diff --git a/.agents/notes/README.i18n.yaml b/.agents/notes/README.i18n.yaml index f92ea2f66d..faf4d69b5b 100644 --- a/.agents/notes/README.i18n.yaml +++ b/.agents/notes/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/README.md README.md: ae8e4724d610c97d74910d7dec5c95af69e93281 -README.zh.md: a16403ae87cd16d758b8303f42f60ad34b9c1eb3 +README.zh.md: 8cecc7068d9efc5e2ebb78d2301d03633f70f137 diff --git a/.agents/notes/README.zh.md b/.agents/notes/README.zh.md index a16403ae87..8cecc7068d 100644 --- a/.agents/notes/README.zh.md +++ b/.agents/notes/README.zh.md @@ -16,13 +16,13 @@ 文件名中的日期是该主题**首次提出**的时间(以 git 历史为准)。Agent Note 之间的交叉引用使用相对 Markdown 链接(`[topic](../../implemented/architecture/2026-…-….md)`),从不使用纯文字或编号,这样既可机械检查,也能在文件夹间移动时保持有效。 -活跃生命周期目录树就是工作清单:浏览其生命周期/类别文件夹,或搜索仓库即可。请勿添加集中式 `INDEX.md`;设计理由见[不设索引的 Agent Note](implemented/process/2026-07-19-remove-generated-agent-note-index.md)。未来指导价值较低的已实施记录会移至下文所述、单独冻结的 [`archived/`](archived/AGENTS.md) 目录树。 +活跃生命周期目录树就是工作清单:浏览其生命周期/类别文件夹,或搜索仓库即可。请勿添加集中式 `INDEX.md`;设计理由见[不设索引的 Agent Note](implemented/process/2026-07-19-remove-generated-agent-note-index.zh.md)。未来指导价值较低的已实施记录会移至下文所述、单独冻结的 [`archived/`](archived/AGENTS.md) 目录树。 ## 分类 -每份 Agent Note 属于 `scripts/agent-note-tree.ts` 中封闭集合里的一个路径编码类别;分类门禁拒绝其他文件夹。新增类别需要同时更新规范集合与本节。见[分类 Agent Note](implemented/process/2026-06-20-agent-note-classification.md)。 +每份 Agent Note 属于 `scripts/agent-note-tree.ts` 中封闭集合里的一个路径编码类别;分类门禁拒绝其他文件夹。新增类别需要同时更新规范集合与本节。见[分类 Agent Note](implemented/process/2026-06-20-agent-note-classification.zh.md)。 | 类别 | 覆盖范围 | |---|---| @@ -41,7 +41,9 @@ 归档路径编码为 `archived/{class}/yyyy-mm-dd-topic-title.md`;其中有意省略 `implemented`,因为只有 implemented Agent Note 可以进入归档。归档变更会移动完整的英文、中文和伴随记录三个文件,保留 `Status: implemented`,在两种语言的文件中紧接该状态行插入相同的 `Archived: YYYY-MM-DD` 行,重新记录伴随记录,并修复或删除入站链接。归档时只允许对内容做这些更改。 -封存后,每组归档文件都永久冻结。禁止编辑、翻译、重新格式化、更新、移动或删除,也不得将其视为当前行为的权威依据。文档门禁会跳过归档源文件,包括其中的出站链接;当活跃文档有意引用历史时,仍可链接到归档 Agent Note。[`verify-archived-agent-notes`](../../scripts/verify-archived-agent-notes.ts) 强制执行封闭的类别目录树、完整的三文件配对、归档元数据、伴随记录 hash,以及仅追加的冻结内容 manifest(元数据清单)。[归档政策 Agent Note](implemented/process/2026-07-26-frozen-agent-note-archive.md) 记录了设计依据。 +封存后,每组归档文件都永久冻结。禁止编辑、翻译、重新格式化、更新、移动或删除,也不得将其视为当前行为的权威依据。文档门禁会跳过归档源文件,包括其中的出站链接;当活跃文档有意引用历史时,仍可链接到归档 Agent Note。[`verify-archived-agent-notes`](../../scripts/verify-archived-agent-notes.ts) 强制执行封闭的类别目录树、完整的三文件配对、归档元数据、伴随记录 hash,以及仅追加的冻结内容 manifest(元数据清单)。[归档政策 Agent Note](implemented/process/2026-07-26-frozen-agent-note-archive.zh.md) 记录了设计依据。 + + ## 何时需要写一份 @@ -57,7 +59,7 @@ ## 文件格式 -每份活跃 Agent Note 遵循统一的文件内格式,由 `pnpm run verify-agent-note-format`([scripts/verify-agent-note-format.ts](../../scripts/verify-agent-note-format.ts),`doc-sync`(文档同步门禁)的一环)强制执行;该格式的设计动机及其否决的替代方案见[统一格式 Agent Note](implemented/process/2026-07-05-uniform-agent-note-format.md)。归档记录保留封存时的格式,并增加上述归档日期行。 +每份活跃 Agent Note 遵循统一的文件内格式,由 `pnpm run verify-agent-note-format`([scripts/verify-agent-note-format.ts](../../scripts/verify-agent-note-format.ts),`doc-sync`(文档同步门禁)的一环)强制执行;该格式的设计动机及其否决的替代方案见[统一格式 Agent Note](implemented/process/2026-07-05-uniform-agent-note-format.zh.md)。归档记录保留封存时的格式,并增加上述归档日期行。 ### 头部块 @@ -126,4 +128,4 @@ Status: ### 中文对侧文件 -`.zh.md` 对侧文件按 [i18n 约定](../../docs/i18n/README.md)逐章节与其英文对侧文件保持相同结构;机器检查的头部标记(`# Agent Note: ` 和 `Status:` 行)保持英文原样不翻译。格式门禁跳过 `.zh.md` 文件;配对门禁检查它们的一致性。 +`.zh.md` 对侧文件按 [i18n 约定](../../docs/i18n/README.zh.md)逐章节与其英文对侧文件保持相同结构;机器检查的头部标记(`# Agent Note: ` 和 `Status:` 行)保持英文原样不翻译。格式门禁跳过 `.zh.md` 文件;配对门禁检查它们的一致性。 diff --git a/.agents/notes/archived/architecture/2026-08-11-plugin-settings-tabs.i18n.yaml b/.agents/notes/archived/architecture/2026-08-11-plugin-settings-tabs.i18n.yaml new file mode 100644 index 0000000000..8b40d4a6bb --- /dev/null +++ b/.agents/notes/archived/architecture/2026-08-11-plugin-settings-tabs.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-11-plugin-settings-tabs.md: 1f1701e000b891b4fc00bc666f603ee00bae50a7 +2026-08-11-plugin-settings-tabs.zh.md: f76a4a9e2317eb6603b48e1a7e451abdc55e00ff diff --git a/.agents/notes/implemented/architecture/2026-08-11-plugin-settings-tabs.md b/.agents/notes/archived/architecture/2026-08-11-plugin-settings-tabs.md similarity index 99% rename from .agents/notes/implemented/architecture/2026-08-11-plugin-settings-tabs.md rename to .agents/notes/archived/architecture/2026-08-11-plugin-settings-tabs.md index 96e4c48926..1f1701e000 100644 --- a/.agents/notes/implemented/architecture/2026-08-11-plugin-settings-tabs.md +++ b/.agents/notes/archived/architecture/2026-08-11-plugin-settings-tabs.md @@ -1,6 +1,7 @@ # Agent Note: Feature-owned tabs in Plugins settings Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-11-plugin-settings-tabs.zh.md) diff --git a/.agents/notes/implemented/architecture/2026-08-11-plugin-settings-tabs.zh.md b/.agents/notes/archived/architecture/2026-08-11-plugin-settings-tabs.zh.md similarity index 99% rename from .agents/notes/implemented/architecture/2026-08-11-plugin-settings-tabs.zh.md rename to .agents/notes/archived/architecture/2026-08-11-plugin-settings-tabs.zh.md index fa8f462640..f76a4a9e23 100644 --- a/.agents/notes/implemented/architecture/2026-08-11-plugin-settings-tabs.zh.md +++ b/.agents/notes/archived/architecture/2026-08-11-plugin-settings-tabs.zh.md @@ -1,6 +1,7 @@ # Agent Note: “插件”设置中的功能自有标签页 Status: implemented +Archived: 2026-08-22 [English](2026-08-11-plugin-settings-tabs.md) | 中文 diff --git a/.agents/notes/archived/bug-fix/2026-08-04-large-history-pagination-call-stack.i18n.yaml b/.agents/notes/archived/bug-fix/2026-08-04-large-history-pagination-call-stack.i18n.yaml new file mode 100644 index 0000000000..4567896a25 --- /dev/null +++ b/.agents/notes/archived/bug-fix/2026-08-04-large-history-pagination-call-stack.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-04-large-history-pagination-call-stack.md: 12e9bbf72c2eea2058bf83fe864e09b4a520d391 +2026-08-04-large-history-pagination-call-stack.zh.md: 288687b05ecdfc2858b4d67e1be199135ea20227 diff --git a/.agents/notes/archived/bug-fix/2026-08-04-large-history-pagination-call-stack.md b/.agents/notes/archived/bug-fix/2026-08-04-large-history-pagination-call-stack.md new file mode 100644 index 0000000000..12e9bbf72c --- /dev/null +++ b/.agents/notes/archived/bug-fix/2026-08-04-large-history-pagination-call-stack.md @@ -0,0 +1,28 @@ +# Agent Note: Large history provenance is scanned without argument expansion + +Status: implemented +Archived: 2026-08-22 + +English | [中文](2026-08-04-large-history-pagination-call-stack.zh.md) + +## Problem + +A finalized assistant message can reference hundreds of thousands of streamed chunks through `sourceEventSeqs`. History pagination found the message group's first event with `Math.min(event.seq, ...sourceEventSeqs)`, so a valid session could exceed the JavaScript engine's function-argument limit and make `session.history` fail with HTTP 500. + +## Decision + +Pagination scans `sourceEventSeqs` and updates the earliest sequence number one element at a time. The algorithm remains linear in the provenance size and preserves the existing page boundary: a page starts before all recorded sources of its oldest included message. + +A regression test rejects multi-argument minimum calls and verifies that every provenance event remains on the page with its finalized message. This exercises the failure mechanism without making the default test suite allocate a production-sized chunk stream. + +## Alternatives considered + +- **Raise the JavaScript stack or argument limit** — rejected: the limit is engine- and deployment-dependent, and array expansion still makes valid history depend on an unrelated runtime ceiling. +- **Truncate `sourceEventSeqs` during pagination** — rejected: this could cut a page inside a message and violate replay grouping. +- **Cap streamed chunk count at the provider boundary** — rejected: providers may legitimately emit long streams, and pagination must handle every valid session representation. + +## Consequences + +- Large provenance arrays no longer make history pagination throw solely because of their length. +- Pagination semantics and wire responses are unchanged. +- This does not bound the byte size of a history page or the browser cost of replaying it; those performance concerns remain separate from the server-side call-stack failure. diff --git a/.agents/notes/archived/bug-fix/2026-08-04-large-history-pagination-call-stack.zh.md b/.agents/notes/archived/bug-fix/2026-08-04-large-history-pagination-call-stack.zh.md new file mode 100644 index 0000000000..288687b05e --- /dev/null +++ b/.agents/notes/archived/bug-fix/2026-08-04-large-history-pagination-call-stack.zh.md @@ -0,0 +1,28 @@ +# Agent Note: 大规模历史记录的溯源信息通过扫描处理,不做参数展开 + +Status: implemented +Archived: 2026-08-22 + +[English](2026-08-04-large-history-pagination-call-stack.md) | 中文 + +## 问题 + +一条已定稿的 assistant 消息可以通过 `sourceEventSeqs` 引用数十万个流式分片。历史记录分页使用 `Math.min(event.seq, ...sourceEventSeqs)` 查找消息组的首个事件,因此,有效会话可能超出 JavaScript 引擎的函数参数数量上限,导致 `session.history` 以 HTTP 500 失败。 + +## 决策 + +分页逻辑逐项扫描 `sourceEventSeqs`,每次使用一个元素更新最早的序号。该算法的复杂度相对溯源信息规模仍为线性,并保留现有的页面边界:页面起点位于其所含最早消息的所有已记录来源之前。 + +回归测试会拒绝以多个参数调用取最小值的做法,并验证每个溯源事件都会与其已定稿消息保留在同一页中。这既覆盖了故障机制,也避免默认测试套件分配生产规模的分片流。 + +## 考虑过的替代方案 + +- **提高 JavaScript 栈或参数上限**:不予采纳,因为该上限取决于引擎和部署环境,而且数组展开仍会让有效历史记录受制于无关的运行时上限。 +- **在分页时截断 `sourceEventSeqs`**:不予采纳,因为这可能会从消息中间切分页面,破坏回放分组。 +- **在提供方边界限制流式分片数量**:不予采纳,因为提供方可能会合理地产生长流,而分页必须处理每一种有效的会话表示。 + +## 后果 + +- 大型溯源数组不再仅因长度而使历史记录分页抛出异常。 +- 分页语义与协议响应保持不变。 +- 本决策不限制历史记录页面的字节大小,也不限制浏览器回放该页面的开销;这两项性能问题仍与服务端调用栈故障分开处理。 diff --git a/.agents/notes/archived/bug-fix/2026-08-06-plan-narrow-viewport-regression.i18n.yaml b/.agents/notes/archived/bug-fix/2026-08-06-plan-narrow-viewport-regression.i18n.yaml new file mode 100644 index 0000000000..a6305c1f5d --- /dev/null +++ b/.agents/notes/archived/bug-fix/2026-08-06-plan-narrow-viewport-regression.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-06-plan-narrow-viewport-regression.md: c42a110bf487ffab8f5975e65a8eb9bff4f1b0ae +2026-08-06-plan-narrow-viewport-regression.zh.md: 3df4f8aca57a1830d7f8062cefe76ac1afa9c2ce diff --git a/.agents/notes/archived/bug-fix/2026-08-06-plan-narrow-viewport-regression.md b/.agents/notes/archived/bug-fix/2026-08-06-plan-narrow-viewport-regression.md new file mode 100644 index 0000000000..c42a110bf4 --- /dev/null +++ b/.agents/notes/archived/bug-fix/2026-08-06-plan-narrow-viewport-regression.md @@ -0,0 +1,34 @@ +# Agent Note: narrow-viewport plan chip click-area regression test + +Status: implemented +Archived: 2026-08-22 + +English | [中文](2026-08-06-plan-narrow-viewport-regression.zh.md) + +## Problem + +The external report dsh-external/issues#107 (clustered internally as deepseek-harness#1406) measured that at viewports between 760px and 850px the plan control and the model selector overlapped, with the model selector covering the plan control's click area so plan mode could not be left by mouse at 800×720. Its acceptance list asked for a browser regression test asserting that the plan center hit-tests to the plan button. + +The browser regression test reproduced the report on current master: at 800×720 the plan chip and the model trigger overlapped by 36.9px and the chip's center hit-tested to the trigger's label. The composer control row is `display: flex; justify-content: space-between` with `.trailing { flex: none }`: when the combined control width exceeds the card, the shrinking `.tools` group keeps its flow children inside its `min-width: 0` box, so the chip — the last flow child before the overflow — is painted over the trailing group. The plan-control form changed since the report (select → chip, `c20b988166`/`fe91919346`) and the row gained adaptive behavior (`c8c75ec891`, [web-composer-shared-width-axis](../feature/2026-08-04-web-composer-shared-width-axis.md)), but the row had no wrap, so the overlap survived both. + +## Decision + +The row wraps instead of shrinking its left group into the right group's area: `.row { flex-wrap: wrap }` plus `margin-left: auto` on `.trailing`, which re-anchors the trailing group (model + send) to the right edge of its wrapped line while `space-between` already pins it right on a single line. Wrapping is the acceptance's "wrap, fold, or re-arrange controls when space runs out" option, keeps every control at full width (no label folding that would hide the model name or the Plan wordmark), and holds at every viewport width by construction instead of at a calibrated container-query threshold. + +Add `apps/web/tests/plan-control-row.e2e.ts`: enter plan mode with the real `/plan` command (no argument — the command handler commits plan/mode active without a model round, the lifecycle-chrome precedent), so the test needs no model call in any mode and no API key in replay/refresh; a providers-only fixture mounts the model catalog without a script to consume. The file joins the host-plane e2e pairing like every sibling: excluded from the client graph in `apps/web/tsconfig.json` (it imports host-plane types) AND included in the host aggregate in `tsconfig.host.json`, so exactly one TypeScript program owns it — the pairing that also gives the lint type service its program. + +The geometry golden records stable facts — viewport membership on both axes and disjoint click areas — never absolute coordinates, whose pixel values depend on installed fonts and differ between macOS and Linux. The behavior assertions implement the acceptance directly: the click areas are disjoint, the click at the chip's center (Playwright's actionability check) leaves plan mode through the real command channel (`/plan off` via `commands.execute`), and the last `plan/mode` event in the session log flips inactive. + +## Alternatives considered + +**Seed a cold session (composer-tab-geometry pattern).** Rejected: the exit path executes `/plan off` through `commands.execute`, which needs the live agent a cold seeded session does not have; `connectFreshWorkspace` keeps one, matching the product's user path. + +**Pin absolute bounding boxes in the golden.** Rejected: chip and trigger widths depend on the installed fonts, so absolute coordinates would churn across platforms without a behavior change. + +**Reuse the plan-review fixture shape (exit_plan_mode review takeover).** Rejected: the takeover replaces the composer's control row, which is the surface under test. + +**Container-query label folding for the chip and/or the model trigger.** Rejected for the fix: two packages (ui-plan, ui-model) would need calibrated thresholds and the chip's own icon-only fold still leaves ~7px of overlap at the reported viewport unless the trigger folds too. Wrapping is one rule in one package and holds at every width. + +## Consequences + +Any future change to the control row layout — fonts, gaps, media or container queries — that re-introduces overlap or moves the chip out of the viewport on either axis fails this test. The test needs no API key in replay/refresh modes: plan mode toggles through the command handler without a model round, and a providers-only replay fixture (no recorded script, consumption check skipped) mounts the model directory so the trigger renders its real long label — the width that made the reported overlap measurable; the test asserts that label before measuring. The golden is compared in replay and record modes and rewritten in refresh mode. diff --git a/.agents/notes/archived/bug-fix/2026-08-06-plan-narrow-viewport-regression.zh.md b/.agents/notes/archived/bug-fix/2026-08-06-plan-narrow-viewport-regression.zh.md new file mode 100644 index 0000000000..3df4f8aca5 --- /dev/null +++ b/.agents/notes/archived/bug-fix/2026-08-06-plan-narrow-viewport-regression.zh.md @@ -0,0 +1,34 @@ +# Agent Note: 窄视口下 Plan chip 点击区域回归测试 + +Status: implemented +Archived: 2026-08-22 + +[English](2026-08-06-plan-narrow-viewport-regression.md) | 中文 + +## 问题 + +外部报告 dsh-external/issues#107(内部聚类为 deepseek-harness#1406)测得视口宽度在 760px 到 850px 之间时 Plan 控件与模型选择器发生重叠,模型选择器覆盖 Plan 控件的点击区域,导致在 800×720 下无法用鼠标退出 Plan 模式。其验收清单要求增加浏览器回归测试,断言 Plan 中心命中 Plan 按钮。 + +浏览器回归测试在当前 master 上复现了报告:800×720 下 Plan chip 与模型 trigger 重叠 36.9px,chip 中心命中 trigger 的 label。composer 控制行是 `display: flex; justify-content: space-between` 且 `.trailing { flex: none }`:当控件总宽超过卡片时,可收缩的 `.tools` 组把流内子项留在 `min-width: 0` 的盒内,于是 chip——溢出前最后一个流内子项——被绘制到 trailing 组上方。报告以来 Plan 控件形态已变(select → chip,`c20b988166`/`fe91919346`),控制行也获得过自适应能力(`c8c75ec891`,[web-composer-shared-width-axis](../feature/2026-08-04-web-composer-shared-width-axis.zh.md)),但该行没有换行,重叠在两次重构后依然存在。 + +## 决策 + +控制行换行而不是把左侧组收缩进右侧组的区域:`.row { flex-wrap: wrap }` 加上 `.trailing` 的 `margin-left: auto`——后者把 trailing 组(模型选择 + 发送)重新锚定到换行后的右缘,单行时 `space-between` 已把它钉在右侧。换行是验收中"空间不足时允许换行、折叠或重新排列控件"的选项,保持每个控件全宽(不做会隐藏模型名或 Plan 字样的 label 折叠),并且按构造在所有视口宽度下成立,而非依赖标定的容器查询阈值。 + +新增 `apps/web/tests/plan-control-row.e2e.ts`:通过真实 `/plan` 命令(无参数——命令 handler 不经模型回合即提交 plan/mode active,lifecycle-chrome 先例)进入 Plan 模式,因此测试在任何模式下都无需模型调用,仅在 replay/refresh 下无需 API key;providers-only fixture 挂载模型目录而无脚本可消费。该文件与所有同类 host 平面 e2e 一样采用成对登记:在 `apps/web/tsconfig.json` 的 exclude 列表(它导入 host 平面类型,client 图绝不编译它),同时在 `tsconfig.host.json` 的 host 聚合 include 中——恰好一个 TypeScript 程序拥有它,这也是 lint 类型服务获得程序的配对方式。 + +几何 golden 记录稳定事实——两个轴上的视口内位置与点击区域不相交——绝不记录绝对坐标,其像素值依赖安装字体且在 macOS 与 Linux 间不同。行为断言直接实现验收:点击区域不相交、点击 chip 中心(Playwright 的可操作性检查)经真实命令通道(`commands.execute` 执行 `/plan off`)退出 Plan 模式,且会话日志中最后一条 `plan/mode` 事件翻转为 inactive。 + +## 备选方案 + +**冷会话 seed(composer-tab-geometry 模式)。** 否决:退出路径经 `commands.execute` 执行 `/plan off`,需要 live agent,而冷 seed 会话没有;`connectFreshWorkspace` 保留一个,与产品的用户路径一致。 + +**golden 固定绝对 bounding box。** 否决:chip 与 trigger 宽度依赖安装字体,绝对坐标会在平台间漂移而不反映行为变化。 + +**复用 plan-review fixture 形态(exit_plan_mode review takeover)。** 否决:takeover 会替换 composer 控制行,而被测表面正是控制行。 + +**chip 与/或模型 trigger 的容器查询 label 折叠。** 否决(作为修复):两个包(ui-plan、ui-model)需要各自标定阈值,且 chip 单独折叠为 icon-only 在报告视口下仍剩约 7px 重叠,除非 trigger 也折叠。换行是一个包中的一条规则,且在所有宽度下成立。 + +## 后果 + +任何改变控制行布局的后续改动——字体、间距、媒体查询或容器查询——一旦重新引入重叠或把 chip 沿任一轴移出视口,本测试即失败。测试在 replay/refresh 模式下无需 API key:Plan 模式经命令 handler 切换,不经模型回合;providers-only replay fixture(无录制脚本,跳过消费检查)挂载模型目录,使触发器渲染真实的长标签——正是使报告重叠可测量的宽度;测试在测量前断言该标签。golden 在 replay 与 record 模式下比较,在 refresh 模式下重写。 diff --git a/.agents/notes/archived/bug-fix/2026-08-11-preset-card-description-clamp.i18n.yaml b/.agents/notes/archived/bug-fix/2026-08-11-preset-card-description-clamp.i18n.yaml new file mode 100644 index 0000000000..644cd813ce --- /dev/null +++ b/.agents/notes/archived/bug-fix/2026-08-11-preset-card-description-clamp.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-11-preset-card-description-clamp.md: b9088b46184cde6f000fa39afbfe2d1145137b24 +2026-08-11-preset-card-description-clamp.zh.md: a7a3c4f26a55405715fe017ec3a7deb164432b0e diff --git a/.agents/notes/implemented/bug-fix/2026-08-11-preset-card-description-clamp.md b/.agents/notes/archived/bug-fix/2026-08-11-preset-card-description-clamp.md similarity index 99% rename from .agents/notes/implemented/bug-fix/2026-08-11-preset-card-description-clamp.md rename to .agents/notes/archived/bug-fix/2026-08-11-preset-card-description-clamp.md index 16ebf371d5..b9088b4618 100644 --- a/.agents/notes/implemented/bug-fix/2026-08-11-preset-card-description-clamp.md +++ b/.agents/notes/archived/bug-fix/2026-08-11-preset-card-description-clamp.md @@ -1,6 +1,7 @@ # Agent Note: Preset cards clamp their description instead of sizing the roster Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-11-preset-card-description-clamp.zh.md) diff --git a/.agents/notes/implemented/bug-fix/2026-08-11-preset-card-description-clamp.zh.md b/.agents/notes/archived/bug-fix/2026-08-11-preset-card-description-clamp.zh.md similarity index 99% rename from .agents/notes/implemented/bug-fix/2026-08-11-preset-card-description-clamp.zh.md rename to .agents/notes/archived/bug-fix/2026-08-11-preset-card-description-clamp.zh.md index 5b7a18f41e..a7a3c4f26a 100644 --- a/.agents/notes/implemented/bug-fix/2026-08-11-preset-card-description-clamp.zh.md +++ b/.agents/notes/archived/bug-fix/2026-08-11-preset-card-description-clamp.zh.md @@ -1,6 +1,7 @@ # Agent Note: 预设卡片截断自身描述,而不是由描述决定整份名单的高度 Status: implemented +Archived: 2026-08-22 [English](2026-08-11-preset-card-description-clamp.md) | 中文 diff --git a/.agents/notes/archived/feature/2026-07-30-versioned-gui-welcome-onboarding.i18n.yaml b/.agents/notes/archived/feature/2026-07-30-versioned-gui-welcome-onboarding.i18n.yaml new file mode 100644 index 0000000000..87a43d82ca --- /dev/null +++ b/.agents/notes/archived/feature/2026-07-30-versioned-gui-welcome-onboarding.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-07-30-versioned-gui-welcome-onboarding.md: 370793dd4e6d744ea61fc9319a94728dbbf3e1fe +2026-07-30-versioned-gui-welcome-onboarding.zh.md: 7b99377d00127174d5bfd8d080107c2cb67e6fff diff --git a/.agents/notes/archived/feature/2026-07-30-versioned-gui-welcome-onboarding.md b/.agents/notes/archived/feature/2026-07-30-versioned-gui-welcome-onboarding.md new file mode 100644 index 0000000000..370793dd4e --- /dev/null +++ b/.agents/notes/archived/feature/2026-07-30-versioned-gui-welcome-onboarding.md @@ -0,0 +1,34 @@ +# Agent Note: Versioned GUI welcome onboarding + +Status: implemented +Archived: 2026-08-22 + +English | [中文](2026-07-30-versioned-gui-welcome-onboarding.zh.md) + +## Problem + +The GUI's credential onboarding begins with a DeepSeek-specific readiness check, but the internal-test notice applies to every user and must precede provider setup even when a credential is already configured. Treating both as independent overlays permits simultaneous dialogs, while a process-local dismissal cannot distinguish a completed notice from a window closed before acknowledgement or intentionally present revised copy once. + +## Decision + +**The Settings shell coordinates ordered steps.** `settings.onboarding` remains a root-scoped list, but `ui-settings` projects its entry ids and order into one coordinator and mounts only the first incomplete step. The active registrant receives `complete()` and `openSection(id)`; no later step mounts until ownership transfers. `ui-settings-models` now registers the restored welcome notice at order `-100` and the conditional DeepSeek credential step at order `0`; their current shared presentation is owned by the [shared-modal onboarding decision](2026-08-13-shared-modal-product-onboarding.md). + +**The product welcome step is versioned and feature-owned.** The notice was historically removed by the [first-run beta notice removal](../simplification/2026-08-13-remove-first-run-beta-notice.md) and is now restored in `ui-settings-models` with new testing-stage copy. `ui-settings-general` still seats no onboarding step; the plugin that owns both current steps also owns the copy, store, and shared modal. + +**The durable `ui-onboarding` section owns acknowledgement.** The Host half registers it in the user-settings seam under the active `$DSH_HOME/settings.yaml`; the current welcome store reads and writes `welcomeNoticeVersion` through the existing public settings API. The connection plugin publishes whether the current page uses a loopback authority as `ctx.connection.isLoopback`; hostname classification remains internal to the connection package, and other client plugins consume the service state instead of importing its implementation. The API proxy exposes this one product namespace through a closed allowlist beside configurable-provider namespaces, without treating its changes as model-catalog invalidations. + +**Visible onboarding uses one shared modal contract.** Both current steps render through the same body-portaled `OnboardingModal`, and the underlying app root stays inert only while a dialog is visible. The shell renders no wrapper while a step loads its private facts. Explicit actions transfer coordinator ownership; Escape and mask clicks do not acknowledge or skip a step. + +## Alternatives considered + +**Browser local storage** — rejected because acknowledgement would follow one browser profile rather than `$DSH_HOME`; a fresh Harness profile could incorrectly inherit a prior acknowledgement, and external profile edits would have no authoritative update stream. Non-loopback fallback therefore remains process-local rather than browser-profile-local. + +**A second independent modal in `ui-settings-general`** — rejected because list registrants would still stack whenever welcome and credential readiness were both true. Ordered ownership belongs to the shell that declares and renders the list. + +**Persisting on render or window close** — rejected because observation is not acknowledgement and close delivery is unreliable. Only the explicit Continue commit may suppress the next launch. + +**A generic public settings-exposure flag** — rejected because one product namespace does not justify widening every settings registrant's public configuration surface. The gateway keeps an explicit closed allowlist. + +## Consequences + +A fresh profile sees the current testing-stage notice, then the conditional DeepSeek key dialog when no provider is usable. Focused store and React tests pin exact-version acknowledgement, coordinator ordering, conditional transfer, shared modal behavior, and HMR cleanup. The real Chromium scenario boots the shipped Web composition with an isolated harness home, verifies both dialogs, writes the key through the existing credential boundary, and checks that no secret reaches the DOM, ARIA, or browser console. diff --git a/.agents/notes/archived/feature/2026-07-30-versioned-gui-welcome-onboarding.zh.md b/.agents/notes/archived/feature/2026-07-30-versioned-gui-welcome-onboarding.zh.md new file mode 100644 index 0000000000..7b99377d00 --- /dev/null +++ b/.agents/notes/archived/feature/2026-07-30-versioned-gui-welcome-onboarding.zh.md @@ -0,0 +1,34 @@ +# Agent Note: 版本化 GUI 欢迎引导 + +Status: implemented +Archived: 2026-08-22 + +[English](2026-07-30-versioned-gui-welcome-onboarding.md) | 中文 + +## 问题 + +GUI 的凭据引导从 DeepSeek 专用的就绪状态检查开始,但内部测试通知适用于每位用户,即使凭据已经配置,也必须先于提供方设置显示。若把两者作为独立浮层处理,多个对话框可能同时出现;仅存于进程内的关闭标记既无法区分通知已完成确认还是窗口在确认前已关闭,也无法在文案有意修订后重新显示一次通知。 + +## 决策 + +**设置外壳协调有序步骤。** `settings.onboarding` 仍是根作用域 list,但 `ui-settings` 会把其中各条目的 id 和顺序投影到一个协调器中,并且只挂载第一个未完成的步骤。当前注册方会收到 `complete()` 和 `openSection(id)`;所有权转移前,不会挂载后续步骤。`ui-settings-models` 现在以顺序 `-100` 注册恢复后的欢迎声明,以顺序 `0` 注册 DeepSeek 条件式凭据步骤;两者当前的共用展示由[共用弹窗引导决策](2026-08-13-shared-modal-product-onboarding.zh.md)持有。 + +**产品欢迎步骤按版本管理并归功能插件所有。** 该声明曾由[移除首次启动内测声明](../simplification/2026-08-13-remove-first-run-beta-notice.zh.md)历史决策移除,现在以新的测试阶段文案恢复在 `ui-settings-models` 中。`ui-settings-general` 仍不注册任何引导步骤;持有当前两个步骤的插件也持有文案、store 和共用弹窗。 + +**持久化的 `ui-onboarding` 分节持有确认状态。** 宿主端在 user-settings seam 中注册它,存入当前 `$DSH_HOME/settings.yaml`;当前欢迎 store 通过既有公开 settings API 读写其中的 `welcomeNoticeVersion`。connection 插件通过 `ctx.connection.isLoopback` 统一发布当前页面是否使用 loopback authority;hostname 判定留在 connection 包内,其他客户端插件只消费服务状态,而不导入其实现。API Proxy 在可配置提供方 namespace 之外,通过封闭的允许列表暴露这一个产品 namespace,同时不会把它的变更视为模型目录失效事件。 + +**可见引导使用同一个弹窗契约。** 当前两个步骤都通过 body portal 的同一个 `OnboardingModal` 渲染,且只在弹窗可见期间把下层应用根节点设为 inert。步骤加载私有事实时,外壳不渲染任何包装。明确操作会移交协调器所有权;Escape 和点击遮罩都不会确认或跳过步骤。 + +## 曾考虑的替代方案 + +**浏览器本地存储**:不予采用,因为确认状态会跟随某个浏览器 profile,而不是 `$DSH_HOME`;全新的 Harness profile 可能错误继承此前的确认状态,外部 profile 编辑也没有权威更新流。因此,非 loopback 的回退保持为进程内状态,而不是浏览器 profile 状态。 + +**在 `ui-settings-general` 中再增加一个独立模态窗口**:不予采用,因为欢迎通知和凭据就绪状态同时为真时,list 注册方仍会堆叠。声明并渲染该 list 的外壳应当持有有序所有权。 + +**在渲染或窗口关闭时持久化**:不予采用,因为看见通知不等于确认,窗口关闭事件也无法可靠送达。只有显式提交「继续」才能阻止通知在下次启动时再次显示。 + +**通用的公开设置暴露标志**:不予采用,因为一个产品 namespace 不足以证明应当扩大每个 settings 注册方的公开配置面。该 API Proxy 保留显式的封闭允许列表。 + +## 后果 + +全新 profile 会先看到当前测试阶段声明;当没有任何可用提供方时,再看到条件式 DeepSeek 密钥弹窗。定向 store 与 React 测试固定精确版本确认、协调器顺序、条件式移交、共用弹窗行为与 HMR 清理。真实 Chromium 场景会在隔离的 harness 家目录下启动已发布 Web 组合,验证两个弹窗,通过既有凭据边界写入密钥,并检查 secret 未进入 DOM、ARIA 或浏览器控制台。 diff --git a/.agents/notes/archived/feature/2026-08-06-bundled-dsh-badge-skill.i18n.yaml b/.agents/notes/archived/feature/2026-08-06-bundled-dsh-badge-skill.i18n.yaml new file mode 100644 index 0000000000..4cb64e37c0 --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-06-bundled-dsh-badge-skill.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-06-bundled-dsh-badge-skill.md: 2909736d53f8aff41ca69e705de44657bc1b4f1e +2026-08-06-bundled-dsh-badge-skill.zh.md: de85e9476d3945cd13335ef596243bc2a126ae55 diff --git a/.agents/notes/implemented/feature/2026-08-06-bundled-dsh-badge-skill.md b/.agents/notes/archived/feature/2026-08-06-bundled-dsh-badge-skill.md similarity index 98% rename from .agents/notes/implemented/feature/2026-08-06-bundled-dsh-badge-skill.md rename to .agents/notes/archived/feature/2026-08-06-bundled-dsh-badge-skill.md index 4c6fbdcb76..2909736d53 100644 --- a/.agents/notes/implemented/feature/2026-08-06-bundled-dsh-badge-skill.md +++ b/.agents/notes/archived/feature/2026-08-06-bundled-dsh-badge-skill.md @@ -1,6 +1,7 @@ # Agent Note: Bundled dsh badge skill Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-06-bundled-dsh-badge-skill.zh.md) diff --git a/.agents/notes/implemented/feature/2026-08-06-bundled-dsh-badge-skill.zh.md b/.agents/notes/archived/feature/2026-08-06-bundled-dsh-badge-skill.zh.md similarity index 83% rename from .agents/notes/implemented/feature/2026-08-06-bundled-dsh-badge-skill.zh.md rename to .agents/notes/archived/feature/2026-08-06-bundled-dsh-badge-skill.zh.md index f249c31f6e..de85e9476d 100644 --- a/.agents/notes/implemented/feature/2026-08-06-bundled-dsh-badge-skill.zh.md +++ b/.agents/notes/archived/feature/2026-08-06-bundled-dsh-badge-skill.zh.md @@ -1,12 +1,13 @@ # Agent Note: 内置 dsh 徽章 skill Status: implemented +Archived: 2026-08-22 [English](2026-08-06-bundled-dsh-badge-skill.md) | 中文 ## 问题 -[Cordis 教程](../../../../docs/cordis-tutorial/index.md)的各个页面都使用官方「powered by dsh」徽章,但交付的 CLI(命令行界面)既没有用于在其他位置应用同样署名的可复用指令,也没有可显式选择加入的提供方。 +[Cordis 教程](../../../../docs/cordis-tutorial/index.zh.md)的各个页面都使用官方「powered by dsh」徽章,但交付的 CLI(命令行界面)既没有用于在其他位置应用同样署名的可复用指令,也没有可显式选择加入的提供方。 ## 决策 diff --git a/.agents/notes/archived/feature/2026-08-07-workspace-picker-composer-entry.i18n.yaml b/.agents/notes/archived/feature/2026-08-07-workspace-picker-composer-entry.i18n.yaml new file mode 100644 index 0000000000..7459c96dae --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-07-workspace-picker-composer-entry.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-07-workspace-picker-composer-entry.md: 023cbe09dd75015a1555103642d1b66ce75aafc9 +2026-08-07-workspace-picker-composer-entry.zh.md: 6b84885b11fb5372a51d620b40ea7d24fe7e45a2 diff --git a/.agents/notes/implemented/feature/2026-08-07-workspace-picker-composer-entry.md b/.agents/notes/archived/feature/2026-08-07-workspace-picker-composer-entry.md similarity index 99% rename from .agents/notes/implemented/feature/2026-08-07-workspace-picker-composer-entry.md rename to .agents/notes/archived/feature/2026-08-07-workspace-picker-composer-entry.md index dc9c26c291..023cbe09dd 100644 --- a/.agents/notes/implemented/feature/2026-08-07-workspace-picker-composer-entry.md +++ b/.agents/notes/archived/feature/2026-08-07-workspace-picker-composer-entry.md @@ -1,6 +1,7 @@ # Agent Note: The no-Workspace composer opens the existing picker Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-07-workspace-picker-composer-entry.zh.md) diff --git a/.agents/notes/implemented/feature/2026-08-07-workspace-picker-composer-entry.zh.md b/.agents/notes/archived/feature/2026-08-07-workspace-picker-composer-entry.zh.md similarity index 89% rename from .agents/notes/implemented/feature/2026-08-07-workspace-picker-composer-entry.zh.md rename to .agents/notes/archived/feature/2026-08-07-workspace-picker-composer-entry.zh.md index 585c59b123..6b84885b11 100644 --- a/.agents/notes/implemented/feature/2026-08-07-workspace-picker-composer-entry.zh.md +++ b/.agents/notes/archived/feature/2026-08-07-workspace-picker-composer-entry.zh.md @@ -1,12 +1,13 @@ # Agent Note: 未选择 Workspace 时从编辑器打开现有选择器 Status: implemented +Archived: 2026-08-22 [English](2026-08-07-workspace-picker-composer-entry.md) | 中文 ## 问题 -[Session scope 决策](../architecture/2026-07-25-web-client-session-scope-and-provide-channel.md)会在 Workspace 存在前保留同一个常驻编辑器,但 textarea 处于禁用状态,只有较小的 Workspace chip 能打开选择器。用户首次点击最显眼、也最熟悉的输入区域时,界面不会响应,尽管同一界面已有继续操作的入口。 +[Session scope 决策](../architecture/2026-07-25-web-client-session-scope-and-provide-channel.zh.md)会在 Workspace 存在前保留同一个常驻编辑器,但 textarea 处于禁用状态,只有较小的 Workspace chip 能打开选择器。用户首次点击最显眼、也最熟悉的输入区域时,界面不会响应,尽管同一界面已有继续操作的入口。 ## 决策 diff --git a/.agents/notes/archived/feature/2026-08-10-creator-guidance-introduce-cue.i18n.yaml b/.agents/notes/archived/feature/2026-08-10-creator-guidance-introduce-cue.i18n.yaml new file mode 100644 index 0000000000..27eba69e62 --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-10-creator-guidance-introduce-cue.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-10-creator-guidance-introduce-cue.md: 2954bb9dca6bd5359ab3ed4b7e32bd8336709a10 +2026-08-10-creator-guidance-introduce-cue.zh.md: 4f818b3a444cbc7bbd4355ac8e7a883aaa4bfa51 diff --git a/.agents/notes/implemented/feature/2026-08-10-creator-guidance-introduce-cue.md b/.agents/notes/archived/feature/2026-08-10-creator-guidance-introduce-cue.md similarity index 99% rename from .agents/notes/implemented/feature/2026-08-10-creator-guidance-introduce-cue.md rename to .agents/notes/archived/feature/2026-08-10-creator-guidance-introduce-cue.md index 888fee7b3d..2954bb9dca 100644 --- a/.agents/notes/implemented/feature/2026-08-10-creator-guidance-introduce-cue.md +++ b/.agents/notes/archived/feature/2026-08-10-creator-guidance-introduce-cue.md @@ -1,6 +1,7 @@ # Agent Note: Creator guidance lands as an introduce cue on the preset chip Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-10-creator-guidance-introduce-cue.zh.md) diff --git a/.agents/notes/implemented/feature/2026-08-10-creator-guidance-introduce-cue.zh.md b/.agents/notes/archived/feature/2026-08-10-creator-guidance-introduce-cue.zh.md similarity index 99% rename from .agents/notes/implemented/feature/2026-08-10-creator-guidance-introduce-cue.zh.md rename to .agents/notes/archived/feature/2026-08-10-creator-guidance-introduce-cue.zh.md index d80260abd1..4f818b3a44 100644 --- a/.agents/notes/implemented/feature/2026-08-10-creator-guidance-introduce-cue.zh.md +++ b/.agents/notes/archived/feature/2026-08-10-creator-guidance-introduce-cue.zh.md @@ -1,6 +1,7 @@ # Agent Note: 创造模式引导以介绍动效落在预设 chip 上 Status: implemented +Archived: 2026-08-22 [English](2026-08-10-creator-guidance-introduce-cue.md) | 中文 diff --git a/.agents/notes/archived/feature/2026-08-11-collapsible-ask-user-question-card.i18n.yaml b/.agents/notes/archived/feature/2026-08-11-collapsible-ask-user-question-card.i18n.yaml new file mode 100644 index 0000000000..6fc55e3a3e --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-11-collapsible-ask-user-question-card.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-11-collapsible-ask-user-question-card.md: 08e87b0d1f05f47c5bc87ef9b32cab05ef229e5c +2026-08-11-collapsible-ask-user-question-card.zh.md: fd3b838ff174dcdb3d4994fe30b193d63f5c4d15 diff --git a/.agents/notes/archived/feature/2026-08-11-collapsible-ask-user-question-card.md b/.agents/notes/archived/feature/2026-08-11-collapsible-ask-user-question-card.md new file mode 100644 index 0000000000..08e87b0d1f --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-11-collapsible-ask-user-question-card.md @@ -0,0 +1,33 @@ +# Agent Note: Collapsible Ask-User Question Card + +Status: implemented +Archived: 2026-08-22 + +English | [中文](2026-08-11-collapsible-ask-user-question-card.zh.md) + +## Problem + +`dsh`'s ask-user takeover renders the pending question set as a bottom card capped at `min(60vh, 520px)`, so a long batch or a user who wants to re-read the conversation above before deciding has no way to reduce the card — the conversation above becomes hard to read because only a few lines peek out at the top. + +## Decision + +Add a minimize/maximize toggle to the question card header, next to the existing dismiss action. Collapsing hides the option body and the footer actions, leaving a header strip (eyebrow, title, both icon buttons) so the user still sees that a question is pending; expanding restores the full card. + +- State lives in `QuestionFlow` local state (`minimized`), so drafts and the current question index survive collapse/expand — nothing is re-derived or reset, and the answers already picked remain submit-ready. +- The toggle is a plain `IconChevronDownOutline14` / `IconChevronUpOutline14` pair on the existing 24px icon-button grid; `aria-expanded` reflects the card state and the label flips between `nav.minimize` / `nav.maximize` (the collapsed button reads "expand" for screen readers). +- While minimized the option body and footer are unmounted (`{!minimized && ...}`), so no hidden interactive surface remains in the a11y tree. +- The collapse button is disabled while a submit/cancel is in flight (`busy !== null`), matching the dismiss button's existing guard. +- CSS: `.cardMinimized` drops the `max-height` cap and hides `.body` / `.footer`; `.header` gains bottom padding so the strip is not cramped. +- Scope: only the generic question flow (`QuestionFlow`) gets the toggle. The plan-review card (`PlanReviewPanel`) is a different shape (one decision over one plan) and keeps its current layout. + +## Consequences + +- Users can shrink the question card to read the conversation, then expand to answer — drafts and position are preserved because the state lives in the flow component, not in the DOM. +- The minimize action is visually adjacent to dismiss; both share the icon button style, so the header stays balanced. +- Product copy additions are confined to the `question` locale namespace (`nav.minimize` / `nav.maximize`), paired zh/en per the dictionary contract. + +## Alternatives considered + +- **Auto-collapse on scroll**: collapsing the card when the user scrolls the conversation would reclaim space without a button, but it fights the user mid-interaction and hides the pending-question signal unexpectedly; an explicit toggle keeps the decision with the user. +- **Resizable card**: a drag handle would let users size the card freely, but it is more machinery than the ask needs and does not address "I want the card out of the way entirely". +- **Persisting the collapsed state per session**: nice-to-have, but the ask is per-interaction; persisting adds storage and sync complexity without a clear win for this surface. diff --git a/.agents/notes/archived/feature/2026-08-11-collapsible-ask-user-question-card.zh.md b/.agents/notes/archived/feature/2026-08-11-collapsible-ask-user-question-card.zh.md new file mode 100644 index 0000000000..fd3b838ff1 --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-11-collapsible-ask-user-question-card.zh.md @@ -0,0 +1,33 @@ +# Agent Note: 可收起的提问卡片 + +Status: implemented +Archived: 2026-08-22 + +[English](2026-08-11-collapsible-ask-user-question-card.md) | 中文 + +## Problem + +`dsh` 的 ask-user 接管界面把待回答的问题组渲染为底部卡片,高度上限为 `min(60vh, 520px)`;当问题批次较长、或用户想先阅读上方的会话记录再决定时,卡片会占满大部分视口且无法缩小——上方会话几乎被遮住,只能看到顶部几行。 + +## Decision + +在提问卡片头部(现有的"放弃整组问题"按钮旁)增加收起/展开切换按钮。收起时隐藏选项主体和底部操作区,只保留一条头部(eyebrow、标题、两个图标按钮),用户仍能看到"有未答问题"的信号;展开后恢复完整卡片。 + +- 状态存放在 `QuestionFlow` 的本地 state(`minimized`),因此收起/展开不会丢失草稿和当前题目索引——已选答案仍可直接提交。 +- 切换按钮使用 `IconChevronDownOutline14` / `IconChevronUpOutline14`,复用现有 24px 图标按钮网格;`aria-expanded` 反映卡片状态,文案在 `nav.minimize` / `nav.maximize` 之间切换(收起后按钮对读屏器显示为"展开")。 +- 收起时选项主体和底部通过 `{!minimized && ...}` 卸载,a11y 树中不残留隐藏的可交互面。 +- 提交/取消进行中(`busy !== null`)时收起按钮禁用,与现有放弃按钮的守卫一致。 +- CSS:`.cardMinimized` 去掉 `max-height` 上限并隐藏 `.body` / `.footer`;`.header` 增加底部 padding,避免折叠后过于局促。 +- 范围:只有通用提问流(`QuestionFlow`)获得该切换。计划评审卡片(`PlanReviewPanel`)是另一种形态(对一个计划做一次决策),保持现有布局。 + +## Consequences + +- 用户可以缩小提问卡片以阅读会话,再展开作答——草稿和位置因状态存放在流程组件中而得以保留。 +- 收起动作紧邻放弃按钮,二者共用图标按钮样式,头部保持平衡。 +- 新增产品文案仅落在 `question` locale 命名空间(`nav.minimize` / `nav.maximize`),按字典契约中英成对。 + +## Alternatives considered + +- **滚动时自动收起**:用户滚动会话时自动折叠卡片可以省空间,但会在交互中途与用户对抗,并意外隐藏"待答问题"信号;显式切换把决定权交给用户。 +- **可拖拽调整大小**:拖拽手柄让用户自由调整卡片大小,但比需求所需的机制更复杂,也没有解决"让卡片完全让开"的诉求。 +- **按会话持久化折叠状态**:锦上添花,但提问是单次交互;持久化引入存储与同步复杂度,对这个界面没有明确收益。 diff --git a/.agents/notes/archived/feature/2026-08-11-web-export-command-and-dialog.i18n.yaml b/.agents/notes/archived/feature/2026-08-11-web-export-command-and-dialog.i18n.yaml new file mode 100644 index 0000000000..da756e7493 --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-11-web-export-command-and-dialog.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-11-web-export-command-and-dialog.md: aba9048f26237ea01e861a5ee6e31b89429629c8 +2026-08-11-web-export-command-and-dialog.zh.md: be4a3a53b1ff75cfbe176e258fb6a73e5abfee22 diff --git a/.agents/notes/implemented/feature/2026-08-11-web-export-command-and-dialog.md b/.agents/notes/archived/feature/2026-08-11-web-export-command-and-dialog.md similarity index 99% rename from .agents/notes/implemented/feature/2026-08-11-web-export-command-and-dialog.md rename to .agents/notes/archived/feature/2026-08-11-web-export-command-and-dialog.md index d28925500a..aba9048f26 100644 --- a/.agents/notes/implemented/feature/2026-08-11-web-export-command-and-dialog.md +++ b/.agents/notes/archived/feature/2026-08-11-web-export-command-and-dialog.md @@ -1,6 +1,7 @@ # Agent Note: Web `/export` shares the streamed Session ZIP download Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-11-web-export-command-and-dialog.zh.md) diff --git a/.agents/notes/implemented/feature/2026-08-11-web-export-command-and-dialog.zh.md b/.agents/notes/archived/feature/2026-08-11-web-export-command-and-dialog.zh.md similarity index 99% rename from .agents/notes/implemented/feature/2026-08-11-web-export-command-and-dialog.zh.md rename to .agents/notes/archived/feature/2026-08-11-web-export-command-and-dialog.zh.md index a9a9c4a5cc..be4a3a53b1 100644 --- a/.agents/notes/implemented/feature/2026-08-11-web-export-command-and-dialog.zh.md +++ b/.agents/notes/archived/feature/2026-08-11-web-export-command-and-dialog.zh.md @@ -1,6 +1,7 @@ # Agent Note: Web `/export` 共用流式 Session ZIP 下载 Status: implemented +Archived: 2026-08-22 [English](2026-08-11-web-export-command-and-dialog.md) | 中文 diff --git a/.agents/notes/archived/feature/2026-08-18-web-home-path-tilde.i18n.yaml b/.agents/notes/archived/feature/2026-08-18-web-home-path-tilde.i18n.yaml new file mode 100644 index 0000000000..d001e4a95c --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-18-web-home-path-tilde.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-18-web-home-path-tilde.md: 108674740c804a42ec3b0491505186215a9d9fcd +2026-08-18-web-home-path-tilde.zh.md: 1f742158f62b9b7fbb8eae8be35c13adc95f3a09 diff --git a/.agents/notes/archived/feature/2026-08-18-web-home-path-tilde.md b/.agents/notes/archived/feature/2026-08-18-web-home-path-tilde.md new file mode 100644 index 0000000000..108674740c --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-18-web-home-path-tilde.md @@ -0,0 +1,38 @@ +# Agent Note: Web UI abbreviates POSIX home paths as `~` + +Status: implemented +Archived: 2026-08-22 + +English | [中文](2026-08-18-web-home-path-tilde.zh.md) + +## Problem + +Workspace hover cards and Tool call summaries showed full POSIX home paths. Those strings are long, repeat the same prefix on every row, and make the sidebar and transcript harder to scan. Windows paths must stay verbatim because `~` is not a Windows filesystem convention. + +## Decision + +`host.describe` reports the host account `home` as a required field. Client and Host ship together, so the field is required rather than optional. ApiProxy fills it from `homedir()` at describe time. + +`abbreviateHomePath` in `dsh-client-runtime` is the display-only helper. It returns `~` or `~/…` when the path is the POSIX home or a descendant, and leaves the path unchanged when `home` is missing, empty, or `/`, when either value is a Windows drive or UNC path, or when the match is only a prefix (`/Users/u` does not claim `/Users/u2`). Tool summaries run workspace-relative shortening first, then this helper, so a path inside the session cwd stays short. `filePath`, Host open, and Workspace hover copy keep the authored filesystem path. + +`ui-tool` and `ui-workspace` inject `connection.hostDescription` at their own slot registrations. ChatView does not grow a Host-description hook. The field is required on `ConnectionHandle`; test fakes supply a source whose snapshot may be undefined before connect. + +The fixture Host home is `/home/fixture`. A second fixture Workspace at `/home/fixture/Documents/project` lets assembled replay hover `~/Documents/project` without moving the existing `/tmp/fixture` account. TerminalBlock's own prompt-label collapse is unchanged. + +## Alternatives considered + +**Guess `/Users` or `/home` without the real home.** Rejected because a shared prefix is not an account home, and `/Users/shared` or `/home/src` would abbreviate incorrectly. + +**Abbreviate Windows `%USERPROFILE%` as `~` as well.** Rejected because the acceptance rule keeps Windows paths verbatim, and `~` is not how Explorer or `cmd` spell those paths. + +**Put the helper in `dsh-home-paths`.** Rejected because that package expands configuration tildes on Node; this helper is a browser display rewrite and must not pull Node `os` into client bundles. + +**Thread `home` from ChatView owner props.** Rejected because it enlarges the conversation inject face and every ChatView test harness for a display fact only Tool and Workspace cards consume. + +## Consequences + +POSIX home-rooted Workspace hover paths and leftover Tool path summaries display as `~`. Copy and open still use the full path. Windows drive and UNC paths never become `~`. A Host that reports `/` as home does not turn the whole filesystem into `~`. Before the first describe, or while reconnecting, the source snapshot is undefined and paths stay unabbreviated. + +## Testing + +Package tests cover `abbreviateHomePath`, `toolRowModel` / `readCardModel` home abbreviation, Workspace hover display versus copy, and `host.describe` schema plus live `homedir()`. Assembled replay `apps/web/tests/home-path-tilde.snapshot.ts` hovers the fixture home-descendant Workspace. Product-GUI PRs still record a real-browser GIF of the hover card. diff --git a/.agents/notes/archived/feature/2026-08-18-web-home-path-tilde.zh.md b/.agents/notes/archived/feature/2026-08-18-web-home-path-tilde.zh.md new file mode 100644 index 0000000000..1f742158f6 --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-18-web-home-path-tilde.zh.md @@ -0,0 +1,38 @@ +# Agent Note: Web UI abbreviates POSIX home paths as `~` + +Status: implemented +Archived: 2026-08-22 + +[English](2026-08-18-web-home-path-tilde.md) | 中文 + +## Problem + +Workspace 悬停卡片和 Tool 调用摘要会显示完整的 POSIX 家目录路径。这些字符串很长,每行重复同一前缀,侧边栏和对话记录更难扫读。Windows 路径必须保持原样,因为 `~` 不是 Windows 文件系统约定。 + +## Decision + +`host.describe` 把宿主账户的 `home` 作为必填字段上报。Client 与 Host 一同发布,因此该字段是必填而不是可选。ApiProxy 在 describe 时用 `homedir()` 填入。 + +`dsh-client-runtime` 中的 `abbreviateHomePath` 是仅用于展示的辅助函数。当路径是 POSIX 家目录或其后代时返回 `~` 或 `~/…`;`home` 缺失、为空或为 `/`,任一侧是 Windows 盘符或 UNC 路径,或只是前缀命中(`/Users/u` 不能收走 `/Users/u2`)时,路径保持不变。Tool 摘要先做工作区相对缩短,再调用该辅助函数,因此会话 cwd 内的路径仍然更短。`filePath`、Host 打开以及 Workspace 悬停复制仍使用作者给出的文件系统路径。 + +`ui-tool` 与 `ui-workspace` 在各自的 slot 注册上注入 `connection.hostDescription`。ChatView 不增加 Host 描述钩子。该字段在 `ConnectionHandle` 上是必填的;测试假对象提供一个来源,其快照在连接完成前可以为 undefined。 + +fixture 的 Host 家目录是 `/home/fixture`。第二个 fixture Workspace 位于 `/home/fixture/Documents/project`,组装回放可以悬停出 `~/Documents/project`,而不必移动现有的 `/tmp/fixture` 账户。TerminalBlock 自有的提示符标签折叠保持不变。 + +## Alternatives considered + +**在没有真实 home 的情况下猜测 `/Users` 或 `/home`。** 否决,因为共享前缀不是账户家目录,`/Users/shared` 或 `/home/src` 会被错误缩写。 + +**同样把 Windows `%USERPROFILE%` 缩写成 `~`。** 否决,因为验收规则要求 Windows 路径保持原样,而且 Explorer 与 `cmd` 并不这样拼写这些路径。 + +**把辅助函数放进 `dsh-home-paths`。** 否决,因为该包在 Node 上展开配置里的波浪号;本辅助函数是浏览器展示改写,不能把 Node `os` 拉进 client 包。 + +**从 ChatView owner props 向下传递 `home`。** 否决,因为它会扩大 conversation 注入面和每一份 ChatView 测试夹具,而只有 Tool 与 Workspace 卡片消费这个展示事实。 + +## Consequences + +POSIX 家目录下的 Workspace 悬停路径,以及缩短 cwd 后仍落在家目录里的 Tool 路径摘要,会显示为 `~`。复制与打开仍使用完整路径。Windows 盘符和 UNC 路径永远不会变成 `~`。若 Host 把 `/` 报成 home,不会把整个文件系统收成 `~`。首次 describe 之前或重连期间,来源快照为 undefined,路径保持未缩写。 + +## Testing + +包测试覆盖 `abbreviateHomePath`、`toolRowModel`/`readCardModel` 的家目录缩写、Workspace 悬停展示与复制,以及 `host.describe` schema 与实时 `homedir()`。组装回放 `apps/web/tests/home-path-tilde.snapshot.ts` 悬停 fixture 中位于家目录下的 Workspace。面向产品 GUI 的 PR 仍需录制悬停卡片的真实浏览器 GIF。 diff --git a/.agents/notes/archived/feature/2026-08-19-high-cache-hit-decimal-display.i18n.yaml b/.agents/notes/archived/feature/2026-08-19-high-cache-hit-decimal-display.i18n.yaml new file mode 100644 index 0000000000..fbbf621db9 --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-19-high-cache-hit-decimal-display.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-19-high-cache-hit-decimal-display.md: 83c031bd1049ec26029d2e9ba0dc5cd623c3f867 +2026-08-19-high-cache-hit-decimal-display.zh.md: 62f27e39d37cf2445ac6796030092e892d82b581 diff --git a/.agents/notes/archived/feature/2026-08-19-high-cache-hit-decimal-display.md b/.agents/notes/archived/feature/2026-08-19-high-cache-hit-decimal-display.md new file mode 100644 index 0000000000..83c031bd10 --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-19-high-cache-hit-decimal-display.md @@ -0,0 +1,51 @@ +# Agent Note: High cache-hit decimal display + +Status: implemented +Archived: 2026-08-22 + +English | [中文](2026-08-19-high-cache-hit-decimal-display.zh.md) + +## Problem + +The Web conversation stats line rounded every non-empty cache-hit ratio to an integer. Once the actual ratio passed 99%, the display hid further progress, and a ratio of at least 99.5% appeared as 100% even while uncached input or cache writes remained. + +Users therefore could not distinguish a nearly complete cache hit from a true full hit. + +## Decision + +`StatsLine` continues to derive the ratio from the whole-session `tokenUsage` projection owned by `@deepseek-ai/dsh-token-meter`; the projection remains the only owner of the uncached-input, cache-read, cache-write, and output counts ([projection decision](../architecture/2026-07-29-projected-token-usage-and-request-context.md)). The presentation layer changes only the text inserted into the existing `stats.cacheHit` locale template. + +| Actual ratio | Display | +|---|---| +| No billed input | Cache-hit group omitted | +| Integer rounding is below 100% | Rounded integer | +| Non-full ratio whose current rounding is 100% | Minimum decimal precision whose rounded result is below 100% | +| 100% | `100%` | + +Every non-empty ratio starts at zero decimal places. A non-full ratio increases precision one place at a time only while rounding would produce 100%, so `99.1%` and `99.49%` remain `99%`, while `99.5%`, `99.95%`, and `99.995%` retain one, two, and three decimal places respectively. `StatsLine` uses exact small-factor comparisons over the safe-integer token counts, then scales the near-full gap only while the intermediate remains within that range. This avoids floating-point tie errors without imposing a precision cap or substitute label. A full hit does not carry a redundant decimal. The same derived string feeds the inline row and its overflow tooltip. + +## Ownership and lifecycle + +Token-meter continues to fold usage from the complete durable session log. `StatsLine` performs a synchronous display derivation whenever the standard projection value changes. It introduces no setting, stored percentage, event, wire field, client state, or recovery path. + +Live updates, reload replay, and reconnect recovery all restore the same `tokenUsage` counts and run the same display function. A missing projection still omits every token group, and a zero input denominator still omits only the cache-hit group. + +## Verification + +The component spec pins the zero denominator, ordinary integer rounding, half-step rounding at several decimal precisions, each precision boundary through three decimal places, a near-full cumulative sample that needs fourteen decimal places, the true `100%` result, both locales, and equality between inline and tooltip values. The assembled `lifecycle-chrome` replay sidecar selects `9,950 / 10,000 = 99.5%` as a deterministic ratio that integer rounding would misreport as 100% while the base session fixture remains recordable; the live assertion and post-reload browser snapshot both display `99.5%` without another model call. + +## Alternatives considered + +**Keep integer rounding for every ratio.** Rejected because it hides all movement above 99% and still reports some non-full hits as 100%. + +**Truncate the high band to one decimal.** Rejected because `99.95%`, `99.995%`, and still closer ratios all collapse to `99.9%` instead of retaining the minimum precision that distinguishes them from a full hit. + +**Cap precision and use a substitute such as `<100%`.** Rejected because the exact cumulative counts can produce the required numeric result, and a cap would make display behavior depend on an arbitrary presentation limit. + +**Show one decimal at every ratio.** Rejected because the additional low-band motion adds noise and changes the established display where integer precision is sufficient. + +**Persist a display percentage in token-meter.** Rejected because the projection already carries the exact counts, while presentation precision belongs to the Web stats line. A second stored value would duplicate derivable state and expand replay and wire responsibilities. + +## Consequences + +High cache-hit sessions remain visually stable until integer rounding would falsely report a full hit, then expose only the decimal places needed to preserve that distinction. Extremely close non-full ratios can therefore produce long decimal strings; this is the accepted cost of having no arbitrary precision cap or nonnumeric fallback. Every delivery and recovery path stays on the existing durable projection lifecycle. diff --git a/.agents/notes/archived/feature/2026-08-19-high-cache-hit-decimal-display.zh.md b/.agents/notes/archived/feature/2026-08-19-high-cache-hit-decimal-display.zh.md new file mode 100644 index 0000000000..62f27e39d3 --- /dev/null +++ b/.agents/notes/archived/feature/2026-08-19-high-cache-hit-decimal-display.zh.md @@ -0,0 +1,51 @@ +# Agent Note: 高缓存命中率的小数显示 + +Status: implemented +Archived: 2026-08-22 + +[English](2026-08-19-high-cache-hit-decimal-display.md) | 中文 + +## 问题 + +Web 会话统计行会把所有非空缓存命中率舍入为整数。真实比率超过 99% 后,显示会隐藏后续提升;比率达到 99.5% 时,即使仍有未缓存输入或缓存写入,也会显示为 100%。 + +用户因此无法区分接近完整的缓存命中与真实满命中。 + +## 决策 + +`StatsLine` 继续从 `@deepseek-ai/dsh-token-meter` 所拥有的完整会话 `tokenUsage` 投影派生比率;该投影仍是未缓存输入、缓存读取、缓存写入与输出计数的唯一所有方([投影决策](../architecture/2026-07-29-projected-token-usage-and-request-context.zh.md))。展示层只改变插入现有 `stats.cacheHit` locale 模板的文本。 + +| 真实比率 | 显示结果 | +|---|---| +| 没有计费输入 | 省略缓存命中分组 | +| 整数舍入结果低于 100% | 舍入后的整数 | +| 当前舍入结果为 100% 的非满命中 | 舍入结果低于 100% 所需的最少小数位 | +| 100% | `100%` | + +所有非空比率都从零位小数开始。非满命中只有在舍入结果会成为 100% 时才逐位增加精度,因此 `99.1%` 与 `99.49%` 仍显示为 `99%`,而 `99.5%`、`99.95%` 与 `99.995%` 分别保留一位、两位与三位小数。`StatsLine` 对安全整数 token 计数执行精确的小因子比较,并且只在中间值仍处于该范围内时缩放接近满命中的差值。该算法既避开浮点临界值误差,也不设置精度上限或替代文案。真实满命中不会携带多余的小数。同一份派生字符串同时用于行内统计与溢出 tooltip。 + +## 归属与生命周期 + +token-meter 继续从完整持久会话日志折叠用量。标准投影值变化时,`StatsLine` 同步派生显示文本。本决策不引入设置、持久百分比、事件、协议字段、客户端状态或恢复路径。 + +实时更新、刷新回放与重连恢复都会还原同一组 `tokenUsage` 计数,并运行同一个显示函数。投影缺失时仍会省略全部 token 分组;输入分母为零时仍只省略缓存命中分组。 + +## 验证 + +组件测试固定了零分母、普通整数舍入、多个小数精度上的半步舍入、直至三位小数的各个精度边界、需要十四位小数的近满累计样本、真实 `100%`、两种 locale,以及行内值与 tooltip 值的一致性。组装后的 `lifecycle-chrome` replay sidecar 将 `9,950 / 10,000 = 99.5%` 选作确定性测试输入;该比率按整数舍入会误报为 100%,同时基础会话 fixture 仍可重录。活跃页面断言与刷新后的浏览器快照都会显示 `99.5%`,且不会产生额外模型调用。 + +## 备选方案 + +**对所有比率继续使用整数舍入。** 不予采纳,因为它会隐藏 99% 以上的全部变化,并继续把部分非满命中显示为 100%。 + +**把高位区间向下截取到一位小数。** 不予采纳,因为 `99.95%`、`99.995%` 以及更接近满命中的比率都会坍缩为 `99.9%`,无法保留区分真实满命中所需的最少精度。 + +**限制精度并使用 `<100%` 等替代文案。** 不予采纳,因为精确累计计数能够产生所需的数值结果,而精度上限会让显示行为依赖任意的展示限制。 + +**所有比率都显示一位小数。** 不予采纳,因为低位区间的额外变化会增加无效抖动,并改变整数精度已经足够的既有显示。 + +**在 token-meter 中持久化显示百分比。** 不予采纳,因为投影已经携带精确计数,而展示精度属于 Web 统计行。第二个持久值会复制可派生状态,并扩大回放与协议职责。 + +## 后果 + +高缓存命中率会保持稳定的整数显示,直到整数舍入会错误地报告满命中;此时界面只展示维持区分所需的小数位。极接近满命中的非满比率可能因此产生较长的小数字符串,这是不设置任意精度上限或非数值回退所接受的代价。所有交付与恢复路径继续沿用既有持久投影生命周期。 diff --git a/.agents/notes/archived/manifest.json b/.agents/notes/archived/manifest.json index f8786470c7..05fb2f3712 100644 --- a/.agents/notes/archived/manifest.json +++ b/.agents/notes/archived/manifest.json @@ -43,6 +43,9 @@ "architecture/2026-07-28-dsh-native-typescript-source-launch.i18n.yaml": "sha256:af071e07bce5d9bc8f3df65fed9dcd9b3779a98c5864badbd530363bda021b55", "architecture/2026-07-28-dsh-native-typescript-source-launch.md": "sha256:1b56e3454277ace713e2a01c4da538c756c45bf633fd24d7b16443d584afac5d", "architecture/2026-07-28-dsh-native-typescript-source-launch.zh.md": "sha256:8c0f97472c2c89d2c19ae5cfa68c6e67f32b50960b08b60b46496f78ea6ffad1", + "architecture/2026-08-11-plugin-settings-tabs.i18n.yaml": "sha256:0365da2b317fc5f94dd190064198565f4c624afc91d2e62161ab9170f79d11bc", + "architecture/2026-08-11-plugin-settings-tabs.md": "sha256:fdd92cfe55b6c4cd31b3f768dd46a2ecf129a04c9818249cbdd33857cf722bbf", + "architecture/2026-08-11-plugin-settings-tabs.zh.md": "sha256:8993df1a0178aba1ea35c460ee67c522900344a4b386287bba9dfac2bfb87efa", "bug-fix/2026-07-20-code-mode-result-card-completeness.i18n.yaml": "sha256:1035dae11d049d32ab09fd7d4f950eceae44bf46ba498b3cfaf3c75102b9fb64", "bug-fix/2026-07-20-code-mode-result-card-completeness.md": "sha256:6ca2c9d4df98be18813ef38b7462db880900b5bcd6944fbcd1b8f2258006b93e", "bug-fix/2026-07-20-code-mode-result-card-completeness.zh.md": "sha256:ed85fa7f935e5f525d566bc37a92014614983e649c75de9a9f244939097a7991", @@ -94,9 +97,18 @@ "bug-fix/2026-08-03-tui-long-session-render-costs.i18n.yaml": "sha256:f65f7bf8fc84c7a1f022ee393c8d969c06d9bde8bed3a0206de86fb35b246ac6", "bug-fix/2026-08-03-tui-long-session-render-costs.md": "sha256:6ecf2ef831f527f361ade18a882d79bc6eccf15cc676d05728e7753f41cde051", "bug-fix/2026-08-03-tui-long-session-render-costs.zh.md": "sha256:5f44e707b332e13fa06d625212173ea055c1c3c0aee60888435a0ff099ec6037", + "bug-fix/2026-08-04-large-history-pagination-call-stack.i18n.yaml": "sha256:9bb1ceec013521116ee73eb9ec28c5708ae9520d6e53dcd4a3621fd2283b215c", + "bug-fix/2026-08-04-large-history-pagination-call-stack.md": "sha256:38c5afd347b131abd6b73634d25210d593bcb1fd72c7ba501bad0b33fb639810", + "bug-fix/2026-08-04-large-history-pagination-call-stack.zh.md": "sha256:2a2790b3b3400c747e20b998edfa96788b4035ccfa5fd4100dbc9a0e694ed30e", + "bug-fix/2026-08-06-plan-narrow-viewport-regression.i18n.yaml": "sha256:fe0539da9ce4015c6deaf585350e586e99d86e0073a36e131b6f1f62cc13382b", + "bug-fix/2026-08-06-plan-narrow-viewport-regression.md": "sha256:ccecdf52213dd1f6ab9935db31906520b83a7d9166f612376877d612230d1331", + "bug-fix/2026-08-06-plan-narrow-viewport-regression.zh.md": "sha256:be10805f0cd5a4f0812c883ab7f3e1e9396b579be455696d5cd726434a26b3e1", "bug-fix/2026-08-10-web-favicon-dark-mode.i18n.yaml": "sha256:859c4399f9a017a68ba89552fdafa05e73c0599d94cee9551c84ea5b749a14f3", "bug-fix/2026-08-10-web-favicon-dark-mode.md": "sha256:4d17e247abd76ae3aed5fb4e075fd66a2838292f89f7021c82a79fe37ed905e6", "bug-fix/2026-08-10-web-favicon-dark-mode.zh.md": "sha256:7bbff8a3b7061c127afcc75cd2a8043b02a999b78c0180edd8f7e4807fcfe71d", + "bug-fix/2026-08-11-preset-card-description-clamp.i18n.yaml": "sha256:d50452503b59aa22c81617888f9391f31f12a779c4b18efb2b4b6de1bd9702a6", + "bug-fix/2026-08-11-preset-card-description-clamp.md": "sha256:7eb8db697f3ad3dea8c0a6045331c05730a010404a89e2b08348cb7b0fad26c8", + "bug-fix/2026-08-11-preset-card-description-clamp.zh.md": "sha256:6d2f7b55ce02275a45adfdd853805e7daf2d6534929b77beb86685a54fc34f85", "bug-fix/2026-08-12-collapsed-sidebar-shared-entry-motion.i18n.yaml": "sha256:3ce4f6e39e173fc304bf64deca9c95bcddc1dbb492e065ca8c267a7a40788588", "bug-fix/2026-08-12-collapsed-sidebar-shared-entry-motion.md": "sha256:7b169aa4543edfc965de5a8b7b9e60aa9d9d5218693cd0b57908e2d482280723", "bug-fix/2026-08-12-collapsed-sidebar-shared-entry-motion.zh.md": "sha256:88db36c698800bf55c3c7531d6f92665576d978c29c15ff7d74215fb93376cb1", @@ -241,6 +253,9 @@ "feature/2026-07-30-tui-details-command.i18n.yaml": "sha256:033cea6df0a16fc68cbdb435babdc6e75c1199a8e70e1a71d87c800c40f5a044", "feature/2026-07-30-tui-details-command.md": "sha256:a13478d4e55ec6d358209b51b541413ec75d0e20dfc22196ace28020f03f0c2d", "feature/2026-07-30-tui-details-command.zh.md": "sha256:de9c449b98468cef34ce4f9a9d2a854a5d8905eecd61f80e27a9a0e4495e9901", + "feature/2026-07-30-versioned-gui-welcome-onboarding.i18n.yaml": "sha256:3d453f1a8f1a642ed569d1900009b785e582614bcabf9fede566ecbd9842e3ef", + "feature/2026-07-30-versioned-gui-welcome-onboarding.md": "sha256:cfaa38cfec722ac3792a4770f7733372805a6c0571f4f08519f367976b0d2379", + "feature/2026-07-30-versioned-gui-welcome-onboarding.zh.md": "sha256:0ce0e4616580c583725aa3d28063dc009889d7f0a260a3cdda53ff48721c1ae4", "feature/2026-07-30-versioned-tui-first-run-welcome.i18n.yaml": "sha256:4c3fc380b0512ad7c00baacd0ac610e1a78ae45374311d9bd43bab6b5e29e630", "feature/2026-07-30-versioned-tui-first-run-welcome.md": "sha256:296f153e6c839f3743078e4f5aab3b2befc211c934835238668c57bdeae52231", "feature/2026-07-30-versioned-tui-first-run-welcome.zh.md": "sha256:82871a9cca1fec46bb08a5b39daad28a44bb2419dea367b4ae41af3cf07bfa65", @@ -259,9 +274,30 @@ "feature/2026-07-31-web-cards-toolrow.i18n.yaml": "sha256:f9a6ab72a77934cdcc02167c7313f08d7e9925362017b34bed7ad56c8c70fbaa", "feature/2026-07-31-web-cards-toolrow.md": "sha256:5058f7cec4497d1cb0a5c8e77b88fddacac6eead034f3edec88e8514919b8a3e", "feature/2026-07-31-web-cards-toolrow.zh.md": "sha256:ba84ef2e1be61211ab5ba6950b78ede3d3a979f252bc068d3e04e2c025f7bc03", + "feature/2026-08-06-bundled-dsh-badge-skill.i18n.yaml": "sha256:4b568d89976a71b7b3864e13b36925bf055479213739ffd4ac81614e00e93e36", + "feature/2026-08-06-bundled-dsh-badge-skill.md": "sha256:7b67f7c09b7e2b2ca756983a8951dad3a15786b8cd3adc6e819f316c67d31b2c", + "feature/2026-08-06-bundled-dsh-badge-skill.zh.md": "sha256:dcc0acb2dca596196ac034a8e86a644131c5b7fb09b184ec9d8493f93019d2b1", + "feature/2026-08-07-workspace-picker-composer-entry.i18n.yaml": "sha256:24a8bb2956371c7c840a662ac16dffbe04a6bb40a7296d01db86cb85da58d238", + "feature/2026-08-07-workspace-picker-composer-entry.md": "sha256:036212fbae6f5d7194e8c7fc9b1e7cd1c35251e9c227e895834a7d00bd5f69f8", + "feature/2026-08-07-workspace-picker-composer-entry.zh.md": "sha256:fb65c3330e8324f90d1270345e1ac941fc800e8caaf3b4bbee1bbb743f713262", "feature/2026-08-08-dsh-run-headless-command.i18n.yaml": "sha256:1c2b4c5b61b9263b6267275d6fc69faeaad3cc887f0728a7ed4172d817af812b", "feature/2026-08-08-dsh-run-headless-command.md": "sha256:7695fe7fd322377d5986f14e35f13337f4cd376405c758218a81230f6d182d1c", "feature/2026-08-08-dsh-run-headless-command.zh.md": "sha256:113c14a36c64d2facc8ae46f37c7aa76359d8cacb9c18fcba26a723f15d036fb", + "feature/2026-08-10-creator-guidance-introduce-cue.i18n.yaml": "sha256:74f519839f0cf82c7304bdeae41ae1cab8bb930bfb94f79ab44708acd3b72128", + "feature/2026-08-10-creator-guidance-introduce-cue.md": "sha256:3e25409dda498de150de18943ee332e1760963e377a40a365902b66f667fdc9f", + "feature/2026-08-10-creator-guidance-introduce-cue.zh.md": "sha256:203847010cab9e9d13c3969f17921d3a5aa0d69377eccfef555c7ff96572f162", + "feature/2026-08-11-collapsible-ask-user-question-card.i18n.yaml": "sha256:9c0873bbb1437bcd5025f5859e1dc447a6b936f19c2c2ad2250521a3aa773a12", + "feature/2026-08-11-collapsible-ask-user-question-card.md": "sha256:4f3b3f5d7020fefbbac8a3c36a97642721ef14a0476a7d8117bde8a128d25f42", + "feature/2026-08-11-collapsible-ask-user-question-card.zh.md": "sha256:e7186c92f77d337a875f981ae39b16331a1c5466a948415bcacfe108ad98fb63", + "feature/2026-08-11-web-export-command-and-dialog.i18n.yaml": "sha256:db7d523a2a1f82a86f532661bd2953ee8538d971d91f886e4bd4e0d88f7226b2", + "feature/2026-08-11-web-export-command-and-dialog.md": "sha256:ec44b47589ca7924018dc24f7fa73379a97b8f053d9e8ccce2aebb600230e47b", + "feature/2026-08-11-web-export-command-and-dialog.zh.md": "sha256:ad28e67d397c87300cfe1705ba3d206cc4d054e07f5647c095c718ac8cf4ec98", + "feature/2026-08-18-web-home-path-tilde.i18n.yaml": "sha256:f151e3e3514f59784fc646c2feb3075dc954c65110d48c2cc482ad486fc0b86f", + "feature/2026-08-18-web-home-path-tilde.md": "sha256:8c7ecf120ff8c81826160acab5fc906a2a0a14213bcd2958343cfea47328d68e", + "feature/2026-08-18-web-home-path-tilde.zh.md": "sha256:3486c5b42aed5bcadf12c62c5e1e6cf7c1b493fc1085ad7d154cdf2ec34076cc", + "feature/2026-08-19-high-cache-hit-decimal-display.i18n.yaml": "sha256:c2cb839ed676040ed62153c2aab65b677fa59739f69253af50246e79a2347620", + "feature/2026-08-19-high-cache-hit-decimal-display.md": "sha256:08cb68bfc379da47a05b816afac26126146d36d6248350d4380f7cb98607d573", + "feature/2026-08-19-high-cache-hit-decimal-display.zh.md": "sha256:9d7afe3e2fc3029fbccc643b432a01bcb3b5671750adb6945ac414ec843ca063", "process/2026-06-11-doc-sync-enforcement.i18n.yaml": "sha256:33b6d5874427bd7a2bd82e7e2f4f482b12448b2464aef15a9c57975edb48554d", "process/2026-06-11-doc-sync-enforcement.md": "sha256:aa2fe83d519fc30d48dff19e596e83c8922aacc9e063e14fe2cc35b769b9100e", "process/2026-06-11-doc-sync-enforcement.zh.md": "sha256:698017bd35f030fdea3eac51df9e43138c48140f504739d687b7251d13fced2b", @@ -322,6 +358,9 @@ "process/2026-08-08-review-driven-issue-lifecycle-triggers.i18n.yaml": "sha256:4c28c59d3fc323e7cd01eff31f1fe759834719c5bede1e82b39f868970bf856d", "process/2026-08-08-review-driven-issue-lifecycle-triggers.md": "sha256:1b0514de5d030170e91e12e4d6ba788a9247f840e82700faa385a1c0c76ab857", "process/2026-08-08-review-driven-issue-lifecycle-triggers.zh.md": "sha256:028d78d61f603d8bac64c4cce20b393a78f8e029d3bb4976e79a47ecaefa6032", + "process/2026-08-12-documentation-site-navigation-and-chrome.i18n.yaml": "sha256:dde0041399b253e3758045f0858488db8178ffc563ce889c8b396c87af6c3730", + "process/2026-08-12-documentation-site-navigation-and-chrome.md": "sha256:56cb836ed862378afd33eb5c1a9dc159958b35a0aed3bf4336fcf26ab0b84b8b", + "process/2026-08-12-documentation-site-navigation-and-chrome.zh.md": "sha256:f2dd4adde38a09fe312866a1e6dad0f465684d809287862f40f1a488acd4fe18", "simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.i18n.yaml": "sha256:ad3d1263cb0051b885173bf064de62065e2c646ccaae2d7250723da3b4eab90c", "simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.md": "sha256:8fb061d51c8c23b47d2367814bab3623c6d5b972f38d207a273caa9030b579bd", "simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.zh.md": "sha256:2ffeaca91f82844a5616d6dcce6b4af514bb8a7c46f78e47f668b204ac6edc04", @@ -334,6 +373,9 @@ "simplification/2026-07-02-remove-stream-chunk-mirror.i18n.yaml": "sha256:eef600eafd70a576b2ac16a74f5dd5010ee601376008a90be10f1da56d746cee", "simplification/2026-07-02-remove-stream-chunk-mirror.md": "sha256:0c22a896260c6eb8991cc8babd8172f7b6889fc41bb891d748f34650b3eee5ec", "simplification/2026-07-02-remove-stream-chunk-mirror.zh.md": "sha256:ae7a3c2450b16fdf2f8da9e1f83cd987bf387671ecc8ebb76d2e7541695e7ee1", + "simplification/2026-07-04-drop-image-content-block.i18n.yaml": "sha256:6af8bc85be1fc23b445b17ff96df69488afa82c7194fcb47beb7a6b7b5a1de6a", + "simplification/2026-07-04-drop-image-content-block.md": "sha256:31e619d09405044a4db4951084f8d12417868b8f255ca26a44af0a5a13d43e6b", + "simplification/2026-07-04-drop-image-content-block.zh.md": "sha256:8fa6558bbb014a38853796102099ffbd5d8151371d68d40b2bf262ad07f4c976", "simplification/2026-07-04-drop-inert-request-knobs.i18n.yaml": "sha256:e4c992a27ae0e37e5ef663c2cddf55eefe20387fd6103bebf655834d8e75e9db", "simplification/2026-07-04-drop-inert-request-knobs.md": "sha256:8735c2b868a85b13235e0491a0fa7b9570dd090eef5170324fc5e93782687b67", "simplification/2026-07-04-drop-inert-request-knobs.zh.md": "sha256:78b243f5d580f2a6fbbdb7d26574295d6ed74feb8d9bba34bbcdf4aa87624b5c", @@ -403,6 +445,18 @@ "simplification/2026-08-03-explicit-config-dsh-entrypoint.i18n.yaml": "sha256:5466161f3fb8f2e8117fe8ff242675cc9fe9ef264d1e29b9bc586891c73c051a", "simplification/2026-08-03-explicit-config-dsh-entrypoint.md": "sha256:f23accae7d05c2e75cb73ec69b492307f1ce7526ecfa9f6b12a621e02fd1a0c3", "simplification/2026-08-03-explicit-config-dsh-entrypoint.zh.md": "sha256:a32d2c6ecf748a16a2c35b59cd2da2fda75769e3ab24be6a2e026d8655466db4", + "simplification/2026-08-11-cmdline-program-action.i18n.yaml": "sha256:e33b6dee66e23beabf03275e4e4f15134d23a82740ae7be6afd28c11c3163fca", + "simplification/2026-08-11-cmdline-program-action.md": "sha256:e6a274bd92a35c98ea24704161b408507876de3162c0f640b4bc70a86ab4d86f", + "simplification/2026-08-11-cmdline-program-action.zh.md": "sha256:bd213ad65ea6129c5360f28b2f52e6f3e224a58d07f56da190702939e7b402ee", + "simplification/2026-08-11-quickstart-documentation-home.i18n.yaml": "sha256:548c0ff16d40fed3b3318b0b9a26e11d53a60582ff457098a212fc64e7d67eac", + "simplification/2026-08-11-quickstart-documentation-home.md": "sha256:21946a828417aca4a214a874a35e88fe5a3e5989c330421f3849937b35bb9a9b", + "simplification/2026-08-11-quickstart-documentation-home.zh.md": "sha256:cb292a428d427cf36aff0a184347f0ab653331edb874daf3c5b58a0d1f8e6964", + "simplification/2026-08-13-remove-first-run-beta-notice.i18n.yaml": "sha256:51267b74e39544991bfe606e3f749a26914162e454cf357f26223a6ed8de5fad", + "simplification/2026-08-13-remove-first-run-beta-notice.md": "sha256:7ef5c712b8dff1152becee6a3f800acd5f7589d7b000f01544f57b175660bcc1", + "simplification/2026-08-13-remove-first-run-beta-notice.zh.md": "sha256:f899c79f838b97d1eea4a118de2cf00a97bae910d86d4aabdc447b4e02fb585f", + "simplification/2026-08-19-knip-config-cleanup.i18n.yaml": "sha256:ca8f5726aed57ce376c3fbd8b70113e235f3ba37dbf290683157fb4bf143ab13", + "simplification/2026-08-19-knip-config-cleanup.md": "sha256:18c61713b3358d3019dac096afc26ce8e5189002f626b25e858dfc5e6f626c8d", + "simplification/2026-08-19-knip-config-cleanup.zh.md": "sha256:b8ff16089c1a331603bb80278544c637cb16c1fa3bcfca3c5e1187152a1a0947", "testing/2026-06-20-remove-redundant-snapshot-log-expected-output.i18n.yaml": "sha256:4177012c0821a8c22499852ecdf096af56d7263cb91c5d9d1bcd552cc26a3e00", "testing/2026-06-20-remove-redundant-snapshot-log-expected-output.md": "sha256:45234e7cc04b6010c6141f8d5924c04547300098f96262d423c50108e7c7011a", "testing/2026-06-20-remove-redundant-snapshot-log-expected-output.zh.md": "sha256:15e5a4ad3dee0bb711480cabe45cd97ec37bbdba19c2c2b47d1e9c203b07a48b", diff --git a/.agents/notes/archived/process/2026-08-12-documentation-site-navigation-and-chrome.i18n.yaml b/.agents/notes/archived/process/2026-08-12-documentation-site-navigation-and-chrome.i18n.yaml new file mode 100644 index 0000000000..439c0c1d47 --- /dev/null +++ b/.agents/notes/archived/process/2026-08-12-documentation-site-navigation-and-chrome.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-12-documentation-site-navigation-and-chrome.md: f33f017d54bcbb3583f37be27dcd6c69952bc66a +2026-08-12-documentation-site-navigation-and-chrome.zh.md: 7f3ff5c829c561167d8e2475cd1c2adf050975be diff --git a/.agents/notes/implemented/process/2026-08-12-documentation-site-navigation-and-chrome.md b/.agents/notes/archived/process/2026-08-12-documentation-site-navigation-and-chrome.md similarity index 99% rename from .agents/notes/implemented/process/2026-08-12-documentation-site-navigation-and-chrome.md rename to .agents/notes/archived/process/2026-08-12-documentation-site-navigation-and-chrome.md index 03cd44b94f..f33f017d54 100644 --- a/.agents/notes/implemented/process/2026-08-12-documentation-site-navigation-and-chrome.md +++ b/.agents/notes/archived/process/2026-08-12-documentation-site-navigation-and-chrome.md @@ -1,6 +1,7 @@ # Agent Note: Documentation-site navigation and repository chrome Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-12-documentation-site-navigation-and-chrome.zh.md) diff --git a/.agents/notes/implemented/process/2026-08-12-documentation-site-navigation-and-chrome.zh.md b/.agents/notes/archived/process/2026-08-12-documentation-site-navigation-and-chrome.zh.md similarity index 99% rename from .agents/notes/implemented/process/2026-08-12-documentation-site-navigation-and-chrome.zh.md rename to .agents/notes/archived/process/2026-08-12-documentation-site-navigation-and-chrome.zh.md index d0972f909e..7f3ff5c829 100644 --- a/.agents/notes/implemented/process/2026-08-12-documentation-site-navigation-and-chrome.zh.md +++ b/.agents/notes/archived/process/2026-08-12-documentation-site-navigation-and-chrome.zh.md @@ -1,6 +1,7 @@ # Agent Note: 文档站导航与仓库 chrome Status: implemented +Archived: 2026-08-22 [English](2026-08-12-documentation-site-navigation-and-chrome.md) | 中文 diff --git a/.agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.i18n.yaml b/.agents/notes/archived/simplification/2026-07-04-drop-image-content-block.i18n.yaml similarity index 69% rename from .agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.i18n.yaml rename to .agents/notes/archived/simplification/2026-07-04-drop-image-content-block.i18n.yaml index f51aaf86b4..f718de7c8a 100644 --- a/.agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.i18n.yaml +++ b/.agents/notes/archived/simplification/2026-07-04-drop-image-content-block.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.md -2026-07-04-drop-image-content-block.md: 5d0357270973ac20234d8ce72c403f27ce2ad0e0 -2026-07-04-drop-image-content-block.zh.md: a13133059cdf4f6e6e8fe44e313a9795a57c10c9 +2026-07-04-drop-image-content-block.md: 362afbbe7c5c1a3b46b1e8abda3b0fec77d1a0e6 +2026-07-04-drop-image-content-block.zh.md: 07fbbd1f165887bf3305ccfa7ba137656ce0a99c diff --git a/.agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.md b/.agents/notes/archived/simplification/2026-07-04-drop-image-content-block.md similarity index 99% rename from .agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.md rename to .agents/notes/archived/simplification/2026-07-04-drop-image-content-block.md index 5d03572709..362afbbe7c 100644 --- a/.agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.md +++ b/.agents/notes/archived/simplification/2026-07-04-drop-image-content-block.md @@ -1,6 +1,7 @@ # Agent Note: Drop the `image` content block until a path can honor it Status: implemented +Archived: 2026-08-19 English | [中文](2026-07-04-drop-image-content-block.zh.md) diff --git a/.agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.zh.md b/.agents/notes/archived/simplification/2026-07-04-drop-image-content-block.zh.md similarity index 99% rename from .agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.zh.md rename to .agents/notes/archived/simplification/2026-07-04-drop-image-content-block.zh.md index a13133059c..07fbbd1f16 100644 --- a/.agents/notes/implemented/simplification/2026-07-04-drop-image-content-block.zh.md +++ b/.agents/notes/archived/simplification/2026-07-04-drop-image-content-block.zh.md @@ -1,6 +1,7 @@ # Agent Note: 移除 `image` 内容块,直到有路径能真正处理它 Status: implemented +Archived: 2026-08-19 [English](2026-07-04-drop-image-content-block.md) | 中文 diff --git a/.agents/notes/archived/simplification/2026-08-11-cmdline-program-action.i18n.yaml b/.agents/notes/archived/simplification/2026-08-11-cmdline-program-action.i18n.yaml new file mode 100644 index 0000000000..69735dfe18 --- /dev/null +++ b/.agents/notes/archived/simplification/2026-08-11-cmdline-program-action.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-11-cmdline-program-action.md: 96cbe2342eef90e68dece3c78b12a2de1bbea7c0 +2026-08-11-cmdline-program-action.zh.md: f5a9fea1447c78c9f099d3b54acd5b90a21aa503 diff --git a/.agents/notes/implemented/simplification/2026-08-11-cmdline-program-action.md b/.agents/notes/archived/simplification/2026-08-11-cmdline-program-action.md similarity index 99% rename from .agents/notes/implemented/simplification/2026-08-11-cmdline-program-action.md rename to .agents/notes/archived/simplification/2026-08-11-cmdline-program-action.md index 40c4dae1d3..96cbe2342e 100644 --- a/.agents/notes/implemented/simplification/2026-08-11-cmdline-program-action.md +++ b/.agents/notes/archived/simplification/2026-08-11-cmdline-program-action.md @@ -1,6 +1,7 @@ # Agent Note: parseCmdline runs the program's own commander action Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-11-cmdline-program-action.zh.md) diff --git a/.agents/notes/implemented/simplification/2026-08-11-cmdline-program-action.zh.md b/.agents/notes/archived/simplification/2026-08-11-cmdline-program-action.zh.md similarity index 85% rename from .agents/notes/implemented/simplification/2026-08-11-cmdline-program-action.zh.md rename to .agents/notes/archived/simplification/2026-08-11-cmdline-program-action.zh.md index 91036f1c52..f5a9fea144 100644 --- a/.agents/notes/implemented/simplification/2026-08-11-cmdline-program-action.zh.md +++ b/.agents/notes/archived/simplification/2026-08-11-cmdline-program-action.zh.md @@ -1,12 +1,13 @@ # Agent Note: parseCmdline 运行 program 自己的 commander action Status: implemented +Archived: 2026-08-22 [English](2026-08-11-cmdline-program-action.md) | 中文 ## Problem -`dsh-cmdline`([应用自有命令行](../architecture/2026-08-06-app-owned-command-line.md))的 `parseCmdline` 曾带着一个自造的回调:`CmdlinePlan = (program, ctx) => T`,在解析成功后于该适配器的 catch 之内调用,使 plan 的 `program.error(...)` 与 help/解析错误共用同一条退出路径;它还带有只被测试使用、类型不健全的默认值 `(() => ({}) as T)`,以及没有任何 plan 读取的 `ctx` 参数。这整条接缝复制了 commander 本就定义的席位:命令的 action 处理器在 `parse` 内部运行,从中抛出的 `program.error(...)` 与语法拒绝一样遵循 `exitOverride`。 +`dsh-cmdline`([应用自有命令行](../architecture/2026-08-06-app-owned-command-line.zh.md))的 `parseCmdline` 曾带着一个自造的回调:`CmdlinePlan = (program, ctx) => T`,在解析成功后于该适配器的 catch 之内调用,使 plan 的 `program.error(...)` 与 help/解析错误共用同一条退出路径;它还带有只被测试使用、类型不健全的默认值 `(() => ({}) as T)`,以及没有任何 plan 读取的 `ctx` 参数。这整条接缝复制了 commander 本就定义的席位:命令的 action 处理器在 `parse` 内部运行,从中抛出的 `program.error(...)` 与语法拒绝一样遵循 `exitOverride`。 ## Decision diff --git a/.agents/notes/archived/simplification/2026-08-11-quickstart-documentation-home.i18n.yaml b/.agents/notes/archived/simplification/2026-08-11-quickstart-documentation-home.i18n.yaml new file mode 100644 index 0000000000..9ad4e8aacb --- /dev/null +++ b/.agents/notes/archived/simplification/2026-08-11-quickstart-documentation-home.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-11-quickstart-documentation-home.md: 653c702eba4c90e52ee0539959d926ae23a98e6c +2026-08-11-quickstart-documentation-home.zh.md: 3d21566f9e4a822d095a115a5e65bfbaa3f947df diff --git a/.agents/notes/implemented/simplification/2026-08-11-quickstart-documentation-home.md b/.agents/notes/archived/simplification/2026-08-11-quickstart-documentation-home.md similarity index 99% rename from .agents/notes/implemented/simplification/2026-08-11-quickstart-documentation-home.md rename to .agents/notes/archived/simplification/2026-08-11-quickstart-documentation-home.md index 3fd98843fc..653c702eba 100644 --- a/.agents/notes/implemented/simplification/2026-08-11-quickstart-documentation-home.md +++ b/.agents/notes/archived/simplification/2026-08-11-quickstart-documentation-home.md @@ -1,6 +1,7 @@ # Agent Note: Route documentation roots to quick start Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-11-quickstart-documentation-home.zh.md) diff --git a/.agents/notes/implemented/simplification/2026-08-11-quickstart-documentation-home.zh.md b/.agents/notes/archived/simplification/2026-08-11-quickstart-documentation-home.zh.md similarity index 88% rename from .agents/notes/implemented/simplification/2026-08-11-quickstart-documentation-home.zh.md rename to .agents/notes/archived/simplification/2026-08-11-quickstart-documentation-home.zh.md index 2e3890f858..3d21566f9e 100644 --- a/.agents/notes/implemented/simplification/2026-08-11-quickstart-documentation-home.zh.md +++ b/.agents/notes/archived/simplification/2026-08-11-quickstart-documentation-home.zh.md @@ -1,6 +1,7 @@ # Agent Note: 将文档根路由指向快速开始 Status: implemented +Archived: 2026-08-22 [English](2026-08-11-quickstart-documentation-home.md) | 中文 @@ -12,7 +13,7 @@ Status: implemented 每个 locale 根路由都是重定向页面。`/` 将读者导向 `./guide/quickstart`,`/en/` 则把同一相对目标解析为 `/en/guide/quickstart`。当网站托管在源站的子路径下时,相对目标仍会保留配置的 `DOCS_BASE`。 -重定向由 `docs/user/index.md` 与 `docs/user/index.zh.md` 的 VitePress frontmatter 维护。对于 locale 首页,[文档网站投影器](../process/2026-07-13-documentation-site-projection.md)只发布这段 frontmatter,因此权威 Markdown 保留中英文语言切换行,且不会渲染第二个首页。投影器测试验证两个 locale 根路由都使用相对于各自 locale 的同一快速开始目标。 +重定向由 `docs/user/index.md` 与 `docs/user/index.zh.md` 的 VitePress frontmatter 维护。对于 locale 首页,[文档网站投影器](../process/2026-07-13-documentation-site-projection.zh.md)只发布这段 frontmatter,因此权威 Markdown 保留中英文语言切换行,且不会渲染第二个首页。投影器测试验证两个 locale 根路由都使用相对于各自 locale 的同一快速开始目标。 文档网站不承载产品定位和功能摘要。快速开始页面仍提供指南、开发、参考、搜索和 locale 导航。 diff --git a/.agents/notes/archived/simplification/2026-08-13-remove-first-run-beta-notice.i18n.yaml b/.agents/notes/archived/simplification/2026-08-13-remove-first-run-beta-notice.i18n.yaml new file mode 100644 index 0000000000..abe87647e3 --- /dev/null +++ b/.agents/notes/archived/simplification/2026-08-13-remove-first-run-beta-notice.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-13-remove-first-run-beta-notice.md: 535d0a20a5805c137551e6047f40fc5cf53153b8 +2026-08-13-remove-first-run-beta-notice.zh.md: 20626bbd9d0bd13c00d4ee66f5dbc267c2e92082 diff --git a/.agents/notes/implemented/simplification/2026-08-13-remove-first-run-beta-notice.md b/.agents/notes/archived/simplification/2026-08-13-remove-first-run-beta-notice.md similarity index 50% rename from .agents/notes/implemented/simplification/2026-08-13-remove-first-run-beta-notice.md rename to .agents/notes/archived/simplification/2026-08-13-remove-first-run-beta-notice.md index 69f47be445..535d0a20a5 100644 --- a/.agents/notes/implemented/simplification/2026-08-13-remove-first-run-beta-notice.md +++ b/.agents/notes/archived/simplification/2026-08-13-remove-first-run-beta-notice.md @@ -1,6 +1,7 @@ # Agent Note: Remove the first-run beta notice Status: implemented +Archived: 2026-08-22 English | [中文](2026-08-13-remove-first-run-beta-notice.zh.md) @@ -10,7 +11,7 @@ Every GUI first launch opened with a full-viewport internal-test statement (内 ## Decision -The first-run notice is removed from the assembled product rather than reworded. `ui-settings-general` seats no `settings.onboarding` step; the notice component, its durable acknowledgement store, its copy owner, and its locale keys are deleted. The `settings.onboarding` coordinator and its takeover stage stay ([ordered onboarding](../feature/2026-07-30-versioned-gui-welcome-onboarding.md)), and the conditional DeepSeek credential step is the only shipped occupant. The Host half still registers the `ui-onboarding` settings namespace: its `welcomeNoticeVersion` field keeps acknowledgements already stored in `$DSH_HOME/settings.yaml` valid, and nothing reads or writes it. Telemetry opt-in remains an explicit deployment environment choice documented in the [CLI reference README](../../../../apps/cli/reference/README.md); the product presents no prompt about enabling it. +This decision removed the first-run notice from the assembled product rather than rewording it. `ui-settings-general` seated no `settings.onboarding` step; the notice component, acknowledgement store, copy owner, and locale keys were deleted, while the Host kept the `ui-onboarding` namespace so stored documents remained valid. The later [shared-modal product onboarding](../feature/2026-08-13-shared-modal-product-onboarding.md) restores a new concise testing-stage notice in `ui-settings-models`, reusing that field and backend contract without restoring the removed takeover layout or telemetry instructions. Telemetry opt-in remains an explicit deployment environment choice documented in the [CLI reference README](../../../../apps/cli/reference/README.md); the restored notice says nothing about enabling it. ## Alternatives considered @@ -22,4 +23,4 @@ The first-run notice is removed from the assembled product rather than reworded. ## Consequences -A fresh profile boots into the credential step when the DeepSeek credential is missing and directly into the product otherwise; no full-viewport notice precedes either. The assembled onboarding scenario starts at the credential step, the remote-notice scenario is deleted with the feature, and the goal-bar fixture keeps the settings shell disabled because the fixture API client rejects settings traffic. Restoring a first-run notice requires a new onboarding registration and a new versioned field; the retained namespace does not resurrect the old acknowledgement semantics. +This removal eliminated the full-viewport notice and its telemetry copy. The later restoration is intentionally a different presentation and copy revision: a shared modal precedes the inline credential dialog, the remote scenario again covers process-local acknowledgement, and the existing `welcomeNoticeVersion` field records the new copy version. The historical telemetry prompt remains absent. diff --git a/.agents/notes/archived/simplification/2026-08-13-remove-first-run-beta-notice.zh.md b/.agents/notes/archived/simplification/2026-08-13-remove-first-run-beta-notice.zh.md new file mode 100644 index 0000000000..20626bbd9d --- /dev/null +++ b/.agents/notes/archived/simplification/2026-08-13-remove-first-run-beta-notice.zh.md @@ -0,0 +1,26 @@ +# Agent Note: 移除首次启动内测声明 + +Status: implemented +Archived: 2026-08-22 + +[English](2026-08-13-remove-first-run-beta-notice.md) | 中文 + +## 问题 + +GUI 每次首启都会先显示占满视口的内测声明:内部测试的定位表述,加上通过 `DSH_TELEMETRY_MODE` 开启 Session Log 上传的说明。会话遥测在 mode 未设置时已解析为 `DISABLED`([遥测默认关闭](../feature/2026-08-10-telemetry-default-off.zh.md)),因此引导流程中关于遥测的全部内容就是一段教用户如何开启的提示,而内部测试的定位表述本身也不应出现在发布版本里。 + +## 决策 + +本决策当时把首启声明从组装后的产品中整体移除,而不是改写。`ui-settings-general` 不再注册任何 `settings.onboarding` 步骤;声明组件、确认 store、文案所有者文件和 locale 键均被删除,Host 则保留 `ui-onboarding` namespace,使既有设置文档继续有效。后续的[共用弹窗产品引导](../feature/2026-08-13-shared-modal-product-onboarding.zh.md)在 `ui-settings-models` 中恢复了一份新的简洁测试阶段声明,复用该字段与后端契约,但不会恢复已移除的接管式布局或遥测说明。遥测的开启仍是显式的部署环境变量选择,记录在 [CLI reference README](../../../../apps/cli/reference/README.zh.md) 中;恢复后的声明不涉及如何开启遥测。 + +## 曾考虑的替代方案 + +**保留声明,只删除其中的遥测段落。** 不予采用:发布版本不应呈现的正是内部测试的定位表述本身,而一个没有实质内容的强制首启插页只剩下打扰。 + +**改为询问上传同意(版本化的同意步骤)。** 本次发布不予采用:首启询问是否开启上传仍然是一个遥测提示。未来的同意流程可以通过保持不变的 `settings.onboarding` seam 注册,并使用新的版本化字段做重新确认。 + +**连 `ui-onboarding` namespace 一起注销。** 不予采用:既有设置文档已经包含该分节,而设置 seam 会用已注册的 namespace 校验存储文档;保留注册就能让这些文档继续有效,且没有额外成本。 + +## 后果 + +这次移除消除了占满视口的声明及其遥测文案。后续恢复有意采用不同的展示与文案版本:共用弹窗先于行内凭据弹窗出现,远程场景重新覆盖进程内确认,既有 `welcomeNoticeVersion` 字段记录新的文案版本。历史上的遥测提示仍未恢复。 diff --git a/.agents/notes/archived/simplification/2026-08-19-knip-config-cleanup.i18n.yaml b/.agents/notes/archived/simplification/2026-08-19-knip-config-cleanup.i18n.yaml new file mode 100644 index 0000000000..6612299cc4 --- /dev/null +++ b/.agents/notes/archived/simplification/2026-08-19-knip-config-cleanup.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-08-19-knip-config-cleanup.md: 56426aeb7ff53caf7828b3f252269559e596940d +2026-08-19-knip-config-cleanup.zh.md: a74fa831f2301d9b95d5e34b003585293501c874 diff --git a/.agents/notes/archived/simplification/2026-08-19-knip-config-cleanup.md b/.agents/notes/archived/simplification/2026-08-19-knip-config-cleanup.md new file mode 100644 index 0000000000..56426aeb7f --- /dev/null +++ b/.agents/notes/archived/simplification/2026-08-19-knip-config-cleanup.md @@ -0,0 +1,32 @@ +# Agent Note: Deleted stale and duplicative knip.json workspace entries + +Status: implemented +Archived: 2026-08-22 + +English | [中文](2026-08-19-knip-config-cleanup.zh.md) + +## Problem + +`knip.json` carried workspace entries that did no work. Some pointed at packages that no longer exist, and some duplicated the `packages/*/*` glob default exactly. Both kinds made the file larger — 790 lines — and signaled a config that had outgrown the packages it described, so a reader could not tell which entries protected real behavior and which were inert. + +## Decision + +Deleted 15 `workspaces` entries: 2 stale keys naming packages absent from the working tree and from `HEAD`, and 13 entries whose `entry`/`project` values were byte-identical to the `packages/*/*` glob default. + +- Stale keys: `packages/util/home` (removed in `4a09d9b34d`, the harness-home resolver collapse) and `packages/client/web-ui` (no directory and no git history, an orphan key). knip 6.16 does not flag stale workspace keys — that stability check arrived in knip 6.18 — so these were inert config that only deleted when their packages disappeared. +- Glob-duplicate entries: `packages/host/webserver`, `packages/client/runtime`, `packages/core/tools`, `packages/context/tmux-context`, `packages/util/timeout`, `packages/util/output-retention`, `packages/goal/goal-round-driver`, `packages/goal/tool-goal`, `packages/util/home-paths`, `packages/fs/tool-fs-search`, `packages/client/ui-settings`, `packages/client/modules`, `packages/client/hmr`. Each declared exactly `entry: ["tests/**/*.spec.ts"]` and `project: ["src/**/*.ts", "tests/**/*.ts"]`, which equals the `packages/*/*` glob, and each package still exists, so the glob now covers it identically. + +The change is a deletion only: `knip.json` went from 790 to 655 lines with no behavioral change. `pnpm run knip` runs clean (zero issues, exit 0) before and after, because knip selects one workspace config per matched key (`getConfigKeyForWorkspace` uses specificity, not array merge), so a removed entry either lost an unresolvable target or fell back to an identical glob config. + +## Alternatives considered + +- Fold `zod` and other workspace-level `ignoreDependencies` up to the root. Rejected: the root `ignoreDependencies` is a repository-wide fallback, and these exemptions are deliberately workspace-scoped (the README of `cordis-host-runner` records why `src` cannot import the flagged dependency while the generated TypeRT face in `lib` needs it). Widening scope would mask a genuinely misplaced dependency in any future package. +- Upgrade knip to 6.18+ to get an automatic stale-workspace check. Deferred: 6.32.2 (latest at the time) re-flags many `@deepseek-ai/...` test dependencies as unused, i.e. it changes analysis semantics, not just adds hints. That is a separate dependency-upgrade decision with its own CI blast radius, not part of this cleanup. +- Keep the entries as documentation of intent. Rejected: an entry identical to the glob it sits under documents nothing beyond the glob itself, and a key naming an absent package actively misleads. + +## Consequences + +- `knip.json` is 135 lines shorter and names only packages that exist with config that differs from the glob default. +- Still-explicit entries (54) all carry a real reason to differ — an `e2e`/fixture/tsx `entry`, a `project` outside the default, or a workspace-scoped `ignoreDependencies`. +- knip 6.16 cannot itself detect the next stale key, so a package removal must still remember to drop its `knip.json` key; upgrading to 6.18+ (after the analysis-semantics change is separately assessed) restores that guard. +- This realizes the "never a restatement of the default stanza" criterion of the package-inventory proposal ([topic](../../proposed/process/2026-06-20-discover-package-inventory.md)); its remaining items — the e2e entry folding and the generated inventory — stay open there. diff --git a/.agents/notes/archived/simplification/2026-08-19-knip-config-cleanup.zh.md b/.agents/notes/archived/simplification/2026-08-19-knip-config-cleanup.zh.md new file mode 100644 index 0000000000..a74fa831f2 --- /dev/null +++ b/.agents/notes/archived/simplification/2026-08-19-knip-config-cleanup.zh.md @@ -0,0 +1,32 @@ +# Agent Note: 删除 knip.json 中失效与重复的 workspace 条目 + +Status: implemented +Archived: 2026-08-22 + +[English](2026-08-19-knip-config-cleanup.md) | 中文 + +## 问题 + +`knip.json` 携带了大量不产生任何作用的 workspace 条目。其中一些指向已经不复存在的包,另一些与 `packages/*/*` 通配默认完全重复。这两类都让文件变大——790 行——并显现出配置已经超出了它所描述的包:读者无法分辨哪些条目在保护真实行为、哪些是惰性的。 + +## 决策 + +删除了 15 个 `workspaces` 条目:2 个指向工作树与 `HEAD` 中都不存在的包的失效键,以及 13 个 `entry`/`project` 与 `packages/*/*` 通配默认逐字节相同的条目。 + +- 失效键:`packages/util/home`(在 `4a09d9b34d`,harness home 解析器的合并改动中删除)和 `packages/client/web-ui`(无对应目录、无 git 历史,是孤儿键)。knip 6.16 不会标记失效的 workspace 键——这项稳定性检查在 knip 6.18 才引入——所以这些是本应在包消失时一并删除、却残留的惰性配置。 +- 通配重复条目:`packages/host/webserver`、`packages/client/runtime`、`packages/core/tools`、`packages/context/tmux-context`、`packages/util/timeout`、`packages/util/output-retention`、`packages/goal/goal-round-driver`、`packages/goal/tool-goal`、`packages/util/home-paths`、`packages/fs/tool-fs-search`、`packages/client/ui-settings`、`packages/client/modules`、`packages/client/hmr`。每个都恰好声明了 `entry: ["tests/**/*.spec.ts"]` 和 `project: ["src/**/*.ts", "tests/**/*.ts"]`,与 `packages/*/*` 通配相等,且这些包仍然存在,因此通配现在以完全相同的方式覆盖它们。 + +本改动只做删除:`knip.json` 从 790 行降到 655 行,行为不变。`pnpm run knip` 在改动前后都干净通过(零问题、退出码 0),因为 knip 为每个已匹配的键选取一条 workspace 配置(`getConfigKeyForWorkspace` 按特定优先、不做数组合并),所以被删条目要么丢掉了无法解析的目标,要么回退到一个完全相同的通配配置。 + +## 备选方案 + +- 把 `zod` 及其它 workspace 级 `ignoreDependencies` 上提到根级。否决:根级 `ignoreDependencies` 是全仓库兜底,而这些豁免是刻意限定在 workspace 的(`cordis-host-runner` 的 README 记录了为什么 `src` 无法 import 被标记的依赖、而生成的 `lib` 里的 TypeRT 契约面需要它)。扩大作用域会掩盖未来任何包里真正放错位置的依赖。 +- 升级 knip 到 6.18+ 以获得自动的失效 workspace 检查。延后:撰写时的最新版 6.32.2 会把大量 `@deepseek-ai/...` 测试依赖重新标记为未使用——也就是改变了分析语义,而不仅是新增提示。那是独立的依赖升级决定,带自己的 CI 影响面,不属于本次清理。 +- 保留这些条目作为意图的文档。否决:与它挂在下面的通配完全相同的条目,除了通配本身外不记录任何东西;而指向不存在包的键确实会误导人。 + +## 结果 + +- `knip.json` 缩短了 135 行,并且只列出确实存在、且配置与通配默认有差异的包。 +- 仍然显式的条目(54 个)都带有真实的特例理由——`e2e`/fixture/tsx 的 `entry`、超出默认的 `project`、或 workspace 级的 `ignoreDependencies`。 +- knip 6.16 自身无法检测下一个失效键,因此删除包时仍须记得清理它的 `knip.json` 键;升级到 6.18+(在分析语义的改动被单独评估之后)会恢复这道守卫。 +- 本改动落实了包清单提案中「绝不复述默认 stanza」的标准([议题](../../proposed/process/2026-06-20-discover-package-inventory.zh.md));其剩余项——e2e 入口折叠与生成的清单——仍在提案中保持开放。 diff --git a/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.i18n.yaml index 0b150d8090..dac9ce6b5c 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.md -2026-06-11-content-block-vocabulary.md: 5228724bb9101307db9929aaf7831b477c2a6022 -2026-06-11-content-block-vocabulary.zh.md: b43de335917c6b5304a94b296fcb8253827b0600 +2026-06-11-content-block-vocabulary.md: a31df6a7d16ea7cba649702fdb474dab34533c1b +2026-06-11-content-block-vocabulary.zh.md: da387b179816cda64791e71ca7affa1fbdfd195b diff --git a/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.md b/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.md index 5228724bb9..a31df6a7d1 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.md +++ b/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.md @@ -22,7 +22,7 @@ In-session context injection (`context/message`) and mid-turn steering originall ## Consequences - Reasoning has a core home without provider-specific shapes. -- Multimodal blocks return only with coordinated adapter, UI, and compaction support; see [the drop-image Agent Note](../simplification/2026-07-04-drop-image-content-block.md). +- Multimodal blocks return only with coordinated adapter, UI, and compaction support; see [the drop-image Agent Note](../../archived/simplification/2026-07-04-drop-image-content-block.md). - Cache hints and assistant prefill remain absent until a shipping adapter can honor them; see the [producer-less variants](../../archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md) and [inert request knobs](../../archived/simplification/2026-07-04-drop-inert-request-knobs.md) Agent Notes. - Every adapter pays a translation cost; the first real adapters have since validated the streaming protocol, and new adapters should continue proving their provider-specific mapping in adapter-local tests. - IDs that cross package boundaries are branded (`CallId`, the shared agent/session `SessionId`) — nominal typing at zero runtime cost. diff --git a/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.zh.md b/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.zh.md index b43de33591..da387b1798 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-11-content-block-vocabulary.zh.md @@ -12,7 +12,7 @@ harness 需要一套统一的内部消息语言,供 agent loop(智能体循 自主拥有词汇:消息是类型化内容块的数组(`text`、`reasoning`、`tool-call`、`tool-result`),其联合类型派生自可合并扩展的 `ContentBlockMap`,插件通过声明合并添加新的块类型。同一可合并扩展映射模式为所有「字符串化」字段提供类型(`MessageSource`、`FinishReason`、`TurnTrigger`、`TurnEndReason`)。流式输出采用原始分片协议;`BlockAssembler` 是唯一的共享组装实现。适配器负责转换为提供方的协议格式(wire format)——映射成本留在适配器中,正是它该在的地方。 -会话内上下文注入(`context/message`)和轮次中途 steering(中途引导)最初渲染为带标签的 user-role 信封(system-reminder 模式),而非引入新角色,因此适配器无需承担额外负担。如今两者都投影为无包装的普通用户内容;见[注入内容信封 Agent Note](../simplification/2026-07-20-unwrap-injected-content-envelopes.md)。实际适配器验证已确认此渲染方式符合当前 DeepSeek 的行为;如果未来某提供方出现不兼容,应在该适配器内处理,而非引入新的规范角色。 +会话内上下文注入(`context/message`)和轮次中途 steering(中途引导)最初渲染为带标签的 user-role 信封(system-reminder 模式),而非引入新角色,因此适配器无需承担额外负担。如今两者都投影为无包装的普通用户内容;见[注入内容信封 Agent Note](../simplification/2026-07-20-unwrap-injected-content-envelopes.zh.md)。实际适配器验证已确认此渲染方式符合当前 DeepSeek 的行为;如果未来某提供方出现不兼容,应在该适配器内处理,而非引入新的规范角色。 ## 曾考虑的替代方案 @@ -22,7 +22,7 @@ harness 需要一套统一的内部消息语言,供 agent loop(智能体循 ## 后果 - 推理(reasoning)在核心层有了归属,无需依赖提供方特有的结构。 -- 多模态块只有在适配器、UI 和上下文压缩(context compaction)三方协同支持后才会回归;见 [drop-image Agent Note](../simplification/2026-07-04-drop-image-content-block.md)。 +- 多模态块只有在适配器、UI 和上下文压缩(context compaction)三方协同支持后才会回归;见 [drop-image Agent Note](../../archived/simplification/2026-07-04-drop-image-content-block.md)。 - 缓存提示与 assistant prefill 在有实际适配器能兑现之前保持缺席;见[无生产者的词汇变体](../../archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)与[无端到端可用路径的请求旋钮](../../archived/simplification/2026-07-04-drop-inert-request-knobs.md) Agent Note。 - 每个适配器都需承担翻译成本;首批真实适配器已验证了流式输出协议,新适配器应继续在适配器本地测试中验证其提供方特有的映射。 - 跨包边界的 ID 使用品牌类型(`CallId`、agent 与会话共享的 `SessionId`)——零运行时开销的名义类型。 diff --git a/.agents/notes/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.i18n.yaml index e5c589dbf9..44272f7ca1 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md 2026-06-11-dev-invariants-over-deep-readonly.md: 66980f1ee09c6112f72786d6c3a147aadbc57f6c -2026-06-11-dev-invariants-over-deep-readonly.zh.md: 0d694ea84e1b7c265491e9adb64757378b4185f1 +2026-06-11-dev-invariants-over-deep-readonly.zh.md: 576e53e0b27e65f6fa071ff649509223a7bc30ff diff --git a/.agents/notes/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.zh.md b/.agents/notes/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.zh.md index 0d694ea84e..576e53e0b2 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.zh.md @@ -32,7 +32,7 @@ TypeScript readonly 类型不是充分的运行时边界。它们在程序运行 ### 包拥有的不变式配套插件检查关系 -`dsh-invariants` 注册可配置的 `ctx.invariants` 服务,本身不包含产品检查。每个包发布一个 `./invariant` 所有权配套插件;`dsh-session`、`dsh-agent`、`dsh-scope` 和 `dsh-agent-loop` 目前添加需要跟踪状态或观察另一个 seam 的规则:单调递增的序列号、轮次与步骤嵌套、工具调用/结果配对、合法的 agent(智能体)状态转换、主体正确的作用域分发,以及循环构建的请求与从其会话日志前缀重建的请求之间的相等性。全局启用和包名 regex 过滤器归该服务所有(见[包拥有的不变式服务](2026-07-19-package-owned-invariant-service.md))。 +`dsh-invariants` 注册可配置的 `ctx.invariants` 服务,本身不包含产品检查。每个包发布一个 `./invariant` 所有权配套插件;`dsh-session`、`dsh-agent`、`dsh-scope` 和 `dsh-agent-loop` 目前添加需要跟踪状态或观察另一个 seam 的规则:单调递增的序列号、轮次与步骤嵌套、工具调用/结果配对、合法的 agent(智能体)状态转换、主体正确的作用域分发,以及循环构建的请求与从其会话日志前缀重建的请求之间的相等性。全局启用和包名 regex 过滤器归该服务所有(见[包拥有的不变式服务](2026-07-19-package-owned-invariant-service.zh.md))。 当会话配套插件附加到已有会话或以种子记录初始化的会话时,它回放不可变日志以重建跟踪状态。服务为每项贡献提供一个可 dispose(资源释放)的子 fiber,因此轮次中途热重载是安全的,同时不赋予诊断逻辑对会话存储的所有权。 diff --git a/.agents/notes/implemented/architecture/2026-06-11-event-sourced-sessions.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-11-event-sourced-sessions.i18n.yaml index 760c3fd099..868d215253 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-event-sourced-sessions.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-11-event-sourced-sessions.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-11-event-sourced-sessions.md 2026-06-11-event-sourced-sessions.md: b6d17d2db1d9b489f3d4224683b61e62be485014 -2026-06-11-event-sourced-sessions.zh.md: a78349ea2385b3c9d32870757bb607f15996196d +2026-06-11-event-sourced-sessions.zh.md: 11e5740dc2636fc002ed7158167b1dfbb39a971d diff --git a/.agents/notes/implemented/architecture/2026-06-11-event-sourced-sessions.zh.md b/.agents/notes/implemented/architecture/2026-06-11-event-sourced-sessions.zh.md index a78349ea23..11e5740dc2 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-event-sourced-sessions.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-11-event-sourced-sessions.zh.md @@ -24,5 +24,5 @@ MVP 要求严格的基于事件的追踪,以及完全可回放的会话(严 - 回放、追踪与遥测在结构上得到保证,而非事后附加。 - 持久化仍是插件关注点;内存存储随 dsh-session 一起提供。 -- 事件词汇可通过合并扩展(插件可添加如压缩(compaction)事件);[会话持久化](2026-06-14-session-persistence.md)在日志具备持久性后固定了其结构。 +- 事件词汇可通过合并扩展(插件可添加如压缩(compaction)事件);[会话持久化](2026-06-14-session-persistence.zh.md)在日志具备持久性后固定了其结构。 - 派生成本随日志长度增长,压缩(dsh-compaction)是预期的缓解手段,而不是改写日志。 diff --git a/.agents/notes/implemented/architecture/2026-06-11-microkernel-event-taxonomy.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-11-microkernel-event-taxonomy.i18n.yaml index 8e97d86ccf..57843018c6 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-microkernel-event-taxonomy.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-11-microkernel-event-taxonomy.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-11-microkernel-event-taxonomy.md 2026-06-11-microkernel-event-taxonomy.md: 6cacbc30f29c863de0f8ddfa4df7251a64bc80ca -2026-06-11-microkernel-event-taxonomy.zh.md: 2e43b57d62d8fca692ffab379d1841e6a3e61fad +2026-06-11-microkernel-event-taxonomy.zh.md: 1c7919392b835c491b9dc606d46f638a708d2948 diff --git a/.agents/notes/implemented/architecture/2026-06-11-microkernel-event-taxonomy.zh.md b/.agents/notes/implemented/architecture/2026-06-11-microkernel-event-taxonomy.zh.md index 2e43b57d62..1c7919392b 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-microkernel-event-taxonomy.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-11-microkernel-event-taxonomy.zh.md @@ -25,7 +25,7 @@ Status: implemented ## 后果 -- 每个 MVP 功能都映射到一个监听器([功能→机制映射](../../../../docs/cookbook/extension-cookbook.md#the-feature--mechanism-map)是证明义务,保持更新)。 +- 每个 MVP 功能都映射到一个监听器([功能→机制映射](../../../../docs/cookbook/extension-cookbook.zh.md#the-feature--mechanism-map)是证明义务,保持更新)。 - HMR 与 dispose 无需额外工作:监听器和注册均为 Cordis effect。 - waterfall 语义(调用 `next()` 或短路)不直观,需要教学——在 AGENTS.md 中记录,并由组合测试覆盖。 - 循环必须具备防御性:插件异常在轮次级别被隔离,来自任何扩展点的 steering(中途引导)永远不会被搁置(有回归测试保障)。 diff --git a/.agents/notes/implemented/architecture/2026-06-11-runtime-arg-validation.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-11-runtime-arg-validation.i18n.yaml index 2e6f1ad2df..821a220d9a 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-runtime-arg-validation.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-11-runtime-arg-validation.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-11-runtime-arg-validation.md 2026-06-11-runtime-arg-validation.md: 520823c76f147205805d306e97b5461f38b84c35 -2026-06-11-runtime-arg-validation.zh.md: 0a6ef40c93ec3550d6e991836950be849e86ede7 +2026-06-11-runtime-arg-validation.zh.md: bc2f743300be78cc2a01f027d5aca81fcb18b468 diff --git a/.agents/notes/implemented/architecture/2026-06-11-runtime-arg-validation.zh.md b/.agents/notes/implemented/architecture/2026-06-11-runtime-arg-validation.zh.md index 0a6ef40c93..bc2f743300 100644 --- a/.agents/notes/implemented/architecture/2026-06-11-runtime-arg-validation.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-11-runtime-arg-validation.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -`defineTool`([统一 schema DSL](2026-07-20-unified-json-value-schema-dsl.md))为工具作者的 `execute(args)` 提供了经 `InferArgs` 映射的类型化参数。但该类型只是对运行时值的编译期声明,而这个值实际上是模型生成的 JSON:没有任何机制强制模型遵守 schema,因此畸形调用(缺少必需键、声明为数字的位置传入字符串,或字面量超出声明的集合)会以「仅在名义上类型化」的状态到达 `execute`。工具函数体随后要么在处理结构错误的数据时崩溃,要么在不报错的情况下行为异常。 +`defineTool`([统一 schema DSL](2026-07-20-unified-json-value-schema-dsl.zh.md))为工具作者的 `execute(args)` 提供了经 `InferArgs` 映射的类型化参数。但该类型只是对运行时值的编译期声明,而这个值实际上是模型生成的 JSON:没有任何机制强制模型遵守 schema,因此畸形调用(缺少必需键、声明为数字的位置传入字符串,或字面量超出声明的集合)会以「仅在名义上类型化」的状态到达 `execute`。工具函数体随后要么在处理结构错误的数据时崩溃,要么在不报错的情况下行为异常。 ## 决策 @@ -17,8 +17,8 @@ Status: implemented ## 后果 - 模型会收到有关自身畸形调用的可操作反馈,而不是遭遇不透明的崩溃,弥合了 `InferArgs` 的承诺与运行时现实之间的鸿沟。 -- 校验器与 `InferArgs` 必须保持一致;一项[属性测试](../testing/2026-06-11-property-based-testing.md)生成满足 spec 的参数并断言它们通过 `validateArgs`(同时通过针对性改坏参数来断言其会被拒绝),通过自动化检查消除这种漂移风险。 -- `ToolArgsError` 是[结构化错误分类体系](2026-06-11-structured-error-taxonomy.md)中 `HarnessError` 的子类,保留其 `code` 字段;读取 `.message` 的调用方不受该层级结构影响。 +- 校验器与 `InferArgs` 必须保持一致;一项[属性测试](../testing/2026-06-11-property-based-testing.zh.md)生成满足 spec 的参数并断言它们通过 `validateArgs`(同时通过针对性改坏参数来断言其会被拒绝),通过自动化检查消除这种漂移风险。 +- `ToolArgsError` 是[结构化错误分类体系](2026-06-11-structured-error-taxonomy.zh.md)中 `HarnessError` 的子类,保留其 `code` 字段;读取 `.message` 的调用方不受该层级结构影响。 - 校验开销相对于一次模型调用可忽略不计。 diff --git a/.agents/notes/implemented/architecture/2026-06-13-capability-seams.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-13-capability-seams.i18n.yaml index 106164a71c..1c4eadabc1 100644 --- a/.agents/notes/implemented/architecture/2026-06-13-capability-seams.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-13-capability-seams.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-13-capability-seams.md -2026-06-13-capability-seams.md: 2a166278ea454895177fa12b58f5493276f19cd1 -2026-06-13-capability-seams.zh.md: 0874af5826960ab9e718eb07b00c12b446edfd78 +2026-06-13-capability-seams.md: 46a2c39e927e859c7eb95956d8586f3bf04c7b1c +2026-06-13-capability-seams.zh.md: f44e3e68d2153149435b0fd0aaa5fd121cf3ecad diff --git a/.agents/notes/implemented/architecture/2026-06-13-capability-seams.md b/.agents/notes/implemented/architecture/2026-06-13-capability-seams.md index 2a166278ea..46a2c39e92 100644 --- a/.agents/notes/implemented/architecture/2026-06-13-capability-seams.md +++ b/.agents/notes/implemented/architecture/2026-06-13-capability-seams.md @@ -6,7 +6,7 @@ English | [中文](2026-06-13-capability-seams.zh.md) ## Problem -The harness has swappable capabilities — bash execution today, sandboxed/remote executors and alternative model providers tomorrow. A capability has three concerns that change at different rates and for different reasons: the *contract* (what the capability is), the *implementation* (how it runs), and the *consumer API* (what the model and other plugins program against). Bundling them in one package couples those rates of change — swapping a local executor for a sandboxed one would churn the tool schemas the model sees, even though the model-facing contract never changed. +The harness has swappable capabilities, including shell execution and model providers. A capability has three concerns that change at different rates and for different reasons: the *contract* (what the capability is), the *implementation* (how it runs), and the *consumer API* (what the model and other plugins program against). Bundling them in one package couples those rates of change — swapping a local executor for a sandboxed one would churn the tool schemas the model sees, even though the model-facing contract never changed. This is distinct from "who provides vs. needs a capability at runtime", which Cordis already answers with services + `inject` (a provider registers `ctx.shell`; a consumer declares `inject: ['bash']` and its fiber pends until the service exists). That mechanism is necessary but doesn't dictate package boundaries; this Agent Note does. diff --git a/.agents/notes/implemented/architecture/2026-06-13-capability-seams.zh.md b/.agents/notes/implemented/architecture/2026-06-13-capability-seams.zh.md index 0874af5826..f44e3e68d2 100644 --- a/.agents/notes/implemented/architecture/2026-06-13-capability-seams.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-13-capability-seams.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -harness 具有可替换的能力:当前是 bash 执行,未来会有沙箱化/远程执行器和替代模型提供方。一项能力涉及三个关注点,它们以不同速率、因不同原因变化:*约定*(这项能力是什么)、*实现*(它如何运行)、*消费方 API*(模型和其他插件面向什么编程)。将三者捆绑在一个包中会耦合这些变化速率——把本地执行器换成沙箱化执行器时,模型看到的工具 schema 也会被搅动,尽管面向模型的约定从未改变。 +harness 具有可替换的能力,包括 shell 执行和模型提供方。一项能力涉及三个关注点,它们以不同速率、因不同原因变化:*约定*(这项能力是什么)、*实现*(它如何运行)、*消费方 API*(模型和其他插件面向什么编程)。将三者捆绑在一个包中会耦合这些变化速率——把本地执行器换成沙箱化执行器时,模型看到的工具 schema 也会被搅动,尽管面向模型的约定从未改变。 这与「谁在运行时提供、谁需要一项能力」是不同的问题,后者 Cordis 已通过服务 + `inject` 解决(提供方注册 `ctx.shell`;消费方声明 `inject: ['bash']`,其 fiber 挂起直到服务存在)。该机制是必要的,但不决定包的边界;本 Agent Note 决定的是包的边界。 @@ -26,7 +26,7 @@ Service Provider 与 Consumer 由此独立演进:沙箱化执行器替换 `dsh ## 术语:seam 指三者组合,而非接口 -一个 **seam** 是完整的能力——三个角色合在一起:**Service Definition**(拥有 `ctx.` 和词汇的 Cordis `Service`)、一个或多个 **Service Provider**,以及一个或多个 **Consumer**。`packages/shell` 是规范范例——`dsh-shell` / `dsh-bash-local`+`dsh-bash-sandbox` / `dsh-tool-bash`。一个包可以承担多个角色,但单个角色本身不是 seam。「seam」一词严格保留给这种完整能力;命名其中一个组成部分时,应使用其角色、类、服务、约定或扩展点。[术语表](../../../../docs/glossary.md#capability-seam)是规范条目。 +一个 **seam** 是完整的能力——三个角色合在一起:**Service Definition**(拥有 `ctx.` 和词汇的 Cordis `Service`)、一个或多个 **Service Provider**,以及一个或多个 **Consumer**。`packages/shell` 是规范范例——`dsh-shell` / `dsh-bash-local`+`dsh-bash-sandbox` / `dsh-tool-bash`。一个包可以承担多个角色,但单个角色本身不是 seam。「seam」一词严格保留给这种完整能力;命名其中一个组成部分时,应使用其角色、类、服务、约定或扩展点。[术语表](../../../../docs/glossary.zh.md#capability-seam)是规范条目。 ## 曾考虑的替代方案 @@ -35,4 +35,4 @@ Service Provider 与 Consumer 由此独立演进:沙箱化执行器替换 `dsh ## 后果 -分离角色会增加包和样板代码(`package.json`、`tsconfig`、README 和注入接线)。换来的是:Service Provider 与 Consumer 独立发布和版本管理,新后端永远不会波及面向模型的约定。[AGENTS.md](../../../../AGENTS.md) 和 [architecture.md](../../../../docs/architecture.md) 载有这项规则;bash 三件套是参考模板。本 Agent Note 记录为什么独立变化的角色通常需要拆分,而确实共享的关注点可以保持合并。 +分离角色会增加包和样板代码(`package.json`、`tsconfig`、README 和注入接线)。换来的是:Service Provider 与 Consumer 独立发布和版本管理,新后端永远不会波及面向模型的约定。[AGENTS.md](../../../../AGENTS.md) 和 [architecture.md](../../../../docs/architecture.zh.md) 载有这项规则;bash 三件套是参考模板。本 Agent Note 记录为什么独立变化的角色通常需要拆分,而确实共享的关注点可以保持合并。 diff --git a/.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.i18n.yaml index d44279eb42..3af4789414 100644 --- a/.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md 2026-06-13-twin-llm-adapters.md: a4c87325a0b0d1ebe6cf8f95672e5de74ef37d57 -2026-06-13-twin-llm-adapters.zh.md: 2af7e7cca4d4c40e18cece1a4dd5c9cdb11901c8 +2026-06-13-twin-llm-adapters.zh.md: 36996750a16cc95393d727cffee3bcc53573eaf2 diff --git a/.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.zh.md b/.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.zh.md index 2af7e7cca4..36996750a1 100644 --- a/.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -`dsh-llm` 拥有一套提供方无关的流式词汇:`StreamChunk` 协议(`block-start`、`text-delta`、`reasoning-delta`、`tool-call-delta`、`block-end`、`usage`、`finish`)以及内容块类型([内容块词汇](2026-06-11-content-block-vocabulary.md))。如果词汇仅针对单个适配器定义,就有可能将该适配器的特异行为固化到「中立」约定中:唯一实现碰巧做了什么,什么就成为事实上的规范;在第二个提供方到来之前,抽象层未经验证——而届时修复这种泄漏的代价已经很高。 +`dsh-llm` 拥有一套提供方无关的流式词汇:`StreamChunk` 协议(`block-start`、`text-delta`、`reasoning-delta`、`tool-call-delta`、`block-end`、`usage`、`finish`)以及内容块类型([内容块词汇](2026-06-11-content-block-vocabulary.zh.md))。如果词汇仅针对单个适配器定义,就有可能将该适配器的特异行为固化到「中立」约定中:唯一实现碰巧做了什么,什么就成为事实上的规范;在第二个提供方到来之前,抽象层未经验证——而届时修复这种泄漏的代价已经很高。 ## 决策 diff --git a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.i18n.yaml index 61f9754431..8f9aa62e49 100644 --- a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-14-session-persistence.md 2026-06-14-session-persistence.md: 62228bd2f5b25b13880a563818d08f3a2d52d956 -2026-06-14-session-persistence.zh.md: b10ceebd95d4d05ae3f7ea620183ad8fffd074ee +2026-06-14-session-persistence.zh.md: ebf004333c383336cd025aa8a4aabc9d1e07f0e5 diff --git a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md index b10ceebd95..ebf004333c 100644 --- a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md @@ -8,28 +8,28 @@ Status: implemented 会话此前仅存在于内存中。示例插件 `session-jsonl.ts`(在两个示例中逐字节重复)是只写的遥测:它缓冲 `session/event` 并追加 JSON 行,没有读取/回放路径,没有崩溃安全性(无 fsync、无原子写入、dispose(资源释放)时采用 fire-and-forget 方式排空),没有列表功能,也没有格式版本控制。没有任何机制能将磁盘上的历史会话重新注入到活跃的 agent(智能体)中,因此持久恢复、持久 fork 以及宿主侧的会话浏览都无法实现。 -[事件溯源模型](2026-06-11-event-sourced-sessions.md)将仅追加日志作为唯一真源,并从中派生 LLM(大语言模型)历史。持久化必须忠实于这一设计:直接持久化现有的 `SessionEvent`,不引入需要来回转换的并行「持久化消息」类型。后端也必须可替换——当前用文件存储,以后用数据库存储——并由同一接口封装。 +[事件溯源模型](2026-06-11-event-sourced-sessions.zh.md)将仅追加日志作为唯一真源,并从中派生 LLM(大语言模型)历史。持久化必须忠实于这一设计:直接持久化现有的 `SessionEvent`,不引入需要来回转换的并行「持久化消息」类型。后端也必须可替换——当前用文件存储,以后用数据库存储——并由同一接口封装。 ## 决策 -持久化是一个具有抽象 Service Definition 的**能力 seam**([能力 seam](2026-06-13-capability-seams.md),`dsh-shell` 模板),而非循环或核心逻辑: +持久化是一个具有抽象 Service Definition 的**能力 seam**([能力 seam](2026-06-13-capability-seams.zh.md),`dsh-shell` 模板),而非循环或核心逻辑: 1. **接口**(`dsh-session-persistence`,`ctx.sessionPersistence`):一个抽象的 `SessionPersistence` 服务,提供 `locate`/`create`/`append`/`prepare`/`load`/`inspect`/`readFrom`/`list`/`listSnapshots`。其持久化单元就是现有的 `SessionEvent`(`{ type, seq, time, data }`),原样复用,无转换类型。 -2. **实现**(`dsh-session-persistence-jsonl`):每个会话一个仅追加的逻辑 JSONL 日志:先是一行 `SessionHeader`,随后是无损表示连续 `SessionEvent` 流的存储记录。符合条件的 `assistant/chunk` 增量连续段默认使用打包行;[带校验和的 Zstandard 帧](2026-07-19-zstandard-jsonl-session-logs.md)是默认物理编码,也可通过配置使用原始行。 +2. **实现**(`dsh-session-persistence-jsonl`):每个会话一个仅追加的逻辑 JSONL 日志:先是一行 `SessionHeader`,随后是无损表示连续 `SessionEvent` 流的存储记录。符合条件的 `assistant/chunk` 增量连续段默认使用打包行;[带校验和的 Zstandard 帧](2026-07-19-zstandard-jsonl-session-logs.zh.md)是默认物理编码,也可通过配置使用原始行。 长期有效、存在争议的关键选择: - **规范的持久日志无损保留每个 `SessionEvent`,包括 `assistant/chunk`。** JSONL 存储可以将一段连续的增量事件编码为一条打包行,但逻辑读取方会重建精确的事件边界、序号与时间戳。`deriveMessages()` 跳过分片,而过滤分片的方案(Codex 的 `policy.rs`)很有吸引力,但 `seq = log.length` 以及 `events[i].seq === i` 验证要求*连续*的逻辑日志;过滤掉分片会留下空洞,同时破坏约定和恢复功能。基于分片过滤的投影可以作为派生视图在后续实现(带有自己的重新编号),但它不是规范日志。 -- **仅追加;崩溃的轮次被关闭,而非截断。** 已刷写的事件永不被重写。[语义检查点策略](../bug-fix/2026-07-21-semantic-session-checkpoints.md)会在调用模型前排空请求、在调用工具前排空已记录的顶层调用,并在步骤结束后排空完整的响应/结果批次;循环则排空最终轮次边界。由于一个被中断的轮次可能包含大量有效工作,冷检查会保留其连续、可解析的事件,并在内存逻辑视图中为未应答的 assistant 调用添加按风险分类的错误结果、补一个缺失的 `step/end`,以及带 `{ kind: 'interrupted' }` 的 `turn/end`。`prepare` 或 `load` 在返回可恢复视图前提交这些收尾事件;合成结果保证恢复后的提供方 transcript(文本记录)仍然有效。只有不完整的最后一条记录会在提交修复时被丢弃;在最后一个真实 `turn/end` 处或之前出现解析错误或序号间隙,属于数据损坏,会使该会话不可加载。 +- **仅追加;崩溃的轮次被关闭,而非截断。** 已刷写的事件永不被重写。[语义检查点策略](../bug-fix/2026-07-21-semantic-session-checkpoints.zh.md)会在调用模型前排空请求、在调用工具前排空已记录的顶层调用,并在步骤结束后排空完整的响应/结果批次;循环则排空最终轮次边界。由于一个被中断的轮次可能包含大量有效工作,冷检查会保留其连续、可解析的事件,并在内存逻辑视图中为未应答的 assistant 调用添加按风险分类的错误结果、补一个缺失的 `step/end`,以及带 `{ kind: 'interrupted' }` 的 `turn/end`。`prepare` 或 `load` 在返回可恢复视图前提交这些收尾事件;合成结果保证恢复后的提供方 transcript(文本记录)仍然有效。只有不完整的最后一条记录会在提交修复时被丢弃;在最后一个真实 `turn/end` 处或之前出现解析错误或序号间隙,属于数据损坏,会使该会话不可加载。 - **文件后端为规范实现,数据库后端为经过验证的直接替换。** `SessionEvent` 1:1 映射到一行 `(session_id, seq, type, time, data)`:`append` 是 INSERT(在一个断言连续 seq 约定的事务中),读取使用 SELECT … ORDER BY seq。`dsh-session-persistence-sqlite` 正是如此:一个 `SessionPersistence` 子类,接口无变化(opencode 在 SQLite/WAL 上采用的正是这种接口形态),且通过与 JSONL 后端相同的 `runPersistenceContract` 测试套件。该约定以相同的语义约束两个后端(惰性物化、逻辑关闭中断轮次、修复只提交一次、连续 seq),一次表达在文件字节上,一次表达在数据库行上。其数据库拥有专用的 application id 与单调递增的 schema 版本。系统会在一个事务中为全新文件创建所有表并写入这两个 header 值;未版本化文件若带有任何用户定义的 schema 对象或应用标识、当前版本文件若带有外部应用标识,以及任何非当前版本文件,都会在修改日志模式之前被拒绝。 -- **元数据在日志之外。** 格式版本、cwd 和谱系是存储关注点,不是可回放的对话状态,因此它们存放在 `dsh-session` 拥有的 `SessionHeader` 中,并通过新的只读属性 `session.header` 附加到 `Session` 上——永远不进入 `SessionEventMap`,永远不到达 `deriveMessages()`。`createdAt` 是以 Unix epoch 毫秒表示的非负安全整数:运行时创建和持久化注册会拒绝小数值,JSONL 会验证解码后的 header,SQLite 则将其存入严格的 `INTEGER` 列。替代方案(一个可合并扩展的 `session/meta` 事件作为日志第 0 行)被否决:日志内事件会自然随 seed/fork 的会话携带,但元数据不是可回放状态,因此显式的日志外 header 边界是更清晰的取舍。(header 最初被拆分为不可变的 `SessionHeader` 加可变的 `SessionSummary`,二者的联合类型为 `SessionMeta`;可变 summary 后来因属于死状态而被移除——见 [移除可变会话摘要](../simplification/2026-06-19-drop-mutable-session-summary.md)。) -- **`ctx.agents.create()` 和 `ctx.agents.resume()` 是异步工厂;恢复还跨越持久化边界。** `ctx.agents.resume({ resumeSessionId })` 通过 `ctx.sessionPersistence.prepare()` 取得精确的未发布 Session,以持久化 id 发布它,并继续其投影。[Session 准备阶段决策](2026-08-05-session-preparation.md)定义历史检查与恢复之间的复用。agent loop(智能体循环)不会硬注入 `sessionPersistence`(那样会让非持久化的演示永远挂起);当它不存在时,`resume` 会以明确的错误拒绝。 +- **元数据在日志之外。** 格式版本、cwd 和谱系是存储关注点,不是可回放的对话状态,因此它们存放在 `dsh-session` 拥有的 `SessionHeader` 中,并通过新的只读属性 `session.header` 附加到 `Session` 上——永远不进入 `SessionEventMap`,永远不到达 `deriveMessages()`。`createdAt` 是以 Unix epoch 毫秒表示的非负安全整数:运行时创建和持久化注册会拒绝小数值,JSONL 会验证解码后的 header,SQLite 则将其存入严格的 `INTEGER` 列。替代方案(一个可合并扩展的 `session/meta` 事件作为日志第 0 行)被否决:日志内事件会自然随 seed/fork 的会话携带,但元数据不是可回放状态,因此显式的日志外 header 边界是更清晰的取舍。(header 最初被拆分为不可变的 `SessionHeader` 加可变的 `SessionSummary`,二者的联合类型为 `SessionMeta`;可变 summary 后来因属于死状态而被移除——见 [移除可变会话摘要](../simplification/2026-06-19-drop-mutable-session-summary.zh.md)。) +- **`ctx.agents.create()` 和 `ctx.agents.resume()` 是异步工厂;恢复还跨越持久化边界。** `ctx.agents.resume({ resumeSessionId })` 通过 `ctx.sessionPersistence.prepare()` 取得精确的未发布 Session,以持久化 id 发布它,并继续其投影。[Session 准备阶段决策](2026-08-05-session-preparation.zh.md)定义历史检查与恢复之间的复用。agent loop(智能体循环)不会硬注入 `sessionPersistence`(那样会让非持久化的演示永远挂起);当它不存在时,`resume` 会以明确的错误拒绝。 ## 曾考虑的替代方案 上述每个关键选择都在陈述处记录了被否决的替代方案:**过滤分片的规范日志**(Codex 的 `policy.rs` 形式)破坏连续 seq 约定;**截断崩溃的轮次**会静默销毁长时间自主运行中的真实工作;**日志内 `session/meta` 事件作为第 0 行**——元数据不是可回放状态;**有限的非整数 `createdAt` 值**没有生产方,且与整数 Unix 毫秒存储及查询列不一致;**接受非全新的未版本化 SQLite 文件**可能覆盖无关对象或应用标识;**将 `sessionPersistence` 硬注入循环**会让非持久化的演示永远挂起。 -格式版本控制:header 携带一个 `version`;冷读取拒绝任何非当前版本。预发布阶段的会话格式仍固定为 `SESSION_FORMAT_VERSION = 0`,不承诺广泛兼容;当持久化用户数据确有需要时,协调器可以负责显式且范围受限的导入升级([消息标识机制引入前的消息恢复](../bug-fix/2026-07-28-load-pre-identity-session-messages.md))。仅追加 + 刷写对尾部的不完整写入具有健壮性(冷准备时可容忍),但无法抵御未使用 fsync 时在行写入中途断电;数据库/WAL 后端是该场景下更强的选项。 +格式版本控制:header 携带一个 `version`;冷读取拒绝任何非当前版本。预发布阶段的会话格式仍固定为 `SESSION_FORMAT_VERSION = 0`,不承诺广泛兼容;当持久化用户数据确有需要时,协调器可以负责显式且范围受限的导入升级([消息标识机制引入前的消息恢复](../bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md))。仅追加 + 刷写对尾部的不完整写入具有健壮性(冷准备时可容忍),但无法抵御未使用 fsync 时在行写入中途断电;数据库/WAL 后端是该场景下更强的选项。 ## 后果 diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml index 9898684b9e..c3984a9fa9 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md 2026-06-17-filesystem-capability-seam.md: 6265aebf5e7ffdd4ec4dc0083aa55adb34ee78a1 -2026-06-17-filesystem-capability-seam.zh.md: 024421ed59fa1e6c29bcc12ab51f69ac19a0c231 +2026-06-17-filesystem-capability-seam.zh.md: f0f1cb132fe06ebf669f0310f0066f4fdd36ad0b diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md index 024421ed59..f0f1cb132f 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md @@ -20,7 +20,7 @@ harness 已有一个具体的 `bash` 能力 seam(`dsh-shell` / `dsh-bash-local ## 决策 -文件系统访问是一个一等的能力 seam,遵循[能力 seam Agent Note](2026-06-13-capability-seams.md): +文件系统访问是一个一等的能力 seam,遵循[能力 seam Agent Note](2026-06-13-capability-seams.zh.md): 1. `@deepseek-ai/dsh-fs`(`packages/fs/fs`)拥有抽象的 `ctx.fs` 服务、文件系统词汇类型,以及 `fs/*` 策略事件词汇。 2. `@deepseek-ai/dsh-fs-local`(`packages/fs/fs-local`)提供第一个实现,以本地文件系统为后端。 @@ -28,7 +28,7 @@ harness 已有一个具体的 `bash` 能力 seam(`dsh-shell` / `dsh-bash-local Consumer 包仅依赖 Service Definition 包,从不依赖 `dsh-fs-local`。需要不同后端的部署只需为 `ctx.fs` 加载不同的提供方,无需改动工具 schema 或面向模型的提示词引导。 -读后写/编辑与观测状态策略是第四个包 `@deepseek-ai/dsh-fs-observation-policy`(`packages/fs/fs-observation-policy`),通过 `fs/*` 事件门控贡献,而非挂在 `ctx.fs` 上;加载 `dsh-tool-fs` 的部署同时加载 `dsh-fs-observation-policy` 以获得读后写/编辑能力。本决策确立了由三个包构成的边界;策略从提供方基类拆出的决策由 [拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md) 做出,其以事件门控插件(而非方法服务)实现的方式由 [事件门控 Agent Note](2026-06-26-file-context-as-event-gate.md) 做出。 +读后写/编辑与观测状态策略是第四个包 `@deepseek-ai/dsh-fs-observation-policy`(`packages/fs/fs-observation-policy`),通过 `fs/*` 事件门控贡献,而非挂在 `ctx.fs` 上;加载 `dsh-tool-fs` 的部署同时加载 `dsh-fs-observation-policy` 以获得读后写/编辑能力。本决策确立了由三个包构成的边界;策略从提供方基类拆出的决策由 [拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.zh.md) 做出,其以事件门控插件(而非方法服务)实现的方式由 [事件门控 Agent Note](2026-06-26-file-context-as-event-gate.zh.md) 做出。 第一个后端有意仅限本地:`dsh-fs-local` 基于宿主文件系统实现 `ctx.fs`。未来的兄弟后端可在同一接口之后提供沙箱、远程、虚拟或项目作用域的文件系统。 @@ -36,7 +36,7 @@ Consumer 包仅依赖 Service Definition 包,从不依赖 `dsh-fs-local`。需 文件系统权限和沙箱并非此拆分所隐含。本地后端从其配置的基目录解析相对路径,但路径包含约束策略是独立的决策:要么由更严格的 `ctx.fs` 实现强制执行,要么由权限/沙箱插件包装 `tools/execute` 并在调用到达消费方之前否决。 -读后写/编辑与观测状态属于 `dsh-fs-observation-policy`,而非 `ctx.fs`。通过 `fs/*` 事件门控,策略按不透明 actor 记录版本,并提供可选的变更期望;提供方原子性地强制新鲜度。`dsh-tool-fs` 发出事件但不依赖策略。见[拆分文件系统 seam](../simplification/2026-06-26-fsspec-style-fs-seam.md)和[事件门控插件](2026-06-26-file-context-as-event-gate.md) Agent Note。 +读后写/编辑与观测状态属于 `dsh-fs-observation-policy`,而非 `ctx.fs`。通过 `fs/*` 事件门控,策略按不透明 actor 记录版本,并提供可选的变更期望;提供方原子性地强制新鲜度。`dsh-tool-fs` 发出事件但不依赖策略。见[拆分文件系统 seam](../simplification/2026-06-26-fsspec-style-fs-seam.zh.md)和[事件门控插件](2026-06-26-file-context-as-event-gate.zh.md) Agent Note。 ## 包拓扑 @@ -84,7 +84,7 @@ Consumer 包仅依赖 Service Definition 包,从不依赖 `dsh-fs-local`。需 - 不透明的 `targetKey`,用于陈旧守护和文件状态查找。本地后端可能使用类似 realpath 的键;远程后端可能使用工作区 URI 或文件 id。消费方禁止解析或假设它是本地绝对路径。 - `displayPath`,用于面向模型/UI 的输出。根据后端不同,它可能是本地绝对路径、工作区相对路径或远程 URI。 -即使另一项能力共享提供方的执行环境,`targetKey` 仍保持不透明。这类消费方通过提供方的 `processPath(target)`、`fileUrl(target)` 或 `contains(parent, child)` 获取所需事实;[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)说明这些事实为何属于文件系统 seam。 +即使另一项能力共享提供方的执行环境,`targetKey` 仍保持不透明。这类消费方通过提供方的 `processPath(target)`、`fileUrl(target)` 或 `contains(parent, child)` 获取所需事实;[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.zh.md)说明这些事实为何属于文件系统 seam。 读取和变更结果必须包含不透明的文件 `version`。本地后端从 bigint stat 元数据(`dev`、`ino`、`size`、`mtimeNs` 和 `ctimeNs`)派生令牌,因此同大小重写和 inode 替换都会可靠地使消费方失效;远程后端可以使用 revision id 或类似 hash 的令牌。`dsh-fs-observation-policy` 插件记录版本用于陈旧检查;消费方可以展示相关元数据但禁止解释版本令牌。 @@ -140,7 +140,9 @@ Consumer 包仅依赖 Service Definition 包,从不依赖 `dsh-fs-local`。需 - **面向模型的工具直接基于 `node:fs`**:工具包将同时承担执行策略、路径解析、原子写入、文本解码和编辑语义,耦合问题部分所列的三个独立变化的关注点,且任何后端替换都会搅动 schema。 - **单一合并包 `dsh-fs-tools`**:seam 之前的形态;以与 bash 相同的 Service Definition / Service Provider / Consumer 拆分理由否决,且合并名称从未成为公开 API。 -- **观测状态放在 `ctx.fs` 上**:本 Agent Note 最初落地的形态;被 [拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md) 和 [事件门控 Agent Note](2026-06-26-file-context-as-event-gate.md) 取代:沙箱/远程后端不应继承面向模型的观测策略,因此提供方只保留版本令牌和可选的版本守护变更。 +- **观测状态放在 `ctx.fs` 上**:本 Agent Note 最初落地的形态;被 [拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.zh.md) 和 [事件门控 Agent Note](2026-06-26-file-context-as-event-gate.zh.md) 取代:沙箱/远程后端不应继承面向模型的观测策略,因此提供方只保留版本令牌和可选的版本守护变更。 + + ## 后果 diff --git a/.agents/notes/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-contracts.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-contracts.i18n.yaml index 0ecda2484f..03a0440ca1 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-contracts.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-contracts.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-contracts.md 2026-06-18-agent-lifecycle-and-ownership-contracts.md: 9bc558bfce75892b0ebdb80e9f8735d440cabaf4 -2026-06-18-agent-lifecycle-and-ownership-contracts.zh.md: 65e0018c3f2ba56043898050c8e0b49929f8e7e7 +2026-06-18-agent-lifecycle-and-ownership-contracts.zh.md: bbd0a7219f902e5ead4d190c58e9a5d4da2131fd diff --git a/.agents/notes/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-contracts.zh.md b/.agents/notes/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-contracts.zh.md index 65e0018c3f..bbd0a7219f 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-contracts.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-contracts.zh.md @@ -14,7 +14,7 @@ ACP(Agent Client Protocol)与 tool-bash 的若干限制是同一个所有权 ### 1. 队列感知的 `Agent.cancel(cause?)` -`Agent` 接口新增 `cancel()` 动词——唯一的公开停止原语。(它最初与范围更窄、仅作用于步骤的 `abort()` 一同交付;后者后来因无人使用而移除,使 `cancel()` 成为唯一公开的停止工作方式。)它清空 inbox 的 queued + steering FIFO,在存在活跃轮次时中止它,并保留一个不带 cause 的 pre-run 标记,使在被领取前被取消的提示词永不运行,而后来的提示词仍保持独立。有效调用会在清空或中止前发出 `agent/cancel-requested`,携带类型化的 `user | parent` cause;空闲取消不发出任何事件,也不会使下一条提示词搁浅。`whenIdle()` 会在取消后达到完全停稳,ACP 的 `session/cancel` 映射到 `user`。[显式轮次取消决策](2026-07-16-explicit-turn-cancellation.md)规定了当前的 cause、signal 生命周期与协作式结算约定。 +`Agent` 接口新增 `cancel()` 动词——唯一的公开停止原语。(它最初与范围更窄、仅作用于步骤的 `abort()` 一同交付;后者后来因无人使用而移除,使 `cancel()` 成为唯一公开的停止工作方式。)它清空 inbox 的 queued + steering FIFO,在存在活跃轮次时中止它,并保留一个不带 cause 的 pre-run 标记,使在被领取前被取消的提示词永不运行,而后来的提示词仍保持独立。有效调用会在清空或中止前发出 `agent/cancel-requested`,携带类型化的 `user | parent` cause;空闲取消不发出任何事件,也不会使下一条提示词搁浅。`whenIdle()` 会在取消后达到完全停稳,ACP 的 `session/cancel` 映射到 `user`。[显式轮次取消决策](2026-07-16-explicit-turn-cancellation.zh.md)规定了当前的 cause、signal 生命周期与协作式结算约定。 ### 2. `AgentHandle` 异步释放器 @@ -43,7 +43,7 @@ bash 所有者 token 比较依赖共享的 `Agent.id`/`SessionId` 在存活 agen - **公开的 `BashTask.owner` 字段**而非 `ShellExecutor.ownerOf(id)` Service Definition 方法:否决。一条读取路径即可,无需冗余 API。 - **为 agent 的会话生命周期使用兄弟 Cordis effect**:否决。fiber 卸载时并发释放兄弟 effect(`Promise.all`),store 拥有的 append 发布钩子的移除与循环的关闭 `session/flush` 产生竞争;单一复合 effect 的有序 LIFO 链才能在两条释放路径上都捕获关闭的 `turn/end`。 -- **在 `cancel()` 之外另设一个仅中止步骤的 `abort()`**:最初发布过,后因无人使用而移除;`cancel()` 是唯一的公开停止原语(见[公开停止接口 Agent Note](../simplification/2026-06-20-public-agent-stop-api.md))。 +- **在 `cancel()` 之外另设一个仅中止步骤的 `abort()`**:最初发布过,后因无人使用而移除;`cancel()` 是唯一的公开停止原语(见[公开停止接口 Agent Note](../simplification/2026-06-20-public-agent-stop-api.zh.md))。 ## 后果 diff --git a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.i18n.yaml index 38a665a0fd..5148c8a648 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md -2026-06-18-shared-persistence-write-coordinator.md: 93b6cd1bd058499e71948d3909de8e4c076b445e -2026-06-18-shared-persistence-write-coordinator.zh.md: 9e1bc736d4dba5e59b763710976425fa0ae76c30 +2026-06-18-shared-persistence-write-coordinator.md: 8392ec726ff44e8a7173f48ef7d5cc4826b7e882 +2026-06-18-shared-persistence-write-coordinator.zh.md: e160f29247ae5cd02aaa8388c141faec64001857 diff --git a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md index 93b6cd1bd0..8392ec726f 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md +++ b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md @@ -16,15 +16,20 @@ Composition, not inheritance. The coordinator is a concrete class the backend ho The coordinator holds one lifecycle entry for each exact live `Session`: initialization plus a package-private write controller that owns pending events, a fixed batching deadline, the active write, failure retention, and the shared flush barrier. Each `session/event` enters that bounded write path, and `session/flush` bypasses the wait to observe quiescence. The [flush-controller simplification](../simplification/2026-07-23-collapse-persistence-flush-state.md) owns controller consolidation; the [bounded batching decision](2026-08-08-bounded-session-persistence-write-batching.md) owns scheduling cadence. +Creation borrows the exact `Session.events` snapshot as its persistence seed. `Session` has already detached, validated, and deeply frozen every event, and the snapshot array remains stable when later appends replace the cached view. The coordinator and its backend hooks only read this typed in-process value, so cloning the complete log again would duplicate the ownership work described by the [agent-scope runtime decision](2026-07-12-agent-scope-runtime-design.md#session-append-materialize-validate-commit-notify). Public persistence `append()` still snapshots caller-owned input at its API boundary. + +Prepared-session suffixes and events admitted to the write-behind queue retain their existing copies. Those paths establish asynchronous queue ownership one suffix or event at a time and have no measured whole-log clone cost; removing their copies remains a separate ownership audit rather than part of creation-seed borrowing. + The coordinator retires a session from `session/disposed`: it waits for the controller's initialization and current flush, serializes a final drain, and removes the controller and owned per-id state only after success. A failure leaves the controller discoverable for backend teardown to retry. Settled per-id chain tails remove themselves only when they are still current, so a completion cannot erase a newer operation for the same id. Backend teardown unregisters write-path listeners, flushes every remaining controller, awaits per-id operations, and then closes the backend. ### The hook interface (`PersistenceBackend`) -Five required members plus an optional lifecycle hook form the only boundary between the coordinator and storage: +Five required members plus optional empty-materialization and lifecycle hooks form the only boundary between the coordinator and storage: - `name` — backend label for the dispose-failure `AggregateError`. - `loadStored(id)` — read one stored prefix by id across every storage scope (every JSONL project directory; SQLite's id is globally unique). Preparation, logical load/inspection, physical suffix reads, live adoption, and the create-collision probe share this lookup. The coordinator asserts the returned id and rejects a stored/live cwd mismatch before repair or state publication. -- `appendBatch(meta, events, isMaterialized)` — durably append a contiguous batch, lazily materializing the session ATOMICALLY when not yet materialized (the materialize-write and the first event batch must commit together — a crash between them must not leave a materialized-but-empty session; this is why there is no separate `materialize` hook). +- `appendBatch(meta, events, isMaterialized)` — durably append a contiguous batch, lazily materializing the session ATOMICALLY when not yet materialized. Ordinary creation therefore cannot leave an abandoned materialized-but-empty session. +- `materializeHeader?(meta)` — explicitly persist a header-only session for `SessionPersistence.ensureMaterialized(session)`. This is reserved for a lifecycle frontend that treats an empty session itself as a resumable durable resource; [standard ACP automation controls](../feature/2026-08-22-standard-acp-automation-controls.md) are the first consumer. Backends that support that lifecycle implement the hook; lazy creation remains the default. - `commitRepair(meta, tornMarker, closers)` — make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined`) and append `closers`. **NOT required to be atomic** — JSONL legitimately truncates-then-appends in two fsync'd steps, SQLite does DELETE+INSERT in one transaction. Used by `prepare`/`load` (truncate + synthetic closers) and live-adoption (truncate only, `closers = []`). - `list()` — list all stored metadata. - `close?()` — optional lifecycle teardown (SQLite closes its db handle; JSONL omits it), awaited in the dispose effect AFTER the quiescence drain so a close failure never masks a drain error. diff --git a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md index 9e1bc736d4..e160f29247 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md @@ -14,17 +14,22 @@ Status: implemented 组合,而非继承。协调器是后端持有的具体类,不是后端继承的基类。协调器让非常规后端与继承层级作斗争的风险由此规避:后端只暴露钩子,无法触及协调器的私有编排状态。第三方后端仍然可以完全不使用协调器、直接实现抽象服务,包括不可变逻辑检查,以及通过 `load` 实现的默认准备回退。 -协调器为每个存活的 `Session` 实例持有一个生命周期条目:初始化,加上一个包私有写入控制器,后者负责待处理事件、固定批处理截止时间、活跃写入、失败保留和共享 flush 屏障。每个 `session/event` 都进入这条有界写入路径,`session/flush` 则绕过等待以观察完全停稳。控制器归并由 [flush 控制器简化](../simplification/2026-07-23-collapse-persistence-flush-state.md)定义;调度节奏由[有界批处理决策](2026-08-08-bounded-session-persistence-write-batching.md)定义。 +协调器为每个存活的 `Session` 实例持有一个生命周期条目:初始化,加上一个包私有写入控制器,后者负责待处理事件、固定批处理截止时间、活跃写入、失败保留和共享 flush 屏障。每个 `session/event` 都进入这条有界写入路径,`session/flush` 则绕过等待以观察完全停稳。控制器归并由 [flush 控制器简化](../simplification/2026-07-23-collapse-persistence-flush-state.zh.md)定义;调度节奏由[有界批处理决策](2026-08-08-bounded-session-persistence-write-batching.zh.md)定义。 + +创建流程将 `Session.events` 的原始快照借作持久化种子。`Session` 已经分离、验证并深度冻结每个事件,后续追加会替换缓存视图,因此该快照数组保持稳定。协调器及其后端钩子只读取这个有类型的进程内值;再次克隆完整日志会重复 [agent scope 运行时决策](2026-07-12-agent-scope-runtime-design.zh.md#session-append-materialize-validate-commit-notify)规定的所有权工作。持久化服务的公开 `append()` 仍在 API 边界为调用方拥有的输入创建快照。 + +已准备 Session 的后缀,以及进入 write-behind 队列的事件,仍保留现有复制。这些路径会逐个后缀或事件建立异步队列所有权,且没有已测得的完整日志克隆成本;移除这些复制属于单独的所有权审计,不属于创建种子的借用决策。 协调器通过 `session/disposed` 退役会话:它等待控制器完成初始化和当前 flush,串行执行最后一次排空,且仅在成功后才移除控制器与其拥有的每 id 状态。失败时保持控制器可被找到,以供后端 teardown(拆除)重试。每个 id 的已结算链尾仅在其仍是当前链尾时才移除自身,因此旧操作完成后不会抹除同一 id 的新操作。后端 teardown 会注销写入路径监听器、flush 每个剩余的控制器、等待所有按 id 串行化的操作,最后关闭后端。 ### 钩子接口(`PersistenceBackend`) -五个必需成员加一个可选的生命周期钩子,构成协调器与存储之间唯一的边界: +五个必需成员加可选的空会话实体化与生命周期钩子,构成协调器与存储之间唯一的边界: - `name`——后端标签,用于 dispose 失败时的 `AggregateError`。 - `loadStored(id)`——按 id 跨所有存储范围读取一个已存储前缀(JSONL 的所有项目目录;SQLite 的 id 全局唯一)。准备、逻辑加载/检查、物理后缀读取、存活会话接管与创建碰撞探测共用此查找。协调器会断言返回的 id,并在修复或发布状态之前拒绝已存储记录与存活会话的 cwd 不匹配。 -- `appendBatch(meta, events, isMaterialized)`——持久追加一个连续批次,在尚未物化时原子地惰性物化会话(物化写入与首批事件必须一起提交——二者之间发生崩溃时,不得留下一个已物化但为空的会话;这就是为什么没有单独的 `materialize` 钩子)。 +- `appendBatch(meta, events, isMaterialized)`——持久追加一个连续批次,在尚未物化时原子地惰性物化会话。因此,普通创建不会留下被放弃的已物化空会话。 +- `materializeHeader?(meta)`——为 `SessionPersistence.ensureMaterialized(session)` 显式持久化仅含 header 的会话。它只供把空会话本身视为可恢复持久资源的生命周期前端使用;[标准 ACP 自动化控制](../feature/2026-08-22-standard-acp-automation-controls.zh.md)是第一个 consumer。支持该生命周期的后端实现此钩子;惰性创建仍是默认行为。 - `commitRepair(meta, tornMarker, closers)`——使崩溃修复持久化:截断损坏的尾部(当且仅当 `tornMarker !== undefined`)并追加 `closers`。**不要求原子性**——JSONL 合理地分两步 fsync(先截断再追加),SQLite 在一个事务中完成 DELETE+INSERT。用于 `prepare`/`load`(截断 + 合成收尾事件)和存活会话接管(仅截断,`closers = []`)。 - `list()`——列出所有已存储的元数据。 - `close?()`——可选的生命周期清理(SQLite 关闭 db 句柄;JSONL 省略),在 dispose effect 中于排空至完全停稳之后被 await,因此 close 失败不会掩盖排空错误。 @@ -44,4 +49,4 @@ Status: implemented ## 后果 -协调器增加了一层间接、一个不透明的 torn marker、脱离会话生命周期的退役任务,以及有界的已准备 Session 状态,但将此前每个后端重复的、对正确性要求很高的编排逻辑集中到一处。会话 dispose 仍是仅观察事件,因此会话所有者不会等待持久化退役;协调器会收容失败、在存活控制器中保留待处理事件,并以后端 teardown 为完全停稳边界。其钩子面保持窄小:标识校验、接管、碰撞检查、准备与不可变检查共用 `loadStored`;物化保持在 `appendBatch` 内原子完成;列举绕过协调器。读模型使用 `inspect` 而非 `load`,因此观察已持久化但仍开放的轮次时不会提交中断收尾事件;复用、预留与发布由 [Session 准备阶段决策](2026-08-05-session-preparation.md)定义。新后端只需实现存储原语,而无需复制有界写入生命周期。 +协调器增加了一层间接、一个不透明的 torn marker、脱离会话生命周期的退役任务,以及有界的已准备 Session 状态,但将此前每个后端重复的、对正确性要求很高的编排逻辑集中到一处。会话 dispose 仍是仅观察事件,因此会话所有者不会等待持久化退役;协调器会收容失败、在存活控制器中保留待处理事件,并以后端 teardown 为完全停稳边界。其钩子面保持窄小:标识校验、接管、碰撞检查、准备与不可变检查共用 `loadStored`;物化保持在 `appendBatch` 内原子完成;列举绕过协调器。读模型使用 `inspect` 而非 `load`,因此观察已持久化但仍开放的轮次时不会提交中断收尾事件;复用、预留与发布由 [Session 准备阶段决策](2026-08-05-session-preparation.zh.md)定义。新后端只需实现存储原语,而无需复制有界写入生命周期。 diff --git a/.agents/notes/implemented/architecture/2026-06-20-branded-ids.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-20-branded-ids.i18n.yaml index b7e2678928..d929b209e3 100644 --- a/.agents/notes/implemented/architecture/2026-06-20-branded-ids.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-20-branded-ids.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-20-branded-ids.md -2026-06-20-branded-ids.md: 29b258b21240c92e74051339f0939a8e70933099 -2026-06-20-branded-ids.zh.md: 288125764d4b279c79b7469080bb0c9f2efdc709 +2026-06-20-branded-ids.md: dda97bbf546ef99083cbe3bd2c7da39070407e04 +2026-06-20-branded-ids.zh.md: 82b79dff9d5018e2ea9f9969148eb25225f6d727 diff --git a/.agents/notes/implemented/architecture/2026-06-20-branded-ids.md b/.agents/notes/implemented/architecture/2026-06-20-branded-ids.md index 29b258b212..dda97bbf54 100644 --- a/.agents/notes/implemented/architecture/2026-06-20-branded-ids.md +++ b/.agents/notes/implemented/architecture/2026-06-20-branded-ids.md @@ -6,7 +6,7 @@ English | [中文](2026-06-20-branded-ids.zh.md) ## Problem -The harness brands `CallId` (`packages/llm/llm/src/brand.ts`) and the shared agent/session `SessionId` (`packages/core/session/src/types.ts`) using the `Branded = string & { readonly [BRAND]: B }` machinery (owned by the type-only `@deepseek-ai/dsh-brand` package at `packages/util/brand/` — see its [README](../../../../packages/util/brand/README.md)) and a zero-cost cast factory per type. `dsh-brand` also states the governing policy: *"Branding is for ids that cross package boundaries and could plausibly be confused; not every string needs a brand."* That policy is right; the problem is that it is only half-applied. Two gaps let a structurally-identical-but-semantically-wrong string slip through the type checker today. +The harness brands `CallId` (`packages/llm/llm/src/brand.ts`) and the shared agent/session `SessionId` (`packages/core/session/src/types.ts`) using the `Branded = string & { readonly [BRAND]: B }` machinery (owned by the type-only `@deepseek-ai/dsh-brand` package at `packages/util/brand/` — see its [README](../../../../packages/util/brand/README.md)) and a zero-cost cast factory per type. `dsh-brand` also states the governing policy: *"Branding is for ids that cross package boundaries and could plausibly be confused; not every string needs a brand."* That policy is right; the problem is that it is only half-applied. Two gaps let a structurally-identical-but-semantically-wrong string slip through the type checker. **Gap 1 — unbranded cross-boundary IDs in the bash seam.** The background-job id is a plain `string`: `BashTask.id: string` (`packages/shell/shell/src/types.ts`), carried as `string` through the whole executor seam (`ShellExecutor.get`/`ownerOf`/`readOutput`/`kill(id: string)` in `packages/shell/shell/src/index.ts`) and validated/passed as `string` by the model-facing tools (`validateJobId`, `assertTaskAccess`, the `job_id` schema arg in `packages/shell/tool-bash/src/index.ts`). It is generated by a per-executor counter — `` `bash-${this.nextTaskId++}` `` in `packages/shell/bash-local/src/index.ts` — which gives it **exactly the same `name-N` shape as `SessionId`'s default** (`` `session-${++counter}` `` in `packages/core/session/src/index.ts`). A bash job id and a session id are trivially swappable at a call site and the compiler says nothing. It is a model-facing id (the model passes `job_id` back to `bash_output`/`bash_kill`), so a confusion here is reachable from untrusted input. @@ -56,7 +56,7 @@ Kept deliberately narrow per the "not every string needs a brand" policy. Each o - **`ToolName`** (the `ToolRuntime` key) — author-defined, human-readable, and rarely confused with another id; the weakest candidate, likely not worth a brand. - **`ErrorCode`** (`HarnessError.code`) — a closed vocabulary (`ABORTED`, `NO_ADAPTER`, …), not a per-instance id; better served by a string-literal union than a brand, if anything. - **Numeric ordinals** — turn number, step number, and the event `seq` are `number`, not `string`, so `Branded` does not apply; a parallel `number & { readonly [BRAND]: B }` variant could brand them, but they are positional ordinals rarely passed across boundaries, so the payoff is low. -- **Validated construction** — the brand factories are pure casts with no runtime check, and every boundary (ACP `sessionId`, provider-issued `call.id`, the empty-string fallback in `dsh-llm-deepseek`) trusts the raw string today. A `SessionId.parse()` / `isValid()` companion that throws on malformed input at boundaries is a genuine gap, but it is a *runtime-behavior* change with its own design (what is "malformed"? what happens on failure?) and belongs in its own decision, not bundled into this type-only change. +- **Validated construction** — the brand factories are pure casts with no runtime check, and every boundary (ACP `sessionId`, provider-issued `call.id`, the empty-string fallback in `dsh-llm-deepseek`) trusts the raw string. A `SessionId.parse()` / `isValid()` companion that throws on malformed input at boundaries is a genuine gap, but it is a *runtime-behavior* change with its own design (what is "malformed"? what happens on failure?) and belongs in its own decision, not bundled into this type-only change. ## Verification diff --git a/.agents/notes/implemented/architecture/2026-06-20-branded-ids.zh.md b/.agents/notes/implemented/architecture/2026-06-20-branded-ids.zh.md index 288125764d..82b79dff9d 100644 --- a/.agents/notes/implemented/architecture/2026-06-20-branded-ids.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-20-branded-ids.zh.md @@ -6,11 +6,11 @@ Status: implemented ## 问题 -harness 使用 `Branded = string & { readonly [BRAND]: B }` 机制,为 `CallId`(`packages/llm/llm/src/brand.ts`)和 agent(智能体)/会话共享的 `SessionId`(`packages/core/session/src/types.ts`)做 brand 处理;该机制由纯类型包 `@deepseek-ai/dsh-brand` 拥有,位于 `packages/util/brand/`,见其 [README](../../../../packages/util/brand/README.md),并为每个类型提供零开销的 cast 工厂。`dsh-brand` 还声明了治理策略:*「Branding 用于跨包边界且可能被混淆的 id;不是每个 string 都需要 brand。」* 这条策略是正确的;问题在于它只落实了一半。两处缺口使得结构相同但语义错误的 string 今天仍能通过类型检查器。 +harness 使用 `Branded = string & { readonly [BRAND]: B }` 机制,为 `CallId`(`packages/llm/llm/src/brand.ts`)和 agent(智能体)/会话共享的 `SessionId`(`packages/core/session/src/types.ts`)做 brand 处理;该机制由纯类型包 `@deepseek-ai/dsh-brand` 拥有,位于 `packages/util/brand/`,见其 [README](../../../../packages/util/brand/README.zh.md),并为每个类型提供零开销的 cast 工厂。`dsh-brand` 还声明了治理策略:*「Branding 用于跨包边界且可能被混淆的 id;不是每个 string 都需要 brand。」* 这条策略是正确的;问题在于它只落实了一半。两处缺口使得结构相同但语义错误的 string 仍能通过类型检查器。 **缺口 1:bash seam 中未 brand 的跨边界 ID。** 后台 job id 是普通 `string`:`BashTask.id: string`(`packages/shell/shell/src/types.ts`),作为 `string` 贯穿整个执行器 seam(`packages/shell/shell/src/index.ts` 中的 `ShellExecutor.get`/`ownerOf`/`readOutput`/`kill(id: string)`),再由面向模型的工具以 `string` 校验并传递(`validateJobId`、`assertTaskAccess`、`packages/shell/tool-bash/src/index.ts` 中 `job_id` 的 schema 参数)。它由每执行器计数器生成——`packages/shell/bash-local/src/index.ts` 中的 `` `bash-${this.nextTaskId++}` ``——其形状与 `SessionId` 的默认值**完全相同,都是 `name-N`**(`packages/core/session/src/index.ts` 中的 `` `session-${++counter}` ``)。bash job id 和会话 id 在调用点轻易就能互换,而编译器毫无反应。它是面向模型的 id(模型会把 `job_id` 传回 `bash_output`/`bash_kill`),所以该混淆可由不受信任的输入触达。 -bash **owner token** 是相关的子情形:`ShellExecRequest.owner?: string` 和 `ShellExecSpec.owner: string | undefined`(`packages/shell/shell/src/types.ts`)被文档描述为刻意*不透明*的隔离键,但在所有实际调用方中,该值就是所属 agent 共享的 `Agent.id`/`SessionId`(`callerToken = (exec) => exec.agent?.id`,位于 `packages/shell/tool-bash/src/index.ts`),只是披着另一个 seam 本地名称。它被用于访问控制比较(`owner !== callerToken(exec)`),因此一个不匹配但类型正确的 string 在此处就是跨会话隔离 bug,而当前类型系统无法捕获。这正是[统一 agent/session 标识决策](../simplification/2026-06-20-unify-agent-and-session-id.md)覆盖的共享 id 别名。 +bash **owner token** 是相关的子情形:`ShellExecRequest.owner?: string` 和 `ShellExecSpec.owner: string | undefined`(`packages/shell/shell/src/types.ts`)被文档描述为刻意*不透明*的隔离键,但在所有实际调用方中,该值就是所属 agent 共享的 `Agent.id`/`SessionId`(`callerToken = (exec) => exec.agent?.id`,位于 `packages/shell/tool-bash/src/index.ts`),只是披着另一个 seam 本地名称。它被用于访问控制比较(`owner !== callerToken(exec)`),因此一个不匹配但类型正确的 string 在此处就是跨会话隔离 bug,而当前类型系统无法捕获。这正是[统一 agent/session 标识决策](../simplification/2026-06-20-unify-agent-and-session-id.zh.md)覆盖的共享 id 别名。 **缺口 2:*已经 brand* 的 ID 在边界处被侵蚀。** 就连 `CallId` 和 `SessionId` 也恰好在最容易混淆的地方退化为裸 `string`:注册表/store 键类型和公开方法参数。代表性位置包括会话存储、agent 注册表(二者都以共享的 `SessionId` 为键)、工具展示层的 call-id map、ACP(Agent Client Protocol)的会话记录,以及持久化协调器。在集合键处丢弃 brand,会让既有 brand 在查找时毫无价值;它们的价值只实现了一部分。 @@ -56,7 +56,7 @@ export function OwnerToken(id: string): OwnerToken { - **`ToolName`**(`ToolRuntime` 的键):由作者定义、人类可读,且很少与其他 id 混淆;最弱的候选,可能不值得加 brand。 - **`ErrorCode`**(`HarnessError.code`):一个封闭词汇(`ABORTED`、`NO_ADAPTER`……),不是逐实例的 id;如果要做,string 字面量联合类型比 brand 更合适。 - **数值序号**:轮次号、步骤号和事件 `seq` 是 `number` 而非 `string`,`Branded` 不适用;可以用并行的 `number & { readonly [BRAND]: B }` 变体来 brand 它们,但它们是位置序号、很少跨边界传递,收益较低。 -- **带校验的构造**:brand 工厂是纯 cast,无运行时检查,且每个边界(ACP `sessionId`、提供方签发的 `call.id`、`dsh-llm-deepseek` 中的空字符串回退)今天都信任裸 string。一个在边界处对格式错误的输入抛异常的 `SessionId.parse()` / `isValid()` 配套工具确实是缺口,但它是*运行时行为*变更,有自己的设计问题(什么算「格式错误」?失败时会怎样?),应在独立决策中处理,不应捆绑进这次纯类型变更。 +- **带校验的构造**:brand 工厂是纯 cast,无运行时检查,且每个边界(ACP `sessionId`、提供方签发的 `call.id`、`dsh-llm-deepseek` 中的空字符串回退)都信任裸 string。一个在边界处对格式错误的输入抛异常的 `SessionId.parse()` / `isValid()` 配套工具确实是缺口,但它是*运行时行为*变更,有自己的设计问题(什么算「格式错误」?失败时会怎样?),应在独立决策中处理,不应捆绑进这次纯类型变更。 ## 验证 @@ -64,6 +64,6 @@ export function OwnerToken(id: string): OwnerToken { ## 后果 -- **两个接口面的机械性改动。** 传播 brand 涉及 bash seam(Service Definition + Service Provider + Consumer)以及 ACP 会话 id 接口和持久化协调器。改动面广但严重度低:遗漏的位置是编译错误而非静默 bug。从可观察行为看,这是一项纯类型变更——无快照或 e2e 行为差异。它与[统一 agent/会话标识决策](../simplification/2026-06-20-unify-agent-and-session-id.md)相邻,因为二者都触及会话 id / owner-token 边界;`OwnerToken` 出于上述解耦理由仍与统一后的 id 保持独立。 +- **两个接口面的机械性改动。** 传播 brand 涉及 bash seam(Service Definition + Service Provider + Consumer)以及 ACP 会话 id 接口和持久化协调器。改动面广但严重度低:遗漏的位置是编译错误而非静默 bug。从可观察行为看,这是一项纯类型变更——无快照或 e2e 行为差异。它与[统一 agent/会话标识决策](../simplification/2026-06-20-unify-agent-and-session-id.zh.md)相邻,因为二者都触及会话 id / owner-token 边界;`OwnerToken` 出于上述解耦理由仍与统一后的 id 保持独立。 - **Brand 不做校验。** Brand 是混淆防护,不是正确性证明:一个*错误的*会话 id 只要仍是格式正确的 string,就和以前一样能通过类型检查器。本决策不关闭这个缺口(见「不在范围内」)——它只阻止这类*类别*错误:传入错误*种类*的 id。 - **「在哪里停下」仍是判断题。** 为 `BashTaskId` 加 brand 但不为 `ToolName` 加,为 `OwnerToken` 加但不为 `ModelId` 加,是对哪些 string「可能被混淆」的品味判断。合理的评审者可能想要更多或更少;`brand.ts` 中的策略是裁决依据,本决策倾向于面向模型或用于访问控制的 id。 diff --git a/.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.i18n.yaml index 477c6df3ae..7d4ccbd4e1 100644 --- a/.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md 2026-06-20-generic-long-running-tool-runtime.md: 7db43323dd83f8e99698a7163412c9b33a607dfb -2026-06-20-generic-long-running-tool-runtime.zh.md: efd306cc8d3b82749635dde302af45235c27b431 +2026-06-20-generic-long-running-tool-runtime.zh.md: 040ea6a01e97c9dc1d93e8f3a0fd9c1f4b0f31fc diff --git a/.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.zh.md b/.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.zh.md index efd306cc8d..040ea6a01e 100644 --- a/.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.zh.md @@ -19,13 +19,13 @@ Status: implemented 长时间运行工具是生产方。`dsh-tool-bash` 将 `ShellProcess` 适配为增量输出与进程取消;`dsh-tool-subagent` 将子运行适配为最终输出与子运行释放。bash 与 subagent 能力 seam 保持独立,不依赖会话或任务注册表。 -`JobRegistry` 是 `@deepseek-ai/dsh-jobs` 中的 Service Definition;进程内 Service Provider 是 `@deepseek-ai/dsh-jobs-local` 中的 `LocalJobRegistry`(该拆分记录在[任务注册表约定 Agent Note](2026-07-26-job-registry-seam.md)中)。 +`JobRegistry` 是 `@deepseek-ai/dsh-jobs` 中的 Service Definition;进程内 Service Provider 是 `@deepseek-ai/dsh-jobs-local` 中的 `LocalJobRegistry`(该拆分记录在[任务注册表约定 Agent Note](2026-07-26-job-registry-seam.zh.md)中)。 ## 运行时约定 -字面类型见[任务子系统页面](../../../../docs/subsystems/jobs.md)。生产方调用 `ctx.jobs.start()`,传入 kind、label、可选的所属 `Agent`、可选的正数 `outputLimitBytes` 与一个 `run()` 函数。运行时会在调用 `run()` 前完成所有可能失败的预检工作,并且只调用一次。`run()` 返回钩子后,注册过程不会再执行可能失败的步骤而直接提交;生产方无法启动没有可收集 job id 的工作。 +字面类型见[任务子系统页面](../../../../docs/subsystems/jobs.zh.md)。生产方调用 `ctx.jobs.start()`,传入 kind、label、可选的所属 `Agent`、可选的正数 `outputLimitBytes` 与一个 `run()` 函数。运行时会在调用 `run()` 前完成所有可能失败的预检工作,并且只调用一次。`run()` 返回钩子后,注册过程不会再执行可能失败的步骤而直接提交;生产方无法启动没有可收集 job id 的工作。 -进程内 Service Provider 还拥有有界准入,其理由记录在[有界后台任务准入决策](../bug-fix/2026-08-11-bounded-background-job-admission.md)中。它的 `maxConcurrentJobsPerOwner` 配置必须是正的安全整数,默认值为 `10`;`start()` 从 `running` 与 `stopping` 记录派生每个确切 `Agent` 对象的活动数量,而全部无 owner 任务共享一个服务级桶。容量拒绝发生在 `run()` 与 id 分配之前,处于 stopping 的任务只有在生产方 `done` 结算时才释放名额。Service Provider 不排队或抢占任务,也不保留第二份可变计数。 +进程内 Service Provider 还拥有有界准入,其理由记录在[有界后台任务准入决策](../bug-fix/2026-08-11-bounded-background-job-admission.zh.md)中。它的 `maxConcurrentJobsPerOwner` 配置必须是正的安全整数,默认值为 `10`;`start()` 从 `running` 与 `stopping` 记录派生每个确切 `Agent` 对象的活动数量,而全部无 owner 任务共享一个服务级桶。容量拒绝发生在 `run()` 与 id 分配之前,处于 stopping 的任务只有在生产方 `done` 结算时才释放名额。Service Provider 不排队或抢占任务,也不保留第二份可变计数。 `outputLimitBytes` 是生产方拥有的呈现策略,而非注册表缓冲区。注册表校验该值,并将其原样投影到 `JobSnapshot`;通用任务控制器添加自身的状态或通知元数据后,再将该上限应用于完整的面向模型输出。省略该值时保持现有控制器行为,因此运行时不会向无关的生产方类别施加隐式默认值。 @@ -79,7 +79,7 @@ job id 在运行时全局可见且可预测,因此注册表会授权每次访 流式读取共享一个任务作用域内的消费游标,因为所属模型是预期读取方。UI 或多个独立读取方需要单独的非消费式观察 API;共享该游标会让读取方彼此消费对方的输出。 -系统提示词要求模型保留 job id、在后台工作运行时继续处理独立工作而非忙轮询或重复启动同一任务、在给出最终答案前收集相关任务,并终止不再重要的工作。完成时,系统会向确切所有者的会话交付一条已记录的消息:繁忙的所有者走注入,空闲的所有者会被唤醒,其有界策略由[空闲所有者唤醒决策](../feature/2026-08-11-background-job-completion-wakes-an-idle-owner.md)负责。 +系统提示词要求模型保留 job id、在后台工作运行时继续处理独立工作而非忙轮询或重复启动同一任务、在给出最终答案前收集相关任务,并终止不再重要的工作。完成时,系统会向确切所有者的会话交付一条已记录的消息:繁忙的所有者走注入,空闲的所有者会被唤醒,其有界策略由[空闲所有者唤醒决策](../feature/2026-08-11-background-job-completion-wakes-an-idle-owner.zh.md)负责。 当读取或等待交付终止任务、尚在等待的等待方在结算时认领了投递,或模型显式终止任务时,运行时将终止任务标为 `reported`。已报告的任务不会注入冗余的完成通知。监听器失败会独立记录,不会阻止后续监听器,也不会被等待方或资源销毁过程等待。当快照携带 `outputLimitBytes` 时,`dsh-tool-jobs` 会保持 UTF-8 边界,并复用生产方已有的截断标记,而不会重复添加。读取会为状态后缀预留空间并保留输出尾部;完成通知会先为稳定的 `background job ` 前缀与 `job_output` 指令预留空间,再截断可变的 kind、label、status、detail,乃至截断标记本身,因此 PTY 的最小上限仍能标识需要收集的任务。任务控制器在策略有机会拒绝或短路分发之前,于最先执行的 pre-execute 监听器中解析调用方可见的生产方上限;随后通过任务定义最后一道的 `finalizeContent` 回调应用该上限,使规范化的工具错误、外层流水线失败与单文本策略结果都无法绕过该边界;经特意结构化的多块策略结果仍由策略拥有其形状与大小。 @@ -105,7 +105,7 @@ bash seam 暴露 `resolve`、`run` 和 `start`。`start(spec)` 返回一个 `She ### 立即抽象任务运行时后端 -当前 `JobStart.run()` 约定传入进程内回调与确切的 `Agent` 对象。持久化后端会改变身份、重启、所有权与观察语义,因此在引入之时注册表保持为单一具体服务,而非固化错误的边界。[任务注册表约定 Agent Note](2026-07-26-job-registry-seam.md)后来在不改变这些进程内语义的前提下,将约定与进程内实现分离。 +当前 `JobStart.run()` 约定传入进程内回调与确切的 `Agent` 对象。持久化后端会改变身份、重启、所有权与观察语义,因此在引入之时注册表保持为单一具体服务,而非固化错误的边界。[任务注册表约定 Agent Note](2026-07-26-job-registry-seam.zh.md)后来在不改变这些进程内语义的前提下,将约定与进程内实现分离。 ### 由消费方负责授权或清理事件 @@ -131,7 +131,7 @@ bash seam 暴露 `resolve`、`run` 和 `start`。`start(spec)` 返回一个 `She ## 后果 -bash 命令与 subagent 共享一套 id 词汇、列表、通知格式、提示词习惯和控制工具。新的长时间运行生产方只需实现执行钩子,而不必再实现一套注册表与工具族。[工具实操手册](../../../../docs/cookbook/adding-a-tool.md)将生产方指向本约定。 +bash 命令与 subagent 共享一套 id 词汇、列表、通知格式、提示词习惯和控制工具。新的长时间运行生产方只需实现执行钩子,而不必再实现一套注册表与工具族。[工具实操手册](../../../../docs/cookbook/adding-a-tool.zh.md)将生产方指向本约定。 单个确切 owner 无法再无限增加进程内由 Task 承载的工作,另一个 owner 也不会消耗它的额度。取消请求会继续占用容量,直到生产方真正释放资源,因此用新工作替换缓慢停止的任务不会突破已配置的实时资源预算。 diff --git a/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.i18n.yaml index f9b830c3b7..dcaeb8c34b 100644 --- a/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md -2026-06-21-bounded-llm-request-recovery.md: 3a91d77fe9c08fbcc3afdce4dad9a38e288b4723 -2026-06-21-bounded-llm-request-recovery.zh.md: a6b342b19236251e638242c1e9df6b3d6ed557c0 +2026-06-21-bounded-llm-request-recovery.md: e725a025f2d8b0d5e8eaf4137f07d8eab4448bf4 +2026-06-21-bounded-llm-request-recovery.zh.md: 9e2263b05888797eaaaeb82859730d1c4a728cea diff --git a/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md index 3a91d77fe9..e725a025f2 100644 --- a/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md +++ b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md @@ -4,13 +4,13 @@ Status: implemented English | [中文](2026-06-21-bounded-llm-request-recovery.zh.md) -The [per-provider request retry policy](../feature/2026-07-24-provider-retry-policies.md) extends this foundation with exact-provider configuration and an explicit unbounded mode. This note continues to own structured failure facts, the closed-step recovery boundary, normal mode's transient defaults, visible single attempts, and durable retry status. [Terminal LLM stream failures](2026-07-29-terminal-llm-stream-failures.md) supersedes its thrown-error identity and stream-sidecar mechanism. +The [per-provider request retry policy](../feature/2026-07-24-provider-retry-policies.md) extends this foundation with exact-provider configuration and an explicit unbounded mode. This note continues to own structured failure facts, the failed-attempt recovery boundary, normal mode's transient defaults, visible single attempts, and durable retry status. [Terminal LLM stream failures](2026-07-29-terminal-llm-stream-failures.md) supersedes its thrown-error identity and stream-sidecar mechanism. ## Problem Provider adapters can fail by throwing during dispatch or iteration or by ending with `finish { kind: 'error' | 'aborted' }`. The final adapter boundary normalizes thrown values to that terminal finish protocol before `dsh-agent-loop` receives them; middleware and result-processing defects remain thrown. The loop offers a terminal model-request failure to `agent/request-error`. An unhandled failure is terminal; a handling listener repairs policy-owned state, returns `{ kind: 'retry' }`, and stops waterfall delegation. The [retry-action decision](../simplification/2026-07-27-request-error-retry-action.md) owns this return contract. -That boundary is already safe for another request attempt. Raw `assistant/chunk` events carry the failed `turn` and `step`, message derivation ignores them unless a successful `assistant/message` cites them, tool calls are dispatched only after a successful terminal finish and assembly, and a retry opens a new numbered turn from the durable log. The harness therefore does not need a second response lifecycle or tentative-output protocol to keep two attempts separate. +That boundary is already safe for another request attempt. Raw `assistant/chunk` events carry the failed `turn` and `step`, message derivation ignores them unless a successful `assistant/message` cites them, tool calls are dispatched only after a successful terminal finish and assembly, and a retry reconstructs its next attempt from the durable log. The harness therefore does not need a second response lifecycle or tentative-output protocol to keep two attempts separate. The prior boundary left three narrower gaps. @@ -52,7 +52,7 @@ The shared transient-code set is intentionally small: adapter mappings for `RATE `@deepseek-ai/dsh-llm-retry` is a function plugin that listens to `agent/request-error`. It introduces no service or new loop branch; the agent-loop package changes only the data carried through its existing failed-step recovery control flow. -The `agent/request-error` waterfall carries the current `LlmFailure`, an immutable list of prior failures that authorized retry turns in the consecutive recovery sequence, and the serving registration's immutable retry policy. The loop transports but does not interpret that policy, owns the consecutive failure history, and clears it after a successful model request. Normal `dsh-llm-retry` policy counts durable retry records scheduled by the same exact-provider policy, while `dsh-compaction-basic` keeps its own context-overflow budget. Alternating transient and context-overflow failures therefore consume their owning finite budgets independently; the maximum request count is one plus the sum of the loaded finite budgets. +The `agent/request-error` waterfall carries the current `LlmFailure`, an immutable list of prior failures that authorized retries in the consecutive recovery sequence, and the serving registration's immutable retry policy. The loop transports but does not interpret that policy, owns the consecutive failure history, and clears it after a successful model request. Normal `dsh-llm-retry` policy counts durable retry records scheduled by the same exact-provider policy, while `dsh-compaction-basic` keeps its own context-overflow budget. Alternating transient and context-overflow failures therefore consume their owning finite budgets independently; the maximum request count is one plus the sum of the loaded finite budgets. The [provider-policy decision](../feature/2026-07-24-provider-retry-policies.md) owns the current configuration shape. Provider adapters register their nested `retryPolicy`; omission uses normal defaults: two transient retries, a 500 millisecond initial delay, a 10 second delay cap, 10 percent jitter, and the five transient codes above. The count and delay bounds match the conservative edge of the inspected implementations: [OpenCode uses two request retries with 500 ms/10 s bounds](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39), [Pi separates three agent-level retries from provider retries and defaults provider retries to zero](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147), and [Codex uses finite request/stream budgets plus a five-minute idle timeout](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33). Ten percent follows [Codex's bounded jitter](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47). @@ -68,7 +68,7 @@ The agent-spine demo bundle loads the plugin so the shared stdio/TUI, one-shot C ### Make one layer own visible attempts -Adapters perform one provider request per `stream()` call. The pi-ai adapter removes public `maxRetries` and `maxRetryDelayMs` profile fields and disables library retries; the hand-written adapter keeps its current single-attempt behavior. This prevents an SDK budget from multiplying the agent budget and ensures every transient retry is represented by a closed failed step plus `llm/retry`. +Adapters perform one provider request per `stream()` call. The pi-ai adapter removes public `maxRetries` and `maxRetryDelayMs` profile fields and disables library retries; the hand-written adapter keeps its current single-attempt behavior. This prevents an SDK budget from multiplying the agent budget and ensures every transient retry is represented by its recorded failed attempt plus `llm/retry`. `ctx.llm.stream()` remains the raw one-attempt waterfall. Direct callers such as compaction summarization receive the structured failure but do not gain automatic retry, because they have no agent step boundary or general durable place to separate attempts. A future direct-call consumer may justify a buffering helper that retries only before emitting a chunk; this decision adds no such helper. @@ -82,9 +82,9 @@ Boundary tests prove termination at both actual transports. The hand-written ada ### Keep attempts separate in the existing log -A failed attempt may leave `assistant/chunk` events in its closed step, but it never appends `assistant/message` and never dispatches a tool. A retry closes the failed turn, opens the next numbered turn, reconstructs the request from the durable surface, and produces its own chunks. UIs may render live chunks while a step is open, then mark or clear that transient view when `llm/retry` identifies the failed step or `turn/end` records failure. Web validates the complete retry payload contract, clears the failed partial at `llm/retry`, projects consecutive retry-turn events into one stable row updated to the latest attempt, and derives scheduled, started, or cancelled status from subsequent turn facts. Its countdown anchors the scheduled delay to browser receipt rather than the Host event clock, uses ceiling-rounded seconds with a one-second floor, animates only while unresolved, and keeps exact latest failure details collapsed behind the row. Retry nodes anchor their own trajectory turn even when the failed attempt has no assistant node. Message derivation continues to ignore the failed chunks, and Web applies the same projection during history rebuild so refreshing cannot resurrect discarded partials or duplicate retry rows. +A failed attempt may leave `assistant/chunk` events in its step, but it never appends `assistant/message` and never dispatches a tool. A retry continues inside the failing turn and step, reconstructs the request from the durable surface, and produces its own chunks; only the final outcome closes the turn. UIs may render live chunks while a step is open, then mark or clear that transient view when `llm/retry` identifies the failed attempt or `turn/end` records failure. Web validates the complete retry payload contract, clears the failed partial at `llm/retry`, projects each producer-correlated `retryId` chain into one stable row updated to the latest attempt, and derives scheduled, started, or cancelled status from `llm/retry-started` and the owning turn and step boundaries' closure. Its countdown anchors the scheduled delay to browser receipt rather than the Host event clock, uses ceiling-rounded seconds with a one-second floor, animates only while unresolved, and keeps exact latest failure details collapsed behind the row. Retry nodes anchor their own trajectory turn even when the failed attempt has no assistant node. Message derivation continues to ignore the failed chunks, and Web applies the same projection during history rebuild so refreshing cannot resurrect discarded partials or duplicate retry rows. -If recovery is exhausted, the final failure is stored once on `turn/end.reason` with the structured facts. Web derives one `turn-error` node at that sequence position and renders its display-safe message and optional code inline; AUTH projections replace provider copy that may echo credential fragments with `API key is invalid`, while the raw diagnostic remains in the session log. The same fold runs for live events and history replay. If transient recovery continues, `llm/retry` is the durable home for that attempt's failure and delay, so its failed turn does not also gain a terminal error row. No standalone final-error event or response-id vocabulary is added. +If recovery is exhausted, the final failure is stored once on `turn/end.reason` with the structured facts. Web derives one `turn-error` node at that sequence position and renders its display-safe message and optional code inline; AUTH projections replace provider copy that may echo credential fragments with `API key is invalid`, while the raw diagnostic remains in the session log. The same fold runs for live events and history replay. While transient recovery continues, `llm/retry` is the durable home for each intermediate failure and delay; the terminal row exists only once `turn/end` records the error, and because exhausted recovery shares the failing turn, the turn's retry history never suppresses that row — the settled retry chain and the terminal error render side by side. No standalone final-error event or response-id vocabulary is added. ## Out of scope @@ -114,15 +114,15 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason` - Each provider adapter validates its nested retry policy at Loader startup, and `ctx.llm` captures it with the route; normal mode delegates ineligible paths and makes at most `maxRetries + 1` provider requests when no other policy applies. - HMR-during-backoff tests prove disposal unregisters the listener, aborts and awaits its captured callbacks, emits no retry decision after disposal, and leaves no timer or promise alive. - Pure unit tests cover transient-code selection, exponential backoff and jitter bounds, valid and over-cap `Retry-After`, exhausted budgets, deterministic timer/random hooks, and abort during backoff. -- Real agent-loop tests cover failure before chunks, partial chunks then failure, thrown and in-band failures, retry to success in a new turn, exhaustion to structured `turn/end.reason`, and composition with `dsh-compaction-basic` context-overflow recovery. +- Real agent-loop tests cover failure before chunks, partial chunks then failure, thrown and in-band failures, retry to success inside the same turn, exhaustion to structured `turn/end.reason`, and composition with `dsh-compaction-basic` context-overflow recovery. - The partial-chunk integration test proves failed chunks remain attributed to the failed step, no assistant message or tool side effect is committed for that step, and the successful retry records its own chunk seqs and provider/model route. -- The plugin-owned `llm/retry` event is non-surface, survives JSONL and SQLite round trips, is ignored by message derivation, and drives TUI and Web retraction plus scheduled-retry rendering. Client tests cover complete wire validation, clock-independent countdown, cancellation versus completed retry labels, and trajectory attribution; keyless UI snapshots cover Web scheduling and success, a real Web composition test covers partial transport failure through recovery, and ACP automation snapshots confirm that a discarded attempt stays off the wire while the recovered reply is emitted. +- The plugin-owned `llm/retry` event is non-surface, survives JSONL and SQLite round trips, is ignored by message derivation, and drives TUI and Web retraction plus scheduled-retry rendering. Client tests cover complete wire validation, clock-independent countdown, cancellation versus completed retry labels, and trajectory attribution; keyless UI snapshots cover Web scheduling and success, real Web composition tests cover partial transport failure through recovery and exhausted recovery's terminal error row beside the settled retry chain, and ACP automation snapshots confirm that a discarded attempt stays off the wire while the recovered reply is emitted. - Idle-watchdog tests prove the stable signal is rearmed only while `next()` is outstanding, disarmed during consumer think time and in `finally`, and classified separately from a total-call deadline and an earlier caller abort; adapter tests prove the signal stops the underlying request rather than merely detaching it. - Direct `ctx.llm.stream()` callers remain single-attempt and receive the same structured failure facts. ## Consequences -- Every retry attempt is visible as a closed failed turn plus `llm/retry`, and adapter-level single-attempt behavior prevents hidden SDK retries from multiplying policy decisions. A retry can still duplicate provider billing even when no chunk arrived; normal mode limits that risk, while explicit always mode accepts it until cancellation or success. +- Every retry attempt is visible inside its owning turn as the failed attempt's chunks plus `llm/retry`, and adapter-level single-attempt behavior prevents hidden SDK retries from multiplying policy decisions. A retry can still duplicate provider billing even when no chunk arrived; normal mode limits that risk, while explicit always mode accepts it until cancellation or success. - Provider SDKs may hide status or retry headers. Those adapters retain the stable facts they expose and otherwise use a coarse code rather than letting recovery policy parse fragile text. - Durable retry events expand the session protocol and UI state machine. Shipping the event and its consumer together prevents an unused telemetry vocabulary, but later schema changes still require persistence and replay work. - Clearing a failed step's live chunks can visibly retract output. That is preferable to presenting discarded text or partial tool JSON as committed history, and snapshots pin the transition. @@ -136,3 +136,4 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason` - [Timeout deadline library](../../implemented/architecture/2026-07-06-timeout-deadline-library.md) separates shared deadline classification from capability-owned termination. - [After-call compaction pressure and context-overflow recovery](../../implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md) owns the current closed-step request-recovery extension point and bounded overflow retry. - [Provider-routed LLM adapters](../../implemented/architecture/2026-07-14-provider-routed-llm-adapters.md) owns explicit provider/model routing and the one-adapter-per-provider invariant. +- [Terminal turn errors survive same-turn retry history](../bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.md) owns the removal of the Web retry-history suppression that hid exhausted recovery's terminal error row. diff --git a/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.zh.md b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.zh.md index a6b342b192..9e2263b058 100644 --- a/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.zh.md @@ -4,13 +4,13 @@ Status: implemented [English](2026-06-21-bounded-llm-request-recovery.md) | 中文 -[按提供方配置的请求重试策略](../feature/2026-07-24-provider-retry-policies.md)在此基础上增加了确切提供方配置与显式无界 mode。本说明继续负责结构化失败事实、已关闭步骤的恢复边界、normal mode 的暂时性默认值、可见的单次尝试和持久重试状态。[LLM(大语言模型)流的终止失败](2026-07-29-terminal-llm-stream-failures.md)取代了其中关于抛出错误身份和流 sidecar 的机制。 +[按提供方配置的请求重试策略](../feature/2026-07-24-provider-retry-policies.zh.md)在此基础上增加了确切提供方配置与显式无界 mode。本说明继续负责结构化失败事实、失败尝试的恢复边界、normal mode 的暂时性默认值、可见的单次尝试和持久重试状态。[LLM(大语言模型)流的终止失败](2026-07-29-terminal-llm-stream-failures.zh.md)取代了其中关于抛出错误身份和流 sidecar 的机制。 ## 问题 -提供方适配器可能在分发或迭代时抛出异常,也可能以 `finish { kind: 'error' | 'aborted' }` 结束。最终适配器边界会在 `dsh-agent-loop` 接收前把抛出值规范化为该终止 finish 协议;middleware 与结果处理缺陷仍会抛出。loop 会将终止模型请求失败交给 `agent/request-error`。未被处理的失败是终态;处理失败的监听器修复策略自有状态,返回 `{ kind: 'retry' }`,并停止 waterfall(瀑布式事件)委托。[重试动作决策](../simplification/2026-07-27-request-error-retry-action.md)规定这一返回约定。 +提供方适配器可能在分发或迭代时抛出异常,也可能以 `finish { kind: 'error' | 'aborted' }` 结束。最终适配器边界会在 `dsh-agent-loop` 接收前把抛出值规范化为该终止 finish 协议;middleware 与结果处理缺陷仍会抛出。loop 会将终止模型请求失败交给 `agent/request-error`。未被处理的失败是终态;处理失败的监听器修复策略自有状态,返回 `{ kind: 'retry' }`,并停止 waterfall(瀑布式事件)委托。[重试动作决策](../simplification/2026-07-27-request-error-retry-action.zh.md)规定这一返回约定。 -该边界已能安全地再次发起请求。原始 `assistant/chunk` 事件携带失败的 `turn` 和 `step`;除非某条成功的 `assistant/message` 引用这些事件,否则消息派生会忽略它们。只有终止性 finish 成功且组装完成后,系统才会分发工具调用;重试则会从持久日志开启新的编号轮次。因此,harness 无需引入第二套响应生命周期或暂定输出协议,即可分隔两次尝试。 +该边界已能安全地再次发起请求。原始 `assistant/chunk` 事件携带失败的 `turn` 和 `step`;除非某条成功的 `assistant/message` 引用这些事件,否则消息派生会忽略它们。只有终止性 finish 成功且组装完成后,系统才会分发工具调用;重试则会从持久日志重建下一次尝试。因此,harness 无需引入第二套响应生命周期或暂定输出协议,即可分隔两次尝试。 此前的边界还留有三个较窄的缺口。 @@ -46,15 +46,15 @@ agent loop(智能体循环)会将终止 finish 的 `LlmFailure` 传给 `agen 适配器会先提取结构化事实,再回退到消息检查。它们会验证 HTTP 状态,将 `Retry-After` 的秒数或日期解析为正的有限毫秒延迟,在提供方公开请求 id 时将其品牌化,并区分自身超时与调用方中止。提供方专用 code 和消息可以细化映射,但恢复监听器不会解析它们。 -共享的暂时性 code 集有意保持很小:适配器针对 `RATE_LIMIT` 和 `SERVER` 的映射,远程失败使用的显式 `TIMEOUT` 和 `TRANSPORT` code,以及提供方响应已完成却没有内容块时使用的 `EMPTY_RESPONSE`。两个适配器都会把最后一种情况归类为错误 finish;详见[空模型响应可重试](../bug-fix/2026-07-24-empty-model-response-is-retryable.md)。身份验证、配额、无效请求、上下文溢出、协议、中止和未知失败都保留不同的稳定 code,且默认不属于暂时性失败。新增 code 需要适配器 fixture(测试前置数据)和已记录的策略决策;无需扩展第二个失败类枚举。 +共享的暂时性 code 集有意保持很小:适配器针对 `RATE_LIMIT` 和 `SERVER` 的映射,远程失败使用的显式 `TIMEOUT` 和 `TRANSPORT` code,以及提供方响应已完成却没有内容块时使用的 `EMPTY_RESPONSE`。两个适配器都会把最后一种情况归类为错误 finish;详见[空模型响应可重试](../bug-fix/2026-07-24-empty-model-response-is-retryable.zh.md)。身份验证、配额、无效请求、上下文溢出、协议、中止和未知失败都保留不同的稳定 code,且默认不属于暂时性失败。新增 code 需要适配器 fixture(测试前置数据)和已记录的策略决策;无需扩展第二个失败类枚举。 ### 将重试策略放在现有失败步骤扩展点上 `@deepseek-ai/dsh-llm-retry` 是监听 `agent/request-error` 的函数插件。它不引入服务或新的循环分支;agent-loop 包仅会更改通过现有失败步骤恢复控制流携带的数据。 -`agent/request-error` waterfall 携带当前 `LlmFailure`、在连续恢复序列中授权重试轮次的不可变先前失败列表,以及提供服务的注册项所携带的不可变重试策略。循环只传递而不解释该策略;它拥有连续失败历史,并在模型请求成功后清除。`dsh-llm-retry` 的 normal 策略统计由同一项确切提供方策略安排的持久重试记录,`dsh-compaction-basic` 则维护自己的上下文溢出预算。因此,暂时性失败与上下文溢出交替出现时,会各自独立消耗其有限预算;最大请求数等于 1 加上所有已加载有限预算之和。 +`agent/request-error` waterfall 携带当前 `LlmFailure`、在连续恢复序列中授权重试的不可变先前失败列表,以及提供服务的注册项所携带的不可变重试策略。循环只传递而不解释该策略;它拥有连续失败历史,并在模型请求成功后清除。`dsh-llm-retry` 的 normal 策略统计由同一项确切提供方策略安排的持久重试记录,`dsh-compaction-basic` 则维护自己的上下文溢出预算。因此,暂时性失败与上下文溢出交替出现时,会各自独立消耗其有限预算;最大请求数等于 1 加上所有已加载有限预算之和。 -当前配置形状由[提供方策略决策](../feature/2026-07-24-provider-retry-policies.md)规定。提供方适配器会注册嵌套的 `retryPolicy`;省略时使用 normal 默认值:两次暂时性重试、500 毫秒初始延迟、10 秒延迟上限、10% 抖动,以及上述五个暂时性 code。计数与延迟边界参考了所调查实现中较保守的一端:[OpenCode 使用两次请求重试,延迟边界为 500 毫秒/10 秒](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39);[Pi 将三次 agent 级重试与提供方重试分开,且提供方重试默认为零](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147);[Codex 使用有限请求/流预算以及五分钟空闲超时](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33)。10% 抖动参考 [Codex 的有界抖动](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47)。 +当前配置形状由[提供方策略决策](../feature/2026-07-24-provider-retry-policies.zh.md)规定。提供方适配器会注册嵌套的 `retryPolicy`;省略时使用 normal 默认值:两次暂时性重试、500 毫秒初始延迟、10 秒延迟上限、10% 抖动,以及上述五个暂时性 code。计数与延迟边界参考了所调查实现中较保守的一端:[OpenCode 使用两次请求重试,延迟边界为 500 毫秒/10 秒](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39);[Pi 将三次 agent 级重试与提供方重试分开,且提供方重试默认为零](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147);[Codex 使用有限请求/流预算以及五分钟空闲超时](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33)。10% 抖动参考 [Codex 的有界抖动](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47)。 对于预算未耗尽的合格失败,从 1 开始的暂时性重试计数使用有界指数退避。有效的 `providerRetryAfterMs` 只有在不超过 `maxDelayMs` 时才会取代指数退避;提供方延迟更长时,系统会委托给下一监听器,而不会违反提供方指令提前重试。本地退避乘以 `[1 - jitterRatio, 1 + jitterRatio]` 内的注入随机因子,并将最终值限制到 `maxDelayMs`;提供方延迟不加抖动。 @@ -68,7 +68,7 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次 ### 由单一层负责可见的尝试 -适配器每次调用 `stream()` 只执行一次提供方请求。pi-ai 适配器移除公开的 `maxRetries` 和 `maxRetryDelayMs` profile 字段,并禁用库内部重试;手写适配器保持现有的单次尝试行为。这样既避免 SDK 预算成倍放大 agent 预算,又能确保每次暂时性重试都由一个已关闭的失败步骤加 `llm/retry` 表示。 +适配器每次调用 `stream()` 只执行一次提供方请求。pi-ai 适配器移除公开的 `maxRetries` 和 `maxRetryDelayMs` profile 字段,并禁用库内部重试;手写适配器保持现有的单次尝试行为。这样既避免 SDK 预算成倍放大 agent 预算,又能确保每次暂时性重试都由其记录在案的失败尝试加 `llm/retry` 表示。 `ctx.llm.stream()` 仍是原始的单次尝试 waterfall。压缩(compaction)摘要等直接调用方会收到结构化失败,但不会自动获得重试,因为它们没有 agent 步骤边界,也没有可供分隔尝试的通用持久位置。未来的直接调用消费方可能会需要一个缓冲辅助函数,仅在尚未发出任何分片时重试;本决策不增加此类辅助函数。 @@ -82,9 +82,9 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次 ### 在现有日志中分隔尝试 -一次失败尝试可以在已关闭的步骤中留下 `assistant/chunk` 事件,但绝不会追加 `assistant/message`,也不会分发工具。重试会关闭失败轮次,开启下一个编号轮次,从持久表层重建请求,并生成自己的分片。步骤仍处于打开状态时,UI 可以渲染实时分片;当 `llm/retry` 标识失败步骤,或 `turn/end` 记录失败时,UI 再标记或清除这份暂时视图。Web 会验证完整的重试载荷约定,在 `llm/retry` 到达时清除失败的部分输出,将连续重试轮次的事件投影为稳定的一行,并用最新一次尝试更新该行,再从后续轮次事实派生 scheduled、started 或 cancelled 状态。倒计时以浏览器收到事件的时刻为计划延迟的起点,而不是使用 Host 事件时钟;它按向上取整且不低于 1 秒的秒数显示,仅在重试尚未结束时显示动画,并把最近一次失败的准确详情折叠在该行之后。即使失败尝试没有 assistant 节点,重试节点也会锚定自身的轨迹轮次。消息派生仍会忽略失败分片;Web 在重建历史时也会应用同一投影,因此刷新页面不会让已丢弃的部分输出重新出现,也不会生成重复的重试行。 +一次失败尝试可以在其步骤中留下 `assistant/chunk` 事件,但绝不会追加 `assistant/message`,也不会分发工具。重试在失败的轮次与步骤内继续,从持久表层重建请求,并生成自己的分片;只有最终结果才会关闭该轮次。步骤仍处于打开状态时,UI 可以渲染实时分片;当 `llm/retry` 标识失败尝试,或 `turn/end` 记录失败时,UI 再标记或清除这份暂时视图。Web 会验证完整的重试载荷约定,在 `llm/retry` 到达时清除失败的部分输出,将每条生产方关联的 `retryId` 重试链投影为稳定的一行,并用最新一次尝试更新该行,再从 `llm/retry-started` 与所属轮次、步骤边界的关闭派生 scheduled、started 或 cancelled 状态。倒计时以浏览器收到事件的时刻为计划延迟的起点,而不是使用 Host 事件时钟;它按向上取整且不低于 1 秒的秒数显示,仅在重试尚未结束时显示动画,并把最近一次失败的准确详情折叠在该行之后。即使失败尝试没有 assistant 节点,重试节点也会锚定自身的轨迹轮次。消息派生仍会忽略失败分片;Web 在重建历史时也会应用同一投影,因此刷新页面不会让已丢弃的部分输出重新出现,也不会生成重复的重试行。 -如果恢复预算耗尽,最终失败会连同结构化事实在 `turn/end.reason` 中存储一次。Web 会在该序列位置派生一个 `turn-error` 节点,并内联渲染适合展示的消息与可选错误码;AUTH 投影会把可能回显凭据片段的提供方文案替换为 `API key is invalid`,原始诊断仍保留在会话日志中。实时事件和历史回放使用同一套折叠逻辑。如果暂时性恢复继续,`llm/retry` 就是该次尝试的失败与延迟的持久归属位置,因此该失败轮次不会再获得终态错误行。本决策不增加独立的最终错误事件或响应 id 词汇。 +如果恢复预算耗尽,最终失败会连同结构化事实在 `turn/end.reason` 中存储一次。Web 会在该序列位置派生一个 `turn-error` 节点,并内联渲染适合展示的消息与可选错误码;AUTH 投影会把可能回显凭据片段的提供方文案替换为 `API key is invalid`,原始诊断仍保留在会话日志中。实时事件和历史回放使用同一套折叠逻辑。暂时性恢复继续期间,`llm/retry` 是每次中间失败与延迟的持久归属位置;终态错误行只在 `turn/end` 记录错误后才存在,而由于耗尽的恢复与失败共享同一轮次,该轮次的重试历史绝不会抑制这一行——定格的重试链与终态错误并列渲染。本决策不增加独立的最终错误事件或响应 id 词汇。 ## 不在范围内 @@ -100,7 +100,7 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次 - **向 `dsh-llm` 增加响应开始、中断、丢弃、失败和提交事件**:拒绝采用,因为 agent 日志已经分隔原始分片、成功消息和编号尝试。第二套状态机会重复归属关系,又不能支持有界的同路由重试。 - **增加逻辑路由、能力矩阵和故障转移选择**:拒绝采用,因为当前请求已经显式指定提供方和模型,每个提供方由一个适配器负责,而且没有当前消费方要求自动回退或能够证明语义兼容性。 - **把 `retryable` 或 `failover` 放在 `LlmFailure` 上**:拒绝采用,因为适配器报告事实,部署策略决定动作。同一个 429 可以在交互式组合包中重试,也可以在成本受限的批处理中被拒绝。 -- **只要调用方仍处于活跃状态就无限重试**:[按提供方配置的策略](../feature/2026-07-24-provider-retry-policies.md)对显式 `always` 配置项推翻了这项拒绝,同时保留有界的 normal mode 作为默认值。 +- **只要调用方仍处于活跃状态就无限重试**:[按提供方配置的策略](../feature/2026-07-24-provider-retry-policies.zh.md)对显式 `always` 配置项推翻了这项拒绝,同时保留有界的 normal mode 作为默认值。 - **只通过进程 logger 记录重试状态**:拒绝采用,因为进程日志无法重建会话行为,也不能驱动回放后的 UI 状态。 - **只保留扁平 code**:拒绝采用,因为重试延迟和提供方请求 id 是结构化的提供方事实,而当不同协议失败共用一个稳定 code 时,诊断还需要 HTTP 状态。 @@ -114,15 +114,15 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次 - 每个提供方适配器都在 Loader 启动时验证其嵌套重试策略,`ctx.llm` 则将该策略与路由一同捕获;normal mode 会委托不合格路径,而且在没有其他策略时最多发起 `maxRetries + 1` 次提供方请求。 - 退避期间执行 HMR 的测试证明:dispose 过程会注销监听器、中止并等待其捕获的回调,dispose 后不发出重试决策,也不留下存活的定时器或 promise。 - 纯单元测试覆盖暂时性 code 选择、指数退避和抖动边界、有效及超出上限的 `Retry-After`、耗尽的预算、确定性定时器/随机数钩子,以及退避期间中止。 -- 真实 agent-loop 测试覆盖分片前失败、部分分片后失败、抛出及带内失败、在新轮次中重试至成功、耗尽后写入结构化 `turn/end.reason`,以及与 `dsh-compaction-basic` 上下文溢出恢复的组合。 +- 真实 agent-loop 测试覆盖分片前失败、部分分片后失败、抛出及带内失败、在同一轮次内重试至成功、耗尽后写入结构化 `turn/end.reason`,以及与 `dsh-compaction-basic` 上下文溢出恢复的组合。 - 部分分片集成测试证明:失败分片仍归属于失败步骤,该步骤不会提交 assistant 消息或工具副作用,成功的重试会记录自己的分片 seq 和提供方/模型路由。 -- 插件拥有的不进入表层的 `llm/retry` 事件可在 JSONL 和 SQLite 往返后保留,被消息派生忽略,并驱动 TUI 和 Web 撤回及计划重试渲染。客户端测试覆盖完整的 wire 验证、独立于时钟的倒计时、已取消与已完成重试标签的区别以及轨迹归属;无密钥 UI 快照覆盖 Web 的调度与成功,真实 Web 组合测试覆盖部分传输失败直至恢复,ACP 自动化快照确认,被丢弃的尝试不会通过协议发出,而恢复后的回复会正常发出。 +- 插件拥有的不进入表层的 `llm/retry` 事件可在 JSONL 和 SQLite 往返后保留,被消息派生忽略,并驱动 TUI 和 Web 撤回及计划重试渲染。客户端测试覆盖完整的 wire 验证、独立于时钟的倒计时、已取消与已完成重试标签的区别以及轨迹归属;无密钥 UI 快照覆盖 Web 的调度与成功,真实 Web 组合测试覆盖部分传输失败直至恢复,以及耗尽后终态错误行与定格重试链并列的画面,ACP 自动化快照确认,被丢弃的尝试不会通过协议发出,而恢复后的回复会正常发出。 - 空闲看门狗测试证明:只有 `next()` 尚未完成时才会重新布防稳定信号;在消费方思考期间及 `finally` 中会解除布防;它与总调用 deadline 以及更早发生的调用方中止分开分类。适配器测试证明该信号会终止底层请求,而不只是与其脱离。 - `ctx.llm.stream()` 的直接调用方仍只尝试一次,并收到相同的结构化失败事实。 ## 后果 -- 每次重试尝试都以一个已关闭失败轮次加 `llm/retry` 的形式可见,适配器级的单次尝试行为会防止隐藏的 SDK 重试成倍增加策略决策。即使没有分片到达,重试仍可能造成提供方重复计费;normal mode 会限制此风险,而显式 always mode 会接受它,直至取消或成功。 +- 每次重试尝试都在其所属轮次内以失败尝试的分片加 `llm/retry` 的形式可见,适配器级的单次尝试行为会防止隐藏的 SDK 重试成倍增加策略决策。即使没有分片到达,重试仍可能造成提供方重复计费;normal mode 会限制此风险,而显式 always mode 会接受它,直至取消或成功。 - 提供方 SDK 可能隐藏状态或重试标头。适配器会保留 SDK 公开的稳定事实,否则使用粗粒度 code,而不会让恢复策略解析脆弱的文本。 - 持久重试事件扩展了会话协议和 UI 状态机。事件与其消费方一同交付,可避免产生无人使用的遥测词汇;但以后更改 schema 仍需要同步完成持久化和回放工作。 - 清除失败步骤的实时分片可能会明显撤回输出。与把丢弃的文本或不完整工具 JSON 呈现为已提交历史相比,这是更好的选择;快照固定这一转换。 @@ -131,8 +131,9 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次 ## 相关资料 -- [结构化错误分类体系](../../implemented/architecture/2026-06-11-structured-error-taxonomy.md)负责稳定、可供机器路由的 code 与 cause chaining。 -- [可重建请求](../../implemented/architecture/2026-07-05-reconstructable-requests.md)使提供方/模型和完整请求输入在分发前持久化。 -- [超时 deadline 库](../../implemented/architecture/2026-07-06-timeout-deadline-library.md)将共享的 deadline 分类与能力自身拥有的终止操作分开。 -- [调用后压缩压力与上下文溢出恢复](../../implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md)负责当前已关闭步骤的请求恢复扩展点与有界溢出重试。 -- [提供方路由的 LLM 适配器](../../implemented/architecture/2026-07-14-provider-routed-llm-adapters.md)负责显式提供方/模型路由与每个提供方仅有一个适配器的不变量。 +- [结构化错误分类体系](../../implemented/architecture/2026-06-11-structured-error-taxonomy.zh.md)负责稳定、可供机器路由的 code 与 cause chaining。 +- [可重建请求](../../implemented/architecture/2026-07-05-reconstructable-requests.zh.md)使提供方/模型和完整请求输入在分发前持久化。 +- [超时 deadline 库](../../implemented/architecture/2026-07-06-timeout-deadline-library.zh.md)将共享的 deadline 分类与能力自身拥有的终止操作分开。 +- [调用后压缩压力与上下文溢出恢复](../../implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md)负责当前已关闭步骤的请求恢复扩展点与有界溢出重试。 +- [提供方路由的 LLM 适配器](../../implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md)负责显式提供方/模型路由与每个提供方仅有一个适配器的不变量。 +- [Terminal turn errors survive same-turn retry history](../bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.zh.md)负责移除曾藏掉耗尽恢复终态错误行的 Web 重试历史抑制。 diff --git a/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.i18n.yaml index 687743c0a4..2b6c63d334 100644 --- a/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md -2026-06-21-mandatory-app-attribution-headers.md: 479d3a46dc41c5cc9ae9b77b81dbef3d6524370b -2026-06-21-mandatory-app-attribution-headers.zh.md: 75162604623099d50ca87aafd5d45567e2121789 +2026-06-21-mandatory-app-attribution-headers.md: 9e0c029dc03c722512680a563c24e470128ee322 +2026-06-21-mandatory-app-attribution-headers.zh.md: 1427daf8f6065ec2dee324075a8381a625d9e960 diff --git a/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md b/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md index 479d3a46dc..9e0c029dc0 100644 --- a/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md +++ b/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md @@ -42,7 +42,7 @@ Wire mapping (`attributionHeaders`; header names lowercase in code - HTTP field |---|---| | All HTTP-based adapters | `User-Agent: {product}/{version} (+{url})` - the parenthesized `+url` comment stays within RFC 9110's conservative product/comment syntax. | | Direct DeepSeek endpoint | `User-Agent` for app attribution; `x-deepseek-harness-user-id` and conditional `x-deepseek-harness-session-id` are separate request identity under the DeepSeek-specific decision. Do not send OpenRouter-only headers unless DeepSeek documents an equivalent contract. | -| OpenRouter endpoints | `User-Agent` only for now. Do not send `HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, or `X-OpenRouter-Categories` under this decision. | +| OpenRouter endpoints | `User-Agent` only. This decision excludes `HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, and `X-OpenRouter-Categories`. | | Future providers | `User-Agent` only unless a later provider-specific Agent Note accepts additional headers. Do not reuse `HTTP-Referer` by analogy. | Endpoint detection is not part of this Agent Note because no endpoint-specific mapping is accepted here. If OpenRouter support lands later, detection must be explicit: either a dedicated OpenRouter provider package or an explicit `provider: 'openrouter'` / `attributionTarget: 'openrouter'` config, not arbitrary path fragments or model names. diff --git a/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.zh.md b/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.zh.md index 7516260462..1427daf8f6 100644 --- a/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -LLM(大语言模型)提供方请求应当标识发出请求的产品。这对提供方侧的技术支持、滥用调查、兼容性调试和流量分析都有价值。在本 Agent Note 之前,harness 只做了部分工作:手写的 DeepSeek 适配器发送了一个手动复制的 `User-Agent` 常量(`packages/llm/llm-deepseek/src/adapter.ts`),而基于 pi-ai 的孪生适配器则完全不发送 harness 自有的头部(`packages/llm/llm-pi-ai/src/adapter.ts`)。因此新适配器可以悄无声息地省略归属标识,而基于库的适配器也可能与手写适配器产生偏差——尽管[孪生适配器 Agent Note](2026-06-13-twin-llm-adapters.md) 的存在正是为了确保两种实现中的提供方约定真实可靠。 +LLM(大语言模型)提供方请求应当标识发出请求的产品。这对提供方侧的技术支持、滥用调查、兼容性调试和流量分析都有价值。在本 Agent Note 之前,harness 只做了部分工作:手写的 DeepSeek 适配器发送了一个手动复制的 `User-Agent` 常量(`packages/llm/llm-deepseek/src/adapter.ts`),而基于 pi-ai 的孪生适配器则完全不发送 harness 自有的头部(`packages/llm/llm-pi-ai/src/adapter.ts`)。因此新适配器可以悄无声息地省略归属标识,而基于库的适配器也可能与手写适配器产生偏差——尽管[孪生适配器 Agent Note](2026-06-13-twin-llm-adapters.zh.md) 的存在正是为了确保两种实现中的提供方约定真实可靠。 直接触发因素来自 OpenRouter 的[应用归属](https://openrouter.ai/docs/app-attribution)文档。OpenRouter 根据 `HTTP-Referer` 加上用于展示和分类的头部来创建应用页面和排名。这有价值,但它不是 HTTP 标准中的应用身份机制。风险在于:把 OpenRouter 的精确头部集当作通用标准来采纳,然后将提供方特有的头部泄漏到直连 DeepSeek 的请求、未来的 OpenAI/Anthropic/Vertex 适配器、测试服务器或无限期记录未知字段的代理中。 @@ -24,7 +24,7 @@ LLM(大语言模型)提供方请求应当标识发出请求的产品。这 ## 决策 -在 LLM 适配器边界,提供方无关的应用归属是强制的,且仅使用标准 `User-Agent` 头部。规则:每个产品级 LLM 适配器在每个提供方 HTTP 请求上发送一个静态、非机密的应用身份,且每个适配器都有测试证明 `User-Agent` 到达了线路(mock 服务器断言收到的头部;对于基于库的适配器,通过库的头部钩子馈入同一个 mock 服务器断言)。这条规则约束应用归属,不约束提供方特有的请求身份;[DeepSeek 请求身份决策](../feature/2026-08-11-deepseek-request-user-id-header.md)另行负责其用户与会话头部。 +在 LLM 适配器边界,提供方无关的应用归属是强制的,且仅使用标准 `User-Agent` 头部。规则:每个产品级 LLM 适配器在每个提供方 HTTP 请求上发送一个静态、非机密的应用身份,且每个适配器都有测试证明 `User-Agent` 到达了线路(mock 服务器断言收到的头部;对于基于库的适配器,通过库的头部钩子馈入同一个 mock 服务器断言)。这条规则约束应用归属,不约束提供方特有的请求身份;[DeepSeek 请求身份决策](../feature/2026-08-11-deepseek-request-user-id-header.zh.md)另行负责其用户与会话头部。 OpenRouter 应用归属刻意未实现。`HTTP-Referer`、`X-OpenRouter-Title`、`X-Title` 和 `X-OpenRouter-Categories` 是 OpenRouter 特有的产品展示头部,不是提供方无关的模型请求归属。它们可以后续由 OpenRouter 适配器或显式 OpenRouter 模式提出,附带自己的隐私/产品决策、测试和文档。在此之前,即使请求指向 OpenRouter,也只发送本决策定义的共享 `User-Agent` 归属。 @@ -42,7 +42,7 @@ OpenRouter 应用归属刻意未实现。`HTTP-Referer`、`X-OpenRouter-Title` |---|---| | 所有基于 HTTP 的适配器 | `User-Agent: {product}/{version} (+{url})`——括号中的 `+url` 注释符合 RFC 9110 保守的 product/comment 语法。 | | 直连 DeepSeek 端点 | `User-Agent` 用于应用归属;`x-deepseek-harness-user-id` 与条件性的 `x-deepseek-harness-session-id` 由 DeepSeek 特有决策作为独立请求身份管理。除非 DeepSeek 文档化了等效约定,否则不发送 OpenRouter 特有头部。 | -| OpenRouter 端点 | 目前仅 `User-Agent`。本决策下不发送 `HTTP-Referer`、`X-OpenRouter-Title`、`X-Title` 或 `X-OpenRouter-Categories`。 | +| OpenRouter 端点 | 仅发送 `User-Agent`。本决策排除 `HTTP-Referer`、`X-OpenRouter-Title`、`X-Title` 与 `X-OpenRouter-Categories`。 | | 未来提供方 | 仅 `User-Agent`,除非后续提供方特有的 Agent Note 接受额外头部。不要类比复用 `HTTP-Referer`。 | 端点检测不在本 Agent Note 范围内,因为此处不接受任何端点特有的映射。如果后续支持 OpenRouter,检测必须是显式的:要么是专门的 OpenRouter 提供方包,要么是显式的 `provider: 'openrouter'` / `attributionTarget: 'openrouter'` 配置,而非任意路径片段或模型名称。 diff --git a/.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.i18n.yaml index bc92549f44..f0e60bbc20 100644 --- a/.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md 2026-06-24-web-capability-seam.md: 7e7b09f19864bd2ad8ad9d69579c1d5c79600cde -2026-06-24-web-capability-seam.zh.md: d6051eec498edb640773ba367038581cebd1f654 +2026-06-24-web-capability-seam.zh.md: dbb41ee42d2c7503955ead2df32abe80b3a4f641 diff --git a/.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md index d6051eec49..dbb41ee42d 100644 --- a/.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md @@ -16,7 +16,7 @@ harness 需要面向模型的 web 工具,但不能将模型约定绑定到某 ## 决策 -Web 访问是一个一等能力 seam,遵循[能力 seam Agent Note](2026-06-13-capability-seams.md): +Web 访问是一个一等能力 seam,遵循[能力 seam Agent Note](2026-06-13-capability-seams.zh.md): 1. `@deepseek-ai/dsh-web`(`packages/web/web`)拥有 `ctx.web`、提供方注册、提供方选择、共享的请求/结果词汇,以及 web 特有的错误。 2. 提供方包实现具体后端并向 `ctx.web` 注册能力,例如 `@deepseek-ai/dsh-web-search-exa`、`@deepseek-ai/dsh-web-search-perplexity`、`@deepseek-ai/dsh-web-search-deepseek` 和 `@deepseek-ai/dsh-web-fetch-http`。 @@ -280,7 +280,7 @@ SSRF/私有网络防护(阻断私有、回环、链路本地、多播及其他 ## 测试 -每一层在自己的边界处固定:`dsh-web` 中的注册/选择/截断/abort 约定与 `WebError` 码;每个提供方基于录制的 fixture(测试前置数据)的请求/响应映射(Perplexity fixture 包含纯 URL 引用,以保持可选 source 字段的诚实性),加上每个真实提供方的自跳过带密钥冒烟测试;`web-fetch-http` 中的真实本地 HTTP 行为;`dsh-tool-web` 中通过真实工具注册表的启用驱动注册、结构化执行错误和结果格式化。一个真实 Loader 冒烟测试守护两种导出形状([事故复盘(postmortem) 0001](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)):`dsh-web` 是默认导出的服务,而提供方和 `tool-web` 是命名空间插件,误加 `export default` 会丢失 `inject`。 +每一层在自己的边界处固定:`dsh-web` 中的注册/选择/截断/abort 约定与 `WebError` 码;每个提供方基于录制的 fixture(测试前置数据)的请求/响应映射(Perplexity fixture 包含纯 URL 引用,以保持可选 source 字段的诚实性),加上每个真实提供方的自跳过带密钥冒烟测试;`web-fetch-http` 中的真实本地 HTTP 行为;`dsh-tool-web` 中通过真实工具注册表的启用驱动注册、结构化执行错误和结果格式化。一个真实 Loader 冒烟测试守护两种导出形状([事故复盘(postmortem) 0001](../../../../docs/postmortem/0001-acp-default-export-drops-inject.zh.md)):`dsh-web` 是默认导出的服务,而提供方和 `tool-web` 是命名空间插件,误加 `export default` 会丢失 `inject`。 ## 曾考虑的替代方案 @@ -329,10 +329,10 @@ SSRF/私有网络防护(阻断私有、回环、链路本地、多播及其他 - `web_fetch` 的 SSRF/私有网络防护:阻断私有、回环、链路本地、多播及其他非公开目的地,使 `web_fetch` 不再是 SSRF 原语。正确实现不仅仅是 URL 字符串检查——需要先 DNS 解析再连接到已验证的 IP(防御 DNS rebinding/TOCTOU)、跨重定向的每跳重新验证,以及 IPv6 边缘处理(私有范围、IPv4 映射地址)。所调研的参考实现均未做 IP 级阻断(OpenCode 做前缀检查后直接 fetch;Claude Code 依赖集中式主机名黑名单加「私有 URL 会失败」的提示词),因此没有可复制的实现,且这是 harness 唯一的 SSRF 防线——值得一次专门的设计/spike。在其落地之前,`web_fetch` 只能在无法触达敏感内部目标的部署中启用。 - `pdf` `WebFetchBody` 类别:`http` 提供方将可文本提取的 PDF 解码(尽力而为、有上限、`truncated`)为 `{ kind: 'pdf'; content; pageCount? }` 分支,`tool-web` 渲染它。这是 fetch 而非 `web_extract`——PDF 获取是具体的 HTTP 200 加确定性的本地解码,不是提供方侧对非 HTTP 资源的提取。添加它是跨 `dsh-web`(声明分支)、提供方(解码 + 将「二进制拒绝」收窄为「拒绝二进制,但可文本提取的 PDF 除外」;需要 OCR 的扫描/图片 PDF 不在范围内)和 `tool-web`(渲染)的协调变更。封闭的 `WebFetchBody` 联合类型使消费方在新分支被处理之前编译失败。 - 提供方支撑的提取作为独立的 `web_extract` 能力,而非静默扩展 `web_fetch`。 -- 权限策略集成:权限系统现已存在([沙箱与审批](../feature/2026-07-06-sandbox.md)、[web 权限预设](../feature/2026-07-23-web-permission-and-approval.md)),但只捆绑了沙箱模式与审批策略;web 权限策略仍未集成。 +- 权限策略集成:权限系统现已存在([沙箱与审批](../feature/2026-07-06-sandbox.zh.md)、[web 权限预设](../feature/2026-07-23-web-permission-and-approval.zh.md)),但只捆绑了沙箱模式与审批策略;web 权限策略仍未集成。 - `query` 和 `maxResults` 之外的提供方无关搜索控制,待 Exa 和 Perplexity 都能诚实遵守时再添加。 ## 开放问题 - 产品应用包是否应在启动时探测 web 配置(当 web 被显式配置时将 `WEB_PROVIDER_CONFIGURED_MISSING`、`WEB_PROVIDER_CONFIGURED_UNAVAILABLE` 和 `WEB_PROVIDER_AMBIGUOUS` 视为致命错误),还是将配置错误留到首次执行时浮出? -- 在已交付的权限系统([沙箱与审批](../feature/2026-07-06-sandbox.md)、[web 权限预设](../feature/2026-07-23-web-permission-and-approval.md))中,公开 web 访问的权限策略应放在哪里:`tools/execute` 上的专用 web 权限插件、提供方配置,还是两者兼有? +- 在已交付的权限系统([沙箱与审批](../feature/2026-07-06-sandbox.zh.md)、[web 权限预设](../feature/2026-07-23-web-permission-and-approval.zh.md))中,公开 web 访问的权限策略应放在哪里:`tools/execute` 上的专用 web 权限插件、提供方配置,还是两者兼有? diff --git a/.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.i18n.yaml index c49cd144b9..421739cb57 100644 --- a/.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md 2026-06-26-file-context-as-event-gate.md: a8316741c3ce8548db5b5def431425589994995c -2026-06-26-file-context-as-event-gate.zh.md: 78b1b759edc0340fd7be103fc9bfec0b125e0800 +2026-06-26-file-context-as-event-gate.zh.md: 618985f33073cf9fd69cf2c7fda7112d9752468f diff --git a/.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.zh.md b/.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.zh.md index 78b1b759ed..618985f330 100644 --- a/.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -[拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md) 在面向模型的工具与 `ctx.fs` 提供方之间放置了 `ctx.fileContext`:`dsh-tool-fs` 注入 `fileContext`,并将每次 `read`/`write`/`edit` 路由到它的方法。这使得 `fileContext` **位于关键路径上且不可省略**。工具不经过它就无法访问 `ctx.fs`,策略层掌控着 fs I/O 和读取窗口,而一个不需要观测状态策略的部署也无法简单地移除该包——`dsh-tool-fs` 会因无法解析 `ctx.fileContext` 而失败。 +[拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.zh.md) 在面向模型的工具与 `ctx.fs` 提供方之间放置了 `ctx.fileContext`:`dsh-tool-fs` 注入 `fileContext`,并将每次 `read`/`write`/`edit` 路由到它的方法。这使得 `fileContext` **位于关键路径上且不可省略**。工具不经过它就无法访问 `ctx.fs`,策略层掌控着 fs I/O 和读取窗口,而一个不需要观测状态策略的部署也无法简单地移除该包——`dsh-tool-fs` 会因无法解析 `ctx.fileContext` 而失败。 这把三件本应可分离的事情耦合在了一起: @@ -33,7 +33,7 @@ provider dsh-fs-local local implementation of ctx.fs 该模型是叠加式的:裸 `ctx.fs` 执行原子化、无约束的文本 I/O,而 `dsh-fs-observation-policy` 叠加观测状态、先读后编辑和版本守卫。因此移除策略层后工具仍可用,只是不受约束。正式发布的 agent(智能体)配置会加载策略;裸模式的存在是为了让策略在服务边界保持可选,而非作为正常部署姿态。 -[文件系统缺失观测后续决策](../bug-fix/2026-08-09-filesystem-absence-observation.md)把记录载荷从仅表示成功的版本细化为显式的存在/缺失状态,并要求带防护的创建以不替换方式发布。事件门控归属与无 I/O 策略边界保持不变。 +[文件系统缺失观测后续决策](../bug-fix/2026-08-09-filesystem-absence-observation.zh.md)把记录载荷从仅表示成功的版本细化为显式的存在/缺失状态,并要求带防护的创建以不替换方式发布。事件门控归属与无 I/O 策略边界保持不变。 `dsh-tool-fs` 不再注入 `fileContext`。它注入 `fs` 和 `tools`/`systemPrompt`。 @@ -152,7 +152,7 @@ interface Events { ## 取代关系 -本 Agent Note 修正——而非推翻——[拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md)。四层拆分、提供方约定和新鲜度*策略*均保留。变更的是**工具与策略层之间的耦合方式**:强制性方法服务变为插件拥有的事件门控,fs I/O + 读取窗口从 `fileContext` 上移至 `dsh-tool-fs`。拆分文件系统 seam Agent Note 中关于 `dsh-tool-fs` 注入 `fileContext` 以及 `fileContext` 拥有 `read`/`write`/`edit` 的描述已在同一变更中更新。 +本 Agent Note 修正——而非推翻——[拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.zh.md)。四层拆分、提供方约定和新鲜度*策略*均保留。变更的是**工具与策略层之间的耦合方式**:强制性方法服务变为插件拥有的事件门控,fs I/O + 读取窗口从 `fileContext` 上移至 `dsh-tool-fs`。拆分文件系统 seam Agent Note 中关于 `dsh-tool-fs` 注入 `fileContext` 以及 `fileContext` 拥有 `read`/`write`/`edit` 的描述已在同一变更中更新。 ## 验证 @@ -160,7 +160,7 @@ interface Events { ## 曾考虑的替代方案 -- **保留 `ctx.fileContext` 作为关键路径上的方法服务**——[拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md) 最初落地的形态;否决,因为工具无法在没有策略层的情况下运行,使策略对基本操作是承重性的,而非可选的收紧。 +- **保留 `ctx.fileContext` 作为关键路径上的方法服务**——[拆分文件系统 seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.zh.md) 最初落地的形态;否决,因为工具无法在没有策略层的情况下运行,使策略对基本操作是承重性的,而非可选的收紧。 - **策略侧版本检查**(`dsh-fs-observation-policy` 在其 waterfall 处理器中 stat 并比较版本)——否决,因为该检查与工具实际写入之间存在 TOCTOU 间隙;提供方的 mutation 临界区是唯一无竞态的位置,因此策略只选择 CAS 基准并对先前观测进行门控。 - **每工具 `/read`/`/write`/`/edit` 子路径插件**——实现时放弃:没有消费方需要单工具部署,且子路径发布迫使引入兄弟工具包都不需要的定制 `tsdown`/`tsconfig`/`files`/workspace-constraint 处理;每工具的注册辅助函数仍作为根插件组合的内部模块保留。 diff --git a/.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.i18n.yaml index 7a5d35f882..04eec8e0e1 100644 --- a/.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.md 2026-06-30-bash-stdin-env-trusted-plugin-api.md: 41be63fff598587ee9b873cf7edf51da788bc02a -2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md: 4f0338cbb6a265ff629a81e9c9a12c4456941ba0 +2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md: 60721539d9d37857e145e289bb262482588139bf diff --git a/.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md b/.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md index 4f0338cbb6..60721539d9 100644 --- a/.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md @@ -8,7 +8,7 @@ Status: implemented 钩子子系统以 Claude Code 和 Codex 的方式运行外部钩子命令:钩子是一条 shell 命令,通过 **stdin 上的 JSON** 接收事件载荷,并从若干**环境变量**(`CLAUDE_PROJECT_DIR`、`CLAUDE_PLUGIN_ROOT`、`PLUGIN_ROOT`……)读取上下文。harness 已经在 `ctx.shell` 能力 seam 后面有一个完善的命令执行器([dsh-shell](../../../../packages/shell/shell) → [dsh-bash-local](../../../../packages/shell/bash-local)),具备进程组终止、输出截断/spill 处理和凭证擦除功能。复用它来执行钩子意味着钩子桥接层无需重新实现子进程底层机制——但该 seam 此前无法写入 stdin 或设置额外 env。本次变更添加这两个输入。 -`stdin` 和 `env` 不构成新的模型能力,因为普通 shell 语法已经能提供两者。环境凭证由 `dsh-bash-local` 的子环境擦除机制保护,而非靠隐藏这些 Service Definition 字段;模型工具参数是静态 JSON,不会展开 shell 变量。因此这些字段服务于受信的进程内调用方(如钩子桥接层),它们需要传递结构化输入和 `CLAUDE_*` 变量,而不必将其嵌入模型可见的 shell 文本。环境变量规则见 [defensive-patterns.md](../../../../docs/defensive-patterns.md)。 +`stdin` 和 `env` 不构成新的模型能力,因为普通 shell 语法已经能提供两者。环境凭证由 `dsh-bash-local` 的子环境擦除机制保护,而非靠隐藏这些 Service Definition 字段;模型工具参数是静态 JSON,不会展开 shell 变量。因此这些字段服务于受信的进程内调用方(如钩子桥接层),它们需要传递结构化输入和 `CLAUDE_*` 变量,而不必将其嵌入模型可见的 shell 文本。环境变量规则见 [defensive-patterns.md](../../../../docs/defensive-patterns.zh.md)。 ## 决策 @@ -16,7 +16,7 @@ Status: implemented 三个有意为之的选择: -1. **模型侧工具不暴露 `stdin` 和 `env`。** Shell 语法已覆盖这些需求,重复参数只会增加接口面而不带来权限隔离。工具仅从声明的模型参数、signal 和 owner 构建请求;受信的进程内调用方可以直接设置请求字段。harness 自有变量使用[托管环境决策](../feature/2026-07-10-agent-session-identity-and-log-location.md)规定的独立 `dshEnv` 通道,因此普通 `env` 无法替换它们。 +1. **模型侧工具不暴露 `stdin` 和 `env`。** Shell 语法已覆盖这些需求,重复参数只会增加接口面而不带来权限隔离。工具仅从声明的模型参数、signal 和 owner 构建请求;受信的进程内调用方可以直接设置请求字段。harness 自有变量使用[托管环境决策](../feature/2026-07-10-agent-session-identity-and-log-location.zh.md)规定的独立 `dshEnv` 通道,因此普通 `env` 无法替换它们。 2. **`env` 在凭证擦除之后合并,因此调用方显式设置的条目即使具有凭证形态的名称也会胜出。** 后续的托管命名空间决策负责管理 `DSH_*`:这类环境条目会被移除,受信的 `dshEnv` 最后合并,因此普通 `env` 条目永远无法顶掉托管值。完整顺序为 `scrub(process.env, including DSH_*)` → `ENV_OVERRIDES` → 普通 `env` → `dshEnv`。 @@ -30,4 +30,4 @@ Status: implemented ## 后果 -钩子桥接层通过既有的 bash seam 传递 JSON 载荷和钩子特定变量,保留其进程组终止、截断和 spill 行为。面向模型的行为不变,bash 工具仍是模型调用请求构建的唯一所有者。相关词汇定义见 [bash 数据结构参考](../../../../docs/subsystems/shell.md)。 +钩子桥接层通过既有的 bash seam 传递 JSON 载荷和钩子特定变量,保留其进程组终止、截断和 spill 行为。面向模型的行为不变,bash 工具仍是模型调用请求构建的唯一所有者。相关词汇定义见 [bash 数据结构参考](../../../../docs/subsystems/shell.zh.md)。 diff --git a/.agents/notes/implemented/architecture/2026-06-30-event-domain-semantics.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-30-event-domain-semantics.i18n.yaml index 171f33fe17..feb512101f 100644 --- a/.agents/notes/implemented/architecture/2026-06-30-event-domain-semantics.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-30-event-domain-semantics.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-30-event-domain-semantics.md 2026-06-30-event-domain-semantics.md: 70da718b5471ce309a090c8aade3e7290cc949dc -2026-06-30-event-domain-semantics.zh.md: 6795e1f6f583d6fe7d34efc5e1619da38ab1c89f +2026-06-30-event-domain-semantics.zh.md: c3b12a167da0a41b792914d82a675a98b3a0b860 diff --git a/.agents/notes/implemented/architecture/2026-06-30-event-domain-semantics.zh.md b/.agents/notes/implemented/architecture/2026-06-30-event-domain-semantics.zh.md index 6795e1f6f5..c3b12a167d 100644 --- a/.agents/notes/implemented/architecture/2026-06-30-event-domain-semantics.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-30-event-domain-semantics.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -harness 通过 Cordis 事件分类体系扩展 agent loop(智能体循环)(见[微内核事件分类体系 Agent Note](2026-06-11-microkernel-event-taxonomy.md))。随着该分类体系的增长,三个事件域之间的界限变得模糊: +harness 通过 Cordis 事件分类体系扩展 agent loop(智能体循环)(见[微内核事件分类体系 Agent Note](2026-06-11-microkernel-event-taxonomy.zh.md))。随着该分类体系的增长,三个事件域之间的界限变得模糊: - `session/*` 承载持久的、事件溯源的日志(`SessionEventMap`)。 - `agent/*` 承载运行时实时信号,向插件传递 `Agent` 句柄。 @@ -26,14 +26,14 @@ harness 通过 Cordis 事件分类体系扩展 agent loop(智能体循环) **边界规则:** 持久的、可回放的事实是 `SessionEvent`;实时拦截或瞬态/活对象信号是 `agent`/`tools` Cordis 事件。轮次或步骤边界是持久事实,因此存在于会话日志中并从 `session/event` 源读取——不会被镜像为 `agent/*` emit。 -**将规则应用于边界镜像:** 全部四个边界镜像——`agent/turn-start`、`agent/turn-end`、`agent/step-start`、`agent/step-end`——被**移除**。没有生产消费方需要在边界处获取活的 `Agent`:ACP(Agent Client Protocol)桥接将其进行中的提示词与精确对应的 `session/event` `turn/start`/`turn/end` 事件对关联,其他 transcript 消费方同样从持久流派生边界。见[移除边界镜像事件 Agent Note](../simplification/2026-06-20-remove-agent-boundary-mirror-events.md),该决策由它负责。移除 emit 也简化了循环的 `closeStep`/`closeTurn`(各只需一次 append,无需配对 emit)。 +**将规则应用于边界镜像:** 全部四个边界镜像——`agent/turn-start`、`agent/turn-end`、`agent/step-start`、`agent/step-end`——被**移除**。没有生产消费方需要在边界处获取活的 `Agent`:ACP(Agent Client Protocol)桥接将其进行中的提示词与精确对应的 `session/event` `turn/start`/`turn/end` 事件对关联,其他 transcript 消费方同样从持久流派生边界。见[移除边界镜像事件 Agent Note](../simplification/2026-06-20-remove-agent-boundary-mirror-events.zh.md),该决策由它负责。移除 emit 也简化了循环的 `closeStep`/`closeTurn`(各只需一次 append,无需配对 emit)。 ## 后果 - 循环不再 emit 任何边界镜像;`closeStep` 仅追加 `step/end`,`closeTurn` 仅追加 `turn/end`。`Session.append` 负责 post-commit observer 隔离,因此抛出异常的边界 observer 无法改变轮次结果或饿死后续消费方;事件接纳失败或内部校验失败仍会在边界进入日志之前向外抛出。 - 之前通过已移除 emit 观察边界的测试,现在观察持久的 `turn/start`/`turn/end`/`step/start`/`step/end` 会话事件——它们所锁定的行为(边界顺序、步骤计数)不变;只是读取的源移到了规范源。那些测试*抛出异常的轮次边界 emit 监听器*的用例被删除,因为该代码路径不再存在(没有 emit 可供抛出)。按照 [AGENTS.md「测试记录行为,而非黄金真相」](../../../../AGENTS.md),行为与其测试一同迁移(或一同消亡)。 - 循环仅在 `append('step/start')` 返回后才标记步骤已打开(`stepOpen = true`)。内部分发校验在日志推入之前运行,可能在不打开步骤的情况下拒绝;post-commit `session/event` observer 的失败被隔离在 `Session.append` 内部。因此该标记精确表示已提交的、欠一个后续 `step/end` 的边界。 -- 完整实现见[简化 Agent Note「停止将持久边界镜像为 agent 事件」](../simplification/2026-06-20-remove-agent-boundary-mirror-events.md):全部四个边界镜像被移除,所有消费方从 `session/event` 读取边界。`agent/steering`(不是边界镜像)不在该 Agent Note 范围内,由其后续 Agent Note [移除 `agent/steering` 镜像 emit](../../archived/simplification/2026-07-04-remove-agent-steering-mirror.md) 单独移除——它镜像的是持久的中途 steering `user/message`。 +- 完整实现见[简化 Agent Note「停止将持久边界镜像为 agent 事件」](../simplification/2026-06-20-remove-agent-boundary-mirror-events.zh.md):全部四个边界镜像被移除,所有消费方从 `session/event` 读取边界。`agent/steering`(不是边界镜像)不在该 Agent Note 范围内,由其后续 Agent Note [移除 `agent/steering` 镜像 emit](../../archived/simplification/2026-07-04-remove-agent-steering-mirror.md) 单独移除——它镜像的是持久的中途 steering `user/message`。 - 生成的 Cordis 事件表面(`docs/subsystems/` 各页)不再列出镜像事件。 diff --git a/.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.i18n.yaml index fa8401fdc8..933b3d2826 100644 --- a/.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.md 2026-07-02-tool-render-intent-union.md: 52626d3d9200146df95aee7836d37bbaf7e6a9ec -2026-07-02-tool-render-intent-union.zh.md: 43fda366694aecda07b276d34c882e75d6b4aa95 +2026-07-02-tool-render-intent-union.zh.md: bc91b89c53c3c43b6c483dc0d4876a00f5a936b7 diff --git a/.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.zh.md b/.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.zh.md index 43fda36669..bc91b89c53 100644 --- a/.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.zh.md @@ -4,7 +4,7 @@ Status: implemented [English](2026-07-02-tool-render-intent-union.md) | 中文 -> render-intent 联合类型对 UI 传输层仍然有效;其 ACP(Agent Client Protocol)映射已被 [ACP 作为仅面向自动化的协议](../simplification/2026-07-23-acp-automation-only-protocol.md)取代。 +> render-intent 联合类型对 UI 传输层仍然有效;其 ACP(Agent Client Protocol)映射已被 [ACP 作为仅面向自动化的协议](../simplification/2026-07-23-acp-automation-only-protocol.zh.md)取代。 ## 问题 @@ -78,5 +78,5 @@ terminal 意图只用于展示。harness 仍通过自身的 bash 服务执行命 ## 相关 - 取代早先被否决的折叠工具自有呈现提案(已否决——「等两个真实工具和两个真实消费方,然后做带标签 render-intent 联合类型」)中的推迟决定。该条件现已满足;本 Agent Note 即为那个联合类型。 -- 被[结果时已应用 hunk 差异](../../archived/architecture/2026-07-02-result-time-applied-hunk-diffs.md)(已归档)扩展:后者添加了一个持久化的 `meta` 通道,使 write/edit 在结果时输出 `DiffResultView`(应用后的变更:带上下文行的 contextual hunk / 每个 `replace_all` 位点一个,或创建时的整文件 diff)——值/呈现拆分与持久化的 `presentationMeta` 通道现由[规范工具输出约定](2026-07-20-canonical-tool-output-contract.md)拥有。 +- 被[结果时已应用 hunk 差异](../../archived/architecture/2026-07-02-result-time-applied-hunk-diffs.md)(已归档)扩展:后者添加了一个持久化的 `meta` 通道,使 write/edit 在结果时输出 `DiffResultView`(应用后的变更:带上下文行的 contextual hunk / 每个 `replace_all` 位点一个,或创建时的整文件 diff)——值/呈现拆分与持久化的 `presentationMeta` 通道现由[规范工具输出约定](2026-07-20-canonical-tool-output-contract.zh.md)拥有。 - 将 `ToolTerminal` 折入当前 UI 传输层使用的带标签 `terminal` 视图。 diff --git a/.agents/notes/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.i18n.yaml index 800449486f..13871d1cff 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md 2026-07-05-prompt-variables-and-tool-guidance-ownership.md: 9a53619d9510e3f4fa561f8420b2da3bedbbf4bb -2026-07-05-prompt-variables-and-tool-guidance-ownership.zh.md: 6174f277f0a4e8c46ff8d732411f28b4ea97728c +2026-07-05-prompt-variables-and-tool-guidance-ownership.zh.md: dc1164198df0f92b843c75b618f140d8aef86e4f diff --git a/.agents/notes/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.zh.md b/.agents/notes/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.zh.md index 6174f277f0..dc1164198d 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.zh.md @@ -40,7 +40,7 @@ Status: implemented ### Subagent 对话历史描述符 -`SubagentProvider.inheritsParentContext` 描述的是对话历史初始化,而非作用域、服务、工具或权限。spawn 和 ACP 将其设为 `false`;fork 设为 `true`。`dsh-tool-subagent` 根据该标志派生工具描述和提示词参数描述,包括 fork 继承已完成轮次但不继承进行中轮次这一点。提供方生命周期事件使该措辞与响应式提供方注册保持同步;其设计动机见[提供方生命周期事件 Agent Note](2026-07-05-subagent-provider-lifecycle-events.md)。 +`SubagentProvider.inheritsParentContext` 描述的是对话历史初始化,而非作用域、服务、工具或权限。spawn 和 ACP 将其设为 `false`;fork 设为 `true`。`dsh-tool-subagent` 根据该标志派生工具描述和提示词参数描述,包括 fork 继承已完成轮次但不继承进行中轮次这一点。提供方生命周期事件使该措辞与响应式提供方注册保持同步;其设计动机见[提供方生命周期事件 Agent Note](2026-07-05-subagent-provider-lifecycle-events.zh.md)。 ## 曾考虑的替代方案 @@ -49,7 +49,7 @@ Status: implemented - **在每个 persona 中手写模型名称**:与上方一行的 `model:` 键重复,配置修改后静默失实;正是本决策要治愈的病症。 - **宽松插值(未知引用保留原样或替换为空)**:一个拼写错误 `{{modle}}`(或一个空洞)会被发送给模型,直到 transcript(文本记录)审查时才会被发现。 - **在配置中为每个 subagent 实例编写措辞**:面向模型的行文回到每个部署 × 实例中,重蹈在 leaf YAML 中手写指导的漂移。**根据提供方名称选择措辞**:`providerName` 本身是配置,重命名提供方后会静默获得错误的措辞。 -- **在 `apply` 时解析提供方(加载顺序要求)**与**仅用 section 承载 subagent 措辞(在 assemble 时惰性解析)**:提供方生命周期事件的替代方案;两者均在[提供方生命周期事件 Agent Note](2026-07-05-subagent-provider-lifecycle-events.md)中被否决。 +- **在 `apply` 时解析提供方(加载顺序要求)**与**仅用 section 承载 subagent 措辞(在 assemble 时惰性解析)**:提供方生命周期事件的替代方案;两者均在[提供方生命周期事件 Agent Note](2026-07-05-subagent-provider-lifecycle-events.zh.md)中被否决。 ## 不在范围内 diff --git a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.i18n.yaml index b2478911dc..23c4ea4142 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md -2026-07-05-reconstructable-requests.md: 63146fa2d392a45543daa32ce2b00158782fddb2 -2026-07-05-reconstructable-requests.zh.md: 94c1d323be0107eb8b6072a05d1e8832ebd1fffc +2026-07-05-reconstructable-requests.md: 3f49ba71a6b98a84b05530c900e902b0cf9f6449 +2026-07-05-reconstructable-requests.zh.md: 7b8a9df65b60f975bc3ae60b2c1b0c3a8cc22e95 diff --git a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md index 63146fa2d3..3f49ba71a6 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md +++ b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md @@ -51,5 +51,6 @@ Like MiniCode, the conversation advances append-only and resets only when model- - What still costs full price at the provider is inherent and logged: compaction (its `compaction/*` events and replacement entry), a real prompt, tool, or config change (`request/header` with reason `change`), or a process boundary with drift (a differing `resume` snapshot). The provider's own reasoning-content exclusion is managed server-side. - `agent/pre-step` is the current-request message channel; direct inbox mutation is the eventual later-request channel. - Tool-result trimming needs no new mechanism: a logged single-entry surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic. +- Unreadable referenced attachment objects still fail model requests; [automatic attachment quarantine](../../proposed/bug-fix/2026-08-20-attachment-read-quarantine.md) records the proposed recovery without weakening byte-exact reconstruction. - Session logs grow one `request/header` snapshot per loop instance plus snapshots on real changes. This is larger than a delta codec but small beside chunk-heavy logs and retains one replay representation. `SESSION_FORMAT_VERSION` stays `0`; legacy delta events are rejected rather than migrated. - Snapshot expected outputs changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths. diff --git a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.zh.md b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.zh.md index 94c1d323be..7b8a9df65b 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.zh.md @@ -51,5 +51,6 @@ Status: implemented - 在提供方处仍需全价计算的内容是固有的且已记录的:压缩(其 `compaction/*` 事件和替换条目)、真正的提示词、工具或配置变更(reason 为 `change` 的 `request/header`),或带漂移的进程边界(不同的 `resume` 快照)。提供方自身的 reasoning-content 排除由服务端管理。 - `agent/pre-step` 是当前请求的消息通道;直接修改 inbox 则是最终进入后续请求的通道。 - 工具结果裁剪无需新机制:一个已记录的单条目 surface replace(`start === end`),携带同一 `callId` 下裁剪后的 `tool/result`——属压缩家族,回放正确,缓存失效由相同的压力逻辑批量处理。 +- 无法读取的被引用附件对象仍会让模型请求失败;[附件自动隔离](../../proposed/bug-fix/2026-08-20-attachment-read-quarantine.zh.md)记录了不削弱字节精确重建的拟议恢复方案。 - 会话日志每个循环实例增长一个 `request/header` 快照,并在真正变更时增加快照。它比 delta 编解码器更大,但相对分片密集型日志仍然很小,并只保留一种回放表示。`SESSION_FORMAT_VERSION` 保持 `0`;旧的 delta 事件被拒绝而非迁移。 - 快照预期输出变更一次(每个 transcript(文本记录)增加其 header 事件);写入文件系统的 fixture(测试前置数据)以规范化的撰写形式存储,工具参数使用 cwd 相对路径,因为回放只对 cwd 无关的参数路径做往返。 diff --git a/.agents/notes/implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.i18n.yaml index 336d9669ed..a66c6f6065 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.md 2026-07-05-subagent-provider-lifecycle-events.md: 503c0d638785e3a4944c8903b8d7d469c68b1881 -2026-07-05-subagent-provider-lifecycle-events.zh.md: e7d1d8c398a6d77efb2dbccc365e0040f38f2d33 +2026-07-05-subagent-provider-lifecycle-events.zh.md: d9d15b4ff688506b84b81c8649e1de4ce50e086d diff --git a/.agents/notes/implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.zh.md b/.agents/notes/implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.zh.md index e7d1d8c398..d9d15b4ff6 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.zh.md @@ -6,9 +6,9 @@ Status: implemented ## 问题 -[提示词变量 Agent Note](2026-07-05-prompt-variables-and-tool-guidance-ownership.md) 让 `dsh-tool-subagent` 从其提供方派生面向模型的措辞:`SubagentProvider.inheritsParentContext`(spawn 和 ACP(Agent Client Protocol)为 `false`,fork 为 `true`)同时驱动工具描述和 `prompt` 参数描述,使 fork 工具不再在上下文继承问题上对模型撒谎。这一修复引入了跨 fiber 的数据依赖:工具描述在工具注册时固定(这是有意为之——描述是 tool-choice 引导所在之处),但提供方在自己的插件 fiber 上到达,时机不确定。 +[提示词变量 Agent Note](2026-07-05-prompt-variables-and-tool-guidance-ownership.zh.md) 让 `dsh-tool-subagent` 从其提供方派生面向模型的措辞:`SubagentProvider.inheritsParentContext`(spawn 和 ACP(Agent Client Protocol)为 `false`,fork 为 `true`)同时驱动工具描述和 `prompt` 参数描述,使 fork 工具不再在上下文继承问题上对模型撒谎。这一修复引入了跨 fiber 的数据依赖:工具描述在工具注册时固定(这是有意为之——描述是 tool-choice 引导所在之处),但提供方在自己的插件 fiber 上到达,时机不确定。 -如果在工具插件的 `apply` 时刻解析提供方,就会产生一个隐式的加载顺序要求(「在 cordis.yml 中把后端列在工具前面」)。这个要求不成立,因为 Cordis Loader 并发启动同级条目,且 `Entry.init()` 不会等待激活完成:延迟到达的后端即使列在前面,也可能让工具 fiber 失败。Loader 不提供同级顺序保证——「异步状态不是同步状态」(见[防御性模式](../../../../docs/defensive-patterns.md))。 +如果在工具插件的 `apply` 时刻解析提供方,就会产生一个隐式的加载顺序要求(「在 cordis.yml 中把后端列在工具前面」)。这个要求不成立,因为 Cordis Loader 并发启动同级条目,且 `Entry.init()` 不会等待激活完成:延迟到达的后端即使列在前面,也可能让工具 fiber 失败。Loader 不提供同级顺序保证——「异步状态不是同步状态」(见[防御性模式](../../../../docs/defensive-patterns.zh.md))。 ## 决策 @@ -31,6 +31,6 @@ Status: implemented ## 后果 - 从命名提供方派生状态的消费方响应 `subagent/provider-added`/`-removed` 事件,而非在 `apply` 时读取注册表;`dsh-tool-subagent` 是参考实现。 -- **添加时大声失败;移除时按监听器隔离。** 添加监听器可以回滚注册。移除在 disposal 期间运行,因此单个监听器抛异常只会被记录日志,不会饿死后续镜像或干扰拆解流程。`start()` 仍在每次运行时按名称解析提供方,防止陈旧工具调用已移除的后端。见[事件目录](../../../../docs/subsystems/subagent.md#cordis-surface)与[生产者/消费方映射](../../../../docs/event-producer-consumer.md)。 +- **添加时大声失败;移除时按监听器隔离。** 添加监听器可以回滚注册。移除在 disposal 期间运行,因此单个监听器抛异常只会被记录日志,不会饿死后续镜像或干扰拆解流程。`start()` 仍在每次运行时按名称解析提供方,防止陈旧工具调用已移除的后端。见[事件目录](../../../../docs/subsystems/subagent.zh.md#cordis-surface)与[生产者/消费方映射](../../../../docs/event-producer-consumer.zh.md)。 - **工具不存在的窗口期。** 在后端 disposal 与重新注册之间(HMR 重载期间),模型看不到 subagent 工具。这是诚实的状态——替代方案是一个向空处分发的工具——工具注册表发出的 `tools/change` 事件会使提示词组装保持最新状态。 - **两个等待中的 fiber 共享同一 `toolName` 是无效配置,被延迟捕获。** 如果两个 `dsh-tool-subagent` 加载实例分别指定了不同的提供方但相同的 `toolName`,两者都会等待,先到达的提供方先注册;第二次注册仅在其提供方到达时才抛异常。插件中的 `TODO(subagent-dup-toolname)` 记录了这一影响范围;工具注册表的重名拒绝机制仍是最终防线。 diff --git a/.agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.i18n.yaml index cf8eaf225c..cc278292d9 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.md 2026-07-05-windows-jsonl-durable-publish.md: 546cde6086f3c84e22b4d4de144a48bb3425cd4e -2026-07-05-windows-jsonl-durable-publish.zh.md: 205460bcd374bd351cfdf541eb4041c09461229d +2026-07-05-windows-jsonl-durable-publish.zh.md: b99032e9d7e72e2028f39c5af91659278fd81343 diff --git a/.agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.zh.md b/.agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.zh.md index 205460bcd3..b99032e9d7 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.zh.md @@ -28,7 +28,7 @@ Windows 通过持久的暂存发布来创建缺失目录:创建一个以固定 ## 影响 -该后端在各平台上维持同一项外部约定:首次追加要么把完整日志发布到最终名称,要么失败且不覆盖已有日志。平台分流只是实现细节;`SessionPersistence` API 和 JSONL 逻辑记录格式均不改变。后续的 [Zstandard 编码决策](2026-07-19-zstandard-jsonl-session-logs.md)会先作用于不透明字节,然后才由任一平台执行发布。 +该后端在各平台上维持同一项外部约定:首次追加要么把完整日志发布到最终名称,要么失败且不覆盖已有日志。平台分流只是实现细节;`SessionPersistence` API 和 JSONL 逻辑记录格式均不改变。后续的 [Zstandard 编码决策](2026-07-19-zstandard-jsonl-session-logs.zh.md)会先作用于不透明字节,然后才由任一平台执行发布。 Windows 测试会在原生 Windows 上执行真实的 Win32 发布路径。断电行为属于 API 约定属性,单元测试无法证明;可测试的不变量包括:Windows 物化不会调用目录 fsync、最终路径冲突会失败、达到最大长度的目标路径组件仍可物化、临时日志在发布前已经执行 fsync,并且生成的日志可以正常加载。 diff --git a/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.i18n.yaml index e7e49aa115..66dd29f3a4 100644 --- a/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md -2026-07-06-timeout-deadline-library.md: 38f048d16ecba0e5278ae34b0c88b7889dcfa47a -2026-07-06-timeout-deadline-library.zh.md: e8b0ab62b19d931c33027d5e6c9919804e80aa99 +2026-07-06-timeout-deadline-library.md: 95adc41bffff6d7711685ebc52cb73b2b455df41 +2026-07-06-timeout-deadline-library.zh.md: 8b7b18a2d1e7757102afc81bea03245de2707d86 diff --git a/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md b/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md index 38f048d16e..95adc41bff 100644 --- a/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md +++ b/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md @@ -8,7 +8,7 @@ English | [中文](2026-07-06-timeout-deadline-library.zh.md) Timeout handling was drifting apart across the tool-bearing capabilities, and the divergence was not superficial — it was the same logic re-implemented three ways, each with its own subtle correctness burden. -- **bash** (then in the bash-local implementation's `run.ts`) had a full, correct timeout inside the process plumbing: a config-clamped `timeoutMs`, two independent triggers — a `killTimer` for the timeout and an `onAbort` listener for upstream cancellation — each calling one `kill()` closure that escalates SIGTERM→grace→SIGKILL on the process group, and two orthogonal outcome booleans (`timedOut`, `aborted`) latched independently. After this consolidation, the plumbing — today [packages/subprocess/subprocess-local/src/spawn.ts](../../../../packages/subprocess/subprocess-local/src/spawn.ts) — only reacts to aborts; [packages/shell/bash-local/src/index.ts](../../../../packages/shell/bash-local/src/index.ts) owns the fused deadline and the `timedOut`/`aborted` classification. +- **bash** (then in the bash-local implementation's `run.ts`) had a full, correct timeout inside the process plumbing: a config-clamped `timeoutMs`, two independent triggers — a `killTimer` for the timeout and an `onAbort` listener for upstream cancellation — each calling one `kill()` closure that escalates SIGTERM→grace→SIGKILL on the process group, and two orthogonal outcome booleans (`timedOut`, `aborted`) latched independently. After this consolidation, the plumbing — [packages/subprocess/subprocess-local/src/spawn.ts](../../../../packages/subprocess/subprocess-local/src/spawn.ts) — only reacts to aborts; [packages/shell/bash-local/src/index.ts](../../../../packages/shell/bash-local/src/index.ts) owns the fused deadline and the `timedOut`/`aborted` classification. - **web_fetch** ([packages/web/web-fetch-http/src/provider.ts](../../../../packages/web/web-fetch-http/src/provider.ts)) had a correct but *hand-rolled* timeout: it constructed an `AbortController`, wired `setTimeout(() => controller.abort(new WebError(…, 'WEB_FETCH_TIMEOUT')))`, manually added and removed the upstream-signal listener, cleared the timer in a `finally`, and recovered the timeout reason from `signal.reason` in a `translateAbortOrNetwork` helper because the reader surfaces a bare `AbortError`. - **web_search** ([packages/web/tool-web/src/search.ts](../../../../packages/web/tool-web/src/search.ts)) had **no timeout at all**: `WebSearchRequest` ([packages/web/web/src/types.ts](../../../../packages/web/web/src/types.ts)) carries no `timeoutMs` field, and each provider's `search()` only forwards `exec.signal`. (web_search stays untimed here — see Consequences.) diff --git a/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.zh.md b/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.zh.md index e8b0ab62b1..8b7b18a2d1 100644 --- a/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.zh.md @@ -8,7 +8,7 @@ Status: implemented 超时处理在各个承载工具的能力之间逐渐分化,而且这种分化并非表面的:同一套逻辑被以三种方式重新实现,各自带有微妙的正确性负担。 -- **bash**(当时位于 bash-local 实现的 `run.ts`)在进程管道内部有一套完整、正确的超时实现:一个经配置钳位的 `timeoutMs`,两个独立触发器(用于超时的 `killTimer` 和用于上游取消的 `onAbort` 监听器),各自调用同一个 `kill()` 闭包对进程组执行 SIGTERM→宽限期→SIGKILL 升级,以及两个正交的结果布尔值(`timedOut`、`aborted`)独立锁存。经此次整合之后,这套管道——今天位于 [packages/subprocess/subprocess-local/src/spawn.ts](../../../../packages/subprocess/subprocess-local/src/spawn.ts)——只响应中止;[packages/shell/bash-local/src/index.ts](../../../../packages/shell/bash-local/src/index.ts) 拥有融合的 deadline 以及 `timedOut`/`aborted` 分类。 +- **bash**(当时位于 bash-local 实现的 `run.ts`)在进程管道内部有一套完整、正确的超时实现:一个经配置钳位的 `timeoutMs`,两个独立触发器(用于超时的 `killTimer` 和用于上游取消的 `onAbort` 监听器),各自调用同一个 `kill()` 闭包对进程组执行 SIGTERM→宽限期→SIGKILL 升级,以及两个正交的结果布尔值(`timedOut`、`aborted`)独立锁存。经此次整合之后,这套管道——位于 [packages/subprocess/subprocess-local/src/spawn.ts](../../../../packages/subprocess/subprocess-local/src/spawn.ts)——只响应中止;[packages/shell/bash-local/src/index.ts](../../../../packages/shell/bash-local/src/index.ts) 拥有融合的 deadline 以及 `timedOut`/`aborted` 分类。 - **web_fetch**([packages/web/web-fetch-http/src/provider.ts](../../../../packages/web/web-fetch-http/src/provider.ts))有一套正确但*手写*的超时:构造一个 `AbortController`,连接 `setTimeout(() => controller.abort(new WebError(…, 'WEB_FETCH_TIMEOUT')))`,手动添加和移除上游信号监听器,在 `finally` 中清除定时器,并在 `translateAbortOrNetwork` 辅助函数中从 `signal.reason` 恢复超时原因(因为 reader 只抛出裸 `AbortError`)。 - **web_search**([packages/web/tool-web/src/search.ts](../../../../packages/web/tool-web/src/search.ts))**完全没有超时**:`WebSearchRequest`([packages/web/web/src/types.ts](../../../../packages/web/web/src/types.ts))不携带 `timeoutMs` 字段,各提供方的 `search()` 只转发 `exec.signal`。(web_search 在本次设计中保持无超时——见「后果」。) @@ -103,7 +103,7 @@ export function timeoutOf(x: AbortSignal | { reason?: unknown }, code?: string): - `AbortSignal.any` 和 `using`/`Symbol.dispose` 在此首次进入本仓库(Node ≥ 24 基线,已满足)。 - 模型流现在共享一个可重启的定时器约定,不会把滑动的空闲间隔变成总调用截止时间,也不会计入消费方思考时间。能够观察到带外传输活动的适配器可以对尚未结算的 demand 调用 `pulse()`;被屏蔽的活动对 watchdog 仍不可见。该原语仍然只做通知;适配器测试证明其传输观察到稳定信号并终止。 -以下内容不在本次范围内,列出以标明边界:`web_search` 可以在其工具 schema 和快照覆盖规划完成后获得可选的面向模型的 `timeout_ms`;基于 ripgrep 的文件系统发现工具([打包的 ripgrep 搜索](2026-08-01-packaged-ripgrep-search.md))通过 `dsh-tool-call-timeout-policy` 和 `exec.signal` 消费同样的提供方自有 deadline 形状;`tools/execute` waterfall(瀑布式事件)中间件可以通过驱动 `exec.signal` 为每次工具调用设置默认 deadline——那将是一个*消费*本库的插件,仍然只做通知,硬终止仍是各能力自己的事。 +以下内容不在本次范围内,列出以标明边界:`web_search` 可以在其工具 schema 和快照覆盖规划完成后获得可选的面向模型的 `timeout_ms`;基于 ripgrep 的文件系统发现工具([打包的 ripgrep 搜索](2026-08-01-packaged-ripgrep-search.zh.md))通过 `dsh-tool-call-timeout-policy` 和 `exec.signal` 消费同样的提供方自有 deadline 形状;`tools/execute` waterfall(瀑布式事件)中间件可以通过驱动 `exec.signal` 为每次工具调用设置默认 deadline——那将是一个*消费*本库的插件,仍然只做通知,硬终止仍是各能力自己的事。 ## 曾考虑的替代方案 diff --git a/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.i18n.yaml index 5e943fed2b..554114fff0 100644 --- a/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.md -2026-07-06-tool-result-retention-library.md: 8d938db8f5fa78398a39e97cc877308200f7d60f -2026-07-06-tool-result-retention-library.zh.md: 0b2fe841beed21511c4088732713d3fd9fd9048f +2026-07-06-tool-result-retention-library.md: 464e3d51a0051487b7c29f0c01a11acb91d160e5 +2026-07-06-tool-result-retention-library.zh.md: 49361eec4b649d2d68dc36929baa0f9ff580eb68 diff --git a/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.md b/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.md index 8d938db8f5..464e3d51a0 100644 --- a/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.md +++ b/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.md @@ -16,7 +16,7 @@ The shared abstraction the tools need is **retention**, not generic collection. The library has two independent retainers: -- `ItemRetainer` handles ordered logical units such as paths, grep matches, or search sources. It supports `head` retention only in v1, while keeping the retainer shape open to additional retention strategies later. +- `ItemRetainer` handles ordered logical units such as paths, grep matches, or search sources. It supports only `head` retention, while keeping the retainer shape open to additional strategies. - `TextRetainer` handles byte-oriented text streams such as bash stdout/stderr or web response bodies. It supports `head`, `tail`, and `headTail` retention while preserving UTF-8 boundaries at `finish()`. Both retainers return a small `PushDecision` after each `push()` so callers can tell whether that unit/chunk was fully retained and whether the accumulated result is now truncated. Omission counts are exact because callers keep feeding every observed item/chunk. @@ -95,7 +95,7 @@ type TextRetentionStrategy = ### Tool mapping -`read` is intentionally outside the v1 retention library. Its `read-render` helper owns a file-specific pagination contract: `offset` / `limit`, line numbers, `totalLines`, offset-out-of-range errors, per-line preview truncation, and a selected-output byte cap that can stop scanning mid-window. That is a line-window renderer, not a generic retention primitive. It may share future neutral notice helpers, but it should not pass its already-selected window through `ItemRetainer`. +`read` is intentionally outside the retention library. Its `read-render` helper owns a file-specific pagination contract: `offset` / `limit`, line numbers, `totalLines`, offset-out-of-range errors, per-line preview truncation, and a selected-output byte cap that can stop scanning mid-window. That is a line-window renderer, not a generic retention primitive. It may share future neutral notice helpers, but it should not pass its already-selected window through `ItemRetainer`. `FsGlobEntry` and `FlatGrepMatch` below are the intended discovery-tool item shapes, not existing retention-library exports. `FsGlobEntry` is one backend-derived path, and `FlatGrepMatch` is one ungrouped grep match before the backend groups retained matches by file. @@ -142,15 +142,15 @@ The formatter hook is deliberately small: a tool turns a `RetentionNotice` into **Boundaries the library holds.** `truncated` means the retainer omitted otherwise-available content because of a budget; it never means the upstream was incomplete. Tool-specific states — `incomplete`, permission failures, provider partial failures, binary skips, bash spill-path recovery, invalid UTF-8 — stay in tool-domain fields, outside the retainer. When a future change migrates a tool, that package's README and tests must prove the model-facing result text is unchanged except for deliberate notice wording. -**Tradeoffs accepted.** The v1 API deliberately supports only item `head` retention and text `head` / `tail` / `headTail`; windows, grouped budgets, sort-aware caps, and upstream-stop control wait until a second consumer proves the need. Text retention counts bytes for process/body safety, leaving character- and line-level preview budgets as separate tool-owned concerns. +**Tradeoffs accepted.** The API deliberately supports only item `head` retention and text `head` / `tail` / `headTail`; windows, grouped budgets, sort-aware caps, and upstream-stop control wait until a second consumer proves the need. Text retention counts bytes for process/body safety, leaving character- and line-level preview budgets as separate tool-owned concerns. ## Alternatives considered **Post-hoc `truncate(text)` only.** Rejected: it matches Codex's history/tool-output truncation use case but loses item counts, grouping boundaries, UTF-8-safe byte windows, and exact omission metadata. -**One generic `Collector` with pluggable callbacks.** Rejected for v1: it hides the two important resource modes. Logical item retention counts items; text retention counts bytes and preserves UTF-8 boundaries. Separate `ItemRetainer` and `TextRetainer` names make that difference explicit while keeping the API small. +**One generic `Collector` with pluggable callbacks.** Rejected: it hides the two important resource modes. Logical item retention counts items; text retention counts bytes and preserves UTF-8 boundaries. Separate `ItemRetainer` and `TextRetainer` names make that difference explicit while keeping the API small. -**Put `read` windowing behind `ItemRetainer`.** Rejected for v1: `read` is the only current window consumer, and its semantics are file pagination rather than generic retention. A single `Omitted` count cannot represent both sides of a line window, and `read` also carries `totalLines`, offset-range errors, per-line preview truncation, and a byte cap over selected output. Keeping `read-render` tool-owned avoids growing the shared library around one special case. +**Put `read` windowing behind `ItemRetainer`.** Rejected: `read` is the only shipped window consumer, and its semantics are file pagination rather than generic retention. A single `Omitted` count cannot represent both sides of a line window, and `read` also carries `totalLines`, offset-range errors, per-line preview truncation, and a byte cap over selected output. Keeping `read-render` tool-owned avoids growing the shared library around one special case. **Make truncation part of `ToolExecutionResult`.** Rejected: the tool registry would have to understand tool-specific recovery guidance, grouping, line numbering, exit status, and provider semantics. Retention is a library used by a tool's Native renderer; the model-facing projection remains tool-owned while the [canonical value](2026-07-20-canonical-tool-output-contract.md) may retain the complete acquired result. diff --git a/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.zh.md b/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.zh.md index 0b2fe841be..49361eec4b 100644 --- a/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.zh.md @@ -16,7 +16,7 @@ Status: implemented 该库包含两个相互独立的 retainer: -- `ItemRetainer` 处理有序逻辑单元,例如路径、grep 匹配项或搜索来源。v1 只支持 `head` 保留,同时维持 retainer 形态,以便未来加入其他保留策略。 +- `ItemRetainer` 处理有序逻辑单元,例如路径、grep 匹配项或搜索来源。它只支持 `head` 保留,同时维持 retainer 形态,以便未来加入其他保留策略。 - `TextRetainer` 处理面向字节的文本流,例如 bash stdout/stderr 或 web 响应正文。它支持 `head`、`tail` 和 `headTail` 保留,并在 `finish()` 时维持 UTF-8 边界。 两个 retainer 都会返回一个小型 `PushDecision`;每次调用 `push()` 后,调用方都能得知该单元/分片是否完整保留,以及累积结果此时是否已被截断。因为调用方会继续输入每一个已观察到的条目/分片,所以省略计数是精确的。 @@ -95,7 +95,7 @@ type TextRetentionStrategy = ### 工具映射 -`read` 被有意排除在 v1 保留库之外。它的 `read-render` 辅助函数拥有文件专用的分页约定:`offset`/`limit`、行号、`totalLines`、offset 越界错误、逐行预览截断,以及能够在窗口中途停止扫描的所选输出字节上限。这是行窗口渲染器,不是通用保留原语。它未来可以共享中性的提示辅助函数,但不应把已经选定的窗口再传入 `ItemRetainer`。 +`read` 被有意排除在保留库之外。它的 `read-render` 辅助函数拥有文件专用的分页约定:`offset`/`limit`、行号、`totalLines`、offset 越界错误、逐行预览截断,以及能够在窗口中途停止扫描的所选输出字节上限。这是行窗口渲染器,不是通用保留原语。它未来可以共享中性的提示辅助函数,但不应把已经选定的窗口再传入 `ItemRetainer`。 下文的 `FsGlobEntry` 与 `FlatGrepMatch` 是预期由发现工具使用的条目形态,不是现有保留库的导出。`FsGlobEntry` 是一个由后端派生的路径;`FlatGrepMatch` 是后端将保留匹配项按文件分组之前的一条未分组 grep 匹配。 @@ -103,7 +103,7 @@ type TextRetentionStrategy = `grep` 在分组前使用 `ItemRetainer`,并将其配置为 `{ kind: 'head', maxItems: grepMaxMatches }`。执行器解析 ripgrep 输出、映射路径、应用逐行预览截断,并输入扁平匹配项。调用 `finish()` 后,工具按文件对保留的匹配项分组;如果行内结果达到上限,还可以通过 spill seam 保存完整匹配列表。分组不属于 retainer,因为上限针对匹配总数,而不是文件数;逐匹配项的预览截断和 `incomplete` 也与结果级保留相互独立。 -`bash` 可以使用 `TextRetainer`,配置为 `tail` 或 `headTail`,并读取至进程结束。bash 执行器仍负责 spill 文件、退出状态、信号、超时与后台任务行为;保留辅助函数只在需要该行为时替换临时实现的内存首尾核算。长时间运行任务的所有权与[通用长时间运行工具的运行时](2026-06-20-generic-long-running-tool-runtime.md)相互独立。 +`bash` 可以使用 `TextRetainer`,配置为 `tail` 或 `headTail`,并读取至进程结束。bash 执行器仍负责 spill 文件、退出状态、信号、超时与后台任务行为;保留辅助函数只在需要该行为时替换临时实现的内存首尾核算。长时间运行任务的所有权与[通用长时间运行工具的运行时](2026-06-20-generic-long-running-tool-runtime.zh.md)相互独立。 `web_fetch` 可以使用 `TextRetainer`,配置为 `head` 或 `headTail`;如果提供方必须在内部读取和解码,也可以保留由提供方负责的正文上限。无论采用哪种方式,fetch 结果中的 `truncated` 仍是提供方/工具事实,该库只提供保留文本与省略元数据。 @@ -138,20 +138,20 @@ const formatGrepNotice = (notice: RetentionNotice): string => **已交付内容。** `@deepseek-ai/dsh-output-retention` 导出 `ItemRetainer`、`TextRetainer`、结果类型(`RetainedItems`、`RetainedText`)、策略类型(`ItemRetentionStrategy`、`TextRetentionStrategy`)、`Omitted`、`PushDecision`、`RetentionNotice`,以及中性的提示辅助函数 `describeOmitted`/`formatRetentionNotice`,且不依赖 Cordis 或任何工具包。单元测试覆盖具有精确省略计数的条目头部保留、文本头部保留、文本尾部保留、首尾字节保留、零预算、UTF-8 边界处理(2、3、4 字节码位,以及每个裁切位置上的无效起始字节)和未知省略量的措辞。 -**已记录但尚未迁移的内容。** `glob`、`grep`、`bash`、`web_fetch` 与 `web_search` 的映射已记录在[包 README](../../../../packages/util/output-retention/README.md) 中,但本次改动并未把每个工具都迁移到该库;迁移工作刻意留作独立的后续任务。`read` 被明确记录为不在范围内:其 `read-render` 行窗口约定(`offset`/`limit`、`totalLines`、offset 范围错误、逐行预览截断,以及针对所选窗口的字节上限)不属于通用保留,而一个 `Omitted` 计数也无法同时表达行窗口两侧。 +**已记录但尚未迁移的内容。** `glob`、`grep`、`bash`、`web_fetch` 与 `web_search` 的映射已记录在[包 README](../../../../packages/util/output-retention/README.zh.md) 中,但本次改动并未把每个工具都迁移到该库;迁移工作刻意留作独立的后续任务。`read` 被明确记录为不在范围内:其 `read-render` 行窗口约定(`offset`/`limit`、`totalLines`、offset 范围错误、逐行预览截断,以及针对所选窗口的字节上限)不属于通用保留,而一个 `Omitted` 计数也无法同时表达行窗口两侧。 **该库维持的边界。** `truncated` 表示 retainer 因预算省略了原本可用的内容,绝不表示上游不完整。工具专用状态,包括 `incomplete`、权限失败、提供方局部失败、跳过二进制文件、bash spill 路径恢复和无效 UTF-8,均留在工具领域字段中、位于 retainer 之外。未来改动迁移某项工具时,该包的 README 与测试必须证明,除了有意改变的提示措辞外,模型可见的结果文本没有变化。 -**接受的取舍。** v1 接口刻意只支持条目的 `head` 保留,以及文本的 `head`/`tail`/`headTail` 保留;窗口、分组预算、感知排序的上限和上游停止控制,要等第二个消费方证明需求后再引入。文本保留按字节计数,以保障进程/正文安全;字符级和行级预览预算继续由具体工具负责。 +**接受的取舍。**接口刻意只支持条目的 `head` 保留,以及文本的 `head`/`tail`/`headTail` 保留;窗口、分组预算、感知排序的上限和上游停止控制,要等第二个消费方证明需求后再引入。文本保留按字节计数,以保障进程/正文安全;字符级和行级预览预算继续由具体工具负责。 ## 考虑过的替代方案 **只进行事后 `truncate(text)`。** 不予采纳:它适合 Codex 的历史/工具输出截断场景,却会丢失条目计数、分组边界、UTF-8 安全的字节窗口与精确省略元数据。 -**使用一个带可插拔回调的通用 `Collector`。** v1 不予采纳,因为它会掩盖两种重要的资源模式。逻辑条目保留按条目计数;文本保留按字节计数并维持 UTF-8 边界。独立的 `ItemRetainer` 与 `TextRetainer` 名称明确表达这种差异,同时保持 API 精简。 +**使用一个带可插拔回调的通用 `Collector`。**不予采纳,因为它会掩盖两种重要的资源模式。逻辑条目保留按条目计数;文本保留按字节计数并维持 UTF-8 边界。独立的 `ItemRetainer` 与 `TextRetainer` 名称明确表达这种差异,同时保持 API 精简。 -**把 `read` 窗口交给 `ItemRetainer`。** v1 不予采纳:`read` 是当前唯一的窗口消费方,其语义属于文件分页,而不是通用保留。一个 `Omitted` 计数无法表示行窗口两侧,而且 `read` 还携带 `totalLines`、offset 范围错误、逐行预览截断和针对所选输出的字节上限。让 `read-render` 由工具所有,可以避免共享库围绕一项特例膨胀。 +**把 `read` 窗口交给 `ItemRetainer`。**不予采纳:`read` 是唯一已交付的窗口消费方,其语义属于文件分页,而不是通用保留。一个 `Omitted` 计数无法表示行窗口两侧,而且 `read` 还携带 `totalLines`、offset 范围错误、逐行预览截断和针对所选输出的字节上限。让 `read-render` 由工具所有,可以避免共享库围绕一项特例膨胀。 -**让截断成为 `ToolExecutionResult` 的一部分。** 不予采纳:工具注册表将不得不理解工具专用的恢复指引、分组、行号、退出状态和提供方语义。保留是由工具的 Native renderer(原生渲染器)使用的库;模型可见投影继续由工具所有,而[规范值](2026-07-20-canonical-tool-output-contract.md)可以保留完整的已采集结果。 +**让截断成为 `ToolExecutionResult` 的一部分。** 不予采纳:工具注册表将不得不理解工具专用的恢复指引、分组、行号、退出状态和提供方语义。保留是由工具的 Native renderer(原生渲染器)使用的库;模型可见投影继续由工具所有,而[规范值](2026-07-20-canonical-tool-output-contract.zh.md)可以保留完整的已采集结果。 **在每个面向模型的工具 schema 中公开上限。** 不作为默认方案:Claude Code 的 grep 公开 `head_limit`/`offset`,但本 harness 会把常规预算保留为部署配置,除非模型确实需要控制分页。未来可以为具体工具增加类似 read 的续传字段;它不属于共享保留原语。 diff --git a/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.i18n.yaml index aad3aa26f0..373b1293ce 100644 --- a/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.md -2026-07-07-tool-call-timeout-policy.md: ce414e541f8e374dd48e46d68cb00121e0004247 -2026-07-07-tool-call-timeout-policy.zh.md: 6fe3c979a3c4e7b7a6ed803a47af45ad32d52cce +2026-07-07-tool-call-timeout-policy.md: 92618cc8c761b38d7e516c9d00eb3de1c37831a8 +2026-07-07-tool-call-timeout-policy.zh.md: 0303fadd134c6eb8c41449823b62f1458f4523ba diff --git a/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.md b/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.md index ce414e541f..92618cc8c7 100644 --- a/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.md +++ b/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.md @@ -77,7 +77,7 @@ No new session event is needed for reconstructability: `TOOL_TIMEOUT` is the fin ### Existing tool adaptation -`web_fetch` and `web_search` are migrated. `dsh-tool-web` keeps ownership of their model-facing schemas, and those schemas expose no timeout knob: `web_fetch` dropped its `timeout_ms` parameter to match the reference-agent shape, and `web_search` stays query-only. The tool bodies do not import `@deepseek-ai/dsh-timeout`; they forward `exec.signal` to `ctx.web`. +`web_fetch` and `web_search` are migrated. `dsh-tool-web` keeps ownership of their model-facing schemas, and those schemas expose no timeout knob: `web_fetch` has no `timeout_ms` parameter, while `web_search` accepts a required `queries` array without a timeout argument. The tool bodies do not import `@deepseek-ai/dsh-timeout`; they forward `exec.signal` to `ctx.web`. `dsh-web-fetch-http` keeps one configured provider-level `timeoutMs` as a large resource backstop for direct `ctx.web.fetch()` callers and misconfigured deployments; it owns no model-facing timeout. When a `TOOL_TIMEOUT` signal reaches the fetch provider first, provider-scoped classification treats it as upstream `WEB_ABORTED`, and the outer `tools/execute` wrapper replaces the final tool result with `TOOL_TIMEOUT`. A shipped web-tool deployment configures the provider backstop above the `timeout-policy` budget so the tool-call policy normally wins for model calls. diff --git a/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.zh.md b/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.zh.md index 6fe3c979a3..0303fadd13 100644 --- a/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -[超时/截止时间 Agent Note](2026-07-06-timeout-deadline-library.md) 将计时与分类原语提取到了 `@deepseek-ai/dsh-timeout`,但超时策略仍然附着在各个能力和面向模型的 schema 上。`bash` 暴露了 `timeoutMs`;`web_fetch` 暴露了 `timeout_ms`;`web_search` 没有面向模型的超时参数,尽管提供方已经遵循 `exec.signal`;未来的 grep/glob 工具要么直接导入超时库,要么自行发明超时策略。对于一个插件 SDK 来说,这是错误的编写范式:工具作者通常只需将 `exec.signal` 转发给其调用的实现,而部署策略来决定预算。 +[超时/截止时间 Agent Note](2026-07-06-timeout-deadline-library.zh.md) 将计时与分类原语提取到了 `@deepseek-ai/dsh-timeout`,但超时策略仍然附着在各个能力和面向模型的 schema 上。`bash` 暴露了 `timeoutMs`;`web_fetch` 暴露了 `timeout_ms`;`web_search` 没有面向模型的超时参数,尽管提供方已经遵循 `exec.signal`;未来的 grep/glob 工具要么直接导入超时库,要么自行发明超时策略。对于一个插件 SDK 来说,这是错误的编写范式:工具作者通常只需将 `exec.signal` 转发给其调用的实现,而部署策略来决定预算。 与此同时,仓库中并非所有超时都是面向模型的工具调用预算。钩子通过直接调用 `ctx.shell` 执行命令钩子,而非通过 `ctx.tools.execute()`;`bash` 模型工具通过同一个后端复用前台执行、后台启动、后台轮询和钩子复用。一步到位地将所有超时移入工具插件会混淆这些路径,并有破坏钩子超时语义的风险。 @@ -16,7 +16,7 @@ Status: implemented - `@deepseek-ai/dsh-timeout` 仍是拥有 `deadline()` 和 `timeoutOf()` 的共享库。 - `@deepseek-ai/dsh-tools` 在 `tools/pre-execute` 和 `tools/post-execute` 之间有一个环绕分发的 waterfall(瀑布式事件)`tools/execute`。 -- [仓库命名约定](2026-08-11-repository-naming-contract-and-rename-ledger.md)使用 `@deepseek-ai/dsh-tool-call-timeout-policy`,准确说明该策略所限制的操作。插件从 runtime 读取每个工具声明的 `timeoutMs`,并通过派生新的 `exec.signal` 来包装有此声明的调用。 +- [仓库命名约定](2026-08-11-repository-naming-contract-and-rename-ledger.zh.md)使用 `@deepseek-ai/dsh-tool-call-timeout-policy`,准确说明该策略所限制的操作。插件从 runtime 读取每个工具声明的 `timeoutMs`,并通过派生新的 `exec.signal` 来包装有此声明的调用。 执行流水线如下: @@ -52,7 +52,7 @@ catch 是基础 `next`(而非 waterfall 之外的东西)这一点至关重 searchTimeoutMs: 30000 ``` -超时放在工具定义上而非自由文本名称映射中,消除了拼错名称导致策略不生效的问题。`defineTool` 校验预算为正有限数。分发期间,执行器派生截止信号并将其赋给 `exec.signal`;注册表依据[工具取消约定](2026-07-19-cooperative-tool-cancellation.md),在执行工具体之前将该截止信号与调用方的原始信号融合。执行器随后恢复调用方信号,并将自身的超时转换为 `TOOL_TIMEOUT`;没有预算的工具原样通过。 +超时放在工具定义上而非自由文本名称映射中,消除了拼错名称导致策略不生效的问题。`defineTool` 校验预算为正有限数。分发期间,执行器派生截止信号并将其赋给 `exec.signal`;注册表依据[工具取消约定](2026-07-19-cooperative-tool-cancellation.zh.md),在执行工具体之前将该截止信号与调用方的原始信号融合。执行器随后恢复调用方信号,并将自身的超时转换为 `TOOL_TIMEOUT`;没有预算的工具原样通过。 信号替换采用**就地修改 `exec.signal`** 的方式,而非向 `next()` 传递新对象。Cordis 的 waterfall `next()` 忽略传入的任何参数,并以共享的 payload 数组重新调用下游监听器(`vendor/cordis/src/events.ts`),因此修改共享对象是包装器向注册表提供截止信号的方式。注册表会在进入工具体前再次融合已捕获的调用方信号;插件则在 `finally` 中将 `exec.signal` 恢复为调用方的原始值,使 `tools/post-execute` 永远不会看到本插件的截止信号。 @@ -77,7 +77,7 @@ function toolTimeoutResult(timeoutMs: number): ToolExecutionResult { ### 现有工具适配 -`web_fetch` 和 `web_search` 已迁移。`dsh-tool-web` 保留对其面向模型 schema 的所有权,这些 schema 不暴露超时旋钮:`web_fetch` 移除了 `timeout_ms` 参数以匹配参考 agent(智能体)的形状,`web_search` 保持仅查询。工具体不导入 `@deepseek-ai/dsh-timeout`;它们将 `exec.signal` 转发给 `ctx.web`。 +`web_fetch` 和 `web_search` 已迁移。`dsh-tool-web` 保留对其面向模型 schema 的所有权,这些 schema 不暴露超时旋钮:`web_fetch` 没有 `timeout_ms` 参数,`web_search` 接受必填的 `queries` 数组,但不接受超时参数。工具体不导入 `@deepseek-ai/dsh-timeout`;它们将 `exec.signal` 转发给 `ctx.web`。 `dsh-web-fetch-http` 保留一个在提供方层面配置的 `timeoutMs`,作为较大的资源兜底值,服务于直接调用 `ctx.web.fetch()` 的调用方和配置错误的部署;它不拥有面向模型的超时。当 `TOOL_TIMEOUT` 信号先到达 fetch 提供方时,提供方作用域的分类将其视为上游 `WEB_ABORTED`,而外层 `tools/execute` 包装器将最终工具结果替换为 `TOOL_TIMEOUT`。一个已发布的 web 工具部署将提供方兜底配置为高于 `timeout-policy` 预算,使工具调用策略在模型调用中通常胜出。 diff --git a/.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.i18n.yaml index efed9ca5ea..20d0e4f85c 100644 --- a/.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.md 2026-07-08-agent-scope-contexts.md: eb3f6f247bac1a1d81aa2644132c7b9cc04d602c -2026-07-08-agent-scope-contexts.zh.md: fe82ac18d07de97330e86461e6bdf86edc37d2ae +2026-07-08-agent-scope-contexts.zh.md: a0f4ffb0ef80dd2fc1ee61c9ab3f4730c28c687e diff --git a/.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.zh.md b/.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.zh.md index fe82ac18d0..a0f4ffb0ef 100644 --- a/.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.zh.md @@ -16,7 +16,7 @@ Status: implemented 每个存活的 agent 拥有一个扁平的注册层,通过 `agent.ctx` 暴露。代码通过拥有某项贡献的上下文进行注册;具备作用域感知的服务将部署全局注册与恰好一个匹配的 agent 层合并;操作从其真实 agent 选择该层;该层在 agent 的完整发布生命周期内存在。 -Cordis 是 SDK 底层的插件框架。Cordis **上下文**是插件用来访问服务和注册效果的对象,效果的清理跟随该上下文。[Cordis 入门](../../../../docs/cordis-primer.md)对该框架有更详细的说明。 +Cordis 是 SDK 底层的插件框架。Cordis **上下文**是插件用来访问服务和注册效果的对象,效果的清理跟随该上下文。[Cordis 入门](../../../../docs/cordis-primer.zh.md)对该框架有更详细的说明。 对大多数贡献者而言,完整约定是四条规则: @@ -45,7 +45,7 @@ flowchart LR 缺失的交叉边即隔离规则:Agent A 的本地注册不会进入 Agent B 的视图,父级的注册也不会仅因父级拥有子级的生命周期就进入子级。 -配套的[运行时设计 Agent Note](2026-07-12-agent-scope-runtime-design.md) 阐述了实现与正确性推理。[subagent 组合控制 Agent Note](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.md) 负责独立的 `persona`、`toolFilter` 和 `maxDepth` 功能。 +配套的[运行时设计 Agent Note](2026-07-12-agent-scope-runtime-design.zh.md) 阐述了实现与正确性推理。[subagent 组合控制 Agent Note](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.zh.md) 负责独立的 `persona`、`toolFilter` 和 `maxDepth` 功能。 ### 注册来源决定可见性与清理 @@ -104,7 +104,7 @@ setup 接收一个完整的受信 Cordis 上下文,因此可以组合普通插 在 Cordis 层面,`Scoped` 是一个不透明的路由接收器。它携带用于选择监听器的过滤器,但本身不是领域对象。因此事件签名将真实的 `Agent`、工具执行、审批请求或其他主体作为显式参数保留,供监听器检查。 -以 `{ global: true }` 注册的监听器有意绕过上下文受众过滤,但其清理仍跟随注册上下文。注册表成员变更通知保持不过滤,因为它们描述的是共享注册表状态而非某个 agent 的操作。详尽的事件参考是各[子系统页面](../../../../docs/subsystems/core.md)上生成的 `cordis-surface` 区块的集合——每个事件作用域在其所属页面上(`agent/*` 与 `agent-loop/*` 在 core.md 本页)。 +以 `{ global: true }` 注册的监听器有意绕过上下文受众过滤,但其清理仍跟随注册上下文。注册表成员变更通知保持不过滤,因为它们描述的是共享注册表状态而非某个 agent 的操作。详尽的事件参考是各[子系统页面](../../../../docs/subsystems/core.zh.md)上生成的 `cordis-surface` 区块的集合——每个事件作用域在其所属页面上(`agent/*` 与 `agent-loop/*` 在 core.md 本页)。 ### 创建最后发布,dispose 最后撤销 @@ -138,6 +138,8 @@ flowchart TB detach --> revoke["Dispose the agent scope"] ``` + + ## 安全与权限是非目标 agent 作用域组合的是受信的同进程注册。它不沙箱化插件、不定义父到子的权限格、不在创建时冻结授权、也不保证子级不能做超出父级的事。 @@ -146,6 +148,8 @@ agent 作用域组合的是受信的同进程注册。它不沙箱化插件、 需要非升权保证的部署需要独立的权限表示、传播规则和执行检查。父级子集授权、创建时授权快照、显式未来授权 API,以及通用的能力/输出/终止标签均不在本决策范围内。 + + ## 曾考虑的替代方案 被否决的设计要么将可见性与清理分离,要么只覆盖一类注册,要么重复共享基础设施,要么将生命周期所有权与继承混为一谈。 diff --git a/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.i18n.yaml index 5f9f8ba421..5ba89a4412 100644 --- a/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md -2026-07-08-tool-output-spill-files.md: 4d1d4b7b665f34b362df2f8c8aeb06d96bf2668f -2026-07-08-tool-output-spill-files.zh.md: 1a3387830861c6d05318024870325523b8eeee36 +2026-07-08-tool-output-spill-files.md: 14667b74ca877622d05196e9bf83945a842fe366 +2026-07-08-tool-output-spill-files.zh.md: db297fa6bee707a1d5a10d20260ce6b8a660d207 diff --git a/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md b/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md index 4d1d4b7b66..14667b74ca 100644 --- a/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md +++ b/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md @@ -147,8 +147,8 @@ Those cases can consume `ctx.spillStore` directly in later work. They are not pa ## Non-goals -- No new model-facing `artifact_read` or `artifact_search` tool in v1. -- No per-tool retention configuration in v1. +- This decision adds no model-facing `artifact_read` or `artifact_search` tool. +- This decision adds no per-tool retention configuration. - No model-facing timeout/truncation arguments. - No migration of `read` output into spill files. - No replacement for provider/resource caps such as `web-fetch-http.maxBodyChars`. @@ -174,9 +174,9 @@ Those cases can consume `ctx.spillStore` directly in later work. They are not pa The default policy only sees final formatted text. It cannot preserve provider-internal content that was already capped or runtime artifacts that were never part of the result. This is acceptable for the first cut because the showcase is final-result spill, not early spill; tool-owned early spill remains deferred work. -Returning real paths from the local backend keeps v1 simple and matches proven agent-tool behavior, while the seam itself only promises an opaque locator plus retrieval hint so remote backends can return non-file locators. +Returning real paths keeps the local backend simple and matches proven agent-tool behavior, while the seam itself only promises an opaque locator plus retrieval hint so remote backends can return non-file locators. -The local-backend value proposition depends on the existing `read`/`grep` tools being able to inspect the returned local path, even when the spill directory is outside the session cwd. That holds today because the filesystem policy records observations and write guards but does not confine reads to the workspace. A future workspace-confinement policy must either allow local spill paths explicitly or use a non-file spill backend whose retrieval hint points at a supported reader. +The local-backend value proposition depends on the existing `read`/`grep` tools being able to inspect the returned local path, even when the spill directory is outside the session cwd. That holds because the filesystem policy records observations and write guards but does not confine reads to the workspace. A future workspace-confinement policy must either allow local spill paths explicitly or use a non-file spill backend whose retrieval hint points at a supported reader. **Snapshot gap.** No ACP snapshot scenario covers the transcript-visible `web_fetch` spill notice yet. The ACP snapshot harness replays keyless and cannot hit the live web, and a `web_fetch` spill requires a real over-cap HTTP body; a deterministic scenario would need a seeded loopback fetch target the replay tree does not currently wire (the examples do not load `tool-web` at all). The behavior is covered instead by the `dsh-tool-web` integration test against a loopback server. Closing the gap is follow-up work: wire `tool-web` + a seeded fetch target into the ACP example, then record a `web-fetch-spill` scenario. @@ -184,7 +184,7 @@ The policy can become too large if it starts owning tool-specific semantics. It ## Alternatives considered -**Require each tool to opt in with a retention declaration.** Rejected for v1: the goal is a default behavior similar to Claude Code's generic tool-result persistence. A single `maxInlineBytes` deployment knob is enough to prove the shape. +**Require each tool to opt in with a retention declaration.** Rejected: the goal is a default behavior similar to Claude Code's generic tool-result persistence. A single `maxInlineBytes` deployment knob is enough to prove the shape. **Make `tool-results` a broad tool-result platform.** Rejected: a broad package name invites retention policy, result replacement, preview wording, search, and early spill into one seam. The shared storage part is smaller: save text and return a locator plus retrieval hint. diff --git a/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md b/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md index 1a33878308..db297fa6be 100644 --- a/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md @@ -8,9 +8,9 @@ Status: implemented 工具输出需要有界的模型可见预览,但部分超大结果仍可能在之后有用。抓取的页面正文或冗长的工具响应不应完整占用下一次模型请求,但模型应能使用现有文件读取工具,在之后查看经过格式化的完整结果。 -这项改动之前的行为并不一致。`dsh-bash-local` 已经会在内存尾部溢出时,把完整 stdout/stderr 流写入私有的临时 spill 文件;普通文本工具结果则仍以内联形式返回,除非工具自行实现上限。[工具结果保留库](2026-07-06-tool-result-retention-library.md)负责预览机制,但不负责存储,也不负责把这些机制应用于最终工具结果的执行流水线策略。 +这项改动之前的行为并不一致。`dsh-bash-local` 已经会在内存尾部溢出时,把完整 stdout/stderr 流写入私有的临时 spill 文件;普通文本工具结果则仍以内联形式返回,除非工具自行实现上限。[工具结果保留库](2026-07-06-tool-result-retention-library.zh.md)负责预览机制,但不负责存储,也不负责把这些机制应用于最终工具结果的执行流水线策略。 -其形态与超时策略设计一致:工具作者声明规范值与 Native renderer(原生渲染器),由策略插件在渲染后的内容上执行部署默认的上下文预算。工具仍可在提供方采集上限处提前 spill;由工具负责的展示 spill 可以保留已完整采集的规范值,而只替换展示内容。[规范工具输出约定](2026-07-20-canonical-tool-output-contract.md)规定了这项区分。 +其形态与超时策略设计一致:工具作者声明规范值与 Native renderer(原生渲染器),由策略插件在渲染后的内容上执行部署默认的上下文预算。工具仍可在提供方采集上限处提前 spill;由工具负责的展示 spill 可以保留已完整采集的规范值,而只替换展示内容。[规范工具输出约定](2026-07-20-canonical-tool-output-contract.zh.md)规定了这项区分。 ## 决策 @@ -147,8 +147,8 @@ ctx.tools.register(defineTool({ ## 非目标 -- v1 不增加面向模型的 `artifact_read` 或 `artifact_search` 工具。 -- v1 不增加逐工具的保留配置。 +- 本决策不增加面向模型的 `artifact_read` 或 `artifact_search` 工具。 +- 本决策不增加逐工具的保留配置。 - 不增加面向模型的超时/截断参数。 - 不把 `read` 输出迁移到 spill 文件。 - 不取代 `web-fetch-http.maxBodyChars` 等提供方/资源上限。 @@ -174,9 +174,9 @@ ctx.tools.register(defineTool({ 默认策略只能看见最终格式化文本。它无法保留已经由提供方限制的内部内容,也无法保留从未成为结果一部分的运行时产物。第一版聚焦最终结果 spill 而不是提前 spill,因此可以接受这一限制;由工具负责的提前 spill 仍属于后续工作。 -本地后端返回真实路径,使 v1 保持简单并符合已经验证的 agent(智能体)工具行为;seam 本身只承诺一个不透明定位符加检索提示,所以远程后端可以返回非文件定位符。 +本地后端返回真实路径,使其保持简单并符合已经验证的 agent(智能体)工具行为;seam 本身只承诺一个不透明定位符加检索提示,所以远程后端可以返回非文件定位符。 -本地后端的价值取决于现有 `read`/`grep` 工具能否检查返回的本地路径,即使 spill 目录位于会话 cwd 之外。目前这一条件成立,因为文件系统策略会记录观察结果并设置写保护,但不会把读取限制在工作区内。未来的工作区限制策略必须显式允许本地 spill 路径,或改用检索提示指向受支持读取器的非文件 spill 后端。 +本地后端的价值取决于现有 `read`/`grep` 工具能否检查返回的本地路径,即使 spill 目录位于会话 cwd 之外。这一条件成立,因为文件系统策略会记录观察结果并设置写保护,但不会把读取限制在工作区内。未来的工作区限制策略必须显式允许本地 spill 路径,或改用检索提示指向受支持读取器的非文件 spill 后端。 **快照缺口。** 目前没有 ACP 快照场景覆盖 transcript(文本记录)可见的 `web_fetch` spill 提示。ACP 快照 harness 在无密钥环境中回放,无法访问实时 web,而 `web_fetch` spill 需要一个真实的超上限 HTTP 正文;确定性场景需要一个预置的 loopback fetch 目标,但当前回放树尚未接线(示例根本没有加载 `tool-web`)。该行为改由 `dsh-tool-web` 针对 loopback server 的集成测试覆盖。弥补该缺口属于后续工作:把 `tool-web` 和预置 fetch 目标接入 ACP 示例,然后录制 `web-fetch-spill` 场景。 @@ -184,7 +184,7 @@ ctx.tools.register(defineTool({ ## 考虑过的替代方案 -**要求每个工具通过保留声明选择加入。** v1 不予采纳,因为目标是实现类似 Claude Code 通用工具结果持久化的默认行为。只需一个 `maxInlineBytes` 部署配置项即可验证该形态。 +**要求每个工具通过保留声明选择加入。**不予采纳,因为目标是实现类似 Claude Code 通用工具结果持久化的默认行为。只需一个 `maxInlineBytes` 部署配置项即可验证该形态。 **把 `tool-results` 建成宽泛的工具结果平台。** 不予采纳:宽泛的包名会诱使系统把保留策略、结果替换、预览措辞、搜索和提前 spill 合并进一个 seam。可共享的存储部分更小:保存文本,并返回定位符与检索提示。 diff --git a/.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml index 0321d494a0..ec2f5318d6 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md 2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: 89c8917484c11057ff2e6b55e2311e83f724f7ed -2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: 96f7aa1b0ea4e44ca6318caca2c785d898b1d00d +2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: 48f954f2c4720894548ecab387965d3a3d1799b0 diff --git a/.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md b/.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md index 96f7aa1b0e..48f954f2c4 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md @@ -22,9 +22,9 @@ Compact-basic 会在每个拟议请求之前包装 `agent/pre-step`。在续步 ### 请求恢复只覆盖最终模型边界 -`agent/request-error` 表示来自最终适配器边界的终止失败。适配器选择、分发、iterator 构造与迭代抛出会在 agent loop(智能体循环)消费前成为终止 `error` 或 `aborted` finish;适配器直接发出的终止 finish 进入同一路径。提示词装配、请求 middleware、请求日志、结果处理、工具、step 监听器与清理仍属于普通失败。[LLM(大语言模型)流的终止失败](2026-07-29-terminal-llm-stream-failures.md)规定这一规范化边界。 +`agent/request-error` 表示来自最终适配器边界的终止失败。适配器选择、分发、iterator 构造与迭代抛出会在 agent loop(智能体循环)消费前成为终止 `error` 或 `aborted` finish;适配器直接发出的终止 finish 进入同一路径。提示词装配、请求 middleware、请求日志、结果处理、工具、step 监听器与清理仍属于普通失败。[LLM(大语言模型)流的终止失败](2026-07-29-terminal-llm-stream-failures.zh.md)规定这一规范化边界。 -恢复运行前,失败 step 已经关闭。负责处理的监听器修复持久状态、返回 `{ kind: 'retry' }`,并停止 waterfall(瀑布式事件)委托。循环随后关闭失败 turn,并从持久日志开启一个重试 turn,中间不发布空闲通知。重试策略与尝试计数由插件自己拥有;compaction-basic 在链路到达终态 `agent/settled` 时清除对应 agent 的溢出计数。两个 DeepSeek 适配器都把识别出的提供方上下文限制错误规范化为 `CONTEXT_WINDOW_EXCEEDED`。[重试动作决策](../simplification/2026-07-27-request-error-retry-action.md)规定这一返回边界。 +恢复运行前,失败 step 已经关闭。负责处理的监听器修复持久状态、返回 `{ kind: 'retry' }`,并停止 waterfall(瀑布式事件)委托。循环随后关闭失败 turn,并从持久日志开启一个重试 turn,中间不发布空闲通知。重试策略与尝试计数由插件自己拥有;compaction-basic 在链路到达终态 `agent/settled` 时清除对应 agent 的溢出计数。两个 DeepSeek 适配器都把识别出的提供方上下文限制错误规范化为 `CONTEXT_WINDOW_EXCEEDED`。[重试动作决策](../simplification/2026-07-27-request-error-retry-action.zh.md)规定这一返回边界。 如果取消发生在 assistant 工具调用已经持久化之后、所有调用完成分发之前,循环会为每个尚未分发的调用记录一对合成的 `tool/call` 与 aborted `tool/result`,随后进入正常中止路径。因此,表层不会仅因取消赢得竞态而留下孤立的持久工具调用。 @@ -58,4 +58,4 @@ Compact-basic 会在每个拟议请求之前包装 `agent/pre-step`。在续步 代价是在共享 pre-step waterfall 中执行压力工作,并需要适配器持续维护溢出分类。提供方措辞与启发式字符密度仍是维护风险。表层压缩依然无法修复仅信封本身就超出窗口的情况,也不能拆分不可分割的非工具节点,或修复不可剪枝的剩余部分仍然过大的工具单元。若可移除的文本工具结果是主要体积,可选剪枝器仍可修复原本不可分割的工具配对。 -[已领取 pre-step 生命周期](2026-07-31-claimed-pre-step-inbox-lifecycle.md)取代了本记录原先的 post-step 触发方式。服务拆分、独立 token meter、平衡范围约定、日志中记录的锁、摘要替换与唯一 `summarize()` 子类 hook 均保持不变。 +[已领取 pre-step 生命周期](2026-07-31-claimed-pre-step-inbox-lifecycle.zh.md)取代了本记录原先的 post-step 触发方式。服务拆分、独立 token meter、平衡范围约定、日志中记录的锁、摘要替换与唯一 `summarize()` 子类 hook 均保持不变。 diff --git a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml index 79be5c180b..68a1a8bbd0 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md -2026-07-10-single-file-executable-sdk-runtime-distribution.md: fa2f86893b730aa1ba020bd568d268ec8d9d6239 -2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: 509bec18edb9923dd4d60d4ecf30d4fbcd9cc6d5 +2026-07-10-single-file-executable-sdk-runtime-distribution.md: 24d95be871b2cb06e07706e6d5e0a628e30ef1bd +2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: eb9ba37c28e89a894ae7f5132592aa98fa56b977 diff --git a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md index fa2f86893b..24d95be871 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md +++ b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md @@ -23,12 +23,12 @@ The exe is packaged with the **`--sea` (enhanced SEA) mode** of [@yao-pkg/pkg](h Terminology reminder: pkg's `/snapshot` VFS has nothing to do with this repo's testing-system "snapshot" (ACP replay expected outputs, `$DSH_SNAPSHOT`); this document says "VFS" for the former. -### The serving interface is a plugin: the two packages sdk/server + examples/jsonrpc-demo +### The serving interface is a plugin: the two packages sdk/server + sdk/python-runtime The deterministic protocol implementation (`server.ts` / `transport.ts`) lands as two packages on the existing `acp/acp` + `examples/acp-demo` pattern — the serving surface is itself a plugin: - [`packages/sdk/server`](../../../../packages/sdk/server/README.md) (`@deepseek-ai/dsh-sdk-jsonrpc-server`): the pure protocol plugin; on apply it mounts `HarnessSdkJsonRpcServer` plus a line-delimited JSON-RPC transport on the process stdio, with disposal through `ctx.effect()`. Whether to serve is decided by `cordis.yml`; a yml that does not mount it is a legitimate process that does not serve. Protocol-level exit belongs to the plugin (after answering and flushing the `shutdown` response it disposes the root runtime so persistence drains, then `exit(0)`; an HMR-style unload only stops the service without exiting the process). -- [`packages/examples/jsonrpc-demo`](../../../../packages/examples/jsonrpc-demo/README.md) (`@deepseek-ai/dsh-sdk-jsonrpc-demo`): a thin app bin — `installFailLoud` + `loadEnv` + config discovery + `boot()` from [`dsh-app-boot`](../../../../packages/boot/app-boot/src/index.ts), done once boot completes; the server is brought up by the `dsh-sdk-jsonrpc-server` entry in the yml. Its only dependency is app-boot. Process-level exit belongs to the bin (stdin EOF/SIGTERM → dispose then 0, SIGINT → 130). +- [`packages/sdk/python-runtime`](../../../../packages/sdk/python-runtime/README.md) (`@deepseek-ai/dsh-sdk-python-runtime`): a private packaged entry — `installFailLoud` + `loadEnv` + config discovery + `boot()` from [`dsh-app-boot`](../../../../packages/boot/app-boot/src/index.ts), done once boot completes; the server is brought up by the `dsh-sdk-jsonrpc-server` entry in the yml. Its only dependency is app-boot. Process-level exit belongs to the packaged entry (stdin EOF/SIGTERM → dispose then 0, SIGINT → 130). Config discovery has two channels and fails loudly when both are missing: the `DSH_CORDIS_CONFIG` environment variable first (the SDK client convention), then an argv positional argument; no default path and no built-in fallback whatsoever — "the plugins actually booted are decided by an external cordis.yml" is a hard semantic. @@ -36,25 +36,27 @@ Config discovery has two channels and fails loudly when both are missing: the `D Inside the exe's VFS sits a **real package tree in build-artifact form** (each package's `lib/` plus a real `node_modules`). The packaged JSON-RPC entry supplies its installed harness base to app-boot's root Include: relative plugin specifiers resolve from the external configuration directory, while bare package names resolve from the VFS, so a configuration inside another Node project cannot shadow the packaged plugin set. The ordinary development bin leaves bare packages configuration-owned. Bare specifiers in the packaged entry resolve upward along `node_modules` from the entry's position inside the VFS and land inside the VFS naturally. The closed set needs no allowlist code — the set is whatever the VFS has installed, and importing a name outside the set fails. -The deploy root is [`python/sdk-runtime/package.json`](../../../../python/sdk-runtime/package.json) (`dsh-jsonrpc-agent-pkg`, a pnpm workspace member and a zero-code pure dependency manifest) — the unified source of truth for "which plugins the exe ships" and "what the Python runtime distributes". Adding a plugin to the exe = adding one dependency line to the manifest and repackaging. [`scripts/verify-runtime-closure.ts`](../../../../scripts/verify-runtime-closure.ts) traverses every workspace package covered by that manifest and requires every non-optional workspace peer at the runtime root, reporting the complete referencing-package → missing-peer chain; `pnpm run hygiene`, CI static, and the single-exe build run it before packaging. Deploy also packs by each package's `files`, so the shared chunks tsdown splits out must be covered by `files`. +The deploy root is [`python/sdk-runtime/package.json`](../../../../python/sdk-runtime/package.json) (`dsh-sdk-python-runtime-closure`, a pnpm workspace member and a zero-code pure dependency manifest) — the unified source of truth for "which plugins the exe ships" and "what the Python runtime distributes". Adding a plugin to the exe = adding one dependency line to the manifest and repackaging. [`scripts/verify-runtime-closure.ts`](../../../../scripts/verify-runtime-closure.ts) reads every shipped `apps/cli/config/agent-presets/*/agent.cordis.yml`, evaluates `disabled` conditions that compare `process.platform` for every target in `python/sdk-runtime/platforms.json`, and requires each active workspace plugin at the runtime root through an explicit `workspace:` dependency. It also traverses every workspace package covered by that manifest and requires every non-optional workspace peer, reporting the complete preset or referencing-package → missing-dependency chain; unknown platform conditions remain active so a plugin cannot be omitted by an unsupported expression. `pnpm run hygiene`, CI static, and the single-exe build run it before packaging. Deploy also packs by each package's `files`, so the shared chunks tsdown splits out must be covered by `files`. + +The deploy root includes `@deepseek-ai/dsh-mcp-client` as an explicitly supported custom-configuration plugin even though no shipped preset mounts it. An external config can therefore connect to user-supplied stdio and Streamable HTTP MCP servers and register their tools; the distribution does not carry those servers or extend the bridge to MCP Resources and Prompts. The executable and installed-wheel smokes start a temporary stdio server, discover its tool, and complete one model-requested call. ### Build pipeline and artifacts -[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → restore any direct workspace package that legacy deploy hoisted back under the source manifest's `node_modules`, omitting its package-local dependency tree and rejecting any remaining manifest gap → replace every staged dependency symlink with its target bytes, remove package-manager `.bin` links, and fail if any symlink remains → inject the pkg configuration (`bin` points at `node_modules/@deepseek-ai/dsh-sdk-jsonrpc-demo/lib/packaged-bin.js` inside the closure, `assets` is a full glob — dynamic import is invisible to pkg's static analysis, so everything must be packed in explicitly) → stage the target `node-pty` addon → one `pkg --sea` per target → the executables `dsh-jsonrpc-agent-pkg--` land in `dist-exe/` and are copied back into the runtime directory. Linux installs build `pty.node` from source; CI rebuilds that addon inside the matching manylinux 2.28 container before packaging, and the builder copies it from the root install into the staged closure because legacy deploy omits that side-effect directory. macOS uses its target prebuild and emits the required `-spawn-helper` beside the executable. CI treats these products as intermediate test inputs and retains their platform wheels. All four deploy flags are grounded in measurement: `--legacy` is the mandatory path with inject-workspace-packages off; hoisted gives pkg a stable single-instance layout that the explicit materialization pass makes symlink-free; disabling automatic peer installation prevents undeclared peers from expanding the closure; link-workspace-packages selects direct workspace dependencies. [`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) overrides the transitive `@deepseek-ai/cosmokit` and `@deepseek-ai/schemastery` semver requests to the pinned vendor sources so legacy deploy never resolves those unpublished names from a registry. +[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-sdk-python-runtime-closure deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → restore any direct workspace package that legacy deploy hoisted back under the source manifest's `node_modules`, omitting its package-local dependency tree and rejecting any remaining manifest gap → replace every staged dependency symlink with its target bytes, remove package-manager `.bin` links, and fail if any symlink remains → inject the pkg configuration (`bin` points at `node_modules/@deepseek-ai/dsh-sdk-python-runtime/lib/packaged-bin.js` inside the closure, `assets` is a full glob — dynamic import is invisible to pkg's static analysis, so everything must be packed in explicitly) → stage the target `node-pty` addon → one `pkg --sea` per target → the executables `dsh-jsonrpc-agent-pkg--` land in `dist-exe/` and are copied back into the runtime directory. Linux installs build `pty.node` from source; CI rebuilds that addon inside the matching manylinux 2.28 container before packaging, and the builder copies it from the root install into the staged closure because legacy deploy omits that side-effect directory. Every target copies its native `@vscode/ripgrep` binary beside the executable as the required `-rg` sidecar; pkg runtimes select that sidecar through `process.pkg`, while ordinary Node execution uses `@vscode/ripgrep` directly. macOS uses its target prebuild and also emits the required `-spawn-helper`. CI treats these products as intermediate test inputs and retains their platform wheels. All four deploy flags are grounded in measurement: `--legacy` is the mandatory path with inject-workspace-packages off; hoisted gives pkg a stable single-instance layout that the explicit materialization pass makes symlink-free; disabling automatic peer installation prevents undeclared peers from expanding the closure; link-workspace-packages selects direct workspace dependencies. [`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) overrides the transitive `@deepseek-ai/cosmokit` and `@deepseek-ai/schemastery` semver requests to the pinned vendor sources so legacy deploy never resolves those unpublished names from a registry. CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml), called for linux-x64 by the [required Python runtime pull-request validation](../testing/2026-08-12-required-python-runtime-pull-request-ci.md), triggered explicitly by `workflow_dispatch` or the `build-exe` label for selected targets, and called for all targets by the [public publication workflow](../process/2026-08-11-python-publication-workflow.md). Native builds run on linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64, with `~/.pkg-cache` cached, and pkg handles macOS ad-hoc signing. Each leg drives a mock SSE model through the SDK with the default config and a custom `cordis.yml`, drives the exe directly over NDJSON JSON-RPC, verifies the JSONL and final response, and installs release-shaped wheels into a clean venv without `runtime_bin`; Linux additionally inspects both the executable and native addon's GLIBC requirements and runs in a manylinux 2.28 container, while macOS verifies that the executable's deployment target fits the wheel tag. A full three-target run retains four artifacts, each containing one release file: the platform-independent SDK wheel and three native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts `python-v` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and three native runtime wheels, then a single serialized job checks and publishes all four to the project PyPI registry. Windows is a non-goal. ### Python SDK distribution: two carriers, exe for production, node for development -The Python SDK lives at [`python/`](../../../../python/README.md): `python/sdk` (the client) + `python/sdk-runtime` (the runtime carrier package). The runtime package's data directory holds the checked-in default `runtime/cordis.yml`, the build-injected platform exe and optional helper, and the build-injected `runtime/node/` closure tree. `resolve_bundled_launch_args()` automatic resolution **finds the exe only**; the node carrier is enabled only by an explicit `DSH_RUNTIME_MODE=node` (running `runtime/node/node_modules/@deepseek-ai/dsh-sdk-jsonrpc-demo/lib/packaged-bin.js`, requiring a system node ≥22.19), positioned as the development-verification channel for members of this repo, and does not enter wheel distributions. +The Python SDK lives at [`python/`](../../../../python/README.md): `python/sdk` (the client) + `python/sdk-runtime` (the runtime carrier package). The runtime package's data directory holds the checked-in default `runtime/cordis.yml`, the build-injected platform exe with its required `-rg` sidecar and optional macOS helper, and the build-injected `runtime/node/` closure tree. `resolve_bundled_launch_args()` automatic resolution **finds the exe only**; the node carrier is enabled only by an explicit `DSH_RUNTIME_MODE=node` (running `runtime/node/node_modules/@deepseek-ai/dsh-sdk-python-runtime/lib/packaged-bin.js`, requiring a system node ≥22.19), positioned as the development-verification channel for members of this repo, and does not enter wheel distributions. -[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) reads the authoritative `X.Y.Z` or prerelease version from the repository root `package.json`, converts prereleases to their PEP 440 spelling, and stages both packages at that wheel version, with `deepseek-harness-sdk` depending exactly on the matching `deepseek-harness-runtime-bin`. An optional `python-v` release tag is a consistency assertion and is rejected when it differs from the repository version; the source `pyproject.toml` development sentinel never determines a release version. Staging also carries the repository license into both wheels and the third-party notices into the bundled runtime wheel. The SDK is a `py3-none-any` wheel; each wheel-only runtime package contains one exe, and the macOS wheel also contains its architecture-matched helper. Runtime wheels use one of `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, or the conservative `py3-none-macosx_14_0_arm64` tag for the Node 24 executable's macOS 13.5 deployment target; the Hatch hook rejects sdists, universal tags, mixed-platform payloads, missing or extra helpers, and unsupported platforms. +[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) reads the authoritative `X.Y.Z` or prerelease version from the repository root `package.json`, converts prereleases to their PEP 440 spelling, and stages both packages at that wheel version, with `deepseek-harness-sdk` depending exactly on the matching `deepseek-harness-runtime-bin`. An optional `python-v` release tag is a consistency assertion and is rejected when it differs from the repository version; the source `pyproject.toml` development sentinel never determines a release version. Staging also carries the repository license into both wheels and the third-party notices into the bundled runtime wheel. The SDK is a `py3-none-any` wheel; each wheel-only runtime package contains one exe and its architecture-matched `-rg` sidecar, and the macOS wheel also contains its architecture-matched spawn helper. Runtime wheels use one of `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, or the conservative `py3-none-macosx_14_0_arm64` tag for the Node 24 executable's macOS 13.5 deployment target; the Hatch hook rejects sdists, universal tags, mixed-platform payloads, missing or extra sidecars, and unsupported platforms. The exe's "must be explicitly configured" hard semantic is unchanged; the zero-config experience is restored by the wrapper: when the caller gave no `cordis`, named no explicit runtime, and the environment has no `DSH_CORDIS_CONFIG`, the client explicitly injects the checked-in default `cordis.yml` (agent-core + preloaded llm-deepseek + JSONL persistence + bash-local + the `dsh-sdk-jsonrpc-server` serving entry, with `!!js` environment-variable fallbacks) via `DSH_CORDIS_CONFIG`. ### Naming lineage -`@deepseek-ai/dsh-sdk-jsonrpc-demo` (the package) → `dsh-jsonrpc-agent` (the bin) → `dsh-jsonrpc-agent-pkg` (the closure manifest; no scope prefix, deliberately sidestepping the constraints' package-shape rules for `@deepseek-ai/dsh-*`) → `dsh-jsonrpc-agent-pkg--` (the exe artifacts). The wire `serverInfo.name` stays `deepseek-harness-sdk-runtime` (a protocol-stable value); the Python distribution names are `deepseek-harness-sdk` / `deepseek-harness-runtime-bin`, while the import modules remain `deepseek_harness` / `deepseek_harness_runtime`. +`@deepseek-ai/dsh-sdk-python-runtime` (the private carrier) → `dsh-sdk-python-runtime-closure` (the deploy manifest; no scope prefix, so it is not a dsh release package) → `dsh-jsonrpc-agent-pkg--` (the exe artifacts). The wire `serverInfo.name` stays `deepseek-harness-sdk-runtime` (a protocol-stable value); the Python distribution names are `deepseek-harness-sdk` / `deepseek-harness-runtime-bin`, while the import modules remain `deepseek_harness` / `deepseek_harness_runtime`. ## Disposition of worker-style plugins @@ -62,7 +64,7 @@ The exe's "must be explicitly configured" hard semantic is unchanged; the zero-c ## Testing -The verification surface has three tiers. Mechanism tier: the measured conclusions for the `--sea` chain are embedded in the Decision sections (ESM dynamic import inside the VFS, single cordis instance, fail-loud config chain, `node:sqlite`, macOS ad-hoc signing runs). SDK tier: the complete keyless pytest suite covers the client protocol against a fake runtime peer, subprocess cleanup, absolute cwd propagation, dual-carrier launch, and carrier resolution; root CI runs it on Python 3.10. End-to-end tier: every platform build completes a turn against a mock endpoint through the default SDK path, a custom config, the checked-in standalone minimal composition, and the direct binary protocol, with final text and JSONL checked. The minimal run asserts its exact system prompt and two-tool catalog, retains Bash state across calls, and invokes the editor. The custom config additionally drives `run_code` and a zero-agent `workflow` through their real worker files inside the packaged VFS. The same build leg runs a committed executable-specific snapshot through the Python SDK: a keyless scripted model mounts a Cordis plugin that registers a tool, invokes that tool from `run_code`, runs a direct spawn subagent and a workflow that starts a second spawn child, then unmounts the plugin. The fixture explicitly disables its unused bundled Bash and local skill discovery so its tool set does not depend on repository-external state, and the comparison normalizes opaque message, agent, workflow-run, and session IDs across the SDK result and notification stream plus the parent and two child JSONL logs. This harness stays separate from ACP's `pnpm run test:snapshot` because the protocols and build artifacts differ. The platform wheel is then installed in a clean venv and run without `runtime_bin`. +The verification surface has three tiers. Mechanism tier: the measured conclusions for the `--sea` chain are embedded in the Decision sections (ESM dynamic import inside the VFS, single cordis instance, fail-loud config chain, `node:sqlite`, macOS ad-hoc signing runs). SDK tier: the complete keyless pytest suite covers the client protocol against a fake runtime peer, subprocess cleanup, absolute cwd propagation, dual-carrier launch, and carrier resolution; root CI runs it on Python 3.10. End-to-end tier: every platform build completes a turn against a mock endpoint through the default SDK path, a custom config, the checked-in standalone minimal composition, and the direct binary protocol, with final text and JSONL checked. The minimal run asserts its exact system prompt and two-tool catalog, retains Bash state across calls, and invokes the editor. The custom config additionally drives `run_code` and a zero-agent `workflow` through their real worker files inside the packaged VFS. The filesystem-search scenario requires the model to call both `glob` and `grep` through the target-native `-rg` sidecar. The MCP scenario starts a temporary external stdio server, deliberately delays its initial `tools/list` response, then immediately starts the first SDK prompt; the prompt must see and call the discovered tool, proving that `initialize` is a real Loader-settlement readiness boundary rather than a timing sleep. The same build leg runs a committed executable-specific snapshot through the Python SDK: a keyless scripted model mounts a Cordis plugin that registers a tool, invokes that tool from `run_code`, runs a direct spawn subagent and a workflow that starts a second spawn child, then unmounts the plugin. The fixture explicitly disables its unused bundled Bash and local skill discovery so its tool set does not depend on repository-external state, and the comparison normalizes opaque message, agent, workflow-run, and session IDs across the SDK result and notification stream plus the parent and two child JSONL logs. This harness stays separate from ACP's `pnpm run test:snapshot` because the protocols and build artifacts differ. The platform wheel is then installed in a clean venv and run without `runtime_bin`. Manual-driving caveat: the bin treats stdin EOF as "the client is gone" and disposes immediately, so a short-lived pipe aborts an in-flight turn — pipe-driven runs must keep stdin open until the turn ends. diff --git a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md index 509bec18ed..eb9ba37c28 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md @@ -23,12 +23,12 @@ exe 使用 [@yao-pkg/pkg](https://github.com/yao-pkg/pkg)(vercel/pkg 归档后 术语提醒:pkg 的 `/snapshot` VFS 与本仓库测试体系的「快照」(ACP(Agent Client Protocol)回放预期输出、`$DSH_SNAPSHOT`)无关,本文用「VFS」指前者。 -### 对外服务接口也是插件:sdk/server + examples/jsonrpc-demo 两个包 +### 对外服务接口也是插件:sdk/server + sdk/python-runtime 两个包 确定性协议实现(`server.ts` / `transport.ts`)按 `acp/acp` + `examples/acp-demo` 的既有模式落为两包——对外服务接口本身也是插件: -- [`packages/sdk/server`](../../../../packages/sdk/server/README.md)(`@deepseek-ai/dsh-sdk-jsonrpc-server`):纯协议插件;执行 `apply` 时,在进程 stdio 上挂载 `HarnessSdkJsonRpcServer` 与按行分隔的 JSON-RPC 传输层,资源释放走 `ctx.effect()`。是否提供服务由 `cordis.yml` 决定;未挂载该插件的配置会启动一个不提供此服务的合法进程。协议级退出归插件所有(应答并确保 `shutdown` 响应发送完毕后,对根运行时执行 dispose(资源释放),让待处理的持久化操作完成,再调用 `exit(0)`;HMR(热模块替换)式卸载只停止服务,不退出进程)。 -- [`packages/examples/jsonrpc-demo`](../../../../packages/examples/jsonrpc-demo/README.md)(`@deepseek-ai/dsh-sdk-jsonrpc-demo`):轻量应用入口——`installFailLoud` + `loadEnv` + 配置发现 + [`dsh-app-boot`](../../../../packages/boot/app-boot/src/index.ts) 的 `boot()`;`boot()` 完成后入口即完成,服务器由 `cordis.yml` 中的 `dsh-sdk-jsonrpc-server` 条目启动。它只依赖 `app-boot`。进程级退出归 `bin` 所有(stdin EOF/SIGTERM → dispose 后返回 0,SIGINT → 130)。 +- [`packages/sdk/server`](../../../../packages/sdk/server/README.zh.md)(`@deepseek-ai/dsh-sdk-jsonrpc-server`):纯协议插件;执行 `apply` 时,在进程 stdio 上挂载 `HarnessSdkJsonRpcServer` 与按行分隔的 JSON-RPC 传输层,资源释放走 `ctx.effect()`。是否提供服务由 `cordis.yml` 决定;未挂载该插件的配置会启动一个不提供此服务的合法进程。协议级退出归插件所有(应答并确保 `shutdown` 响应发送完毕后,对根运行时执行 dispose(资源释放),让待处理的持久化操作完成,再调用 `exit(0)`;HMR(热模块替换)式卸载只停止服务,不退出进程)。 +- [`packages/sdk/python-runtime`](../../../../packages/sdk/python-runtime/README.zh.md)(`@deepseek-ai/dsh-sdk-python-runtime`):私有打包入口——`installFailLoud` + `loadEnv` + 配置发现 + [`dsh-app-boot`](../../../../packages/boot/app-boot/src/index.ts) 的 `boot()`;`boot()` 完成后入口即完成,服务器由 `cordis.yml` 中的 `dsh-sdk-jsonrpc-server` 条目启动。它只依赖 `app-boot`。进程级退出归打包入口所有(stdin EOF/SIGTERM → dispose 后返回 0,SIGINT → 130)。 配置发现有两个通道,均缺失时立即报错:优先使用 `DSH_CORDIS_CONFIG` 环境变量(SDK 客户端约定),其次使用 argv 位置参数;没有默认路径或内置回退——「实际启动的插件由外部 `cordis.yml` 决定」是硬语义。 @@ -36,25 +36,27 @@ exe 使用 [@yao-pkg/pkg](https://github.com/yao-pkg/pkg)(vercel/pkg 归档后 exe 的 VFS 内是**构建产物形态的真实包树**(各包的 `lib/` + 真实 `node_modules`)。打包专用 JSON-RPC 入口会向 app-boot 的根 Include 提供自身已安装 harness 的基准位置:相对插件说明符从外部配置目录解析,裸包名则从 VFS 解析,因此位于另一个 Node 项目内的配置无法遮蔽已打包的插件集合。普通开发 bin 仍由配置项目提供裸包。打包入口中的裸包名从该入口在 VFS 内的位置沿 `node_modules` 向上解析,自然落在 VFS 内。封闭集不需要白名单代码——VFS 中安装了什么,集合中就有什么;`import()` 集合外的名称会失败。 -部署根目录是 [`python/sdk-runtime/package.json`](../../../../python/sdk-runtime/package.json)(`dsh-jsonrpc-agent-pkg`,pnpm 工作区成员、零代码纯依赖 manifest),也是「exe 安装哪些插件」与「Python 运行时分发什么」的统一真源。向 exe 添加插件,就是在 manifest 中增加一行依赖后重新打包。[`scripts/verify-runtime-closure.ts`](../../../../scripts/verify-runtime-closure.ts) 遍历该 manifest 覆盖的全部工作区包,要求每个非可选的工作区对等依赖(peer dependency)都显式列在运行时根目录,并报告“引用包 → 缺失对等依赖”的完整链路;`pnpm run hygiene`、CI 静态检查与 single-exe 构建都会在打包前运行该门禁。部署还会依据各包的 `files` 字段打包,因此 tsdown 拆出的共享分片必须被 `files` 覆盖。 +部署根目录是 [`python/sdk-runtime/package.json`](../../../../python/sdk-runtime/package.json)(`dsh-sdk-python-runtime-closure`,pnpm 工作区成员、零代码纯依赖 manifest),也是「exe 安装哪些插件」与「Python 运行时分发什么」的统一真源。向 exe 添加插件,就是在 manifest 中增加一行依赖后重新打包。[`scripts/verify-runtime-closure.ts`](../../../../scripts/verify-runtime-closure.ts) 读取每个已发布的 `apps/cli/config/agent-presets/*/agent.cordis.yml`,针对 `python/sdk-runtime/platforms.json` 中的每个目标解析比较 `process.platform` 的 `disabled` 条件,并要求该目标启用的每个工作区插件都通过显式的 `workspace:` 依赖列在运行时根目录。它还遍历该 manifest 覆盖的全部工作区包,要求每个非可选的工作区对等依赖(peer dependency)都显式列出,并报告“preset 或引用包 → 缺失依赖”的完整链路;无法识别的平台条件会保持启用,避免因不支持的表达式遗漏插件。`pnpm run hygiene`、CI 静态检查与 single-exe 构建都会在打包前运行该门禁。部署还会依据各包的 `files` 字段打包,因此 tsdown 拆出的共享分片必须被 `files` 覆盖。 + +部署根目录显式包含 `@deepseek-ai/dsh-mcp-client`,将其作为自定义配置可用的插件,即使随附 preset 均未挂载该插件。外部配置因此可以连接由用户提供的 stdio 与 Streamable HTTP MCP server 并注册其工具;分发物不包含这些 server,也不将桥接范围扩展到 MCP Resources 和 Prompts。可执行程序与已安装 wheel 包的冒烟测试会启动临时 stdio server,发现其工具,并完成一次由模型请求的调用。 ### 构建流水线与产物 -[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts):运行时闭包校验 → `pnpm run build` →(清空后)`pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → 恢复被 legacy deploy 提升回源 manifest 的 `node_modules` 下的任何直接工作区包,同时省略其包内依赖树,并拒绝剩余的 manifest 缺口 → 将暂存依赖中的每个符号链接替换为目标文件内容,删除包管理器的 `.bin` 链接,并在仍有任何符号链接时失败 → 注入 pkg 配置(`bin` 指向闭包内的 `node_modules/@deepseek-ai/dsh-sdk-jsonrpc-demo/lib/packaged-bin.js`;`assets` 使用全量 glob,因为动态 `import()` 对 pkg 静态分析不可见,必须显式打入全部内容)→ 暂存目标平台的 `node-pty` addon → 每个构建目标调用一次 `pkg --sea` → 可执行文件 `dsh-jsonrpc-agent-pkg--` 写入 `dist-exe/`,并拷回运行时目录。Linux 安装会从源码构建 `pty.node`;CI 会在打包前进入匹配架构的 manylinux 2.28 容器重新构建该 addon,而 `--legacy` 部署会省略这一副作用目录,因此构建器会把它从根安装目录复制到暂存闭包。macOS 使用对应目标的预构建产物,并在可执行文件旁生成所需的 `-spawn-helper`。CI 将这些产物作为测试中间输入,只保留对应平台的 wheel 包。四个部署标志都有实测依据:未启用 `inject-workspace-packages` 时必须使用 `--legacy`;`hoisted` 为 pkg 提供稳定的单实例布局,再由显式物化步骤消除符号链接;关闭对等依赖自动安装可防止未声明的对等依赖扩大闭包;`link-workspace-packages` 选择直接工作区依赖。[`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) 将传递的 `@deepseek-ai/cosmokit` 与 `@deepseek-ai/schemastery` semver 请求覆盖到固定的 vendor 源码,使 legacy deploy 不会从注册表解析这些未发布名称。 +[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts):运行时闭包校验 → `pnpm run build` →(清空后)`pnpm --filter dsh-sdk-python-runtime-closure deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → 恢复被 legacy deploy 提升回源 manifest 的 `node_modules` 下的任何直接工作区包,同时省略其包内依赖树,并拒绝剩余的 manifest 缺口 → 将暂存依赖中的每个符号链接替换为目标文件内容,删除包管理器的 `.bin` 链接,并在仍有任何符号链接时失败 → 注入 pkg 配置(`bin` 指向闭包内的 `node_modules/@deepseek-ai/dsh-sdk-python-runtime/lib/packaged-bin.js`;`assets` 使用全量 glob,因为动态 `import()` 对 pkg 静态分析不可见,必须显式打入全部内容)→ 暂存目标平台的 `node-pty` addon → 每个构建目标调用一次 `pkg --sea` → 可执行文件 `dsh-jsonrpc-agent-pkg--` 写入 `dist-exe/`,并拷回运行时目录。Linux 安装会从源码构建 `pty.node`;CI 会在打包前进入匹配架构的 manylinux 2.28 容器重新构建该 addon,而 `--legacy` 部署会省略这一副作用目录,因此构建器会把它从根安装目录复制到暂存闭包。每个目标都会把对应的原生 `@vscode/ripgrep` 二进制复制到可执行文件旁,作为必需的 `-rg` 伴随文件;pkg 运行时通过 `process.pkg` 选择该伴随文件,普通 Node 执行则直接使用 `@vscode/ripgrep`。macOS 使用对应目标的预构建产物,并额外生成所需的 `-spawn-helper`。CI 将这些产物作为测试中间输入,只保留对应平台的 wheel 包。四个部署标志都有实测依据:未启用 `inject-workspace-packages` 时必须使用 `--legacy`;`hoisted` 为 pkg 提供稳定的单实例布局,再由显式物化步骤消除符号链接;关闭对等依赖自动安装可防止未声明的对等依赖扩大闭包;`link-workspace-packages` 选择直接工作区依赖。[`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) 将传递的 `@deepseek-ai/cosmokit` 与 `@deepseek-ai/schemastery` semver 请求覆盖到固定的 vendor 源码,使 legacy deploy 不会从注册表解析这些未发布名称。 -CI 使用 [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml):[必需的 Python 运行时拉取请求验证](../testing/2026-08-12-required-python-runtime-pull-request-ci.md)调用它构建 linux-x64,手动派发 `workflow_dispatch` 或 PR(Pull Request)的 `build-exe` 标签可以显式选择构建目标,[公开发布工作流](../process/2026-08-11-python-publication-workflow.md)则调用它构建全部目标。linux-x64、linux-arm64(`ubuntu-24.04-arm`)和 macos-arm64 三个平台分别进行原生构建,并缓存 `~/.pkg-cache`;macOS 的 ad-hoc 签名由 pkg 处理。每个平台都使用 mock SSE(Server-Sent Events)模型,分别通过默认配置和自定义 `cordis.yml` 驱动 SDK,再通过 NDJSON JSON-RPC 直接驱动 exe,校验 JSONL 与最终响应;最后把发布形态的 wheel 包安装到干净的 venv 中,并在不传 `runtime_bin` 的情况下运行。Linux 还会检查可执行文件和原生 addon 各自的 GLIBC 依赖,并在 manylinux 2.28 容器中运行;macOS 则验证可执行文件的部署目标符合 wheel 包标签。完整构建三个目标时保留 4 个产物,每个产物只含一个发布文件:平台无关的 SDK wheel 包与 3 个原生运行时 wheel 包;手动选择部分目标时保留 SDK wheel 与所选运行时 wheel。裸 exe 与源码包只作为测试中间输入。[`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) 只接受版本与根目录 `package.json` 匹配的 `python-v` 标签流水线,构建一个 SDK wheel 包和 3 个原生运行时 wheel 包,再由单个串行任务校验并将这 4 个文件发布到项目的 PyPI 注册表。Windows 不在目标范围内。 +CI 使用 [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml):[必需的 Python 运行时拉取请求验证](../testing/2026-08-12-required-python-runtime-pull-request-ci.zh.md)调用它构建 linux-x64,手动派发 `workflow_dispatch` 或 PR(Pull Request)的 `build-exe` 标签可以显式选择构建目标,[公开发布工作流](../process/2026-08-11-python-publication-workflow.zh.md)则调用它构建全部目标。linux-x64、linux-arm64(`ubuntu-24.04-arm`)和 macos-arm64 三个平台分别进行原生构建,并缓存 `~/.pkg-cache`;macOS 的 ad-hoc 签名由 pkg 处理。每个平台都使用 mock SSE(Server-Sent Events)模型,分别通过默认配置和自定义 `cordis.yml` 驱动 SDK,再通过 NDJSON JSON-RPC 直接驱动 exe,校验 JSONL 与最终响应;最后把发布形态的 wheel 包安装到干净的 venv 中,并在不传 `runtime_bin` 的情况下运行。Linux 还会检查可执行文件和原生 addon 各自的 GLIBC 依赖,并在 manylinux 2.28 容器中运行;macOS 则验证可执行文件的部署目标符合 wheel 包标签。完整构建三个目标时保留 4 个产物,每个产物只含一个发布文件:平台无关的 SDK wheel 包与 3 个原生运行时 wheel 包;手动选择部分目标时保留 SDK wheel 与所选运行时 wheel。裸 exe 与源码包只作为测试中间输入。[`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) 只接受版本与根目录 `package.json` 匹配的 `python-v` 标签流水线,构建一个 SDK wheel 包和 3 个原生运行时 wheel 包,再由单个串行任务校验并将这 4 个文件发布到项目的 PyPI 注册表。Windows 不在目标范围内。 ### Python SDK 分发:双载体,exe 用于生产,`node` 用于开发 -Python SDK 位于 [`python/`](../../../../python/README.md):`python/sdk` 是客户端,`python/sdk-runtime` 是运行时载体包。运行时包的数据目录包含检入的默认 `runtime/cordis.yml`、构建注入的平台 exe 与可选 helper,以及构建注入的 `runtime/node/` 闭包树。`resolve_bundled_launch_args()` 的自动解析**只查找 exe**;`node` 载体仅在显式设置 `DSH_RUNTIME_MODE=node` 时启用(运行 `runtime/node/node_modules/@deepseek-ai/dsh-sdk-jsonrpc-demo/lib/packaged-bin.js`,需要系统 Node ≥22.19),定位为本仓库成员的开发验证通道,不随 wheel 包分发。 +Python SDK 位于 [`python/`](../../../../python/README.zh.md):`python/sdk` 是客户端,`python/sdk-runtime` 是运行时载体包。运行时包的数据目录包含检入的默认 `runtime/cordis.yml`、构建注入的平台 exe 及其必需的 `-rg` 伴随文件和可选的 macOS helper,以及构建注入的 `runtime/node/` 闭包树。`resolve_bundled_launch_args()` 的自动解析**只查找 exe**;`node` 载体仅在显式设置 `DSH_RUNTIME_MODE=node` 时启用(运行 `runtime/node/node_modules/@deepseek-ai/dsh-sdk-python-runtime/lib/packaged-bin.js`,需要系统 Node ≥22.19),定位为本仓库成员的开发验证通道,不随 wheel 包分发。 -[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) 从仓库根目录的 `package.json` 读取权威的 `X.Y.Z` 或预发布版本,把预发布版本转换为 PEP 440 写法,并以该 wheel 包版本暂存两个包,让 `deepseek-harness-sdk` 精确依赖匹配版本的 `deepseek-harness-runtime-bin`。可选的 `python-v` 发布标签只是一项一致性断言,与仓库版本不同时会被拒绝;源码 `pyproject.toml` 中的开发占位版本从不决定发布版本。暂存过程还会把仓库许可证放入两个 wheel 包,并把第三方声明放入内置运行时 wheel 包。SDK 是 `py3-none-any` wheel 包;每个只提供 wheel 包的运行时包都包含一个 exe,macOS wheel 包还包含与其架构匹配的 helper。运行时 wheel 包使用 `py3-none-manylinux_2_28_x86_64`、`py3-none-manylinux_2_28_aarch64`,或针对 Node 24 可执行文件 macOS 13.5 部署目标而保守选择的 `py3-none-macosx_14_0_arm64` 标签;Hatch 钩子拒绝 sdist、通用标签、混合平台载荷、helper 缺失或多余,以及不支持的平台。 +[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) 从仓库根目录的 `package.json` 读取权威的 `X.Y.Z` 或预发布版本,把预发布版本转换为 PEP 440 写法,并以该 wheel 包版本暂存两个包,让 `deepseek-harness-sdk` 精确依赖匹配版本的 `deepseek-harness-runtime-bin`。可选的 `python-v` 发布标签只是一项一致性断言,与仓库版本不同时会被拒绝;源码 `pyproject.toml` 中的开发占位版本从不决定发布版本。暂存过程还会把仓库许可证放入两个 wheel 包,并把第三方声明放入内置运行时 wheel 包。SDK 是 `py3-none-any` wheel 包;每个只提供 wheel 包的运行时包都包含一个 exe 及其架构匹配的 `-rg` 伴随文件,macOS wheel 包还包含与其架构匹配的 spawn helper。运行时 wheel 包使用 `py3-none-manylinux_2_28_x86_64`、`py3-none-manylinux_2_28_aarch64`,或针对 Node 24 可执行文件 macOS 13.5 部署目标而保守选择的 `py3-none-macosx_14_0_arm64` 标签;Hatch 钩子拒绝 sdist、通用标签、混合平台载荷、伴随文件缺失或多余,以及不支持的平台。 exe「必须显式配置」的硬语义不变;零配置体验由包装层恢复:调用方没有提供 `cordis`、没有显式指定运行时,且环境中没有 `DSH_CORDIS_CONFIG` 时,客户端将检入的默认 `cordis.yml`(`agent-core` + 预载的 `llm-deepseek` + JSONL 持久化 + `bash-local` + `dsh-sdk-jsonrpc-server` 对外服务条目,并通过 `!!js` 使用环境变量兜底)显式注入 `DSH_CORDIS_CONFIG`。 ### 命名血统 -`@deepseek-ai/dsh-sdk-jsonrpc-demo`(包)→ `dsh-jsonrpc-agent`(`bin`)→ `dsh-jsonrpc-agent-pkg`(闭包 manifest;没有作用域前缀,刻意避开 `constraints` 对 `@deepseek-ai/dsh-*` 的包形状规则)→ `dsh-jsonrpc-agent-pkg--`(exe 产物)。协议字段 `serverInfo.name` 保持为 `deepseek-harness-sdk-runtime`(协议稳定值);Python 分发包名为 `deepseek-harness-sdk` / `deepseek-harness-runtime-bin`,导入模块名仍为 `deepseek_harness` / `deepseek_harness_runtime`。 +`@deepseek-ai/dsh-sdk-python-runtime`(私有载体)→ `dsh-sdk-python-runtime-closure`(部署 manifest;没有作用域前缀,因此不属于 dsh 发布包)→ `dsh-jsonrpc-agent-pkg--`(exe 产物)。协议字段 `serverInfo.name` 保持为 `deepseek-harness-sdk-runtime`(协议稳定值);Python 分发包名为 `deepseek-harness-sdk` / `deepseek-harness-runtime-bin`,导入模块名仍为 `deepseek_harness` / `deepseek_harness_runtime`。 ## 工作线程插件 @@ -62,7 +64,7 @@ exe 内支持 `dsh-workflow-worker-thread` 与 `dsh-code-runtime-worker-thread` ## 测试 -验证面分三层。机制层:`--sea` 链路的实测结论内嵌在「决策」各节(VFS 内 ESM 动态 `import()`、单一 Cordis 实例、明确报错的配置链路、`node:sqlite`、macOS ad-hoc 签名可运行)。SDK 层:完整的无密钥 pytest 套件以 mock 运行时对端覆盖客户端协议、子进程清理、绝对 `cwd` 传递、双载体启动与载体解析;根 CI 在 Python 3.10 上运行全部用例。端到端层:每个平台构建都通过默认 SDK 路径、自定义配置、仓库内置的独立 minimal 组合和直接二进制协议,对 mock 端点完成一个轮次,并校验最终文本与 JSONL。minimal 运行会断言其精确系统提示词与双工具目录,跨调用保留 Bash 状态,并调用编辑器。自定义配置还会通过打包进 VFS 的真实工作线程文件执行 `run_code` 和不启动 agent 的 `workflow`。同一构建任务还会经 Python SDK 运行一组检入的 exe 专用快照:无密钥脚本化模型挂载一个会注册工具的 Cordis 插件,从 `run_code` 调用该工具,运行一个直接 spawn 的 subagent 和一个会通过 spawn 启动第二个 subagent 的工作流,随后卸载该插件。该 fixture(测试前置数据)会显式禁用组合包中未使用的 Bash 和本地 skill(技能)发现,使其工具集不依赖仓库外部状态;比较时会规范化 SDK 结果与通知流,以及父会话和两个子会话 JSONL 日志中不透明的消息、agent、工作流运行与会话 ID。该 harness 与 ACP 的 `pnpm run test:snapshot` 保持独立,因为二者的协议和构建产物不同。随后把平台 wheel 包安装进干净的 venv,并在不传 `runtime_bin` 的情况下运行。 +验证面分三层。机制层:`--sea` 链路的实测结论内嵌在「决策」各节(VFS 内 ESM 动态 `import()`、单一 Cordis 实例、明确报错的配置链路、`node:sqlite`、macOS ad-hoc 签名可运行)。SDK 层:完整的无密钥 pytest 套件以 mock 运行时对端覆盖客户端协议、子进程清理、绝对 `cwd` 传递、双载体启动与载体解析;根 CI 在 Python 3.10 上运行全部用例。端到端层:每个平台构建都通过默认 SDK 路径、自定义配置、仓库内置的独立 minimal 组合和直接二进制协议,对 mock 端点完成一个轮次,并校验最终文本与 JSONL。minimal 运行会断言其精确系统提示词与双工具目录,跨调用保留 Bash 状态,并调用编辑器。自定义配置还会通过打包进 VFS 的真实工作线程文件执行 `run_code` 和不启动 agent 的 `workflow`。文件系统搜索场景要求模型通过目标平台的 `-rg` 伴随文件调用 `glob` 与 `grep`。MCP 场景会启动临时外部 stdio server,刻意延迟首次 `tools/list` 响应,随后立即启动第一个 SDK 提示词;该提示词必须看到并调用已发现的工具,从而证明 `initialize` 是真正以 Loader 插件树完全稳定为准的就绪边界,而不是依赖定时 sleep。同一构建任务还会经 Python SDK 运行一组检入的 exe 专用快照:无密钥脚本化模型挂载一个会注册工具的 Cordis 插件,从 `run_code` 调用该工具,运行一个直接 spawn 的 subagent 和一个会通过 spawn 启动第二个 subagent 的工作流,随后卸载该插件。该 fixture(测试前置数据)会显式禁用组合包中未使用的 Bash 和本地 skill(技能)发现,使其工具集不依赖仓库外部状态;比较时会规范化 SDK 结果与通知流,以及父会话和两个子会话 JSONL 日志中不透明的消息、agent、工作流运行与会话 ID。该 harness 与 ACP 的 `pnpm run test:snapshot` 保持独立,因为二者的协议和构建产物不同。随后把平台 wheel 包安装进干净的 venv,并在不传 `runtime_bin` 的情况下运行。 手工驱动注意:`bin` 将 stdin EOF 视为「客户端已离开」并立即 dispose,生命周期较短的管道会中止进行中的轮次——管道驱动必须保持 stdin 打开,直到轮次结束。 diff --git a/.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.i18n.yaml index 02aeec5ad6..7d8d2db1cf 100644 --- a/.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.md 2026-07-12-agent-scope-runtime-design.md: 9d70b8048b1d2bb50290d34158d9deb329d5e15e -2026-07-12-agent-scope-runtime-design.zh.md: a505f5337559294795c1080937c6079cee235baa +2026-07-12-agent-scope-runtime-design.zh.md: 278bcede47fee9f67d3d2d2d7135e5357c120161 diff --git a/.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.zh.md b/.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.zh.md index a505f53375..278bcede47 100644 --- a/.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -[agent(智能体)作用域约定](2026-07-08-agent-scope-contexts.md)对贡献者而言很简单:通过 `agent.ctx` 注册,解析出一个全局加单 agent 的视图,仅在 setup 完成后发布,并保持作用域直到工作停止。运行时必须在协作式插件框架、异步创建、可重入监听器、持久化会话提交以及 worker 或进程故障等场景下维护这份约定。 +[agent(智能体)作用域约定](2026-07-08-agent-scope-contexts.zh.md)对贡献者而言很简单:通过 `agent.ctx` 注册,解析出一个全局加单 agent 的视图,仅在 setup 完成后发布,并保持作用域直到工作停止。运行时必须在协作式插件框架、异步创建、可重入监听器、持久化会话提交以及 worker 或进程故障等场景下维护这份约定。 主要的设计风险是为每个竞态条件引入第二套机制。独立的预留、就绪哨兵、取消中继、快照层和保护注册表可能镜像同一个事实,直到没有读者能分辨哪个才是权威的。这些机制还会诱使运行时把可信的类型化调用当作敌对的序列化边界来处理。 @@ -30,7 +30,7 @@ Status: implemented 本 Agent Note 余下部分按依赖顺序展开这些选择:Cordis 机制、作用域路由、创建与会话提交、工具与提示词、subagent 与工作流,最后是可执行检查。 -[7 月 8 日 Agent Note](2026-07-08-agent-scope-contexts.md) 仍然是贡献者约定。独立的 [subagent 组合控制 Agent Note](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.md) 拥有 `persona`、`toolFilter` 和 `maxDepth`;本文仅讨论它们的 setup 如何融入生命周期。 +[7 月 8 日 Agent Note](2026-07-08-agent-scope-contexts.zh.md) 仍然是贡献者约定。独立的 [subagent 组合控制 Agent Note](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.zh.md) 拥有 `persona`、`toolFilter` 和 `maxDepth`;本文仅讨论它们的 setup 如何融入生命周期。 ## Cordis 模型:上下文、fiber、effect、receiver 与 waterfall @@ -58,6 +58,8 @@ Cordis 使用 dispatch receiver(`this`)过滤监听器,而 harness 的监 Cordis waterfall 是中间件风格的 dispatch。每个监听器接收 `next()`:调用它则委托给剩余监听器和基础操作,不调用则短路或替换下游结果。Waterfall 驱动提示词组装和工具策略;普通 emit 事件同步通知,parallel 事件等待所有监听器但没有否决结果。 + + ## 作用域路由:一个不透明键选择一层 scope 包实现了 Cordis 路由所需的最小对象。其载体仅持有一个组合的服务过滤器和作用域谓词,而包私有地记录不透明键,并单独暴露会等待作用域 fiber 完全停稳的 disposer。 @@ -72,7 +74,7 @@ Receiver 是一个小型载体而非领域对象的透明代理。需要 agent ### 注册表读取叠加一个精确 layer -作用域感知的注册表使用 `ScopedLayers`,拥有一个即时创建的全局 aggregate 和按标识键惰性创建的 aggregate。读取解析全局 layer 和至多一个精确局部 layer;它不创建状态,也从不遍历父级链。注册可见性与 Cordis effect 所有权都从同一个上下文派生,而回收会等待具体 layer 的完整 aggregate 变空(见[决策](2026-07-12-scoped-layers-store.md))。 +作用域感知的注册表使用 `ScopedLayers`,拥有一个即时创建的全局 aggregate 和按标识键惰性创建的 aggregate。读取解析全局 layer 和至多一个精确局部 layer;它不创建状态,也从不遍历父级链。注册可见性与 Cordis effect 所有权都从同一个上下文派生,而回收会等待具体 layer 的完整 aggregate 变空(见[决策](2026-07-12-scoped-layers-store.zh.md))。 每个服务保留其领域规则。命名 command 和提示词视图使用共享的、保持插入顺序的 shadow 合并;工具保留更丰富的 resolver,因为限制会在加入局部工具前过滤全局工具,保留的 Code Mode transport 则单独插入。提示词变量和工具 guard 保持实时迭代,而工具提供方成员关系按每次 assembly 物化。Scope 提供存储生命周期和命名遮蔽,而非通用的注册表视图。 @@ -158,6 +160,8 @@ sequenceDiagram 此顺序让最终的 agent 和会话事件能使用匹配的作用域监听器,并使持久化观察者在最终刷新完成前保持附加。作用域 dispose 放在最后,因为注册撤销是外部可见的生命期边界。 + + ## 会话追加:物化、验证、提交、通知 会话事件跨越持久化边界,因此追加操作拥有其数据。算法的其余部分使用一条已附加的注册表条目和一个提交点。 @@ -208,7 +212,7 @@ Session 头部、种子和追加的事件是无损 JSON 数据。Session 构造 私有解析器应用当前展示模式、活跃的全局限制、精确的局部叠加和局部遮蔽。Schema、查找、执行、Code Mode SDK 生成和限制验证都使用该解析器或其限制前的全局名称视图。 -[subagent 组合控制 Agent Note](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.md#tool-filtering-is-one-live-global-view-rule) 拥有用户可见的 allow/deny 语义。实现要求是一致性:被过滤掉的全局工具不能通过另一条查找路径仍可执行,局部遮蔽的定义就是被展示和执行的同一个定义。 +[subagent 组合控制 Agent Note](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.zh.md#tool-filtering-is-one-live-global-view-rule) 拥有用户可见的 allow/deny 语义。实现要求是一致性:被过滤掉的全局工具不能通过另一条查找路径仍可执行,局部遮蔽的定义就是被展示和执行的同一个定义。 `ToolRestriction` 接受 readonly 的 allow/deny 名称并将其编译为内部集合。多个限制取交集。公开的 `visible()` 和 `knownNames()` 方法是不必要的,因为只有注册表需要中间视图。 @@ -230,6 +234,8 @@ SystemPrompt 首先将全局加 agent 的段、变量和工具提供方解析为 Scope 直接解决了真正的隔离问题。结构化输出贡献注册在子级的精确作用域中,而 Code Mode 从同一个已解析的工具视图派生其传输和 SDK。第二套命名保护系统需要另一套所有权和碰撞规则来覆盖任意 schema 提供方(包括有意贡献重复名称的提供方),却不创建新的信任边界。 + + ### 结构化输出仅提交权威结果 结构化输出将子作用域组合与两阶段执行提交相结合。子级在发布前注册其 `structured_output` 工具和指令;可信的 assembly 监听器可以变换这些普通贡献,并有责任在期望子级完成时保持协议。工具体验证候选值并按当前 `ToolExecution` 暂存,但成功捕获仅由不可变的 `tools/result` 观察决定。 @@ -242,6 +248,8 @@ Scope 直接解决了真正的隔离问题。结构化输出贡献注册在子 纯 Code Mode 的注册表贡献从原生 wire schema 中省略 `structured_output`,并通过生成的 SDK 暴露它。Assembly waterfall 可以有意改变该展示;执行仍然针对子作用域定义进行验证,监听器拥有其创建的任何替代模型可见路由的一致性。 + + ### 三个执行边界有意设为单向 提示词组装有意是协作式的,但三个执行事实在其可扩展阶段之后需要单向结算: @@ -266,7 +274,7 @@ subagent 启动有一次所有权转移。提供方拥有未发布资源,直 ### 服务约定有一个取消通道 -`SubagentProvider.start()` 和 `SubagentRuntime.start()` 返回 `Promise`。Promise 会在后端跨过发布边界后兑现,因此调用方和 `subagent/start` 观察者从不需要第二个 `run.started` promise。提供方工作如果在发布前失败,`start()` 就会被拒绝;发布后的提示词、轮次、取消与基础设施结果会通过 `SubagentRun.result` 结算,且不会隐藏 child id,这也是[持久化目录决策](../feature/2026-07-22-durable-subagent-catalog-and-list-agents.md)所要求的约定。 +`SubagentProvider.start()` 和 `SubagentRuntime.start()` 返回 `Promise`。Promise 会在后端跨过发布边界后兑现,因此调用方和 `subagent/start` 观察者从不需要第二个 `run.started` promise。提供方工作如果在发布前失败,`start()` 就会被拒绝;发布后的提示词、轮次、取消与基础设施结果会通过 `SubagentRun.result` 结算,且不会隐藏 child id,这也是[持久化目录决策](../feature/2026-07-22-durable-subagent-catalog-and-list-agents.zh.md)所要求的约定。 `SubagentStartRequest.signal` 是必需的。中止它会在启动期间,以及已发布 run 的剩余就绪或轮次工作中请求取消。`SubagentRun.dispose()` 也请求取消并等待完全停稳。没有单独的公开 `run.cancel()` 通道。 @@ -292,6 +300,8 @@ Start 仅在 `initialize` 和 `newSession` 成功后才 resolve。Abort、spawn Worker 和子进程桥接比同进程注册表需要更多状态,因为消息、进程死亡和清理可以独立结算。它们的状态围绕这些真实事实组织,而非重复的取消协议。 + + ### 工作流子级是待定 start 或已发布记录 工作流宿主保持待定的提供方 start promise 和已发布的子级记录。子级仅在异步 `SubagentRuntime.start()` 兑现时才从待定变为已发布;被拒绝的 start 清理其部分提供方工作且不产生子级生命周期对。 @@ -308,7 +318,7 @@ Worker 边界仍然序列化请求和结果。宿主保留首个终端结果仲 ### ACP 提示词结算不依赖更新投递 -[仅面向自动化的 ACP 桥接层](../simplification/2026-07-23-acp-automation-only-protocol.md)直接将一个进行中的提示词与其观察到的用户消息轮次关联。它不从日志水位线扫描,也不使用会话状态作为第二个调和预言机。 +[仅面向自动化的 ACP 桥接层](../simplification/2026-07-23-acp-automation-only-protocol.zh.md)直接将一个进行中的提示词与其观察到的用户消息轮次关联。它不从日志水位线扫描,也不使用会话状态作为第二个调和预言机。 即使已提交消息的更新无法送达客户端,会话事件监听器也会从匹配的 `turn/end` 结算关联。因此更新投递不能让会话永久处于进行中状态。ACP 创建由服务器分配 id 的全新会话,并拥有由此产生的每个 agent 句柄,直到连接拆除。 @@ -330,13 +340,13 @@ TypeScript 无法管控 JavaScript 强制转换、直接 Cordis dispatch、进 ### 生成的产物使公开约定保持对齐 -事件目录、服务目录、生产者/消费方矩阵、配置目录、模块图、工具目录、type-equiv 块和作用域事件解析器映射都是从源码生成或受新鲜度门禁约束的。[TypeScript 语义门禁 Agent Note](../process/2026-07-14-typescript-program-backed-semantic-gates.md) 拥有 Program 构造、语义事件发现和解析器生成规则。 +事件目录、服务目录、生产者/消费方矩阵、配置目录、模块图、工具目录、type-equiv 块和作用域事件解析器映射都是从源码生成或受新鲜度门禁约束的。[TypeScript 语义门禁 Agent Note](../process/2026-07-14-typescript-program-backed-semantic-gates.zh.md) 拥有 Program 构造、语义事件发现和解析器生成规则。 行为测试固定了作用域路由和 dispose、最终写入注册表时的碰撞清理、发布回滚、有序完全停稳、持久化前/后提交行为、跨展示和执行的活跃工具过滤、协作式提示词组装、原生和 Code Mode 中的结构化输出提交、异步 subagent 启动和信号取消、worker 终端仲裁、ACP 结算和进程拆除。 ## 曾考虑的替代方案 -[7 月 8 日 Agent Note](2026-07-08-agent-scope-contexts.md#alternatives-considered) 拥有公开扁平作用域约定的替代方案。此处的替代方案关注实现形态。 +[7 月 8 日 Agent Note](2026-07-08-agent-scope-contexts.zh.md#alternatives-considered) 拥有公开扁平作用域约定的替代方案。此处的替代方案关注实现形态。 ### 使用透明代理作为作用域载体 @@ -389,4 +399,4 @@ Worker 消息、进程死亡和持久化输入确实跨越所有权和序列化 该设计信任同进程中的类型化插件。它不防御任意强制转换、有状态 getter、违反 readonly 约定的修改,或插件有意在支持的组合 API 之外使用环境服务访问。 -[安全与权限非目标](2026-07-08-agent-scope-contexts.md#security-and-authority-are-non-goals)仍然是根本性的。这些机制证明注册组合、发布和生命期所有权;它们不证明隔离或父到子的非升权。 +[安全与权限非目标](2026-07-08-agent-scope-contexts.zh.md#security-and-authority-are-non-goals)仍然是根本性的。这些机制证明注册组合、发布和生命期所有权;它们不证明隔离或父到子的非升权。 diff --git a/.agents/notes/implemented/architecture/2026-07-12-scoped-layers-store.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-12-scoped-layers-store.i18n.yaml index 86871f2bcd..205c3c4e39 100644 --- a/.agents/notes/implemented/architecture/2026-07-12-scoped-layers-store.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-12-scoped-layers-store.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-12-scoped-layers-store.md 2026-07-12-scoped-layers-store.md: 221711748007d5f909aec3fdfcf3919884d11cdb -2026-07-12-scoped-layers-store.zh.md: 7f3ad11e23f02eeafe955efe08525bda8a9e9996 +2026-07-12-scoped-layers-store.zh.md: 97ad637b91a2b9b2cad9847d8b4cdbe23ff14a48 diff --git a/.agents/notes/implemented/architecture/2026-07-12-scoped-layers-store.zh.md b/.agents/notes/implemented/architecture/2026-07-12-scoped-layers-store.zh.md index 7f3ad11e23..97ad637b91 100644 --- a/.agents/notes/implemented/architecture/2026-07-12-scoped-layers-store.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-12-scoped-layers-store.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -agent(智能体)作用域机制([决策](2026-07-08-agent-scope-contexts.md)、[运行时设计](2026-07-12-agent-scope-runtime-design.md))让支持作用域的注册表反复呈现同一种形态:一个全局注册层,加上一个与具体 agent 精确对应的层。七个注册门面都采用这一形态:`tools.register`、`tools.restrict` 和 `tools.guard`(位于 `dsh-tools`);`SystemPrompt.section`、`SystemPrompt.tools` 和 `SystemPrompt.variable`(位于 `dsh-system-prompt`);以及 `CommandRuntime.register`(位于 `dsh-commands`)。 +agent(智能体)作用域机制([决策](2026-07-08-agent-scope-contexts.zh.md)、[运行时设计](2026-07-12-agent-scope-runtime-design.zh.md))让支持作用域的注册表反复呈现同一种形态:一个全局注册层,加上一个与具体 agent 精确对应的层。七个注册门面都采用这一形态:`tools.register`、`tools.restrict` 和 `tools.guard`(位于 `dsh-tools`);`SystemPrompt.section`、`SystemPrompt.tools` 和 `SystemPrompt.variable`(位于 `dsh-system-prompt`);以及 `CommandRuntime.register`(位于 `dsh-commands`)。 如果没有共享原语,每个门面都要围绕自己的领域状态重复相同的生命周期编排:从调用方上下文导出可见性,按需创建专属容器,把属主绑定到同一个 Cordis fiber,先装入 undo 再通知观察者,原样返回 Cordis 的 disposer,并回收空的专属状态。各自分离的映射与集合类型也会让服务缺少一个表示某个 scope 完整贡献的对象。 diff --git a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml index a79104468c..34523f2723 100644 --- a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md -2026-07-14-provider-routed-llm-adapters.md: e1eaf52f21481a7c65e85effb7607b16f9b0ffdd -2026-07-14-provider-routed-llm-adapters.zh.md: 4e620408dabffc8368293635613afb5778b6e822 +2026-07-14-provider-routed-llm-adapters.md: 78c8d6788006c503b532ff2bbddd30342415f0a4 +2026-07-14-provider-routed-llm-adapters.zh.md: af8bc4fe27a50d47d7b49b51eada67afe889fc13 diff --git a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md index e1eaf52f21..78c8d67880 100644 --- a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md +++ b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md @@ -40,9 +40,9 @@ pi-ai's common stream options do not expose stop sequences. `dsh-llm-pi-ai` reje Assistant messages carry the request's `provider` and `model`, plus an optional JSON-serializable adapter replay state. A successful `assistant/message` session event records those fields and `deriveMessages()` returns them with the assistant message. User, system, context, and tool-result messages carry no assistant route fields. The provider/model fields are authoritative loop data; an adapter owns only its opaque replay-state payload. -A terminal successful `finish` chunk may carry replay state, and `BlockAssembler` retains it alongside usage and finish reason. The loop attaches that state to the assembled assistant message's model source without exposing a response-rewrite hook. Error and aborted responses do not produce a normal assistant message and therefore do not enter future model history. +A terminal successful `finish` chunk may carry replay state as a `ReplayEnvelope`: opaque response-level metadata plus optional per-block entries aligned with the emitted block sequence. `BlockAssembler` makes one keep/drop decision for content and metadata — when max-token assembly drops a tool call, the envelope loses the entry at the same position — so the state the loop attaches to the assembled assistant message's model source always describes the stored blocks, per the [max-token replay-state alignment decision](../bug-fix/2026-08-15-max-token-replay-state-alignment.md). The loop exposes no response-rewrite hook. Error and aborted responses do not produce a normal assistant message and therefore do not enter future model history. -The pi-ai replay state is a versioned, minimal projection of its successful `AssistantMessage`: source API/provider/model, response id/model, stop reason, and index-aligned text, thinking, and tool-call signatures. It does not duplicate text or tool arguments already carried by Harness content blocks, and it omits diagnostics, timestamps, usage, and errors. On a later request, `LlmRuntime` gives replay state to the target adapter only when the historical provider and target provider are currently owned by the same adapter instance. That adapter combines the logged Harness content with replay state when it can restore the historical response, and owns any required cross-model or cross-provider conversion. An adapter receiving replay state with an unknown version or mismatched block shape fails explicitly; a different adapter receives only provider-neutral content plus provider/model fields. +The pi-ai replay state fills that envelope with a versioned, minimal projection of its successful `AssistantMessage`: a response half (source API/provider/model, response id/model, stop reason) and per-block text, thinking, and tool-call signatures. It does not duplicate text or tool arguments already carried by Harness content blocks, and it omits diagnostics, timestamps, usage, and errors. On a later request, `LlmRuntime` gives replay state to the target adapter only when the historical provider and target provider are currently owned by the same adapter instance. That adapter combines the logged Harness content with replay state when it can restore the historical response, and owns any required cross-model or cross-provider conversion. Durable content stays authoritative: an adapter receiving replay state it cannot use — an unknown kind or version, malformed metadata, or a block shape that no longer matches the content — degrades that message to provider-neutral conversion with a diagnostic; a different adapter receives only provider-neutral content plus provider/model fields. This state is model-visible replay input and therefore follows the existing [reconstructable-request rule](2026-07-05-reconstructable-requests.md): it is present in both the terminal `finish` chunk and the assembled `assistant/message` model source that drives derivation. Resume and fork preserve it verbatim. Compaction that shadows the assistant message also removes its replay state from the active surface; the summary is ordinary provider-neutral content. diff --git a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md index 4e620408da..af8bc4fe27 100644 --- a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md @@ -20,7 +20,7 @@ Status: implemented `GenerateOptions` 与 `LlmCallConfig` 在 `model: string` 之外携带 `provider: string`,`AgentOptions` 则携带对应的可选创建字段。只有两个值都非空时,agent loop(智能体循环)请求才有效;两个值也都会写入请求头日志。`agent/request` 可以在任意步骤返回替换后的字段组合,因此会话可以切换提供方与模型,无需改变 Cordis 插件生命周期。 -`LlmRuntime` 按提供方注册和解析适配器。`registerAdapter(providers, adapter)` 在修改注册表前检查整个提供方列表,遇到重复项时返回 `DUPLICATE_ADAPTER`,并以一个 effect 为单位整体 dispose(资源释放)。模型 ID 不作为注册键;仍由选中的适配器负责验证或转发。后续的 [LLM 目录与 ACP 模型选择 Agent Note](2026-07-15-llm-model-catalog-and-acp-selection.md) 增加了建议性的 `listProviders()` / `listModels()` 发现接口,但不会把目录成员关系变成请求校验规则。 +`LlmRuntime` 按提供方注册和解析适配器。`registerAdapter(providers, adapter)` 在修改注册表前检查整个提供方列表,遇到重复项时返回 `DUPLICATE_ADAPTER`,并以一个 effect 为单位整体 dispose(资源释放)。模型 ID 不作为注册键;仍由选中的适配器负责验证或转发。后续的 [LLM 目录与 ACP 模型选择 Agent Note](2026-07-15-llm-model-catalog-and-acp-selection.zh.md) 增加了建议性的 `listProviders()` / `listModels()` 发现接口,但不会把目录成员关系变成请求校验规则。 在一个 Cordis 上下文中,一个提供方只能有一个适配器所有者。`dsh-llm-deepseek` 注册 `deepseek`;`dsh-llm-pi-ai` 也可以注册 `deepseek`,但同时加载两个所有者属于配置错误,不采用顺序规则或回退行为。若部署选择手写的 DeepSeek 实现,需从 pi-ai 配置中排除 `deepseek`;若部署选择 pi-ai 的 DeepSeek 实现,则不挂载 `dsh-llm-deepseek`。 @@ -40,11 +40,11 @@ pi-ai 的通用流选项不支持停止序列。若 Harness `stop` 选项已定 助手消息携带请求的 `provider` 和 `model`,以及可选的 JSON 可序列化适配器回放状态。成功的 `assistant/message` 会话事件记录这些字段,`deriveMessages()` 返回助手消息时也会包含它们。用户、系统、上下文与工具结果消息不携带助手路由字段。提供方/模型字段是 agent loop 的权威数据;适配器仅拥有其不透明回放状态 payload。 -成功的终止 `finish` 分片可以携带回放状态,`BlockAssembler` 会将其与 token 用量和结束原因一起保留。agent loop 会把该状态附加到已组装助手消息的模型来源中,但不公开响应改写钩子。错误或中止响应不会生成正常助手消息,因此不会进入后续模型历史。 +成功的终止 `finish` 分片可以以 `ReplayEnvelope` 形式携带回放状态:不透明的响应级元数据,加上与发射块序列对齐的可选逐块条目。`BlockAssembler` 对内容与元数据只做一次保留/丢弃决定——max-token 组装丢弃工具调用时,数据同一位置的条目一并丢弃——因此 agent loop 附加到已组装助手消息模型来源中的状态始终描述存储的块,见 [max-token 回放状态对齐决定](../bug-fix/2026-08-15-max-token-replay-state-alignment.zh.md)。agent loop 不公开响应改写钩子。错误或中止响应不会生成正常助手消息,因此不会进入后续模型历史。 -pi-ai 回放状态是其成功 `AssistantMessage` 的带版本最小投影,包含源 API/提供方/模型、响应 ID/模型、停止原因,以及按索引对齐的文本签名、thinking 签名和工具调用签名。它不会重复 Harness 内容块中已有的文本或工具参数,也不包含诊断信息、时间戳、用量或错误。后续请求中,只有历史提供方和目标提供方当前归同一个适配器实例所有时,`LlmRuntime` 才会把回放状态交给目标适配器。适配器在能够恢复历史响应时,将 Harness 记录的内容与回放状态组合,并负责所需的跨模型或跨提供方转换。适配器收到未知版本或块形状不匹配的回放状态时会显式失败;其他适配器只能收到提供方无关的内容以及提供方/模型字段。 +pi-ai 回放状态用其成功 `AssistantMessage` 的带版本最小投影填充该结构:一个响应半区(源 API/提供方/模型、响应 ID/模型、停止原因),以及逐块的文本签名、thinking 签名和工具调用签名。它不会重复 Harness 内容块中已有的文本或工具参数,也不包含诊断信息、时间戳、用量或错误。后续请求中,只有历史提供方和目标提供方当前归同一个适配器实例所有时,`LlmRuntime` 才会把回放状态交给目标适配器。适配器在能够恢复历史响应时,将 Harness 记录的内容与回放状态组合,并负责所需的跨模型或跨提供方转换。持久化内容保持权威:适配器收到无法使用的回放状态——未知 kind 或版本、格式错误的元数据、或与内容不再匹配的块结构——会把该消息降级为提供方无关转换并带出诊断;其他适配器只能收到提供方无关的内容以及提供方/模型字段。 -该状态属于模型可见的回放输入,因此遵循现有的[请求可重建规则](2026-07-05-reconstructable-requests.md):它同时存在于终止 `finish` 分片和驱动派生的已组装 `assistant/message` 模型来源中。恢复和 fork 会原样保留该状态。压缩(compaction)遮蔽助手消息时,也会从活动 surface 中移除其回放状态;摘要属于普通的提供方无关内容。 +该状态属于模型可见的回放输入,因此遵循现有的[请求可重建规则](2026-07-05-reconstructable-requests.zh.md):它同时存在于终止 `finish` 分片和驱动派生的已组装 `assistant/message` 模型来源中。恢复和 fork 会原样保留该状态。压缩(compaction)遮蔽助手消息时,也会从活动 surface 中移除其回放状态;摘要属于普通的提供方无关内容。 ### 在所有请求生产方中传播目标 diff --git a/.agents/notes/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml index 458999f905..570b68263c 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-agent-initiator-scope.md 2026-07-15-agent-initiator-scope.md: 63540c0ec6b29a10613e01f1ed9ced24e8f2d277 -2026-07-15-agent-initiator-scope.zh.md: a0e3638081d875adc2412191829ab84c8dcd697c +2026-07-15-agent-initiator-scope.zh.md: 3ea893aa5f6992bf09965436c1db3144d2fae5ac diff --git a/.agents/notes/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md b/.agents/notes/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md index a0e3638081..3ea893aa5f 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md @@ -12,7 +12,7 @@ harness 中存在两种有用但不同的上下文概念。Cordis `Context` 负 ## 决策 -必需的 `ctx.agents` 服务使用 Node `AsyncLocalStorage` 携带发起 Agent。它直接存储同一个 `Agent`,不引入只有一个字段的帧;另一个私有运行标记只记录嵌套边界的谱系,供 teardown 记账使用,不携带身份。[核心数据目录](../../../../docs/subsystems/core.md#initiating-agent)标明了所携带的类型。 +必需的 `ctx.agents` 服务使用 Node `AsyncLocalStorage` 携带发起 Agent。它直接存储同一个 `Agent`,不引入只有一个字段的帧;另一个私有运行标记只记录嵌套边界的谱系,供 teardown 记账使用,不携带身份。[核心数据目录](../../../../docs/subsystems/core.zh.md#initiating-agent)标明了所携带的类型。 `currentInitiator()` 用于可选读取,`requireInitiator()` 抛出 `no initiating agent is active`,`withInitiator(agent, operation)` 保留操作返回的同步值或 Promise 本身。`withoutInitiator(operation)` 会建立清空边界,供不得继承 Agent 的工作使用。会话仍通过 `agent.session` 推导;轮次、步骤、工具调用、`signal`、模型、`cwd`、沙箱和授权继续由现有归属方管理。 @@ -28,7 +28,7 @@ harness 中存在两种有用但不同的上下文概念。Cordis `Context` 负 宿主感知的传输层可以从 `ctx.agents.requireInitiator().session.id` 推导由部署方拥有的 `X-Harness-Session-Id` 等请求头;模型可见 schema 和参数中不包含该请求头。本决策不让现有生产 MCP 或 Web 传输层采用此请求头。测试替身传输层用于证明可信边界,而不会把宿主路由策略分配给现有的提供方无关 seam。 -本决策扩展 [Agent 注册作用域约定](2026-07-08-agent-scope-contexts.md)及其[运行时设计](2026-07-12-agent-scope-runtime-design.md),不会改变其中 `agent.ctx` 的静态含义。 +本决策扩展 [Agent 注册作用域约定](2026-07-08-agent-scope-contexts.zh.md)及其[运行时设计](2026-07-12-agent-scope-runtime-design.zh.md),不会改变其中 `agent.ctx` 的静态含义。 ## 验证 diff --git a/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.i18n.yaml index 69af08605c..eb7a9adfab 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.md -2026-07-15-llm-model-catalog-and-acp-selection.md: 3dddbe7e9fae74e4ce1ec1c8e93a40c3352b5a54 -2026-07-15-llm-model-catalog-and-acp-selection.zh.md: 145fd0bc379ff8132d09ec628b6381a18f755cfa +2026-07-15-llm-model-catalog-and-acp-selection.md: fef23711a9214eae414809833bcd9ee9e26105ae +2026-07-15-llm-model-catalog-and-acp-selection.zh.md: 85e9fa35ac99b72a7df207fdb4971b57cf6dc525 diff --git a/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.md b/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.md index 3dddbe7e9f..fef23711a9 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.md +++ b/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.md @@ -4,7 +4,7 @@ Status: implemented English | [中文](2026-07-15-llm-model-catalog-and-acp-selection.zh.md) -> The catalog decision remains current. Per-session ACP model selection is superseded by [ACP as an automation-only protocol](../simplification/2026-07-23-acp-automation-only-protocol.md). +> The catalog and scoped-selection decisions remain current. The temporary removal of ACP selection is superseded by [standard ACP v1 automation controls](../feature/2026-08-22-standard-acp-automation-controls.md), which exposes the catalog through standard session configuration without restoring UI projections. ## Problem @@ -24,17 +24,17 @@ ACP selection must also preserve the provider dimension. The same model id may a Catalog membership is advisory. It drives selectors and diagnostics but never changes `stream()` routing and never rejects an otherwise valid request. Provider ownership remains exclusive and lifecycle-bound; model ids remain request-time adapter input. -`dsh-llm-pi-ai` maps the configured provider's installed `getModels(provider)` entries into the neutral catalog. Its existing request-time catalog lookup remains authoritative and still rejects unknown models with `UNKNOWN_MODEL`. `dsh-llm-deepseek` accepts an optional `models` config containing display entries, defaulting to `deepseek-v4-flash` named `DeepSeek-V4-Flash` and `deepseek-v4-pro` named `DeepSeek-V4-Pro`. An explicit list replaces those defaults and an empty list disables discovery. The entries improve selector UX for known public or private models, while every unlisted model id continues to pass through unchanged. +`dsh-llm-pi-ai` maps the configured provider's installed `getModels(provider)` entries into the neutral catalog. Its existing request-time catalog lookup remains authoritative and still rejects unknown models with `UNKNOWN_MODEL`. `dsh-llm-deepseek` accepts an optional `models` config containing display entries, defaulting to `deepseek-v4-flash` named `DeepSeek-V4-Flash`, `deepseek-v4-pro` named `DeepSeek-V4-Pro`, and image-capable `deepseek-v4-flash-vision-exp` named `DeepSeek-V4-Flash-Vision-Exp`. An explicit list replaces those defaults and an empty list disables discovery. The entries improve selector UX for known public or private models, while every unlisted model id continues to pass through unchanged. ### Per-session selection in the front end -A selection is owned by the front end that offers it (today the TUI `/model` selector), never by `LlmRuntime` or `AgentOptions`: those are deployment-wide or creation-wide objects, and mutating them would couple concurrent sessions. Each opaque choice carries the full provider/model pair, because the same model id may appear under multiple routes. +A selection is owned by the front end that offers it, never by `LlmRuntime` or `AgentOptions`: those are deployment-wide or creation-wide objects, and mutating them would couple concurrent sessions. Each opaque choice carries the full provider/model pair, because the same model id may appear under multiple routes. -The ACP automation transport is not a catalog consumer. Its deployment config supplies one optional provider/model target for newly created agents, and it advertises no model selector or configuration-option interface. +The ACP automation transport consumes the advisory catalog through standard session configuration options. Its deployment config still supplies the initial provider/model target; each session owns an opaque provider/model choice and a dependent exact-model reasoning-effort choice. Adapter topology changes publish the complete option state. Catalog absence never invalidates the configured route: the current unlisted route is synthesized into the choices. ### Prompt/request consistency and durability -`installModelSelection` (in `dsh-agent`) installs scoped `system-prompt/assemble` and `agent/request` listeners for a front-end-owned selection. Prompt assembly snapshots the selected pair once per step, overwrites the assembled `provider` and `model` variables after downstream prompt listeners, and the request listener applies that same snapshot after downstream request listeners. A selection during asynchronous assembly therefore starts on the next step rather than splitting prompt text from routing. Other call-config fields remain untouched. +`installModelSelection` (in `dsh-agent`) installs scoped `system-prompt/assemble` and `agent/request` listeners for a front-end-owned selection. Ordinary consumers snapshot the selection once per step. ACP associates its admission snapshot with the identified message in the per-session module until inbox claim, then pins that selection for the complete admitted turn, so asynchronous image admission, prompt variables, and every request step remain aligned without changing the durable user source. A concurrent selection starts on the next ACP turn. Other call-config fields remain untouched. The request header remains the durable source of truth. When a selection is actually used, the existing full `request/header` snapshot records it, and a front end initializes its selection from the folded last request header before falling back to creation options. A selection that is never used by a request is intentionally in-memory only because it never became model-visible state. @@ -53,10 +53,10 @@ The request header remains the durable source of truth. When a selection is actu - Any adapter can expose a dynamic model list without leaking provider-library types into the LLM Service Definition. - Catalog consumers must treat absence as “not advertised,” never “invalid request.” - pi-ai adapters expose their installed provider catalogs; hand-written DeepSeek deployments list known choices explicitly and retain arbitrary model support. -- Human-facing catalog consumers own their selection interaction. ACP uses its fixed deployment target and does not widen the protocol with model discovery. +- Each catalog consumer owns its selection interaction. ACP uses standard session configuration options and emits no DSH-specific selector or UI metadata. - Request headers remain compatible with the provider-routed session shape; no new JSONL event or format version is required. - A catalog read can be asynchronous, and every caller receives detached values. ## Testing -Unit coverage validates catalog detachment and malformed metadata, pi-ai and DeepSeek catalog projection, provider/model request routing, and prompt-variable alignment; per-agent isolation follows from installing the listeners on the agent-scoped context. ACP transport tests validate fixed provider/model forwarding independently of catalog discovery; the TUI suite covers selector interaction and header-based restoration. +Unit coverage validates catalog detachment and malformed metadata, pi-ai and DeepSeek catalog projection, provider/model request routing, and prompt-variable alignment; per-agent isolation follows from installing the listeners on the agent-scoped context. ACP tests validate grouped discovery, invalid and concurrent changes, topology updates, header-based restoration, per-turn route pinning, and image-route consistency; human clients test their own selector presentation. diff --git a/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.zh.md b/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.zh.md index 145fd0bc37..85e9fa35ac 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-llm-model-catalog-and-acp-selection.zh.md @@ -4,7 +4,7 @@ Status: implemented [English](2026-07-15-llm-model-catalog-and-acp-selection.md) | 中文 -> 目录决策仍然有效。ACP(Agent Client Protocol)会话级模型选择已由 [ACP 作为仅面向自动化的协议](../simplification/2026-07-23-acp-automation-only-protocol.md)取代。 +> Catalog 和 scoped selection 决策仍然有效。ACP selection 的暂时移除已由[标准 ACP v1 自动化控制](../feature/2026-08-22-standard-acp-automation-controls.zh.md)取代;后者通过标准会话配置公开 catalog,但不会恢复 UI 投影。 ## 问题 @@ -24,17 +24,17 @@ ACP 选择还必须保留提供方维度。同一个模型 ID 可能存在于多 目录成员关系仅提供建议。它驱动选择器与诊断,但不会改变 `stream()` 路由,也不会拒绝原本有效的请求。提供方所有权仍然具有排他性并绑定生命周期;模型 ID 仍是请求时传给适配器的输入。 -`dsh-llm-pi-ai` 将已配置提供方的 `getModels(provider)` 返回的已安装条目映射为提供方无关的目录。其现有请求时目录查询仍是权威依据,未知模型仍以 `UNKNOWN_MODEL` 失败。`dsh-llm-deepseek` 接受包含展示条目的可选 `models` 配置,默认包含名为 `DeepSeek-V4-Flash` 的 `deepseek-v4-flash` 和名为 `DeepSeek-V4-Pro` 的 `deepseek-v4-pro`。显式列表会替换这些默认值,空列表则关闭发现。这些条目改善已知公开或私有模型的选择体验,而所有未列出的模型 ID 仍会原样透传。 +`dsh-llm-pi-ai` 将已配置提供方的 `getModels(provider)` 返回的已安装条目映射为提供方无关的目录。其现有请求时目录查询仍是权威依据,未知模型仍以 `UNKNOWN_MODEL` 失败。`dsh-llm-deepseek` 接受包含展示条目的可选 `models` 配置,默认包含名为 `DeepSeek-V4-Flash` 的 `deepseek-v4-flash`、名为 `DeepSeek-V4-Pro` 的 `deepseek-v4-pro`,以及名为 `DeepSeek-V4-Flash-Vision-Exp`、支持图片输入的 `deepseek-v4-flash-vision-exp`。显式列表会替换这些默认值,空列表则关闭发现。这些条目改善已知公开或私有模型的选择体验,而所有未列出的模型 ID 仍会原样透传。 ### 前端内的会话级选择 -选择由提供它的前端拥有(今天是 TUI 的 `/model` 选择器),而不由 `LlmRuntime` 或 `AgentOptions` 拥有:它们是部署级或创建级对象,改动它们会把并发会话耦合在一起。每个不透明选项都携带完整的提供方/模型对,因为同一模型 ID 可能出现在多个路由下。 +选择由提供它的前端拥有,而不由 `LlmRuntime` 或 `AgentOptions` 拥有:它们是部署级或创建级对象,改动它们会把并发会话耦合在一起。每个不透明选项都携带完整的提供方/模型对,因为同一模型 ID 可能出现在多个路由下。 -ACP 自动化传输层不是目录消费方。它通过部署配置为新创建的 agent 提供一个可选的提供方/模型目标,不展示模型选择器或配置选项接口。 +ACP 自动化传输层通过标准会话配置选项消费建议性 catalog。部署配置仍提供初始提供方/模型目标;每个会话拥有一个不透明的提供方/模型选择,以及一个依赖确切模型的 reasoning-effort 选择。Adapter 拓扑变化会公布完整选项状态。Catalog 中缺少条目不会使配置路由失效:当前未列出的路由会合成到选项中。 ### 提示词/请求一致性与持久化 -`installModelSelection`(位于 `dsh-agent`)为前端拥有的选择安装 agent 作用域的 `system-prompt/assemble` 与 `agent/request` 监听器。提示词组装在每个步骤对所选组合做一次快照,在下游提示词监听器之后覆写组装出的 `provider` 与 `model` 变量;请求监听器在下游请求监听器之后应用同一快照。因此,发生在异步组装期间的选择会从下一个步骤生效,而不会让提示词文本与路由分裂。其他调用配置字段保持不变。 +`installModelSelection`(位于 `dsh-agent`)为前端拥有的选择安装 agent 作用域的 `system-prompt/assemble` 与 `agent/request` 监听器。普通 consumer 每个步骤快照一次选择。ACP 会在 per-session 模块中把准入快照与已识别消息关联到 inbox claim 时刻,再在完整已准入轮次中固定该选择,使异步图片准入、提示词变量和每个请求步骤保持一致,同时不改变持久用户 source。并发选择变更从下一个 ACP 轮次开始。其他调用配置字段保持不变。 请求头仍是持久化的真源。当某个选择真正被使用时,现有的完整 `request/header` 快照会记录它;前端先从折叠后的最后一个请求头初始化其选择,然后才回退到创建选项。从未被请求使用的选择有意只保留在内存中,因为它从未成为模型可见状态。 @@ -53,10 +53,10 @@ ACP 自动化传输层不是目录消费方。它通过部署配置为新创建 - 任意适配器都能暴露动态模型列表,无需把提供方库类型泄漏到 LLM Service Definition。 - 目录消费方必须把缺失理解为「未展示」,而不是「请求无效」。 - pi-ai 适配器会暴露其已安装的提供方目录;手写 DeepSeek 部署显式列出已知选项,同时保留对任意模型的支持。 -- 面向人类的目录消费方拥有各自的选择交互。ACP 使用固定部署目标,不会为模型发现扩大协议范围。 +- 每个 catalog consumer 拥有自己的选择交互。ACP 使用标准会话配置选项,不发出 DSH 专用 selector 或 UI 元数据。 - 请求头与基于提供方路由的会话形态保持兼容;不需要新的 JSONL 事件或格式版本。 - 目录读取可以是异步的,且每个调用方都会收到值的独立副本。 ## 测试 -单元测试覆盖目录值副本与格式错误的元数据、pi-ai 和 DeepSeek 目录投影、提供方/模型请求路由,以及提示词变量对齐;监听器安装在 agent 作用域的上下文中,因此能够实现 agent 间隔离。ACP 传输测试独立验证固定提供方/模型的转发行为;TUI 套件覆盖选择器交互与基于请求头的恢复。 +单元测试覆盖 catalog 值副本与格式错误的元数据、pi-ai 和 DeepSeek catalog 投影、提供方/模型请求路由,以及提示词变量对齐;监听器安装在 agent 作用域的上下文中,因此能够实现 agent 间隔离。ACP 测试覆盖分组发现、无效和并发变更、拓扑更新、基于请求 header 的恢复、逐轮路由固定以及图片路由一致性;人工客户端测试自己的 selector 展示。 diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml index aee9c76d55..277e0b097d 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md 2026-07-15-lsp-capability-seam.md: 90f9daf4b890bd53621916e492d4d82baaa3e8cc -2026-07-15-lsp-capability-seam.zh.md: b85b478b25a375b04e1301d893e53198554641e3 +2026-07-15-lsp-capability-seam.zh.md: bdb5e812e94a4aec4402fe83ca9c818bfa00f9f9 diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md index b85b478b25..bdb5e812e9 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md @@ -79,7 +79,7 @@ interface LspService { 映射键规范化为带前导点的小写扩展名,并按 `filePath` 的最后一个扩展名选择;语言 id 仅用于文档同步。seam 中的位置和范围从零开始按 UTF-16 计数。`findReferences` 始终包含声明:提供方在内部执行该约束,本地映射设置 `context.includeDeclaration: true`,调用方不能配置。封闭结果联合将导航统一为位置,将 `hover` 统一为内容或 `null`;导航结果携带提供方的规范工作区 URI,使消费方在执行世界的命名空间内相对化文件 URI。seam 不公开协议类型、进程或文档控制,也不提供通用请求逃生口。 -`dsh-lsp-stdio` 负责服务器配置、JSON-RPC、进程与临时文档状态和协议转换。它通过 `ctx.fs` 读取,通过 `ctx.subprocess` 启动,只依赖二者的 Service Definition 包而非具体提供方;[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)负责定义这种配对。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置;如果后续映射无效或发生冲突,插件会撤销此前的注册,并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`,通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区,其取值方式与文件系统工具一致,也不导入提供方。 +`dsh-lsp-stdio` 负责服务器配置、JSON-RPC、进程与临时文档状态和协议转换。它通过 `ctx.fs` 读取,通过 `ctx.subprocess` 启动,只依赖二者的 Service Definition 包而非具体提供方;[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.zh.md)负责定义这种配对。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置;如果后续映射无效或发生冲突,插件会撤销此前的注册,并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`,通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区,其取值方式与文件系统工具一致,也不导入提供方。 ## 面向模型的约定 diff --git a/.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml index 47e7156ab8..6ad1773f6f 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md 2026-07-15-replay-token-meter-service.md: 10261189fca621b305b3ec6347b54471d81c8ba3 -2026-07-15-replay-token-meter-service.zh.md: 4e460b3c32630f205f447a16205bd4c9b1d89faf +2026-07-15-replay-token-meter-service.zh.md: 370d3cf932d37702c5298bc79e9dc8f5a37835b9 diff --git a/.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md b/.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md index 4e460b3c32..370d3cf932 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md @@ -16,7 +16,7 @@ Status: implemented `@deepseek-ai/dsh-token-meter` 是 `packages/llm/` 下的单个具体包,并注册 `ctx.tokenMeter`。在第二种实现出现之前,它不会被拆成接口与后端。`TokenMeter` 本身公开 `measure(session, requestHeader?)` 与 `estimateMessage(message)`;消费方直接调用这个单例服务。 -服务没有配置。估算采用固定的每 token 四个字符启发式规则,并加上结构开销。服务不提供模型 profile、容量设置、密度设置、分词器后端或语言专用策略。对精确提供方/模型容量的查询由适配器单独负责,具体见[路由模型上下文与压缩策略 Agent Note](2026-07-20-routed-model-context-and-compaction-policy.md)。 +服务没有配置。估算采用固定的每 token 四个字符启发式规则,并加上结构开销。服务不提供模型 profile、容量设置、密度设置、分词器后端或语言专用策略。对精确提供方/模型容量的查询由适配器单独负责,具体见[路由模型上下文与压缩策略 Agent Note](2026-07-20-routed-model-context-and-compaction-policy.zh.md)。 ### 逐会话回放折叠 diff --git a/.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.i18n.yaml index 342f2073ea..fcca895513 100644 --- a/.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md 2026-07-16-explicit-turn-cancellation.md: 86faad9929d3eb5b00e66bb1c46a2e35b135d954 -2026-07-16-explicit-turn-cancellation.zh.md: 5397b368628c196efc2b35c00855267246ac6e85 +2026-07-16-explicit-turn-cancellation.zh.md: acf56e0629668a227324045ffd0521619dc45f33 diff --git a/.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.zh.md b/.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.zh.md index 5397b36862..acf56e0629 100644 --- a/.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.zh.md @@ -8,7 +8,7 @@ Status: implemented 取消是一种生命周期短于 Agent(智能体)驱动器的控制能力。自由文本字符串无法完整区分所有调用方,步骤级控制器也无法中断提示词提交、提示词组装、继续决策或轮次终止策略。持久化 `Error`、`AbortSignal.reason` 或后端私有对象还会向持久化回放暴露不稳定的运行时细节。 -[发起 Agent 作用域决策](2026-07-15-agent-initiator-scope.md)有意让 AsyncLocalStorage 只携带同一个 Agent。若把轮次、步骤或 signal 状态加入这个与驱动器同生命周期的边界,陈旧的异步后代就会看似仍对后续轮次拥有权限。因此,取消需要一个轮次归属方并显式传播,且不创建另一套环境上下文或公开的轮次包装层。 +[发起 Agent 作用域决策](2026-07-15-agent-initiator-scope.zh.md)有意让 AsyncLocalStorage 只携带同一个 Agent。若把轮次、步骤或 signal 状态加入这个与驱动器同生命周期的边界,陈旧的异步后代就会看似仍对后续轮次拥有权限。因此,取消需要一个轮次归属方并显式传播,且不创建另一套环境上下文或公开的轮次包装层。 ## 决策 @@ -18,9 +18,9 @@ Agent 拥有仅用于运行时的 `AgentCancelCause` 联合类型 `{ kind: 'user AgentLoop 为每个待启动轮次私有地持有一个 `TurnCancellation`。它在通知 `agent/status = running` 前安装该持有者,使其中唯一的 `AbortController` 持续覆盖 inbox 领取、`agent/pre-step`、提示词组装、每个步骤、模型与工具执行以及 `agent/turn-stopping`;随后在发布 `turn/end` 前立即清除所安装的那个持有者。因此,即使驱动器状态可能在持久化刷新结算前保持 `running`,终态事件观察者及其后的持久化刷新也无法取消已完成的轮次工作。所有参与的方法、事件和请求值都会收到同一个显式 signal;下一个轮次会收到全新的 signal。 -对于轮次被认领前已取消的排队工作,驱动器只保留一个不携带取消原因的运行前标记。实际生效的 `cancel()` 会先发出仅供观察的 `agent/cancel-requested` 通知并携带最终确定的类型化取消原因,然后才清除排队工作和 steering(中途引导)工作或中止持有者;通知失败不能阻止此次停止,空闲状态下调用则不发出任何通知。通知观察者同步加入队列的工作也会被这次清除,而稍后由 signal 中止观察者加入队列的工作会被锁存,并在被中止的活动收敛到空闲时执行——`disposed` 取消则将其停放([取消收敛窗口唤醒锁存](../bug-fix/2026-08-07-cancel-convergence-wake-latch.md))。若 `running` 监听器同步取消旧工作并发送替代提示词,驱动器会丢弃已中止的持有者,并为替代提示词创建全新的持有者。同一活跃持有者上的重复取消遵循首次请求优先,后续调用仍可清除新入队的待处理工作。 +对于轮次被认领前已取消的排队工作,驱动器只保留一个不携带取消原因的运行前标记。实际生效的 `cancel()` 会先发出仅供观察的 `agent/cancel-requested` 通知并携带最终确定的类型化取消原因,然后才清除排队工作和 steering(中途引导)工作或中止持有者;通知失败不能阻止此次停止,空闲状态下调用则不发出任何通知。通知观察者同步加入队列的工作也会被这次清除,而稍后由 signal 中止观察者加入队列的工作会被锁存,并在被中止的活动收敛到空闲时执行——`disposed` 取消则将其停放([取消收敛窗口唤醒锁存](../bug-fix/2026-08-07-cancel-convergence-wake-latch.zh.md))。若 `running` 监听器同步取消旧工作并发送替代提示词,驱动器会丢弃已中止的持有者,并为替代提示词创建全新的持有者。同一活跃持有者上的重复取消遵循首次请求优先,后续调用仍可清除新入队的待处理工作。 -显式事件签名传递单个 payload 对象:agent 作用域事件在 payload 中携带 `agent` 和 `signal`,`next` 位于最后;其余 API 保持 `signal` 紧邻 waterfall(瀑布式事件)的最终 `next` 之前。`PreStepContext` 与 `RequestFailureContext` 已退役,其字段并入 `agent/pre-step` 与 `agent/request-error` 的 payload([payload-object 事件](2026-08-06-agent-event-payload-objects.md))。进入 pre-step 时、请求配置、请求错误恢复、模型生成、工具执行、审批、轮次停止以及 subagent 或工作流请求都会收到当前 signal。钩子桥接器也必须提供 `RunHookOptions.signal`,使轮次取消能够到达 Bash 执行器终止进程组并等待其退出的边界。`SystemPrompt.assemble()` 在 `AssembleContext` 中携带 `signal?: AbortSignal`,因为该对象是显式请求值,也可表示轮次之外不携带 signal 的组装。监听器可以配合该 signal 取消,但不得保留它来控制其他轮次。 +显式事件签名传递单个 payload 对象:agent 作用域事件在 payload 中携带 `agent` 和 `signal`,`next` 位于最后;其余 API 保持 `signal` 紧邻 waterfall(瀑布式事件)的最终 `next` 之前。`PreStepContext` 与 `RequestFailureContext` 已退役,其字段并入 `agent/pre-step` 与 `agent/request-error` 的 payload([payload-object 事件](2026-08-06-agent-event-payload-objects.zh.md))。进入 pre-step 时、请求配置、请求错误恢复、模型生成、工具执行、审批、轮次停止以及 subagent 或工作流请求都会收到当前 signal。钩子桥接器也必须提供 `RunHookOptions.signal`,使轮次取消能够到达 Bash 执行器终止进程组并等待其退出的边界。`SystemPrompt.assemble()` 在 `AssembleContext` 中携带 `signal?: AbortSignal`,因为该对象是显式请求值,也可表示轮次之外不携带 signal 的组装。监听器可以配合该 signal 取消,但不得保留它来控制其他轮次。 `ctx.agents` 仍只携带发起 Agent。环境中的 Agent 并不代表存活、当前轮次或取消权限。cause 读取器是 loop 私有的,它直接陈述机器私有的 slot 不变量(只有 `cancel()` 会中止轮次控制器,且总是携带规范的冻结 cause),而不是对 reason 做结构化再校验;不存在从任意 signal 读取 cause 的公开辅助函数。并发 Agent 会同时隔离各自的发起方身份和轮次 signal;子驱动会遮蔽父发起方,而父请求 signal 仍通过 subagent seam 传递。 diff --git a/.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.i18n.yaml index e772ff9036..29807cfb7c 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.md 2026-07-19-cooperative-tool-cancellation.md: 781202688a5cbcd7076ee694fc7dd9489683d8e8 -2026-07-19-cooperative-tool-cancellation.zh.md: a5ce2e671e92f4758ddeb3d3556d8af574467f7f +2026-07-19-cooperative-tool-cancellation.zh.md: ec35734eef91c5c774d1be814b221e9fdb8f65fa diff --git a/.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.zh.md b/.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.zh.md index a5ce2e671e..ec35734eef 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.zh.md @@ -42,7 +42,7 @@ Status: implemented 工具主体一旦启动,注册表就会等待它完成。取消通过融合信号到达工具主体,但注册表不会与其 promise 竞速或丢弃该 promise。协作式实现会停止自身工作或继续转发取消,并在所持有的工作完全停稳后完成;不协作的同进程实现可能让注册表无限期保持等待。进程、worker、网络和提供方层仍负责各自的终止机制。 -这项决策只要求工具调用边界携带取消信号。让工具主体可达的异步能力也必须接收信号,属于另一项迁移,见提议中的[工具可达能力 seam 中的必填取消](../../proposed/architecture/2026-07-19-required-cancellation-through-tool-capability-seams.md)。 +这项决策只要求工具调用边界携带取消信号。让工具主体可达的异步能力也必须接收信号,属于另一项迁移,见提议中的[工具可达能力 seam 中的必填取消](../../proposed/architecture/2026-07-19-required-cancellation-through-tool-capability-seams.zh.md)。 ## 验证 @@ -62,7 +62,7 @@ Status: implemented **禁止环绕包装层替换信号。** 不予采纳,因为截止时间和嵌套操作作用域需要词法派生信号。捕获并融合调用方信号既保留组合能力,也不允许切断调用方取消。 -**让工具 promise 与取消竞速。** 不予采纳,因为这种方式会在副作用仍可能存活时报告完成,违反[dispose(资源释放)必须完全停稳的规则](../../../../docs/defensive-patterns.md#dispose-must-reach-quiescence-not-just-request-it)。 +**让工具 promise 与取消竞速。** 不予采纳,因为这种方式会在副作用仍可能存活时报告完成,违反[dispose(资源释放)必须完全停稳的规则](../../../../docs/defensive-patterns.zh.md#dispose-must-reach-quiescence-not-just-request-it)。 ## 后果 diff --git a/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.i18n.yaml index 1423e0f404..38e07804b7 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.md -2026-07-19-gui-layering-and-rpc-protocol.md: 22bebb0951ca3ffa0a0679396c4c54ef32eca279 -2026-07-19-gui-layering-and-rpc-protocol.zh.md: c38328cc2e692e8eed44a3dbf4149207820265c9 +2026-07-19-gui-layering-and-rpc-protocol.md: 372bf4926011835999ebae9b1e2d1f5beb8eb663 +2026-07-19-gui-layering-and-rpc-protocol.zh.md: ecce57c01c155c2a0b19b7729da13c39d1a520a6 diff --git a/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.md b/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.md index 22bebb0951..372bf49260 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.md +++ b/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.md @@ -26,7 +26,7 @@ Directories layer as follows: - `packages/host/*`: packages provide host-side capability only (representing the Node.js engineering core built on the existing harness plugin system), and additionally - the unified backend protocol (fetch, HTTP, streaming interfaces…) — definitions and support, see the "Message protocol" sections below - `packages/client/*`: packages provide client-side capability only; every package stays single-sided. Three kinds live here (the axes are owned by the [client plugin loading note](2026-07-23-client-plugin-loading-model.md)): - - **Pure libraries** (`ui-slots`, `web-react`, `ui-primitives`, plus the `loader` kernel package): ordinary root-index packages, statically bundled into the shell; the first three are seeded into the module table. + - **Pure libraries** (`ui-slots`, `ui-primitives`, plus the `loader` kernel package): ordinary root-index packages, statically bundled into the shell; the two client libraries are seeded into the module table. - **Static-arrival entry packages** (`connection`, `runtime`, `ui-theme`, `i18n`, `hmr`): no `dsh.client` key and no browser bundle — the shell bundles their `src/client/` half and registers it with `ctx.modules`; they are governed as entries of the host-authored graph like everything else. - **Fetch-arrival plugin packages** (`ui-layout`, `ui-sidebar`, `ui-conversation`, `ui-trajectory`): dual-entry — the root index is the node half (an empty `apply`, existing so the host Loader governs lifecycle and the web plugin registry discovers the package.json `dsh.client` declaration); the implementation lives under `src/client/`, shipped as the `./client` subpath (a tsdown closure-factory bundle). Cross-plugin consumption of `/client` is type-only; value cooperation goes through cordis services. - `apps/` holds the externally exported applications, assembled from Client / Host mixtures. @@ -39,7 +39,7 @@ apps/* (applications: apps/web = vite app, apps/cli = bin dispatch) │ consume ▼ packages/host/* packages/client/* - apiproxy front layer: protocol pure libs: ui-slots / web-react / ui-primitives + apiproxy front layer: protocol pure libs: ui-slots / ui-primitives runtime assembly / host entity dsh.client plugins ×8 (node half = empty apply, webserver Web HTTP carriage client half = src/client/) │ ctx.plugin(...) ▲ import only apiproxy's /api /client subpaths @@ -65,8 +65,8 @@ On the protocol side: TS interfaces (`packages/host/apiproxy/src/api/`, zero Nod | Front layer | `dsh-host-apiproxy` | TS/zod definitions (api/) + the fetch abstraction (fetch/: handler + client base class) | Keep it simple — every consumer needs it; importable from Node and browser alike; protocol content in the "Message protocol" sections below; clients must not bypass api through ctx | | Assembly layer | `dsh-host-runtime` | Plugin composition + ApiProxy integration + the web UI plugin mount (in-memory Loader tree over the eight dsh.client packages); home of host-level configuration (defaults/persistenceRoot, future user profile) | Which plugins mount and with what defaults is decided only here; shells must not alter the assembly | | Carrier layer | `dsh-host-webserver` | Web HTTP and upgrade: static serving + `/api/*`→handler forwarding + WebSocket upgrade route + close semantics; plugin bundle endpoint + `__DSH_BOOT__` manifest injection (fed by the web plugin registry) | Web (browser access) only; zero workspace dependencies (the registry arrives by structural injection); Electron does not reuse it | -| Client libraries | `dsh-client-ui-slots` / `dsh-client-web-react` / `dsh-client-ui-primitives` | Slot registry core / ctx↔React glue / pure React atoms | Zero cordis runtime dependency in components; seeded into the loader module table by the shell | -| Client plugins | `dsh-client-connection` / `dsh-client-runtime` / `dsh-client-ui-theme` / `dsh-client-i18n` / `dsh-client-ui-layout` / `dsh-client-ui-sidebar` / `dsh-client-ui-conversation` / `dsh-client-ui-trajectory` | Browser-side cordis plugin tree (wire consumer, core services, theme, i18n, layout, sidebar, conversation, trajectory) — see the web client architecture note | Dual entry (node half = empty apply; implementation in `src/client/`); the consumption face goes exclusively through ApiProxy | +| Client libraries | `dsh-client-ui-slots` / `dsh-client-ui-primitives` | Slot contracts / pure React atoms | Seeded into the loader module table by the shell | +| Client plugins | `dsh-client-connection` / `dsh-client-runtime` / `dsh-client-ui-theme` / `dsh-client-ui-renderer` / feature UI packages | Browser-side Cordis plugin tree: wire consumer, core services, theme, React rendering, and feature composition — see the web client architecture note | Dual entry (node half = empty apply; implementation in `src/client/`); cross-plugin value cooperation uses services and slots | | Application | `@deepseek-ai/dsh` (apps/cli) + `dsh-web-frontend` (apps/web, the vite application) | Coarse bin dispatch + one assembly module per application (web.ts / headless.ts); the vite app is a thin main over the `dsh-client-web` shell surface | Applications use dynamic imports so they never load each other; workspace knowledge like dist location stays in the app | #### Naming rule @@ -209,7 +209,7 @@ The same domain tree as `ApiProxy`, but unary methods **take the business payloa ### The instance-level envelope observation aspect -All four quadrant full forms pass through `onEnvelope`; the base implementation is an **instance-owned microtask-batched buffer** (frame storms must not disturb consumers per frame; module-level state would leak across instances/tests, hence instance-owned). Observers subscribe via `subscribeEnvelopes(listener)` (receiving whole batches as `readonly RpcMessage[]`, returning an unsubscribe function); a listener throw is isolated (observation must never bite the carrier). With no subscribers the buffering costs nothing. No shipped consumer subscribes today — the aspect is the designated seat for wire diagnostics (the retired RPC debug panel was its first consumer, and a future one plugs in without touching the carrier). +All four quadrant full forms pass through `onEnvelope`; the base implementation is an **instance-owned microtask-batched buffer** (frame storms must not disturb consumers per frame; module-level state would leak across instances/tests, hence instance-owned). Observers subscribe via `subscribeEnvelopes(listener)` (receiving whole batches as `readonly RpcMessage[]`, returning an unsubscribe function); a listener throw is isolated (observation must never bite the carrier). With no subscribers the buffering costs nothing. No shipped consumer subscribes — the aspect is the designated seat for wire diagnostics (the retired RPC debug panel was its first consumer, and a future one plugs in without touching the carrier). ### The subclass table (transport carriage) diff --git a/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.zh.md b/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.zh.md index c38328cc2e..ecce57c01c 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.zh.md @@ -4,7 +4,7 @@ Status: implemented [English](2026-07-19-gui-layering-and-rpc-protocol.md) | 中文 -> 分工线:本篇 = 分层模型 + 通道无关的 RPC 协议;协议的 Web 实现由 HTTP 上行加 [WebSocket 下行载体](2026-08-04-websocket-downlink-carrier.md)组成,浏览器对象层见 [Web 客户端架构笔记](2026-07-19-gui-web-client-architecture.md)。 +> 分工线:本篇 = 分层模型 + 通道无关的 RPC 协议;协议的 Web 实现由 HTTP 上行加 [WebSocket 下行载体](2026-08-04-websocket-downlink-carrier.zh.md)组成,浏览器对象层见 [Web 客户端架构笔记](2026-07-19-gui-web-client-architecture.zh.md)。 ## Problem @@ -23,13 +23,13 @@ Status: implemented 目录按照如下分层: - `packages/host/*`:包只提供 Host 侧能力(代表了以现在 Harness 实体插件系统为主体的 Node.js 代码核心工程),除此之外,还包含 - 统一后端协议(fetch、HTTP、流式接口等)定义和支持,见本篇「消息协议」起各节 -- `packages/client/*`:包只提供 Client 侧能力,每包单边不混。这里住三类包(两条轴归 [client 插件装载笔记](2026-07-23-client-plugin-loading-model.md) 所有): - - **纯库**(`ui-slots`、`web-react`、`ui-primitives`,外加内核包 `loader`):普通根入口包,静态打包进壳;前三者播种进模块表。 +- `packages/client/*`:包只提供 Client 侧能力,每包单边不混。这里住三类包(两条轴归 [client 插件装载笔记](2026-07-23-client-plugin-loading-model.zh.md) 所有): + - **纯库**(`ui-slots`、`ui-primitives`,外加内核包 `loader`):普通根入口包,静态打包进壳;两个客户端库播种进模块表。 - **静态到达 entry 包**(`connection`、`runtime`、`ui-theme`、`i18n`、`hmr`):无 `dsh.client` 键、无浏览器 bundle——壳把它们的 `src/client/` 半边打进自己的 bundle 并向 `ctx.modules` 登记;它们与其余单元一样,作为 host 独家撰写的图里的 entry 受治理。 - **fetch 到达插件包**(`ui-layout`、`ui-sidebar`、`ui-conversation`、`ui-trajectory`):双入口——根入口是 node 半边(空 `apply`,其存在是为了让 host Loader 管辖生命周期、让 web 插件注册表发现 package.json 的 `dsh.client` 声明);实现住在 `src/client/` 下,经 `./client` 子路径发布(tsdown 闭包工厂 bundle)。跨插件消费 `/client` 只限类型;值层面的协作走 cordis 服务。 - `apps/` 作为对外导出的应用入口,可以由 Client / Host 混合组装。 - `apps/web`(`dsh-web-frontend`)是 vite 应用:`dsh-client-web` 导出的壳 API 之上的一层薄 `main.ts`。 - - `apps/cli`(`@deepseek-ai/dsh`)分发命令:`dsh web` = Host + webserver + 构建出的 `dsh-web-frontend` dist;`dsh --profile headless` = [直接使用核心 Agent/Session 的入口](2026-08-09-headless-direct-core-entry-point.md),不含 Host、HTTP 或浏览器层。 + - `apps/cli`(`@deepseek-ai/dsh`)分发命令:`dsh web` = Host + webserver + 构建出的 `dsh-web-frontend` dist;`dsh --profile headless` = [直接使用核心 Agent/Session 的入口](2026-08-09-headless-direct-core-entry-point.zh.md),不含 Host、HTTP 或浏览器层。 - 将来的 Electron 应用经由 IPC fetch 载体复用同一套 web client 包。 ``` @@ -37,7 +37,7 @@ apps/* (applications: apps/web = vite app, apps/cli = bin dispatch) │ consume ▼ packages/host/* packages/client/* - apiproxy front layer: protocol pure libs: ui-slots / web-react / ui-primitives + apiproxy front layer: protocol pure libs: ui-slots / ui-primitives runtime assembly / host entity dsh.client plugins ×8 (node half = empty apply, webserver Web HTTP carriage client half = src/client/) │ ctx.plugin(...) ▲ import only apiproxy's /api /client subpaths @@ -50,9 +50,9 @@ harness core packages ──────────────────┘ - `runtime → apiproxy` 单向;apiproxy 仅依赖类型定义。 - client 侧包**永不 import** host 侧包的运行时(只吃 `/api`、`/client` 两个浏览器安全子路径)。 - `webserver` 不依赖 `runtime`:它提供 `{ fetch }` 特定实现 ——「webserver ← runtime」只是运行时注入关系,不是包依赖。 -- client 侧跨包 import 插件包一律走 `/client` 子路径,且插件包之间只限类型 import——跨插件值 import 在 tsdown 纯度门禁处即构建错误(值层面的协作走 cordis 服务;边规则归 [client 插件装载笔记](2026-07-23-client-plugin-loading-model.md) 所有)。 +- client 侧跨包 import 插件包一律走 `/client` 子路径,且插件包之间只限类型 import——跨插件值 import 在 tsdown 纯度门禁处即构建错误(值层面的协作走 cordis 服务;边规则归 [client 插件装载笔记](2026-07-23-client-plugin-loading-model.zh.md) 所有)。 -TypeScript 以 solution 根引用的**两个聚合 program** 检查(`tsconfig.json` = solution;`tsconfig.host.json` = host 侧 + 测试,排除 `packages/client`;`tsconfig.client.json` = client 各包及其测试):两侧在相同键(`sessions`、`loader`)下以不同服务合并 cordis `Context` 接口,单一 program 会同时看到两份声明合并而报冲突。共享叶子包(session/llm/tools/apiproxy 等)只构建一次,由两个 program 共同引用([拓扑](../process/2026-07-22-tsconfig-solution-root-two-aggregates.md))。 +TypeScript 以 solution 根引用的**两个聚合 program** 检查(`tsconfig.json` = solution;`tsconfig.host.json` = host 侧 + 测试,排除 `packages/client`;`tsconfig.client.json` = client 各包及其测试):两侧在相同键(`sessions`、`loader`)下以不同服务合并 cordis `Context` 接口,单一 program 会同时看到两份声明合并而报冲突。共享叶子包(session/llm/tools/apiproxy 等)只构建一次,由两个 program 共同引用([拓扑](../process/2026-07-22-tsconfig-solution-root-two-aggregates.zh.md))。 协议侧:TS interface(`packages/host/apiproxy/src/api/`,零 Node 依赖,浏览器可 import);wire 消息统一为**双向模型**——每条逻辑消息按「谁发起 × request/response」分类(两轴四格,后文称四象限),与物理通道解耦;客户端统一继承 `AbstractApiClient`(协议不变量全在基类,平台差异只是 `doFetch` 传输切面)。 @@ -63,8 +63,8 @@ TypeScript 以 solution 根引用的**两个聚合 program** 检查(`tsconfig. | 前置层 | `dsh-host-apiproxy` | TS/zod 定义 (api/)+ fetch 抽象 (fetch/:handler + 客户端基类) | 做简单、每个消费方都要;Node/浏览器皆可 import;协议内容见下文「消息协议」起各节;client 不得经 ctx 绕开 api | | 装配层 | `dsh-host-runtime` | 插件组合 + ApiProxy 集成 + web UI 插件挂载(覆盖八个 dsh.client 包的内存 Loader 树);host 级配置归属地(defaults/persistenceRoot,将来用户 profile) | 装什么插件、给什么默认值只在这里定;壳不得改装配 | | 承载层 | `dsh-host-webserver` | Web HTTP 与 upgrade:静态服务 + `/api/*`→handler 转发 + WebSocket upgrade route + close 语义;插件 bundle 端点 + `__DSH_BOOT__` manifest(元数据清单)注入(由 web 插件注册表供给) | Web(浏览器访问)专用;零 workspace 依赖(注册表经结构注入到达);Electron 不复用它 | -| client 库 | `dsh-client-ui-slots` / `dsh-client-web-react` / `dsh-client-ui-primitives` | slot 注册表核心 / ctx↔React 胶合 / 纯 React 原子组件 | 组件零 cordis 运行时依赖;由壳播种进 loader 模块表 | -| client 插件 | `dsh-client-connection` / `dsh-client-runtime` / `dsh-client-ui-theme` / `dsh-client-i18n` / `dsh-client-ui-layout` / `dsh-client-ui-sidebar` / `dsh-client-ui-conversation` / `dsh-client-ui-trajectory` | 浏览器侧 cordis 插件树(wire 消费方、核心服务、主题、i18n、布局、侧栏、对话、轨迹)——见 Web 客户端架构笔记 | 双入口(node 半边=空 apply;实现在 `src/client/`);消费面唯一经 ApiProxy | +| client 库 | `dsh-client-ui-slots` / `dsh-client-ui-primitives` | slot 约定 / 纯 React 原子组件 | 由壳播种进 loader 模块表 | +| client 插件 | `dsh-client-connection` / `dsh-client-runtime` / `dsh-client-ui-theme` / `dsh-client-ui-renderer` / 功能 UI 包 | 浏览器侧 Cordis 插件树:wire 消费方、核心服务、主题、React 渲染与功能组合——见 Web 客户端架构笔记 | 双入口(node 半边=空 apply;实现在 `src/client/`);跨插件值协作经服务与 slot 完成 | | 应用 | `@deepseek-ai/dsh`(apps/cli)+ `dsh-web-frontend`(apps/web,vite 应用) | bin 粗分发 + 每个应用一个拼装模块(web.ts / headless.ts);vite 应用是 `dsh-client-web` 壳表面之上的薄 main | 各应用使用动态 import,因此不会互相加载;dist 定位等 workspace 知识留在 app | #### 命名规则 @@ -167,7 +167,7 @@ export type ResponseValue = ### 帧(server→client,具名 union) -两条逻辑流:mux 流(`/api/events.mux`,全 session 聚合)与 host 流(`/api/events.host`,host 级事件)。浏览器通过每流一条下行 WebSocket 消费,进程内 fetch 载体以 SSE 保持同构;物理边界见 [WebSocket 下行载体](2026-08-04-websocket-downlink-carrier.md)。帧示例一行: +两条逻辑流:mux 流(`/api/events.mux`,全 session 聚合)与 host 流(`/api/events.host`,host 级事件)。浏览器通过每流一条下行 WebSocket 消费,进程内 fetch 载体以 SSE 保持同构;物理边界见 [WebSocket 下行载体](2026-08-04-websocket-downlink-carrier.zh.md)。帧示例一行: | 帧 type | 载荷 | 何时发 | |---|---|---| @@ -207,14 +207,14 @@ export type ResponseValue = ### 实例级 envelope 观测切面 -四象限全形均过 `onEnvelope`;基类实现是**实例持有的微任务合批缓冲**(帧风暴不逐帧惊扰消费方;模块级状态会跨实例/测试泄漏,故实例持有)。观测者经 `subscribeEnvelopes(listener)` 订阅(收整批 `readonly RpcMessage[]`,返回退订函数);listener 抛异常被隔离(观测不得反噬载体)。无订阅者时零缓冲成本。当前没有任何现役消费方订阅——该切面是 wire 诊断的预留位(已退役的 RPC 调试面板是它的首个消费方,将来的诊断消费方接入时不动载体)。 +四象限全形均过 `onEnvelope`;基类实现是**实例持有的微任务合批缓冲**(帧风暴不逐帧惊扰消费方;模块级状态会跨实例/测试泄漏,故实例持有)。观测者经 `subscribeEnvelopes(listener)` 订阅(收整批 `readonly RpcMessage[]`,返回退订函数);listener 抛异常被隔离(观测不得反噬载体)。无订阅者时零缓冲成本。没有任何已交付消费方订阅——该切面是 wire 诊断的预留位(已退役的 RPC 调试面板是它的首个消费方,将来的诊断消费方接入时不动载体)。 ### 子类表(传输承载) | 子类 | 所在包 | doFetch | 用途 | |---|---|---|---| | `InProcessApiClient` | apiproxy 本包 | 注入的 `{ fetch }` handler | **同构点**:`new InProcessApiClient(toFetchHandler(api))` 全程不过网络但真跑 wire 序列化/zod/SSE 帧;载体测试与调用方可以在不打开端口的情况下运行这套协议,而产品 `dsh --profile headless` 直接驱动 core | -| `WebApiClient` | dsh-client-connection | `globalThis.fetch` 上行 + 每逻辑流一条同源 WebSocket 下行 | 浏览器客户端;物理边界见 [WebSocket 下行载体](2026-08-04-websocket-downlink-carrier.md) | +| `WebApiClient` | dsh-client-connection | `globalThis.fetch` 上行 + 每逻辑流一条同源 WebSocket 下行 | 浏览器客户端;物理边界见 [WebSocket 下行载体](2026-08-04-websocket-downlink-carrier.zh.md) | | `FixtureApiClient` | dsh-client-connection | 不用(协议层覆写) | 无 server 的 UI 开发(`?fixture`):覆写 `callUnary`/`openMux`/`openHost`/`respond` 虚方法,自己就是假 server(帧 rpcId 由它 mint,语义自洽) | | IPC 桥子类(假想示例——尚无此形态) | Electron 壳 | IPC 序列化往返 | 只需换 doFetch,约定/基类零改 | diff --git a/.agents/notes/implemented/architecture/2026-07-19-gui-web-client-architecture.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-19-gui-web-client-architecture.i18n.yaml index c48579cb0d..9ee365f446 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-gui-web-client-architecture.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-19-gui-web-client-architecture.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-19-gui-web-client-architecture.md -2026-07-19-gui-web-client-architecture.md: 070b857f14007f429826ab83b17b5c8fbd3d3d0b -2026-07-19-gui-web-client-architecture.zh.md: 37e082985b3e6dbf3effd33a5aaeb54fce96d713 +2026-07-19-gui-web-client-architecture.md: 8b4f940299cbba78d403c34b1e5fc9740e44f2c2 +2026-07-19-gui-web-client-architecture.zh.md: 705b1337dd97ac37bd01bdcc5aa22484b7971908 diff --git a/.agents/notes/implemented/architecture/2026-07-19-gui-web-client-architecture.md b/.agents/notes/implemented/architecture/2026-07-19-gui-web-client-architecture.md index 070b857f14..8b4f940299 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-gui-web-client-architecture.md +++ b/.agents/notes/implemented/architecture/2026-07-19-gui-web-client-architecture.md @@ -22,23 +22,23 @@ Both ends run cordis. The host is a cordis plugin tree; the browser runs a secon │ ├ GET /plugins//client.js │ │ │ ui-theme/i18n(fetch bundle,boot 预拉) │ │ └ GET / 注入 __DSH_BOOT__ 图 │ │ ├ lazy entries: layout/sidebar/ │ │ │ │ │ conversation/trajectory(fetch bundle,按需) │ -└────────────────────────────────┘ │ ├ app-shell 伪行(壳内静态注册,同一治理) │ +└────────────────────────────────┘ │ ├ ui-renderer(fetch bundle,React 根) │ │ └ session scope ×N(观看驱动,惰性建) │ - │ React: loading 页 → settled → 整 UI 一次成型 │ + │ DOM loading 页 → settled → React UI 一次成型 │ └────────────────────────────────────────────────────┘ ``` ## The client cordis tree and the loading chain -The loading chain — the two package kinds (plain vs dsh.client plugin), the module-system/plugin-governor split, the two-phase boot over the host-authored entry graph with revisions, and hot reload — is owned by the [client plugin loading note](2026-07-23-client-plugin-loading-model.md). The load-bearing facts for this document: the browser boots the same vendored `@cordisjs/plugin-loader` as the host with a client module system (`ctx.modules`, `packages/client/modules`) filling its `internal` contract; every unit with product behavior is an entry in the host-authored `__DSH_BOOT__` graph — every production plugin package (infrastructure included) carries the `dsh.client` declaration and arrives as a fetched `./client` tsdown closure bundle, `immediately` rows differing only in boot phase-one prefetch, while plain packages (react family, cordis, the not-yet-promoted libraries) stay shell-bundled, seeded, and invisible to the graph; bundles execute `window.__ModuleLoader__.load({ id, factory })` and their `require` is answered from the lazy CJS module table (seed words + registered factories, materialized and memoized on first require — cross-plugin value imports are a build error, cooperation goes through cordis services); plugin CSS is inlined in the bundle and injected as `` } + case 'html': + return { placement: row.placement, markup: row.html } + default: + return assertNever(row) + } +} + +/** Insert `markup` into `html` at `at`. */ +function splice(html: string, at: number, markup: string): string { + return `${html.slice(0, at)}${markup}${html.slice(at)}` +} + +/** + * Tail script settling the boot-readiness deferred (`__DSH_BOOT_READY__`): + * the client entry awaits its `.promise` before reading any injected state. + * Whichever side runs first creates the deferred (`??=`), so a bootstrap that + * applies the table asynchronously installs it ahead of the entry module and + * settles it after the last row; the served form below creates and resolves + * it in one statement, because every row is already in the document text. + */ +const READY_MARKUP = '' + +/** + * Render rows into an index.html body: head rows immediately after the + * opening head tag, body rows immediately after the opening body tag, each + * group in table order, and the boot-readiness tail after the last body row. + * @param html - the raw index.html body. + * @param rows - the collected injection table. + * @returns the html with every row rendered. + */ +export function renderIndexInjections(html: string, rows: readonly IndexInjection[]): string { + let head = '' + let body = '' + for (const row of rows) { + const rendered = renderRow(row) + if (rendered.placement === 'head') head += rendered.markup + else body += rendered.markup + } + body += READY_MARKUP + let out = html + if (head !== '') { + const open = /]*)?>/i.exec(out) + // Headless fixture pages may lack ; prepending keeps the rows ahead + // of every document script. + out = open === null ? `${head}${out}` : splice(out, open.index + open[0].length, head) + } + if (body !== '') { + const open = /]*)?>/i.exec(out) + // Body-less fragments receive the rows at the end, where the HTML parser + // has already synthesized a body. + out = open === null ? `${out}${body}` : splice(out, open.index + open[0].length, body) + } + return out +} diff --git a/packages/host/webserver/tests/webserver.spec.ts b/packages/host/webserver/tests/webserver.spec.ts index 2cbd285856..e8fa315ecc 100644 --- a/packages/host/webserver/tests/webserver.spec.ts +++ b/packages/host/webserver/tests/webserver.spec.ts @@ -15,7 +15,7 @@ import { afterEach, describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' import Include from '@deepseek-ai/cordis-plugin-include' -import HttpServer from '../src/index.ts' +import HttpServer, { renderIndexInjections } from '../src/index.ts' let root: string | undefined let context: Context | undefined @@ -200,6 +200,56 @@ describe('real Loader composition', () => { await expect(request(port, '/probe')).rejects.toThrow() }) + it('collects injection rows fresh per render and layers taps over the rendered rows', { timeout: 60_000 }, async () => { + const loaded = await loadComposition() + const server = loaded.webServer + let flag = 'dark' + loaded.on('webserver/index-inject', (table) => { + table.push( + { kind: 'script', placement: 'head', text: 'window.__Q__=1' }, + { kind: 'script-src', placement: 'head', src: '/plugins/a.js?rev="1"&x=' }, + { kind: 'global', name: '__DSH_BOOT__', value: { rev: '' } }, + { kind: 'style', text: 'body{margin:0}' }, + { kind: 'html', placement: 'head', html: '' }, + { kind: 'script', placement: 'body', text: `window.__P__=${JSON.stringify(flag)}` }, + ) + }) + + const html = server.renderIndex('shell') + // Head rows land right after the opening head tag in table order; the body + // row lands right after the opening body tag. + const order = [ + '', + '', + '', + 'globalThis["__DSH_BOOT__"] = {"rev":"\\u003c/script>\\u003cb>"}', + '', + '', + '', + '', + 'shell', + ].map(part => html.indexOf(part)) + expect(order).toEqual([...order].sort((a, b) => a - b)) + expect(order.every(at => at !== -1)).toBe(true) + + // Fresh collection per render: the listener reads live state at emit time. + flag = 'light' + expect(server.renderIndex('')).toContain('window.__P__="light"') + + // Raw taps still run, over the already-rendered rows. + const untap = server.tapIndex(h => h.replace('window.__Q__=1', 'window.__Q__=2')) + expect(server.renderIndex('')).toContain('window.__Q__=2') + untap() + + // Tag-less fragments: head rows prepend, body rows append, and the + // boot-readiness tail lands after the last body row. + expect(renderIndexInjections('
x
', [ + { kind: 'script', placement: 'head', text: 'H' }, + { kind: 'script', placement: 'body', text: 'B' }, + ])).toBe('
x
' + + '') + }) + it('fails the fiber when the port is already taken (fail-loud at activation)', { timeout: 60_000 }, async () => { const first = await loadComposition() const takenPort = first.webServer.port diff --git a/packages/identity/README.i18n.yaml b/packages/identity/README.i18n.yaml index 653704a83c..f1f3b0c384 100644 --- a/packages/identity/README.i18n.yaml +++ b/packages/identity/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/identity/README.md README.md: ebbc7d937dfa793edf9a8617d86a1820e87866da -README.zh.md: 3d29fc3dee7599e366cb91c09d6a73c8acddab96 +README.zh.md: cd5fa06a0e9c6c277292b5904a1617acaa8dcfc3 diff --git a/packages/identity/README.zh.md b/packages/identity/README.zh.md index 3d29fc3dee..cd5fa06a0e 100644 --- a/packages/identity/README.zh.md +++ b/packages/identity/README.zh.md @@ -6,4 +6,4 @@ | 包 | 职责 | ctx key | |---|---|---| -| [`anonymous-user-id/`](anonymous-user-id/README.md) | 为遥测、反馈和 DeepSeek 请求持久化一个限定于 Harness home 的匿名关联 id | — | +| [`anonymous-user-id/`](anonymous-user-id/README.zh.md) | 为遥测、反馈和 DeepSeek 请求持久化一个限定于 Harness home 的匿名关联 id | — | diff --git a/packages/identity/anonymous-user-id/package.json b/packages/identity/anonymous-user-id/package.json index 604b308f15..35019e6c7b 100644 --- a/packages/identity/anonymous-user-id/package.json +++ b/packages/identity/anonymous-user-id/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-anonymous-user-id", "description": "Shared anonymous user identity for DeepSeek Harness telemetry and feedback correlation", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/interaction/README.i18n.yaml b/packages/interaction/README.i18n.yaml index 116db6b260..7086f59493 100644 --- a/packages/interaction/README.i18n.yaml +++ b/packages/interaction/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/interaction/README.md README.md: a7842e40aa708ee9ec159dd51b8a94a6b2b18539 -README.zh.md: ec329365331efd2769078078d1da4a8e1b3b57e0 +README.zh.md: 1a760484f887a459348612c867213d95e7257420 diff --git a/packages/interaction/README.zh.md b/packages/interaction/README.zh.md index ec32936533..1a760484f8 100644 --- a/packages/interaction/README.zh.md +++ b/packages/interaction/README.zh.md @@ -6,12 +6,12 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`commands/`](commands/README.md) | 为交互式适配器注册并分派用户命令。 | `ctx.commands` | -| [`user-approval/`](user-approval/README.md) | 协调一次性审批决策。 | `ctx.approval` | -| [`permission/`](permission-presets/README.md) | 呈现并持久化面向用户的权限预设。 | `ctx.permissionPresets` | -| [`user-questions/`](user-questions/README.md) | 定义与提供方无关的用户问答 seam。 | `ctx.userQuestions` | -| [`tool-ask-user/`](tool-ask-user/README.md) | 向模型提供用户问题。 | (注册到 `ctx.tools`) | +| [`commands/`](commands/README.zh.md) | 为交互式适配器注册并分派用户命令。 | `ctx.commands` | +| [`user-approval/`](user-approval/README.zh.md) | 协调一次性审批决策。 | `ctx.approval` | +| [`permission/`](permission-presets/README.zh.md) | 呈现并持久化面向用户的权限预设。 | `ctx.permissionPresets` | +| [`user-questions/`](user-questions/README.zh.md) | 定义与提供方无关的用户问答 seam。 | `ctx.userQuestions` | +| [`tool-ask-user/`](tool-ask-user/README.zh.md) | 向模型提供用户问题。 | (注册到 `ctx.tools`) | -这些包通过现有的 agent 和会话约定集成,而不改变循环。交互式应用提供具体的命令、审批和提问适配器;自动化使用 [`acp/`](../acp/README.md),可运行的演示组合包位于 [`examples/`](../examples/README.md)。产品 [`dsh`](../../apps/cli/README.md) CLI(命令行界面)直接组合这些包。 +这些包通过现有的 agent 和会话约定集成,而不改变循环。交互式应用提供具体的命令、审批和提问适配器;自动化使用 [`acp/`](../acp/README.zh.md),可运行的演示组合包位于 [`examples/`](../examples/README.zh.md)。产品 [`dsh`](../../apps/cli/README.zh.md) CLI(命令行界面)直接组合这些包。 -子系统参考:[approval.md](../../docs/subsystems/approval.md)、[permission-presets.md](../../docs/subsystems/permission-presets.md)、[user-questions.md](../../docs/subsystems/user-questions.md)与 [commands.md](../../docs/subsystems/commands.md)。仅自动化的 ACP 传输是 [`acp/`](../acp/README.md),SDK 的 JSON-RPC 服务器端是 [`sdk/server`](../sdk/README.md),共享 bin 启动胶水是 [`boot/`](../boot/README.md)。 +子系统参考:[approval.md](../../docs/subsystems/approval.zh.md)、[permission-presets.md](../../docs/subsystems/permission-presets.zh.md)、[user-questions.md](../../docs/subsystems/user-questions.zh.md)与 [commands.md](../../docs/subsystems/commands.zh.md)。仅自动化的 ACP 传输是 [`acp/`](../acp/README.zh.md),SDK 的 JSON-RPC 服务器端是 [`sdk/server`](../sdk/README.zh.md),共享 bin 启动胶水是 [`boot/`](../boot/README.zh.md)。 diff --git a/packages/interaction/commands/README.i18n.yaml b/packages/interaction/commands/README.i18n.yaml index 9a53413b85..cde56d4ef9 100644 --- a/packages/interaction/commands/README.i18n.yaml +++ b/packages/interaction/commands/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/interaction/commands/README.md -README.md: e1d7e5f4f626d062a16840bce6354ee3aac921c9 -README.zh.md: 6ce3ca9016e26537003ba85e0b2217bd37448fc5 +README.md: 4a4cb2a70b56ba1a18e9f4719541a50a9683510c +README.zh.md: d331e3e2f359572e4953831ea0c4dbe8d2fd64c3 diff --git a/packages/interaction/commands/README.md b/packages/interaction/commands/README.md index e1d7e5f4f6..4a4cb2a70b 100644 --- a/packages/interaction/commands/README.md +++ b/packages/interaction/commands/README.md @@ -6,9 +6,9 @@ Plugin-owned human-command registry consumed by interactive UI adapters. The [pl ## Service contract -`ctx.commands.register(definition)` registers one lowercase command name, description, optional unstructured-input hint, optional `recordInput` policy, and abortable handler. `recordInput` defaults to true; a command whose authoritative domain event owns the payload sets it to false so `command/run` omits `args` instead of duplicating the input. A registered command is available to every composed command adapter; a plugin that is incompatible with a deployment does not register there. A plain-context registration is global. A command-producing plugin mounted beneath `agent.ctx` declares its own `commands` injection and creates an exact agent-scoped definition; it shadows a global definition with the same name. This child-injection shape preserves the agent scope without making the core agent loop depend on a UI service. Duplicate names within one layer fail during registration. Every disposer is the exact Cordis effect disposer, and registration or removal notifies every `commands/change` observer so live adapters can refresh discovery; observer failures are logged and cannot veto the registry mutation or starve later observers. +`ctx.commands.register(definition)` registers one lowercase command name, description, optional unstructured-input descriptor (`hint` plus an `images` flag declaring whether composer image attachments may accompany an invocation), optional `recordInput` policy, and abortable handler. `recordInput` defaults to true; a command whose authoritative domain event owns the payload sets it to false so `command/run` omits `args` instead of duplicating the input. A registered command is available to every composed command adapter; a plugin that is incompatible with a deployment does not register there. A plain-context registration is global. A command-producing plugin mounted beneath `agent.ctx` declares its own `commands` injection and creates an exact agent-scoped definition; it shadows a global definition with the same name. This child-injection shape preserves the agent scope without making the core agent loop depend on a UI service. Duplicate names within one layer fail during registration. Every disposer is the exact Cordis effect disposer, and registration or removal notifies every `commands/change` observer so live adapters can refresh discovery; observer failures are logged and cannot veto the registry mutation or starve later observers. -`list(agent)` returns immutable, name-sorted descriptors after scoped shadowing. `find(agent, name)` returns the corresponding definition. `execute(agent, line, signal)` uses `parseCommand()` and runs only a known command, returning the settled `CommandExecution` (the normalized result plus the lifecycle pairing `commandId`) or `undefined` for invalid syntax or unknown names. A resolved command's lifecycle is logged on the receiving agent's session as the log-only pair `command/run` (before the handler, with a minted `commandId`, the parser's structured name, the issuing `CommandSource`, and `args` unless `recordInput` is false) and `command/done` (at settlement, with the outcome kind and verbatim text; a successful result may also name an earlier non-command authoritative domain event through `sourceEventSeq`; a thrown or aborted handler settles as `kind: 'error'`). Admission misses log nothing. Both are direct standalone appends on the receiving agent's session: no turn wraps them, and persistence drains them through ordinary checkpoints and teardown. +`list(agent)` returns immutable, name-sorted descriptors after scoped shadowing (the descriptor carries `input.images` so composers can refuse image submissions to non-declaring commands before dispatch). `find(agent, name)` returns the corresponding definition. `execute(agent, line, images, signal)` uses `parseCommand()` and runs only a known command, returning the settled `CommandExecution` (the normalized result plus the lifecycle pairing `commandId`) or `undefined` for invalid syntax or unknown names. `images` carries the submission's base64-encoded composer images (`EncodedImageAttachment` from `@deepseek-ai/dsh-attachment/types`); the executor enforces the declaration — images sent to a non-declaring command, an absent `attachments` store, or an exceeded batch limit each settle as an error result before the handler runs, and a rejected batch publishes no durable object. An admitted batch is committed through `admitEncodedImages` and handed to the handler as frozen ordered `ImageBlock`s on `invocation.attachments`; the handler owns their model-visible use and returns an error when its grammar cannot use them, so the dispatching composer keeps the originals. A resolved command's lifecycle is logged on the receiving agent's session as the log-only pair `command/run` (before the handler, with a minted `commandId`, the parser's structured name, the issuing `CommandSource`, and `args` unless `recordInput` is false) and `command/done` (at settlement, with the outcome kind and verbatim text; a successful result may also name an earlier non-command authoritative domain event through `sourceEventSeq`; a thrown or aborted handler settles as `kind: 'error'`). Admission misses log nothing. Both are direct standalone appends on the receiving agent's session: no turn wraps them, and persistence drains them through ordinary checkpoints and teardown. `parseCommand()` recognizes a slash at byte zero, a lowercase name containing letters, digits, `_`, or `-`, and either end-of-input or whitespace. It returns every byte after the name as `rawInput`, including separator whitespace; consumers own their command-specific grammar and may normalize only what that grammar permits. @@ -24,7 +24,7 @@ The shipped `dsh` base mounts this service and the Web client dispatches through #### What the model sees -The registry itself submits nothing. Known slash commands execute in the UI command plane, and their `CommandResult` text is not submitted as a user message. Unknown slash-command input is rejected by shipped adapters instead of becoming a model prompt. A command producer may explicitly use the receiving `Agent`; for example, [`dsh-plan-mode`](../../plan/plan-mode/README.md#model-and-human-interactions) submits the optional message in `/plan [message]` after selecting plan mode. +The registry itself submits nothing. Known slash commands execute in the UI command plane, and their `CommandResult` text is not submitted as a user message. Unknown slash-command input is rejected by shipped adapters instead of becoming a model prompt. A command producer may explicitly use the receiving `Agent`; for example, [`dsh-plan-mode`](../../plan/plan-mode/README.md#model-and-human-interactions) submits the optional message in `/plan [message]` after selecting plan mode. Image attachments follow the same rule: the executor only admits them into durable attachment objects, and a declaring producer decides whether and how they become model-visible message content. #### Token effect diff --git a/packages/interaction/commands/README.zh.md b/packages/interaction/commands/README.zh.md index 6ce3ca9016..d331e3e2f3 100644 --- a/packages/interaction/commands/README.zh.md +++ b/packages/interaction/commands/README.zh.md @@ -2,13 +2,13 @@ [English](README.md) | 中文 -由插件负责、供交互式 UI 适配器使用的面向用户命令注册表。[插件命令注册 Agent Note](../../../.agents/notes/implemented/feature/2026-07-19-plugin-command-registration.md)定义了其边界与分发约定。 +由插件负责、供交互式 UI 适配器使用的面向用户命令注册表。[插件命令注册 Agent Note](../../../.agents/notes/implemented/feature/2026-07-19-plugin-command-registration.zh.md)定义了其边界与分发约定。 ## 服务约定 -`ctx.commands.register(definition)` 注册一个小写命令名称、描述、可选的非结构化输入提示、可选的 `recordInput` 策略,以及可中止的处理器。`recordInput` 默认为 true;若载荷由命令的权威领域事件持有,该命令会将 `recordInput` 设为 false,让 `command/run` 省略 `args`,避免重复记录输入。每个已注册命令都可供所有已组合的命令适配器使用;与某项部署不兼容的插件不会在此注册。普通上下文中的注册全局生效。在 `agent.ctx` 下挂载的命令生产插件会声明自身的 `commands` 注入,并创建精确限定到该 agent(智能体)的定义;该定义会遮蔽同名的全局定义。这种子级注入形态保留了 agent 作用域,同时不会让核心 agent loop(智能体循环)依赖 UI 服务。同一层中的名称重复会在注册时失败。每个 disposer 都是 Cordis effect 返回的确切 disposer;注册或移除命令时,系统会通知每个 `commands/change` 观察者,使运行中的适配器能够刷新发现结果。观察者失败会写入日志,既不能否决注册表变更,也不能阻止后续观察者运行。 +`ctx.commands.register(definition)` 注册一个小写命令名称、描述、可选的非结构化输入描述符(`hint`,以及声明调用是否可携带 composer 图片附件的 `images` 标志)、可选的 `recordInput` 策略,以及可中止的处理器。`recordInput` 默认为 true;若载荷由命令的权威领域事件持有,该命令会将 `recordInput` 设为 false,让 `command/run` 省略 `args`,避免重复记录输入。每个已注册命令都可供所有已组合的命令适配器使用;与某项部署不兼容的插件不会在此注册。普通上下文中的注册全局生效。在 `agent.ctx` 下挂载的命令生产插件会声明自身的 `commands` 注入,并创建精确限定到该 agent(智能体)的定义;该定义会遮蔽同名的全局定义。这种子级注入形态保留了 agent 作用域,同时不会让核心 agent loop(智能体循环)依赖 UI 服务。同一层中的名称重复会在注册时失败。每个 disposer 都是 Cordis effect 返回的确切 disposer;注册或移除命令时,系统会通知每个 `commands/change` 观察者,使运行中的适配器能够刷新发现结果。观察者失败会写入日志,既不能否决注册表变更,也不能阻止后续观察者运行。 -`list(agent)` 在应用作用域遮蔽后,返回按名称排序的不可变描述符。`find(agent, name)` 返回相应定义。`execute(agent, line, signal)` 使用 `parseCommand()`,且只运行已知命令,返回已结算的 `CommandExecution`(规范化结果加生命周期配对 `commandId`);语法无效或名称未知时返回 `undefined`。已解析命令的生命周期会以 log-only 事件对的形式记录在接收 agent 的会话日志中:`command/run`(进入处理器前记录,携带新生成的 `commandId`、解析器的结构化名称、发起方 `CommandSource`,以及 `args`(`recordInput` 为 false 时省略))与 `command/done`(结算时记录,携带结果类型与原样文本;成功结果还可通过 `sourceEventSeq` 指向更早的一条非命令权威领域事件;处理器抛出或被中止时以 `kind: 'error'` 结算)。未通过准入的输入不记录任何事件。两者都直接独立追加到接收 agent 的会话中:没有轮次包裹它们,持久化机制会在常规检查点和销毁期间排空这些事件。 +`list(agent)` 在应用作用域遮蔽后,返回按名称排序的不可变描述符(描述符携带 `input.images`,使 composer 能在分发前就拒绝把图片提交给未声明的命令)。`find(agent, name)` 返回相应定义。`execute(agent, line, images, signal)` 使用 `parseCommand()`,且只运行已知命令,返回已结算的 `CommandExecution`(规范化结果加生命周期配对 `commandId`);语法无效或名称未知时返回 `undefined`。`images` 携带本次提交的 base64 编码 composer 图片(来自 `@deepseek-ai/dsh-attachment/types` 的 `EncodedImageAttachment`);执行器负责声明的强制执行:把图片发给未声明的命令、`attachments` 存储缺失、或批量超出限制,都会在处理器运行前以错误结果结算,被拒绝的批量不会发布任何持久化对象。通过准入的批量经 `admitEncodedImages` 提交,并以冻结的有序 `ImageBlock` 数组挂在 `invocation.attachments` 上交给处理器;处理器负责它们的模型可见用途,当其语法无法使用这些图片时返回错误,使分发方 composer 保留原件。已解析命令的生命周期会以 log-only 事件对的形式记录在接收 agent 的会话日志中:`command/run`(进入处理器前记录,携带新生成的 `commandId`、解析器的结构化名称、发起方 `CommandSource`,以及 `args`(`recordInput` 为 false 时省略))与 `command/done`(结算时记录,携带结果类型与原样文本;成功结果还可通过 `sourceEventSeq` 指向更早的一条非命令权威领域事件;处理器抛出或被中止时以 `kind: 'error'` 结算)。未通过准入的输入不记录任何事件。两者都直接独立追加到接收 agent 的会话中:没有轮次包裹它们,持久化机制会在常规检查点和销毁期间排空这些事件。 `parseCommand()` 识别位于第 0 字节的斜杠、由小写字母、数字、`_` 或 `-` 构成的名称,以及名称后紧接输入末尾或空白的形式。它将名称后的每个字节作为 `rawInput` 返回,其中包括分隔空白;消费方负责各命令专用的语法,只能执行该语法允许的规范化。 @@ -24,7 +24,7 @@ #### 模型看到的内容 -注册表自身不会提交任何内容。已知斜杠命令在 UI 命令平面执行,其 `CommandResult` 文本不会作为用户消息提交。已交付的适配器会拒绝未知斜杠命令输入,而不是将其变成模型提示词。命令生产方可以显式使用接收命令的 `Agent`;例如,[`dsh-plan-mode`](../../plan/plan-mode/README.md#model-and-human-interactions)在选择 plan mode 后,会提交 `/plan [message]` 中的可选消息。 +注册表自身不会提交任何内容。已知斜杠命令在 UI 命令平面执行,其 `CommandResult` 文本不会作为用户消息提交。已交付的适配器会拒绝未知斜杠命令输入,而不是将其变成模型提示词。命令生产方可以显式使用接收命令的 `Agent`;例如,[`dsh-plan-mode`](../../plan/plan-mode/README.zh.md#model-and-human-interactions)在选择 plan mode 后,会提交 `/plan [message]` 中的可选消息。图片附件遵循同一规则:执行器只负责把它们准入为持久化附件对象,是否以及如何成为模型可见的消息内容由声明接受的生产方决定。 #### Token 影响 diff --git a/packages/interaction/commands/package.json b/packages/interaction/commands/package.json index c08b1294a4..f5eebc76d1 100644 --- a/packages/interaction/commands/package.json +++ b/packages/interaction/commands/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-commands", "description": "Plugin-owned human command registry for DeepSeek Harness UIs", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -54,20 +54,25 @@ "license": "MIT", "peerDependencies": { "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-attachment": "workspace:^", "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-scope": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-typert-protocol": "workspace:^", "@deepseek-ai/cordis": "workspace:^" }, "dependencies": { + "@deepseek-ai/dsh-util-crypto": "workspace:^", "zod": "^4.4.3" }, "devDependencies": { "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-attachment": "workspace:^", "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-scope": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-typert-protocol": "workspace:^", diff --git a/packages/interaction/commands/src/index.ts b/packages/interaction/commands/src/index.ts index 4e179435bb..b0b3a0dd45 100644 --- a/packages/interaction/commands/src/index.ts +++ b/packages/interaction/commands/src/index.ts @@ -4,7 +4,11 @@ */ import { Context } from '@deepseek-ai/cordis' +import { randomUUID } from '@deepseek-ai/dsh-util-crypto' import type { Agent } from '@deepseek-ai/dsh-agent' +import { AttachmentError, admitEncodedImages } from '@deepseek-ai/dsh-attachment' +import type { EncodedImageAttachment } from '@deepseek-ai/dsh-attachment/types' +import type { ImageBlock } from '@deepseek-ai/dsh-llm' import { NamedEntries, ScopedLayers } from '@deepseek-ai/dsh-scope' import type { ScopeKey, ScopeLayer } from '@deepseek-ai/dsh-scope' import type { Session, SessionEvent, SessionEventMap } from '@deepseek-ai/dsh-session' @@ -24,6 +28,9 @@ export const name = 'commands' const COMMAND_NAME = /^[a-z][a-z0-9_-]*$/u +/** Shared frozen attachments value for image-free invocations. */ +const NO_ATTACHMENTS: readonly ImageBlock[] = Object.freeze([]) + /** Invocation passed to one registered command handler. */ export interface CommandInvocation { /** Pairing id already written to this invocation's `command/run` event. */ @@ -32,6 +39,14 @@ export interface CommandInvocation { readonly agent: Agent /** Exact text following the registered command name, including separator whitespace. */ readonly rawInput: string + /** + * Durably admitted image blocks accompanying this invocation, in submission + * order; empty unless the definition declares `input.images`. The handler + * owns their model-visible use — the registry never schedules them itself — + * and a handler whose grammar cannot use them in this invocation returns an + * error so the dispatching composer retains the originals. + */ + readonly attachments: readonly ImageBlock[] /** Cancellation signal owned by the dispatching UI request. */ readonly signal: AbortSignal } @@ -114,6 +129,11 @@ function abortError(signal: AbortSignal): Error { return new Error(typeof signal.reason === 'string' ? signal.reason : 'command aborted') } +/** The signal's normalized abort error when it is already aborted. */ +function cancellationOf(signal: AbortSignal): Error | undefined { + return signal.aborted ? abortError(signal) : undefined +} + /** Render arbitrary thrown values without trusting their string coercion. */ function renderThrown(value: unknown): string { try { @@ -171,7 +191,13 @@ function normalizeDefinition(definition: CommandDefinition): RegisteredCommand { if (rawInput.hint.trim().length === 0) { throw new TypeError(`command "${definition.name}" input hint must not be empty`) } - input = Object.freeze({ hint: rawInput.hint }) + if ('images' in rawInput && rawInput.images !== undefined && typeof rawInput.images !== 'boolean') { + throw new TypeError(`command "${definition.name}" input images flag must be a boolean`) + } + input = Object.freeze({ + hint: rawInput.hint, + ...('images' in rawInput && rawInput.images === true) ? { images: true } : {}, + }) } const normalized = Object.freeze({ name: definition.name, @@ -231,7 +257,7 @@ export class CommandRuntime extends TypertRemoteService { /** Monotonic per-instance counter behind {@link mintCommandId}. */ private commandSeq = 0 /** Instance token keeping minted ids unique across process restarts over one resumed log. */ - private readonly instanceToken = crypto.randomUUID().slice(0, 8) + private readonly instanceToken = randomUUID().slice(0, 8) constructor(ctx: Context) { super(ctx, 'commands') @@ -287,8 +313,15 @@ export class CommandRuntime extends TypertRemoteService { * handler-failure path is contained so the handler's own error stays the * reported failure. * + * Image admission is enforced here, not in the composer: images sent to a + * command that does not declare `input.images`, an absent attachment store, + * and an exceeded attachment limit each settle as an error result before + * the handler runs, and a rejected batch publishes no durable object. + * * @param agent - exact receiving agent. * @param line - complete slash-command line. + * @param images - base64-encoded composer images accompanying the line, in + * submission order; empty for a plain invocation. * @param signal - cancellation signal owned by the UI request. * @returns the settled execution (result + lifecycle pairing id), or * `undefined` when syntax or name does not resolve. @@ -297,6 +330,7 @@ export class CommandRuntime extends TypertRemoteService { async execute( agent: Agent, line: string, + images: readonly EncodedImageAttachment[], signal: AbortSignal, ): Promise { const parsed = parseCommand(line) @@ -311,30 +345,67 @@ export class CommandRuntime extends TypertRemoteService { ...command.definition.recordInput === false ? {} : { args: parsed.rawInput }, source: { kind: 'user' }, }) - const invocation = Object.freeze({ commandId, agent, rawInput: parsed.rawInput, signal }) + const settle = (result: CommandResult): CommandExecution => { + this.appendLifecycle(agent.session, 'command/done', { + commandId, kind: result.kind, + ...result.text === undefined ? {} : { text: result.text }, + ...result.kind === 'success' && result.sourceEventSeq !== undefined + ? { sourceEventSeq: result.sourceEventSeq } + : {}, + }) + return Object.freeze({ commandId, result: Object.freeze(result) }) + } + let attachments: readonly ImageBlock[] = NO_ATTACHMENTS + if (images.length > 0) { + if (command.definition.input?.images !== true) { + return settle({ kind: 'error', text: `/${parsed.name} does not accept image attachments` }) + } + const store = this.ctx.get('attachments') + if (store === undefined) { + return settle({ kind: 'error', text: `/${parsed.name}: image attachments are unavailable because no attachment store is composed` }) + } + try { + const refs = await admitEncodedImages(store, images) + attachments = Object.freeze(refs.map(ref => Object.freeze({ type: 'image' as const, attachment: ref }))) + } catch (error: unknown) { + if (error instanceof AttachmentError) { + return settle({ kind: 'error', text: error.message }) + } + this.settleThrown(agent.session, parsed.name, commandId, error) + throw error + } + // Cancellation must be honored BEFORE the handler runs: admission may + // await slow storage, and a handler entered after the caller cancelled + // would mutate state the retrying caller then duplicates. (The committed + // image objects stay unreferenced and are deferred-GC territory.) + const cancelledDuringAdmission = cancellationOf(signal) + if (cancelledDuringAdmission !== undefined) { + this.settleThrown(agent.session, parsed.name, commandId, cancelledDuringAdmission) + throw cancelledDuringAdmission + } + } + const invocation = Object.freeze({ commandId, agent, rawInput: parsed.rawInput, attachments, signal }) let result: CommandResult try { const output = command.definition.handler(invocation) result = normalizeResult(parsed.name, await withAbort(Promise.resolve(output), signal)) } catch (error: unknown) { - try { - this.appendLifecycle(agent.session, 'command/done', { - commandId, kind: 'error', - text: error instanceof Error ? error.message : renderThrown(error), - }) - } catch (appendError: unknown) { - this.ctx.logger.warn(`command "${parsed.name}": command/done append failed: ${renderThrown(appendError)}`) - } + this.settleThrown(agent.session, parsed.name, commandId, error) throw error } - this.appendLifecycle(agent.session, 'command/done', { - commandId, kind: result.kind, - ...result.text === undefined ? {} : { text: result.text }, - ...result.kind === 'success' && result.sourceEventSeq !== undefined - ? { sourceEventSeq: result.sourceEventSeq } - : {}, - }) - return Object.freeze({ commandId, result }) + return settle(result) + } + + /** Contained `command/done` error append for a thrown handler or admission failure. */ + private settleThrown(session: Session, command: string, commandId: CommandId, error: unknown): void { + try { + this.appendLifecycle(session, 'command/done', { + commandId, kind: 'error', + text: error instanceof Error ? error.message : renderThrown(error), + }) + } catch (appendError: unknown) { + this.ctx.logger.warn(`command "${command}": command/done append failed: ${renderThrown(appendError)}`) + } } /** Mint the next pairing id (monotonic; instance-token-prefixed so a resumed log never repeats one). */ diff --git a/packages/interaction/commands/src/types.ts b/packages/interaction/commands/src/types.ts index 32f1dbcc43..621a5971ba 100644 --- a/packages/interaction/commands/src/types.ts +++ b/packages/interaction/commands/src/types.ts @@ -13,6 +13,14 @@ import type { CommandId } from './brand.ts' export interface CommandInputDescriptor { /** Placeholder shown before the user supplies free-form input. */ readonly hint: string + /** + * Whether composer image attachments may accompany an invocation. Absent or + * false = the executor rejects an invocation carrying images and capable + * composers refuse the submission before dispatch. A declaring command's + * handler receives the admitted durable blocks and owns every further + * grammar decision, including rejecting sub-commands that cannot use them. + */ + readonly images?: boolean } /** Expected command outcome rendered directly by the dispatching UI. */ @@ -51,7 +59,7 @@ export interface CommandDescriptor { /** * Producer record for one command invocation (the `command/run` event's * source slot). Merge-extensible sum type mirroring `MessageSourceMap`'s - * shape; minimal today because every executor caller is a human-facing UI + * shape; minimal because every executor caller is a human-facing UI * surface dispatching a human-typed line, so the sole variant is `user`. */ export interface CommandSourceMap { diff --git a/packages/interaction/commands/tests/commands.spec.ts b/packages/interaction/commands/tests/commands.spec.ts index c00f938cac..a95ee024dc 100644 --- a/packages/interaction/commands/tests/commands.spec.ts +++ b/packages/interaction/commands/tests/commands.spec.ts @@ -5,6 +5,7 @@ import type { Scope } from '@deepseek-ai/dsh-scope' import type { Agent } from '@deepseek-ai/dsh-agent' import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import CommandRuntime, { parseCommand, type CommandDefinition } from '@deepseek-ai/dsh-commands' +import { AttachmentStore } from '@deepseek-ai/dsh-attachment' function command(name: string, text = `ran:${name}`): CommandDefinition { return { @@ -96,11 +97,11 @@ describe('CommandRuntime', () => { expect(ctx.commands.list(agent).map(item => item.name)).toEqual(['shared']) expect(ctx.commands.find(agent, 'shared')?.handler).toBeDefined() expect(ctx.commands.list(other).map(item => item.name)).toEqual(['shared']) - expect((await ctx.commands.execute(agent, '/shared', new AbortController().signal))?.result) + expect((await ctx.commands.execute(agent, '/shared', [], new AbortController().signal))?.result) .toEqual({ kind: 'success', text: 'scoped' }) await scope.dispose() - expect((await ctx.commands.execute(agent, '/shared', new AbortController().signal))?.result.text).toBe('global') + expect((await ctx.commands.execute(agent, '/shared', [], new AbortController().signal))?.result.text).toBe('global') }) it('removes a registration when its contributing plugin fiber is disposed', async () => { @@ -176,7 +177,7 @@ describe('CommandRuntime', () => { ctx.commands.register({ name: 'run', description: 'Run it', handler: seen }) const controller = new AbortController() - const execution = await ctx.commands.execute(agent, '/run untouched ', controller.signal) + const execution = await ctx.commands.execute(agent, '/run untouched ', [], controller.signal) expect(execution?.result).toEqual({ kind: 'success', text: 'ok' }) expect(execution?.commandId).toBeTruthy() @@ -187,8 +188,8 @@ describe('CommandRuntime', () => { rawInput: ' untouched ', signal: controller.signal, })) - await expect(ctx.commands.execute(agent, 'run', controller.signal)).resolves.toBeUndefined() - await expect(ctx.commands.execute(agent, '/missing', controller.signal)).resolves.toBeUndefined() + await expect(ctx.commands.execute(agent, 'run', [], controller.signal)).resolves.toBeUndefined() + await expect(ctx.commands.execute(agent, '/missing', [], controller.signal)).resolves.toBeUndefined() }) it('stops awaiting an aborted handler and handles an already-aborted signal', async () => { @@ -201,18 +202,18 @@ describe('CommandRuntime', () => { handler: () => new Promise((resolve) => { release = resolve }), }) const running = new AbortController() - const promise = ctx.commands.execute(agent, '/wait', running.signal) + const promise = ctx.commands.execute(agent, '/wait', [], running.signal) running.abort('operator cancelled command') await expect(promise).rejects.toThrow('operator cancelled command') release({ kind: 'success', text: 'late' }) const already = new AbortController() already.abort(new Error('already gone')) - await expect(ctx.commands.execute(agent, '/wait', already.signal)).rejects.toThrow('already gone') + await expect(ctx.commands.execute(agent, '/wait', [], already.signal)).rejects.toThrow('already gone') const defaultReason = new AbortController() defaultReason.abort({ source: 'test' }) - await expect(ctx.commands.execute(agent, '/wait', defaultReason.signal)).rejects.toThrow('command aborted') + await expect(ctx.commands.execute(agent, '/wait', [], defaultReason.signal)).rejects.toThrow('command aborted') }) it('propagates an asynchronously rejected handler', async () => { @@ -223,7 +224,7 @@ describe('CommandRuntime', () => { description: 'Reject', handler: () => Promise.reject(new Error('handler rejected')), }) - await expect(ctx.commands.execute(agent, '/reject', new AbortController().signal)) + await expect(ctx.commands.execute(agent, '/reject', [], new AbortController().signal)) .rejects.toThrow('handler rejected') ctx.commands.register({ @@ -232,7 +233,7 @@ describe('CommandRuntime', () => { // oxlint-disable-next-line typescript/prefer-promise-reject-errors -- exercise untyped plugin normalization handler: () => Promise.reject('not an Error'), }) - await expect(ctx.commands.execute(agent, '/reject-value', new AbortController().signal)) + await expect(ctx.commands.execute(agent, '/reject-value', [], new AbortController().signal)) .rejects.toThrow('command handler rejected with a non-Error value: not an Error') const hostile = { toString(): string { throw new Error('cannot render') } } @@ -242,7 +243,7 @@ describe('CommandRuntime', () => { // oxlint-disable-next-line typescript/prefer-promise-reject-errors -- exercise hostile plugin normalization handler: () => Promise.reject(hostile), }) - await expect(ctx.commands.execute(agent, '/reject-hostile', new AbortController().signal)) + await expect(ctx.commands.execute(agent, '/reject-hostile', [], new AbortController().signal)) .rejects.toMatchObject({ message: 'command handler rejected with a non-Error value: ', cause: hostile, @@ -261,7 +262,7 @@ describe('CommandRuntime', () => { return { kind: 'success' } }, }) - await expect(ctx.commands.execute(agent, '/self-abort', controller.signal)) + await expect(ctx.commands.execute(agent, '/self-abort', [], controller.signal)) .rejects.toThrow('aborted in handler') }) @@ -273,7 +274,7 @@ describe('CommandRuntime', () => { description: 'Denied', handler: () => ({ kind: 'error', text: 'not now' }), }) - const execution = await ctx.commands.execute(agent, '/denied', new AbortController().signal) + const execution = await ctx.commands.execute(agent, '/denied', [], new AbortController().signal) expect(execution?.result).toEqual({ kind: 'error', text: 'not now' }) expect(Object.isFrozen(execution?.result)).toBe(true) @@ -282,7 +283,7 @@ describe('CommandRuntime', () => { description: 'No output', handler: () => ({ kind: 'success' }), }) - const silent = await ctx.commands.execute(agent, '/silent', new AbortController().signal) + const silent = await ctx.commands.execute(agent, '/silent', [], new AbortController().signal) expect(silent?.result).toEqual({ kind: 'success' }) expect(Object.isFrozen(silent?.result)).toBe(true) }) @@ -302,7 +303,7 @@ describe('CommandRuntime', () => { const { agent } = await mintAgentScope(ctx, 'a') ctx.commands.register(command('deploy', 'deployed')) - const execution = await ctx.commands.execute(agent, '/deploy now', new AbortController().signal) + const execution = await ctx.commands.execute(agent, '/deploy now', [], new AbortController().signal) const lifecycle = lifecycleOf(agent) expect(lifecycle).toMatchObject([ @@ -330,7 +331,7 @@ describe('CommandRuntime', () => { handler: () => ({ kind: 'success', text: 'linked', sourceEventSeq: source.seq }), }) - const execution = await ctx.commands.execute(agent, '/linked', new AbortController().signal) + const execution = await ctx.commands.execute(agent, '/linked', [], new AbortController().signal) expect(execution?.result).toEqual({ kind: 'success', text: 'linked', sourceEventSeq: source.seq }) expect(lifecycleOf(agent)).toMatchObject([ @@ -350,7 +351,7 @@ describe('CommandRuntime', () => { handler: seen, }) - await ctx.commands.execute(agent, '/private keep this once', new AbortController().signal) + await ctx.commands.execute(agent, '/private keep this once', [], new AbortController().signal) expect(seen).toHaveBeenCalledWith(expect.objectContaining({ rawInput: ' keep this once' })) const run = agent.session.events.find(event => event.type === 'command/run') @@ -363,8 +364,8 @@ describe('CommandRuntime', () => { const { agent } = await mintAgentScope(ctx, 'a') ctx.commands.register(command('first')) ctx.commands.register(command('second')) - await ctx.commands.execute(agent, '/first', new AbortController().signal) - await ctx.commands.execute(agent, '/second', new AbortController().signal) + await ctx.commands.execute(agent, '/first', [], new AbortController().signal) + await ctx.commands.execute(agent, '/second', [], new AbortController().signal) const ids = lifecycleOf(agent) .filter(event => event.type === 'command/run') .map(event => (event.data as { commandId: string }).commandId) @@ -375,7 +376,7 @@ describe('CommandRuntime', () => { const ctx = await mount() const { agent } = await mintAgentScope(ctx, 'a') ctx.commands.register({ name: 'denied', description: 'Denied', handler: () => ({ kind: 'error', text: 'not now' }) }) - await ctx.commands.execute(agent, '/denied', new AbortController().signal) + await ctx.commands.execute(agent, '/denied', [], new AbortController().signal) expect(lifecycleOf(agent)).toMatchObject([ { type: 'command/run', data: { name: 'denied' } }, { type: 'command/done', data: { kind: 'error', text: 'not now' } }, @@ -390,7 +391,7 @@ describe('CommandRuntime', () => { description: 'Throw', handler: () => { throw new Error('handler exploded') }, }) - await expect(ctx.commands.execute(agent, '/boom', new AbortController().signal)) + await expect(ctx.commands.execute(agent, '/boom', [], new AbortController().signal)) .rejects.toThrow('handler exploded') expect(lifecycleOf(agent)).toMatchObject([ { type: 'command/run', data: { name: 'boom' } }, @@ -407,7 +408,7 @@ describe('CommandRuntime', () => { handler: () => new Promise(() => undefined), }) const controller = new AbortController() - const pending = ctx.commands.execute(agent, '/hang', controller.signal) + const pending = ctx.commands.execute(agent, '/hang', [], controller.signal) // The run append must land before the abort so the pair stays complete. await vi.waitFor(() => { expect(lifecycleOf(agent)).toHaveLength(1) }) controller.abort('operator cancelled command') @@ -425,8 +426,8 @@ describe('CommandRuntime', () => { const { agent } = await mintAgentScope(ctx, 'a') ctx.commands.register(command('real')) const signal = new AbortController().signal - await ctx.commands.execute(agent, 'not a command', signal) - await ctx.commands.execute(agent, '/missing', signal) + await ctx.commands.execute(agent, 'not a command', [], signal) + await ctx.commands.execute(agent, '/missing', [], signal) expect(agent.session.events).toEqual([]) }) @@ -435,7 +436,7 @@ describe('CommandRuntime', () => { const { agent } = await mintAgentScope(ctx, 'a') ctx.commands.register(command('mid')) agent.session.append('turn/start', { turn: 1 }) - await ctx.commands.execute(agent, '/mid', new AbortController().signal) + await ctx.commands.execute(agent, '/mid', [], new AbortController().signal) expect(agent.session.events.map(event => event.type)).toEqual([ 'turn/start', 'command/run', 'command/done', ]) @@ -460,6 +461,167 @@ describe('CommandRuntime', () => { description: 'Broken', handler: () => output as never, }) - await expect(ctx.commands.execute(agent, '/broken', new AbortController().signal)).rejects.toThrow(expected) + await expect(ctx.commands.execute(agent, '/broken', [], new AbortController().signal)).rejects.toThrow(expected) + }) +}) + +describe('image attachments', () => { + const PNG = 'AAAA' + + function storeOf() { + let saved = 0 + const store = { + imageLimits: { + maxImageBytes: 1024, maxImagesPerMessage: 2, maxMessageImageBytes: 1024, + maxImagePixels: 1_000_000, maxImageDimension: 2000, mediaTypes: ['image/png'], + }, + validateImage: vi.fn(() => Promise.resolve()), + saveImage: vi.fn((input: { mediaType: string; name?: string }) => { + saved += 1 + return Promise.resolve({ + attachmentId: `att-${saved}`, mediaType: input.mediaType, bytes: 3, width: 1, height: 1, + ...input.name === undefined ? {} : { name: input.name }, + }) + }), + validateImageBatch(inputs: readonly unknown[]) { + const validate = AttachmentStore.prototype as unknown as { + validateImageBatch(this: unknown, batch: readonly unknown[]): void + } + validate.validateImageBatch.call(this, inputs) + }, + // The real base-class batch method over this double's limits and members. + saveImages(inputs: readonly unknown[]) { + return (AttachmentStore.prototype.saveImages as (this: unknown, batch: readonly unknown[]) => Promise).call(this, inputs) + }, + } + return store + } + + function accepting(handler: CommandDefinition['handler']): CommandDefinition { + return { + name: 'vision', + description: 'accepts images', + input: { hint: '', images: true }, + handler, + } + } + + it('rejects a boolean-typed images flag violation at registration', async () => { + const ctx = await mount() + expect(() => ctx.commands.register({ + ...command('flag-type'), + input: { hint: 'x', images: 'yes' }, + } as unknown as CommandDefinition)).toThrow('command "flag-type" input images flag must be a boolean') + }) + + it('lists images acceptance on the descriptor and omits a false flag', async () => { + const ctx = await mount() + const { agent } = await mintAgentScope(ctx, 'a') + ctx.commands.register(accepting(() => ({ kind: 'success' }))) + ctx.commands.register({ ...command('plain-input'), input: { hint: 'x', images: false } }) + const byName = new Map(ctx.commands.list(agent).map(descriptor => [descriptor.name, descriptor])) + expect(byName.get('vision')?.input).toEqual({ hint: '', images: true }) + expect(byName.get('plain-input')?.input).toEqual({ hint: 'x' }) + }) + + it('settles images sent to a non-declaring command as a logged error before the handler', async () => { + const ctx = await mount() + const { agent } = await mintAgentScope(ctx, 'a') + const handler = vi.fn(() => ({ kind: 'success' as const })) + ctx.commands.register({ ...command('deploy'), handler }) + const execution = await ctx.commands.execute( + agent, '/deploy now', [{ mediaType: 'image/png', data: PNG }], new AbortController().signal) + expect(execution?.result).toEqual({ kind: 'error', text: '/deploy does not accept image attachments' }) + expect(handler).not.toHaveBeenCalled() + expect(lifecycleOf(agent)).toMatchObject([ + { type: 'command/run', data: { name: 'deploy' } }, + { type: 'command/done', data: { kind: 'error', text: '/deploy does not accept image attachments' } }, + ]) + }) + + it('settles a declaring command as a logged error when no attachment store is composed', async () => { + const ctx = await mount() + const { agent } = await mintAgentScope(ctx, 'a') + ctx.commands.register(accepting(() => ({ kind: 'success' }))) + const execution = await ctx.commands.execute( + agent, '/vision x', [{ mediaType: 'image/png', data: PNG }], new AbortController().signal) + expect(execution?.result).toEqual({ + kind: 'error', + text: '/vision: image attachments are unavailable because no attachment store is composed', + }) + }) + + it('admits and hands the handler frozen ordered image blocks; plain invocations stay empty', async () => { + const ctx = await mount() + ctx.provide('attachments', storeOf()) + const { agent } = await mintAgentScope(ctx, 'a') + const seen = vi.fn((invocation: { attachments: readonly unknown[] }) => { + expect(Object.isFrozen(invocation.attachments)).toBe(true) + return { kind: 'success' as const } + }) + ctx.commands.register(accepting(seen)) + await ctx.commands.execute(agent, '/vision x', [ + { mediaType: 'image/png', data: PNG, name: 'a.png' }, + { mediaType: 'image/png', data: PNG, name: 'b.png' }, + ], new AbortController().signal) + const invocation = seen.mock.calls[0]?.[0] as { attachments: ReadonlyArray<{ type: string; attachment: { name?: string } }> } + expect(invocation.attachments.map(block => [block.type, block.attachment.name])).toEqual([ + ['image', 'a.png'], ['image', 'b.png'], + ]) + await ctx.commands.execute(agent, '/vision y', [], new AbortController().signal) + expect((seen.mock.calls[1]?.[0] as { attachments: readonly unknown[] }).attachments).toEqual([]) + }) + + it('settles an admission limit failure as a logged error result', async () => { + const ctx = await mount() + ctx.provide('attachments', storeOf()) + const { agent } = await mintAgentScope(ctx, 'a') + const handler = vi.fn(() => ({ kind: 'success' as const })) + ctx.commands.register(accepting(handler)) + const three = [1, 2, 3].map(() => ({ mediaType: 'image/png' as const, data: PNG })) + const execution = await ctx.commands.execute(agent, '/vision x', three, new AbortController().signal) + expect(execution?.result).toEqual({ kind: 'error', text: 'Image batch exceeds the configured image-count limit.' }) + expect(handler).not.toHaveBeenCalled() + expect(lifecycleOf(agent).at(-1)).toMatchObject({ type: 'command/done', data: { kind: 'error' } }) + }) + + it('honors a cancellation that lands during admission before entering the handler', async () => { + const ctx = await mount() + const controller = new AbortController() + const store = storeOf() + store.saveImage.mockImplementationOnce((input: { mediaType: string }) => { + controller.abort('operator cancelled during admission') + return Promise.resolve({ + attachmentId: 'att-late', mediaType: input.mediaType, bytes: 3, width: 1, height: 1, + }) + }) + ctx.provide('attachments', store) + const { agent } = await mintAgentScope(ctx, 'a') + const handler = vi.fn(() => ({ kind: 'success' as const })) + ctx.commands.register(accepting(handler)) + await expect(ctx.commands.execute( + agent, '/vision x', [{ mediaType: 'image/png', data: PNG }], controller.signal, + )).rejects.toThrow('operator cancelled during admission') + expect(handler).not.toHaveBeenCalled() + expect(lifecycleOf(agent).at(-1)).toMatchObject({ + type: 'command/done', + data: { kind: 'error', text: 'operator cancelled during admission' }, + }) + }) + + it('logs and rethrows a non-attachment admission failure', async () => { + const ctx = await mount() + const store = storeOf() + store.saveImage.mockRejectedValueOnce(new Error('disk gone')) + ctx.provide('attachments', store) + const { agent } = await mintAgentScope(ctx, 'a') + ctx.commands.register(accepting(() => ({ kind: 'success' }))) + await expect(ctx.commands.execute( + agent, '/vision x', [{ mediaType: 'image/png', data: PNG }], new AbortController().signal, + )).rejects.toThrow('disk gone') + expect(lifecycleOf(agent).at(-1)).toMatchObject({ + type: 'command/done', + data: { kind: 'error', text: 'disk gone' }, + }) }) }) diff --git a/packages/interaction/commands/tsconfig.json b/packages/interaction/commands/tsconfig.json index 0504815c60..436f5c9472 100644 --- a/packages/interaction/commands/tsconfig.json +++ b/packages/interaction/commands/tsconfig.json @@ -14,12 +14,18 @@ { "path": "../../../vendor/cordis" }, + { + "path": "../../attachment/attachment" + }, { "path": "../../core/agent" }, { "path": "../../core/scope" }, + { + "path": "../../llm/llm" + }, { "path": "../../core/session" }, @@ -31,6 +37,9 @@ }, { "path": "../../typert/protocol" + }, + { + "path": "../../util/crypto" } ] } diff --git a/packages/interaction/permission-presets/README.i18n.yaml b/packages/interaction/permission-presets/README.i18n.yaml index 7e7359526c..58938e7883 100644 --- a/packages/interaction/permission-presets/README.i18n.yaml +++ b/packages/interaction/permission-presets/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/interaction/permission-presets/README.md README.md: 2b671f9e6e835c529453dc7d4ca7bc2eff01f2b6 -README.zh.md: 686138c1f2b0b3770771d0e6d6a785ed17cc8621 +README.zh.md: 2475740a1c7e3240a12f1f462bab05cc7e908b38 diff --git a/packages/interaction/permission-presets/README.zh.md b/packages/interaction/permission-presets/README.zh.md index 686138c1f2..2475740a1c 100644 --- a/packages/interaction/permission-presets/README.zh.md +++ b/packages/interaction/permission-presets/README.zh.md @@ -8,7 +8,7 @@ 该服务拥有 `permissionPresets` Settings namespace。其 `defaultPreset` 是未来会话的默认值:组合项使用 `Config.defaultPreset`;省略时,则推断与组合后的沙箱和审批默认值匹配的 preset。已提交的 Settings 变更会在下一个会话创建时读取;创建过程将 `permissionPresets/preset`、`sandbox/mode` 和 `approval/policy` 固定到该会话中,因此后续变更绝不会改变现有会话。恢复的 seed,包括由 `session/end-seed` 标记的显式空 seed,都会保留其有效权限,只补齐缺失的持久事实,而不会采用最新的用户默认值。挂载服务时还会遍历所有已存活会话,因此 HMR(热模块替换)会固定插件缺席期间创建的所有会话。 -该服务要求存在具有约束能力的 `ctx.shell` 执行器和 `ctx.approval`。表中名为 `custom` 的条目会在加载时抛出异常。当组合默认值与任何 preset 都不匹配时,插件要求显式配置 `defaultPreset`;独立构造的零事件会话仍可能推导出 `custom`。详见[沙箱切换设计](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md)。 +该服务要求存在具有约束能力的 `ctx.shell` 执行器和 `ctx.approval`。表中名为 `custom` 的条目会在加载时抛出异常。当组合默认值与任何 preset 都不匹配时,插件要求显式配置 `defaultPreset`;独立构造的零事件会话仍可能推导出 `custom`。详见[沙箱切换设计](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.zh.md)。 两个可选子功能在同一服务之上提供产品界面:`permissions` 会话投影单元(`src/types.ts` 声明该 key;单元以组合默认值为基础折叠三个全量值可调参数事件,并生成选择器视图,其中包含表内选项和仅作当前值的 `custom`)与 `/permissionPresets` 命令(不带参数调用时报告当前预设与表;预设参数经 `set` 切换)。每个子功能仅在其注册表(`ctx.sessionProjections` / `ctx.commands`)被组合时激活。 diff --git a/packages/interaction/permission-presets/package.json b/packages/interaction/permission-presets/package.json index 3b3160baae..a9e3313204 100644 --- a/packages/interaction/permission-presets/package.json +++ b/packages/interaction/permission-presets/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-permission-presets", "description": "User-facing permission presets (ctx.permissionPresets) for the DeepSeek Harness: one product-level Permissions select bundling the sandbox-mode and approval-policy knobs, written through to their own session events", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/interaction/permission-presets/src/index.ts b/packages/interaction/permission-presets/src/index.ts index fee29c2644..2366ff13f9 100644 --- a/packages/interaction/permission-presets/src/index.ts +++ b/packages/interaction/permission-presets/src/index.ts @@ -100,6 +100,22 @@ export interface KnobState { approval: ApprovalPolicy | null } +declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + permissions: KnobState + } +} + +const knobStateSchema: zod.ZodType = zod.object({ + preset: zod.string().nullable(), + sandbox: zod.union([ + zod.literal('read-only'), + zod.literal('workspace-write'), + zod.literal('danger-full-access'), + ]).nullable(), + approval: zod.union([zod.literal('ask'), zod.literal('never')]).nullable(), +}).strict() + /** State for the empty log: every knob at its composition default. */ const EMPTY_KNOBS: KnobState = { preset: null, sandbox: null, approval: null } @@ -243,10 +259,10 @@ export class PermissionPresetService extends Service { ctx.inject(['sessionProjections'], (projectionCtx) => { projectionCtx.sessionProjections.register<'permissions', KnobState>({ key: 'permissions', - schema: selectSchema, + stateSchema: knobStateSchema, init: () => EMPTY_KNOBS, apply: applyKnobEvent, - view: state => this.selectFor(state), + wire: { viewSchema: selectSchema, view: state => this.selectFor(state) }, stateVersion: 1, }) }) diff --git a/packages/interaction/permission-presets/tests/projection.spec.ts b/packages/interaction/permission-presets/tests/projection.spec.ts index 2c068594cc..f9bba33958 100644 --- a/packages/interaction/permission-presets/tests/projection.spec.ts +++ b/packages/interaction/permission-presets/tests/projection.spec.ts @@ -90,7 +90,7 @@ describe('/permission command', () => { it('switches through permission.set and logs the lifecycle pair', async () => { const { ctx, session } = await harness() const { agent, inject } = await agentFor(ctx, session) - const execution = await ctx.commands.execute(agent, '/permission danger-full-access', new AbortController().signal) + const execution = await ctx.commands.execute(agent, '/permission danger-full-access', [], new AbortController().signal) expect(execution?.result).toEqual({ kind: 'success', text: 'preset danger-full-access' }) expect(ctx.permissionPresets.current(session.events)).toBe('danger-full-access') expect(inject.mock.calls[0]?.[0]).toMatchObject({ @@ -106,7 +106,7 @@ describe('/permission command', () => { it('reports the current preset and the table on bare invocation', async () => { const { ctx, session } = await harness() const { agent } = await agentFor(ctx, session) - const execution = await ctx.commands.execute(agent, '/permission', new AbortController().signal) + const execution = await ctx.commands.execute(agent, '/permission', [], new AbortController().signal) expect(execution?.result).toEqual({ kind: 'success', text: 'current preset workspace-write (available: workspace-write, danger-full-access)', @@ -119,7 +119,7 @@ describe('/permission command', () => { const { agent } = await agentFor(ctx, session) const before = session.events.filter(event => event.type !== 'command/run' && event.type !== 'command/done') - const execution = await ctx.commands.execute(agent, '/permission yolo', new AbortController().signal) + const execution = await ctx.commands.execute(agent, '/permission yolo', [], new AbortController().signal) // The error text carries the same no-self-labelling rule as the success // texts: `permission · unknown preset "yolo" (…)`, not `unknown permission // preset`, which the row's own title already says. diff --git a/packages/interaction/tool-ask-user/README.i18n.yaml b/packages/interaction/tool-ask-user/README.i18n.yaml index ed6cd6a35f..dd69a79586 100644 --- a/packages/interaction/tool-ask-user/README.i18n.yaml +++ b/packages/interaction/tool-ask-user/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/interaction/tool-ask-user/README.md README.md: bf2ba369fca36c462c6155ed2354ab3d0e930dd0 -README.zh.md: 671bea0132093b890de4ce27095fa60dac396276 +README.zh.md: 067235d84d70d2f3278fd39f5c01f130373d3ac2 diff --git a/packages/interaction/tool-ask-user/README.zh.md b/packages/interaction/tool-ask-user/README.zh.md index 671bea0132..067235d84d 100644 --- a/packages/interaction/tool-ask-user/README.zh.md +++ b/packages/interaction/tool-ask-user/README.zh.md @@ -27,7 +27,7 @@ #### 模型看到的内容 -模型会看到生成的 [`ask_user_question` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-ask-user),其中包含问题 id、提示语、标题、选项和多选标志。 +模型会看到生成的 [`ask_user_question` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-ask-user),其中包含问题 id、提示语、标题、选项和多选标志。 #### Token 影响 diff --git a/packages/interaction/tool-ask-user/package.json b/packages/interaction/tool-ask-user/package.json index af45d56ab1..067115a5d2 100644 --- a/packages/interaction/tool-ask-user/package.json +++ b/packages/interaction/tool-ask-user/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-ask-user", "description": "Model-facing ask_user_question tool over the ctx.userQuestions seam", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/interaction/user-approval/README.i18n.yaml b/packages/interaction/user-approval/README.i18n.yaml index 07843b5f58..ba340c5273 100644 --- a/packages/interaction/user-approval/README.i18n.yaml +++ b/packages/interaction/user-approval/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/interaction/user-approval/README.md README.md: 0cf5d458863194e29f8c84168a6f089baabbf3d2 -README.zh.md: ccd8641e30d8a78611eabf602fa46d2705f24487 +README.zh.md: a93f9c17c89ea50622e354eb7729547660e877e2 diff --git a/packages/interaction/user-approval/README.zh.md b/packages/interaction/user-approval/README.zh.md index ccd8641e30..a93f9c17c8 100644 --- a/packages/interaction/user-approval/README.zh.md +++ b/packages/interaction/user-approval/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -与通道无关的一次性审批 seam。`ctx.approval.request(req)` 返回 `allowed-once`、`rejected`、`cancelled` 或 `unavailable`;应答者缺失或失败时会以拒绝方式关闭,授权也只适用于所请求的操作。确切事件签名见 [approval.md](../../../docs/subsystems/approval.md#cordis-surface) 的生成区块。 +与通道无关的一次性审批 seam。`ctx.approval.request(req)` 返回 `allowed-once`、`rejected`、`cancelled` 或 `unavailable`;应答者缺失或失败时会以拒绝方式关闭,授权也只适用于所请求的操作。确切事件签名见 [approval.md](../../../docs/subsystems/approval.zh.md#cordis-surface) 的生成区块。 每个请求都必须属于一个尚未结束的 agent(智能体)轮次。服务会追加一对 `approval/asked` 与 `approval/decided` 审计记录,而模型只会看到由此产生且已写入日志的工具结果。已中止的请求会解析为 `cancelled`;如果审计记录的追加在提交前失败,Promise 会被拒绝,而不会返回一项未记录的决定。 @@ -10,7 +10,7 @@ `ApprovalPolicy` 为 `'ask'` 或 `'never'`。实际值取最后一条 `approval/policy` 事件,并回退到配置;`setApprovalPolicy()` 是写入路径。`'never'` 会在交互式分发之前拒绝请求。两种策略都会将各自完整的当前含义贡献给缓存安全的运行时上下文快照。 -工具流水线通过此 seam 路由 `ask` 决定,并在该 seam 缺失时以拒绝方式关闭;沙箱 bash 工具也会将它用于升权重试。ACP 自动化桥接层根据客户端的机器策略,回答其自有 agent 的调用。审计事件仍只写入日志,因此模型只会看到发起请求的消费方所返回的结果。详见[审批 seam Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-approval-seam.md)和[沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md)。 +工具流水线通过此 seam 路由 `ask` 决定,并在该 seam 缺失时以拒绝方式关闭;沙箱 bash 工具也会将它用于升权重试。ACP 自动化桥接层根据客户端的机器策略,回答其自有 agent 的调用。审计事件仍只写入日志,因此模型只会看到发起请求的消费方所返回的结果。详见[审批 seam Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-approval-seam.zh.md)和[沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.zh.md)。 ## 模型体验 diff --git a/packages/interaction/user-approval/package.json b/packages/interaction/user-approval/package.json index aa1e6a5ee7..5ec59a32d6 100644 --- a/packages/interaction/user-approval/package.json +++ b/packages/interaction/user-approval/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-user-approval", "description": "User-approval seam (ctx.approval) for the DeepSeek Harness: one-shot permission decisions dispatched to composed answerers over the approval/request waterfall, fail-closed by default", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/interaction/user-questions/package.json b/packages/interaction/user-questions/package.json index 04b7355dd1..07dafa3341 100644 --- a/packages/interaction/user-questions/package.json +++ b/packages/interaction/user-questions/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-user-questions", "description": "Abstract user-questions seam (ctx.userQuestions) for asking the human during agent runs", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/jobs/README.i18n.yaml b/packages/jobs/README.i18n.yaml index 6c032dcbf0..121481d6ab 100644 --- a/packages/jobs/README.i18n.yaml +++ b/packages/jobs/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/jobs/README.md README.md: daad96ae561f8f943759332b6b1ea1e20fcaf40e -README.zh.md: 4e733b1da09656c9d779c43a0ee77703e1c10bfa +README.zh.md: 586f195f2603cd1b8e4cd5e23174841cb1a39910 diff --git a/packages/jobs/README.zh.md b/packages/jobs/README.zh.md index 4e733b1da0..586f195f26 100644 --- a/packages/jobs/README.zh.md +++ b/packages/jobs/README.zh.md @@ -6,10 +6,10 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`jobs/`](jobs/README.md) | 定义任务注册表和生命周期约定 | `ctx.jobs` | -| [`jobs-local/`](jobs-local/README.md) | 实现进程本地任务注册表 | 注册到 `ctx.jobs` | -| [`tool-jobs/`](tool-jobs/README.md) | 向模型公开任务控制和完成通知 | 注册到 `ctx.tools` | +| [`jobs/`](jobs/README.zh.md) | 定义任务注册表和生命周期约定 | `ctx.jobs` | +| [`jobs-local/`](jobs-local/README.zh.md) | 实现进程本地任务注册表 | 注册到 `ctx.jobs` | +| [`tool-jobs/`](tool-jobs/README.zh.md) | 向模型公开任务控制和完成通知 | 注册到 `ctx.tools` | -参见[后台任务运行时](../../.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md)和[任务注册表](../../.agents/notes/implemented/architecture/2026-07-26-job-registry-seam.md)决策。 +参见[后台任务运行时](../../.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.zh.md)和[任务注册表](../../.agents/notes/implemented/architecture/2026-07-26-job-registry-seam.zh.md)决策。 -子系统参考文档——id 方案、所有者隔离约定、快照——见 [docs/subsystems/jobs.md](../../docs/subsystems/jobs.md);设计见[后台任务运行时](../../.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md)与[任务注册表约定](../../.agents/notes/implemented/architecture/2026-07-26-job-registry-seam.md)两篇 Agent Note。 +子系统参考文档——id 方案、所有者隔离约定、快照——见 [docs/subsystems/jobs.md](../../docs/subsystems/jobs.zh.md);设计见[后台任务运行时](../../.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.zh.md)与[任务注册表约定](../../.agents/notes/implemented/architecture/2026-07-26-job-registry-seam.zh.md)两篇 Agent Note。 diff --git a/packages/jobs/jobs-local/README.i18n.yaml b/packages/jobs/jobs-local/README.i18n.yaml index 7d7f6d2c9b..8aa1a4ea4b 100644 --- a/packages/jobs/jobs-local/README.i18n.yaml +++ b/packages/jobs/jobs-local/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/jobs/jobs-local/README.md README.md: e55d9d1747e0dbb12d22f2528527312334f352e6 -README.zh.md: 78979d9518b1b588c689ce5184b33d79177a1ec0 +README.zh.md: 86fd3e8b9db9d0073ce5e6999f5a1fbce12059d2 diff --git a/packages/jobs/jobs-local/README.zh.md b/packages/jobs/jobs-local/README.zh.md index 78979d9518..86fd3e8b9d 100644 --- a/packages/jobs/jobs-local/README.zh.md +++ b/packages/jobs/jobs-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-jobs`](../jobs/README.md) 注册表约定的进程本地实现:`LocalJobRegistry` 把每条记录保存在内存中,按 kind 签发 `-N` id,并且只交出全新快照,从不交出实时状态。作为插件加载后即注册为 `ctx.jobs`。 +[`@deepseek-ai/dsh-jobs`](../jobs/README.zh.md) 注册表约定的进程本地实现:`LocalJobRegistry` 把每条记录保存在内存中,按 kind 签发 `-N` id,并且只交出全新快照,从不交出实时状态。作为插件加载后即注册为 `ctx.jobs`。 ## 准入 @@ -22,7 +22,7 @@ ## 模型体验 -通过生产方插件和 [`dsh-tool-jobs`](../tool-jobs/README.md) 间接影响;它们会呈现 job id、输出、状态、取消和完成通知。 +通过生产方插件和 [`dsh-tool-jobs`](../tool-jobs/README.zh.md) 间接影响;它们会呈现 job id、输出、状态、取消和完成通知。 #### KV Cache 影响 diff --git a/packages/jobs/jobs-local/package.json b/packages/jobs/jobs-local/package.json index 4b1120a92d..117f9d9e99 100644 --- a/packages/jobs/jobs-local/package.json +++ b/packages/jobs/jobs-local/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-jobs-local", "description": "Process-local implementation of the DeepSeek Harness background job registry seam", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/jobs/jobs/README.i18n.yaml b/packages/jobs/jobs/README.i18n.yaml index 0473d974f8..8129e170ed 100644 --- a/packages/jobs/jobs/README.i18n.yaml +++ b/packages/jobs/jobs/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/jobs/jobs/README.md README.md: f4469286603f687be5789bd016ab462dba26e664 -README.zh.md: 30d8e4cb1e298bb4a3a0a3e65bb0858b2458a277 +README.zh.md: 6fd97aa3fe4cbbd555c2475f4132ec00c6a97390 diff --git a/packages/jobs/jobs/README.zh.md b/packages/jobs/jobs/README.zh.md index 30d8e4cb1e..6fd97aa3fe 100644 --- a/packages/jobs/jobs/README.zh.md +++ b/packages/jobs/jobs/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -后台任务注册表约定(`ctx.jobs`)。抽象的 `JobRegistry` 及其词汇类型在同一份约定下为长时间运行的生产方提供共享 id、owner 隔离、读取、取消、等待、通知和清理;进程局部注册表位于 [`dsh-jobs-local`](../jobs-local/README.md)。生产方插件使用其不透明 id namespace 扩展 `JobKindMap`。 +后台任务注册表约定(`ctx.jobs`)。抽象的 `JobRegistry` 及其词汇类型在同一份约定下为长时间运行的生产方提供共享 id、owner 隔离、读取、取消、等待、通知和清理;进程局部注册表位于 [`dsh-jobs-local`](../jobs-local/README.zh.md)。生产方插件使用其不透明 id namespace 扩展 `JobKindMap`。 ## 服务约定 @@ -23,11 +23,11 @@ 实现还必须兑现约定的生命周期语义:注册的存续期长于生产方 fiber 与控制器 fiber,owner 释放和服务释放会取消仍在运行的工作并等待守约的生产方,结算遵循首次结果优先(一条终止记录、一轮异常受到隔离的监听器通知,然后释放等待方)。 -参见[任务类型目录](../../../docs/subsystems/jobs.md)、[运行时 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md)和 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-job-registry-seam.md)。 +参见[任务类型目录](../../../docs/subsystems/jobs.zh.md)、[运行时 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.zh.md)和 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-job-registry-seam.zh.md)。 ## 模型体验 -通过生产方插件和 [`dsh-tool-jobs`](../tool-jobs/README.md) 间接影响;它们会渲染 job id、输出、状态、取消和完成通知。 +通过生产方插件和 [`dsh-tool-jobs`](../tool-jobs/README.zh.md) 间接影响;它们会渲染 job id、输出、状态、取消和完成通知。 #### KV Cache 影响 diff --git a/packages/jobs/jobs/package.json b/packages/jobs/jobs/package.json index 83235ec22b..c658d2d49f 100644 --- a/packages/jobs/jobs/package.json +++ b/packages/jobs/jobs/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-jobs", "description": "Background job registry (ctx.jobs) for the DeepSeek Harness — shared ids, owner isolation, polling, cancellation, and completion listeners for long-running tool work", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/jobs/tool-jobs/README.i18n.yaml b/packages/jobs/tool-jobs/README.i18n.yaml index e1a51a2cbe..6b5cc133dc 100644 --- a/packages/jobs/tool-jobs/README.i18n.yaml +++ b/packages/jobs/tool-jobs/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/jobs/tool-jobs/README.md README.md: 8d6a00651f69258d764904ac99d000c2055982dc -README.zh.md: 15903b6f263fe9d225c48bf567c46f587d23cb00 +README.zh.md: bd1045ecf6f2b7c9ed1f8a8cdfe4f965af418a37 diff --git a/packages/jobs/tool-jobs/README.zh.md b/packages/jobs/tool-jobs/README.zh.md index 15903b6f26..bd1045ecf6 100644 --- a/packages/jobs/tool-jobs/README.zh.md +++ b/packages/jobs/tool-jobs/README.zh.md @@ -63,7 +63,7 @@ Track every background job id you start. You are notified in-session when a job #### 模型看到的内容 -该工具集可见时,会看到生成的 [`job_output`、`job_list` 和 `job_kill` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-jobs)。 +该工具集可见时,会看到生成的 [`job_output`、`job_list` 和 `job_kill` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-jobs)。 #### Token 影响 diff --git a/packages/jobs/tool-jobs/package.json b/packages/jobs/tool-jobs/package.json index 73ddb5f657..fc1fc6877c 100644 --- a/packages/jobs/tool-jobs/package.json +++ b/packages/jobs/tool-jobs/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-jobs", "description": "Model-facing background job control tools (job_output, job_list, job_kill) over the ctx.jobs registry", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/llm/README.i18n.yaml b/packages/llm/README.i18n.yaml index a655518047..a404fdb6eb 100644 --- a/packages/llm/README.i18n.yaml +++ b/packages/llm/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/README.md -README.md: 15f90024abf3256560d67099af79be7d4e0ee310 -README.zh.md: 6be4123e2e5de5c41937c4b966580278c78e5a42 +README.md: edef08e594c1bb46179996342b05662af981b98a +README.zh.md: a454a1e1eb33484274005d7689e8a44ae0e062ba diff --git a/packages/llm/README.md b/packages/llm/README.md index 15f90024ab..edef08e594 100644 --- a/packages/llm/README.md +++ b/packages/llm/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The LLM seam and its provider adapters. The `llm` package owns both the Service Definition and Consumer roles: the abstract service, content-block vocabulary, and stream-chunk assembler. Provider adapters register on `ctx.llm`. All **product** packages. +The LLM seam, provider adapters, and provider-specific request metadata plugins. The `llm` package owns both the Service Definition and Consumer roles: the abstract service, content-block vocabulary, and stream-chunk assembler. Provider adapters register on `ctx.llm`. All **product** packages. | Package | Role | ctx key | |---|---|---| @@ -11,6 +11,8 @@ The LLM seam and its provider adapters. The `llm` package owns both the Service | [`llm-retry/`](llm-retry/README.md) | Provider-scoped retry policy | listens to `agent/request-error` | | [`llm-deepseek/`](llm-deepseek/README.md) | Direct DeepSeek adapter | registers on `ctx.llm` | | [`llm-pi-ai/`](llm-pi-ai/README.md) | Multi-provider pi-ai adapter | registers on `ctx.llm` | +| [`deepseek-llm-api-extensions/`](deepseek-llm-api-extensions/README.md) | Official DeepSeek request-field registry | `ctx.deepseekLlmApiExtensions` | +| [`plugin-package-inventory-deepseek/`](plugin-package-inventory-deepseek/README.md) | Active package metadata for official DeepSeek requests | contributes `dsh_plugin_packages` | Adapters register provider routes on the seam; retry and token measurement remain separate consumers. The child READMEs own routing, metadata, replay, and provider-wire details; the [LLM architecture decisions](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) own the rationale. diff --git a/packages/llm/README.zh.md b/packages/llm/README.zh.md index 6be4123e2e..a454a1e1eb 100644 --- a/packages/llm/README.zh.md +++ b/packages/llm/README.zh.md @@ -2,16 +2,18 @@ [English](README.md) | 中文 -LLM(大语言模型)seam 及其提供方适配器。`llm` 包同时承担 Service Definition 和 Consumer 角色:抽象服务、内容块词汇和流式分片组装器。提供方适配器注册到 `ctx.llm`。这些全是**产品**包。 +LLM(大语言模型)seam、提供方适配器及提供方特定请求元数据插件。`llm` 包同时承担 Service Definition 和 Consumer 角色:抽象服务、内容块词汇和流式分片组装器。提供方适配器注册到 `ctx.llm`。这些全是**产品**包。 | 包 | 职责 | ctx key | |---|---|---| -| [`llm/`](llm/README.md) | LLM 服务和共享流式词汇 | `ctx.llm` | -| [`token-meter/`](token-meter/README.md) | 可感知回放的 token 测量 | `ctx.tokenMeter` | -| [`llm-retry/`](llm-retry/README.md) | 提供方作用域的重试策略 | 监听 `agent/request-error` | -| [`llm-deepseek/`](llm-deepseek/README.md) | 直接 DeepSeek 适配器 | 注册到 `ctx.llm` | -| [`llm-pi-ai/`](llm-pi-ai/README.md) | 多提供方 pi-ai 适配器 | 注册到 `ctx.llm` | +| [`llm/`](llm/README.zh.md) | LLM 服务和共享流式词汇 | `ctx.llm` | +| [`token-meter/`](token-meter/README.zh.md) | 可感知回放的 token 测量 | `ctx.tokenMeter` | +| [`llm-retry/`](llm-retry/README.zh.md) | 提供方作用域的重试策略 | 监听 `agent/request-error` | +| [`llm-deepseek/`](llm-deepseek/README.zh.md) | 直接 DeepSeek 适配器 | 注册到 `ctx.llm` | +| [`llm-pi-ai/`](llm-pi-ai/README.zh.md) | 多提供方 pi-ai 适配器 | 注册到 `ctx.llm` | +| [`deepseek-llm-api-extensions/`](deepseek-llm-api-extensions/README.zh.md) | DeepSeek 官方请求字段注册表 | `ctx.deepseekLlmApiExtensions` | +| [`plugin-package-inventory-deepseek/`](plugin-package-inventory-deepseek/README.zh.md) | DeepSeek 官方请求的存活包元数据 | 贡献 `dsh_plugin_packages` | -适配器在 seam 上注册提供方路由;重试与 token 测量仍是独立消费方。子 README 负责路由、元数据、回放和提供方协议细节;[LLM 架构决策](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)说明设计原理。 +适配器在 seam 上注册提供方路由;重试与 token 测量仍是独立消费方。子 README 负责路由、元数据、回放和提供方协议细节;[LLM 架构决策](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.zh.md)说明设计原理。 -子系统参考——消息与内容块、模型请求、`StreamChunk` 协议、适配器约定(adapter contract)——见 [docs/subsystems/llm-streaming.md](../../docs/subsystems/llm-streaming.md)(token 计量:[token-meter.md](../../docs/subsystems/token-meter.md));另见[孪生适配器](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)、[回放 token 计量](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md)与[按路由模型上下文](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) Agent Note。 +子系统参考——消息与内容块、模型请求、`StreamChunk` 协议、适配器约定(adapter contract)——见 [docs/subsystems/llm-streaming.md](../../docs/subsystems/llm-streaming.zh.md)(token 计量:[token-meter.md](../../docs/subsystems/token-meter.zh.md));另见[孪生适配器](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.zh.md)、[回放 token 计量](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md)与[按路由模型上下文](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.zh.md) Agent Note。 diff --git a/packages/llm/deepseek-llm-api-extensions/README.i18n.yaml b/packages/llm/deepseek-llm-api-extensions/README.i18n.yaml new file mode 100644 index 0000000000..85596da12b --- /dev/null +++ b/packages/llm/deepseek-llm-api-extensions/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/llm/deepseek-llm-api-extensions/README.md +README.md: 5680b58ee2baff6a6ee2ba0249a7ea145ee7eb78 +README.zh.md: 9e7ae393b0cc72328101c89a245b8a3ace173503 diff --git a/packages/llm/deepseek-llm-api-extensions/README.md b/packages/llm/deepseek-llm-api-extensions/README.md new file mode 100644 index 0000000000..5680b58ee2 --- /dev/null +++ b/packages/llm/deepseek-llm-api-extensions/README.md @@ -0,0 +1,28 @@ +# @deepseek-ai/dsh-deepseek-llm-api-extensions + +English | [中文](README.zh.md) + +Provider-specific registry for additive top-level fields on official DeepSeek LLM API requests. `DeepSeekLlmApiExtensionRegistry` registers `ctx.deepseekLlmApiExtensions`; contributor plugins claim one declaration-merged field, and `dsh-llm-deepseek` prepares the current contributions after serializing its base request. + +## Service + +- `register(field, provider)` reserves one field for the calling fiber. Duplicate or malformed names fail synchronously; disposing the registration releases it for a later provider. +- `prepare(request)` snapshots the registered providers, prepares them concurrently, clones and freezes returned JSON values, and returns `{ fields, accept }`. A preparation failure rejects before HTTP dispatch; request cancellation stops awaiting providers even when one ignores its signal. +- `accept()` runs every captured post-2xx callback once. Concurrent calls join the same settlement, every callback settles before failures are reported, and several failures become one `AggregateError`. + +Each provider sees the exact serialized base body, the request `AbortSignal`, plus optional `sessionId` and auxiliary-call `purpose`. It must stop its own work promptly after cancellation and returns `undefined` when its field does not apply to that request. A prepared operation retains the providers it captured even if HMR removes their registrations before HTTP acceptance. + +The registry owns addition and lifecycle, not field semantics. `@deepseek-ai/dsh-session-log-deepseek` owns `dsh_session_log`; `@deepseek-ai/dsh-plugin-package-inventory-deepseek` owns `dsh_plugin_packages`. The provider-neutral LLM seam and `llm-pi-ai` do not consume this registry. + +## Model Experience + +Indirectly, through `@deepseek-ai/dsh-llm-deepseek`, which sends registered fields outside the model's `messages`, system prompt, and tool schemas. + +#### KV Cache effect + +None; registry fields are model-hidden provider metadata and do not alter the serialized model-input prefix. + +## Known Limitations and Deferred Work + +- **Official DeepSeek requests only** — the registry intentionally has no provider-neutral routing or pi-ai adapter integration. +- **No field ordering contract** — JSON object member order follows registration preparation but receivers address fields by name. diff --git a/packages/llm/deepseek-llm-api-extensions/README.zh.md b/packages/llm/deepseek-llm-api-extensions/README.zh.md new file mode 100644 index 0000000000..9e7ae393b0 --- /dev/null +++ b/packages/llm/deepseek-llm-api-extensions/README.zh.md @@ -0,0 +1,28 @@ +# @deepseek-ai/dsh-deepseek-llm-api-extensions + +[English](README.md) | 中文 + +用于向 DeepSeek 官方 LLM API 请求添加顶层字段的提供方特定注册表。`DeepSeekLlmApiExtensionRegistry` 注册 `ctx.deepseekLlmApiExtensions`;贡献插件分别认领一个经声明合并的字段,`dsh-llm-deepseek` 则在序列化基础请求后准备当前贡献。 + +## 服务 + +- `register(field, provider)` 为调用 fiber 保留一个字段。重复或格式错误的名称会同步失败;dispose(资源释放)该注册后,后续提供方可以再次认领。 +- `prepare(request)` 对已注册提供方取快照,并发准备贡献,克隆并冻结返回的 JSON 值,然后返回 `{ fields, accept }`。准备失败会在 HTTP 分发前拒绝请求;请求取消后,即使某个提供方忽略信号,注册表也会停止等待。 +- `accept()` 对每个捕获的 2xx 后回调只运行一次。并发调用会等待同一次结算,所有回调都在报告失败前完成,多个失败会合并为一个 `AggregateError`。 + +每个提供方都会看到确切的已序列化基础正文、请求 `AbortSignal`,以及可选的 `sessionId` 与辅助调用 `purpose`。提供方必须在取消后迅速停止自身工作;字段不适用于当前请求时返回 `undefined`。即使 HMR(热模块替换)在 HTTP 接受前移除了注册,已准备的操作仍会保留其捕获的提供方。 + +注册表拥有字段添加与生命周期,不拥有字段语义。`@deepseek-ai/dsh-session-log-deepseek` 拥有 `dsh_session_log`;`@deepseek-ai/dsh-plugin-package-inventory-deepseek` 拥有 `dsh_plugin_packages`。提供方无关的 LLM seam 与 `llm-pi-ai` 都不消费该注册表。 + +## 模型体验 + +通过 `@deepseek-ai/dsh-llm-deepseek` 间接生效;该包在模型的 `messages`、系统提示词与工具 schema 之外发送已注册字段。 + +#### KV Cache 影响 + +无;注册表字段是模型不可见的提供方元数据,不改变已序列化的模型输入前缀。 + +## 已知限制与暂缓事项 + +- **仅限 DeepSeek 官方请求**——该注册表刻意不提供提供方无关的路由,也不集成 pi-ai 适配器。 +- **不约定字段顺序**——JSON 对象成员顺序取决于注册准备顺序,但接收方按名称寻址字段。 diff --git a/packages/client/web-react/package.json b/packages/llm/deepseek-llm-api-extensions/package.json similarity index 65% rename from packages/client/web-react/package.json rename to packages/llm/deepseek-llm-api-extensions/package.json index 0fcc91e373..b82d01328d 100644 --- a/packages/client/web-react/package.json +++ b/packages/llm/deepseek-llm-api-extensions/package.json @@ -1,14 +1,14 @@ { - "name": "@deepseek-ai/dsh-client-web-react", - "description": "Shell-side React glue: createSlotRenderer, SessionProvider, bindSnapshotSelector (uSES bridge), useInvoke", - "version": "0.0.1-rc.5", + "name": "@deepseek-ai/dsh-deepseek-llm-api-extensions", + "description": "Additive request-field registry for the official DeepSeek LLM API adapter", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", - "directory": "packages/client/web-react" + "directory": "packages/llm/deepseek-llm-api-extensions" }, "type": "module", "main": "lib/index.js", @@ -22,27 +22,26 @@ "types": "./lib/types/invariant.d.ts", "default": "./lib/invariant.js" }, + "./types": { + "types": "./lib/types/types.d.ts", + "default": "./lib/types/types.js" + }, "./src/*": "./src/*", "./package.json": "./package.json" }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.js", + "lib/types/**/*.d.ts" + ], "license": "MIT", - "dependencies": { - "@deepseek-ai/dsh-client-ui-slots": "workspace:^", - "react": "^18.2.0", - "use-sync-external-store": "1.2.0" - }, "peerDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/cordis": "workspace:^" }, "devDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", - "@types/react": "~18.3.1", "@deepseek-ai/cordis": "workspace:^" - }, - "files": [ - "lib/index.js", - "lib/invariant.js", - "lib/types/**/*.d.ts" - ] + } } diff --git a/packages/llm/deepseek-llm-api-extensions/src/index.ts b/packages/llm/deepseek-llm-api-extensions/src/index.ts new file mode 100644 index 0000000000..cf548f7bee --- /dev/null +++ b/packages/llm/deepseek-llm-api-extensions/src/index.ts @@ -0,0 +1,132 @@ +/** + * DeepSeek LLM API extension registry: plugins own independent top-level request + * fields while the official adapter performs one preparation and acceptance transaction. + * @module @deepseek-ai/dsh-deepseek-llm-api-extensions + */ + +import { Context, Service } from '@deepseek-ai/cordis' +import type { + DeepSeekLlmApiExtensionMap, + DeepSeekLlmApiExtensionProvider, + DeepSeekLlmApiExtensionRequest, + DeepSeekLlmApiJson, + PreparedDeepSeekLlmApiExtensions, +} from './types.ts' + +export type * from './types.ts' + +declare module '@deepseek-ai/cordis' { + interface Context { + deepseekLlmApiExtensions: DeepSeekLlmApiExtensionRegistry + } +} + +interface ErasedProvider { + prepare(request: DeepSeekLlmApiExtensionRequest): + | { readonly value: DeepSeekLlmApiJson; accept?(): void | Promise } + | undefined + | Promise<{ readonly value: DeepSeekLlmApiJson; accept?(): void | Promise } | undefined> +} + +/** Recursively freeze a fresh structured clone. */ +function freezeJson(value: T): T { + if (value !== null && typeof value === 'object') { + for (const child of Array.isArray(value) ? value : Object.values(value)) freezeJson(child) + Object.freeze(value) + } + return value +} + +/** Settle every acceptance callback before reporting failures. */ +async function acceptAll(callbacks: readonly (() => void | Promise)[]): Promise { + const outcomes = await Promise.allSettled(callbacks.map(callback => Promise.resolve().then(callback))) + const failures: unknown[] = outcomes + .filter((outcome): outcome is PromiseRejectedResult => outcome.status === 'rejected') + .map(outcome => outcome.reason as unknown) + if (failures.length === 1) throw failures[0] + if (failures.length > 1) throw new AggregateError(failures, 'DeepSeek LLM API extension acceptance failed') +} + +/** Stop awaiting provider work when the containing model request is cancelled. */ +async function abortable(work: Promise, signal: AbortSignal): Promise { + signal.throwIfAborted() + const aborted = Promise.withResolvers() + const onAbort = (): void => { aborted.reject(signal.reason) } + signal.addEventListener('abort', onAbort, { once: true }) + try { + const result = await Promise.race([work, aborted.promise]) + signal.throwIfAborted() + return result + } finally { + signal.removeEventListener('abort', onAbort) + } +} + +/** Registry of independently owned top-level fields for official DeepSeek requests. */ +export class DeepSeekLlmApiExtensionRegistry extends Service { + private readonly providers = new Map() + + constructor(ctx: Context) { + super(ctx, 'deepseekLlmApiExtensions') + } + + /** + * Register the sole provider of one top-level request field. Registration is effect-scoped. + * @param field - declaration-merged field owned by the provider. + * @param provider - request-time field preparation and optional acceptance behavior. + * @returns disposer that releases the field. + */ + register( + field: K, + provider: DeepSeekLlmApiExtensionProvider, + ): () => Promise { + const fieldName = field as string + if (fieldName.length === 0 || fieldName.trim() !== fieldName) { + throw new Error('deepseek-llm-api-extensions: field must be a non-blank trimmed string') + } + const providers = this.providers + const erased = provider as ErasedProvider + const dispose = this.ctx.effect(() => { + if (providers.has(fieldName)) { + throw new Error(`deepseek-llm-api-extensions: field ${JSON.stringify(fieldName)} is already registered`) + } + providers.set(fieldName, erased) + return () => { + providers.delete(fieldName) + } + }, `deepseekLlmApiExtensions.register(${JSON.stringify(fieldName)})`) + return dispose + } + + /** + * Prepare every currently registered field from one immutable base request. + * Preparation failures reject before HTTP dispatch. Field values are cloned and frozen; + * providers retain no mutable alias to the outgoing request. + * @param request - exact serialized request facts before extension fields. + * @returns detached fields and their idempotent joint acceptance transaction. + */ + async prepare(request: DeepSeekLlmApiExtensionRequest): Promise { + request.signal.throwIfAborted() + const entries = [...this.providers.entries()] + const prepared = await abortable(Promise.all(entries.map(async ([field, provider]) => ({ + field, + result: await provider.prepare(request), + }))), request.signal) + const fields: Record = Object.create(null) as Record + const callbacks: Array<() => void | Promise> = [] + for (const { field, result } of prepared) { + if (result === undefined) continue + fields[field] = freezeJson(structuredClone(result.value)) + const accept = result.accept + if (accept !== undefined) callbacks.push(accept.bind(result)) + } + Object.freeze(fields) + let acceptance: Promise | undefined + return { + fields, + accept: () => acceptance ??= acceptAll(callbacks), + } + } +} + +export default DeepSeekLlmApiExtensionRegistry diff --git a/packages/llm/deepseek-llm-api-extensions/src/invariant.ts b/packages/llm/deepseek-llm-api-extensions/src/invariant.ts new file mode 100644 index 0000000000..ed743eed65 --- /dev/null +++ b/packages/llm/deepseek-llm-api-extensions/src/invariant.ts @@ -0,0 +1,27 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-deepseek-llm-api-extensions`. */ + +/* jscpd:ignore-start */ +import type { Context } from '@deepseek-ai/cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-deepseek-llm-api-extensions' + +/** Cordis companion plugin name. */ +export const name = 'deepseek-llm-api-extensions-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: duplicate ownership, detached output, and one acceptance + * settlement are enforced inside the registry operation that owns each decision. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/llm/deepseek-llm-api-extensions/src/types.ts b/packages/llm/deepseek-llm-api-extensions/src/types.ts new file mode 100644 index 0000000000..1e3ccc492b --- /dev/null +++ b/packages/llm/deepseek-llm-api-extensions/src/types.ts @@ -0,0 +1,59 @@ +/** Provider-specific JSON and contribution types for DeepSeek request extensions. */ + +/** Lossless JSON value accepted by the DeepSeek request body. */ +export type DeepSeekLlmApiJson = + | null + | boolean + | number + | string + | DeepSeekLlmApiJson[] + | { [key: string]: DeepSeekLlmApiJson } + +/** + * Merge-extensible table of top-level DeepSeek request extension fields. + * Contributor packages declaration-merge the field they own. + */ +export interface DeepSeekLlmApiExtensionMap {} + +/** Exact serialized request facts visible to extension providers. */ +export interface DeepSeekLlmApiExtensionRequest { + /** Base DeepSeek request body before extension fields are merged. */ + readonly body: Readonly> + /** Session identity carried by the model request, when present. */ + readonly sessionId?: string + /** Auxiliary request classification, when present. */ + readonly purpose?: 'compaction' | 'session-title' + /** Cancellation for request preparation; providers must stop promptly after abort. */ + readonly signal: AbortSignal +} + +/** One prepared field value and its optional post-2xx commit. */ +export interface PreparedDeepSeekLlmApiExtension { + /** Detached value merged under the provider's registered field. */ + readonly value: T + /** Commit state that depends on confirmed provider acceptance. */ + accept?(): void | Promise +} + +/** Provider registered under one key of {@link DeepSeekLlmApiExtensionMap}. */ +export interface DeepSeekLlmApiExtensionProvider { + /** + * Prepare one field for an exact serialized request. + * @param request - immutable base request facts. + * @returns the prepared field, or `undefined` when this request has no value for it. + */ + prepare( + request: DeepSeekLlmApiExtensionRequest, + ): PreparedDeepSeekLlmApiExtension | undefined | Promise | undefined> +} + +/** All fields prepared for one request plus their joint acceptance transaction. */ +export interface PreparedDeepSeekLlmApiExtensions { + /** Detached top-level fields to merge into the base request. */ + readonly fields: Readonly> + /** + * Commit every captured provider after HTTP 2xx. Repeated calls join the same settlement. + * @returns fulfillment after every commit succeeds. + */ + accept(): Promise +} diff --git a/packages/llm/deepseek-llm-api-extensions/tests/registry.spec.ts b/packages/llm/deepseek-llm-api-extensions/tests/registry.spec.ts new file mode 100644 index 0000000000..85cc97525c --- /dev/null +++ b/packages/llm/deepseek-llm-api-extensions/tests/registry.spec.ts @@ -0,0 +1,154 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import DeepSeekLlmApiExtensionRegistry from '../src/index.ts' + +declare module '@deepseek-ai/dsh-deepseek-llm-api-extensions/types' { + interface DeepSeekLlmApiExtensionMap { + test_alpha: { readonly value: string } + test_beta: readonly number[] + } +} + +const contexts: Context[] = [] +const SIGNAL = new AbortController().signal + +afterEach(async () => { + await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose())) +}) + +async function harness(): Promise { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(DeepSeekLlmApiExtensionRegistry) + return ctx +} + +describe('DeepSeekLlmApiExtensionRegistry', () => { + it('prepares detached fields and accepts every provider exactly once', async () => { + const ctx = await harness() + const first = vi.fn() + const second = vi.fn() + const mutable = { value: 'original' } + ctx.deepseekLlmApiExtensions.register('test_alpha', { + prepare: () => ({ + value: mutable, + accept: first, + }), + }) + ctx.deepseekLlmApiExtensions.register('test_beta', { + prepare: async request => ({ + value: [request.body.messages === undefined ? 0 : 1], + accept: async () => { second() }, + }), + }) + + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL, sessionId: 's' }) + mutable.value = 'changed' + expect(prepared.fields).toEqual({ test_alpha: { value: 'original' }, test_beta: [1] }) + expect(Object.isFrozen(prepared.fields)).toBe(true) + expect(Object.isFrozen(prepared.fields.test_alpha)).toBe(true) + + await Promise.all([prepared.accept(), prepared.accept()]) + expect(first).toHaveBeenCalledTimes(1) + expect(second).toHaveBeenCalledTimes(1) + }) + + it('preserves the prepared result as an acceptance method receiver', async () => { + const ctx = await harness() + const result = { + value: { value: 'receiver' }, + accepted: 0, + accept(): void { + this.accepted += 1 + }, + } + ctx.deepseekLlmApiExtensions.register('test_alpha', { prepare: () => result }) + + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: {}, signal: SIGNAL }) + await prepared.accept() + expect(result.accepted).toBe(1) + }) + + it('rejects duplicate fields and releases ownership with the registering fiber', async () => { + const ctx = await harness() + const owner = ctx.extend() + const dispose = owner.deepseekLlmApiExtensions.register('test_alpha', { + prepare: () => ({ value: { value: 'one' } }), + }) + expect(() => ctx.deepseekLlmApiExtensions.register('test_alpha', { + prepare: () => ({ value: { value: 'two' } }), + })).toThrow(/already registered/) + + await dispose() + ctx.deepseekLlmApiExtensions.register('test_alpha', { + prepare: () => ({ value: { value: 'replacement' } }), + }) + await expect(ctx.deepseekLlmApiExtensions.prepare({ body: {}, signal: SIGNAL })) + .resolves.toMatchObject({ fields: { test_alpha: { value: 'replacement' } } }) + }) + + it('settles every acceptance callback before reporting one or several failures', async () => { + const ctx = await harness() + const later = vi.fn() + ctx.deepseekLlmApiExtensions.register('test_alpha', { + prepare: () => ({ + value: { value: 'x' }, + accept: () => { throw new Error('alpha failed') }, + }), + }) + ctx.deepseekLlmApiExtensions.register('test_beta', { + prepare: () => ({ + value: [2], + accept: () => { later(); throw new Error('beta failed') }, + }), + }) + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: {}, signal: SIGNAL }) + await expect(prepared.accept()).rejects.toMatchObject({ + errors: [expect.objectContaining({ message: 'alpha failed' }), expect.objectContaining({ message: 'beta failed' })], + }) + expect(later).toHaveBeenCalledOnce() + }) + + it('reports a single acceptance failure verbatim and omits an undefined contribution', async () => { + const ctx = await harness() + const failure = new Error('single failure') + ctx.deepseekLlmApiExtensions.register('test_alpha', { + prepare: () => ({ value: { value: 'x' }, accept: () => { throw failure } }), + }) + ctx.deepseekLlmApiExtensions.register('test_beta', { prepare: () => undefined }) + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: {}, signal: SIGNAL }) + expect(prepared.fields).toEqual({ test_alpha: { value: 'x' } }) + await expect(prepared.accept()).rejects.toBe(failure) + }) + + it('rejects invalid field names and preparation failures before returning fields', async () => { + const ctx = await harness() + expect(() => ctx.deepseekLlmApiExtensions.register('' as 'test_alpha', { + prepare: () => ({ value: { value: 'x' } }), + })).toThrow(/non-blank trimmed/) + ctx.deepseekLlmApiExtensions.register('test_alpha', { + prepare: () => { throw new Error('prepare failed') }, + }) + await expect(ctx.deepseekLlmApiExtensions.prepare({ body: {}, signal: SIGNAL })).rejects.toThrow('prepare failed') + }) + + it('stops waiting for a provider that ignores request cancellation', async () => { + const ctx = await harness() + const controller = new AbortController() + const started = Promise.withResolvers() + ctx.deepseekLlmApiExtensions.register('test_alpha', { + prepare: () => { + started.resolve(undefined) + return new Promise(() => {}) + }, + }) + + const pending = ctx.deepseekLlmApiExtensions.prepare({ + body: {}, + signal: controller.signal, + }) + await started.promise + controller.abort(new Error('cancelled during extension preparation')) + await expect(pending).rejects.toBe(controller.signal.reason) + }, 500) +}) diff --git a/packages/llm/deepseek-llm-api-extensions/tsconfig.json b/packages/llm/deepseek-llm-api-extensions/tsconfig.json new file mode 100644 index 0000000000..bb46910c07 --- /dev/null +++ b/packages/llm/deepseek-llm-api-extensions/tsconfig.json @@ -0,0 +1,21 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../runtime-diagnostics/invariants" + } + ] +} diff --git a/packages/llm/llm-deepseek/README.i18n.yaml b/packages/llm/llm-deepseek/README.i18n.yaml index 9f1f57ef90..8076a692e6 100644 --- a/packages/llm/llm-deepseek/README.i18n.yaml +++ b/packages/llm/llm-deepseek/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md -README.md: 1923be694c20149fe5387bd7bb4dac5da32a63a4 -README.zh.md: bdce7f2b2fdff0e1542deb3e7fae5c5e7139d13b +README.md: 7d50a8e99863637a06abf49d26a6eeb419cf1bbd +README.zh.md: c1571149529a2d4e54d67b10f63b60bb2aa1abfd diff --git a/packages/llm/llm-deepseek/README.md b/packages/llm/llm-deepseek/README.md index 1923be694c..7d50a8e998 100644 --- a/packages/llm/llm-deepseek/README.md +++ b/packages/llm/llm-deepseek/README.md @@ -17,50 +17,82 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire apiKeyEnv: DEEPSEEK_API_KEY # default; resolved per request via ctx.credentials, then the environment baseURL: https://api.deepseek.com # optional; $DEEPSEEK_BASE_URL then the public API when omitted thinking: enabled # optional; provider default is enabled - reasoningEffort: high # optional; off | high | max — omitted ⇒ high + reasoningEffort: high # optional; off | low | high | max — omitted ⇒ high maxTokens: 256000 # optional positive per-request output cap; this is the default streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default - retryPolicy: # optional; omission uses bounded normal defaults + maxRequestFilesBytes: 134217728 # optional positive integer; 128 MiB raw request-image default + maxInlineRequestImageBytes: 20971520 # base64 fallback high watermark; 20 MiB default + maxImagesPerRequest: 600 # provider request image-count limit + imageOffloadByteQuantum: 67108864 # oldest-image removal advances in 64 MiB steps + inlineImageOffloadByteQuantum: 10485760 # fallback removal advances in 10 MiB steps + imageOffloadCountQuantum: 20 # count overflow advances in 20-image steps + filesApiTimeoutMs: 60000 # per-image Files resolution deadline; one-minute default + fileExpiresAfterSeconds: 604800 # uploaded image lifetime; 1 hour to 30 days + fileRefreshMarginSeconds: 3600 # replace ids with less lifetime remaining + fileQuotaCleanupBatch: 100 # oldest harness-owned files deleted before one quota retry + retryPolicy: # optional; omission uses normal mode with five retries mode: always # normal | always backoff: initialDelayMs: 500 maxDelayMs: 10000 jitterRatio: 0.1 defaultContextWindow: 1000000 # optional positive-integer fallback; this is the default - models: # optional; defaults to V4 Flash and V4 Pro + models: # optional; defaults to V4 Flash, V4 Pro, and V4 Flash Vision Exp - id: deepseek-v4-flash name: DeepSeek-V4-Flash + - id: deepseek-v4-flash-vision-exp + name: DeepSeek-V4-Flash-Vision-Exp + inputModalities: [text, image] + imagePixelBudget: 640000 + imageMaxBytes: 1048576 - id: private-reasoner description: Company-hosted reasoning model contextWindow: 512000 ``` -The plugin registers the single provider route `deepseek-official` together with its resolved `retryPolicy`. A request selects it with `provider: deepseek-official`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` as `DeepSeek-V4-Flash` and `deepseek-v4-pro` as `DeepSeek-V4-Pro`, each with a 1,000,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek-official')` for clients such as ACP editors and the Web selector, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id. +The plugin registers the single provider route `deepseek-official` together with its resolved `retryPolicy`; omission resolves to normal mode with five retries. A request selects it with `provider: deepseek-official`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash`, `deepseek-v4-pro`, and the image-capable `deepseek-v4-flash-vision-exp`, each with a 1,000,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek-official')` for clients such as ACP editors and the Web selector, but remain advisory: unlisted model ids still pass through unchanged as text-only routes. An omitted entry name defaults to its id, and omitted `inputModalities` means `text` only. + +An image-capable catalog entry declares `inputModalities: [text, image]` and may set `imagePixelBudget`, `imageMaxBytes`, or `imageDetail: low`. The ordinary default is 640,000 total pixels and 1MiB encoded bytes; low detail defaults to 512 by 512 total pixels. The attachment store scales by `min(1, sqrt(pixelBudget / (width * height)))` and rounds inward to keep the pixel count at or below the hard cap, so a 2048 by 1024 normalized attachment becomes about 1130 by 565 instead of a forced square. Request encoders run lazily: low-color images try PNG (palette only without alpha) then WebP 85 and 80, other alpha images try WebP 85 then 80, and other opaque images try JPEG 85 then 80; dimensions shrink only when both quality attempts exceed 1MiB. Concurrent generation of one `variantId` shares one transform. A caller can cancel its own wait without interrupting other waiters; the transform stops when no waiter remains. The adapter normally uploads the exact derived request bytes through `POST /files` and sends `{type: "file", file_id}` blocks. A failed or timed-out file-id resolution rebuilds the whole chat request with those same request versions as base64 data URLs; one request never mixes file ids and inline images. Every retained image is preceded by stable text naming the complete attachment id and actual request dimensions. User, tool-result, agent-loop, compaction, and direct `ctx.llm.stream` requests all use this projection. Text-only routes receive stable attachment placeholders while durable history keeps its image references. + +`maxRequestFilesBytes` and `maxImagesPerRequest` bound the retained request versions at 128MiB and 600 images by default. The byte and count quanta must not exceed their corresponding bounds. Before attachment reads, the adapter uses each route's request-version byte cap as a conservative upper bound and removes the oldest over-budget prefix; only retained normalized attachments are read and transformed. Exact derived lengths are checked again without restoring omitted images. When the byte bound is crossed, the oldest prefix advances past the next 64MiB boundary; 129 one-megabyte images remove the oldest 65 and retain 64MiB, and that prefix stays unchanged until durable history exceeds 192MiB. Count overflow advances independently in `imageOffloadCountQuantum` steps. Removed images become the fixed model-visible placeholder `[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]`. This high-watermark projection avoids changing an old request prefix after every new image. + +Inline fallback has an independent base64 budget. `maxInlineRequestImageBytes` defaults to 20MiB and `inlineImageOffloadByteQuantum` to 10MiB, so a history of 21 one-megabyte base64 payloads removes the oldest 11 and retains 10MiB. The calculation uses base64-expanded lengths. The prepared request versions are reused byte-for-byte; fallback does not decode or compress an image again. Successful mappings created before a later image fails remain indexed for future requests. + +Uploaded ids are indexed below `DSH_HOME` by endpoint/API-key scope and request `variantId`. The variant covers the normalized attachment id, transform version, route pixel and byte budgets, and encoder parameters, so Files API and inline fallback refer to the same deterministic bytes. Uploads request a seven-day lifetime by default and store the server's `expires_at`. A local mapping with no more than one hour remaining is replaced before use; the adapter does not retrieve every remote file before chat. If chat reports expired, deleted, missing, or invalid file ids and names one or more ids used by the request, the adapter removes exactly those mappings. If the provider identifies stale file state without naming an id, it removes every file mapping used by that chat attempt. It then uploads the affected request versions again and retries chat once. A second stale-file rejection clears the mappings identified by that response and is returned without a third chat attempt. An upload response without a complete file object, matching byte count, and `expires_at` is never indexed; a later request therefore uploads again instead of trusting inconsistent local state. A malformed local upload index is treated as an empty cache and replaced by the next successful upload. File resolution, including local index access and remote upload, has a per-image one-minute deadline by default. The default five-minute stream idle deadline therefore leaves time for inline fallback; a deployment may configure a shorter stream idle deadline when it wants that outer deadline to terminate the request first. Each successful resolution refreshes the outer idle watchdog. Any resolution failure switches that request to inline mode, while explicit public file-management operations continue to report their own failures. + +Concurrent resolution of one scoped `variantId` shares one Files upload with waiter-local cancellation. One quota upload failure first paginates and collects the configured number of oldest `dsh-` files, then deletes that set before one upload retry. `DeepSeekFilesClient.delete`, `DeepSeekFileStore.release`, and `releaseAll` expose explicit remote-space reclamation. The current provider limits represented by this package are 128MiB per Files upload, 32MiB per chat-referenced image, 10,000 stored files, and 25GiB per API key; the default 1MiB request version remains below the two per-file limits. `contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelInfo('deepseek-official', model).context` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. The adapter default is 1,000,000; pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek-official` throws `LlmError('DUPLICATE_ADAPTER')`. `maxTokens` is the adapter-configured output cap for conversation requests and defaults to 256,000. A catalog entry may carry its own `maxTokens`, which wins for that model; an entry without one, and any unlisted pass-through id, resolve to the profile value, so adding a per-model cap changes one model rather than the route. Exact-model resolution exposes the winner as `defaultMaxTokens`; `LlmRuntime` materializes that value into `GenerateOptions.maxTokens` before the agent loop writes `request/header`, so the wire request remains reconstructable. An explicit request or `AgentOptions.maxTokens` value wins and is serialized as `max_tokens`. The adapter does not clamp this request budget against `contextWindow`; deployments with a smaller context or provider output limit must configure a compatible `maxTokens`. -The same exact-model result exposes ordered `off`, `high`, and `max` efforts under `reasoning` for every pass-through model when deployment policy permits thinking. `reasoningEffort` selects the deployment default and falls back to `high` when omitted. `agent/request` can replace it on each conversation step; the resolved value is logged in `request/header`. `high` and `max` enable thinking and serialize as the official top-level `reasoning_effort`; adapter-owned `off` instead serializes `thinking.type: disabled` and omits `reasoning_effort`. An unsupported value fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O. +The same exact-model result exposes ordered `off`, `low`, `high`, and `max` efforts under `reasoning` for every pass-through model when deployment policy permits thinking. `reasoningEffort` selects the deployment default and falls back to `high` when omitted. `agent/request` can replace it on each conversation step; the resolved value is logged in `request/header`. `low`, `high`, and `max` enable thinking and serialize as the same official top-level `reasoning_effort` value; adapter-owned `off` instead serializes `thinking.type: disabled` and omits `reasoning_effort`. An unsupported value fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O. -`thinking: disabled` is a deployment lock that publishes only `off` with `off` as its default. Omitting `reasoningEffort` or configuring it as `off` is valid; configuring `high` or `max` fails plugin loading, and a direct per-request attempt to enable thinking fails before network I/O. A request with `GenerateOptions.purpose: 'session-title'` also forces thinking disabled and omits the already-resolved effort, reserving its bounded output for visible title text without changing conversation or compaction defaults. +`thinking: disabled` is a deployment lock that publishes only `off` with `off` as its default. Omitting `reasoningEffort` or configuring it as `off` is valid; configuring `low`, `high`, or `max` fails plugin loading, and a direct per-request attempt to enable thinking fails before network I/O. A request with `GenerateOptions.purpose: 'session-title'` also forces thinking disabled and omits the already-resolved effort, reserving its bounded output for visible title text without changing conversation or compaction defaults. -`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. DeepSeek SSE comments rearm an outstanding read as transport activity but never become `StreamChunk` values or session-log events. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; it registers the configured policy as provider metadata, and `dsh-llm-retry` separately executes it at durable agent-step boundaries. +`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. DeepSeek SSE comments and successful file resolutions rearm an outstanding read as transport activity but never become `StreamChunk` values or session-log events. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter normally makes one chat request per `stream()` call and makes a second only for stale-file recovery. A file-resolution failure before the first chat sends one inline request. If replacement resolution fails after a stale-file response, the inline request is the one permitted retry. It registers the configured retry policy as provider metadata, and `dsh-llm-retry` separately executes that policy at durable agent-step boundaries. ## Dynamic configuration (settings + credentials) -Connection facts are not frozen at load. `resolveAdapterOptions` is the one explicit resolve step from raw config to validated facts, and the adapter re-reads them through a thunk **once per operation**: base URL, catalog, request defaults, and idle budget all take effect on the next request, while an in-flight stream keeps the facts it started with. Two optional seams feed that thunk: +Connection facts are not frozen at load. `resolveAdapterOptions` is the one explicit resolve step from raw config to validated facts, and the adapter re-reads them through a thunk **once per operation**: base URL, catalog, request defaults, image and Files policies, and idle budget all take effect on the next request, while an in-flight stream keeps the facts it started with. Three optional seams feed that thunk: - **`ctx.settings`** — the plugin registers the `llm-deepseek` namespace with this same `Config` schema and its `cordis.yml` entry as the composition `base`, so a `llm-deepseek:` section in the user settings document overrides any field without a restart. Without a mounted settings service the entry config alone drives the adapter, unchanged. A live settings snapshot that passes the schema but fails a beyond-schema bound (a duplicate catalog id, a broken thinking/effort pair) keeps the last good facts and logs the failure; the entry config itself still fails plugin load. - **`ctx.credentials`** — the API key resolves per stream call, from the *same* resolved snapshot that supplies the endpoint. Configuration carries only `apiKeyEnv`, never a literal key: the reference resolves through the credential seam, and without a mounted seam through the trusted environment layers. Because credential facts travel with the connection facts, a settings snapshot the resolver rejects contributes neither its endpoint nor its key: the whole previous generation keeps serving. Every resolved key is format-checked before use, so a value no HTTP header can carry is refused with `LlmError('INVALID_CREDENTIAL')` naming the failing entry point — never any part of the key — instead of surfacing as an opaque `fetch` `TypeError`. A request with no key anywhere fails with `MISSING_CREDENTIAL` naming every configuration entry point, while the route stays registered and the catalog stays browsable — first-run onboarding is "browse models, store the key, prompt again", with no restart between. +- **`ctx.attachments`** — image requests resolve this service at request time, so Cordis load order does not freeze optional image availability. Absence rejects image input with `UNSUPPORTED_CONTENT`; text-only calls do not require the service. The one registration-captured fact is the retry policy: when its resolved value changes, the plugin re-registers the route in place (same adapter instance, one synchronous section), so `ctx.llm.providerRetryPolicy('deepseek-official')` always reports the current policy. The plugin also declares its route in the configurable-provider directory (`ctx.llm.listConfigurableProviders()`): provider `deepseek-official`, settings namespace `llm-deepseek`, empty settings path — the whole section is the profile. Configuration surfaces use that entry to offer this adapter alongside dormant pi-ai providers. +## DeepSeek request extensions + +When `ctx.deepseekLlmApiExtensions` is present, the adapter prepares its registered top-level fields after serializing the exact wire messages and before `fetch`. The same request signal reaches providers, and cancellation stops waiting even when a provider ignores it. Preparation failures and field collisions fail before HTTP with `REQUEST_EXTENSION`. After HTTP 2xx, the adapter awaits the prepared acceptance transaction before consuming the SSE body; an acceptance failure uses the same code, while transport and non-2xx failures do not accept the fields. Fields go to the resolved `baseURL`, including a configured gateway. A composition without the registry sends the base DeepSeek request unchanged. + +Shipped profiles and runnable examples mount [`@deepseek-ai/dsh-session-log-deepseek`](../../session/session-log-deepseek/README.md) for the incremental `dsh_session_log` field and [`@deepseek-ai/dsh-plugin-package-inventory-deepseek`](../plugin-package-inventory-deepseek/README.md) for the complete active `dsh_plugin_packages` field. Session-log upload defaults off and requires `session-log-deepseek.enabled: true`; package metadata defaults on. Both remain model-hidden. `llm-pi-ai` neither imports nor calls this provider-specific registry. + ## App attribution -Every request carries the shared attribution header from dsh-llm's `attributionHeaders()` - the mandatory `User-Agent` baseline identifying the harness (see [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts)). Direct DeepSeek requests and OpenAI-compatible gateway requests get no provider-specific app-attribution headers under this adapter contract; OpenRouter app attribution is deferred to a future explicit OpenRouter adapter or mode. A request whose `GenerateOptions.purpose` is `compaction` (dsh-compaction-basic's auxiliary summarization call) additionally carries `x-deepseek-harness-compact: 1`, so the host can separate compaction traffic from conversation requests. +Every chat and Files API request carries the shared attribution header from dsh-llm's `attributionHeaders()`, the mandatory `User-Agent` baseline identifying the harness (see [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts)). Direct DeepSeek requests and OpenAI-compatible gateway requests get no provider-specific app-attribution headers under this adapter contract; OpenRouter app attribution is deferred to a future explicit OpenRouter adapter or mode. A request whose `GenerateOptions.purpose` is `compaction` (dsh-compaction-basic's auxiliary summarization call) additionally carries `x-deepseek-harness-compact: 1`, so the host can separate compaction traffic from conversation requests. DeepSeek request identity is separate from app attribution. After credential resolution, every provider request carries `x-deepseek-harness-user-id` with the stable anonymous id from [`@deepseek-ai/dsh-anonymous-user-id`](../../identity/anonymous-user-id/README.md); a request carrying `GenerateOptions.sessionId` also sends that exact value as `x-deepseek-harness-session-id`, while a direct call without a session omits the session header. Both headers go to the resolved `baseURL`, including a configured gateway, and remain outside the request body and model-visible content. @@ -69,12 +101,13 @@ DeepSeek request identity is separate from app attribution. After credential res - Streaming only (`stream_options.include_usage` always on). `usage` may arrive attached to the finish chunk or as a trailing usage-only chunk — the translator defers both to `[DONE]`, so `usage` always precedes `finish` and nothing follows `finish`. - The adapter-owned `off` effort maps to `thinking: {type: 'disabled'}` and never crosses the wire as `reasoning_effort: 'off'`. - The first thinking-mode chunk carries `reasoning_content: ""` — handled (no spurious reasoning block). -- **Reasoning passback rule**: on assistant turns that carried tool calls, `reasoning_content` is serialized back in history (required by the API in thinking mode); on tool-call-free turns it is dropped (ignored anyway — saves tokens). +- **Reasoning passback rule**: every assistant turn that carried reasoning serializes `reasoning_content` back in history. Thinking mode requires it on tool-call turns; DeepSeek ignores it elsewhere, while a gateway re-encoding the conversation for another vendor recovers that turn's upstream thinking signature by hashing the replayed text. +- Image-capable user messages preserve text/image order. Tool-role content remains a string; consecutive tool-result images are grouped into the following user message with `Attached image(s) from tool result:`. - Cache accounting: `cacheReadTokens` ← `prompt_cache_hit_tokens` / `prompt_tokens_details.cached_tokens`; DeepSeek reports no cache-write metric. ## Errors -Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA` (a response whose provider details identify exhausted quota, balance, or credits), `RATE_LIMIT` (other 429s), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_` otherwise. Its serializable `failure` retains the HTTP status plus a valid positive `Retry-After` seconds/date delay and `x-request-id` / `x-deepseek-request-id` when present. A pre-response transport failure (DNS, refused connection, TLS, proxy) throws `TRANSPORT` naming the configured endpoint and chaining the original rejection as `cause`; caller aborts throw `ABORTED`, and the loop's cancellation signal remains authoritative. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', failure}` chunks, and a completed stream whose `stop` (or absent) finish opened no content blocks becomes a `finish {kind: 'error'}` with code `EMPTY_RESPONSE` (retried by default policy). +Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA` (a response whose provider details identify exhausted quota, balance, or credits), `RATE_LIMIT` (other 429s), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s and 413), `SERVER` (5xx), `HTTP_` otherwise. Its serializable `failure` retains the HTTP status plus a valid positive `Retry-After` seconds/date delay and `x-request-id` / `x-deepseek-request-id` when present. Extension preparation, base-field collision, or post-2xx acceptance fails with `REQUEST_EXTENSION`; no extension failure is relabelled as transport. If DeepSeek rejects a normalized image, the primary message names the attachment or display name, durable message and image position, normalized media type, 8-bit sRGB/sRGBA depth, dimensions, and provider message. With several candidates and no file id in the provider detail, it lists each possible image instead of assigning the failure to the first one. The raw response remains the error `cause`; it is never the only user-visible diagnostic. Attachment reads retain their stable attachment failure code rather than becoming transport failures. A pre-response transport failure (DNS, refused connection, TLS, proxy) throws `TRANSPORT` naming the configured endpoint and chaining the original rejection as `cause`; caller aborts throw `ABORTED`, and the loop's cancellation signal remains authoritative. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', failure}` chunks, and a completed stream whose `stop` (or absent) finish opened no content blocks becomes a `finish {kind: 'error'}` with code `EMPTY_RESPONSE` (retried by default policy). ## Model Experience @@ -82,15 +115,15 @@ Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA` #### What the model sees -The selected DeepSeek model receives the harness system prompt, message history, tool schemas, stop sequences, and call config without adapter-authored prompt prose. On a prior assistant turn with tool calls, its reasoning content is passed back as required; reasoning from tool-call-free turns is omitted. +The selected DeepSeek model receives the harness system prompt, message history, tool schemas, stop sequences, and call config without adapter-authored prompt prose. Provider-specific request extension fields remain outside that model input. The vision model normally receives retained user and tool-result images as Files API references beside stable attachment handles and request-image dimensions; a Files resolution failure sends all retained images as inline data URLs instead. An over-budget older image is represented by the documented placeholder. Reasoning content from a prior assistant turn is passed back verbatim, whether or not that turn called a tool. #### Token effect -Provider tokenization governs exact input. Conditional reasoning passback increases tool-round-trip context, while dropping other reasoning avoids paying those tokens again; cache-read usage is reported when available. +Provider tokenization governs exact text and image-token input. Reasoning passback carries every reasoned turn's chain of thought into later requests, while dropping over-budget images avoids paying those tokens again; cache-read usage is reported when available. #### KV Cache effect -An unchanged assembled prefix is eligible for DeepSeek cache reuse, which this adapter reports in usage. A model-route change or any upstream prompt, schema, prefix, or history change may prevent reuse from the first changed token; reasoning passback appends during tool round trips. +An unchanged assembled prefix, including deterministically encoded retained images and placeholders, is eligible for DeepSeek cache reuse, which this adapter reports in usage. A model-route change or any upstream prompt, schema, prefix, history, or image-budget change may prevent reuse from the first changed token; reasoning passback appends on every reasoned turn. ### DeepSeek response @@ -111,4 +144,5 @@ Loop-retained response blocks append to the next request and preserve its earlie - **A settings `models` list replaces the composition list wholesale** — settings-layer merging is per-field, and arrays are one field; per-entry catalog merging would need a keyed shape. - **`tool_choice` is not mapped** — not part of the core vocabulary (MVP cut, shared with the pi-ai twin). - **Requests use raw `fetch`, not `@cordisjs/plugin-http`** — no shared proxy/interception configuration; adoption is deferred until a second adapter wants it (`TODO(http)`). -- **Serialization flattens user and tool-result content to text blocks** — plugin-added block types are skipped, and empty tool output crosses the wire as the literal `(no output)`. +- **Plugin-added content block types are skipped** — core text and supported image blocks are serialized, and empty tool output crosses the wire as the literal `(no output)`. +- **Images are input-only durable attachments** — direct external URLs and assistant image output are not supported; DeepSeek input normally uses the Files API and uses inline base64 only for per-request recovery. diff --git a/packages/llm/llm-deepseek/README.zh.md b/packages/llm/llm-deepseek/README.zh.md index bdce7f2b2f..c157114952 100644 --- a/packages/llm/llm-deepseek/README.zh.md +++ b/packages/llm/llm-deepseek/README.zh.md @@ -17,64 +17,97 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器: apiKeyEnv: DEEPSEEK_API_KEY # default; resolved per request via ctx.credentials, then the environment baseURL: https://api.deepseek.com # optional; $DEEPSEEK_BASE_URL then the public API when omitted thinking: enabled # optional; provider default is enabled - reasoningEffort: high # optional; off | high | max — omitted ⇒ high + reasoningEffort: high # optional; off | low | high | max — omitted ⇒ high maxTokens: 256000 # optional positive per-request output cap; this is the default streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default - retryPolicy: # optional; omission uses bounded normal defaults + maxRequestFilesBytes: 134217728 # optional positive integer; 128 MiB raw request-image default + maxInlineRequestImageBytes: 20971520 # base64 fallback high watermark; 20 MiB default + maxImagesPerRequest: 600 # provider request image-count limit + imageOffloadByteQuantum: 67108864 # oldest-image removal advances in 64 MiB steps + inlineImageOffloadByteQuantum: 10485760 # fallback removal advances in 10 MiB steps + imageOffloadCountQuantum: 20 # count overflow advances in 20-image steps + filesApiTimeoutMs: 60000 # per-image Files resolution deadline; one-minute default + fileExpiresAfterSeconds: 604800 # uploaded image lifetime; 1 hour to 30 days + fileRefreshMarginSeconds: 3600 # replace ids with less lifetime remaining + fileQuotaCleanupBatch: 100 # oldest harness-owned files deleted before one quota retry + retryPolicy: # optional; omission uses normal mode with five retries mode: always # normal | always backoff: initialDelayMs: 500 maxDelayMs: 10000 jitterRatio: 0.1 defaultContextWindow: 1000000 # optional positive-integer fallback; this is the default - models: # optional; defaults to V4 Flash and V4 Pro + models: # optional; defaults to V4 Flash, V4 Pro, and V4 Flash Vision Exp - id: deepseek-v4-flash name: DeepSeek-V4-Flash + - id: deepseek-v4-flash-vision-exp + name: DeepSeek-V4-Flash-Vision-Exp + inputModalities: [text, image] + imagePixelBudget: 640000 + imageMaxBytes: 1048576 - id: private-reasoner description: Company-hosted reasoning model contextWindow: 512000 ``` -该插件注册唯一提供方路由 `deepseek-official`,同时注册解析后的 `retryPolicy`。请求使用 `provider: deepseek-official` 选择该路由;其 `model` 会作为协议 `model` 字符串原样传递,因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash`(名称为 `DeepSeek-V4-Flash`)和 `deepseek-v4-pro`(名称为 `DeepSeek-V4-Pro`),两者的上下文窗口均为 1,000,000 token;显式列表会替换这些默认值,`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek-official')` 公开给 ACP(Agent Client Protocol)编辑器和 Web 选择器等客户端,但仍只提供建议:未列出模型 id 仍原样传递。省略配置项 name 默认为其 id。 +该插件注册唯一提供方路由 `deepseek-official`,并一同注册解析后的 `retryPolicy`;省略时会解析为 normal 模式并重试五次。请求使用 `provider: deepseek-official` 选择该路由;其 `model` 会作为协议 `model` 字符串原样传递,因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash`、`deepseek-v4-pro` 与支持图片输入的 `deepseek-v4-flash-vision-exp`,三者的上下文窗口均为 1,000,000 token;显式列表会替换这些默认值,`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek-official')` 公开给 ACP(Agent Client Protocol)编辑器和 Web 选择器等客户端,但仍只提供建议:未列出模型 id 仍原样传递,并按纯文本路由处理。省略配置项 name 默认为其 id,省略 `inputModalities` 则表示仅支持 `text`。 + +支持图片的 catalog 配置项声明 `inputModalities: [text, image]`,并可设置 `imagePixelBudget`、`imageMaxBytes` 或 `imageDetail: low`。普通默认值为总像素 640,000、编码字节 1MiB;low detail 的默认总像素为 512×512。附件存储按 `min(1, sqrt(pixelBudget / (width * height)))` 缩放,并向预算内取整,确保总像素不超过硬上限。因此 2048×1024 规范化附件会得到约 1130×565 的请求版本,而不会被强制变成正方形。请求编码按需执行:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。两个质量档均超过 1MiB 时才缩小尺寸。同一 `variantId` 的并发生成共享一次变换。调用方可以单独取消等待,不会中断其他等待方;没有等待方时才会停止变换。适配器通常通过 `POST /files` 上传确切的派生请求字节,再发送 `{type: "file", file_id}` 块。File ID 解析失败或超时后,适配器会用相同请求版本的 base64 data URL 重新组装整个 chat 请求;同一请求不会混用 file ID 和内联图片。每张保留图片前都有稳定文本,写明完整附件 ID 和实际请求尺寸。User、工具结果、agent loop、压缩和直接 `ctx.llm.stream` 请求都使用该投影。纯文本路由会收到稳定的附件占位文本,持久历史继续保留图片引用。 + +`maxRequestFilesBytes` 和 `maxImagesPerRequest` 限制请求中保留的请求版本,默认值分别为 128MiB 和 600 张。字节和数量步长不得超过对应上限。读取附件前,适配器以路由的请求版本字节上限作为保守上界,移除超预算的最旧前缀,只读取并转换保留的规范化附件。系统随后用确切派生长度再次检查,但不会重新加入已省略图片。字节数越过上限时,被移除的最旧前缀会越过下一个 64MiB 边界。由 1MiB 图片组成的历史达到 129MiB 时会移除最旧的 65 张并保留 64MiB;直到持久历史超过 192MiB,这个前缀才再次变化。图片数量超限时则按 `imageOffloadCountQuantum` 独立递增。移除的图片会变成固定模型可见占位文本 `[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]`。这种定量投影不会因每新增一张图片就改写较早的请求前缀。 + +内联回退使用独立的 base64 预算。`maxInlineRequestImageBytes` 默认为 20MiB,`inlineImageOffloadByteQuantum` 默认为 10MiB,因此由 21 个 1MiB base64 负载组成的历史会移除最旧的 11 个并保留 10MiB。计算使用 base64 膨胀后的长度。系统逐字节复用已经准备好的请求版本;回退不会再次解码或压缩图片。前面图片已经成功写入的上传映射会保留,供后续请求复用。 + +上传 ID 按端点和 API key 作用域以及请求 `variantId` 记录在 `DSH_HOME` 下。变体身份覆盖规范化附件 ID、变换策略版本、路由像素和字节预算及编码参数,因此 Files API 和内联回退引用同一份确定性字节。上传默认请求 7 天有效期,并保存服务端返回的 `expires_at`。本地映射剩余时间不超过一小时时会在使用前替换;适配器不会在每次 chat 前查询远端文件。如果 chat 报告文件 ID 已过期、删除、缺失或无效,并指出本次请求使用的一个或多个 ID,适配器只删除这些映射。如果响应只说明文件状态失效而没有指出 ID,适配器会删除该次 chat 使用的全部文件映射。随后重新上传受影响的请求版本,并重试一次 chat。第二次 chat 仍报告文件失效时,适配器会按该响应清理映射并返回错误,不会发起第三次 chat。上传响应若没有完整文件对象、匹配的字节数和 `expires_at`,就不会写入索引;后续请求会再次上传,而不是信任不一致的本地状态。本地上传索引格式损坏时按空缓存处理,并由下一次成功上传替换。文件解析包括本地索引访问和远端上传,默认每张图片的时限为一分钟。默认的 stream idle 时限为五分钟,因此通常有时间执行内联回退;部署可以设置更短的 stream idle 时限,让外层时限先终止请求。每次成功解析都会刷新外层 idle watchdog。任何解析失败都会把该请求切换到内联模式;显式公共文件管理操作仍会报告自身错误。 + +同一作用域和 `variantId` 的并发解析共享一次 Files 上传,每个等待方可以单独取消。一次上传配额错误会先分页收集配置数量的最旧 `dsh-` 文件,再删除这些文件并重试一次上传。`DeepSeekFilesClient.delete`、`DeepSeekFileStore.release` 和 `releaseAll` 提供主动远端空间回收。本包记录的当前提供方限制为 Files 单次上传 128MiB、chat 单图引用 32MiB、每个 API key 最多 10,000 个文件和 25GiB;默认 1MiB 请求版本低于两个单文件上限。 `contextWindow` 对每个已配置模型都可选,不会通过建议 catalog 公开。`ctx.llm.resolveModelInfo('deepseek-official', model).context` 先返回精确模型值,再对不含容量的配置项或未列出原样传递 id 返回 `defaultContextWindow`。适配器默认值为 1,000,000;因此,压力敏感插件可以获得由部署决定的容量,不会将模型 selector 视为权威。为 `deepseek-official` 注册另一个适配器会抛出 `LlmError('DUPLICATE_ADAPTER')`。 `maxTokens` 是适配器为对话请求配置的输出上限,默认值为 256,000。Catalog 配置项可以自带 `maxTokens`,它对该模型胜出;不含该上限的配置项以及任何未列出原样传递 id 都解析为 profile 值,因此新增按模型的上限只改变一个模型,而非整条路由。确切模型解析会将胜出值公开为 `defaultMaxTokens`;`LlmRuntime` 会在 agent loop(智能体循环)写入 `request/header` 前,将该值填入 `GenerateOptions.maxTokens`,从而仍可根据持久记录重建协议请求。显式的请求值或 `AgentOptions.maxTokens` 值优先,并会序列化为 `max_tokens`。适配器不会根据 `contextWindow` 自动调低该请求预算;上下文或提供方输出上限较小的部署必须配置与其相容的 `maxTokens`。 -同一确切模型结果会在部署策略允许思考时,为每个原样传递模型在 `reasoning` 下公开有序的 `off`、`high` 和 `max` 推理(reasoning)强度。`reasoningEffort` 选择部署默认值,省略时回退为 `high`。`agent/request` 可以在每个会话步骤替换它;解析后的值会记录在 `request/header`。`high` 和 `max` 会启用思考,并序列化为官方顶层 `reasoning_effort`;适配器持有的 `off` 则序列化为 `thinking.type: disabled`,且省略 `reasoning_effort`。不支持的值会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。 +同一确切模型结果会在部署策略允许思考时,为每个原样传递模型在 `reasoning` 下公开有序的 `off`、`low`、`high` 和 `max` 推理(reasoning)强度。`reasoningEffort` 选择部署默认值,省略时回退为 `high`。`agent/request` 可以在每个会话步骤替换它;解析后的值会记录在 `request/header`。`low`、`high` 和 `max` 会启用思考,并以同名值序列化为官方顶层 `reasoning_effort`;适配器持有的 `off` 则序列化为 `thinking.type: disabled`,且省略 `reasoning_effort`。不支持的值会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。 -`thinking: disabled` 是部署锁定:它只公布 `off`,并以 `off` 为默认值。省略 `reasoningEffort` 或将其配置为 `off` 均有效;配置 `high` 或 `max` 会使插件加载失败,直接按请求启用思考也会在网络 I/O 前失败。携带 `GenerateOptions.purpose: 'session-title'` 的请求也会强制禁用思考并省略已解析的推理强度,将有界输出保留给可见标题文本,不改变会话或压缩(compaction)默认值。 +`thinking: disabled` 是部署锁定:它只公布 `off`,并以 `off` 为默认值。省略 `reasoningEffort` 或将其配置为 `off` 均有效;配置 `low`、`high` 或 `max` 会使插件加载失败,直接按请求启用思考也会在网络 I/O 前失败。携带 `GenerateOptions.purpose: 'session-title'` 的请求也会强制禁用思考并省略已解析的推理强度,将有界输出保留给可见标题文本,不改变会话或压缩(compaction)默认值。 -`streamIdleTimeoutMs` 会限制每次未完成提供方读取,包括初始 `fetch`,但不计入消费方在分片间花费的时间。DeepSeek SSE 注释会作为传输活动使尚未完成的读取重新布防,但绝不会成为 `StreamChunk` 值或会话日志事件。同一个稳定的 abort 信号会在整个调用期间传递给请求与 body reader;过期会停止传输并抛出 `LlmError('TIMEOUT')`,较早的调用方 abort 则抛出 `LlmError('ABORTED')`。适配器每次 `stream()` 调用恰好发起一次提供方请求;它把已配置策略注册为提供方元数据,再由 `dsh-llm-retry` 在持久化的 agent(智能体)步骤边界单独执行该策略。 +`streamIdleTimeoutMs` 会限制每次未完成提供方读取,包括初始 `fetch`,但不计入消费方在分片间花费的时间。DeepSeek SSE 注释和成功的文件解析会作为传输活动使尚未完成的读取重新计时,但绝不会成为 `StreamChunk` 值或会话日志事件。同一个稳定的 abort 信号会在整个调用期间传递给请求与 body reader;过期会停止传输并抛出 `LlmError('TIMEOUT')`,较早的调用方 abort 则抛出 `LlmError('ABORTED')`。适配器通常每次 `stream()` 调用发起一次 chat 请求,只有失效文件恢复会发起第二次。首次 chat 前的文件解析失败会发送一次内联请求。如果失效文件响应后的替换解析失败,该内联请求就是唯一允许的重试。适配器把已配置重试策略注册为提供方元数据,再由 `dsh-llm-retry` 在持久化的 agent(智能体)步骤边界单独执行该策略。 ## 动态配置(settings + credentials) -连接事实不在加载时冻结。`resolveAdapterOptions` 是从原始配置到已校验事实的唯一显式 resolve 步骤,适配器经由一个 thunk **每操作重读一次**:base URL、catalog、请求默认值与 idle 预算都在下一次请求生效,进行中的流则保持其起始事实。两个可选 seam 供给该 thunk: +连接事实不在加载时冻结。`resolveAdapterOptions` 是从原始配置到已校验事实的唯一显式 resolve 步骤,适配器经由一个 thunk **每操作重读一次**:base URL、catalog、请求默认值、图片和 Files 策略与 idle 预算都在下一次请求生效,进行中的流则保持其起始事实。三个可选 seam 供给该 thunk: - **`ctx.settings`**——插件用同一份 `Config` schema 注册 `llm-deepseek` namespace,并以其 `cordis.yml` 条目为组合 `base`,因此用户设置文档中的 `llm-deepseek:` 分节可以免重启覆盖任何字段。未挂载 settings 服务时,仅由 entry 配置驱动适配器,行为不变。存活 settings 快照若通过 schema 却违反 schema 之外的约束(重复的 catalog id、无法成立的 thinking/推理强度组合),则保留最后可用事实并记录失败;entry 配置本身仍会使插件加载失败。 - **`ctx.credentials`**——API 密钥按每次 stream 调用解析,取自与端点*同一*份解析后的快照。配置只携带 `apiKeyEnv`,从不携带字面密钥:该引用经凭据 seam 解析,未挂载 seam 时则经受信环境层解析。由于凭据事实与连接事实同行,被 resolver 拒绝的 settings 快照既不贡献自己的端点,也不贡献自己的密钥:整个先前世代继续服务。每个解析出的密钥在使用前都会被校验格式,因此 HTTP 标头无法承载的值会以 `LlmError('INVALID_CREDENTIAL')` 被拒绝,点名失败的入口,但绝不透露密钥的任何部分,而不是以语义不明的 `fetch` `TypeError` 形式浮现。任何地方都没有密钥的请求以 `MISSING_CREDENTIAL` 失败,并点名每个配置入口,同时路由保持注册、catalog 保持可浏览——首次运行的上手流程就是「浏览模型、存入密钥、再次发起提示」,中间无需任何重启。 +- **`ctx.attachments`**——图片请求会在请求时解析该服务,因此 Cordis 加载顺序不会冻结可选图片能力。服务缺失时,图片输入以 `UNSUPPORTED_CONTENT` 失败;纯文本调用不依赖该服务。 唯一在注册期捕获的事实是重试策略:其解析值变化时,插件原地重新注册该路由(同一适配器实例、一个同步区段),因此 `ctx.llm.providerRetryPolicy('deepseek-official')` 始终报告当前策略。 该插件还会在可配置提供方目录(`ctx.llm.listConfigurableProviders()`)中声明自己的路由:提供方为 `deepseek-official`,settings namespace 为 `llm-deepseek`,settings path 为空——整个分节就是 profile。配置界面借助该条目,把本适配器与休眠的 pi-ai 提供方一并呈现。 +## DeepSeek 请求扩展 + +存在 `ctx.deepseekLlmApiExtensions` 时,适配器会在序列化确切协议消息后、`fetch` 前准备其中已注册的顶层字段。提供方会收到同一个请求信号;即使某个提供方忽略信号,取消也会停止等待。准备失败与字段冲突会在 HTTP 前以 `REQUEST_EXTENSION` 失败。HTTP 2xx 后,适配器会先等待已准备的接受事务,再消费 SSE 正文;接受失败使用同一 code,而传输失败与非 2xx 失败不会接受字段。字段会发往解析后的 `baseURL`,包括已配置的网关。未组合该注册表的部署会发送未改变的 DeepSeek 基础请求。 + +随附 profile 与可运行示例会挂载 [`@deepseek-ai/dsh-session-log-deepseek`](../../session/session-log-deepseek/README.zh.md) 以提供增量 `dsh_session_log` 字段,并挂载 [`@deepseek-ai/dsh-plugin-package-inventory-deepseek`](../plugin-package-inventory-deepseek/README.zh.md) 以提供完整存活 `dsh_plugin_packages` 字段。会话日志上传默认关闭,需要设置 `session-log-deepseek.enabled: true`;插件包元数据默认开启。两个字段都对模型不可见。`llm-pi-ai` 既不导入也不调用该提供方特定注册表。 + ## 应用归因 -每个请求都携带 dsh-llm `attributionHeaders()` 的共享归因标头,即用于识别 harness 的必需 `User-Agent` 基线(见 [dsh-llm § 应用归因](../llm/README.md#app-attribution-attributionts))。在该适配器约定(adapter contract)下,直接 DeepSeek 请求与 OpenAI 兼容 gateway 请求都不会获得提供方特定应用归因标头;OpenRouter 应用归因暂缓到未来的显式 OpenRouter 适配器或模式。`GenerateOptions.purpose` 为 `compaction` 的请求(dsh-compaction-basic 的辅助摘要调用)还会携带 `x-deepseek-harness-compact: 1`,让宿主可以将压缩流量与会话请求分开。 +每个 chat 和 Files API 请求都携带 dsh-llm `attributionHeaders()` 的共享归因标头,即用于识别 harness 的必需 `User-Agent` 基线(见 [dsh-llm § 应用归因](../llm/README.zh.md#app-attribution-attributionts))。在该适配器约定(adapter contract)下,直接 DeepSeek 请求与 OpenAI 兼容 gateway 请求都不会获得提供方特定应用归因标头;OpenRouter 应用归因暂缓到未来的显式 OpenRouter 适配器或模式。`GenerateOptions.purpose` 为 `compaction` 的请求(dsh-compaction-basic 的辅助摘要调用)还会携带 `x-deepseek-harness-compact: 1`,让宿主可以将压缩流量与会话请求分开。 -DeepSeek 请求身份独立于应用归因。凭据解析成功后,每个提供方请求都会通过 `x-deepseek-harness-user-id` 携带来自 [`@deepseek-ai/dsh-anonymous-user-id`](../../identity/anonymous-user-id/README.md) 的稳定匿名 id;携带 `GenerateOptions.sessionId` 的请求还会通过 `x-deepseek-harness-session-id` 发送该确切值,缺少会话的直接调用则省略会话标头。两个标头都会发送至解析后的 `baseURL`(包括已配置的 gateway),且不会进入请求正文或模型可见内容。 +DeepSeek 请求身份独立于应用归因。凭据解析成功后,每个提供方请求都会通过 `x-deepseek-harness-user-id` 携带来自 [`@deepseek-ai/dsh-anonymous-user-id`](../../identity/anonymous-user-id/README.zh.md) 的稳定匿名 id;携带 `GenerateOptions.sessionId` 的请求还会通过 `x-deepseek-harness-session-id` 发送该确切值,缺少会话的直接调用则省略会话标头。两个标头都会发送至解析后的 `baseURL`(包括已配置的 gateway),且不会进入请求正文或模型可见内容。 ## 协议格式说明 - 只支持流式输出(`stream_options.include_usage` 始终开启)。`usage` 可能附着在 finish 分片上,也可能作为尾随的纯 usage 分片到达;转换器会将两者都延迟到 `[DONE]`,因此 `usage` 始终位于 `finish` 之前,`finish` 之后不会出现任何内容。 - 适配器持有的 `off` 推理强度映射为 `thinking: {type: 'disabled'}`,绝不会以 `reasoning_effort: 'off'` 通过协议发送。 - 第一个思考模式分片携带 `reasoning_content: ""`,系统会处理它(不会产生多余 reasoning 块)。 -- **推理回传规则**:对携带工具调用的 assistant 轮次,会将 `reasoning_content` 序列化回历史(思考模式 API 必需);对不含工具调用的轮次,它会被丢弃(不会使用,可节省 token)。 +- **推理回传规则**:每个携带推理内容的 assistant 轮次都会将 `reasoning_content` 序列化回历史。思考模式在工具调用轮次上必需它;DeepSeek 在其他轮次上会忽略它,而将该对话重新编码转发给其他厂商的网关,要靠对回传原文取哈希来恢复该轮次上游的思考签名。 +- 支持图片的 user 消息会保留文本/图片顺序。Tool role 内容仍为字符串;连续工具结果中的图片会用 `Attached image(s) from tool result:` 汇总到随后一条 user 消息。 - Cache 计量:`cacheReadTokens` ← `prompt_cache_hit_tokens` / `prompt_tokens_details.cached_tokens`;DeepSeek 不报告 cache-write 指标。 ## 错误 -非 2xx 响应会抛出稳定 code 的 `LlmError`:`AUTH`(401/403)、`QUOTA`(提供方详细信息标识配额、余额或点数耗尽的响应)、`RATE_LIMIT`(其他 429)、`CONTEXT_WINDOW_EXCEEDED`(提供方 code、type 或 message 标识上下文溢出的 400)、`INVALID_REQUEST`(其他 400)、`SERVER`(5xx),其他情况为 `HTTP_`。其可序列化 `failure` 保留 HTTP 状态,以及有效的正 `Retry-After` 秒数/日期延迟和存在时的 `x-request-id` / `x-deepseek-request-id`。响应前传输失败(DNS、连接被拒绝、TLS、proxy)会抛出命名已配置端点的 `TRANSPORT`,并将原始拒绝作为 `cause`;调用方 abort 抛出 `ABORTED`,仍以 loop 的取消信号为准。协议违例抛出 `STREAM_CLOSED`(没有 `[DONE]`)或 `MALFORMED_RESPONSE`(JSON payload 格式错误)。未知协议 `finish_reason`(例如 `content_filter`、`insufficient_system_resource`)会变为 `finish {kind: 'error', failure}` 分片;已完成流如果使用 `stop`(或缺失)finish 但没有开启内容块,就会变为 `finish {kind: 'error'}`,code 为 `EMPTY_RESPONSE`(默认策略会重试)。 +非 2xx 响应会抛出稳定 code 的 `LlmError`:`AUTH`(401/403)、`QUOTA`(提供方详细信息标识配额、余额或点数耗尽的响应)、`RATE_LIMIT`(其他 429)、`CONTEXT_WINDOW_EXCEEDED`(提供方 code、type 或 message 标识上下文溢出的 400)、`INVALID_REQUEST`(其他 400 和 413)、`SERVER`(5xx),其他情况为 `HTTP_`。其可序列化 `failure` 保留 HTTP 状态,以及有效的正 `Retry-After` 秒数/日期延迟和存在时的 `x-request-id` / `x-deepseek-request-id`。扩展准备、基础字段冲突或 2xx 后接受失败会使用 `REQUEST_EXTENSION`;扩展失败绝不会被重新标记为传输失败。如果 DeepSeek 拒绝一张已规范化图片,主错误会写明附件 ID 或显示名称、持久消息和图片位置、规范化后的媒体类型、8-bit sRGB/sRGBA 位深、尺寸和提供方消息。存在多张候选图片且提供方详细信息没有 file id 时,错误会列出全部可能图片,不会把错误归给第一张。原始响应保留为错误 `cause`,不会成为唯一的用户可见诊断。附件读取会保留稳定的附件失败 code,不会变成传输失败。响应前传输失败(DNS、连接被拒绝、TLS、proxy)会抛出命名已配置端点的 `TRANSPORT`,并将原始拒绝作为 `cause`;调用方 abort 抛出 `ABORTED`,仍以 loop 的取消信号为准。协议违例抛出 `STREAM_CLOSED`(没有 `[DONE]`)或 `MALFORMED_RESPONSE`(JSON payload 格式错误)。未知协议 `finish_reason`(例如 `content_filter`、`insufficient_system_resource`)会变为 `finish {kind: 'error', failure}` 分片;已完成流如果使用 `stop`(或缺失)finish 但没有开启内容块,就会变为 `finish {kind: 'error'}`,code 为 `EMPTY_RESPONSE`(默认策略会重试)。 ## 模型体验 @@ -82,15 +115,15 @@ DeepSeek 请求身份独立于应用归因。凭据解析成功后,每个提 #### 模型看到的内容 -所选 DeepSeek 模型会收到 harness 系统提示词、消息历史、工具 schema、stop sequence 和调用配置,不含适配器撰写的提示词文本。当之前的 assistant 轮次包含工具调用时,会按要求回传其推理内容;不含工具调用的轮次会省略推理。 +所选 DeepSeek 模型会收到 harness 系统提示词、消息历史、工具 schema、stop sequence 和调用配置,不含适配器撰写的提示词文本。提供方特定请求扩展字段仍位于该模型输入之外。视觉模型通常通过 Files API 引用收到保留的 user 与工具结果图片,旁边带有稳定附件句柄和请求图片尺寸;Files 解析失败时,所有保留图片改用内联 data URL。超出上限的较旧图片由已记录的占位文本表示。之前 assistant 轮次的推理内容会原文回传,无论该轮次是否调用了工具。 #### Token 影响 -精确输入取决于提供方 tokenization。有条件推理回传会增加工具往返上下文,丢弃其他推理则避免再次支付这些 token;可用时会报告 cache-read 用量。 +精确文本与图片 token 输入取决于提供方 tokenization。推理回传会把每个含推理轮次的思维链带入后续请求,丢弃超出上限的图片则避免再次支付这些 token;可用时会报告 cache-read 用量。 #### KV Cache 影响 -未更改的已组装前缀可使用 DeepSeek cache 复用,适配器会在 usage 中报告它。模型路由变更,或任何上游提示词、schema、前缀或历史变更,都可能使从首个发生变化的 token 起的复用失效;推理回传会在工具往返期间追加。 +未更改的已组装前缀,包括确定性编码的保留图片与占位文本,可使用 DeepSeek cache 复用,适配器会在 usage 中报告它。模型路由变更,或任何上游提示词、schema、前缀、历史或图片上限变更,都可能使从首个发生变化的 token 起的复用失效;推理回传会在每个含推理的轮次上追加。 ### DeepSeek 响应 @@ -111,4 +144,5 @@ loop 保留的响应块会追加到下一个请求,并保留其较早可复用 - **settings 的 `models` 列表会整体替换组合列表**:settings 层按字段合并,而数组是单个字段;按条目合并 catalog 需要带键的形状。 - **未映射 `tool_choice`**:它不属于核心词汇(MVP 取舍,与 pi-ai twin 共享)。 - **请求使用原始 `fetch`,而非 `@cordisjs/plugin-http`**:没有共享 proxy/拦截配置;采用暂缓到第二个适配器需要该功能时(`TODO(http)`)。 -- **序列化会将 user 与工具结果内容展平为文本块**:会跳过插件添加的块类型,空工具输出会以字面 `(no output)` 通过协议发送。 +- **会跳过插件添加的内容块类型**:核心文本与支持的图片块会被序列化,空工具输出会以字面 `(no output)` 通过协议发送。 +- **图片是仅输入的持久附件**:不支持直接外部 URL 和 assistant 图片输出;DeepSeek 图片输入通常使用 Files API,仅在单次请求恢复时使用内联 base64。 diff --git a/packages/llm/llm-deepseek/package.json b/packages/llm/llm-deepseek/package.json index 9c86b3c98e..eba8fbc5a7 100644 --- a/packages/llm/llm-deepseek/package.json +++ b/packages/llm/llm-deepseek/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-llm-deepseek", "description": "DeepSeek chat-completions adapter for the DeepSeek Harness LLM seam", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -32,10 +32,15 @@ ], "license": "MIT", "peerDependencies": { + "@deepseek-ai/dsh-attachment": "workspace:^", + "@deepseek-ai/dsh-atomic-write": "workspace:^", + "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-credentials": "workspace:^", + "@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^", "@deepseek-ai/dsh-launch-environment": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-home-paths": "workspace:^", "@deepseek-ai/dsh-settings": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-anonymous-user-id": "workspace:^", @@ -46,11 +51,20 @@ "@deepseek-ai/schemastery": "workspace:^" }, "devDependencies": { + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-attachment": "workspace:^", + "@deepseek-ai/dsh-atomic-write": "workspace:^", + "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-credentials": "workspace:^", + "@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^", + "@deepseek-ai/dsh-plugin-package-inventory-deepseek": "workspace:^", "@deepseek-ai/dsh-launch-environment": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-home-paths": "workspace:^", "@deepseek-ai/dsh-settings": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-session-log-deepseek": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-anonymous-user-id": "workspace:^", "@deepseek-ai/cordis": "workspace:^" diff --git a/packages/llm/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts index 5fa62d30ff..fe461fdb35 100644 --- a/packages/llm/llm-deepseek/src/adapter.ts +++ b/packages/llm/llm-deepseek/src/adapter.ts @@ -8,23 +8,41 @@ * @module dsh-llm-deepseek/adapter */ -import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE, ReasoningEffortId } from '@deepseek-ai/dsh-llm' +import { attributionHeaders, contentHasImage, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, offloadRequestImagesWithPolicy, ProviderRequestId, QUOTA_EXCEEDED_CODE, ReasoningEffortId } from '@deepseek-ai/dsh-llm' import type { + ContentBlock, GenerateOptions, LlmModelInfo, LlmProviderInfo, + PreparedAdapterCall, LlmResolvedModelInfo, + ModelModality, ResolvedRetryPolicy, StreamChunk, } from '@deepseek-ai/dsh-llm' +import type { + AttachmentId, + AttachmentStore, + ImageAttachmentRef, + ImageRequestPolicy, + RequestImageAttachment, +} from '@deepseek-ai/dsh-attachment' import type { CredentialRef } from '@deepseek-ai/dsh-credentials' -import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout' +import { deadline, idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout' import type { AnonymousUserId } from '@deepseek-ai/dsh-anonymous-user-id' -import { serializeRequest } from './serialize.ts' -import type { RequestDefaults } from './serialize.ts' +import type { + DeepSeekLlmApiExtensionRequest, + DeepSeekLlmApiJson, + PreparedDeepSeekLlmApiExtensions, +} from '@deepseek-ai/dsh-deepseek-llm-api-extensions' +import { serializeRequest, serializeRequestWithImages } from './serialize.ts' +import type { ImageWireLocation, RequestDefaults } from './serialize.ts' +import { DeepSeekFileStore } from './file-store.ts' +import type { DeepSeekFilePolicy } from './file-store.ts' +import type { DeepSeekFileId } from './file-id.ts' import { parseSse } from './sse.ts' import { translate } from './translate.ts' -import type { WireError } from './types.ts' +import type { WireError, WireRequest } from './types.ts' /** One optional model entry advertised by the direct-fetch adapter. */ export interface DeepSeekCatalogModel { @@ -38,6 +56,14 @@ export interface DeepSeekCatalogModel { contextWindow?: number /** Per-request output cap for this model; omission falls back to the profile's {@link DeepSeekConnectionOptions.maxTokens}. */ maxTokens?: number + /** Accepted request modalities; omission is text-only. */ + inputModalities?: ModelModality[] + /** Total-pixel budget for one deterministic request preview. */ + imagePixelBudget?: number + /** Encoded-byte cap for one deterministic request preview. */ + imageMaxBytes?: number + /** Provider detail tier; `low` uses the 512-by-512 total-pixel default. */ + imageDetail?: 'auto' | 'low' } /** @@ -66,6 +92,22 @@ export interface DeepSeekConnectionOptions { models: readonly DeepSeekCatalogModel[] /** Maximum provider idle time while one stream read is outstanding. */ streamIdleTimeoutMs: number + /** Maximum accumulated file-referenced image bytes in one request. */ + maxRequestFilesBytes: number + /** Maximum accumulated base64 image payload after Files API fallback. */ + maxInlineRequestImageBytes: number + /** Maximum number of represented images in one request. */ + maxImagesPerRequest: number + /** Raw-byte removal step after the file-reference bound is exceeded. */ + imageOffloadByteQuantum: number + /** Base64-byte removal step after the inline fallback bound is exceeded. */ + inlineImageOffloadByteQuantum: number + /** Image-count removal step after the count bound is exceeded. */ + imageOffloadCountQuantum: number + /** Maximum duration of one request-image Files API resolution. */ + filesApiTimeoutMs: number + /** Upload expiry, refresh, and quota-recovery policy. */ + filePolicy: DeepSeekFilePolicy /** Provider-owned model-request retry policy, already resolved. */ retryPolicy: ResolvedRetryPolicy } @@ -83,6 +125,12 @@ export interface DeepSeekAdapterOptions { resolveApiKey: (connection: DeepSeekConnectionOptions) => Promise /** Resolve the harness-home anonymous id shared with telemetry and feedback. */ resolveUserId: () => AnonymousUserId + /** Resolve the current durable attachment service; absence rejects image input. */ + resolveAttachments?: () => AttachmentStore | undefined + /** Resolve the process-wide upload reuse store. */ + resolveFiles?: () => DeepSeekFileStore + /** Prepare the official API's plugin-contributed top-level fields for one exact wire request. */ + prepareExtensions: (request: DeepSeekLlmApiExtensionRequest) => Promise } /** Default maximum idle interval while an adapter stream read is outstanding. */ @@ -91,12 +139,41 @@ export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000 export const DEFAULT_CONTEXT_WINDOW = 1_000_000 /** Default per-request output-token cap. */ export const DEFAULT_MAX_TOKENS = 256_000 +/** Default bound on accumulated file-referenced image bytes per request. */ +export const DEFAULT_MAX_REQUEST_FILES_BYTES = 128 * 1024 * 1024 +/** Default bound on accumulated base64 image payload after Files API fallback. */ +export const DEFAULT_MAX_INLINE_REQUEST_IMAGE_BYTES = 20 * 1024 * 1024 +/** Provider request image-count limit. */ +export const DEFAULT_MAX_IMAGES_PER_REQUEST = 600 +/** Total-pixel budget matching DeepSeek's normal vision projection. */ +export const DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET = 640_000 +/** Total-pixel budget matching provider low-detail image input. */ +export const DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET = 512 * 512 +/** Encoded-byte cap for one deterministic model-request image. */ +export const DEFAULT_REQUEST_IMAGE_MAX_BYTES = 1024 * 1024 +/** Deterministic raw-byte removal step. */ +export const DEFAULT_IMAGE_OFFLOAD_BYTE_QUANTUM = 64 * 1024 * 1024 +/** Deterministic base64-byte removal step after Files API fallback. */ +export const DEFAULT_INLINE_IMAGE_OFFLOAD_BYTE_QUANTUM = 10 * 1024 * 1024 +/** Deterministic image-count removal step. */ +export const DEFAULT_IMAGE_OFFLOAD_COUNT_QUANTUM = 20 +/** Default explicit lifetime for uploaded images. */ +export const DEFAULT_FILE_EXPIRY_SECONDS = 7 * 24 * 60 * 60 +/** Default proactive refresh window for indexed file ids. */ +export const DEFAULT_FILE_REFRESH_MARGIN_SECONDS = 60 * 60 +/** Default number of oldest harness-owned files removed on quota recovery. */ +export const DEFAULT_FILE_QUOTA_CLEANUP_BATCH = 100 +/** Default deadline for resolving one request image through the Files API. */ +export const DEFAULT_FILES_API_TIMEOUT_MS = 60_000 const STREAM_IDLE_TIMEOUT_CODE = 'LLM_STREAM_IDLE_TIMEOUT' +const FILES_API_TIMEOUT_CODE = 'DEEPSEEK_FILES_API_TIMEOUT' const OFF_REASONING_EFFORT = ReasoningEffortId('off') +const LOW_REASONING_EFFORT = ReasoningEffortId('low') const HIGH_REASONING_EFFORT = ReasoningEffortId('high') const MAX_REASONING_EFFORT = ReasoningEffortId('max') const REASONING_EFFORTS = [ { id: OFF_REASONING_EFFORT, name: 'Off' }, + { id: LOW_REASONING_EFFORT, name: 'Low' }, { id: HIGH_REASONING_EFFORT, name: 'High' }, { id: MAX_REASONING_EFFORT, name: 'Max' }, ] as const @@ -104,13 +181,138 @@ const OFF_ONLY_REASONING_EFFORTS = [ { id: OFF_REASONING_EFFORT, name: 'Off' }, ] as const +/** Marks a failed file-id resolution that may be retried as an inline request. */ +class FileResolutionFailure extends Error { + constructor(cause: unknown) { + super('DeepSeek Files API could not resolve a request image.', { cause }) + this.name = 'FileResolutionFailure' + } +} + +function collectImageRefs( + content: readonly ContentBlock[], + refs: Map, +): void { + for (const block of content) { + if (block.type === 'image') refs.set(block.attachment.attachmentId, block.attachment) + else if (block.type === 'tool-result') collectImageRefs(block.content, refs) + } +} + +/** + * Resolve the request-image budgets owned by one DeepSeek model route. + * @param model - Advertised model route and its optional image overrides. + * @returns Complete pixel and encoded-byte budgets. + * @internal + */ +export function resolveRequestImagePolicy(model: DeepSeekCatalogModel): ImageRequestPolicy { + let maxPixels: number + if (model.imagePixelBudget !== undefined) maxPixels = model.imagePixelBudget + else if (model.imageDetail === 'low') maxPixels = DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET + else maxPixels = DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET + return { + maxPixels, + maxBytes: model.imageMaxBytes === undefined + ? DEFAULT_REQUEST_IMAGE_MAX_BYTES + : model.imageMaxBytes, + } +} + +async function prepareRequestImages( + options: GenerateOptions, + attachments: AttachmentStore, + model: DeepSeekCatalogModel, + signal: AbortSignal, +): Promise> { + const refs = new Map() + for (const message of options.messages) collectImageRefs(message.content, refs) + const policy = resolveRequestImagePolicy(model) + const orderedRefs = [...refs.values()] + const projected = await Promise.all(orderedRefs.map( + ref => attachments.readImageRequest(ref, policy, signal), + )) + return new Map(orderedRefs.map((ref, index) => ( + [ref.attachmentId, projected[index] as RequestImageAttachment] + ))) +} + +function providerRejectedNormalizedImage(detail: string): boolean { + const reasonBeforeImage = /(?:unsupported|invalid|cannot read|failed to (?:decode|process)).{0,40}image/iu + const imageBeforeReason = /image.{0,40}(?:unsupported|invalid|cannot be decoded)/iu + return reasonBeforeImage.test(detail) || imageBeforeReason.test(detail) +} + +interface UsedRequestFile { + version: RequestImageAttachment + fileId: DeepSeekFileId + location: ImageWireLocation +} + +function providerRejectedFileId(detail: string): boolean { + const file = /\bfile(?:[_ -]?(?:id|api|not[_ -]?found|deleted|expired))?/iu.test(detail) + const missing = /(?:expired|not[_ -]?found|deleted|do(?:es)? not exist|not created under (?:this|your) account)/iu.test(detail) + const invalidId = /(?:invalid.{0,20}file[_ -]?(?:id|api)|file[_ -]?(?:id|api).{0,20}invalid)/iu.test(detail) + return file && (missing || invalidId) +} + +function detailNamesFileId(detail: string, fileId: DeepSeekFileId): boolean { + let index = detail.indexOf(fileId) + while (index >= 0) { + const before = detail[index - 1] + const after = detail[index + fileId.length] + if ((before === undefined || !/[\p{L}\p{N}_-]/u.test(before)) + && (after === undefined || !/[\p{L}\p{N}_-]/u.test(after))) return true + index = detail.indexOf(fileId, index + 1) + } + return false +} + +function staleMappings( + files: readonly UsedRequestFile[], + detail: string, +): UsedRequestFile[] { + const unique = [...new Map(files.map(file => [`${file.version.variantId}\0${file.fileId}`, file])).values()] + const exact = unique.filter(file => detailNamesFileId(detail, file.fileId)) + return exact.length > 0 ? exact : unique +} + +function normalizedImageFacts( + file: { version: RequestImageAttachment; location: ImageWireLocation }, +): string { + const version = file.version + const name = version.attachment.name ?? version.attachment.attachmentId + const colour = version.hasAlpha ? 'sRGBA' : 'sRGB' + return `"${name}" at message ${file.location.message}, image ${file.location.image} ` + + `(${version.mediaType}, 8-bit ${colour}, ${version.width}x${version.height})` +} + +function normalizedImageDiagnostic( + files: readonly UsedRequestFile[], + providerMessage: string, + providerDetail: string, +): string { + const exact = files.find(file => detailNamesFileId(providerDetail, file.fileId)) + const target = exact ?? (files.length === 1 ? files[0] : undefined) + if (target !== undefined) { + return `DeepSeek rejected normalized image ${normalizedImageFacts(target)}: ${providerMessage}. ` + + 'The provider rejected bytes already normalized by the harness; PNG, JPEG, WebP, and GIF remain supported input formats.' + } + const candidates = [...new Map(files.map(file => [ + `${file.version.variantId}\0${file.location.message}\0${file.location.image}`, + file, + ])).values()] + return `DeepSeek rejected a normalized request image: ${providerMessage}. Candidate images: ` + + `${candidates.map(normalizedImageFacts).join('; ')}. ` + + 'The provider rejected bytes already normalized by the harness; PNG, JPEG, WebP, and GIF remain supported input formats.' +} + function modelInfo(provider: string, model: DeepSeekCatalogModel): LlmModelInfo { return { provider, id: model.id, name: model.name ?? model.id, ...model.description === undefined ? {} : { description: model.description }, - inputModalities: ['text'], + inputModalities: model.inputModalities ?? ['text'], } } @@ -137,6 +339,7 @@ function requestId(headers: Headers): ReturnType | und */ export function httpErrorCode(status: number, error?: WireError['error']): string { if (status === 401 || status === 403) return 'AUTH' + if (status === 413) return 'INVALID_REQUEST' const detail = [error?.code, error?.type, error?.message].filter(Boolean).join(' ') if (isQuotaExceededError(detail)) return QUOTA_EXCEEDED_CODE if (status === 429) return 'RATE_LIMIT' @@ -156,8 +359,11 @@ export function httpErrorCode(status: number, error?: WireError['error']): strin * map to `ABORTED`; the configured per-read idle watchdog maps to `TIMEOUT`. */ export class DeepSeekAdapter extends LlmAdapter { + private readonly files: DeepSeekFileStore + constructor(private readonly config: DeepSeekAdapterOptions) { super() + this.files = config.resolveFiles?.() ?? new DeepSeekFileStore() } override providerInfo(provider: string): LlmProviderInfo { @@ -177,15 +383,21 @@ export class DeepSeekAdapter extends LlmAdapter { model: string, _signal?: AbortSignal, ): Promise { - const connection = this.config.options() + return Promise.resolve(this.modelInfoFor(this.config.options(), provider, model)) + } + + private modelInfoFor( + connection: DeepSeekConnectionOptions, + provider: string, + model: string, + ): LlmResolvedModelInfo { const configured = connection.models.find(entry => entry.id === model) const contextWindow = configured?.contextWindow ?? connection.defaultContextWindow - return Promise.resolve({ - // The chat-completions wire route is text-only regardless of catalog - // membership, so the uncatalogued fallback declares the same negative - // capability — "unknown" here would let the host accept and persist - // images the serializer must then reject. + return { + // An uncatalogued endpoint is safely treated as text-only. Declaring an + // unverified image capability would let the host persist input that the + // endpoint may reject on every later turn. ...configured === undefined ? { provider, id: model, name: model, inputModalities: ['text' as const] } : modelInfo(provider, configured), @@ -203,21 +415,55 @@ export class DeepSeekAdapter extends LlmAdapter { efforts: REASONING_EFFORTS, defaultEffort: connection.defaults.reasoningEffort === 'off' ? OFF_REASONING_EFFORT - : connection.defaults.reasoningEffort === 'max' - ? MAX_REASONING_EFFORT - : HIGH_REASONING_EFFORT, + : connection.defaults.reasoningEffort === 'low' + ? LOW_REASONING_EFFORT + : connection.defaults.reasoningEffort === 'max' + ? MAX_REASONING_EFFORT + : HIGH_REASONING_EFFORT, }, }, + } + } + + override prepareCall(provider: string, model: string, _signal?: AbortSignal): Promise { + const connection = this.config.options() + return Promise.resolve({ + model: this.modelInfoFor(connection, provider, model), + stream: options => this.streamWithConnection(options, connection), }) } - async * stream(options: GenerateOptions): AsyncIterable { + stream(options: GenerateOptions): AsyncIterable { + return this.streamWithConnection(options, this.config.options()) + } + + private async * streamWithConnection( + options: GenerateOptions, + connection: DeepSeekConnectionOptions, + ): AsyncIterable { // One resolution per stream call: connection facts and the credential // freeze here and hold for this whole request, so an in-flight stream // never observes a configuration change and the next call re-resolves. // The key resolves *from this snapshot*, so an endpoint and the secret // sent to it can never come from different configuration generations. - const connection = this.config.options() + const hasImages = options.messages.some(message => contentHasImage(message.content)) + let attachments: AttachmentStore | undefined + if (hasImages) { + const model = connection.models.find(entry => entry.id === options.model) + if (model?.inputModalities?.includes('image') !== true) { + throw new LlmError( + `DeepSeek model "${options.model}" does not accept image input.`, + 'UNSUPPORTED_CONTENT', + ) + } + attachments = this.config.resolveAttachments?.() + if (attachments === undefined) { + throw new LlmError( + 'DeepSeek image conversion requires the durable attachment service.', + 'UNSUPPORTED_CONTENT', + ) + } + } const apiKey = await this.config.resolveApiKey(connection) const userId = this.config.resolveUserId() const consumer = new AbortController() @@ -231,6 +477,7 @@ export class DeepSeekAdapter extends LlmAdapter { connection, apiKey, userId, + attachments, () => { watchdog.pulse() }, )[Symbol.asyncIterator]() let exhausted = false @@ -274,12 +521,9 @@ export class DeepSeekAdapter extends LlmAdapter { connection: DeepSeekConnectionOptions, apiKey: string, userId: AnonymousUserId, - onComment: () => void, + attachments: AttachmentStore | undefined, + onActivity: () => void, ): AsyncIterable { - const body = serializeRequest(options, connection.defaults) - // Prepared outside the try so the TRANSPORT label below covers exactly the - // transport boundary, never a serialization failure. - const payload = JSON.stringify(body) const headers = { 'authorization': `Bearer ${apiKey}`, 'content-type': 'application/json', @@ -294,53 +538,159 @@ export class DeepSeekAdapter extends LlmAdapter { : {}, } - // TODO(http): adopt the Cordis HTTP service when shared transport configuration - // outweighs its additional runtime dependencies. - let response: Response - try { - response = await fetch(`${connection.baseURL}/chat/completions`, { - method: 'POST', - headers, - body: payload, - signal, - }) - } catch (error: unknown) { - // The outer stream distinguishes caller cancellation and watchdog expiry. - if (signal.aborted) throw error - // fetch wraps every transport failure (DNS, refused connection, TLS, - // proxy) in a bare `TypeError: fetch failed` whose actionable detail - // lives on `cause`. Wrapping with the endpoint and chaining the cause - // lets `errorChain` render the full diagnosis at every reporting boundary. - throw new LlmError( - `DeepSeek API request to ${connection.baseURL} failed`, - 'TRANSPORT', - { cause: error }, - ) - } - - if (!response.ok) { - let message = `DeepSeek API error (HTTP ${response.status})` - let providerError: WireError['error'] - try { - const parsed = await response.json() as WireError - providerError = parsed.error - if (providerError?.message) message = providerError.message - } catch { - // Only swallow error-body parsing: the HTTP status still identifies the - // failure, so malformed gateway JSON must not mask it. + const fileConnection = { baseURL: connection.baseURL, apiKey } + const model = connection.models.find(entry => entry.id === options.model) + const policy = model === undefined ? undefined : resolveRequestImagePolicy(model) + const requestMessages = policy === undefined ? options.messages : offloadRequestImagesWithPolicy(options.messages, { + representation: 'raw', + maxBytes: connection.maxRequestFilesBytes, + maxImages: connection.maxImagesPerRequest, + byteQuantum: connection.imageOffloadByteQuantum, + countQuantum: connection.imageOffloadCountQuantum, + byteLength: ref => Math.min(ref.bytes, policy.maxBytes), + }) + const requestOptions = requestMessages === options.messages ? options : { ...options, messages: [...requestMessages] } + const requestImages = attachments === undefined || model === undefined + ? new Map() + : await prepareRequestImages(requestOptions, attachments, model, signal) + let representation: 'file' | 'base64' = 'file' + let fileAttempt = 0 + while (true) { + const usedFiles: UsedRequestFile[] = [] + let body: WireRequest + if (attachments === undefined) { + body = serializeRequest(requestOptions, connection.defaults) + } else if (representation === 'base64') { + body = await serializeRequestWithImages(requestOptions, { + representation: { kind: 'base64' }, + requestImages, + maxRequestImageBytes: connection.maxInlineRequestImageBytes, + maxImagesPerRequest: connection.maxImagesPerRequest, + byteQuantum: connection.inlineImageOffloadByteQuantum, + countQuantum: connection.imageOffloadCountQuantum, + }, connection.defaults) + } else { + try { + body = await serializeRequestWithImages(requestOptions, { + representation: { + kind: 'file', + resolveFileId: async (version, _block, location) => { + using filesDeadline = deadline(signal, connection.filesApiTimeoutMs, FILES_API_TIMEOUT_CODE) + let resolved: Awaited> + try { + resolved = await this.files.ensureUploaded( + version, + fileConnection, + connection.filePolicy, + filesDeadline.signal, + ) + } catch (error: unknown) { + if (signal.aborted) throw error + throw new FileResolutionFailure(error) + } + onActivity() + usedFiles.push({ version, fileId: resolved.record.fileId, location }) + return resolved.record.fileId + }, + }, + requestImages, + maxRequestImageBytes: connection.maxRequestFilesBytes, + maxImagesPerRequest: connection.maxImagesPerRequest, + byteQuantum: connection.imageOffloadByteQuantum, + countQuantum: connection.imageOffloadCountQuantum, + }, connection.defaults) + } catch (error: unknown) { + if (!(error instanceof FileResolutionFailure)) throw error + representation = 'base64' + continue + } } - const delay = providerRetryAfterMs(response.headers.get('retry-after')) - const id = requestId(response.headers) - throw new LlmError(message, httpErrorCode(response.status, providerError), { - status: response.status, - ...delay === undefined ? {} : { providerRetryAfterMs: delay }, - ...id === undefined ? {} : { requestId: id }, - }) - } - if (!response.body) { - throw new LlmError('DeepSeek API returned no response body', 'EMPTY_RESPONSE') - } + let extensions: PreparedDeepSeekLlmApiExtensions + try { + extensions = await this.config.prepareExtensions({ + body: body as unknown as Readonly>, + signal, + ...options.sessionId === undefined ? {} : { sessionId: String(options.sessionId) }, + ...options.purpose === undefined ? {} : { purpose: options.purpose }, + }) + } catch (error) { + throw new LlmError('DeepSeek request extension preparation failed', 'REQUEST_EXTENSION', { cause: error }) + } + for (const field of Object.keys(extensions.fields)) { + if (Object.hasOwn(body, field)) { + throw new LlmError(`DeepSeek request extension field ${JSON.stringify(field)} collides with the base request`, 'REQUEST_EXTENSION') + } + } + // Prepared outside the try so the TRANSPORT label below covers exactly the + // transport boundary, never a serialization failure. + const payload = JSON.stringify({ ...body, ...extensions.fields }) - yield* translate(parseSse(response.body, onComment)) + // TODO(http): adopt the Cordis HTTP service when shared transport configuration + // outweighs its additional runtime dependencies. + let response: Response + try { + response = await fetch(`${connection.baseURL}/chat/completions`, { + method: 'POST', + headers, + body: payload, + signal, + }) + } catch (error: unknown) { + if (signal.aborted) throw error + throw new LlmError( + `DeepSeek API request to ${connection.baseURL} failed`, + 'TRANSPORT', + { cause: error }, + ) + } + + if (!response.ok) { + let message = `DeepSeek API error (HTTP ${response.status})` + let providerError: WireError['error'] + const rawResponse = await response.text() + try { + const parsed = JSON.parse(rawResponse) as WireError + providerError = parsed.error + if (providerError?.message) message = providerError.message + } catch { + // The HTTP status remains authoritative when a gateway returns malformed JSON. + } + const detail = [providerError?.code, providerError?.type, providerError?.message] + .filter((field): field is string => typeof field === 'string') + .join(' ') + const staleFile = usedFiles.length > 0 && providerRejectedFileId(detail) + if (staleFile) { + await Promise.all(staleMappings(usedFiles, detail).map(file => ( + this.files.invalidate(file.version, file.fileId, fileConnection) + ))) + if (fileAttempt === 0) { + fileAttempt += 1 + continue + } + } + if (response.status === 400 && usedFiles.length > 0 && providerRejectedNormalizedImage(detail)) { + message = normalizedImageDiagnostic(usedFiles, message, detail) + } + const delay = providerRetryAfterMs(response.headers.get('retry-after')) + const id = requestId(response.headers) + throw new LlmError(message, httpErrorCode(response.status, providerError), { + cause: new Error(rawResponse.length > 0 ? rawResponse : `DeepSeek HTTP ${response.status}`), + status: response.status, + ...delay === undefined ? {} : { providerRetryAfterMs: delay }, + ...id === undefined ? {} : { requestId: id }, + }) + } + try { + await extensions.accept() + } catch (error) { + throw new LlmError('DeepSeek request extension acceptance failed', 'REQUEST_EXTENSION', { cause: error }) + } + if (!response.body) { + throw new LlmError('DeepSeek API returned no response body', 'EMPTY_RESPONSE') + } + + yield* translate(parseSse(response.body, onActivity)) + return + } } } diff --git a/packages/llm/llm-deepseek/src/file-id.ts b/packages/llm/llm-deepseek/src/file-id.ts new file mode 100644 index 0000000000..fd77de372f --- /dev/null +++ b/packages/llm/llm-deepseek/src/file-id.ts @@ -0,0 +1,27 @@ +/** DeepSeek Files API identifiers. @module dsh-llm-deepseek/file-id */ + +import type { Branded } from '@deepseek-ai/dsh-brand' + +/** Opaque identifier returned by the DeepSeek Files API. */ +export type DeepSeekFileId = Branded<'DeepSeekFileId'> + +/** + * Brand a provider-returned file identifier after wire validation. + * @param id - non-empty Files API identifier. + * @returns the same string with its provider identity attached at type level. + */ +export function DeepSeekFileId(id: string): DeepSeekFileId { + return id as DeepSeekFileId +} + +/** Non-secret digest identifying one endpoint and API-key file namespace. */ +export type DeepSeekFileScope = Branded<'DeepSeekFileScope'> + +/** + * Brand a locally derived namespace digest. + * @param scope - SHA-256 digest of endpoint and API key. + * @returns the same string with namespace identity attached at type level. + */ +export function DeepSeekFileScope(scope: string): DeepSeekFileScope { + return scope as DeepSeekFileScope +} diff --git a/packages/llm/llm-deepseek/src/file-store.ts b/packages/llm/llm-deepseek/src/file-store.ts new file mode 100644 index 0000000000..fde86fa44e --- /dev/null +++ b/packages/llm/llm-deepseek/src/file-store.ts @@ -0,0 +1,331 @@ +/** DeepSeek Files API upload reuse, invalidation, and quota recovery. @module dsh-llm-deepseek/file-store */ + +import type { RequestImageAttachment } from '@deepseek-ai/dsh-attachment' +import { LlmError } from '@deepseek-ai/dsh-llm' +import { DeepSeekFilesClient, isFilesQuotaError } from './files-api.ts' +import type { DeepSeekFileId } from './file-id.ts' +import { deepSeekFileScope, DeepSeekUploadIndex } from './upload-index.ts' +import type { DeepSeekUploadRecord } from './upload-index.ts' + +/** DeepSeek chat accepts at most 32 MiB per image even when it is referenced by file id. */ +export const MAX_CHAT_IMAGE_BYTES = 32 * 1024 * 1024 +const OWNED_FILE_PREFIX = 'dsh-' + +/** Resolved file-store policy from the plugin configuration. */ +export interface DeepSeekFilePolicy { + expiresAfterSeconds: number + refreshMarginSeconds: number + quotaCleanupBatch: number +} + +/** Connection facts needed by file operations. */ +export interface DeepSeekFileConnection { + baseURL: string + apiKey: string +} + +/** Result of one file-id resolution. */ +export interface DeepSeekFileReference { + record: DeepSeekUploadRecord + uploaded: boolean +} + +interface FileStoreOptions { + index?: DeepSeekUploadIndex + now?: () => number + fetch?: typeof fetch +} + +interface SharedUpload { + controller: AbortController + promise: Promise + settled: boolean + waiters: number +} + +function abortReason(signal: AbortSignal): Error { + const reason: unknown = signal.reason + return reason instanceof Error + ? reason + : new Error('DeepSeek file upload cancelled with a non-Error reason.', { cause: reason }) +} + +function uploadFailure(error: unknown): Error { + return error instanceof Error + ? error + : new Error('DeepSeek file upload failed with a non-Error reason.', { cause: error }) +} + +function waitForUpload(operation: SharedUpload, signal: AbortSignal | undefined): Promise { + signal?.throwIfAborted() + operation.waiters += 1 + let released = false + const release = (cancelledReason?: Error): void => { + if (released) return + released = true + operation.waiters -= 1 + if (cancelledReason !== undefined && operation.waiters === 0 && !operation.settled) { + operation.controller.abort(cancelledReason) + } + } + if (signal === undefined) { + return operation.promise.finally(() => { + release() + }) + } + return new Promise((resolve, reject) => { + const abort = (): void => { + const reason = abortReason(signal) + release(reason) + reject(reason) + } + signal.addEventListener('abort', abort, { once: true }) + void operation.promise.then((value) => { + signal.removeEventListener('abort', abort) + release() + resolve(value) + }, (error: unknown) => { + signal.removeEventListener('abort', abort) + release() + reject(uploadFailure(error)) + }) + }) +} + +function extension(mediaType: RequestImageAttachment['mediaType']): 'png' | 'jpeg' | 'webp' | 'gif' { + switch (mediaType) { + case 'image/png': return 'png' + case 'image/jpeg': return 'jpeg' + case 'image/webp': return 'webp' + case 'image/gif': return 'gif' + } +} + +function filename(version: RequestImageAttachment): string { + const attachment = String(version.attachment.attachmentId).slice('sha256:'.length, 'sha256:'.length + 16) + const variant = String(version.variantId).slice('sha256:'.length, 'sha256:'.length + 8) + return `${OWNED_FILE_PREFIX}${attachment}-${variant}.${extension(version.mediaType)}` +} + +/** User-scoped durable file-id reuse for the DeepSeek route. */ +export class DeepSeekFileStore { + private readonly index: DeepSeekUploadIndex + private readonly now: () => number + private readonly fetchImpl: typeof fetch | undefined + private readonly inflight = new Map() + + /** + * @param options - testable index, clock, and transport boundaries. + */ + constructor(options: FileStoreOptions = {}) { + this.index = options.index ?? new DeepSeekUploadIndex() + this.now = options.now ?? Date.now + this.fetchImpl = options.fetch + } + + private client(connection: DeepSeekFileConnection): DeepSeekFilesClient { + return new DeepSeekFilesClient({ + baseURL: connection.baseURL, + apiKey: connection.apiKey, + ...this.fetchImpl === undefined ? {} : { fetch: this.fetchImpl }, + }) + } + + /** + * Resolve or upload one deterministic request image. Concurrent calls share one upload while retaining independent waits. + * @param version - deterministic model-request bytes and complete transformation identity. + * @param connection - endpoint and API-key snapshot. + * @param policy - expiry and quota-recovery policy. + * @param signal - cancellation of this wait; shared transport stops when no waiter remains. + * @returns a reusable file id and whether this call published a new upload. + */ + ensureUploaded( + version: RequestImageAttachment, + connection: DeepSeekFileConnection, + policy: DeepSeekFilePolicy, + signal?: AbortSignal, + ): Promise { + signal?.throwIfAborted() + const scope = deepSeekFileScope(connection.baseURL, connection.apiKey) + const key = `${scope}\0${version.variantId}` + let active = this.inflight.get(key) + if (active?.controller.signal.aborted) { + this.inflight.delete(key) + active = undefined + } + if (active !== undefined) return waitForUpload(active, signal) + const controller = new AbortController() + const shared: SharedUpload = { + controller, + settled: false, + waiters: 0, + promise: undefined as never, + } + shared.promise = this.ensureUploadedOnce(version, connection, policy, controller.signal).then((value) => { + shared.settled = true + return value + }, (error: unknown) => { + shared.settled = true + throw uploadFailure(error) + }) + this.inflight.set(key, shared) + void shared.promise.finally(() => { + if (this.inflight.get(key) === shared) this.inflight.delete(key) + }).catch(() => {}) + return waitForUpload(shared, signal) + } + + private async ensureUploadedOnce( + version: RequestImageAttachment, + connection: DeepSeekFileConnection, + policy: DeepSeekFilePolicy, + signal: AbortSignal, + ): Promise { + if (version.bytes > MAX_CHAT_IMAGE_BYTES) { + throw new LlmError('DeepSeek chat image exceeds the 32 MiB per-image limit.', 'INVALID_REQUEST') + } + const scope = deepSeekFileScope(connection.baseURL, connection.apiKey) + const now = this.now() + const marginMs = policy.refreshMarginSeconds * 1_000 + const cached = await this.index.get(scope, version.variantId, now, marginMs) + if (cached !== undefined) return { record: cached, uploaded: false } + + const client = this.client(connection) + const upload = async (): Promise => { + const remote = await client.upload({ + data: version.data, + mediaType: version.mediaType, + filename: filename(version), + expiresAfterSeconds: policy.expiresAfterSeconds, + signal, + }) + if (remote.bytes !== version.data.byteLength) { + throw new LlmError('DeepSeek Files API upload response does not match the submitted image.', 'INVALID_RESPONSE') + } + return { + scope, + attachmentId: version.attachment.attachmentId, + variantId: version.variantId, + fileId: remote.id, + bytes: remote.bytes, + createdAt: remote.createdAt * 1_000, + expiresAt: remote.expiresAt * 1_000, + } + } + + let candidate: DeepSeekUploadRecord + try { + candidate = await upload() + } catch (error: unknown) { + if (!isFilesQuotaError(error)) throw error + const deleted = await this.reclaimOldestOwned(connection, policy.quotaCleanupBatch, signal) + if (deleted === 0) throw error + candidate = await upload() + } + const committed = await this.index.commit(candidate, this.now(), marginMs) + if (!committed.accepted) { + try { + await client.delete(candidate.fileId, signal) + } catch { + // The winning mapping is durable. A failed duplicate cleanup affects quota only and is retried by recovery. + } + } + return { record: committed.record, uploaded: committed.accepted } + } + + /** + * Invalidate one exact local mapping after the chat endpoint rejects its remote id. + * @param version - request-image version whose remote generation failed. + * @param fileId - exact rejected file id. + * @param connection - endpoint and API-key snapshot. + */ + async invalidate( + version: RequestImageAttachment, + fileId: DeepSeekFileId, + connection: DeepSeekFileConnection, + ): Promise { + await this.index.remove( + deepSeekFileScope(connection.baseURL, connection.apiKey), + version.variantId, + fileId, + ) + } + + /** + * Delete the indexed remote file for one attachment and remove its local mapping. + * @param version - exact request-image version to release. + * @param connection - endpoint and API-key snapshot. + * @param policy - expiry policy used to locate a reusable mapping. + * @param signal - request cancellation. + * @returns whether an indexed file existed and was deleted. + */ + async release( + version: RequestImageAttachment, + connection: DeepSeekFileConnection, + policy: DeepSeekFilePolicy, + signal?: AbortSignal, + ): Promise { + const scope = deepSeekFileScope(connection.baseURL, connection.apiKey) + const record = await this.index.get( + scope, + version.variantId, + this.now(), + policy.refreshMarginSeconds * 1_000, + ) + if (record === undefined) return false + await this.client(connection).delete(record.fileId, signal) + await this.index.remove(scope, version.variantId, record.fileId) + return true + } + + /** + * Delete the oldest provider files whose names identify harness ownership. + * @param connection - endpoint and API-key snapshot. + * @param count - positive maximum number of files to delete. + * @param signal - request cancellation. + * @returns number of successfully deleted files. + */ + async reclaimOldestOwned( + connection: DeepSeekFileConnection, + count: number, + signal?: AbortSignal, + ): Promise { + const client = this.client(connection) + let after: DeepSeekFileId | undefined + const owned: DeepSeekFileId[] = [] + while (owned.length < count) { + const page = await client.list({ + ...after === undefined ? {} : { after }, + limit: 1_000, + order: 'asc', + ...signal === undefined ? {} : { signal }, + }) + for (const file of page.data) { + if (!file.filename.startsWith(OWNED_FILE_PREFIX)) continue + owned.push(file.id) + if (owned.length === count) break + } + if (!page.hasMore || page.lastId === undefined || page.lastId === after) break + after = page.lastId + } + for (const fileId of owned) await client.delete(fileId, signal) + return owned.length + } + + /** + * Delete every remote harness-owned file in the active API-key namespace and clear its index. + * @param connection - endpoint and API-key snapshot. + * @param signal - request cancellation. + * @returns number of deleted files. + */ + async releaseAll(connection: DeepSeekFileConnection, signal?: AbortSignal): Promise { + let total = 0 + for (;;) { + const deleted = await this.reclaimOldestOwned(connection, 1_000, signal) + total += deleted + if (deleted < 1_000) break + } + await this.index.clear(deepSeekFileScope(connection.baseURL, connection.apiKey)) + return total + } +} diff --git a/packages/llm/llm-deepseek/src/files-api.ts b/packages/llm/llm-deepseek/src/files-api.ts new file mode 100644 index 0000000000..cc998b2e7e --- /dev/null +++ b/packages/llm/llm-deepseek/src/files-api.ts @@ -0,0 +1,257 @@ +/** OpenAI-compatible DeepSeek Files API transport. @module dsh-llm-deepseek/files-api */ + +import { attributionHeaders, LlmError } from '@deepseek-ai/dsh-llm' +import type { ImageMediaType } from '@deepseek-ai/dsh-attachment' +import { DeepSeekFileId } from './file-id.ts' +import type { DeepSeekFileId as DeepSeekFileIdType } from './file-id.ts' + +/** Minimum provider-supported file lifetime. */ +export const MIN_FILE_EXPIRY_SECONDS = 3_600 +/** Maximum provider-supported file lifetime. */ +export const MAX_FILE_EXPIRY_SECONDS = 2_592_000 +/** Maximum Files API upload size. */ +export const MAX_FILE_UPLOAD_BYTES = 128 * 1024 * 1024 +/** Current per-key file-count quota. */ +export const MAX_STORED_FILE_COUNT = 10_000 +/** Current per-key storage quota. */ +export const MAX_STORED_FILE_BYTES = 25 * 1024 * 1024 * 1024 + +/** Validated file object returned by the OpenAI-compatible endpoint. */ +export interface DeepSeekFileObject { + id: DeepSeekFileIdType + bytes: number + createdAt: number + filename: string + purpose: 'user_data' + expiresAt?: number +} + +/** One page returned by `GET /files`. */ +export interface DeepSeekFilePage { + data: DeepSeekFileObject[] + firstId?: DeepSeekFileIdType + lastId?: DeepSeekFileIdType + hasMore: boolean +} + +/** Files API operation failure with its HTTP status retained for recovery policy. */ +export class DeepSeekFilesError extends LlmError { + /** Parsed provider detail used only for error classification. */ + readonly detail: string + + /** + * @param message - user-readable provider failure. + * @param status - HTTP status returned by the Files API. + * @param detail - provider error fields joined for classification. + */ + constructor(message: string, status: number, detail: string) { + super(message, status === 401 || status === 403 + ? 'AUTH' + : status === 429 + ? 'RATE_LIMIT' + : status >= 500 + ? 'SERVER' + : 'FILES_API', { status }) + this.name = 'DeepSeekFilesError' + this.detail = detail + } +} + +/** + * Whether an upload failure reports a provider storage or file-count quota. + * @param error - Files API operation failure. + * @returns whether one bounded remote cleanup and upload retry may recover. + */ +export function isFilesQuotaError(error: unknown): error is DeepSeekFilesError { + return error instanceof DeepSeekFilesError + && /(?:quota|storage|stored files|file count|too many files)/iu.test(error.detail) +} + +interface FilesApiOptions { + baseURL: string + apiKey: string + fetch?: typeof fetch +} + +interface WireFileObject { + id?: unknown + object?: unknown + bytes?: unknown + created_at?: unknown + filename?: unknown + purpose?: unknown + expires_at?: unknown +} + +function invalidResponse(operation: string): LlmError { + return new LlmError(`DeepSeek Files API returned an invalid ${operation} response.`, 'INVALID_RESPONSE') +} + +function parseFileObject(value: unknown, operation: string): DeepSeekFileObject { + if (value === null || typeof value !== 'object' || Array.isArray(value)) throw invalidResponse(operation) + const wire = value as WireFileObject + if (typeof wire.id !== 'string' || wire.id.length === 0 + || wire.object !== 'file' + || !Number.isSafeInteger(wire.bytes) || (wire.bytes as number) < 0 + || !Number.isSafeInteger(wire.created_at) || (wire.created_at as number) < 0 + || typeof wire.filename !== 'string' || wire.filename.length === 0 + || wire.purpose !== 'user_data' + || (wire.expires_at !== undefined + && (!Number.isSafeInteger(wire.expires_at) || (wire.expires_at as number) < 0))) { + throw invalidResponse(operation) + } + return { + id: DeepSeekFileId(wire.id), + bytes: wire.bytes as number, + createdAt: wire.created_at as number, + filename: wire.filename, + purpose: 'user_data', + ...wire.expires_at === undefined ? {} : { expiresAt: wire.expires_at as number }, + } +} + +function providerErrorDetail(value: unknown): { message?: string; detail: string } { + if (value === null || typeof value !== 'object' || Array.isArray(value)) return { detail: '' } + const error = (value as { error?: unknown }).error + if (error === null || typeof error !== 'object' || Array.isArray(error)) return { detail: '' } + const fields = error as { message?: unknown; type?: unknown; code?: unknown } + const message = typeof fields.message === 'string' ? fields.message : undefined + return { + ...message === undefined ? {} : { message }, + detail: [fields.code, fields.type, fields.message] + .filter((field): field is string => typeof field === 'string') + .join(' '), + } +} + +/** Direct client for the OpenAI-compatible `/files` endpoints. */ +export class DeepSeekFilesClient { + private readonly baseURL: string + private readonly apiKey: string + private readonly fetchImpl: typeof fetch + + /** + * @param options - endpoint, API-key snapshot, and optional test transport. + */ + constructor(options: FilesApiOptions) { + this.baseURL = options.baseURL.replace(/\/+$/u, '') + this.apiKey = options.apiKey + this.fetchImpl = options.fetch ?? globalThis.fetch + } + + private async request(path: string, init: RequestInit, signal?: AbortSignal): Promise { + let response: Response + try { + const headers = new Headers(attributionHeaders()) + headers.set('authorization', `Bearer ${this.apiKey}`) + response = await this.fetchImpl(`${this.baseURL}${path}`, { + ...init, + headers, + ...signal === undefined ? {} : { signal }, + }) + } catch (error: unknown) { + if (signal?.aborted) throw error + throw new LlmError(`DeepSeek Files API request to ${this.baseURL} failed`, 'TRANSPORT', { cause: error }) + } + if (response.ok) return response + let parsed: unknown + try { + parsed = await response.json() + } catch { + // A status remains sufficient to report the provider failure. + } + const { message, detail } = providerErrorDetail(parsed) + throw new DeepSeekFilesError( + message ?? `DeepSeek Files API error (HTTP ${response.status})`, + response.status, + detail, + ) + } + + /** + * Upload one image with an explicit expiry. + * @param input - deterministic request-version bytes, media type, filename, lifetime, and cancellation. + * @returns the validated provider file object, including `expires_at`. + */ + async upload(input: { + data: Uint8Array + mediaType: ImageMediaType + filename: string + expiresAfterSeconds: number + signal?: AbortSignal + }): Promise { + if (input.data.byteLength > MAX_FILE_UPLOAD_BYTES) { + throw new LlmError('DeepSeek Files API upload exceeds 128 MiB.', 'INVALID_REQUEST') + } + if (!Number.isSafeInteger(input.expiresAfterSeconds) + || input.expiresAfterSeconds < MIN_FILE_EXPIRY_SECONDS + || input.expiresAfterSeconds > MAX_FILE_EXPIRY_SECONDS) { + throw new LlmError('DeepSeek file expiry must be between 3600 and 2592000 seconds.', 'INVALID_REQUEST') + } + const form = new FormData() + form.set('purpose', 'user_data') + form.set('expires_after[anchor]', 'created_at') + form.set('expires_after[seconds]', String(input.expiresAfterSeconds)) + form.set('file', new Blob([Uint8Array.from(input.data).buffer], { type: input.mediaType }), input.filename) + const response = await this.request('/files', { method: 'POST', body: form }, input.signal) + const file = parseFileObject(await response.json(), 'upload') + if (file.expiresAt === undefined) throw invalidResponse('upload') + return { ...file, expiresAt: file.expiresAt } + } + + /** + * List one ascending or descending page of user-data files. + * @param options - pagination, ordering, and cancellation. + * @returns the validated page. + */ + async list(options: { + after?: DeepSeekFileIdType + limit?: number + order?: 'asc' | 'desc' + signal?: AbortSignal + } = {}): Promise { + const query = new URLSearchParams({ purpose: 'user_data' }) + if (options.after !== undefined) query.set('after', options.after) + if (options.limit !== undefined) query.set('limit', String(options.limit)) + if (options.order !== undefined) query.set('order', options.order) + const response = await this.request(`/files?${query.toString()}`, { method: 'GET' }, options.signal) + const value = await response.json() as unknown + if (value === null || typeof value !== 'object' || Array.isArray(value)) throw invalidResponse('list') + const wire = value as { object?: unknown; data?: unknown; first_id?: unknown; last_id?: unknown; has_more?: unknown } + if (wire.object !== 'list' || !Array.isArray(wire.data) || typeof wire.has_more !== 'boolean' + || (wire.first_id !== undefined && typeof wire.first_id !== 'string') + || (wire.last_id !== undefined && typeof wire.last_id !== 'string')) { + throw invalidResponse('list') + } + return { + data: wire.data.map(item => parseFileObject(item, 'list')), + ...typeof wire.first_id === 'string' ? { firstId: DeepSeekFileId(wire.first_id) } : {}, + ...typeof wire.last_id === 'string' ? { lastId: DeepSeekFileId(wire.last_id) } : {}, + hasMore: wire.has_more, + } + } + + /** + * Retrieve one file object. + * @param fileId - provider file identifier. + * @param signal - request cancellation. + * @returns the validated file object. + */ + async retrieve(fileId: DeepSeekFileIdType, signal?: AbortSignal): Promise { + const response = await this.request(`/files/${encodeURIComponent(fileId)}`, { method: 'GET' }, signal) + return parseFileObject(await response.json(), 'retrieve') + } + + /** + * Delete one provider file. + * @param fileId - provider file identifier. + * @param signal - request cancellation. + */ + async delete(fileId: DeepSeekFileIdType, signal?: AbortSignal): Promise { + const response = await this.request(`/files/${encodeURIComponent(fileId)}`, { method: 'DELETE' }, signal) + const value = await response.json() as unknown + if (value === null || typeof value !== 'object' || Array.isArray(value)) throw invalidResponse('delete') + const wire = value as { id?: unknown; object?: unknown; deleted?: unknown } + if (wire.id !== fileId || wire.object !== 'file' || wire.deleted !== true) throw invalidResponse('delete') + } +} diff --git a/packages/llm/llm-deepseek/src/index.ts b/packages/llm/llm-deepseek/src/index.ts index 8d01d9d6bc..e6faad74ce 100644 --- a/packages/llm/llm-deepseek/src/index.ts +++ b/packages/llm/llm-deepseek/src/index.ts @@ -14,7 +14,7 @@ import type { Context } from '@deepseek-ai/cordis' import z from '@deepseek-ai/schemastery' import { assertUsableApiKey, LlmError, resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm' -import type { RetryPolicyConfig } from '@deepseek-ai/dsh-llm' +import type { ModelModality, RetryPolicyConfig } from '@deepseek-ai/dsh-llm' import { credentialRef } from '@deepseek-ai/dsh-credentials' import { launchEnvironmentOf, type LaunchEnvironmentSnapshot } from '@deepseek-ai/dsh-launch-environment' import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings' @@ -22,7 +22,20 @@ import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { getOrCreateAnonymousUserId, type AnonymousUserId } from '@deepseek-ai/dsh-anonymous-user-id' import { DEFAULT_CONTEXT_WINDOW, + DEFAULT_FILE_EXPIRY_SECONDS, + DEFAULT_FILE_QUOTA_CLEANUP_BATCH, + DEFAULT_FILE_REFRESH_MARGIN_SECONDS, + DEFAULT_FILES_API_TIMEOUT_MS, + DEFAULT_IMAGE_OFFLOAD_BYTE_QUANTUM, + DEFAULT_IMAGE_OFFLOAD_COUNT_QUANTUM, + DEFAULT_INLINE_IMAGE_OFFLOAD_BYTE_QUANTUM, + DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET, + DEFAULT_MAX_INLINE_REQUEST_IMAGE_BYTES, + DEFAULT_MAX_IMAGES_PER_REQUEST, + DEFAULT_MAX_REQUEST_FILES_BYTES, DEFAULT_MAX_TOKENS, + DEFAULT_REQUEST_IMAGE_MAX_BYTES, + DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET, DEFAULT_STREAM_IDLE_TIMEOUT_MS, DeepSeekAdapter, } from './adapter.ts' @@ -30,11 +43,32 @@ import type { DeepSeekCatalogModel, DeepSeekConnectionOptions } from './adapter. export { DEFAULT_CONTEXT_WINDOW, + DEFAULT_FILE_EXPIRY_SECONDS, + DEFAULT_FILE_QUOTA_CLEANUP_BATCH, + DEFAULT_FILE_REFRESH_MARGIN_SECONDS, + DEFAULT_FILES_API_TIMEOUT_MS, + DEFAULT_IMAGE_OFFLOAD_BYTE_QUANTUM, + DEFAULT_IMAGE_OFFLOAD_COUNT_QUANTUM, + DEFAULT_INLINE_IMAGE_OFFLOAD_BYTE_QUANTUM, + DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET, + DEFAULT_MAX_INLINE_REQUEST_IMAGE_BYTES, + DEFAULT_MAX_IMAGES_PER_REQUEST, + DEFAULT_MAX_REQUEST_FILES_BYTES, DEFAULT_MAX_TOKENS, + DEFAULT_REQUEST_IMAGE_MAX_BYTES, + DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET, DEFAULT_STREAM_IDLE_TIMEOUT_MS, DeepSeekAdapter, } from './adapter.ts' export type { DeepSeekAdapterOptions, DeepSeekCatalogModel, DeepSeekConnectionOptions } from './adapter.ts' +export { DeepSeekFileStore, MAX_CHAT_IMAGE_BYTES } from './file-store.ts' +export type { DeepSeekFileConnection, DeepSeekFilePolicy, DeepSeekFileReference } from './file-store.ts' +export { DeepSeekFilesClient, MAX_FILE_EXPIRY_SECONDS, MAX_FILE_UPLOAD_BYTES, MAX_STORED_FILE_BYTES, MAX_STORED_FILE_COUNT, MIN_FILE_EXPIRY_SECONDS } from './files-api.ts' +export type { DeepSeekFileObject, DeepSeekFilePage } from './files-api.ts' +export { DeepSeekFileId } from './file-id.ts' +export type { DeepSeekFileId as DeepSeekFileIdType } from './file-id.ts' +export { DeepSeekUploadIndex, deepSeekFileScope } from './upload-index.ts' +export type { DeepSeekUploadRecord } from './upload-index.ts' export type { RequestDefaults } from './serialize.ts' export type * from './types.ts' @@ -49,8 +83,18 @@ const PROVIDER = 'deepseek-official' const DEFAULT_MODELS: DeepSeekCatalogModel[] = [ { id: 'deepseek-v4-flash', name: 'DeepSeek-V4-Flash', contextWindow: DEFAULT_CONTEXT_WINDOW }, { id: 'deepseek-v4-pro', name: 'DeepSeek-V4-Pro', contextWindow: DEFAULT_CONTEXT_WINDOW }, + { + id: 'deepseek-v4-flash-vision-exp', + name: 'DeepSeek-V4-Flash-Vision-Exp', + contextWindow: DEFAULT_CONTEXT_WINDOW, + inputModalities: ['text', 'image'], + imagePixelBudget: DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET, + imageMaxBytes: DEFAULT_REQUEST_IMAGE_MAX_BYTES, + }, ] +const MODEL_MODALITIES = ['text', 'image'] as const satisfies readonly ModelModality[] + /** * Plugin config, validated by the same-named schemastery schema and doubling * as the `llm-deepseek` settings-section shape. Every field is optional in @@ -67,16 +111,36 @@ export interface Config { /** Deployment thinking policy; `disabled` limits every conversation request to `off`. */ thinking?: 'enabled' | 'disabled' /** Default thinking effort (default `high`); `off` disables thinking per request. */ - reasoningEffort?: 'off' | 'high' | 'max' + reasoningEffort?: 'off' | 'low' | 'high' | 'max' /** Default per-request output cap (default 256,000); a model's own cap and explicit request values win. */ maxTokens?: number /** Positive context capacity used when the selected model has no exact value (default 1,000,000). */ defaultContextWindow?: number - /** Advisory models shown by discovery consumers; defaults to V4 Flash and V4 Pro. */ + /** Advisory models shown by discovery consumers; defaults to V4 Flash, V4 Pro, and V4 Flash Vision Exp. */ models?: DeepSeekCatalogModel[] /** Maximum provider idle time while one stream read is outstanding (default five minutes). */ streamIdleTimeoutMs?: number - /** Provider-owned model-request retry policy; omission uses normal defaults. */ + /** Maximum accumulated file-referenced image bytes per chat request (default 128 MiB). */ + maxRequestFilesBytes?: number + /** Maximum accumulated base64 image payload after Files API fallback (default 20 MiB). */ + maxInlineRequestImageBytes?: number + /** Maximum number of represented images per chat request (default 600). */ + maxImagesPerRequest?: number + /** Raw-byte removal step after the request exceeds its file bound (default 64 MiB). */ + imageOffloadByteQuantum?: number + /** Base64-byte removal step after inline fallback exceeds its bound (default 10 MiB). */ + inlineImageOffloadByteQuantum?: number + /** Image-count removal step after the request exceeds its count bound (default 20). */ + imageOffloadCountQuantum?: number + /** Maximum duration of one request-image Files API resolution (default one minute). */ + filesApiTimeoutMs?: number + /** Explicit lifetime assigned to each uploaded image (default seven days). */ + fileExpiresAfterSeconds?: number + /** Remaining lifetime below which an indexed file is replaced (default one hour). */ + fileRefreshMarginSeconds?: number + /** Oldest harness-owned files deleted before one quota-recovery upload retry (default 100). */ + fileQuotaCleanupBatch?: number + /** Provider-owned model-request retry policy; omission uses normal mode with five retries. */ retryPolicy?: RetryPolicyConfig } @@ -86,17 +150,31 @@ const catalogModel: z = z.object({ description: z.string(), contextWindow: z.number().step(1).min(1), maxTokens: z.number().step(1).min(1), + inputModalities: z.array(z.union(MODEL_MODALITIES)).min(1).default(['text']), + imagePixelBudget: z.number().step(1).min(1), + imageMaxBytes: z.number().step(1).min(1), + imageDetail: z.union(['auto', 'low']), }) export const Config: z = z.object({ apiKeyEnv: z.string().role('credential-ref').default(DEFAULT_API_KEY_ENV), baseURL: z.string(), thinking: z.union(['enabled', 'disabled']), - reasoningEffort: z.union(['off', 'high', 'max']), + reasoningEffort: z.union(['off', 'low', 'high', 'max']), maxTokens: z.number().step(1).min(1).max(Number.MAX_SAFE_INTEGER).default(DEFAULT_MAX_TOKENS), defaultContextWindow: z.number().step(1).min(1).default(DEFAULT_CONTEXT_WINDOW), models: z.array(catalogModel).default(DEFAULT_MODELS), streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS), + maxRequestFilesBytes: z.number().step(1).min(1).default(DEFAULT_MAX_REQUEST_FILES_BYTES), + maxInlineRequestImageBytes: z.number().step(1).min(1).default(DEFAULT_MAX_INLINE_REQUEST_IMAGE_BYTES), + maxImagesPerRequest: z.number().step(1).min(1).default(DEFAULT_MAX_IMAGES_PER_REQUEST), + imageOffloadByteQuantum: z.number().step(1).min(1).default(DEFAULT_IMAGE_OFFLOAD_BYTE_QUANTUM), + inlineImageOffloadByteQuantum: z.number().step(1).min(1).default(DEFAULT_INLINE_IMAGE_OFFLOAD_BYTE_QUANTUM), + imageOffloadCountQuantum: z.number().step(1).min(1).default(DEFAULT_IMAGE_OFFLOAD_COUNT_QUANTUM), + filesApiTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_FILES_API_TIMEOUT_MS), + fileExpiresAfterSeconds: z.number().step(1).min(3_600).max(2_592_000).default(DEFAULT_FILE_EXPIRY_SECONDS), + fileRefreshMarginSeconds: z.number().step(1).min(0).default(DEFAULT_FILE_REFRESH_MARGIN_SECONDS), + fileQuotaCleanupBatch: z.number().step(1).min(1).max(1_000).default(DEFAULT_FILE_QUOTA_CLEANUP_BATCH), retryPolicy: RetryPolicySchema, }) @@ -134,6 +212,31 @@ function resolveModels(models: readonly DeepSeekCatalogModel[] | undefined): Dee `llm-deepseek: catalog model "${model.id}" maxTokens must be a positive integer`, ) } + const inputModalities = model.inputModalities ?? ['text'] + if (inputModalities.length === 0) { + throw new Error(`llm-deepseek: catalog model "${model.id}" inputModalities must not be empty`) + } + if (inputModalities.some(modality => !MODEL_MODALITIES.includes(modality))) { + throw new Error( + `llm-deepseek: catalog model "${model.id}" inputModalities must contain only "text" and "image"`, + ) + } + if (new Set(inputModalities).size !== inputModalities.length) { + throw new Error(`llm-deepseek: catalog model "${model.id}" inputModalities must not contain duplicates`) + } + const hasImage = inputModalities.includes('image') + if (!hasImage && (model.imagePixelBudget !== undefined + || model.imageMaxBytes !== undefined || model.imageDetail !== undefined)) { + throw new Error(`llm-deepseek: text-only catalog model "${model.id}" cannot declare image request limits`) + } + if (model.imagePixelBudget !== undefined + && (!Number.isSafeInteger(model.imagePixelBudget) || model.imagePixelBudget <= 0)) { + throw new Error(`llm-deepseek: catalog model "${model.id}" imagePixelBudget must be a positive safe integer`) + } + if (model.imageMaxBytes !== undefined + && (!Number.isSafeInteger(model.imageMaxBytes) || model.imageMaxBytes <= 0)) { + throw new Error(`llm-deepseek: catalog model "${model.id}" imageMaxBytes must be a positive safe integer`) + } if (seen.has(model.id)) throw new Error(`llm-deepseek: duplicate catalog model "${model.id}"`) seen.add(model.id) return { @@ -142,6 +245,17 @@ function resolveModels(models: readonly DeepSeekCatalogModel[] | undefined): Dee ...model.description === undefined ? {} : { description: model.description }, ...model.contextWindow === undefined ? {} : { contextWindow: model.contextWindow }, ...model.maxTokens === undefined ? {} : { maxTokens: model.maxTokens }, + inputModalities: [...inputModalities], + ...hasImage + ? { + imagePixelBudget: model.imagePixelBudget + ?? (model.imageDetail === 'low' + ? DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET + : DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET), + imageMaxBytes: model.imageMaxBytes ?? DEFAULT_REQUEST_IMAGE_MAX_BYTES, + ...model.imageDetail === undefined ? {} : { imageDetail: model.imageDetail }, + } + : {}, } }) } @@ -180,6 +294,66 @@ export function resolveAdapterOptions(config: Config, environment?: LaunchEnviro `llm-deepseek: streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`, ) } + const maxRequestFilesBytes = config.maxRequestFilesBytes ?? DEFAULT_MAX_REQUEST_FILES_BYTES + if (!Number.isSafeInteger(maxRequestFilesBytes) || maxRequestFilesBytes <= 0) { + throw new Error('llm-deepseek: maxRequestFilesBytes must be a positive safe integer') + } + const maxInlineRequestImageBytes = config.maxInlineRequestImageBytes ?? DEFAULT_MAX_INLINE_REQUEST_IMAGE_BYTES + if (!Number.isSafeInteger(maxInlineRequestImageBytes) || maxInlineRequestImageBytes <= 0) { + throw new Error('llm-deepseek: maxInlineRequestImageBytes must be a positive safe integer') + } + const maxImagesPerRequest = config.maxImagesPerRequest ?? DEFAULT_MAX_IMAGES_PER_REQUEST + if (!Number.isSafeInteger(maxImagesPerRequest) || maxImagesPerRequest <= 0) { + throw new Error('llm-deepseek: maxImagesPerRequest must be a positive safe integer') + } + const imageOffloadByteQuantum = config.imageOffloadByteQuantum ?? DEFAULT_IMAGE_OFFLOAD_BYTE_QUANTUM + if (!Number.isSafeInteger(imageOffloadByteQuantum) || imageOffloadByteQuantum <= 0) { + throw new Error('llm-deepseek: imageOffloadByteQuantum must be a positive safe integer') + } + if (imageOffloadByteQuantum > maxRequestFilesBytes) { + throw new Error('llm-deepseek: imageOffloadByteQuantum must not exceed maxRequestFilesBytes') + } + const inlineImageOffloadByteQuantum = config.inlineImageOffloadByteQuantum + ?? DEFAULT_INLINE_IMAGE_OFFLOAD_BYTE_QUANTUM + if (!Number.isSafeInteger(inlineImageOffloadByteQuantum) || inlineImageOffloadByteQuantum <= 0) { + throw new Error('llm-deepseek: inlineImageOffloadByteQuantum must be a positive safe integer') + } + if (inlineImageOffloadByteQuantum > maxInlineRequestImageBytes) { + throw new Error('llm-deepseek: inlineImageOffloadByteQuantum must not exceed maxInlineRequestImageBytes') + } + const imageOffloadCountQuantum = config.imageOffloadCountQuantum ?? DEFAULT_IMAGE_OFFLOAD_COUNT_QUANTUM + if (!Number.isSafeInteger(imageOffloadCountQuantum) || imageOffloadCountQuantum <= 0) { + throw new Error('llm-deepseek: imageOffloadCountQuantum must be a positive safe integer') + } + if (imageOffloadCountQuantum > maxImagesPerRequest) { + throw new Error('llm-deepseek: imageOffloadCountQuantum must not exceed maxImagesPerRequest') + } + const filesApiTimeoutMs = config.filesApiTimeoutMs ?? DEFAULT_FILES_API_TIMEOUT_MS + if (!Number.isFinite(filesApiTimeoutMs) + || filesApiTimeoutMs <= 0 + || filesApiTimeoutMs > MAX_TIMER_DELAY_MS) { + throw new Error( + `llm-deepseek: filesApiTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`, + ) + } + const fileExpiresAfterSeconds = config.fileExpiresAfterSeconds ?? DEFAULT_FILE_EXPIRY_SECONDS + if (!Number.isSafeInteger(fileExpiresAfterSeconds) + || fileExpiresAfterSeconds < 3_600 + || fileExpiresAfterSeconds > 2_592_000) { + throw new Error('llm-deepseek: fileExpiresAfterSeconds must be an integer from 3600 through 2592000') + } + const fileRefreshMarginSeconds = config.fileRefreshMarginSeconds ?? DEFAULT_FILE_REFRESH_MARGIN_SECONDS + if (!Number.isSafeInteger(fileRefreshMarginSeconds) + || fileRefreshMarginSeconds < 0 + || fileRefreshMarginSeconds >= fileExpiresAfterSeconds) { + throw new Error('llm-deepseek: fileRefreshMarginSeconds must be a non-negative integer below fileExpiresAfterSeconds') + } + const fileQuotaCleanupBatch = config.fileQuotaCleanupBatch ?? DEFAULT_FILE_QUOTA_CLEANUP_BATCH + if (!Number.isSafeInteger(fileQuotaCleanupBatch) + || fileQuotaCleanupBatch < 1 + || fileQuotaCleanupBatch > 1_000) { + throw new Error('llm-deepseek: fileQuotaCleanupBatch must be an integer from 1 through 1000') + } return { apiKeyEnv: credentialRef(config.apiKeyEnv ?? DEFAULT_API_KEY_ENV), baseURL: config.baseURL @@ -193,6 +367,18 @@ export function resolveAdapterOptions(config: Config, environment?: LaunchEnviro defaultContextWindow: config.defaultContextWindow ?? DEFAULT_CONTEXT_WINDOW, models: resolveModels(config.models), streamIdleTimeoutMs, + maxRequestFilesBytes, + maxInlineRequestImageBytes, + maxImagesPerRequest, + imageOffloadByteQuantum, + inlineImageOffloadByteQuantum, + imageOffloadCountQuantum, + filesApiTimeoutMs, + filePolicy: { + expiresAfterSeconds: fileExpiresAfterSeconds, + refreshMarginSeconds: fileRefreshMarginSeconds, + quotaCleanupBatch: fileQuotaCleanupBatch, + }, retryPolicy: resolveRetryPolicy(config.retryPolicy, 'llm-deepseek: retryPolicy'), } } @@ -247,7 +433,17 @@ export function apply(ctx: Context, config: Config): void { let userId: AnonymousUserId | undefined const resolveUserId = (): AnonymousUserId => userId ??= getOrCreateAnonymousUserId() - const adapter = new DeepSeekAdapter({ options, resolveApiKey, resolveUserId }) + const adapter = new DeepSeekAdapter({ + options, + resolveApiKey, + resolveUserId, + resolveAttachments: () => ctx.get('attachments'), + prepareExtensions: (request) => { + const extensions = ctx.get('deepseekLlmApiExtensions') + return extensions?.prepare(request) + ?? Promise.resolve({ fields: {}, accept: () => Promise.resolve() }) + }, + }) ctx.llm.registerConfigurableProviders([ { provider: PROVIDER, displayName: 'DeepSeek', settingsNs: NS, settingsPath: [] }, ]) diff --git a/packages/llm/llm-deepseek/src/serialize.ts b/packages/llm/llm-deepseek/src/serialize.ts index 34fa214bb9..3b22967d96 100644 --- a/packages/llm/llm-deepseek/src/serialize.ts +++ b/packages/llm/llm-deepseek/src/serialize.ts @@ -1,31 +1,75 @@ /** - * Serialize harness messages into DeepSeek chat completions. User text is joined; assistant text - * becomes `content`, tool calls become `tool_calls`, and tool results become separate tool messages. - * Assistant reasoning is replayed as `reasoning_content` only on tool-call turns, as required by - * thinking-mode passback. Core image blocks are rejected explicitly because this wire route is text-only; - * unknown declaration-merged block types retain the adapter's documented extension fallback. + * Serialize harness messages into DeepSeek chat completions. Text-only + * requests retain string user content; the image path resolves durable + * attachments into ordered file-id or inline parts. Tool-result images follow their + * string-only tool messages in a separate user message. * @module dsh-llm-deepseek/serialize */ -import { contentHasImage, LlmError } from '@deepseek-ai/dsh-llm' +import { contentHasImage, LlmError, offloadRequestImagesWithPolicy, requestImageHandleText } from '@deepseek-ai/dsh-llm' import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm' -import type { WireMessage, WireRequest, WireTool } from './types.ts' +import type { ImageAttachmentRef, RequestImageAttachment } from '@deepseek-ai/dsh-attachment' +import type { + WireImageContentPart, + WireMessage, + WireRequest, + WireTextContentPart, + WireTool, + WireUserContentPart, +} from './types.ts' /** Adapter-level request defaults (from plugin config). */ export interface RequestDefaults { thinking?: 'enabled' | 'disabled' | undefined - reasoningEffort?: 'off' | 'high' | 'max' | undefined + reasoningEffort?: 'off' | 'low' | 'high' | 'max' | undefined } interface ResolvedThinking { thinking?: 'enabled' | 'disabled' - reasoningEffort?: 'high' | 'max' + reasoningEffort?: 'low' | 'high' | 'max' } +/** Provider representation for every retained image in one request. */ +export type ImageRequestRepresentation = + | { + kind: 'file' + /** Resolve a retained request version to a reusable DeepSeek file id. */ + resolveFileId: ( + version: RequestImageAttachment, + block: Extract, + location: ImageWireLocation, + ) => Promise + } + | { kind: 'base64' } + +/** Dependencies required only when the request contains image input. */ +export interface ImageSerializationOptions { + /** One representation used for every retained image in this request. */ + representation: ImageRequestRepresentation + /** Request versions prepared for the conservatively retained normalized attachments, keyed by attachment id. */ + requestImages: ReadonlyMap + /** Positive bound on accumulated represented image bytes. */ + maxRequestImageBytes: number + /** Maximum represented images in one request. */ + maxImagesPerRequest?: number + /** Represented-byte removal step applied after the request exceeds its byte bound. */ + byteQuantum?: number + /** Image-count removal step applied after the request exceeds its count bound. */ + countQuantum?: number +} + +/** Durable message and image ordinal used in provider diagnostics. */ +export interface ImageWireLocation { + message: number + image: number +} + +const TOOL_RESULT_IMAGE_TEXT = 'Attached image(s) from tool result:' + /** Validate the adapter-owned effort before resolving its DeepSeek wire fields. */ -function reasoningEffort(effort: NonNullable): 'off' | 'high' | 'max' { - if (effort === 'off' || effort === 'high' || effort === 'max') { - return effort as 'off' | 'high' | 'max' +function reasoningEffort(effort: NonNullable): 'off' | 'low' | 'high' | 'max' { + if (effort === 'off' || effort === 'low' || effort === 'high' || effort === 'max') { + return effort as 'off' | 'low' | 'high' | 'max' } throw new LlmError( `DeepSeek does not support reasoning effort "${effort}"`, @@ -46,7 +90,7 @@ function resolveThinking(options: GenerateOptions, defaults: RequestDefaults): R ) } if (effort === 'off') return { thinking: 'disabled' } - if (effort === 'high' || effort === 'max') { + if (effort === 'low' || effort === 'high' || effort === 'max') { return { thinking: 'enabled', reasoningEffort: effort } } return defaults.thinking === undefined ? {} : { thinking: defaults.thinking } @@ -67,6 +111,90 @@ function assertTextOnly(blocks: readonly ContentBlock[]): void { } } +/** Reject roles whose DeepSeek history format cannot carry image input. */ +function assertSupportedImageRoles(messages: readonly Message[]): void { + for (const message of messages) { + if (message.role !== 'user' && contentHasImage(message.content)) { + throw new LlmError( + `The DeepSeek chat-completions adapter cannot represent image content in a ${message.role} message.`, + 'UNSUPPORTED_CONTENT', + ) + } + } +} + +/** Describe the exact request preview and its model-callable coordinate system. */ +function imageHandle( + version: RequestImageAttachment, + precededByContent: boolean, +): WireTextContentPart { + return { + type: 'text', + text: `${precededByContent ? '\n' : ''}${requestImageHandleText(version)}`, + } +} + +/** Resolve one durable image into its descriptor and transient DeepSeek image part. */ +async function imageParts( + block: Extract, + images: ImageSerializationOptions, + location: ImageWireLocation, + precededByContent: boolean, +): Promise<[WireTextContentPart, WireImageContentPart]> { + const version = images.requestImages.get(block.attachment.attachmentId) + if (version === undefined) { + throw new LlmError( + `DeepSeek request image ${block.attachment.attachmentId} was not prepared.`, + 'INVALID_REQUEST', + ) + } + const image: WireImageContentPart = images.representation.kind === 'file' + ? { type: 'file', file_id: await images.representation.resolveFileId(version, block, location) } + : { + type: 'image_url', + image_url: { url: `data:${version.mediaType};base64,${Buffer.from(version.data).toString('base64')}` }, + } + return [imageHandle(version, precededByContent), image] +} + +/** Convert user or nested tool-result blocks into ordered wire parts. */ +async function contentParts( + blocks: readonly ContentBlock[], + images: ImageSerializationOptions, + message: number, + nextImage: { value: number }, +): Promise { + const parts: WireUserContentPart[] = [] + for (const block of blocks) { + switch (block.type) { + case 'text': + if (block.text.length > 0) parts.push({ type: 'text', text: block.text }) + break + case 'image': + nextImage.value += 1 + parts.push(...await imageParts(block, images, { message, image: nextImage.value }, parts.length > 0)) + break + case 'tool-result': + parts.push(...await contentParts(block.content, images, message, nextImage)) + break + default: + // Other merge-extensible blocks are not DeepSeek user-input vocabulary. + break + } + } + return parts +} + +/** Keep text-only user messages on the compact string wire form. */ +function userContent(parts: readonly WireUserContentPart[]): string | WireUserContentPart[] { + const text: string[] = [] + for (const part of parts) { + if (part.type !== 'text') return [...parts] + text.push(part.text) + } + return text.join('') +} + /** Serialize one assistant message (text + reasoning + tool calls). */ function serializeAssistant(message: Message): WireMessage { const text = flattenText(message.content) @@ -93,10 +221,12 @@ function serializeAssistant(message: Message): WireMessage { // the message sits durably in the session log, a null here bricks every // later turn of that session. content: text, - // Official passback rule (guides/thinking_mode.mdx): reasoning_content - // must return on tool-call turns; it is ignored on plain turns, so we - // drop it there to save tokens. - ...toolCalls.length > 0 && reasoning.length > 0 ? { reasoning_content: reasoning } : {}, + // CoT passback on every reasoning-carrying turn. The official rule + // (guides/thinking_mode.mdx) requires it on tool-call turns and ignores it + // elsewhere; a gateway re-encoding the conversation for another vendor + // recovers that turn's upstream thinking signature by hashing this exact + // text, which a tool-call-free turn carries nowhere else. + ...reasoning.length > 0 ? { reasoning_content: reasoning } : {}, ...toolCalls.length > 0 ? { tool_calls: toolCalls } : {}, } } @@ -140,6 +270,102 @@ export function serializeMessages(messages: Message[]): WireMessage[] { return wire } +/** + * Serialize image-capable history after resolving durable attachments. + * Consecutive tool results keep string `tool` messages and share one following + * user message containing their images. + * @param messages - transient request history after request-size offloading. + * @param images - prepared request versions, one provider representation, and its budget. + * @returns ordered DeepSeek wire messages. + */ +export async function serializeMessagesWithImages( + messages: readonly Message[], + images: ImageSerializationOptions, +): Promise { + assertSupportedImageRoles(messages) + const wire: WireMessage[] = [] + let pendingToolImages: WireImageContentPart[] = [] + const flushToolImages = (): void => { + if (pendingToolImages.length === 0) return + wire.push({ + role: 'user', + content: [{ type: 'text', text: TOOL_RESULT_IMAGE_TEXT }, ...pendingToolImages], + }) + pendingToolImages = [] + } + + for (const [messageIndex, message] of messages.entries()) { + const nextImage = { value: 0 } + if (message.role === 'system') { + flushToolImages() + wire.push({ role: 'system', content: flattenText(message.content) }) + continue + } + if (message.role === 'assistant') { + flushToolImages() + wire.push(serializeAssistant(message)) + continue + } + + const regular = message.content.filter(block => block.type !== 'tool-result') + const toolResults = message.content.filter((block): block is Extract => ( + block.type === 'tool-result' + )) + const content = userContent(await contentParts(regular, images, messageIndex + 1, nextImage)) + if (content.length > 0 || toolResults.length === 0) { + flushToolImages() + wire.push({ + role: 'user', + content, + }) + } + for (const result of toolResults) { + const parts = await contentParts(result.content, images, messageIndex + 1, nextImage) + const imageParts = parts.filter((part): part is WireImageContentPart => part.type !== 'text') + const text = parts.filter(part => part.type === 'text').map(part => part.text).join('') + wire.push({ + role: 'tool', + tool_call_id: result.toolCallId, + content: text || '(no output)', + }) + pendingToolImages.push(...imageParts) + } + } + flushToolImages() + return wire +} + +/** Assemble request fields shared by text-only and image-capable conversion. */ +function requestWithMessages( + options: GenerateOptions, + messages: WireMessage[], + defaults: RequestDefaults, +): WireRequest { + const tools: WireTool[] | undefined = options.tools?.map(tool => ({ + type: 'function', + function: { + name: tool.name, + description: tool.description, + parameters: tool.parameters, + }, + })) + const resolvedThinking = resolveThinking(options, defaults) + return { + model: options.model, + messages, + stream: true, + stream_options: { include_usage: true }, + ...resolvedThinking.thinking !== undefined ? { thinking: { type: resolvedThinking.thinking } } : {}, + ...resolvedThinking.reasoningEffort !== undefined + ? { reasoning_effort: resolvedThinking.reasoningEffort } + : {}, + ...tools !== undefined && tools.length > 0 ? { tools } : {}, + ...options.temperature !== undefined ? { temperature: options.temperature } : {}, + ...options.maxTokens === undefined ? {} : { max_tokens: options.maxTokens }, + ...options.stop !== undefined ? { stop: options.stop } : {}, + } +} + /** * Build the full wire request. Always streaming (`stream: true`, usage * reporting on); optional fields are omitted rather than sent as null, so @@ -158,30 +384,42 @@ export function serializeRequest( } messages.push(...serializeMessages(options.messages)) - const tools: WireTool[] | undefined = options.tools?.map(tool => ({ - type: 'function', - function: { - name: tool.name, - description: tool.description, - parameters: tool.parameters, + return requestWithMessages(options, messages, defaults) +} + +/** + * Build one image-capable request while keeping durable bytes out of session + * messages. Oversized oldest images become deterministic text after their + * exact request-version byte lengths are known and before provider serialization. + * @param options - harness request containing image-capable user content. + * @param images - attachment resolver, request bound, and cancellation. + * @param defaults - adapter-level thinking defaults. + * @returns the fully materialized DeepSeek request body. + */ +export async function serializeRequestWithImages( + options: GenerateOptions, + images: ImageSerializationOptions, + defaults: RequestDefaults = {}, +): Promise { + assertSupportedImageRoles(options.messages) + const requestMessages = offloadRequestImagesWithPolicy(options.messages, { + representation: images.representation.kind === 'file' ? 'raw' : 'base64', + byteLength: (ref) => { + const version = images.requestImages.get(ref.attachmentId) + if (version === undefined) { + throw new LlmError(`DeepSeek request image ${ref.attachmentId} was not prepared.`, 'INVALID_REQUEST') + } + return version.bytes }, - })) - // A short title budget must produce visible text; conversation and - // compaction calls continue to inherit the adapter's thinking defaults. - const resolvedThinking = resolveThinking(options, defaults) - - return { - model: options.model, - messages, - stream: true, - stream_options: { include_usage: true }, - ...resolvedThinking.thinking !== undefined ? { thinking: { type: resolvedThinking.thinking } } : {}, - ...resolvedThinking.reasoningEffort !== undefined - ? { reasoning_effort: resolvedThinking.reasoningEffort } - : {}, - ...tools !== undefined && tools.length > 0 ? { tools } : {}, - ...options.temperature !== undefined ? { temperature: options.temperature } : {}, - ...options.maxTokens === undefined ? {} : { max_tokens: options.maxTokens }, - ...options.stop !== undefined ? { stop: options.stop } : {}, + maxBytes: images.maxRequestImageBytes, + ...images.maxImagesPerRequest === undefined ? {} : { maxImages: images.maxImagesPerRequest }, + ...images.byteQuantum === undefined ? {} : { byteQuantum: images.byteQuantum }, + ...images.countQuantum === undefined ? {} : { countQuantum: images.countQuantum }, + }) + const messages: WireMessage[] = [] + if (options.system !== undefined) { + messages.push({ role: 'system', content: options.system }) } + messages.push(...await serializeMessagesWithImages(requestMessages, images)) + return requestWithMessages(options, messages, defaults) } diff --git a/packages/llm/llm-deepseek/src/types.ts b/packages/llm/llm-deepseek/src/types.ts index a9d7403a35..f5dd5df0aa 100644 --- a/packages/llm/llm-deepseek/src/types.ts +++ b/packages/llm/llm-deepseek/src/types.ts @@ -17,8 +17,8 @@ export interface WireRequest { stream_options: { include_usage: true } /** Thinking-mode toggle (top level, NOT inside extra_body on the wire). */ thinking?: { type: 'enabled' | 'disabled' } - /** Thinking effort (official levels; low/medium map to high server-side). */ - reasoning_effort?: 'high' | 'max' + /** Thinking effort (official levels). */ + reasoning_effort?: 'low' | 'high' | 'max' tools?: WireTool[] temperature?: number max_tokens?: number @@ -35,10 +35,34 @@ export interface WireSystemMessage { content: string } -/** User-role message: a single string of user input. */ +/** Text part inside a multimodal user message. */ +export interface WireTextContentPart { + type: 'text' + text: string +} + +/** Files API reference inside a multimodal user message. */ +export interface WireFileContentPart { + type: 'file' + file_id: string +} + +/** Inline base64 data URL inside a multimodal user message. */ +export interface WireImageUrlContentPart { + type: 'image_url' + image_url: { url: string } +} + +/** One image representation accepted by a multimodal user message. */ +export type WireImageContentPart = WireFileContentPart | WireImageUrlContentPart + +/** Ordered input part accepted by a multimodal user message. */ +export type WireUserContentPart = WireTextContentPart | WireImageContentPart + +/** User-role message: text-only string or ordered multimodal input. */ export interface WireUserMessage { role: 'user' - content: string + content: string | WireUserContentPart[] } /** Tool-role message: the result of one tool call, keyed by its call id. */ @@ -64,9 +88,11 @@ export interface WireAssistantMessage { role: 'assistant' content: string | null /** - * CoT passback. REQUIRED on assistant turns that carried tool calls - * (thinking mode); ignored on tool-call-free turns (we omit it there to - * save tokens). See guides/thinking_mode.mdx § Tool Calls. + * CoT passback, present on every turn whose assistant content carried + * reasoning. REQUIRED on tool-call turns in thinking mode (see + * guides/thinking_mode.mdx § Tool Calls); DeepSeek ignores it elsewhere, + * while a gateway re-encoding for another vendor recovers that turn's + * thinking signature by hashing it. */ reasoning_content?: string tool_calls?: WireToolCall[] diff --git a/packages/llm/llm-deepseek/src/upload-index.ts b/packages/llm/llm-deepseek/src/upload-index.ts new file mode 100644 index 0000000000..d442bb55fe --- /dev/null +++ b/packages/llm/llm-deepseek/src/upload-index.ts @@ -0,0 +1,225 @@ +/** Durable DeepSeek attachment-to-file-id index. @module dsh-llm-deepseek/upload-index */ + +import { createHash } from 'node:crypto' +import { readFile, mkdir } from 'node:fs/promises' +import { dirname, join } from 'node:path' +import { withFileLock, writeFileAtomic } from '@deepseek-ai/dsh-atomic-write' +import { resolveDshHome } from '@deepseek-ai/dsh-home-paths' +import { ImageVariantId } from '@deepseek-ai/dsh-attachment' +import type { AttachmentId, ImageVariantId as ImageVariantIdType } from '@deepseek-ai/dsh-attachment' +import { DeepSeekFileId, DeepSeekFileScope } from './file-id.ts' +import type { DeepSeekFileId as DeepSeekFileIdType, DeepSeekFileScope as DeepSeekFileScopeType } from './file-id.ts' + +/** One durable remote upload mapping. Unix times are milliseconds. */ +export interface DeepSeekUploadRecord { + scope: DeepSeekFileScopeType + /** Provider-independent normalized attachment from which the uploaded request version was derived. */ + attachmentId: AttachmentId + /** Complete request transformation identity, including route budgets and encoder parameters. */ + variantId: ImageVariantIdType + fileId: DeepSeekFileIdType + bytes: number + createdAt: number + expiresAt: number +} + +interface StoredIndex { + formatVersion: 3 + records: DeepSeekUploadRecord[] +} + +class InvalidUploadIndexError extends Error {} + +/** Candidate commit outcome when another process already published a reusable upload. */ +export interface UploadIndexCommit { + record: DeepSeekUploadRecord + accepted: boolean +} + +/** + * Derive a non-secret stable index namespace without persisting or logging the API key. + * @param baseURL - normalized provider endpoint namespace. + * @param apiKey - resolved credential used only as hash input. + * @returns branded SHA-256 namespace digest. + */ +export function deepSeekFileScope(baseURL: string, apiKey: string): DeepSeekFileScopeType { + const digest = createHash('sha256') + .update(baseURL.replace(/\/+$/u, '')) + .update('\0') + .update(apiKey) + .digest('hex') + return DeepSeekFileScope(digest) +} + +function absent(error: unknown): boolean { + return (error as NodeJS.ErrnoException | null)?.code === 'ENOENT' +} + +function parseRecord(value: unknown): DeepSeekUploadRecord { + if (value === null || typeof value !== 'object' || Array.isArray(value)) { + throw new InvalidUploadIndexError('llm-deepseek: upload index contains a non-object record') + } + const record = value as Record + if (typeof record.scope !== 'string' || !/^[0-9a-f]{64}$/u.test(record.scope) + || typeof record.attachmentId !== 'string' || !/^sha256:[0-9a-f]{64}$/u.test(record.attachmentId) + || typeof record.variantId !== 'string' || !/^sha256:[0-9a-f]{64}$/u.test(record.variantId) + || typeof record.fileId !== 'string' || record.fileId.length === 0 + || !Number.isSafeInteger(record.bytes) || (record.bytes as number) < 0 + || !Number.isSafeInteger(record.createdAt) || (record.createdAt as number) < 0 + || !Number.isSafeInteger(record.expiresAt) || (record.expiresAt as number) < 0) { + throw new InvalidUploadIndexError('llm-deepseek: upload index contains an invalid record') + } + return { + scope: DeepSeekFileScope(record.scope), + attachmentId: record.attachmentId as AttachmentId, + variantId: ImageVariantId(record.variantId), + fileId: DeepSeekFileId(record.fileId), + bytes: record.bytes as number, + createdAt: record.createdAt as number, + expiresAt: record.expiresAt as number, + } +} + +function parseIndex(text: string): StoredIndex { + let value: unknown + try { + value = JSON.parse(text) + } catch (error: unknown) { + throw new InvalidUploadIndexError('llm-deepseek: upload index is not valid JSON', { cause: error }) + } + if (value === null || typeof value !== 'object' || Array.isArray(value)) { + throw new InvalidUploadIndexError('llm-deepseek: upload index is not an object') + } + const index = value as { formatVersion?: unknown; records?: unknown } + if (index.formatVersion !== 3 || !Array.isArray(index.records)) { + throw new InvalidUploadIndexError('llm-deepseek: unsupported upload index format') + } + const records = index.records.map(parseRecord) + const keys = new Set() + for (const record of records) { + const key = `${record.scope}\0${record.variantId}` + if (keys.has(key)) throw new InvalidUploadIndexError('llm-deepseek: upload index contains duplicate mappings') + keys.add(key) + } + return { formatVersion: 3, records } +} + +function reusable(record: DeepSeekUploadRecord, now: number, refreshMarginMs: number): boolean { + return record.expiresAt - now > refreshMarginMs +} + +/** Atomic local index shared by every DeepSeek session in this DSH home. */ +export class DeepSeekUploadIndex { + /** Absolute owner-private JSON index path. */ + readonly path: string + + /** + * @param path - explicit test path; omission uses `DSH_HOME/llm-deepseek/files-v3.json`. + */ + constructor(path = join(resolveDshHome(), 'llm-deepseek', 'files-v3.json')) { + this.path = path + } + + private async load(): Promise { + try { + return parseIndex(await readFile(this.path, 'utf8')) + } catch (error: unknown) { + if (absent(error) || error instanceof InvalidUploadIndexError) { + return { formatVersion: 3, records: [] } + } + throw error + } + } + + private async save(index: StoredIndex): Promise { + await writeFileAtomic(this.path, `${JSON.stringify(index, undefined, 2)}\n`, { + mode: 0o600, + dirMode: 0o700, + }) + } + + /** + * Read one reusable mapping. + * @param scope - endpoint/API-key namespace. + * @param variantId - complete request-image transformation identity. + * @param now - current Unix time in milliseconds. + * @param refreshMarginMs - remaining lifetime below which a mapping is not reused. + * @returns the mapping when it has enough lifetime remaining. + */ + async get( + scope: DeepSeekFileScopeType, + variantId: ImageVariantIdType, + now: number, + refreshMarginMs: number, + ): Promise { + const record = (await this.load()).records.find(candidate => ( + candidate.scope === scope && candidate.variantId === variantId + )) + return record !== undefined && reusable(record, now, refreshMarginMs) ? record : undefined + } + + /** + * Publish a completed upload unless another process already published a reusable mapping. + * @param candidate - completed remote upload. + * @param now - current Unix time in milliseconds. + * @param refreshMarginMs - minimum reusable remaining lifetime. + * @returns the winning record and whether the candidate entered the index. + */ + async commit( + candidate: DeepSeekUploadRecord, + now: number, + refreshMarginMs: number, + ): Promise { + await mkdir(dirname(this.path), { recursive: true, mode: 0o700 }) + return withFileLock(this.path, async () => { + const index = await this.load() + const existing = index.records.find(record => ( + record.scope === candidate.scope + && record.variantId === candidate.variantId + && reusable(record, now, refreshMarginMs) + )) + if (existing !== undefined) return { record: existing, accepted: false } + const records = index.records.filter(record => ( + reusable(record, now, refreshMarginMs) + && !(record.scope === candidate.scope && record.variantId === candidate.variantId) + )) + records.push(candidate) + await this.save({ formatVersion: 3, records }) + return { record: candidate, accepted: true } + }) + } + + /** + * Remove one exact mapping without deleting a concurrently installed successor. + * @param scope - endpoint/API-key namespace. + * @param variantId - complete request-image transformation identity. + * @param fileId - exact remote generation being invalidated. + */ + async remove( + scope: DeepSeekFileScopeType, + variantId: ImageVariantIdType, + fileId: DeepSeekFileIdType, + ): Promise { + await mkdir(dirname(this.path), { recursive: true, mode: 0o700 }) + await withFileLock(this.path, async () => { + const index = await this.load() + const records = index.records.filter(record => !( + record.scope === scope && record.variantId === variantId && record.fileId === fileId + )) + if (records.length !== index.records.length) await this.save({ formatVersion: 3, records }) + }) + } + + /** + * Remove every local mapping for one remote namespace. + * @param scope - endpoint/API-key namespace. + */ + async clear(scope: DeepSeekFileScopeType): Promise { + await mkdir(dirname(this.path), { recursive: true, mode: 0o700 }) + await withFileLock(this.path, async () => { + const index = await this.load() + const records = index.records.filter(record => record.scope !== scope) + if (records.length !== index.records.length) await this.save({ formatVersion: 3, records }) + }) + } +} diff --git a/packages/llm/llm-deepseek/tests/adapter.e2e.ts b/packages/llm/llm-deepseek/tests/adapter.e2e.ts index 269169c73a..6e7c3307f7 100644 --- a/packages/llm/llm-deepseek/tests/adapter.e2e.ts +++ b/packages/llm/llm-deepseek/tests/adapter.e2e.ts @@ -1,26 +1,100 @@ +import { readFileSync } from 'node:fs' import { mkdtemp, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' +import { randomBytes } from 'node:crypto' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import LlmRuntime, { createUserMessage, CallId, ReasoningEffortId , createMessage } from '@deepseek-ai/dsh-llm' +import Loader from '@deepseek-ai/cordis-plugin-loader' +import AgentRegistry from '@deepseek-ai/dsh-agent' +import LlmRuntime, { createUserMessage, CallId, ReasoningEffortId, createMessage } from '@deepseek-ai/dsh-llm' import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm' +import AttachmentStore, { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment' +import type { + ImageAttachmentLimits, + ImageAttachmentRef, + ImageRequestPolicy, + RequestImageAttachment, + SaveImageAttachment, + StoredImageAttachment, +} from '@deepseek-ai/dsh-attachment' import { LocalCredentialProvider } from '@deepseek-ai/dsh-credentials-local' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import DeepSeekLlmApiExtensionRegistry from '@deepseek-ai/dsh-deepseek-llm-api-extensions' +import * as PluginPackageInventoryDeepSeek from '@deepseek-ai/dsh-plugin-package-inventory-deepseek' +import * as SessionLogDeepSeek from '@deepseek-ai/dsh-session-log-deepseek' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' import type { Config } from '@deepseek-ai/dsh-llm-deepseek' import { assemble, type AssembledResult } from './assemble.ts' /** * Real-API e2e for the direct-fetch adapter: V4 Flash + V4 Pro across - * thinking modes and both official effort levels. Key-gated — skips - * entirely without $DEEPSEEK_API_KEY (see vitest.e2e.config.ts). + * thinking modes and all official effort levels. The suite skips entirely + * without $DEEPSEEK_API_KEY; the pre-release vision smoke additionally + * requires $DEEPSEEK_VISION_E2E=1 (see vitest.e2e.config.ts). */ const FLASH = 'deepseek-v4-flash' const PRO = 'deepseek-v4-pro' +const VISION = 'deepseek-v4-flash-vision-exp' +const VISION_E2E_ENABLED = process.env.DEEPSEEK_VISION_E2E === '1' +const TEST_PNG = Uint8Array.from(readFileSync( + new URL('../../llm-pi-ai/tests/fixtures/qr-code.png', import.meta.url), +)) const contexts: Context[] = [] let identityHome: string +class E2eAttachmentStore extends AttachmentStore { + readonly imageLimits: ImageAttachmentLimits = { + maxImageBytes: TEST_PNG.byteLength, + maxImagesPerMessage: 1, + maxMessageImageBytes: TEST_PNG.byteLength, + maxImagePixels: 256 * 256, + maxImageDimension: 256, + mediaTypes: ['image/png'], + } + readonly ref: ImageAttachmentRef = { + attachmentId: AttachmentId(`sha256:${randomBytes(32).toString('hex')}`), + mediaType: 'image/png', + bytes: TEST_PNG.byteLength, + width: 256, + height: 256, + name: 'files-api-e2e.png', + } + readonly version: RequestImageAttachment = { + variantId: ImageVariantId(`sha256:${randomBytes(32).toString('hex')}`), + attachment: this.ref, + data: TEST_PNG, + mediaType: 'image/png', + bytes: TEST_PNG.byteLength, + width: 256, + height: 256, + depth: 'uchar', + space: 'srgb', + hasAlpha: false, + } + + validateImage(_input: SaveImageAttachment): Promise { + return Promise.resolve() + } + + saveImage(_input: SaveImageAttachment): Promise { + return Promise.resolve(this.ref) + } + + readImage(ref: ImageAttachmentRef, _signal?: AbortSignal): Promise { + return Promise.resolve({ ref, data: TEST_PNG }) + } + + override readImageRequest( + _ref: ImageAttachmentRef, + _policy: ImageRequestPolicy, + _signal?: AbortSignal, + ): Promise { + return Promise.resolve(this.version) + } +} + beforeEach(async () => { identityHome = await mkdtemp(join(tmpdir(), 'dsh-e2e-user-id-')) vi.stubEnv('DSH_HOME', identityHome) @@ -30,12 +104,14 @@ async function harness(_model: string, config: Partial = {}) { const ctx = new Context() contexts.push(ctx) await ctx.plugin(LlmRuntime) + await ctx.plugin(E2eAttachmentStore) await ctx.plugin(LlmDeepSeek, config) return ctx } afterEach(async () => { await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose())) + vi.unstubAllGlobals() vi.unstubAllEnvs() await rm(identityHome, { recursive: true, force: true }) }) @@ -65,6 +141,76 @@ const weatherTool: ToolSchema = { } describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () => { + it.skipIf(!VISION_E2E_ENABLED)('uses the built-in official route to upload, reference, and delete one image', async () => { + const key = process.env.DEEPSEEK_API_KEY + if (key === undefined) throw new Error('e2e ran without DEEPSEEK_API_KEY') + const baseURL = process.env.DEEPSEEK_BASE_URL ?? LlmDeepSeek.PUBLIC_BASE_URL + const ctx = await harness(VISION, { baseURL }) + await ctx.plugin(E2eAttachmentStore) + const attachments = ctx.attachments as E2eAttachmentStore + let uploadedFile: LlmDeepSeek.DeepSeekFileIdType | undefined + const nativeFetch = globalThis.fetch + const observedFetch: typeof fetch = async (input, init) => { + const response = await nativeFetch(input, init) + const url = new URL(input instanceof Request ? input.url : input) + const method = init?.method ?? (input instanceof Request ? input.method : 'GET') + if (method === 'POST' && url.pathname.endsWith('/files') && response.ok) { + const value = await response.clone().json() as { id?: unknown } + if (typeof value.id === 'string') uploadedFile = LlmDeepSeek.DeepSeekFileId(value.id) + } + return response + } + vi.stubGlobal('fetch', observedFetch) + const files = new LlmDeepSeek.DeepSeekFilesClient({ baseURL, apiKey: key }) + + try { + const result = await assemble(ctx, { + model: VISION, + messages: [createUserMessage({ + content: [ + { type: 'text', text: 'Briefly describe this image.' }, + { type: 'image', attachment: attachments.ref }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + maxTokens: 100, + }) + expect( + result.finish.kind, + `DeepSeek vision result: ${JSON.stringify(result.finish)}`, + ).toBe('stop') + expect(textOf(result).trim().length).toBeGreaterThan(0) + expect(uploadedFile).toMatch(/^file-api-/u) + } finally { + if (uploadedFile !== undefined) await files.delete(uploadedFile) + } + }) + + it('accepts the session-log and plugin-package request extension fields', async () => { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(Loader) + await ctx.plugin(AgentRegistry) + await ctx.plugin(LlmRuntime) + await ctx.plugin(SessionStore) + await ctx.plugin(DeepSeekLlmApiExtensionRegistry) + await ctx.plugin(SessionLogDeepSeek, { enabled: true }) + await ctx.plugin(PluginPackageInventoryDeepSeek) + await ctx.plugin(LlmDeepSeek, { thinking: 'disabled' }) + const session = ctx.sessions.create(SessionId('real-extension-fields')) + session.append('turn/start', { turn: 1 }) + + const result = await assemble(ctx, { + model: FLASH, + messages: ask('Reply with exactly the word: pong'), + maxTokens: 50, + sessionId: session.id, + }) + expect(result.finish.kind).toBe('stop') + expect(textOf(result).toLowerCase()).toContain('pong') + expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(0) + }) + it('serves a real request with the key held only by a credentials-local document', async () => { const key = process.env.DEEPSEEK_API_KEY if (key === undefined) throw new Error('e2e ran without DEEPSEEK_API_KEY') @@ -72,7 +218,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () try { // JSON.stringify quotes the value: YAML is a JSON superset, so a real // key survives whatever characters it happens to carry. - await writeFile(join(dir, '.credentials.yaml'), `DEEPSEEK_API_KEY: ${JSON.stringify(key)}\n`, { mode: 0o600 }) + await writeFile(join(dir, '.credentials.yaml'), `version: 1\nrefs:\n DEEPSEEK_API_KEY: ${JSON.stringify(key)}\n`, { mode: 0o600 }) // Scrub the ambient variable so only the credential seam can supply the // key: this request proves the per-request resolution path end to end. vi.stubEnv('DEEPSEEK_API_KEY', '') @@ -95,7 +241,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () } }) - it('flash dynamically switches from off to high', async () => { + it('flash dynamically switches from off to low', async () => { const ctx = await harness(FLASH, { reasoningEffort: 'off' }) const withoutThinking = await assemble(ctx,{ model: FLASH, @@ -110,7 +256,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () const withThinking = await assemble(ctx,{ model: FLASH, - reasoningEffort: ReasoningEffortId('high'), + reasoningEffort: ReasoningEffortId('low'), messages: ask('Which is larger, 9.11 or 9.8? Answer with just the number.'), maxTokens: 2000, }) diff --git a/packages/llm/llm-deepseek/tests/adapter.spec.ts b/packages/llm/llm-deepseek/tests/adapter.spec.ts index 387abf583f..5a32a35664 100644 --- a/packages/llm/llm-deepseek/tests/adapter.spec.ts +++ b/packages/llm/llm-deepseek/tests/adapter.spec.ts @@ -3,9 +3,12 @@ import { mkdtempSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { Context } from '@deepseek-ai/cordis' +import { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment' +import type { AttachmentStore, ImageAttachmentRef, RequestImageAttachment } from '@deepseek-ai/dsh-attachment' import { createLaunchEnvironmentSnapshot } from '@deepseek-ai/dsh-launch-environment' -import LlmRuntime, { createUserMessage, +import LlmRuntime, { CallId, createUserMessage, CONTEXT_WINDOW_EXCEEDED_CODE, + LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE, ReasoningEffortId, @@ -14,9 +17,11 @@ import LlmRuntime, { createUserMessage, import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { getOrCreateAnonymousUserId, type AnonymousUserId } from '@deepseek-ai/dsh-anonymous-user-id' import { SessionId } from '@deepseek-ai/dsh-session' +import DeepSeekLlmApiExtensionRegistry from '@deepseek-ai/dsh-deepseek-llm-api-extensions' +import type { PreparedDeepSeekLlmApiExtensions } from '@deepseek-ai/dsh-deepseek-llm-api-extensions' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' import { DeepSeekAdapter, resolveAdapterOptions } from '@deepseek-ai/dsh-llm-deepseek' -import { httpErrorCode } from '../src/adapter.ts' +import { httpErrorCode, resolveRequestImagePolicy } from '../src/adapter.ts' import { assemble } from './assemble.ts' import { closeMockServers, mockServer, textEvents } from './mock-server.ts' import type { Behavior } from './mock-server.ts' @@ -42,21 +47,264 @@ async function harness(baseURL: string, config: object = {}) { vi.stubEnv('DEEPSEEK_API_KEY', 'test-key') const ctx = new Context() await ctx.plugin(LlmRuntime) + await ctx.plugin(DeepSeekLlmApiExtensionRegistry) await ctx.plugin(LlmDeepSeek, { baseURL, ...config }) return ctx } /** Direct adapter over the plugin's real resolve step, with a static key. */ -function adapterOf(config: Partial & { apiKey?: string } = {}): DeepSeekAdapter { +function noExtensions(): Promise { + return Promise.resolve({ fields: {}, accept: () => Promise.resolve() }) +} + +/** Direct adapter over the plugin's real resolve step, with a static key. */ +function adapterOf( + config: Partial & { apiKey?: string } = {}, + attachments?: AttachmentStore, + files?: LlmDeepSeek.DeepSeekFileStore, +): DeepSeekAdapter { const { apiKey, ...rest } = config return new DeepSeekAdapter({ options: () => resolveAdapterOptions(rest), resolveApiKey: () => Promise.resolve(apiKey ?? 'k'), resolveUserId: () => TEST_USER_ID, + resolveAttachments: () => attachments, + ...files === undefined ? {} : { resolveFiles: () => files }, + prepareExtensions: noExtensions, }) } +async function drain(stream: AsyncIterable): Promise { + for await (const _chunk of stream) { /* drain */ } +} + +const imageRef: ImageAttachmentRef = { + attachmentId: AttachmentId(`sha256:${'a'.repeat(64)}`), + mediaType: 'image/png', + bytes: 3, + width: 1, + height: 1, +} + +function requestImage(ref = imageRef): RequestImageAttachment { + return { + variantId: ImageVariantId(`sha256:${'b'.repeat(64)}`), + attachment: ref, + data: Uint8Array.of(1, 2, 3), + mediaType: 'image/png', + bytes: 3, + width: 1, + height: 1, + depth: 'uchar', + space: 'srgb', + hasAlpha: true, + } +} + +function attachmentStoreOf( + project: (ref: ImageAttachmentRef, policy: unknown, signal?: AbortSignal) => Promise, +): { + store: AttachmentStore + readImageRequest: ReturnType> +} { + const readImageRequest = vi.fn(project) + return { + store: { readImageRequest } as unknown as AttachmentStore, + readImageRequest, + } +} + +function fileStoreOf( + implementation: (...args: Parameters) => ReturnType, +) { + const ensureUploaded = vi.fn(implementation) + const invalidate = vi.fn(() => Promise.resolve()) + return { + store: { ensureUploaded, invalidate } as unknown as LlmDeepSeek.DeepSeekFileStore, + ensureUploaded, + invalidate, + } +} + +function fileReference(fileId: string): Awaited> { + return { + record: { fileId: LlmDeepSeek.DeepSeekFileId(fileId) }, + uploaded: true, + } as Awaited> +} + +function successfulSseResponse(): Response { + return new Response(textEvents.map(event => `data: ${event}\n\n`).join(''), { + status: 200, + headers: { 'content-type': 'text/event-stream' }, + }) +} + +describe('request image policy', () => { + it.each([ + [ + { id: 'default' }, + { maxPixels: 640_000, maxBytes: 1024 * 1024 }, + ], + [ + { id: 'low', imageDetail: 'low' as const }, + { maxPixels: 512 * 512, maxBytes: 1024 * 1024 }, + ], + [ + { id: 'custom', imagePixelBudget: 320_000, imageMaxBytes: 512_000 }, + { maxPixels: 320_000, maxBytes: 512_000 }, + ], + ])('resolves route-owned defaults and overrides for %s', (model, expected) => { + expect(resolveRequestImagePolicy(model)).toEqual(expected) + }) +}) + describe('DeepSeekAdapter against a mock server', () => { + it('merges prepared extension fields and accepts them once after HTTP 2xx', async () => { + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const accept = vi.fn() + const prepareExtensions = vi.fn(async () => ({ + fields: { dsh_test: { version: 1 } }, + accept: async () => { accept() }, + })) + const adapter = new DeepSeekAdapter({ + options: () => resolveAdapterOptions({ baseURL: server.url }), + resolveApiKey: () => Promise.resolve('k'), + resolveUserId: () => TEST_USER_ID, + prepareExtensions: prepareExtensions as never, + }) + + await drain(adapter.stream({ provider: 'deepseek-official', model: 'm', messages: [], sessionId: SessionId('s') })) + expect(server.requests[0]).toMatchObject({ dsh_test: { version: 1 } }) + expect(prepareExtensions).toHaveBeenCalledWith(expect.objectContaining({ sessionId: 's' })) + expect(accept).toHaveBeenCalledOnce() + }) + + it('fails before fetch on extension preparation or base-field collision', async () => { + const server = await mockServer([]) + const base = { + options: () => resolveAdapterOptions({ baseURL: server.url }), + resolveApiKey: () => Promise.resolve('k'), + resolveUserId: () => TEST_USER_ID, + } + const failed = new DeepSeekAdapter({ + ...base, + prepareExtensions: () => Promise.reject(new Error('metadata unavailable')), + }) + await expect(drain(failed.stream({ provider: 'deepseek-official', model: 'm', messages: [] }))) + .rejects.toMatchObject({ code: 'REQUEST_EXTENSION' }) + + const collision = new DeepSeekAdapter({ + ...base, + prepareExtensions: (() => Promise.resolve({ fields: { model: 'replacement' }, accept: () => Promise.resolve() })) as never, + }) + await expect(drain(collision.stream({ provider: 'deepseek-official', model: 'm', messages: [] }))) + .rejects.toMatchObject({ code: 'REQUEST_EXTENSION' }) + expect(server.requests).toHaveLength(0) + }) + + it('passes cancellation into extension preparation and aborts before fetch', async () => { + const server = await mockServer([]) + const controller = new AbortController() + const started = Promise.withResolvers() + let signalSeen: AbortSignal | undefined + const adapter = new DeepSeekAdapter({ + options: () => resolveAdapterOptions({ baseURL: server.url }), + resolveApiKey: () => Promise.resolve('k'), + resolveUserId: () => TEST_USER_ID, + prepareExtensions: ((request: { signal: AbortSignal }) => { + signalSeen = request.signal + started.resolve(undefined) + if (request.signal === undefined) return new Promise(() => {}) + return new Promise((_resolve, reject) => { + request.signal.addEventListener('abort', () => { + const reason: unknown = request.signal.reason + reject(reason instanceof Error ? reason : new Error('extension preparation aborted', { cause: reason })) + }, { once: true }) + }) + }) as never, + }) + + const pending = drain(adapter.stream({ + provider: 'deepseek-official', + model: 'm', + messages: [], + signal: controller.signal, + })) + await started.promise + expect(signalSeen).toBeInstanceOf(AbortSignal) + controller.abort() + await expect(pending).rejects.toMatchObject({ code: 'ABORTED' }) + expect(server.requests).toHaveLength(0) + }) + + it('passes cancellation through an outstanding fetch', async () => { + const controller = new AbortController() + const started = Promise.withResolvers() + const fetch = vi.spyOn(globalThis, 'fetch').mockImplementation((_input, init) => { + const signal = init?.signal + return new Promise((_resolve, reject) => { + started.resolve(undefined) + signal?.addEventListener('abort', () => { + const reason: unknown = signal.reason + reject(reason instanceof Error ? reason : new Error('fetch aborted', { cause: reason })) + }, { once: true }) + }) + }) + try { + const adapter = adapterOf({ baseURL: 'https://provider.invalid' }) + const pending = drain(adapter.stream({ + provider: 'deepseek-official', + model: 'm', + messages: [], + signal: controller.signal, + })) + await started.promise + controller.abort() + await expect(pending).rejects.toMatchObject({ code: 'ABORTED' }) + } finally { + fetch.mockRestore() + } + }) + + it('does not accept extensions on non-2xx and does accept before a later stream failure', async () => { + const server = await mockServer([ + { kind: 'http-error', status: 500, body: '{}' }, + { kind: 'close-early', events: ['{"choices":[{"delta":{"content":"partial"}}]}'] }, + ]) + const accept = vi.fn() + const adapter = new DeepSeekAdapter({ + options: () => resolveAdapterOptions({ baseURL: server.url }), + resolveApiKey: () => Promise.resolve('k'), + resolveUserId: () => TEST_USER_ID, + prepareExtensions: () => Promise.resolve({ fields: { dsh_test: 1 }, accept: async () => { accept() } }) as never, + }) + const request = { provider: 'deepseek-official', model: 'm', messages: [] } + + await expect(drain(adapter.stream(request))).rejects.toMatchObject({ code: 'SERVER' }) + expect(accept).not.toHaveBeenCalled() + await expect(drain(adapter.stream(request))).rejects.toBeDefined() + expect(accept).toHaveBeenCalledOnce() + }) + + it('reports a post-2xx extension acceptance failure without relabelling it as transport', async () => { + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const failure = new Error('watermark append failed') + const adapter = new DeepSeekAdapter({ + options: () => resolveAdapterOptions({ baseURL: server.url }), + resolveApiKey: () => Promise.resolve('k'), + resolveUserId: () => TEST_USER_ID, + prepareExtensions: () => Promise.resolve({ + fields: { dsh_test: 1 }, + accept: () => Promise.reject(failure), + }) as never, + }) + + await expect(drain(adapter.stream({ provider: 'deepseek-official', model: 'm', messages: [] }))) + .rejects.toMatchObject({ code: 'REQUEST_EXTENSION', cause: failure }) + expect(server.requests).toHaveLength(1) + }) + it('streams a text generation end to end through the assembler', async () => { const server = await mockServer([{ kind: 'sse', events: textEvents }]) const ctx = await harness(server.url) @@ -90,6 +338,745 @@ describe('DeepSeekAdapter against a mock server', () => { expect(server.headers[0]).not.toHaveProperty('x-deepseek-harness-compact') }) + it('uploads a durable image once and sends only its Files API id to the vision model', async () => { + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const signalSeen: (AbortSignal | undefined)[] = [] + const policies: unknown[] = [] + const attachmentMocks = attachmentStoreOf((ref, policy, signal) => { + signalSeen.push(signal) + policies.push(policy) + return Promise.resolve(requestImage(ref)) + }) + const adapter = adapterOf({ baseURL: server.url }, attachmentMocks.store) + + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'text', text: 'describe ' }, + { type: 'image', attachment: imageRef }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + expect(server.requests[0]).toMatchObject({ + model: 'deepseek-v4-flash-vision-exp', + messages: [{ + role: 'user', + content: [ + { type: 'text', text: 'describe ' }, + { type: 'text', text: expect.stringContaining(`Image ${imageRef.attachmentId}; request image 1x1px.`) as string }, + { type: 'file', file_id: 'file-api-1' }, + ], + }], + }) + expect(server.fileRequests).toEqual([{ + method: 'POST', + path: '/files', + filename: `dsh-${'a'.repeat(16)}-${'b'.repeat(8)}.png`, + bytes: 3, + }]) + expect(signalSeen[0]).toBeInstanceOf(AbortSignal) + expect(policies).toEqual([{ maxPixels: 640_000, maxBytes: 1024 * 1024 }]) + }) + + it('falls back to one all-base64 request when Files API resolution fails', async () => { + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const secondRef = { ...imageRef, attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`) } + const attachments = attachmentStoreOf(ref => Promise.resolve({ + ...requestImage(ref), + variantId: ImageVariantId(`sha256:${(ref.attachmentId === imageRef.attachmentId ? 'b' : 'd').repeat(64)}`), + })).store + const files = fileStoreOf(() => Promise.reject(new LlmError('Files unavailable', 'SERVER'))) + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments, files.store) + + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'image', attachment: imageRef }, + { type: 'image', attachment: secondRef }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + const body = server.requests[0] as { messages: Array<{ content: unknown }> } + expect(JSON.stringify(body.messages[0]?.content).match(/"type":"image_url"/g)).toHaveLength(2) + expect(JSON.stringify(body)).not.toContain('file_id') + expect(files.ensureUploaded).toHaveBeenCalledTimes(1) + }) + + it('reduces base64 fallback history from the configured high watermark to its half-size quantum', async () => { + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const attachments = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))).store + const files = fileStoreOf(() => Promise.reject(new LlmError('Files unavailable', 'SERVER'))) + const adapter = adapterOf({ + baseURL: server.url, + maxInlineRequestImageBytes: 80, + inlineImageOffloadByteQuantum: 40, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments, files.store) + + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: Array.from({ length: 21 }, () => ({ type: 'image' as const, attachment: imageRef })), + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + const body = JSON.stringify(server.requests[0]) + expect(body.match(/older images are omitted first/g)).toHaveLength(11) + expect(body.match(/"type":"image_url"/g)).toHaveLength(10) + }) + + it('discards partially resolved file ids and falls back with every retained image inline', async () => { + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const secondRef = { ...imageRef, attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`) } + const attachments = attachmentStoreOf(ref => Promise.resolve({ + ...requestImage(ref), + variantId: ImageVariantId(`sha256:${(ref.attachmentId === imageRef.attachmentId ? 'b' : 'd').repeat(64)}`), + })).store + const files = fileStoreOf(() => Promise.reject(new Error('unused'))) + files.ensureUploaded + .mockResolvedValueOnce(fileReference('file-api-partial')) + .mockRejectedValueOnce(new LlmError('Files unavailable', 'TRANSPORT')) + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments, files.store) + + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'image', attachment: imageRef }, + { type: 'image', attachment: secondRef }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + const body = server.requests[0] as { messages: Array<{ content: unknown }> } + expect(JSON.stringify(body.messages[0]?.content).match(/"type":"image_url"/g)).toHaveLength(2) + expect(JSON.stringify(body)).not.toContain('file-api-partial') + }) + + it('falls back after the configured Files API deadline without aborting chat', async () => { + vi.useFakeTimers() + const started = Promise.withResolvers() + const files = fileStoreOf((_version, _connection, _policy, signal) => new Promise((_resolve, reject) => { + started.resolve(undefined) + signal?.addEventListener('abort', () => { + const reason: unknown = signal.reason + reject(reason instanceof Error ? reason : new Error('files operation aborted')) + }, { once: true }) + })) + const attachments = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))).store + const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue(successfulSseResponse()) + const adapter = adapterOf({ + baseURL: 'https://deepseek.invalid', + filesApiTimeoutMs: 50, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments, files.store) + + const pending = drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: imageRef }], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + await started.promise + await vi.advanceTimersByTimeAsync(50) + await pending + + expect(fetchSpy).toHaveBeenCalledTimes(1) + expect(fetchSpy.mock.calls[0]?.[1]?.body).toEqual(expect.stringContaining('image_url')) + fetchSpy.mockRestore() + }) + + it('does not turn caller cancellation during file resolution into base64 fallback', async () => { + const started = Promise.withResolvers() + const files = fileStoreOf((_version, _connection, _policy, signal) => new Promise((_resolve, reject) => { + started.resolve(undefined) + signal?.addEventListener('abort', () => { reject(new Error('cancelled')) }, { once: true }) + })) + const attachments = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))).store + const fetchSpy = vi.spyOn(globalThis, 'fetch') + const controller = new AbortController() + const adapter = adapterOf({ + baseURL: 'https://deepseek.invalid', + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments, files.store) + + const pending = drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + signal: controller.signal, + messages: [createUserMessage({ + content: [{ type: 'image', attachment: imageRef }], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + await started.promise + controller.abort() + + await expect(pending).rejects.toMatchObject({ code: 'ABORTED' }) + expect(fetchSpy).not.toHaveBeenCalled() + fetchSpy.mockRestore() + }) + + it('does not retry a generic chat failure through base64 fallback', async () => { + const server = await mockServer([{ + kind: 'http-error', + status: 503, + body: JSON.stringify({ error: { message: 'chat unavailable' } }), + }]) + const attachments = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))).store + const files = fileStoreOf(() => Promise.resolve(fileReference('file-api-ready'))) + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments, files.store) + + await expect(drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: imageRef }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }))).rejects.toMatchObject({ code: 'SERVER', message: 'chat unavailable' }) + + expect(server.requests).toHaveLength(1) + expect(JSON.stringify(server.requests[0])).toContain('file-api-ready') + expect(JSON.stringify(server.requests[0])).not.toContain('image_url') + }) + + it('does not prepare an old image removed by request offload', async () => { + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const old = { ...imageRef, attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`), bytes: 3 } + const recent = { ...imageRef, attachmentId: AttachmentId(`sha256:${'d'.repeat(64)}`), bytes: 3 } + const attachmentMocks = attachmentStoreOf((ref) => { + if (ref.attachmentId === old.attachmentId) throw new Error('old image must not be read') + return Promise.resolve(requestImage(ref)) + }) + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + maxRequestFilesBytes: 4, + imageOffloadByteQuantum: 2, + }, attachmentMocks.store) + + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'image', attachment: old }, + { type: 'image', attachment: recent }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + expect(attachmentMocks.readImageRequest).toHaveBeenCalledWith( + recent, + { maxPixels: 640_000, maxBytes: 1024 * 1024 }, + expect.any(AbortSignal), + ) + const body = server.requests[0] as { messages: unknown[] } + expect(body.messages[0]).toMatchObject({ + role: 'user', + content: [ + { type: 'text', text: expect.stringContaining('older images are omitted first') as string }, + { type: 'text', text: expect.stringContaining(String(recent.attachmentId)) as string }, + { type: 'file', file_id: 'file-api-1' }, + ], + }) + }) + + it('projects nested tool-result images with route-owned request budgets', async () => { + const server = await mockServer([ + { kind: 'sse', events: textEvents }, + { kind: 'sse', events: textEvents }, + ]) + const attachmentMocks = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))) + const adapter = adapterOf({ + baseURL: server.url, + models: [ + { + id: 'vision-low', + inputModalities: ['text', 'image'], + imageDetail: 'low', + imageMaxBytes: 512_000, + }, + { + id: 'vision-custom', + inputModalities: ['text', 'image'], + imagePixelBudget: 320_000, + }, + ], + }, attachmentMocks.store) + const nested = createUserMessage({ + content: [{ + type: 'tool-result', + toolCallId: CallId('image-result'), + content: [{ type: 'image', attachment: imageRef }], + }], + source: { kind: 'plugin', plugin: 'test' }, + }) + + await drain(adapter.stream({ provider: 'deepseek-official', model: 'vision-low', messages: [nested] })) + await drain(adapter.stream({ provider: 'deepseek-official', model: 'vision-custom', messages: [nested] })) + + expect(attachmentMocks.readImageRequest).toHaveBeenNthCalledWith( + 1, + imageRef, + { maxPixels: 512 * 512, maxBytes: 512_000 }, + expect.any(AbortSignal), + ) + expect(attachmentMocks.readImageRequest).toHaveBeenNthCalledWith( + 2, + imageRef, + { maxPixels: 320_000, maxBytes: 1024 * 1024 }, + expect.any(AbortSignal), + ) + }) + + it('reuses the exact request version between agent and compaction calls', async () => { + const server = await mockServer([ + { kind: 'sse', events: textEvents }, + { kind: 'sse', events: textEvents }, + ]) + const attachments = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))).store + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments) + const messages = [createUserMessage({ + content: [{ type: 'image' as const, attachment: imageRef }], + source: { kind: 'plugin' as const, plugin: 'test' }, + })] + + await drain(adapter.stream({ provider: 'deepseek-official', model: 'deepseek-v4-flash-vision-exp', messages })) + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages, + purpose: 'compaction', + })) + + expect(server.fileRequests.filter(request => request.method === 'POST')).toHaveLength(1) + expect(server.requests).toMatchObject([ + { messages: [{ content: [expect.objectContaining({ type: 'text' }), { file_id: 'file-api-1' }] }] }, + { messages: [{ content: [expect.objectContaining({ type: 'text' }), { file_id: 'file-api-1' }] }] }, + ]) + expect(server.headers[1]?.['x-deepseek-harness-compact']).toBe('1') + }) + + it('explains a provider rejection of a normalized image and retains the raw response as cause', async () => { + const raw = JSON.stringify({ error: { message: 'unsupported image payload for file-api-1' } }) + const server = await mockServer([{ kind: 'http-error', status: 400, body: raw }]) + const attachments = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))).store + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments) + + let failure: unknown + try { + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: imageRef }], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + } catch (error: unknown) { + failure = error + } + expect(failure).toMatchObject({ + code: 'INVALID_REQUEST', + message: expect.stringContaining( + `normalized image "${imageRef.attachmentId}" at message 1, image 1`, + ) as string, + cause: { message: raw }, + }) + expect((failure as Error).message).toContain('image/png, 8-bit sRGBA, 1x1') + expect((failure as Error).message).toContain('unsupported image payload for file-api-1') + expect((failure as Error).message).not.toBe(raw) + }) + + it('identifies the sole image when a normalized rejection omits its file id', async () => { + const raw = JSON.stringify({ error: { message: 'unsupported image payload' } }) + const server = await mockServer([{ kind: 'http-error', status: 400, body: raw }]) + const attachments = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))).store + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments) + + await expect(drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: imageRef }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }))).rejects.toMatchObject({ + message: expect.stringContaining(`normalized image "${imageRef.attachmentId}"`) as string, + }) + }) + + it('lists every candidate when a normalized multi-image rejection names no file id', async () => { + const secondRef: ImageAttachmentRef = { + ...imageRef, + attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`), + } + const raw = JSON.stringify({ error: { message: 'unsupported image payload' } }) + const server = await mockServer([{ kind: 'http-error', status: 400, body: raw }]) + const attachments = attachmentStoreOf((ref) => { + const first = ref.attachmentId === imageRef.attachmentId + return Promise.resolve({ + ...requestImage(ref), + variantId: ImageVariantId(`sha256:${(first ? 'b' : 'd').repeat(64)}`), + attachment: first ? { ...ref, name: 'diagram.png' } : ref, + hasAlpha: false, + }) + }).store + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments) + + await expect(drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'image', attachment: imageRef }, + { type: 'image', attachment: secondRef }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + }))).rejects.toMatchObject({ + code: 'INVALID_REQUEST', + message: expect.stringContaining('Candidate images: "diagram.png"') as string, + cause: { message: raw }, + }) + }) + + it.each([ + 'file-api-1 expired', + 'file_id file-api-10 invalid; file_id file-api-1 expired', + 'file_id file-api-1 expired', + 'file_not_found', + 'file_id file-api-1 deleted', + 'invalid file_id file-api-1', + ])('reuploads once when chat rejects a Files API reference as %s', async (providerMessage) => { + const server = await mockServer([ + { + kind: 'http-error', + status: 400, + body: JSON.stringify({ error: { message: providerMessage } }), + }, + { kind: 'sse', events: textEvents }, + ]) + const attachmentMocks = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))) + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachmentMocks.store) + const options = { + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image' as const, attachment: imageRef }], + source: { kind: 'plugin' as const, plugin: 'test' }, + })], + } + + await drain(adapter.stream(options)) + + expect(server.fileRequests.filter(request => request.method === 'POST')).toHaveLength(2) + expect(server.requests).toMatchObject([ + { messages: [{ content: [expect.objectContaining({ type: 'text' }), { file_id: 'file-api-1' }] }] }, + { messages: [{ content: [expect.objectContaining({ type: 'text' }), { file_id: 'file-api-2' }] }] }, + ]) + expect(attachmentMocks.readImageRequest).toHaveBeenCalledTimes(1) + }) + + it('uses inline fallback when stale-id recovery cannot resolve a replacement file', async () => { + const server = await mockServer([ + { + kind: 'http-error', + status: 400, + body: JSON.stringify({ error: { message: 'file_id file-api-stale expired' } }), + }, + { kind: 'sse', events: textEvents }, + ]) + const attachments = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))).store + const files = fileStoreOf(() => Promise.reject(new Error('unused'))) + files.ensureUploaded + .mockResolvedValueOnce(fileReference('file-api-stale')) + .mockRejectedValueOnce(new LlmError('Files unavailable', 'SERVER')) + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments, files.store) + + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: imageRef }], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + expect(files.invalidate).toHaveBeenCalledTimes(1) + expect(server.requests).toHaveLength(2) + expect(JSON.stringify(server.requests[0])).toContain('file-api-stale') + expect(JSON.stringify(server.requests[1])).toContain('image_url') + }) + + it('invalidates only the identified mapping when a multi-image request names one stale file id', async () => { + const secondRef: ImageAttachmentRef = { + ...imageRef, + attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`), + } + const server = await mockServer([ + { + kind: 'http-error', + status: 400, + body: JSON.stringify({ error: { message: 'file_id file-api-2 expired' } }), + }, + { kind: 'sse', events: textEvents }, + ]) + const attachments = attachmentStoreOf(ref => Promise.resolve({ + ...requestImage(ref), + variantId: ImageVariantId(`sha256:${(ref.attachmentId === imageRef.attachmentId ? 'b' : 'd').repeat(64)}`), + })).store + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments) + + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'image', attachment: imageRef }, + { type: 'image', attachment: secondRef }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + expect(server.fileRequests.filter(request => request.method === 'POST')).toHaveLength(3) + const retries = server.requests as Array<{ messages: Array<{ content: Array<{ type: string; file_id?: string }> }> }> + expect(retries[0]?.messages[0]?.content.filter(block => block.type === 'file')) + .toEqual([{ type: 'file', file_id: 'file-api-1' }, { type: 'file', file_id: 'file-api-2' }]) + expect(retries[1]?.messages[0]?.content.filter(block => block.type === 'file')) + .toEqual([{ type: 'file', file_id: 'file-api-1' }, { type: 'file', file_id: 'file-api-3' }]) + }) + + it('invalidates every listed missing file id and preserves unlisted mappings', async () => { + const secondRef: ImageAttachmentRef = { + ...imageRef, + attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`), + } + const thirdRef: ImageAttachmentRef = { + ...imageRef, + attachmentId: AttachmentId(`sha256:${'e'.repeat(64)}`), + } + const server = await mockServer([ + { + kind: 'http-error', + status: 400, + body: JSON.stringify({ + error: { + message: 'path.to.object[index]: the following file_ids do not exist or are not created under your account: ' + + 'file-api-1, file-api-3, file-api-unknown', + }, + }), + }, + { kind: 'sse', events: textEvents }, + ]) + const attachments = attachmentStoreOf((ref) => { + let digest = 'f' + if (ref.attachmentId === imageRef.attachmentId) digest = 'b' + else if (ref.attachmentId === secondRef.attachmentId) digest = 'd' + return Promise.resolve({ + ...requestImage(ref), + variantId: ImageVariantId(`sha256:${digest.repeat(64)}`), + }) + }).store + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments) + + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'image', attachment: imageRef }, + { type: 'image', attachment: secondRef }, + { type: 'image', attachment: thirdRef }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + expect(server.fileRequests.filter(request => request.method === 'POST')).toHaveLength(5) + const retries = server.requests as Array<{ messages: Array<{ content: Array<{ type: string; file_id?: string }> }> }> + expect(retries[0]?.messages[0]?.content.filter(block => block.type === 'file')) + .toEqual([ + { type: 'file', file_id: 'file-api-1' }, + { type: 'file', file_id: 'file-api-2' }, + { type: 'file', file_id: 'file-api-3' }, + ]) + expect(retries[1]?.messages[0]?.content.filter(block => block.type === 'file')) + .toEqual([ + { type: 'file', file_id: 'file-api-4' }, + { type: 'file', file_id: 'file-api-2' }, + { type: 'file', file_id: 'file-api-5' }, + ]) + }) + + it('invalidates every used mapping when a stale-file response does not identify one file id', async () => { + const secondRef: ImageAttachmentRef = { + ...imageRef, + attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`), + } + const server = await mockServer([ + { + kind: 'http-error', + status: 400, + body: JSON.stringify({ error: { message: 'file reference expired' } }), + }, + { kind: 'sse', events: textEvents }, + ]) + const attachments = attachmentStoreOf(ref => Promise.resolve({ + ...requestImage(ref), + variantId: ImageVariantId(`sha256:${(ref.attachmentId === imageRef.attachmentId ? 'b' : 'd').repeat(64)}`), + })).store + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments) + + await drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'image', attachment: imageRef }, + { type: 'image', attachment: secondRef }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + expect(server.fileRequests.filter(request => request.method === 'POST')).toHaveLength(4) + const retries = server.requests as Array<{ messages: Array<{ content: Array<{ type: string; file_id?: string }> }> }> + expect(retries[1]?.messages[0]?.content.filter(block => block.type === 'file')) + .toEqual([{ type: 'file', file_id: 'file-api-3' }, { type: 'file', file_id: 'file-api-4' }]) + }) + + it('returns the second stale-file rejection without a third chat attempt', async () => { + const stale = JSON.stringify({ error: { message: 'file_id file-api-1 expired' } }) + const server = await mockServer([ + { kind: 'http-error', status: 400, body: stale }, + { kind: 'http-error', status: 400, body: stale }, + ]) + const attachments = attachmentStoreOf(ref => Promise.resolve(requestImage(ref))).store + const adapter = adapterOf({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }, attachments) + + await expect(drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: imageRef }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }))).rejects.toMatchObject({ code: 'INVALID_REQUEST', message: 'file_id file-api-1 expired' }) + expect(server.requests).toHaveLength(2) + expect(server.fileRequests.filter(request => request.method === 'POST')).toHaveLength(2) + }) + + it.each(['deepseek-v4-flash', 'unlisted-pass-through'])( + 'rejects image input for text-only model %s before credentials, attachments, or fetch', + async (model) => { + const server = await mockServer([]) + const resolveApiKey = vi.fn(() => Promise.resolve('k')) + const resolveAttachments = vi.fn(() => ({}) as AttachmentStore) + const adapter = new DeepSeekAdapter({ + options: () => resolveAdapterOptions({ baseURL: server.url }), + resolveApiKey, + resolveUserId: () => TEST_USER_ID, + resolveAttachments, + prepareExtensions: noExtensions, + }) + + await expect(drain(adapter.stream({ + provider: 'deepseek-official', + model, + messages: [createUserMessage({ + content: [{ type: 'image', attachment: imageRef }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }))).rejects.toMatchObject({ code: 'UNSUPPORTED_CONTENT' }) + expect(resolveApiKey).not.toHaveBeenCalled() + expect(resolveAttachments).not.toHaveBeenCalled() + expect(server.requests).toHaveLength(0) + }, + ) + + it('rejects vision input without an attachment provider before credentials or fetch', async () => { + const server = await mockServer([]) + const resolveApiKey = vi.fn(() => Promise.resolve('k')) + const adapter = new DeepSeekAdapter({ + options: () => resolveAdapterOptions({ + baseURL: server.url, + models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }], + }), + resolveApiKey, + resolveUserId: () => TEST_USER_ID, + prepareExtensions: noExtensions, + }) + + await expect(drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: imageRef }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }))).rejects.toMatchObject({ code: 'UNSUPPORTED_CONTENT' }) + expect(resolveApiKey).not.toHaveBeenCalled() + expect(server.requests).toHaveLength(0) + }) + it('streams raw chunks through ctx.llm.stream', async () => { const server = await mockServer([{ kind: 'sse', events: textEvents, delayMs: 2 }]) const ctx = await harness(server.url) @@ -141,13 +1128,13 @@ describe('DeepSeekAdapter against a mock server', () => { expect(server.headers[0]?.['x-deepseek-harness-compact']).toBe('1') }) - it('switches dynamically from the configured high default through off to max', async () => { + it('switches dynamically from the configured low default through off to max', async () => { const server = await mockServer([ { kind: 'sse', events: textEvents }, { kind: 'sse', events: textEvents }, { kind: 'sse', events: textEvents }, ]) - const ctx = await harness(server.url, { thinking: 'enabled', reasoningEffort: 'high' }) + const ctx = await harness(server.url, { thinking: 'enabled', reasoningEffort: 'low' }) await assemble(ctx,{ model: 'deepseek-v4-flash', @@ -174,7 +1161,7 @@ describe('DeepSeekAdapter against a mock server', () => { }) expect(server.requests[0]).toMatchObject({ thinking: { type: 'enabled' }, - reasoning_effort: 'high', + reasoning_effort: 'low', }) expect(server.requests[1]).toMatchObject({ thinking: { type: 'disabled' }, @@ -287,6 +1274,20 @@ describe('DeepSeekAdapter against a mock server', () => { }) }) + it('uses the HTTP status as the cause when an error response has no body', async () => { + const server = await mockServer([{ kind: 'http-error', status: 500, body: '' }]) + const adapter = adapterOf({ baseURL: server.url }) + + await expect(drain(adapter.stream({ + provider: 'deepseek-official', + model: 'deepseek-v4-flash', + messages: [], + }))).rejects.toMatchObject({ + code: 'SERVER', + cause: { message: 'DeepSeek HTTP 500' }, + }) + }) + it('classifies an HTTP context-window failure with the canonical code', async () => { const server = await mockServer([{ kind: 'http-error', @@ -386,7 +1387,7 @@ describe('DeepSeekAdapter against a mock server', () => { .toBe(CONTEXT_WINDOW_EXCEEDED_CODE) expect(httpErrorCode(400, { message: 'invalid input: temperature exceeds maximum allowed value' })) .toBe('INVALID_REQUEST') - expect(httpErrorCode(413, { code: 'context_length_exceeded' })).toBe('HTTP_413') + expect(httpErrorCode(413, { code: 'context_length_exceeded' })).toBe('INVALID_REQUEST') }) it('distinguishes terminal quota exhaustion from transient HTTP 429 throttling', () => { @@ -550,7 +1551,10 @@ describe('DeepSeekAdapter against a mock server', () => { }) return Promise.resolve(new Response(body, { status: 200 })) }) - const adapter = adapterOf({ baseURL: 'https://example.invalid', streamIdleTimeoutMs: 100 }) + const adapter = adapterOf({ + baseURL: 'https://example.invalid', + streamIdleTimeoutMs: 100, + }) try { const drain = (async () => { for await (const _chunk of adapter.stream({ provider: 'deepseek-official', model: 'm', messages: [] })) { /* drain */ } @@ -581,7 +1585,10 @@ describe('DeepSeekAdapter against a mock server', () => { }) return Promise.resolve(new Response(body, { status: 200 })) }) - const adapter = adapterOf({ baseURL: 'https://example.invalid', streamIdleTimeoutMs: 100 }) + const adapter = adapterOf({ + baseURL: 'https://example.invalid', + streamIdleTimeoutMs: 100, + }) try { const chunks: string[] = [] const drain = (async () => { @@ -660,6 +1667,7 @@ describe('plugin registration and config', () => { await expect(ctx.llm.listModels('deepseek-official')).resolves.toEqual([ { provider: 'deepseek-official', id: 'deepseek-v4-flash', name: 'DeepSeek-V4-Flash', inputModalities: ['text'] }, { provider: 'deepseek-official', id: 'deepseek-v4-pro', name: 'DeepSeek-V4-Pro', inputModalities: ['text'] }, + { provider: 'deepseek-official', id: 'deepseek-v4-flash-vision-exp', name: 'DeepSeek-V4-Flash-Vision-Exp', inputModalities: ['text', 'image'] }, ]) await expect(ctx.llm.resolveModelInfo('deepseek-official', 'deepseek-v4-flash')) .resolves.toMatchObject({ @@ -671,15 +1679,25 @@ describe('plugin registration and config', () => { reasoning: { efforts: [ { id: ReasoningEffortId('off'), name: 'Off' }, + { id: ReasoningEffortId('low'), name: 'Low' }, { id: ReasoningEffortId('high'), name: 'High' }, { id: ReasoningEffortId('max'), name: 'Max' }, ], defaultEffort: ReasoningEffortId('high'), }, }) + await expect(ctx.llm.resolveModelInfo('deepseek-official', 'deepseek-v4-flash-vision-exp')) + .resolves.toMatchObject({ + provider: 'deepseek-official', + id: 'deepseek-v4-flash-vision-exp', + name: 'DeepSeek-V4-Flash-Vision-Exp', + inputModalities: ['text', 'image'], + context: { contextWindow: 1_000_000 }, + defaultMaxTokens: 256_000, + }) }) - it.each(['off', 'max'] as const)('uses the configured %s reasoning default', async (effort) => { + it.each(['off', 'low', 'max'] as const)('uses the configured %s reasoning default', async (effort) => { const ctx = new Context() await ctx.plugin(LlmRuntime) await ctx.plugin(LlmDeepSeek, { @@ -691,6 +1709,7 @@ describe('plugin registration and config', () => { reasoning: { efforts: [ { id: ReasoningEffortId('off'), name: 'Off' }, + { id: ReasoningEffortId('low'), name: 'Low' }, { id: ReasoningEffortId('high'), name: 'High' }, { id: ReasoningEffortId('max'), name: 'Max' }, ], @@ -716,7 +1735,7 @@ describe('plugin registration and config', () => { }) }) - it.each(['high', 'max'] as const)( + it.each(['low', 'high', 'max'] as const)( 'rejects configured reasoning effort %s when thinking is disabled', async (reasoningEffort) => { const ctx = new Context() @@ -730,7 +1749,7 @@ describe('plugin registration and config', () => { }, ) - it.each(['high', 'max'] as const)( + it.each(['low', 'high', 'max'] as const)( 'rejects disabled-thinking effort %s at the resolver boundary', (reasoningEffort) => { expect(() => resolveAdapterOptions({ thinking: 'disabled', reasoningEffort })) @@ -755,9 +1774,26 @@ describe('plugin registration and config', () => { await expect(ctx.llm.listModels('deepseek-official')).resolves.toEqual([ { provider: 'deepseek-official', id: 'deepseek-v4-flash', name: 'DeepSeek-V4-Flash', inputModalities: ['text'] }, { provider: 'deepseek-official', id: 'deepseek-v4-pro', name: 'DeepSeek-V4-Pro', inputModalities: ['text'] }, + { provider: 'deepseek-official', id: 'deepseek-v4-flash-vision-exp', name: 'DeepSeek-V4-Flash-Vision-Exp', inputModalities: ['text', 'image'] }, ]) }) + it('defaults an adapter-supplied catalog entry to text input', async () => { + const connection = resolveAdapterOptions({ models: [] }) + const adapter = new DeepSeekAdapter({ + options: () => ({ ...connection, models: [{ id: 'adapter-model' }] }), + resolveApiKey: () => Promise.resolve('k'), + resolveUserId: () => TEST_USER_ID, + prepareExtensions: noExtensions, + }) + await expect(adapter.listModels('deepseek-official')).resolves.toEqual([{ + provider: 'deepseek-official', + id: 'adapter-model', + name: 'adapter-model', + inputModalities: ['text'], + }]) + }) + it('advertises configured models without restricting arbitrary request ids', async () => { const ctx = new Context() await ctx.plugin(LlmRuntime) @@ -770,12 +1806,13 @@ describe('plugin registration and config', () => { name: 'Private Reasoner', description: 'Higher reasoning budget', contextWindow: 64_000, + inputModalities: ['text', 'image'], }, ], }) await expect(ctx.llm.listModels('deepseek-official')).resolves.toEqual([ { provider: 'deepseek-official', id: 'private-fast', name: 'private-fast', inputModalities: ['text'] }, - { provider: 'deepseek-official', id: 'private-reasoner', name: 'Private Reasoner', description: 'Higher reasoning budget', inputModalities: ['text'] }, + { provider: 'deepseek-official', id: 'private-reasoner', name: 'Private Reasoner', description: 'Higher reasoning budget', inputModalities: ['text', 'image'] }, ]) await expect(ctx.llm.resolveModelInfo('deepseek-official', 'private-fast')) .resolves.toMatchObject({ context: { contextWindow: 32_000 } }) @@ -783,6 +1820,7 @@ describe('plugin registration and config', () => { .resolves.toMatchObject({ name: 'Private Reasoner', description: 'Higher reasoning budget', + inputModalities: ['text', 'image'], }) await expect(ctx.llm.resolveModelInfo('deepseek-official', 'arbitrary-unlisted')) .resolves.toMatchObject({ @@ -821,13 +1859,21 @@ describe('plugin registration and config', () => { await expect(ctx.llm.listModels('deepseek-official')).resolves.toEqual([]) }) - it.each([ + const invalidModels: Array<[LlmDeepSeek.DeepSeekCatalogModel[], RegExp]> = [ [[{ id: '' }], /ids must be non-empty/], [[{ id: 'm', name: '' }], /empty name/], [[{ id: 'm', contextWindow: 0 }], /contextWindow/], [[{ id: 'm', contextWindow: 1.5 }], /contextWindow/], + [[{ id: 'm', inputModalities: [] }], /inputModalities/], + [[{ id: 'm', inputModalities: ['text', 'text'] }], /inputModalities must not contain duplicates/], + [[{ + id: 'm', + inputModalities: ['audio'] as unknown as NonNullable, + }], /expected "text" \| "image"/], [[{ id: 'm' }, { id: 'm' }], /duplicate catalog model/], - ] as const)('rejects invalid advisory model config', async (models, message) => { + ] + + it.each(invalidModels)('rejects invalid advisory model config', async (models, message) => { const ctx = new Context() await ctx.plugin(LlmRuntime) await expect(ctx.plugin(LlmDeepSeek, { @@ -837,11 +1883,40 @@ describe('plugin registration and config', () => { expect(ctx.llm.listProviders()).toEqual([]) }) + const invalidProgrammaticModalities: Array<[LlmDeepSeek.DeepSeekCatalogModel[], RegExp]> = [ + [[{ id: 'm', inputModalities: [] }], /inputModalities must not be empty/], + [[{ + id: 'm', + inputModalities: ['audio'] as unknown as NonNullable, + }], /inputModalities must contain only "text" and "image"/], + ] + + it.each(invalidProgrammaticModalities)('rejects programmatic modality config that bypasses the schema', (models, message) => { + expect(() => resolveAdapterOptions({ models: [...models] })).toThrow(message) + }) + it.each([0, 1.5])('rejects a per-model output cap of %s', (maxTokens) => { expect(() => resolveAdapterOptions({ models: [{ id: 'bad-cap', maxTokens }] })) .toThrow(/maxTokens must be a positive integer/) }) + it('rejects image request limits on a text-only catalog model', () => { + expect(() => resolveAdapterOptions({ + models: [{ id: 'text-only', inputModalities: ['text'], imagePixelBudget: 1 }], + })).toThrow(/text-only catalog model .* cannot declare image request limits/) + }) + + it.each([ + ['imagePixelBudget', 0, /imagePixelBudget must be a positive safe integer/], + ['imagePixelBudget', Number.MAX_SAFE_INTEGER + 1, /imagePixelBudget must be a positive safe integer/], + ['imageMaxBytes', 0, /imageMaxBytes must be a positive safe integer/], + ['imageMaxBytes', 1.5, /imageMaxBytes must be a positive safe integer/], + ] as const)('rejects per-model %s=%s', (field, value, message) => { + expect(() => resolveAdapterOptions({ + models: [{ id: 'vision', inputModalities: ['image'], [field]: value }], + })).toThrow(message) + }) + it('prefers a model\'s own output cap over the profile default', async () => { // The profile default stays what an unlisted or uncapped model resolves // to, so adding a per-model cap changes one model rather than the route. @@ -901,6 +1976,72 @@ describe('plugin registration and config', () => { }, ) + it('rejects offload quanta larger than their request bounds', () => { + expect(() => resolveAdapterOptions({ + maxRequestFilesBytes: 10, + imageOffloadByteQuantum: 11, + })).toThrow(/imageOffloadByteQuantum must not exceed maxRequestFilesBytes/) + expect(() => resolveAdapterOptions({ + maxInlineRequestImageBytes: 10, + inlineImageOffloadByteQuantum: 11, + })).toThrow(/inlineImageOffloadByteQuantum must not exceed maxInlineRequestImageBytes/) + expect(() => resolveAdapterOptions({ + maxImagesPerRequest: 10, + imageOffloadCountQuantum: 11, + })).toThrow(/imageOffloadCountQuantum must not exceed maxImagesPerRequest/) + }) + + it.each([ + ['maxImagesPerRequest', 0, /maxImagesPerRequest must be a positive safe integer/], + ['maxImagesPerRequest', 1.5, /maxImagesPerRequest must be a positive safe integer/], + ['imageOffloadByteQuantum', 0, /imageOffloadByteQuantum must be a positive safe integer/], + ['imageOffloadByteQuantum', Number.MAX_SAFE_INTEGER + 1, /imageOffloadByteQuantum must be a positive safe integer/], + ['inlineImageOffloadByteQuantum', 0, /inlineImageOffloadByteQuantum must be a positive safe integer/], + ['inlineImageOffloadByteQuantum', Number.MAX_SAFE_INTEGER + 1, /inlineImageOffloadByteQuantum must be a positive safe integer/], + ['imageOffloadCountQuantum', 0, /imageOffloadCountQuantum must be a positive safe integer/], + ['imageOffloadCountQuantum', 1.5, /imageOffloadCountQuantum must be a positive safe integer/], + ['fileExpiresAfterSeconds', 3_599, /fileExpiresAfterSeconds must be an integer from 3600 through 2592000/], + ['fileExpiresAfterSeconds', 2_592_001, /fileExpiresAfterSeconds must be an integer from 3600 through 2592000/], + ['fileRefreshMarginSeconds', -1, /fileRefreshMarginSeconds must be a non-negative integer/], + ['fileRefreshMarginSeconds', 604_800, /fileRefreshMarginSeconds must be a non-negative integer/], + ['fileQuotaCleanupBatch', 0, /fileQuotaCleanupBatch must be an integer from 1 through 1000/], + ['fileQuotaCleanupBatch', 1_001, /fileQuotaCleanupBatch must be an integer from 1 through 1000/], + ] as const)('rejects %s=%s', (field, value, message) => { + expect(() => resolveAdapterOptions({ [field]: value })).toThrow(message) + }) + + it.each([0, 1.5, Number.MAX_SAFE_INTEGER + 1])( + 'rejects invalid request file bound %s', + async (maxRequestFilesBytes) => { + expect(() => resolveAdapterOptions({ maxRequestFilesBytes })) + .toThrow(/maxRequestFilesBytes must be a positive safe integer/) + + const ctx = new Context() + await ctx.plugin(LlmRuntime) + await expect(ctx.plugin(LlmDeepSeek, { + baseURL: 'http://127.0.0.1:1', + maxRequestFilesBytes, + })).rejects.toThrow(/maxRequestFilesBytes/) + expect(ctx.llm.listProviders()).toEqual([]) + }, + ) + + it.each([0, 1.5, Number.MAX_SAFE_INTEGER + 1])( + 'rejects invalid inline request image bound %s', + async (maxInlineRequestImageBytes) => { + expect(() => resolveAdapterOptions({ maxInlineRequestImageBytes })) + .toThrow(/maxInlineRequestImageBytes must be a positive safe integer/) + + const ctx = new Context() + await ctx.plugin(LlmRuntime) + await expect(ctx.plugin(LlmDeepSeek, { + baseURL: 'http://127.0.0.1:1', + maxInlineRequestImageBytes, + })).rejects.toThrow(/maxInlineRequestImageBytes/) + expect(ctx.llm.listProviders()).toEqual([]) + }, + ) + it('falls back to DEEPSEEK_API_KEY and DEEPSEEK_BASE_URL env vars', async () => { vi.stubEnv('DEEPSEEK_API_KEY', 'env-key') vi.stubEnv('DEEPSEEK_BASE_URL', 'http://127.0.0.1:1') @@ -918,7 +2059,7 @@ describe('plugin registration and config', () => { // First-boot onboarding: the route registers so models stay discoverable; // only the request itself needs a key. expect(ctx.llm.listProviders()).toEqual([{ id: 'deepseek-official', name: 'DeepSeek' }]) - await expect(ctx.llm.listModels('deepseek-official')).resolves.toHaveLength(2) + await expect(ctx.llm.listModels('deepseek-official')).resolves.toHaveLength(3) const first = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }) expect(first.finish).toMatchObject({ kind: 'error', failure: { code: 'MISSING_CREDENTIAL' } }) // The guidance leads with the managed credential store. @@ -1006,7 +2147,7 @@ describe('plugin registration and config', () => { expect(adapter).toBeInstanceOf(DeepSeekAdapter) // Direct embedding shares the plugin's one resolve step, so it advertises // the same default catalog instead of a divergent empty one. - await expect(adapter.listModels('deepseek-official')).resolves.toHaveLength(2) + await expect(adapter.listModels('deepseek-official')).resolves.toHaveLength(3) }) it('resolves connection facts and the credential exactly once per stream call', async () => { @@ -1014,7 +2155,7 @@ describe('plugin registration and config', () => { const options = vi.fn(() => resolveAdapterOptions({ baseURL: server.url })) const resolveApiKey = vi.fn(() => Promise.resolve('per-request-key')) const resolveUserId = vi.fn(() => TEST_USER_ID) - const adapter = new DeepSeekAdapter({ options, resolveApiKey, resolveUserId }) + const adapter = new DeepSeekAdapter({ options, resolveApiKey, resolveUserId, prepareExtensions: noExtensions }) for await (const _chunk of adapter.stream({ provider: 'deepseek-official', model: 'm', messages: [] })) { /* drain */ } @@ -1042,6 +2183,26 @@ describe('plugin registration and config', () => { })).rejects.toThrow(/streamIdleTimeoutMs/) }) + it('validates Files API timeout bounds independently of the stream idle deadline', async () => { + expect(() => resolveAdapterOptions({ filesApiTimeoutMs: Number.POSITIVE_INFINITY })) + .toThrow(/filesApiTimeoutMs.*positive finite/) + expect(() => resolveAdapterOptions({ filesApiTimeoutMs: MAX_TIMER_DELAY_MS + 1 })) + .toThrow(/filesApiTimeoutMs.*no greater/) + + const ctx = new Context() + await ctx.plugin(LlmRuntime) + await expect(ctx.plugin(LlmDeepSeek, { + baseURL: 'http://127.0.0.1:1', + filesApiTimeoutMs: 0, + })).rejects.toThrow(/filesApiTimeoutMs/) + await expect(ctx.plugin(LlmDeepSeek, { + baseURL: 'http://127.0.0.1:1', + filesApiTimeoutMs: MAX_TIMER_DELAY_MS + 1, + })).rejects.toThrow(/filesApiTimeoutMs/) + expect(resolveAdapterOptions({ filesApiTimeoutMs: 100, streamIdleTimeoutMs: 100 })) + .toMatchObject({ filesApiTimeoutMs: 100, streamIdleTimeoutMs: 100 }) + }) + it('rejects invalid nested retryPolicy before registering the provider', async () => { const ctx = new Context() await ctx.plugin(LlmRuntime) diff --git a/packages/llm/llm-deepseek/tests/dynamic-config.spec.ts b/packages/llm/llm-deepseek/tests/dynamic-config.spec.ts index 84e78819b0..4617ebdfed 100644 --- a/packages/llm/llm-deepseek/tests/dynamic-config.spec.ts +++ b/packages/llm/llm-deepseek/tests/dynamic-config.spec.ts @@ -3,7 +3,16 @@ import { Context } from '@deepseek-ai/cordis' import { access, mkdtemp, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' -import LlmRuntime, { INVALID_CREDENTIAL_CODE } from '@deepseek-ai/dsh-llm' +import LlmRuntime, { createUserMessage, INVALID_CREDENTIAL_CODE } from '@deepseek-ai/dsh-llm' +import AttachmentStore, { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment' +import type { + ImageAttachmentLimits, + ImageAttachmentRef, + ImageRequestPolicy, + RequestImageAttachment, + SaveImageAttachment, + StoredImageAttachment, +} from '@deepseek-ai/dsh-attachment' import { credentialRef } from '@deepseek-ai/dsh-credentials' import { LocalCredentialProvider } from '@deepseek-ai/dsh-credentials-local' import { settingsNamespace } from '@deepseek-ai/dsh-settings' @@ -14,6 +23,55 @@ import { closeMockServers, mockServer, textEvents } from './mock-server.ts' const NS = settingsNamespace('llm-deepseek') const KEY_REF = credentialRef('DEEPSEEK_API_KEY') +const IMAGE_REF: ImageAttachmentRef = { + attachmentId: AttachmentId(`sha256:${'a'.repeat(64)}`), + mediaType: 'image/png', + bytes: 3, + width: 1, + height: 1, +} + +class StaticAttachmentStore extends AttachmentStore { + readonly imageLimits: ImageAttachmentLimits = { + maxImageBytes: 16, + maxImagesPerMessage: 4, + maxMessageImageBytes: 64, + maxImagePixels: 4, + maxImageDimension: 4, + mediaTypes: ['image/png'], + } + + validateImage(_input: SaveImageAttachment): Promise { + return Promise.resolve() + } + + saveImage(_input: SaveImageAttachment): Promise { + return Promise.resolve(IMAGE_REF) + } + + readImage(ref: ImageAttachmentRef, _signal?: AbortSignal): Promise { + return Promise.resolve({ ref, data: Uint8Array.of(1, 2, 3) }) + } + + override readImageRequest( + ref: ImageAttachmentRef, + _policy: ImageRequestPolicy, + _signal?: AbortSignal, + ): Promise { + return Promise.resolve({ + variantId: ImageVariantId(`sha256:${'b'.repeat(64)}`), + attachment: ref, + data: Uint8Array.of(1, 2, 3), + mediaType: ref.mediaType, + bytes: 3, + width: ref.width, + height: ref.height, + depth: 'uchar', + space: 'srgb', + hasAlpha: true, + }) + } +} const cleanups: Array<() => Promise> = [] @@ -47,6 +105,7 @@ async function boot(dir: string, config: object): Promise { await ctx.fiber.dispose() }) await ctx.plugin(LlmRuntime) + await ctx.plugin(StaticAttachmentStore) const settingsFiber = ctx.plugin(FileSettingsProvider, { path: join(dir, 'settings.yaml'), watch: false }) await settingsFiber await ctx.plugin(LocalCredentialProvider, { path: join(dir, '.credentials.yaml'), watch: false }) @@ -62,7 +121,7 @@ describe('request-level dynamic configuration', () => { it('routes the next request with the freshly resolved base URL and credential', async () => { vi.stubEnv('DEEPSEEK_API_KEY', '') const dir = await home() - await writeFile(join(dir, '.credentials.yaml'), 'DEEPSEEK_API_KEY: first-key\n', { mode: 0o600 }) + await writeFile(join(dir, '.credentials.yaml'), 'version: 1\nrefs:\n DEEPSEEK_API_KEY: first-key\n', { mode: 0o600 }) const serverA = await mockServer([{ kind: 'sse', events: textEvents }]) const serverB = await mockServer([{ kind: 'sse', events: textEvents }]) const { ctx } = await boot(dir, { baseURL: serverA.url }) @@ -117,13 +176,42 @@ describe('request-level dynamic configuration', () => { const dir = await home() const { ctx } = await boot(dir, { baseURL: 'http://127.0.0.1:1' }) - await expect(ctx.llm.listModels('deepseek-official')).resolves.toHaveLength(2) - await ctx.settings.update(NS, { models: [{ id: 'settings-model', name: 'From Settings' }] }) + await expect(ctx.llm.listModels('deepseek-official')).resolves.toHaveLength(3) + await ctx.settings.update(NS, { + models: [{ id: 'settings-model', name: 'From Settings', inputModalities: ['text', 'image'] }], + }) await expect(ctx.llm.listModels('deepseek-official')).resolves.toEqual([ - { provider: 'deepseek-official', id: 'settings-model', name: 'From Settings', inputModalities: ['text'] }, + { provider: 'deepseek-official', id: 'settings-model', name: 'From Settings', inputModalities: ['text', 'image'] }, ]) }) + it('applies changed request file limits to the next request', async () => { + vi.stubEnv('DEEPSEEK_API_KEY', 'test-key') + const dir = await home() + const server = await mockServer([ + { kind: 'sse', events: textEvents }, + { kind: 'sse', events: textEvents }, + ]) + const { ctx } = await boot(dir, { baseURL: server.url }) + const messages = [createUserMessage({ + content: [ + { type: 'image', attachment: IMAGE_REF }, + { type: 'image', attachment: IMAGE_REF }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })] + + await assemble(ctx, { model: 'deepseek-v4-flash-vision-exp', messages }) + await ctx.settings.update(NS, { maxRequestFilesBytes: 4, imageOffloadByteQuantum: 2 }) + await assemble(ctx, { model: 'deepseek-v4-flash-vision-exp', messages }) + + const first = (server.requests[0] as { messages: Array<{ content: unknown }> }).messages[0]?.content + const second = (server.requests[1] as { messages: Array<{ content: unknown }> }).messages[0]?.content + expect(JSON.stringify(first).match(/"type":"file"/g)).toHaveLength(2) + expect(JSON.stringify(second)).toContain('[image omitted to keep the request within its image limit') + expect(JSON.stringify(second).match(/"type":"file"/g)).toHaveLength(1) + }) + it('re-registers the route in place when the captured retry policy changes, without an empty-registry window', async () => { const dir = await home() const { ctx } = await boot(dir, { baseURL: 'http://127.0.0.1:1' }) @@ -156,7 +244,7 @@ describe('request-level dynamic configuration', () => { // Schema-valid but resolver-invalid: duplicate catalog ids pass the array // schema and fail the explicit resolve step. await ctx.settings.update(NS, { models: [{ id: 'dup' }, { id: 'dup' }] }) - await expect(ctx.llm.listModels('deepseek-official')).resolves.toHaveLength(2) + await expect(ctx.llm.listModels('deepseek-official')).resolves.toHaveLength(3) await ctx.settings.update(NS, { models: [{ id: 'recovered' }] }) await expect(ctx.llm.listModels('deepseek-official')).resolves.toEqual([ { provider: 'deepseek-official', id: 'recovered', name: 'recovered', inputModalities: ['text'] }, @@ -188,7 +276,7 @@ describe('request-level dynamic configuration', () => { it('falls back to the composition entry when settings detach', async () => { vi.stubEnv('DEEPSEEK_API_KEY', '') const dir = await home() - await writeFile(join(dir, '.credentials.yaml'), 'DEEPSEEK_API_KEY: steady-key\n', { mode: 0o600 }) + await writeFile(join(dir, '.credentials.yaml'), 'version: 1\nrefs:\n DEEPSEEK_API_KEY: steady-key\n', { mode: 0o600 }) const serverA = await mockServer([{ kind: 'sse', events: textEvents }]) const serverB = await mockServer([{ kind: 'sse', events: textEvents }]) const { ctx, settingsFiber } = await boot(dir, { baseURL: serverA.url }) diff --git a/packages/llm/llm-deepseek/tests/file-store.spec.ts b/packages/llm/llm-deepseek/tests/file-store.spec.ts new file mode 100644 index 0000000000..d6d154033b --- /dev/null +++ b/packages/llm/llm-deepseek/tests/file-store.spec.ts @@ -0,0 +1,496 @@ +import { mkdtemp } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { describe, expect, it, vi } from 'vitest' +import { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment' +import type { ImageAttachmentRef, RequestImageAttachment } from '@deepseek-ai/dsh-attachment' +import { DeepSeekFileStore, MAX_CHAT_IMAGE_BYTES } from '../src/file-store.ts' +import { DeepSeekFileId } from '../src/file-id.ts' +import { deepSeekFileScope, DeepSeekUploadIndex } from '../src/upload-index.ts' + +const REF: ImageAttachmentRef = { + attachmentId: AttachmentId(`sha256:${'a'.repeat(64)}`), + mediaType: 'image/png', + bytes: 3, + width: 1, + height: 1, +} +const VERSION: RequestImageAttachment = { + variantId: ImageVariantId(`sha256:${'b'.repeat(64)}`), + attachment: REF, + data: Uint8Array.of(1, 2, 3), + mediaType: 'image/png', + bytes: 3, + width: 1, + height: 1, + depth: 'uchar', + space: 'srgb', + hasAlpha: true, +} +const CONNECTION = { baseURL: 'https://api.deepseek.com', apiKey: 'key' } +const POLICY = { expiresAfterSeconds: 604_800, refreshMarginSeconds: 3_600, quotaCleanupBatch: 100 } +const NOW = 1_700_000_000_000 + +function requestUrl(input: string | URL | Request): string { + if (typeof input === 'string') return input + return input instanceof URL ? input.href : input.url +} + +function uploadFetch(now: () => number = () => NOW) { + let uploads = 0 + const fetchImpl = vi.fn(async (_url: string | URL | Request, init?: RequestInit) => { + if (init?.method === 'POST') { + uploads += 1 + const createdAt = now() / 1_000 + return new Response(JSON.stringify({ + id: `file-api-${uploads}`, + object: 'file', + bytes: 3, + created_at: createdAt, + filename: `dsh-${'a'.repeat(16)}-${'b'.repeat(8)}.png`, + purpose: 'user_data', + expires_at: createdAt + POLICY.expiresAfterSeconds, + }), { status: 200 }) + } + if (init?.method === 'DELETE') { + const id = requestUrl(_url).split('/').at(-1) + return new Response(JSON.stringify({ id, object: 'file', deleted: true }), { status: 200 }) + } + throw new Error('unexpected Files API request') + }) as typeof fetch + return { fetchImpl, uploads: () => uploads } +} + +describe('DeepSeekFileStore', () => { + it('singleflights the first upload and reuses the durable mapping across store instances', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + const remote = uploadFetch() + const first = new DeepSeekFileStore({ index, now: () => NOW, fetch: remote.fetchImpl }) + + const [a, b] = await Promise.all([ + first.ensureUploaded(VERSION, CONNECTION, POLICY), + first.ensureUploaded(VERSION, CONNECTION, POLICY), + ]) + expect(a.record.fileId).toBe('file-api-1') + expect(b.record.fileId).toBe('file-api-1') + expect(remote.uploads()).toBe(1) + + const resumed = new DeepSeekFileStore({ index, now: () => NOW, fetch: remote.fetchImpl }) + await expect(resumed.ensureUploaded(VERSION, CONNECTION, POLICY)) + .resolves.toMatchObject({ record: { fileId: 'file-api-1' }, uploaded: false }) + expect(remote.uploads()).toBe(1) + }) + + it('keeps a shared upload alive while another waiter remains', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + let complete: ((response: Response) => void) | undefined + let uploadSignal: AbortSignal | undefined + const fetchImpl = vi.fn((_url: string | URL | Request, init?: RequestInit) => { + uploadSignal = init?.signal ?? undefined + return new Promise((resolve, reject) => { + complete = resolve + uploadSignal?.addEventListener('abort', () => { + reject(new Error('upload aborted', { cause: uploadSignal?.reason })) + }, { once: true }) + }) + }) as typeof fetch + const store = new DeepSeekFileStore({ index, now: () => NOW, fetch: fetchImpl }) + const controller = new AbortController() + + const cancelled = store.ensureUploaded(VERSION, CONNECTION, POLICY, controller.signal) + const completed = store.ensureUploaded(VERSION, CONNECTION, POLICY) + await vi.waitFor(() => { + expect(fetchImpl).toHaveBeenCalledTimes(1) + }) + const reason = new Error('cancel one upload waiter') + controller.abort(reason) + + await expect(cancelled).rejects.toBe(reason) + expect(uploadSignal?.aborted).toBe(false) + complete?.(new Response(JSON.stringify({ + id: 'file-api-shared', + object: 'file', + bytes: 3, + created_at: NOW / 1_000, + filename: `dsh-${'a'.repeat(16)}-${'b'.repeat(8)}.png`, + purpose: 'user_data', + expires_at: NOW / 1_000 + POLICY.expiresAfterSeconds, + }), { status: 200 })) + await expect(completed).resolves.toMatchObject({ record: { fileId: 'file-api-shared' } }) + }) + + it('aborts the shared upload after its only waiter cancels', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + let uploadSignal: AbortSignal | undefined + const fetchImpl = vi.fn((_url: string | URL | Request, init?: RequestInit) => { + uploadSignal = init?.signal ?? undefined + return new Promise((_resolve, reject) => { + uploadSignal?.addEventListener('abort', () => { + reject(new Error('upload aborted', { cause: uploadSignal?.reason })) + }, { once: true }) + }) + }) as typeof fetch + const store = new DeepSeekFileStore({ index, now: () => NOW, fetch: fetchImpl }) + const controller = new AbortController() + const upload = store.ensureUploaded(VERSION, CONNECTION, POLICY, controller.signal) + await vi.waitFor(() => { + expect(fetchImpl).toHaveBeenCalledTimes(1) + }) + + const reason = new Error('cancel only upload waiter') + controller.abort(reason) + + await expect(upload).rejects.toBe(reason) + expect(uploadSignal?.reason).toBe(reason) + }) + + it('normalizes a non-Error cancellation reason', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const fetchImpl = vi.fn((_url: string | URL | Request, init?: RequestInit) => ( + new Promise((_resolve, reject) => { + init?.signal?.addEventListener('abort', () => { + reject(new Error('upload aborted', { cause: init.signal?.reason })) + }, { once: true }) + }) + )) as typeof fetch + const store = new DeepSeekFileStore({ + index: new DeepSeekUploadIndex(join(dir, 'index.json')), + now: () => NOW, + fetch: fetchImpl, + }) + const controller = new AbortController() + const upload = store.ensureUploaded(VERSION, CONNECTION, POLICY, controller.signal) + await vi.waitFor(() => { + expect(fetchImpl).toHaveBeenCalledOnce() + }) + controller.abort('cancelled') + + await expect(upload).rejects.toMatchObject({ + message: 'DeepSeek file upload cancelled with a non-Error reason.', + cause: 'cancelled', + }) + }) + + it('starts a fresh upload while the cancelled transport is settling', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + let requests = 0 + const fetchImpl = vi.fn((_url: string | URL | Request, init?: RequestInit) => { + requests += 1 + if (requests === 1) { + return new Promise((_resolve, reject) => { + init?.signal?.addEventListener('abort', () => { + queueMicrotask(() => { + reject(new Error('upload aborted', { cause: init.signal?.reason })) + }) + }, { once: true }) + }) + } + return Promise.resolve(new Response(JSON.stringify({ + id: 'file-api-retry', object: 'file', bytes: 3, created_at: NOW / 1_000, + filename: 'dsh-retry.png', purpose: 'user_data', + expires_at: NOW / 1_000 + POLICY.expiresAfterSeconds, + }), { status: 200 })) + }) as typeof fetch + const store = new DeepSeekFileStore({ + index: new DeepSeekUploadIndex(join(dir, 'index.json')), + now: () => NOW, + fetch: fetchImpl, + }) + const controller = new AbortController() + const cancelled = store.ensureUploaded(VERSION, CONNECTION, POLICY, controller.signal) + await vi.waitFor(() => { + expect(fetchImpl).toHaveBeenCalledOnce() + }) + controller.abort(new Error('cancel first')) + const retried = store.ensureUploaded(VERSION, CONNECTION, POLICY) + + await expect(cancelled).rejects.toThrow('cancel first') + await expect(retried).resolves.toMatchObject({ record: { fileId: 'file-api-retry' } }) + }) + + it('rejects a request version above the chat per-image limit before transport', async () => { + const fetchImpl = vi.fn() as typeof fetch + const store = new DeepSeekFileStore({ now: () => NOW, fetch: fetchImpl }) + const oversized = { ...VERSION, bytes: MAX_CHAT_IMAGE_BYTES + 1 } + await expect(store.ensureUploaded(oversized, CONNECTION, POLICY)) + .rejects.toMatchObject({ code: 'INVALID_REQUEST' }) + expect(fetchImpl).not.toHaveBeenCalled() + }) + + it('does not persist an upload whose response is missing and retries on the next request', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + const good = uploadFetch() + let first = true + const fetchImpl = vi.fn((url: string | URL | Request, init?: RequestInit) => { + if (first) { + first = false + return Promise.resolve(new Response('', { status: 204 })) + } + return good.fetchImpl(url, init) + }) as typeof fetch + const store = new DeepSeekFileStore({ index, now: () => NOW, fetch: fetchImpl }) + + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)) + .rejects.toBeInstanceOf(Error) + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)) + .resolves.toMatchObject({ record: { fileId: 'file-api-1' }, uploaded: true }) + }) + + it('rejects an upload response whose byte count differs from the request version', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const fetchImpl = vi.fn(() => Promise.resolve(new Response(JSON.stringify({ + id: 'file-api-wrong-size', object: 'file', bytes: 2, created_at: NOW / 1_000, + filename: 'dsh-wrong.png', purpose: 'user_data', + expires_at: NOW / 1_000 + POLICY.expiresAfterSeconds, + }), { status: 200 }))) as typeof fetch + const store = new DeepSeekFileStore({ + index: new DeepSeekUploadIndex(join(dir, 'index.json')), + now: () => NOW, + fetch: fetchImpl, + }) + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)) + .rejects.toMatchObject({ code: 'INVALID_RESPONSE' }) + }) + + it.each([ + ['image/jpeg', 'jpeg'], + ['image/webp', 'webp'], + ['image/gif', 'gif'], + ] as const)('uses the %s filename extension for uploads', async (mediaType, extension) => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const remote = uploadFetch() + const store = new DeepSeekFileStore({ + index: new DeepSeekUploadIndex(join(dir, `${extension}.json`)), + now: () => NOW, + fetch: remote.fetchImpl, + }) + await store.ensureUploaded({ ...VERSION, mediaType }, CONNECTION, POLICY) + const form = vi.mocked(remote.fetchImpl).mock.calls[0]?.[1]?.body + expect(form).toBeInstanceOf(FormData) + const file = (form as FormData).get('file') + expect(file).toBeInstanceOf(File) + if (!(file instanceof File)) throw new Error('expected multipart file') + expect(file.name).toMatch(new RegExp(`\\.${extension}$`, 'u')) + }) + + it('normalizes a non-Error failure from the durable upload index', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + vi.spyOn(index, 'get').mockRejectedValue('index unavailable') + const store = new DeepSeekFileStore({ index, now: () => NOW, fetch: vi.fn() as typeof fetch }) + + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)).rejects.toMatchObject({ + message: 'DeepSeek file upload failed with a non-Error reason.', + cause: 'index unavailable', + }) + }) + + it('reuses local expires_at above the refresh margin and uploads again at the margin', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + let now = NOW + const remote = uploadFetch(() => now) + const store = new DeepSeekFileStore({ index, now: () => now, fetch: remote.fetchImpl }) + + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)) + .resolves.toMatchObject({ record: { fileId: 'file-api-1' }, uploaded: true }) + now = NOW + (POLICY.expiresAfterSeconds - POLICY.refreshMarginSeconds) * 1_000 - 1 + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)) + .resolves.toMatchObject({ record: { fileId: 'file-api-1' }, uploaded: false }) + now += 1 + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)) + .resolves.toMatchObject({ record: { fileId: 'file-api-2' }, uploaded: true }) + + expect(remote.uploads()).toBe(2) + expect(vi.mocked(remote.fetchImpl).mock.calls.every(([, init]) => init?.method === 'POST')).toBe(true) + }) + + it('releases an indexed file through DELETE and removes only that mapping', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + const remote = uploadFetch() + const store = new DeepSeekFileStore({ index, now: () => NOW, fetch: remote.fetchImpl }) + await store.ensureUploaded(VERSION, CONNECTION, POLICY) + + await expect(store.release(VERSION, CONNECTION, POLICY)).resolves.toBe(true) + await expect(store.release(VERSION, CONNECTION, POLICY)).resolves.toBe(false) + expect(remote.fetchImpl).toHaveBeenCalledTimes(2) + }) + + it('removes a losing upload and keeps the winning durable mapping when duplicate cleanup fails', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + vi.spyOn(index, 'commit').mockResolvedValue({ + accepted: false, + record: { + scope: deepSeekFileScope(CONNECTION.baseURL, CONNECTION.apiKey), + attachmentId: VERSION.attachment.attachmentId, + variantId: VERSION.variantId, + fileId: DeepSeekFileId('file-api-winner'), + bytes: 3, + createdAt: NOW, + expiresAt: NOW + POLICY.expiresAfterSeconds * 1_000, + }, + }) + const remote = uploadFetch() + const fetchImpl = vi.fn((url: string | URL | Request, init?: RequestInit) => { + if (init?.method === 'DELETE') return Promise.resolve(new Response('failed', { status: 500 })) + return remote.fetchImpl(url, init) + }) as typeof fetch + const store = new DeepSeekFileStore({ index, now: () => NOW, fetch: fetchImpl }) + + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)).resolves.toMatchObject({ + record: { fileId: 'file-api-winner' }, + uploaded: false, + }) + expect(fetchImpl).toHaveBeenCalledTimes(2) + }) + + it('reclaims one owned file after quota rejection and retries the upload once', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + let uploads = 0 + const fetchImpl = vi.fn((input: string | URL | Request, init?: RequestInit) => { + if (init?.method === 'POST') { + uploads += 1 + if (uploads === 1) return Promise.resolve(new Response(JSON.stringify({ + error: { message: 'stored file quota exceeded', code: 'file_quota' }, + }), { status: 400 })) + return Promise.resolve(new Response(JSON.stringify({ + id: 'file-api-recovered', object: 'file', bytes: 3, created_at: NOW / 1_000, + filename: 'dsh-recovered.png', purpose: 'user_data', + expires_at: NOW / 1_000 + POLICY.expiresAfterSeconds, + }), { status: 200 })) + } + if (init?.method === 'DELETE') { + return Promise.resolve(new Response(JSON.stringify({ + id: 'file-api-old', object: 'file', deleted: true, + }), { status: 200 })) + } + expect(new URL(requestUrl(input)).pathname).toBe('/files') + return Promise.resolve(new Response(JSON.stringify({ + object: 'list', + data: [{ + id: 'file-api-old', object: 'file', bytes: 3, created_at: NOW / 1_000, + filename: 'dsh-old.png', purpose: 'user_data', + }], + first_id: 'file-api-old', last_id: 'file-api-old', has_more: false, + }), { status: 200 })) + }) as typeof fetch + const store = new DeepSeekFileStore({ + index: new DeepSeekUploadIndex(join(dir, 'index.json')), + now: () => NOW, + fetch: fetchImpl, + }) + + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)).resolves.toMatchObject({ + record: { fileId: 'file-api-recovered' }, uploaded: true, + }) + expect(uploads).toBe(2) + }) + + it('preserves a quota error when no harness-owned file can be reclaimed', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const fetchImpl = vi.fn((_input: string | URL | Request, init?: RequestInit) => { + if (init?.method === 'POST') return Promise.resolve(new Response(JSON.stringify({ + error: { message: 'file count quota exceeded', code: 'file_quota' }, + }), { status: 400 })) + return Promise.resolve(new Response(JSON.stringify({ + object: 'list', + data: [{ + id: 'file-api-foreign', object: 'file', bytes: 3, created_at: NOW / 1_000, + filename: 'foreign.png', purpose: 'user_data', + }], + has_more: false, + }), { status: 200 })) + }) as typeof fetch + const store = new DeepSeekFileStore({ + index: new DeepSeekUploadIndex(join(dir, 'index.json')), + now: () => NOW, + fetch: fetchImpl, + }) + + await expect(store.ensureUploaded(VERSION, CONNECTION, POLICY)).rejects.toMatchObject({ code: 'FILES_API' }) + }) + + it('finishes pagination before deleting cursor files during quota recovery', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const deleted = new Set() + const fetchImpl = vi.fn(async (input: string | URL | Request, init?: RequestInit) => { + const target = new URL(requestUrl(input)) + if (init?.method === 'DELETE') { + const id = target.pathname.split('/').at(-1) ?? '' + deleted.add(id) + return new Response(JSON.stringify({ id, object: 'file', deleted: true }), { status: 200 }) + } + const after = target.searchParams.get('after') + if (after !== null && deleted.has(after)) throw new Error('deleted cursor cannot be reused') + const id = after === null ? 'file-api-oldest' : 'file-api-next' + return new Response(JSON.stringify({ + object: 'list', + data: [{ + id, + object: 'file', + bytes: 3, + created_at: NOW / 1_000, + filename: `dsh-${id}.png`, + purpose: 'user_data', + }], + first_id: id, + last_id: id, + has_more: after === null, + }), { status: 200 }) + }) as typeof fetch + const store = new DeepSeekFileStore({ + index: new DeepSeekUploadIndex(join(dir, 'index.json')), + now: () => NOW, + fetch: fetchImpl, + }) + + await expect(store.reclaimOldestOwned(CONNECTION, 2)).resolves.toBe(2) + expect([...deleted]).toEqual(['file-api-oldest', 'file-api-next']) + }) + + it('stops pagination when a page omits or repeats its cursor', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + for (const mode of ['missing', 'repeated'] as const) { + let page = 0 + const fetchImpl = vi.fn((input: string | URL | Request, init?: RequestInit) => { + if (init?.method === 'DELETE') { + const id = requestUrl(input).split('/').at(-1) + return Promise.resolve(new Response(JSON.stringify({ id, object: 'file', deleted: true }), { status: 200 })) + } + page += 1 + const lastId = mode === 'missing' ? undefined : 'file-api-same' + return Promise.resolve(new Response(JSON.stringify({ + object: 'list', data: [], has_more: true, + ...lastId === undefined ? {} : { last_id: lastId }, + }), { status: 200 })) + }) as typeof fetch + const store = new DeepSeekFileStore({ + index: new DeepSeekUploadIndex(join(dir, `${mode}.json`)), + now: () => NOW, + fetch: fetchImpl, + }) + await expect(store.reclaimOldestOwned(CONNECTION, 1)).resolves.toBe(0) + expect(page).toBe(mode === 'missing' ? 1 : 2) + } + }) + + it('releases every batch and clears the scoped upload index', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + const store = new DeepSeekFileStore({ index, now: () => NOW, fetch: vi.fn() as typeof fetch }) + const reclaim = vi.spyOn(store, 'reclaimOldestOwned') + .mockResolvedValueOnce(1_000) + .mockResolvedValueOnce(2) + const clear = vi.spyOn(index, 'clear') + + await expect(store.releaseAll(CONNECTION)).resolves.toBe(1_002) + expect(reclaim).toHaveBeenCalledTimes(2) + expect(clear).toHaveBeenCalledOnce() + }) +}) diff --git a/packages/llm/llm-deepseek/tests/files-api.spec.ts b/packages/llm/llm-deepseek/tests/files-api.spec.ts new file mode 100644 index 0000000000..e6a1aa59ea --- /dev/null +++ b/packages/llm/llm-deepseek/tests/files-api.spec.ts @@ -0,0 +1,260 @@ +import { describe, expect, it, vi } from 'vitest' +import { userAgent } from '@deepseek-ai/dsh-llm' +import { DeepSeekFileId } from '../src/file-id.ts' +import { + DeepSeekFilesClient, + DeepSeekFilesError, + isFilesQuotaError, + MAX_FILE_UPLOAD_BYTES, +} from '../src/files-api.ts' + +function requestUrl(input: string | URL | Request): string { + if (typeof input === 'string') return input + return input instanceof URL ? input.href : input.url +} + +function file(overrides: Record = {}) { + return { + id: 'file-api-one', + object: 'file', + bytes: 3, + created_at: 1_700_000_000, + filename: 'image.png', + purpose: 'user_data', + expires_at: 1_700_604_800, + ...overrides, + } +} + +describe('DeepSeekFilesClient', () => { + it('uploads multipart bytes with the required purpose and explicit expiry', async () => { + const fetchImpl = vi.fn(async (url: string | URL | Request, init?: RequestInit) => { + expect(requestUrl(url)).toBe('https://api.deepseek.com/files') + expect(init?.method).toBe('POST') + const headers = new Headers(init?.headers) + expect(headers.get('authorization')).toBe('Bearer key') + expect(headers.get('user-agent')).toBe(userAgent()) + const form = init?.body + expect(form).toBeInstanceOf(FormData) + if (!(form instanceof FormData)) throw new Error('expected multipart body') + expect(form.get('purpose')).toBe('user_data') + expect(form.get('expires_after[anchor]')).toBe('created_at') + expect(form.get('expires_after[seconds]')).toBe('604800') + const blob = form.get('file') + expect(blob).toBeInstanceOf(Blob) + expect((blob as Blob).size).toBe(3) + return new Response(JSON.stringify(file()), { status: 200 }) + }) as typeof fetch + const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com/', apiKey: 'key', fetch: fetchImpl }) + + await expect(client.upload({ + data: Uint8Array.of(1, 2, 3), + mediaType: 'image/png', + filename: 'image.png', + expiresAfterSeconds: 604_800, + })).resolves.toEqual({ + id: DeepSeekFileId('file-api-one'), + bytes: 3, + createdAt: 1_700_000_000, + filename: 'image.png', + purpose: 'user_data', + expiresAt: 1_700_604_800, + }) + }) + + it('validates list, retrieve, and delete responses', async () => { + const fetchImpl = vi.fn(async (url: string | URL | Request, init?: RequestInit) => { + const target = requestUrl(url) + if (target.includes('?')) { + return new Response(JSON.stringify({ + object: 'list', data: [file()], first_id: 'file-api-one', last_id: 'file-api-one', has_more: false, + }), { status: 200 }) + } + if (init?.method === 'DELETE') { + return new Response(JSON.stringify({ id: 'file-api-one', object: 'file', deleted: true }), { status: 200 }) + } + return new Response(JSON.stringify(file()), { status: 200 }) + }) as typeof fetch + const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com', apiKey: 'key', fetch: fetchImpl }) + + await expect(client.list({ after: DeepSeekFileId('file-api-before'), limit: 20, order: 'desc' })).resolves.toMatchObject({ + data: [{ id: 'file-api-one' }], firstId: 'file-api-one', lastId: 'file-api-one', hasMore: false, + }) + await expect(client.retrieve(DeepSeekFileId('file-api-one'))).resolves.toMatchObject({ id: 'file-api-one' }) + await expect(client.delete(DeepSeekFileId('file-api-one'))).resolves.toBeUndefined() + }) + + it('refuses an upload response that omits the requested expiry', async () => { + const fetchImpl = vi.fn(() => Promise.resolve(new Response( + JSON.stringify(file({ expires_at: undefined })), + { status: 200 }, + ))) as typeof fetch + const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com', apiKey: 'key', fetch: fetchImpl }) + + await expect(client.upload({ + data: Uint8Array.of(1), mediaType: 'image/png', filename: 'image.png', expiresAfterSeconds: 3_600, + })).rejects.toMatchObject({ code: 'INVALID_RESPONSE' }) + }) + + it('retains quota error detail for the one cleanup retry policy', async () => { + const fetchImpl = vi.fn(() => Promise.resolve(new Response(JSON.stringify({ + error: { message: 'user storage quota exceeded', type: 'invalid_request_error', code: 'file_quota' }, + }), { status: 400 }))) as typeof fetch + const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com', apiKey: 'key', fetch: fetchImpl }) + + const error = await client.upload({ + data: Uint8Array.of(1), mediaType: 'image/png', filename: 'image.png', expiresAfterSeconds: 3_600, + }).catch((caught: unknown) => caught) + expect(isFilesQuotaError(error)).toBe(true) + expect(isFilesQuotaError(new Error('storage quota'))).toBe(false) + }) + + it.each([ + [401, 'AUTH'], + [403, 'AUTH'], + [429, 'RATE_LIMIT'], + [500, 'SERVER'], + [400, 'FILES_API'], + ] as const)('classifies HTTP %i Files failures as %s', async (status, code) => { + const client = new DeepSeekFilesClient({ + baseURL: 'https://api.deepseek.com', + apiKey: 'key', + fetch: vi.fn(() => Promise.resolve(new Response('not-json', { status }))), + }) + await expect(client.retrieve(DeepSeekFileId('missing'))).rejects.toMatchObject({ + name: 'DeepSeekFilesError', + code, + detail: '', + }) + }) + + it.each([ + null, + [], + {}, + { error: null }, + { error: [] }, + { error: { message: 1, type: 2, code: 3 } }, + ])('falls back to the HTTP status for an unstructured provider error %#', async (body) => { + const client = new DeepSeekFilesClient({ + baseURL: 'https://api.deepseek.com', + apiKey: 'key', + fetch: vi.fn(() => Promise.resolve(new Response(JSON.stringify(body), { status: 400 }))), + }) + const error = await client.retrieve(DeepSeekFileId('missing')).catch((caught: unknown) => caught) + expect(error).toBeInstanceOf(DeepSeekFilesError) + expect(error).toMatchObject({ message: 'DeepSeek Files API error (HTTP 400)', detail: '' }) + }) + + it('wraps transport failures but preserves an aborted request reason', async () => { + const transport = new Error('socket closed') + const client = new DeepSeekFilesClient({ + baseURL: 'https://api.deepseek.com', + apiKey: 'key', + fetch: vi.fn(() => Promise.reject(transport)), + }) + await expect(client.retrieve(DeepSeekFileId('one'))).rejects.toMatchObject({ + code: 'TRANSPORT', + cause: transport, + }) + + const controller = new AbortController() + const reason = new Error('cancelled') + controller.abort(reason) + await expect(client.retrieve(DeepSeekFileId('one'), controller.signal)).rejects.toBe(transport) + }) + + it.each([ + null, + [], + file({ id: 1 }), + file({ id: '' }), + file({ object: 'wrong' }), + file({ bytes: 1.5 }), + file({ bytes: -1 }), + file({ created_at: 1.5 }), + file({ created_at: -1 }), + file({ filename: 1 }), + file({ filename: '' }), + file({ purpose: 'assistants' }), + file({ expires_at: 1.5 }), + file({ expires_at: -1 }), + ])('rejects an invalid file object %#', async (body) => { + const client = new DeepSeekFilesClient({ + baseURL: 'https://api.deepseek.com', + apiKey: 'key', + fetch: vi.fn(() => Promise.resolve(new Response(JSON.stringify(body), { status: 200 }))), + }) + await expect(client.retrieve(DeepSeekFileId('one'))).rejects.toMatchObject({ code: 'INVALID_RESPONSE' }) + }) + + it.each([ + 3_599, + 2_592_001, + 3_600.5, + ])('refuses invalid file expiry %s before transport', async (expiresAfterSeconds) => { + const fetchImpl = vi.fn() as typeof fetch + const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com', apiKey: 'key', fetch: fetchImpl }) + await expect(client.upload({ + data: Uint8Array.of(1), mediaType: 'image/png', filename: 'image.png', expiresAfterSeconds, + })).rejects.toMatchObject({ code: 'INVALID_REQUEST' }) + expect(fetchImpl).not.toHaveBeenCalled() + }) + + it('refuses a file larger than the upload limit before transport', async () => { + const fetchImpl = vi.fn() as typeof fetch + const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com', apiKey: 'key', fetch: fetchImpl }) + const data = { byteLength: MAX_FILE_UPLOAD_BYTES + 1 } as Uint8Array + await expect(client.upload({ + data, mediaType: 'image/png', filename: 'image.png', expiresAfterSeconds: 3_600, + })).rejects.toMatchObject({ code: 'INVALID_REQUEST' }) + expect(fetchImpl).not.toHaveBeenCalled() + }) + + it.each([ + null, + [], + {}, + { object: 'wrong', data: [], has_more: false }, + { object: 'list', data: null, has_more: false }, + { object: 'list', data: [], has_more: 0 }, + { object: 'list', data: [], has_more: false, first_id: 1 }, + { object: 'list', data: [], has_more: false, last_id: 1 }, + ])('rejects an invalid list response %#', async (body) => { + const client = new DeepSeekFilesClient({ + baseURL: 'https://api.deepseek.com', + apiKey: 'key', + fetch: vi.fn(() => Promise.resolve(new Response(JSON.stringify(body), { status: 200 }))), + }) + await expect(client.list()).rejects.toMatchObject({ code: 'INVALID_RESPONSE' }) + }) + + it('accepts a list without cursors and uses the global fetch default', async () => { + const fetchImpl = vi.fn(() => Promise.resolve(new Response(JSON.stringify({ + object: 'list', data: [], has_more: false, + }), { status: 200 }))) + vi.stubGlobal('fetch', fetchImpl) + try { + const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com///', apiKey: 'key' }) + await expect(client.list()).resolves.toEqual({ data: [], hasMore: false }) + } finally { + vi.unstubAllGlobals() + } + }) + + it.each([ + null, + [], + {}, + { id: 'wrong', object: 'file', deleted: true }, + { id: 'file-api-one', object: 'wrong', deleted: true }, + { id: 'file-api-one', object: 'file', deleted: false }, + ])('rejects an invalid delete response %#', async (body) => { + const client = new DeepSeekFilesClient({ + baseURL: 'https://api.deepseek.com', + apiKey: 'key', + fetch: vi.fn(() => Promise.resolve(new Response(JSON.stringify(body), { status: 200 }))), + }) + await expect(client.delete(DeepSeekFileId('file-api-one'))).rejects.toMatchObject({ code: 'INVALID_RESPONSE' }) + }) +}) diff --git a/packages/llm/llm-deepseek/tests/loader-composition.spec.ts b/packages/llm/llm-deepseek/tests/loader-composition.spec.ts index 1398d36110..6237221aae 100644 --- a/packages/llm/llm-deepseek/tests/loader-composition.spec.ts +++ b/packages/llm/llm-deepseek/tests/loader-composition.spec.ts @@ -8,7 +8,7 @@ * behavior — the documented optional-inject fallback. */ -import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { pathToFileURL } from 'node:url' @@ -17,11 +17,16 @@ import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' import Include from '@deepseek-ai/cordis-plugin-include' import LlmRuntime from '@deepseek-ai/dsh-llm' +import AgentRegistry from '@deepseek-ai/dsh-agent' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import { credentialRef } from '@deepseek-ai/dsh-credentials' import LocalCredentialProvider from '@deepseek-ai/dsh-credentials-local' import { settingsNamespace } from '@deepseek-ai/dsh-settings' import FileSettingsProvider from '@deepseek-ai/dsh-settings-file' import { getOrCreateAnonymousUserId } from '@deepseek-ai/dsh-anonymous-user-id' +import DeepSeekLlmApiExtensionRegistry from '@deepseek-ai/dsh-deepseek-llm-api-extensions' +import * as SessionLogDeepSeek from '@deepseek-ai/dsh-session-log-deepseek' +import * as DeepSeekPluginPackageInventory from '@deepseek-ai/dsh-plugin-package-inventory-deepseek' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' import { assemble } from './assemble.ts' import { closeMockServers, mockServer, textEvents } from './mock-server.ts' @@ -42,7 +47,7 @@ afterEach(async () => { }) async function loadComposition( - options: { withDynamic: boolean; baseURL: string; reuseRoot?: string }, + options: { withDynamic: boolean; baseURL: string; reuseRoot?: string; enableSessionLog?: boolean }, ): Promise<{ ctx: Context; settingsPath: string; credentialsPath: string }> { // A reused root is the restart case: the same harness home, its documents // exactly as the previous process left them. @@ -53,13 +58,26 @@ async function loadComposition( const credentialsPath = join(root, '.credentials.yaml') if (options.withDynamic && fresh) { await writeFile(settingsPath, '# personal settings\n') - await writeFile(credentialsPath, 'DEEPSEEK_API_KEY: boot-key\n', { mode: 0o600 }) + await writeFile(credentialsPath, 'version: 1\nrefs:\n DEEPSEEK_API_KEY: boot-key\n', { mode: 0o600 }) } const configPath = join(root, 'cordis.yml') await writeFile(configPath, [ '- id: llm', - " name: 'test-llm-service'", + " name: '@deepseek-ai/dsh-llm'", + '- id: session', + " name: '@deepseek-ai/dsh-session'", + '- id: agents', + " name: '@deepseek-ai/dsh-agent'", + '- id: deepseek-llm-api-extensions', + " name: '@deepseek-ai/dsh-deepseek-llm-api-extensions'", + '- id: session-log-deepseek', + " name: '@deepseek-ai/dsh-session-log-deepseek'", + ...options.enableSessionLog === true + ? [' config:', ' enabled: true'] + : [], + '- id: plugin-package-inventory-deepseek', + " name: '@deepseek-ai/dsh-plugin-package-inventory-deepseek'", ...options.withDynamic ? [ '- id: settings', @@ -87,11 +105,27 @@ async function loadComposition( await ctx.plugin(Loader) ctx.loader.builtins.include = Include const modules = new Map([ - ['test-llm-service', LlmRuntime], + ['@deepseek-ai/dsh-llm', LlmRuntime], + ['@deepseek-ai/dsh-session', SessionStore], + ['@deepseek-ai/dsh-agent', AgentRegistry], + ['@deepseek-ai/dsh-deepseek-llm-api-extensions', DeepSeekLlmApiExtensionRegistry], + ['@deepseek-ai/dsh-session-log-deepseek', SessionLogDeepSeek], + ['@deepseek-ai/dsh-plugin-package-inventory-deepseek', DeepSeekPluginPackageInventory], ['@deepseek-ai/dsh-settings-file', FileSettingsProvider], ['@deepseek-ai/dsh-credentials-local', LocalCredentialProvider], ['@deepseek-ai/dsh-llm-deepseek', LlmDeepSeek], ]) + // The custom importer bypasses Node resolution; mirror the package manifests + // a deployed cordis.yml has beside its declared dependencies. + await Promise.all([...modules.keys()].map(async (packageName) => { + const packageDir = join(root!, 'node_modules', ...packageName.split('/')) + await mkdir(packageDir, { recursive: true }) + await writeFile(join(packageDir, 'package.json'), `${JSON.stringify({ + name: packageName, + version: '0.1.0-rc.8', + type: 'module', + })}\n`) + })) ctx.loader.internal = { version: 'v2', async import(specifier: string) { @@ -108,6 +142,56 @@ async function loadComposition( } describe('llm-deepseek real dynamic composition', () => { + it('keeps session upload off and package inventory on by default in the real Loader composition', async () => { + vi.stubEnv('DEEPSEEK_API_KEY', 'entry-key') + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const { ctx } = await loadComposition({ withDynamic: false, baseURL: server.url }) + const session = ctx.sessions.create(SessionId('extension-composition')) + session.append('turn/start', { turn: 1 }) + + await assemble(ctx, { model: 'deepseek-v4-flash', messages: [], sessionId: session.id }) + const request = server.requests[0] as { dsh_plugin_packages: { version: number; packages: unknown[] } } + expect(request).not.toHaveProperty('dsh_session_log') + expect(request.dsh_plugin_packages.packages).toEqual(expect.arrayContaining([ + { name: '@deepseek-ai/dsh-deepseek-llm-api-extensions', version: '0.1.0-rc.8' }, + { name: '@deepseek-ai/dsh-llm-deepseek', version: '0.1.0-rc.8' }, + { name: '@deepseek-ai/dsh-session-log-deepseek', version: '0.1.0-rc.8' }, + ])) + expect(request.dsh_plugin_packages.version).toBe(1) + expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(-1) + }) + + it('sends the canonical session suffix when the Loader composition explicitly enables upload', async () => { + vi.stubEnv('DEEPSEEK_API_KEY', 'entry-key') + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const { ctx } = await loadComposition({ + withDynamic: false, + baseURL: server.url, + enableSessionLog: true, + }) + const session = ctx.sessions.create(SessionId('extension-composition-enabled')) + session.append('turn/start', { turn: 1 }) + + await assemble(ctx, { model: 'deepseek-v4-flash', messages: [], sessionId: session.id }) + const request = server.requests[0] as { + dsh_session_log?: { + version: number + session: { id: string } + afterSeq: number + throughSeq: number + events: Array<{ type: string; seq: number }> + } + } + expect(request.dsh_session_log).toMatchObject({ + version: 1, + session: { id: 'extension-composition-enabled' }, + afterSeq: -1, + throughSeq: 0, + events: [{ type: 'turn/start', seq: 0 }], + }) + expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(0) + }) + it('boots from cordis.yml and routes the next request after external settings and credential edits', async () => { vi.stubEnv('DEEPSEEK_API_KEY', '') const serverA = await mockServer([{ kind: 'sse', events: textEvents }]) @@ -124,7 +208,7 @@ describe('llm-deepseek real dynamic composition', () => { await vi.waitFor(() => { expect((ctx.get('settings')!.get(NS) as { baseURL?: string }).baseURL).toBe(serverB.url) }, { timeout: 5000 }) - await writeFile(credentialsPath, 'DEEPSEEK_API_KEY: rotated-key\n', { mode: 0o600 }) + await writeFile(credentialsPath, 'version: 1\nrefs:\n DEEPSEEK_API_KEY: rotated-key\n', { mode: 0o600 }) await vi.waitFor(async () => { expect(await ctx.get('credentials')!.resolve(KEY_REF)).toEqual({ value: 'rotated-key', source: 'file' }) }, { timeout: 5000 }) diff --git a/packages/llm/llm-deepseek/tests/mock-server.ts b/packages/llm/llm-deepseek/tests/mock-server.ts index cdb499e143..819a945e93 100644 --- a/packages/llm/llm-deepseek/tests/mock-server.ts +++ b/packages/llm/llm-deepseek/tests/mock-server.ts @@ -13,6 +13,8 @@ export interface MockServer { requests: unknown[] /** Header bags of received requests, in order (parallel to `requests`). */ headers: IncomingMessage['headers'][] + /** Parsed Files API operations, excluded from chat request ordering. */ + fileRequests: Array<{ method: string; path: string; filename?: string; bytes?: number }> script: Behavior[] close(): Promise } @@ -36,36 +38,111 @@ export const textEvents = [ export async function mockServer(script: Behavior[]): Promise { const requests: unknown[] = [] const headers: IncomingMessage['headers'][] = [] + const fileRequests: MockServer['fileRequests'] = [] + const files = new Map() + let nextFile = 1 const server = createServer((request: IncomingMessage, response: ServerResponse) => { - let body = '' - request.on('data', (chunk: Buffer) => { body += chunk.toString('utf8') }) + const chunks: Buffer[] = [] + request.on('data', (chunk: Buffer) => { chunks.push(chunk) }) request.on('end', () => { - requests.push(JSON.parse(body)) - headers.push(request.headers) - const behavior = script.shift() - if (!behavior) { - response.writeHead(500).end('mock script exhausted') - return - } - if (behavior.kind === 'http-error') { - response.writeHead(behavior.status, { - 'content-type': behavior.contentType ?? 'application/json', - ...behavior.headers, - }) - response.end(behavior.body) - return - } - response.writeHead(200, { 'content-type': 'text/event-stream' }) - const write = (index: number): void => { - if (index >= behavior.events.length) { - if (behavior.kind === 'sse') response.end() - else response.destroy() // close-early: drop the socket mid-stream + void (async () => { + const url = new URL(request.url ?? '/', 'http://localhost') + const body = Buffer.concat(chunks) + if (url.pathname === '/files' && request.method === 'POST') { + const headers = new Headers() + for (const [name, value] of Object.entries(request.headers)) { + if (value !== undefined) headers.set(name, Array.isArray(value) ? value.join(', ') : value) + } + const form = await new Request('http://localhost/files', { + method: 'POST', + headers, + body, + }).formData() + const blob = form.get('file') + if (!(blob instanceof Blob)) throw new Error('mock upload omitted file') + const name = 'name' in blob && typeof blob.name === 'string' ? blob.name : 'uploaded_file' + const id = `file-api-${nextFile}` + const createdAt = Math.floor(Date.now() / 1_000) + nextFile += 1 + const expiresSeconds = Number(form.get('expires_after[seconds]')) + const file = { + id, + object: 'file' as const, + bytes: blob.size, + created_at: createdAt, + filename: name, + purpose: 'user_data' as const, + expires_at: createdAt + expiresSeconds, + } + files.set(id, file) + fileRequests.push({ method: 'POST', path: url.pathname, filename: name, bytes: blob.size }) + response.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify(file)) return } - response.write(`data: ${behavior.events[index]}\n\n`) - setTimeout(() => { write(index + 1) }, behavior.kind === 'sse' ? behavior.delayMs ?? 0 : 5) - } - write(0) + if (url.pathname === '/files' && request.method === 'GET') { + fileRequests.push({ method: 'GET', path: `${url.pathname}${url.search}` }) + const data = [...files.values()] + response.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify({ + object: 'list', + data, + first_id: data[0]?.id, + last_id: data.at(-1)?.id, + has_more: false, + })) + return + } + if (url.pathname.startsWith('/files/') && request.method === 'DELETE') { + const id = decodeURIComponent(url.pathname.slice('/files/'.length)) + files.delete(id) + fileRequests.push({ method: 'DELETE', path: url.pathname }) + response.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify({ + id, object: 'file', deleted: true, + })) + return + } + if (url.pathname.startsWith('/files/') && request.method === 'GET') { + const id = decodeURIComponent(url.pathname.slice('/files/'.length)) + fileRequests.push({ method: 'GET', path: url.pathname }) + const file = files.get(id) + if (file === undefined) { + response.writeHead(404, { 'content-type': 'application/json' }).end(JSON.stringify({ + error: { message: 'file not found', code: 'file_not_found' }, + })) + } else { + response.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify(file)) + } + return + } + + requests.push(JSON.parse(body.toString('utf8'))) + headers.push(request.headers) + const behavior = script.shift() + if (!behavior) { + response.writeHead(500).end('mock script exhausted') + return + } + if (behavior.kind === 'http-error') { + response.writeHead(behavior.status, { + 'content-type': behavior.contentType ?? 'application/json', + ...behavior.headers, + }) + response.end(behavior.body) + return + } + response.writeHead(200, { 'content-type': 'text/event-stream' }) + const write = (index: number): void => { + if (index >= behavior.events.length) { + if (behavior.kind === 'sse') response.end() + else response.destroy() // close-early: drop the socket mid-stream + return + } + response.write(`data: ${behavior.events[index]}\n\n`) + setTimeout(() => { write(index + 1) }, behavior.kind === 'sse' ? behavior.delayMs ?? 0 : 5) + } + write(0) + })().catch((error: unknown) => { + response.writeHead(500, { 'content-type': 'text/plain' }).end(String(error)) + }) }) }) servers.push(server) @@ -76,6 +153,7 @@ export async function mockServer(script: Behavior[]): Promise { url: `http://127.0.0.1:${address.port}`, requests, headers, + fileRequests, script, close: () => new Promise(resolve => server.close(() => { resolve() })), } diff --git a/packages/llm/llm-deepseek/tests/serialize.spec.ts b/packages/llm/llm-deepseek/tests/serialize.spec.ts index e7e95703fb..968ceabdaf 100644 --- a/packages/llm/llm-deepseek/tests/serialize.spec.ts +++ b/packages/llm/llm-deepseek/tests/serialize.spec.ts @@ -1,13 +1,83 @@ -import { describe, expect, it } from 'vitest' -import { AttachmentId } from '@deepseek-ai/dsh-attachment' +import { describe, expect, it, vi } from 'vitest' +import { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment' +import type { ImageAttachmentRef, ImageMediaType, RequestImageAttachment } from '@deepseek-ai/dsh-attachment' import { createUserMessage, CallId, ReasoningEffortId, createMessage } from '@deepseek-ai/dsh-llm' import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm' -import { serializeMessages, serializeRequest } from '../src/serialize.ts' +import { + serializeMessages, + serializeMessagesWithImages, + serializeRequest, + serializeRequestWithImages, +} from '../src/serialize.ts' +import type { ImageSerializationOptions } from '../src/serialize.ts' + +type FileResolver = Extract['resolveFileId'] function request(overrides: Partial = {}): GenerateOptions { return { provider: 'deepseek-official', model: 'deepseek-v4-flash', messages: [], ...overrides } } +function imageRef(mediaType: ImageMediaType = 'image/png', bytes = 3): ImageAttachmentRef { + const digit = ({ + 'image/png': 'a', + 'image/jpeg': 'b', + 'image/webp': 'c', + 'image/gif': 'd', + } as const)[mediaType] + return { + attachmentId: AttachmentId(`sha256:${digit.repeat(64)}`), + mediaType, + bytes, + width: 1, + height: 1, + } +} + +function fileResolver(id = 'file-api-image') { + return vi.fn(() => Promise.resolve(id)) +} + +function requestVersion(ref: ImageAttachmentRef): RequestImageAttachment { + const hash = String(ref.attachmentId).slice('sha256:'.length) + return { + variantId: ImageVariantId(`sha256:${hash}`), + attachment: ref, + data: new Uint8Array(ref.bytes), + mediaType: ref.mediaType, + bytes: ref.bytes, + width: ref.width, + height: ref.height, + depth: 'uchar', + space: 'srgb', + hasAlpha: ref.mediaType === 'image/png', + } +} + +function imageOptions( + refs: readonly ImageAttachmentRef[], + resolveFileId: FileResolver = fileResolver(), + maxRequestImageBytes = 20 * 1024 * 1024, +) { + return { + representation: { kind: 'file' as const, resolveFileId }, + requestImages: new Map(refs.map(ref => [ref.attachmentId, requestVersion(ref)])), + maxRequestImageBytes, + } +} + +function inlineImageOptions( + refs: readonly ImageAttachmentRef[], + maxRequestImageBytes = 20 * 1024 * 1024, + byteQuantum = 10 * 1024 * 1024, +): ImageSerializationOptions { + return { + representation: { kind: 'base64' }, + requestImages: new Map(refs.map(ref => [ref.attachmentId, requestVersion(ref)])), + maxRequestImageBytes, + byteQuantum, + } +} + describe('serializeMessages', () => { it('maps user text to string content', () => { const wire = serializeMessages([ @@ -29,7 +99,7 @@ describe('serializeMessages', () => { expect(wire).toEqual([{ role: 'system', content: 'be brief' }]) }) - it('maps plain assistant text without reasoning_content', () => { + it('passes reasoning_content back on tool-call-free turns', () => { const wire = serializeMessages([ createMessage({ role: 'assistant', @@ -40,8 +110,10 @@ describe('serializeMessages', () => { source: { kind: 'plugin', plugin: 'test' }, }), ]) - // Tool-call-free turn: reasoning is dropped (ignored by the API anyway). - expect(wire).toEqual([{ role: 'assistant', content: 'answer' }]) + // A gateway that re-encodes the conversation for another vendor recovers + // the upstream thinking signature by hashing this exact text, and a turn + // that called no tool carries it nowhere else. + expect(wire).toEqual([{ role: 'assistant', content: 'answer', reasoning_content: 'thinking…' }]) }) it('passes reasoning_content back on tool-call turns (official passback rule)', () => { @@ -203,13 +275,13 @@ describe('serializeRequest', () => { expect(wire.tools).toBeUndefined() }) - it('maps adapter-default thinking and the request reasoning effort', () => { + it.each(['low', 'high', 'max'] as const)('maps adapter-default thinking and request effort %s', (effort) => { const wire = serializeRequest( - request({ messages: history, reasoningEffort: ReasoningEffortId('max') }), + request({ messages: history, reasoningEffort: ReasoningEffortId(effort) }), { thinking: 'enabled', reasoningEffort: 'high' }, ) expect(wire.thinking).toEqual({ type: 'enabled' }) - expect(wire.reasoning_effort).toBe('max') + expect(wire.reasoning_effort).toBe(effort) }) it('maps off to disabled thinking without a wire reasoning effort', () => { @@ -270,6 +342,334 @@ describe('serializeRequest', () => { }) }) +describe('image serialization', () => { + it.each([ + 'image/png', + 'image/jpeg', + 'image/webp', + 'image/gif', + ] as const)('preserves ordered text and %s image parts', async (mediaType) => { + const resolveFileId = fileResolver() + const ref = imageRef(mediaType) + const wire = await serializeRequestWithImages(request({ + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'text', text: 'before' }, + { type: 'image', attachment: ref }, + { type: 'text', text: 'after' }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + }), imageOptions([ref], resolveFileId)) + + expect(wire.messages).toEqual([{ + role: 'user', + content: [ + { type: 'text', text: 'before' }, + { type: 'text', text: expect.stringContaining(`Image ${ref.attachmentId}; request image 1x1px`) as string }, + { type: 'file', file_id: 'file-api-image' }, + { type: 'text', text: 'after' }, + ], + }]) + }) + + it.each([ + ['image/png', 'data:image/png;base64,AAAA'], + ['image/jpeg', 'data:image/jpeg;base64,AAAA'], + ['image/webp', 'data:image/webp;base64,AAAA'], + ['image/gif', 'data:image/gif;base64,AAAA'], + ] as const)('serializes every retained %s request version as an inline data URL', async (mediaType, url) => { + const ref = imageRef(mediaType) + const wire = await serializeRequestWithImages(request({ + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: ref }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }), inlineImageOptions([ref])) + + expect(wire.messages).toEqual([{ + role: 'user', + content: [ + { type: 'text', text: `Image ${ref.attachmentId}; request image 1x1px.` }, + { type: 'image_url', image_url: { url } }, + ], + }]) + }) + + it('gives image-only input a stable handle and request dimensions', async () => { + const ref = imageRef() + const wire = await serializeRequestWithImages(request({ + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: ref }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }), imageOptions([ref])) + + expect(wire.messages).toEqual([{ + role: 'user', + content: [ + { type: 'text', text: `Image ${ref.attachmentId}; request image 1x1px.` }, + { type: 'file', file_id: 'file-api-image' }, + ], + }]) + }) + + it('rejects an image whose prepared request version is absent', async () => { + const ref = imageRef() + await expect(serializeMessagesWithImages([createUserMessage({ + content: [{ type: 'image', attachment: ref }], + source: { kind: 'plugin', plugin: 'test' }, + })], imageOptions([]))).rejects.toMatchObject({ code: 'INVALID_REQUEST' }) + }) + + it('keeps tool content textual and groups consecutive tool-result images afterward', async () => { + const messages = [ + createUserMessage({ + content: [{ + type: 'tool-result', + toolCallId: CallId('first'), + content: [{ type: 'image', attachment: imageRef() }], + }], + source: { kind: 'plugin', plugin: 'test' }, + }), + createUserMessage({ + content: [{ + type: 'tool-result', + toolCallId: CallId('second'), + content: [ + { type: 'text', text: 'caption' }, + { type: 'image', attachment: imageRef('image/jpeg') }, + ], + }], + source: { kind: 'plugin', plugin: 'test' }, + }), + ] + + const png = imageRef() + const jpeg = imageRef('image/jpeg') + await expect(serializeMessagesWithImages(messages, imageOptions( + [png, jpeg], + vi.fn((version: RequestImageAttachment) => Promise.resolve(`file-api-${version.mediaType}`)), + ))).resolves.toEqual([ + { + role: 'tool', + tool_call_id: 'first', + content: expect.stringContaining(`Image ${png.attachmentId}`) as string, + }, + { + role: 'tool', + tool_call_id: 'second', + content: expect.stringContaining(`caption\nImage ${jpeg.attachmentId}`) as string, + }, + { + role: 'user', + content: [ + { type: 'text', text: 'Attached image(s) from tool result:' }, + { type: 'file', file_id: 'file-api-image/png' }, + { type: 'file', file_id: 'file-api-image/jpeg' }, + ], + }, + ]) + }) + + it('does not emit an empty user message for ignored content beside a tool result', async () => { + const messages = [createUserMessage({ + content: [ + { type: 'text', text: '' }, + { type: 'chart', data: 'ignored' } as unknown as ContentBlock, + { + type: 'tool-result', + toolCallId: CallId('result'), + content: [{ type: 'text', text: 'ok' }], + }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })] + + await expect(serializeMessagesWithImages(messages, imageOptions([], fileResolver()))).resolves.toEqual([ + { role: 'tool', tool_call_id: 'result', content: 'ok' }, + ]) + }) + + it('recursively converts nested tool-result content and preserves the empty fallback', async () => { + const messages = [createUserMessage({ + content: [ + { + type: 'tool-result', + toolCallId: CallId('nested'), + content: [{ + type: 'tool-result', + toolCallId: CallId('inner'), + content: [{ type: 'text', text: 'inside' }], + }], + }, + { type: 'tool-result', toolCallId: CallId('empty'), content: [] }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })] + + await expect(serializeMessagesWithImages(messages, imageOptions([], fileResolver()))).resolves.toEqual([ + { role: 'tool', tool_call_id: 'nested', content: 'inside' }, + { role: 'tool', tool_call_id: 'empty', content: '(no output)' }, + ]) + }) + + it('flushes tool-result images before system and assistant history', async () => { + const imageResult = (id: string) => createUserMessage({ + content: [{ + type: 'tool-result', + toolCallId: CallId(id), + content: [{ type: 'image', attachment: imageRef() }], + }], + source: { kind: 'plugin' as const, plugin: 'test' }, + }) + const messages = [ + imageResult('before-system'), + createMessage({ + role: 'system', + content: [{ type: 'text', text: 'system history' }], + source: { kind: 'plugin', plugin: 'test' }, + }), + imageResult('before-assistant'), + createMessage({ + role: 'assistant', + content: [{ type: 'text', text: 'assistant history' }], + source: { kind: 'plugin', plugin: 'test' }, + }), + ] + + const wire = await serializeMessagesWithImages(messages, imageOptions([imageRef()], fileResolver())) + expect(wire).toEqual([ + { + role: 'tool', + tool_call_id: 'before-system', + content: expect.stringContaining('request image 1x1px') as string, + }, + expect.objectContaining({ role: 'user' }), + { role: 'system', content: 'system history' }, + { + role: 'tool', + tool_call_id: 'before-assistant', + content: expect.stringContaining('request image 1x1px') as string, + }, + expect.objectContaining({ role: 'user' }), + { role: 'assistant', content: 'assistant history' }, + ]) + }) + + it('offloads oldest images before reads and keeps the newest image', async () => { + const resolveFileId = fileResolver() + const png = imageRef('image/png', 3) + const jpeg = imageRef('image/jpeg', 3) + const wire = await serializeRequestWithImages(request({ + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [ + { type: 'image', attachment: png }, + { type: 'image', attachment: jpeg }, + ], + source: { kind: 'plugin', plugin: 'test' }, + })], + }), imageOptions([png, jpeg], resolveFileId, 4)) + + expect(wire.messages[0]).toMatchObject({ + role: 'user', + content: [ + { type: 'text', text: expect.stringContaining('older images are omitted first') as string }, + { type: 'text', text: expect.stringContaining(`Image ${jpeg.attachmentId}`) as string }, + { type: 'file', file_id: 'file-api-image' }, + ], + }) + expect(resolveFileId).toHaveBeenCalledTimes(1) + expect(resolveFileId.mock.calls[0]?.[0]).toMatchObject({ attachment: { mediaType: 'image/jpeg' } }) + }) + + it('drops base64 history from a 20-unit high watermark to a 10-unit low watermark', async () => { + const ref = imageRef('image/png', 3) + const wire = await serializeRequestWithImages(request({ + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: Array.from({ length: 21 }, () => ({ type: 'image' as const, attachment: ref })), + source: { kind: 'plugin', plugin: 'test' }, + })], + }), inlineImageOptions([ref], 80, 40)) + + const content = wire.messages[0]?.content + expect(JSON.stringify(content).match(/older images are omitted first/g)).toHaveLength(11) + expect(JSON.stringify(content).match(/"type":"image_url"/g)).toHaveLength(10) + }) + + it('rejects an unprepared image while computing exact request bytes', async () => { + const ref = imageRef() + await expect(serializeRequestWithImages(request({ + model: 'deepseek-v4-flash-vision-exp', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: ref }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }), imageOptions([]))).rejects.toMatchObject({ code: 'INVALID_REQUEST' }) + }) + + it.each(['system', 'assistant'] as const)('rejects an image in %s history before reading attachments', async (role) => { + const resolveFileId = vi.fn() + await expect(serializeMessagesWithImages([createMessage({ + role, + content: [{ type: 'image', attachment: imageRef() }], + source: { kind: 'plugin', plugin: 'test' }, + })], imageOptions([imageRef()], resolveFileId))) + .rejects.toMatchObject({ code: 'UNSUPPORTED_CONTENT' }) + expect(resolveFileId).not.toHaveBeenCalled() + }) + + it('rejects unsupported image history before request offloading can replace it', async () => { + const resolveFileId = vi.fn() + await expect(serializeRequestWithImages(request({ + messages: [createMessage({ + role: 'system', + content: [{ type: 'image', attachment: imageRef('image/png', 300) }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }), imageOptions([imageRef('image/png', 300)], resolveFileId, 1))) + .rejects.toMatchObject({ code: 'UNSUPPORTED_CONTENT' }) + expect(resolveFileId).not.toHaveBeenCalled() + }) + + it('prepends the request system prompt on the image path', async () => { + const ref = imageRef() + const wire = await serializeRequestWithImages(request({ + system: 'system prompt', + messages: [createUserMessage({ + content: [{ type: 'image', attachment: ref }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }), imageOptions([ref])) + expect(wire.messages[0]).toEqual({ role: 'system', content: 'system prompt' }) + }) + + it('preserves stable file-resolution failure codes', async () => { + const failure = new Error('Stored attachment bytes are corrupt.') as Error & { code: string } + failure.code = 'ATTACHMENT_CORRUPT' + const resolveFileId = vi.fn(() => Promise.reject(failure)) + await expect(serializeMessagesWithImages([createUserMessage({ + content: [{ type: 'image', attachment: imageRef() }], + source: { kind: 'plugin', plugin: 'test' }, + })], imageOptions([imageRef()], resolveFileId))) + .rejects.toMatchObject({ code: 'ATTACHMENT_CORRUPT' }) + }) + + it('preserves non-attachment resolver failures', async () => { + const failure = new Error('resolver failed') + const resolveFileId = vi.fn(() => Promise.reject(failure)) + await expect(serializeMessagesWithImages([createUserMessage({ + content: [{ type: 'image', attachment: imageRef() }], + source: { kind: 'plugin', plugin: 'test' }, + })], imageOptions([imageRef()], resolveFileId))).rejects.toBe(failure) + }) +}) + describe('review fixes: assistant content shapes', () => { it('serializes a content-less, tool-call-less assistant message as "" content, never null', () => { // Aborted/empty assistant turns: no text, no calls → "". The earlier @@ -282,16 +682,17 @@ describe('review fixes: assistant content shapes', () => { expect(wire).toEqual([{ role: 'assistant', content: '' }]) }) - it('serializes a reasoning-ONLY assistant message as "" content with the reasoning dropped', () => { + it('serializes a reasoning-ONLY assistant message as "" content beside its reasoning', () => { // The model can answer entirely in the reasoning channel (a v4-flash - // greeting did, live). The passback rule keeps reasoning_content off - // plain turns, and content must still be SET — a null here poisoned the - // session log and bricked every later turn of that session. + // greeting did, live). Content must still be SET — a null here poisoned + // the session log and bricked every later turn of that session. const wire = serializeMessages([createMessage({ role: 'assistant', content: [{ type: 'reasoning', text: '你好!有什么我可以帮你的吗?' }], source: { kind: 'plugin', plugin: 'test' }, })]) - expect(wire).toEqual([{ role: 'assistant', content: '' }]) + expect(wire).toEqual([{ + role: 'assistant', content: '', reasoning_content: '你好!有什么我可以帮你的吗?', + }]) }) it('serializes tool-call turns with empty string content, not null', () => { diff --git a/packages/llm/llm-deepseek/tests/upload-index.spec.ts b/packages/llm/llm-deepseek/tests/upload-index.spec.ts new file mode 100644 index 0000000000..2cad8a22be --- /dev/null +++ b/packages/llm/llm-deepseek/tests/upload-index.spec.ts @@ -0,0 +1,181 @@ +import { mkdir, mkdtemp, readFile, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { describe, expect, it } from 'vitest' +import { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment' +import { DeepSeekFileId } from '../src/file-id.ts' +import { deepSeekFileScope, DeepSeekUploadIndex } from '../src/upload-index.ts' + +const ATTACHMENT = AttachmentId(`sha256:${'a'.repeat(64)}`) +const VARIANT = ImageVariantId(`sha256:${'b'.repeat(64)}`) + +describe('DeepSeekUploadIndex', () => { + it('normalizes trailing endpoint slashes in the credential scope', () => { + expect(deepSeekFileScope('https://api.deepseek.com///', 'key')) + .toBe(deepSeekFileScope('https://api.deepseek.com', 'key')) + }) + + it('isolates API-key namespaces and reuses only records above the refresh margin', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + const first = deepSeekFileScope('https://api.deepseek.com', 'first-key') + const second = deepSeekFileScope('https://api.deepseek.com', 'second-key') + const record = { + scope: first, + attachmentId: ATTACHMENT, + variantId: VARIANT, + fileId: DeepSeekFileId('file-api-one'), + bytes: 3, + createdAt: 1_000, + expiresAt: 10_000, + } + + await expect(index.commit(record, 1_000, 1_000)).resolves.toMatchObject({ accepted: true }) + await expect(index.get(first, VARIANT, 1_000, 1_000)).resolves.toEqual(record) + await expect(index.get(second, VARIANT, 1_000, 1_000)).resolves.toBeUndefined() + await expect(index.get(first, VARIANT, 9_000, 1_000)).resolves.toBeUndefined() + }) + + it('keeps a reusable cross-process winner and removes only an exact generation', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + const scope = deepSeekFileScope('https://api.deepseek.com', 'key') + const first = { + scope, attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: DeepSeekFileId('file-api-first'), bytes: 3, createdAt: 1, expiresAt: 10_000, + } + const duplicate = { ...first, fileId: DeepSeekFileId('file-api-duplicate') } + await index.commit(first, 1, 1) + + await expect(index.commit(duplicate, 2, 1)).resolves.toEqual({ record: first, accepted: false }) + await index.remove(scope, VARIANT, duplicate.fileId) + await expect(index.get(scope, VARIANT, 2, 1)).resolves.toEqual(first) + await index.remove(scope, VARIANT, first.fileId) + await expect(index.get(scope, VARIANT, 2, 1)).resolves.toBeUndefined() + }) + + it('treats a corrupt upload cache as empty and repairs it on the next commit', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-')) + const path = join(dir, 'index.json') + await writeFile(path, '{bad', 'utf8') + const index = new DeepSeekUploadIndex(path) + const scope = deepSeekFileScope('https://api.deepseek.com', 'key') + const record = { + scope, + attachmentId: ATTACHMENT, + variantId: VARIANT, + fileId: DeepSeekFileId('file-api-repaired'), + bytes: 3, + createdAt: 1, + expiresAt: 10_000, + } + + await expect(index.get(scope, VARIANT, 1, 1)).resolves.toBeUndefined() + await expect(index.commit(record, 1, 1)).resolves.toEqual({ record, accepted: true }) + await expect(index.get(scope, VARIANT, 1, 1)).resolves.toEqual(record) + expect(JSON.parse(await readFile(path, 'utf8'))).toMatchObject({ formatVersion: 3 }) + }) + + it.each([ + 'null', + '[]', + '{}', + '{"formatVersion":1,"records":[]}', + '{"formatVersion":2,"records":[]}', + '{"formatVersion":3,"records":null}', + '{"formatVersion":3,"records":[null]}', + '{"formatVersion":3,"records":[[]]}', + '{"formatVersion":3,"records":[{}]}', + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'x'.repeat(64), attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: 10_000, + })}]}`, + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'a'.repeat(64), attachmentId: 'wrong', variantId: VARIANT, + fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: 10_000, + })}]}`, + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'a'.repeat(64), attachmentId: ATTACHMENT, variantId: 'wrong', + fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: 10_000, + })}]}`, + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'a'.repeat(64), attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: '', bytes: 3, createdAt: 1, expiresAt: 10_000, + })}]}`, + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'a'.repeat(64), attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: 'file-api-one', bytes: -1, createdAt: 1, expiresAt: 10_000, + })}]}`, + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'a'.repeat(64), attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: 'file-api-one', bytes: 1.5, createdAt: 1, expiresAt: 10_000, + })}]}`, + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'a'.repeat(64), attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: 'file-api-one', bytes: 3, createdAt: -1, expiresAt: 10_000, + })}]}`, + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'a'.repeat(64), attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: 'file-api-one', bytes: 3, createdAt: 1.5, expiresAt: 10_000, + })}]}`, + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'a'.repeat(64), attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: -1, + })}]}`, + `{"formatVersion":3,"records":[${JSON.stringify({ + scope: 'a'.repeat(64), attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: 1.5, + })}]}`, + ])('treats an invalid persisted index as empty %#', async (text) => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-')) + const path = join(dir, 'index.json') + await writeFile(path, text, 'utf8') + const index = new DeepSeekUploadIndex(path) + await expect(index.get( + deepSeekFileScope('https://api.deepseek.com', 'key'), VARIANT, 1, 1, + )).resolves.toBeUndefined() + }) + + it('rejects duplicate persisted mappings as a corrupt cache', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-')) + const path = join(dir, 'index.json') + const scope = deepSeekFileScope('https://api.deepseek.com', 'key') + const record = { + scope, attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: DeepSeekFileId('file-api-one'), bytes: 3, createdAt: 1, expiresAt: 10_000, + } + await writeFile(path, JSON.stringify({ formatVersion: 3, records: [record, record] }), 'utf8') + const index = new DeepSeekUploadIndex(path) + await expect(index.get(scope, VARIANT, 1, 1)).resolves.toBeUndefined() + }) + + it('drops expired records on commit and clears only the selected namespace', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-')) + const index = new DeepSeekUploadIndex(join(dir, 'index.json')) + const first = deepSeekFileScope('https://api.deepseek.com', 'first') + const second = deepSeekFileScope('https://api.deepseek.com', 'second') + const expired = { + scope: first, attachmentId: ATTACHMENT, variantId: VARIANT, + fileId: DeepSeekFileId('file-api-expired'), bytes: 3, createdAt: 1, expiresAt: 2, + } + const live = { + ...expired, scope: second, fileId: DeepSeekFileId('file-api-live'), expiresAt: 10_000, + } + await index.commit(expired, 0, 0) + await index.commit(live, 3, 1) + await index.clear(first) + await index.clear(second) + await expect(index.get(second, VARIANT, 3, 1)).resolves.toBeUndefined() + await index.clear(second) + }) + + it('propagates non-cache filesystem read failures', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-')) + const path = join(dir, 'directory') + await mkdir(path) + const index = new DeepSeekUploadIndex(path) + await expect(index.get( + deepSeekFileScope('https://api.deepseek.com', 'key'), VARIANT, 1, 1, + )).rejects.toBeInstanceOf(Error) + }) +}) diff --git a/packages/llm/llm-deepseek/tsconfig.json b/packages/llm/llm-deepseek/tsconfig.json index 0a1751aab1..2f75c10b9e 100644 --- a/packages/llm/llm-deepseek/tsconfig.json +++ b/packages/llm/llm-deepseek/tsconfig.json @@ -20,9 +20,24 @@ { "path": "../../llm/llm" }, + { + "path": "../../attachment/attachment" + }, + { + "path": "../../util/atomic-write" + }, + { + "path": "../../util/brand" + }, + { + "path": "../../util/home-paths" + }, { "path": "../../credentials/credentials" }, + { + "path": "../deepseek-llm-api-extensions" + }, { "path": "../../util/launch-environment" }, diff --git a/packages/llm/llm-pi-ai/README.i18n.yaml b/packages/llm/llm-pi-ai/README.i18n.yaml index 31e6ac3b8a..42364c1d5c 100644 --- a/packages/llm/llm-pi-ai/README.i18n.yaml +++ b/packages/llm/llm-pi-ai/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md -README.md: 6120f8d982c6d475cd508e6cf9e41cabfc9ba159 -README.zh.md: 4b47976c6c6c67968b5b93edbdfd5dfa9530eb1d +README.md: 43472de90803481deebb9bc91586a4b85e443db5 +README.zh.md: 76b3dc9dec2a4bf83933319aa065954a191e595b diff --git a/packages/llm/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md index 6120f8d982..43472de908 100644 --- a/packages/llm/llm-pi-ai/README.md +++ b/packages/llm/llm-pi-ai/README.md @@ -8,7 +8,7 @@ The package root exposes the Cordis plugin contract, `PiAiAdapter`, and `support ## Config -Configure credentials, the model catalog, and deployment-specific transport settings per provider, keyed by the provider route itself. `apiKeyEnv` is a credential *reference* resolved per request, so no secret enters this file. Omitting it leaves the route unauthenticated, which for an installed catalog route means pi-ai's provider-native ambient discovery; a configured reference that resolves to nothing fails the request with `MISSING_CREDENTIAL` instead, because falling through would authenticate with whatever unrelated key the environment happens to hold. One credential serves every model on its route. +Configure credentials, the model catalog, and deployment-specific transport settings per provider, keyed by the provider route itself. Each profile may set a `retryPolicy`; omission uses normal mode with five retries. `apiKeyEnv` is a credential *reference* resolved per request, so no secret enters this file. Omitting it leaves the route unauthenticated, which for an installed catalog route means pi-ai's provider-native ambient discovery; a configured reference that resolves to nothing fails the request with `MISSING_CREDENTIAL` instead, because falling through would authenticate with whatever unrelated key the environment happens to hold. One credential serves every model on its route. ```yaml - id: llm @@ -20,6 +20,9 @@ Configure credentials, the model catalog, and deployment-specific transport sett apiKeyEnv: OPENAI_API_KEY baseURL: https://proxy.example.com:8443 reasoning: high + requestImagePixelBudget: 4194304 # total pixels; 2048 by 2048 default + requestImageMaxBytes: 1048576 # raw bytes before base64 expansion + maxRequestImageBytes: 20971520 # accumulated base64 payload retryPolicy: mode: normal maxRetries: 3 @@ -51,9 +54,12 @@ Configure credentials, the model catalog, and deployment-specific transport sett apiKeyEnv: ACME_GATEWAY_API_KEY api: openai-completions baseURL: https://gateway.acme.example/v1 - # Reasoning dialect for an endpoint whose URL pi-ai cannot recognize. + # Request shape for an endpoint whose URL pi-ai cannot recognize; it + # would otherwise be addressed as though it were OpenAI itself. compat: thinkingFormat: deepseek + supportsDeveloperRole: false + maxTokensField: max_tokens models: - id: acme-large name: Acme Large @@ -85,9 +91,13 @@ A profile's `models` list *replaces* the route's installed catalog rather than e The declaration translates to pi-ai's `Model.reasoning` + `thinkingLevelMap` with every level decided explicitly — undeclared levels are pinned unsupported rather than left to pi-ai's own defaulting, which is asymmetric (an absent key means "supported" for the five base levels but "unsupported" for `xhigh`/`max`) and which a profile author should not need to know. `off` is the one three-state key: left out, selectors offer no Off and an explicit Off request is refused — a request naming no effort still goes out without the parameter, so what the provider then does is its own default; declared with no value (`off:`), Off is offered and selecting it sends nothing — for the `deepseek` dialect an explicit `thinking: {type: "disabled"}` — which also covers a request naming no effort at all; declared with a value (`off: none`), that value goes on the wire as the effort parameter. There is no spelling for restoring a catalog map key to "unset": the declaration is the whole offer, so restate the catalog levels you keep. -### Reasoning-dispatch compat switches +### Wire-compatibility switches -How a thinking level travels — `reasoning_effort` alone, DeepSeek's `thinking: {type}` plus effort, z.ai's `thinking` object, and so on — is pi-ai's `compat.thinkingFormat`, which pi-ai guesses from the endpoint URL; a private gateway's URL says nothing, so a DeepSeek-dialect gateway would be spoken to in the OpenAI dialect with no way to correct it. `compat.thinkingFormat` and `compat.supportsReasoningEffort` are therefore configurable on the route (its models' default) and per model (winning per field), resolving model → route → installed catalog entry → pi-ai's URL-derived guess; setting a route-level switch shadows the catalog entry's value for every model on the route, and there is no spelling for handing a field back to the catalog short of restating its value. `thinkingFormat` accepts pi-ai's dispatchable formats except the two `chat-template` variants, which need `chatTemplateKwargs` this configuration does not expose. Both switches exist only on `openai-completions` — the other protocols carry their reasoning shape in the protocol itself — so a model-level switch elsewhere fails resolution, a route-level one skips models of other protocols, and a route with no `openai-completions` model at all is refused. The rest of pi-ai's compat surface (`supportsStore`, `maxTokensField`, …) stays auto-detected and is deliberately not configurable here. +pi-ai shapes each request from the provider id and baseURL: which role carries the system prompt, which field caps output, how a thinking level travels. A private gateway's URL says nothing, and for an endpoint pi-ai does not recognize the detection answers as though it were OpenAI itself — a reasoning model's system prompt goes out as `developer`, the output cap as `max_completion_tokens`, the thinking level as a bare `reasoning_effort` — and most OpenAI-compatible gateways reject at least one of those. `compat` is therefore configurable on the route (its models' default) and per model (winning per field), resolving model → route → installed catalog entry → pi-ai's own detection; a route-level switch shadows the catalog entry's value for every model that reads it, and there is no spelling for handing a field back to the catalog short of restating its value. + +Each switch belongs to the protocols whose pi-ai compat type declares it, and grouping follows the compat *type* rather than the protocol name: the three Responses protocols (`openai-responses`, `azure-openai-responses`, `openai-codex-responses`) share one compat type, so a switch settable on one is settable on all three. `supportsDeveloperRole` is settable on `openai-completions` and on those three; `thinkingFormat` only on `openai-completions`; `supportsTemperature` only on `anthropic-messages`; `supportsStrictMode` also reaches `bedrock-converse-stream`. A model-level switch its protocol does not take fails resolution naming what that protocol does offer; a route-level one lands on the models that read it and skips the rest, and is refused only when no model on the route could read it at all. + +Three kinds of key are refused rather than dropped: one no protocol declares (a misspelling), one pi-ai's installed catalog owns for a named vendor (`openRouterRouting`, `zaiToolStream`, `deferredToolsMode`, `sessionAffinityFormat`, `supportsOpenAIGrammarTools`, `supportsToolSearch`, `supportsExplicitPromptCacheMode`, `supportsToolReferences`, `vercelGatewayRouting`, `sendSessionAffinityHeaders`) — a route needing a vendor's own switch is a catalog route that should be named as such — and one written with no value at all (`supportsDeveloperRole:`), which schemastery passes through as null and which would otherwise replace the installed catalog's value with nothing. The offered set is pinned to pi-ai's four compat types by drift gates, the protocols carrying them are derived from `Model.compat` itself, and each field's type is derived from upstream rather than restated, so an upgrade that adds a field, gives a further protocol a compat type, or widens a value union fails the build until someone classifies it. A model neither the entry nor the installed catalog sizes takes the route's `defaultContextWindow` (262,144) and `defaultMaxTokens` (32,768), so a listing that discloses nothing but ids still yields a serviceable route. Both fallbacks are guesses by construction, which is why they are route fields a deployment whose gateway serves smaller models corrects once rather than constants buried in the adapter; the fallback sizes the model and never becomes a per-request cap. @@ -113,7 +123,7 @@ A model that carries reasoning metadata — from the installed catalog or from i A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`. -Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name. +Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, `maxRequestImageBytes`, `requestImagePixelBudget`, `requestImageMaxBytes`, and `retryPolicy`. Each resolved profile retry policy is captured with that provider route; omission uses the shared bounded normal default of five retries. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Every image route derives a deterministic request version from the provider-independent normalized attachment under `requestImagePixelBudget` (default 2048 by 2048 total pixels) and `requestImageMaxBytes` (default 1MiB raw bytes). Before reading attachments, `maxRequestImageBytes` applies to conservative request-version upper bounds and replaces the oldest over-budget images with fixed text; exact base64 lengths are checked again after retained versions are generated. The 20MiB default can retain fifteen maximum-size 1MiB versions after base64 expansion while leaving request-body headroom. The same version feeds inline base64, and its stable descriptor exposes the attachment id and actual request-image dimensions. Harness app attribution wins a conflicting configured header name. The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`. @@ -133,13 +143,13 @@ Most listings disclose an id and nothing else; `context_window`/`context_length` Each resolution produces one **immutable** snapshot — the profiles plus a `createModels()` collection holding the `Provider` each route built — and every operation captures a whole snapshot before its first `await`. A configuration change builds a *new* collection rather than mutating the one in use: `Models.streamSimple()` resolves its provider lazily, when the stream is first consumed, which is after the credential await, so a mutated collection would let a request that started under one configuration finish under another or fail on a provider that no longer exists. This is what makes the seam's per-step call freeze (`llm.prepareCall()`) hold end to end — switching models mid-reply takes effect on the next step, never inside the one in flight. Requests reach their provider through `Models.streamSimple()`. A catalog route that keeps its catalog protocol **reuses** the installed provider with its model list replaced, because that provider owns API implementations this package cannot reconstruct — Bedrock loads its Smithy module through a separate entry point — so rebuilding it from parts would silently narrow which providers work. Every other route is built by `createProvider()` over the protocol table behind `supportedProtocols()`, whose entries are the same factories pi-ai's own provider factories use. -Credentials never enter that collection. The harness resolves a route's key through its own seam before the request reaches pi-ai and passes it as the request's `apiKey` option, which pi-ai treats as the highest-priority auth override; `Models` therefore holds no credential store, and the harness keeps its fail-loud reference semantics. A route naming no credential resolves as configured-but-keyless and leaves the requirement to the protocol, which is where it actually lives. +A route's `apiKeyEnv` key still resolves through the harness seam before the request reaches pi-ai and rides as the request's `apiKey` option, which pi-ai treats as the highest-priority auth override — the fail-loud reference semantics stay the harness's. Beneath that override the collection carries this plugin's credential store and ambient auth context: a stored sign-in (an OAuth grant, or a key typed into pi-ai's own login prompt) authenticates its route through them and refreshes itself under the store's cross-process lock. The store addresses records as `llm-pi-ai/`, and a hand-declared route key outside that record grammar (uppercase, dots, underscores) reads as holding nothing stored rather than as an addressing error; such a route cannot be signed into — a record write for it refuses with `LlmError('UNSTORABLE_PROVIDER_ID')` — and authenticates through `apiKeyEnv` or ambient provider settings. A route naming no credential at all resolves as configured-but-keyless and leaves the requirement to the protocol, which is where it actually lives. The selected model descriptor supplies the protocol implementation. This includes native API differences such as OpenAI models whose descriptor uses the Responses API rather than Chat Completions; the harness adapter does not hardcode endpoint selection by model name. -Successful assistant responses store a versioned, lossless-JSON replay state beside the provider and model that produced them. At request time, `LlmRuntime` passes replay state only when the historical provider route and target provider route are currently owned by this same `PiAiAdapter` instance. The adapter validates the state and restores pi-ai response ids and provider signatures even when the target provider or model changes; pi-ai then decides which metadata its target API can reuse. History without replay state is translated as foreign provider-neutral content and never impersonates a native pi-ai response. +Successful assistant responses store a versioned, lossless-JSON replay state beside the provider and model that produced them, as a `ReplayEnvelope`: a response-level half (kind, version, API, route, response ids, native stop reason) plus one per-block entry per streamed block carrying that block's signatures. The per-block alignment is what `BlockAssembler` prunes when assembly drops a block (a `max-tokens` tool call), so the stored entries always describe the stored content — the retained blocks keep their signatures. At request time, `LlmRuntime` passes replay state only when the historical provider route and target provider route are currently owned by this same `PiAiAdapter` instance. The adapter validates the state and restores pi-ai response ids and provider signatures even when the target provider or model changes; pi-ai then decides which metadata its target API can reuse. History without replay state is translated as foreign provider-neutral content and never impersonates a native pi-ai response. -If a listener rewrites assembled assistant content, the loop drops replay state before logging the message because its provider metadata no longer describes the content. Invalid versions, malformed metadata, provider/model mismatches between the message and replay state, and content/block mismatches fail explicitly with `LlmError('INVALID_REPLAY_STATE')`. +Durable content is the authoritative record; replay state only restores native fidelity. A stored state this build cannot use — another adapter's kind, another version (including the flat pre-envelope form older logs carry), malformed metadata, provider/model mismatches between the message and replay state, or content/block mismatches — degrades that one assistant message to the same foreign provider-neutral conversion instead of failing the request, and the plugin logs the `INVALID_REPLAY_STATE` diagnostic through its `onReplayDegrade` hook. ## Vocabulary differences @@ -163,15 +173,15 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata #### What the model sees -The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. This package adds no prompt prose. Provider-native replay metadata is restored only when the adapter validates it for the historical content. +The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. Each retained image is preceded by stable text naming its complete attachment id and actual request dimensions. When accumulated base64 image payload exceeds the route's `maxRequestImageBytes`, each offloaded image (oldest first) is replaced by fixed text that tells the model to read the file again when a path is available or ask the user to attach it again. Offloaded normalized attachments are not read or transformed. Provider-native replay metadata is restored only when the adapter validates it for the historical content. #### Token effect -Provider tokenization governs exact input. Conversion adds no model-visible text; replay metadata may let a native API reuse provider-side state. +Provider tokenization governs exact input. Retained images add the stable attachment and coordinate descriptor; the offload placeholder replaces an omitted image's visual tokens. Replay metadata may let a native API reuse provider-side state. #### KV Cache effect -Conversion preserves logical request order without adding text, while the selected provider's serialization and replay state determine reuse. Changing adapter instance, provider, model, or any upstream request token may prevent reuse from the first difference. +Conversion preserves logical request order without adding text, while the selected provider's serialization and replay state determine reuse. Changing adapter instance, provider, model, or any upstream request token may prevent reuse from the first difference. Crossing the image bound rewrites an early message (the newly offloaded image becomes placeholder text), so reuse ends at that message until the offloaded prefix stabilizes. ### Provider response @@ -189,16 +199,17 @@ Recorded response content appends to the next request and does not invalidate it ## Known Limitations and Deferred Work -- **A provider that authenticates through OAuth alone is not offered** — pi-ai resolves OAuth from a *stored* OAuth credential, and this adapter builds its `Models` collection with no credential store and runs no login flow, so every request on such a route fails `Provider is not configured` before it goes out. The configurable-provider directory withholds them; `openai-codex` is the only one the installed catalog ships. A route a settings document already names keeps its entry so a configuration surface can edit or delete it, and `apiKeyEnv` still authenticates it with that key — which for Codex is a token that expires with nothing here to refresh it. -- **Provider-native discovery reads the process environment only** — a route naming no credential defers to the catalog provider's own resolution, which interrogates environment variables (`AZURE_OPENAI_API_KEY`, `AWS_PROFILE`, `AWS_ACCESS_KEY_ID`, and each provider's own set). It reads no local credential directory, so `~/.aws/credentials` without an exported `AWS_PROFILE` resolves as unconfigured, and a value held by the harness credential seam is invisible to it unless the process environment carries it too. +- **`maxRequestImageBytes` counts base64 image payload only** — text, tools, descriptors, and JSON structure ride outside the bound, so it must sit below the gateway's request-body cap with headroom. Offload is a deterministic request projection and is not recorded as a session event. +- **A sign-in lives only in the process that started it** — an authorization attempt is not durable, so reloading the page mid-login abandons it and the human starts over. Signing out is `deleteRecord` on the stored record, which forgets it locally without telling the issuer. +- **Provider-native discovery answers through this plugin's ambient context** — a route naming no credential defers to the catalog provider's own resolution, which asks for environment values (`AZURE_OPENAI_API_KEY`, `AWS_PROFILE`, and each provider's own set) and for local credential files. Both questions are answered here: the credential seam is consulted before the process environment, and file existence is checked against the host process's filesystem with `~` expanded. What it cannot do is *read* a credential file's contents — a provider that parses `~/.aws/credentials` itself does so directly, outside the seam. - **Settings can add or override routes, not remove composition routes** — the user layer merges over the composition `base`, so deleting a `cordis.yml`-provided provider is a composition change; `replace` on the namespace only resets the user layer. - **The layered merge has no delete for dict keys** — the settings seam merges the composition `base` and the user layer per key, recursively, so a `reasoningEfforts` level, `modelOverrides` entry, or `compat` field the base declares cannot be removed by the user layer, only overridden — and for `reasoningEfforts` absence *is* the meaning ("not offered"), so a base-declared level stays offered. This only triggers when a `cordis.yml` entry config declares per-model reasoning fields for the same model the user layer edits; the supported posture is to leave those to the settings document (the shipped composition mounts the adapter dormant), and a `models` list is an array replacing wholesale, which is the in-band escape. - **`headers` can carry a credential the redactor never sees** — the profile's `headers` dict is plain strings, so `Authorization` or `api-key` set there is returned verbatim by a redacted `describe()` and rendered by any configuration UI. Store credentials as `apiKeyEnv` references; making the dict write-only is deferred with the rest of the [wire-boundary work](../llm/README.md#known-limitations-and-deferred-work). - **A route's catalog never refreshes itself** — the catalog is whatever `settings.yaml` says, so a model list is only as current as its last edit. Nothing here queries a provider for the models it serves; a route gains a model when someone writes one. - **One wire protocol per route** — `api` applies to the whole route, so a mixed-protocol catalog route (an OpenAI-style catalog spanning Responses and Chat Completions) cannot host a model of the other protocol, and adding a model such a route does not describe requires naming `api` and moving every model onto it. Splitting the provider across two route keys is the workaround. -- **A modality declaration is not verified, and over-claiming outlives the turn** — nothing interrogates an endpoint for what it accepts, so a model declaring `image` its gateway does not serve is refused by the provider mid-turn rather than here. Prompt admission commits the user message durably before the request is built, so the rejected image stays in the session log: that model keeps re-sending it, and model selection refuses a switch to any text-only model. Recovery is another image-capable model, a fork before the image, or a new session; rolling an unconsumed image message back out of the log on a failed send is deferred. +- **A modality declaration is not verified** — nothing interrogates an endpoint for what it accepts, so a model declaring `image` its gateway does not serve is refused by the provider after prompt admission. The durable image remains in history and the same misdeclared model can fail again. Switching to a text-only model remains possible because the shared LLM runtime projects image references into stable text for that exact request. - **An unauthenticated route depends on its protocol** — naming no credential resolves the route as configured-but-keyless, but pi-ai's OpenAI-compatible implementation still requires an API key or an `Authorization` header, so a keyless local server needs a placeholder credential referenced by `apiKeyEnv` or an `Authorization` entry in `headers`. - **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field. - **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override. - **Provider HTTP status is unavailable** — pi-ai error events do not expose a stable HTTP status across providers; failures expose only stable harness error codes. -- **Retry policy is provider-owned, not an SDK retry** — each provider profile may configure nested `retryPolicy`, which `dsh-llm-retry` executes at the agent failed-step extension point; pi-ai SDK retries stay disabled so durable agent steps and `llm/retry` events own every visible attempt, and direct `ctx.llm.stream()` calls remain single-attempt. +- **Retry policy is provider-owned, not an SDK retry** — each provider profile may supply nested `retryPolicy`; omission resolves to normal mode with five retries, and the effective route policy is what `dsh-llm-retry` executes at the agent failed-step extension point. pi-ai SDK retries stay disabled so durable agent steps and `llm/retry` events own every visible attempt, and direct `ctx.llm.stream()` calls remain single-attempt. diff --git a/packages/llm/llm-pi-ai/README.zh.md b/packages/llm/llm-pi-ai/README.zh.md index 4b47976c6c..76b3dc9dec 100644 --- a/packages/llm/llm-pi-ai/README.zh.md +++ b/packages/llm/llm-pi-ai/README.zh.md @@ -8,7 +8,7 @@ ## 配置 -按提供方配置凭据、模型 catalog 与部署特定传输设置,并以提供方路由本身为键。`apiKeyEnv` 是按请求解析的凭据*引用*,因此机密不进入该文件。省略它会让该路由处于未认证状态;对已安装 catalog 路由而言,这意味着交给 pi-ai 的提供方原生环境发现。已配置却解析不出任何值的引用则相反,会让请求以 `MISSING_CREDENTIAL` 失败,因为放行下去就会用环境里恰好持有的某个无关密钥完成认证。一条凭据服务该路由下的全部模型。 +按提供方配置凭据、模型 catalog 与部署特定传输设置,并以提供方路由本身为键。每个 profile 都可以设置 `retryPolicy`;省略时使用 normal 模式并重试五次。`apiKeyEnv` 是按请求解析的凭据*引用*,因此机密不进入该文件。省略它会让该路由处于未认证状态;对已安装 catalog 路由而言,这意味着交给 pi-ai 的提供方原生环境发现。已配置却解析不出任何值的引用则相反,会让请求以 `MISSING_CREDENTIAL` 失败,因为放行下去就会用环境里恰好持有的某个无关密钥完成认证。一条凭据服务该路由下的全部模型。 ```yaml - id: llm @@ -20,6 +20,9 @@ apiKeyEnv: OPENAI_API_KEY baseURL: https://proxy.example.com:8443 reasoning: high + requestImagePixelBudget: 4194304 # total pixels; 2048 by 2048 default + requestImageMaxBytes: 1048576 # raw bytes before base64 expansion + maxRequestImageBytes: 20971520 # accumulated base64 payload retryPolicy: mode: normal maxRetries: 3 @@ -51,9 +54,12 @@ apiKeyEnv: ACME_GATEWAY_API_KEY api: openai-completions baseURL: https://gateway.acme.example/v1 - # Reasoning dialect for an endpoint whose URL pi-ai cannot recognize. + # Request shape for an endpoint whose URL pi-ai cannot recognize; it + # would otherwise be addressed as though it were OpenAI itself. compat: thinkingFormat: deepseek + supportsDeveloperRole: false + maxTokensField: max_tokens models: - id: acme-large name: Acme Large @@ -85,9 +91,13 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog,而不是扩 该声明会转换为 pi-ai 的 `Model.reasoning` + `thinkingLevelMap`,其中每个档位都被显式决定——未声明的档位一律固定为不支持,而不是留给 pi-ai 自己的默认规则:那套规则并不对称(键缺席对五个基础档位意味着「支持」,对 `xhigh`/`max` 却意味着「不支持」),也本不该要求 profile 作者了解。`off` 是唯一的三态键:不写它,选择器不提供 Off,显式请求 Off 会被拒绝——不点名任何档位的请求仍会在不带该参数的情况下发出,提供方随后做什么是它自己的默认行为;声明而不给值(`off:`),则会提供 Off,选中它时什么也不发送——对 `deepseek` 方言则是一个显式的 `thinking: {type: "disabled"}`——这同时覆盖完全不点名任何档位的请求;声明并给值(`off: none`),该值就会作为档位参数在协议中发送。没有任何写法能把 catalog 映射中的键恢复为「未设置」:这份声明就是对外提供的全部,因此把你要保留的 catalog 档位重述出来。 -### 推理分派的 compat 开关 +### 协议兼容开关 -思考级别如何在协议中传输——单独一个 `reasoning_effort`、DeepSeek 的 `thinking: {type}` 加上档位、z.ai 的 `thinking` 对象,诸如此类——就是 pi-ai 的 `compat.thinkingFormat`,pi-ai 会从端点 URL 猜测它;私有网关的 URL 什么也说明不了,于是说 DeepSeek 方言的网关只会收到 OpenAI 方言的请求,且无从更正。因此 `compat.thinkingFormat` 与 `compat.supportsReasoningEffort` 既可配置在路由上(作为其模型的默认值),也可按模型配置(逐字段胜出),解析顺序为模型 → 路由 → 已安装 catalog 条目 → pi-ai 按 URL 得出的猜测;设置路由级开关会为路由上的每个模型遮蔽 catalog 条目的值,而且除了重述其值,没有任何写法能把某个字段交还给 catalog。`thinkingFormat` 接受 pi-ai 可分派的各种格式,但不含两个 `chat-template` 变体:它们需要的 `chatTemplateKwargs` 本配置并不暴露。两个开关都只存在于 `openai-completions` 上——其余协议的推理形状由协议本身承载——因此在其他协议的模型上设置模型级开关会使解析失败,路由级开关会跳过其他协议的模型,而完全没有 `openai-completions` 模型的路由则会被拒绝。pi-ai compat 面的其余部分(`supportsStore`、`maxTokensField`……)保持自动检测,特意不在此处开放配置。 +pi-ai 依据提供方 id 与 baseURL 决定每个请求的形状:系统提示词由哪个角色承载、输出上限写在哪个字段、思考级别如何传输。私有网关的 URL 什么也说明不了,而对于 pi-ai 无法识别的端点,其检测会当作 OpenAI 本身来回答——推理模型的系统提示词以 `developer` 发出、输出上限写作 `max_completion_tokens`、思考级别只发一个裸的 `reasoning_effort`——而多数 OpenAI 兼容网关至少会拒绝其中之一。因此 `compat` 既可配置在路由上(作为其模型的默认值),也可按模型配置(逐字段胜出),解析顺序为模型 → 路由 → 已安装 catalog 条目 → pi-ai 自身的检测;路由级开关会为每个读取它的模型遮蔽 catalog 条目的值,而且除了重述其值,没有任何写法能把某个字段交还给 catalog。 + +每个开关归属于其 pi-ai compat 类型声明了它的那些协议,且归组依据是 compat **类型**而非协议名:三个 Responses 协议(`openai-responses`、`azure-openai-responses`、`openai-codex-responses`)共用同一个 compat 类型,因此可设在其中之一的开关,三者皆可设。`supportsDeveloperRole` 可设在 `openai-completions` 与这三者上;`thinkingFormat` 只能设在 `openai-completions`;`supportsTemperature` 只能设在 `anthropic-messages`;`supportsStrictMode` 还可达 `bedrock-converse-stream`。模型级开关若其协议并不接受,解析失败并点名该协议实际提供哪些开关;路由级开关则落在读取它的模型上、跳过其余模型,只有当路由上没有任何模型能读取它时才被拒绝。 + +三类键会被拒绝而非丢弃:没有任何协议声明的键(笔误);pi-ai 已安装 catalog 为具名厂商掌管的键(`openRouterRouting`、`zaiToolStream`、`deferredToolsMode`、`sessionAffinityFormat`、`supportsOpenAIGrammarTools`、`supportsToolSearch`、`supportsExplicitPromptCacheMode`、`supportsToolReferences`、`vercelGatewayRouting`、`sendSessionAffinityHeaders`)——需要某厂商专属开关的路由,本就是一条应当以该厂商命名的 catalog 路由;以及完全没有写值的键(`supportsDeveloperRole:`),schemastery 会把它放行为 null,若照单收下就会用空值替换已安装 catalog 的值。开放集由漂移门禁钉在 pi-ai 的四个 compat 类型上,承载它们的协议集派生自 `Model.compat` 本身,每个字段的类型也派生自上游而非重述,因此上游新增字段、给别的协议加上 compat 类型、或拓宽某个值并集,都会使构建失败,直到有人为它做出分类。 条目与已安装 catalog 都没有给出尺寸的模型,会采用该路由的 `defaultContextWindow`(262,144)与 `defaultMaxTokens`(32,768),因此一份只公布 id 的列表同样能产出可服务的路由。两个回退值本质上都是猜测,这正是它们作为路由字段、供网关服务更小模型的部署一次性更正的原因,而不是埋在适配器里的常量;回退值只用于给模型定尺寸,绝不会变成单次请求上限。 @@ -114,7 +124,7 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog,而不是扩 **没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型,以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档,但 `off` 会被翻译成*省略* reasoning 选项,而那与「不点名任何档位」产出的请求逐字节相同:选它关不掉任何东西,于是自身默认就在思考的提供方,会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用,界面就只剩提供方默认这一项,不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值(包括 `off`)在存在时是部署默认值;省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先;未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败,而不会被自动调整。**描述**一个模型则从不这样失败:同一提供方下各模型接受的档位并不一致,因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」,而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处,而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。 -受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获;省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。若已配置标头中有同名项,则以 Harness 应用归因为准。 +受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs`、`maxRequestImageBytes`、`requestImagePixelBudget`、`requestImageMaxBytes` 和 `retryPolicy`。每条 profile 解析后的重试策略会随该提供方路由一同捕获;省略时使用共享的有界 normal 默认值并重试五次。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。每条图片路由从提供方无关的规范化附件派生确定性请求版本,受 `requestImagePixelBudget`(默认总像素 2048×2048)和 `requestImageMaxBytes`(默认原始字节 1MiB)约束。读取附件前,`maxRequestImageBytes` 先按请求版本的保守上界替换超预算的最旧图片;保留版本生成后再用确切 base64 长度检查。20MiB 默认值可保留十五个按 1MiB 上限生成的请求版本,并为请求正文留下余量。同一版本用于内联 base64,其稳定描述会公开附件 ID 和实际请求图片尺寸。若已配置标头中有同名项,则以 Harness 应用归因为准。 适配器强制 pi-ai SDK `maxRetries` 为零,因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败,而不是静默倍增或隐藏单独组合的 agent(智能体)级重试预算。空闲超时会 abort SDK 的稳定请求信号,并以 `TIMEOUT` 呈现;较早的调用方 abort 仍为 `ABORTED`。 @@ -134,13 +144,13 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog,而不是扩 每次解析产出一份**不可变**快照——profiles 加上一个持有各路由所建 `Provider` 的 `createModels()` 集合——每个操作都在自己第一个 `await` 之前整体捕获一份快照。配置变化会构造**新**集合,而不是改动正在被使用的那个:`Models.streamSimple()` 是惰性的,它在流首次被消费时才解析 provider,而那已在 credential await 之后,因此改动共享集合会让一个在旧配置下开始的请求在新配置下结束,或者撞上一个已不存在的 provider。这正是 seam 的每步调用冻结(`llm.prepareCall()`)能贯通到底的原因——回复途中切换模型会在下一步生效,绝不会影响在途的那一步。请求经 `Models.streamSimple()` 抵达提供方。保持 catalog 协议不变的 catalog 路由会**复用**已安装提供方,只替换其模型列表,因为该提供方持有本包无法重建的 API 实现——Bedrock 经由独立入口加载其 Smithy 模块——从零件重建会静默收窄可用提供方的范围。其余路由都由 `createProvider()` 基于 `supportedProtocols()` 背后的协议表构造,表中条目正是 pi-ai 自己的提供方工厂所用的同一批 factory。 -凭据绝不进入该集合。harness 在请求抵达 pi-ai 之前经自身 seam 解析路由密钥,并作为请求的 `apiKey` 选项传入,而 pi-ai 将其视为优先级最高的 auth 覆盖;因此 `Models` 不持有任何凭据存储,harness 也保住了自己明确失败的引用语义。没有点名任何凭据的路由会解析为「已配置但无密钥」,把该要求留给协议——那才是它真正所在的位置。 +路由的 `apiKeyEnv` 密钥仍在请求抵达 pi-ai 之前经 harness 自身 seam 解析,并作为请求的 `apiKey` 选项传入——pi-ai 将其视为优先级最高的 auth 覆盖,harness 因此保住自己明确失败的引用语义。在该覆盖之下,集合携带本插件的凭据存储与 ambient auth context:已存储的登录(OAuth grant,或在 pi-ai 自己的登录提示里键入的密钥)经由它们为路由完成认证,并在存储的跨进程锁下自行刷新。存储以 `llm-pi-ai/` 为记录地址;手写路由键若落在记录文法之外(大写、点、下划线),读取时视为「没有存储任何东西」而不是寻址错误——这样的路由无法登录(对它的记录写入会以 `LlmError('UNSTORABLE_PROVIDER_ID')` 拒绝),只能经 `apiKeyEnv` 或提供方 ambient 设置认证。完全没有点名任何凭据的路由会解析为「已配置但无密钥」,把该要求留给协议——那才是它真正所在的位置。 所选模型 descriptor 提供协议实现。这包括原生 API 差异,例如 descriptor 使用 Responses API 而非 Chat Completions 的 OpenAI 模型;harness 适配器不会按模型名称硬编码端点选择。 -成功的 assistant 响应会将经版本化的无损 JSON 回放状态与生成该响应的提供方和模型一同存储。请求时,`LlmRuntime` 只有在历史提供方路由与目标提供方路由当前由同一个 `PiAiAdapter` 实例拥有时,才会传递回放状态。即使目标提供方或模型改变,适配器也会验证状态并恢复 pi-ai 响应 id 与提供方 signature;随后由 pi-ai 判定目标 API 可以复用哪些元数据。没有回放状态的历史会被转换为外来的、与提供方无关的内容,绝不伪装为原生 pi-ai 响应。 +成功的 assistant 响应会将经版本化的无损 JSON 回放状态与生成该响应的提供方和模型一同存储,其形式是 `ReplayEnvelope`:一个响应级半区(kind、版本、API、路由、响应 id、原生停止原因),加上每个流式块一条、携带该块 signature 的逐块条目。逐块对齐正是 `BlockAssembler` 在组装丢弃某个块(`max-tokens` 下的工具调用)时裁剪的对象,因此存储的条目始终描述存储的内容——保留的块保有其 signature。请求时,`LlmRuntime` 只有在历史提供方路由与目标提供方路由当前由同一个 `PiAiAdapter` 实例拥有时,才会传递回放状态。即使目标提供方或模型改变,适配器也会验证状态并恢复 pi-ai 响应 id 与提供方 signature;随后由 pi-ai 判定目标 API 可以复用哪些元数据。没有回放状态的历史会被转换为外来的、与提供方无关的内容,绝不伪装为原生 pi-ai 响应。 -如果 listener 改写已组装 assistant 内容,loop 会在记录消息前丢弃回放状态,因为其提供方元数据不再描述该内容。无效版本、格式错误元数据、消息与回放状态之间的提供方/模型不匹配,以及内容/块不匹配都会显式以 `LlmError('INVALID_REPLAY_STATE')` 失败。 +持久化内容是权威记录;回放状态只负责恢复原生保真度。当前构建无法使用的已存状态——其他适配器的 kind、其他版本(包括旧日志携带的平铺前信封形式)、格式错误的元数据、消息与回放状态之间的提供方/模型不匹配,或内容/块不匹配——会把这一条 assistant 消息降级为同样的外来提供方无关转换而不是让请求失败,插件通过其 `onReplayDegrade` 钩子记录 `INVALID_REPLAY_STATE` 诊断。 ## 词汇差异 @@ -152,7 +162,7 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog,而不是扩 ## 应用归因 -每个请求都携带 dsh-llm `attributionHeaders()` 的共享归因标头,并通过 pi-ai `headers` 流选项合并。不会合成提供方特定应用归因标头。详见 [dsh-llm § 应用归因](../llm/README.md#app-attribution-attributionts)。 +每个请求都携带 dsh-llm `attributionHeaders()` 的共享归因标头,并通过 pi-ai `headers` 流选项合并。不会合成提供方特定应用归因标头。详见 [dsh-llm § 应用归因](../llm/README.zh.md#app-attribution-attributionts)。 ## 依赖体量 @@ -164,15 +174,15 @@ pi-ai 会安装多个提供方 SDK,并延迟加载 catalog 模型所选的 SDK #### 模型看到的内容 -所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具,以及 pi-ai 通用流式 API 支持的采样字段。本包不添加提示词文本。只有当适配器验证提供方原生回放元数据与历史内容匹配时,才会恢复这些元数据。 +所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具,以及 pi-ai 通用流式 API 支持的采样字段。每张保留图片前都有稳定文本,写明完整附件 ID 和实际请求尺寸。请求累积的 base64 图片载荷超过路由的 `maxRequestImageBytes` 时,被 offload 的图片会从最老开始替换为固定文本,要求模型在有路径时重新读取文件,否则请用户重新附上图片。系统不会读取或转换被 offload 的规范化附件。只有当适配器验证提供方原生回放元数据与历史内容匹配时,才会恢复这些元数据。 #### Token 影响 -精确输入取决于提供方 tokenization。转换不添加模型可见文本;回放元数据可能让原生 API 复用提供方侧状态。 +精确输入取决于提供方 tokenization。保留图片会增加稳定的附件与坐标描述;offload 占位文本替代被省略图片的视觉 token。回放元数据可能让原生 API 复用提供方侧状态。 #### KV Cache 影响 -转换保留逻辑请求顺序,不添加文本;复用取决于所选提供方的序列化与回放状态。更改适配器实例、提供方、模型或任何上游请求 token,都可能使复用从首个出现差异的 token 起失效。 +转换保留逻辑请求顺序,不添加文本;复用取决于所选提供方的序列化与回放状态。更改适配器实例、提供方、模型或任何上游请求 token,都可能使复用从首个出现差异的 token 起失效。跨过图片上限会改写较早的一条消息(新被 offload 的图片变为占位文本),复用在该消息处截止,直到被 offload 的前缀稳定。 ### 提供方响应 @@ -190,16 +200,17 @@ pi-ai 事件会变为 harness 推理、文本、工具调用、usage 与 finish ## 已知限制与暂缓事项 -- **仅以 OAuth 认证的提供方不予提供**:pi-ai 的 OAuth 只从*已存储*的 OAuth 凭据解析,而本适配器构造 `Models` 集合时不注入凭据存储、也不运行登录流程,因此这类路由的每个请求都会在发出之前以 `Provider is not configured` 失败。可配置提供方目录因此不列出它们;已安装 catalog 中只有 `openai-codex` 属于此类。settings 文档已经写过的路由仍保留目录条目,配置界面据此可以编辑或删除;`apiKeyEnv` 也仍能用该密钥完成认证——对 Codex 而言那是一个会过期、且这里没有任何环节会去刷新的 token。 -- **提供方自带的凭据发现只读进程环境**:不指定凭据的路由交由 catalog 提供方自行解析,而它探测的是环境变量(`AZURE_OPENAI_API_KEY`、`AWS_PROFILE`、`AWS_ACCESS_KEY_ID` 以及各提供方自己的那一组)。它不读任何本地凭据目录,因此只有 `~/.aws/credentials` 而未导出 `AWS_PROFILE` 会被解析为未配置;由 harness 凭据 seam 保管的值,除非进程环境里也有,否则对它不可见。 +- **`maxRequestImageBytes` 只统计 base64 图片载荷**:文本、工具、图片描述和 JSON 结构不计入上限,因此该值必须低于网关请求体上限并留出余量。offload 是确定性请求投影,不记录为会话事件。 +- **一次登录只存活于发起它的进程中**:授权尝试不可持久,登录途中刷新页面会丢弃它,人需要重来。登出即对已存储记录执行 `deleteRecord`,它只在本地遗忘而不通知签发方。 +- **提供方自带的凭据发现经由本插件的 ambient context 作答**:不指定凭据的路由交由 catalog 提供方自行解析,它会询问环境值(`AZURE_OPENAI_API_KEY`、`AWS_PROFILE` 以及各提供方自己的那一组)与本地凭据文件是否存在。两类问题都在这里作答:先查凭据 seam 再查进程环境,文件存在性则按宿主进程的文件系统判断并展开 `~`。它做不到的是*读取*凭据文件的内容——自行解析 `~/.aws/credentials` 的提供方是直接读盘的,不经过 seam。 - **settings 能新增或覆盖路由,但不能移除组合路由**:用户层合并在组合 `base` 之上,因此删除 `cordis.yml` 提供的提供方属于组合变更;对该 namespace 执行 `replace` 只会重置用户层。 - **分层合并对字典键没有删除语义**:settings seam 把组合 `base` 与用户层按键递归合并,因此 base 声明的某个 `reasoningEfforts` 档位、`modelOverrides` 条目或 `compat` 字段,用户层只能覆盖、无法移除——而 `reasoningEfforts` 里缺席本身*就是*语义(「不提供」),于是 base 声明过的档位会一直被提供。只有 `cordis.yml` entry config 为用户层正在编辑的同一模型声明了按模型推理字段才会触发;受支持的姿态是把这些字段留给 settings 文档(shipped 组合以 dormant 方式挂载该适配器),且 `models` 列表是数组、整体替换,这是带内的解决办法。 -- **`headers` 可能承载一条脱敏器看不见的凭据**:profile 的 `headers` 是纯字符串字典,因此设在其中的 `Authorization` 或 `api-key` 会被脱敏后的 `describe()` 原样返回,并被任何配置 UI 渲染出来。请把凭据存为 `apiKeyEnv` 引用;把该字典整体改为只写与其余[协议边界工作](../llm/README.md#known-limitations-and-deferred-work)一并暂缓。 +- **`headers` 可能承载一条脱敏器看不见的凭据**:profile 的 `headers` 是纯字符串字典,因此设在其中的 `Authorization` 或 `api-key` 会被脱敏后的 `describe()` 原样返回,并被任何配置 UI 渲染出来。请把凭据存为 `apiKeyEnv` 引用;把该字典整体改为只写与其余[协议边界工作](../llm/README.zh.md#known-limitations-and-deferred-work)一并暂缓。 - **路由的 catalog 不会自我刷新**:catalog 就是 `settings.yaml` 所写的内容,因此模型列表的新鲜度只到最近一次编辑为止。这里没有任何环节会去问提供方它服务哪些模型;路由要多一个模型,得有人写进去。 - **每条路由只有一种协议格式**:`api` 作用于整条路由,因此混合协议的 catalog 路由(跨 Responses 与 Chat Completions 的 OpenAI 式 catalog)无法承载另一种协议的模型,向这类路由添加它未描述的模型必须点名 `api` 并把全部模型一起迁过去。把该提供方拆成两个路由键是变通办法。 -- **模态声明不经验证,且多声明的后果超出本轮**:没有任何环节会去询问端点接受什么,因此声明了网关并不提供的 `image` 的模型不会在这里被拦下,而是由提供方在轮次中途拒绝。prompt 准入在构造请求之前就把用户消息持久化提交,于是被拒绝的图片留在会话日志里:该模型会不断重发它,而模型选择拒绝切换到任何纯文本模型。恢复途径是换一个确实支持图片的模型、fork 到图片之前,或开启新会话;发送失败时把尚未消费的图片消息从日志中回滚出去这件事已暂缓。 +- **模态声明不经验证**:没有任何环节会去询问端点接受什么,因此声明了网关并不提供的 `image` 的模型会在 prompt 准入后被提供方拒绝。持久图片会留在历史中,同一个错误声明的模型可能再次失败。系统仍允许切换到纯文本模型,因为共享 LLM 运行时会在该次请求中把图片引用投影为稳定文本。 - **未认证路由取决于其协议**:不点名凭据会让路由解析为「已配置但无密钥」,但 pi-ai 的 OpenAI 兼容实现仍要求 API key 或 `Authorization` 标头,因此无鉴权的本地服务需要一个由 `apiKeyEnv` 引用的占位凭据,或在 `headers` 中给出 `Authorization` 条目。 - **不支持 `GenerateOptions.stop`**:pi-ai 的通用流选项无法保证所有提供方都支持 stop sequence,因此适配器会拒绝该字段。 - **历史中的 `system` 消息使用 pi-ai 通用上下文转换**:提供方特定位置由 pi-ai 决定,而非由 harness 拥有的协议覆盖决定。 - **无法获取提供方 HTTP 状态**:pi-ai 错误事件不会在所有提供方上公开稳定 HTTP 状态;失败只公开稳定 harness 错误 code。 -- **重试策略由提供方持有,而不是 SDK 重试**:每个提供方 profile 都可以配置嵌套的 `retryPolicy`,由 `dsh-llm-retry` 在 agent 的失败步骤扩展点上执行;pi-ai SDK 重试仍保持禁用,因此持久化的 agent 步骤与 `llm/retry` 事件记录每次可见尝试,直接 `ctx.llm.stream()` 调用仍只尝试一次。 +- **重试策略由提供方持有,而不是 SDK 重试**:每个提供方 profile 都可以提供嵌套的 `retryPolicy`;省略时解析为 normal 模式并重试五次,`dsh-llm-retry` 会在 agent 的失败步骤扩展点上执行有效路由策略。pi-ai SDK 重试仍保持禁用,因此持久化的 agent 步骤与 `llm/retry` 事件记录每次可见尝试,直接 `ctx.llm.stream()` 调用仍只尝试一次。 diff --git a/packages/llm/llm-pi-ai/package.json b/packages/llm/llm-pi-ai/package.json index 5c2845394e..0fd59afaa1 100644 --- a/packages/llm/llm-pi-ai/package.json +++ b/packages/llm/llm-pi-ai/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-llm-pi-ai", "description": "pi-ai-backed DeepSeek adapter for the DeepSeek Harness LLM seam (design-verification twin of dsh-llm-deepseek)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -33,6 +33,7 @@ "license": "MIT", "peerDependencies": { "@deepseek-ai/dsh-attachment": "workspace:^", + "@deepseek-ai/dsh-authorization": "workspace:^", "@deepseek-ai/dsh-credentials": "workspace:^", "@deepseek-ai/dsh-launch-environment": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", @@ -47,6 +48,7 @@ }, "devDependencies": { "@deepseek-ai/dsh-attachment": "workspace:^", + "@deepseek-ai/dsh-authorization": "workspace:^", "@deepseek-ai/dsh-credentials": "workspace:^", "@deepseek-ai/dsh-launch-environment": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", diff --git a/packages/llm/llm-pi-ai/src/adapter.ts b/packages/llm/llm-pi-ai/src/adapter.ts index 66964c5339..f06b27a1ea 100644 --- a/packages/llm/llm-pi-ai/src/adapter.ts +++ b/packages/llm/llm-pi-ai/src/adapter.ts @@ -13,10 +13,15 @@ * way down: switching models mid-reply takes effect on the next step, never * inside the one in flight. * - * Credentials stay outside that collection. The harness resolves a route's key - * through its own seam and passes it as the request's `apiKey` option, which - * pi-ai treats as the highest-priority auth override — so `Models` never holds - * a credential store and the harness keeps its fail-loud reference semantics. + * A route naming a credential reference still resolves it through the harness + * seam and passes it as the request's `apiKey` option, which pi-ai treats as + * the highest-priority auth override — that is what keeps the fail-loud + * reference semantics. Everything that override does not cover reaches pi-ai + * through the collection's own auth: the credential store holds the records a + * login wrote and a refresh rotates, and the auth context answers the ambient + * questions a provider asks while resolving. Both are stable across snapshots, + * so a configuration change rebuilds the collection without forgetting who is + * signed in. * * @module dsh-llm-pi-ai/adapter */ @@ -24,6 +29,8 @@ import { createModels, getSupportedThinkingLevels } from '@earendil-works/pi-ai' import type { Api, + AuthContext, + CredentialStore, Model, Models, ModelThinkingLevel, @@ -43,6 +50,7 @@ import type { LlmModelInfo, LlmProviderInfo, LlmResolvedModelInfo, + PreparedAdapterCall, ReasoningEffortId as ReasoningEffortIdType, ResolvedRetryPolicy, StreamChunk, @@ -74,8 +82,30 @@ export interface PiAiAdapterOptions { * `MISSING_CREDENTIAL` rather than falling back. */ resolveApiKey: (provider: string, profile: ResolvedPiAiProviderProfile) => Promise + /** + * How every collection this adapter builds resolves auth the request-level + * `apiKey` override does not cover. Required rather than optional: a + * collection built without them gets pi-ai's in-memory default store, which + * is empty at every boot and discarded on every configuration change, so a + * route whose only method is a login would report itself unconfigured on + * every request no matter how often the human signed in. + */ + auth: PiAiAuthInjection /** Resolve the optional durable attachment service at request time. */ resolveAttachments?: () => AttachmentStore | undefined + /** + * Observe one assistant history message degrading to provider-neutral + * conversion because its stored replay state is unusable by this build. + */ + onReplayDegrade?: (detail: { provider: string; model: string; reason: string }) => void +} + +/** The two auth injectables a pi-ai collection is built with. */ +export interface PiAiAuthInjection { + /** Durable storage for credentials pi-ai itself writes: logins, and the refreshes it runs under its own lock. */ + credentials: CredentialStore + /** Ambient lookups a provider performs while resolving its own auth. */ + authContext: AuthContext } /** Copy profile stream knobs into pi-ai's common option vocabulary. */ @@ -199,7 +229,7 @@ export class PiAiAdapter extends LlmAdapter { private current(): PiAiSnapshot { const profiles = this.config.profiles() if (this.snapshot?.profiles === profiles) return this.snapshot - const models: MutableModels = createModels() + const models: MutableModels = createModels(this.config.auth) for (const profile of profiles.values()) models.setProvider(profile.piProvider) this.snapshot = { profiles, models } return this.snapshot @@ -255,25 +285,44 @@ export class PiAiAdapter extends LlmAdapter { ): Promise { return Promise.resolve().then(() => { const snapshot = this.current() - const profile = this.profileOf(snapshot, provider) - const resolvedModel = this.modelOf(snapshot, provider, model) - const defaultLevel = describableReasoningLevel(resolvedModel, profile.reasoning) - // Only a cap the deployment configured is a request default; the - // catalog's `maxTokens` sizes the model and stops there. - const configuredMaxTokens = profile.configuredMaxTokens.get(model) - return { - provider, - id: model, - name: resolvedModel.name, - inputModalities: [...resolvedModel.input], - context: { contextWindow: resolvedModel.contextWindow }, - ...configuredMaxTokens === undefined ? {} : { defaultMaxTokens: configuredMaxTokens }, - ...reasoningInfo(resolvedModel, defaultLevel), - } + return this.modelInfo(snapshot, provider, model) }) } - async * stream(options: GenerateOptions): AsyncIterable { + private modelInfo(snapshot: PiAiSnapshot, provider: string, model: string): LlmResolvedModelInfo { + const profile = this.profileOf(snapshot, provider) + const resolvedModel = this.modelOf(snapshot, provider, model) + const defaultLevel = describableReasoningLevel(resolvedModel, profile.reasoning) + // Only a cap the deployment configured is a request default; the + // catalog's `maxTokens` sizes the model and stops there. + const configuredMaxTokens = profile.configuredMaxTokens.get(model) + return { + provider, + id: model, + name: resolvedModel.name, + inputModalities: [...resolvedModel.input], + context: { contextWindow: resolvedModel.contextWindow }, + ...configuredMaxTokens === undefined ? {} : { defaultMaxTokens: configuredMaxTokens }, + ...reasoningInfo(resolvedModel, defaultLevel), + } + } + + override prepareCall(provider: string, model: string, _signal?: AbortSignal): Promise { + const snapshot = this.current() + return Promise.resolve({ + model: this.modelInfo(snapshot, provider, model), + stream: options => this.streamWithSnapshot(options, snapshot), + }) + } + + stream(options: GenerateOptions): AsyncIterable { + return this.streamWithSnapshot(options, this.current()) + } + + private async * streamWithSnapshot( + options: GenerateOptions, + snapshot: PiAiSnapshot, + ): AsyncIterable { if (options.stop !== undefined) { throw new LlmError('llm-pi-ai does not support GenerateOptions.stop', 'UNSUPPORTED_OPTION') } @@ -282,7 +331,6 @@ export class PiAiAdapter extends LlmAdapter { // snapshot, and the credential freezes with them. A configuration change // mid-request builds a separate snapshot, so this request finishes under // the one it started with and the next call picks up the new one. - const snapshot = this.current() const profile = this.profileOf(snapshot, options.provider) const model = this.modelOf(snapshot, options.provider, options.model) const reasoning = resolveReasoningLevel( @@ -307,9 +355,15 @@ export class PiAiAdapter extends LlmAdapter { if (containsImage && attachments === undefined) { throw new LlmError('pi-ai image input requires the durable attachment service', 'UNSUPPORTED_CONTENT') } + const onReplayDegrade = (reason: string): void => { + this.config.onReplayDegrade?.({ provider: options.provider, model: options.model, reason }) + } const context = attachments === undefined - ? toPiContext(options) - : await toPiContext(options, attachments) + ? toPiContext(options, undefined, onReplayDegrade) + : await toPiContext({ ...options, signal: watchdog.signal }, attachments, onReplayDegrade, profile.maxRequestImageBytes, { + maxPixels: profile.requestImagePixelBudget, + maxBytes: profile.requestImageMaxBytes, + }) const events = snapshot.models.streamSimple(model, context, { ...profileOptions(profile, reasoning, apiKey), ...options.temperature === undefined ? {} : { temperature: options.temperature }, diff --git a/packages/llm/llm-pi-ai/src/auth.ts b/packages/llm/llm-pi-ai/src/auth.ts new file mode 100644 index 0000000000..f729a77f85 --- /dev/null +++ b/packages/llm/llm-pi-ai/src/auth.ts @@ -0,0 +1,207 @@ +/** + * The three adapters between pi-ai's auth model and the harness credential + * plane. Every pi-ai-specific concept stays on this side of them: the harness + * seams they consume — `ctx.credentials` records and `ctx.authorization` flows — + * name nothing from this library, so another adapter family can arrive with a + * different auth model and share the same two seams. + * + * @module dsh-llm-pi-ai/auth + */ + +import { homedir } from 'node:os' +import { access } from 'node:fs/promises' +import { resolve as resolvePath } from 'node:path' +import type { AuthContext, Credential, CredentialInfo, CredentialStore } from '@earendil-works/pi-ai' +import type { Context } from '@deepseek-ai/cordis' +import { + credentialKey, credentialKeyId, credentialKeyScope, credentialRef, isCredentialKeySegment, isCredentialRefName, +} from '@deepseek-ai/dsh-credentials' +import type { CredentialKey, CredentialProvider, CredentialRecord } from '@deepseek-ai/dsh-credentials' +import { launchEnvironmentOf } from '@deepseek-ai/dsh-launch-environment' +import { LlmError } from '@deepseek-ai/dsh-llm' + +/** + * The record scope every credential this adapter family stores is written + * under. It is the plugin's registered name, which is what tells a later + * reader — a configuration UI, or a second adapter family serving the same + * provider name — that this plugin owns the format inside the record. + */ +export const RECORD_SCOPE = 'llm-pi-ai' + +/** + * The record address for one pi-ai provider id. + * @param providerId - pi-ai's own provider id, which is also the harness route key. + * @returns the scoped credential key this adapter family reads and writes. + */ +export function recordKeyFor(providerId: string): CredentialKey { + return credentialKey(RECORD_SCOPE, providerId) +} + +/** + * Translate a stored record into the credential pi-ai expects. + * + * An `api-key` record is structural on both sides, so it is rebuilt field by + * field. A `grant` payload is pi-ai's own OAuth credential, stored verbatim: + * the seam treats it as opaque JSON precisely so a library that owns a token + * format keeps owning it, refresh fields and all. + * @param record - the stored record, or undefined when nothing is stored. + * @returns the pi-ai credential, or undefined for an absent record. + */ +function toPiCredential(record: CredentialRecord | undefined): Credential | undefined { + if (record === undefined) return undefined + if (record.kind === 'api-key') { + return { + type: 'api_key', + ...record.key === undefined ? {} : { key: record.key }, + ...record.env === undefined ? {} : { env: { ...record.env } }, + } + } + return record.payload as Credential +} + +/** + * Translate a pi-ai credential into the record to store. + * @param credential - what a login or refresh produced. + * @returns the record to commit, in the union the credential seam stores. + */ +function toRecord(credential: Credential): CredentialRecord { + if (credential.type === 'api_key') { + return { + kind: 'api-key', + ...credential.key === undefined ? {} : { key: credential.key }, + ...credential.env === undefined ? {} : { env: { ...credential.env } }, + } + } + return { kind: 'grant', payload: credential } +} + +/** + * The credential service, or the failure that names what is missing. Reads + * answer "nothing stored" without a service, because a composition with no + * credential plane genuinely holds no credential; writes refuse, because a + * login whose grant silently evaporated would report success and then fail + * every request. + * @param ctx - the plugin context. + * @returns the live service. + * @throws {LlmError} code `NO_CREDENTIAL_STORE` when none is mounted. + */ +function writableStore(ctx: Context): CredentialProvider { + const credentials = ctx.get('credentials') + if (credentials === undefined) { + throw new LlmError( + 'llm-pi-ai: this composition mounts no credentials service, so there is nowhere to store the' + + ' credential a sign-in produces; mount one (dsh-credentials-local) to sign in', + 'NO_CREDENTIAL_STORE', + ) + } + return credentials +} + +/** + * A pi-ai `CredentialStore` over the harness credential records. + * + * pi-ai runs OAuth refresh *inside* `modify()`, so this store's exclusion has + * to cover a network round trip rather than a file rename — which is why the + * record write path takes a wait limit of its own rather than the short one a + * local write would need. + * + * pi-ai asks this store about every provider in the collection, hand-declared + * routes included, and a route key is an arbitrary settings dict key while a + * record id is not. An id outside the record grammar can never have stored a + * record, so reads answer "nothing stored" and a delete has nothing to remove; + * only `modify` refuses it, because a write that cannot land must not report + * that it did. + * @param ctx - the plugin context carrying the optional `ctx.credentials`. + * @returns the store to hand `createModels()`. + */ +export function credentialStoreFrom(ctx: Context): CredentialStore { + return { + async read(providerId) { + const credentials = ctx.get('credentials') + if (credentials === undefined) return undefined + if (!isCredentialKeySegment(providerId)) return undefined + return toPiCredential(await credentials.readRecord(recordKeyFor(providerId))) + }, + async list(): Promise { + const stored = await ctx.get('credentials')?.listRecords() ?? [] + const mine: CredentialInfo[] = [] + for (const entry of stored) { + // Records another plugin owns are not this collection's to report: + // their payloads are written in a format pi-ai never agreed to. + if (credentialKeyScope(entry.key) !== RECORD_SCOPE) continue + mine.push({ + providerId: credentialKeyId(entry.key), + type: entry.kind === 'api-key' ? 'api_key' : 'oauth', + }) + } + return mine + }, + async modify(providerId, mutate) { + if (!isCredentialKeySegment(providerId)) { + throw new LlmError( + `llm-pi-ai: provider id "${providerId}" cannot address a stored credential record (a record id is a` + + ' lowercase hyphenated identifier); authenticate this route through apiKeyEnv instead of a stored' + + ' credential', + 'UNSTORABLE_PROVIDER_ID', + ) + } + const stored = await writableStore(ctx).modifyRecord(recordKeyFor(providerId), async (current) => { + const next = await mutate(toPiCredential(current)) + return next === undefined ? undefined : toRecord(next) + }) + return toPiCredential(stored) + }, + // `async` so a missing service reaches the caller as a rejection: pi-ai's + // store contract is promise-returning, and a synchronous throw would + // escape the `ModelsError` wrapper every other storage failure gets. + async delete(providerId) { + if (!isCredentialKeySegment(providerId)) return + await writableStore(ctx).deleteRecord(recordKeyFor(providerId)) + }, + } +} + +/** + * A pi-ai `AuthContext` over the harness credential plane and the host + * filesystem. + * + * `env()` answers from the credential seam first, so a value a deployment + * stored through the harness is found by a provider's own ambient discovery — + * without this, that discovery reads only the process environment and a stored + * `AWS_ACCESS_KEY_ID` is invisible to it. `fileExists()` answers about the host + * process's own filesystem rather than the workspace `ctx.fs` seam, because the + * paths it is asked about (`~/.aws/credentials`, application-default + * credentials) are facts about where this process runs, not about the project + * under edit. + * @param ctx - the plugin context carrying the optional `ctx.credentials`. + * @returns the auth context to hand `createModels()`. + */ +export function authContextFrom(ctx: Context): AuthContext { + return { + async env(name) { + // pi-ai asks about arbitrary provider-declared names; one that is not a + // POSIX identifier can never have been stored as a reference, and asking + // the seam would throw instead of answering "not set". + if (isCredentialRefName(name)) { + const credentials = ctx.get('credentials') + const hit = await credentials?.resolve(credentialRef(name)) + if (hit !== undefined) return hit.value + } + return launchEnvironmentOf(ctx).get(name)?.value + }, + async fileExists(path) { + const expanded = path.startsWith('~/') || path === '~' + ? resolvePath(homedir(), path.slice(1).replace(/^\//, '')) + : path + try { + await access(expanded) + return true + } catch { + // Absent, unreadable, or a broken symlink — every one of which means + // this ambient credential source cannot be used, which is the only + // distinction the caller makes. + return false + } + }, + } +} diff --git a/packages/llm/llm-pi-ai/src/catalog.ts b/packages/llm/llm-pi-ai/src/catalog.ts index 7a4ff7e8a2..1e9b126dd5 100644 --- a/packages/llm/llm-pi-ai/src/catalog.ts +++ b/packages/llm/llm-pi-ai/src/catalog.ts @@ -15,11 +15,16 @@ import { builtinProviders, getBuiltinModels, getBuiltinProviders } from '@earendil-works/pi-ai/providers/all' import type { BuiltinProvider } from '@earendil-works/pi-ai/providers/all' import type { + AnthropicMessagesCompat, Api, + BedrockCompat, + ChatTemplateKwargValue, + KnownApi, Model, ModelCost, ModelThinkingLevel, OpenAICompletionsCompat, + OpenAIResponsesCompat, Provider, ThinkingLevelMap, } from '@earendil-works/pi-ai' @@ -79,23 +84,16 @@ const THINKING_LEVEL_GATE: Record = { /** Every pi-ai thinking level a profile may declare, in escalation order. */ export const THINKING_LEVELS = Object.keys(THINKING_LEVEL_GATE) as readonly ModelThinkingLevel[] -/** The `compat.thinkingFormat` spellings pi-ai accepts on an `openai-completions` model. */ -type PiThinkingFormat = NonNullable - -/** - * pi-ai thinking formats a profile cannot name: both drive the request through - * `chatTemplateKwargs`, which this configuration does not expose. - */ -type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template' - /** One reasoning-dispatch wire format a profile may name. */ -export type PiAiThinkingFormat = Exclude +export type PiAiThinkingFormat = NonNullable /** * The nameable reasoning-dispatch formats, most-reached first. The `Record` * key type is a drift gate: a pi-ai upgrade that adds a format (0.84 added - * `baseten`) fails compilation here until the format is classified as offered - * here or withheld above, so the offer never silently lags the upstream set. + * `baseten`) fails compilation here until the new format is named, so the + * offer never silently lags the upstream set. The two `chat-template` variants + * are nameable because {@link PiAiCompatProfile.chatTemplateKwargs} carries + * the kwargs they dispatch through. */ const THINKING_FORMAT_GATE: Record = { 'openai': true, @@ -104,6 +102,8 @@ const THINKING_FORMAT_GATE: Record = { 'together': true, 'zai': true, 'qwen': true, + 'chat-template': true, + 'qwen-chat-template': true, 'string-thinking': true, 'ant-ling': true, } @@ -111,6 +111,41 @@ const THINKING_FORMAT_GATE: Record = { /** Reasoning-dispatch wire formats a profile may name, most-reached first. */ export const SUPPORTED_THINKING_FORMATS = Object.keys(THINKING_FORMAT_GATE) as readonly PiAiThinkingFormat[] +/** The output-cap field spellings pi-ai accepts. */ +export type PiAiMaxTokensField = NonNullable + +/** Drift gate over {@link PiAiMaxTokensField}; an upstream spelling added here fails compilation until named. */ +const MAX_TOKENS_FIELD_GATE: Record = { + max_completion_tokens: true, + max_tokens: true, +} + +/** The output-cap field spellings a profile may name. */ +export const MAX_TOKENS_FIELDS = Object.keys(MAX_TOKENS_FIELD_GATE) as readonly PiAiMaxTokensField[] + +/** The prompt-cache marker conventions pi-ai accepts. */ +export type PiAiCacheControlFormat = NonNullable + +/** Drift gate over {@link PiAiCacheControlFormat}; a new upstream convention fails compilation until named. */ +const CACHE_CONTROL_FORMAT_GATE: Record = { + anthropic: true, +} + +/** The prompt-cache marker conventions a profile may name. */ +export const CACHE_CONTROL_FORMATS = Object.keys(CACHE_CONTROL_FORMAT_GATE) as readonly PiAiCacheControlFormat[] + +/** The request-state placeholders a `chat_template_kwargs` value may name. */ +export type PiAiChatTemplateVar = Extract['$var'] + +/** Drift gate over {@link PiAiChatTemplateVar}; a new upstream placeholder fails compilation until named. */ +const CHAT_TEMPLATE_VAR_GATE: Record = { + 'thinking.enabled': true, + 'thinking.effort': true, +} + +/** The request-state placeholders a profile may name. */ +export const CHAT_TEMPLATE_VARS = Object.keys(CHAT_TEMPLATE_VAR_GATE) as readonly PiAiChatTemplateVar[] + let providerIndex: Map | undefined /** @@ -141,26 +176,6 @@ export function catalogProviderIds(): readonly string[] { return getBuiltinProviders() } -/** - * Whether the installed catalog provider for one route declares an api-key - * method — the only authentication this adapter obtains on its own. - * - * A key is what the harness resolves through its own credential seam and hands - * pi-ai per request. pi-ai's other method, OAuth, resolves from a *stored* - * OAuth credential alone: `resolveProviderAuth` has no ambient path for it, - * this adapter builds its `Models` collection with no credential store, and - * nothing here runs a login flow. So a provider offering OAuth by itself - * leaves nothing for this adapter to authenticate with, and the posture such a - * provider invites — no key configured, credentials discovered by the provider - * — fails every request with `Provider is not configured`. - * @param provider - provider route key. - * @returns whether the catalog provider takes an api key; false for a route - * pi-ai does not ship, which the caller answers for separately. - */ -export function catalogProviderTakesApiKey(provider: string): boolean { - return catalogProvider(provider)?.auth.apiKey !== undefined -} - /** * The installed catalog models for one route, indexed by model id. * @param provider - provider route key. @@ -183,19 +198,336 @@ export function catalogModels(provider: string): Map> { export type PiAiReasoningEfforts = Partial> /** - * Reasoning-dispatch compatibility switches, set on the route (its models' - * default) or per model (winning over the route). Only the switches pi-ai's - * reasoning dispatch reads are offered; the rest of pi-ai's compat surface - * keeps its baseURL-derived auto-detection. pi-ai types both fields only on - * `OpenAICompletionsCompat` — the other wire protocols define their reasoning - * fields in the protocol itself — so resolution rejects a model-level switch - * anywhere else, while a route-level default skips past models it cannot fit. + * Whether one pi-ai compat field is configurable on a profile. + * + * `withhold` is the disposition for a field pi-ai's installed catalog already + * sets for a named vendor. Reaching for one of those on a hand-declared route + * means configuring a provider that should have been named as a catalog route + * instead, where the installed entry carries the right value already. + */ +type CompatDisposition = 'offer' | 'withhold' + +/** + * Disposition of every `OpenAICompletionsCompat` field. The `Record` key type + * is a drift gate: a pi-ai upgrade that adds a field fails compilation here + * until it is classified, so the offer never silently lags the upstream set. + */ +const COMPLETIONS_COMPAT_GATE = { + supportsStore: 'offer', + supportsDeveloperRole: 'offer', + supportsReasoningEffort: 'offer', + supportsUsageInStreaming: 'offer', + maxTokensField: 'offer', + requiresToolResultName: 'offer', + requiresAssistantAfterToolResult: 'offer', + requiresThinkingAsText: 'offer', + requiresReasoningContentOnAssistantMessages: 'offer', + thinkingFormat: 'offer', + chatTemplateKwargs: 'offer', + supportsStrictMode: 'offer', + cacheControlFormat: 'offer', + supportsLongCacheRetention: 'offer', + openRouterRouting: 'withhold', + vercelGatewayRouting: 'withhold', + zaiToolStream: 'withhold', + supportsOpenAIGrammarTools: 'withhold', + sendSessionAffinityHeaders: 'withhold', + deferredToolsMode: 'withhold', + sessionAffinityFormat: 'withhold', +} as const satisfies Record + +/** Disposition of every `OpenAIResponsesCompat` field; a drift gate like the one above. */ +const RESPONSES_COMPAT_GATE = { + supportsDeveloperRole: 'offer', + supportsStrictMode: 'offer', + supportsLongCacheRetention: 'offer', + sessionAffinityFormat: 'withhold', + supportsOpenAIGrammarTools: 'withhold', + supportsToolSearch: 'withhold', + supportsExplicitPromptCacheMode: 'withhold', +} as const satisfies Record + +/** Disposition of every `AnthropicMessagesCompat` field; a drift gate like the one above. */ +const ANTHROPIC_COMPAT_GATE = { + supportsEagerToolInputStreaming: 'offer', + supportsLongCacheRetention: 'offer', + supportsCacheControlOnTools: 'offer', + supportsTemperature: 'offer', + forceAdaptiveThinking: 'offer', + allowEmptySignature: 'offer', + supportsStrictTools: 'offer', + sendSessionAffinityHeaders: 'withhold', + supportsToolReferences: 'withhold', +} as const satisfies Record + +/** Disposition of every `BedrockCompat` field; a drift gate like the one above. */ +const BEDROCK_COMPAT_GATE = { + supportsStrictMode: 'offer', +} as const satisfies Record + +/** + * Every wire protocol pi-ai gives a compat type. Derived from `Model.compat`'s + * own conditional rather than listed by hand, so a pi-ai release that gives a + * further protocol a compat type fails the {@link COMPAT_GATES} entry list + * until someone classifies its fields. A protocol pi-ai gives no compat type + * resolves away here and takes no configured compat at all. + */ +type ApiWithCompat = { [K in KnownApi]: NonNullable['compat']> extends never ? never : K }[KnownApi] + +/** + * The compat gate of every wire protocol a profile may configure. + * + * Keyed by protocol, but grouped by pi-ai's compat *type*: the three Responses + * protocols share `OpenAIResponsesCompat`, so a switch settable on one is + * settable on all three. Keying by protocol alone would refuse + * `azure-openai-responses` and `openai-codex-responses` the fields their own + * models declare. + */ +const COMPAT_GATES: Readonly>>> = { + 'openai-completions': COMPLETIONS_COMPAT_GATE, + 'openai-responses': RESPONSES_COMPAT_GATE, + 'azure-openai-responses': RESPONSES_COMPAT_GATE, + 'openai-codex-responses': RESPONSES_COMPAT_GATE, + 'anthropic-messages': ANTHROPIC_COMPAT_GATE, + 'bedrock-converse-stream': BEDROCK_COMPAT_GATE, +} + +/** + * The compat gate of one resolved protocol. A `string` lookup rather than a + * keyed read: a route's `api` is configuration, so it may name a protocol + * pi-ai gives no compat type — or none at all. + * @param api - resolved wire protocol. + * @returns that protocol's field gate, or `undefined` when it takes no compat. + */ +function compatGate(api: string): Readonly> | undefined { + return (COMPAT_GATES as Readonly>>>)[api] +} + +/** The field names one gate offers. */ +type OfferedIn = { [K in keyof G]: G[K] extends 'offer' ? K : never }[keyof G] + +/** Every compat field name a profile may set, on whichever protocol takes it. */ +type OfferedCompatField = + | OfferedIn + | OfferedIn + | OfferedIn + | OfferedIn + +/** + * pi-ai wire-compatibility switches, set on the route (its models' default) or + * per model (winning over the route, field by field). + * + * pi-ai decides each of these from the provider id and baseURL when no layer + * sets it, and a private gateway's URL says nothing: for an endpoint it does + * not recognize the detection answers as though it were OpenAI itself, which + * is wrong for most OpenAI-compatible gateways. So every field here is one a + * deployment must be able to state because nothing can infer it, while the + * fields pi-ai's catalog sets for a named vendor stay withheld. + * + * A field belongs to the protocols whose upstream compat type declares it: a + * model-level switch its protocol does not take fails resolution, and a + * route-level one skips past models it cannot fit. "The three Responses + * protocols" below means `openai-responses`, `azure-openai-responses`, and + * `openai-codex-responses`, which pi-ai gives one shared compat type, so a + * switch settable on one is settable on all three. */ export interface PiAiCompatProfile { - /** Reasoning parameter format the endpoint expects; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */ - thinkingFormat?: PiAiThinkingFormat - /** Whether the endpoint accepts `reasoning_effort`; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */ + /** Whether the endpoint accepts `store`; `openai-completions`. */ + supportsStore?: boolean + /** + * Whether the endpoint accepts the `developer` role for the system prompt, + * which pi-ai sends only to a reasoning model; `false` keeps `system`. + * `openai-completions` and the three Responses protocols. + */ + supportsDeveloperRole?: boolean + /** Whether the endpoint accepts `reasoning_effort`; `openai-completions`. */ supportsReasoningEffort?: boolean + /** Whether the endpoint accepts `stream_options: {include_usage: true}`; `openai-completions`. */ + supportsUsageInStreaming?: boolean + /** Which output-cap field the endpoint reads; `openai-completions`. */ + maxTokensField?: NonNullable + /** Whether tool results must carry `name`; `openai-completions`. */ + requiresToolResultName?: boolean + /** Whether a user message after tool results needs an assistant message between; `openai-completions`. */ + requiresAssistantAfterToolResult?: boolean + /** Whether thinking blocks must travel as text in `` delimiters; `openai-completions`. */ + requiresThinkingAsText?: boolean + /** Whether replayed assistant messages need an empty `reasoning_content` while reasoning is on; `openai-completions`. */ + requiresReasoningContentOnAssistantMessages?: boolean + /** Reasoning parameter format the endpoint expects; `openai-completions`. */ + thinkingFormat?: PiAiThinkingFormat + /** + * Kwargs sent as `chat_template_kwargs`, which pi-ai reads only under the + * two `chat-template` thinking formats; `openai-completions`. Nothing checks + * that pairing: the format in force may come from the installed catalog + * entry or from pi-ai's own baseURL detection, neither of which resolution + * can read, so kwargs set beside another format are sent nowhere. + */ + chatTemplateKwargs?: NonNullable + /** + * Whether the endpoint accepts `strict` in tool definitions; + * `openai-completions`, the three Responses protocols, `bedrock-converse-stream`. + */ + supportsStrictMode?: boolean + /** Prompt-cache marker convention; `openai-completions`. */ + cacheControlFormat?: NonNullable + /** + * Whether the endpoint accepts long prompt-cache retention; + * `openai-completions`, the three Responses protocols, `anthropic-messages`. + */ + supportsLongCacheRetention?: boolean + /** Whether the endpoint accepts per-tool `eager_input_streaming`; `anthropic-messages`. */ + supportsEagerToolInputStreaming?: boolean + /** Whether the endpoint accepts `cache_control` on tool definitions; `anthropic-messages`. */ + supportsCacheControlOnTools?: boolean + /** Whether the endpoint accepts the `temperature` request field; `anthropic-messages`. */ + supportsTemperature?: boolean + /** Whether to force adaptive thinking regardless of model id; `anthropic-messages`. */ + forceAdaptiveThinking?: boolean + /** Whether to replay an empty thinking signature instead of converting thinking to text; `anthropic-messages`. */ + allowEmptySignature?: boolean + /** Whether the endpoint accepts Anthropic strict tool schemas; `anthropic-messages`. */ + supportsStrictTools?: boolean +} + +/** Compile-time constraint that `T` is `never`. */ +type AssertNever = T + +/** + * Proof that every documented field is one a gate offers. A field the profile + * declares past the gates fails compilation with its own name in the error. + */ +export type EveryProfileFieldIsOffered = AssertNever> + +/** + * Proof that every offered field is documented. A gate entry flipped to + * `offer` without a profile field fails compilation with its own name in the + * error, which is the half a schema alone cannot catch. + */ +export type EveryOfferedFieldIsDocumented = AssertNever> + +/** Compile-time constraint that `T` is `true`. */ +type AssertTrue = T + +/** Every compat type a gate classifies, merged so one `Pick` reaches all offered fields. */ +type UpstreamCompat = OpenAICompletionsCompat & OpenAIResponsesCompat & AnthropicMessagesCompat & BedrockCompat + +/** + * Proof that each documented field carries its upstream type, not a hand-copied + * restatement of it. The name gates above pin *which* fields exist; this pins + * their types, in both directions because each catches a different drift. A + * profile field wider than upstream accepts a value the provider rejects, and + * `resolveModelCompat`'s cast to `ModelCompat` would hide it; a narrower one + * refuses a value the provider accepts, which is how an upgrade that widens a + * union would otherwise leave configuration silently behind. + */ +export type EveryProfileFieldMatchesUpstream = AssertTrue< + PiAiCompatProfile extends Partial> + ? Partial> extends PiAiCompatProfile ? true : false + : false +> + +/** + * The compat entries a profile actually set. + * + * schemastery materializes an absent dict as `{}` — the behavior + * `reasoningEfforts` works around with a union — so every parsed profile + * carries a `chatTemplateKwargs` key whether or not anyone wrote one. An empty + * one states nothing here: it would send no kwargs, which is exactly what + * leaving the field out does, so absent and empty are the same request and + * neither may make a route look like it configured a switch. A valueless + * scalar is the other thing schemastery lets through, and it is refused by + * {@link assertOfferedCompatFields} before this runs rather than filtered. + * @param compat - the configured switches, when any. + * @returns the entries carrying a value, in declaration order. + */ +function configuredCompatEntries(compat: PiAiCompatProfile | undefined): readonly (readonly [string, unknown])[] { + return Object.entries(compat ?? {}).flatMap(([field, value]) => { + const empty = typeof value === 'object' && value !== null && !Array.isArray(value) + && Object.keys(value as object).length === 0 + return empty ? [] : [[field, value] as const] + }) +} + +/** + * The protocols offering one compat field, in {@link COMPAT_GATES} order. + * @param field - configured compat field name. + * @returns the protocols whose compat takes it; empty when none does, which + * is either a withheld field or a name no upstream compat type declares. + */ +function compatProtocols(field: string): readonly string[] { + return Object.entries(COMPAT_GATES).flatMap(([api, gate]) => gate[field] === 'offer' ? [api] : []) +} + +/** + * The compat fields one protocol offers, for a diagnostic that has to show + * what was available instead of the name that missed. + * @param api - wire protocol. + * @returns the offered field names, or an empty list for a protocol taking no compat. + */ +function offeredCompatFields(api: string): readonly string[] { + return Object.entries(compatGate(api) ?? {}).flatMap(([field, disposition]) => disposition === 'offer' ? [field] : []) +} + +/** + * Every offered field name, deduplicated, for the one diagnostic that cannot + * narrow by protocol: the vocabulary check runs before any protocol resolves, + * which is what lets it refuse a misspelling on a route whose models would + * never have reached the protocol that declares the intended field. + * @returns the offered field names across every protocol, in gate order. + */ +function allOfferedCompatFields(): readonly string[] { + const fields = new Set() + for (const api of Object.keys(COMPAT_GATES)) { + for (const field of offeredCompatFields(api)) fields.add(field) + } + return [...fields] +} + +/** + * Reject a compat key no protocol offers. Runs before any protocol is + * resolved, so a withheld field or a misspelling fails even on a route whose + * models never reach the protocol that would have taken it — the alternative + * being the silent drop that let an unreadable switch look applied. + * @param provider - provider route key, for diagnostics. + * @param site - the configuration site, for diagnostics. + * @param compat - the configured switches, when any. + * @throws Error naming the offending key. + */ +function assertOfferedCompatFields( + provider: string, + site: string, + compat: PiAiCompatProfile | undefined, +): void { + // Every key, not only the ones carrying a value: a withheld or undeclared + // name is never in the schema, so schemastery cannot have materialized it — + // whatever its value, a person wrote it and expects it to do something. + for (const [field, value] of Object.entries(compat ?? {})) { + // The name is judged before the value, so a withheld or misspelled key + // written bare is refused for being that name rather than for being empty: + // the other order sends someone to supply a value the key would be refused + // with anyway. + if (compatProtocols(field).length === 0) { + const declared = Object.values(COMPAT_GATES).some(gate => gate[field] !== undefined) + if (declared) { + invalid(provider, `${site} sets compat "${field}", which is not configurable here: pi-ai's installed` + + ' catalog sets it for the vendors that need it, so name that provider as the route instead') + } + invalid(provider, `${site} sets compat "${field}", which no wire protocol declares; the configurable` + + ` switches are ${allOfferedCompatFields().join(', ')}`) + } + // A valueless key (`supportsDeveloperRole:`) survives schemastery, which + // passes nullable data through before any member schema runs — the same + // behavior `reasoningEfforts` documents — and a `cordis.yml` entry may + // reach the same state through `!!js undefined`. Either way the key is + // kept, so carrying it forward writes nothing over whatever the next layer + // resolved, leaving pi-ai's `??` at its baseURL detection: the "written but + // not applied" outcome this surface exists to refuse. + if (value == null) { + invalid(provider, `${site} sets compat "${field}" with no value; give it one, or remove the key to` + + ' leave the field to the next layer — the installed catalog entry, then pi-ai\'s own detection') + } + } } /** One configured model entry: an id plus the catalog fields it overrides. */ @@ -233,7 +565,7 @@ export interface PiAiModelProfile { * declares the offered levels and their wire spellings. */ reasoningEfforts?: false | PiAiReasoningEfforts - /** Reasoning-dispatch switches for this model, winning over the route's. */ + /** pi-ai wire-compatibility switches for this model, winning over the route's per field; one its protocol does not declare is refused. */ compat?: PiAiCompatProfile } @@ -258,7 +590,7 @@ export interface RouteCatalogRequest { models?: readonly PiAiModelProfile[] /** Installed-catalog customizations by model id; only meaningful while `models` is absent. */ modelOverrides?: Readonly> - /** Reasoning-dispatch switches for every `openai-completions` model on the route; entries override per field. */ + /** Route-level wire-compatibility switches, landing on each model whose protocol declares them; entries override per field. */ compat?: PiAiCompatProfile /** Context capacity for a model neither the entry nor the catalog sizes. */ defaultContextWindow: number @@ -368,16 +700,20 @@ function resolveModelReasoning( return { reasoning: true, thinkingLevelMap: map } } +/** The compat block a materialized model carries, whichever protocol it speaks. */ +type ModelCompat = OpenAICompletionsCompat | OpenAIResponsesCompat | AnthropicMessagesCompat | BedrockCompat + /** - * Resolve one model's compat block from the profile's reasoning switches. + * Resolve one model's compat block from the profile's switches. * - * A model switch wins over the route switch; whatever neither sets keeps the - * installed entry's value, and a field no layer decides falls through to - * pi-ai's baseURL-derived detection. Only an `openai-completions` model takes - * the switches at all: a model-level switch on any other protocol fails - * resolution, while a route-level default skips past such models — the same - * posture as the route-level `reasoning` default, which also must not fail - * models it does not fit. + * A model switch wins over the route switch field by field; whatever neither + * sets keeps the installed entry's value, and a field no layer decides falls + * through to pi-ai's own detection. A model-level switch its protocol does not + * take fails resolution — about one named model it can only be a mistake — + * while a route-level one skips past such models, since a route default must + * stay settable on a route whose models do not all speak one protocol. Every + * field reaching here is offered by some protocol; {@link + * assertOfferedCompatFields} has already refused the rest. * @param provider - provider route key, for diagnostics. * @param entry - the configured model entry. * @param route - the route-level switches, when any. @@ -391,31 +727,31 @@ function resolveModelCompat( route: PiAiCompatProfile | undefined, base: Model | undefined, api: string, -): { compat: OpenAICompletionsCompat } | Record { - const thinkingFormat = entry.compat?.thinkingFormat ?? route?.thinkingFormat - const supportsReasoningEffort = entry.compat?.supportsReasoningEffort ?? route?.supportsReasoningEffort - if (thinkingFormat === undefined && supportsReasoningEffort === undefined) return {} - if (api !== 'openai-completions') { - if (entry.compat?.thinkingFormat !== undefined || entry.compat?.supportsReasoningEffort !== undefined) { - invalid(provider, `model "${entry.id}" sets compat reasoning switches, but its api is "${api}";` - + ' thinkingFormat and supportsReasoningEffort exist only on openai-completions') - } - return {} +): { compat: ModelCompat } | Record { + const gate = compatGate(api) + const configured: Record = {} + for (const [field, value] of configuredCompatEntries(route)) { + if (gate?.[field] !== 'offer') continue + configured[field] = value } + for (const [field, value] of configuredCompatEntries(entry.compat)) { + if (gate?.[field] !== 'offer') { + const offered = offeredCompatFields(api) + invalid(provider, `model "${entry.id}" sets compat "${field}", but its api is "${api}", which does not` + + ` take it; that switch exists on ${compatProtocols(field).join(', ')}, and "${api}" offers` + + ` ${offered.length === 0 ? 'no configurable compat' : offered.join(', ')}`) + } + configured[field] = value + } + if (Object.keys(configured).length === 0) return {} // The installed entry's compat matches the entry's OWN api — a route-level // `api` repoint (an anthropic catalog served through an OpenAI-compatible // gateway) leaves `base.compat` in the other protocol's shape, so it is // inherited only while the resolved api still is the entry's. A repointed // model starts from pi-ai's baseURL-derived detection instead, which is // what a protocol change means for every other compat field too. - const inherited: OpenAICompletionsCompat | undefined = base?.api === api ? base.compat : undefined - return { - compat: { - ...inherited, - ...thinkingFormat === undefined ? {} : { thinkingFormat }, - ...supportsReasoningEffort === undefined ? {} : { supportsReasoningEffort }, - }, - } + const inherited = base?.api === api ? base.compat : undefined + return { compat: { ...inherited, ...configured } as ModelCompat } } /** One route's materialized catalog, plus the request caps its profile chose. */ @@ -485,8 +821,13 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog { + ' must be listed in configuration') } const routeApi = sharedCatalogApi(defaults) - const routeCompatDefined = request.compat?.thinkingFormat !== undefined - || request.compat?.supportsReasoningEffort !== undefined + // Vocabulary before protocols: a withheld or undeclared switch is refused + // wherever it is written, so it cannot look applied on a route whose models + // never reach the protocol that would have taken it. + assertOfferedCompatFields(provider, 'route', request.compat) + for (const entry of entries) { + assertOfferedCompatFields(provider, `model "${entry.id}"`, entry.compat) + } const seen = new Set() const configuredMaxTokens = new Map() const models = entries.map((entry) => { @@ -538,9 +879,15 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog { ...resolveModelCompat(provider, entry, request.compat, base, api), } }) - if (routeCompatDefined && !models.some(model => model.api === 'openai-completions')) { - invalid(provider, 'sets compat reasoning switches, but no model on the route speaks openai-completions;' - + ' thinkingFormat and supportsReasoningEffort exist only on that protocol') + // Per field, not per block: a route may default a switch its completions + // models take beside one only its anthropic models do, and neither should + // fail for the other's sake. What is refused is a route default no model on + // the route could ever read, which is a route that will not behave as written. + for (const [field] of configuredCompatEntries(request.compat)) { + const takers = compatProtocols(field) + if (models.some(model => takers.includes(model.api))) continue + invalid(provider, `sets compat "${field}", but no model on the route speaks a protocol that takes it;` + + ` it exists on ${takers.join(', ')}`) } return { models, configuredMaxTokens } } diff --git a/packages/llm/llm-pi-ai/src/config.ts b/packages/llm/llm-pi-ai/src/config.ts index 4e8e032df0..5fdcc2cdb2 100644 --- a/packages/llm/llm-pi-ai/src/config.ts +++ b/packages/llm/llm-pi-ai/src/config.ts @@ -14,14 +14,22 @@ * @module dsh-llm-pi-ai/config */ -import type { CacheRetention, ModelThinkingLevel, Provider, ThinkingBudgets, Transport } from '@earendil-works/pi-ai' +import type { CacheRetention, ChatTemplateKwargValue, ModelThinkingLevel, Provider, ThinkingBudgets, Transport } from '@earendil-works/pi-ai' import z from '@deepseek-ai/schemastery' import { credentialRef } from '@deepseek-ai/dsh-credentials' import type { CredentialRef } from '@deepseek-ai/dsh-credentials' import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm' import type { ResolvedRetryPolicy, RetryPolicyConfig } from '@deepseek-ai/dsh-llm' -import { MODALITIES, resolveRouteModels, SUPPORTED_THINKING_FORMATS, THINKING_LEVELS } from './catalog.ts' +import { + CACHE_CONTROL_FORMATS, + CHAT_TEMPLATE_VARS, + MAX_TOKENS_FIELDS, + MODALITIES, + resolveRouteModels, + SUPPORTED_THINKING_FORMATS, + THINKING_LEVELS, +} from './catalog.ts' import type { PiAiCompatProfile, PiAiModality, @@ -34,6 +42,21 @@ import { buildProvider, supportedProtocols } from './provider.ts' /** Default maximum idle interval while an adapter stream read is outstanding. */ export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000 +/** + * Default request-level bound on base64-encoded image payload. Every image in + * history is re-encoded into every request body, so an unbounded conversation + * eventually exceeds a provider or gateway request-size cap and the session + * can never complete another request. The 20MiB default admits fifteen 1MiB + * request versions after base64 expansion and reserves request capacity for + * system prompts, history, tools, and JSON. + * Deployments behind stricter gateways lower it per route. + */ +export const DEFAULT_MAX_REQUEST_IMAGE_BYTES = 20 * 1024 * 1024 +/** Default total-pixel budget preserves the complete 2048px normalized attachment. */ +export const DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET = 2048 * 2048 +/** Default raw encoded-byte cap before inline base64 expansion. */ +export const DEFAULT_REQUEST_IMAGE_MAX_BYTES = 1024 * 1024 + /** Context capacity assumed for a model neither configuration nor the catalog sizes. */ export const DEFAULT_CONTEXT_WINDOW = 262_144 @@ -91,10 +114,11 @@ export interface PiAiProviderProfile { */ modelOverrides?: Record /** - * Reasoning-dispatch switches for every `openai-completions` model on this - * route; each model's own `compat` overrides per field. What neither sets - * keeps the installed catalog entry's value, then pi-ai's baseURL-derived - * detection. + * pi-ai wire-compatibility switches defaulting every model on this route + * whose protocol declares them; each model's own `compat` overrides per + * field. What neither sets keeps the installed catalog entry's value, then + * pi-ai's own detection. A switch no model on the route could read is + * refused rather than left looking applied. */ compat?: PiAiCompatProfile /** @@ -136,7 +160,18 @@ export interface PiAiProviderProfile { websocketConnectTimeoutMs?: number /** Maximum provider idle time while one stream read is outstanding. */ streamIdleTimeoutMs?: number - /** Provider-owned model-request retry policy; omission uses normal defaults. */ + /** + * Maximum base64-encoded image payload per request. When a request's + * accumulated images exceed it, the oldest images are replaced by text + * placeholders until the request fits, so a long session keeps completing + * requests instead of being rejected by a request-size cap. + */ + maxRequestImageBytes?: number + /** Total-pixel budget for each deterministic inline request version. */ + requestImagePixelBudget?: number + /** Raw encoded-byte cap for each deterministic inline request version. */ + requestImageMaxBytes?: number + /** Provider-owned model-request retry policy; omission uses normal mode with five retries. */ retryPolicy?: RetryPolicyConfig } @@ -151,6 +186,12 @@ export interface ResolvedPiAiProviderProfile apiKeyEnv?: CredentialRef /** Positive finite provider-idle interval after defaulting. */ streamIdleTimeoutMs: number + /** Positive request-level base64 image payload bound after defaulting. */ + maxRequestImageBytes: number + /** Positive total-pixel request-version budget after defaulting. */ + requestImagePixelBudget: number + /** Positive raw request-version byte cap after defaulting. */ + requestImageMaxBytes: number /** Immutable retry policy captured with this provider route. */ retryPolicy: ResolvedRetryPolicy /** @@ -185,9 +226,43 @@ const thinkingBudgets = z.object({ high: z.number(), }) +/** + * One `chat_template_kwargs` value. The `$var` member is pi-ai's placeholder + * for a value dispatch fills from the request's thinking state, which is what + * makes a chat-template gateway configurable without restating its template. + */ +const chatTemplateKwarg: z = z.union([ + z.string(), + z.number(), + z.boolean(), + z.const(null), + z.object({ + $var: z.union(CHAT_TEMPLATE_VARS).required(), + omitWhenOff: z.boolean(), + }), +]) + const compatProfile: z = z.object({ - thinkingFormat: z.union(SUPPORTED_THINKING_FORMATS), + supportsStore: z.boolean(), + supportsDeveloperRole: z.boolean(), supportsReasoningEffort: z.boolean(), + supportsUsageInStreaming: z.boolean(), + maxTokensField: z.union(MAX_TOKENS_FIELDS), + requiresToolResultName: z.boolean(), + requiresAssistantAfterToolResult: z.boolean(), + requiresThinkingAsText: z.boolean(), + requiresReasoningContentOnAssistantMessages: z.boolean(), + thinkingFormat: z.union(SUPPORTED_THINKING_FORMATS), + chatTemplateKwargs: z.dict(chatTemplateKwarg), + supportsStrictMode: z.boolean(), + cacheControlFormat: z.union(CACHE_CONTROL_FORMATS), + supportsLongCacheRetention: z.boolean(), + supportsEagerToolInputStreaming: z.boolean(), + supportsCacheControlOnTools: z.boolean(), + supportsTemperature: z.boolean(), + forceAdaptiveThinking: z.boolean(), + allowEmptySignature: z.boolean(), + supportsStrictTools: z.boolean(), }) /** @@ -248,6 +323,9 @@ const profile = z.object({ timeoutMs: z.natural(), websocketConnectTimeoutMs: z.natural(), streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS), + maxRequestImageBytes: z.number().step(1).min(1).default(DEFAULT_MAX_REQUEST_IMAGE_BYTES), + requestImagePixelBudget: z.number().step(1).min(1).default(DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET), + requestImageMaxBytes: z.number().step(1).min(1).default(DEFAULT_REQUEST_IMAGE_MAX_BYTES), retryPolicy: RetryPolicySchema, }) @@ -323,6 +401,18 @@ export function resolveProfiles( `llm-pi-ai: provider "${provider}" streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`, ) } + const maxRequestImageBytes = source.maxRequestImageBytes ?? DEFAULT_MAX_REQUEST_IMAGE_BYTES + if (!Number.isInteger(maxRequestImageBytes) || maxRequestImageBytes <= 0) { + throw new Error(`llm-pi-ai: provider "${provider}" maxRequestImageBytes must be a positive integer`) + } + const requestImagePixelBudget = source.requestImagePixelBudget ?? DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET + if (!Number.isSafeInteger(requestImagePixelBudget) || requestImagePixelBudget <= 0) { + throw new Error(`llm-pi-ai: provider "${provider}" requestImagePixelBudget must be a positive safe integer`) + } + const requestImageMaxBytes = source.requestImageMaxBytes ?? DEFAULT_REQUEST_IMAGE_MAX_BYTES + if (!Number.isSafeInteger(requestImageMaxBytes) || requestImageMaxBytes <= 0) { + throw new Error(`llm-pi-ai: provider "${provider}" requestImageMaxBytes must be a positive safe integer`) + } // Detached from the configuration object because pi-ai types `Model.input` // mutable. The schema's explicit default covers an absent key, so an empty // list here is always one someone typed — and unlike an entry's, nothing @@ -354,6 +444,9 @@ export function resolveProfiles( displayName, ...apiKeyEnv === undefined ? {} : { apiKeyEnv: credentialRef(apiKeyEnv) }, streamIdleTimeoutMs, + maxRequestImageBytes, + requestImagePixelBudget, + requestImageMaxBytes, retryPolicy: resolveRetryPolicy(retryPolicy, `llm-pi-ai: provider "${provider}" retryPolicy`), ...rest.headers === undefined ? {} : { headers: { ...rest.headers } }, ...rest.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...rest.thinkingBudgets } }, diff --git a/packages/llm/llm-pi-ai/src/context.ts b/packages/llm/llm-pi-ai/src/context.ts index 678820510e..9faf457c9a 100644 --- a/packages/llm/llm-pi-ai/src/context.ts +++ b/packages/llm/llm-pi-ai/src/context.ts @@ -4,11 +4,18 @@ * @module dsh-llm-pi-ai/context */ -import { CallId, contentHasImage, LlmError } from '@deepseek-ai/dsh-llm' +import { CallId, contentHasImage, LlmError, offloadRequestImagesWithPolicy, requestImageHandleText } from '@deepseek-ai/dsh-llm' import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm' -import type { AttachmentStore } from '@deepseek-ai/dsh-attachment' +import type { + AttachmentId, + AttachmentStore, + ImageAttachmentRef, + ImageRequestPolicy, + RequestImageAttachment, +} from '@deepseek-ai/dsh-attachment' import type { Context as PiContext, ImageContent, Message as PiMessage, TextContent, Tool as PiTool } from '@earendil-works/pi-ai' import { toPiAssistant } from './replay.ts' +import { DEFAULT_REQUEST_IMAGE_MAX_BYTES, DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET } from './config.ts' /** Join the text blocks of a harness message. */ function flattenText(message: Message): string { @@ -26,9 +33,21 @@ function toolResultText(blocks: readonly ContentBlock[]): string { : block.type === 'tool-result' ? toolResultText(block.content) : '').join('') } +/** Reject image roles that pi-ai cannot replay before request-size offloading can replace them. */ +function assertSupportedImageRoles(messages: readonly Message[]): void { + for (const message of messages) { + if (message.role !== 'user' && contentHasImage(message.content)) { + throw new LlmError( + `pi-ai cannot represent an image in an in-history ${message.role} message`, + 'UNSUPPORTED_CONTENT', + ) + } + } +} + async function userContent( blocks: readonly ContentBlock[], - attachments: AttachmentStore, + requestImages: ReadonlyMap, ): Promise { const content: (TextContent | ImageContent)[] = [] for (const block of blocks) { @@ -37,17 +56,18 @@ async function userContent( if (block.text.length > 0) content.push({ type: 'text', text: block.text }) break case 'image': { - const stored = await attachments.readImage(block.attachment) + const version = requestImages.get(block.attachment.attachmentId) as RequestImageAttachment + content.push({ type: 'text', text: requestImageHandleText(version) }) content.push({ type: 'image', - data: Buffer.from(stored.data).toString('base64'), - mimeType: stored.ref.mediaType, + data: Buffer.from(version.data).toString('base64'), + mimeType: version.mediaType, }) break } case 'tool-result': { - const nested = await userContent(block.content, attachments) + const nested = await userContent(block.content, requestImages) if (typeof nested === 'string') { if (nested.length > 0) content.push({ type: 'text', text: nested }) } else { @@ -64,6 +84,35 @@ async function userContent( return content } +function collectImageRefs( + blocks: readonly ContentBlock[], + refs: Map, +): void { + for (const block of blocks) { + if (block.type === 'image') refs.set(block.attachment.attachmentId, block.attachment) + else if (block.type === 'tool-result') collectImageRefs(block.content, refs) + } +} + +async function prepareRequestImages( + messages: readonly Message[], + attachments: AttachmentStore, + policy: ImageRequestPolicy, + signal?: AbortSignal, +): Promise> { + const refs = new Map() + for (const message of messages) collectImageRefs(message.content, refs) + const orderedRefs = [...refs.values()] + const prepared = await Promise.all(orderedRefs.map( + ref => attachments.readImageRequest(ref, policy, signal), + )) + const versions = new Map() + for (const [index, ref] of orderedRefs.entries()) { + versions.set(ref.attachmentId, prepared[index] as RequestImageAttachment) + } + return versions +} + function toolsOf(options: GenerateOptions): PiTool[] | undefined { return options.tools?.map(tool => ({ name: tool.name, @@ -84,7 +133,7 @@ function piContext(options: GenerateOptions, messages: PiMessage[]): PiContext { } } -function textOnlyContext(options: GenerateOptions): PiContext { +function textOnlyContext(options: GenerateOptions, onReplayDegrade?: (reason: string) => void): PiContext { const toolNames = new Map() const messages: PiMessage[] = [] for (const message of options.messages) { @@ -96,7 +145,7 @@ function textOnlyContext(options: GenerateOptions): PiContext { continue } if (message.role === 'assistant') { - const assistant = toPiAssistant(message) + const assistant = toPiAssistant(message, onReplayDegrade) for (const block of assistant.content) if (block.type === 'toolCall') toolNames.set(CallId(block.id), block.name) messages.push(assistant) continue @@ -125,30 +174,76 @@ function textOnlyContext(options: GenerateOptions): PiContext { * Convert text-only harness history to a synchronous pi-ai Context. Tool * result names are recovered from preceding assistant tool calls. * @param options - the harness request; `options.system` maps to pi-ai's single `systemPrompt` slot. + * @param attachments - absent; selects the synchronous conversion. + * @param onReplayDegrade - forwarded to {@link toPiAssistant} for each assistant message. * @returns the pi-ai context; `tools` is omitted when the request declares none. */ -export function toPiContext(options: GenerateOptions): PiContext +export function toPiContext( + options: GenerateOptions, + attachments?: undefined, + onReplayDegrade?: (reason: string) => void, +): PiContext /** * Convert harness history to a pi-ai Context while resolving durable images. - * Tool result names are recovered from preceding assistant tool calls. + * Tool result names are recovered from preceding assistant tool calls. When + * the accumulated base64 image payload exceeds `maxRequestImageBytes`, the + * oldest images are replaced by text placeholders until the request fits, so + * an image-heavy session keeps clearing gateway request-size caps. * @param options - the harness request; `options.system` maps to pi-ai's single `systemPrompt` slot. * @param attachments - durable byte resolver for image references. + * @param onReplayDegrade - forwarded to {@link toPiAssistant} for each assistant message. + * @param maxRequestImageBytes - request-level bound on base64-encoded image payload; omission leaves every image in place. + * @param requestImagePolicy - route pixel and raw encoded-byte budgets. * @returns the asynchronously resolved pi-ai context. */ -export function toPiContext(options: GenerateOptions, attachments: AttachmentStore): Promise -export function toPiContext(options: GenerateOptions, attachments?: AttachmentStore): PiContext | Promise { - return attachments === undefined ? textOnlyContext(options) : toPiContextWithImages(options, attachments) +export function toPiContext( + options: GenerateOptions, + attachments: AttachmentStore, + onReplayDegrade?: (reason: string) => void, + maxRequestImageBytes?: number, + requestImagePolicy?: ImageRequestPolicy, +): Promise +export function toPiContext( + options: GenerateOptions, + attachments?: AttachmentStore, + onReplayDegrade?: (reason: string) => void, + maxRequestImageBytes?: number, + requestImagePolicy?: ImageRequestPolicy, +): PiContext | Promise { + return attachments === undefined + ? textOnlyContext(options, onReplayDegrade) + : toPiContextWithImages(options, attachments, onReplayDegrade, maxRequestImageBytes, requestImagePolicy) } -async function toPiContextWithImages(options: GenerateOptions, attachments: AttachmentStore): Promise { +async function toPiContextWithImages( + options: GenerateOptions, + attachments: AttachmentStore, + onReplayDegrade?: (reason: string) => void, + maxRequestImageBytes?: number, + requestImagePolicy: ImageRequestPolicy = { + maxPixels: DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET, + maxBytes: DEFAULT_REQUEST_IMAGE_MAX_BYTES, + }, +): Promise { + assertSupportedImageRoles(options.messages) + const requestMessages = offloadRequestImagesWithPolicy(options.messages, { + representation: 'base64', + ...maxRequestImageBytes === undefined ? {} : { maxBytes: maxRequestImageBytes }, + byteQuantum: 1, + byteLength: ref => Math.min(ref.bytes, requestImagePolicy.maxBytes), + }) + const requestImages = await prepareRequestImages(requestMessages, attachments, requestImagePolicy, options.signal) + const exactMessages = offloadRequestImagesWithPolicy(requestMessages, { + representation: 'base64', + ...maxRequestImageBytes === undefined ? {} : { maxBytes: maxRequestImageBytes }, + byteQuantum: 1, + byteLength: ref => (requestImages.get(ref.attachmentId) as RequestImageAttachment).bytes, + }) const toolNames = new Map() const messages: PiMessage[] = [] - for (const message of options.messages) { + for (const message of exactMessages) { if (message.role === 'system') { - if (contentHasImage(message.content)) { - throw new LlmError('pi-ai cannot represent an image in an in-history system message', 'UNSUPPORTED_CONTENT') - } // pi-ai has a single systemPrompt slot; in-history system messages are // folded into user messages to preserve order (rare in practice — the // harness sends the system prompt via options.system). @@ -156,7 +251,7 @@ async function toPiContextWithImages(options: GenerateOptions, attachments: Atta continue } if (message.role === 'assistant') { - const assistant = toPiAssistant(message) + const assistant = toPiAssistant(message, onReplayDegrade) for (const block of assistant.content) { if (block.type === 'toolCall') toolNames.set(CallId(block.id), block.name) } @@ -165,13 +260,15 @@ async function toPiContextWithImages(options: GenerateOptions, attachments: Atta } // user role: text + tool results (each result becomes its own message). const regular = message.content.filter(block => block.type !== 'tool-result') - const content = await userContent(regular, attachments) - const results = message.content.filter(block => block.type === 'tool-result') + const content = await userContent(regular, requestImages) + const results = message.content.filter((block): block is Extract => ( + block.type === 'tool-result' + )) if (content.length > 0 || results.length === 0) { messages.push({ role: 'user', content, timestamp: 0 }) } for (const result of results) { - const resultContent = await userContent(result.content, attachments) + const resultContent = await userContent(result.content, requestImages) messages.push({ role: 'toolResult', toolCallId: result.toolCallId, diff --git a/packages/llm/llm-pi-ai/src/index.ts b/packages/llm/llm-pi-ai/src/index.ts index 2e550771fc..846509f8c6 100644 --- a/packages/llm/llm-pi-ai/src/index.ts +++ b/packages/llm/llm-pi-ai/src/index.ts @@ -61,10 +61,12 @@ import { assertUsableApiKey, LlmError } from '@deepseek-ai/dsh-llm' import type { AdapterRegistrationHandle, DirectoryRegistrationHandle, LlmConfigurableProvider } from '@deepseek-ai/dsh-llm' import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings' import { PiAiAdapter } from './adapter.ts' -import { catalogProviderIds, catalogProviderTakesApiKey } from './catalog.ts' +import { authContextFrom, credentialStoreFrom } from './auth.ts' +import { catalogProviderIds } from './catalog.ts' import { assertServiceable, Config, resolveProfiles } from './config.ts' import type { ResolvedPiAiProviderProfile } from './config.ts' import { discoverModels } from './discovery.ts' +import { registerPiAiFlows } from './login.ts' export { PiAiAdapter } from './adapter.ts' export type { PiAiAdapterOptions } from './adapter.ts' @@ -79,6 +81,7 @@ export type { PiAiThinkingFormat, ResolvedPiAiProviderProfile, } from './config.ts' +export { recordKeyFor } from './auth.ts' export { supportedProtocols } from './provider.ts' export const name = 'llm-pi-ai' @@ -105,15 +108,10 @@ function registrationFacts(profiles: ReadonlyMap ctx.get('attachments'), + onReplayDegrade: ({ provider, model, reason }) => { + ctx.logger.warn( + `llm-pi-ai: unusable replay state on assistant history for route "${provider}/${model}";` + + ` sending that message as provider-neutral content (${reason})`, + ) + }, }) + // Independent of the route set: signing in is what makes a route worth + // adding, so the flows are offered before any profile names their provider. + // Scoped to the authorization seam rather than injected outright, because a + // composition without it (headless, ACP) simply has no surface to sign in + // from, while everything else this plugin does still works. + ctx.inject(['authorization'], (authorized) => { registerPiAiFlows(authorized, auth) }) // The full installed catalog is configurable from the moment the plugin // mounts — dormant or not — so configuration surfaces can offer every // pi-ai provider before any route exists. Hand-declared routes join it as diff --git a/packages/llm/llm-pi-ai/src/login.ts b/packages/llm/llm-pi-ai/src/login.ts new file mode 100644 index 0000000000..a94dcc93ab --- /dev/null +++ b/packages/llm/llm-pi-ai/src/login.ts @@ -0,0 +1,161 @@ +/** + * Authorization flows for the pi-ai providers that ship a login. This is the + * whole of the translation between the harness's neutral notice/prompt + * vocabulary and pi-ai's `AuthInteraction`; nothing above it knows which + * library ran the conversation. + * + * @module dsh-llm-pi-ai/login + */ + +import { createModels } from '@earendil-works/pi-ai' +import type { AuthEvent, AuthPrompt, AuthType, Provider } from '@earendil-works/pi-ai' +import type { Context } from '@deepseek-ai/cordis' +import type { AuthorizationMethod, AuthorizationPrompt, AuthorizationSession } from '@deepseek-ai/dsh-authorization' +import { isCredentialKeySegment } from '@deepseek-ai/dsh-credentials' +import { catalogProvider, catalogProviderIds } from './catalog.ts' +import { recordKeyFor } from './auth.ts' +import type { PiAiAuthInjection } from './adapter.ts' + +/** + * The login methods one catalog provider offers. + * + * A method appears only when pi-ai can actually run it: `oauth` always carries + * a `login`, while an api-key method has one only when the provider collects + * its key interactively — which every installed one currently does, so a key is + * typed into pi-ai's own prompt rather than into the settings form. + * @param provider - the installed catalog provider, if pi-ai ships one. + * @returns its methods, most preferred first; empty when it offers no login. + */ +function loginMethods(provider: Provider | undefined): AuthorizationMethod[] { + const methods: AuthorizationMethod[] = [] + const oauth = provider?.auth.oauth + if (oauth !== undefined) methods.push({ id: 'oauth', label: oauth.loginLabel ?? oauth.name }) + const apiKey = provider?.auth.apiKey + if (apiKey?.login !== undefined) methods.push({ id: 'api-key', label: apiKey.name }) + return methods +} + +/** + * Restate one pi-ai login event in the seam's vocabulary. + * + * A device-code grant is the one event carrying two things the human needs at + * once — where to go and what to type there — which is why the neutral notice + * has a `code` beside its `url` rather than folding the code into the message. + * @param event - what pi-ai reported. + * @param session - the attempt to report it to. + */ +function relay(event: AuthEvent, session: AuthorizationSession): void { + switch (event.type) { + case 'info': { + const link = event.links?.[0] + session.notify({ message: event.message, ...link === undefined ? {} : { url: link.url } }) + return + } + case 'auth_url': + session.notify({ + message: event.instructions ?? 'Open this page to continue signing in.', + url: event.url, + }) + return + case 'device_code': + session.notify({ + message: 'Enter this code on the verification page to finish signing in.', + url: event.verificationUri, + code: event.userCode, + }) + return + case 'progress': + session.notify({ message: event.message }) + return + default: + // pi-ai's event union is open to new members: a build that meets one it + // does not know still shows the human that something is happening rather + // than going silent mid-login. + session.notify({ message: 'Signing in…' }) + } +} + +/** + * Restate one pi-ai prompt in the seam's vocabulary. + * + * `manual_code` becomes a plain text question because the difference pi-ai + * draws — a code the human copies from a browser rather than a value they know + * — changes nothing a surface renders. Its own `signal` is carried through, and + * that is the part which matters: it is how a flow racing a typed code against + * a browser callback withdraws the losing question. + * @param prompt - what pi-ai asked. + * @returns the neutral prompt to put to the human. + */ +function restate(prompt: AuthPrompt): AuthorizationPrompt { + const signal = prompt.signal === undefined ? {} : { signal: prompt.signal } + switch (prompt.type) { + case 'select': + return { ...signal, kind: 'select', message: prompt.message, options: prompt.options } + case 'secret': + return { + ...signal, + kind: 'secret', + message: prompt.message, + ...prompt.placeholder === undefined ? {} : { placeholder: prompt.placeholder }, + } + default: + return { + ...signal, + kind: 'text', + message: prompt.message, + ...prompt.placeholder === undefined ? {} : { placeholder: prompt.placeholder }, + } + } +} + +/** + * Register one authorization flow per installed provider that ships a login. + * + * Registration is unconditional on configuration: a provider has to be signed + * into before a route for it is worth adding, so the flow exists from the + * moment the plugin mounts rather than appearing once a profile does. + * @param ctx - the plugin context carrying `ctx.authorization`. + * @param auth - the injectables every collection here is built with. + */ +export function registerPiAiFlows(ctx: Context, auth: PiAiAuthInjection): void { + for (const providerId of catalogProviderIds()) { + const provider = catalogProvider(providerId) + const [first, ...rest] = loginMethods(provider) + /* v8 ignore next 3 -- every id here names an installed provider and every + installed provider ships a login, so no entry is skipped; the guard + is what keeps that from becoming a crash if either stops being true. */ + if (provider === undefined || first === undefined) continue + /* v8 ignore next 7 -- every installed catalog id is a lowercase + hyphenated identifier; the guard keeps a future upstream id outside the + record grammar (dotted or uppercase, as vendor ids elsewhere already + are) from throwing in `recordKeyFor` and failing the whole mount. */ + if (!isCredentialKeySegment(providerId)) { + ctx.logger.warn( + 'llm-pi-ai: catalog provider "%s" cannot address a credential record; its sign-in is not offered', + providerId) + continue + } + ctx.authorization.registerFlow({ + key: recordKeyFor(providerId), + label: provider.name, + methods: [first, ...rest], + async run(session) { + // A collection of its own, holding only the provider being signed + // into: login is not serving requests, and the credential it produces + // lands in the shared store either way. + const models = createModels(auth) + models.setProvider(provider) + // Total over the two ids declared above, and the seam only ever hands + // back one a flow declared. + const type: AuthType = session.method === 'oauth' ? 'oauth' : 'api_key' + // pi-ai persists what the login returns through that same store, which + // is what makes it the single writer of this record. + await models.login(providerId, type, { + signal: session.signal, + notify: (event) => { relay(event, session) }, + prompt: prompt => session.prompt(restate(prompt)), + }) + }, + }) + } +} diff --git a/packages/llm/llm-pi-ai/src/provider.ts b/packages/llm/llm-pi-ai/src/provider.ts index b4331b26ef..f03c83ce92 100644 --- a/packages/llm/llm-pi-ai/src/provider.ts +++ b/packages/llm/llm-pi-ai/src/provider.ts @@ -33,7 +33,7 @@ import { catalogProvider } from './catalog.ts' * catalog route would. * * The table is deliberately narrow: the protocols a hand-declared route - * actually reaches for today, each completely describable with a key, an + * actually reads, each completely describable with a key, an * endpoint, and headers. Bedrock signs with SigV4 over AWS credentials and a * region, Vertex needs a project, a location, and application-default * credentials, Azure needs provider environment plus an api-version, and Codex diff --git a/packages/llm/llm-pi-ai/src/replay.ts b/packages/llm/llm-pi-ai/src/replay.ts index 10a39c655b..bae4cfc40d 100644 --- a/packages/llm/llm-pi-ai/src/replay.ts +++ b/packages/llm/llm-pi-ai/src/replay.ts @@ -9,24 +9,30 @@ */ import { LlmError } from '@deepseek-ai/dsh-llm' -import type { Message, ModelMessageSource } from '@deepseek-ai/dsh-llm' +import type { Message, ModelMessageSource, ReplayEnvelope } from '@deepseek-ai/dsh-llm' import type { Api, AssistantMessage, Usage as PiUsage } from '@earendil-works/pi-ai' -type PiAiReplayBlock = +/** Per-block half of the pi-ai replay envelope, one entry per content block. */ +export type PiAiReplayBlock = | { type: 'text'; textSignature?: string } | { type: 'reasoning'; thinkingSignature?: string; redacted?: boolean } | { type: 'tool-call'; thoughtSignature?: string } -/** Versioned adapter-private projection required to replay a pi-ai response. */ -export interface PiAiReplayState { +/** Versioned response-level half of the pi-ai replay envelope. */ +export interface PiAiReplayResponse { kind: 'pi-ai' - version: 1 + version: 2 api: Api provider: string model: string responseModel?: string responseId?: string stopReason: AssistantMessage['stopReason'] +} + +/** The validated halves of one pi-ai replay envelope. */ +interface PiAiReplayState { + response: PiAiReplayResponse blocks: PiAiReplayBlock[] } @@ -57,19 +63,25 @@ function emptyPiUsage(): PiUsage { /** * Project a successful pi-ai response into the minimal durable replay state. + * The per-block half is index-aligned with the streamed blocks (pi-ai content + * order), so `BlockAssembler` prunes an entry with its block whenever assembly + * removes one. * @param message - completed native pi-ai assistant response. * @returns the versioned lossless-JSON replay projection. */ -export function toPiReplayState(message: AssistantMessage): PiAiReplayState { - return { +export function toPiReplayState(message: AssistantMessage): ReplayEnvelope { + const response: PiAiReplayResponse = { kind: 'pi-ai', - version: 1, + version: 2, api: message.api, provider: message.provider, model: message.model, ...message.responseModel === undefined ? {} : { responseModel: message.responseModel }, ...message.responseId === undefined ? {} : { responseId: message.responseId }, stopReason: message.stopReason, + } + return { + response, blocks: message.content.map((block): PiAiReplayBlock => { switch (block.type) { case 'text': return { @@ -94,22 +106,26 @@ function invalidReplay(message: string): never { throw new LlmError(`invalid pi-ai replay state: ${message}`, 'INVALID_REPLAY_STATE') } -/** Validate the adapter-private state before it reaches pi-ai. */ +/** Validate the durable adapter-private envelope before it reaches pi-ai. */ function readReplayState(value: unknown): PiAiReplayState { - if (typeof value !== 'object' || value === null || Array.isArray(value)) return invalidReplay('expected an object') - const state = value as Record - if (state['kind'] !== 'pi-ai') return invalidReplay('unknown state kind') - if (state['version'] !== 1) return invalidReplay(`unsupported version ${String(state['version'])}`) + if (typeof value !== 'object' || value === null || Array.isArray(value)) return invalidReplay('expected a replay envelope') + const envelope = value as Record + const rawResponse = envelope['response'] + if (typeof rawResponse !== 'object' || rawResponse === null || Array.isArray(rawResponse)) return invalidReplay('expected a response object') + const response = rawResponse as Record + if (response['kind'] !== 'pi-ai') return invalidReplay('unknown state kind') + if (response['version'] !== 2) return invalidReplay(`unsupported version ${String(response['version'])}`) for (const key of ['api', 'provider', 'model'] as const) { - if (typeof state[key] !== 'string' || state[key].length === 0) return invalidReplay(`${key} must be a non-empty string`) + if (typeof response[key] !== 'string' || response[key].length === 0) return invalidReplay(`${key} must be a non-empty string`) } - if (!['stop', 'length', 'toolUse', 'error', 'aborted'].includes(String(state['stopReason']))) { + if (!['stop', 'length', 'toolUse', 'error', 'aborted'].includes(String(response['stopReason']))) { return invalidReplay('unknown stopReason') } - if (state['responseModel'] !== undefined && typeof state['responseModel'] !== 'string') return invalidReplay('responseModel must be a string') - if (state['responseId'] !== undefined && typeof state['responseId'] !== 'string') return invalidReplay('responseId must be a string') - if (!Array.isArray(state['blocks'])) return invalidReplay('blocks must be an array') - for (const [index, value] of state['blocks'].entries()) { + if (response['responseModel'] !== undefined && typeof response['responseModel'] !== 'string') return invalidReplay('responseModel must be a string') + if (response['responseId'] !== undefined && typeof response['responseId'] !== 'string') return invalidReplay('responseId must be a string') + const blocks = envelope['blocks'] + if (!Array.isArray(blocks)) return invalidReplay('blocks must be an array') + for (const [index, value] of blocks.entries()) { if (typeof value !== 'object' || value === null || Array.isArray(value)) return invalidReplay(`block ${index} must be an object`) const block = value as Record if (!['text', 'reasoning', 'tool-call'].includes(String(block['type']))) return invalidReplay(`block ${index} has an unknown type`) @@ -118,7 +134,10 @@ function readReplayState(value: unknown): PiAiReplayState { } if (block['redacted'] !== undefined && typeof block['redacted'] !== 'boolean') return invalidReplay(`block ${index} redacted must be boolean`) } - return state as unknown as PiAiReplayState + return { + response: response as unknown as PiAiReplayResponse, + blocks: blocks as PiAiReplayBlock[], + } } /** Convert provider-neutral blocks without trusting them as same-model replay. */ @@ -159,8 +178,8 @@ function foreignAssistant(message: Message): AssistantMessage { /** Recombine durable Harness content with validated pi-ai replay metadata. */ function replayedAssistant(message: Message, source: ModelMessageSource, rawState: unknown): AssistantMessage { const state = readReplayState(rawState) - if (state.provider !== source.provider) return invalidReplay('provider does not match assistant source') - if (state.model !== source.model) return invalidReplay('model does not match assistant source') + if (state.response.provider !== source.provider) return invalidReplay('provider does not match assistant source') + if (state.response.model !== source.model) return invalidReplay('model does not match assistant source') if (state.blocks.length !== message.content.length) return invalidReplay('block count does not match assistant content') const content: AssistantMessage['content'] = message.content.map((block, index) => { const replay = state.blocks[index] @@ -191,25 +210,40 @@ function replayedAssistant(message: Message, source: ModelMessageSource, rawStat return { role: 'assistant', content, - api: state.api, - provider: state.provider, - model: state.model, - ...state.responseModel === undefined ? {} : { responseModel: state.responseModel }, - ...state.responseId === undefined ? {} : { responseId: state.responseId }, + api: state.response.api, + provider: state.response.provider, + model: state.response.model, + ...state.response.responseModel === undefined ? {} : { responseModel: state.response.responseModel }, + ...state.response.responseId === undefined ? {} : { responseId: state.response.responseId }, usage: emptyPiUsage(), - stopReason: state.stopReason, + stopReason: state.response.stopReason, timestamp: 0, } } /** * Convert one durable Harness assistant message into pi-ai history. + * + * Durable content is the authoritative record; replay metadata only restores + * native fidelity (ids, signatures). A replay state this build cannot use — + * another adapter's kind, another version, a malformed value, or metadata that + * no longer matches the content — therefore degrades the one message to + * provider-neutral history instead of failing the request. * @param message - assistant content with required source and optional adapter-owned replay metadata. + * @param onDegrade - called with the diagnostic reason when an unusable replay + * state falls back to provider-neutral conversion. * @returns a native pi-ai assistant message reconstructed from durable content. */ -export function toPiAssistant(message: Message): AssistantMessage { +export function toPiAssistant(message: Message, onDegrade?: (reason: string) => void): AssistantMessage { const source = message.source - return source.kind !== 'model' || source.replayState === undefined - ? foreignAssistant(message) - : replayedAssistant(message, source, source.replayState) + if (source.kind !== 'model' || source.replayState === undefined) return foreignAssistant(message) + try { + return replayedAssistant(message, source, source.replayState) + } catch (error: unknown) { + /* v8 ignore next -- replayedAssistant throws only INVALID_REPLAY_STATE LlmErrors; the + guard keeps a future non-replay failure loud instead of silently degrading it */ + if (!(error instanceof LlmError) || error.code !== 'INVALID_REPLAY_STATE') throw error + onDegrade?.(error.message) + return foreignAssistant(message) + } } diff --git a/packages/llm/llm-pi-ai/src/stream.ts b/packages/llm/llm-pi-ai/src/stream.ts index 22aa7cb579..d45df45a39 100644 --- a/packages/llm/llm-pi-ai/src/stream.ts +++ b/packages/llm/llm-pi-ai/src/stream.ts @@ -40,6 +40,9 @@ function classifyPiAiError(message: string): string { if (/\b(?:401|403)\b/.test(message)) return 'AUTH' if (isQuotaExceededError(message)) return QUOTA_EXCEEDED_CODE if (/\b429\b|rate.?limit/i.test(message)) return 'RATE_LIMIT' + // A rejected request body (gateway or provider size cap): resending the + // same request cannot succeed, so it is invalid, not transient. + if (/\b413\b|failed to buffer the request body:\s*length limit exceeded|payload too large|request body too large/i.test(message)) return 'INVALID_REQUEST' if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST' if (/\b5\d\d\b/.test(message)) return 'SERVER' if (/\btime(?:d)?\s*out\b|timeout/i.test(message)) return 'TIMEOUT' diff --git a/packages/llm/llm-pi-ai/tests/adapter.spec.ts b/packages/llm/llm-pi-ai/tests/adapter.spec.ts index 51ce347e9a..29e55e0553 100644 --- a/packages/llm/llm-pi-ai/tests/adapter.spec.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.spec.ts @@ -1,9 +1,11 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import { AttachmentId, AttachmentStore } from '@deepseek-ai/dsh-attachment' +import { AttachmentId, AttachmentStore, ImageVariantId } from '@deepseek-ai/dsh-attachment' import type { ImageAttachmentLimits, ImageAttachmentRef, + ImageRequestPolicy, + RequestImageAttachment, SaveImageAttachment, StoredImageAttachment, } from '@deepseek-ai/dsh-attachment' @@ -12,7 +14,8 @@ import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai' import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all' -import { resolveProfiles } from '../src/config.ts' +import { DEFAULT_MAX_REQUEST_IMAGE_BYTES, resolveProfiles } from '../src/config.ts' +import { memoryAuth } from './auth-double.ts' import { assemble } from './assemble.ts' import { closeMockServers, mockServer, textEvents } from './mock-server.ts' @@ -47,6 +50,7 @@ function adapterOf( return new PiAiAdapter({ profiles: () => resolveProfiles(providers), resolveApiKey: () => Promise.resolve(apiKey), + auth: memoryAuth(), }) } @@ -73,6 +77,30 @@ describe('PiAiAdapter provider routing', () => { expect(server.paths).toEqual(['/chat/completions']) }) + it('keeps prepared model metadata and dispatch on one profile snapshot', async () => { + const first = await mockServer([{ events: textEvents }]) + const second = await mockServer([]) + let providers: Record = { + deepseek: { apiKeyEnv: 'PI_TEST_KEY', baseURL: first.url }, + } + const ctx = new Context() + await ctx.plugin(LlmRuntime) + ctx.llm.registerAdapter(['deepseek'], new PiAiAdapter({ + profiles: () => resolveProfiles(providers), + resolveApiKey: () => Promise.resolve('test-key'), + auth: memoryAuth(), + })) + + const prepared = await ctx.llm.prepareCall({ provider: 'deepseek', model: 'deepseek-v4-flash' }) + providers = { deepseek: { apiKeyEnv: 'PI_TEST_KEY', baseURL: second.url } } + const chunks: unknown[] = [] + for await (const chunk of prepared.stream({ ...prepared.config, messages: [] })) chunks.push(chunk) + + expect(chunks.length).toBeGreaterThan(0) + expect(first.requests).toHaveLength(1) + expect(second.requests).toHaveLength(0) + }) + it('merges profile headers with Harness attribution winning', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url, { @@ -108,6 +136,8 @@ describe('PiAiAdapter provider routing', () => { thinking: { type: 'enabled' }, reasoning_effort: 'max', }) + expect(server.requests[0]).not.toHaveProperty('dsh_session_log') + expect(server.requests[0]).not.toHaveProperty('dsh_plugin_packages') }) it('uses a dynamic request effort and reports unsupported efforts before network I/O', async () => { @@ -210,6 +240,24 @@ describe('PiAiAdapter provider routing', () => { } const readImage = vi.fn((_ref: ImageAttachmentRef): Promise => Promise.resolve({ ref, data: Uint8Array.of(1) })) + const readImageRequest = vi.fn(( + value: ImageAttachmentRef, + _policy: ImageRequestPolicy, + _signal?: AbortSignal, + ): Promise => ( + Promise.resolve({ + variantId: ImageVariantId(`sha256:${'b'.repeat(64)}`), + attachment: value, + data: Uint8Array.of(1), + mediaType: value.mediaType, + bytes: 1, + width: value.width, + height: value.height, + depth: 'uchar', + space: 'srgb', + hasAlpha: true, + }) + )) class LateAttachmentStore extends AttachmentStore { readonly imageLimits: ImageAttachmentLimits = { @@ -217,6 +265,7 @@ describe('PiAiAdapter provider routing', () => { maxImagesPerMessage: 1, maxMessageImageBytes: 1, maxImagePixels: 1, + maxImageDimension: 2000, mediaTypes: ['image/png'], } @@ -231,6 +280,14 @@ describe('PiAiAdapter provider routing', () => { readImage(value: ImageAttachmentRef): Promise { return readImage(value) } + + override readImageRequest( + value: ImageAttachmentRef, + policy: ImageRequestPolicy, + signal?: AbortSignal, + ): Promise { + return readImageRequest(value, policy, signal) + } } const ctx = new Context() @@ -250,7 +307,10 @@ describe('PiAiAdapter provider routing', () => { }) expect(result.finish.kind).toBe('error') - expect(readImage).toHaveBeenCalledWith(ref) + expect(readImageRequest).toHaveBeenCalledWith(ref, { + maxPixels: 2048 * 2048, + maxBytes: 1024 * 1024, + }, expect.any(AbortSignal)) expect(server.paths).toEqual(['/v1/responses']) }) @@ -391,7 +451,7 @@ describe('provider profile lifecycle', () => { }) expect(ctx.llm.providerRetryPolicy('anthropic')).toMatchObject({ mode: 'normal', - maxRetries: 2, + maxRetries: 5, }) await fiber.dispose() expect(ctx.llm.listProviders()).toEqual([]) @@ -596,6 +656,46 @@ describe('provider profile lifecycle', () => { expect(server.requests[1]).not.toHaveProperty('reasoning_effort') }) + it('keeps the system role on a declared route whose gateway rejects the developer one', async () => { + vi.stubEnv('PI_TEST_KEY', 'test-key') + const server = await mockServer([{ events: textEvents }, { events: textEvents }]) + const ctx = new Context() + await ctx.plugin(LlmRuntime) + await ctx.plugin(LlmPiAi, { + providers: { + 'acme-gateway': { + apiKeyEnv: 'PI_TEST_KEY', + api: 'openai-completions', + baseURL: `${server.url}/v1`, + models: [ + // pi-ai sends the system prompt as `developer` to a reasoning + // model whenever its URL detection says the endpoint is OpenAI — + // which is what an unrecognized private URL resolves to. Most + // OpenAI-compatible gateways reject that role. + { id: 'acme-think', reasoningEfforts: { off: null, high: 'high' }, compat: { supportsDeveloperRole: false } }, + { id: 'acme-guess', reasoningEfforts: { off: null, high: 'high' } }, + ], + }, + }, + }) + const roles = async (model: string): Promise => { + await assemble(ctx, { + provider: 'acme-gateway', + model, + reasoningEffort: ReasoningEffortId('high'), + system: 'you are a harness', + messages: [], + }) + const request = server.requests.at(-1) as { messages: { role: string }[] } + return request.messages.map(message => message.role) + } + + expect(await roles('acme-think')).toEqual(['system']) + // The switch is the only thing that changes it: the same route, same + // endpoint, same reasoning declaration still gets pi-ai's guess. + expect(await roles('acme-guess')).toEqual(['developer']) + }) + it('sends a declared off value as the effort parameter instead of omitting it', async () => { vi.stubEnv('PI_TEST_KEY', 'test-key') const server = await mockServer([{ events: textEvents }]) @@ -696,6 +796,7 @@ describe('provider profile lifecycle', () => { }) it('validates empty, underspecified, legacy-shaped, and explicitly blank profiles', () => { + expect(DEFAULT_MAX_REQUEST_IMAGE_BYTES).toBe(20 * 1024 * 1024) // Empty and omitted dicts are the dormant zero-route posture, not errors. expect(resolveProfiles({}).size).toBe(0) expect(resolveProfiles(undefined).size).toBe(0) @@ -709,6 +810,11 @@ describe('provider profile lifecycle', () => { expect(() => resolveProfiles({ openai: { provider: 'openai' } as never })).toThrow(/moved to the providers dict key/) expect(() => resolveProfiles({ openai: { baseURL: '' } })).toThrow(/empty baseURL/) expect(() => resolveProfiles({ openai: { apiKeyEnv: 'not-a-var!' } })).toThrow(/must match/) + expect(() => resolveProfiles({ openai: { maxRequestImageBytes: 0 } })).toThrow(/maxRequestImageBytes/) + expect(resolveProfiles({ openai: {} }).get('openai')?.maxRequestImageBytes) + .toBe(DEFAULT_MAX_REQUEST_IMAGE_BYTES) + expect(resolveProfiles({ openai: { maxRequestImageBytes: 1024 } }).get('openai')?.maxRequestImageBytes) + .toBe(1024) }) it.each(['maxRetries', 'maxRetryDelayMs'] as const)( @@ -730,6 +836,9 @@ describe('provider profile lifecycle', () => { { streamIdleTimeoutMs: 0 }, { streamIdleTimeoutMs: Number.NaN }, { streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1 }, + { maxRequestImageBytes: 0 }, + { maxRequestImageBytes: 1.5 }, + { maxRequestImageBytes: Number.NaN }, ] for (const entry of invalid) { const ctx = new Context() diff --git a/packages/llm/llm-pi-ai/tests/auth-double.ts b/packages/llm/llm-pi-ai/tests/auth-double.ts new file mode 100644 index 0000000000..a05c017d04 --- /dev/null +++ b/packages/llm/llm-pi-ai/tests/auth-double.ts @@ -0,0 +1,39 @@ +import type { Credential } from '@earendil-works/pi-ai' +import type { PiAiAuthInjection } from '../src/adapter.ts' + +/** + * The auth injectables for tests that exercise streaming rather than + * authentication: an in-process credential store and an ambient context that + * finds nothing. A test needing real records builds the store over + * `ctx.credentials` instead, through `credentialStoreFrom`. + * @param seed - credentials to start with, by pi-ai provider id. + * @returns the injection to hand `PiAiAdapter`, with its store readable. + */ +export function memoryAuth(seed: Record = {}): PiAiAuthInjection & { + stored: Map +} { + const stored = new Map(Object.entries(seed)) + return { + stored, + credentials: { + read: id => Promise.resolve(stored.get(id)), + list: () => Promise.resolve([...stored].map(([providerId, credential]) => ({ + providerId, + type: credential.type, + }))), + async modify(id, mutate) { + const next = await mutate(stored.get(id)) + if (next !== undefined) stored.set(id, next) + return stored.get(id) + }, + delete: (id) => { + stored.delete(id) + return Promise.resolve() + }, + }, + authContext: { + env: () => Promise.resolve(undefined), + fileExists: () => Promise.resolve(false), + }, + } +} diff --git a/packages/llm/llm-pi-ai/tests/auth.spec.ts b/packages/llm/llm-pi-ai/tests/auth.spec.ts new file mode 100644 index 0000000000..27a496a2c3 --- /dev/null +++ b/packages/llm/llm-pi-ai/tests/auth.spec.ts @@ -0,0 +1,172 @@ +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import LocalCredentialProvider from '@deepseek-ai/dsh-credentials-local' +import { credentialKey, credentialRef } from '@deepseek-ai/dsh-credentials' +import { authContextFrom, credentialStoreFrom, recordKeyFor } from '../src/auth.ts' + +const CODEX = recordKeyFor('openai-codex') + +const dirs: string[] = [] + +/** A context whose credential records live in a throwaway `$DSH_HOME`. */ +async function stored(): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dsh-pi-auth-')) + dirs.push(dir) + const ctx = new Context() + await ctx.plugin(LocalCredentialProvider, { path: join(dir, '.credentials.yaml'), watch: false }) + return ctx +} + +afterEach(async () => { + vi.unstubAllEnvs() + await Promise.all(dirs.splice(0).map(dir => rm(dir, { recursive: true, force: true }))) +}) + +describe('pi-ai credential store over harness records', () => { + it('reads nothing for a provider with no record', async () => { + const store = credentialStoreFrom(await stored()) + + await expect(store.read('openai-codex')).resolves.toBeUndefined() + }) + + it('round-trips an api-key credential field by field', async () => { + const ctx = await stored() + const store = credentialStoreFrom(ctx) + + await store.modify('cloudflare', () => + Promise.resolve({ type: 'api_key', key: 'sk-live', env: { ACCOUNT_ID: 'acct-1' } })) + + await expect(store.read('cloudflare')) + .resolves.toEqual({ type: 'api_key', key: 'sk-live', env: { ACCOUNT_ID: 'acct-1' } }) + await expect(ctx.credentials.readRecord(recordKeyFor('cloudflare'))) + .resolves.toEqual({ kind: 'api-key', key: 'sk-live', env: { ACCOUNT_ID: 'acct-1' } }) + }) + + it('stores an api-key credential carrying neither a key nor env', async () => { + const store = credentialStoreFrom(await stored()) + + await store.modify('bedrock', () => Promise.resolve({ type: 'api_key' })) + + await expect(store.read('bedrock')).resolves.toEqual({ type: 'api_key' }) + }) + + it('keeps an OAuth credential verbatim, refresh fields and all', async () => { + const ctx = await stored() + const store = credentialStoreFrom(ctx) + const granted = { type: 'oauth' as const, access: 'at', refresh: 'rt', expires: 42, accountId: 'acc' } + + await store.modify('openai-codex', () => Promise.resolve(granted)) + + await expect(store.read('openai-codex')).resolves.toEqual(granted) + await expect(ctx.credentials.readRecord(CODEX)).resolves.toEqual({ kind: 'grant', payload: granted }) + }) + + it('shows the mutation the current credential and leaves it alone when declined', async () => { + const store = credentialStoreFrom(await stored()) + await store.modify('openai-codex', () => + Promise.resolve({ type: 'oauth', access: 'first', refresh: 'r', expires: 1 })) + const seen: unknown[] = [] + + const unchanged = await store.modify('openai-codex', (current) => { + seen.push(current) + return Promise.resolve(undefined) + }) + + expect(seen).toEqual([{ type: 'oauth', access: 'first', refresh: 'r', expires: 1 }]) + expect(unchanged).toEqual({ type: 'oauth', access: 'first', refresh: 'r', expires: 1 }) + }) + + it('lists only the records this adapter family owns', async () => { + const ctx = await stored() + const store = credentialStoreFrom(ctx) + await store.modify('openai-codex', () => + Promise.resolve({ type: 'oauth', access: 'at', refresh: 'rt', expires: 1 })) + await store.modify('cloudflare', () => Promise.resolve({ type: 'api_key', key: 'k' })) + // Another plugin's record for a provider name this one also serves: its + // payload is written in a format pi-ai never agreed to. + await ctx.credentials.modifyRecord(credentialKey('llm-kimi', 'openai-codex'), () => + Promise.resolve({ kind: 'grant', payload: { theirs: true } })) + + await expect(store.list()).resolves.toEqual([ + { providerId: 'openai-codex', type: 'oauth' }, + { providerId: 'cloudflare', type: 'api_key' }, + ]) + }) + + it('forgets a credential on delete, and stays quiet when there was none', async () => { + const store = credentialStoreFrom(await stored()) + await store.modify('openai-codex', () => + Promise.resolve({ type: 'oauth', access: 'at', refresh: 'rt', expires: 1 })) + + await store.delete('openai-codex') + await store.delete('openai-codex') + + await expect(store.read('openai-codex')).resolves.toBeUndefined() + }) + + it('reads empty but refuses to write without a credentials service', async () => { + const store = credentialStoreFrom(new Context()) + + await expect(store.read('openai-codex')).resolves.toBeUndefined() + await expect(store.list()).resolves.toEqual([]) + await expect(store.modify('openai-codex', () => Promise.resolve({ type: 'api_key', key: 'k' }))) + .rejects.toThrow(/mounts no credentials service/) + await expect(store.delete('openai-codex')).rejects.toThrow(/mounts no credentials service/) + }) + + it('treats a provider id outside the record grammar as holding nothing', async () => { + const store = credentialStoreFrom(await stored()) + + // A hand-declared route key is an arbitrary settings dict key, and pi-ai + // reads it during auth resolution: the answer is "not signed in", never a + // thrown address error… + await expect(store.read('My_Proxy')).resolves.toBeUndefined() + // …nothing can ever be stored under it, so a logout has nothing to remove… + await expect(store.delete('My_Proxy')).resolves.toBeUndefined() + // …while a write that cannot land must refuse rather than report success. + await expect(store.modify('My_Proxy', () => Promise.resolve({ type: 'api_key', key: 'k' }))) + .rejects.toThrow(/cannot address a stored credential record/) + }) +}) + +describe('pi-ai ambient auth context', () => { + beforeEach(() => { + vi.stubEnv('PI_AUTH_AMBIENT', 'from-environment') + }) + + it('answers an environment name from the credential seam first', async () => { + const ctx = await stored() + await ctx.credentials.set(credentialRef('PI_AUTH_SEAM'), 'from-seam') + + await expect(authContextFrom(ctx).env('PI_AUTH_SEAM')).resolves.toBe('from-seam') + }) + + it('falls back to the launch environment when nothing is stored', async () => { + await expect(authContextFrom(await stored()).env('PI_AUTH_AMBIENT')).resolves.toBe('from-environment') + }) + + it('answers "not set" for a name no reference could ever address', async () => { + // pi-ai asks about provider-declared names; one outside the reference + // grammar has no reference to miss, and must not throw. + await expect(authContextFrom(await stored()).env('not a var')).resolves.toBeUndefined() + }) + + it('answers about the host filesystem, expanding a leading ~', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-pi-home-')) + dirs.push(dir) + await writeFile(join(dir, 'creds'), 'x') + // Both spellings of "home": os.homedir() reads HOME on POSIX and + // USERPROFILE on Windows, and the expansion under test goes through it. + vi.stubEnv('HOME', dir) + vi.stubEnv('USERPROFILE', dir) + const context = authContextFrom(await stored()) + + await expect(context.fileExists('~/creds')).resolves.toBe(true) + await expect(context.fileExists('~/missing')).resolves.toBe(false) + await expect(context.fileExists(join(dir, 'creds'))).resolves.toBe(true) + await expect(context.fileExists('~')).resolves.toBe(true) + }) +}) diff --git a/packages/llm/llm-pi-ai/tests/catalog.spec.ts b/packages/llm/llm-pi-ai/tests/catalog.spec.ts index eb4ba511d4..dd4d9e3480 100644 --- a/packages/llm/llm-pi-ai/tests/catalog.spec.ts +++ b/packages/llm/llm-pi-ai/tests/catalog.spec.ts @@ -3,7 +3,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import LlmRuntime, { createUserMessage } from '@deepseek-ai/dsh-llm' +import LlmRuntime, { createUserMessage, ReasoningEffortId } from '@deepseek-ai/dsh-llm' import type { StreamChunk } from '@deepseek-ai/dsh-llm' import FileSettingsProvider from '@deepseek-ai/dsh-settings-file' import { settingsNamespace } from '@deepseek-ai/dsh-settings' @@ -15,6 +15,7 @@ import type { Api, Model, OpenAICompletionsCompat, Provider } from '@earendil-wo import { resolveProfiles } from '../src/config.ts' import { buildProvider, supportedProtocols } from '../src/provider.ts' import { assemble } from './assemble.ts' +import { memoryAuth } from './auth-double.ts' import { closeMockServers, mockServer, textEvents } from './mock-server.ts' const homes: string[] = [] @@ -755,7 +756,7 @@ describe('modelOverrides', () => { }) }) -describe('reasoning-dispatch compat switches', () => { +describe('compat switches', () => { /** The materialized models of one route, keyed by id. */ function modelsOf(providers: Record, route: string): Map> { const models = resolveProfiles(providers).get(route)?.piProvider.getModels() ?? [] @@ -813,18 +814,244 @@ describe('reasoning-dispatch compat switches', () => { expect(models.get(responses.id)?.compat).toEqual(responses.compat) }) - it('rejects a model-level switch on a protocol that has no such field', () => { + it('rejects a model-level switch on a protocol that has no such field, naming what it offers', () => { expect(() => resolveProfiles({ anthropic: { models: [{ id: 'claude-sonnet-4-5', compat: { thinkingFormat: 'openai' } }], }, - })).toThrow(/exist only on openai-completions/) + })).toThrow(/its api is "anthropic-messages", which does not take it.*exists on openai-completions/s) }) it('rejects route switches no model on the route can take', () => { expect(() => resolveProfiles({ anthropic: { compat: { thinkingFormat: 'openai' } }, - })).toThrow(/no model on the route speaks openai-completions/) + })).toThrow(/no model on the route speaks a protocol that takes it/) + }) + + it('carries the developer-role switch onto a hand-declared reasoning model', () => { + // pi-ai reads this switch only for a reasoning model, and detects it from + // the endpoint URL — which for a private gateway answers as though it were + // OpenAI itself, so the route must be able to say otherwise. + const models = modelsOf({ + 'acme-gateway': { + api: 'openai-completions', + baseURL: 'https://acme.test', + compat: { supportsDeveloperRole: false, maxTokensField: 'max_tokens' }, + models: [{ id: 'acme-think', reasoningEfforts: { off: null, high: 'high' } }], + }, + }, 'acme-gateway') + + expect(models.get('acme-think')?.compat).toEqual({ + supportsDeveloperRole: false, + maxTokensField: 'max_tokens', + }) + }) + + it('carries a switch both OpenAI protocols declare onto an openai-responses route', () => { + const models = modelsOf({ + 'acme-responses': { + api: 'openai-responses', + baseURL: 'https://acme.test', + compat: { supportsDeveloperRole: false }, + models: [{ id: 'acme-r', reasoningEfforts: { off: null, high: 'high' } }], + }, + }, 'acme-responses') + + expect(models.get('acme-r')?.compat).toEqual({ supportsDeveloperRole: false }) + }) + + it('carries an anthropic-only switch onto an anthropic-messages route', () => { + const models = modelsOf({ + 'acme-claude': { + api: 'anthropic-messages', + baseURL: 'https://acme.test', + compat: { supportsTemperature: false, supportsCacheControlOnTools: false }, + models: [{ id: 'acme-opus' }], + }, + }, 'acme-claude') + + expect(models.get('acme-opus')?.compat).toEqual({ + supportsTemperature: false, + supportsCacheControlOnTools: false, + }) + }) + + it('lands each route switch only on the models whose protocol declares it', () => { + const catalog = getBuiltinModels('xai') as readonly Model[] + const completions = catalog.find(model => model.api === 'openai-completions') + const responses = catalog.find(model => model.api === 'openai-responses') + if (completions === undefined || responses === undefined) throw new Error('xai no longer ships a mixed catalog') + + const models = modelsOf({ + xai: { + // Both protocols take the first switch; only completions takes the second. + compat: { supportsDeveloperRole: false, thinkingFormat: 'openai' }, + models: [{ id: completions.id }, { id: responses.id }], + }, + }, 'xai') + + const onCompletions = models.get(completions.id)?.compat as OpenAICompletionsCompat + expect(onCompletions.supportsDeveloperRole).toBe(false) + expect(onCompletions.thinkingFormat).toBe('openai') + const onResponses = models.get(responses.id)?.compat as { supportsDeveloperRole?: boolean; thinkingFormat?: string } + expect(onResponses.supportsDeveloperRole).toBe(false) + expect(onResponses.thinkingFormat).toBeUndefined() + }) + + it('carries chat-template kwargs beside the thinking format that dispatches through them', () => { + const models = modelsOf({ + 'acme-qwen': { + api: 'openai-completions', + baseURL: 'https://acme.test', + models: [{ + id: 'qwen-local', + reasoningEfforts: { off: null, medium: 'medium' }, + compat: { + thinkingFormat: 'qwen-chat-template', + chatTemplateKwargs: { enable_thinking: { $var: 'thinking.enabled' } }, + }, + }], + }, + }, 'acme-qwen') + + expect(models.get('qwen-local')?.compat).toEqual({ + thinkingFormat: 'qwen-chat-template', + chatTemplateKwargs: { enable_thinking: { $var: 'thinking.enabled' } }, + }) + }) + + it('rejects a model switch on an unrecognized protocol as having no configurable compat', () => { + expect(() => resolveProfiles({ + 'acme-gateway': { + api: 'acme-chat', + baseURL: 'https://acme.test', + models: [{ id: 'acme-a', compat: { supportsStore: false } }], + }, + })).toThrow(/its api is "acme-chat", which does not take it.*"acme-chat" offers no configurable compat/s) + }) + + it('refuses a valueless compat key written through the composed settings path', async () => { + // The write path an operator reaches: a section resolved by schemastery, + // judged by this adapter's section validator before it is stored. + // schemastery keeps the null, so nothing but that check stands between it + // and `Model.compat`. + const dir = await home() + const ctx = await bootWithSettings(dir, {}) + await expect(ctx.settings.update(settingsNamespace('llm-pi-ai'), { + providers: { + 'acme-gateway': { + api: 'openai-completions', + baseURL: 'https://acme.test/v1', + compat: { supportsDeveloperRole: null }, + models: [{ id: 'acme-a' }], + }, + }, + })).rejects.toThrow(/compat "supportsDeveloperRole" with no value/) + }) + + it('carries a compat switch from a written settings section onto the wire', async () => { + // End to end for the reported gap: the switch enters as configuration and + // changes the request the provider receives, not merely the resolved model. + vi.stubEnv(KEY_ENV, 'test-key') + const server = await mockServer([{ events: textEvents }]) + const dir = await home() + const ctx = await bootWithSettings(dir, {}) + await ctx.settings.update(settingsNamespace('llm-pi-ai'), { + providers: { + 'acme-gateway': { + apiKeyEnv: KEY_ENV, + api: 'openai-completions', + baseURL: `${server.url}/v1`, + compat: { supportsDeveloperRole: false }, + models: [{ id: 'acme-think', reasoningEfforts: { off: null, high: 'high' } }], + }, + }, + }) + + await assemble(ctx, { + provider: 'acme-gateway', + model: 'acme-think', + reasoningEffort: ReasoningEffortId('high'), + system: 'you are a harness', + messages: [], + }) + + const request = server.requests[0] as { messages: { role: string }[] } + expect(request.messages.map(message => message.role)).toEqual(['system']) + }) + + it('refuses a valueless compat key rather than writing null over the catalog', () => { + // schemastery passes a YAML bare key through as null. Carried forward it + // would replace the installed entry's value, and pi-ai's `??` would then + // reach for its baseURL detection — the "written but not applied" outcome. + expect(() => resolveProfiles({ + 'acme-gateway': { + api: 'openai-completions', + baseURL: 'https://acme.test', + compat: { supportsDeveloperRole: null } as never, + models: [{ id: 'acme-a' }], + }, + })).toThrow(/compat "supportsDeveloperRole" with no value/) + }) + + it('refuses a compat key whose value is undefined, as a cordis.yml entry can write', () => { + // `!!js undefined` reaches the same state as a YAML bare key, and + // schemastery keeps the key either way, so both are refused together. + expect(() => resolveProfiles({ + 'acme-gateway': { + api: 'openai-completions', + baseURL: 'https://acme.test', + compat: { supportsDeveloperRole: undefined } as never, + models: [{ id: 'acme-a' }], + }, + })).toThrow(/compat "supportsDeveloperRole" with no value/) + }) + + it('refuses a valueless compat key on a model entry too', () => { + expect(() => resolveProfiles({ + deepseek: { + modelOverrides: { 'deepseek-v4-flash': { compat: { requiresReasoningContentOnAssistantMessages: null } } as never }, + }, + })).toThrow(/model "deepseek-v4-flash" sets compat "requiresReasoningContentOnAssistantMessages" with no value/) + }) + + it('serves the Responses compat type on every protocol pi-ai gives it to', () => { + // pi-ai types azure-openai-responses and openai-codex-responses with the + // same OpenAIResponsesCompat, so a switch settable on one is settable on all. + for (const route of ['azure-openai-responses', 'openai-codex']) { + const models = modelsOf({ [route]: { compat: { supportsDeveloperRole: false } } }, route) + const [first] = [...models.values()] + expect((first?.compat as { supportsDeveloperRole?: boolean }).supportsDeveloperRole).toBe(false) + } + }) + + it('serves the Bedrock compat type on its own protocol', () => { + const models = modelsOf({ 'amazon-bedrock': { compat: { supportsStrictMode: false } } }, 'amazon-bedrock') + const [first] = [...models.values()] + expect((first?.compat as { supportsStrictMode?: boolean }).supportsStrictMode).toBe(false) + }) + + it('refuses a compat key no wire protocol declares instead of dropping it', () => { + // Schemastery passes unknown keys through, so silently dropping one would + // make an unreadable switch look applied; the resolver must refuse it. + expect(() => resolveProfiles({ + 'acme-gateway': { + api: 'openai-completions', + baseURL: 'https://acme.test', + compat: { supportsDevelperRole: false } as never, + models: [{ id: 'acme-a' }], + }, + })).toThrow(/compat "supportsDevelperRole", which no wire protocol declares; the configurable switches are .*\bsupportsDeveloperRole\b/) + }) + + it('refuses a compat key pi-ai’s catalog owns, pointing at the catalog route', () => { + expect(() => resolveProfiles({ + 'acme-gateway': { + api: 'openai-completions', + baseURL: 'https://acme.test', + models: [{ id: 'acme-a', compat: { openRouterRouting: {} } as never }], + }, + })).toThrow(/compat "openRouterRouting", which is not configurable here/) }) }) @@ -839,6 +1066,7 @@ describe('resolution snapshots', () => { // Credential resolution is the real await inside a stream call, and the // window a configuration change has to land in. resolveApiKey: async () => { await held; return 'k' }, + auth: memoryAuth(), }) const chunks: StreamChunk[] = [] @@ -867,7 +1095,11 @@ describe('resolution snapshots', () => { const first = await mockServer([{ events: textEvents }]) const second = await mockServer([{ events: textEvents }]) let current = resolveProfiles({ deepseek: { baseURL: `${first.url}/v1` } }) - const adapter = new PiAiAdapter({ profiles: () => current, resolveApiKey: () => Promise.resolve('k') }) + const adapter = new PiAiAdapter({ + profiles: () => current, + resolveApiKey: () => Promise.resolve('k'), + auth: memoryAuth(), + }) const drain = async (): Promise => { for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'deepseek-v4-flash', messages: [], @@ -887,7 +1119,6 @@ describe('configurable-provider directory', () => { it('keeps the previous directory when a route collides with another adapter family', async () => { const dir = await home() const ctx = await bootWithSettings(dir, {}) - // Another adapter family owns this route id, exactly as llm-deepseek does. ctx.llm.registerConfigurableProviders([ { provider: 'deepseek-official', displayName: 'DeepSeek', settingsNs: 'llm-deepseek', settingsPath: [] }, ]) @@ -934,30 +1165,22 @@ describe('configurable-provider directory', () => { expect(ctx.llm.listConfigurableProviders()).toHaveLength(catalogOnly) }) - it('withholds a catalog route this adapter cannot authenticate', async () => { + it('offers every installed catalog route, including one that only signs in', async () => { const ctx = await harness({}) const offered = ctx.llm.listConfigurableProviders().map(entry => entry.provider) // `openai-codex` is the one installed provider that authenticates through - // OAuth alone. pi-ai resolves OAuth only from a *stored* credential, this - // adapter constructs its collection with no credential store, and nothing - // here runs a login flow — so every request on such a route fails with - // `Provider is not configured` before it goes out. Offering it would put a - // provider on the settings page that no amount of configuration can make - // work. - expect(offered).not.toContain('openai-codex') - // A provider that offers OAuth *beside* an api-key method keeps its entry: - // the key is a path this adapter can serve. + // OAuth alone. It is offered like any other because the collection now + // carries a durable credential store and a login flow writes into it, so + // the route has a posture that works rather than only one that fails. + expect(offered).toContain('openai-codex') expect(offered).toContain('anthropic') expect(offered).toContain('openai') }) - it('still lists a withheld route a stored profile names, as a catalog route', async () => { - // Withholding the offer must not strand a profile someone already stored: - // the route keeps its entry so a configuration surface can edit or delete - // it, and `declared` still answers catalog membership rather than the - // offer, so the page does not mislabel it as a route this deployment - // invented. + it('lists a route a stored profile names as a catalog route, not a declared one', async () => { + // `declared` answers catalog membership, so a profile stored against a + // route pi-ai ships is not mislabelled as one this deployment invented. const ctx = await harness({ providers: { 'openai-codex': { apiKeyEnv: KEY_ENV } } }) expect(ctx.llm.listConfigurableProviders()).toContainEqual({ diff --git a/packages/llm/llm-pi-ai/tests/config.spec.ts b/packages/llm/llm-pi-ai/tests/config.spec.ts index 55444228de..61a2535a9d 100644 --- a/packages/llm/llm-pi-ai/tests/config.spec.ts +++ b/packages/llm/llm-pi-ai/tests/config.spec.ts @@ -64,3 +64,26 @@ describe('modality schema boundary', () => { expect(absent.providers['acme-gateway']?.defaultInput).toEqual(['text']) }) }) + +describe('request image policy bounds', () => { + it.each([ + ['requestImagePixelBudget', 0, /requestImagePixelBudget must be a positive safe integer/], + ['requestImagePixelBudget', Number.MAX_SAFE_INTEGER + 1, /requestImagePixelBudget must be a positive safe integer/], + ['requestImageMaxBytes', 0, /requestImageMaxBytes must be a positive safe integer/], + ['requestImageMaxBytes', 1.5, /requestImageMaxBytes must be a positive safe integer/], + ] as const)('rejects %s=%s at service resolution', (field, value, message) => { + const programmatic = { + providers: { + 'acme-gateway': { + api: 'openai-completions', + baseURL: 'https://acme.test', + models: [{ id: 'm' }], + [field]: value, + }, + }, + } as unknown as Config + expect(() => { + assertServiceable(programmatic) + }).toThrow(message) + }) +}) diff --git a/packages/llm/llm-pi-ai/tests/context.spec.ts b/packages/llm/llm-pi-ai/tests/context.spec.ts index 01ed3d080c..026ca2c608 100644 --- a/packages/llm/llm-pi-ai/tests/context.spec.ts +++ b/packages/llm/llm-pi-ai/tests/context.spec.ts @@ -1,7 +1,12 @@ import { describe, expect, it, vi } from 'vitest' -import { AttachmentId } from '@deepseek-ai/dsh-attachment' -import type { AttachmentStore, ImageAttachmentRef } from '@deepseek-ai/dsh-attachment' -import { CallId, createMessage, createUserMessage } from '@deepseek-ai/dsh-llm' +import { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment' +import type { + AttachmentStore, + ImageAttachmentRef, + ImageRequestPolicy, + RequestImageAttachment, +} from '@deepseek-ai/dsh-attachment' +import { CallId, createMessage, createUserMessage, OFFLOADED_IMAGE_TEXT } from '@deepseek-ai/dsh-llm' import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm' import { toPiContext } from '../src/context.ts' import { toPiAssistant } from '../src/replay.ts' @@ -14,9 +19,34 @@ const ref: ImageAttachmentRef = { height: 1, } -const attachments = { - readImage: vi.fn(() => Promise.resolve({ ref, data: Uint8Array.of(1) })), -} as unknown as AttachmentStore +function requestImage(value: ImageAttachmentRef, data: Uint8Array): RequestImageAttachment { + return { + variantId: ImageVariantId(`sha256:${'b'.repeat(64)}`), + attachment: value, + data, + mediaType: value.mediaType, + bytes: data.byteLength, + width: value.width, + height: value.height, + depth: 'uchar', + space: 'srgb', + hasAlpha: value.mediaType === 'image/png', + } +} + +function projectionStore( + readImageRequest: ( + value: ImageAttachmentRef, + policy: ImageRequestPolicy, + signal?: AbortSignal, + ) => Promise = vi.fn((value: ImageAttachmentRef) => ( + Promise.resolve(requestImage(value, Uint8Array.of(1))) + )), +): AttachmentStore { + return { readImageRequest } as unknown as AttachmentStore +} + +const attachments = projectionStore() function request(messages: GenerateOptions['messages']): GenerateOptions { return { @@ -116,6 +146,7 @@ describe('pi-ai request context conversion', () => { { role: 'user', content: [ + { type: 'text', text: expect.stringContaining(`Image ${ref.attachmentId}`) as string }, { type: 'image', data: 'AQ==', mimeType: 'image/png' }, { type: 'text', text: 'caption' }, ], @@ -133,13 +164,204 @@ describe('pi-ai request context conversion', () => { role: 'toolResult', toolCallId: 'missing-call', toolName: 'unknown', - content: [{ type: 'image', data: 'AQ==', mimeType: 'image/png' }], + content: [ + { type: 'text', text: expect.stringContaining(`Image ${ref.attachmentId}`) as string }, + { type: 'image', data: 'AQ==', mimeType: 'image/png' }, + ], isError: true, timestamp: 0, }, ]) }) + it('recursively converts nested tool-result text and images', async () => { + const callId = CallId('nested-call') + const context = await toPiContext(request([user([{ + type: 'tool-result', + toolCallId: callId, + content: [ + { + type: 'tool-result', + toolCallId: callId, + content: [{ type: 'text', text: 'nested text' }], + }, + { + type: 'tool-result', + toolCallId: callId, + content: [{ type: 'image', attachment: ref }], + }, + ], + }])]), attachments) + + expect(context.messages).toEqual([{ + role: 'toolResult', + toolCallId: 'nested-call', + toolName: 'unknown', + content: [ + { type: 'text', text: 'nested text' }, + { type: 'text', text: expect.stringContaining(`Image ${ref.attachmentId}`) as string }, + { type: 'image', data: 'AQ==', mimeType: 'image/png' }, + ], + isError: false, + timestamp: 0, + }]) + }) + + it('flattens nested text-only tool results and ignores other block types without storage', () => { + const callId = CallId('nested-text') + expect(toPiContext(request([user([{ + type: 'tool-result', + toolCallId: callId, + content: [ + { type: 'chart', data: 'ignored' } as unknown as ContentBlock, + { + type: 'tool-result', + toolCallId: callId, + content: [{ type: 'text', text: 'nested' }], + }, + ], + }])]))).toMatchObject({ + messages: [{ + role: 'toolResult', + content: [{ type: 'text', text: 'nested' }], + }], + }) + }) + + it('replaces the oldest images with placeholders once the request payload bound is exceeded', async () => { + const readImageRequest = vi.fn((value: ImageAttachmentRef) => ( + Promise.resolve(requestImage(value, Uint8Array.of(1, 2, 3))) + )) + const store = projectionStore(readImageRequest) + const sized: ImageAttachmentRef = { ...ref, bytes: 3 } + const callId = CallId('shot-call') + // Three 3-byte images cost 4 base64 characters each (12 total); a bound of + // 8 forces exactly the oldest one out, including one nested in a tool result. + const context = await toPiContext(request([ + user([{ + type: 'tool-result', + toolCallId: callId, + content: [{ type: 'image', attachment: sized }], + }]), + user([{ type: 'image', attachment: sized }, { type: 'text', text: 'newer' }]), + user([{ type: 'image', attachment: sized }]), + ]), store, undefined, 8) + + expect(context.messages).toEqual([ + { + role: 'toolResult', + toolCallId: 'shot-call', + toolName: 'unknown', + content: [{ type: 'text', text: OFFLOADED_IMAGE_TEXT }], + isError: false, + timestamp: 0, + }, + { + role: 'user', + content: [ + { type: 'text', text: expect.stringContaining(`Image ${sized.attachmentId}`) as string }, + { type: 'image', data: 'AQID', mimeType: 'image/png' }, + { type: 'text', text: 'newer' }, + ], + timestamp: 0, + }, + { + role: 'user', + content: [ + { type: 'text', text: expect.stringContaining(`Image ${sized.attachmentId}`) as string }, + { type: 'image', data: 'AQID', mimeType: 'image/png' }, + ], + timestamp: 0, + }, + ]) + expect(readImageRequest).toHaveBeenCalledTimes(1) + }) + + it('does not prepare an old image removed by the conservative request projection', async () => { + const old = { ...ref, attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`), bytes: 3 } + const recent = { ...ref, attachmentId: AttachmentId(`sha256:${'d'.repeat(64)}`), bytes: 3 } + const readImageRequest = vi.fn((value: ImageAttachmentRef) => { + if (value.attachmentId === old.attachmentId) throw new Error('old image must not be read') + return Promise.resolve(requestImage(value, Uint8Array.of(1, 2, 3))) + }) + + const context = await toPiContext(request([user([ + { type: 'image', attachment: old }, + { type: 'image', attachment: recent }, + ])]), projectionStore(readImageRequest), undefined, 4) + + expect(context.messages[0]).toMatchObject({ + role: 'user', + content: [ + { type: 'text', text: OFFLOADED_IMAGE_TEXT }, + { type: 'text', text: expect.stringContaining(String(recent.attachmentId)) as string }, + { type: 'image' }, + ], + }) + expect(readImageRequest).toHaveBeenCalledTimes(1) + expect(readImageRequest.mock.calls[0]?.[0]).toEqual(recent) + }) + + it('keeps every image at exactly the payload bound and drops all of them when even the newest cannot fit', async () => { + const sized: ImageAttachmentRef = { ...ref, bytes: 3 } + const exact = await toPiContext(request([ + user([{ type: 'image', attachment: sized }]), + user([{ type: 'image', attachment: sized }]), + ]), attachments, undefined, 8) + expect(exact.messages).toEqual([ + { + role: 'user', + content: [expect.objectContaining({ type: 'text' }), expect.objectContaining({ type: 'image' })], + timestamp: 0, + }, + { + role: 'user', + content: [expect.objectContaining({ type: 'text' }), expect.objectContaining({ type: 'image' })], + timestamp: 0, + }, + ]) + + const readImageRequest = vi.fn((value: ImageAttachmentRef) => ( + Promise.resolve(requestImage(value, new Uint8Array(300))) + )) + const store = projectionStore(readImageRequest) + const oversized = await toPiContext(request([ + user([{ type: 'image', attachment: { ...ref, bytes: 300 } }]), + ]), store, undefined, 8) + // All-text content collapses to the string form; the placeholder still reaches the model. + expect(oversized.messages).toEqual([ + { role: 'user', content: OFFLOADED_IMAGE_TEXT, timestamp: 0 }, + ]) + expect(readImageRequest).not.toHaveBeenCalled() + }) + + it('offloads repeated image-block occurrences by position rather than shared object identity', async () => { + const sized: ImageAttachmentRef = { ...ref, bytes: 3 } + const shared: ContentBlock = { type: 'image', attachment: sized } + const readImageRequest = vi.fn((value: ImageAttachmentRef) => ( + Promise.resolve(requestImage(value, Uint8Array.of(1, 2, 3))) + )) + const store = projectionStore(readImageRequest) + const aliased = await toPiContext(request([user([shared, shared])]), store, undefined, 4) + const replayed = await toPiContext(request([user([ + { type: 'image', attachment: { ...sized } }, + { type: 'image', attachment: { ...sized } }, + ])]), store, undefined, 4) + + const expected = [{ + role: 'user', + content: [ + { type: 'text', text: OFFLOADED_IMAGE_TEXT }, + { type: 'text', text: expect.stringContaining(`Image ${sized.attachmentId}`) as string }, + { type: 'image', data: 'AQID', mimeType: 'image/png' }, + ], + timestamp: 0, + }] + expect(aliased.messages).toEqual(expected) + expect(replayed.messages).toEqual(expected) + expect(readImageRequest).toHaveBeenCalledTimes(2) + }) + it('keeps empty text-only users while separating result-only messages', () => { const callId = CallId('unknown-call') expect(toPiContext(request([ @@ -163,9 +385,14 @@ describe('pi-ai request context conversion', () => { }) it('handles in-history system and assistant messages explicitly on the image path', async () => { - await expect(toPiContext(request([ - history('system', [{ type: 'image', attachment: ref }]), - ]), attachments)).rejects.toMatchObject({ code: 'UNSUPPORTED_CONTENT' }) + for (const role of ['system', 'assistant'] as const) { + const readImageRequest = vi.fn() + const store = projectionStore(readImageRequest) + await expect(toPiContext(request([ + history(role, [{ type: 'image', attachment: ref }]), + ]), store, undefined, 1)).rejects.toMatchObject({ code: 'UNSUPPORTED_CONTENT' }) + expect(readImageRequest).not.toHaveBeenCalled() + } await expect(toPiContext(request([ history('system', [{ type: 'text', text: 'history system' }]), @@ -183,4 +410,5 @@ describe('pi-ai request context conversion', () => { history('assistant', [{ type: 'image', attachment: ref }]), )).toThrow(/assistant image output/) }) + }) diff --git a/packages/llm/llm-pi-ai/tests/convert.spec.ts b/packages/llm/llm-pi-ai/tests/convert.spec.ts index 5af58e6630..c540f2d50b 100644 --- a/packages/llm/llm-pi-ai/tests/convert.spec.ts +++ b/packages/llm/llm-pi-ai/tests/convert.spec.ts @@ -1,7 +1,7 @@ import { describe, expect, it, vi } from 'vitest' -import { AttachmentId } from '@deepseek-ai/dsh-attachment' -import type { AttachmentStore } from '@deepseek-ai/dsh-attachment' -import { createUserMessage, CallId, CONTEXT_WINDOW_EXCEEDED_CODE, EMPTY_RESPONSE_CODE, LlmError, createMessage } from '@deepseek-ai/dsh-llm' +import { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment' +import type { AttachmentStore, ImageAttachmentRef, ImageRequestPolicy, RequestImageAttachment } from '@deepseek-ai/dsh-attachment' +import { createUserMessage, CallId, CONTEXT_WINDOW_EXCEEDED_CODE, EMPTY_RESPONSE_CODE, createMessage } from '@deepseek-ai/dsh-llm' import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm' import type { AssistantMessage, AssistantMessageEvent, Usage } from '@earendil-works/pi-ai' import { toPiContext } from '../src/context.ts' @@ -43,6 +43,29 @@ async function collect(stream: AsyncIterable): Promise Promise): AttachmentStore { + return { readImageRequest } as unknown as AttachmentStore +} + describe('toPiContext', () => { it('maps system prompt, user text, and tools', () => { const context = toPiContext({ @@ -76,7 +99,9 @@ describe('toPiContext', () => { width: 1, height: 1, } - const readImage = vi.fn().mockResolvedValue({ ref: attachment, data: Uint8Array.of(1, 2, 3) }) + const readImageRequest = vi.fn((value: ImageAttachmentRef, _policy: ImageRequestPolicy) => ( + Promise.resolve(requestVersion(value)) + )) const context = await toPiContext({ provider: 'openai', model: 'gpt-4.1', @@ -84,13 +109,18 @@ describe('toPiContext', () => { content: [{ type: 'text', text: 'describe' }, { type: 'image', attachment }], source: { kind: 'plugin', plugin: 'test' }, })], - }, { readImage } as unknown as AttachmentStore) + }, attachmentStore(readImageRequest)) - expect(readImage).toHaveBeenCalledWith(attachment) + expect(readImageRequest).toHaveBeenCalledWith( + attachment, + { maxPixels: 2048 * 2048, maxBytes: 1024 * 1024 }, + undefined, + ) expect(context.messages[0]).toEqual({ role: 'user', content: [ { type: 'text', text: 'describe' }, + { type: 'text', text: expect.stringContaining(`Image ${attachment.attachmentId}`) as string }, { type: 'image', data: 'AQID', mimeType: 'image/png' }, ], timestamp: 0, @@ -105,7 +135,9 @@ describe('toPiContext', () => { width: 1, height: 1, } - const readImage = vi.fn().mockResolvedValue({ ref: attachment, data: Uint8Array.of(1, 2, 3) }) + const readImageRequest = vi.fn((value: ImageAttachmentRef, _policy: ImageRequestPolicy) => ( + Promise.resolve(requestVersion(value)) + )) const context = await toPiContext({ provider: 'openai', model: 'gpt-4.1', @@ -129,7 +161,7 @@ describe('toPiContext', () => { }], source: { kind: 'plugin', plugin: 'test' }, })], - }, { readImage } as unknown as AttachmentStore) + }, attachmentStore(readImageRequest)) expect(context.messages).toEqual([{ role: 'toolResult', @@ -138,6 +170,7 @@ describe('toPiContext', () => { content: [ { type: 'text', text: 'before' }, { type: 'text', text: 'middle' }, + { type: 'text', text: expect.stringContaining(`Image ${attachment.attachmentId}`) as string }, { type: 'image', data: 'AQID', mimeType: 'image/png' }, { type: 'text', text: 'after' }, ], @@ -415,35 +448,68 @@ describe('toPiContext', () => { expect(context.messages[0]).not.toHaveProperty('responseId') }) - it('rejects unsupported replay-state versions with a stable error code', () => { - try { - toPiContext({ - provider: 'deepseek', - model: 'm', - messages: [createMessage({ - role: 'assistant', - content: [{ type: 'text', text: 'done' }], - source: { - kind: 'model', - ...{ - provider: 'deepseek', - model: 'old', - replayState: { kind: 'pi-ai', version: 2 }, - }, + it('degrades unsupported replay-state versions to provider-neutral history', () => { + const onDegrade = vi.fn() + const context = toPiContext({ + provider: 'deepseek', + model: 'm', + messages: [createMessage({ + role: 'assistant', + content: [{ type: 'text', text: 'done' }], + source: { + kind: 'model', + ...{ + provider: 'deepseek', + model: 'old', + replayState: { response: { kind: 'pi-ai', version: 3 }, blocks: [] }, }, - })], - }) - expect.fail('expected invalid replay state') - } catch (error: unknown) { - expect(error).toBeInstanceOf(LlmError) - expect((error as LlmError).code).toBe('INVALID_REPLAY_STATE') - expect((error as Error).message).toContain('unsupported version 2') - } + }, + })], + }, undefined, onDegrade) + expect(context.messages[0]).toMatchObject({ + role: 'assistant', + api: 'dsh-foreign', + provider: 'deepseek', + model: 'old', + content: [{ type: 'text', text: 'done' }], + }) + expect(onDegrade).toHaveBeenCalledWith(expect.stringContaining('unsupported version 3')) }) - it('rejects replay metadata whose blocks do not match the durable content', () => { + it('degrades the flat pre-envelope replay state a legacy session log carries', () => { + const onDegrade = vi.fn() + const context = toPiContext({ + provider: 'deepseek', + model: 'm', + messages: [createMessage({ + role: 'assistant', + content: [{ type: 'text', text: 'done' }], + source: { + kind: 'model', + ...{ + provider: 'deepseek', + model: 'deepseek-v4-flash', + replayState: { + kind: 'pi-ai', + version: 1, + api: 'openai-completions', + provider: 'deepseek', + model: 'deepseek-v4-flash', + stopReason: 'stop', + blocks: [{ type: 'text' }], + }, + }, + }, + })], + }, undefined, onDegrade) + expect(context.messages[0]).toMatchObject({ role: 'assistant', api: 'dsh-foreign' }) + expect(onDegrade).toHaveBeenCalledWith(expect.stringContaining('expected a response object')) + }) + + it('degrades replay metadata whose blocks do not match the durable content', () => { + const onDegrade = vi.fn() const state = toPiReplayState(assistant({ content: [{ type: 'text', text: 'done' }] })) - expect(() => toPiContext({ + const context = toPiContext({ provider: 'deepseek', model: 'm', messages: [createMessage({ @@ -454,12 +520,19 @@ describe('toPiContext', () => { ...{ provider: 'deepseek', model: 'deepseek-v4-flash', replayState: state }, }, })], - })).toThrow(/block 0 does not match assistant content/) + }, undefined, onDegrade) + expect(context.messages[0]).toMatchObject({ + role: 'assistant', + api: 'dsh-foreign', + content: [{ type: 'thinking', thinking: 'done' }], + }) + expect(onDegrade).toHaveBeenCalledWith(expect.stringContaining('block 0 does not match assistant content')) }) - it('rejects replay metadata whose block count differs from durable content', () => { + it('degrades replay metadata whose block count differs from durable content', () => { + const onDegrade = vi.fn() const state = toPiReplayState(assistant()) - expect(() => toPiContext({ + const context = toPiContext({ provider: 'deepseek', model: 'm', messages: [createMessage({ @@ -470,66 +543,34 @@ describe('toPiContext', () => { ...{ provider: 'deepseek', model: 'deepseek-v4-flash', replayState: state }, }, })], - })).toThrow(/block count does not match assistant content/) + }, undefined, onDegrade) + expect(context.messages[0]).toMatchObject({ + role: 'assistant', + api: 'dsh-foreign', + provider: 'deepseek', + model: 'deepseek-v4-flash', + content: [{ type: 'text', text: 'done' }], + stopReason: 'stop', + }) + expect(onDegrade).toHaveBeenCalledWith(expect.stringContaining('block count does not match assistant content')) }) - const validReplay = { + const validResponse = { kind: 'pi-ai', - version: 1, + version: 2, api: 'openai-completions', provider: 'deepseek', model: 'deepseek-v4-flash', stopReason: 'stop', - blocks: [{ type: 'text' }], } + const validReplay = { response: validResponse, blocks: [{ type: 'text' }] } - it.each([ - ['provider', { ...validReplay, provider: 'openai' }], - ['model', { ...validReplay, model: 'deepseek-v4-pro' }], - ])('rejects replay metadata whose %s differs from assistant source', (field, replayState) => { - try { - toPiContext({ - provider: 'deepseek', - model: 'next-model', - messages: [createMessage({ - role: 'assistant', - content: [{ type: 'text', text: 'done' }], - source: { - kind: 'model', - ...{ provider: 'deepseek', model: 'deepseek-v4-flash', replayState }, - }, - })], - }) - expect.fail('expected invalid replay state') - } catch (error: unknown) { - expect(error).toBeInstanceOf(LlmError) - expect((error as LlmError).code).toBe('INVALID_REPLAY_STATE') - expect((error as Error).message).toContain(`${field} does not match assistant source`) - } - }) - - it.each([ - ['number state', 1, 'expected an object'], - ['null state', null, 'expected an object'], - ['array state', [], 'expected an object'], - ['unknown kind', { ...validReplay, kind: 'other' }, 'unknown state kind'], - ['non-string api', { ...validReplay, api: 1 }, 'api must be a non-empty string'], - ['empty provider', { ...validReplay, provider: '' }, 'provider must be a non-empty string'], - ['missing model', { ...validReplay, model: undefined }, 'model must be a non-empty string'], - ['unknown stop reason', { ...validReplay, stopReason: 'pause' }, 'unknown stopReason'], - ['non-string response model', { ...validReplay, responseModel: 1 }, 'responseModel must be a string'], - ['non-string response id', { ...validReplay, responseId: 1 }, 'responseId must be a string'], - ['non-array blocks', { ...validReplay, blocks: 'text' }, 'blocks must be an array'], - ['number block', { ...validReplay, blocks: [1] }, 'block 0 must be an object'], - ['null block', { ...validReplay, blocks: [null] }, 'block 0 must be an object'], - ['array block', { ...validReplay, blocks: [[]] }, 'block 0 must be an object'], - ['unknown block type', { ...validReplay, blocks: [{ type: 'audio' }] }, 'block 0 has an unknown type'], - ['non-string signature', { ...validReplay, blocks: [{ type: 'text', textSignature: 1 }] }, 'textSignature must be a string'], - ['non-boolean redaction', { ...validReplay, blocks: [{ type: 'reasoning', redacted: 'yes' }] }, 'redacted must be boolean'], - ])('rejects malformed replay state: %s', (_name, replayState, message) => { - expect(() => toPiContext({ + /** Convert with the given state and assert the message degraded to foreign with the given reason. */ + function expectDegraded(replayState: unknown, message: string): void { + const onDegrade = vi.fn() + const context = toPiContext({ provider: 'deepseek', - model: 'm', + model: 'next-model', messages: [createMessage({ role: 'assistant', content: [{ type: 'text', text: 'done' }], @@ -538,7 +579,45 @@ describe('toPiContext', () => { ...{ provider: 'deepseek', model: 'deepseek-v4-flash', replayState }, }, })], - })).toThrow(message) + }, undefined, onDegrade) + expect(context.messages[0]).toMatchObject({ + role: 'assistant', + api: 'dsh-foreign', + content: [{ type: 'text', text: 'done' }], + }) + expect(onDegrade).toHaveBeenCalledWith(expect.stringContaining(message)) + } + + it.each([ + ['provider', { ...validReplay, response: { ...validResponse, provider: 'openai' } }], + ['model', { ...validReplay, response: { ...validResponse, model: 'deepseek-v4-pro' } }], + ])('degrades replay metadata whose %s differs from assistant source', (field, replayState) => { + expectDegraded(replayState, `${field} does not match assistant source`) + }) + + it.each([ + ['number state', 1, 'expected a replay envelope'], + ['null state', null, 'expected a replay envelope'], + ['array state', [], 'expected a replay envelope'], + ['missing response', { blocks: [] }, 'expected a response object'], + ['array response', { ...validReplay, response: [] }, 'expected a response object'], + ['unknown kind', { ...validReplay, response: { ...validResponse, kind: 'other' } }, 'unknown state kind'], + ['non-string api', { ...validReplay, response: { ...validResponse, api: 1 } }, 'api must be a non-empty string'], + ['empty provider', { ...validReplay, response: { ...validResponse, provider: '' } }, 'provider must be a non-empty string'], + ['missing model', { ...validReplay, response: { ...validResponse, model: undefined } }, 'model must be a non-empty string'], + ['unknown stop reason', { ...validReplay, response: { ...validResponse, stopReason: 'pause' } }, 'unknown stopReason'], + ['non-string response model', { ...validReplay, response: { ...validResponse, responseModel: 1 } }, 'responseModel must be a string'], + ['non-string response id', { ...validReplay, response: { ...validResponse, responseId: 1 } }, 'responseId must be a string'], + ['missing blocks', { response: validResponse }, 'blocks must be an array'], + ['non-array blocks', { ...validReplay, blocks: 'text' }, 'blocks must be an array'], + ['number block', { ...validReplay, blocks: [1] }, 'block 0 must be an object'], + ['null block', { ...validReplay, blocks: [null] }, 'block 0 must be an object'], + ['array block', { ...validReplay, blocks: [[]] }, 'block 0 must be an object'], + ['unknown block type', { ...validReplay, blocks: [{ type: 'audio' }] }, 'block 0 has an unknown type'], + ['non-string signature', { ...validReplay, blocks: [{ type: 'text', textSignature: 1 }] }, 'textSignature must be a string'], + ['non-boolean redaction', { ...validReplay, blocks: [{ type: 'reasoning', redacted: 'yes' }] }, 'redacted must be boolean'], + ])('degrades malformed replay state: %s', (_name, replayState, message) => { + expectDegraded(replayState, message) }) }) @@ -565,12 +644,14 @@ describe('toStreamChunks', () => { type: 'finish', reason: { kind: 'stop' }, replayState: { - kind: 'pi-ai', - version: 1, - api: 'openai-completions', - provider: 'deepseek', - model: 'deepseek-v4-flash', - stopReason: 'stop', + response: { + kind: 'pi-ai', + version: 2, + api: 'openai-completions', + provider: 'deepseek', + model: 'deepseek-v4-flash', + stopReason: 'stop', + }, blocks: [{ type: 'text' }], }, }, @@ -614,12 +695,14 @@ describe('toStreamChunks', () => { type: 'finish', reason: { kind: 'tool-calls' }, replayState: { - kind: 'pi-ai', - version: 1, - api: 'openai-completions', - provider: 'deepseek', - model: 'deepseek-v4-flash', - stopReason: 'toolUse', + response: { + kind: 'pi-ai', + version: 2, + api: 'openai-completions', + provider: 'deepseek', + model: 'deepseek-v4-flash', + stopReason: 'toolUse', + }, blocks: [{ type: 'tool-call' }], }, }, @@ -729,6 +812,16 @@ describe('mapStopReason / mapUsage', () => { stopReason: 'error', errorMessage: 'HTTP 400: invalid input: temperature exceeds maximum allowed value', }))).toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } }) + expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 413: Payload Too Large' }))) + .toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } }) + expect(mapStopReason(assistant({ + stopReason: 'error', + errorMessage: 'Failed to buffer the request body: length limit exceeded', + }))).toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } }) + expect(mapStopReason(assistant({ + stopReason: 'error', + errorMessage: 'vector length limit exceeded', + }))).toMatchObject({ kind: 'error', failure: { code: 'PI_AI_ERROR' } }) }) it.each([ diff --git a/packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts b/packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts index 1019354156..2289d56574 100644 --- a/packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts +++ b/packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts @@ -9,6 +9,7 @@ import { LocalCredentialProvider } from '@deepseek-ai/dsh-credentials-local' import { settingsNamespace } from '@deepseek-ai/dsh-settings' import { FileSettingsProvider } from '@deepseek-ai/dsh-settings-file' import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' +import AuthorizationService from '@deepseek-ai/dsh-authorization' import { assemble } from './assemble.ts' import { closeMockServers, mockServer, textEvents } from './mock-server.ts' @@ -36,8 +37,11 @@ async function home(): Promise { return dir } -/** Real dynamic composition mirroring the deepseek twin's harness. */ -async function boot(dir: string, config: LlmPiAi.Config): Promise { +async function boot( + dir: string, + config: LlmPiAi.Config, + options: { authorization?: boolean } = {}, +): Promise { const ctx = new Context() cleanups.push(async () => { await ctx.fiber.dispose() @@ -45,17 +49,38 @@ async function boot(dir: string, config: LlmPiAi.Config): Promise { await ctx.plugin(LlmRuntime) await ctx.plugin(FileSettingsProvider, { path: join(dir, 'settings.yaml'), watch: false }) await ctx.plugin(LocalCredentialProvider, { path: join(dir, '.credentials.yaml'), watch: false }) + if (options.authorization === true) await ctx.plugin(AuthorizationService) await ctx.plugin(LlmPiAi, config) return ctx } +describe('login flows in a real composition', () => { + it('offers a sign-in for a provider no route names, once the seam is mounted', async () => { + const ctx = await boot(await home(), {}, { authorization: true }) + + // Zero routes configured: signing in is what makes a route worth adding, + // so the offer cannot wait for a profile to name the provider. + const codex = ctx.authorization.describe(LlmPiAi.recordKeyFor('openai-codex')) + expect(codex?.methods.map(method => method.id)).toEqual(['oauth']) + }) + + it('mounts without the seam, and simply offers no sign-in', async () => { + const ctx = await boot(await home(), {}) + + // A headless or ACP composition has no surface to sign in from; everything + // else this plugin does still works. + expect(ctx.get('authorization')).toBeUndefined() + expect(ctx.llm.listConfigurableProviders().length).toBeGreaterThan(0) + }) +}) + describe('request-level dynamic profiles', () => { it('mounts bare and dormant, then registers routes the moment settings supply providers', async () => { vi.stubEnv('PI_DYNAMIC_KEY', '') const dir = await home() await writeFile( join(dir, '.credentials.yaml'), - 'PI_DYNAMIC_KEY: pk-from-settings\nPI_LIVE_KEY: live-key\nPI_OTHER_KEY: other\n', + 'version: 1\nrefs:\n PI_DYNAMIC_KEY: pk-from-settings\n PI_LIVE_KEY: live-key\n PI_OTHER_KEY: other\n', { mode: 0o600 }, ) const server = await mockServer([{ events: textEvents }]) @@ -93,7 +118,7 @@ describe('request-level dynamic profiles', () => { const dir = await home() await writeFile( join(dir, '.credentials.yaml'), - 'PI_LIVE_KEY: live-key\nPI_OTHER_KEY: other\n', + 'version: 1\nrefs:\n PI_LIVE_KEY: live-key\n PI_OTHER_KEY: other\n', { mode: 0o600 }, ) const server = await mockServer([{ events: textEvents }]) @@ -122,7 +147,7 @@ describe('request-level dynamic profiles', () => { it('rotates the per-request credential referenced by apiKeyEnv', async () => { vi.stubEnv('PI_DYNAMIC_KEY', '') const dir = await home() - await writeFile(join(dir, '.credentials.yaml'), 'PI_DYNAMIC_KEY: pk-one\n', { mode: 0o600 }) + await writeFile(join(dir, '.credentials.yaml'), 'version: 1\nrefs:\n PI_DYNAMIC_KEY: pk-one\n', { mode: 0o600 }) const server = await mockServer([{ events: textEvents }, { events: textEvents }]) const ctx = await boot(dir, { providers: { deepseek: { apiKeyEnv: 'PI_DYNAMIC_KEY', baseURL: server.url } }, @@ -173,7 +198,7 @@ describe('request-level dynamic profiles', () => { const dir = await home() await writeFile( join(dir, '.credentials.yaml'), - 'PI_LIVE_KEY: live-key\nPI_OTHER_KEY: other\n', + 'version: 1\nrefs:\n PI_LIVE_KEY: live-key\n PI_OTHER_KEY: other\n', { mode: 0o600 }, ) const server = await mockServer([{ events: textEvents }, { events: textEvents }]) diff --git a/assets/community-wecom-survey.png b/packages/llm/llm-pi-ai/tests/fixtures/qr-code.png similarity index 100% rename from assets/community-wecom-survey.png rename to packages/llm/llm-pi-ai/tests/fixtures/qr-code.png diff --git a/packages/llm/llm-pi-ai/tests/loader-composition.spec.ts b/packages/llm/llm-pi-ai/tests/loader-composition.spec.ts index 0ed1eee440..6ca02f281a 100644 --- a/packages/llm/llm-pi-ai/tests/loader-composition.spec.ts +++ b/packages/llm/llm-pi-ai/tests/loader-composition.spec.ts @@ -16,13 +16,22 @@ import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' import Include from '@deepseek-ai/cordis-plugin-include' -import LlmRuntime from '@deepseek-ai/dsh-llm' +import LlmRuntime, { createMessage, createUserMessage } from '@deepseek-ai/dsh-llm' import LocalCredentialProvider from '@deepseek-ai/dsh-credentials-local' import FileSettingsProvider from '@deepseek-ai/dsh-settings-file' import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' import { assemble } from './assemble.ts' import { closeMockServers, mockServer, textEvents } from './mock-server.ts' +/** One text block, then a tool call truncated by the output-token ceiling. */ +const truncatedToolCallEvents = [ + '{"choices":[{"delta":{"role":"assistant","content":""},"index":0,"finish_reason":null}]}', + '{"choices":[{"delta":{"content":"partial"},"index":0,"finish_reason":null}]}', + '{"choices":[{"delta":{"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"echo","arguments":"{\\"text\\":"}}]},"index":0,"finish_reason":null}]}', + '{"choices":[{"delta":{},"index":0,"finish_reason":"length"}],"usage":{"prompt_tokens":3,"completion_tokens":4}}', + '[DONE]', +] + let root: string | undefined let context: Context | undefined @@ -40,7 +49,7 @@ async function loadComposition(): Promise<{ ctx: Context; settingsPath: string } root = await mkdtemp(join(tmpdir(), 'dsh-pi-composition-')) const settingsPath = join(root, 'settings.yaml') await writeFile(settingsPath, '# personal settings\n') - await writeFile(join(root, '.credentials.yaml'), 'PI_COMPOSITION_KEY: key-from-store\n', { mode: 0o600 }) + await writeFile(join(root, '.credentials.yaml'), 'version: 1\nrefs:\n PI_COMPOSITION_KEY: key-from-store\n', { mode: 0o600 }) const configPath = join(root, 'cordis.yml') await writeFile(configPath, [ @@ -113,4 +122,123 @@ describe('llm-pi-ai real dormant composition', () => { expect(result.message.content).toEqual([{ type: 'text', text: 'hello' }]) expect(server.headers[0]?.authorization).toBe('Bearer key-from-store') }) + + it('continues natively after max-token assembly drops a tool call, with pruned replay metadata', async () => { + vi.stubEnv('PI_COMPOSITION_KEY', '') + const server = await mockServer([ + { events: truncatedToolCallEvents }, + { events: textEvents }, + ]) + const { ctx, settingsPath } = await loadComposition() + await writeFile(settingsPath, [ + 'llm-pi-ai:', + ' providers:', + ' deepseek:', + ' apiKeyEnv: PI_COMPOSITION_KEY', + ` baseURL: ${server.url}`, + '', + ].join('\n')) + await vi.waitFor(() => { + expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['deepseek']) + }, { timeout: 5000 }) + + const truncated = await assemble(ctx, { + provider: 'deepseek', + model: 'deepseek-v4-flash', + messages: [], + }) + expect(truncated.finish).toEqual({ kind: 'max-tokens' }) + expect(truncated.message.content).toEqual([{ type: 'text', text: 'partial' }]) + expect(truncated.message.source).toEqual({ + kind: 'model', + provider: 'deepseek', + model: 'deepseek-v4-flash', + replayState: { + response: { + kind: 'pi-ai', + version: 2, + api: 'openai-completions', + provider: 'deepseek', + model: 'deepseek-v4-flash', + stopReason: 'length', + }, + blocks: [{ type: 'text' }], + }, + }) + + const continued = await assemble(ctx, { + provider: 'deepseek', + model: 'deepseek-v4-flash', + messages: [ + truncated.message, + createUserMessage({ content: [{ type: 'text', text: 'continue' }], source: { kind: 'user' } }), + ], + }) + expect(continued.message.content).toEqual([{ type: 'text', text: 'hello' }]) + expect(server.requests).toHaveLength(2) + expect(server.requests[1]).toMatchObject({ + messages: [ + { role: 'assistant', content: 'partial' }, + { role: 'user', content: 'continue' }, + ], + }) + const followup = server.requests[1] as { messages?: unknown[] } + expect(followup.messages?.[0]).not.toHaveProperty('tool_calls') + }) + + it('continues a legacy session whose stored replay state no longer matches its content', async () => { + vi.stubEnv('PI_COMPOSITION_KEY', '') + const server = await mockServer([{ events: textEvents }]) + const { ctx, settingsPath } = await loadComposition() + await writeFile(settingsPath, [ + 'llm-pi-ai:', + ' providers:', + ' deepseek:', + ' apiKeyEnv: PI_COMPOSITION_KEY', + ` baseURL: ${server.url}`, + '', + ].join('\n')) + await vi.waitFor(() => { + expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['deepseek']) + }, { timeout: 5000 }) + + // A pre-envelope session log entry: max-token assembly dropped the tool + // call from content while the flat v1 state still describes both blocks. + const poisoned = createMessage({ + role: 'assistant', + content: [{ type: 'text', text: 'partial' }], + source: { + kind: 'model', + ...{ + provider: 'deepseek', + model: 'deepseek-v4-flash', + replayState: { + kind: 'pi-ai', + version: 1, + api: 'openai-completions', + provider: 'deepseek', + model: 'deepseek-v4-flash', + stopReason: 'length', + blocks: [{ type: 'text' }, { type: 'tool-call' }], + }, + }, + }, + }) + const continued = await assemble(ctx, { + provider: 'deepseek', + model: 'deepseek-v4-flash', + messages: [ + poisoned, + createUserMessage({ content: [{ type: 'text', text: 'continue' }], source: { kind: 'user' } }), + ], + }) + expect(continued.finish).toEqual({ kind: 'stop' }) + expect(continued.message.content).toEqual([{ type: 'text', text: 'hello' }]) + expect(server.requests[0]).toMatchObject({ + messages: [ + { role: 'assistant', content: 'partial' }, + { role: 'user', content: 'continue' }, + ], + }) + }) }) diff --git a/packages/llm/llm-pi-ai/tests/login.spec.ts b/packages/llm/llm-pi-ai/tests/login.spec.ts new file mode 100644 index 0000000000..beb65ce188 --- /dev/null +++ b/packages/llm/llm-pi-ai/tests/login.spec.ts @@ -0,0 +1,198 @@ +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import AuthorizationService from '@deepseek-ai/dsh-authorization' +import type { AuthorizationInteraction, AuthorizationNotice, AuthorizationPrompt } from '@deepseek-ai/dsh-authorization' +import LocalCredentialProvider from '@deepseek-ai/dsh-credentials-local' +import type { CredentialKey } from '@deepseek-ai/dsh-credentials' +import type { AuthEvent, AuthInteraction, AuthPrompt, AuthType, Credential } from '@earendil-works/pi-ai' + +const login = vi.hoisted(() => vi.fn()) + +// The whole of what this module does with pi-ai is run one provider's login +// against a collection built with the harness store, so the collection is the +// boundary worth observing; a real login would open a browser. +vi.mock('@earendil-works/pi-ai', async importOriginal => ({ + ...await importOriginal(), + createModels: () => ({ setProvider: () => {}, login }), +})) + +const { credentialStoreFrom, authContextFrom, recordKeyFor } = await import('../src/auth.ts') +const { registerPiAiFlows } = await import('../src/login.ts') + +const CODEX = recordKeyFor('openai-codex') +const dirs: string[] = [] + +/** A context with the record store, the seam, and every pi-ai login flow. */ +async function harness(): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dsh-pi-login-')) + dirs.push(dir) + const ctx = new Context() + await ctx.plugin(LocalCredentialProvider, { path: join(dir, '.credentials.yaml'), watch: false }) + await ctx.plugin(AuthorizationService) + registerPiAiFlows(ctx, { credentials: credentialStoreFrom(ctx), authContext: authContextFrom(ctx) }) + return ctx +} + +/** An interaction recording everything a flow says, answering every question. */ +function surface(answer = 'typed'): AuthorizationInteraction & { + notices: AuthorizationNotice[] + prompts: AuthorizationPrompt[] +} { + const notices: AuthorizationNotice[] = [] + const prompts: AuthorizationPrompt[] = [] + return { + notices, + prompts, + notify: (notice) => { notices.push(notice) }, + prompt: (prompt) => { + prompts.push(prompt) + return Promise.resolve(answer) + }, + } +} + +/** Drive one attempt, letting the mocked login talk back through `converse`. */ +async function attempt( + ctx: Context, + converse: (interaction: AuthInteraction) => Promise, + request: { key?: CredentialKey; method?: string } = {}, +): Promise> { + const ui = surface() + login.mockImplementation(async (providerId: string, _type: AuthType, interaction: AuthInteraction) => { + await converse(interaction) + const granted: Credential = { type: 'oauth', access: 'at', refresh: 'rt', expires: 1 } + await credentialStoreFrom(ctx).modify(providerId, () => Promise.resolve(granted)) + return granted + }) + await expect(ctx.authorization.begin({ + key: request.key ?? CODEX, + interaction: ui, + ...request.method === undefined ? {} : { method: request.method }, + })).resolves.toEqual({ status: 'authorized' }) + return ui +} + +afterEach(async () => { + login.mockReset() + await Promise.all(dirs.splice(0).map(dir => rm(dir, { recursive: true, force: true }))) +}) + +describe('pi-ai login flows', () => { + it('offers one flow per installed provider, with the methods that provider ships', async () => { + const ctx = await harness() + const offered = ctx.authorization.list() + + // The OAuth-only provider is exactly the case this exists for: nothing + // else could ever configure it. + expect(offered.find(entry => entry.key === CODEX)?.methods) + .toEqual([{ id: 'oauth', label: expect.stringContaining('ChatGPT') as string }]) + // A provider offering both keeps both, the subscription login first. + expect(offered.find(entry => entry.key === recordKeyFor('anthropic'))?.methods.map(one => one.id)) + .toEqual(['oauth', 'api-key']) + // A key-only provider still gets a flow, because pi-ai collects the key + // through its own prompt rather than leaving it to the settings form. + expect(offered.find(entry => entry.key === recordKeyFor('deepseek'))?.methods.map(one => one.id)) + .toEqual(['api-key']) + }) + + it('runs the pi-ai auth type the chosen method names', async () => { + const ctx = await harness() + + await attempt(ctx, () => Promise.resolve()) + expect(login).toHaveBeenLastCalledWith('openai-codex', 'oauth', expect.anything()) + + await attempt(ctx, () => Promise.resolve(), { key: recordKeyFor('anthropic'), method: 'api-key' }) + expect(login).toHaveBeenLastCalledWith('anthropic', 'api_key', expect.anything()) + }) + + it('commits what the login produced, where the adapter reads it back', async () => { + const ctx = await harness() + + await attempt(ctx, () => Promise.resolve()) + + await expect(ctx.credentials.readRecord(CODEX)).resolves.toEqual({ + kind: 'grant', + payload: { type: 'oauth', access: 'at', refresh: 'rt', expires: 1 }, + }) + }) + + it('restates every pi-ai login event in the neutral vocabulary', async () => { + const ctx = await harness() + const events: AuthEvent[] = [ + { type: 'info', message: 'Read this first', links: [{ url: 'https://help.example' }] }, + { type: 'info', message: 'Nothing to open' }, + { type: 'auth_url', url: 'https://auth.example/start', instructions: 'Approve in the tab' }, + { type: 'auth_url', url: 'https://auth.example/plain' }, + { type: 'device_code', userCode: 'WXYZ-1234', verificationUri: 'https://device.example' }, + { type: 'progress', message: 'Exchanging the code' }, + // pi-ai's event union is open; an unrecognised member must still show + // the human that something is happening. + { type: 'quantum-handshake' } as unknown as AuthEvent, + ] + + const ui = await attempt(ctx, (interaction) => { + for (const event of events) interaction.notify(event) + return Promise.resolve() + }) + + expect(ui.notices).toEqual([ + { message: 'Read this first', url: 'https://help.example' }, + { message: 'Nothing to open' }, + { message: 'Approve in the tab', url: 'https://auth.example/start' }, + { message: 'Open this page to continue signing in.', url: 'https://auth.example/plain' }, + { + message: 'Enter this code on the verification page to finish signing in.', + url: 'https://device.example', + code: 'WXYZ-1234', + }, + { message: 'Exchanging the code' }, + { message: 'Signing in…' }, + ]) + }) + + it('restates every pi-ai prompt, carrying the per-prompt withdrawal signal', async () => { + const ctx = await harness() + const withdraw = new AbortController() + const prompts: AuthPrompt[] = [ + { type: 'text', message: 'Your workspace', placeholder: 'acme' }, + { type: 'secret', message: 'Paste the key' }, + { type: 'secret', message: 'Paste the token', placeholder: 'sk-…' }, + { type: 'select', message: 'Which account?', options: [{ id: 'a', label: 'Work' }] }, + // The manual-code question a browser callback can win the race against. + { type: 'manual_code', message: 'Paste the code', signal: withdraw.signal }, + ] + + const ui = await attempt(ctx, async (interaction) => { + for (const prompt of prompts) await interaction.prompt(prompt) + }) + + expect(ui.prompts).toEqual([ + { kind: 'text', message: 'Your workspace', placeholder: 'acme' }, + { kind: 'secret', message: 'Paste the key' }, + { kind: 'secret', message: 'Paste the token', placeholder: 'sk-…' }, + { kind: 'select', message: 'Which account?', options: [{ id: 'a', label: 'Work' }] }, + { kind: 'text', message: 'Paste the code', signal: withdraw.signal }, + ]) + }) + + it('hands the flow the attempt-wide cancellation signal', async () => { + const ctx = await harness() + let seen: AbortSignal | undefined + const controller = new AbortController() + login.mockImplementation((_id: string, _type: AuthType, interaction: AuthInteraction) => { + seen = interaction.signal + controller.abort() + return new Promise(() => {}) + }) + + await expect(ctx.authorization.begin({ + key: CODEX, + interaction: surface(), + signal: controller.signal, + })).resolves.toEqual({ status: 'cancelled' }) + expect(seen?.aborted).toBe(true) + }) +}) diff --git a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts index b1731d7071..f651aa1f9a 100644 --- a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts +++ b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts @@ -1,17 +1,19 @@ import { readFile } from 'node:fs/promises' import { afterEach, describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import { AttachmentId, AttachmentStore } from '@deepseek-ai/dsh-attachment' +import { AttachmentId, AttachmentStore, ImageVariantId } from '@deepseek-ai/dsh-attachment' import type { ImageAttachmentLimits, ImageAttachmentRef, + ImageRequestPolicy, + RequestImageAttachment, SaveImageAttachment, StoredImageAttachment, } from '@deepseek-ai/dsh-attachment' import LlmRuntime, { createUserMessage, CallId } from '@deepseek-ai/dsh-llm' import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm' import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' -import type { PiAiReplayState } from '../src/replay.ts' +import type { PiAiReplayResponse } from '../src/replay.ts' import { assemble, type AssembledResult } from './assemble.ts' interface ProviderCase { @@ -70,6 +72,7 @@ async function harness(image?: StoredImageAttachment): Promise { maxImagesPerMessage: 1, maxMessageImageBytes: fixture.data.byteLength, maxImagePixels: fixture.ref.width * fixture.ref.height, + maxImageDimension: Math.max(fixture.ref.width, fixture.ref.height), mediaTypes: [fixture.ref.mediaType], } @@ -87,6 +90,24 @@ async function harness(image?: StoredImageAttachment): Promise { } return Promise.resolve(fixture) } + + override readImageRequest(ref: ImageAttachmentRef, _policy: ImageRequestPolicy): Promise { + if (ref.attachmentId !== fixture.ref.attachmentId) { + return Promise.reject(new Error('unknown e2e attachment fixture')) + } + return Promise.resolve({ + variantId: ImageVariantId(`sha256:${'f'.repeat(64)}`), + attachment: fixture.ref, + data: fixture.data, + mediaType: fixture.ref.mediaType, + bytes: fixture.data.byteLength, + width: fixture.ref.width, + height: fixture.ref.height, + depth: 'uchar', + space: 'srgb', + hasAlpha: fixture.ref.mediaType === 'image/png', + }) + } } await ctx.plugin(E2eAttachmentStore) } @@ -118,18 +139,20 @@ function expectFinish(result: AssembledResult, expected: 'stop' | 'tool-calls'): expect(result.finish.kind).toBe(expected) } -function expectNativeReplay(result: AssembledResult, profile: ProviderCase): PiAiReplayState { +function expectNativeReplay(result: AssembledResult, profile: ProviderCase): PiAiReplayResponse { const replayState = result.message.source.kind === 'model' ? result.message.source.replayState : undefined expect(replayState).toMatchObject({ - kind: 'pi-ai', - version: 1, - api: profile.api, - provider: profile.provider, - model: profile.model, + response: { + kind: 'pi-ai', + version: 2, + api: profile.api, + provider: profile.provider, + model: profile.model, + }, }) - return replayState as PiAiReplayState + return (replayState as { response: PiAiReplayResponse }).response } const lookupTool: ToolSchema = { @@ -207,7 +230,7 @@ for (const profile of providerCases) { if (profile.provider === 'anthropic') { it('sends a real image through the authenticated Anthropic visual path', async () => { const data = new Uint8Array(await readFile( - new URL('../../../../assets/community-wecom-survey.png', import.meta.url), + new URL('./fixtures/qr-code.png', import.meta.url), )) const ref: ImageAttachmentRef = { attachmentId: AttachmentId(`sha256:${'a'.repeat(64)}`), diff --git a/packages/llm/llm-pi-ai/tests/sdk-options.spec.ts b/packages/llm/llm-pi-ai/tests/sdk-options.spec.ts index 317b803f0c..2f409c6d9d 100644 --- a/packages/llm/llm-pi-ai/tests/sdk-options.spec.ts +++ b/packages/llm/llm-pi-ai/tests/sdk-options.spec.ts @@ -13,6 +13,7 @@ vi.mock('@earendil-works/pi-ai/api/openai-completions.lazy', () => ({ import { PiAiAdapter } from '../src/adapter.ts' import { resolveProfiles } from '../src/config.ts' +import { memoryAuth } from './auth-double.ts' afterEach(() => { streamSimple.mockReset() }) @@ -27,6 +28,7 @@ function gatewayAdapter(): PiAiAdapter { }, }), resolveApiKey: () => Promise.resolve('test-key'), + auth: memoryAuth(), }) } diff --git a/packages/llm/llm-pi-ai/tsconfig.json b/packages/llm/llm-pi-ai/tsconfig.json index 6f0b821d36..172dbe8a6e 100644 --- a/packages/llm/llm-pi-ai/tsconfig.json +++ b/packages/llm/llm-pi-ai/tsconfig.json @@ -29,6 +29,9 @@ { "path": "../../credentials/credentials" }, + { + "path": "../../credentials/authorization" + }, { "path": "../../settings/settings" }, diff --git a/packages/llm/llm-retry/README.i18n.yaml b/packages/llm/llm-retry/README.i18n.yaml index fd2395c33f..2c468f9bc1 100644 --- a/packages/llm/llm-retry/README.i18n.yaml +++ b/packages/llm/llm-retry/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/llm-retry/README.md -README.md: 0a907b5505650f7c2cd5e9933750be3701f8e34b -README.zh.md: 024545e1ed52bb8d4aeac8114ffafc26a04a2ddb +README.md: 1dd530051e387230510414e35701db10c7c7c063 +README.zh.md: 52badf6a6e7d2c2571fa951865f6b4b3f08ae4ad diff --git a/packages/llm/llm-retry/README.md b/packages/llm/llm-retry/README.md index 0a907b5505..1dd530051e 100644 --- a/packages/llm/llm-retry/README.md +++ b/packages/llm/llm-retry/README.md @@ -4,7 +4,7 @@ English | [中文](README.zh.md) Function plugin that applies exact-provider retry policy through the agent loop's closed-step `agent/request-error` waterfall. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered turn. -Each provider adapter owns an optional nested `retryPolicy`, captured when its route registers on `ctx.llm` and carried with each call that reaches that registration's final adapter boundary. An in-flight failure retains that serving policy if the route is later disposed or replaced; a failure before any final adapter is selected has no provider policy and delegates. Omission uses normal mode: two retries for `EMPTY_RESPONSE`, `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, with bounded exponential backoff from 500 ms to 10 seconds and 10 percent jitter. `EMPTY_RESPONSE` is the adapters' classification of a degenerate provider completion that produced no durable content, so repeating it is safe. A normal policy can change its finite budget, eligible codes, and backoff. Always mode asks downstream recovery first, then retries every model-request failure without an attempt limit; success, cancellation, or plugin disposal stops it after active delegated recovery reaches quiescence. +Each provider adapter owns an optional nested `retryPolicy`, captured when its route registers on `ctx.llm` and carried with each call that reaches that registration's final adapter boundary. An in-flight failure retains that serving policy if the route is later disposed or replaced; a failure before any final adapter is selected has no provider policy and delegates. Omission uses normal mode: five retries for `EMPTY_RESPONSE`, `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, with bounded exponential backoff from 500 ms to 10 seconds and 10 percent jitter. `EMPTY_RESPONSE` is the adapters' classification of a degenerate provider completion that produced no durable content, so repeating it is safe. A normal policy can change its finite budget, eligible codes, and backoff. Always mode asks downstream recovery first, then retries every model-request failure without an attempt limit; success, cancellation, or plugin disposal stops it after active delegated recovery reaches quiescence. Both modes use bounded exponential backoff with symmetric jitter. A valid `providerRetryAfterMs` at or below `maxDelayMs` replaces local backoff without jitter. An over-cap provider delay makes normal mode delegate, while always mode uses its configured local backoff so it cannot terminate on that instruction. diff --git a/packages/llm/llm-retry/README.zh.md b/packages/llm/llm-retry/README.zh.md index 024545e1ed..52badf6a6e 100644 --- a/packages/llm/llm-retry/README.zh.md +++ b/packages/llm/llm-retry/README.zh.md @@ -4,7 +4,7 @@ 一个函数插件,通过 agent loop(智能体循环)在已关闭步骤上触发的 `agent/request-error` waterfall(瀑布式事件)应用确切提供方重试策略。它不包装 `ctx.llm.stream()`:每次适配器调用仍是一次提供方尝试,每次重试都会开启新的编号轮次。 -每个提供方适配器都拥有可选的嵌套 `retryPolicy`;路由在 `ctx.llm` 上注册时会捕获该策略,任何到达该注册最终适配器边界的调用都会携带它。如果之后释放或替换路由,进行中的失败仍会保留当时为其提供服务的策略;在选中任何最终适配器前发生的失败没有提供方策略,会继续委托。省略策略时使用 normal mode:为 `EMPTY_RESPONSE`、`RATE_LIMIT`、`SERVER`、`TIMEOUT` 和 `TRANSPORT` 重试两次,并采用从 500 ms 到 10 秒的有界指数退避与 10% jitter。`EMPTY_RESPONSE` 是适配器对未产生任何持久内容的退化提供方完成所作的分类,因此可安全重复。normal 策略可以更改其有限预算、符合条件的 code 和退避配置。always mode 会先请求下游恢复,再无次数上限地重试每个模型请求失败;成功、取消或插件 dispose(资源释放)会在活跃的委托恢复完全停稳后终止它。 +每个提供方适配器都拥有可选的嵌套 `retryPolicy`;路由在 `ctx.llm` 上注册时会捕获该策略,任何到达该注册最终适配器边界的调用都会携带它。如果之后释放或替换路由,进行中的失败仍会保留当时为其提供服务的策略;在选中任何最终适配器前发生的失败没有提供方策略,会继续委托。省略策略时使用 normal mode:为 `EMPTY_RESPONSE`、`RATE_LIMIT`、`SERVER`、`TIMEOUT` 和 `TRANSPORT` 重试五次,并采用从 500 ms 到 10 秒的有界指数退避与 10% jitter。`EMPTY_RESPONSE` 是适配器对未产生任何持久内容的退化提供方完成所作的分类,因此可安全重复。normal 策略可以更改其有限预算、符合条件的 code 和退避配置。always mode 会先请求下游恢复,再无次数上限地重试每个模型请求失败;成功、取消或插件 dispose(资源释放)会在活跃的委托恢复完全停稳后终止它。 两种 mode 都使用带对称 jitter 的有界指数退避。有效 `providerRetryAfterMs` 不超过 `maxDelayMs` 时会替换本地退避,并且不加 jitter。超出上限的提供方延迟会使 normal mode 继续委托;always mode 则改用已配置的本地退避,避免该指令终止重试。 diff --git a/packages/llm/llm-retry/package.json b/packages/llm/llm-retry/package.json index e6c3013da1..e66c909078 100644 --- a/packages/llm/llm-retry/package.json +++ b/packages/llm/llm-retry/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-llm-retry", "description": "Provider-routed LLM request retry policy for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/llm/llm/README.i18n.yaml b/packages/llm/llm/README.i18n.yaml index 4ac51d768e..08820bbfd9 100644 --- a/packages/llm/llm/README.i18n.yaml +++ b/packages/llm/llm/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/llm/README.md -README.md: 2cae9a05a58295b06d25382729a3304dbdc9a6fa -README.zh.md: bf86e703fc15231c6845364dc383b7fc6ad77297 +README.md: 59c5303bdae8d6391f3bb1595a527d677e2378bc +README.zh.md: 313fc23590de2119bf07dd5c4c4fafe9daa94c56 diff --git a/packages/llm/llm/README.md b/packages/llm/llm/README.md index 2cae9a05a5..59c5303bda 100644 --- a/packages/llm/llm/README.md +++ b/packages/llm/llm/README.md @@ -8,6 +8,10 @@ Provider-neutral LLM vocabulary and abstract service. This package defines the c An adapter registry plus a single streaming call API, interceptable via a waterfall event. +### Retry policy + +Each provider adapter supplies its resolved route policy. Omitting provider configuration uses bounded normal mode with five retries after the first request. Layered configuration may retain `maxRetries` or `retryableCodes` after changing `mode` to `always`; resolution ignores those inactive normal-mode fields and captures a pure always policy. This service stores the effective policy but does not execute retries. + ### Public API - `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber. The returned disposer also carries `replace(providers)`: the candidate route set is validated in full before anything moves, so a conflict with another adapter leaves the current routes registered and serving, and the swap itself is one synchronous section with no observable gap. `replace([])` is legal — a registration holding zero routes — unlike an empty initial registration. @@ -21,7 +25,7 @@ An adapter registry plus a single streaming call API, interceptable via a waterf - `ctx.llm.listModels(provider: string): Promise` Discover the models one registered provider currently advertises. - `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise` Resolve validated exact-model identity plus available context, output-default, and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters. - `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise` Validate an explicit effort and materialize adapter-configured call defaults without clamping. -- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise` Resolve a config plus detached context metadata and markers for fields supplied by adapter defaults in one exact-model lookup, then capture its current adapter registration and immutable retry policy as one cancellable, one-shot call. +- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise` Resolve a config plus detached context and modality metadata and markers for fields supplied by adapter defaults in one exact-model lookup, then capture the adapter's matching dispatch generation and immutable retry policy as one cancellable, one-shot call. - `ctx.llm.stream(options: GenerateOptions): AsyncIterable` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`. `LlmRuntime` normalizes failures from final adapter selection, synchronous dispatch, iterator construction, and iteration into the stream protocol's single terminal form: `finish { kind: 'error' | 'aborted', failure }`. A failure after partial deltas may leave content blocks open; consumers discard that incomplete output. Errors from `llm/stream` middleware, nested calls, adapter cleanup, and downstream consumers remain thrown because they are plugin or consumer failures rather than model-request outcomes. A prepared call exposes the immutable retry policy captured with its exact adapter registration; a route handled entirely by middleware has no serving policy. @@ -34,7 +38,7 @@ Every topology commit point — adapter routes registering or disposing, directo Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context`, `defaultMaxTokens`, or `reasoning` fields preserve unknown capacity, provider-owned output defaults, or unavailable reasoning capability. Invalid identity, context, output default, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, `INVALID_MODEL_MAX_TOKENS`, or `INVALID_MODEL_REASONING`. -`defaultMaxTokens` is an adapter-configured per-request output cap, not a model hard limit. `resolveCallConfig()` materializes it only when the request omits `maxTokens`; an explicit cap wins. Reasoning identifiers are opaque adapter-owned strings rather than a core enum: the same resolution accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally exposes detached context metadata from the same lookup, reports which `maxTokens` and `reasoningEffort` fields it materialized in `adapterDefaults`, and retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O. +`defaultMaxTokens` is an adapter-configured per-request output cap, not a model hard limit. `resolveCallConfig()` materializes it only when the request omits `maxTokens`; an explicit cap wins. Reasoning identifiers are opaque adapter-owned strings rather than a core enum: the same resolution accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally exposes detached context and input-modality metadata, reports which `maxTokens` and `reasoningEffort` fields it materialized in `adapterDefaults`, and binds those facts to the adapter generation that performs terminal dispatch. HMR or dynamic settings therefore cannot combine one generation's image capability with another generation's endpoint; reusing the one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O. ### Events @@ -51,9 +55,11 @@ Exact-model metadata is a separate correctness query, not a catalog decoration o `Message` is the shared immutable value used by delivery, durable history, and model requests. Every message has a required `MessageId`, role, content, and typed source from creation onward. `createMessage(input)` mints the identity and returns a detached deep-frozen value; `createUserMessage({ content, source })` fixes the user role; `createAssistantMessage({ content, source })` fixes the assistant role and model source kind; `createToolResultMessage({ callId, content, isError })` fixes the user role and couples the tool source to its result block; `freezeMessage(message)` imports an identity that already exists and never replaces it. Message rewrites preserve the identity and produce another frozen value. Browser code imports these value constructors from the dependency-minimal `@deepseek-ai/dsh-llm/message` entry instead of the service-bearing package root. -Message content is an array of typed blocks: `text`, `reasoning`, `tool-call`, `tool-result`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. Assistant messages use a model source carrying the provider and model that produced them plus optional adapter-private replay state. Before dispatch, `LlmRuntime` retains that state only when the historical provider route and target provider route are currently owned by the exact same adapter instance; the adapter then decides whether it can restore or convert the state across models/providers. The core block set is limited to blocks every shipping path honors — multimodal content (images, audio, …) has no core block type; a feature that needs one adds it via the map together with the adapter/UI/compaction support that honors it. +Message content is an array of typed blocks: `text`, `reasoning`, `image`, `tool-call`, `tool-result`. An `ImageBlock` carries only a durable `ImageAttachmentRef`; provider bytes and request dimensions are resolved later. The union remains merge-extensible through `ContentBlockMap`, so plugins can add further block types via declaration merging. Assistant messages use a model source carrying the provider and model that produced them plus optional adapter-private replay state. Before dispatch, `LlmRuntime` retains that state only when the historical provider route and target provider route are currently owned by the exact same adapter instance; the adapter then decides whether it can restore or convert the state across models/providers. -Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). Every adapter outcome reaches consumers as one terminal `finish`; operational failure uses its `error` or `aborted` reason rather than throwing across the stream API. `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages. +Every dispatch uses the exact model modalities captured with its adapter generation. An image-capable adapter projects durable image references into route-specific request versions. A text-only route instead receives deterministic attachment placeholders, including nested tool-result images, without changing append-only session history. `offloadRequestImagesWithPolicy()` provides deterministic oldest-first image removal with raw or base64 accounting and count or byte quanta; adapters supply the exact derived-version byte length. + +Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). Every adapter outcome reaches consumers as one terminal `finish`; operational failure uses its `error` or `aborted` reason rather than throwing across the stream API. `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages. A successful `finish` may carry a `ReplayEnvelope` — opaque response-level replay metadata plus optional per-block entries aligned with the emitted block sequence. Assembly makes one keep/drop decision for content and metadata together: a `max-tokens` finish drops tool calls that may have been truncated, and the envelope loses the entry at each dropped position, so stored metadata always describes stored content. ### Call configuration (`call-config.ts`) diff --git a/packages/llm/llm/README.zh.md b/packages/llm/llm/README.zh.md index bf86e703fc..313fc23590 100644 --- a/packages/llm/llm/README.zh.md +++ b/packages/llm/llm/README.zh.md @@ -2,39 +2,43 @@ [English](README.md) | 中文 -提供方无关的 LLM(大语言模型)词汇与抽象服务。本包定义 agent loop(智能体循环)、会话日志和每个插件使用的规范语言。 +提供方无关的 LLM(大语言模型)词汇与抽象服务。本包定义 agent loop(智能体循环)、会话日志和所有插件共同使用的规范词汇。 ## 服务:`LlmRuntime`(ctx key:`llm`) 一个适配器注册表加单一流式调用接口,可通过 waterfall(瀑布式事件)拦截。 +### 重试策略 + +每个提供方适配器都会提供解析后的路由策略。省略提供方配置时使用有界 normal mode,在首次请求后最多重试五次。分层配置把 `mode` 改为 `always` 后可能残留 `maxRetries` 或 `retryableCodes`;解析过程会忽略这些不再生效的 normal-mode 字段,并捕获纯 always 策略。本服务存储有效策略,但不执行重试。 + ### 公开 API -- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber 一起 dispose(资源释放)。返回的释放器还携带 `replace(providers)`:候选路由集合会在任何东西变动之前完整校验,因此与另一适配器冲突时,当前路由保持注册且继续服务,而替换本身是一个同步区段,不存在可观察的空档。`replace([])` 合法——一个持有零条路由的注册——这与空的初始注册不同。 +- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber 一起 dispose(资源释放)。返回的句柄还提供 `replace(providers)`:候选路由集合会在注册状态发生任何变化前完成整体验证,因此与另一适配器发生冲突时,当前路由仍保持注册并继续提供服务。替换会在一次同步操作中完成,不会出现可观察的空档。`replace([])` 合法,表示保留注册但不持有任何路由;初始注册则不得为空。 - `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。 - `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle` 声明适配器插件可通过配置激活的提供方路由——无论已注册还是休眠——每个条目指明其所属 settings namespace,以及 profile 在该分节内的路径。要么全部成功,要么全部不生效(`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`),并随调用 fiber dispose。该句柄还带 `replace(entries)`:候选集合会先被整体校验,因此其中若有条目已被另一个注册声明,当前集合原封不动;此处允许传空数组。声明集合随配置变化的插件必须使用 `replace`,而不是先 dispose 再重新注册——后者会在新集合被拒时让目录整个落空。 -- `ctx.llm.listConfigurableProviders(): LlmConfigurableProvider[]` 按声明顺序列出已声明的目录;配置界面将其与 `listProviders()` 合并,为每个条目标注存活或休眠。条目可携带 `declared`——拥有该路由的适配器是否只因配置点名才知道它。只有适配器能回答,因此缺席意味着「这个适配器不作此区分」,绝不等于「内置」。 -- `ctx.llm.registerModelDiscovery(settingsNs: string, discover): () => void` 为本插件拥有的 settings namespace 提供「询问提供方端点」的能力。每个 namespace 只能有一个(`INVALID_DISCOVERY`/`DUPLICATE_DISCOVERY`),并随调用 fiber dispose。 +- `ctx.llm.listConfigurableProviders(): LlmConfigurableProvider[]` 按声明顺序列出可配置提供方目录;配置界面将其与 `listProviders()` 合并,为每个条目标注存活或休眠。条目可携带 `declared`,表示拥有该路由的适配器是否只因配置点名才知道它。只有适配器能回答这一点:该字段缺失时,只表示该适配器不区分这两种来源,不能据此判断路由是否随产品交付。 +- `ctx.llm.registerModelDiscovery(settingsNs: string, discover): () => void` 为本插件拥有的 settings namespace 提供查询提供方端点的能力。每个 namespace 只能有一个(`INVALID_DISCOVERY`/`DUPLICATE_DISCOVERY`),并随调用 fiber dispose。 - `ctx.llm.listModelDiscoveryNamespaces(): string[]` 列出可以询问端点的 namespace,让界面只在可用之处提供该动作。 - `ctx.llm.discoverModels(settingsNs: string, request: LlmModelDiscoveryRequest): Promise` 询问某个端点它公布了哪些模型。 -- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的提供方重试策略,并解析 normal 默认值。 +- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的提供方自身的重试策略,并解析 normal 默认值。 - `ctx.llm.listModels(provider: string): Promise` 发现某个已注册提供方当前公布的模型。 -- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise` 从拥有精确路由的适配器解析经校验的确切模型身份,以及可用上下文、输出默认值和推理(reasoning)元数据;异步适配器可选地支持取消。 +- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise` 从拥有该精确路由的适配器中,解析并校验确切模型身份,以及可用上下文、输出默认值和推理(reasoning)元数据;异步适配器可选地支持取消。 - `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise` 校验显式推理强度,并填入适配器配置的调用默认值,但不自动调整。 -- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise` 在一次精确模型查询中解析配置、脱耦的上下文元数据以及标明哪些字段由适配器默认值填入的标记,再将当前适配器注册和不可变重试策略捕获为一次可取消、一次性调用。 +- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise` 在一次精确模型查询中解析配置、脱耦的上下文与模态元数据以及标明哪些字段由适配器默认值填入的标记,再把适配器匹配的分发世代和不可变重试策略捕获为一次可取消、一次性调用。 - `ctx.llm.stream(options: GenerateOptions): AsyncIterable` 将一次模型调用流式输出为原始分片(token 级增量)。消费方使用 `BlockAssembler` 将分片组装为块/消息。 -`LlmRuntime` 将最终适配器选择、同步 dispatch、iterator 构造与迭代中的失败规范化为流协议唯一的终止形式:`finish { kind: 'error' | 'aborted', failure }`。部分增量输出后发生失败时,内容块可能仍未闭合;消费方会丢弃这些不完整输出。`llm/stream` middleware、嵌套调用、适配器清理和下游消费方的错误仍会抛出,因为它们属于插件或消费方失败,而非模型请求结果。已准备调用会暴露随其确切适配器注册一同捕获的不可变重试策略;完全由 middleware 处理的路由没有服务策略。 +`LlmRuntime` 将最终适配器选择、同步分发、迭代器构造和迭代期间的失败,统一转换为流协议唯一的终止形式:`finish { kind: 'error' | 'aborted', failure }`。部分增量输出后发生失败时,内容块可能仍未闭合;消费方会丢弃这些不完整输出。`llm/stream` middleware、嵌套调用、适配器清理和下游消费方的错误仍会抛出,因为它们属于插件或消费方失败,而非模型请求结果。已准备调用会暴露随其确切适配器注册一同捕获的不可变重试策略;完全由 middleware 处理的路由没有服务策略。 -询问端点属于配置期针对**草稿**的操作,以 settings namespace 而非提供方路由为键——界面正在新增的提供方还不存在,也就没有路由可点名。但请求仍可**点名**它正在编辑的路由,而已经描述该路由的适配器会用自己的知识作答,无需联网;`baseURL` 可选,两者至少要有一个。除此之外,请求携带端点、协议,以及一条 harness 只用于这一次询问、绝不存储的凭据——这里既不读也不写 settings 与 credentials,回复是界面可供用户采纳的候选元数据,而不是已注册的 catalog。`LlmDiscoveredModel` 除 `id` 外每个字段都是可选的,因为大多数提供方列表只公布 id;采纳其中一条的界面仍要补上其适配器所需的容量。重复与不可用的 id 会被丢弃,无人服务的 namespace 以 `NO_DISCOVERY` 失败,既不点名路由也不给端点的请求以 `INVALID_DISCOVERY` 失败。 +询问端点属于配置期针对**草稿**的操作,以 settings namespace 而非提供方路由为键——界面正在新增的提供方还不存在,也就没有路由可点名。但请求仍可**点名**它正在编辑的路由,而已经描述该路由的适配器会用自己的知识作答,无需联网;路由名称和 `baseURL` 至少需要提供一项。除此之外,请求携带端点、协议,以及一条 harness 只用于这一次询问、绝不存储的凭据。这里既不读取也不写入 settings 或 credentials;返回内容是界面可以提供给用户采纳的候选元数据,而不是已注册的 catalog。`LlmDiscoveredModel` 除 `id` 外每个字段都是可选的,因为大多数提供方列表只公布 id;采纳其中一条的界面仍要补上其适配器所需的容量。重复与不可用的 id 会被丢弃,无人服务的 namespace 以 `NO_DISCOVERY` 失败,既不点名路由也不给端点的请求以 `INVALID_DISCOVERY` 失败。 -提供方与模型元数据是发现接口,不是路由白名单。`registerAdapter()` 仍拥有提供方排他性,并为每条路由捕获适配器的重试策略;适配器则可以接受 `listModels()` 中不存在的模型 id,消费方禁止因模型未列出而拒绝请求。返回的 selector 元数据与输入脱离,无效或重复的适配器条目会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。 +提供方和模型元数据用于发现,不构成路由白名单。`registerAdapter()` 仍拥有提供方路由的排他性,并为每条路由捕获适配器的重试策略;适配器可以接受未出现在 `listModels()` 中的模型 id,消费方不得仅因模型未列出而拒绝请求。返回的 selector 元数据已分离;无效或重复的适配器条目会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。 -每个拓扑提交点——适配器路由注册或 dispose、目录条目出现或撤回——都会在变更之后发出无载荷的 `llm/adapters-updated` 事件,消费方因此重读 `listProviders()`/`listModels()`/`listConfigurableProviders()` 而非轮询。观察者故障会被隔离(记录日志、不否决);只有带 `INVARIANT` 码的故障会在扇出后重新抛出。 +每个拓扑提交点——适配器路由注册或 dispose、目录条目出现或撤回——都会在变更之后发出无载荷的 `llm/adapters-updated` 事件,消费方因此会重新读取 `listProviders()`/`listModels()`/`listConfigurableProviders()`,而不是轮询。观察者故障会被记录并隔离,不能否决变更;只有带 `INVARIANT` 码的故障会在通知完所有观察者后重新抛出。 -确切模型元数据是独立的正确性查询,不是 catalog 装饰或全局 LLM 设置。`resolveModelInfo()` 会向拥有精确提供方/模型路由的适配器查询一次;适配器可以描述未列出的动态模型,缺少 `context`、`defaultMaxTokens` 或 `reasoning` 字段会分别保留未知容量、提供方持有的输出默认值或不可用的推理能力。无效的身份、上下文、输出默认值或推理元数据会以 `INVALID_MODEL_INFO`、`INVALID_MODEL_CONTEXT`、`INVALID_MODEL_MAX_TOKENS` 或 `INVALID_MODEL_REASONING` 失败。 +确切模型元数据是独立的正确性查询,不是 catalog 装饰或全局 LLM 设置。`resolveModelInfo()` 会向拥有精确提供方/模型路由的适配器查询一次;适配器可以描述未列出的动态模型。缺少 `context` 表示模型容量未知;缺少 `defaultMaxTokens` 表示继续沿用提供方自身的输出默认值;缺少 `reasoning` 则表示推理能力不可用。无效的身份、上下文、输出默认值或推理元数据会以 `INVALID_MODEL_INFO`、`INVALID_MODEL_CONTEXT`、`INVALID_MODEL_MAX_TOKENS` 或 `INVALID_MODEL_REASONING` 失败。 -`defaultMaxTokens` 是适配器配置的单次请求输出上限,不是模型硬上限。仅当请求省略 `maxTokens` 时,`resolveCallConfig()` 才会填入该值;显式上限优先。推理标识符是由适配器持有的不透明字符串,而非核心枚举:同一次解析只接受与已公布标识符完全一致的值,在存在 `defaultEffort` 时填入它,否则保留提供方默认值。异步模型解析器会接收调用方的 signal,并且必须在取消后迅速结束。`prepareCall()` 还会公开同一次查询得到的脱耦上下文元数据,通过 `adapterDefaults` 报告它填入了哪些 `maxTokens` 和 `reasoningEffort` 字段,并让精确适配器注册跨越请求头记录和最终分派,因此 HMR(热模块替换)不会将一个适配器的能力结果与另一个适配器的请求混用;复用其一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。 +`defaultMaxTokens` 是适配器配置的单次请求输出上限,不是模型硬上限。仅当请求省略 `maxTokens` 时,`resolveCallConfig()` 才会填入该值;显式上限优先。推理标识符是由适配器定义的不透明字符串,而非核心枚举:同一次解析只接受与已公布标识符完全一致的值,在存在 `defaultEffort` 时填入它,否则保留提供方默认值。异步模型解析器会接收调用方信号,并且必须在取消后尽快结算。`prepareCall()` 还会公开脱离内部状态的上下文和输入模态元数据,通过 `adapterDefaults` 标明填入了哪些 `maxTokens` 和 `reasoningEffort` 字段,并把这些事实绑定到执行最终分发的适配器世代。因此,HMR(热模块替换)或动态 settings 不会把一个世代的图片能力与另一个世代的端点组合;复用一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。 ### 事件 @@ -44,28 +48,32 @@ ### 扩展点 -- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerRetryPolicy()` 以提供由提供方持有的恢复配置,覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量、输出默认值或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现使用有界的 normal 重试策略,将路由和模型 id 用作名称,不公布模型,也不返回容量、输出默认值或推理元数据。 -- 包装 `llm/stream` 时,通过 `ctx.on()` waterfall listener 实现缓存、日志或路由。发出分片后重试的包装层没有持久尝试边界;因此已发布 agent 重试策略改用 `agent/request-error`。 +- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerRetryPolicy()` 以提供由提供方定义的恢复配置,覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量、输出默认值或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现使用有界的 normal 重试策略,将路由和模型 id 用作名称,不公布模型,也不返回容量、输出默认值或推理元数据。 +- 包装 `llm/stream` 时,通过 `ctx.on()` waterfall listener 实现缓存、日志或路由。包装层如果在已经发出分片后重试,就没有可持久记录的尝试边界;因此,随产品交付的 agent 重试策略改用 `agent/request-error`。 ### 消息(`message.ts`)与内容块(`types.ts`) `Message` 是投递、持久历史和模型请求共享的不可变值。每条消息从创建起都必须具有 `MessageId`、角色、内容和带类型的来源。`createMessage(input)` 生成标识,并返回与输入分离且深度冻结的值;`createUserMessage({ content, source })` 固定 user 角色;`createAssistantMessage({ content, source })` 固定 assistant 角色与模型来源类别;`createToolResultMessage({ callId, content, isError })` 固定 user 角色,并将工具来源与其结果块耦合;`freezeMessage(message)` 导入已有标识,绝不将其替换。改写消息时会保留标识,并产生另一个冻结值。浏览器端代码会从依赖最少的 `@deepseek-ai/dsh-llm/message` 入口导入这些值构造函数,而不是从包含服务的包根入口导入。 -消息内容是类型化内容块数组:`text`、`reasoning`、`tool-call`、`tool-result`。联合从可合并扩展的 `ContentBlockMap` 派生,因此插件可以通过 declaration merging 添加块类型。assistant 消息使用模型来源,其中携带生成该消息的提供方和模型,以及可选的适配器私有回放状态。dispatch 前,`LlmRuntime` 只在历史提供方路由与目标提供方路由当前由完全相同的适配器实例拥有时才保留该状态;随后由适配器判定能否在模型/提供方间恢复或转换该状态。核心块集只包含每条已发布路径都支持的块。多模态内容(图像、音频等)没有核心块类型;需要它的功能会通过 map 添加,并一并添加相应的适配器/UI/压缩(compaction)支持。 +消息内容是类型化内容块数组:`text`、`reasoning`、`image`、`tool-call`、`tool-result`。`ImageBlock` 只携带持久 `ImageAttachmentRef`;提供方字节和请求尺寸之后再解析。联合仍从可合并扩展的 `ContentBlockMap` 派生,因此插件可以通过 declaration merging 添加其他块类型。assistant 消息使用模型来源,其中携带生成该消息的提供方和模型,以及可选的适配器私有回放状态。dispatch 前,`LlmRuntime` 只在历史提供方路由与目标提供方路由当前由完全相同的适配器实例拥有时才保留该状态;随后由适配器判定能否在模型或提供方间恢复或转换该状态。 -流式输出是原始分片协议(`block-start`、`text-delta`、`reasoning-delta`、`tool-call-delta`、`block-end`、`usage`、`finish`)。每个适配器结果都以一个终止 `finish` 到达消费方;运行故障使用 `error` 或 `aborted` 作为结束原因,而不会跨流 API 抛出。`BlockAssembler` 是将分片组装为块/消息的唯一共享实现。 +每次分发都使用随适配器世代捕获的确切模型模态。支持图片的适配器把持久图片引用投影为路由专用请求版本。纯文本路由则收到确定性的附件占位文本,其中也包括嵌套工具结果图片,追加式会话历史不会改变。`offloadRequestImagesWithPolicy()` 提供确定性的从旧到新图片移除,支持按原始字节或 base64 计数,也支持图片数量或字节量步长;适配器提供确切派生版本的字节长度。 + +流式输出是原始分片协议(`block-start`、`text-delta`、`reasoning-delta`、`tool-call-delta`、`block-end`、`usage`、`finish`)。每个适配器结果都以一个终止 `finish` 到达消费方;运行故障使用 `error` 或 `aborted` 作为结束原因,而不会跨流 API 抛出。`BlockAssembler` 是将分片组装为块/消息的唯一共享实现。成功的 `finish` 可以携带 `ReplayEnvelope`——不透明的响应级回放元数据,加上与发射块序列对齐的可选逐块条目。组装对内容与元数据只做一次保留/丢弃决定:`max-tokens` 结束会丢弃可能被截断的工具调用,数据在每个被丢弃的位置同步失去对应条目,因此存储的元数据始终描述存储的内容。 ### 调用配置(`call-config.ts`) -`LlmCallConfig` 是一个会话中各次请求的提供方、模型、可选的适配器持有推理强度和采样标量(`provider`、`model`、`reasoningEffort`、`temperature`、`maxTokens`、`stop`,每个都与同名 `GenerateOptions` 字段 1:1 映射)。它是作为请求标头一部分记录在会话日志中的每会话状态(见 dsh-session `request/header` 事件),绝不是可静默调整的每次调用旋钮:`agent/request` waterfall 会提议替换,`prepareCall()` 在轮次 signal 控制下校验它并填入适配器默认值,loop 随后记录生效值以及标明哪些字段由适配器默认值填入的标记,再使用已准备调用中与注册绑定的流。下一次提议会省略带标记的默认值,使变更后的路由解析自身的值;未带标记的显式字段会保留。`callConfigEquals(a, b)` 是逐字段真实变更检测器;`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper(`llm/stream` listener 与适配器只读,绝不改写)。`markAgentLoopRequest()` 将该精确对象标记为由进程本地 agent loop 创建,`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类,让适配器可以应用目的特定传输策略,而不改变普通会话请求。 +`LlmCallConfig` 记录一个会话的模型请求所使用的提供方、模型、由适配器定义的可选推理强度,以及采样参数(`provider`、`model`、`reasoningEffort`、`temperature`、`maxTokens`、`stop`,分别与同名 `GenerateOptions` 字段一一对应)。它是作为请求标头一部分记录在会话日志中的每会话状态(见 dsh-session `request/header` 事件),绝不是可静默调整的每次调用旋钮:`agent/request` waterfall 会提议替换,`prepareCall()` 在轮次 signal 控制下校验它并填入适配器默认值,loop 随后记录生效值以及标明哪些字段由适配器默认值填入的标记,再使用已准备调用中与注册绑定的流。下一次提议会省略带标记的默认值,使变更后的路由解析自身的值;未带标记的显式字段会保留。`callConfigEquals(a, b)` 是逐字段真实变更检测器;`deepFreeze(value)` 是 loop 使用的请求所有权辅助函数:每个构造完成的请求都会在分发前深度冻结;`llm/stream` 监听器和适配器只能读取,绝不能改写。`markAgentLoopRequest()` 将该精确对象标记为由进程本地 agent loop 创建,`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 会对已记录的辅助压缩和会话标题调用进行分类,使适配器可以按调用目的应用不同的传输策略,而不改变普通会话请求。 + + ### 应用归因(`attribution.ts`) -每个产品适配器都会在提供方 HTTP 请求上发送应用身份。`attributionHeaders(identity?)` 构建标准 `User-Agent`,默认为公开 `APP_IDENTITY`;白标部署可以替换它,但不能抑制它。适配器会直接验证 wire 标头,或通过自身库 hook 验证。详见 [归因 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md)。 +每个产品适配器都会在提供方 HTTP 请求上发送应用身份。`attributionHeaders(identity?)` 构建标准 `User-Agent`,默认为公开 `APP_IDENTITY`;白标部署可以替换它,但不能抑制它。适配器会直接验证 wire 标头,或通过自身库 hook 验证。详见 [归因 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.zh.md)。 ### API 密钥校验(`api-key.ts`) -每个要把凭据放进 HTTP 标头的适配器,使用前都以同一套规则校验它。`normalizeApiKey(raw)` 先去除首尾空白,再接受任意非空的可打印 ASCII 值(`/^[\x21-\x7E]+$/`,不含空格),否则以 `ApiKeyRejection`(`'empty'` | `'illegalCharacters'`)说明拒绝原因,二者一并包含在 `ApiKeyCheck` 结果中。缺失从不参与校验:调用方会在询问之前自行判断是否提供了值——未点名凭据的 profile 会转由提供方自身的环境发现或 OAuth 完成认证。 +每个要把凭据放进 HTTP 标头的适配器,使用前都以同一套规则校验它。`normalizeApiKey(raw)` 会先去除首尾空白,然后接受任意非空的可打印 ASCII 值(`/^[\x21-\x7E]+$/`,不含空格),或通过 `ApiKeyRejection`(`'empty'` | `'illegalCharacters'`)说明拒绝原因。这些结果一并包含在 `ApiKeyCheck` 中。缺失从不参与校验:调用方会在询问之前自行判断是否提供了值——未点名凭据的 profile 会转由提供方自身的环境发现或 OAuth 完成认证。 ### 类 @@ -73,7 +81,7 @@ - `BlockAssembler`:将原始分片逐步组装为完整内容块,并能据此创建带标识且冻结的 assistant 消息。agent loop 向它提供原始分片(同时记录以供回放),并读取已组装块以构建历史。 - `HarnessError`:harness 错误分类体系的基类,包含稳定 `code` 字符串(与面向人的 `message` 不同)以及 `cause` 链。它位于所有其他包都从中导入的叶子包中,因此可以共享单一基类,无需新的依赖边。各包的错误(`LlmError`、`ToolArgsError`、`InvariantError` 等)都继承自它。`isHarnessError(value)` 在进程边界处收窄类型。 - `LlmError`:继承自 `HarnessError`;其稳定 `code` 字符串(`NO_ADAPTER`、`DUPLICATE_ADAPTER` 与 `AUTH`/`RATE_LIMIT` 等适配器 code)与冻结可序列化 `failure.code` 匹配。Payload 还可以保留已验证状态、`Retry-After` 和品牌化提供方请求 id 事实;策略位于错误之外。 -- `errorChain(value)`:渲染抛出值的完整 `cause` 链与 AggregateError 成员,供诊断输出使用,包括 UI 通知、logger 行和持久 `turn/end` 消息。因此 undici 的 `TypeError: fetch failed` 等传输包装层会显示底层 `ECONNREFUSED`/DNS/TLS 详细信息,而不是将其遮蔽。该函数只负责渲染:请按 `code` 路由,绝不解析结果。 +- `errorChain(value)`:渲染抛出值的完整 `cause` 链与 AggregateError 成员,供诊断输出使用,包括 UI 通知、logger 行和持久 `turn/end` 消息。因此 undici 的 `TypeError: fetch failed` 等传输包装层会显示底层 `ECONNREFUSED`/DNS/TLS 详细信息,而不是将其遮蔽。该函数只负责生成诊断文本。调用方必须依据稳定的 `code` 选择错误处理路径,绝不能通过解析渲染后的文本作出判断。 - `CONTEXT_WINDOW_EXCEEDED_CODE`:当请求超过模型上下文窗口时,无论通过 HTTP 异常抛出还是带内 finish 交付,两个 DeepSeek 适配器都使用的提供方无关 code。`isContextWindowExceededError(detail)` 是它们针对 OpenAI 兼容提供方详细信息的共享保守分类器。 - `QUOTA_EXCEEDED_CODE`:帐户配额、余额、点数、预算或用量限制耗尽时使用的非暂时性提供方无关 code。`isQuotaExceededError(detail)` 使这些失败与请求速率限制保持区分。 - `EMPTY_RESPONSE_CODE`:两个适配器都使用的提供方无关 code,用于表示退化的提供方生成结果:一个未携带任何内容块的终止 `stop`。它会被分类为错误 finish(而非成功空消息),因为尝试未产生持久内容;`dsh-llm-retry` 默认重试它。 @@ -81,7 +89,7 @@ ### 真实适配器 -两个适配器使用不同内部机制实现 `LlmAdapter`:[`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) 针对 `deepseek-official` 路由使用直接 fetch 加 `eventsource-parser` SSE(Server-Sent Events)分帧,[`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) 则通过 `@earendil-works/pi-ai` 动态解析已配置提供方/模型对。两者都遵循 `types.ts` 中的 `StreamChunk` 约定:usage 先于 finish,工具参数保持原始字符串。适配器实现在内部可以抛出异常或发出失败 finish;`LlmRuntime` 会将两者都暴露为终止失败 finish。适配器理由见[双 LLM 适配器](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md),服务边界见[终止失败决策](../../../.agents/notes/implemented/architecture/2026-07-29-terminal-llm-stream-failures.md)。 +两个适配器使用不同内部机制实现 `LlmAdapter`:[`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) 针对 `deepseek-official` 路由使用直接 fetch 加 `eventsource-parser` SSE(Server-Sent Events)分帧,[`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) 则通过 `@earendil-works/pi-ai` 动态解析已配置提供方/模型对。两者都遵循 `types.ts` 中的 `StreamChunk` 约定:usage 先于 finish,工具参数保持原始字符串。适配器实现在内部可以抛出异常或发出失败 finish;`LlmRuntime` 会将两者都暴露为终止失败 finish。适配器理由见[双 LLM 适配器](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.zh.md),服务边界见[终止失败决策](../../../.agents/notes/implemented/architecture/2026-07-29-terminal-llm-stream-failures.zh.md)。 ## 模型体验 @@ -91,11 +99,13 @@ 透传;注册表保留已组装请求前缀,cache 复用与路由边界属于所选适配器和提供方。 + + ## 已知限制与暂缓事项 - **本服务不执行重试、缓存或速率限制**:提供方注册会存储重试策略,但 `llm/stream` 仍是单次尝试调用包装层。agent loop 会将已验证模型请求失败单独提供给 `agent/request-error`,其默认行为是保留原始失败;`@deepseek-ai/dsh-llm-retry` 是共享示例主干加载的可选执行器。 - **`GenerateOptions` 采样只包含 `temperature`/`maxTokens`/`stop`**:没有 `tool_choice`、`top_p` 或 penalty 字段;有产生方落地时词汇才会增长(见 [已删除惰性旋钮](../../../.agents/notes/archived/simplification/2026-07-04-drop-inert-request-knobs.md))。 -- **受产生方约束的变体在实际产生前不会加入**:`prefill`、每工具 `strict`、块 `cache` 提示与 `agent` 消息源变体因没有产生方而被剪除(见 [Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md))。 +- **只有出现实际产生方后,相应变体才会加入**:`prefill`、逐工具 `strict`、内容块 `cache` 提示和 `agent` 消息来源变体,都因当前没有产生方而被移除(见 [Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md))。 - **`BlockAssembler` 只处理核心块类型**:如果插件添加块类型的流从未由 `block-end` 关闭,`blocks()` 会抛出异常。 - **`APP_IDENTITY.url` 指向一个尚不存在的仓库**:该公开主页必须在发布前可访问。 - **`GenerateOptions.sessionId` 是本地声明的品牌类型**:导入 dsh-session 的 `SessionId` 会产生循环;未来拥有 id 的包可以消除该权宜之计。 diff --git a/packages/llm/llm/package.json b/packages/llm/llm/package.json index c6018de902..cf032be43e 100644 --- a/packages/llm/llm/package.json +++ b/packages/llm/llm/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-llm", "description": "Provider-neutral LLM service interface for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -52,6 +52,7 @@ "@deepseek-ai/cordis": "workspace:^" }, "dependencies": { + "@deepseek-ai/dsh-util-crypto": "workspace:^", "@deepseek-ai/schemastery": "workspace:^" }, "devDependencies": { diff --git a/packages/llm/llm/src/assembler.ts b/packages/llm/llm/src/assembler.ts index a0e1332417..7dc5221c9b 100644 --- a/packages/llm/llm/src/assembler.ts +++ b/packages/llm/llm/src/assembler.ts @@ -10,7 +10,7 @@ import { CallId } from './brand.ts' import { assertNever } from './never.ts' import { createMessage } from './message.ts' import type { Message, MessageSource } from './message.ts' -import type { ContentBlock, FinishReason, StreamChunk, TokenUsage } from './types.ts' +import type { ContentBlock, FinishReason, ReplayEnvelope, StreamChunk, TokenUsage } from './types.ts' interface PartialBlock { blockType: string @@ -27,7 +27,8 @@ interface PartialBlock { * {@link ContentBlock}s and a final assistant {@link Message}. * * The agent loop feeds it while logging raw chunks for replay fidelity, then - * reads `blocks()` / `message()` / `usage` / `finish` once the stream ends. + * reads `blocks()` / `message()` / `usage` / `finish` once the stream ends, + * or `interruptedBlocks()` when cancellation cut the stream short. * * Tolerant of delta-only protocols (no block-start/end); deltas arriving for * an index already closed by `block-end` are ignored (malformed stream) so a @@ -38,7 +39,7 @@ export class BlockAssembler { private order: number[] = [] private _usage: TokenUsage | undefined private _finish: FinishReason | undefined - private _replayState: unknown = undefined + private _replayState: ReplayEnvelope | undefined /** * Feed one chunk into the assembly state. @@ -125,6 +126,28 @@ export class BlockAssembler { return partial } + /** + * The one shared keep/drop decision over all seen blocks: max-token + * truncation drops tool calls that cannot be executed safely. Emitted blocks + * and replay metadata both derive from this result, so they cannot disagree. + */ + private assembled(): { blocks: ContentBlock[]; replay: ReplayEnvelope | undefined } { + const all = this.order.map(index => this.assemble(this.mustGet(index), index)) + const kept = this.finish.kind === 'max-tokens' + ? all.map(block => block.type !== 'tool-call') + : undefined + const blocks = kept === undefined ? all : all.filter((_, position) => kept[position]) + const envelope = this._replayState + if (envelope?.blocks === undefined) return { blocks, replay: envelope } + if (envelope.blocks.length !== all.length) return { blocks, replay: undefined } + return { + blocks, + replay: kept === undefined || blocks.length === all.length + ? envelope + : { response: envelope.response, blocks: envelope.blocks.filter((_, position) => kept[position]) }, + } + } + /** * Assemble all blocks seen so far, in stream order. * @returns one block per seen index, except that max-token truncation drops @@ -132,10 +155,26 @@ export class BlockAssembler { * its accumulated deltas (an unknown block type never closed by `block-end` throws). */ blocks(): ContentBlock[] { - const blocks = this.order.map(index => this.assemble(this.mustGet(index), index)) - return this.finish.kind === 'max-tokens' - ? blocks.filter(block => block.type !== 'tool-call') - : blocks + return this.assembled().blocks + } + + /** + * Assemble the prefix an interrupted stream can safely finalize: closed and + * open text/reasoning blocks with non-whitespace content, in stream order. + * Tool calls are omitted because interruption precedes dispatch; retaining + * one would require a fabricated result. Open unknown blocks are also omitted. + * @returns the kept blocks; empty when nothing streamed before the interruption. + */ + interruptedBlocks(): ContentBlock[] { + return this.order + .map((index) => { + const partial = this.mustGet(index) + const type = partial.block?.type ?? partial.blockType + if (type !== 'text' && type !== 'reasoning') return undefined + return this.assemble(partial, index) + }) + .filter((block): block is ContentBlock => + (block?.type === 'text' || block?.type === 'reasoning') && block.text.trim() !== '') } /** Usage from the `usage` chunk; undefined until one arrives. */ @@ -148,9 +187,13 @@ export class BlockAssembler { return this._finish ?? { kind: 'stop' } } - /** Adapter-private replay state from the terminal finish chunk, if any. */ - get replayState(): unknown { - return this._replayState + /** + * Replay metadata from the terminal finish chunk, if any, with per-block + * entries pruned in step with {@link blocks}. Undefined when the envelope's + * entries do not align with the emitted blocks. + */ + get replayState(): ReplayEnvelope | undefined { + return this.assembled().replay } /** diff --git a/packages/llm/llm/src/content.ts b/packages/llm/llm/src/content.ts index 19b760a02a..4620275429 100644 --- a/packages/llm/llm/src/content.ts +++ b/packages/llm/llm/src/content.ts @@ -1,6 +1,31 @@ /** Content-block structure helpers. @module @deepseek-ai/dsh-llm/content */ import type { ContentBlock } from './types.ts' +import type { Message } from './message.ts' +import type { ImageAttachmentRef, RequestImageAttachment } from '@deepseek-ai/dsh-attachment' + +/** Model-facing stand-in for an image removed to fit a provider request bound. */ +export const OFFLOADED_IMAGE_TEXT + = '[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]' + +/** + * Stable text shown to a model that cannot accept one durable image reference. + * @param ref - durable master reference omitted from the request. + * @returns deterministic text-only placeholder. + */ +export function textOnlyImageText(ref: ImageAttachmentRef): string { + const digest = String(ref.attachmentId).slice('sha256:'.length, 'sha256:'.length + 8) + return `[image omitted because this model accepts text only; attachment sha256:${digest}]` +} + +/** + * Stable model-facing handle for one exact request image. + * @param version - exact request image shown beside the text. + * @returns attachment handle and request-image dimensions. + */ +export function requestImageHandleText(version: RequestImageAttachment): string { + return `Image ${version.attachment.attachmentId}; request image ${version.width}x${version.height}px.` +} /** * True when typed model content contains an image block, walking nested @@ -14,3 +39,164 @@ export function contentHasImage(content: readonly ContentBlock[]): boolean { return content.some(block => block.type === 'image' || (block.type === 'tool-result' && contentHasImage(block.content))) } + +/** Base64 length of raw image bytes, including padding. */ +function base64Length(bytes: number): number { + return Math.ceil(bytes / 3) * 4 +} + +/** Byte accounting and quantized removal policy for one request representation. */ +export interface RequestImageOffloadPolicy { + /** Image count accepted by the route; omission leaves count unbounded. */ + maxImages?: number + /** Accumulated image bytes accepted by the route; omission leaves bytes unbounded. */ + maxBytes?: number + /** Number of excess images removed as one deterministic step. */ + countQuantum?: number + /** Number of excess bytes removed as one deterministic step. */ + byteQuantum?: number + /** Whether byte accounting uses raw file bytes or inline base64 length. */ + representation: 'raw' | 'base64' + /** Resolve the encoded request-version length; omission uses master attachment bytes. */ + byteLength?: (ref: ImageAttachmentRef) => number +} + +/** Collect represented image lengths in request and nested-block order. */ +function collectImageLengths( + blocks: readonly ContentBlock[], + lengths: number[], + policy: RequestImageOffloadPolicy, +): void { + for (const block of blocks) { + if (block.type === 'image') { + const bytes = policy.byteLength === undefined + ? block.attachment.bytes + : policy.byteLength(block.attachment) + lengths.push(policy.representation === 'base64' ? base64Length(bytes) : bytes) + } else if (block.type === 'tool-result') { + collectImageLengths(block.content, lengths, policy) + } + } +} + +/** Replace the first `remaining.count` image occurrences without mutating durable messages. */ +function replaceOldestImages( + blocks: readonly ContentBlock[], + remaining: { count: number }, +): ContentBlock[] { + let next: ContentBlock[] | undefined + for (const [index, block] of blocks.entries()) { + if (block.type === 'image' && remaining.count > 0) { + remaining.count -= 1 + next ??= blocks.slice(0, index) + next.push({ type: 'text', text: OFFLOADED_IMAGE_TEXT }) + continue + } + if (block.type === 'tool-result') { + const content = replaceOldestImages(block.content, remaining) + if (content !== block.content) { + next ??= blocks.slice(0, index) + next.push({ ...block, content }) + continue + } + } + next?.push(block) + } + return next ?? blocks as ContentBlock[] +} + +/** Replace every image occurrence, including nested tool results, for a text-only model. */ +function replaceImagesForTextModel(blocks: readonly ContentBlock[]): ContentBlock[] { + let next: ContentBlock[] | undefined + for (const [index, block] of blocks.entries()) { + if (block.type === 'image') { + next ??= blocks.slice(0, index) + next.push({ type: 'text', text: textOnlyImageText(block.attachment) }) + continue + } + if (block.type === 'tool-result') { + const content = replaceImagesForTextModel(block.content) + if (content !== block.content) { + next ??= blocks.slice(0, index) + next.push({ ...block, content }) + continue + } + } + next?.push(block) + } + return next ?? blocks as ContentBlock[] +} + +/** + * Project durable image history into deterministic text for an exact text-only model. + * @param messages - complete request history. + * @returns the original list without images, otherwise shallow message copies with stable placeholders. + */ +export function projectImagesForTextModel(messages: readonly Message[]): readonly Message[] { + if (!messages.some(message => contentHasImage(message.content))) return messages + return messages.map((message) => { + const content = replaceImagesForTextModel(message.content) + return content === message.content ? message : { ...message, content } + }) +} + +/** + * Return transient request messages whose oldest images are replaced until + * their accumulated base64 payload fits the configured bound. The selection + * is deterministic from durable message order and attachment metadata; a + * provider can serialize the returned messages without reading omitted bytes. + * @param messages - complete request history, oldest first. + * @param maxRequestImageBytes - positive bound on total base64 image payload; undefined preserves every image. + * @returns the original messages when they already fit, otherwise shallow message copies with replaced content trees. + */ +export function offloadRequestImages( + messages: readonly Message[], + maxRequestImageBytes: number | undefined, +): readonly Message[] { + return offloadRequestImagesWithPolicy(messages, { + representation: 'base64', + ...maxRequestImageBytes === undefined ? {} : { maxBytes: maxRequestImageBytes }, + byteQuantum: 1, + }) +} + +/** + * Return a deterministic transient projection whose oldest images are replaced + * in whole count and byte quanta after a route budget is exceeded. The target + * depends only on complete durable history: at 129 one-megabyte images under + * a 128 MiB bound with a 64 MiB quantum, the oldest 65 images are removed so + * 64 MiB remain; that removed prefix stays fixed until total history exceeds + * 192 MiB. + * @param messages - complete request history, oldest first. + * @param policy - route representation, budgets, and removal quanta. + * @returns original messages below both bounds, otherwise shallow copies with deterministic placeholders. + */ +export function offloadRequestImagesWithPolicy( + messages: readonly Message[], + policy: RequestImageOffloadPolicy, +): readonly Message[] { + const lengths: number[] = [] + for (const message of messages) collectImageLengths(message.content, lengths, policy) + const total = lengths.reduce((sum, bytes) => sum + bytes, 0) + const excessCount = policy.maxImages === undefined ? 0 : Math.max(0, lengths.length - policy.maxImages) + const excessBytes = policy.maxBytes === undefined ? 0 : Math.max(0, total - policy.maxBytes) + if (excessCount === 0 && excessBytes === 0) return messages + const countQuantum = policy.countQuantum ?? 1 + const byteQuantum = policy.byteQuantum ?? 1 + const removeCount = excessCount === 0 ? 0 : Math.ceil(excessCount / countQuantum) * countQuantum + const removeBytes = excessBytes === 0 ? 0 : Math.ceil(excessBytes / byteQuantum) * byteQuantum + let count = 0 + let removedBytes = 0 + for (const imageBytes of lengths) { + const byteTargetMet = removeBytes === 0 + || (byteQuantum === 1 ? removedBytes >= removeBytes : removedBytes > removeBytes) + if (count >= removeCount && byteTargetMet) break + removedBytes += imageBytes + count += 1 + } + const remaining = { count } + return messages.map((message) => { + const content = replaceOldestImages(message.content, remaining) + return content === message.content ? message : { ...message, content } + }) +} diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index e87c428d06..82b64bf4ce 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -29,6 +29,7 @@ import type { LlmCallConfig, LlmCallConfigAdapterDefaults } from './call-config. import { HarnessError, INVALID_CREDENTIAL_CODE } from './error.ts' import { normalizeLlmFailure } from './adapter-failure.ts' import { normalizeApiKey } from './api-key.ts' +import { contentHasImage, projectImagesForTextModel } from './content.ts' export * from './attribution.ts' export * from './brand.ts' @@ -159,6 +160,8 @@ export interface PreparedLlmCall { readonly retryPolicy: ResolvedRetryPolicy /** Detached context metadata resolved with the registration-bound call. */ readonly context?: LlmModelContext + /** Exact model modalities captured with the adapter dispatch generation. */ + readonly inputModalities?: readonly ModelModality[] /** Config fields materialized by the captured adapter rather than proposed by the caller. */ readonly adapterDefaults: LlmCallConfigAdapterDefaults /** @@ -171,6 +174,14 @@ export interface PreparedLlmCall { stream(options: GenerateOptions): AsyncIterable } +/** One adapter-owned model-resolution generation bound to its eventual stream call. */ +export interface PreparedAdapterCall { + /** Exact model metadata from the same adapter generation as {@link stream}. */ + readonly model: LlmResolvedModelInfo + /** Dispatch through that generation without re-reading dynamic connection facts. */ + stream(options: GenerateOptions): AsyncIterable +} + /** * Provider-wire adapter for the harness message and stream vocabulary. Register implementations * with `ctx.llm.registerAdapter(providers, adapter)`. Every provider HTTP request must include @@ -224,6 +235,22 @@ export abstract class LlmAdapter { return Promise.resolve({ provider, id: model, name: model }) } + /** + * Bind exact model metadata and the eventual request dispatch to one adapter generation. + * Dynamic adapters override this so settings changes between preparation and + * dispatch cannot combine one generation's capabilities with another's endpoint. + * @param provider - registered provider route. + * @param model - exact model id. + * @param signal - cancellation for model resolution. + * @returns model metadata and a one-generation stream entry point. + */ + async prepareCall(provider: string, model: string, signal?: AbortSignal): Promise { + return { + model: await this.resolveModel(provider, model, signal), + stream: options => this.stream(options), + } + } + /** * Stream one model call as raw chunks. The only required method. * @param options - the fully-assembled request; implementations must honor `options.signal`. @@ -629,8 +656,17 @@ export class LlmRuntime extends Service { model: string, signal?: AbortSignal, ): Promise { + const resolved = await registration.adapter.resolveModel(registration.provider.id, model, signal) + return this.normalizeModelInfo(registration, model, resolved) + } + + /** Validate and detach one adapter-returned exact model result. */ + private normalizeModelInfo( + registration: AdapterRegistration, + model: string, + resolved: LlmResolvedModelInfo, + ): LlmResolvedModelInfo { const provider = registration.provider.id - const resolved = await registration.adapter.resolveModel(provider, model, signal) if ( typeof resolved.provider !== 'string' || resolved.provider !== provider @@ -735,8 +771,16 @@ export class LlmRuntime extends Service { registration: AdapterRegistration, config: LlmCallConfig, signal?: AbortSignal, - ): Promise<{ config: LlmCallConfig; context?: LlmModelContext }> { + ): Promise<{ config: LlmCallConfig; context?: LlmModelContext; modelInfo: LlmResolvedModelInfo }> { const info = await this.resolveModelInfoFor(registration, config.model, signal) + return this.resolveCallWithInfo(config, info) + } + + /** Validate request controls against one already-bound exact model result. */ + private resolveCallWithInfo( + config: LlmCallConfig, + info: LlmResolvedModelInfo, + ): { config: LlmCallConfig; context?: LlmModelContext; modelInfo: LlmResolvedModelInfo } { const defaulted = config.maxTokens === undefined && info.defaultMaxTokens !== undefined ? { ...config, maxTokens: info.defaultMaxTokens } : config @@ -765,6 +809,7 @@ export class LlmRuntime extends Service { return { config: resolvedConfig, ...info.context === undefined ? {} : { context: info.context }, + modelInfo: info, } } @@ -778,7 +823,9 @@ export class LlmRuntime extends Service { */ async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise { const registration = this.registration(config.provider) - const resolved = await this.resolveCallFor(registration, config, signal) + const adapterCall = await registration.adapter.prepareCall(config.provider, config.model, signal) + const modelInfo = this.normalizeModelInfo(registration, config.model, adapterCall.model) + const resolved = this.resolveCallWithInfo(config, modelInfo) const resolvedConfig = deepFreeze(structuredClone(resolved.config)) const context = resolved.context === undefined ? undefined @@ -797,6 +844,9 @@ export class LlmRuntime extends Service { retryPolicy: registration.retryPolicy, adapterDefaults, ...context === undefined ? {} : { context }, + ...modelInfo.inputModalities === undefined + ? {} + : { inputModalities: Object.freeze([...modelInfo.inputModalities]) }, stream: (options: GenerateOptions): AsyncIterable => { if (dispatched) { throw new LlmError('a prepared LLM call can only be dispatched once', 'INVALID_PREPARED_CALL') @@ -808,7 +858,12 @@ export class LlmRuntime extends Service { ) } dispatched = true - return this.streamWithRegistration(options, { registration, config: resolvedConfig }) + return this.streamWithRegistration(options, { + registration, + config: resolvedConfig, + modelInfo, + dispatch: options => adapterCall.stream(options), + }) }, }) } @@ -842,14 +897,25 @@ export class LlmRuntime extends Service { */ private async * adapterStream( options: GenerateOptions, - prepared?: { registration: AdapterRegistration; config: LlmCallConfig }, + prepared?: PreparedDispatch, ): AsyncGenerator { let iterator: AsyncIterator try { const registration = prepared?.registration ?? this.registration(options.provider) - const resolvedConfig = prepared === undefined - ? (await this.resolveCallFor(registration, options, options.signal)).config - : prepared.config + const adapter = registration.adapter + let modelInfo: LlmResolvedModelInfo + let resolvedConfig: LlmCallConfig + let dispatch: (options: GenerateOptions) => AsyncIterable + if (prepared === undefined) { + const adapterCall = await adapter.prepareCall(options.provider, options.model, options.signal) + modelInfo = this.normalizeModelInfo(registration, options.model, adapterCall.model) + resolvedConfig = this.resolveCallWithInfo(options, modelInfo).config + dispatch = options => adapterCall.stream(options) + } else { + modelInfo = prepared.modelInfo + resolvedConfig = prepared.config + dispatch = prepared.dispatch + } if (prepared !== undefined && !callConfigEquals(options, resolvedConfig)) { throw new LlmError( 'prepared LLM call config changed before adapter dispatch', @@ -861,8 +927,14 @@ export class LlmRuntime extends Service { : Object.isFrozen(options) ? deepFreeze({ ...options, ...resolvedConfig }) : { ...options, ...resolvedConfig } - const adapter = registration.adapter - const stream = adapter.stream(this.forAdapter(resolvedOptions, adapter)) + const projectedOptions = modelInfo.inputModalities !== undefined + && !modelInfo.inputModalities.includes('image') + && resolvedOptions.messages.some(message => contentHasImage(message.content)) + ? Object.isFrozen(resolvedOptions) + ? deepFreeze({ ...resolvedOptions, messages: projectImagesForTextModel(resolvedOptions.messages) as Message[] }) + : { ...resolvedOptions, messages: projectImagesForTextModel(resolvedOptions.messages) as Message[] } + : resolvedOptions + const stream = dispatch(this.forAdapter(projectedOptions, adapter)) iterator = stream[Symbol.asyncIterator]() } catch (error: unknown) { yield adapterFailureChunk(error, options.signal) @@ -916,7 +988,7 @@ export class LlmRuntime extends Service { private streamWithRegistration( options: GenerateOptions, - prepared?: { registration: AdapterRegistration; config: LlmCallConfig }, + prepared?: PreparedDispatch, ): AsyncIterable { return this.ctx.waterfall( this, @@ -944,4 +1016,11 @@ interface AdapterRegistration { readonly retryPolicy: ResolvedRetryPolicy } +interface PreparedDispatch { + readonly registration: AdapterRegistration + readonly config: LlmCallConfig + readonly modelInfo: LlmResolvedModelInfo + readonly dispatch: (options: GenerateOptions) => AsyncIterable +} + export default LlmRuntime diff --git a/packages/llm/llm/src/message.ts b/packages/llm/llm/src/message.ts index 608b56475d..30df8157a8 100644 --- a/packages/llm/llm/src/message.ts +++ b/packages/llm/llm/src/message.ts @@ -1,5 +1,6 @@ /** Message value types, identity, and immutable construction helpers. */ +import { randomUUID } from '@deepseek-ai/dsh-util-crypto' import { MessageId, type CallId } from './brand.ts' import { deepFreeze } from './call-config.ts' import type { ContentBlock, StreamChunk, ToolResultBlock } from './types.ts' @@ -180,7 +181,7 @@ export function createMessage( ): T & Pick { return freezeMessage({ ...input, - id: MessageId(crypto.randomUUID()), + id: MessageId(randomUUID()), }) } diff --git a/packages/llm/llm/src/retry-policy.ts b/packages/llm/llm/src/retry-policy.ts index ad9c7af65c..f6e6175cb9 100644 --- a/packages/llm/llm/src/retry-policy.ts +++ b/packages/llm/llm/src/retry-policy.ts @@ -11,7 +11,7 @@ import z from '@deepseek-ai/schemastery' import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { EMPTY_RESPONSE_CODE } from './error.ts' -const DEFAULT_MAX_RETRIES = 2 +const DEFAULT_MAX_RETRIES = 5 const DEFAULT_INITIAL_DELAY_MS = 500 const DEFAULT_MAX_DELAY_MS = 10_000 const DEFAULT_JITTER_RATIO = 0.1 @@ -37,7 +37,7 @@ export interface BackoffConfig { export interface NormalRetryPolicyConfig { /** Retry only configured transient failure codes. */ mode: 'normal' - /** Maximum eligible retries after the first request (default 2). */ + /** Maximum eligible retries after the first request (default 5). */ maxRetries?: number /** Stable failure codes eligible for this policy. */ retryableCodes?: string[] @@ -105,7 +105,11 @@ export const RetryPolicySchema: z = z.union([ const NORMAL_POLICY_KEYS: ReadonlySet = new Set([ 'mode', 'maxRetries', 'retryableCodes', 'backoff', ]) -const ALWAYS_POLICY_KEYS: ReadonlySet = new Set(['mode', 'backoff']) +// Layered configuration can retain normal-only fields after switching modes; +// always mode ignores those inactive values while still rejecting unknown keys. +const ALWAYS_POLICY_KEYS: ReadonlySet = new Set([ + 'mode', 'maxRetries', 'retryableCodes', 'backoff', +]) const BACKOFF_KEYS: ReadonlySet = new Set(['initialDelayMs', 'maxDelayMs', 'jitterRatio']) function validateKeys(value: object, allowed: ReadonlySet, path: string): void { diff --git a/packages/llm/llm/src/types.ts b/packages/llm/llm/src/types.ts index 326db1cb14..bfd3d076d6 100644 --- a/packages/llm/llm/src/types.ts +++ b/packages/llm/llm/src/types.ts @@ -66,7 +66,7 @@ export interface ReasoningBlock { * A durable raster image reference, valid in user or assistant content. The * block is deliberately role-neutral; assistant-side rendering is forward * compatibility — the current production adapters declare text-only output, - * so only user content carries images today. + * so only user messages may carry images. */ export interface ImageBlock { type: 'image' @@ -280,6 +280,27 @@ export interface LlmResolvedModelInfo extends LlmModelInfo { reasoning?: LlmModelReasoningInfo } +/** + * Adapter-private lossless-JSON state for replaying a successful response, + * carried by a terminal `finish` chunk and stored on the assembled assistant + * message's model source. Both halves stay opaque to the harness; only the + * split is shared vocabulary, so assembly can keep stored metadata aligned + * with stored content without reading either half. + */ +export interface ReplayEnvelope { + /** Response-level adapter-private metadata (ids, native stop reason). */ + response: unknown + /** + * Per-block adapter-private metadata, one entry per emitted block in + * first-seen stream order. When assembly drops a block it drops the entry at + * the same position; entries whose length does not match the emitted block + * count discard the whole envelope. An adapter whose metadata is independent + * of block structure omits this field and the envelope passes through + * assembly unchanged. + */ + blocks?: readonly unknown[] +} + /** * Raw streaming protocol emitted by adapters. * Block indexes correlate interleaved deltas, and `block-end` carries the @@ -298,8 +319,8 @@ export type StreamChunk = | { type: 'finish' reason: FinishReason - /** Adapter-private lossless-JSON state for replaying a successful response. */ - replayState?: unknown + /** Replay metadata for a successful response; see {@link ReplayEnvelope}. */ + replayState?: ReplayEnvelope } /** diff --git a/packages/llm/llm/tests/assembler.spec.ts b/packages/llm/llm/tests/assembler.spec.ts index bf2276a218..2ecfcfd860 100644 --- a/packages/llm/llm/tests/assembler.spec.ts +++ b/packages/llm/llm/tests/assembler.spec.ts @@ -118,6 +118,85 @@ describe('BlockAssembler', () => { }) }) +describe('BlockAssembler replay metadata', () => { + const response = { responseId: 'resp-1' } + + it('prunes per-block replay entries with the tool calls a max-tokens finish drops', () => { + const assembler = new BlockAssembler() + assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'lead' } }) + assembler.push({ + type: 'block-end', + index: 1, + block: { type: 'tool-call', id: CallId('c1'), name: 'echo', arguments: '{"text":' }, + }) + assembler.push({ type: 'block-end', index: 2, block: { type: 'reasoning', text: 'tail' } }) + assembler.push({ + type: 'finish', + reason: { kind: 'max-tokens' }, + replayState: { response, blocks: ['meta-0', 'meta-1', 'meta-2'] }, + }) + + expect(assembler.blocks()).toEqual([ + { type: 'text', text: 'lead' }, + { type: 'reasoning', text: 'tail' }, + ]) + expect(assembler.replayState).toEqual({ response, blocks: ['meta-0', 'meta-2'] }) + }) + + it('omits replay metadata whose per-block entries misalign with the emitted blocks', () => { + const assembler = new BlockAssembler() + assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'one' } }) + assembler.push({ type: 'block-end', index: 1, block: { type: 'text', text: 'two' } }) + assembler.push({ + type: 'finish', + reason: { kind: 'stop' }, + replayState: { response, blocks: ['meta-0'] }, + }) + + expect(assembler.blocks()).toHaveLength(2) + expect(assembler.replayState).toBeUndefined() + }) + + it('passes replay metadata through unchanged when assembly drops nothing', () => { + const replayState = { response, blocks: ['meta-0', 'meta-1'] } + const assembler = new BlockAssembler() + assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'partial' } }) + assembler.push({ + type: 'block-end', + index: 1, + block: { type: 'tool-call', id: CallId('c1'), name: 'echo', arguments: '{}' }, + }) + assembler.push({ type: 'finish', reason: { kind: 'tool-calls' }, replayState }) + + expect(assembler.replayState).toBe(replayState) + }) + + it('keeps a max-tokens replay state with no per-block entries across a tool-call drop', () => { + const replayState = { response } + const assembler = new BlockAssembler() + assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'partial' } }) + assembler.push({ + type: 'block-end', + index: 1, + block: { type: 'tool-call', id: CallId('c1'), name: 'echo', arguments: '{"text":' }, + }) + assembler.push({ type: 'finish', reason: { kind: 'max-tokens' }, replayState }) + + expect(assembler.blocks()).toEqual([{ type: 'text', text: 'partial' }]) + expect(assembler.replayState).toBe(replayState) + }) + + it('keeps a text-only max-tokens response and its replay metadata intact', () => { + const replayState = { response, blocks: ['meta-0'] } + const assembler = new BlockAssembler() + assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'partial' } }) + assembler.push({ type: 'finish', reason: { kind: 'max-tokens' }, replayState }) + + expect(assembler.blocks()).toEqual([{ type: 'text', text: 'partial' }]) + expect(assembler.replayState).toBe(replayState) + }) +}) + describe('assertNever', () => { it('throws with diagnostics when a value escapes a closed union at runtime', async () => { const { assertNever } = await import('@deepseek-ai/dsh-llm') @@ -145,3 +224,41 @@ describe('BlockAssembler duplicate-close contract', () => { expect(assembler.blocks()).toEqual([{ type: 'reasoning', text: 'first' }]) }) }) + +describe('BlockAssembler.interruptedBlocks', () => { + it('keeps closed and open text/reasoning blocks with streamed content, in order', () => { + const assembler = new BlockAssembler() + assembler.push({ type: 'block-start', index: 0, blockType: 'reasoning' }) + assembler.push({ type: 'reasoning-delta', index: 0, text: 'planning' }) + assembler.push({ type: 'block-end', index: 0, block: { type: 'reasoning', text: 'planning' } }) + assembler.push({ type: 'block-start', index: 1, blockType: 'text' }) + assembler.push({ type: 'text-delta', index: 1, text: 'half an ans' }) + expect(assembler.interruptedBlocks()).toEqual([ + { type: 'reasoning', text: 'planning' }, + { type: 'text', text: 'half an ans' }, + ]) + }) + + it('drops tool calls whether open or closed — interruption precedes dispatch', () => { + const assembler = new BlockAssembler() + assembler.push({ type: 'block-start', index: 0, blockType: 'text' }) + assembler.push({ type: 'text-delta', index: 0, text: 'calling' }) + assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'calling' } }) + assembler.push({ type: 'block-start', index: 1, blockType: 'tool-call' }) + assembler.push({ type: 'tool-call-delta', index: 1, id: CallId('c1'), name: 'read', argumentsDelta: '{"a":1}' }) + assembler.push({ type: 'block-end', index: 1, block: { type: 'tool-call', id: CallId('c1'), name: 'read', arguments: '{"a":1}' } }) + assembler.push({ type: 'block-start', index: 2, blockType: 'tool-call' }) + assembler.push({ type: 'tool-call-delta', index: 2, id: CallId('c2'), name: 'read', argumentsDelta: '{"pa' }) + expect(assembler.interruptedBlocks()).toEqual([{ type: 'text', text: 'calling' }]) + }) + + it('drops empty and whitespace-only text/reasoning blocks and unknown open block types', () => { + const assembler = new BlockAssembler() + assembler.push({ type: 'block-start', index: 0, blockType: 'text' }) + assembler.push({ type: 'text-delta', index: 0, text: ' \n' }) + // A merge-extended block kind this build does not know how to assemble. + assembler.push({ type: 'block-start', index: 1, blockType: 'mystery' } as unknown as StreamChunk) + assembler.push({ type: 'block-start', index: 2, blockType: 'reasoning' }) + expect(assembler.interruptedBlocks()).toEqual([]) + }) +}) diff --git a/packages/llm/llm/tests/content.spec.ts b/packages/llm/llm/tests/content.spec.ts new file mode 100644 index 0000000000..6a0eb02c63 --- /dev/null +++ b/packages/llm/llm/tests/content.spec.ts @@ -0,0 +1,185 @@ +import { describe, expect, it } from 'vitest' +import { AttachmentId } from '@deepseek-ai/dsh-attachment' +import { + CallId, + createUserMessage, + OFFLOADED_IMAGE_TEXT, + offloadRequestImages, + offloadRequestImagesWithPolicy, + projectImagesForTextModel, +} from '../src/index.ts' +import type { ContentBlock } from '../src/index.ts' + +const source = { kind: 'plugin' as const, plugin: 'test' } + +function image(bytes: number): ContentBlock { + return { + type: 'image', + attachment: { + attachmentId: AttachmentId(`sha256:${'a'.repeat(64)}`), + mediaType: 'image/png', + bytes, + width: 1, + height: 1, + }, + } +} + +describe('offloadRequestImages', () => { + it('preserves every image when no payload bound is configured', () => { + const messages = [createUserMessage({ content: [image(300)], source })] + expect(offloadRequestImages(messages, undefined)).toBe(messages) + }) + + it('preserves the original request when its base64 payload fits exactly', () => { + const messages = [createUserMessage({ content: [image(3), image(3)], source })] + expect(offloadRequestImages(messages, 8)).toBe(messages) + }) + + it('keeps five 3 MiB images at 20 MiB and offloads the oldest after one more raw byte', () => { + const rawImageBytes = 3 * 1024 * 1024 + const maxRequestImageBytes = 20 * 1024 * 1024 + const exact = [createUserMessage({ + content: Array.from({ length: 5 }, () => image(rawImageBytes)), + source, + })] + expect(offloadRequestImages(exact, maxRequestImageBytes)).toBe(exact) + + const over = [createUserMessage({ + content: [image(rawImageBytes + 1), ...Array.from({ length: 4 }, () => image(rawImageBytes))], + source, + })] + expect(offloadRequestImages(over, maxRequestImageBytes)[0]?.content).toEqual([ + { type: 'text', text: OFFLOADED_IMAGE_TEXT }, + ...Array.from({ length: 4 }, () => image(rawImageBytes)), + ]) + }) + + it('replaces the oldest nested occurrences without mutating durable messages', () => { + const shared = image(3) + const messages = [ + createUserMessage({ + content: [{ + type: 'tool-result', + toolCallId: CallId('shot'), + content: [shared], + }], + source, + }), + createUserMessage({ content: [shared, image(3)], source }), + ] + + const fitted = offloadRequestImages(messages, 8) + expect(fitted).not.toBe(messages) + expect(fitted[0]?.content).toEqual([{ + type: 'tool-result', + toolCallId: CallId('shot'), + content: [{ type: 'text', text: OFFLOADED_IMAGE_TEXT }], + }]) + expect(fitted[1]?.content).toEqual([shared, image(3)]) + expect(messages[0]?.content[0]).toMatchObject({ type: 'tool-result', content: [shared] }) + }) + + it('replaces a single image that cannot fit', () => { + const messages = [createUserMessage({ content: [image(300)], source })] + expect(offloadRequestImages(messages, 8)[0]?.content) + .toEqual([{ type: 'text', text: OFFLOADED_IMAGE_TEXT }]) + }) + + it('keeps unchanged nested content while replacing a later image', () => { + const nested = { + type: 'tool-result' as const, + toolCallId: CallId('text-only'), + content: [{ type: 'text' as const, text: 'kept' }], + } + const messages = [createUserMessage({ content: [nested, image(3)], source })] + expect(offloadRequestImages(messages, 1)[0]?.content).toEqual([ + nested, + { type: 'text', text: OFFLOADED_IMAGE_TEXT }, + ]) + }) +}) + +describe('offloadRequestImagesWithPolicy', () => { + it('drops 129 MiB to 64 MiB and keeps the removed prefix stable through 192 MiB', () => { + const mib = 1024 * 1024 + const project = (count: number) => offloadRequestImagesWithPolicy([ + createUserMessage({ content: Array.from({ length: count }, () => image(mib)), source }), + ], { + representation: 'raw', + maxBytes: 128 * mib, + byteQuantum: 64 * mib, + })[0]?.content + + expect(project(128)?.filter(block => block.type === 'image')).toHaveLength(128) + expect(project(129)?.filter(block => block.type === 'text')).toHaveLength(65) + expect(project(192)?.filter(block => block.type === 'text')).toHaveLength(65) + expect(project(193)?.filter(block => block.type === 'text')).toHaveLength(129) + }) + + it('rounds a count excess up to a 20-image removal step', () => { + const projected = offloadRequestImagesWithPolicy([ + createUserMessage({ content: Array.from({ length: 601 }, () => image(1)), source }), + ], { + representation: 'raw', + maxImages: 600, + countQuantum: 20, + }) + expect(projected[0]?.content.filter(block => block.type === 'text')).toHaveLength(20) + expect(projected[0]?.content.filter(block => block.type === 'image')).toHaveLength(581) + }) + + it('uses route-owned request byte lengths when supplied', () => { + const messages = [createUserMessage({ content: [image(100), image(100)], source })] + const projected = offloadRequestImagesWithPolicy(messages, { + representation: 'raw', + maxBytes: 3, + byteLength: () => 2, + }) + expect(projected[0]?.content).toEqual([ + { type: 'text', text: OFFLOADED_IMAGE_TEXT }, + image(100), + ]) + }) +}) + +describe('projectImagesForTextModel', () => { + it('returns image-free history unchanged', () => { + const messages = [createUserMessage({ content: [{ type: 'text', text: 'plain' }], source })] + expect(projectImagesForTextModel(messages)).toBe(messages) + }) + + it('replaces direct and nested images while retaining unaffected messages and blocks', () => { + const plain = createUserMessage({ content: [{ type: 'text', text: 'plain' }], source }) + const nested = { + type: 'tool-result' as const, + toolCallId: CallId('nested-image'), + content: [{ type: 'text' as const, text: 'before' }, image(3), { type: 'text' as const, text: 'after' }], + } + const unchangedNested = { + type: 'tool-result' as const, + toolCallId: CallId('text-only'), + content: [{ type: 'text' as const, text: 'unchanged' }], + } + const visual = createUserMessage({ + content: [{ type: 'text', text: 'lead' }, image(3), unchangedNested, nested], + source, + }) + + const projected = projectImagesForTextModel([plain, visual]) + expect(projected[0]).toBe(plain) + expect(projected[1]?.content).toEqual([ + { type: 'text', text: 'lead' }, + { type: 'text', text: '[image omitted because this model accepts text only; attachment sha256:aaaaaaaa]' }, + unchangedNested, + { + ...nested, + content: [ + { type: 'text', text: 'before' }, + { type: 'text', text: '[image omitted because this model accepts text only; attachment sha256:aaaaaaaa]' }, + { type: 'text', text: 'after' }, + ], + }, + ]) + }) +}) diff --git a/packages/llm/llm/tests/retry-policy.spec.ts b/packages/llm/llm/tests/retry-policy.spec.ts index 1860625147..cc7ebb8fa7 100644 --- a/packages/llm/llm/tests/retry-policy.spec.ts +++ b/packages/llm/llm/tests/retry-policy.spec.ts @@ -12,7 +12,7 @@ describe('provider retry policy', () => { expect(policy).toEqual({ mode: 'normal', - maxRetries: 2, + maxRetries: 5, retryableCodes: ['EMPTY_RESPONSE', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'], initialDelayMs: 500, maxDelayMs: 10_000, @@ -59,6 +59,21 @@ describe('provider retry policy', () => { expect(RetryPolicySchema).toBeDefined() }) + it('ignores normal-only fields retained after switching to always mode', () => { + const layered = { + mode: 'always', + maxRetries: 5, + retryableCodes: ['SERVER'], + } as unknown as RetryPolicyConfig + + expect(resolveRetryPolicy(layered, 'provider.retryPolicy')).toEqual({ + mode: 'always', + initialDelayMs: 500, + maxDelayMs: 10_000, + jitterRatio: 0.1, + }) + }) + it.each([ [{ mode: 'normal', maxRetries: -1 }, /maxRetries/], [{ mode: 'normal', maxRetries: 1.5 }, /maxRetries/], @@ -74,7 +89,6 @@ describe('provider retry policy', () => { [{ mode: 'normal', retryableCodes: [''] }, /non-empty strings/], [{ mode: 'normal', retryableCodes: [429] }, /non-empty strings/], [{ mode: 'normal', maxRetires: 1 }, /unknown key "maxRetires"/], - [{ mode: 'always', maxRetries: 1 }, /unknown key "maxRetries"/], [{ mode: 'always', backoff: { initialDelay: 1 } }, /unknown key "initialDelay"/], [{ mode: 'sometimes' }, /mode must be "normal" or "always"/], ] as const)('rejects invalid policy %#', (config, message) => { diff --git a/packages/llm/llm/tests/service.spec.ts b/packages/llm/llm/tests/service.spec.ts index ed7d8d9bf8..4c624fa862 100644 --- a/packages/llm/llm/tests/service.spec.ts +++ b/packages/llm/llm/tests/service.spec.ts @@ -1,5 +1,6 @@ import { describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' +import { AttachmentId } from '@deepseek-ai/dsh-attachment' import LlmRuntime, { errorChain, GenerateOptions, @@ -13,6 +14,7 @@ import LlmRuntime, { resolveRetryPolicy, StreamChunk, createMessage, + createUserMessage, } from '@deepseek-ai/dsh-llm' import type { LlmModelContext, @@ -217,7 +219,7 @@ describe('LlmRuntime', () => { expect(ctx.llm.providerRetryPolicy('configured')).toBe(configured) expect(ctx.llm.providerRetryPolicy('defaulted')).toMatchObject({ mode: 'normal', - maxRetries: 2, + maxRetries: 5, }) expect(() => ctx.llm.providerRetryPolicy('missing')).toThrow( expect.objectContaining({ code: 'NO_ADAPTER' }), @@ -915,6 +917,89 @@ describe('LlmRuntime', () => { expect(resolutions).toBe(2) }) + it('binds adapter-owned capabilities and dispatch to one prepared generation', async () => { + const ctx = new Context() + await ctx.plugin(LlmRuntime) + let generation = 'first' + let dispatched: string | undefined + const adapter = new class extends ScriptedAdapter { + override prepareCall(provider: string, model: string) { + const captured = generation + return Promise.resolve({ + model: { provider, id: model, name: model, inputModalities: ['text'] as const }, + stream: (options: GenerateOptions) => { + dispatched = captured + return super.stream(options) + }, + }) + } + }(SCRIPT) + ctx.llm.registerAdapter(['route'], adapter) + + const prepared = await ctx.llm.prepareCall({ provider: 'route', model: 'model' }) + generation = 'second' + expect(prepared.inputModalities).toEqual(['text']) + expect(Object.isFrozen(prepared.inputModalities)).toBe(true) + await collect(prepared.stream({ ...prepared.config, messages: [] })) + expect(dispatched).toBe('first') + }) + + it('projects historical images to stable text only after the loop-visible waterfall', async () => { + const ctx = new Context() + await ctx.plugin(LlmRuntime) + const seen: GenerateOptions[] = [] + const adapter = new class extends ScriptedAdapter { + override resolveModel(provider: string, model: string): Promise { + return Promise.resolve({ provider, id: model, name: model, inputModalities: ['text'] }) + } + + override async * stream(options: GenerateOptions): AsyncIterable { + seen.push(options) + yield * super.stream(options) + } + }(SCRIPT) + ctx.llm.registerAdapter(['route'], adapter) + const attachment = { + attachmentId: AttachmentId(`sha256:${'a'.repeat(64)}`), + mediaType: 'image/png' as const, + bytes: 3, + width: 1, + height: 1, + } + const waterfall: GenerateOptions[] = [] + ctx.on('llm/stream', async function* (options, next) { + waterfall.push(options) + yield * next() + }) + + await collect(ctx.llm.stream({ + provider: 'route', + model: 'text-only', + messages: [createUserMessage({ + content: [{ type: 'image', attachment }], + source: { kind: 'plugin', plugin: 'test' }, + })], + })) + + expect(waterfall[0]?.messages[0]?.content).toEqual([{ type: 'image', attachment }]) + expect(seen[0]?.messages[0]?.content).toEqual([{ + type: 'text', + text: '[image omitted because this model accepts text only; attachment sha256:aaaaaaaa]', + }]) + + const frozen = Object.freeze({ + provider: 'route', + model: 'text-only', + messages: [createUserMessage({ + content: [{ type: 'image', attachment }], + source: { kind: 'plugin' as const, plugin: 'test' }, + })], + }) + await collect(ctx.llm.stream(frozen)) + expect(Object.isFrozen(seen[1])).toBe(true) + expect(Object.isFrozen(seen[1]?.messages)).toBe(true) + }) + it('passes cancellation through exact-model resolution', async () => { const ctx = new Context() await ctx.plugin(LlmRuntime) diff --git a/packages/llm/llm/tsconfig.json b/packages/llm/llm/tsconfig.json index 21044b821b..2206a6028e 100644 --- a/packages/llm/llm/tsconfig.json +++ b/packages/llm/llm/tsconfig.json @@ -25,6 +25,9 @@ }, { "path": "../../util/timeout" + }, + { + "path": "../../util/crypto" } ] } diff --git a/packages/llm/plugin-package-inventory-deepseek/README.i18n.yaml b/packages/llm/plugin-package-inventory-deepseek/README.i18n.yaml new file mode 100644 index 0000000000..6a8127b284 --- /dev/null +++ b/packages/llm/plugin-package-inventory-deepseek/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/llm/plugin-package-inventory-deepseek/README.md +README.md: 3100c89b3abf317c6dfceb9aa5e77f8f6717d5c4 +README.zh.md: 64592eb0d4a0a92fbd110cb5fdcd4f46d8258b5d diff --git a/packages/llm/plugin-package-inventory-deepseek/README.md b/packages/llm/plugin-package-inventory-deepseek/README.md new file mode 100644 index 0000000000..3100c89b3a --- /dev/null +++ b/packages/llm/plugin-package-inventory-deepseek/README.md @@ -0,0 +1,43 @@ +# @deepseek-ai/dsh-plugin-package-inventory-deepseek + +English | [中文](README.zh.md) + +Complete active Loader-backed plugin package inventory for official DeepSeek LLM API requests. This function plugin injects the Loader, live Agent registry, and `ctx.deepseekLlmApiExtensions`, then owns the `dsh_plugin_packages` field. + +## Configuration + +| Key | Default | Meaning | +|---|---:|---| +| `enabled` | `true` | Register the `dsh_plugin_packages` contribution. Set it to `false` to omit package metadata. | + +Shipped profiles use the default, so every official DeepSeek request carries the package inventory when preparation succeeds. + +## Collection + +Every request re-reads active non-group entries from the host Loader tree. When optional `ctx.agentPresets` is present and `sessionId` resolves to a live Agent joined to a standing preset, that preset's separate Loader tree joins the same collection; deployments without the service report the host tree only. Entries are included only while their root fiber is `ACTIVE` and their effective Loader state is enabled. + +Bare package and package-subpath specifiers resolve through Node's package search paths without requiring a `./package.json` export. Each ordinary entry uses its owning Loader tree base. A standing preset's root entries use the harness base, matching the preset Loader's deliberate bare-package override; nested includes retain their own bases. Relative and absolute modules walk to their nearest manifest; a manifest without `name` marks a loose module and contributes no package identity. A named package manifest must also declare a non-empty `version`, and malformed package metadata fails request preparation. Exact name/version pairs are deduplicated and sorted with a locale-independent comparison, while simultaneously active different versions remain separate. + +The version-1 `dsh_plugin_packages` field contains only `{ name, version }` pairs. Disabled, pending, failed, disposed, unloading, structural `cordis:` rows, ordinary dependencies, loose files without an owning package identity, programmatically mounted child fibers, and in-memory dynamic plugins are excluded. + +## Model Experience + +### Package inventory metadata + +#### What the model sees + +Nothing. `dsh_plugin_packages` is provider metadata outside the model's messages, system prompt, and tool schemas. + +#### Token effect + +Zero model-input tokens; the complete inventory adds only HTTP request bytes. + +#### KV Cache effect + +None; package lifecycle changes do not alter the model-visible prefix. + +## Known Limitations and Deferred Work + +- **Loader package provenance only** — programmatic child fibers and in-memory dynamic plugins do not have authoritative npm name/version provenance and remain outside this inventory. +- **Loose modules are omitted** — a relative file without a named and versioned owning manifest is a plugin module, not a plugin package. +- **In-place package replacement requires restart** — manifest identities are cached for the process lifetime. Loader enable, disable, mount, unmount, and ordinary source HMR still refresh the active entry set, but replacing a mounted package's manifest with another version in the same process is not a supported upgrade path. diff --git a/packages/llm/plugin-package-inventory-deepseek/README.zh.md b/packages/llm/plugin-package-inventory-deepseek/README.zh.md new file mode 100644 index 0000000000..64592eb0d4 --- /dev/null +++ b/packages/llm/plugin-package-inventory-deepseek/README.zh.md @@ -0,0 +1,43 @@ +# @deepseek-ai/dsh-plugin-package-inventory-deepseek + +[English](README.md) | 中文 + +用于 DeepSeek 官方 LLM API 请求的完整存活 Loader 插件包清单。该函数插件注入 Loader、存活 Agent 注册表与 `ctx.deepseekLlmApiExtensions`,并拥有 `dsh_plugin_packages` 字段。 + +## 配置 + +| 配置键 | 默认值 | 含义 | +|---|---:|---| +| `enabled` | `true` | 注册 `dsh_plugin_packages` 贡献。将其设为 `false` 可省略包元数据。 | + +随附 profile 使用该默认值,因此只要准备成功,每个 DeepSeek 官方请求都会携带包清单。 + +## 收集 + +每次请求都会重读宿主 Loader 树中的存活非 group 配置项。存在可选 `ctx.agentPresets` 且 `sessionId` 解析到已加入 standing preset 的存活 Agent 时,该 preset 的独立 Loader 树也会加入同一次收集;未挂载该服务的部署只报告宿主树。只有根 fiber 处于 `ACTIVE` 且 Loader 有效状态为启用的配置项才会纳入。 + +裸包与包子路径 specifier 通过 Node 包搜索路径解析,无需包导出 `./package.json`。每个普通配置项使用其所属 Loader 树的基址。standing preset 的根配置项使用宿主基址,与 preset Loader 对裸包的显式覆写保持一致;嵌套 include 仍使用自身基址。相对与绝对模块会向上查找最近的 manifest(元数据清单);没有 `name` 的 manifest 只标记松散模块,不贡献包身份。具名包 manifest 还必须声明非空 `version`,格式错误的包元数据会使请求准备失败。系统使用与 locale 无关的比较按确切名称/版本对去重并排序,同时存活的不同版本仍会分开保留。 + +版本 1 的 `dsh_plugin_packages` 字段只包含 `{ name, version }` 对。系统会排除禁用、pending、failed、disposed、unloading 状态,结构性 `cordis:` 配置项,普通依赖,没有所属包身份的松散文件,以编程方式挂载的子 fiber,以及内存动态插件。 + +## 模型体验 + +### 包清单元数据 + +#### 模型看到的内容 + +无。`dsh_plugin_packages` 是位于模型消息、系统提示词与工具 schema 之外的提供方元数据。 + +#### Token 影响 + +模型输入 token 为零;完整清单只会增加 HTTP 请求字节数。 + +#### KV Cache 影响 + +无;包生命周期变化不会改变模型可见前缀。 + +## 已知限制与暂缓事项 + +- **仅含 Loader 包来源**——以编程方式创建的子 fiber 与内存动态插件没有权威 NPM 名称/版本来源,因此不在该清单内。 +- **省略松散模块**——没有具名且带版本所属 manifest 的相对文件是插件模块,不是插件包。 +- **原地替换包需要重启**——manifest 身份会在进程存活期内缓存。Loader 的启用、禁用、挂载、卸载与普通源码 HMR 仍会刷新存活配置项集合,但在同一进程中把已挂载包的 manifest 替换为另一版本并不是受支持的升级路径。 diff --git a/packages/llm/plugin-package-inventory-deepseek/package.json b/packages/llm/plugin-package-inventory-deepseek/package.json new file mode 100644 index 0000000000..4b8eea1427 --- /dev/null +++ b/packages/llm/plugin-package-inventory-deepseek/package.json @@ -0,0 +1,67 @@ +{ + "name": "@deepseek-ai/dsh-plugin-package-inventory-deepseek", + "description": "Active Loader-backed plugin package inventory for official DeepSeek LLM API requests", + "version": "0.1.1-rc.2", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/llm/plugin-package-inventory-deepseek" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./types": { + "types": "./lib/types/types.d.ts", + "default": "./lib/types/types.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.js", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "dependencies": { + "@deepseek-ai/schemastery": "workspace:^" + }, + "peerDependencies": { + "@deepseek-ai/cordis-plugin-loader": "workspace:^", + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-agent-presets": "workspace:^", + "@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + }, + "peerDependenciesMeta": { + "@deepseek-ai/dsh-agent-presets": { + "optional": true + } + }, + "devDependencies": { + "@deepseek-ai/cordis-plugin-include": "workspace:^", + "@deepseek-ai/cordis-plugin-loader": "workspace:^", + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-agent-presets": "workspace:^", + "@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-scope": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + } +} diff --git a/packages/llm/plugin-package-inventory-deepseek/src/index.ts b/packages/llm/plugin-package-inventory-deepseek/src/index.ts new file mode 100644 index 0000000000..40aeadfafa --- /dev/null +++ b/packages/llm/plugin-package-inventory-deepseek/src/index.ts @@ -0,0 +1,198 @@ +/** + * Active Loader-backed plugin package inventory for official DeepSeek requests. + * Host entries and the requesting agent's standing preset are resolved at request time; + * installed dependencies and plugin fibers without Loader package provenance are excluded. + * @module @deepseek-ai/dsh-plugin-package-inventory-deepseek + */ + +import { existsSync, readFileSync } from 'node:fs' +import { createRequire } from 'node:module' +import { dirname, isAbsolute, join, parse } from 'node:path' +import { fileURLToPath, pathToFileURL } from 'node:url' +import { FiberState, type Context } from '@deepseek-ai/cordis' +import z from '@deepseek-ai/schemastery' +import type { Entry, EntryTree } from '@deepseek-ai/cordis-plugin-loader' +import type {} from '@deepseek-ai/dsh-agent' +import type {} from '@deepseek-ai/dsh-deepseek-llm-api-extensions' +import { SessionId } from '@deepseek-ai/dsh-session' +import type {} from '@deepseek-ai/dsh-agent-presets' +import type { DeepSeekPluginPackageIdentity, DeepSeekPluginPackageInventoryExtension } from './types.ts' +import type {} from './types.ts' + +export type * from './types.ts' + +/** Cordis plugin name. */ +export const name = 'plugin-package-inventory-deepseek' +/** Services required to locate host/requesting-agent entries and contribute the field. */ +export const inject = ['agents', 'deepseekLlmApiExtensions', 'loader'] + +/** Plugin-package request contribution configuration. */ +export interface Config { + /** Contribute `dsh_plugin_packages` to official DeepSeek requests. Defaults to `true`. */ + enabled?: boolean +} + +/** Validated plugin-package request contribution configuration. */ +export const Config: z = z.object({ + enabled: z.boolean().default(true), +}) + +interface PackageManifest { + readonly name?: unknown + readonly version?: unknown +} + +interface ActiveEntry { + readonly entry: Entry + /** Bare-package base used by the Loader path that activated this entry. */ + readonly bareBaseUrl?: string +} + +/** Parse a bare package or package-subpath specifier into its package name. */ +function barePackageName(specifier: string): string | undefined { + if (specifier.startsWith('.') || specifier.includes(':') || isAbsolute(specifier)) return undefined + const [first = '', second = ''] = specifier.split('/') + // An active Loader entry already passed module resolution, so a scoped bare name has its package segment. + return first.startsWith('@') ? `${first}/${second}` : first +} + +/** Read one manifest identity, optionally treating an absent name as a loose-module marker. */ +function identityFromManifest(path: string, allowAnonymous: boolean): DeepSeekPluginPackageIdentity | undefined { + const manifest = JSON.parse(readFileSync(path, 'utf8')) as PackageManifest + if (allowAnonymous && manifest.name === undefined) return undefined + if (typeof manifest.name !== 'string' || manifest.name.length === 0 + || typeof manifest.version !== 'string' || manifest.version.length === 0) { + throw new Error(`plugin-package-inventory-deepseek: ${path} must declare non-empty name and version`) + } + return { name: manifest.name, version: manifest.version } +} + +/** Resolve a bare package without requiring it to export `./package.json`. */ +function barePackageManifest(packageName: string, anchors: readonly string[]): string | undefined { + for (const anchor of anchors) { + const searchPaths = createRequire(anchor).resolve.paths(packageName) + /* v8 ignore next -- active non-builtin package entries always have Node package search paths */ + if (searchPaths === null) continue + for (const searchPath of searchPaths) { + const manifest = join(searchPath, packageName, 'package.json') + if (existsSync(manifest)) return manifest + } + } + return undefined +} + +/** Find the nearest owning manifest for a relative or absolute plugin module. */ +function nearestManifest(modulePath: string): string | undefined { + let current = dirname(modulePath) + const root = parse(current).root + while (true) { + const manifest = join(current, 'package.json') + if (existsSync(manifest)) return manifest + if (current === root) return undefined + current = dirname(current) + } +} + +/** Exact package identity resolver with immutable per-process manifest caching. */ +class PackageIdentityResolver { + // TODO: Invalidate manifest identities if in-process package-version replacement becomes a supported upgrade path. + private readonly cache = new Map() + + constructor(private readonly hostBaseUrl: string) {} + + /** Resolve one Loader entry's owning package, or absence for a non-package loose module. */ + resolve({ entry, bareBaseUrl }: ActiveEntry): DeepSeekPluginPackageIdentity | undefined { + /* v8 ignore next -- Loader entry trees inherit a base URL; the fallback supports direct embedders. */ + const treeBase = entry.parent.tree.ctx.baseUrl ?? this.hostBaseUrl + const anchors = [...new Set([bareBaseUrl ?? treeBase, treeBase, this.hostBaseUrl, import.meta.url])] + const key = `${anchors.join('\u0000')}\u0000${entry.options.name}` + if (this.cache.has(key)) return this.cache.get(key) + + const packageName = barePackageName(entry.options.name) + let manifest: string | undefined + if (packageName !== undefined) { + manifest = barePackageManifest(packageName, anchors) + if (manifest === undefined) { + throw new Error(`plugin-package-inventory-deepseek: cannot resolve active package ${JSON.stringify(packageName)}`) + } + } else if (!entry.options.name.startsWith('cordis:')) { + const moduleUrl = isAbsolute(entry.options.name) + ? pathToFileURL(entry.options.name) + : new URL(entry.options.name, treeBase) + if (moduleUrl.protocol === 'file:') manifest = nearestManifest(fileURLToPath(moduleUrl)) + } + const identity = manifest === undefined ? undefined : identityFromManifest(manifest, packageName === undefined) + this.cache.set(key, identity) + return identity + } +} + +/** Yield active, non-structural entries from one Loader tree. */ +function activeEntries(tree: EntryTree, rootBareBaseUrl?: string): ActiveEntry[] { + return [...tree.entries()] + .filter(entry => !entry.options.group + && !entry.disabled + && entry.fiber?.state === FiberState.ACTIVE) + .map(entry => ({ + entry, + ...entry.parent.tree === tree && rootBareBaseUrl !== undefined + ? { bareBaseUrl: rootBareBaseUrl } + : {}, + })) +} + +/** Deterministic text order independent of the host's ICU data and locale. */ +function compareWireText(left: string, right: string): number { + return left < right ? -1 : left > right ? 1 : 0 +} + +/** Collect the full active package set for one request. */ +async function collectActivePluginPackages( + ctx: Context, + resolver: PackageIdentityResolver, + hostBaseUrl: string, + sessionId?: string, +): Promise { + const entries = activeEntries(ctx.loader) + if (sessionId !== undefined && ctx.get('agentPresets') !== undefined) { + const agent = ctx.agents.get(SessionId(sessionId)) + if (agent !== undefined) { + // The optional peer is loaded only when its service is present. Its existing + // mount query keeps Loader internals off the public AgentPresets service. + const { standingMountFor } = await import('@deepseek-ai/dsh-agent-presets') + const presetTree = standingMountFor(agent.ctx)?.tree + // PresetTree deliberately resolves its root bare rows from the harness; + // nested ordinary includes retain their own tree base. + if (presetTree !== undefined) entries.push(...activeEntries(presetTree, hostBaseUrl)) + } + } + const unique = new Map() + for (const activeEntry of entries) { + const identity = resolver.resolve(activeEntry) + if (identity === undefined) continue + unique.set(`${identity.name}\u0000${identity.version}`, identity) + } + return [...unique.values()].sort((left, right) => ( + compareWireText(left.name, right.name) || compareWireText(left.version, right.version) + )) +} + +/** + * Register the complete `dsh_plugin_packages` request contribution when enabled. + * @param ctx - plugin context carrying Loader provenance and the DeepSeek request-extension registry. + * @param config - validated default-on configuration. + */ +export function apply(ctx: Context, config: Config): void { + if (config.enabled === false) return + const hostBaseUrl = ctx.baseUrl ?? import.meta.url + const resolver = new PackageIdentityResolver(hostBaseUrl) + ctx.deepseekLlmApiExtensions.register('dsh_plugin_packages', { + prepare: async (request) => { + const value: DeepSeekPluginPackageInventoryExtension = { + version: 1, + packages: await collectActivePluginPackages(ctx, resolver, hostBaseUrl, request.sessionId), + } + return { value } + }, + }) +} diff --git a/packages/llm/plugin-package-inventory-deepseek/src/invariant.ts b/packages/llm/plugin-package-inventory-deepseek/src/invariant.ts new file mode 100644 index 0000000000..0325dc3c4f --- /dev/null +++ b/packages/llm/plugin-package-inventory-deepseek/src/invariant.ts @@ -0,0 +1,27 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-plugin-package-inventory-deepseek`. */ + +/* jscpd:ignore-start */ +import type { Context } from '@deepseek-ai/cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-plugin-package-inventory-deepseek' + +/** Cordis companion plugin name. */ +export const name = 'plugin-package-inventory-deepseek-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: each request reads authoritative Loader fiber state and + * package manifests directly; the plugin retains no independently mutable inventory. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/llm/plugin-package-inventory-deepseek/src/types.ts b/packages/llm/plugin-package-inventory-deepseek/src/types.ts new file mode 100644 index 0000000000..8135a119d7 --- /dev/null +++ b/packages/llm/plugin-package-inventory-deepseek/src/types.ts @@ -0,0 +1,19 @@ +/** Wire types for the active DeepSeek plugin package inventory. */ + +/** One exact active plugin package version. */ +export interface DeepSeekPluginPackageIdentity { + readonly name: string + readonly version: string +} + +/** Versioned full package inventory carried by each official DeepSeek request. */ +export interface DeepSeekPluginPackageInventoryExtension { + readonly version: 1 + readonly packages: readonly DeepSeekPluginPackageIdentity[] +} + +declare module '@deepseek-ai/dsh-deepseek-llm-api-extensions/types' { + interface DeepSeekLlmApiExtensionMap { + dsh_plugin_packages: DeepSeekPluginPackageInventoryExtension + } +} diff --git a/packages/llm/plugin-package-inventory-deepseek/tests/inventory.spec.ts b/packages/llm/plugin-package-inventory-deepseek/tests/inventory.spec.ts new file mode 100644 index 0000000000..aabbe7362f --- /dev/null +++ b/packages/llm/plugin-package-inventory-deepseek/tests/inventory.spec.ts @@ -0,0 +1,233 @@ +import { afterEach, describe, expect, it } from 'vitest' +import { mkdtemp, mkdir, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { pathToFileURL } from 'node:url' +import { Context } from '@deepseek-ai/cordis' +import Loader from '@deepseek-ai/cordis-plugin-loader' +import Include from '@deepseek-ai/cordis-plugin-include' +import AgentRegistry, { type Agent } from '@deepseek-ai/dsh-agent' +import { SessionId } from '@deepseek-ai/dsh-session' +import { createScope } from '@deepseek-ai/dsh-scope' +import AgentPresets, { mountPreset } from '@deepseek-ai/dsh-agent-presets' +import DeepSeekLlmApiExtensionRegistry from '@deepseek-ai/dsh-deepseek-llm-api-extensions' +import * as PluginInventory from '../src/index.ts' + +const contexts: Context[] = [] +const roots: string[] = [] +const SIGNAL = new AbortController().signal + +afterEach(async () => { + await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose())) + await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true }))) +}) + +async function packagePlugin( + root: string, + dir: string, + manifest: object, + source = 'export default () => {}\n', +): Promise { + const packageDir = join(root, dir) + await mkdir(packageDir, { recursive: true }) + await writeFile(join(packageDir, 'package.json'), `${JSON.stringify({ type: 'module', ...manifest })}\n`) + await writeFile(join(packageDir, 'plugin.mjs'), source) + return `./${dir}/plugin.mjs` +} + +async function harness(enabled?: boolean): Promise<{ ctx: Context; root: string; disposeInventory: () => Promise }> { + const root = await mkdtemp(join(tmpdir(), 'dsh-plugin-packages-')) + roots.push(root) + const ctx = new Context() + contexts.push(ctx) + ctx.baseUrl = pathToFileURL(join(root, 'cordis.yml')).href + await ctx.plugin(Loader) + ctx.loader.builtins.include = Include + await ctx.plugin(AgentRegistry) + await ctx.plugin(AgentPresets, { default: 'fixture', roots: [], includeUserRoot: false }) + await ctx.plugin(DeepSeekLlmApiExtensionRegistry) + const inventory = enabled === undefined + ? ctx.plugin(PluginInventory) + : ctx.plugin(PluginInventory, { enabled }) + await inventory + return { ctx, root, disposeInventory: () => inventory.dispose() } +} + +describe('DeepSeek plugin package inventory', () => { + it('contributes by default and can be explicitly disabled', async () => { + const defaultHarness = await harness() + const defaultFields = await defaultHarness.ctx.deepseekLlmApiExtensions.prepare({ + body: { messages: [] }, signal: SIGNAL, + }) + expect(defaultFields.fields).toHaveProperty('dsh_plugin_packages') + + const disabledHarness = await harness(false) + const disabledFields = await disabledHarness.ctx.deepseekLlmApiExtensions.prepare({ + body: { messages: [] }, signal: SIGNAL, + }) + expect(disabledFields.fields).not.toHaveProperty('dsh_plugin_packages') + }) + + it('reports active package versions once, retains parallel versions, and excludes inactive or loose entries', async () => { + const { ctx, root } = await harness() + const oneA = await packagePlugin(root, 'one-a', { name: 'one', version: '1.0.0' }) + const oneB = await packagePlugin(root, 'one-b', { name: 'one', version: '2.0.0' }) + const disabled = await packagePlugin(root, 'disabled', { name: 'disabled', version: '1.0.0' }) + await mkdir(join(root, 'loose'), { recursive: true }) + await writeFile(join(root, 'loose/plugin.mjs'), 'export default () => {}\n') + + await ctx.loader.create({ name: oneA }) + await ctx.loader.create({ name: oneA }) + await ctx.loader.create({ name: oneB }) + await ctx.loader.create({ name: disabled, disabled: true }) + await ctx.loader.create({ name: './loose/plugin.mjs' }) + + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL }) + expect(prepared.fields.dsh_plugin_packages).toEqual({ + version: 1, + packages: [ + { name: 'one', version: '1.0.0' }, + { name: 'one', version: '2.0.0' }, + ], + }) + }) + + it('fails request preparation for an active package with malformed identity metadata', async () => { + const { ctx, root } = await harness() + const bad = await packagePlugin(root, 'bad', { name: 'bad' }) + await ctx.loader.create({ name: bad }) + await expect(ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL })) + .rejects.toThrow(/must declare non-empty name and version/) + }) + + it('omits a loose ESM module whose nearest manifest only marks the module type', async () => { + const { ctx, root } = await harness() + const marker = await packagePlugin(root, 'marker-only', {}) + await ctx.loader.create({ name: marker }) + await expect(ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL })) + .resolves.toMatchObject({ fields: { dsh_plugin_packages: { version: 1, packages: [] } } }) + }) + + it('uses the host inventory when a request has no matching or joined live agent', async () => { + const { ctx, root } = await harness() + const plugin = await packagePlugin(root, 'host-only', { name: 'host-only', version: '3.0.0' }) + await ctx.loader.create({ name: plugin }) + const missing = await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL, sessionId: 'missing' }) + expect(missing.fields.dsh_plugin_packages?.packages).toEqual([{ name: 'host-only', version: '3.0.0' }]) + + const id = SessionId('bare-agent') + const agentScope = createScope(ctx, {}) + ctx.agents.register({ id, ctx: agentScope.ctx, session: { id } } as unknown as Agent) + const bare = await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL, sessionId: id }) + expect(bare.fields.dsh_plugin_packages?.packages).toEqual([{ name: 'host-only', version: '3.0.0' }]) + }) + + it('resolves scoped and unscoped bare subpaths, absolute/file modules, and skips URL or Cordis modules', async () => { + const { ctx, root } = await harness() + await packagePlugin(root, 'node_modules/plain-package', { name: 'plain-package', version: '1.0.0' }) + await packagePlugin(root, 'node_modules/@scope/scoped-package', { name: '@scope/scoped-package', version: '2.0.0' }) + await packagePlugin(root, 'absolute-package', { name: 'absolute-package', version: '3.0.0' }) + const absolute = join(root, 'absolute-package/plugin.mjs') + const internal = ctx.loader.internal + ctx.loader.internal = { + version: 'v2', + import: async (specifier: string, ...args: unknown[]) => { + if (specifier === 'https://plugins.example/test.mjs') return { default: () => {} } + // Node ESM on Windows requires a file URL; retain the raw Loader name for package attribution. + const portableSpecifier = specifier === absolute ? pathToFileURL(specifier).href : specifier + return await (internal as never as { import(specifier: string, ...args: unknown[]): Promise }) + .import(portableSpecifier, ...args) + }, + } as unknown as NonNullable + + await ctx.loader.create({ name: 'plain-package/plugin.mjs' }) + await ctx.loader.create({ name: '@scope/scoped-package/plugin.mjs' }) + await ctx.loader.create({ name: absolute }) + await ctx.loader.create({ name: pathToFileURL(absolute).href }) + ctx.loader.builtins.noop = () => {} + await ctx.loader.create({ name: 'cordis:noop' }) + await ctx.loader.create({ name: 'https://plugins.example/test.mjs' }) + + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL }) + expect(prepared.fields.dsh_plugin_packages?.packages).toEqual([ + { name: '@scope/scoped-package', version: '2.0.0' }, + { name: 'absolute-package', version: '3.0.0' }, + { name: 'plain-package', version: '1.0.0' }, + ]) + }) + + it('fails when a Loader-resolved bare entry has no package manifest', async () => { + const { ctx } = await harness() + ctx.loader.internal = { + version: 'v2', + import: async () => ({ default: () => {} }), + } as unknown as NonNullable + await ctx.loader.create({ name: 'missing-package' }) + await expect(ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL })) + .rejects.toThrow(/cannot resolve active package/) + }) + + it('supports a direct embedding whose context has no base URL', async () => { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(Loader) + await ctx.plugin(AgentRegistry) + await ctx.plugin(DeepSeekLlmApiExtensionRegistry) + await ctx.plugin(PluginInventory) + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL }) + expect(prepared.fields.dsh_plugin_packages).toEqual({ version: 1, packages: [] }) + }) + + it('uses each ordinary Loader tree base for conflicting bare package versions', async () => { + const { ctx, root } = await harness() + await packagePlugin(root, 'node_modules/versioned-plugin', { + name: 'versioned-plugin', version: '1.0.0', + }) + const nestedRoot = join(root, 'nested') + await packagePlugin(nestedRoot, 'node_modules/versioned-plugin', { + name: 'versioned-plugin', version: '2.0.0', + }) + const composition = join(nestedRoot, 'cordis.yml') + await writeFile(composition, '- id: nested\n name: versioned-plugin/plugin.mjs\n') + + await ctx.loader.create({ name: 'versioned-plugin/plugin.mjs' }) + await ctx.loader.create({ name: 'cordis:include', config: { path: pathToFileURL(composition).href } }) + + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL }) + expect(prepared.fields.dsh_plugin_packages?.packages).toEqual([ + { name: 'versioned-plugin', version: '1.0.0' }, + { name: 'versioned-plugin', version: '2.0.0' }, + ]) + }) + + it('mirrors the standing preset bare-package override instead of its local node_modules', async () => { + const { ctx, root } = await harness() + await packagePlugin(root, 'node_modules/preset-only', { name: 'preset-only', version: '4.0.0' }) + const presetDir = join(root, 'preset') + await mkdir(presetDir, { recursive: true }) + await packagePlugin(presetDir, 'node_modules/preset-only', { name: 'preset-only', version: '9.0.0' }) + const composition = join(presetDir, 'agent.cordis.yml') + await writeFile(composition, '- id: preset-only\n name: preset-only/plugin.mjs\n') + + const standingKey = {} + const standing = createScope(ctx, standingKey) + await mountPreset(standing.ctx, { id: 'fixture', trust: 'user', path: composition }) + const agentKey = {} + const agentScope = createScope(ctx, agentKey, { parent: standingKey }) + const id = SessionId('preset-agent') + const agent = { id, ctx: agentScope.ctx, session: { id } } as unknown as Agent + ctx.agents.register(agent) + + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL, sessionId: id }) + expect(prepared.fields.dsh_plugin_packages?.packages).toEqual([{ name: 'preset-only', version: '4.0.0' }]) + }) + + it('withdraws the inventory field when the contributing plugin reloads', async () => { + const { ctx, disposeInventory } = await harness() + expect((await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL })).fields) + .toHaveProperty('dsh_plugin_packages') + await disposeInventory() + expect((await ctx.deepseekLlmApiExtensions.prepare({ body: { messages: [] }, signal: SIGNAL })).fields) + .not.toHaveProperty('dsh_plugin_packages') + }) +}) diff --git a/packages/llm/plugin-package-inventory-deepseek/tsconfig.json b/packages/llm/plugin-package-inventory-deepseek/tsconfig.json new file mode 100644 index 0000000000..4edcfd192d --- /dev/null +++ b/packages/llm/plugin-package-inventory-deepseek/tsconfig.json @@ -0,0 +1,39 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../../vendor/loader" + }, + { + "path": "../../core/agent" + }, + { + "path": "../../core/session" + }, + { + "path": "../../preset/agent-presets" + }, + { + "path": "../deepseek-llm-api-extensions" + }, + { + "path": "../../runtime-diagnostics/invariants" + } + ] +} diff --git a/packages/llm/token-meter/README.i18n.yaml b/packages/llm/token-meter/README.i18n.yaml index 44db6530cd..98b96e634d 100644 --- a/packages/llm/token-meter/README.i18n.yaml +++ b/packages/llm/token-meter/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/token-meter/README.md README.md: a2deab11a31285ba598b8864d3a734ecf7c56620 -README.zh.md: e0c3a8aa123ca0acc374277cd92b415e0b0b3cb5 +README.zh.md: d14cded74691f88db7267ea470f536db85a39218 diff --git a/packages/llm/token-meter/README.zh.md b/packages/llm/token-meter/README.zh.md index e0c3a8aa12..d14cded746 100644 --- a/packages/llm/token-meter/README.zh.md +++ b/packages/llm/token-meter/README.zh.md @@ -41,7 +41,7 @@ fold 跟踪完整请求标头快照、步骤边界、表层追加与替换、成 这是刻意的选择。占用率百分比是面向用户的参考数字,既不是计费记录,也不是门控输入:harness 中没有任何环节依据它做决策,压缩改为直接读取 `measure()`。UI 用测得的压力除以为所选模型单独解析出的容量来计算占用率。 -[Agent Note](../../../.agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.md) 记录了否决「让这对值保持原子」方案的那次对比。需要同一边界精确数字的消费方应在自己的请求边界调用 `measure()`,而不是读取该投影。 +[Agent Note](../../../.agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.zh.md) 记录了否决「让这对值保持原子」方案的那次对比。需要同一边界精确数字的消费方应在自己的请求边界调用 `measure()`,而不是读取该投影。 ## 组合 diff --git a/packages/llm/token-meter/package.json b/packages/llm/token-meter/package.json index efaf61ef86..60a21c7a8e 100644 --- a/packages/llm/token-meter/package.json +++ b/packages/llm/token-meter/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-token-meter", "description": "Replay-aware token measurement service (ctx.tokenMeter) for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/llm/token-meter/src/breakdown-projection.ts b/packages/llm/token-meter/src/breakdown-projection.ts index c83879c63a..e0c980e843 100644 --- a/packages/llm/token-meter/src/breakdown-projection.ts +++ b/packages/llm/token-meter/src/breakdown-projection.ts @@ -10,22 +10,35 @@ import { canonicalHeader } from '@deepseek-ai/dsh-session' import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection' import { estimateSystemTokens, estimateToolsTokens } from './estimate.ts' import { foldSurfaceProjection } from './surface-projection.ts' -import type { ShadowPriceClaim } from './surface-projection.ts' -// Import for the `contextBreakdown` SessionProjectionMap key merge. +// Import for the `contextBreakdown` SessionProjectionStateMap key merge. import type {} from './projection.ts' -interface ContextBreakdownState { - systemTokens: number - toolsTokens: number - messageTokens: number - /** Shadow price armed by the immediately preceding metering event. */ - claim?: ShadowPriceClaim +declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + contextBreakdown: ContextBreakdownState + } } +/** Non-negative integer token count (the shared figure shape). */ +const tokenCount = z.number().int().nonnegative() + +/** The context-breakdown state schema and source of its inferred type. */ +const contextBreakdownStateSchema = z.object({ + systemTokens: tokenCount, + toolsTokens: tokenCount, + messageTokens: tokenCount, + claim: z.object({ + start: tokenCount, + end: tokenCount, + tokens: tokenCount, + }).optional(), +}).strict() +type ContextBreakdownState = z.infer + const breakdownSchema = z.object({ - systemTokens: z.number().int().nonnegative(), - toolsTokens: z.number().int().nonnegative(), - messageTokens: z.number().int().nonnegative(), + systemTokens: tokenCount, + toolsTokens: tokenCount, + messageTokens: tokenCount, }).strict() /** @@ -39,10 +52,10 @@ const breakdownSchema = z.object({ * state is a fixed handful of numbers, so the persisted checkpoint stays * O(1) over the session's life. */ -export const contextBreakdownProjectionDefinition: -ProjectionDefinition<'contextBreakdown', ContextBreakdownState> = { +export const contextBreakdownProjectionDefinition = { key: 'contextBreakdown', - schema: breakdownSchema, + stateVersion: 2, + stateSchema: contextBreakdownStateSchema, init: () => ({ systemTokens: 0, toolsTokens: 0, messageTokens: 0 }), apply: (state, event) => { const fold = foldSurfaceProjection(state.claim, event) @@ -65,6 +78,8 @@ ProjectionDefinition<'contextBreakdown', ContextBreakdownState> = { ...fold.claim === undefined ? {} : { claim: fold.claim }, } }, - view: ({ systemTokens, toolsTokens, messageTokens }) => ({ systemTokens, toolsTokens, messageTokens }), - stateVersion: 2, -} + wire: { + viewSchema: breakdownSchema, + view: ({ systemTokens, toolsTokens, messageTokens }) => ({ systemTokens, toolsTokens, messageTokens }), + }, +} satisfies ProjectionDefinition<'contextBreakdown', ContextBreakdownState> diff --git a/packages/llm/token-meter/src/usage-projection.ts b/packages/llm/token-meter/src/usage-projection.ts index a7fc9debf0..864b1669ce 100644 --- a/packages/llm/token-meter/src/usage-projection.ts +++ b/packages/llm/token-meter/src/usage-projection.ts @@ -8,18 +8,6 @@ import type { SessionEvent } from '@deepseek-ai/dsh-session' import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection' import type { ContextPressureProjection, TokenUsageProjection } from './projection.ts' import { foldSurfaceProjection } from './surface-projection.ts' -import type { ShadowPriceClaim } from './surface-projection.ts' - -interface UsageSample { - turn: number - step: number - buckets: TokenUsageProjection -} - -interface TokenUsageState { - totals: TokenUsageProjection - last: UsageSample | null -} const zeroBuckets = (): TokenUsageProjection => ({ uncachedInputTokens: 0, @@ -59,13 +47,30 @@ const projectionSchema = z.object({ cacheWriteTokens: z.number().int().nonnegative(), }).strict() -// Cast for the optional values: under exactOptionalPropertyTypes zod infers -// `number | undefined` where the interface declares absent-or-number fields. -const pressureSchema = z.object({ +/** + * The token-usage unit's state schema — the one definition of the state + * shape; the state type is inferred from it. + */ +const tokenUsageStateSchema = z.object({ + totals: projectionSchema, + last: z.object({ + turn: z.number().int().nonnegative(), + step: z.number().int().nonnegative(), + buckets: projectionSchema, + }).nullable(), +}).strict() + +type TokenUsageState = z.infer + +const pressureSchema: z.ZodType = z.object({ pressureTokens: z.number().int().nonnegative().optional(), projectedTokens: z.number().int().nonnegative().optional(), contextWindow: z.number().int().positive().optional(), -}).strict() as unknown as z.ZodType +}).strict().transform(({ pressureTokens, projectedTokens, contextWindow }) => ({ + ...pressureTokens === undefined ? {} : { pressureTokens }, + ...projectedTokens === undefined ? {} : { projectedTokens }, + ...contextWindow === undefined ? {} : { contextWindow }, +})) /** Prompt-side pressure of one request: input plus cache traffic, no output. */ const pressureFrom = (usage: TokenUsage): number => @@ -79,21 +84,28 @@ const usageOf = (event: SessionEvent): TokenUsage | undefined => ? event.data.usage : undefined -/** - * Context-occupancy state: the two independent last-wins records plus the - * O(1) running surface total needed to carry the newest sample forward. - */ -interface ContextPressureState { - contextWindow?: number - pressureTokens?: number - /** Running heuristic total over the current surface ({@link foldSurfaceProjection}). */ - surfaceTokens: number - /** {@link surfaceTokens} at the newest usage sample; absent until one lands. */ - sampledSurfaceTokens?: number - /** Shadow price armed by the immediately preceding metering event. */ - claim?: ShadowPriceClaim +declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + tokenUsage: TokenUsageState + contextPressure: ContextPressureState + } } +/** The context-pressure state schema and source of its inferred type. */ +const contextPressureStateSchema = z.object({ + contextWindow: z.number().int().positive().optional(), + pressureTokens: z.number().int().nonnegative().optional(), + surfaceTokens: z.number().int().nonnegative(), + sampledSurfaceTokens: z.number().int().nonnegative().optional(), + claim: z.object({ + start: z.number().int().nonnegative(), + end: z.number().int().nonnegative(), + tokens: z.number().int().nonnegative(), + }).optional(), +}).strict() + +type ContextPressureState = z.infer + /** * Token-meter's session projection unit. * @@ -104,10 +116,10 @@ interface ContextPressureState { * that usage reports for one turn/step are adjacent: once a later step begins, * a legal log never reports usage for an earlier step again. */ -export const tokenUsageProjectionDefinition: -ProjectionDefinition<'tokenUsage', TokenUsageState> = { +export const tokenUsageProjectionDefinition = { key: 'tokenUsage', - schema: projectionSchema, + stateVersion: 1, + stateSchema: tokenUsageStateSchema, init: () => ({ totals: zeroBuckets(), last: null }), apply: (state, event) => { let turn: number @@ -135,9 +147,8 @@ ProjectionDefinition<'tokenUsage', TokenUsageState> = { last: { turn, step, buckets }, } }, - view: state => state.totals, - stateVersion: 1, -} + wire: { viewSchema: projectionSchema, view: state => state.totals }, +} satisfies ProjectionDefinition<'tokenUsage', TokenUsageState> /** * Token-meter's context-occupancy projection unit. @@ -160,10 +171,10 @@ ProjectionDefinition<'tokenUsage', TokenUsageState> = { * BEFORE the same event joins the surface, so an `assistant/message` anchors * against the surface its own request saw. */ -export const contextPressureProjectionDefinition: -ProjectionDefinition<'contextPressure', ContextPressureState> = { +export const contextPressureProjectionDefinition = { key: 'contextPressure', - schema: pressureSchema, + stateVersion: 4, + stateSchema: contextPressureStateSchema, init: () => ({ surfaceTokens: 0 }), apply: (state, event) => { const fold = foldSurfaceProjection(state.claim, event) @@ -195,12 +206,14 @@ ProjectionDefinition<'contextPressure', ContextPressureState> = { const { claim: _expired, ...withoutClaim } = next return fold.claim === undefined ? withoutClaim : { ...withoutClaim, claim: fold.claim } }, - view: ({ contextWindow, pressureTokens, surfaceTokens, sampledSurfaceTokens }) => ({ - ...contextWindow === undefined ? {} : { contextWindow }, - ...pressureTokens === undefined ? {} : { pressureTokens }, - ...pressureTokens === undefined || sampledSurfaceTokens === undefined - ? {} - : { projectedTokens: Math.max(0, pressureTokens + surfaceTokens - sampledSurfaceTokens) }, - }), - stateVersion: 4, -} + wire: { + viewSchema: pressureSchema, + view: ({ contextWindow, pressureTokens, surfaceTokens, sampledSurfaceTokens }) => ({ + ...contextWindow === undefined ? {} : { contextWindow }, + ...pressureTokens === undefined ? {} : { pressureTokens }, + ...pressureTokens === undefined || sampledSurfaceTokens === undefined + ? {} + : { projectedTokens: Math.max(0, pressureTokens + surfaceTokens - sampledSurfaceTokens) }, + }), + }, +} satisfies ProjectionDefinition<'contextPressure', ContextPressureState> diff --git a/packages/llm/token-meter/tests/context-breakdown-projection.spec.ts b/packages/llm/token-meter/tests/context-breakdown-projection.spec.ts index 350ed335c9..6ab566d085 100644 --- a/packages/llm/token-meter/tests/context-breakdown-projection.spec.ts +++ b/packages/llm/token-meter/tests/context-breakdown-projection.spec.ts @@ -94,7 +94,7 @@ describe('contextBreakdown session projection', () => { header: { config: CONFIG, system: 'You are terse.', tools: TOOLS }, reason: 'change', }) - session.append('todo/write', { todos: [] }) + session.append('session/end-seed', {}) expect(changed).not.toContain('contextBreakdown') // A system-less, tool-less envelope prices back to zero. @@ -209,20 +209,20 @@ describe('contextBreakdown session projection', () => { state = definition.apply(state, append(1)) state = definition.apply(state, append(3)) // No metering event: the replacement contributes zero instead of throwing. - expect(definition.view(definition.apply(state, replace(1, 3))).messageTokens) - .toBe(definition.view(state).messageTokens) + expect(definition.wire.view(definition.apply(state, replace(1, 3))).messageTokens) + .toBe(definition.wire.view(state).messageTokens) // An adjacent claim for another range contradicts the replacement. const mismatched = definition.apply(state, meter(1, 1, 8)) expect(() => definition.apply(mismatched, replace(1, 3))).toThrow('no adjacent shadow price') // A claim expires after one intervening event, so replacement delta is zero. let expired = definition.apply(state, meter(1, 3, 8)) - expired = definition.apply(expired, { type: 'todo/write', seq: 9, time: 0, data: { todos: [] } } as unknown as SessionEvent) - expect(definition.view(definition.apply(expired, replace(1, 3))).messageTokens) - .toBe(definition.view(state).messageTokens) + expired = definition.apply(expired, { type: 'session/end-seed', seq: 9, time: 0, data: {} }) + expect(definition.wire.view(definition.apply(expired, replace(1, 3))).messageTokens) + .toBe(definition.wire.view(state).messageTokens) // The armed claim prices exactly the next event's matching replacement. const armed = definition.apply(state, meter(1, 3, 8)) - expect(definition.view(definition.apply(armed, replace(1, 3))).messageTokens) - .toBe(definition.view(state).messageTokens - 5 + estimateMessage( + expect(definition.wire.view(definition.apply(armed, replace(1, 3))).messageTokens) + .toBe(definition.wire.view(state).messageTokens - 5 + estimateMessage( createUserMessage({ content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } }), )) }) diff --git a/packages/llm/token-meter/tests/token-meter.spec.ts b/packages/llm/token-meter/tests/token-meter.spec.ts index 7805d39278..074f18fb76 100644 --- a/packages/llm/token-meter/tests/token-meter.spec.ts +++ b/packages/llm/token-meter/tests/token-meter.spec.ts @@ -4,6 +4,7 @@ import { createUserMessage, CallId, createMessage } from '@deepseek-ai/dsh-llm' import type { ContentBlock, Message, TokenUsage } from '@deepseek-ai/dsh-llm' import SessionStore, { Session, SessionId, canonicalHeader } from '@deepseek-ai/dsh-session' import type { EpochHeader, SessionEvent } from '@deepseek-ai/dsh-session' +import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import TokenMeter from '@deepseek-ai/dsh-token-meter' import type { TokenMeasurement, TokenMeterConfig } from '@deepseek-ai/dsh-token-meter' @@ -90,7 +91,11 @@ function appendSuccessfulCall( } function meter(config: TokenMeterConfig = {}): TokenMeter { - return new TokenMeter(new Context(), config) + const ctx = new Context() + // The registry is a required injection of the service (its three projection + // units register in the constructor); mount it synchronously. + new SessionProjectionRegistry(ctx) + return new TokenMeter(ctx, config) } function expectSurfaceTotal(measurement: TokenMeasurement): void { @@ -115,6 +120,7 @@ describe('TokenMeter configuration and registration', () => { it('registers and unregisters ctx.tokenMeter with its plugin fiber', async () => { const ctx = new Context() await ctx.plugin(SessionStore) + await ctx.plugin(SessionProjectionRegistry) const fiber = await ctx.plugin(TokenMeter) expect(ctx.get('tokenMeter')).toBeInstanceOf(TokenMeter) await fiber.dispose() @@ -660,6 +666,7 @@ describe('malformed replay and listener lifecycle', () => { it('handles earlier-reader catch-up, eager observation, and service reload', async () => { const ctx = new Context() await ctx.plugin(SessionStore) + await ctx.plugin(SessionProjectionRegistry) let activeMeter: TokenMeter | undefined const revisions: number[] = [] ctx.on('session/event', (session) => { diff --git a/packages/llm/token-meter/tests/token-usage-projection.spec.ts b/packages/llm/token-meter/tests/token-usage-projection.spec.ts index d72ba7f1f6..d076459559 100644 --- a/packages/llm/token-meter/tests/token-usage-projection.spec.ts +++ b/packages/llm/token-meter/tests/token-usage-projection.spec.ts @@ -8,6 +8,7 @@ import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import TokenMeter from '@deepseek-ai/dsh-token-meter' import type { ContextPressureProjection, TokenUsageProjection } from '@deepseek-ai/dsh-token-meter/client' import { CompactionId } from '@deepseek-ai/dsh-compaction' +import type {} from '../src/usage-projection.ts' const ZERO: TokenUsageProjection = { uncachedInputTokens: 0, @@ -356,7 +357,7 @@ describe('contextPressure session projection', () => { const changed: string[] = [] ctx.sessionProjections.onChanged((_session, key) => { changed.push(key) }) - session.append('todo/write', { todos: [] }) + session.append('session/end-seed', {}) expect(changed).not.toContain('contextPressure') // A repeated capacity record for the same window is also a no-op. recordContext(session, 'small', 64_000) diff --git a/packages/lsp/README.i18n.yaml b/packages/lsp/README.i18n.yaml index 195967e3a2..2670636cb5 100644 --- a/packages/lsp/README.i18n.yaml +++ b/packages/lsp/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/README.md README.md: 414311fd3f537634651daf6f313baf33daede26b -README.zh.md: 8791b0ea9c01ed637586cee5c131690a73263a6a +README.zh.md: 8128bd410daeaa12186191a2dccddddc2a0e5b37 diff --git a/packages/lsp/README.zh.md b/packages/lsp/README.zh.md index 8791b0ea9c..8128bd410d 100644 --- a/packages/lsp/README.zh.md +++ b/packages/lsp/README.zh.md @@ -12,6 +12,6 @@ Service Definition 位于 `lsp/lsp/`。该 seam 恰好公开四种语义操作:`goToDefinition`、`findReferences`、`goToImplementation`、`hover`,且不提供通用 JSON-RPC 逃生口;因此,替换提供方不会改变模型请求导航的方式,也不会让协议载荷或未经评审的修改进入模型约定。提供方注册的是**能力**而非工具;`tool-lsp` 是面向模型的名称、schema、提示词指引和呈现的唯一 owner。 -设计原理见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md),其中也解释了文档为何在每次查询时临时打开、stdio 主机为何使用共享的文件系统/子进程执行环境,以及扩展名归属为何在同一运行时内互斥。 +设计原理见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md),其中也解释了文档为何在每次查询时临时打开、stdio 主机为何使用共享的文件系统/子进程执行环境,以及扩展名归属为何在同一运行时内互斥。 -子系统参考——操作、坐标、请求/结果、`LspError`——见 [docs/subsystems/lsp.md](../../docs/subsystems/lsp.md);设计依据见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)。 +子系统参考——操作、坐标、请求/结果、`LspError`——见 [docs/subsystems/lsp.md](../../docs/subsystems/lsp.zh.md);设计依据见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md)。 diff --git a/packages/lsp/lsp-stdio/package.json b/packages/lsp/lsp-stdio/package.json index 04088296fb..3dca6f9433 100644 --- a/packages/lsp/lsp-stdio/package.json +++ b/packages/lsp/lsp-stdio/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-lsp-stdio", "description": "Generic stdio language-server provider for the DeepSeek Harness LSP capability seam (ctx.lsp) — spawns configured servers, translates JSON-RPC, and serves transient-open goToDefinition/findReferences/goToImplementation/hover queries in the host filesystem namespace", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/lsp/lsp-stdio/tests/fixture-server.ts b/packages/lsp/lsp-stdio/tests/fixture-server.ts index 428abe0268..e6b3800916 100644 --- a/packages/lsp/lsp-stdio/tests/fixture-server.ts +++ b/packages/lsp/lsp-stdio/tests/fixture-server.ts @@ -200,7 +200,6 @@ function send(message: Record): void { process.stdout.write(Buffer.concat([Buffer.from(`Content-Length: ${body.length}\r\n\r\n`, 'ascii'), body])) } -// Keep the event loop alive. process.stdin.resume() if (pauseStdinAfterInitialized) { setInterval(() => {}, 1000) diff --git a/packages/lsp/lsp/README.i18n.yaml b/packages/lsp/lsp/README.i18n.yaml index 1beb022f57..0fb8bcab4e 100644 --- a/packages/lsp/lsp/README.i18n.yaml +++ b/packages/lsp/lsp/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp/README.md README.md: cf32f9cc60a346def48d9720bf0695db36553b0e -README.zh.md: 6f19d1f649eb1483782b6ebb4c82f05da7a42a80 +README.zh.md: eba25a7494f8c624b0e9d0aa5b5c7dd484d1f0f9 diff --git a/packages/lsp/lsp/README.zh.md b/packages/lsp/lsp/README.zh.md index 6f19d1f649..eba25a7494 100644 --- a/packages/lsp/lsp/README.zh.md +++ b/packages/lsp/lsp/README.zh.md @@ -39,6 +39,6 @@ ## 已知限制与暂缓事项 -- **同一运行时内扩展名归属互斥**:两个提供方不能同时声明 `.ts`,即使 language id 不同;重叠会使注册失败。预期扩展是在注册之上增加部署配置的 selector;它可以放宽互斥保留,而无需把提供方选择加入模型输入(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md))。 +- **同一运行时内扩展名归属互斥**:两个提供方不能同时声明 `.ts`,即使 language id 不同;重叠会使注册失败。预期扩展是在注册之上增加部署配置的 selector;它可以放宽互斥保留,而无需把提供方选择加入模型输入(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md))。 - **仅四种操作**:symbol 与 call hierarchy 暂缓(它们需要不同 schema);diagnostics 需要独立的新鲜度/累积规则;修改操作(rename、code action、formatting)需要独立工具,并集成预览、权限和写入策略。 - **没有观测表层**:可用性只能通过运行 `query()` 并按抛出的 `LspError` code 路由来观测;没有提供方变更事件或能力状态查询。 diff --git a/packages/lsp/lsp/package.json b/packages/lsp/lsp/package.json index 4f16df5a2f..249b8249da 100644 --- a/packages/lsp/lsp/package.json +++ b/packages/lsp/lsp/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-lsp", "description": "Abstract LSP capability seam (ctx.lsp) for the DeepSeek Harness — language-server provider registry keyed by branded id and extension mapping, order-independent per-query selection, normalized definition/references/implementation/hover requests and results, and the LspError taxonomy", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/lsp/tool-lsp/README.i18n.yaml b/packages/lsp/tool-lsp/README.i18n.yaml index 2eb5cb2f17..8287d4b40b 100644 --- a/packages/lsp/tool-lsp/README.i18n.yaml +++ b/packages/lsp/tool-lsp/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/tool-lsp/README.md README.md: e0fd5738cf114314a8483125d20142d5f1ffac40 -README.zh.md: 8c908c1e8766ab47e9cefbc9d1f72f05e8c2709f +README.zh.md: 71d6a4b5479b2da187b78a388d617afe7214e78c diff --git a/packages/lsp/tool-lsp/README.zh.md b/packages/lsp/tool-lsp/README.zh.md index 8c908c1e87..71d6a4b547 100644 --- a/packages/lsp/tool-lsp/README.zh.md +++ b/packages/lsp/tool-lsp/README.zh.md @@ -46,7 +46,7 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu #### 模型看到的内容 -模型会看到生成的 [`lsp` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-lsp)。 +模型会看到生成的 [`lsp` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-lsp)。 #### Token 影响 @@ -86,5 +86,5 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu ## 已知限制与暂缓事项 -- **UTF-16 光标坐标**:列坐标与协议精确一致,但模型难以在非 BMP 字符周围计数;未落在符号上的位置可能返回空结果,因此提示词解释了该约定,但不鼓励广泛使用 LSP(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md))。 +- **UTF-16 光标坐标**:列坐标与协议精确一致,但模型难以在非 BMP 字符周围计数;未落在符号上的位置可能返回空结果,因此提示词解释了该约定,但不鼓励广泛使用 LSP(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md))。 - **不承诺跨服务器完整性**:受支持的服务器仍可能根据索引就绪情况返回空或部分结果;该工具不承诺跨语言或服务器的完整性。 diff --git a/packages/lsp/tool-lsp/package.json b/packages/lsp/tool-lsp/package.json index bd4bda1388..de95b13daa 100644 --- a/packages/lsp/tool-lsp/package.json +++ b/packages/lsp/tool-lsp/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-lsp", "description": "Model-facing lsp tool over the DeepSeek Harness LSP capability seam (ctx.lsp) — one read-only tool with goToDefinition/findReferences/goToImplementation/hover operations, one-based UTF-16 cursor coordinates, bounded location rendering, and hover normalization", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/lsp/tool-lsp/src/session-cwd.ts b/packages/lsp/tool-lsp/src/session-cwd.ts index 7fc41785de..00453843e2 100644 --- a/packages/lsp/tool-lsp/src/session-cwd.ts +++ b/packages/lsp/tool-lsp/src/session-cwd.ts @@ -1,9 +1,7 @@ /** - * Derive the workspace root an `lsp` call resolves against: the calling agent's per-session - * workspace (`exec.agent.session.header.cwd`), mirroring how the filesystem tools resolve paths. - * Unlike those tools, LSP has NO provider fallback — a missing cwd fails the call as - * `LSP_WORKSPACE_REQUIRED`, because the local provider must canonicalize a real workspace before it - * can start a server. + * Derive the workspace root an `lsp` call resolves against from the calling + * agent's session. A missing cwd fails as `LSP_WORKSPACE_REQUIRED` because the + * local provider must canonicalize a real workspace before starting a server. * @module @deepseek-ai/dsh-tool-lsp/session-cwd */ diff --git a/packages/mcp/README.i18n.yaml b/packages/mcp/README.i18n.yaml index 3ce6f13ac3..93fa4aa4aa 100644 --- a/packages/mcp/README.i18n.yaml +++ b/packages/mcp/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/mcp/README.md README.md: cc3440f6cb2e507277e6141393ccf39fc7022d7d -README.zh.md: 0a74f3e9016be95f76104303396dc690888d645d +README.zh.md: 9cdc01b80dce63e06bb36fe92cfaf62ed5adce51 diff --git a/packages/mcp/README.zh.md b/packages/mcp/README.zh.md index 0a74f3e901..9cdc01b80d 100644 --- a/packages/mcp/README.zh.md +++ b/packages/mcp/README.zh.md @@ -6,4 +6,4 @@ | 包 | 职责 | |---|---| -| [`mcp-client/`](mcp-client/README.md) | MCP 客户端桥接,将外部服务器工具注册到 `ctx.tools` | +| [`mcp-client/`](mcp-client/README.zh.md) | MCP 客户端桥接,将外部服务器工具注册到 `ctx.tools` | diff --git a/packages/mcp/mcp-client/README.i18n.yaml b/packages/mcp/mcp-client/README.i18n.yaml index 67b937e0e2..b97f1b92b0 100644 --- a/packages/mcp/mcp-client/README.i18n.yaml +++ b/packages/mcp/mcp-client/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/mcp/mcp-client/README.md -README.md: 266c3b7c2b38406800ae5dad1eb065c9dcbf50e6 -README.zh.md: 1b5b5c523e0a477db30f97a748651dbe7e6992ea +README.md: 46ea451e2e28513981a96470fe93506dd3298d96 +README.zh.md: ab72bfa1649efac17c0a56c2d18941cbb6c6f925 diff --git a/packages/mcp/mcp-client/README.md b/packages/mcp/mcp-client/README.md index 266c3b7c2b..46ea451e2e 100644 --- a/packages/mcp/mcp-client/README.md +++ b/packages/mcp/mcp-client/README.md @@ -36,7 +36,7 @@ The model sees `mcp__github__create_issue`, `mcp__web__search`, … — the same | Field | Transport | Required | Description | |---|---|---|---| | `transport` | both | yes | `"stdio"` or `"streamable-http"` | -| `serverName` | both | yes | Namespace for this server's model-facing tool names; `[A-Za-z0-9_-]{1,32}`, unique across live instances | +| `serverName` | both | yes | Namespace for this server's model-facing tool names; `[A-Za-z0-9_-]{1,32}`, unique inside one registration scope | | `command` | stdio | yes | Executable to spawn | | `args` | stdio | no | Arguments passed to the command | | `env` | stdio | no | Extra env vars merged on top of scrubbed ambient env | @@ -55,7 +55,7 @@ The model sees `mcp__github__create_issue`, `mcp__web__search`, … — the same Every MCP tool has two names: the raw MCP name (sent on the wire in `tools/call`) and the public name `mcp____` registered on `ctx.tools`. Public names are normalized to the DeepSeek function-name contract (64 chars, `[A-Za-z0-9_-]`); when replacement or truncation changes the name, a deterministic 12-hex-char hash of `(serverName, rawName)` is appended so distinct tools never collapse into one name. Names are pure functions of `(serverName, rawName)` — connection order, re-syncs, and other servers never rename a tool. - Two servers publishing the same raw name (e.g. `search`) coexist under their namespaces. -- A duplicate `serverName` across live instances fails the later plugin instance at load. +- A duplicate `serverName` inside one live registration scope fails the later plugin instance at load. Independent Agent scopes may reuse the same namespace because their tools and transports are isolated. - A server listing the same tool name twice is rejected as an invalid tool list. - A foreign registration squatting on this server's namespace rolls back the whole generation (never a partial set), with a loud error. @@ -65,7 +65,7 @@ Every MCP tool has two names: the raw MCP name (sent on the wire in `tools/call` - Listens for `notifications/tools/list_changed` → re-syncs; a fetch-phase failure keeps the previous generation registered, while a registration conflict rolls back the attempted generation and leaves no tools from that server. - Tool execute: `client.callTool({ name: rawName, arguments }, { signal })` with timeout + abort support—the public name is never sent to the server. - Canonical success is `{ content: JsonValue[], structuredContent? }`; complete JSON MCP blocks survive for programmatic callers. A supported advertised `outputSchema` validates `structuredContent`; unsupported schema vocabulary falls back to unconstrained `JsonValue`. -- Native/model rendering keeps the existing text projection: text blocks join with newlines while image, audio, resource, and unsupported blocks become placeholders. +- Native/model rendering preserves MCP block order. Text-like runs join with newlines; resource links keep their name and URI as text; supported images become durable core image blocks only when `ctx.attachments` is mounted and the exact calling model route explicitly declares image input. The whole image batch is decoded and admitted before any member is saved. A malformed/refused image batch, audio, embedded resources, and unsupported blocks become explicit diagnostic text rather than disappearing. - On disconnect/crash: the supervisor restarts the original server config with exponential backoff (`reconnect.initialDelayMs` doubling up to `reconnect.maxDelayMs`) and re-runs discovery on success — the recovered generation replaces the previous one, so tools neither duplicate nor leak. During the outage the last good generation stays registered; calls against it fail until recovery. - Reconnection is budgeted per outage: after `reconnect.maxAttempts` consecutive failures the server's tools are unregistered and reconnection stops until an HMR reload or Host restart. A connection that survives past `maxDelayMs` resets the budget, so an occasionally-crashing server recovers indefinitely while a crash-looping one — even with briefly successful connects — still exhausts the cap instead of restarting forever. - Reconnect states are user-visible in logs: reconnecting (warn, with attempt count and delay), recovered (info), final failure and disabled-loss (error). Disposal cancels any pending reconnect. With `reconnect.enabled: false`, a lost connection keeps tools registered but failing until a reload — the manual-recovery behavior. @@ -75,6 +75,8 @@ Every MCP tool has two names: the raw MCP name (sent on the wire in `tools/call` | Service | Usage | |---|---| | `ctx.tools` | Register/unregister MCP tools | +| `ctx.attachments` | Optionally validate and persist image result batches before model projection | +| `ctx.llm` | Optionally prove the exact calling route explicitly supports image input | ## Model Experience @@ -96,11 +98,11 @@ Prefix-stable while the discovered tool set and schemas are unchanged. A re-sync #### What the model sees -The public tool name and JSON arguments remain in assistant history. Text result blocks are joined with newlines into one retained Native text result; image, audio, resource, and unsupported blocks become short placeholders there. Their full JSON blocks and optional structured content remain in the execution-local canonical value, and MCP `isError` rejects the call through the registry's error path. +The public tool name and JSON arguments remain in assistant history. The execution-local canonical value always retains the complete JSON MCP blocks and optional structured content for programmatic and Code Mode callers. In Native context, supported image blocks are durably projected beside text in their original order after exact route-capability proof; Code Mode additionally ferries that settled rich projection through the outer `run_code` result without changing the canonical binding value. Refused images, audio, embedded resources, resource links, and unknown blocks remain visible as bounded text diagnostics, and MCP `isError` rejects the call before image persistence. #### Token effect -Arguments and mapped text are retained until compaction. Binary and resource payloads are discarded rather than added to context. +Arguments, mapped text, and durable image references are retained until compaction. Inline MCP base64 stays only in the execution-local canonical value and is never copied into a session event; the provider reads verified bytes from the attachment store. Audio and embedded-resource payloads stay out of model context. #### KV Cache effect @@ -111,5 +113,5 @@ Append-only; newly visible content follows the reusable request prefix and does - **Tools are the only bridged MCP capability** — Resources and Prompts have no harness consumer and are deferred. - **Startup timeout is inherited from the MCP SDK** — DSH does not yet expose a connection/discovery timeout. Each initialize or paginated `tools/list` request uses the SDK's 60-second default, so an unresponsive server or cursor chain can delay both activation and teardown while the initial synchronization settles. - **Reconnect triggers on transport close** — a crashed stdio child fires it; Streamable HTTP failures surface per request and through the SDK transport's own SSE-stream recovery, so an unreachable HTTP server is retried per call rather than respawned by the supervisor. -- **Native non-text rendering is lossy** — image, audio, and resource payloads become placeholders in model context even though the execution-local canonical value preserves their JSON blocks. Richer Native multimedia projection is deferred. +- **Image is the only durable rich-result bridge** — PNG, JPEG, WebP, and GIF can enter Native context after exact capability proof. Audio and embedded-resource payloads remain execution-local with explicit diagnostics, while resource links preserve only their name and URI as text. - **Unsupported MCP output schemas are not enforced** — `structuredContent` falls back to `JsonValue` when the advertised schema uses vocabulary outside the harness subset. diff --git a/packages/mcp/mcp-client/README.zh.md b/packages/mcp/mcp-client/README.zh.md index 1b5b5c523e..ab72bfa164 100644 --- a/packages/mcp/mcp-client/README.zh.md +++ b/packages/mcp/mcp-client/README.zh.md @@ -36,7 +36,7 @@ MCP 客户端桥接插件:连接外部 [Model Context Protocol](https://modelc | 字段 | 传输 | 必填 | 描述 | |---|---|---|---| | `transport` | 两者 | 是 | `"stdio"` 或 `"streamable-http"` | -| `serverName` | 两者 | 是 | 该服务器面向模型工具名称的 namespace;`[A-Za-z0-9_-]{1,32}`,在存活实例中唯一 | +| `serverName` | 两者 | 是 | 该服务器面向模型工具名称的 namespace;`[A-Za-z0-9_-]{1,32}`,在单个 registration scope 内唯一 | | `command` | stdio | 是 | 要 spawn 的可执行文件 | | `args` | stdio | 否 | 传给命令的参数 | | `env` | stdio | 否 | 合并到已清理环境中的额外环境变量 | @@ -55,7 +55,7 @@ MCP 客户端桥接插件:连接外部 [Model Context Protocol](https://modelc 每个 MCP 工具都有两个名称:通过 `tools/call` 在协议上传送的原始 MCP 名称,以及公开名称 `mcp____`,后者注册到 `ctx.tools`。公开名称会规范化为 DeepSeek 函数名称约定(64 个字符、`[A-Za-z0-9_-]`);如果替换或截断改变名称,就会追加 `(serverName, rawName)` 的确定性 12 位十六进制 hash,确保不同工具绝不会折叠为同一个名称。名称是 `(serverName, rawName)` 的纯函数:连接顺序、重新同步和其他服务器永远不会重命名工具。 - 发布相同原始名称(例如 `search`)的两个服务器会在各自 namespace 下共存。 -- 存活实例中的重复 `serverName` 会使后加载的插件实例失败。 +- 同一个存活 registration scope 内重复的 `serverName` 会使后加载的插件实例失败。独立 Agent scope 的工具和传输彼此隔离,因此可以复用同一 namespace。 - 服务器在工具列表中两次列出同一工具名称时,该列表会作为无效工具列表被拒绝。 - 外部注册抢占该服务器 namespace 时,会回滚整个世代(绝不保留部分集合),并明确报错。 @@ -65,7 +65,7 @@ MCP 客户端桥接插件:连接外部 [Model Context Protocol](https://modelc - 监听 `notifications/tools/list_changed` → 重新同步;获取阶段失败时保留上一世代的注册,注册冲突则会回滚本次尝试的世代,并且不保留该服务器的任何工具。 - 工具执行:`client.callTool({ name: rawName, arguments }, { signal })`,支持超时 + 中止;公开名称绝不会发给服务器。 - 规范成功值是 `{ content: JsonValue[], structuredContent? }`;完整的 JSON MCP 块会保留给编程调用方。受支持且已声明的 `outputSchema` 会验证 `structuredContent`;不受支持的 schema 词汇会回退为不受约束的 `JsonValue`。 -- Native/模型渲染保留现有文本投影:文本块以换行连接,图片、音频、资源和不受支持的块会变成占位符。 +- Native/模型渲染会保留 MCP 块顺序。文本类连续块以换行连接;资源链接以文本保留名称和 URI;只有挂载 `ctx.attachments` 且确切调用模型路由明确声明支持图片输入时,受支持的图片才会成为持久核心图片块。整个图片批次会先完成解码与准入,再保存任一成员。格式错误或被拒绝的图片批次、音频、嵌入资源和不受支持的块会成为明确诊断文本,而不会消失。 - 断开/崩溃时:supervisor 以指数退避(`reconnect.initialDelayMs` 逐次翻倍,上限 `reconnect.maxDelayMs`)重启原始服务器配置,成功后重新执行发现——恢复的世代会替换前一个,因此工具既不会重复也不会泄漏。中断期间最后一个正常世代保持注册;针对它的调用在恢复前会失败。 - 重连按中断预算控制:连续失败达到 `reconnect.maxAttempts` 次后,该服务器的工具会被注销,重连停止,直到 HMR 重载或重启 Host。连接存活超过 `maxDelayMs` 会重置预算,因此偶尔崩溃的服务器可以无限恢复,而崩溃循环的服务器——即使短暂连接成功——仍会耗尽上限而非永远重启。 - 重连状态在日志中对用户可见:reconnecting(warn,含尝试次数和延迟)、recovered(info)、最终失败和 disabled-loss(error)。dispose(资源释放)会取消任何待执行的重连。设置 `reconnect.enabled: false` 时,连接丢失后工具保持注册但调用失败,直到重载——即手动恢复行为。 @@ -75,6 +75,8 @@ MCP 客户端桥接插件:连接外部 [Model Context Protocol](https://modelc | 服务 | 用途 | |---|---| | `ctx.tools` | 注册/注销 MCP 工具 | +| `ctx.attachments` | 可选;在模型投影前校验并持久保存图片结果批次 | +| `ctx.llm` | 可选;证明确切调用路由明确支持图片输入 | ## 模型体验 @@ -96,11 +98,11 @@ MCP 客户端桥接插件:连接外部 [Model Context Protocol](https://modelc #### 模型看到的内容 -公开工具名称和 JSON 参数会保留在 assistant 历史中。文本结果块会以换行连接为一个保留的 Native 文本结果;图片、音频、资源和不受支持的块在其中变为简短占位符。它们的完整 JSON 块及可选结构化内容保留在执行局部的规范值中;MCP `isError` 会通过注册表的错误路径拒绝调用。 +公开工具名称和 JSON 参数会保留在 assistant 历史中。执行局部的规范值始终为程序化调用方和 Code Mode 保留完整 JSON MCP 块及可选结构化内容。在 Native 上下文中,受支持的图片块会在确切路由能力得到证明后,按原始顺序与文本一起持久投影;Code Mode 还会经外层 `run_code` 结果转运这份已经结算的丰富投影,而不改变规范绑定值。被拒绝的图片、音频、嵌入资源、资源链接和未知块会继续以有界文本诊断可见;MCP `isError` 会在持久化图片前拒绝调用。 #### Token 影响 -参数和映射后的文本会保留到压缩(compaction)发生时。二进制与资源载荷会被丢弃,而不会加入上下文。 +参数、映射后的文本和持久图片引用会保留到压缩(compaction)发生时。内联 MCP base64 只存在于执行局部的规范值中,绝不会复制进会话事件;提供方会从附件存储读取经过校验的字节。音频和嵌入资源载荷仍不会进入模型上下文。 #### KV Cache 影响 @@ -111,5 +113,5 @@ MCP 客户端桥接插件:连接外部 [Model Context Protocol](https://modelc - **只桥接 MCP 的工具能力**:资源和提示词没有 harness 消费接口,暂缓实现。 - **启动超时继承自 MCP SDK**:DSH 尚未公开连接/发现超时。每次 initialize 请求或分页 `tools/list` 请求都使用 SDK 默认的 60 秒,因此在初始同步完成期间,无响应的 server 或 cursor chain 可能同时延迟激活与 teardown。 - **重连在传输关闭时触发**:崩溃的 stdio 子进程会触发重连;Streamable HTTP 失败通过每次请求以及 SDK 传输自身的 SSE(Server-Sent Events)流恢复机制暴露,因此不可达的 HTTP 服务器会按调用重试,而非由 supervisor 重新 spawn。 -- **Native 非文本渲染有损**:图片、音频与资源载荷在模型上下文中会变成占位符,即使执行局部的规范值保留了其 JSON 块。更丰富的 Native 多媒体投影暂缓实现。 +- **图片是唯一的持久丰富结果桥接**:PNG、JPEG、WebP 和 GIF 可以在确切能力得到证明后进入 Native 上下文。音频和嵌入资源载荷仍只存在于执行局部,并配有明确诊断;资源链接只以文本保留名称和 URI。 - **不强制执行不受支持的 MCP 输出 schema**:已声明 schema 使用 harness 子集之外的词汇时,`structuredContent` 会回退到 `JsonValue`。 diff --git a/packages/mcp/mcp-client/package.json b/packages/mcp/mcp-client/package.json index d42b425730..37af73f583 100644 --- a/packages/mcp/mcp-client/package.json +++ b/packages/mcp/mcp-client/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-mcp-client", "description": "MCP client bridge: connects to MCP servers and registers their tools on ctx.tools", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -32,8 +32,10 @@ ], "license": "MIT", "peerDependencies": { + "@deepseek-ai/dsh-attachment": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-scope": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-tools": "workspace:^", @@ -45,8 +47,11 @@ "zod": "^4.4.3" }, "devDependencies": { + "@deepseek-ai/dsh-attachment": "workspace:^", + "@deepseek-ai/dsh-attachment-local": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-scope": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-tools": "workspace:^", diff --git a/packages/mcp/mcp-client/src/index.ts b/packages/mcp/mcp-client/src/index.ts index 5a2a52f55d..59ef928751 100644 --- a/packages/mcp/mcp-client/src/index.ts +++ b/packages/mcp/mcp-client/src/index.ts @@ -15,6 +15,7 @@ import type { Context } from '@deepseek-ai/cordis' import z from '@deepseek-ai/schemastery' +import { scopeOf } from '@deepseek-ai/dsh-scope' import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { RECONNECT_DEFAULTS, resolveReconnectPolicy, startConnection } from './connection.ts' import type { ReconnectConfig } from './connection.ts' @@ -37,12 +38,11 @@ const DEFAULT_TOOL_CALL_TIMEOUT_MS = 60_000 const SERVER_NAME_PATTERN = /^[A-Za-z0-9_-]{1,32}$/ /** - * Live `serverName` reservations per app, keyed off `ctx.root` (multiple apps - * in one process — tests — must not see each other's names). A duplicate - * namespace is a configuration error surfaced at plugin load, never silent - * shadowing. + * Live `serverName` reservations per registration scope. Agent-scoped MCP + * servers may reuse a namespace in another Agent, while global instances and + * duplicates inside one Agent remain mutually exclusive. */ -const activeServerNames = new WeakMap>() +const activeServerNames = new WeakMap>() // ---- Config ---- @@ -97,6 +97,12 @@ export interface StreamableHttpConfig { /** Configuration for one stdio or Streamable HTTP MCP server. */ export type Config = StdioConfig | StreamableHttpConfig +type StdioConfigInput = Omit + & Partial> +type StreamableHttpConfigInput = Omit + & Partial> +type ConfigInput = StdioConfigInput | StreamableHttpConfigInput + const Reconnect: z = z.object({ enabled: z.boolean().default(RECONNECT_DEFAULTS.enabled), initialDelayMs: z.number().min(1).max(MAX_TIMER_DELAY_MS).default(RECONNECT_DEFAULTS.initialDelayMs), @@ -125,7 +131,7 @@ export const Config = z.union([ failOnStartupError: z.boolean().default(false), reconnect: Reconnect, }), -]) as unknown as z +]) as unknown as z // ---- Plugin apply ---- @@ -146,10 +152,11 @@ export async function apply(ctx: Context, config: Config): Promise { // Reserve the namespace next: a duplicate `serverName` fails THIS instance // at load with an actionable error and leaves the earlier instance intact. ctx.effect(() => { - let names = activeServerNames.get(ctx.root) + const owner = scopeOf(ctx) ?? ctx.root + let names = activeServerNames.get(owner) if (!names) { names = new Set() - activeServerNames.set(ctx.root, names) + activeServerNames.set(owner, names) } if (names.has(config.serverName)) { throw new Error( diff --git a/packages/mcp/mcp-client/src/tools.ts b/packages/mcp/mcp-client/src/tools.ts index 52f2a73738..8c4780f71d 100644 --- a/packages/mcp/mcp-client/src/tools.ts +++ b/packages/mcp/mcp-client/src/tools.ts @@ -13,11 +13,15 @@ */ import { createHash } from 'node:crypto' +import { isDeepStrictEqual } from 'node:util' import type { Client } from '@modelcontextprotocol/sdk/client/index.js' import { ListToolsResultSchema } from '@modelcontextprotocol/sdk/types.js' import { z } from 'zod' import type { Context } from '@deepseek-ai/cordis' -import type { ToolDefinition, ToolExecution } from '@deepseek-ai/dsh-tools' +import { isImageAdmissionError } from '@deepseek-ai/dsh-attachment' +import type { AttachmentStore, ImageAttachmentRef, ImageMediaType, SaveImageAttachment } from '@deepseek-ai/dsh-attachment' +import type { ContentBlock } from '@deepseek-ai/dsh-llm' +import type { ToolDefinition, ToolExecution, ToolExecutionResult } from '@deepseek-ai/dsh-tools' import { assertSupportedJsonSchema } from '@deepseek-ai/dsh-tools' import type { JsonSchemaNode, JsonValue } from '@deepseek-ai/dsh-tools' @@ -53,6 +57,17 @@ const HASH_LENGTH = 12 /** Raw result record: the bridge owns JSON-value validation after transport. */ const RawCallToolResultSchema = z.record(z.string(), z.unknown()) +/** Raster formats supported by the durable attachment vocabulary. */ +const IMAGE_MEDIA_TYPES: readonly ImageMediaType[] = [ + 'image/png', + 'image/jpeg', + 'image/webp', + 'image/gif', +] + +/** Canonical RFC 4648 base64, excluding whitespace and URL-safe aliases. */ +const CANONICAL_BASE64 = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/ + /** List without mutating the SDK's per-page output-validator cache. */ function listToolsUncached(client: Client, cursor?: string) { return client.request( @@ -143,13 +158,17 @@ export async function syncTools( `mcp-client(${opts.serverName}): server listed tool "${tool.name}" more than once — invalid tool list`, ) } - definitions.set(publicName, { - name: publicName, - description: tool.description ?? '', - parameters: tool.inputSchema, - output: createOutput(tool.name, supportedOutputSchema(tool.outputSchema)), - execute: createExecutor(client, tool.name, tool.execution?.taskSupport === 'required', opts), - }) + definitions.set(publicName, createDefinition( + client, + ctx, + publicName, + tool.name, + tool.description ?? '', + tool.inputSchema, + supportedOutputSchema(tool.outputSchema), + tool.execution?.taskSupport === 'required', + opts, + )) } cursor = response.nextCursor } while (cursor) @@ -183,6 +202,19 @@ interface McpContentBlock { type: string text?: string mimeType?: string + data?: string + name?: string + uri?: string +} + +/** Async rich projection staged for one exact ToolRuntime execution. */ +interface PreparedProjection { + /** Canonical MCP value returned by execute before registry materialization. */ + value: McpResult + /** Synchronous output.render projection expected before finalization. */ + fallback: ContentBlock[] + /** Image-enriched or explicit-refusal projection prepared during execute. */ + content: ContentBlock[] } /** Keep a supported advertised schema; unsupported MCP vocabulary falls back to JsonValue. */ @@ -196,6 +228,49 @@ function supportedOutputSchema(candidate: unknown): JsonSchemaNode | undefined { } } +/** + * Build one generation-local tool definition and its execution-local rich projections. + * @param client - connected MCP client used for calls. + * @param ctx - plugin context carrying optional attachment and model services. + * @param publicName - registry-qualified public tool name. + * @param rawName - MCP wire tool name. + * @param description - model-facing tool description. + * @param parameters - MCP input schema. + * @param structuredSchema - supported structured-output schema, when advertised. + * @param taskRequired - whether this MCP tool requires unsupported task execution. + * @param opts - bridge timeout and namespace options. + * @returns a complete ToolRuntime definition. + */ +function createDefinition( + client: Client, + ctx: Context, + publicName: string, + rawName: string, + description: string, + parameters: Record, + structuredSchema: JsonSchemaNode | undefined, + taskRequired: boolean, + opts: ToolBridgeOptions, +): ToolDefinition { + const projections = new WeakMap() + return { + name: publicName, + description, + parameters, + output: createOutput(rawName, structuredSchema), + execute: createExecutor(client, ctx, rawName, taskRequired, opts, projections), + finalizeContent(exec: Readonly, result: Readonly) { + const projection = projections.get(exec) + if (projection === undefined) return undefined + projections.delete(exec) + if (result.isError) return undefined + if (!isDeepStrictEqual(result.value, projection.value)) return undefined + if (!isDeepStrictEqual(result.content, projection.fallback)) return undefined + return projection.content + }, + } +} + /** Build the canonical result schema and existing Native text projection. */ function createOutput(rawName: string, structuredSchema: JsonSchemaNode | undefined): ToolDefinition['output'] { return { @@ -208,7 +283,7 @@ function createOutput(rawName: string, structuredSchema: JsonSchemaNode | undefi required: structuredSchema === undefined ? ['content'] : ['content', 'structuredContent'], additionalProperties: false, }, - render(_args, value) { + render(_args: unknown, value: JsonValue) { const result = value as unknown as McpResult return [{ type: 'text', text: extractText(result.content, rawName) }] }, @@ -227,9 +302,11 @@ function createOutput(rawName: string, structuredSchema: JsonSchemaNode | undefi */ function createExecutor( client: Client, + ctx: Context, rawName: string, taskRequired: boolean, opts: ToolBridgeOptions, + projections: WeakMap, ): ToolDefinition['execute'] { return async (args: unknown, exec: ToolExecution) => { if (taskRequired) { @@ -268,12 +345,144 @@ function createExecutor( throw new Error(text) } - return { + const value: McpResult = { content, ...result.structuredContent !== undefined ? { structuredContent: result.structuredContent as JsonValue } : {}, } + if (containsImage(content)) { + const fallback: ContentBlock[] = [{ type: 'text', text: extractText(content, rawName) }] + const projected = await prepareImageProjection(ctx, exec, content, rawName) + projections.set(exec, { value, fallback, content: projected }) + } + return value + } +} + +/** Whether an untrusted MCP content array contains a declared image block. */ +function containsImage(content: JsonValue[]): boolean { + return content.some(value => isRecord(value) && value.type === 'image') +} + +/** Narrow one JSON value to a string-keyed object. */ +function isRecord(value: JsonValue): value is { [key: string]: JsonValue } { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +/** Narrow a declared MIME string to the durable image vocabulary. */ +function isImageMediaType(value: string): value is ImageMediaType { + return IMAGE_MEDIA_TYPES.includes(value as ImageMediaType) +} + +/** Decode one untrusted MCP image block without accepting base64 aliases. */ +function decodeImage(block: McpContentBlock): SaveImageAttachment { + if (block.mimeType === undefined || !isImageMediaType(block.mimeType)) { + throw new Error('the declared media type is not PNG, JPEG, WebP, or GIF') + } + if (block.data === undefined || !CANONICAL_BASE64.test(block.data)) { + throw new Error('the image data is not canonical base64') + } + const data = Buffer.from(block.data, 'base64') + if (data.toString('base64') !== block.data) { + throw new Error('the image data is not canonical base64') + } + return { data, mediaType: block.mimeType } +} + +/** + * Resolve the active model route and durable store for an image-bearing result. + * @param ctx - plugin context with optional services. + * @param exec - exact tool execution whose agent supplies the latest route. + * @returns the attachment store after exact positive image-capability proof. + */ +async function resolveImageAdmission(ctx: Context, exec: ToolExecution): Promise { + const attachments = ctx.get('attachments') + if (attachments === undefined) throw new Error('no attachment store is mounted') + const routed = exec.agent?.session.requestHeader()?.config + const provider = routed?.provider ?? exec.agent?.options.provider + const model = routed?.model ?? exec.agent?.options.model + const llm = ctx.get('llm') + if (provider === undefined || model === undefined || llm === undefined) { + throw new Error('the current model route could not be resolved') + } + let info: Awaited> + try { + info = await llm.resolveModelInfo(provider, model, exec.signal) + } catch { + throw new Error('the current model route could not be verified') + } + if (info.inputModalities === undefined || !info.inputModalities.includes('image')) { + throw new Error(`model "${model}" does not declare image input`) + } + if (exec.signal.aborted) throw new Error('the tool call was canceled before image storage') + return attachments +} + +/** Stable diagnostic text for an image block that was not admitted. */ +function imageDiagnostic(block: McpContentBlock, reason: string): string { + const mediaType = block.mimeType ?? 'unknown media type' + return `[image unavailable: ${mediaType}; ${reason}; raw image data remains available to programmatic callers]` +} + +/** + * Decode, preflight, and durably save one MCP result's ordered image batch. + * Any refusal projects every image as text while retaining the canonical raw + * value for programmatic callers. + */ +async function prepareImageProjection( + ctx: Context, + exec: ToolExecution, + content: JsonValue[], + toolName: string, +): Promise { + const decoded: SaveImageAttachment[] = [] + const validationErrors = new Map() + const imageIndexes: number[] = [] + for (const [index, value] of content.entries()) { + if (!isRecord(value) || value.type !== 'image') continue + imageIndexes.push(index) + try { + decoded.push(decodeImage(value as unknown as McpContentBlock)) + } catch (error: unknown) { + // decodeImage owns every throw above and always produces Error. + validationErrors.set(index, (error as Error).message) + } + } + if (validationErrors.size > 0) { + return projectContent(content, toolName, (block, index) => ({ + type: 'text', + text: imageDiagnostic( + block, + validationErrors.get(index) ?? 'another image in the same result was invalid', + ), + })) + } + + let attachments: AttachmentStore + try { + attachments = await resolveImageAdmission(ctx, exec) + } catch (error: unknown) { + // resolveImageAdmission contains provider failures and throws Error only. + const reason = (error as Error).message + return projectContent(content, toolName, block => ({ type: 'text', text: imageDiagnostic(block, reason) })) + } + + try { + const refs = await attachments.saveImages(decoded) + const byIndex = new Map(imageIndexes.map((index, offset) => [index, refs[offset] as ImageAttachmentRef] as const)) + return projectContent(content, toolName, (_block, index) => ({ + type: 'image', + attachment: byIndex.get(index) as ImageAttachmentRef, + })) + } catch (error: unknown) { + const reason = isImageAdmissionError(error) + ? `image admission rejected the result: ${error.message}` + : 'durable image storage rejected the result' + return projectContent(content, toolName, block => ({ + type: 'text', + text: imageDiagnostic(block, reason), + })) } } @@ -286,32 +495,65 @@ function createExecutor( * guarded with fallbacks because this is a network trust boundary. */ function extractText(mcpContent: JsonValue[], toolName: string): string { - const parts: string[] = [] + const content = projectContent(mcpContent, toolName) + // The default image projector below also returns text, so this local call + // cannot produce a core image block. + return content.map(block => (block as Extract).text).join('\n') +} - for (const value of mcpContent) { - if (typeof value !== 'object' || value === null || Array.isArray(value)) { - parts.push('[unsupported content type: unknown]') +/** + * Project ordered MCP blocks into the core content vocabulary. + * Text-like runs are newline-coalesced; admitted images split those runs at + * their original position. + */ +function projectContent( + mcpContent: JsonValue[], + toolName: string, + image: (block: McpContentBlock, index: number) => ContentBlock = block => ({ + type: 'text', + text: imageDiagnostic(block, 'this result was not admitted to durable model context'), + }), +): ContentBlock[] { + const projected: ContentBlock[] = [] + const text: string[] = [] + const flushText = (): void => { + if (text.length === 0) return + projected.push({ type: 'text', text: text.splice(0).join('\n') }) + } + + for (const [index, value] of mcpContent.entries()) { + if (!isRecord(value)) { + text.push('[unsupported MCP content block: expected an object]') continue } const block = value as unknown as McpContentBlock switch (block.type) { case 'text': - if (block.text !== undefined) parts.push(block.text) + if (block.text !== undefined) text.push(block.text) break case 'image': - parts.push(`[image: ${block.mimeType ?? 'unknown'}, content discarded]`) + flushText() + projected.push(image(block, index)) + break + case 'resource_link': + if (block.name === undefined || block.uri === undefined) { + text.push('[resource link unavailable: the MCP block is missing its name or URI]') + } else { + text.push(`Resource link: ${block.name} (${block.uri})`) + } break case 'audio': - parts.push(`[audio: ${block.mimeType ?? 'unknown'}, content discarded]`) + text.push(`[audio result unsupported: ${block.mimeType ?? 'unknown media type'}; raw audio data remains available to programmatic callers]`) break case 'resource': - case 'resource_link': - parts.push('[resource: content discarded]') + text.push('[embedded resource unsupported; raw resource data remains available to programmatic callers]') break default: - parts.push(`[unsupported content type: ${block.type}]`) + text.push(`[unsupported MCP content type: ${block.type}]`) } } - - return parts.join('\n') || `(${toolName} returned no text content)` + flushText() + return projected.length > 0 + ? projected + : [{ type: 'text', text: `(${toolName} returned no model-visible content)` }] } diff --git a/packages/mcp/mcp-client/tests/apply.spec.ts b/packages/mcp/mcp-client/tests/apply.spec.ts index 9cda54aeab..1b1716e51b 100644 --- a/packages/mcp/mcp-client/tests/apply.spec.ts +++ b/packages/mcp/mcp-client/tests/apply.spec.ts @@ -6,6 +6,7 @@ import { describe, expect, it, vi, beforeEach } from 'vitest' import { Context } from '@deepseek-ai/cordis' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRuntime from '@deepseek-ai/dsh-tools' +import { createScope } from '@deepseek-ai/dsh-scope' import type { Config } from '@deepseek-ai/dsh-mcp-client' // ---- Mock MCP SDK ---- @@ -208,6 +209,16 @@ describe('apply (plugin lifecycle)', () => { expect(ctx.tools.get('mcp__srv__remote')).toBeDefined() }) + it('allows one serverName in each independent registration scope', async () => { + const first = createScope(ctx, {}) + const second = createScope(ctx, {}) + + await Promise.all([apply(first.ctx, stdioConfig), apply(second.ctx, stdioConfig)]) + + expect(mockConnect).toHaveBeenCalledTimes(2) + await Promise.all([first.dispose(), second.dispose()]) + }) + it('releases the serverName reservation on dispose', async () => { const first = new Context() await first.plugin(SystemPrompt) diff --git a/packages/mcp/mcp-client/tests/fixture-server.ts b/packages/mcp/mcp-client/tests/fixture-server.ts index 974e2a26b0..a2e9b5b5f0 100644 --- a/packages/mcp/mcp-client/tests/fixture-server.ts +++ b/packages/mcp/mcp-client/tests/fixture-server.ts @@ -46,7 +46,7 @@ server.registerTool('image', { }, async () => ({ content: [ { type: 'text', text: 'Here is an image:' }, - { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }, + { type: 'image', data: 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC', mimeType: 'image/png' }, { type: 'text', text: 'End of image.' }, ], })) diff --git a/packages/mcp/mcp-client/tests/http-fixture.ts b/packages/mcp/mcp-client/tests/http-fixture.ts new file mode 100644 index 0000000000..32ef516a0e --- /dev/null +++ b/packages/mcp/mcp-client/tests/http-fixture.ts @@ -0,0 +1,52 @@ +/** Keyless stateless Streamable HTTP MCP fixture for integration tests. */ + +import { createServer, type IncomingMessage, type ServerResponse } from 'node:http' +import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js' +import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js' +import type { Transport } from '@modelcontextprotocol/sdk/shared/transport.js' + +/** Running HTTP fixture and the request headers it observed. */ +export interface HttpMcpFixture { + url: string + authorization: Array + close: () => Promise +} + +/** Start a local stateless MCP endpoint exposing one `ping` tool. */ +export async function startHttpMcpFixture(): Promise { + const authorization: Array = [] + const handleRequest = async (request: IncomingMessage, response: ServerResponse): Promise => { + authorization.push(request.headers.authorization) + const mcp = new McpServer( + { name: 'http-fixture', version: '1.0.0' }, + { capabilities: { tools: {} } }, + ) + mcp.registerTool('ping', { description: 'Replies pong.', inputSchema: {} }, async () => ({ + content: [{ type: 'text', text: 'pong' }], + })) + const transport = new StreamableHTTPServerTransport({}) + response.on('close', () => { + void transport.close() + void mcp.close() + }) + await mcp.connect(transport as Transport) + await transport.handleRequest(request, response) + } + const server = createServer((request, response) => { + handleRequest(request, response).catch((error: unknown) => { + response.writeHead(500).end(String(error)) + }) + }) + const listening: PromiseWithResolvers = Promise.withResolvers() + server.listen(0, '127.0.0.1', listening.resolve) + await listening.promise + const address = server.address() + if (address === null || typeof address === 'string') throw new Error('HTTP MCP fixture has no TCP address') + return { + url: `http://127.0.0.1:${address.port}/mcp`, + authorization, + close: () => new Promise((resolve, reject) => { + server.close((error) => { if (error === undefined) resolve(); else reject(error) }) + }), + } +} diff --git a/packages/mcp/mcp-client/tests/mcp-client.e2e.ts b/packages/mcp/mcp-client/tests/mcp-client.e2e.ts index d25e3eb771..cec7451db2 100644 --- a/packages/mcp/mcp-client/tests/mcp-client.e2e.ts +++ b/packages/mcp/mcp-client/tests/mcp-client.e2e.ts @@ -19,9 +19,11 @@ import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js' import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js' import { z } from 'zod' import type { Transport } from '@modelcontextprotocol/sdk/shared/transport.js' +import LocalAttachmentStore from '@deepseek-ai/dsh-attachment-local' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRuntime from '@deepseek-ai/dsh-tools' -import { CallId } from '@deepseek-ai/dsh-llm' +import { CallId, LlmAdapter, LlmRuntime } from '@deepseek-ai/dsh-llm' +import type { GenerateOptions, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm' import { apply } from '@deepseek-ai/dsh-mcp-client/src/index.ts' import { publicToolName } from '@deepseek-ai/dsh-mcp-client/src/tools.ts' import type { Config } from '@deepseek-ai/dsh-mcp-client' @@ -43,6 +45,33 @@ async function mountRegistry(): Promise { return ctx } +/** Exact-route adapter used to prove real MCP image admission without an API key. */ +class ImageAdapter extends LlmAdapter { + override resolveModel(provider: string, model: string): Promise { + return Promise.resolve({ provider, id: model, name: model, inputModalities: ['text', 'image'] }) + } + + stream(_options: GenerateOptions): AsyncIterable { + throw new Error('MCP image e2e never streams') + } +} + +async function mountImageRegistry(dshHome: string): Promise { + const ctx = await mountRegistry() + await ctx.plugin(LocalAttachmentStore, { dshHome }) + await ctx.plugin(LlmRuntime) + ctx.llm.registerAdapter(['visual'], new ImageAdapter()) + return ctx +} + +/** Calling-agent stand-in pinned to the keyless image-capable route. */ +function imageAgent(): object { + return { + options: { provider: 'visual', model: 'vision' }, + session: { requestHeader: () => undefined }, + } +} + function sleep(ms: number): Promise { const gate: PromiseWithResolvers = Promise.withResolvers() setTimeout(gate.resolve, ms) @@ -66,6 +95,7 @@ function nextCallId(): CallId { describe('fixture server — controlled scenarios', () => { let ctx: Context + let home: string const fixtureConfig: Config = { transport: 'stdio', @@ -79,13 +109,15 @@ describe('fixture server — controlled scenarios', () => { } beforeAll(async () => { - ctx = await mountRegistry() + home = await mkdtemp(join(tmpdir(), 'mcp-image-e2e-')) + ctx = await mountImageRegistry(home) await apply(ctx, fixtureConfig) }, 30_000) afterAll(async () => { if (ctx) await ctx.fiber.dispose() await sleep(200) + await rm(home, { recursive: true, force: true }) }) it('discovers all fixture tools under the server namespace', () => { @@ -141,16 +173,23 @@ describe('fixture server — controlled scenarios', () => { expect(result.content[0]).toMatchObject({ type: 'text' }) }) - it('executes image() → image placeholder', async () => { + it('executes image() → ordered durable image content', async () => { const result = await ctx.tools.execute({ - signal: testToolSignal, + signal: testToolSignal, agent: imageAgent() as never, callId: nextCallId(), name: 'mcp__fixture__image', arguments: {}, }) expect(result.isError).toBe(false) - const text = textOf(result.content[0]) - expect(text).toContain('Here is an image:') - expect(text).toContain('[image: image/png, content discarded]') - expect(text).toContain('End of image.') + expect(result.content).toHaveLength(3) + expect(result.content[0]).toEqual({ type: 'text', text: 'Here is an image:' }) + expect(result.content[2]).toEqual({ type: 'text', text: 'End of image.' }) + const image = result.content[1] + if (image?.type !== 'image') throw new Error(`expected an image block, got ${JSON.stringify(image)}`) + expect(image.attachment).toMatchObject({ mediaType: 'image/png', width: 1, height: 1 }) + const stored = await ctx.attachments.readImage(image.attachment) + expect(stored.data.byteLength).toBe(image.attachment.bytes) + if (result.isError) throw new Error('expected MCP image success') + expect(JSON.stringify(result.value)).toContain('iVBORw0KGgo') + expect(JSON.stringify(result.content)).not.toContain('iVBORw0KGgo') }) }) @@ -334,13 +373,14 @@ describe('server-everything — official test server', () => { expect(textOf(result.content[0])).toContain('10') }) - it('executes get-tiny-image → image placeholder', async () => { + it('executes get-tiny-image → explicit refusal without a durable route', async () => { const result = await ctx.tools.execute({ signal: testToolSignal, callId: nextCallId(), name: 'mcp__everything__get-tiny-image', arguments: {}, }) expect(result.isError).toBe(false) - expect(textOf(result.content[0])).toContain('[image: image/png, content discarded]') + expect(result.content.map(block => block.type === 'text' ? block.text : '').join('\n')) + .toContain('[image unavailable: image/png; no attachment store is mounted;') }) }) diff --git a/packages/mcp/mcp-client/tests/mcp-client.spec.ts b/packages/mcp/mcp-client/tests/mcp-client.spec.ts index 016ebb97bb..b72b8faad4 100644 --- a/packages/mcp/mcp-client/tests/mcp-client.spec.ts +++ b/packages/mcp/mcp-client/tests/mcp-client.spec.ts @@ -2,9 +2,14 @@ import { describe, expect, it, vi, beforeEach } from 'vitest' import { Client } from '@modelcontextprotocol/sdk/client/index.js' import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js' import { Context } from '@deepseek-ai/cordis' -import { CallId } from '@deepseek-ai/dsh-llm' +import AttachmentStore, { AttachmentError, AttachmentId } from '@deepseek-ai/dsh-attachment' +import type { ImageAttachmentLimits, ImageAttachmentRef, SaveImageAttachment, StoredImageAttachment } from '@deepseek-ai/dsh-attachment' +import { CallId, LlmAdapter, LlmRuntime } from '@deepseek-ai/dsh-llm' +import type { ContentBlock } from '@deepseek-ai/dsh-llm' +import type { GenerateOptions, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRuntime, { type JsonValue } from '@deepseek-ai/dsh-tools' +import type { PostToolDecision } from '@deepseek-ai/dsh-tools' import { publicToolName, syncTools, type ToolBridgeOptions } from '@deepseek-ai/dsh-mcp-client/src/tools.ts' import { createTransport } from '@deepseek-ai/dsh-mcp-client/src/transport.ts' import type { Config } from '@deepseek-ai/dsh-mcp-client' @@ -63,6 +68,81 @@ async function mountRegistry(): Promise { return ctx } +const IMAGE_LIMITS: ImageAttachmentLimits = { + maxImageBytes: 1024, + maxImagesPerMessage: 4, + maxMessageImageBytes: 2048, + maxImagePixels: 1024, + maxImageDimension: 2000, + mediaTypes: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'], +} + +/** Attachment fake that records exact decoded batches while using the real batch contract. */ +class RecordingAttachmentStore extends AttachmentStore { + readonly imageLimits = IMAGE_LIMITS + readonly saved: SaveImageAttachment[] = [] + + validateImage(_input: SaveImageAttachment): Promise { + return Promise.resolve() + } + + saveImage(input: SaveImageAttachment): Promise { + this.saved.push(input) + const marker = input.data[0] ?? 0 + const ref: ImageAttachmentRef = { + attachmentId: AttachmentId(`sha256:${marker.toString(16).padStart(64, '0')}`), + mediaType: input.mediaType, + bytes: input.data.byteLength, + width: 1, + height: 1, + } + return Promise.resolve(ref) + } + + readImage(_ref: ImageAttachmentRef): Promise { + throw new Error('not used') + } +} + +/** Exact-route fake used only for image-capability admission. */ +class ImageCatalogAdapter extends LlmAdapter { + override resolveModel(provider: string, model: string): Promise { + return Promise.resolve({ + provider, + id: model, + name: model, + inputModalities: model === 'vision' ? ['text', 'image'] : ['text'], + }) + } + + stream(_options: GenerateOptions): AsyncIterable { + throw new Error('MCP bridge tests never stream') + } +} + +async function mountRichRegistry(): Promise<{ ctx: Context; attachments: RecordingAttachmentStore }> { + const ctx = await mountRegistry() + await ctx.plugin(RecordingAttachmentStore) + await ctx.plugin(LlmRuntime) + ctx.llm.registerAdapter(['visual'], new ImageCatalogAdapter()) + return { ctx, attachments: ctx.attachments as RecordingAttachmentStore } +} + +/** Calling-agent stand-in with no durable request header yet. */ +function agentOn(model: string | undefined = 'vision'): object { + return { + options: model === undefined ? {} : { provider: 'visual', model }, + session: { requestHeader: () => undefined }, + } +} + +/** Require one text block and return its text for diagnostic assertions. */ +function textAt(content: readonly ContentBlock[], index = 0): string { + const block = content[index] + if (block?.type !== 'text') throw new Error(`expected text content at index ${index}`) + return block.text +} + const defaultOpts: ToolBridgeOptions = { registrationFailure: 'contain', serverName: 'srv', @@ -363,24 +443,329 @@ describe('tool execution', () => { expect(result.content).toEqual([{ type: 'text', text: 'line1\nline2' }]) }) - it('preserves full JSON MCP blocks while Native rendering uses placeholders', async () => { + it('preserves canonical MCP JSON while admitting an ordered mixed image result', async () => { + const rich = await mountRichRegistry() const blocks = [ { type: 'text', text: 'before' }, - { type: 'image', mimeType: 'image/png', data: 'base64-data', annotations: { audience: ['assistant'] } }, + { type: 'image', mimeType: 'image/png', data: 'AQ==', annotations: { audience: ['assistant'] } }, + { type: 'text', text: 'between' }, + { type: 'image', mimeType: 'image/jpeg', data: 'Ag==' }, + { type: 'text', text: 'after' }, ] satisfies JsonValue[] const client = createMockClient( [{ name: 'img', inputSchema: { type: 'object' } }], { content: blocks }, ) - await syncTools(client as never, ctx, defaultOpts, new Map()) - const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__img', arguments: {} }) + await syncTools(client as never, rich.ctx, defaultOpts, new Map()) + const result = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('c1'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) - expect(result.content[0]).toEqual({ type: 'text', text: 'before\n[image: image/png, content discarded]' }) + expect(result.content.map(block => block.type)).toEqual(['text', 'image', 'text', 'image', 'text']) + expect(result.content[0]).toEqual({ type: 'text', text: 'before' }) + expect(result.content[2]).toEqual({ type: 'text', text: 'between' }) + expect(result.content[4]).toEqual({ type: 'text', text: 'after' }) + const firstImage = result.content[1] + const secondImage = result.content[3] + if (firstImage?.type !== 'image' || secondImage?.type !== 'image') throw new Error('expected ordered image blocks') + expect(firstImage.attachment.mediaType).toBe('image/png') + expect(firstImage.attachment.bytes).toBe(1) + expect(secondImage.attachment.mediaType).toBe('image/jpeg') + expect(secondImage.attachment.bytes).toBe(1) + expect(rich.attachments.saved.map(input => [...input.data])).toEqual([[1], [2]]) + expect(JSON.stringify(result.content)).not.toContain('AQ==') + expect(JSON.stringify(result.content)).not.toContain('Ag==') if (result.isError) throw new Error('expected MCP success') expect(result.value).toEqual({ content: blocks }) }) + it('keeps a valid raw image result while explicitly refusing it without a durable route', async () => { + const blocks = [{ type: 'image', mimeType: 'image/png', data: 'AQ==' }] satisfies JsonValue[] + const client = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: blocks }, + ) + + await syncTools(client as never, ctx, defaultOpts, new Map()) + const result = await ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('no-store'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + + expect(result.content).toEqual([{ + type: 'text', + text: '[image unavailable: image/png; no attachment store is mounted; raw image data remains available to programmatic callers]', + }]) + if (result.isError) throw new Error('image refusal must preserve MCP success') + expect(result.value).toEqual({ content: blocks }) + }) + + it('rejects a malformed image batch before storing any member', async () => { + const rich = await mountRichRegistry() + const blocks = [ + { type: 'image', mimeType: 'image/png', data: 'AQ==' }, + { type: 'image', mimeType: 'image/png', data: 'not base64' }, + ] satisfies JsonValue[] + const client = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: blocks }, + ) + + await syncTools(client as never, rich.ctx, defaultOpts, new Map()) + const result = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('bad-batch'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + + expect(rich.attachments.saved).toEqual([]) + expect(result.content).toHaveLength(2) + expect(textAt(result.content, 0)).toContain('another image in the same result was invalid') + expect(textAt(result.content, 1)).toContain('not canonical base64') + }) + + it('rejects non-canonical and incomplete image blocks as one atomic batch', async () => { + const rich = await mountRichRegistry() + const client = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: [ + { type: 'image', mimeType: 'image/tiff', data: 'AQ==' }, + { type: 'image', mimeType: 'image/png', data: 'AB==' }, + { type: 'image', mimeType: 'image/png' }, + ] }, + ) + + await syncTools(client as never, rich.ctx, defaultOpts, new Map()) + const result = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('strict-batch'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + + expect(rich.attachments.saved).toEqual([]) + expect(result.content).toHaveLength(3) + expect(textAt(result.content, 0)).toContain('not PNG, JPEG, WebP, or GIF') + expect(textAt(result.content, 1)).toContain('not canonical base64') + expect(textAt(result.content, 2)).toContain('not canonical base64') + }) + + it('does not admit images for a route without declared image input', async () => { + const rich = await mountRichRegistry() + const client = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: [{ type: 'image', mimeType: 'image/png', data: 'AQ==' }] }, + ) + + await syncTools(client as never, rich.ctx, defaultOpts, new Map()) + const result = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('text-route'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn('text') as never, + }) + + expect(rich.attachments.saved).toEqual([]) + expect(textAt(result.content)).toContain('does not declare image input') + }) + + it('refuses images when the exact route is missing, unverifiable, or canceled', async () => { + const rich = await mountRichRegistry() + const client = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: [{ type: 'image', mimeType: 'image/png', data: 'AQ==' }] }, + ) + await syncTools(client as never, rich.ctx, defaultOpts, new Map()) + + const noProvider = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('no-provider'), + name: 'mcp__srv__img', + arguments: {}, + agent: { options: { model: 'vision' }, session: { requestHeader: () => undefined } } as never, + }) + expect(textAt(noProvider.content)).toContain('route could not be resolved') + + const noModel = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('no-model'), + name: 'mcp__srv__img', + arguments: {}, + agent: { options: { provider: 'visual' }, session: { requestHeader: () => undefined } } as never, + }) + expect(textAt(noModel.content)).toContain('route could not be resolved') + + const noLlmCtx = await mountRegistry() + await noLlmCtx.plugin(RecordingAttachmentStore) + await syncTools(client as never, noLlmCtx, defaultOpts, new Map()) + const noLlm = await noLlmCtx.tools.execute({ + signal: testToolSignal, + callId: CallId('no-llm'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + expect(textAt(noLlm.content)).toContain('route could not be resolved') + + vi.spyOn(rich.ctx.llm, 'resolveModelInfo').mockRejectedValueOnce(new Error('catalog down')) + const unverified = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('unverified'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + expect(textAt(unverified.content)).toContain('route could not be verified') + + vi.spyOn(rich.ctx.llm, 'resolveModelInfo').mockResolvedValueOnce({ + provider: 'visual', id: 'vision', name: 'vision', + }) + const unknown = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('unknown-modalities'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + expect(textAt(unknown.content)).toContain('does not declare image input') + + const controller = new AbortController() + vi.spyOn(rich.ctx.llm, 'resolveModelInfo').mockImplementationOnce(async (provider, model) => { + controller.abort(new Error('stop')) + return { provider, id: model, name: model, inputModalities: ['text', 'image'] } + }) + const canceled = await rich.ctx.tools.execute({ + signal: controller.signal, + callId: CallId('canceled'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + expect(canceled.isError).toBe(true) + expect(canceled.content[0]).toEqual({ type: 'text', text: 'Error: tool call aborted' }) + expect(rich.attachments.saved).toEqual([]) + }) + + it('refuses images when attachment storage rejects the admitted batch', async () => { + const rich = await mountRichRegistry() + vi.spyOn(rich.attachments, 'saveImages').mockRejectedValueOnce(new Error('disk full')) + const client = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: [{ type: 'image', mimeType: 'image/png', data: 'AQ==' }] }, + ) + + await syncTools(client as never, rich.ctx, defaultOpts, new Map()) + const result = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('store-rejected'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + + expect(textAt(result.content)).toContain('durable image storage rejected the result') + }) + + it('reports attachment policy rejection as image admission rather than storage failure', async () => { + const rich = await mountRichRegistry() + vi.spyOn(rich.attachments, 'saveImages').mockRejectedValueOnce( + new AttachmentError('too many images', 'TOO_MANY_IMAGES'), + ) + const client = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: [{ type: 'image', mimeType: 'image/png', data: 'AQ==' }] }, + ) + + await syncTools(client as never, rich.ctx, defaultOpts, new Map()) + const result = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('policy-rejected'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + + expect(textAt(result.content)).toContain('image admission rejected the result: too many images') + expect(textAt(result.content)).not.toContain('storage rejected') + }) + + it('lets post-execute replacement win over a prepared image projection', async () => { + const rich = await mountRichRegistry() + rich.ctx.on('tools/post-execute', async (): Promise => ({ + kind: 'accept', + content: [{ type: 'text', text: 'policy replacement' }], + })) + const client = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: [{ type: 'image', mimeType: 'image/png', data: 'AQ==' }] }, + ) + + await syncTools(client as never, rich.ctx, defaultOpts, new Map()) + const result = await rich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('replaced'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + + expect(rich.attachments.saved).toHaveLength(1) + expect(result.content).toEqual([{ type: 'text', text: 'policy replacement' }]) + }) + + it('lets post-execute value replacement and blocking discard prepared projections', async () => { + const valueRich = await mountRichRegistry() + valueRich.ctx.on('tools/post-execute', async (): Promise => ({ + kind: 'accept', + value: { content: [{ type: 'text', text: 'value replacement' }] }, + })) + const valueClient = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: [{ type: 'image', mimeType: 'image/png', data: 'AQ==' }] }, + ) + await syncTools(valueClient as never, valueRich.ctx, defaultOpts, new Map()) + const replaced = await valueRich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('value-replaced'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + expect(replaced.content).toEqual([{ type: 'text', text: 'value replacement' }]) + + const blockedRich = await mountRichRegistry() + blockedRich.ctx.on('tools/post-execute', async (): Promise => ({ + kind: 'block', + feedback: [{ type: 'text', text: 'blocked by policy' }], + })) + const blockedClient = createMockClient( + [{ name: 'img', inputSchema: { type: 'object' } }], + { content: [{ type: 'image', mimeType: 'image/png', data: 'Ag==' }] }, + ) + await syncTools(blockedClient as never, blockedRich.ctx, defaultOpts, new Map()) + const blocked = await blockedRich.ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('blocked'), + name: 'mcp__srv__img', + arguments: {}, + agent: agentOn() as never, + }) + expect(blocked.isError).toBe(true) + expect(blocked.content).toEqual([{ type: 'text', text: 'blocked by policy' }]) + }) + it('preserves primitive JSON MCP blocks while Native rendering marks them unsupported', async () => { const blocks = [42, null, ['nested']] satisfies JsonValue[] const client = createMockClient( @@ -396,7 +781,7 @@ describe('tool execution', () => { expect(result.content[0]).toEqual({ type: 'text', - text: '[unsupported content type: unknown]\n[unsupported content type: unknown]\n[unsupported content type: unknown]', + text: '[unsupported MCP content block: expected an object]\n[unsupported MCP content block: expected an object]\n[unsupported MCP content block: expected an object]', }) if (result.isError) throw new Error('expected primitive MCP blocks to remain a successful JSON value') expect(result.value).toEqual({ content: blocks }) @@ -547,7 +932,7 @@ describe('tool execution edge cases', () => { ctx = await mountRegistry() }) - it('handles audio content with placeholder', async () => { + it('reports unsupported audio without claiming the raw block was discarded', async () => { const client = createMockClient( [{ name: 'audio_tool', inputSchema: { type: 'object' } }], { content: [{ type: 'audio', mimeType: 'audio/mp3' }] }, @@ -556,10 +941,13 @@ describe('tool execution edge cases', () => { await syncTools(client as never, ctx, defaultOpts, new Map()) const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__audio_tool', arguments: {} }) - expect(result.content[0]).toEqual({ type: 'text', text: '[audio: audio/mp3, content discarded]' }) + expect(result.content[0]).toEqual({ + type: 'text', + text: '[audio result unsupported: audio/mp3; raw audio data remains available to programmatic callers]', + }) }) - it('handles resource content with placeholder', async () => { + it('reports unsupported embedded resources without discarding the raw block', async () => { const client = createMockClient( [{ name: 'res_tool', inputSchema: { type: 'object' } }], { content: [{ type: 'resource' }] }, @@ -568,19 +956,36 @@ describe('tool execution edge cases', () => { await syncTools(client as never, ctx, defaultOpts, new Map()) const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__res_tool', arguments: {} }) - expect(result.content[0]).toEqual({ type: 'text', text: '[resource: content discarded]' }) + expect(result.content[0]).toEqual({ + type: 'text', + text: '[embedded resource unsupported; raw resource data remains available to programmatic callers]', + }) }) - it('handles resource_link content with placeholder', async () => { + it('preserves resource-link name and URI in the model projection', async () => { const client = createMockClient( [{ name: 'link_tool', inputSchema: { type: 'object' } }], - { content: [{ type: 'resource_link' }] }, + { content: [{ type: 'resource_link', name: 'Design', uri: 'https://example.test/design' }] }, ) await syncTools(client as never, ctx, defaultOpts, new Map()) const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__link_tool', arguments: {} }) - expect(result.content[0]).toEqual({ type: 'text', text: '[resource: content discarded]' }) + expect(result.content[0]).toEqual({ type: 'text', text: 'Resource link: Design (https://example.test/design)' }) + }) + + it('diagnoses an incomplete resource link', async () => { + const client = createMockClient( + [{ name: 'link_tool', inputSchema: { type: 'object' } }], + { content: [{ type: 'resource_link', name: 'Missing URI' }] }, + ) + + await syncTools(client as never, ctx, defaultOpts, new Map()) + const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('missing-link'), name: 'mcp__srv__link_tool', arguments: {} }) + + expect(result.content[0]).toEqual({ + type: 'text', text: '[resource link unavailable: the MCP block is missing its name or URI]', + }) }) it('handles unknown content types', async () => { @@ -592,7 +997,7 @@ describe('tool execution edge cases', () => { await syncTools(client as never, ctx, defaultOpts, new Map()) const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__unknown_tool', arguments: {} }) - expect(result.content[0]).toEqual({ type: 'text', text: '[unsupported content type: video]' }) + expect(result.content[0]).toEqual({ type: 'text', text: '[unsupported MCP content type: video]' }) }) it('handles image with missing mimeType (buggy server)', async () => { @@ -604,7 +1009,10 @@ describe('tool execution edge cases', () => { await syncTools(client as never, ctx, defaultOpts, new Map()) const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__img2', arguments: {} }) - expect(result.content[0]).toEqual({ type: 'text', text: '[image: unknown, content discarded]' }) + expect(result.content[0]).toEqual({ + type: 'text', + text: '[image unavailable: unknown media type; the declared media type is not PNG, JPEG, WebP, or GIF; raw image data remains available to programmatic callers]', + }) }) it('handles audio with missing mimeType (buggy server)', async () => { @@ -616,7 +1024,10 @@ describe('tool execution edge cases', () => { await syncTools(client as never, ctx, defaultOpts, new Map()) const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__audio_no_mime', arguments: {} }) - expect(result.content[0]).toEqual({ type: 'text', text: '[audio: unknown, content discarded]' }) + expect(result.content[0]).toEqual({ + type: 'text', + text: '[audio result unsupported: unknown media type; raw audio data remains available to programmatic callers]', + }) }) it('handles text block with missing text (buggy server)', async () => { @@ -628,7 +1039,7 @@ describe('tool execution edge cases', () => { await syncTools(client as never, ctx, defaultOpts, new Map()) const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__notext', arguments: {} }) - expect(result.content[0]).toEqual({ type: 'text', text: '(notext returned no text content)' }) + expect(result.content[0]).toEqual({ type: 'text', text: '(notext returned no model-visible content)' }) }) it('handles empty content array', async () => { @@ -640,7 +1051,7 @@ describe('tool execution edge cases', () => { await syncTools(client as never, ctx, defaultOpts, new Map()) const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__empty_tool', arguments: {} }) - expect(result.content[0]).toEqual({ type: 'text', text: '(empty_tool returned no text content)' }) + expect(result.content[0]).toEqual({ type: 'text', text: '(empty_tool returned no model-visible content)' }) }) @@ -678,7 +1089,10 @@ describe('tool execution edge cases', () => { const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('c1'), name: 'mcp__srv__err_notext', arguments: {} }) expect(result.isError).toBe(true) - expect(result.content[0]).toEqual({ type: 'text', text: 'Error: [image: image/png, content discarded]' }) + expect(result.content[0]).toEqual({ + type: 'text', + text: 'Error: [image unavailable: image/png; this result was not admitted to durable model context; raw image data remains available to programmatic callers]', + }) }) diff --git a/packages/mcp/mcp-client/tsconfig.json b/packages/mcp/mcp-client/tsconfig.json index 7dbe351b30..b48aea0922 100644 --- a/packages/mcp/mcp-client/tsconfig.json +++ b/packages/mcp/mcp-client/tsconfig.json @@ -21,6 +21,9 @@ { "path": "../../core/tools" }, + { + "path": "../../core/scope" + }, { "path": "../../subprocess/subprocess" }, diff --git a/packages/plan/README.i18n.yaml b/packages/plan/README.i18n.yaml index c2a60d7d2d..51a2fb09eb 100644 --- a/packages/plan/README.i18n.yaml +++ b/packages/plan/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/plan/README.md README.md: 535a2b02e297d7c3bc68538fa1d03226089372e0 -README.zh.md: 0ee1c658ee0a7863eb16b99d956e882f320a3717 +README.zh.md: 288daf7a7b19a14372ab9396363bf558f1bcc240 diff --git a/packages/plan/README.zh.md b/packages/plan/README.zh.md index 0ee1c658ee..288daf7a7b 100644 --- a/packages/plan/README.zh.md +++ b/packages/plan/README.zh.md @@ -6,8 +6,8 @@ Plan mode 是按 agent(智能体)记录的协作状态,而不是通用模 | 包 | 职责 | ctx 键 | |---|---|---| -| [`plan-mode/`](plan-mode/README.md) | 负责 plan mode 状态、指引、命令和评审流程 | `ctx.planMode` | +| [`plan-mode/`](plan-mode/README.zh.md) | 负责 plan mode 状态、指引、命令和评审流程 | `ctx.planMode` | -[plan 专用协作状态](../../.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.md)决策记录了该家族的设计。 +[plan 专用协作状态](../../.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.zh.md)决策记录了该家族的设计。 -子系统参考——`plan/mode` 折叠、步骤边界刷写、配置、退出工具——见 [docs/subsystems/plan.md](../../docs/subsystems/plan.md);设计见[计划专属协作状态](../../.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.md)。 +子系统参考——`plan/mode` 折叠、步骤边界刷写、配置、退出工具——见 [docs/subsystems/plan.md](../../docs/subsystems/plan.zh.md);设计见[计划专属协作状态](../../.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.zh.md)。 diff --git a/packages/plan/plan-mode/README.i18n.yaml b/packages/plan/plan-mode/README.i18n.yaml index fb59fa576d..b1c7671e8b 100644 --- a/packages/plan/plan-mode/README.i18n.yaml +++ b/packages/plan/plan-mode/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/plan/plan-mode/README.md -README.md: 7171997406ea43487762d9947d07426df400c78a -README.zh.md: 5c04cdabc293c9abda6bb5e77f596715ee137c6f +README.md: 3eabe2cb3f04b434b7f908f7beca869f1022a59e +README.zh.md: b3548da880af5d0cacfebba4486294d9d5085626 diff --git a/packages/plan/plan-mode/README.md b/packages/plan/plan-mode/README.md index 7171997406..3eabe2cb3f 100644 --- a/packages/plan/plan-mode/README.md +++ b/packages/plan/plan-mode/README.md @@ -16,13 +16,13 @@ While active, `plan:policy` renders the configured `section`. The plugin always The review question declares the `plan-review` presentation intent, naming `Approve` as the label that approves it, so a capable UI presents the plan as a decision instead of a generic question; the answer the tool reads is the same either way. A dismissed review — the user closing the request to speak instead — is reported to the model as such, telling it to stay in plan mode and wait for the message; every other review failure keeps the seam's own message. -When `ctx.commands` is composed, the package registers `/plan [message]` and reserves the exact argument `off` for direct exit. Bare `/plan` selects plan mode; any other non-empty argument selects it first and is then submitted through `agent.steer()`, so it becomes the next step's ordinary logged user message under plan guidance. `/plan off` selects inactive without sending model input; it also cancels a pending entry before plan mode reaches a request. +When `ctx.commands` is composed, the package registers `/plan [message]` and reserves the exact argument `off` for direct exit. Bare `/plan` selects plan mode; any other non-empty argument selects it first and is then submitted through `agent.steer()`, so it becomes the next step's ordinary logged user message under plan guidance. `/plan off` selects inactive without sending model input; it also cancels a pending entry before plan mode reaches a request. The command declares `input.images`: composer image attachments ride the steered message ahead of its text block. Bare `/plan` with images steers an image-only user message, while `/plan off` with images returns a direct error before any mode change so the composer keeps them. The Web client consumes the plugin-owned `/plan` command; other entry points may drive the same service directly without defining a second mode vocabulary. ## Session projection -When the composition mounts `ctx.sessionProjections` ([`@deepseek-ai/dsh-session-projection`](../../session/session-projection/README.md)), this package registers the `plan` projection unit under an injected child. The unit folds two event kinds: a `command/run` record named `plan` with recorded `args` sets the wanted target (`off` → inactive, anything else → active), and `plan/mode` commits the logged state and clears it; every other event returns the same state reference. `view` derives `{ active, pending }`, where `pending` is true only while an outstanding selection differs from the logged state — a pure replay quantity, so host restarts, other tabs, and cold reads all recover it from the log alone (the `/plan` handler calls `set()` before any failing path, so a failed handler cannot leave a recorded command without its plan selection). The key merges into `SessionProjectionMap` from `src/types.ts` (served to host consumers via `./types` and client aggregates via `./client`); the framework drives the unit and carriers serve the value on the history tail page and the `session/projection` push frame. Compositions without the registry are unaffected. +When the composition mounts `ctx.sessionProjections` ([`@deepseek-ai/dsh-session-projection`](../../session/session-projection/README.md)), this package registers the `plan` projection unit under an injected child. A `command/run` record named `plan` with recorded `args` starts a candidate target (`off` → inactive, anything else → active); its paired `command/done` retains a successful selection and drops an error; `plan/mode` commits the logged state and clears the retained selection. Every other event returns the same state reference. `view` derives `{ active, pending }`, where `pending` is true only while an unsettled or successful selection differs from the logged state. This remains a pure replay quantity, so host restarts, other tabs, and cold reads recover it from the log alone, and a rejected `/plan off` with images cannot leave a pending exit. The key merges into `SessionProjectionMap` from `src/types.ts` (served to host consumers via `./types` and client aggregates via `./client`); the framework drives the unit and carriers serve the value on the history tail page and the `session/projection` push frame. Compositions without the registry are unaffected. ## Configuration @@ -65,11 +65,11 @@ The section is stable within plan mode, but entering or leaving changes the syst #### What the model sees -`/plan`, `/plan off`, and their terminal results stay outside model history. A non-empty suffix other than the exact `off` argument becomes one trimmed user text block through `agent.steer()` after plan mode is selected. An active `/plan off` selection contributes the standard logged user-switch notice only when the last request header described plan mode; cancelling a pending entry contributes none because no request observed it. +`/plan`, `/plan off`, and their terminal results stay outside model history. A non-empty suffix other than the exact `off` argument becomes one user message through `agent.steer()` after plan mode is selected: any admitted image attachments as leading image blocks, then the trimmed text block. Bare `/plan` with admitted images steers one user message containing only those image blocks. An active `/plan off` selection contributes the standard logged user-switch notice only when the last request header described plan mode; cancelling a pending entry contributes none because no request observed it. #### Token effect -The optional message costs the same history tokens as submitting that text separately; bare `/plan` and `/plan off` add none. A narrated active exit adds the small retained switch notice. +The optional message costs the same history tokens as submitting that content separately. Bare `/plan` without images and `/plan off` add none; bare `/plan` with images has the normal image-prompt cost. A narrated active exit adds the small retained switch notice. #### KV Cache effect diff --git a/packages/plan/plan-mode/README.zh.md b/packages/plan/plan-mode/README.zh.md index 5c04cdabc2..b3548da880 100644 --- a/packages/plan/plan-mode/README.zh.md +++ b/packages/plan/plan-mode/README.zh.md @@ -10,19 +10,21 @@ `ctx.planMode.set(agent, active)` 会在 agent 空闲时立即追加独立的 `plan/mode` 事件,因为下一个提示词之前不会运行轮内 pre-step。agent 运行时,该方法会保留待生效选择,直到下一个被接受的轮内 pre-step。返回值区分 `committed`、`queued`、表示反转的 `cancelled` 和 `noop`。`get(agent)` 返回 `{ active, pending? }`,将用于组装当前步骤的日志状态与用户的轮中选择分开。初始与续步 pre-step 都会应用待生效选择;同一步骤的请求恢复重试会复用已冻结的 assembly,并将该选择保留到下一个被接受的轮内 pre-step。当最后记录的请求头描述了另一状态时,用户选择的变更会贡献一条插件来源的 `user/message` 通知(两条提交路径皆然)。 + + ## 模型与人类交互 激活时,`plan:policy` 会渲染已配置的 `section`。插件始终注册 `exit_plan_mode`,使工具 schema 在转换期间保持稳定;其 execute 路径只接受已激活的 plan mode,且只有通过 `ctx.userQuestions` 获得用户明确批准后才退出。 评审问题声明 `plan-review` 呈现意图,并指名 `Approve` 为表示批准的标签,因此有能力的 UI 会把计划呈现为一次决定而非通用问题;两种情况下该工具读到的回答完全相同。放弃审阅——用户关闭请求,转而发言——会如实报告给模型,要求它留在 plan mode 中等待那条消息;其余每一种评审失败都保留 seam 自身的消息。 -组合 `ctx.commands` 时,该包会注册 `/plan [message]`,并将参数恰好为 `off` 的情况保留给直接退出。不带参数的 `/plan` 会启用 plan mode;任何其他非空参数都会先启用 plan mode,再通过 `agent.steer()` 提交,因此它会在 plan 引导下成为下一步骤的常规已记录用户消息。`/plan off` 会选择停用状态,不发送模型输入;它还可以在启用 plan mode 的待处理选择由轮内 pre-step 追加之前将其取消。 +组合 `ctx.commands` 时,该包会注册 `/plan [message]`,并将参数恰好为 `off` 的情况保留给直接退出。不带参数的 `/plan` 会启用 plan mode;任何其他非空参数都会先启用 plan mode,再通过 `agent.steer()` 提交,因此它会在 plan 引导下成为下一步骤的常规已记录用户消息。`/plan off` 会选择停用状态,不发送模型输入;它还可以在启用 plan mode 的待处理选择由轮内 pre-step 追加之前将其取消。该命令声明了 `input.images`:composer 图片附件会随被 steer 的消息一起提交,位于文本块之前。不带参数的 `/plan` 若附有图片,会 steer 一条只含图片的用户消息;`/plan off` 若附有图片,会在任何模式变更前直接返回错误,composer 保留图片。 Web 客户端使用该插件提供的 `/plan` 命令;其他入口可以直接驱动同一服务,无需定义第二套 mode 词汇。 ## 会话投影 -当组合挂载 `ctx.sessionProjections`([`@deepseek-ai/dsh-session-projection`](../../session/session-projection/README.md))时,本包会在一个注入的子插件中注册 `plan` 投影单元。该单元折叠两类事件:名为 `plan` 且携带已记录 `args` 的 `command/run` 记录会设置目标状态(`off` → 未激活,其余 → 激活),`plan/mode` 会提交已记录状态并清除该目标;其他任何事件都返回同一个状态引用。`view` 推导 `{ active, pending }`,其中 `pending` 仅在尚未落实的选择与已记录状态不同时为 true。该值完全由日志回放得出,因此 host 重启、其他标签页和冷读都能仅凭日志恢复它。`/plan` 处理器会在任何可能失败的路径之前调用 `set()`,因此处理器失败时不会留下缺少对应 plan 选择的已记录命令。key 由 `src/types.ts` 通过声明合并加入 `SessionProjectionMap`:host 消费方经 `./types` 获取,client 聚合经 `./client` 获取。框架负责驱动该单元,载体通过历史尾页和 `session/projection` 推送帧提供其值。未挂载注册表的组合不受影响。 +当组合挂载 `ctx.sessionProjections`([`@deepseek-ai/dsh-session-projection`](../../session/session-projection/README.zh.md))时,本包会在一个注入的子插件中注册 `plan` 投影单元。名为 `plan` 且携带已记录 `args` 的 `command/run` 记录会开始一个候选目标(`off` → 未激活,其余 → 激活);与它配对的 `command/done` 保留成功选择并丢弃错误选择;`plan/mode` 提交已记录状态并清除已保留的选择。其他任何事件都返回同一个状态引用。`view` 推导 `{ active, pending }`,其中 `pending` 仅在未结算或已成功的选择与已记录状态不同时为 true。该值仍完全由日志回放得出,因此 host 重启、其他标签页和冷读都能仅凭日志恢复它,被拒绝的带图 `/plan off` 也不会留下待退出状态。key 由 `src/types.ts` 通过声明合并加入 `SessionProjectionMap`:host 消费方经 `./types` 获取,client 聚合经 `./client` 获取。框架负责驱动该单元,载体通过历史尾页和 `session/projection` 推送帧提供其值。未挂载注册表的组合不受影响。 ## 配置 @@ -37,7 +39,7 @@ Web 客户端使用该插件提供的 `/plan` 命令;其他入口可以直接 `section` 必填且非空。出现未知键时,插件会加载失败。该包不接受任意命名的 mode、工具过滤器、沙箱设置或批准策略。 -设计:[plan 专用协作状态](../../../.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.md)。 +设计:[plan 专用协作状态](../../../.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.zh.md)。 ## 模型体验 @@ -65,11 +67,11 @@ You are in plan mode. Explore and design before presenting the complete plan thr #### 模型所见内容 -`/plan`、`/plan off` 及其终端结果留在模型历史之外。除恰好为 `off` 以外的非空后缀会在选择 plan mode 后,通过 `agent.steer()` 成为一个已去除首尾空白的用户文本块。plan mode 已激活时,选择 `/plan off` 只会在最后一个请求头描述了 plan mode 的情况下追加标准的已记录用户切换通知;取消待生效进入不会贡献通知,因为没有请求观测到它。 +`/plan`、`/plan off` 及其终端结果留在模型历史之外。除恰好为 `off` 以外的非空后缀会在选择 plan mode 后,通过 `agent.steer()` 成为一条用户消息:任何已准入的图片附件作为前置图片块,之后是已去除首尾空白的文本块。不带参数的 `/plan` 若带有已准入图片,会 steer 一条只含这些图片块的用户消息。plan mode 已激活时,选择 `/plan off` 只会在最后一个请求头描述了 plan mode 的情况下追加标准的已记录用户切换通知;取消待生效进入不会贡献通知,因为没有请求观测到它。 #### Token 影响 -可选消息的历史 token 成本与单独提交该文本相同;不带参数的 `/plan` 和 `/plan off` 不增加 token。一次带有切换通知的已激活状态退出会追加一条简短且会保留的通知。 +可选消息的历史 token 成本与单独提交该内容相同。不带图片和参数的 `/plan` 与 `/plan off` 不增加 token;不带参数但带图的 `/plan` 产生常规图片提示词成本。一次带有切换通知的已激活状态退出会追加一条简短且会保留的通知。 #### KV Cache 影响 @@ -79,7 +81,7 @@ You are in plan mode. Explore and design before presenting the complete plan thr #### 模型所见内容 -[`exit_plan_mode` schema](../../../docs/tool-catalog.md#deepseek-aidsh-plan-mode) 在两种状态下均可用;在 plan mode 外执行会失败,而 plan mode 内经批准的评审会返回规范的 `{ approved: true }` 值,并渲染既有的确认文本。拒绝仍是携带评审反馈的失败调用,放弃审阅则是一次指明用户接手的失败调用。 +[`exit_plan_mode` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-plan-mode) 在两种状态下均可用;在 plan mode 外执行会失败,而 plan mode 内经批准的评审会返回规范的 `{ approved: true }` 值,并渲染既有的确认文本。拒绝仍是携带评审反馈的失败调用,放弃审阅则是一次指明用户接手的失败调用。 #### Token 影响 @@ -89,6 +91,8 @@ You are in plan mode. Explore and design before presenting the complete plan thr mode 转换不改变工具目录;plan 参数与评审结果按常规方式扩展对话。 + + ## 已知限制与暂缓事项 - Plan mode 只进行引导,而不强制执行;需要强制限制的部署必须分别配置沙箱与批准控制。 diff --git a/packages/plan/plan-mode/package.json b/packages/plan/plan-mode/package.json index d13c7606b1..07e7d0cf4c 100644 --- a/packages/plan/plan-mode/package.json +++ b/packages/plan/plan-mode/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-plan-mode", "description": "Logged per-agent plan mode with deployment guidance, a direct slash command, and a user-reviewed exit", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/plan/plan-mode/src/index.ts b/packages/plan/plan-mode/src/index.ts index d8399ee3e1..d81865932c 100644 --- a/packages/plan/plan-mode/src/index.ts +++ b/packages/plan/plan-mode/src/index.ts @@ -33,7 +33,7 @@ import { defineTool } from '@deepseek-ai/dsh-tools' import type {} from '@deepseek-ai/dsh-system-prompt' import { UserQuestionError } from '@deepseek-ai/dsh-user-questions' // Type-only edge: resolves `ctx.commands` for the optional command child. -import type {} from '@deepseek-ai/dsh-commands' +import type { CommandId } from '@deepseek-ai/dsh-commands' // Type-only: resolves ctx.sessionProjections for the optional unit child. import type {} from '@deepseek-ai/dsh-session-projection' import type { PlanProjection } from './types.ts' @@ -138,16 +138,34 @@ export function foldPlanMode(events: readonly SessionEvent[], end = events.lengt } /** - * Projection unit state: the logged mode plus the latest logged `/plan` - * selection (`command/run`) not yet resolved by a `plan/mode` commit. Plain - * JSON (persisted-cache precondition). + * Projection unit state: the logged mode, the latest successful `/plan` + * selection not yet resolved by a `plan/mode` commit, and an execution whose + * paired `command/done` has not settled. Plain JSON (persisted-cache + * precondition). */ interface PlanUnitState { active: boolean /** The selection's target mode; null when no selection is outstanding. */ wanted: boolean | null + /** The latest plan command awaiting its paired settlement. */ + running: { commandId: CommandId; wanted: boolean } | null } +declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + plan: PlanUnitState + } +} + +const planUnitStateSchema: ZodType = zod.object({ + active: zod.boolean(), + wanted: zod.boolean().nullable(), + running: zod.object({ + commandId: zod.string() as unknown as ZodType, + wanted: zod.boolean(), + }).strict().nullable(), +}).strict() + /** Wire payload schema of the `plan` projection. */ const planProjectionSchema: ZodType = zod.object({ active: zod.boolean(), @@ -232,36 +250,44 @@ export class PlanModeController extends Service { }, }) - // The plan projection unit (session-projection RFC): a pure double-event - // fold serving clients the whole {active, pending} value. `command/run` - // records the user's logged /plan selection (the handler calls `set()` - // before any failing path, so a failed handler cannot leave the recorded - // command without its plan selection); `plan/mode` records that selection - // and clears it. Pending is thereby a pure + // The plan projection unit (session-projection RFC): a pure event fold + // serving clients the whole {active, pending} value. `command/run` + // records the user's logged /plan selection, its paired `command/done` + // keeps only successful selections, and `plan/mode` records that + // selection and clears it. Pending is thereby a pure // replay quantity: host restarts, other tabs, and cold reads all recover // it from the log alone. The unit child activates only when a projection // registry is composed (headless assemblies stay unaffected). ctx.inject(['sessionProjections'], (projectionCtx) => { projectionCtx.sessionProjections.register<'plan', PlanUnitState>({ key: 'plan', - schema: planProjectionSchema, - init: () => ({ active: false, wanted: null }), + stateSchema: planUnitStateSchema, + init: () => ({ active: false, wanted: null, running: null }), apply: (state, event) => { if (event.type === 'command/run' && event.data.name === 'plan') { if (event.data.args === undefined) return state const wanted = event.data.args.trim() !== 'off' - return wanted === state.wanted ? state : { active: state.active, wanted } + return { ...state, running: { commandId: event.data.commandId, wanted } } + } + if (event.type === 'command/done' && event.data.commandId === state.running?.commandId) { + const wanted = event.data.kind === 'success' && state.running.wanted !== state.active + ? state.running.wanted + : null + return { ...state, wanted, running: null } } if (event.type === 'plan/mode') { - return { active: event.data.active, wanted: null } + return { ...state, active: event.data.active, wanted: null } } return state }, - view: state => ({ - active: state.active, - pending: state.wanted !== null && state.wanted !== state.active, - }), - stateVersion: 1, + wire: { + viewSchema: planProjectionSchema, + view: (state) => { + const wanted = state.running?.wanted ?? state.wanted + return { active: state.active, pending: wanted !== null && wanted !== state.active } + }, + }, + stateVersion: 2, }) }) @@ -270,9 +296,12 @@ export class PlanModeController extends Service { commandCtx.commands.register({ name: 'plan', description: 'Enter or leave plan mode', - input: { hint: '[off|message]' }, - handler: ({ agent, rawInput }) => { + input: { hint: '[off|message]', images: true }, + handler: ({ agent, rawInput, attachments }) => { const message = rawInput.trim() + if (message === 'off' && attachments.length > 0) { + return { kind: 'error', text: 'Image attachments cannot accompany /plan off.' } + } if (message === 'off') { switch (this.set(agent, false)) { case 'committed': @@ -291,7 +320,15 @@ export class PlanModeController extends Service { } } const outcome = this.set(agent, true) - if (message !== '') agent.steer(createUserMessage({ content: [{ type: 'text', text: message }], source: { kind: 'user' } })) + if (message !== '' || attachments.length > 0) { + agent.steer(createUserMessage({ + content: [ + ...attachments, + ...(message === '' ? [] : [{ type: 'text' as const, text: message }]), + ], + source: { kind: 'user' }, + })) + } return { kind: 'success', text: outcome === 'committed' diff --git a/packages/plan/plan-mode/src/types.ts b/packages/plan/plan-mode/src/types.ts index a3c10d2252..8efd32f22a 100644 --- a/packages/plan/plan-mode/src/types.ts +++ b/packages/plan/plan-mode/src/types.ts @@ -11,9 +11,10 @@ /** * The plan projection's wire value. `active` is the logged state in force * (the last `plan/mode`, inactive before the first); `pending` is true while - * a logged `/plan` selection (`command/run`) targets a state other than - * `active` and no later `plan/mode` event has recorded that state. Capability - * absence (plan-mode not composed) is the key's absence, never a value. + * a logged `/plan` selection targets a state other than `active`, has not + * failed through its paired `command/done`, and no later `plan/mode` event has + * recorded that state. Capability absence (plan-mode not composed) is the + * key's absence, never a value. */ export interface PlanProjection { active: boolean @@ -22,7 +23,7 @@ export interface PlanProjection { declare module '@deepseek-ai/dsh-session-projection/types' { interface SessionProjectionMap { - /** Plan collaboration state folded from `command/run` (name `plan`) and `plan/mode` events. */ + /** Plan collaboration state folded from the plan command lifecycle and `plan/mode` events. */ plan: PlanProjection } } diff --git a/packages/plan/plan-mode/tests/plan-mode.spec.ts b/packages/plan/plan-mode/tests/plan-mode.spec.ts index c4d41b13d8..eea2beddb0 100644 --- a/packages/plan/plan-mode/tests/plan-mode.spec.ts +++ b/packages/plan/plan-mode/tests/plan-mode.spec.ts @@ -60,7 +60,6 @@ async function agentWithSession( return agent } -/** Assemble exactly as the loop does: the agent is both subject and scope. */ function assembleFor(ctx: Context, agent: Agent) { return ctx.systemPrompt.assemble({ agent, scope: agent }) } @@ -561,13 +560,13 @@ describe('/plan', () => { const plainSteer = vi.fn() ;(plainAgent as unknown as { steer: typeof plainSteer }).steer = plainSteer expect(ctx.commands.list(plainAgent)).toEqual([ - { name: 'plan', description: 'Enter or leave plan mode', input: { hint: '[off|message]' } }, + { name: 'plan', description: 'Enter or leave plan mode', input: { hint: '[off|message]', images: true } }, ]) const signal = new AbortController().signal - expect(await ctx.commands.execute(plainAgent, '/mode', signal)).toBeUndefined() - expect(await ctx.commands.execute(plainAgent, '/review', signal)).toBeUndefined() - const plain = await ctx.commands.execute(plainAgent, '/plan', signal) + expect(await ctx.commands.execute(plainAgent, '/mode', [], signal)).toBeUndefined() + expect(await ctx.commands.execute(plainAgent, '/review', [], signal)).toBeUndefined() + const plain = await ctx.commands.execute(plainAgent, '/plan', [], signal) expect(plain?.result).toEqual({ kind: 'success', text: 'Entering plan mode (applies from the next step). Use /plan off to leave.', @@ -579,7 +578,7 @@ describe('/plan', () => { openTurn(messageAgent.session) const messageSteer = vi.fn() ;(messageAgent as unknown as { steer: typeof messageSteer }).steer = messageSteer - const plan = await ctx.commands.execute(messageAgent, '/plan draft the migration ', signal) + const plan = await ctx.commands.execute(messageAgent, '/plan draft the migration ', [], signal) expect(plan?.result).toEqual({ kind: 'success', text: 'Entering plan mode (applies from the next step). Use /plan off to leave.', @@ -600,7 +599,7 @@ describe('/plan', () => { const signal = new AbortController().signal const inactive = await agentWithSession(ctx, 'inactive-plan-command') - expect((await ctx.commands.execute(inactive, '/plan off', signal))?.result) + expect((await ctx.commands.execute(inactive, '/plan off', [], signal))?.result) .toEqual({ kind: 'success', text: 'Plan mode is already inactive.' }) expect(ctx.planMode.get(inactive)).toEqual({ active: false }) @@ -608,8 +607,8 @@ describe('/plan', () => { openTurn(entering.session) const enteringSteer = vi.fn() ;(entering as unknown as { steer: typeof enteringSteer }).steer = enteringSteer - await ctx.commands.execute(entering, '/plan', signal) - expect((await ctx.commands.execute(entering, '/plan off', signal))?.result) + await ctx.commands.execute(entering, '/plan', [], signal) + expect((await ctx.commands.execute(entering, '/plan off', [], signal))?.result) .toEqual({ kind: 'success', text: 'Plan mode entry cancelled.' }) expect(ctx.planMode.get(entering)).toEqual({ active: false, pending: false }) expect(enteringSteer).not.toHaveBeenCalled() @@ -621,10 +620,10 @@ describe('/plan', () => { openTurn(active.session) const activeSteer = vi.fn() ;(active as unknown as { steer: typeof activeSteer }).steer = activeSteer - expect((await ctx.commands.execute(active, '/plan off', signal))?.result) + expect((await ctx.commands.execute(active, '/plan off', [], signal))?.result) .toEqual({ kind: 'success', text: 'Leaving plan mode (applies from the next step).' }) expect(ctx.planMode.get(active)).toEqual({ active: true, pending: false }) - expect((await ctx.commands.execute(active, '/plan off', signal))?.result) + expect((await ctx.commands.execute(active, '/plan off', [], signal))?.result) .toEqual({ kind: 'success', text: 'Leaving plan mode (applies from the next step).' }) expect(activeSteer).not.toHaveBeenCalled() await boundary(ctx, active, 'step-start') @@ -637,14 +636,80 @@ describe('/plan', () => { await new Promise(resolve => setImmediate(resolve)) const signal = new AbortController().signal const agent = await agentWithSession(ctx, 'idle-plan-command') - expect((await ctx.commands.execute(agent, '/plan', signal))?.result) + expect((await ctx.commands.execute(agent, '/plan', [], signal))?.result) .toEqual({ kind: 'success', text: 'Plan mode on. Use /plan off to leave.' }) expect(foldPlanMode(agent.session.events)).toBe(true) - expect((await ctx.commands.execute(agent, '/plan off', signal))?.result) + expect((await ctx.commands.execute(agent, '/plan off', [], signal))?.result) .toEqual({ kind: 'success', text: 'Plan mode off.' }) expect(foldPlanMode(agent.session.events)).toBe(false) }) + it('steers image attachments with or without text and refuses them on /plan off', async () => { + const ctx = await setup() + await ctx.plugin(CommandRuntime) + await new Promise(resolve => setImmediate(resolve)) + let saved = 0 + const saveImage = (input: { mediaType: string }) => { + saved += 1 + return Promise.resolve({ + attachmentId: `att-${saved}`, mediaType: input.mediaType, bytes: 3, width: 1, height: 1, + }) + } + ctx.provide('attachments', { + imageLimits: { + maxImageBytes: 1024, maxImagesPerMessage: 4, maxMessageImageBytes: 1024, + maxImagePixels: 1_000_000, mediaTypes: ['image/png'], + }, + validateImage: () => Promise.resolve(), + saveImage, + async saveImages(inputs: readonly { mediaType: string }[]) { + const refs = [] + for (const input of inputs) refs.push(await saveImage(input)) + return refs + }, + }) + const signal = new AbortController().signal + const images = [{ mediaType: 'image/png' as const, data: 'AAAA' }] + + const agent = await agentWithSession(ctx, 'imaged-plan-command') + openTurn(agent.session) + const steer = vi.fn() + ;(agent as unknown as { steer: typeof steer }).steer = steer + const withMessage = await ctx.commands.execute(agent, '/plan sketch the layout', images, signal) + expect(withMessage?.result.kind).toBe('success') + expect(steer).toHaveBeenCalledExactlyOnceWith({ + id: expect.any(String) as unknown, + role: 'user', + content: [ + { type: 'image', attachment: expect.objectContaining({ attachmentId: 'att-1' }) as unknown }, + { type: 'text', text: 'sketch the layout' }, + ], + source: { kind: 'user' }, + }) + + const bareAgent = await agentWithSession(ctx, 'imaged-bare-plan-command') + openTurn(bareAgent.session) + const bareSteer = vi.fn() + ;(bareAgent as unknown as { steer: typeof bareSteer }).steer = bareSteer + expect((await ctx.commands.execute(bareAgent, '/plan', images, signal))?.result) + .toEqual({ kind: 'success', text: 'Entering plan mode (applies from the next step). Use /plan off to leave.' }) + expect(bareSteer).toHaveBeenCalledExactlyOnceWith({ + id: expect.any(String) as unknown, + role: 'user', + content: [{ type: 'image', attachment: expect.objectContaining({ attachmentId: 'att-2' }) as unknown }], + source: { kind: 'user' }, + }) + expect(ctx.planMode.get(bareAgent)).toEqual({ active: false, pending: true }) + + const activeAgent = await agentWithSession(ctx, 'imaged-off-plan-command', { active: true }) + const offSteer = vi.fn() + ;(activeAgent as unknown as { steer: typeof offSteer }).steer = offSteer + expect((await ctx.commands.execute(activeAgent, '/plan off', images, signal))?.result) + .toEqual({ kind: 'error', text: 'Image attachments cannot accompany /plan off.' }) + expect(offSteer).not.toHaveBeenCalled() + expect(ctx.planMode.get(activeAgent)).toEqual({ active: true }) + }) + it('removes the contributed command when the plan-mode plugin is disposed', async () => { const ctx = new Context() await ctx.plugin(SystemPrompt) diff --git a/packages/plan/plan-mode/tests/projection.spec.ts b/packages/plan/plan-mode/tests/projection.spec.ts index 25f9a5197c..e0ad20148c 100644 --- a/packages/plan/plan-mode/tests/projection.spec.ts +++ b/packages/plan/plan-mode/tests/projection.spec.ts @@ -1,9 +1,10 @@ /** * The `plan` projection unit (session-projection RFC's complete example): a - * double-event fold over the session log. `command/run` records named `plan` - * with recorded input set the wanted target (`off` → false, anything else - * → true); `plan/mode` commits and clears it. `view` reports pending only - * while an outstanding selection differs from the logged state. + * event fold over the session log. `command/run` records named `plan` with + * recorded input set the candidate target (`off` → false, anything else → + * true); `command/done` keeps successful candidates and drops failures; + * `plan/mode` commits and clears a selection. `view` reports pending only while + * an outstanding selection differs from the logged state. * Pending is thereby a pure replay quantity — a cold fold answers it without * the service's in-memory intent. Composition without plan-mode has no `plan` * key; unloading the fiber removes it (HMR safety). @@ -47,13 +48,20 @@ async function harness(withPlanMode: boolean): Promise { } /** Append one logged /plan selection record (the executor's command/run shape). */ -function runPlanCommand(session: Session, args: string, index: number): void { +function runPlanCommand(session: Session, args: string, index: number): CommandId { + const commandId = CommandId(`plan-proj-${String(index)}`) session.append('command/run', { - commandId: CommandId(`plan-proj-${String(index)}`), + commandId, name: 'plan', args, source: { kind: 'user' }, }) + return commandId +} + +/** Append the paired settlement for one projected plan command. */ +function settlePlanCommand(session: Session, commandId: CommandId, kind: 'success' | 'error'): void { + session.append('command/done', { commandId, kind }) } /** Commit one plan/mode flip inside an open turn (the invariant's turn-enclosure rule). */ @@ -71,15 +79,23 @@ describe('plan projection unit', () => { it('a logged /plan selection reads pending until plan/mode records it', async () => { const bench = await harness(true) - runPlanCommand(bench.session, '', 0) + const commandId = runPlanCommand(bench.session, '', 0) expect(bench.values().plan).toEqual({ active: false, pending: true }) - // A repeated identical selection returns the same state reference (no frame). - runPlanCommand(bench.session, '', 1) + settlePlanCommand(bench.session, commandId, 'success') expect(bench.values().plan).toEqual({ active: false, pending: true }) commitPlanMode(bench.session, true, 0) expect(bench.values().plan).toEqual({ active: true, pending: false }) }) + it('drops a plan selection when its command settles with an error', async () => { + const bench = await harness(true) + commitPlanMode(bench.session, true, 0) + const commandId = runPlanCommand(bench.session, 'off', 0) + expect(bench.values().plan).toEqual({ active: true, pending: true }) + settlePlanCommand(bench.session, commandId, 'error') + expect(bench.values().plan).toEqual({ active: true, pending: false }) + }) + it('folds `off` args and non-plan commands correctly, and a matching selection is not pending', async () => { const bench = await harness(true) commitPlanMode(bench.session, true, 0) @@ -128,7 +144,7 @@ describe('plan projection unit', () => { // memory involved, the fold alone answers {active:false, pending:true}. const cold = await harness(true) for (const event of bench.session.events) { - if (event.type === 'command/run' || event.type === 'plan/mode') { + if (event.type === 'command/run' || event.type === 'command/done' || event.type === 'plan/mode') { cold.session.append(event.type, event.data) } } diff --git a/packages/preset/README.i18n.yaml b/packages/preset/README.i18n.yaml index b8d5dd62d9..bf3583068a 100644 --- a/packages/preset/README.i18n.yaml +++ b/packages/preset/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/preset/README.md README.md: 5e7805eaa303b78c4d8a1a8972e1d031ee304fae -README.zh.md: 9afb0a4f585fc445977f81839695f15cf8b21327 +README.zh.md: 217d807f40abc0908fb0c67fa0c94dc4d0806592 diff --git a/packages/preset/README.zh.md b/packages/preset/README.zh.md index 9afb0a4f58..217d807f40 100644 --- a/packages/preset/README.zh.md +++ b/packages/preset/README.zh.md @@ -13,4 +13,4 @@ 本组假定的组装划分是:注册表与跨会话设施是进程单例,留在宿主组装中;preset 只承载单个 agent 对它们的贡献。若 preset 中某一行发布了进程级全局服务,挂载时即被拒绝,而不是留到与下一个会话相撞。 -设计详见 [按会话组装 agent preset 的 Agent Note](../../.agents/notes/implemented/architecture/2026-08-03-per-session-agent-presets.md)。 +设计详见 [按会话组装 agent preset 的 Agent Note](../../.agents/notes/implemented/architecture/2026-08-03-per-session-agent-presets.zh.md)。 diff --git a/packages/preset/agent-presets/README.i18n.yaml b/packages/preset/agent-presets/README.i18n.yaml index f3fef85bbd..25ebbe1b15 100644 --- a/packages/preset/agent-presets/README.i18n.yaml +++ b/packages/preset/agent-presets/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/preset/agent-presets/README.md README.md: 2438d751f000297cc4ede7b2a34736a62cc39dda -README.zh.md: c8ab8e167d624f639f6657d4c0ade7205b23821b +README.zh.md: a786afbc3790595edbd64ec16686717ef3d8140f diff --git a/packages/preset/agent-presets/README.zh.md b/packages/preset/agent-presets/README.zh.md index c8ab8e167d..a786afbc37 100644 --- a/packages/preset/agent-presets/README.zh.md +++ b/packages/preset/agent-presets/README.zh.md @@ -4,7 +4,7 @@ 按 preset 组装 agent(智能体)。**preset** 是一个目录,其中放置一份 `agent.cordis.yml`;roster 在整个进程内只把它挂载一次(常驻 scope),命名它的每个会话通过把自己 agent 的 scope key 认父到该挂载(`dsh-scope` 的父链)来加入。挂载的工具、提示词段落与投影单元只存在一份,覆盖所有已加入的 agent——其插件本就按 Session/Agent 分键存状态,会话在共享实例内互不串扰——而完全没有 agent 的宿主读取方(冷读记录)也能按 preset id 解析到同一份常驻注册。 -其机制是两条 seam。entry 上下文沿原型链连到子树被挂载时所在的上下文,而 [`dsh-tools`](../../core/tools/README.md) 与 [`dsh-system-prompt`](../../core/system-prompt/README.md) 本就按调用方上下文的 scope 分层归档注册——因此常驻挂载的贡献落在 **preset 的分层**里。把它们送达每个会话的是 `dsh-scope` 的父链:agent 的视图按 `agent → preset → global` 解析(近者遮蔽远者),挂载的监听器对认父到它的每个 agent 放行,而兄弟 preset 的监听器保持失聪。 +其机制是两条 seam。entry 上下文沿原型链连到子树被挂载时所在的上下文,而 [`dsh-tools`](../../core/tools/README.zh.md) 与 [`dsh-system-prompt`](../../core/system-prompt/README.zh.md) 本就按调用方上下文的 scope 分层归档注册——因此常驻挂载的贡献落在 **preset 的分层**里。把它们送达每个会话的是 `dsh-scope` 的父链:agent 的视图按 `agent → preset → global` 解析(近者遮蔽远者),挂载的监听器对认父到它的每个 agent 放行,而兄弟 preset 的监听器保持失聪。 ## 服务:`AgentPresets`(ctx 键:`agentPresets`) @@ -36,7 +36,7 @@ subagent 的子 agent 通过 `composeFrom()` 加入其父方的常驻组装, 按 id 重新挂载父方的 preset 与认父有两处差别,且两处都要紧。父方启动后被编辑过的组装文件会把与父方历史所产出时**不同**的一个代际交给子 agent;而此后被删除的 preset 会让子 agent 直接失败,尽管其父方仍在正常运行。认父还是同步的,这正是进程内 subagent 驱动能够使用它的前提——它们在同步的创建窗口里组装子 agent。 -子 agent 会把所加入的 id 记在自己的持久化 header 上(见 [`dsh-subagent`](../../subagent/subagent/README.md)),因此冷读子 agent 的历史时重建的是它实际运行过的组装,而不是部署默认值。 +子 agent 会把所加入的 id 记在自己的持久化 header 上(见 [`dsh-subagent`](../../subagent/subagent/README.zh.md)),因此冷读子 agent 的历史时重建的是它实际运行过的组装,而不是部署默认值。 ### 会话实际运行的是哪个 preset @@ -48,7 +48,7 @@ subagent 的子 agent 通过 `composeFrom()` 加入其父方的常驻组装, `recompose()` 先卸载已装入的子树、再装入新的,因为两份组装无法共存——它们会把相同的工具名注册进同一个层。挂载失败会恢复先前的组装,而不是让 agent 一无所有;未知 id 则在任何东西被拆除之前就被拒绝。 -"仅限尚未产出任何内容的 agent"是一条产品规则而非机制约束:在对话进行中调换工具,会留下新组装无法执行的、已被记录的工具调用。该规则由网关在传输层执行([`dsh-apiproxy`](../../host/apiproxy/README.md) 返回 `agent-preset-locked`),因为会话历史在那里才拿得到。 +"仅限尚未产出任何内容的 agent"是一条产品规则而非机制约束:在对话进行中调换工具,会留下新组装无法执行的、已被记录的工具调用。该规则由网关在传输层执行([`dsh-apiproxy`](../../host/apiproxy/README.zh.md) 返回 `agent-preset-locked`),因为会话历史在那里才拿得到。 ## 创作 @@ -93,7 +93,7 @@ description: 仅提供持久 bash 与 str_replace_editor 的双工具编码 Agen ### 可写根目录属于本包,随附根目录属于 app -`/.agent-presets` 是个人自有 preset 的所在,正如 `/skills` 是其自有 skill 的所在([`dsh-skill-filesystem`](../../skill/skill-filesystem/README.md)),因此 roster 自行推导它,而不等某个部署记得配置——一个什么都没配的启动器同样能发现并创作 preset。它追加在全部已配置根目录**之后**,从而保持靠前的根目录赢得重复 id:随附的 `standard` 仍然遮蔽一个占用该名字的家目录目录,而 `copy()` 会拒绝该 id,不会落下一个无人解析得到的 preset。 +`/.agent-presets` 是个人自有 preset 的所在,正如 `/skills` 是其自有 skill 的所在([`dsh-skill-filesystem`](../../skill/skill-filesystem/README.zh.md)),因此 roster 自行推导它,而不等某个部署记得配置——一个什么都没配的启动器同样能发现并创作 preset。它追加在全部已配置根目录**之后**,从而保持靠前的根目录赢得重复 id:随附的 `standard` 仍然遮蔽一个占用该名字的家目录目录,而 `copy()` 会拒绝该 id,不会落下一个无人解析得到的 preset。 根目录在服务构造时解析一次。若根目录集合在一次 `list()` 与依据其答案执行的 `copy()` 之间发生变化,写入的将是调用方从未见过的目录。 diff --git a/packages/preset/agent-presets/package.json b/packages/preset/agent-presets/package.json index 241cf74ce7..95e0144108 100644 --- a/packages/preset/agent-presets/package.json +++ b/packages/preset/agent-presets/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-agent-presets", "description": "Per-session agent composition from preset cordis.yml files for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/preset/agent-presets/src/index.ts b/packages/preset/agent-presets/src/index.ts index 6a24a89d76..5467116cf8 100644 --- a/packages/preset/agent-presets/src/index.ts +++ b/packages/preset/agent-presets/src/index.ts @@ -160,9 +160,9 @@ export class AgentPresets extends Service { // Note](../../../../.agents/notes/implemented/architecture/2026-08-10-host-plane-ownership-after-presets.md). // // Known false positive: a session created bare and bound later by - // `recompose` is warned about once, before its first bind. No shipped flow - // does that today — the Web surface mounts in `setup` and children join - // through `composeFrom` before publication. + // `recompose` is warned about once, before its first bind. Shipped Web + // sessions mount in `setup`, and children join through `composeFrom` + // before publication. ctx.on('agent/created', ({ agent }) => { if (this.resolvedRoots.length === 0) return if (this.composedPreset(agent.ctx) !== undefined) return diff --git a/packages/preset/agent-presets/src/mount.ts b/packages/preset/agent-presets/src/mount.ts index d9a7108fd7..57a5c6f341 100644 --- a/packages/preset/agent-presets/src/mount.ts +++ b/packages/preset/agent-presets/src/mount.ts @@ -103,7 +103,7 @@ class PresetTree extends Include { * backs every session that names it. * * Dropping the write drops the `loader/config-update` the inherited method - * emits with it. Nothing observes one for a preset subtree today, and a + * emits with it. No consumer observes one for a preset subtree, and a * future "edit your preset while it runs" flow needs a deliberate * persistence path rather than this method's return. */ @@ -117,6 +117,8 @@ export interface PresetMount { readonly presetId: string /** The mounted subtree's fiber. */ readonly fiber: Fiber + /** Loader entry tree whose active rows form this standing composition. */ + readonly tree: EntryTree /** The standing scope key agents are parented to (undefined only in torn-down records). */ readonly key: ScopeKey | undefined } @@ -365,7 +367,7 @@ export async function mountPreset(agentCtx: Context, preset: AgentPreset): Promi + 'a preset service must sit behind an `isolate` realm or move to the host composition', ) } - mounts.add({ presetId: preset.id, fiber, key: scopeOf(agentCtx) }) + mounts.add({ presetId: preset.id, fiber, tree, key: scopeOf(agentCtx) }) } catch (error) { try { await handle.dispose() diff --git a/packages/preset/agent-presets/tests/mount.spec.ts b/packages/preset/agent-presets/tests/mount.spec.ts index 824308e009..49e16e6ab3 100644 --- a/packages/preset/agent-presets/tests/mount.spec.ts +++ b/packages/preset/agent-presets/tests/mount.spec.ts @@ -53,7 +53,6 @@ async function harness(roster: Config = { default: 'standard', roots: ROOTS, inc return ctx } -/** Create one agent composed from `presetId`, exactly as a factory `setup` would. */ async function agentOn(ctx: Context, id: string, presetId?: string): Promise { const handle = await ctx.agents.create({ sessionId: SessionId(id), diff --git a/packages/preset/persona/README.i18n.yaml b/packages/preset/persona/README.i18n.yaml index 0323741aed..560a90256a 100644 --- a/packages/preset/persona/README.i18n.yaml +++ b/packages/preset/persona/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/preset/persona/README.md README.md: 978d01ddbd52095bc2486d27d08fde2d4a8fa79b -README.zh.md: d36af6c028f1258d044608535c6da4eaf2fb92e8 +README.zh.md: a6e682711c0d6bba742cd1bb192fb9f38bf32cd0 diff --git a/packages/preset/persona/README.zh.md b/packages/preset/persona/README.zh.md index d36af6c028..a6e682711c 100644 --- a/packages/preset/persona/README.zh.md +++ b/packages/preset/persona/README.zh.md @@ -4,7 +4,7 @@ 把 agent(智能体)人设做成一个可组装的行:它既可以遮蔽部署级人设,也可以拥有完整系统提示词。 -[`dsh-system-prompt`](../../core/system-prompt/README.md) 以自身配置持有部署级人设,并且无条件注册该段落,因此一个进程只有一份。[agent preset](../agent-presets/README.md) 无法自行挂载提示词注册表——若没有属于自己的行,preset 能改变 agent 的工具,却永远改不了它的身份。本包就是那一行。 +[`dsh-system-prompt`](../../core/system-prompt/README.zh.md) 以自身配置持有部署级人设,并且无条件注册该段落,因此一个进程只有一份。[agent preset](../agent-presets/README.zh.md) 无法自行挂载提示词注册表——若没有属于自己的行,preset 能改变 agent 的工具,却永远改不了它的身份。本包就是那一行。 ## 仅限 scope 内使用 diff --git a/packages/preset/persona/package.json b/packages/preset/persona/package.json index 24fb8660a1..8eb81d930b 100644 --- a/packages/preset/persona/package.json +++ b/packages/preset/persona/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-persona", "description": "Composition-authored deployment persona section for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/runtime-diagnostics/invariants/package.json b/packages/runtime-diagnostics/invariants/package.json index 41d8c631e1..9cbd4110e7 100644 --- a/packages/runtime-diagnostics/invariants/package.json +++ b/packages/runtime-diagnostics/invariants/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-invariants", "description": "Registry service for package-owned DeepSeek Harness runtime invariants", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/sandbox/README.i18n.yaml b/packages/sandbox/README.i18n.yaml index ab79d37170..cc93658129 100644 --- a/packages/sandbox/README.i18n.yaml +++ b/packages/sandbox/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/sandbox/README.md README.md: 61127118c895cce728ab0b4eed387f51054a83f1 -README.zh.md: 984a677aad6ab97579abf861ea8b99f149c07d08 +README.zh.md: c5b8f12c544e2b6b2c7ce1ec3fc63d274df18609 diff --git a/packages/sandbox/README.zh.md b/packages/sandbox/README.zh.md index 984a677aad..c5b8f12c54 100644 --- a/packages/sandbox/README.zh.md +++ b/packages/sandbox/README.zh.md @@ -6,10 +6,10 @@ | 包 | 职责 | ctx key | |---|---|---| -| [`sandbox/`](sandbox/README.md) | 定义进程沙箱服务和共享升权词汇 | `ctx.sandbox` | -| [`sandbox-local/`](sandbox-local/README.md) | 提供本地平台限制后端 | 注册到 `ctx.sandbox` | -| [`sandbox-policy/`](sandbox-policy/README.md) | 解析持久的逐会话沙箱策略 | `ctx.sandboxPolicy` | +| [`sandbox/`](sandbox/README.zh.md) | 定义进程沙箱服务和共享升权词汇 | `ctx.sandbox` | +| [`sandbox-local/`](sandbox-local/README.zh.md) | 提供本地平台限制后端 | 注册到 `ctx.sandbox` | +| [`sandbox-policy/`](sandbox-policy/README.zh.md) | 解析持久的逐会话沙箱策略 | `ctx.sandboxPolicy` | -[沙箱决策](../../.agents/notes/implemented/feature/2026-07-06-sandbox.md)记录了能力边界,[文件系统集成决策](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)记录了跨家族策略的使用方式。 +[沙箱决策](../../.agents/notes/implemented/feature/2026-07-06-sandbox.zh.md)记录了能力边界,[文件系统集成决策](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.zh.md)记录了跨家族策略的使用方式。 -子系统参考——模式与强制执行、按调用策略、包装 argv 方言、故障关闭错误——见 [docs/subsystems/sandbox.md](../../docs/subsystems/sandbox.md);边界与跨家族阶段见[沙箱](../../.agents/notes/implemented/feature/2026-07-06-sandbox.md)与[跨家族 fs 沙箱](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md) Agent Note。 +子系统参考——模式与强制执行、按调用策略、包装 argv 方言、故障关闭错误——见 [docs/subsystems/sandbox.md](../../docs/subsystems/sandbox.zh.md);边界与跨家族阶段见[沙箱](../../.agents/notes/implemented/feature/2026-07-06-sandbox.zh.md)与[跨家族 fs 沙箱](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.zh.md) Agent Note。 diff --git a/packages/sandbox/sandbox-local/README.i18n.yaml b/packages/sandbox/sandbox-local/README.i18n.yaml index d8155add50..8c63ddc717 100644 --- a/packages/sandbox/sandbox-local/README.i18n.yaml +++ b/packages/sandbox/sandbox-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/sandbox/sandbox-local/README.md -README.md: e1b59385a1380e0918e69c06dd86834c00cf2395 -README.zh.md: f4e31e220f3f69612243066a3183ebf79ed6f73a +README.md: 7f96ebd725280cc5b767c40dcf52e9353aaafe79 +README.zh.md: 8cb14128ae4b97c3089d72bb6126bc4fd835ac31 diff --git a/packages/sandbox/sandbox-local/README.md b/packages/sandbox/sandbox-local/README.md index e1b59385a1..7f96ebd725 100644 --- a/packages/sandbox/sandbox-local/README.md +++ b/packages/sandbox/sandbox-local/README.md @@ -6,10 +6,12 @@ Local implementation of the [`dsh-sandbox`](../sandbox/) seam. It selects and ca The package root exports the default and named `LocalSandboxProvider` plugin and `Config`; platform profile builders stay internal. -Unsupported platforms and unusable runners fail closed with `SANDBOX_UNAVAILABLE`; execution never silently falls through unconfined. Each wrap carries structured runner-failure rules so consumers can distinguish a broken sandbox from a command failure. The [sandbox Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md) owns selection rationale and profile differences. +Unsupported platforms and unusable runners fail closed with `SANDBOX_UNAVAILABLE`; execution never silently falls through unconfined. Each wrap carries structured runner-failure rules so consumers can distinguish a broken sandbox from a command failure. Policy is per call; the provider stores only the mechanism and cached runner verdict. Each wrap reports enforcement completeness plus backend-specific denial signatures and runner-failure rules. Landlock requires exit 125 and a `landlock-run:` fatal line after excluding only the exact partial-enforcement notice; a notice with child exit 1, 2, or 125 remains a child outcome. Bubblewrap and Seatbelt remain signature-only because neither public contract reserves a launcher-failure status. Consumers spawn the returned argv directly, so a missing or unexecutable runner is an out-of-band spawn failure while a successfully launched child exit 126 or 127 remains ordinary. `runnerCommand` skips probes and requires one or more non-empty, single-line, case-insensitive `runnerFailureSignatures` entries for the custom runner's own fatal dialect. Because its mechanism is unknown, it carries both Linux denial dialects. `probeTimeoutMs` bounds functional probes. The [sandbox Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md) owns selection and failure semantics. +The bwrap profile combines a read-only host root, fresh `/dev`, and private-PID `/proc`. Commands manage descendants but cannot see host processes; hiding host `/proc/` entries prevents magic links such as `root` and `fd` from bypassing its mounts. `workspace-write` adds ephemeral `/tmp` and a writable workspace bind. The [private-PID Agent Note](../../../.agents/notes/implemented/bug-fix/2026-08-06-bwrap-private-pid-namespace.md) records the boundary. + The Seatbelt profile is allow-default with `(deny file-write*)` plus write allow-lists, so exactly the mode's promised file effects are governed: `read-only` grants the `/dev/null` literal alone; `workspace-write` adds the workspace root, `/tmp`, and the per-user darwin temp dir (`os.tmpdir()` — the platform's real temp area for mkstemp-family tools), every root canonicalized because Seatbelt matches resolved paths (`/tmp` IS `/private/tmp`). Apple marks the `sandbox-exec` CLI deprecated but ships it on every macOS; the functional probe is what fails closed if that ever changes. The Windows rung keeps one deterministic write SID and standing ACE per workspace, but gives every live session/workspace pair a random private temp directory with a distinct SID and revocable ACE. Sessions sharing a workspace therefore share its intended write authority without inheriting one another's temp authority. A fresh provider always chooses a new temp path and SID, so crash residue cannot block or authorize a resumed session; agentless calls receive the same per-invocation isolation from the runner. A workspace equal to or containing the platform temp root fails before any ACL mutation because its inheritable workspace ACE would otherwise reach every private temp child. diff --git a/packages/sandbox/sandbox-local/README.zh.md b/packages/sandbox/sandbox-local/README.zh.md index f4e31e220f..8cb14128ae 100644 --- a/packages/sandbox/sandbox-local/README.zh.md +++ b/packages/sandbox/sandbox-local/README.zh.md @@ -6,9 +6,11 @@ 包根目录导出默认及命名的 `LocalSandboxProvider` 插件和 `Config`;平台 profile builder 仍为内部实现。 -不受支持的平台和不可用 runner 会以 `SANDBOX_UNAVAILABLE` 拒绝执行;执行绝不会静默回退为不受限制。每次包装都携带结构化 runner 失败规则,使消费方能够区分损坏的沙箱与命令失败。[沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md) 负责说明选择依据与 profile 差异。 +不受支持的平台和不可用 runner 会以 `SANDBOX_UNAVAILABLE` 拒绝执行;执行绝不会静默回退为不受限制。每次包装都携带结构化 runner 失败规则,使消费方能够区分损坏的沙箱与命令失败。 -策略逐调用传入;提供方只存储机制与缓存的 runner 结论。每次包装都会报告强制执行完整度,以及后端专用的拒绝签名和 runner 失败规则。Landlock 只有在退出码为 125,且仅排除完全匹配的部分强制执行通知后仍存在一行 `landlock-run:` 致命诊断时,才判定 runner 失败;携带该通知的子进程即使以 1、2 或 125 退出,也仍按子进程结果处理。Bubblewrap 和 Seatbelt 仍仅依据签名,因为两者的公开约定均未保留 launcher 失败状态。消费方会直接 spawn 返回的 argv,因此 runner 缺失或不可执行属于带外 spawn 失败,而成功启动的子进程以 126 或 127 退出时仍按普通结果处理。`runnerCommand` 会跳过探测,并要求为自定义 runner 自身的致命方言提供一个或多个非空、单行、不区分大小写的 `runnerFailureSignatures` 条目。由于其机制未知,它会同时携带两种 Linux 拒绝方言。`probeTimeoutMs` 限定功能探测的时长。[沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md) 负责说明选择与失败语义。 +策略逐调用传入;提供方只存储机制与缓存的 runner 结论。每次包装都会报告强制执行完整度,以及后端专用的拒绝签名和 runner 失败规则。Landlock 只有在退出码为 125,且仅排除完全匹配的部分强制执行通知后仍存在一行 `landlock-run:` 致命诊断时,才判定 runner 失败;携带该通知的子进程即使以 1、2 或 125 退出,也仍按子进程结果处理。Bubblewrap 和 Seatbelt 仍仅依据签名,因为两者的公开约定均未保留 launcher 失败状态。消费方会直接 spawn 返回的 argv,因此 runner 缺失或不可执行属于带外 spawn 失败,而成功启动的子进程以 126 或 127 退出时仍按普通结果处理。`runnerCommand` 会跳过探测,并要求为自定义 runner 自身的致命方言提供一个或多个非空、单行、不区分大小写的 `runnerFailureSignatures` 条目。由于其机制未知,它会同时携带两种 Linux 拒绝方言。`probeTimeoutMs` 限定功能探测的时长。[沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.zh.md) 负责说明选择与失败语义。 + +bwrap profile 将只读宿主根目录、全新的 `/dev` 与使用私有 PID 命名空间的 `/proc` 组合起来。命令可管理后代进程,但看不到宿主进程;隐藏宿主的 `/proc/` 条目,可以防止 `root`、`fd` 等魔法链接绕过其挂载约束。`workspace-write` 另加临时的 `/tmp` 与可写工作区绑定挂载。[私有 PID Agent Note](../../../.agents/notes/implemented/bug-fix/2026-08-06-bwrap-private-pid-namespace.zh.md)记录该边界。 Seatbelt profile 默认允许,但带 `(deny file-write*)` 和写入 allow-list,因此恰好约束相应模式承诺的文件操作:`read-only` 只授予 `/dev/null` 字面路径;`workspace-write` 另加工作区根目录、`/tmp` 和逐用户 darwin 临时目录(`os.tmpdir()`,即平台供 mkstemp 家族工具使用的真实临时区域)。每个根目录都经过规范化,因为 Seatbelt 匹配解析后的路径(`/tmp` 就是 `/private/tmp`)。Apple 将 `sandbox-exec` CLI(命令行界面)标为 deprecated,但所有 macOS 系统仍会提供它;若情况发生变化,功能探测会使执行被拒绝。 @@ -25,7 +27,7 @@ Windows 档为每个工作区保留一个确定性写入 SID 和常驻 ACE,但 ## 模型体验 -通过 [`dsh-bash-sandbox`](../../shell/bash-sandbox/README.md) 和 [`dsh-tool-bash`](../../shell/tool-bash/README.md) 间接影响;它们渲染该提供方的强制执行与拒绝事实,而 [`dsh-sandbox`](../sandbox/README.md) seam 负责定义 `SANDBOX_UNAVAILABLE` 文本,runner 选择与 profile 则不进入上下文。 +通过 [`dsh-bash-sandbox`](../../shell/bash-sandbox/README.zh.md) 和 [`dsh-tool-bash`](../../shell/tool-bash/README.zh.md) 间接影响;它们渲染该提供方的强制执行与拒绝事实,而 [`dsh-sandbox`](../sandbox/README.zh.md) seam 负责定义 `SANDBOX_UNAVAILABLE` 文本,runner 选择与 profile 则不进入上下文。 #### KV Cache 影响 diff --git a/packages/sandbox/sandbox-local/package.json b/packages/sandbox/sandbox-local/package.json index dfe13545ad..12c948b7c6 100644 --- a/packages/sandbox/sandbox-local/package.json +++ b/packages/sandbox/sandbox-local/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-sandbox-local", "description": "Local process-sandbox backends for the DeepSeek Harness sandbox seam: bwrap, the npm-distributed landlock-run launcher, macOS Seatbelt, or the Windows ACL restricted-token runner — functionally probed, fail-closed", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/sandbox/sandbox-local/src/index.ts b/packages/sandbox/sandbox-local/src/index.ts index 8880da8103..849aa560dd 100644 --- a/packages/sandbox/sandbox-local/src/index.ts +++ b/packages/sandbox/sandbox-local/src/index.ts @@ -66,7 +66,7 @@ export interface Config { /** Probe whether `bwrap` can create the profile; the provider caches the bounded result. */ function defaultProbeBwrap(timeoutMs: number): boolean { - const probe = spawnSync('bwrap', ['--ro-bind', '/', '/', '--dev', '/dev', '--proc', '/proc', '--die-with-parent', '--', 'true'], { + const probe = spawnSync('bwrap', [...bwrapProfileArgs({ mode: 'read-only', workspaceRoot: '/' }), '--', 'true'], { timeout: timeoutMs, stdio: 'ignore', }) @@ -169,7 +169,7 @@ const PLATFORM_CHAINS: Record = { * Enforcement completeness a rung claims when selected WITHOUT a probe (a * chain of one). `bwrap` and Seatbelt govern every promised file effect by * construction, so the claim is a profile fact; `landlock` is listed for the - * table's totality but is unreachable unprobed today (the Linux chain has + * table's totality but is unreachable without a probe (the Linux chain has * two rungs, so it is only ever selected through its probe, whose report is * what distinguishes full from per-ABI-partial — and the launcher additionally * self-reports partial enforcement on stderr at every confined run). diff --git a/packages/sandbox/sandbox-local/src/profiles.ts b/packages/sandbox/sandbox-local/src/profiles.ts index 5b76390319..4d647580c5 100644 --- a/packages/sandbox/sandbox-local/src/profiles.ts +++ b/packages/sandbox/sandbox-local/src/profiles.ts @@ -14,7 +14,7 @@ import type { SandboxPolicy } from '@deepseek-ai/dsh-sandbox' * @returns profile arguments before the trailing separator and command argv. */ export function bwrapProfileArgs(policy: SandboxPolicy): string[] { - const args = ['--ro-bind', '/', '/', '--dev', '/dev', '--proc', '/proc', '--die-with-parent'] + const args = ['--ro-bind', '/', '/', '--dev', '/dev', '--unshare-pid', '--proc', '/proc', '--die-with-parent'] if (policy.mode === 'workspace-write') { args.push('--tmpfs', '/tmp') args.push('--bind', policy.workspaceRoot, policy.workspaceRoot) diff --git a/packages/sandbox/sandbox-local/tests/bwrap.e2e.ts b/packages/sandbox/sandbox-local/tests/bwrap.e2e.ts index 3da6787049..9a8289b95a 100644 --- a/packages/sandbox/sandbox-local/tests/bwrap.e2e.ts +++ b/packages/sandbox/sandbox-local/tests/bwrap.e2e.ts @@ -1,5 +1,5 @@ import { spawnSync } from 'node:child_process' -import { existsSync, readFileSync, rmSync } from 'node:fs' +import { existsSync, readFileSync, readlinkSync, rmSync } from 'node:fs' import { mkdtemp, rm } from 'node:fs/promises' import { homedir, tmpdir } from 'node:os' import { join } from 'node:path' @@ -78,6 +78,44 @@ describe.skipIf(!bwrapUsable)('sandbox-local: real bwrap confinement', () => { expect(result.stdout).toBe('dev-ok\n') }) + it.each(['read-only', 'workspace-write'] as const)( + '%s runs in a private PID namespace and blocks writes through procfs root magic links', + async (mode) => { + const workdir = await tempDir(homedir()) + const outside = await tempDir(homedir()) + const target = join(outside, 'escaped.txt') + const sandbox = await provider() + // Compare PID-namespace identity, not PID numbers: numeric /proc entries + // recur inside a private namespace, and the /proc/1/root write below is + // denied even in a shared namespace (host init is root-owned), so this + // comparison is the assertion that fails when --unshare-pid is lost. + const hostPidNamespace = readlinkSync('/proc/self/ns/pid') + const visibility = runConfined(sandbox, 'readlink /proc/self/ns/pid', { mode, workspaceRoot: workdir }) + expect(visibility.result.status).toBe(0) + expect(visibility.result.stdout.trim()).not.toBe('') + expect(visibility.result.stdout.trim()).not.toBe(hostPidNamespace) + + const escape = runConfined( + sandbox, + `printf escaped > /proc/1/root${target}`, + { mode, workspaceRoot: workdir }, + ) + expect(escape.result.status).not.toBe(0) + expect(existsSync(target)).toBe(false) + }, + ) + + it('keeps descendants observable and controllable inside the private PID namespace', async () => { + const workdir = await tempDir(homedir()) + const sandbox = await provider() + const { result } = runConfined( + sandbox, + 'sleep 30 & child=$!; kill -0 "$child" && kill "$child"; wait "$child"; status=$?; test "$status" -ge 128', + { mode: 'read-only', workspaceRoot: workdir }, + ) + expect(result.status).toBe(0) + }) + it('workspace-write lands a write inside the workspace root and still denies one beside it', async () => { const workdir = await tempDir(homedir()) const outside = await tempDir(homedir()) diff --git a/packages/sandbox/sandbox-local/tests/local.spec.ts b/packages/sandbox/sandbox-local/tests/local.spec.ts index 389f23a011..285afb865b 100644 --- a/packages/sandbox/sandbox-local/tests/local.spec.ts +++ b/packages/sandbox/sandbox-local/tests/local.spec.ts @@ -62,13 +62,13 @@ function fakeSeatbeltExec(status: number): string { const SEATBELT_RO_PROFILE = '(version 1) (allow default) (deny file-write*) (allow file-write* (literal "/dev/null"))' describe('profile dialects', () => { - it('bwrap read-only: whole tree read-only with fresh /dev and /proc, no writable mounts', () => { - expect(bwrapProfileArgs(RO)).toEqual(['--ro-bind', '/', '/', '--dev', '/dev', '--proc', '/proc', '--die-with-parent']) + it('bwrap read-only: whole tree read-only with fresh /dev and private PID-scoped /proc, no writable mounts', () => { + expect(bwrapProfileArgs(RO)).toEqual(['--ro-bind', '/', '/', '--dev', '/dev', '--unshare-pid', '--proc', '/proc', '--die-with-parent']) }) it('bwrap workspace-write: adds an ephemeral /tmp and rebinds the workspace root', () => { expect(bwrapProfileArgs(WW)).toEqual([ - '--ro-bind', '/', '/', '--dev', '/dev', '--proc', '/proc', '--die-with-parent', + '--ro-bind', '/', '/', '--dev', '/dev', '--unshare-pid', '--proc', '/proc', '--die-with-parent', '--tmpfs', '/tmp', '--bind', '/ws', '/ws', ]) }) diff --git a/packages/sandbox/sandbox-local/tests/packed-install.e2e.ts b/packages/sandbox/sandbox-local/tests/packed-install.e2e.ts index eded2b8d70..135ebb8494 100644 --- a/packages/sandbox/sandbox-local/tests/packed-install.e2e.ts +++ b/packages/sandbox/sandbox-local/tests/packed-install.e2e.ts @@ -32,6 +32,7 @@ const WORKSPACE_CLOSURE = [ // consumer resolves it like any other @deepseek-ai peer (koffi arrives // from the registry). 'packages/sandbox/sandbox-windows-acl', + 'packages/subprocess/win32-process', 'packages/sandbox/sandbox', 'packages/core/session', 'packages/core/scope', diff --git a/packages/sandbox/sandbox-policy/README.i18n.yaml b/packages/sandbox/sandbox-policy/README.i18n.yaml index f12249398a..31ef984c5e 100644 --- a/packages/sandbox/sandbox-policy/README.i18n.yaml +++ b/packages/sandbox/sandbox-policy/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/sandbox/sandbox-policy/README.md README.md: c85420019b490d311360f310c04212ac6cd737d1 -README.zh.md: 58354adf55a55951cad1ea9adb88aea2457c7f7b +README.zh.md: 46e1deea680122622e58490064dcc4ea6699d1d3 diff --git a/packages/sandbox/sandbox-policy/README.zh.md b/packages/sandbox/sandbox-policy/README.zh.md index 58354adf55..46e1deea68 100644 --- a/packages/sandbox/sandbox-policy/README.zh.md +++ b/packages/sandbox/sandbox-policy/README.zh.md @@ -2,11 +2,11 @@ [English](README.md) | 中文 -沙箱策略解析的唯一归属位置:部署默认 [`SandboxMode`](../sandbox/README.md) 与回退根目录,加上每个会话的持久模式覆盖和不可变工作区根目录。每项负责强制执行的能力在每次调用时都会收到一项解析完成的模式与根目录策略;模型在每次请求前会收到当前策略,而不会另收一份能力清单。 +沙箱策略解析的唯一归属位置:部署默认 [`SandboxMode`](../sandbox/README.zh.md) 与回退根目录,加上每个会话的持久模式覆盖和不可变工作区根目录。每项负责强制执行的能力在每次调用时都会收到一项解析完成的模式与根目录策略;模型在每次请求前会收到当前策略,而不会另收一份能力清单。 ## 为何需要共享归属位置 -文件系统工具、一次性 bash 命令和终端会话可以用不同组合强制执行同一套模式词汇。如果各自解析 `mode` + `workspaceRoot`,就可能漂移成分裂世界,正是[沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md)所警告的情况。每个强制执行后端都会消费归属方解析出的完整策略,而当前上下文只说明该策略对于任何受 DSH 文件沙箱强制执行的可用操作有何含义。[跨家族 fs 沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)记录了共享策略决策。 +文件系统工具、一次性 bash 命令和终端会话可以用不同组合强制执行同一套模式词汇。如果各自解析 `mode` + `workspaceRoot`,就可能漂移成分裂世界,正是[沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.zh.md)所警告的情况。每个强制执行后端都会消费归属方解析出的完整策略,而当前上下文只说明该策略对于任何受 DSH 文件沙箱强制执行的可用操作有何含义。[跨家族 fs 沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.zh.md)记录了共享策略决策。 ## 配置 diff --git a/packages/sandbox/sandbox-policy/package.json b/packages/sandbox/sandbox-policy/package.json index 06ea90d1cf..eda4650b47 100644 --- a/packages/sandbox/sandbox-policy/package.json +++ b/packages/sandbox/sandbox-policy/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-sandbox-policy", "description": "Per-call sandbox policy resolver and current model context: deployment fallbacks plus each session's mode and workspace root, shared by every enforcing capability family", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/sandbox/sandbox-windows-acl/README.i18n.yaml b/packages/sandbox/sandbox-windows-acl/README.i18n.yaml index a394957f49..978f8f40e5 100644 --- a/packages/sandbox/sandbox-windows-acl/README.i18n.yaml +++ b/packages/sandbox/sandbox-windows-acl/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/sandbox/sandbox-windows-acl/README.md -README.md: a78f334342196ec1848a4a360e5c60b375a28057 -README.zh.md: 8962653b69b23b92fe763e4fcc90bf45911865f7 +README.md: 055f2e92a42acfc707207306d31b51ea765cb3df +README.zh.md: 0163c1235ec42c4967a0071258d8dcc68b278370 diff --git a/packages/sandbox/sandbox-windows-acl/README.md b/packages/sandbox/sandbox-windows-acl/README.md index a78f334342..055f2e92a4 100644 --- a/packages/sandbox/sandbox-windows-acl/README.md +++ b/packages/sandbox/sandbox-windows-acl/README.md @@ -38,7 +38,7 @@ sandbox.dispose() // revokes the revocable (temp) grant, keeps the standing work rmSync(tempDir, { recursive: true, force: true }) ``` -A direct `AclSandbox` requires an explicit private temp directory (or `tempDir: null`; the ambient temp root is never an implicit grant), grants the workspace ACEs STANDING (dispose() leaves them — they are the cross-instance reuse cache), and grants the distinct temp SID revocably. The server-side reuse is the `AclWriteGrant` class: `add(path, standing)` per directory, `dispose()` revokes the revocable paths and frees the SID — see the runner contract below. Every Win32 API call in this package is checked; failures throw `Win32Error` carrying the API name, the exact Win32 code, the `FormatMessageW` system text, and the failing path/context. This is deliberate: the POC ignored every return value and, when `CreateRestrictedToken` failed, silently ran the child with the FULL unrestricted token (fail-open). This port fails closed by construction. +A direct `AclSandbox` requires an explicit private temp directory (or `tempDir: null`; the ambient temp root is never an implicit grant), grants the workspace ACEs STANDING (dispose() leaves them — they are the cross-instance reuse cache), and grants the distinct temp SID revocably. The server-side reuse is the `AclWriteGrant` class: `add(path, standing)` per directory, `dispose()` revokes the revocable paths and frees the SID — see the runner contract below. Every policy-specific Win32 call and every process primitive from [`dsh-win32-process`](../../subprocess/win32-process/README.md) is checked; failures throw `Win32Error` carrying the API name, the exact Win32 code, the `FormatMessageW` system text, and the failing path/context. This is deliberate: the POC ignored every return value and, when `CreateRestrictedToken` failed, silently ran the child with the FULL unrestricted token (fail-open). This port fails closed by construction. ## The confinement runner @@ -62,14 +62,12 @@ The `AclSandbox` class (explicit private `tempDir` + `tempWriteSid`, or `tempDir ## Header verification -All constants, signatures, and struct layouts were verified against the Windows headers on the development machine (MinGW `winnt.h` / `accctrl.h` / `aclapi.h` / `securitybaseapi.h` / `sddl.h` / `processthreadsapi.h` / `fileapi.h` / `namedpipeapi.h` / `synchapi.h` / `winbase.h`) and are cross-checked at runtime by [`verify/abi-probe.cpp`](verify/abi-probe.cpp) (sizes, offsets, enum values, static asserts): +The sandbox-owned SID, ACL, token, file, and lock constants and layouts were verified against the Windows headers on the development machine (MinGW `winnt.h` / `accctrl.h` / `aclapi.h` / `securitybaseapi.h` / `sddl.h` / `fileapi.h`) and are cross-checked by [`verify/abi-probe.cpp`](verify/abi-probe.cpp). The shared process, stdio, and Job ABI is owned and verified by [`@deepseek-ai/dsh-win32-process`](../../subprocess/win32-process/README.md#header-verification). ```sh g++ -std=c++20 -municode -O2 -o abi-probe.exe verify/abi-probe.cpp -ladvapi32 && ./abi-probe.exe ``` -The koffi struct definitions assert their sizes against the probe at module load, so a header/koffi layout drift fails loudly instead of corrupting memory. - ## Verified boundaries (inherent to restricted tokens, not this port) - **Everyone grants remain ambient write authority.** Everyone must stay in both restricting lists: removing it breaks early DLL initialization and CNG. An external NTFS object whose normal DACL grants Everyone a requested write right therefore clears both access checks and stays writable under both modes. The real runner suite provisions an external `Everyone:Modify` directory and pins that behavior; the provider reports `enforcement: 'partial'` so callers can reject or surface the weaker boundary. @@ -95,7 +93,6 @@ None directly; the denial surface belongs to the tool layer. - **One write allowlist per workspace** — the write SID is the unit of the allowlist and IS the workspace identity; reusing one sandbox instance across two workspaces widens both grants to both roots (the same SID would then name two roots). Create one instance per workspace root — the seam does exactly this, keyed by the workspace path. - **Cleanup is best-effort by design** — `dispose()` attempts every temp revocation and aggregates failures into an `AggregateError`; a cleanup failure can leave the random directory and its temp-SID-only ACE behind. Once the process exits no future token carries that SID, so the residue is inert until OS temp hygiene or manual directory removal reclaims it. - **Standing workspace ACEs are invisible residue.** Renaming a workspace derives a new SID; the old ACEs on the old path stay (inert, write-SID-only). A future cleanup command may reap them; nothing re-propagates because of them. -- **NULL-DACL directories are not identity-preserving under grant+revoke.** A directory with a NULL DACL (rare — Windows-created directories carry real DACLs) means "everyone full control"; `grantWrite` builds the new ACL from that null, and the revoke round-trip leaves an EMPTY (deny-all) DACL rather than the original NULL DACL. The POC shares the behavior; real workspace and temp directories carry real DACLs, so this stays a documented edge rather than a guarded path. - **Piped stdio capture is impossible for confined grandchildren (the named-pipe default SD template).** libuv's pipe stdio uses NAMED pipes; `CreateNamedPipeW` without security attributes installs the Win32 layer's user-mode default SD template (built by KernelBase — owner/SYSTEM/Admins full, Everyone/ANONYMOUS read-only, the fixed template [MS documents](https://learn.microsoft.com/en-us/windows/win32/ipc/named-pipe-security-and-access-rights)) — NOT the token default DACL, which is what the kernel applies to a raw SD-null create — so the client-end open requests write access no restricting SID is granted: `spawn(..., { stdio: 'pipe' })` inside a confined process fails with EPERM, the POC-documented "no output redirection" boundary of WRITE_RESTRICTED tokens. Inherited (`inherit`/fd) and ignored (`ignore`) stdio spawns work, and anonymous pipes (CreatePipe — a token-default-DACL consumer, e.g. PowerShell pipelines) work because the restricted token's default DACL carries a full-access restricting-SID ACE (set at init). A confined process therefore cannot capture a grandchild's output through a pipe; tools that must capture output cannot run confined. - **Grant materialization is an eager full-tree propagation.** `SetNamedSecurityInfoW` on a directory with inheritable ACEs walks every descendant immediately (NOT lazily per access — measured at tens of seconds on large workspace trees). The per-workspace identity pays it once per workspace per machine (lazily at the first confined execution ever, skipped entirely on every later provision when the exact ACE stands). Private temp directories start empty, so their distinct grant is cheap. If a workspace is huge, the first confined write on this host is correspondingly slow. - **Read-side confinement and network policy are out of scope** — `WRITE_RESTRICTED` intersects write accesses only; pair this backend with a read-side policy for stronger confinement. diff --git a/packages/sandbox/sandbox-windows-acl/README.zh.md b/packages/sandbox/sandbox-windows-acl/README.zh.md index 8962653b69..0163c1235e 100644 --- a/packages/sandbox/sandbox-windows-acl/README.zh.md +++ b/packages/sandbox/sandbox-windows-acl/README.zh.md @@ -6,7 +6,7 @@ 一句话机制:把调用者令牌复制为 `WRITE_RESTRICTED` 受限令牌,其 restricting SIDs 携带彼此独立的工作区能力与私有临时目录能力。工作区 SID 由规范工作区路径确定性派生(`workspaceWriteSid`),因此工作区根目录 ACE 每台机器每个工作区只物化一次,之后每次会话、调用或重启都命中精确 ACE 跳过。每个活跃的会话/工作区对则获得一个随机临时目录,以及一个从该路径派生的 SID(`tempWriteSid`),因此各会话共享预期的工作区权限,却不会继承彼此的临时目录权限。此后 Windows 只在「调用者正常权限」与「restricting SID 交集」同时允许时才放行写入。这些 SID 是主要白名单,在系统其余位置不授予任何权限;但该检查还会继承**其他** restricting SID 的环境写 ACE(保活组登录 SID + Everyone),而 NTFS ACL 属于文件对象而非路径。Everyone 与硬链接边界正是该档报告部分而非完整强制执行的原因。 -直接构建在原生 ACL 机制上是记录在案的设计选择:它实现两种隔离模式,且不背负被否决的容器方案的问题——见[设计笔记](../../../.agents/notes/implemented/feature/2026-08-08-windows-acl-restricted-token-sandbox.md)([mxc](https://github.com/microsoft/mxc/blob/main/docs/process-container/os-version-support.md) 要求 Windows 11 24H2 的 OS 下限,且任意路径读取需要整体改写宿主 DACL;AppContainer 根本无法任意路径读取)。 +直接构建在原生 ACL 机制上是记录在案的设计选择:它实现两种隔离模式,且不背负被否决的容器方案的问题——见[设计笔记](../../../.agents/notes/implemented/feature/2026-08-08-windows-acl-restricted-token-sandbox.zh.md)([mxc](https://github.com/microsoft/mxc/blob/main/docs/process-container/os-version-support.md) 要求 Windows 11 24H2 的 OS 下限,且任意路径读取需要整体改写宿主 DACL;AppContainer 根本无法任意路径读取)。 ## 用法 @@ -38,7 +38,7 @@ sandbox.dispose() // revokes the revocable (temp) grant, keeps the standing work rmSync(tempDir, { recursive: true, force: true }) ``` -直接使用 `AclSandbox` 时,必须显式提供私有临时目录(或通过 `tempDir: null` 禁用临时写入;环境临时根目录绝不会被隐式授权),工作区 ACE 以**常驻**方式授予(`dispose()` 保留它们——它们是跨实例的复用缓存),不同的临时 SID 则以**可回收**方式授予。服务端复用则是 `AclWriteGrant` 类:每个目录一次 `add(path, standing)`,`dispose()` 撤销可回收路径并释放 SID——见下方 runner 契约。本包中的每个 Win32 API 调用都有检查;失败抛出 `Win32Error`,携带 API 名、精确 Win32 错误码、`FormatMessageW` 系统文本和失败的路径/上下文。这是刻意的:POC 忽略每个返回值,当 `CreateRestrictedToken` 失败时用完整无限制令牌静默运行子进程(fail-open)。本移植从构造上 fail-closed。 +直接使用 `AclSandbox` 时,必须显式提供私有临时目录(或通过 `tempDir: null` 禁用临时写入;环境临时根目录绝不会被隐式授权),工作区 ACE 以**常驻**方式授予(`dispose()` 保留它们——它们是跨实例的复用缓存),不同的临时 SID 则以**可回收**方式授予。服务端复用则是 `AclWriteGrant` 类:每个目录一次 `add(path, standing)`,`dispose()` 撤销可回收路径并释放 SID——见下方 runner 契约。每个 policy-specific Win32 调用和 [`dsh-win32-process`](../../subprocess/win32-process/README.zh.md) 提供的 process primitive 都有检查;失败抛出 `Win32Error`,携带 API 名、精确 Win32 错误码、`FormatMessageW` 系统文本和失败的路径/上下文。这是刻意的:POC 忽略每个返回值,当 `CreateRestrictedToken` 失败时用完整无限制令牌静默运行子进程(fail-open)。本移植从构造上 fail-closed。 @@ -64,14 +64,12 @@ Authenticated Users 在**两种**列表中都不存在——WMI 命名空间安 ## 头部验证 -所有常量、签名与结构体布局都在开发机上对照 Windows 头文件(MinGW `winnt.h` / `accctrl.h` / `aclapi.h` / `securitybaseapi.h` / `sddl.h` / `processthreadsapi.h` / `fileapi.h` / `namedpipeapi.h` / `synchapi.h` / `winbase.h`)验证过,并在运行时由 [`verify/abi-probe.cpp`](verify/abi-probe.cpp)(大小、偏移、枚举值、静态断言)交叉检查: +sandbox 自有的 SID、ACL、token、文件与锁常量和布局均已在开发机上对照 Windows 头文件(MinGW `winnt.h` / `accctrl.h` / `aclapi.h` / `securitybaseapi.h` / `sddl.h` / `fileapi.h`)验证,并由 [`verify/abi-probe.cpp`](verify/abi-probe.cpp) 交叉检查。共享的 process、stdio 与 Job ABI 由 [`@deepseek-ai/dsh-win32-process`](../../subprocess/win32-process/README.zh.md#header-verification) 归属并验证。 ```sh g++ -std=c++20 -municode -O2 -o abi-probe.exe verify/abi-probe.cpp -ladvapi32 && ./abi-probe.exe ``` -koffi 结构体定义在模块加载时对照探针断言其大小,因此头文件/koffi 布局漂移会大声失败而不是破坏内存。 - ## 已验证边界(受限令牌固有,非本移植引入) - **Everyone 授权仍是环境中的写权限来源。** Everyone 必须保留在两种 restricting 列表中:移除它会破坏早期 DLL 初始化与 CNG。因此,如果外部 NTFS 对象的正常 DACL 向 Everyone 授予所请求的写权限,它就会同时通过两次访问检查,并在两种模式下保持可写。真实 runner 套件配置一个外部 `Everyone:Modify` 目录并钉住该行为;提供方报告 `enforcement: 'partial'`,使调用方能够拒绝或向上暴露这项较弱的边界。 @@ -86,7 +84,7 @@ koffi 结构体定义在模块加载时对照探针断言其大小,因此头 ## 模型体验 -间接地通过 [`dsh-bash-sandbox`](../../shell/bash-sandbox/README.md)、[`dsh-pwsh-sandbox`](../../shell/pwsh-sandbox/README.md) 及其工具呈现:它们渲染此后端的部分强制执行与拒绝事实(工具层通过 `denialSignatures` 分类的受限 stderr),而 [`dsh-sandbox`](../sandbox/README.md) seam 拥有 `SANDBOX_UNAVAILABLE` 文本与 runner 选择。 +间接地通过 [`dsh-bash-sandbox`](../../shell/bash-sandbox/README.zh.md)、[`dsh-pwsh-sandbox`](../../shell/pwsh-sandbox/README.zh.md) 及其工具呈现:它们渲染此后端的部分强制执行与拒绝事实(工具层通过 `denialSignatures` 分类的受限 stderr),而 [`dsh-sandbox`](../sandbox/README.zh.md) seam 拥有 `SANDBOX_UNAVAILABLE` 文本与 runner 选择。 #### KV Cache 影响 @@ -97,7 +95,6 @@ koffi 结构体定义在模块加载时对照探针断言其大小,因此头 - **每个工作区一个写入白名单** —— 写入 SID 是白名单的基本单位,且**就是**工作区身份;同一沙盒实例跨两个工作区复用时,两个根目录会互相扩大授权面(同一个 SID 将命名两个根)。请按工作区根目录各建一个实例——seam 正是这样做的,以工作区路径为键。 - **清理按设计尽力而为** —— `dispose()` 会尝试全部临时撤销并把失败聚合为 `AggregateError`;清理失败可能留下随机目录及其仅含临时 SID 的 ACE。进程退出后,不会再有令牌携带该 SID,因此残留保持失效,直到 OS 临时目录卫生或手动移除目录将其回收。 - **常驻工作区 ACE 是不可见残留。** 工作区改名会派生新的 SID;旧路径上的旧 ACE 留在原地(失效、仅含写入 SID)。未来的清理命令可以回收它们;它们不会引起任何重新传播。 -- **NULL-DACL 目录在 grant+revoke 往返下不保持身份。** 带 NULL DACL 的目录(罕见——Windows 创建的目录都带真实 DACL)意味着「所有人完全控制」;`grantWrite` 从该 null 构建新 ACL,撤销往返后留下的是 EMPTY(全部拒绝)DACL 而非原始 NULL DACL。POC 行为相同;真实工作区与临时目录都带真实 DACL,因此这仍是记录在案的边界情形而非守护路径。 - **受限孙进程的管道 stdio 捕获不可用(named pipe 的默认 SD 模板)。** libuv 的管道 stdio 用的是 NAMED pipe;不带安全属性调用 `CreateNamedPipeW` 时,其默认安全描述符不是内核的模板,而是 Win32 层在用户态安装的默认 SD 模板(由 KernelBase 构建——owner/SYSTEM/Admins 全权,Everyone/ANONYMOUS 只读,即 [MS 文档](https://learn.microsoft.com/en-us/windows/win32/ipc/named-pipe-security-and-access-rights)记载的固定模板)——**不是**令牌默认 DACL(后者才是内核在原始 SD-null 创建时应用的)——因此 client 端打开所请求的写访问没有任何 restricting SID 被授予:受限进程内 `spawn(..., { stdio: 'pipe' })` 以 EPERM 失败,这是 POC 记载的 WRITE_RESTRICTED「无法重定向输出」边界。继承(`inherit`/fd)与忽略(`ignore`)stdio 的 spawn 可用;匿名管道(CreatePipe——令牌默认 DACL 的消费者,例如 PowerShell 的管道)因受限令牌默认 DACL 携带 restricting SID 全权 ACE(init 时写入)而可用。受限进程因此无法用管道捕获孙进程输出;必须捕获输出的工具无法在受限下运行。 - **授权物化是急切的全树传播。** 在带可继承 ACE 的目录上调用 `SetNamedSecurityInfoW` 会立即遍历每个后代(**不是**按访问惰性进行——大型工作区树上实测数十秒)。按工作区身份每台机器每个工作区只付一次(在首次受限执行时惰性进行,之后每次供给在精确 ACE 常驻时完全跳过)。私有临时目录创建时为空,因此其独立授权开销很小。如果工作区巨大,该主机上的第一次受限写入相应变慢。 - **读侧隔离与网络策略不在范围内** —— `WRITE_RESTRICTED` 只交叉检查写访问;将此后端与读侧策略配对以获得更强隔离。 diff --git a/packages/sandbox/sandbox-windows-acl/package.json b/packages/sandbox/sandbox-windows-acl/package.json index 7faad86ea0..fc3eb85c85 100644 --- a/packages/sandbox/sandbox-windows-acl/package.json +++ b/packages/sandbox/sandbox-windows-acl/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-sandbox-windows-acl", "description": "Windows ACL write-restriction sandbox backend (restricted-token spawn with capability-SID write allowlist) for the DeepSeek Harness sandbox seam", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -42,6 +42,7 @@ "@deepseek-ai/cordis": "workspace:^" }, "dependencies": { + "@deepseek-ai/dsh-win32-process": "workspace:^", "koffi": "^3.1.0" }, "devDependencies": { diff --git a/packages/sandbox/sandbox-windows-acl/src/errors.ts b/packages/sandbox/sandbox-windows-acl/src/errors.ts deleted file mode 100644 index b57d6dd466..0000000000 --- a/packages/sandbox/sandbox-windows-acl/src/errors.ts +++ /dev/null @@ -1,21 +0,0 @@ -/** - * Fail-closed Win32 error type. Every backend API failure raises this with the - * API name and the exact Win32 code; the original POC silently ignored every - * failed call and would run children UNRESTRICTED (fail-open) — that is the - * failure mode this class exists to prevent. - * @module @deepseek-ai/dsh-sandbox-windows-acl/errors - */ - -export class Win32Error extends Error { - /** The failing Win32 API name, e.g. `CreateRestrictedToken`. */ - readonly api: string - /** The Win32 error code (`GetLastError` for BOOL APIs, the HRESULT-style return for ACL APIs). */ - readonly win32Code: number - - constructor(api: string, win32Code: number, detail?: string) { - super(`${api} failed (Win32 ${win32Code})${detail === undefined ? '' : `: ${detail}`}`) - this.name = 'Win32Error' - this.api = api - this.win32Code = win32Code - } -} diff --git a/packages/sandbox/sandbox-windows-acl/src/ffi.ts b/packages/sandbox/sandbox-windows-acl/src/ffi.ts index 698f0dc2ee..983f5953f8 100644 --- a/packages/sandbox/sandbox-windows-acl/src/ffi.ts +++ b/packages/sandbox/sandbox-windows-acl/src/ffi.ts @@ -1,512 +1,299 @@ -/** - * Lazy koffi bindings for the Win32 ACL-sandbox backend. Koffi loads lazily so - * non-Windows processes never open Win32 libraries. Every function signature - * below was verified against the MinGW Windows headers on this machine - * (winnt.h / accctrl.h / aclapi.h / securitybaseapi.h / sddl.h / - * processthreadsapi.h / fileapi.h / namedpipeapi.h / synchapi.h / winbase.h); - * struct layouts are asserted at load time against verify/abi-probe.cpp. - * @module @deepseek-ai/dsh-sandbox-windows-acl/ffi - */ +/** ACL/token bindings layered on the shared Win32 process owner. */ import koffi from 'koffi' -import { Win32Error } from './errors.ts' +import { + ERROR_INSUFFICIENT_BUFFER, + Win32Error, + extendWin32ProcessBindings, + isNullPtr, + throwLastError, +} from '@deepseek-ai/dsh-win32-process' +import type { NativePtr, Win32ProcessBindings } from '@deepseek-ai/dsh-win32-process' import * as abi from './win32-abi.ts' -/** Branded koffi 3 native pointer. Koffi 3 pointers are BigInt values; the brand keeps them out of numeric contexts. */ -declare const nativePtr: unique symbol -/** Koffi 3 native pointer (a BigInt address), branded so it cannot silently enter numeric contexts. */ -export type NativePtr = bigint & { readonly [nativePtr]: true } +export { + allocPtrSlot, + allocUint32, + decodePtr, + decodeUint32, + isNullPtr, + throwLastError, + throwWin32, +} from '@deepseek-ai/dsh-win32-process' +export type { NativePtr } from '@deepseek-ai/dsh-win32-process' -/** - * True for NULL pointers, however koffi returns them (null or 0n). - * @param value - a pointer as koffi may hand it back (pointer, null, or 0n). - * @returns a type guard narrowing to the NULL shapes. - */ -export function isNullPtr(value: NativePtr | null | undefined): value is null | undefined { - return value === null || value === undefined || (value as bigint) === 0n +type Ptr = ReturnType +const PVOID: Ptr = koffi.pointer('void') +const PPVOID: Ptr = koffi.pointer(PVOID) + +/** ACL/token calls composed with the generic Win32 process binding table. */ +export interface Win32Bindings extends Win32ProcessBindings { + openProcess(desiredAccess: number, inheritHandle: number, pid: number): NativePtr + openProcessToken(process: NativePtr, desiredAccess: number, tokenHandle: NativePtr): number + localAlloc(flags: number, bytes: number): NativePtr + localFree(memory: NativePtr): NativePtr + convertStringSidToSidW(stringSid: string, sid: NativePtr): number + createWellKnownSid(type: number, domainSid: null, sid: NativePtr, size: NativePtr): number + isValidSid(sid: NativePtr): number + getLengthSid(sid: NativePtr): number + copySid(length: number, destination: NativePtr, source: NativePtr): number + getTokenInformation(token: NativePtr, cls: number, info: Buffer | null, length: number, needed: NativePtr): number + setTokenInformation(token: NativePtr, cls: number, info: Buffer, length: number): number + createRestrictedToken( + existing: NativePtr, + flags: number, + disableCount: number, + disableSids: null, + deletePrivilegeCount: number, + privilegesToDelete: null, + restrictCount: number, + restrictingSids: Buffer, + newToken: NativePtr, + ): number + setEntriesInAclW(count: number, entries: Buffer, oldAcl: NativePtr | null, newAcl: NativePtr): number + setNamedSecurityInfoW( + path: string, + objectType: number, + information: number, + owner: null, + group: null, + dacl: NativePtr | null, + sacl: null, + ): number + getNamedSecurityInfoW( + path: string, + objectType: number, + information: number, + owner: NativePtr, + group: NativePtr, + dacl: NativePtr, + sacl: NativePtr, + descriptor: NativePtr, + ): number + getTempPathW(length: number, buffer: Buffer): number + setEnvironmentVariableW(name: string, value: string): number + setConsoleCtrlHandler(handler: null, add: number): number + createFileW( + fileName: string, + desiredAccess: number, + shareMode: number, + attributes: null, + creationDisposition: number, + flagsAndAttributes: number, + templateFile: null, + ): NativePtr + lockFileEx( + file: NativePtr, + flags: number, + reserved: number, + bytesLow: number, + bytesHigh: number, + overlapped: NativePtr, + ): number + unlockFileEx( + file: NativePtr, + reserved: number, + bytesLow: number, + bytesHigh: number, + overlapped: NativePtr, + ): number } /** - * True for CreateFileW's INVALID_HANDLE_VALUE failure marker (-1, which - * koffi hands back as the unsigned 64-bit all-ones pointer). - * @param handle - the handle CreateFileW returned. - * @returns whether the handle signals failure. + * Return whether CreateFileW produced INVALID_HANDLE_VALUE. + * @param handle - handle returned by CreateFileW. + * @returns true for null, zero, or the all-bits-one sentinel. */ export function isInvalidHandle(handle: NativePtr | null | undefined): boolean { if (isNullPtr(handle)) return true return (handle as bigint) === 0xFFFFFFFFFFFFFFFFn || (handle as bigint) === -1n } -type Ptr = ReturnType - -/** Field subset written into a zeroed STARTUPINFOW (layout verified: size 104). */ -export interface StartupInfoInput { - cb: number - dwFlags: number - hStdInput: NativePtr - hStdOutput: NativePtr - hStdError: NativePtr -} - -/** Decoded PROCESS_INFORMATION (layout verified: size 24). */ -export interface ProcessInfoOutput { - hProcess: NativePtr | null - hThread: NativePtr | null - dwProcessId: number - dwThreadId: number -} - -/** The lazy koffi binding table: every Win32 call the ACL backend uses, signature-verified against the real headers. */ -export interface Win32Bindings { - // ---- process / token handles -------------------------------------------- - openProcess(desiredAccess: number, inheritHandle: number, pid: number): NativePtr - openProcessToken(process: NativePtr, desiredAccess: number, tokenHandle: NativePtr): number - closeHandle(handle: NativePtr): number - // ---- errors / diagnostics ------------------------------------------------ - getLastError(): number - formatMessageW(flags: number, source: null, messageId: number, languageId: number, buffer: Buffer, size: number, args: null): number - // ---- memory -------------------------------------------------------------- - localAlloc(flags: number, bytes: number): NativePtr - localFree(memory: NativePtr): NativePtr - // ---- SIDs ---------------------------------------------------------------- - convertStringSidToSidW(stringSid: string, sid: NativePtr): number - createWellKnownSid(type: number, domainSid: null, sid: NativePtr, size: NativePtr): number - isValidSid(sid: NativePtr): number - getLengthSid(sid: NativePtr): number - copySid(length: number, destination: NativePtr, source: NativePtr): number - // ---- token information --------------------------------------------------- - getTokenInformation(token: NativePtr, cls: number, info: Buffer | null, length: number, needed: NativePtr): number - setTokenInformation(token: NativePtr, cls: number, info: Buffer, length: number): number - // ---- restricted token ---------------------------------------------------- - createRestrictedToken( - existing: NativePtr, flags: number, - disableCount: number, disableSids: null, - deletePrivilegeCount: number, privilegesToDelete: null, - restrictCount: number, restrictingSids: Buffer, - newToken: NativePtr, - ): number - // ---- ACL editing --------------------------------------------------------- - setEntriesInAclW(count: number, entries: Buffer, oldAcl: NativePtr | null, newAcl: NativePtr): number - setNamedSecurityInfoW( - path: string, objectType: number, information: number, - owner: null, group: null, dacl: NativePtr | null, sacl: null, - ): number - getNamedSecurityInfoW( - path: string, objectType: number, information: number, - owner: NativePtr, group: NativePtr, dacl: NativePtr, sacl: NativePtr, descriptor: NativePtr, - ): number - // ---- environment / io ---------------------------------------------------- - getTempPathW(length: number, buffer: Buffer): number - createFileW( - fileName: string, desiredAccess: number, shareMode: number, attributes: null, - creationDisposition: number, flagsAndAttributes: number, templateFile: null, - ): NativePtr - lockFileEx(file: NativePtr, flags: number, reserved: number, bytesLow: number, bytesHigh: number, overlapped: NativePtr): number - unlockFileEx(file: NativePtr, reserved: number, bytesLow: number, bytesHigh: number, overlapped: NativePtr): number - createPipe(readHandle: NativePtr, writeHandle: NativePtr, attributes: null, size: number): number - setHandleInformation(handle: NativePtr, mask: number, flags: number): number - createProcessAsUserW( - token: NativePtr, applicationName: null, commandLine: string, - processAttributes: null, threadAttributes: null, - inheritHandles: number, creationFlags: number, environment: null, - currentDirectory: string | null, startupInfo: NativePtr, processInfo: NativePtr, - ): number - setEnvironmentVariableW(name: string, value: string): number - readFile(file: NativePtr, buffer: Buffer, count: number, bytesRead: NativePtr, overlapped: null): number - peekNamedPipe( - pipe: NativePtr, buffer: null, size: number, - bytesRead: NativePtr, totalAvail: NativePtr, leftThisMessage: NativePtr, - ): number - waitForSingleObject(handle: NativePtr, milliseconds: number): number - getExitCodeProcess(process: NativePtr, exitCode: NativePtr): number - resumeThread(thread: NativePtr): number - // ---- job object (runner kill-on-close) ----------------------------------- - createJobObjectW(attributes: null, name: null): NativePtr - setInformationJobObject(job: NativePtr, cls: number, information: Buffer, length: number): number - assignProcessToJobObject(job: NativePtr, process: NativePtr): number - // Terminate a suspended child that could not be placed in the kill-on-close - // job — closing handles alone would leave it hanging forever. - terminateProcess(process: NativePtr, exitCode: number): number - // ---- console ------------------------------------------------------------- - // HandlerRoutine=null + add=1 makes this process ignore CTRL+C (wincon.h): - // the runner survives console Ctrl+C so the child handles its own and the - // runner can clean up grants after the child exits. - setConsoleCtrlHandler(handler: null, add: number): number - getStdHandle(stdHandle: number): NativePtr -} - -const PVOID: Ptr = koffi.pointer('void') -const PPVOID: Ptr = koffi.pointer(PVOID) - -/** koffi STARTUPINFOW layout; its size is asserted against abi.STARTUPINFOW_SIZE at load. */ -export const STARTUPINFOW = koffi.struct('STARTUPINFOW', { - cb: 'uint32', - lpReserved: 'str16', - lpDesktop: 'str16', - lpTitle: 'str16', - dwX: 'uint32', - dwY: 'uint32', - dwXSize: 'uint32', - dwYSize: 'uint32', - dwXCountChars: 'uint32', - dwYCountChars: 'uint32', - dwFillAttribute: 'uint32', - dwFlags: 'uint32', - wShowWindow: 'uint16', - cbReserved2: 'uint16', - lpReserved2: koffi.pointer('uint8'), - hStdInput: PVOID, - hStdOutput: PVOID, - hStdError: PVOID, -}) - -/** koffi PROCESS_INFORMATION layout; its size is asserted against abi.PROCESS_INFORMATION_SIZE at load. */ -export const PROCESS_INFORMATION = koffi.struct('PROCESS_INFORMATION', { - hProcess: PVOID, - hThread: PVOID, - dwProcessId: 'uint32', - dwThreadId: 'uint32', -}) - -/* v8 ignore start -- layout-mismatch guards fire only on ABI breakage; verify/abi-probe.cpp pins both sizes. */ -if (STARTUPINFOW.size !== abi.STARTUPINFOW_SIZE) { - throw new Error(`STARTUPINFOW layout mismatch: koffi computed ${STARTUPINFOW.size}, header probe says ${abi.STARTUPINFOW_SIZE}`) -} -if (PROCESS_INFORMATION.size !== abi.PROCESS_INFORMATION_SIZE) { - throw new Error(`PROCESS_INFORMATION layout mismatch: koffi computed ${PROCESS_INFORMATION.size}, header probe says ${abi.PROCESS_INFORMATION_SIZE}`) -} -/* v8 ignore stop */ - /** - * Allocate one pointer-sized slot (for `T **` out-parameters). - * @returns the allocated slot pointer. - */ -export function allocPtrSlot(): NativePtr { - const value: unknown = koffi.alloc(PVOID, 1) - return value as NativePtr -} - -/** - * Allocate one uint32 slot. - * @returns the allocated slot pointer. - */ -export function allocUint32(): NativePtr { - const value: unknown = koffi.alloc('uint32', 1) - return value as NativePtr -} - -/** - * Write a uint32 value into a slot pointer. - * @param slot - the slot allocated by {@link allocUint32}. - * @param value - the uint32 to encode. + * Encode a uint32 into an allocated slot. + * @param slot - slot allocated by allocUint32. + * @param value - unsigned value to store. */ export function encodeUint32(slot: NativePtr, value: number): void { koffi.encode(slot, 'uint32', value) } /** - * Decode the pointer stored in a pointer-sized slot (NULL becomes null). - * @param slot - the pointer-sized slot holding the out-parameter value. - * @returns the decoded pointer, or null for NULL. - */ -export function decodePtr(slot: NativePtr): NativePtr | null { - const value: unknown = koffi.decode(slot, PVOID) - if (isNullPtr(value as NativePtr | null | undefined)) return null - return value as NativePtr -} - -/** - * Decode a uint32 at a slot pointer. - * @param slot - the uint32 slot holding the out-parameter value. - * @returns the decoded uint32. - */ -export function decodeUint32(slot: NativePtr): number { - const value: unknown = koffi.decode(slot, 'uint32') - return value as number -} - -/** - * Cast a koffi pointer to its numeric address (bigint, used for raw struct packing). - * @param ptr - the koffi pointer. - * @returns the pointer's numeric address. + * Return a Koffi pointer's numeric address for struct packing. + * @param ptr - native pointer. + * @returns pointer address. */ export function ptrAddress(ptr: NativePtr): bigint { return koffi.address(ptr) } /** - * Allocate a raw byte block (used for SID copies and variable-length arrays). - * @param length - the block size in bytes. - * @returns the allocated block pointer. + * Allocate a raw byte block. + * @param length - byte count. + * @returns allocated pointer. */ export function allocBytes(length: number): NativePtr { - const value: unknown = koffi.alloc('uint8', length) - return value as NativePtr + return koffi.alloc('uint8', length) as NativePtr } /** - * Allocate one zeroed OVERLAPPED (32 bytes on x64: Internal@0, InternalHigh@8, - * Offset@16, OffsetHigh@20, hEvent@24). LockFileEx/UnlockFileEx receive this - * instead of a NULL lpOverlapped: koffi 3.1.1 crashes on NULL there, and a - * zeroed OVERLAPPED on a synchronous file handle is the documented equivalent - * (the byte range locks from offset 0, hEvent stays NULL). - * @returns the zeroed block pointer. + * Allocate one zeroed x64 OVERLAPPED record. + * @returns allocated pointer. + * @remarks Koffi 3.1.1 crashes when LockFileEx or UnlockFileEx receives NULL; + * a zeroed OVERLAPPED is equivalent for the synchronous lock-file handle. */ export function allocOverlapped(): NativePtr { return allocBytes(32) } /** - * Decode a pointer VALUE stored in memory at `buffer[offset]` (e.g. TOKEN_GROUPS entries). - * @param buffer - the buffer holding the pointer value. - * @param offset - byte offset of the pointer inside the buffer. - * @returns the decoded pointer, or null for NULL. + * Decode a pointer value from a Buffer field. + * @param buffer - encoded native record. + * @param offset - pointer field byte offset. + * @returns decoded pointer, or null for address zero. */ export function decodePtrAt(buffer: Buffer, offset: number): NativePtr | null { - const value: unknown = koffi.decode(buffer, offset, PVOID) - if (isNullPtr(value as NativePtr | null | undefined)) return null - return value as NativePtr + const value = koffi.decode(buffer, offset, PVOID) as NativePtr | null + return isNullPtr(value) ? null : value } /** - * Decode a uint8 at a native pointer plus byte offset — the ACL walk's - * field-read primitive (koffi.decode with an offset, no memcpy, no pointer - * arithmetic). - * @param ptr - the native pointer to read from. - * @param offset - byte offset from the pointer. - * @returns the decoded uint8. + * Decode a uint8 field at a native pointer offset. + * @param ptr - native record pointer. + * @param offset - field byte offset. + * @returns decoded value. */ export function decodeUint8At(ptr: NativePtr, offset: number): number { - const value: unknown = koffi.decode(ptr, offset, 'uint8') - return value as number + return koffi.decode(ptr, offset, 'uint8') as number } /** - * Decode a uint16 at a native pointer plus byte offset (see {@link decodeUint8At}). - * @param ptr - the native pointer to read from. - * @param offset - byte offset from the pointer. - * @returns the decoded uint16. + * Decode a uint16 field at a native pointer offset. + * @param ptr - native record pointer. + * @param offset - field byte offset. + * @returns decoded value. */ export function decodeUint16At(ptr: NativePtr, offset: number): number { - const value: unknown = koffi.decode(ptr, offset, 'uint16') - return value as number + return koffi.decode(ptr, offset, 'uint16') as number } /** - * Decode a uint32 at a native pointer plus byte offset (see {@link decodeUint8At}). - * @param ptr - the native pointer to read from. - * @param offset - byte offset from the pointer. - * @returns the decoded uint32. + * Decode a uint32 field at a native pointer offset. + * @param ptr - native record pointer. + * @param offset - field byte offset. + * @returns decoded value. */ export function decodeUint32At(ptr: NativePtr, offset: number): number { - const value: unknown = koffi.decode(ptr, offset, 'uint32') - return value as number + return koffi.decode(ptr, offset, 'uint32') as number } /** - * Compare two SIDs field-by-field via BOUNDED offset reads (revision, count, - * identifier authority, subauthorities up to the count) — never a fixed-size - * struct decode, which would read past a short SID allocation (a SID with - * fewer than 8 subauthorities is smaller than `SID_STRUCT`). An implausible - * subauthority count reads as unequal. - * @param left - pointer to one SID (offset 0). - * @param leftOffset - byte offset of the SID structure within `left`. - * @param right - pointer to the other SID. - * @param rightOffset - byte offset of the SID structure within `right`. - * @returns whether the SIDs are identical. + * Compare two in-memory SID records without allocating strings. + * @param left - first native buffer. + * @param leftOffset - first SID byte offset. + * @param right - second native buffer. + * @param rightOffset - second SID byte offset. + * @returns true when revision, authority, and every sub-authority match. */ -export function sameSidAt(left: NativePtr, leftOffset: number, right: NativePtr, rightOffset: number): boolean { - const leftRevision = decodeUint8At(left, leftOffset) - const rightRevision = decodeUint8At(right, rightOffset) - if (leftRevision !== rightRevision) return false +export function sameSidAt( + left: NativePtr, + leftOffset: number, + right: NativePtr, + rightOffset: number, +): boolean { + if (decodeUint8At(left, leftOffset) !== decodeUint8At(right, rightOffset)) return false const leftCount = decodeUint8At(left, leftOffset + 1) const rightCount = decodeUint8At(right, rightOffset + 1) if (leftCount !== rightCount || leftCount > abi.SID_MAX_SUB_AUTHORITIES) return false - for (let index = 0; index < 6; index++) { - if (decodeUint8At(left, leftOffset + 2 + index) !== decodeUint8At(right, rightOffset + 2 + index)) return false + for (let index = 0; index < 6; index += 1) { + if (decodeUint8At(left, leftOffset + 2 + index) !== decodeUint8At(right, rightOffset + 2 + index)) { + return false + } } - for (let index = 0; index < leftCount; index++) { - if (decodeUint32At(left, leftOffset + 8 + index * 4) !== decodeUint32At(right, rightOffset + 8 + index * 4)) return false + for (let index = 0; index < leftCount; index += 1) { + if (decodeUint32At(left, leftOffset + 8 + index * 4) !== + decodeUint32At(right, rightOffset + 8 + index * 4)) return false } return true } -/** - * Allocate a zeroed STARTUPINFOW. - * @returns the allocated struct pointer. - */ -export function allocStartupInfo(): NativePtr { - const value: unknown = koffi.alloc(STARTUPINFOW, 1) - return value as NativePtr -} - -/** - * Write the stdio-relevant fields into a zeroed STARTUPINFOW (others stay default-initialized). - * @param startupInfo - the allocated STARTUPINFOW to encode into. - * @param fields - the field subset to write. - */ -export function encodeStartupInfo(startupInfo: NativePtr, fields: StartupInfoInput): void { - koffi.encode(startupInfo, STARTUPINFOW, fields) -} - -/** - * Allocate a zeroed PROCESS_INFORMATION. - * @returns the allocated struct pointer. - */ -export function allocProcessInfo(): NativePtr { - const value: unknown = koffi.alloc(PROCESS_INFORMATION, 1) - return value as NativePtr -} - -/** - * Decode a PROCESS_INFORMATION after CreateProcessAsUserW. - * @param processInfo - the PROCESS_INFORMATION filled by the spawn call. - * @returns the decoded handle/id fields. - */ -export function decodeProcessInfo(processInfo: NativePtr): ProcessInfoOutput { - const value: unknown = koffi.decode(processInfo, PROCESS_INFORMATION) - return value as ProcessInfoOutput -} - let cached: Win32Bindings | undefined function bindings(): Win32Bindings { if (cached !== undefined) return cached - const kernel32 = koffi.load('kernel32.dll') - const advapi32 = koffi.load('advapi32.dll') - - // Each binding shape is verified by verify/abi-probe.cpp against the real - // Windows headers and exercised end-to-end by tests/probe.spec.ts; the - // single cast keeps the per-binding noise out of this table. - const bind = (lib: ReturnType, name: string, result: Ptr | string, args: Array): unknown => - lib.func('__stdcall', name, result, args) - - cached = { + cached = extendWin32ProcessBindings(({ kernel32, advapi32, bind }) => ({ openProcess: bind(kernel32, 'OpenProcess', PVOID, ['uint32', 'int', 'uint32']), openProcessToken: bind(advapi32, 'OpenProcessToken', 'int', [PVOID, 'uint32', PPVOID]), - closeHandle: bind(kernel32, 'CloseHandle', 'int', [PVOID]), - getLastError: bind(kernel32, 'GetLastError', 'uint32', []), - formatMessageW: bind(kernel32, 'FormatMessageW', 'uint32', ['uint32', PVOID, 'uint32', 'uint32', PVOID, 'uint32', PVOID]), localAlloc: bind(kernel32, 'LocalAlloc', PVOID, ['uint32', 'size_t']), localFree: bind(kernel32, 'LocalFree', PVOID, [PVOID]), convertStringSidToSidW: bind(advapi32, 'ConvertStringSidToSidW', 'int', ['str16', PPVOID]), - createWellKnownSid: bind(advapi32, 'CreateWellKnownSid', 'int', ['int', PVOID, PVOID, koffi.pointer('uint32')]), + createWellKnownSid: bind(advapi32, 'CreateWellKnownSid', 'int', [ + 'int', PVOID, PVOID, koffi.pointer('uint32'), + ]), isValidSid: bind(advapi32, 'IsValidSid', 'int', [PVOID]), getLengthSid: bind(advapi32, 'GetLengthSid', 'uint32', [PVOID]), copySid: bind(advapi32, 'CopySid', 'int', ['uint32', PVOID, PVOID]), - getTokenInformation: bind(advapi32, 'GetTokenInformation', 'int', [PVOID, 'int', PVOID, 'uint32', koffi.pointer('uint32')]), - setTokenInformation: bind(advapi32, 'SetTokenInformation', 'int', [PVOID, 'int', PVOID, 'uint32']), - createRestrictedToken: bind(advapi32, 'CreateRestrictedToken', 'int', [PVOID, 'uint32', 'uint32', PVOID, 'uint32', PVOID, 'uint32', PVOID, PPVOID]), - setEntriesInAclW: bind(advapi32, 'SetEntriesInAclW', 'uint32', ['uint32', PVOID, PVOID, PPVOID]), - setNamedSecurityInfoW: bind(advapi32, 'SetNamedSecurityInfoW', 'uint32', ['str16', 'int', 'uint32', PVOID, PVOID, PVOID, PVOID]), - getNamedSecurityInfoW: bind(advapi32, 'GetNamedSecurityInfoW', 'uint32', ['str16', 'int', 'uint32', PPVOID, PPVOID, PPVOID, PPVOID, PPVOID]), - getTempPathW: bind(kernel32, 'GetTempPathW', 'uint32', ['uint32', PVOID]), - // fileapi.h line ~64: HANDLE CreateFileW(LPCWSTR, DWORD, DWORD, - // LPSECURITY_ATTRIBUTES, DWORD, DWORD, HANDLE). - createFileW: bind(kernel32, 'CreateFileW', PVOID, ['str16', 'uint32', 'uint32', PVOID, 'uint32', 'uint32', PVOID]), - // fileapi.h lines ~177/~185: BOOL LockFileEx(HANDLE, DWORD, DWORD, DWORD, - // DWORD, LPOVERLAPPED); BOOL UnlockFileEx(HANDLE, DWORD, DWORD, DWORD, - // LPOVERLAPPED). lpOverlapped is NULL for synchronous locking. - lockFileEx: bind(kernel32, 'LockFileEx', 'int', [PVOID, 'uint32', 'uint32', 'uint32', 'uint32', PVOID]), - unlockFileEx: bind(kernel32, 'UnlockFileEx', 'int', [PVOID, 'uint32', 'uint32', 'uint32', PVOID]), - createPipe: bind(kernel32, 'CreatePipe', 'int', [PPVOID, PPVOID, PVOID, 'uint32']), - setHandleInformation: bind(kernel32, 'SetHandleInformation', 'int', [PVOID, 'uint32', 'uint32']), - createProcessAsUserW: bind(advapi32, 'CreateProcessAsUserW', 'int', [ - PVOID, 'str16', 'str16', PVOID, PVOID, 'int', 'uint32', PVOID, 'str16', - koffi.pointer(STARTUPINFOW), koffi.pointer(PROCESS_INFORMATION), + getTokenInformation: bind(advapi32, 'GetTokenInformation', 'int', [ + PVOID, 'int', PVOID, 'uint32', koffi.pointer('uint32'), ]), + setTokenInformation: bind(advapi32, 'SetTokenInformation', 'int', [PVOID, 'int', PVOID, 'uint32']), + createRestrictedToken: bind(advapi32, 'CreateRestrictedToken', 'int', [ + PVOID, 'uint32', 'uint32', PVOID, 'uint32', PVOID, 'uint32', PVOID, PPVOID, + ]), + setEntriesInAclW: bind(advapi32, 'SetEntriesInAclW', 'uint32', ['uint32', PVOID, PVOID, PPVOID]), + setNamedSecurityInfoW: bind(advapi32, 'SetNamedSecurityInfoW', 'uint32', [ + 'str16', 'int', 'uint32', PVOID, PVOID, PVOID, PVOID, + ]), + getNamedSecurityInfoW: bind(advapi32, 'GetNamedSecurityInfoW', 'uint32', [ + 'str16', 'int', 'uint32', PPVOID, PPVOID, PPVOID, PPVOID, PPVOID, + ]), + getTempPathW: bind(kernel32, 'GetTempPathW', 'uint32', ['uint32', PVOID]), setEnvironmentVariableW: bind(kernel32, 'SetEnvironmentVariableW', 'int', ['str16', 'str16']), - readFile: bind(kernel32, 'ReadFile', 'int', [PVOID, PVOID, 'uint32', koffi.pointer('uint32'), PVOID]), - peekNamedPipe: bind(kernel32, 'PeekNamedPipe', 'int', [PVOID, PVOID, 'uint32', koffi.pointer('uint32'), koffi.pointer('uint32'), koffi.pointer('uint32')]), - waitForSingleObject: bind(kernel32, 'WaitForSingleObject', 'uint32', [PVOID, 'uint32']), - getExitCodeProcess: bind(kernel32, 'GetExitCodeProcess', 'int', [PVOID, koffi.pointer('uint32')]), - resumeThread: bind(kernel32, 'ResumeThread', 'uint32', [PVOID]), - createJobObjectW: bind(kernel32, 'CreateJobObjectW', PVOID, [PVOID, 'str16']), - setInformationJobObject: bind(kernel32, 'SetInformationJobObject', 'int', [PVOID, 'int', PVOID, 'uint32']), - assignProcessToJobObject: bind(kernel32, 'AssignProcessToJobObject', 'int', [PVOID, PVOID]), - terminateProcess: bind(kernel32, 'TerminateProcess', 'int', [PVOID, 'uint32']), setConsoleCtrlHandler: bind(kernel32, 'SetConsoleCtrlHandler', 'int', [PVOID, 'int']), - getStdHandle: bind(kernel32, 'GetStdHandle', PVOID, ['int']), - } as unknown as Win32Bindings + createFileW: bind(kernel32, 'CreateFileW', PVOID, [ + 'str16', 'uint32', 'uint32', PVOID, 'uint32', 'uint32', PVOID, + ]), + lockFileEx: bind(kernel32, 'LockFileEx', 'int', [ + PVOID, 'uint32', 'uint32', 'uint32', 'uint32', PVOID, + ]), + unlockFileEx: bind(kernel32, 'UnlockFileEx', 'int', [ + PVOID, 'uint32', 'uint32', 'uint32', PVOID, + ]), + })) as unknown as Win32Bindings return cached } /** - * Resolve the lazy Win32 bindings (throws the first binding failure, fail-closed). - * @returns the cached binding table. + * Resolve the cached ACL/token binding table asynchronously. + * @returns generic process plus ACL/token bindings. */ export function win32(): Promise { return Promise.resolve(bindings()) } /** - * Resolve the lazy Win32 bindings SYNCHRONOUSLY — the sandbox seam's - * server-side per-session grant materializes ACEs inside the synchronous - * `confine()` call, which cannot await. Same cached table as {@link win32} - * (the underlying koffi loads are synchronous; the async wrapper exists for - * the runner's await-shaped call sites). - * @returns the cached binding table. + * Resolve the cached ACL/token binding table synchronously. + * @returns generic process plus ACL/token bindings. */ export function win32Sync(): Win32Bindings { return bindings() } /** - * Turn a Win32 error code into readable text via FormatMessageW. - * @param api - the binding table. - * @param win32Code - the error code to format. - * @returns the formatted message text, or '' when formatting fails. - */ -export function errorText(api: Win32Bindings, win32Code: number): string { - const buffer = Buffer.alloc(1024) - const length = api.formatMessageW( - abi.FORMAT_MESSAGE_FROM_SYSTEM | abi.FORMAT_MESSAGE_IGNORE_INSERTS, - null, win32Code, 0, buffer, buffer.length / 2, null, - ) - if (length === 0) return '' - return buffer.subarray(0, length * 2).toString('utf16le').trim() -} - -/** - * Read the process temp directory via GetTempPathW (fileapi.h line ~188). - * Defensive against an overlong system temp path: GetTempPathW reports the - * REQUIRED length (including NUL) without writing the buffer when it is too - * small, so a reported length beyond the buffer's capacity means the buffer - * was never filled and must not be decoded. - * @param api - the binding table. - * @returns the NUL-terminated temp path decoded as a string. + * Resolve the current Windows temporary directory. + * @param api - active ACL/token binding table. + * @returns UTF-16 path reported by GetTempPathW. */ export function getTempPath(api: Win32Bindings): string { const buffer = Buffer.alloc((abi.MAX_PATH + 1) * 2) const length = api.getTempPathW(buffer.length / 2, buffer) if (length === 0) throwLastError(api, 'GetTempPathW') if (length > buffer.length / 2) { - throw new Win32Error('GetTempPathW', abi.ERROR_INSUFFICIENT_BUFFER, `required ${length} chars exceed the ${buffer.length / 2}-char buffer; nothing was written`) + throw new Win32Error( + 'GetTempPathW', + ERROR_INSUFFICIENT_BUFFER, + `required ${length} chars exceed the ${buffer.length / 2}-char buffer; nothing was written`, + ) } return buffer.subarray(0, length * 2).toString('utf16le') } - -/** - * Throw a Win32Error for a BOOL-style API failure. MUST be called immediately - * after the failed call so GetLastError is not clobbered by other Win32 calls. - * @param api - the binding table. - * @param name - the failed API's name for the error message. - * @param detail - optional detail overriding the formatted system message. - * @returns never — always throws. - */ -export function throwLastError(api: Win32Bindings, name: string, detail?: string): never { - const win32Code = api.getLastError() - throw new Win32Error(name, win32Code, detail ?? errorText(api, win32Code)) -} - -/** - * Throw a Win32Error for an HRESULT-style API return value (the value IS the error code). - * @param api - the binding table. - * @param name - the failed API's name for the error message. - * @param win32Code - the API's returned error code. - * @param detail - optional detail overriding the formatted system message. - * @returns never — always throws. - */ -export function throwWin32(api: Win32Bindings, name: string, win32Code: number, detail?: string): never { - throw new Win32Error(name, win32Code, detail ?? errorText(api, win32Code)) -} diff --git a/packages/sandbox/sandbox-windows-acl/src/index.ts b/packages/sandbox/sandbox-windows-acl/src/index.ts index cf304b1904..5ebdccbd6a 100644 --- a/packages/sandbox/sandbox-windows-acl/src/index.ts +++ b/packages/sandbox/sandbox-windows-acl/src/index.ts @@ -42,9 +42,9 @@ import { existsSync, statSync } from 'node:fs' import { resolve } from 'node:path' +import { Win32Error } from '@deepseek-ai/dsh-win32-process' import { grantWrite, revokeWrite } from './acl.ts' -import { Win32Error } from './errors.ts' import { allocPtrSlot, decodePtr, isNullPtr, throwLastError, win32 } from './ffi.ts' import type { NativePtr, Win32Bindings } from './ffi.ts' import { assertPrivateTempDisjoint } from './path-boundary.ts' @@ -52,12 +52,9 @@ import { drainPipe, spawnSandboxed, spawnSandboxedInherited, waitForExit } from import { createRestrictedToken, findLogonSid, makeWellKnownSid, openCurrentProcessToken, setTokenDefaultDaclGrant } from './token.ts' import * as abi from './win32-abi.ts' -export { quoteArg } from './spawn.ts' export { AclWriteGrant } from './grant.ts' export { assertTempRootOutsideWorkspace } from './path-boundary.ts' export { tempWriteSid, workspaceWriteSid } from './workspace-sid.ts' -export { Win32Error } from './errors.ts' - /** Construction options: the workspace/temp allowlists and their distinct SID identities. */ export interface AclSandboxOptions { /** Directories the confined child may write into (must exist and be caller-owned). */ diff --git a/packages/sandbox/sandbox-windows-acl/src/spawn.ts b/packages/sandbox/sandbox-windows-acl/src/spawn.ts index eafcb252ce..a36b0253c4 100644 --- a/packages/sandbox/sandbox-windows-acl/src/spawn.ts +++ b/packages/sandbox/sandbox-windows-acl/src/spawn.ts @@ -1,357 +1,60 @@ -/** - * Restricted-process spawning: anonymous pipes for stdio, STARTUPINFOW with - * STARTF_USESTDHANDLES, CreateProcessAsUserW under the restricted token, then - * asynchronous pipe draining and exit waiting. Console isolation - * (CREATE_NO_WINDOW / CREATE_NEW_CONSOLE) is intentionally absent: under this - * restriction scheme hidden-console children die with STATUS_DLL_INIT_FAILED - * (0xC0000142) — verified empirically, see win32-abi.ts. Stdio redirection is - * pipe-based and unaffected; the child shares the host console. - * @module @deepseek-ai/dsh-sandbox-windows-acl/spawn - */ +/** Restricted-token adapters over the shared Win32 process owner. */ -import { allocPtrSlot, allocProcessInfo, allocStartupInfo, allocUint32, decodePtr, decodeProcessInfo, decodeUint32, encodeStartupInfo, isNullPtr, throwLastError, throwWin32 } from './ffi.ts' -import type { NativePtr, Win32Bindings } from './ffi.ts' -import * as abi from './win32-abi.ts' +import { + spawnInheritedJobProcess, + spawnPipedProcess, + waitForProcessExit, +} from '@deepseek-ai/dsh-win32-process' +import type { + NativePtr, + SpawnedJobProcess, + SpawnedPipedProcess, +} from '@deepseek-ai/dsh-win32-process' +import type { Win32Bindings } from './ffi.ts' + +export { drainPipe } from '@deepseek-ai/dsh-win32-process' + +/** Restricted-token child with piped stdio resources. */ +export interface SpawnedNative extends SpawnedPipedProcess {} +/** Restricted-token child assigned to a kill-on-close Job. */ +export interface SpawnedInherited extends SpawnedJobProcess {} /** - * Quote one argument per the CommandLineToArgvW parsing rules: backslashes - * are doubled only before a quote character — including the closing quote - * this function appends, so a trailing backslash run is doubled as well - * (otherwise an odd run would escape the closing quote into a literal - * character and corrupt the rest of the command line). Mirrors the CRT - * ArgvQuote behavior Microsoft documents for command-line arguments. - * @param argument - one argv entry to quote. - * @returns the quoted entry (bare when quoting is unnecessary). - */ -export function quoteArg(argument: string): string { - if (argument === '') return '""' - if (!/[\s"]/u.test(argument)) return argument - let quoted = '"' - for (let index = 0; index < argument.length; index++) { - let backslashes = 0 - while (index < argument.length && argument.charAt(index) === '\\') { - backslashes++ - index++ - } - if (index === argument.length) { - // Trailing backslash run: doubled so it cannot escape the closing quote. - quoted += '\\'.repeat(backslashes * 2) - } else if (argument.charAt(index) === '"') { - quoted += '\\'.repeat(backslashes * 2 + 1) + '"' - } else { - quoted += '\\'.repeat(backslashes) + argument.charAt(index) - } - } - return quoted + '"' -} - -/** - * Build the single command line CreateProcess parses from program + argv. - * @param program - the executable (argv[0]). - * @param args - the remaining argv entries. - * @returns the joined, quoted command line. - */ -export function buildCommandLine(program: string, args: readonly string[]): string { - return [program, ...args].map(quoteArg).join(' ') -} - -interface PipePair { - read: NativePtr - write: NativePtr -} - -function createPipe(api: Win32Bindings): PipePair { - const readSlot = allocPtrSlot() - const writeSlot = allocPtrSlot() - if (api.createPipe(readSlot, writeSlot, null, 0) === 0) throwLastError(api, 'CreatePipe') - const read = decodePtr(readSlot) - const write = decodePtr(writeSlot) - if (read === null || write === null) throwLastError(api, 'CreatePipe', 'null pipe handle') - return { read, write } -} - -function setInheritable(api: Win32Bindings, handle: NativePtr, label: string): void { - if (api.setHandleInformation(handle, abi.HANDLE_FLAG_INHERIT, abi.HANDLE_FLAG_INHERIT) === 0) { - throwLastError(api, 'SetHandleInformation', label) - } -} - -/** A confined child spawned with piped stdio: process handle plus the pipe read ends to drain. */ -export interface SpawnedNative { - pid: number - process: NativePtr - stdoutRead: NativePtr - stderrRead: NativePtr -} - -/** - * Create a process under the restricted token with piped stdio. The child's - * stdin is closed immediately (EOF), matching the POC; stdout/stderr read ends - * are returned for draining. The child inherits the caller's environment block - * (lpEnvironment NULL); the caller rewrites entries through - * SetEnvironmentVariableW before spawning (the runner's per-session temp - * contract) — passing an explicit block through koffi trips - * ERROR_INVALID_PARAMETER in CreateProcessAsUserW (verified empirically). - * @param api - the binding table. - * @param token - the restricted token the child runs under. + * Spawn a restricted-token child with piped stdout/stderr. + * @param api - ACL/token binding table. + * @param token - restricted primary token. * @param options - command, args, and working directory. - * @returns the spawned child's handles. + * @returns process and caller-owned pipe handles. */ export function spawnSandboxed( api: Win32Bindings, token: NativePtr, options: { command: string; args: readonly string[]; cwd: string }, ): SpawnedNative { - const stdIn = createPipe(api) - const stdOut = createPipe(api) - const stdErr = createPipe(api) - // Child side of each pipe must be inheritable (POC lines 262-268). - setInheritable(api, stdIn.read, 'stdin read end') - setInheritable(api, stdOut.write, 'stdout write end') - setInheritable(api, stdErr.write, 'stderr write end') - - const startupInfo = allocStartupInfo() - encodeStartupInfo(startupInfo, { - cb: abi.STARTUPINFOW_SIZE, - dwFlags: abi.STARTF_USESTDHANDLES, - hStdInput: stdIn.read, - hStdOutput: stdOut.write, - hStdError: stdErr.write, - }) - - const processInfo = allocProcessInfo() - const commandLine = buildCommandLine(options.command, options.args) - const created = api.createProcessAsUserW( - token, null, commandLine, - null, null, - 1, // bInheritHandles: required for redirection - 0, // no creation flags: suspended/no-window variants are unusable under the restriction - null, options.cwd, - startupInfo, processInfo, - ) - // Capture the failure before CloseHandle calls clobber GetLastError, then - // close every pipe handle created so far — the six-close contract this test - // surface pins (tests/failure-paths.spec.ts). - if (created === 0) { - const win32Code = api.getLastError() - api.closeHandle(stdIn.read) - api.closeHandle(stdIn.write) - api.closeHandle(stdOut.read) - api.closeHandle(stdOut.write) - api.closeHandle(stdErr.read) - api.closeHandle(stdErr.write) - throwWin32(api, 'CreateProcessAsUserW', win32Code, `command: ${options.command}, cwd: ${options.cwd}`) - } - - const info = decodeProcessInfo(processInfo) - const processHandle = info.hProcess - const threadHandle = info.hThread - if (processHandle === null || threadHandle === null) { - throw new Error(`CreateProcessAsUserW succeeded but returned null process/thread handles (pid ${info.dwProcessId})`) - } - - // Host-side cleanup: child handles are now duplicated in the child; the - // host closes its copies so ReadFile sees EOF when the child exits. - api.closeHandle(stdIn.read) - api.closeHandle(stdOut.write) - api.closeHandle(stdErr.write) - api.closeHandle(stdIn.write) - api.closeHandle(threadHandle) - - return { - pid: info.dwProcessId, - process: processHandle, - stdoutRead: stdOut.read, - stderrRead: stdErr.read, - } + return spawnPipedProcess(api, { ...options, token }) } /** - * Drain one pipe read end to a Buffer via non-blocking PeekNamedPipe polling. - * @param api - the binding table. - * @param handle - the pipe read end to drain (closed when done). - * @returns the complete pipe contents. - */ -export async function drainPipe(api: Win32Bindings, handle: NativePtr): Promise { - const chunks: Buffer[] = [] - for (;;) { - const bytesReadSlot = allocUint32() - const totalAvailSlot = allocUint32() - const leftThisMessageSlot = allocUint32() - const peeked = api.peekNamedPipe(handle, null, 0, bytesReadSlot, totalAvailSlot, leftThisMessageSlot) - if (peeked === 0) { - const win32Code = api.getLastError() - if (win32Code === abi.ERROR_BROKEN_PIPE || win32Code === abi.ERROR_NO_DATA) break // child closed its end: clean EOF - throwLastError(api, 'PeekNamedPipe', `drain failure after ${chunks.length} chunk(s)`) - } - const available = decodeUint32(totalAvailSlot) - if (available > 0) { - const chunk = Buffer.alloc(available) - const readSlot = allocUint32() - if (api.readFile(handle, chunk, chunk.length, readSlot, null) === 0) { - throwLastError(api, 'ReadFile', `drain failure after ${chunks.length} chunk(s)`) - } - chunks.push(chunk.subarray(0, decodeUint32(readSlot))) - } - // Small backoff instead of setImmediate: a bare next-tick would busy-poll - // the pipe at full event-loop speed while the child produces no output. - await new Promise(resolve => setTimeout(resolve, 1)) - } - api.closeHandle(handle) - return Buffer.concat(chunks) -} - -/** - * Wait for process exit and return its exit code. Call only after both drains - * have resolved — the drains finish when the child closed its pipe ends, i.e. - * the child has already exited, so this wait returns immediately. Calling it - * earlier would block the event loop and starve the drains (the pipe-buffer - * deadlock the POC comments warn about). - * @param api - the binding table. - * @param process - the child process handle (closed when done). - * @returns the child's exit code. - */ -export function waitForExit(api: Win32Bindings, process: NativePtr): number { - const waitResult = api.waitForSingleObject(process, abi.INFINITE) - if (waitResult === 0xFFFFFFFF) throwLastError(api, 'WaitForSingleObject') - const exitCodeSlot = allocUint32() - if (api.getExitCodeProcess(process, exitCodeSlot) === 0) throwLastError(api, 'GetExitCodeProcess') - api.closeHandle(process) - return decodeUint32(exitCodeSlot) -} - -/** - * Create a kill-on-close job object (JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE at - * LimitFlags offset 16 of JOBOBJECT_EXTENDED_LIMIT_INFORMATION, layout - * verified by abi-probe.cpp). When the caller dies with the job handle open, - * Windows terminates every process in the job — the orphan-child backstop. - * The caller keeps the returned handle open for the child's lifetime. - */ -function createKillOnCloseJob(api: Win32Bindings): NativePtr { - const job = api.createJobObjectW(null, null) - if (isNullPtr(job)) throwLastError(api, 'CreateJobObjectW') - const information = Buffer.alloc(abi.JOBOBJECT_EXTENDED_LIMIT_SIZE) - information.writeUInt32LE(abi.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, abi.JOBOBJECT_EXTENDED_LIMIT_FLAGS_OFFSET) - if (api.setInformationJobObject(job, abi.JobObjectExtendedLimitInformation, information, information.length) === 0) { - const win32Code = api.getLastError() - api.closeHandle(job) - throwWin32(api, 'SetInformationJobObject', win32Code) - } - return job -} - -/** A confined child spawned with inherited stdio: process handle plus its kill-on-close job. */ -export interface SpawnedInherited { - pid: number - process: NativePtr - /** Kill-on-close job the child was placed in; caller closes it after the child exits. */ - job: NativePtr -} - -/** - * Create a process under the restricted token whose stdio passes straight - * through to the caller's pipes. This is the runner shape: the harness spawns - * the runner with piped stdio, and the runner's confined child writes to - * those same pipes. - * - * Node clears the inheritability of its stdio handles at startup - * (uv_disable_stdio_inheritance), so raw spawns must re-enable the inherit - * bit around the call (libuv instead duplicates the handles; re-enabling is - * equivalent here and cheaper) and pass them explicitly via - * STARTF_USESTDHANDLES — otherwise the child receives INVALID std handles - * ("The handle is invalid", verified the hard way). The child starts - * suspended so it can be assigned to a kill-on-close job before it runs. - * @param api - the binding table. - * @param token - the restricted token the child runs under. + * Spawn a restricted-token child in a kill-on-close Job with inherited stdio. + * @param api - ACL/token binding table. + * @param token - restricted primary token. * @param options - command, args, and working directory. - * @returns the spawned child's handles and job. + * @returns process and Job handles after assignment and resume. */ export function spawnSandboxedInherited( api: Win32Bindings, token: NativePtr, options: { command: string; args: readonly string[]; cwd: string }, ): SpawnedInherited { - const job = createKillOnCloseJob(api) - const stdIn = api.getStdHandle(abi.STD_INPUT_HANDLE) - const stdOut = api.getStdHandle(abi.STD_OUTPUT_HANDLE) - const stdErr = api.getStdHandle(abi.STD_ERROR_HANDLE) - if (isNullPtr(stdIn) || isNullPtr(stdOut) || isNullPtr(stdErr)) { - api.closeHandle(job) - throwLastError(api, 'GetStdHandle', 'null standard handle') - } - - const makeInheritable = (handle: NativePtr, label: string): void => { - if (api.setHandleInformation(handle, abi.HANDLE_FLAG_INHERIT, abi.HANDLE_FLAG_INHERIT) === 0) { - throwLastError(api, 'SetHandleInformation', `${label} (enable inherit)`) - } - } - const restoreInherit = (handle: NativePtr): void => { - // Best-effort hygiene: the runner spawns nothing else; failures here must - // not mask the child outcome, so the result is deliberately unchecked. - api.setHandleInformation(handle, abi.HANDLE_FLAG_INHERIT, 0) - } - makeInheritable(stdIn, 'stdin') - makeInheritable(stdOut, 'stdout') - makeInheritable(stdErr, 'stderr') - - const startupInfo = allocStartupInfo() - encodeStartupInfo(startupInfo, { - cb: abi.STARTUPINFOW_SIZE, - dwFlags: abi.STARTF_USESTDHANDLES, - hStdInput: stdIn, - hStdOutput: stdOut, - hStdError: stdErr, - }) - - const processInfo = allocProcessInfo() - const commandLine = buildCommandLine(options.command, options.args) - const created = api.createProcessAsUserW( - token, null, commandLine, - null, null, - 1, // bInheritHandles: the re-enabled std handles must be inheritable - abi.CREATE_SUSPENDED, // suspended so job assignment precedes any execution - null, options.cwd, - startupInfo, processInfo, - ) - restoreInherit(stdIn) - restoreInherit(stdOut) - restoreInherit(stdErr) - if (created === 0) { - const win32Code = api.getLastError() - api.closeHandle(job) - throwWin32(api, 'CreateProcessAsUserW', win32Code, `command: ${options.command}, cwd: ${options.cwd}`) - } - - const info = decodeProcessInfo(processInfo) - const processHandle = info.hProcess - const threadHandle = info.hThread - if (processHandle === null || threadHandle === null) { - api.closeHandle(job) - throw new Error(`CreateProcessAsUserW succeeded but returned null process/thread handles (pid ${info.dwProcessId})`) - } - - if (api.assignProcessToJobObject(job, processHandle) === 0) { - // The child was created suspended and is NOT in the kill-on-close job: - // closing handles would leave it suspended forever. Terminate it first, - // then drop the handles and throw. - const win32Code = api.getLastError() - api.terminateProcess(processHandle, 1) - api.closeHandle(threadHandle) - api.closeHandle(processHandle) - api.closeHandle(job) - throwWin32(api, 'AssignProcessToJobObject', win32Code, `pid ${info.dwProcessId}`) - } - if (api.resumeThread(threadHandle) === 0xFFFFFFFF) { - // Closing the job triggers kill-on-close, so the suspended child dies - // instead of hanging until this process exits; the process/thread handles - // must go too. - const win32Code = api.getLastError() - api.closeHandle(threadHandle) - api.closeHandle(processHandle) - api.closeHandle(job) - throwWin32(api, 'ResumeThread', win32Code, `pid ${info.dwProcessId}`) - } - api.closeHandle(threadHandle) - - return { pid: info.dwProcessId, process: processHandle, job } + return spawnInheritedJobProcess(api, { ...options, token }) +} + +/** + * Wait for a restricted child and close its process handle. + * @param api - ACL/token binding table. + * @param process - caller-owned process handle. + * @returns direct process exit code. + */ +export function waitForExit(api: Win32Bindings, process: NativePtr): number { + return waitForProcessExit(api, process) } diff --git a/packages/sandbox/sandbox-windows-acl/src/token.ts b/packages/sandbox/sandbox-windows-acl/src/token.ts index abd0c73617..f1f2befbdd 100644 --- a/packages/sandbox/sandbox-windows-acl/src/token.ts +++ b/packages/sandbox/sandbox-windows-acl/src/token.ts @@ -180,8 +180,9 @@ export interface RestrictingSidSet { * `AU:(AD)` + `AU:(OI)(CI)(IO)(M)` ACEs) is closed in both — documented in * README. INTERACTIVE/LOCAL are absent from BOTH lists too — the host's * Public tree grants write to INTERACTIVE, so removing it closes that - * escape. S-1-2-1 (console logon) is intentionally absent: see win32-abi.ts - * for the verified failure modes. FAILS CLOSED: any failure throws — never + * escape. S-1-2-1 (console logon) is intentionally absent: the package + * README's "Console isolation is unavailable" entry records the verified + * failure modes. FAILS CLOSED: any failure throws — never * spawn unrestricted. * @param api - the binding table. * @param currentToken - the process token to restrict. diff --git a/packages/sandbox/sandbox-windows-acl/src/win32-abi.ts b/packages/sandbox/sandbox-windows-acl/src/win32-abi.ts index 5af4496af7..478fc71e1d 100644 --- a/packages/sandbox/sandbox-windows-acl/src/win32-abi.ts +++ b/packages/sandbox/sandbox-windows-acl/src/win32-abi.ts @@ -1,258 +1,98 @@ -/** - * Windows ABI constants for the ACL-sandbox backend. - * - * Every value was verified against the actual MinGW Windows headers on this - * machine (C:\Strawberry\c\x86_64-w64-mingw32\include\) and cross-checked at - * runtime by verify/abi-probe.cpp (same numbers; static_asserts passed). - * Regenerate the probe with: - * g++ -std=c++20 -municode -O2 -o abi-probe.exe abi-probe.cpp -ladvapi32 && .\abi-probe.exe - * - * The port intentionally excludes two pieces of the original POC - * (github.com/huoyaoyuan/windows-acl-restrict-poc @ 10e4dfb), both verified - * empirically on Windows 11 build 26200: - * - S-1-2-1 (console logon SID) in the restricting list: the POC created it - * via CreateWellKnownSid(WinLocalLogonSid) which fails here with - * ERROR_INVALID_PARAMETER (87), leaving a garbage SID that makes - * CreateRestrictedToken fail with ERROR_INVALID_SID (1337); using the - * correct WinConsoleLogonSid does produce a valid S-1-2-1, but the child - * then still dies with STATUS_DLL_INIT_FAILED (0xC0000142) whenever - * CREATE_NO_WINDOW / CREATE_NEW_CONSOLE is used. - * - Console isolation: under this restriction scheme a hidden console is not - * attainable, so children share the host console (stdio redirection is - * pipe-based and unaffected). - * @module @deepseek-ai/dsh-sandbox-windows-acl/win32-abi - */ +/** ACL/token-specific Win32 constants. */ -// ---- winnt.h --------------------------------------------------------------- - -// TOKEN_* access rights (winnt.h lines ~3928) -/** TOKEN_ASSIGN_PRIMARY: required to create a process with the token (CreateProcessAsUser). */ -export const TOKEN_ASSIGN_PRIMARY = 0x0001 -/** TOKEN_DUPLICATE: required to duplicate a token (DuplicateTokenEx). */ -export const TOKEN_DUPLICATE = 0x0002 -/** TOKEN_QUERY: required to read token information (GetTokenInformation). */ -export const TOKEN_QUERY = 0x0008 -/** TOKEN_ADJUST_DEFAULT: required to change a token's default DACL. */ -export const TOKEN_ADJUST_DEFAULT = 0x0080 - -// SID_AND_ATTRIBUTES.Attributes flags (winnt.h lines ~3446) -/** - * SE_GROUP_LOGON_ID: marks a token group SID as the logon SID (compared with - * `>>> 0` — the flag's high bit makes it negative as a signed 32-bit number). - */ -export const SE_GROUP_LOGON_ID = 0xC0000000 - -// Generic file access (winnt.h lines ~5893-5913): -// FILE_GENERIC_WRITE = STANDARD_RIGHTS_WRITE | FILE_WRITE_DATA | FILE_WRITE_ATTRIBUTES -// | FILE_WRITE_EA | FILE_APPEND_DATA | SYNCHRONIZE -/** STANDARD_RIGHTS_WRITE (== READ_CONTROL): the standard-rights component of generic write access. */ -export const STANDARD_RIGHTS_WRITE = 0x00020000 // == READ_CONTROL -/** FILE_GENERIC_WRITE: every file-write permission bit plus SYNCHRONIZE. */ -export const FILE_GENERIC_WRITE = 0x00120116 -/** DELETE: remove or rename the object (winnt.h line ~3009). */ -export const DELETE = 0x00010000 -/** FILE_DELETE_CHILD: remove or rename a directory's children (winnt.h line ~5907). */ -export const FILE_DELETE_CHILD = 0x0040 -// The POC granted FILE_GENERIC_WRITE minus READ_CONTROL, which displays as -// "Write" in Explorer/icacls (windows-acl-restrict-poc.cpp line 16). The -// sandbox grant adds DELETE and FILE_DELETE_CHILD so confined -// delete/rename/git operations inside the granted trees pass the token's -// access check too; Write+DELETE displays as "Modify" in icacls. -// WRITE_DAC/WRITE_OWNER stay OUT deliberately — granting them would let the -// child take ownership or rewrite DACLs and escape the allowlist (the -// security boundary). -/** - * GRANT_MASK: FILE_GENERIC_WRITE minus READ_CONTROL plus DELETE and - * FILE_DELETE_CHILD — the write+delete access mask the capability-SID ACEs grant - * (displays as "Modify" in Explorer/icacls). WRITE_DAC/WRITE_OWNER are - * deliberately excluded: they would let the confined child take ownership or - * rewrite DACLs. - */ -export const GRANT_MASK = (FILE_GENERIC_WRITE | DELETE | FILE_DELETE_CHILD) & ~STANDARD_RIGHTS_WRITE // 0x00110156 - -/** - * FILE_ALL_ACCESS (winnt.h line ~2789: STANDARD_RIGHTS_REQUIRED | SYNCHRONIZE - * | 0x1FF): full file-object access. The mask of the ACE merged into the - * restricted token's DEFAULT DACL — the token holder must keep full access to - * every NEW object it creates (pipes included), and the ACE must name a - * restricting SID so the write pass-2 check passes at creation. - */ -export const FILE_ALL_ACCESS = 0x1F01FF - -// CreateRestrictedToken flags (winnt.h lines ~4284) -/** DISABLE_MAX_PRIVILEGE: strip the token's maximum-privilege elevation so the confined child cannot escalate. */ -export const DISABLE_MAX_PRIVILEGE = 0x1 -/** LUA_TOKEN: produce a limited-user (filtered admin) token. */ -export const LUA_TOKEN = 0x4 -/** WRITE_RESTRICTED: intersect write access with the restricting SIDs' ACL grants — the sandbox's core mechanism. */ -export const WRITE_RESTRICTED = 0x8 - -// WELL_KNOWN_SID_TYPE (winnt.h lines ~3369-3407) -/** WinWorldSid: S-1-1-0 (Everyone) — the only well-known SID the restricted tokens use (keep-alive group; see token.ts). */ -export const WinWorldSid = 1 - -// TOKEN_INFORMATION_CLASS (winnt.h line ~3963: TokenUser=1, TokenGroups=2) -/** TokenGroups: GetTokenInformation class returning the token's group SIDs. */ -export const TokenGroups = 2 -/** TokenDefaultDacl: the token's default DACL — the DACL every NEW object created without an explicit SD takes. */ -export const TokenDefaultDacl = 6 - -// SECURITY_INFORMATION (winnt.h line ~4293) -/** DACL_SECURITY_INFORMATION: read/write only the DACL of a security descriptor. */ -export const DACL_SECURITY_INFORMATION = 0x00000004 - -// PROCESS access rights (winnt.h lines ~4364) -/** PROCESS_QUERY_INFORMATION: read exit status and times of a process handle. */ +/** OpenProcess access required to query the current process token. */ export const PROCESS_QUERY_INFORMATION = 0x0400 - -// ---- accctrl.h ------------------------------------------------------------- - -// SE_OBJECT_TYPE (accctrl.h line ~22: SE_UNKNOWN_OBJECT_TYPE=0, SE_FILE_OBJECT=1) -/** SE_FILE_OBJECT: the trustee path names a filesystem object. */ +/** Token right required by CreateProcessAsUserW. */ +export const TOKEN_ASSIGN_PRIMARY = 0x0001 +/** Token right required by DuplicateTokenEx. */ +export const TOKEN_DUPLICATE = 0x0002 +/** Token right required to read token information. */ +export const TOKEN_QUERY = 0x0008 +/** Token right required to replace the token default DACL. */ +export const TOKEN_ADJUST_DEFAULT = 0x0080 +/** Group attribute identifying the token logon SID. */ +export const SE_GROUP_LOGON_ID = 0xC0000000 +/** Standard-rights portion excluded from the write capability grant. */ +export const STANDARD_RIGHTS_WRITE = 0x00020000 +/** Generic file write access bits. */ +export const FILE_GENERIC_WRITE = 0x00120116 +/** Delete or rename an object. */ +export const DELETE = 0x00010000 +/** Delete or rename a directory child. */ +export const FILE_DELETE_CHILD = 0x0040 +/** + * Capability-SID access mask granting write, delete, and child deletion. + * WRITE_DAC and WRITE_OWNER stay excluded so a confined child cannot rewrite + * DACLs or take ownership to escape the allowlist. + */ +export const GRANT_MASK = (FILE_GENERIC_WRITE | DELETE | FILE_DELETE_CHILD) & ~STANDARD_RIGHTS_WRITE +/** Full access used in the restricted token default DACL. */ +export const FILE_ALL_ACCESS = 0x1F01FF +/** CreateRestrictedToken flag that disables maximum privileges. */ +export const DISABLE_MAX_PRIVILEGE = 0x1 +/** CreateRestrictedToken limited-user flag. */ +export const LUA_TOKEN = 0x4 +/** Restrict write access to the listed restricting SIDs. */ +export const WRITE_RESTRICTED = 0x8 +/** WELL_KNOWN_SID_TYPE value for Everyone. */ +export const WinWorldSid = 1 +/** TOKEN_INFORMATION_CLASS value for token groups. */ +export const TokenGroups = 2 +/** TOKEN_INFORMATION_CLASS value for the token default DACL. */ +export const TokenDefaultDacl = 6 +/** SECURITY_INFORMATION flag selecting the DACL. */ +export const DACL_SECURITY_INFORMATION = 0x00000004 +/** SE_OBJECT_TYPE value for filesystem objects. */ export const SE_FILE_OBJECT = 1 - -// TRUSTEE_FORM / TRUSTEE_TYPE (accctrl.h lines ~38-55): both enums start at 0 -/** TRUSTEE_IS_UNKNOWN: TRUSTEE_TYPE unknown (TrusteeForm carries the shape). */ +/** TRUSTEE_TYPE value used when trustee classification is unknown. */ export const TRUSTEE_IS_UNKNOWN = 0 -/** TRUSTEE_IS_SID: TRUSTEE_FORM — Trustee.ptstrName is a SID pointer. */ +/** TRUSTEE_FORM value indicating a SID pointer. */ export const TRUSTEE_IS_SID = 0 -/** NO_MULTIPLE_TRUSTEE: Trustee.pMultipleTrustee is null. */ +/** Trustee record has no chained trustee. */ export const NO_MULTIPLE_TRUSTEE = 0 - -// ACCESS_MODE (accctrl.h line ~127: NOT_USED_ACCESS=0, GRANT_ACCESS=1, REVOKE_ACCESS=4) -/** GRANT_ACCESS: SetEntriesInAclW adds the entry as an allow ACE. */ +/** EXPLICIT_ACCESS mode that grants access. */ export const GRANT_ACCESS = 1 -/** REVOKE_ACCESS: SetEntriesInAclW removes the matching allow ACE. */ +/** EXPLICIT_ACCESS mode that revokes access. */ export const REVOKE_ACCESS = 4 - -// grfInheritance (accctrl.h lines ~137-142) -/** - * SUB_CONTAINERS_AND_OBJECTS_INHERIT: the ACE applies to the directory, its - * subdirectories, and files (OBJECT_INHERIT_ACE | CONTAINER_INHERIT_ACE). - */ -export const SUB_CONTAINERS_AND_OBJECTS_INHERIT = 0x3 // == OBJECT_INHERIT_ACE | CONTAINER_INHERIT_ACE - -// ---- winbase.h ------------------------------------------------------------- - -/** - * STARTF_USESTDHANDLES: STARTUPINFOW dwFlags — the child uses the hStd* - * handles, required because Node clears stdio inheritability at startup. - */ -export const STARTF_USESTDHANDLES = 0x00000100 -/** HANDLE_FLAG_INHERIT: SetHandleInformation flag re-enabling handle inheritance for the spawned child's stdio handles. */ -export const HANDLE_FLAG_INHERIT = 0x1 -/** INFINITE: never-timeout wait value. */ -export const INFINITE = 0xFFFFFFFF -/** MAX_PATH: legacy path length bound. */ +/** ACE inheritance flags for child containers and objects. */ +export const SUB_CONTAINERS_AND_OBJECTS_INHERIT = 0x3 +/** Legacy Win32 maximum path character count used by GetTempPathW. */ export const MAX_PATH = 260 -// winbase.h line ~410: the confined child starts suspended so the runner can -// assign it to the kill-on-close job before any of its code runs. -/** CREATE_SUSPENDED: create the child with its primary thread suspended until ResumeThread. */ -export const CREATE_SUSPENDED = 0x4 -// winbase.h lines ~497-499: GetStdHandle selectors. -/** STD_INPUT_HANDLE: GetStdHandle selector for the standard input. */ -export const STD_INPUT_HANDLE = -10 -/** STD_OUTPUT_HANDLE: GetStdHandle selector for the standard output. */ -export const STD_OUTPUT_HANDLE = -11 -/** STD_ERROR_HANDLE: GetStdHandle selector for the standard error. */ -export const STD_ERROR_HANDLE = -12 - -// FormatMessageW flags (winbase.h lines ~1446-1469) -/** FORMAT_MESSAGE_FROM_SYSTEM: format the message from the system message table. */ -export const FORMAT_MESSAGE_FROM_SYSTEM = 0x00001000 -/** FORMAT_MESSAGE_IGNORE_INSERTS: skip insert-sequence substitution. */ -export const FORMAT_MESSAGE_IGNORE_INSERTS = 0x00000200 - -// ---- error codes ----------------------------------------------------------- - -/** ERROR_SUCCESS: the operation succeeded. */ +/** Successful Win32 status code. */ export const ERROR_SUCCESS = 0 -/** ERROR_INSUFFICIENT_BUFFER: a size-probe call succeeded but needs a larger buffer. */ -export const ERROR_INSUFFICIENT_BUFFER = 122 -/** ERROR_BROKEN_PIPE: the pipe's other end has closed. */ -export const ERROR_BROKEN_PIPE = 109 -/** ERROR_NO_DATA: the pipe is being closed. */ -export const ERROR_NO_DATA = 232 -/** ERROR_LOCK_VIOLATION: a byte-range lock conflicts with an existing lock (winerror.h line ~78). */ +/** Win32 error reported when an immediate byte-range lock cannot be obtained. */ export const ERROR_LOCK_VIOLATION = 33 - -// ---- lock files (fileapi.h / minwinbase.h / winnt.h) ----------------------- - -// CreateFileW dwDesiredAccess for the ACL lock files: plain read+write is -// enough to take byte-range locks. -/** GENERIC_READ: generic read access (winnt.h line ~3028). */ +/** Generic read access bit. */ export const GENERIC_READ = 0x80000000 -/** GENERIC_WRITE: generic write access (winnt.h line ~3029). */ +/** Generic write access bit. */ export const GENERIC_WRITE = 0x40000000 -// CreateFileW dwShareMode: the lock file is shared for read/write but NOT -// for delete — if a locked file could be deleted and recreated underneath the -// lock holder, two processes could hold "the same" lock on different files. -/** FILE_SHARE_READ: other opens may read (winnt.h line ~5949). */ +/** CreateFile share-read flag. */ export const FILE_SHARE_READ = 0x00000001 -/** FILE_SHARE_WRITE: other opens may write (winnt.h line ~5950). */ +/** CreateFile share-write flag. */ export const FILE_SHARE_WRITE = 0x00000002 -/** FILE_SHARE_DELETE: other opens may delete (winnt.h line ~5951) — deliberately NOT used for lock files. */ +/** CreateFile share-delete flag. */ export const FILE_SHARE_DELETE = 0x00000004 -/** OPEN_ALWAYS: create the lock file if absent, open it otherwise (fileapi.h line ~21). */ +/** CreateFile disposition that opens or creates the file. */ export const OPEN_ALWAYS = 4 -// LockFileEx dwFlags (minwinbase.h lines ~180-181, included by winbase.h). -/** LOCKFILE_EXCLUSIVE_LOCK: request an exclusive byte-range lock. */ +/** LockFileEx exclusive-lock flag. */ export const LOCKFILE_EXCLUSIVE_LOCK = 0x2 -/** LOCKFILE_FAIL_IMMEDIATELY: fail with ERROR_LOCK_VIOLATION instead of waiting. */ +/** LockFileEx immediate-failure flag. */ export const LOCKFILE_FAIL_IMMEDIATELY = 0x1 - -// ACE_HEADER.AceType (winnt.h lines ~3449-3463) -/** ACCESS_ALLOWED_ACE_TYPE: an access-allowed ACE granting the mask to the trustee. */ +/** ACE type for an allowed-access entry. */ export const ACCESS_ALLOWED_ACE_TYPE = 0 - -// SID structure (winnt.h line ~280 SID_IDENTIFIER_AUTHORITY; line ~286 -// #define SID_MAX_SUB_AUTHORITIES 15). -/** SID_MAX_SUB_AUTHORITIES: the most subauthorities a SID may carry. */ +/** Maximum SID sub-authority count. */ export const SID_MAX_SUB_AUTHORITIES = 15 - -// ACE_HEADER.AceFlags (winnt.h lines ~3477-3524): inherited ACEs shown when -// reading a DACL are marked with this bit and are not part of the explicit -// DACL edits this module makes. -/** INHERITED_ACE: the ACE was inherited from the parent object, not stored explicitly. */ +/** ACE flag marking inherited entries. */ export const INHERITED_ACE = 0x10 - -// ---- job object (winnt.h lines ~4859-4866, ~5138, ~5190-5199) -------------- - -// JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE: the child dies when the runner's last -// job handle closes — the orphan-child backstop for the runner design. -/** JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE: the child dies when the runner's last job handle closes — the orphan-child backstop. */ -export const JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE = 0x00002000 -// JOBOBJECTINFOCLASS: JobObjectBasicAccountingInformation=1, ..., ExtendedLimit=9. -/** JobObjectExtendedLimitInformation: JOBOBJECTINFOCLASS for the extended limit structure. */ -export const JobObjectExtendedLimitInformation = 9 -// sizeof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION), verified by abi-probe. -/** sizeof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION), verified by abi-probe. */ -export const JOBOBJECT_EXTENDED_LIMIT_SIZE = 144 -// LimitFlags offset inside JOBOBJECT_EXTENDED_LIMIT_INFORMATION -// (BasicLimitInformation@0 + PerProcessUserTimeLimit@0 + PerJobUserTimeLimit@8), -// verified by abi-probe. -/** - * LimitFlags offset inside JOBOBJECT_EXTENDED_LIMIT_INFORMATION - * (BasicLimitInformation@0 + PerProcessUserTimeLimit@0 + - * PerJobUserTimeLimit@8), verified by abi-probe. - */ -export const JOBOBJECT_EXTENDED_LIMIT_FLAGS_OFFSET = 16 - -// ---- ABI layout, verified by verify/abi-probe.cpp (x64) -------------------- - -/** SECURITY_MAX_SID_SIZE: maximum SID byte size. */ +/** Maximum SID allocation size in bytes. */ export const SECURITY_MAX_SID_SIZE = 68 -/** SID_AND_ATTRIBUTES stride: { PSID Sid @0 (8); DWORD Attributes @8 (4) } + pad. */ +/** x64 SID_AND_ATTRIBUTES byte size. */ export const SID_AND_ATTRIBUTES_SIZE = 16 -/** TOKEN_GROUPS.Groups[] starts at offset 8 (GroupCount @0 + alignment). */ +/** x64 TOKEN_GROUPS offset of the first group entry. */ export const TOKEN_GROUPS_OFFSET = 8 -/** sizeof(EXPLICIT_ACCESS_W): perms@0 mode@4 inheritance@8 Trustee@16. */ +/** x64 EXPLICIT_ACCESS_W byte size. */ export const EXPLICIT_ACCESS_W_SIZE = 48 -/** Trustee offset inside EXPLICIT_ACCESS_W. */ +/** x64 offset of TRUSTEE_W inside EXPLICIT_ACCESS_W. */ export const TRUSTEE_W_OFFSET = 16 -/** ptstrName offset inside TRUSTEE_W (=> 40 inside EXPLICIT_ACCESS_W). */ +/** x64 offset of ptstrName inside TRUSTEE_W. */ export const TRUSTEE_W_PTSTRNAME_OFFSET = 24 -/** sizeof(STARTUPINFOW), verified by abi-probe. */ -export const STARTUPINFOW_SIZE = 104 -/** sizeof(PROCESS_INFORMATION), verified by abi-probe. */ -export const PROCESS_INFORMATION_SIZE = 24 diff --git a/packages/sandbox/sandbox-windows-acl/tests/acl-failure-paths.spec.ts b/packages/sandbox/sandbox-windows-acl/tests/acl-failure-paths.spec.ts index 005f522fda..fce0c3562a 100644 --- a/packages/sandbox/sandbox-windows-acl/tests/acl-failure-paths.spec.ts +++ b/packages/sandbox/sandbox-windows-acl/tests/acl-failure-paths.spec.ts @@ -1,6 +1,6 @@ /** - * ACL failure-path tests with stub binding tables (the failure-paths.spec.ts - * pattern): every checked Win32 call in the lock, read-merge-write, and + * ACL failure-path tests with minimal stub binding tables: every checked + * Win32 call in the lock, read-merge-write, and * grant-skip sequence has a failing counterpart, and each failure closes the * handles it created before throwing. The exact-ACE skip and the DACL-walk * defenses are driven through crafted in-memory ACL/SID buffers. Pure @@ -9,13 +9,13 @@ */ import { tmpdir } from 'node:os' +import { Win32Error } from '@deepseek-ai/dsh-win32-process' import { describe, expect, it, vi } from 'vitest' import koffi from 'koffi' import { grantWrite, revokeWrite, withPathLock } from '../src/acl.ts' import { allocBytes, ptrAddress } from '../src/ffi.ts' import type { NativePtr, Win32Bindings } from '../src/ffi.ts' -import { Win32Error } from '../src/errors.ts' import * as abi from '../src/win32-abi.ts' const PVOID = koffi.pointer('void') diff --git a/packages/sandbox/sandbox-windows-acl/tests/ffi.spec.ts b/packages/sandbox/sandbox-windows-acl/tests/ffi.spec.ts index 903f56afc3..5dab1058c2 100644 --- a/packages/sandbox/sandbox-windows-acl/tests/ffi.spec.ts +++ b/packages/sandbox/sandbox-windows-acl/tests/ffi.spec.ts @@ -1,8 +1,7 @@ /** - * FFI helper tests with stub binding tables (the failure-paths.spec.ts - * pattern): error formatting and temp-path decoding defenses, the - * last-error throwers' detail fallback, pointer decode NULL handling, and - * the bounded SID comparison's early exits. Pure stubs — no real Win32 + * Sandbox-specific FFI tests with stub binding tables: temp-path decoding, + * invalid-handle checks, pointer-at-offset decoding, and the bounded SID + * comparison's early exits. Pure stubs — no real Win32 * calls, so these run on every platform; the real-FFI round-trip lives in * acl.spec.ts and probe.spec.ts (win32 only). */ @@ -10,17 +9,15 @@ import { describe, expect, it, vi } from 'vitest' import koffi from 'koffi' -import { Win32Error } from '../src/errors.ts' +import { Win32Error } from '@deepseek-ai/dsh-win32-process' import { - allocBytes, decodePtr, decodePtrAt, errorText, getTempPath, - isInvalidHandle, isNullPtr, sameSidAt, throwLastError, throwWin32, + allocBytes, decodePtrAt, getTempPath, + isInvalidHandle, sameSidAt, } from '../src/ffi.ts' import type { NativePtr, Win32Bindings } from '../src/ffi.ts' import * as abi from '../src/win32-abi.ts' -const PVOID = koffi.pointer('void') - -/** A stub whose formatMessageW writes real UTF-16 text (the errorText round-trip). */ +/** A stub whose formatMessageW supplies text to the GetTempPath failure path. */ function formatApi(): { api: Win32Bindings; formatMessageW: ReturnType } { const formatMessageW = vi.fn((_flags: number, _source: null, _id: number, _lang: number, buffer: Buffer, _size: number, _args: null) => { const text = 'access denied' @@ -48,18 +45,6 @@ function craftSid(revision: number, count: number, authority: number[] = [0, 0, return sid } -describe('errorText', () => { - it('decodes the formatted UTF-16 message and trims it', () => { - const { api } = formatApi() - expect(errorText(api, 5)).toBe('access denied') - }) - - it('returns an empty string when FormatMessageW formats nothing', () => { - const api = { formatMessageW: vi.fn(() => 0) } as unknown as Win32Bindings - expect(errorText(api, 5)).toBe('') - }) -}) - describe('getTempPath', () => { it('decodes the NUL-terminated temp path GetTempPathW wrote', () => { const api = { @@ -83,55 +68,14 @@ describe('getTempPath', () => { expect(caught).toBeInstanceOf(Win32Error) expect((caught as Win32Error).api).toBe('GetTempPathW') }) -}) -describe('throwLastError and throwWin32', () => { - it('throwLastError formats the system message when no detail is given', () => { - const { api } = formatApi() - let caught: unknown - try { - throwLastError(api, 'Probe') - } catch (error) { - caught = error - } - expect(caught).toBeInstanceOf(Win32Error) - expect((caught as Win32Error).message).toContain('Probe failed (Win32 5): access denied') - }) - - it('throwWin32 formats the system message when no detail is given', () => { - const { api } = formatApi() - let caught: unknown - try { - throwWin32(api, 'Probe', 5) - } catch (error) { - caught = error - } - expect(caught).toBeInstanceOf(Win32Error) - expect((caught as Win32Error).message).toContain('Probe failed (Win32 5): access denied') - }) - - it('Win32Error appends the detail when one is given', () => { - const error = new Win32Error('Probe', 5, 'the lock file path') - expect(error.name).toBe('Win32Error') - expect(error.api).toBe('Probe') - expect(error.win32Code).toBe(5) - expect(error.message).toBe('Probe failed (Win32 5): the lock file path') - }) - - it('Win32Error omits the detail suffix when none is given', () => { - const error = new Win32Error('Probe', 5) - expect(error.message).toBe('Probe failed (Win32 5)') + it('rejects a required length larger than the fixed buffer', () => { + const api = { getTempPathW: vi.fn(() => 300) } as unknown as Win32Bindings + expect(() => getTempPath(api)).toThrow(/GetTempPathW failed \(Win32 122\): required 300/u) }) }) -describe('pointer NULL handling', () => { - it('isNullPtr accepts null, undefined, and the zero pointer', () => { - expect(isNullPtr(null)).toBe(true) - expect(isNullPtr(undefined)).toBe(true) - expect(isNullPtr(0n as NativePtr)).toBe(true) - expect(isNullPtr(42n as NativePtr)).toBe(false) - }) - +describe('sandbox pointer handling', () => { it('isInvalidHandle treats NULL as failure', () => { expect(isInvalidHandle(null)).toBe(true) expect(isInvalidHandle(undefined)).toBe(true) @@ -150,11 +94,6 @@ describe('pointer NULL handling', () => { buffer.writeBigUInt64LE(42n, 0) expect(decodePtrAt(buffer, 0)).toBe(42n) }) - - it('decodePtr returns null for an unset out-parameter slot', () => { - const slot = koffi.alloc(PVOID, 1) as unknown as NativePtr - expect(decodePtr(slot)).toBeNull() - }) }) describe('sameSidAt bounded comparison', () => { diff --git a/packages/sandbox/sandbox-windows-acl/tests/grant-failure-paths.spec.ts b/packages/sandbox/sandbox-windows-acl/tests/grant-failure-paths.spec.ts index fe803025b9..e22a46060c 100644 --- a/packages/sandbox/sandbox-windows-acl/tests/grant-failure-paths.spec.ts +++ b/packages/sandbox/sandbox-windows-acl/tests/grant-failure-paths.spec.ts @@ -1,6 +1,6 @@ /** - * AclWriteGrant failure-path tests with stub binding tables (the - * failure-paths.spec.ts pattern): create fails closed on SID-parse failure, + * AclWriteGrant failure-path tests with minimal stub binding tables: create + * fails closed on SID-parse failure, * dispose aggregates revocation and SID-free failures into an * AggregateError. Pure stubs — no real Win32 calls, so these run on every * platform; the real-FFI round-trip lives in grant.spec.ts (win32 only). diff --git a/packages/sandbox/sandbox-windows-acl/tests/index-failure-paths.spec.ts b/packages/sandbox/sandbox-windows-acl/tests/index-failure-paths.spec.ts index c171d30084..64899d9db8 100644 --- a/packages/sandbox/sandbox-windows-acl/tests/index-failure-paths.spec.ts +++ b/packages/sandbox/sandbox-windows-acl/tests/index-failure-paths.spec.ts @@ -11,12 +11,13 @@ import { mkdtempSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' import { join, resolve } from 'node:path' +import { Win32Error } from '@deepseek-ai/dsh-win32-process' +import { ERROR_BROKEN_PIPE } from '@deepseek-ai/dsh-win32-process/src/abi.ts' +import { PROCESS_INFORMATION } from '@deepseek-ai/dsh-win32-process/src/ffi.ts' import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' import koffi from 'koffi' -import { PROCESS_INFORMATION } from '../src/ffi.ts' import type { NativePtr, Win32Bindings } from '../src/ffi.ts' -import { Win32Error } from '../src/errors.ts' import { AclSandbox } from '../src/index.ts' import * as abi from '../src/win32-abi.ts' @@ -146,10 +147,11 @@ function happyStubs(): HappyStubs { const setInformationJobObject = vi.fn(() => 1) const assignProcessToJobObject = vi.fn(() => 1) const resumeThread = vi.fn(() => 0) + const terminateProcess = vi.fn(() => 1) const getStdHandle = vi.fn(() => fresh()) const localFree = vi.fn(() => 0n) const closeHandle = vi.fn(() => 1) - const getLastError = vi.fn(() => abi.ERROR_BROKEN_PIPE) // the drains' clean EOF + const getLastError = vi.fn(() => ERROR_BROKEN_PIPE) // the drains' clean EOF const formatMessageW = vi.fn(() => 0) const api = { @@ -159,7 +161,8 @@ function happyStubs(): HappyStubs { getLengthSid, copySid, createWellKnownSid, isValidSid, createRestrictedToken, setTokenInformation, createPipe, setHandleInformation, createProcessAsUserW, peekNamedPipe, readFile, waitForSingleObject, getExitCodeProcess, createJobObjectW, - setInformationJobObject, assignProcessToJobObject, resumeThread, getStdHandle, + setInformationJobObject, assignProcessToJobObject, resumeThread, terminateProcess, + getStdHandle, localFree, closeHandle, getLastError, formatMessageW, } as unknown as Win32Bindings return { diff --git a/packages/sandbox/sandbox-windows-acl/tests/quote.spec.ts b/packages/sandbox/sandbox-windows-acl/tests/quote.spec.ts deleted file mode 100644 index 5af00fb9cb..0000000000 --- a/packages/sandbox/sandbox-windows-acl/tests/quote.spec.ts +++ /dev/null @@ -1,88 +0,0 @@ -/** - * quoteArg unit tests plus a round-trip through the REAL CommandLineToArgvW - * parser (shell32.dll, shellapi.h line ~867: - * `LPWSTR *CommandLineToArgvW(LPCWSTR lpCmdLine, int *pNumArgs)`) on win32. - * - * CommandLineToArgvW applies the documented backslash rule (2n backslashes - * before a quote produce n backslashes and toggle quoting; 2n+1 produce n - * backslashes and a literal quote) to every token EXCEPT the first — the - * first token is parsed with backslashes literal and quotes toggling - * (verified empirically on this machine, Windows 11 build 26200). The - * round-trip therefore prepends a plain program token, exactly like - * buildCommandLine's real callers do, so the arguments under test land on - * the rule-applying tokens. - * - * Reading argv from CommandLineToArgvW: koffi cannot decode the returned - * LPWSTR* contents directly (the pointed-to strings are not koffi-registered - * references), so each string is copied with lstrcpynW (winbase.h line - * ~1500) into a Node Buffer and read as UTF-16LE; lengths come from - * lstrlenW (winbase.h line ~1506); the argv block is freed with LocalFree - * (winbase.h line ~1127) — CommandLineToArgvW's documented contract. - */ - -import { describe, expect, it } from 'vitest' - -import { buildCommandLine, quoteArg } from '../src/spawn.ts' - -const isWin32 = process.platform === 'win32' - -/** - * Table cases: input argv entry → the exact command-line fragment quoteArg - * must produce. Trailing-backslash inputs are the regression: the closing - * quote must be preceded by DOUBLED backslashes, or the parser reads them as - * escaping the closing quote. - */ -const cases: Array<[input: string, quoted: string]> = [ - ['', '""'], - ['a', 'a'], - ['a b', '"a b"'], - ['a"b', '"a\\"b"'], - ['a\\b', 'a\\b'], - ['a b\\', '"a b\\\\"'], - ['a b\\\\', '"a b\\\\\\\\"'], - ['a b\\\\\\', '"a b\\\\\\\\\\\\"'], - ['a\\\\"b', '"a\\\\\\\\\\"b"'], -] - -describe('quoteArg', () => { - it.each(cases)('quotes %j as %j', (input, quoted) => { - expect(quoteArg(input)).toBe(quoted) - }) -}) - -describe.skipIf(!isWin32)('CommandLineToArgvW round-trip', () => { - it('parses quoteArg+join back to the exact original argv', async () => { - const { default: koffi } = await import('koffi') - const PVOID = koffi.pointer('void') - const shell32 = koffi.load('shell32.dll') - const kernel32 = koffi.load('kernel32.dll') - const commandLineToArgvW = shell32.func('__stdcall', 'CommandLineToArgvW', PVOID, ['str16', koffi.pointer('int')]) - const lstrcpynW = kernel32.func('__stdcall', 'lstrcpynW', PVOID, [PVOID, PVOID, 'int']) - const lstrlenW = kernel32.func('__stdcall', 'lstrlenW', 'int', [PVOID]) - const localFree = kernel32.func('__stdcall', 'LocalFree', PVOID, [PVOID]) - - const parse = (commandLine: string): string[] => { - const countSlot = koffi.alloc('int', 1) as unknown - const argvBlock = commandLineToArgvW(commandLine, countSlot) as unknown - try { - if (argvBlock === null) throw new Error('CommandLineToArgvW returned NULL') - const count = koffi.decode(countSlot, 0, 'int') as number - const table = Buffer.from(koffi.view(argvBlock, count * 8)) - const parsed: string[] = [] - for (let index = 0; index < count; index++) { - const stringAddress = table.readBigUInt64LE(index * 8) - const copied = Buffer.alloc(2048) - lstrcpynW(copied, stringAddress, copied.length / 2) - const length = lstrlenW(copied) as number - parsed.push(copied.subarray(0, length * 2).toString('utf16le')) - } - return parsed - } finally { - localFree(argvBlock) - } - } - - const argv = ['', 'a', 'a b', 'a"b', 'a\\b', 'a b\\', 'a b\\\\', 'a b\\\\\\', 'a\\\\"b'] - expect(parse(buildCommandLine('prog.exe', argv))).toEqual(['prog.exe', ...argv]) - }) -}) diff --git a/packages/sandbox/sandbox-windows-acl/tests/runner.spec.ts b/packages/sandbox/sandbox-windows-acl/tests/runner.spec.ts index 19dfdaf106..b0d364202b 100644 --- a/packages/sandbox/sandbox-windows-acl/tests/runner.spec.ts +++ b/packages/sandbox/sandbox-windows-acl/tests/runner.spec.ts @@ -331,9 +331,9 @@ describe.skipIf(!isWin32 || !pwshAvailable())('windows-acl runner', () => { // workspace-write keeps the ACE standing for the server lifetime. After // switching to read-only, the restricted token's read-only list must carry NO // capability SID — the standing ACE stays but the pass-2 check cannot use - // it, so the workspace write is denied (previously it LEAKED). The - // switch back reuses the SAME standing ACE: the re-upgrade write lands - // without any re-grant. + // it, so the workspace write is denied instead of leaking through the + // standing ACE. The switch back reuses the SAME standing ACE: the + // re-upgrade write lands without any re-grant. const writeSid = workspaceWriteSid(writableDir) const privateTemp = join(isolatedTemp, 'mode-switch-temp') mkdirSync(privateTemp) diff --git a/packages/sandbox/sandbox-windows-acl/tests/token-failure-paths.spec.ts b/packages/sandbox/sandbox-windows-acl/tests/token-failure-paths.spec.ts index b3559ee781..6149d75277 100644 --- a/packages/sandbox/sandbox-windows-acl/tests/token-failure-paths.spec.ts +++ b/packages/sandbox/sandbox-windows-acl/tests/token-failure-paths.spec.ts @@ -1,6 +1,6 @@ /** - * Restricted-token failure-path tests with stub binding tables (the - * failure-paths.spec.ts pattern): every checked Win32 call in the token + * Restricted-token failure-path tests with minimal stub binding tables: every + * checked Win32 call in the token * pipeline — open, logon-SID scan, well-known SID creation, default-DACL * merge, restricted-token creation — has a failing counterpart, and each * failure closes or frees what it created before throwing. Pure stubs — no @@ -8,12 +8,12 @@ * lives in acl.spec.ts (win32 only). */ +import { Win32Error } from '@deepseek-ai/dsh-win32-process' import { describe, expect, it, vi } from 'vitest' import koffi from 'koffi' import { allocBytes, isNullPtr } from '../src/ffi.ts' import type { NativePtr, Win32Bindings } from '../src/ffi.ts' -import { Win32Error } from '../src/errors.ts' import { createRestrictedToken, findLogonSid, makeWellKnownSid, openCurrentProcessToken, setTokenDefaultDaclGrant, } from '../src/token.ts' diff --git a/packages/sandbox/sandbox-windows-acl/tsconfig.json b/packages/sandbox/sandbox-windows-acl/tsconfig.json index 1d7a70a5b1..97f5530dd8 100644 --- a/packages/sandbox/sandbox-windows-acl/tsconfig.json +++ b/packages/sandbox/sandbox-windows-acl/tsconfig.json @@ -17,6 +17,9 @@ }, { "path": "../../runtime-diagnostics/invariants" + }, + { + "path": "../../subprocess/win32-process" } ] } diff --git a/packages/sandbox/sandbox-windows-acl/verify/abi-probe.cpp b/packages/sandbox/sandbox-windows-acl/verify/abi-probe.cpp index a74afe9d80..55a3d9c9d8 100644 --- a/packages/sandbox/sandbox-windows-acl/verify/abi-probe.cpp +++ b/packages/sandbox/sandbox-windows-acl/verify/abi-probe.cpp @@ -1,6 +1,3 @@ -// ABI probe: prints sizeof/offsetof/enum values from the actual MinGW Windows -// headers on this machine. These numbers are the source of truth for the -// koffi FFI definitions in the Node.js port. #include #include #include @@ -11,185 +8,67 @@ int wmain() { - P(sizeof(void*)); - P(sizeof(HANDLE)); - P(sizeof(DWORD)); - P(sizeof(WORD)); - P(sizeof(BOOL)); + P(sizeof(TRUSTEE_W)); + P(offsetof(TRUSTEE_W, ptstrName)); + P(sizeof(EXPLICIT_ACCESS_W)); + P(offsetof(EXPLICIT_ACCESS_W, Trustee)); + P(sizeof(SID_AND_ATTRIBUTES)); + P(offsetof(SID_AND_ATTRIBUTES, Attributes)); + P(sizeof(TOKEN_GROUPS)); + P(offsetof(TOKEN_GROUPS, Groups)); + P(SECURITY_MAX_SID_SIZE); + P(SID_MAX_SUB_AUTHORITIES); + P(TOKEN_ASSIGN_PRIMARY); + P(TOKEN_DUPLICATE); + P(TOKEN_QUERY); + P(TOKEN_ADJUST_DEFAULT); + P(SE_GROUP_LOGON_ID); + P(FILE_GENERIC_WRITE); + P(STANDARD_RIGHTS_WRITE); + P(DELETE); + P(FILE_DELETE_CHILD); + P(((FILE_GENERIC_WRITE | DELETE | FILE_DELETE_CHILD) & ~STANDARD_RIGHTS_WRITE)); + P(FILE_SHARE_READ); + P(FILE_SHARE_WRITE); + P(FILE_SHARE_DELETE); + P(GENERIC_READ); + P(GENERIC_WRITE); + P(OPEN_ALWAYS); + P(LOCKFILE_EXCLUSIVE_LOCK); + P(LOCKFILE_FAIL_IMMEDIATELY); + P(ERROR_LOCK_VIOLATION); + P(INHERITED_ACE); + P(DISABLE_MAX_PRIVILEGE); + P(LUA_TOKEN); + P(WRITE_RESTRICTED); + P((int)WinWorldSid); + P((int)TokenGroups); + P((int)SE_FILE_OBJECT); + P(DACL_SECURITY_INFORMATION); + P((int)TRUSTEE_IS_UNKNOWN); + P((int)TRUSTEE_IS_SID); + P((int)GRANT_ACCESS); + P((int)REVOKE_ACCESS); + P(SUB_CONTAINERS_AND_OBJECTS_INHERIT); + P(MAX_PATH); + P(ERROR_SUCCESS); - P(sizeof(STARTUPINFOW)); - P(offsetof(STARTUPINFOW, cb)); - P(offsetof(STARTUPINFOW, lpReserved)); - P(offsetof(STARTUPINFOW, lpDesktop)); - P(offsetof(STARTUPINFOW, lpTitle)); - P(offsetof(STARTUPINFOW, dwX)); - P(offsetof(STARTUPINFOW, dwY)); - P(offsetof(STARTUPINFOW, dwXSize)); - P(offsetof(STARTUPINFOW, dwYSize)); - P(offsetof(STARTUPINFOW, dwXCountChars)); - P(offsetof(STARTUPINFOW, dwYCountChars)); - P(offsetof(STARTUPINFOW, dwFillAttribute)); - P(offsetof(STARTUPINFOW, dwFlags)); - P(offsetof(STARTUPINFOW, wShowWindow)); - P(offsetof(STARTUPINFOW, cbReserved2)); - P(offsetof(STARTUPINFOW, lpReserved2)); - P(offsetof(STARTUPINFOW, hStdInput)); - P(offsetof(STARTUPINFOW, hStdOutput)); - P(offsetof(STARTUPINFOW, hStdError)); - - P(sizeof(PROCESS_INFORMATION)); - P(offsetof(PROCESS_INFORMATION, hProcess)); - P(offsetof(PROCESS_INFORMATION, hThread)); - P(offsetof(PROCESS_INFORMATION, dwProcessId)); - P(offsetof(PROCESS_INFORMATION, dwThreadId)); - - P(sizeof(SECURITY_ATTRIBUTES)); - P(offsetof(SECURITY_ATTRIBUTES, nLength)); - P(offsetof(SECURITY_ATTRIBUTES, lpSecurityDescriptor)); - P(offsetof(SECURITY_ATTRIBUTES, bInheritHandle)); - - P(sizeof(TRUSTEE_W)); - P(offsetof(TRUSTEE_W, pMultipleTrustee)); - P(offsetof(TRUSTEE_W, MultipleTrusteeOperation)); - P(offsetof(TRUSTEE_W, TrusteeForm)); - P(offsetof(TRUSTEE_W, TrusteeType)); - P(offsetof(TRUSTEE_W, ptstrName)); - - P(sizeof(EXPLICIT_ACCESS_W)); - P(offsetof(EXPLICIT_ACCESS_W, grfAccessPermissions)); - P(offsetof(EXPLICIT_ACCESS_W, grfAccessMode)); - P(offsetof(EXPLICIT_ACCESS_W, grfInheritance)); - P(offsetof(EXPLICIT_ACCESS_W, Trustee)); - - P(sizeof(SID_AND_ATTRIBUTES)); - P(offsetof(SID_AND_ATTRIBUTES, Sid)); - P(offsetof(SID_AND_ATTRIBUTES, Attributes)); - - P(sizeof(TOKEN_GROUPS)); - P(offsetof(TOKEN_GROUPS, GroupCount)); - P(offsetof(TOKEN_GROUPS, Groups)); - - P(sizeof(TOKEN_MANDATORY_LABEL)); - - P(sizeof(SID)); - P(SECURITY_MAX_SID_SIZE); - P(SID_MAX_SUB_AUTHORITIES); - P(SID_REVISION); - - P(TOKEN_ASSIGN_PRIMARY); - P(TOKEN_DUPLICATE); - P(TOKEN_QUERY); - P(TOKEN_ADJUST_DEFAULT); - - P(SE_GROUP_LOGON_ID); - P(SE_GROUP_INTEGRITY); - P(SE_GROUP_INTEGRITY_ENABLED); - - P(FILE_GENERIC_WRITE); - P((FILE_GENERIC_WRITE & ~STANDARD_RIGHTS_WRITE)); - P(STANDARD_RIGHTS_WRITE); - P(DELETE); - P(FILE_DELETE_CHILD); - P(((FILE_GENERIC_WRITE | DELETE | FILE_DELETE_CHILD) & ~STANDARD_RIGHTS_WRITE)); - - P(FILE_SHARE_READ); - P(FILE_SHARE_WRITE); - P(FILE_SHARE_DELETE); - P(GENERIC_READ); - P(GENERIC_WRITE); - P(OPEN_ALWAYS); - P(LOCKFILE_EXCLUSIVE_LOCK); - P(LOCKFILE_FAIL_IMMEDIATELY); - P(ERROR_LOCK_VIOLATION); - P(INHERITED_ACE); - - P(DISABLE_MAX_PRIVILEGE); - P(SANDBOX_INERT); - P(LUA_TOKEN); - P(WRITE_RESTRICTED); - - P((int)WinWorldSid); - P((int)WinLocalLogonSid); - P((int)WinConsoleLogonSid); - - P((int)TokenUser); - P((int)TokenGroups); - P((int)TokenIntegrityLevel); - - P((int)SE_FILE_OBJECT); - P(DACL_SECURITY_INFORMATION); - - P((int)TRUSTEE_IS_UNKNOWN); - P((int)TRUSTEE_IS_SID); - P((int)NOT_USED_ACCESS); - P((int)GRANT_ACCESS); - P((int)REVOKE_ACCESS); - P(SUB_CONTAINERS_AND_OBJECTS_INHERIT); - P(OBJECT_INHERIT_ACE); - P(CONTAINER_INHERIT_ACE); - - P(CREATE_SUSPENDED); - P(CREATE_NO_WINDOW); - P(DETACHED_PROCESS); - P(CREATE_NEW_CONSOLE); - P(STARTF_USESTDHANDLES); - P(HANDLE_FLAG_INHERIT); - P(INFINITE); - - P(LMEM_FIXED); - P(LMEM_ZEROINIT); - P(LPTR); - - P(FORMAT_MESSAGE_ALLOCATE_BUFFER); - P(FORMAT_MESSAGE_FROM_SYSTEM); - P(FORMAT_MESSAGE_IGNORE_INSERTS); - P(MAX_PATH); - - P(ERROR_SUCCESS); - P(ERROR_INSUFFICIENT_BUFFER); - P(ERROR_NO_MORE_ITEMS); - P(ERROR_INVALID_PARAMETER); - P(ERROR_INVALID_SID); - P(ERROR_NONE_MAPPED); - P(ERROR_BROKEN_PIPE); - - // Job object (runner kill-on-close hardening) - P(sizeof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION)); - P(sizeof(JOBOBJECT_BASIC_LIMIT_INFORMATION)); - P(sizeof(IO_COUNTERS)); - P(offsetof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION, BasicLimitInformation)); - P(offsetof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION, BasicLimitInformation) + offsetof(JOBOBJECT_BASIC_LIMIT_INFORMATION, LimitFlags)); - P(offsetof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION, ProcessMemoryLimit)); - P((int)JobObjectExtendedLimitInformation); - P(JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE); - - // static assertions for the values the koffi module will hardcode - static_assert(sizeof(STARTUPINFOW) == 104, "STARTUPINFOW size"); - static_assert(sizeof(PROCESS_INFORMATION) == 24, "PROCESS_INFORMATION size"); - static_assert(sizeof(SECURITY_ATTRIBUTES) == 24, "SECURITY_ATTRIBUTES size"); - static_assert(sizeof(EXPLICIT_ACCESS_W) == 48, "EXPLICIT_ACCESS_W size"); - static_assert(sizeof(TRUSTEE_W) == 32, "TRUSTEE_W size"); - static_assert(sizeof(SID_AND_ATTRIBUTES) == 16, "SID_AND_ATTRIBUTES size"); - static_assert(SECURITY_MAX_SID_SIZE == 68, "SECURITY_MAX_SID_SIZE"); - static_assert(TOKEN_QUERY == 0x8 && TOKEN_DUPLICATE == 0x2 && TOKEN_ADJUST_DEFAULT == 0x80 && TOKEN_ASSIGN_PRIMARY == 0x1, "token rights"); - static_assert(SE_GROUP_LOGON_ID == 0xC0000000, "logon id attr"); - static_assert(FILE_GENERIC_WRITE == 0x120116, "generic write"); - static_assert((FILE_GENERIC_WRITE & ~STANDARD_RIGHTS_WRITE) == 0x100116, "poc grant mask"); - static_assert(DELETE == 0x10000 && FILE_DELETE_CHILD == 0x40, "delete rights"); - static_assert(((FILE_GENERIC_WRITE | DELETE | FILE_DELETE_CHILD) & ~STANDARD_RIGHTS_WRITE) == 0x110156, "sandbox grant mask"); - static_assert(FILE_SHARE_READ == 0x1 && FILE_SHARE_WRITE == 0x2 && FILE_SHARE_DELETE == 0x4, "share modes"); - static_assert(OPEN_ALWAYS == 4, "open always"); - static_assert(LOCKFILE_EXCLUSIVE_LOCK == 0x2 && LOCKFILE_FAIL_IMMEDIATELY == 0x1, "lockfile flags"); - static_assert(ERROR_LOCK_VIOLATION == 33, "lock violation"); - static_assert(INHERITED_ACE == 0x10, "inherited ace flag"); - static_assert(GRANT_ACCESS == 1 && REVOKE_ACCESS == 4, "access modes"); - static_assert(SUB_CONTAINERS_AND_OBJECTS_INHERIT == 0x3, "inheritance"); - static_assert(CREATE_NO_WINDOW == 0x08000000, "create no window"); - static_assert(STARTF_USESTDHANDLES == 0x100, "std handles flag"); - static_assert(sizeof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION) == 144, "job extended limit size"); - static_assert(offsetof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION, BasicLimitInformation) + offsetof(JOBOBJECT_BASIC_LIMIT_INFORMATION, LimitFlags) == 16, "job LimitFlags offset"); - static_assert(JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE == 0x2000, "kill on job close flag"); - static_assert(JobObjectExtendedLimitInformation == 9, "extended limit class"); - printf("\nstatic_asserts passed\n"); - return 0; + static_assert(sizeof(EXPLICIT_ACCESS_W) == 48, "EXPLICIT_ACCESS_W size"); + static_assert(sizeof(TRUSTEE_W) == 32, "TRUSTEE_W size"); + static_assert(sizeof(SID_AND_ATTRIBUTES) == 16, "SID_AND_ATTRIBUTES size"); + static_assert(SECURITY_MAX_SID_SIZE == 68, "SECURITY_MAX_SID_SIZE"); + static_assert(TOKEN_QUERY == 0x8 && TOKEN_DUPLICATE == 0x2 && TOKEN_ADJUST_DEFAULT == 0x80 && TOKEN_ASSIGN_PRIMARY == 0x1, "token rights"); + static_assert(SE_GROUP_LOGON_ID == 0xC0000000, "logon id attr"); + static_assert(FILE_GENERIC_WRITE == 0x120116, "generic write"); + static_assert(DELETE == 0x10000 && FILE_DELETE_CHILD == 0x40, "delete rights"); + static_assert(((FILE_GENERIC_WRITE | DELETE | FILE_DELETE_CHILD) & ~STANDARD_RIGHTS_WRITE) == 0x110156, "sandbox grant mask"); + static_assert(FILE_SHARE_READ == 0x1 && FILE_SHARE_WRITE == 0x2 && FILE_SHARE_DELETE == 0x4, "share modes"); + static_assert(OPEN_ALWAYS == 4, "open always"); + static_assert(LOCKFILE_EXCLUSIVE_LOCK == 0x2 && LOCKFILE_FAIL_IMMEDIATELY == 0x1, "lockfile flags"); + static_assert(ERROR_LOCK_VIOLATION == 33, "lock violation"); + static_assert(INHERITED_ACE == 0x10, "inherited ace flag"); + static_assert(GRANT_ACCESS == 1 && REVOKE_ACCESS == 4, "access modes"); + static_assert(SUB_CONTAINERS_AND_OBJECTS_INHERIT == 0x3, "inheritance"); + printf("\nstatic_asserts passed\n"); + return 0; } diff --git a/packages/sandbox/sandbox/README.i18n.yaml b/packages/sandbox/sandbox/README.i18n.yaml index 7e7ed765b2..d92c0552ea 100644 --- a/packages/sandbox/sandbox/README.i18n.yaml +++ b/packages/sandbox/sandbox/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/sandbox/sandbox/README.md README.md: 3ebbffd623eee51dd6fcb588eec374e6d09be2f0 -README.zh.md: bde83955a85cbe2d60c521e3677b5b305a29ecb1 +README.zh.md: b9215b59d5ed5314275e91261c5541be1e5586fe diff --git a/packages/sandbox/sandbox/README.zh.md b/packages/sandbox/sandbox/README.zh.md index bde83955a8..b9215b59d5 100644 --- a/packages/sandbox/sandbox/README.zh.md +++ b/packages/sandbox/sandbox/README.zh.md @@ -2,23 +2,25 @@ [English](README.md) | 中文 -进程沙箱 Service Definition。负责定义 `ctx.sandbox` 服务约定([`SandboxProvider`](src/index.ts))与 harness 共享的限制词汇:`SandboxMode`(`read-only`/`workspace-write`/`danger-full-access`,仅限文件操作)、`SandboxEnforcement`(`full`/`partial`,针对每种内核 ABI)、`SandboxExecutionPolicy`(每次调用的完整模式及工作区根目录)、`SandboxPolicy`(其中受限制的子集),以及故障时拒绝放行的 `SANDBOX_UNAVAILABLE` 错误。作为[能力 seam 拆分](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)中的 Service Definition 角色,它只依赖 cordis(及 harness 错误基类),绝不依赖后端。 +进程沙箱 Service Definition。负责定义 `ctx.sandbox` 服务约定([`SandboxProvider`](src/index.ts))与 harness 共享的限制词汇:`SandboxMode`(`read-only`/`workspace-write`/`danger-full-access`,仅限文件操作)、`SandboxEnforcement`(`full`/`partial`,针对每种内核 ABI)、`SandboxExecutionPolicy`(每次调用的完整模式及工作区根目录)、`SandboxPolicy`(其中受限制的子集),以及故障时拒绝放行的 `SANDBOX_UNAVAILABLE` 错误。作为[能力 seam 拆分](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.zh.md)中的 Service Definition 角色,它只依赖 cordis(及 harness 错误基类),绝不依赖后端。 -用一句话概括约定:`ctx.sandbox.confine(argv, policy)` 返回用于 spawn、应当取代调用方原始 argv 的 argv。返回值经过包装,使进程及其派生的所有进程都在限制下运行;还会附带所选后端达到的强制执行完整度、拒绝方言(`denialSignatures`)和结构化 runner 失败证据(`runnerFailureRules`)。没有可用后端时,它会抛出异常,绝不会原样传递 argv 使其不受限制地运行。[核心类型目录](../../../docs/subsystems/sandbox.md#wrapped-argv-and-classification-dialects)负责定义分类器的精确结构。 +用一句话概括约定:`ctx.sandbox.confine(argv, policy)` 返回用于 spawn、应当取代调用方原始 argv 的 argv。返回值经过包装,使进程及其派生的所有进程都在限制下运行;还会附带所选后端达到的强制执行完整度、拒绝方言(`denialSignatures`)和结构化 runner 失败证据(`runnerFailureRules`)。没有可用后端时,它会抛出异常,绝不会原样传递 argv 使其不受限制地运行。[核心类型目录](../../../docs/subsystems/sandbox.zh.md#wrapped-argv-and-classification-dialects)负责定义分类器的精确结构。 策略随调用传递,而不属于提供方:两个消费方可以同时按不同策略施加限制(bash 使用 `read-only`,而受限制的子 agent(智能体)保持其状态目录可写);获批的升权重试只是使用更宽策略发起的新调用。 -**只支持与宿主共享文件系统和内核的限制。** 后端与宿主共享文件系统和内核(`bwrap`、Landlock、Seatbelt);`workspaceRoot` 指向文件系统规范化后的真实主机目录。系统先解析工作区所指的目录,再做词法规范化,因此包含 `symlink/..` 的有效 cwd 会授权 `chdir` 实际到达的目录,而非无关的词法父目录。容器、microVM 与远程执行器都不是该 seam 的后端:它们会以环境一致的分组替换整个能力 seam 的 Service Provider(`ctx.shell`、`ctx.fs`)。边界及其设计理由见[沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md)。 +**只支持与宿主共享文件系统和内核的限制。** 后端与宿主共享文件系统和内核(`bwrap`、Landlock、Seatbelt);`workspaceRoot` 指向文件系统规范化后的真实主机目录。系统先解析工作区所指的目录,再做词法规范化,因此包含 `symlink/..` 的有效 cwd 会授权 `chdir` 实际到达的目录,而非无关的词法父目录。容器、microVM 与远程执行器都不是该 seam 的后端:它们会以环境一致的分组替换整个能力 seam 的 Service Provider(`ctx.shell`、`ctx.fs`)。边界及其设计理由见[沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.zh.md)。 实现:[`@deepseek-ai/dsh-sandbox-local`](../sandbox-local/)(Linux:`bwrap`,否则使用相应平台的 Landlock launcher;macOS:`sandbox-exec`/Seatbelt)。消费方:[`@deepseek-ai/dsh-bash-sandbox`](../../shell/bash-sandbox/)(包装 `['bash', '-c', command]`)。 ## 模型体验 + + ### 间接的限制错误 #### 模型看到的内容 -通过 [`dsh-bash-sandbox`](../../shell/bash-sandbox/README.md) 和 [`dsh-tool-bash`](../../shell/tool-bash/README.md),无法强制执行所请求模式时会产生错误码 `SANDBOX_UNAVAILABLE` 及以下精确错误。执行期 runner 失败会追加 ` Runner failure: `。 +通过 [`dsh-bash-sandbox`](../../shell/bash-sandbox/README.zh.md) 和 [`dsh-tool-bash`](../../shell/tool-bash/README.zh.md),无法强制执行所请求模式时会产生错误码 `SANDBOX_UNAVAILABLE` 及以下精确错误。执行期 runner 失败会追加 ` Runner failure: `。 ##### 精确错误 diff --git a/packages/sandbox/sandbox/package.json b/packages/sandbox/sandbox/package.json index 00ebcfd612..3bec839e4d 100644 --- a/packages/sandbox/sandbox/package.json +++ b/packages/sandbox/sandbox/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-sandbox", "description": "Abstract process-sandbox seam (ctx.sandbox) for the DeepSeek Harness: same-world confinement vocabulary and the SandboxProvider contract", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/schedule/README.i18n.yaml b/packages/schedule/README.i18n.yaml index 4d04edf194..90aa318ed0 100644 --- a/packages/schedule/README.i18n.yaml +++ b/packages/schedule/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/schedule/README.md README.md: 2fac190cc19f40e5d5e87acacaddce970e4b8474 -README.zh.md: 3db72e7e6650e38ab51fd75087d95cb2faf121d4 +README.zh.md: af4cd96dcb5f095a7ea83556e5f73eca532d0dee diff --git a/packages/schedule/README.zh.md b/packages/schedule/README.zh.md index 3db72e7e66..af4cd96dcb 100644 --- a/packages/schedule/README.zh.md +++ b/packages/schedule/README.zh.md @@ -10,4 +10,4 @@ Schedule 家族负责管理提醒,其持久状态保存在原 Session 日志 本包有意不公开 Schedule service 或可变数据库。工具与 runtime 向 Session stream 追加事件;到期工作通过 Agent 的普通 follow-up 队列进入同一对话。 -有关持久记录、转换、视图与交付约定,请参阅[仅限 Session 内的 Schedule](../../docs/subsystems/schedule.md)。 +有关持久记录、转换、视图与交付约定,请参阅[仅限 Session 内的 Schedule](../../docs/subsystems/schedule.zh.md)。 diff --git a/packages/schedule/schedule/README.i18n.yaml b/packages/schedule/schedule/README.i18n.yaml index 76577697c4..8dc406becf 100644 --- a/packages/schedule/schedule/README.i18n.yaml +++ b/packages/schedule/schedule/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/schedule/schedule/README.md README.md: f56c689198b7e17a85f43a1ecb6e5f1527da6a91 -README.zh.md: 75425c77d2b43c6511c400a55bb5f8f898f978cc +README.zh.md: 7180565370d83a7207c73aa9a2539219974dace0 diff --git a/packages/schedule/schedule/README.zh.md b/packages/schedule/schedule/README.zh.md index 75425c77d2..7180565370 100644 --- a/packages/schedule/schedule/README.zh.md +++ b/packages/schedule/schedule/README.zh.md @@ -26,7 +26,7 @@ Schedule 负责确定性的日历规范化。落在夏令时缺口内的本地 ## 管理工具 -生成的[工具目录](../../../docs/tool-catalog.md)负责 `schedule_create`、`schedule_list` 和 `schedule_delete` 的参数与输出 schema。虽然模型输入使用 `after_seconds` 和 `time_zone`,但其规范值中的记录字段使用 camelCase。 +生成的[工具目录](../../../docs/tool-catalog.zh.md)负责 `schedule_create`、`schedule_list` 和 `schedule_delete` 的参数与输出 schema。虽然模型输入使用 `after_seconds` 和 `time_zone`,但其规范值中的记录字段使用 camelCase。 一条 Agent-scoped 队列会将每项已接纳的管理事务与 live owner 的到期事务从 preflight 到任何 post-append barrier 全程串行化。`schedule_create` 要求 `after_seconds`、`at` 与 `every_seconds` 有且只有一项;它会在进入队列前验证只依赖输入形状的失败,随后执行检查点、分配永不复用的 id、追加 create,再次执行检查点。`schedule_list` 按创建顺序返回活动记录,其中包含 `state: "scheduled" | "overdue"` 与 `deliveryMode: "session-local"`。`schedule_delete` 会在进入队列前拒绝空 id 或前后带空白的 id,并只为活动 id 追加事件;未知或已终结的 id 会在 preflight 后返回 `{ id, deleted: false, code: "schedule_not_found" }`。 diff --git a/packages/schedule/schedule/package.json b/packages/schedule/schedule/package.json index f7e2fd6406..9cf315d544 100644 --- a/packages/schedule/schedule/package.json +++ b/packages/schedule/schedule/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-schedule", "description": "Agent-scoped durable after, at, and fixed-rate reminders over the session event log", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/schedule/schedule/tests/runtime.spec.ts b/packages/schedule/schedule/tests/runtime.spec.ts index d49f33b715..609c68a6cd 100644 --- a/packages/schedule/schedule/tests/runtime.spec.ts +++ b/packages/schedule/schedule/tests/runtime.spec.ts @@ -679,7 +679,9 @@ describe('Schedule runtime failure and teardown boundaries', () => { const runFailure = await harness() appendAfter(runFailure, 'schedule-1', 1, Date.now() - 1_000) - const uuidSpy = vi.spyOn(globalThis.crypto, 'randomUUID').mockImplementation(() => { throw 'message failed' }) + // The reminder message mints its id through dsh-util-crypto, whose + // entropy source is getRandomValues — the failure injection follows it. + const uuidSpy = vi.spyOn(globalThis.crypto, 'getRandomValues').mockImplementation(() => { throw 'message failed' }) const failingRuntime = runtimeFor(runFailure) failingRuntime.start() for (let index = 0; index < 12; index += 1) await Promise.resolve() @@ -690,7 +692,7 @@ describe('Schedule runtime failure and teardown boundaries', () => { const departedRun = await harness() appendAfter(departedRun, 'schedule-1', 1, Date.now() - 1_000) - const departedUuidSpy = vi.spyOn(globalThis.crypto, 'randomUUID').mockImplementation(() => { + const departedUuidSpy = vi.spyOn(globalThis.crypto, 'getRandomValues').mockImplementation(() => { departedRun.disposeAgent() throw 'message failed after detach' }) diff --git a/packages/sdk/README.i18n.yaml b/packages/sdk/README.i18n.yaml index c9fc6e2600..05788b05a5 100644 --- a/packages/sdk/README.i18n.yaml +++ b/packages/sdk/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/sdk/README.md -README.md: 052ac933defae8766e99d79b79bc4cdc6c6d74db -README.zh.md: 364f752fcbed06cae8b0675cae852be2760bd3f4 +README.md: 663fa95dc56ecd71c7a2639b6edb45caacbab860 +README.zh.md: 0343ad477f9fa522666cd8d4440bfe645f309c2e diff --git a/packages/sdk/README.md b/packages/sdk/README.md index 052ac933de..663fa95dc5 100644 --- a/packages/sdk/README.md +++ b/packages/sdk/README.md @@ -2,10 +2,11 @@ English | [中文](README.zh.md) -This group contains the protocol stack for driving a Harness runtime from another process. Callers supply the runtime executable and its `cordis.yml`; this group does not create, configure, build, or launch developer projects. The [TypeScript SDK decision](../../.agents/notes/implemented/feature/2026-07-27-typescript-sdk-and-sdk-subagent-backend.md) owns the client contract, and the [toolchain removal](../../.agents/notes/implemented/simplification/2026-08-11-remove-sdk-project-toolchain.md) owns the product boundary. +This group contains the protocol stack for driving a Harness runtime from another process. The TypeScript client launches the matching `dsh` CLI with a named profile and ordered patches; the private Python carrier preserves the current packaged direct-config runtime until Python moves through the same profile path. The [TypeScript SDK decision](../../.agents/notes/implemented/feature/2026-07-27-typescript-sdk-and-sdk-subagent-backend.md) owns the client contract, and the [toolchain removal](../../.agents/notes/implemented/simplification/2026-08-11-remove-sdk-project-toolchain.md) owns the product boundary. | Package | Role | |---|---| | [`protocol/`](protocol/README.md) | Defines the SDK runtime wire protocol | | [`client/`](client/README.md) | Drives a Harness runtime through the TypeScript client API | | [`server/`](server/README.md) | Serves out-of-process SDK clients over stdio JSON-RPC | +| [`python-runtime/`](python-runtime/README.md) | Private direct-config carrier for the temporarily unchanged Python SDK runtime | diff --git a/packages/sdk/README.zh.md b/packages/sdk/README.zh.md index 364f752fcb..0343ad477f 100644 --- a/packages/sdk/README.zh.md +++ b/packages/sdk/README.zh.md @@ -2,10 +2,11 @@ [English](README.md) | 中文 -本组包含用于从另一进程驱动 Harness 运行时的协议栈。调用方提供运行时可执行文件及其 `cordis.yml`;本组不创建、配置、构建或启动开发者项目。[TypeScript SDK 决策](../../.agents/notes/implemented/feature/2026-07-27-typescript-sdk-and-sdk-subagent-backend.md)负责客户端约定,[工具链移除](../../.agents/notes/implemented/simplification/2026-08-11-remove-sdk-project-toolchain.md)负责产品边界。 +本组包含用于从另一进程驱动 Harness 运行时的协议栈。TypeScript 客户端通过具名 profile 与有序 patch 启动匹配版本的 `dsh` CLI;私有 Python 载体在 Python 迁移到同一 profile 路径之前,保留当前打包后的直读配置运行时。[TypeScript SDK 决策](../../.agents/notes/implemented/feature/2026-07-27-typescript-sdk-and-sdk-subagent-backend.zh.md)负责客户端约定,[工具链移除](../../.agents/notes/implemented/simplification/2026-08-11-remove-sdk-project-toolchain.zh.md)负责产品边界。 | 包 | 职责 | |---|---| -| [`protocol/`](protocol/README.md) | 定义 SDK 运行时通信协议 | -| [`client/`](client/README.md) | 通过 TypeScript 客户端 API 驱动 Harness 运行时 | -| [`server/`](server/README.md) | 通过 stdio JSON-RPC 为进程外 SDK 客户端提供服务 | +| [`protocol/`](protocol/README.zh.md) | 定义 SDK 运行时通信协议 | +| [`client/`](client/README.zh.md) | 通过 TypeScript 客户端 API 驱动 Harness 运行时 | +| [`server/`](server/README.zh.md) | 通过 stdio JSON-RPC 为进程外 SDK 客户端提供服务 | +| [`python-runtime/`](python-runtime/README.zh.md) | 为暂时保持不变的 Python SDK 运行时提供私有直读配置载体 | diff --git a/packages/sdk/client/README.i18n.yaml b/packages/sdk/client/README.i18n.yaml index da2067692b..de336b04da 100644 --- a/packages/sdk/client/README.i18n.yaml +++ b/packages/sdk/client/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/sdk/client/README.md -README.md: b33457875f81d11d09bab2e5aa5ce730e233c78a -README.zh.md: 271f07ffb0f97abe005971962beb517acfdc05a4 +README.md: bf4f6bcaf2f0928cc7aa95d18f0cbbff3cdbe37d +README.zh.md: ba64ab19c6ba1e9ad685585330fe8369b2ccb381 diff --git a/packages/sdk/client/README.md b/packages/sdk/client/README.md index b33457875f..bf4f6bcaf2 100644 --- a/packages/sdk/client/README.md +++ b/packages/sdk/client/README.md @@ -2,9 +2,11 @@ English | [中文](README.zh.md) -The TypeScript client SDK for driving a DeepSeek Harness runtime as a subprocess over stdio JSON-RPC — the design twin of the [Python SDK](../../../python/README.md) (`deepseek-harness`), sharing the same runtime peer, protocol, and layering: `DeepSeekHarness` is the high-level owned-run API, `HarnessClient` the lower-level protocol client. The package root enumerates the consumer interface: the two client layers, caller-facing types, and `JsonRpcResponseError`; source modules, normalization helpers, and subscription-delivery machinery are not consumer imports. A pure library: it registers nothing on a Cordis context; the runtime process it spawns is a complete harness whose composition its own `cordis.yml` decides. +The TypeScript client SDK for driving the same-version [`dsh`](../../../apps/cli/README.md) runtime over stdio JSON-RPC. `DeepSeekHarness` is the high-level owned-run API; `HarnessClient` is the lower-level protocol client. The package depends on `@deepseek-ai/dsh` and resolves that installed CLI directly, so ordinary consumers do not discover a runtime executable or maintain a second application configuration. A clean source checkout whose `lib/bin.js` does not exist launches the same package's `src/bin.ts` through its resolved `tsx/esm` loader and an internal patch that omits build-generated Typert contribution loading; the SDK JSON-RPC application does not consume that remote gateway. An installed package uses the complete built entry. -Unlike the Python SDK, the launch spec is fully explicit (`command`/`args`): this package is for repo-adjacent TypeScript consumers — including the [`dsh-subagent-dsh-sdk`](../../subagent/subagent-dsh-sdk/README.md) backend and automation — that know which runtime they are launching. Bundled-runtime resolution (finding a packaged executable) remains the Python distribution's concern. +Both client layers accept the same launch fields: `dshBin?`, `profile?` (default `sdk`), ordered `patches?`, `dshHome?`, `processCwd?`, `env?`, and request/initialize/shutdown/disposal timeouts. Caller-relative CLI-module, patch, explicit home, and process-cwd paths become absolute before spawn. The client runs the dsh CLI module through its current Node executable on every platform. An omitted home keeps normal dsh resolution (`DSH_HOME`, then `~/.dsh`); an explicit home overrides the child environment. `env` replaces the child environment when supplied; the client reads either that object or `process.env` when `start()` actually spawns, so mutations before the first start are visible. + +Composition customization stays in the profile system. Install persistent bundles and plugin dependencies with `dsh plugin --profile …`, edit that profile's `cordis.patch.yml`, and select it with `profile`. Use `patches` for ordered per-launch overrides. A patch replaces a row's complete config, and a custom profile must retain `@deepseek-ai/dsh-sdk-app` or another SDK server row. ## DeepSeekHarness @@ -12,7 +14,8 @@ Unlike the Python SDK, the launch spec is fully explicit (`command`/`args`): thi import { DeepSeekHarness } from '@deepseek-ai/dsh-sdk-client' await using harness = new DeepSeekHarness({ - launch: { command: 'node', args: ['lib/bin.js', 'cordis.yml'] }, + profile: 'sdk', + patches: ['./automation.cordis.yml'], provider: 'deepseek-official', model: 'deepseek-v4-flash', maxTokens: 49_152, @@ -21,29 +24,27 @@ const result = await harness.run('say hi') console.log(result.finalResponse) ``` -The subprocess starts lazily on first use and stays owned by the instance across `run()` calls; `close()` (or `await using`) is required so the child is always reaped. `start()` memoizes the `initialize` handshake (the workspace cwd — resolved absolute before it crosses the wire — plus the provider/model route and optional positive `maxTokens` output cap); a failed handshake reaps the runtime and swaps in a fresh client, so a later call retries with a new subprocess (until `close()`, which is terminal). The cap applies to each root-agent request and is inherited by in-process descendants; compaction plugins own their separate summary limits. `session(id?)` opens a named or fresh session handle. +The dsh process starts lazily on first use and stays owned across `run()` calls. `close()` (or `await using`) is required. `start()` memoizes the bounded `initialize` handshake; `initializeTimeoutMs` defaults to 10 seconds and its diagnostic names the selected profile with the retained stderr tail. A failed handshake reaps the runtime and lets a later call retry with a fresh process until terminal `close()`. -`run(input, { sessionId?, onNotification? })` owns one activity interval: it queues the prompt, waits until its `MessageId` appears in a durable `agent/inbox/spliced` receipt, then collects through the next whole-agent `idle`. It returns `RunResult { sessionId, finalResponse, events, notifications }`. `finalResponse` is the last committed root-session assistant text in that interval, not a response causally assigned to the prompt; steering, injected context, and other queued work may contribute before idle. `events` contains root-session events, while `notifications` also contains descendants discovered from `subagent.started`, all in wire order. The result carries no prompt-level status or turn reason. Transport loss, timeout, and protocol violations reject; model outcomes remain observable in the event stream without being attributed to one input. +The handshake carries the absolute session workspace plus provider/model and optional positive `maxTokens`. `run(input, { sessionId?, onNotification? })` queues a prompt, waits for its durable inbox receipt, and collects until the whole root agent next becomes idle. It returns `RunResult { sessionId, finalResponse, events, notifications }`; `events` is root-scoped, while notifications also contain discovered descendants. ## HarnessClient -The protocol client under the owned-run API: explicit `start()`/`initialize()`/`prompt()`/`request()`/`close()`, plus notification subscriptions. `prompt()` returns the queued message id as soon as the runtime accepts it; it never waits for agent activity. `subscribe(filter?)` returns a `NotificationSubscription` (awaitable `next()`, non-blocking `tryNext()`, async iteration); `subscribeSessionTree(id)` scopes to one session and the descendants discovered from `subagent.started` lineage edges — the runtime notifies for every session in its context, and scoping is client-side, exactly like the Python SDK. Error surfaces are typed and exported from this package: `JsonRpcResponseError` (wire error response, code/data preserved), `RequestTimeoutError` (a configured bound elapsed), `SdkProtocolError` (a response outside the documented protocol), `TransportClosedError` (the runtime is gone — message carries the exit code and a bounded stderr tail). +The low-level client exposes `start()`/`initialize()`/`prompt()`/`request()`/`close()` and notification subscriptions. `prompt()` returns the durable message id after enqueue, not a prompt result. `subscribeSessionTree(id)` scopes the process-wide notification stream to one session lineage. Exported failures are `JsonRpcResponseError`, `RequestTimeoutError`, `SdkProtocolError`, and `TransportClosedError`. -`close()` requests protocol `shutdown` (bounded by `shutdownTimeoutMs`, default 1000 ms), then walks a stdin-EOF → SIGTERM → SIGKILL ladder (`disposeEofGraceMs` default 6000, `disposeGraceMs` default 3000) until the process has actually exited. The ladder is private to this client: it runs outside any harness context, so it cannot ride the [`dsh-subprocess`](../../subprocess/README.md) service — the seam's documented exception for SDK-managed transports. It is idempotent, and a closed client refuses reuse. - -`HarnessClientOptions.env` replaces the child environment entirely when given (`undefined` inherits the parent's); callers own credential policy — `scrubbedParentEnv` from `dsh-subprocess` is the shared scrub base for isolation-minded launches. +`close()` requests protocol `shutdown` (default bound 1000 ms), then uses stdin EOF → SIGTERM → SIGKILL (`disposeEofGraceMs` 6000, `disposeGraceMs` 3000) until the process exits. This client lives outside any Harness context, so its private process adapter is the documented SDK-managed transport exception to `dsh-subprocess`; generic command/argv launching is package-test machinery, not a consumer interface. ## Model Experience -None, as this is a client-process library; the model runs in the spawned runtime, whose experience is owned by the plugins its `cordis.yml` composes. +None, as this library adds no model-visible content; the selected dsh profile owns the spawned model's prompt, tools, policy, and cache prefix (see [`dsh-sdk-app`](../../bundle/sdk-app/README.md)). #### KV Cache effect -None; this package neither assembles nor sends a provider request. +None in the client process. Profile, patch, provider, model, and history choices determine cache reuse in the child. ## Known Limitations and Deferred Work -- **No bundled-runtime resolution** — callers name the runtime executable explicitly; packaged-executable discovery stays Python-side until a TypeScript distribution consumer exists. -- **No mid-turn cancel** — the wire has no prompt-cancel method; abandoning a turn means closing the runtime (see the protocol's [Known Limitations](../protocol/README.md)). -- **No per-prompt result or cancel** — low-level `prompt()` returns only an enqueue receipt; high-level `run()` owns receipt-to-idle collection, and abandoning it means closing the runtime. -- **Client→server notifications and server→client requests are unimplemented** on both wire ends; the transport carries them for future approval flows. +- **A selected profile can omit the SDK server** — initialization fails at its configured bound and names that profile; retain the SDK app bundle or an equivalent server row. +- **No mid-turn cancel or per-prompt result** — abandoning an owned activity means closing the runtime; model outcomes remain in session events. +- **Trusted patches can violate stdout purity** — the shipped SDK profile writes only protocol frames, but arbitrary user plugins own their output behavior. +- **Client→server notifications and server→client requests are unimplemented** on both wire ends. diff --git a/packages/sdk/client/README.zh.md b/packages/sdk/client/README.zh.md index 271f07ffb0..ba64ab19c6 100644 --- a/packages/sdk/client/README.zh.md +++ b/packages/sdk/client/README.zh.md @@ -2,9 +2,11 @@ [English](README.md) | 中文 -以子进程方式驱动 DeepSeek Harness 运行时、走 stdio JSON-RPC 的 TypeScript 客户端 SDK——[Python SDK](../../../python/README.md)(`deepseek-harness`)的设计孪生,共享同一个运行时对端、协议与分层:`DeepSeekHarness` 是高层自有运行 API,`HarnessClient` 是低层协议客户端。包(package)根枚举消费方接口:两层客户端、面向调用方的类型和 `JsonRpcResponseError`;源模块、规范化辅助函数与订阅投递机制不供消费方导入。纯库:不在任何 Cordis 上下文注册;它所 spawn 的运行时进程是一个完整 harness,其组成由自己的 `cordis.yml` 决定。 +通过 stdio JSON-RPC 驱动同版本 [`dsh`](../../../apps/cli/README.zh.md) runtime 的 TypeScript client SDK。`DeepSeekHarness` 是高层自有运行 API,`HarnessClient` 是低层协议 client。本包依赖 `@deepseek-ai/dsh` 并直接解析随安装的 CLI,因此普通消费方无需发现 runtime 可执行文件,也无需维护第二套应用配置。干净源码 checkout 中若不存在 `lib/bin.js`,client 会通过已解析的 `tsx/esm` loader 启动同一包的 `src/bin.ts`,并应用一个省略构建期生成 Typert 贡献加载的内部 patch;SDK JSON-RPC 应用不消费该远程网关。已安装包使用完整的构建后入口。 -与 Python SDK 不同,启动规格完全显式(`command`/`args`):本包面向仓库近旁的 TypeScript 消费方,包括 [`dsh-subagent-dsh-sdk`](../../subagent/subagent-dsh-sdk/README.md) 后端和自动化;它们知道自己要启动哪个运行时。捆绑运行时解析(寻找打包可执行文件)仍归 Python 发行版负责。 +两层 client 接受同一组启动字段:`dshBin?`、`profile?`(默认 `sdk`)、有序 `patches?`、`dshHome?`、`processCwd?`、`env?`,以及请求、初始化、关闭和 dispose 超时。相对调用方的 CLI 模块、patch、显式 home 与进程 cwd 路径会在 spawn 前转为绝对路径。client 会在所有平台上通过自身当前的 Node 可执行文件运行 dsh CLI 模块。省略 home 时保留 dsh 的普通解析(先 `DSH_HOME`,再 `~/.dsh`);显式 home 会覆盖子进程环境。提供 `env` 时,它整体替换子进程环境;client 会在 `start()` 实际 spawn 时读取该对象或 `process.env`,所以首次启动前的修改会生效。 + +组合自定义属于 profile 系统。使用 `dsh plugin --profile …` 安装持久组合包与插件依赖,编辑该 profile 的 `cordis.patch.yml`,再通过 `profile` 选择。`patches` 用于有序的逐次启动覆盖。patch 会替换配置行的完整 config;自定义 profile 必须保留 `@deepseek-ai/dsh-sdk-app` 或另一个 SDK server 配置行。 ## DeepSeekHarness @@ -12,7 +14,8 @@ import { DeepSeekHarness } from '@deepseek-ai/dsh-sdk-client' await using harness = new DeepSeekHarness({ - launch: { command: 'node', args: ['lib/bin.js', 'cordis.yml'] }, + profile: 'sdk', + patches: ['./automation.cordis.yml'], provider: 'deepseek-official', model: 'deepseek-v4-flash', maxTokens: 49_152, @@ -21,29 +24,27 @@ const result = await harness.run('say hi') console.log(result.finalResponse) ``` -子进程在首次使用时惰性启动,并在多次 `run()` 之间持续归实例所有;必须 `close()`(或 `await using`),子进程才总能被回收。`start()` 记忆化 `initialize` 握手(工作区 cwd——在通过协议传输之前解析为绝对路径——加 provider/model 路由和可选的正整数 `maxTokens` 输出上限);握手失败会回收运行时并换入全新客户端,后续调用用新子进程重试(直到终结性的 `close()`)。该上限作用于根 agent(智能体)的每次请求,并由进程内后代继承;压缩(compaction)插件单独持有摘要上限。`session(id?)` 打开具名或全新的会话句柄。 +dsh 进程在首次使用时惰性启动,并在多次 `run()` 之间持续归实例所有;必须调用 `close()`(或使用 `await using`)。`start()` 会记忆化有界的 `initialize` 握手;`initializeTimeoutMs` 默认 10 秒,诊断会写明所选 profile 并附带保留的 stderr 尾部。握手失败会回收 runtime,之后的调用可以用新进程重试,直至终结性的 `close()`。 -`run(input, { sessionId?, onNotification? })` 拥有一个活动区间:它将提示词排入队列,等待其 `MessageId` 出现在持久的 `agent/inbox/spliced` 回执中,然后持续收集到整个 agent 下一次进入 `idle`。它返回 `RunResult { sessionId, finalResponse, events, notifications }`。`finalResponse` 是该区间内根会话最后提交的助手文本,并非因果上归属于该提示词的响应;steering(中途引导)、注入的上下文和其他排队工作都可能在 idle 前参与其中。`events` 包含根会话事件,`notifications` 还包含通过 `subagent.started` 发现的后代,均按协议传输顺序排列。结果不携带提示词级状态或轮次原因。传输丢失、超时和协议违例会导致 Promise 被拒绝;模型结果仍可在事件流中观察,但不会归属于某一输入。 +握手携带绝对 session workspace、provider/model 和可选的正整数 `maxTokens`。`run(input, { sessionId?, onNotification? })` 将 prompt 入队,等待持久 inbox 回执,并收集到整个根 agent 下次 idle。它返回 `RunResult { sessionId, finalResponse, events, notifications }`;`events` 仅限根 session,notification 还包括发现的后代。 ## HarnessClient -自有运行 API 之下的协议客户端:显式 `start()`/`initialize()`/`prompt()`/`request()`/`close()`,外加通知订阅。`prompt()` 在运行时接受排队消息后立即返回该消息的 ID,绝不等待 agent 活动。`subscribe(filter?)` 返回 `NotificationSubscription`(可等待的 `next()`、非阻塞 `tryNext()`、异步迭代);`subscribeSessionTree(id)` 把范围限定到一个会话及从 `subagent.started` 血缘边发现的后代——运行时对上下文内每个会话都发通知,范围限定在客户端完成,与 Python SDK 完全一致。本包导出有明确类型的错误:`JsonRpcResponseError`(协议错误响应,保留 code/data)、`RequestTimeoutError`(配置的时限已到)、`SdkProtocolError`(响应超出文档化协议)、`TransportClosedError`(运行时已消失——消息携带退出码与有界 stderr 尾部)。 +低层 client 提供 `start()`/`initialize()`/`prompt()`/`request()`/`close()` 与 notification 订阅。`prompt()` 返回入队后的持久消息 ID,而不是 prompt 结果。`subscribeSessionTree(id)` 把进程级 notification 流限定到一个 session 血缘。导出的失败类型为 `JsonRpcResponseError`、`RequestTimeoutError`、`SdkProtocolError` 与 `TransportClosedError`。 -`close()` 先请求协议 `shutdown`(受 `shutdownTimeoutMs` 约束,默认 1000 毫秒),然后走 stdin-EOF → SIGTERM → SIGKILL 阶梯(`disposeEofGraceMs` 默认 6000,`disposeGraceMs` 默认 3000)直到进程真正退出。该阶梯为本客户端私有:它运行在任何 harness 上下文之外,无法搭乘 [`dsh-subprocess`](../../subprocess/README.md) 服务——即该 seam 所记录的 SDK 托管传输例外。幂等,已关闭的客户端拒绝复用。 - -`HarnessClientOptions.env` 给定时整体替换子进程环境(`undefined` 原样继承父进程环境);凭据策略归调用方——`dsh-subprocess` 的 `scrubbedParentEnv` 是面向隔离启动的共享擦除基底。 +`close()` 先请求协议 `shutdown`(默认上限 1000 毫秒),再执行 stdin EOF → SIGTERM → SIGKILL(`disposeEofGraceMs` 6000、`disposeGraceMs` 3000),直到进程退出。client 位于任何 Harness context 之外,因此其私有进程 adapter 是 `dsh-subprocess` 中记录的 SDK 托管传输例外;通用 command/argv 启动仅是包测试机制,不是消费方接口。 ## 模型体验 -无,因为这是一个客户端进程库;模型运行在 spawn 出的运行时中,其体验由该运行时的 `cordis.yml` 所组合的插件决定。 +本库不会增加任何模型可见内容;所选 dsh profile 负责子进程模型的 prompt、工具、策略和缓存前缀(见 [`dsh-sdk-app`](../../bundle/sdk-app/README.zh.md))。 #### KV Cache 影响 -无;本包既不组装也不发送提供方请求。 +client 进程中无影响。子进程的 profile、patch、provider、model 与历史决定缓存复用。 ## 已知限制与暂缓事项 -- **无捆绑运行时解析**——调用方显式指定运行时可执行文件;打包可执行文件的发现留在 Python 侧,直到出现 TypeScript 发行版消费方。 -- **无轮次中取消**——协议层没有提示词取消方法;放弃轮次意味着关闭运行时(见协议的 [已知限制](../protocol/README.md))。 -- **没有逐提示词结果或取消**——低层 `prompt()` 只返回入队回执;高层 `run()` 负责从回执收集到 idle,放弃该过程意味着关闭运行时。 -- **客户端→服务端通知与服务端→客户端请求**在协议两端都未实现;传输层为未来审批流保留了承载能力。 +- **所选 profile 可以省略 SDK server**:初始化会在配置的上限失败并写明 profile;请保留 SDK app 组合包或等价 server 配置行。 +- **没有轮次中取消或逐 prompt 结果**:放弃自有活动意味着关闭 runtime;模型结果保留在 session event 中。 +- **受信任 patch 可能破坏 stdout 纯净性**:随附 SDK profile 只写协议 frame,但任意用户插件负责自己的输出行为。 +- **client→server notification 与 server→client request 尚未实现**。 diff --git a/packages/sdk/client/package.json b/packages/sdk/client/package.json index 6e8d4d5dd9..b733286d89 100644 --- a/packages/sdk/client/package.json +++ b/packages/sdk/client/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-sdk-client", "description": "TypeScript client SDK for driving a DeepSeek Harness runtime subprocess over stdio JSON-RPC: the DeepSeekHarness high-level turns API and the lower-level HarnessClient", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -30,6 +30,9 @@ "lib/types/**/*.d.ts" ], "license": "MIT", + "dependencies": { + "@deepseek-ai/dsh": "workspace:*" + }, "peerDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", diff --git a/packages/sdk/client/src/api.ts b/packages/sdk/client/src/api.ts index d615caece5..3104f77891 100644 --- a/packages/sdk/client/src/api.ts +++ b/packages/sdk/client/src/api.ts @@ -2,7 +2,6 @@ * High-level run API over {@link HarnessClient}: `DeepSeekHarness` owns one * runtime subprocess across many sessions; `HarnessSession.run` sends a * prompt and settles when the whole agent next becomes idle. - * Mirrors the Python SDK's `DeepSeekHarness`/`Session` pair. * * @module @deepseek-ai/dsh-sdk-client/api */ @@ -10,8 +9,9 @@ import { randomUUID } from 'node:crypto' import { resolve } from 'node:path' import type { SessionEvent } from '@deepseek-ai/dsh-session' -import { HarnessClient, isRecord, SdkProtocolError } from './client.ts' -import type { ContentBlock, DeepSeekHarnessOptions, HarnessClientOptions, HarnessNotification, RunResult } from './types.ts' +import { createProcessHarnessClient, HarnessClient, isRecord, SdkProtocolError } from './client.ts' +import type { RuntimeProcessOptions } from './launch.ts' +import type { ContentBlock, DeepSeekHarnessOptions, HarnessNotification, RunResult } from './types.ts' /** * Reusable SDK for running DeepSeek Harness agent turns in a runtime @@ -21,7 +21,7 @@ import type { ContentBlock, DeepSeekHarnessOptions, HarnessClientOptions, Harnes */ export class DeepSeekHarness implements AsyncDisposable { private clientInstance: HarnessClient - private readonly launch: HarnessClientOptions + private readonly createClient: () => HarnessClient private readonly cwd: string private readonly provider: string private readonly model: string @@ -29,14 +29,15 @@ export class DeepSeekHarness implements AsyncDisposable { private initialized: Promise | undefined private closed = false - /** @param options - runtime launch spec plus the session route (cwd/provider/model). */ - constructor(options: DeepSeekHarnessOptions) { - this.launch = options.launch - this.clientInstance = new HarnessClient(options.launch) + /** @param options - dsh launch configuration plus the session route. */ + constructor(options?: DeepSeekHarnessOptions) + constructor(options: DeepSeekHarnessOptions = {}, clientFactory?: () => HarnessClient) { + this.createClient = clientFactory ?? (() => new HarnessClient(options)) + this.clientInstance = this.createClient() // Absolute before the handshake: the child spawns relative to THIS // process's cwd, but the wire cwd is resolved again inside the child — a // relative value would double-resolve (e.g. `worker` → `worker/worker`). - this.cwd = resolve(options.cwd ?? options.launch.cwd ?? process.cwd()) + this.cwd = resolve(options.cwd ?? options.processCwd ?? process.cwd()) this.provider = options.provider ?? 'deepseek-official' this.model = options.model ?? 'deepseek-v4-flash' this.maxTokens = options.maxTokens @@ -72,7 +73,7 @@ export class DeepSeekHarness implements AsyncDisposable { } catch (error) { this.initialized = undefined await this.clientInstance.close() - if (!this.closed) this.clientInstance = new HarnessClient(this.launch) + if (!this.closed) this.clientInstance = this.createClient() throw error } })() @@ -118,6 +119,21 @@ export class DeepSeekHarness implements AsyncDisposable { } } +/** Construct the high-level API against a generic process for package-local fake-runtime tests. */ +export function createProcessDeepSeekHarness( + runtime: RuntimeProcessOptions, + options: DeepSeekHarnessOptions = {}, +): DeepSeekHarness { + const Constructor = DeepSeekHarness as unknown as new ( + publicOptions: DeepSeekHarnessOptions, + clientFactory: () => HarnessClient, + ) => DeepSeekHarness + return new Constructor({ + ...runtime.cwd === undefined ? {} : { processCwd: runtime.cwd }, + ...options, + }, () => createProcessHarnessClient(runtime)) +} + /** Per-run options: target session and streaming observer. */ export interface RunOptions { /** Session id to run on; omitted mints a fresh session per call. */ diff --git a/packages/sdk/client/src/client.ts b/packages/sdk/client/src/client.ts index 1937f4a1dc..a4482b4a89 100644 --- a/packages/sdk/client/src/client.ts +++ b/packages/sdk/client/src/client.ts @@ -22,6 +22,7 @@ import { } from '@deepseek-ai/dsh-sdk-protocol' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import { disposeRuntimeProcess } from './dispose.ts' +import { resolveDshLaunch, type RuntimeProcessOptions } from './launch.ts' import type { HarnessClientOptions, HarnessNotification, NotificationFilter } from './types.ts' /** Retained stderr lines used to diagnose an unexpected runtime death. */ @@ -144,7 +145,7 @@ class NotificationSubscriptionImpl implements NotificationSubscription { * Deliver one notification to a waiter or the queue when the filter * matches. A throwing filter fails only THIS subscription (detached, the * throw becomes its terminal error) — it never disturbs sibling - * subscriptions or the transport's read loop, mirroring the Python client. + * subscriptions or the transport's read loop. * @param notification - the wire notification to deliver. */ push(notification: HarnessNotification): void { @@ -182,6 +183,9 @@ class NotificationSubscriptionImpl implements NotificationSubscription { * runtime is closed. */ export class HarnessClient { + /** Original public dsh launch and timeout options for this client. */ + readonly options: HarnessClientOptions + private readonly runtime: RuntimeProcessOptions private child: ChildProcess | undefined private transport: JsonRpcLineTransport | undefined private readonly stderrTail: string[] = [] @@ -193,8 +197,12 @@ export class HarnessClient { private streamsSettled: Promise = Promise.resolve() private closeTask: Promise | undefined - /** @param options - launch spec, complete child environment, and timeouts. */ - constructor(readonly options: HarnessClientOptions) {} + /** @param options - dsh profile, patch, home, process, environment, and timeout options. */ + constructor(options?: HarnessClientOptions) + constructor(options: HarnessClientOptions = {}, runtime?: RuntimeProcessOptions) { + this.options = options + this.runtime = runtime ?? resolveDshLaunch(options) + } /** * Spawn the runtime subprocess and start reading frames. Idempotent while @@ -203,9 +211,9 @@ export class HarnessClient { start(): void { if (this.closeTask !== undefined) throw new TransportClosedError('DeepSeek Harness runtime client is closed') if (this.child !== undefined) return - const child = spawn(this.options.command, this.options.args ?? [], { - cwd: this.options.cwd, - env: this.options.env ?? process.env, + const child = spawn(this.runtime.command, this.runtime.args, { + cwd: this.runtime.cwd, + env: this.runtime.environment(), stdio: ['pipe', 'pipe', 'pipe'], }) this.child = child @@ -266,7 +274,7 @@ export class HarnessClient { * @returns the runtime's wire identity. */ async initialize(params: InitializeParams): Promise { - const result = await this.request('initialize', { ...params }) + const result = await this.request('initialize', { ...params }, this.runtime.initializeTimeoutMs) if (!isRecord(result) || !isRecord(result.serverInfo) || typeof result.serverInfo.name !== 'string' || typeof result.serverInfo.version !== 'string') { throw new SdkProtocolError(`initialize returned no server identity: ${JSON.stringify(result)}`) @@ -309,7 +317,7 @@ export class HarnessClient { const transport = this.transport /* v8 ignore next -- start() either sets the transport or throws */ if (transport === undefined) throw new TransportClosedError('DeepSeek Harness runtime is not running') - const timeout = timeoutMs ?? this.options.requestTimeoutMs + const timeout = timeoutMs ?? this.runtime.requestTimeoutMs try { if (timeout === undefined) return await transport.request(method, params ?? {}) // The abort signal makes the timeout an abandonment: the transport drops @@ -317,7 +325,8 @@ export class HarnessClient { // retain no per-call state (the server-side work still runs to close). const abandon = new AbortController() const timer = setTimeout(() => { - abandon.abort(new RequestTimeoutError(`${method} timed out after ${timeout}ms waiting for the DeepSeek Harness runtime`)) + const stderr = this.stderrTail.length === 0 ? '' : `; stderr tail:\n${this.stderrTail.join('\n')}` + abandon.abort(new RequestTimeoutError(`${method} timed out after ${timeout}ms waiting for ${this.runtime.description}${stderr}`)) }, timeout) try { return await transport.request(method, params ?? {}, abandon.signal) @@ -353,8 +362,8 @@ export class HarnessClient { /** * Subscribe to one session and the descendants discovered from - * `subagent.started` lineage edges (the runtime notifies for every session - * in its context; scoping is client-side, mirroring the Python SDK). + * `subagent.started` lineage edges. The runtime notifies for every session + * in its context, so this client applies the scope. * @param sessionId - the root session id. * @returns the filtered subscription handle. */ @@ -386,15 +395,15 @@ export class HarnessClient { const child = this.child if (child === undefined) return try { - await this.request('shutdown', undefined, this.options.shutdownTimeoutMs ?? 1_000) + await this.request('shutdown', undefined, this.runtime.shutdownTimeoutMs ?? 1_000) } catch (error) { // Diagnostic only: the dispose ladder below is the authoritative teardown // for a runtime that cannot answer shutdown anymore. this.appendStderr([`shutdown request failed: ${errorMessage(error)}`]) } await disposeRuntimeProcess(child, { - disposeEofGraceMs: this.options.disposeEofGraceMs ?? 6_000, - disposeGraceMs: this.options.disposeGraceMs ?? 3_000, + disposeEofGraceMs: this.runtime.disposeEofGraceMs ?? 6_000, + disposeGraceMs: this.runtime.disposeGraceMs ?? 3_000, }) this.transport?.close() this.failSubscriptions(this.closedError('DeepSeek Harness runtime closed')) @@ -449,7 +458,7 @@ export class HarnessClient { } private closedError(reason: string): TransportClosedError { - const parts = [reason] + const parts = [`${this.runtime.description}: ${reason}`] if (this.spawnError !== undefined) parts.push(`spawn error: ${this.spawnError.message}`) if (this.exitCode !== undefined) parts.push(`exit code: ${String(this.exitCode)}`) if (this.stderrTail.length > 0) parts.push(`stderr tail:\n${this.stderrTail.join('\n')}`) @@ -457,6 +466,15 @@ export class HarnessClient { } } +/** Construct the transport against a generic process for package-local fake-runtime tests. */ +export function createProcessHarnessClient(options: RuntimeProcessOptions): HarnessClient { + const Constructor = HarnessClient as unknown as new ( + publicOptions: HarnessClientOptions, + runtime: RuntimeProcessOptions, + ) => HarnessClient + return new Constructor({}, options) +} + /** * Whether `value` is a plain JSON object (the wire-boundary shape probe). * @param value - the wire value to probe. diff --git a/packages/sdk/client/src/index.ts b/packages/sdk/client/src/index.ts index 128cfbb898..9180a27f81 100644 --- a/packages/sdk/client/src/index.ts +++ b/packages/sdk/client/src/index.ts @@ -1,10 +1,10 @@ /** * TypeScript client SDK for the DeepSeek Harness runtime: spawn the - * `dsh-jsonrpc-agent` runtime as a subprocess and drive agent turns over - * stdio JSON-RPC. `DeepSeekHarness` is the high-level run API; + * same-version `dsh --profile sdk` runtime as a subprocess and drive agent + * turns over stdio JSON-RPC. `DeepSeekHarness` is the high-level run API; * `HarnessClient` is the lower-level protocol client. A pure library — it - * registers nothing on a Cordis context; the runtime process it spawns is a - * complete harness configured by its own `cordis.yml`. + * registers nothing on a Cordis context; named profiles and ordered patch + * files customize the runtime process it spawns. * * @module @deepseek-ai/dsh-sdk-client */ diff --git a/packages/sdk/client/src/launch.ts b/packages/sdk/client/src/launch.ts new file mode 100644 index 0000000000..b4a6419d19 --- /dev/null +++ b/packages/sdk/client/src/launch.ts @@ -0,0 +1,157 @@ +/** + * Resolve the public SDK launch configuration to one dsh subprocess. + * @module @deepseek-ai/dsh-sdk-client/launch + */ + +import { existsSync, readFileSync } from 'node:fs' +import { dirname, resolve } from 'node:path' +import { fileURLToPath } from 'node:url' +import type { HarnessClientOptions } from './types.ts' + +/** Default bound for a profile to answer the SDK initialize handshake. */ +export const DEFAULT_INITIALIZE_TIMEOUT_MS = 10_000 + +/** Internal generic process launch used by the transport and fake-runtime tests. */ +export interface RuntimeProcessOptions { + command: string + args: string[] + cwd?: string + /** Materialize the complete child environment when the client starts its subprocess. */ + environment: () => NodeJS.ProcessEnv + description: string + initializeTimeoutMs: number + requestTimeoutMs?: number + shutdownTimeoutMs?: number + disposeEofGraceMs?: number + disposeGraceMs?: number +} + +/** Node argv plus internal profile patches required by one resolved dsh entry. */ +export interface DshNodeLaunch { + /** Arguments before the profile selector. */ + nodeArgs: string[] + /** Internal patches applied below caller-supplied patches. */ + patches: string[] + /** Environment values required by the resolved entry mode. */ + environment: NodeJS.ProcessEnv +} + +interface PackageManifest { + version?: unknown + bin?: unknown +} + +/** Read a package manifest from one resolved package.json URL. */ +function manifest(url: string): PackageManifest { + return JSON.parse(readFileSync(fileURLToPath(url), 'utf8')) as PackageManifest +} + +/** + * Resolve and version-check a dsh executable from package manifests. + * @param dshManifestUrl - resolved URL of the dsh package manifest. + * @param clientManifestUrl - resolved URL of the SDK client manifest. + * @returns the absolute dsh executable path. + */ +export function resolveDshBinFromManifests(dshManifestUrl: string, clientManifestUrl: string): string { + const dshManifest = manifest(dshManifestUrl) + const clientManifest = manifest(clientManifestUrl) + if (typeof dshManifest.version !== 'string' || dshManifest.version !== clientManifest.version) { + throw new Error(`dsh SDK client ${String(clientManifest.version)} requires the same dsh version, got ${String(dshManifest.version)}`) + } + const bin = typeof dshManifest.bin === 'object' && dshManifest.bin !== null + ? (dshManifest.bin as Record).dsh + : dshManifest.bin + if (typeof bin !== 'string' || bin === '') throw new Error('@deepseek-ai/dsh declares no dsh executable') + return resolve(dirname(fileURLToPath(dshManifestUrl)), bin) +} + +/** + * Resolve and version-check the built dsh executable installed with this SDK. + * @returns the absolute built executable path, whether or not it exists in a source checkout. + */ +export function installedDshBin(): string { + return resolveDshBinFromManifests( + import.meta.resolve('@deepseek-ai/dsh/package.json'), + new URL('../package.json', import.meta.url).href, + ) +} + +/** + * Resolve the Node launch for one same-version dsh package. + * @param dshManifestUrl - resolved URL of the dsh package manifest. + * @param clientManifestUrl - resolved URL of the SDK client manifest. + * @param sourceLoaderUrl - optional absolute tsx loader URL for deterministic tests. + * @returns built output, or the source entry plus its compatibility patch and tsx environment. + */ +export function resolveDshNodeLaunchFromManifests( + dshManifestUrl: string, + clientManifestUrl: string, + sourceLoaderUrl?: string, +): DshNodeLaunch { + const bin = resolveDshBinFromManifests(dshManifestUrl, clientManifestUrl) + if (existsSync(bin)) return { nodeArgs: [bin], patches: [], environment: {} } + + const packageDir = dirname(fileURLToPath(dshManifestUrl)) + const sourceBin = resolve(packageDir, 'src/bin.ts') + const sourcePatch = resolve(packageDir, 'src/sdk-source.cordis.patch.yml') + const sourceTsconfig = resolve(packageDir, 'tsconfig.json') + if (!existsSync(sourceBin) || !existsSync(sourcePatch) || !existsSync(sourceTsconfig)) { + throw new Error( + `@deepseek-ai/dsh is missing its built executable ${bin} and complete source launch files ${sourceBin}, ${sourcePatch}, ${sourceTsconfig}`, + ) + } + const loader = sourceLoaderUrl ?? import.meta.resolve('tsx/esm') + return { + nodeArgs: ['--import', loader, sourceBin], + patches: [sourcePatch], + environment: { TSX_TSCONFIG_PATH: sourceTsconfig }, + } +} + +/** + * Resolve the installed dsh package to a built or source Node launch. + * @returns the launch descriptor for the current checkout or installed package. + */ +function installedDshNodeLaunch(): DshNodeLaunch { + return resolveDshNodeLaunchFromManifests( + import.meta.resolve('@deepseek-ai/dsh/package.json'), + new URL('../package.json', import.meta.url).href, + ) +} + +/** + * Resolve caller-relative filesystem inputs and construct canonical dsh argv. + * @param options - public SDK launch options. + * @param callerCwd - parent-process directory used for lexical resolution. + * @returns one generic subprocess spec for the JSON-RPC transport. + */ +export function resolveDshLaunch( + options: HarnessClientOptions = {}, + callerCwd: string = process.cwd(), +): RuntimeProcessOptions { + const profile = options.profile ?? 'sdk' + const dshLaunch = options.dshBin === undefined + ? installedDshNodeLaunch() + : { nodeArgs: [resolve(callerCwd, options.dshBin)], patches: [], environment: {} } + const patches = [ + ...dshLaunch.patches, + ...(options.patches ?? []).map(path => resolve(callerCwd, path)), + ] + const dshHome = options.dshHome === undefined ? undefined : resolve(callerCwd, options.dshHome) + return { + command: process.execPath, + args: [...dshLaunch.nodeArgs, '--profile', profile, ...patches.flatMap(path => ['--patch', path])], + ...options.processCwd === undefined ? {} : { cwd: resolve(callerCwd, options.processCwd) }, + environment: () => ({ + ...(options.env ?? process.env), + ...dshLaunch.environment, + ...dshHome === undefined ? {} : { DSH_HOME: dshHome }, + }), + description: `dsh profile ${JSON.stringify(profile)}`, + initializeTimeoutMs: options.initializeTimeoutMs ?? DEFAULT_INITIALIZE_TIMEOUT_MS, + ...options.requestTimeoutMs === undefined ? {} : { requestTimeoutMs: options.requestTimeoutMs }, + ...options.shutdownTimeoutMs === undefined ? {} : { shutdownTimeoutMs: options.shutdownTimeoutMs }, + ...options.disposeEofGraceMs === undefined ? {} : { disposeEofGraceMs: options.disposeEofGraceMs }, + ...options.disposeGraceMs === undefined ? {} : { disposeGraceMs: options.disposeGraceMs }, + } +} diff --git a/packages/sdk/client/src/types.ts b/packages/sdk/client/src/types.ts index 1ac92f43a3..0750c78d1c 100644 --- a/packages/sdk/client/src/types.ts +++ b/packages/sdk/client/src/types.ts @@ -21,19 +21,26 @@ export type NotificationFilter = (notification: HarnessNotification) => boolean /** Launch and timeout options for {@link HarnessClient}. */ export interface HarnessClientOptions { - /** The runtime executable (the `dsh-jsonrpc-agent` bin, a packaged exe, or `node`). */ - command: string - /** Arguments passed to {@link command}. */ - args?: string[] - /** Working directory for the runtime process itself. */ - cwd?: string + /** Absolute or caller-relative dsh CLI module; omitted resolves this package's same-version dependency. */ + dshBin?: string + /** Named profile serving the SDK protocol (default `sdk`). */ + profile?: string + /** Ordered per-launch profile patches; relative paths resolve before spawn. */ + patches?: string[] + /** Explicit Harness home for this child; relative paths resolve before spawn. */ + dshHome?: string + /** Working directory for the dsh process itself. */ + processCwd?: string /** - * The complete child environment. `undefined` inherits the parent env - * verbatim; passing an object replaces it entirely, so callers own + * The complete child environment, read when {@link HarnessClient.start} + * spawns. `undefined` reads the parent env at that time; passing an object + * reads that object at spawn and replaces the parent environment entirely, so callers own * credential policy (see `scrubbedParentEnv` in `@deepseek-ai/dsh-subprocess` * for the shared scrub-then-merge base). */ env?: NodeJS.ProcessEnv + /** Bound (ms) on the initial profile handshake (default 10000). */ + initializeTimeoutMs?: number /** Per-request timeout (ms); `undefined` waits indefinitely (a turn can legitimately run long). */ requestTimeoutMs?: number /** Bound (ms) on the protocol `shutdown` exchange inside `close()` (default 1000). */ @@ -45,10 +52,8 @@ export interface HarnessClientOptions { } /** Options for the high-level {@link DeepSeekHarness} wrapper. */ -export interface DeepSeekHarnessOptions { - /** Launch spec for the runtime subprocess (command, args, cwd, env, timeouts). */ - launch: HarnessClientOptions - /** Workspace cwd recorded on every SDK-created session (default: the launch cwd, else `process.cwd()`). */ +export interface DeepSeekHarnessOptions extends HarnessClientOptions { + /** Workspace cwd recorded on every SDK-created session (default: the process cwd, else `process.cwd()`). */ cwd?: string /** Provider route for SDK-created agents (default `deepseek-official`). */ provider?: string diff --git a/packages/sdk/client/tests/launch.spec.ts b/packages/sdk/client/tests/launch.spec.ts new file mode 100644 index 0000000000..0bbf859882 --- /dev/null +++ b/packages/sdk/client/tests/launch.spec.ts @@ -0,0 +1,169 @@ +/** Public dsh launch resolution for the TypeScript SDK. */ + +import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join, resolve } from 'node:path' +import { pathToFileURL } from 'node:url' +import { afterEach, describe, expect, it } from 'vitest' +import { + DEFAULT_INITIALIZE_TIMEOUT_MS, + installedDshBin, + resolveDshNodeLaunchFromManifests, + resolveDshBinFromManifests, + resolveDshLaunch, +} from '../src/launch.ts' + +const cleanups: string[] = [] +afterEach(() => { + for (const path of cleanups.splice(0)) rmSync(path, { recursive: true, force: true }) +}) + +function manifestPair(dsh: object, client: object): { dshUrl: string; clientUrl: string; root: string } { + const root = mkdtempSync(join(tmpdir(), 'dsh-sdk-manifests-')) + cleanups.push(root) + const dshPath = join(root, 'dsh-package.json') + const clientPath = join(root, 'client-package.json') + writeFileSync(dshPath, JSON.stringify(dsh)) + writeFileSync(clientPath, JSON.stringify(client)) + return { + dshUrl: pathToFileURL(dshPath).href, + clientUrl: pathToFileURL(clientPath).href, + root, + } +} + +describe('SDK dsh launch resolution', () => { + it('resolves the same-version installed dsh entry by default', () => { + const bin = installedDshBin() + expect(bin.endsWith(join('apps', 'cli', 'lib', 'bin.js'))).toBe(true) + const launch = resolveDshLaunch() + expect(launch.command).toBe(process.execPath) + expect(launch.args).toEqual(existsSync(bin) + ? [bin, '--profile', 'sdk'] + : [ + '--import', import.meta.resolve('tsx/esm'), resolve(bin, '..', '..', 'src/bin.ts'), + '--profile', 'sdk', + '--patch', resolve(bin, '..', '..', 'src/sdk-source.cordis.patch.yml'), + ]) + expect(launch.initializeTimeoutMs).toBe(DEFAULT_INITIALIZE_TIMEOUT_MS) + expect(launch.description).toBe('dsh profile "sdk"') + }) + + it('makes every filesystem input absolute before spawn and preserves patch order', () => { + const caller = resolve('/tmp', 'sdk-launch-caller') + const launch = resolveDshLaunch({ + dshBin: './bin/dsh', + profile: 'custom-sdk', + patches: ['./first.yml', '../second.yml'], + dshHome: './home', + processCwd: './worker', + env: { PATH: '/bin', DSH_HOME: '/stale' }, + initializeTimeoutMs: 123, + requestTimeoutMs: 456, + shutdownTimeoutMs: 789, + disposeEofGraceMs: 12, + disposeGraceMs: 34, + }, caller) + expect(launch).toMatchObject({ + command: process.execPath, + args: [ + join(caller, 'bin/dsh'), + '--profile', 'custom-sdk', + '--patch', join(caller, 'first.yml'), + '--patch', resolve(caller, '../second.yml'), + ], + cwd: join(caller, 'worker'), + description: 'dsh profile "custom-sdk"', + initializeTimeoutMs: 123, + requestTimeoutMs: 456, + shutdownTimeoutMs: 789, + disposeEofGraceMs: 12, + disposeGraceMs: 34, + }) + expect(launch.environment()).toEqual({ PATH: '/bin', DSH_HOME: join(caller, 'home') }) + }) + + it('falls back to the same package source entry through an absolute tsx loader', () => { + const pair = manifestPair({ version: '1.0.0', bin: 'lib/bin.js' }, { version: '1.0.0' }) + const sourceBin = join(pair.root, 'src/bin.ts') + const sourcePatch = join(pair.root, 'src/sdk-source.cordis.patch.yml') + const sourceTsconfig = join(pair.root, 'tsconfig.json') + mkdirSync(join(pair.root, 'src')) + writeFileSync(sourceBin, '') + writeFileSync(sourcePatch, '[]\n') + writeFileSync(sourceTsconfig, '{}\n') + + expect(resolveDshNodeLaunchFromManifests(pair.dshUrl, pair.clientUrl, 'file:///tsx-loader.mjs')) + .toEqual({ + nodeArgs: ['--import', 'file:///tsx-loader.mjs', sourceBin], + patches: [sourcePatch], + environment: { TSX_TSCONFIG_PATH: sourceTsconfig }, + }) + expect(resolveDshNodeLaunchFromManifests(pair.dshUrl, pair.clientUrl)) + .toEqual({ + nodeArgs: ['--import', import.meta.resolve('tsx/esm'), sourceBin], + patches: [sourcePatch], + environment: { TSX_TSCONFIG_PATH: sourceTsconfig }, + }) + }) + + it('uses the built entry when the manifest bin exists', () => { + const pair = manifestPair({ version: '1.0.0', bin: 'lib/bin.js' }, { version: '1.0.0' }) + const bin = join(pair.root, 'lib/bin.js') + mkdirSync(join(pair.root, 'lib')) + writeFileSync(bin, '') + + expect(resolveDshNodeLaunchFromManifests(pair.dshUrl, pair.clientUrl)).toEqual({ + nodeArgs: [bin], + patches: [], + environment: {}, + }) + }) + + it.each([0, 1, 2])('fails loud when a source launch is missing required file set %s', (presentCount) => { + const pair = manifestPair({ version: '1.0.0', bin: 'lib/bin.js' }, { version: '1.0.0' }) + mkdirSync(join(pair.root, 'src')) + const sourceFiles = ['src/bin.ts', 'src/sdk-source.cordis.patch.yml', 'tsconfig.json'] + for (const source of sourceFiles.slice(0, presentCount)) writeFileSync(join(pair.root, source), '') + expect(() => resolveDshNodeLaunchFromManifests(pair.dshUrl, pair.clientUrl, 'file:///tsx-loader.mjs')) + .toThrow('is missing its built executable') + }) + + it('reads explicit and inherited environments when the child starts', () => { + const explicit: NodeJS.ProcessEnv = { MARKER: 'before' } + const explicitLaunch = resolveDshLaunch({ dshBin: '/bin/dsh', env: explicit }) + explicit.MARKER = 'after' + expect(explicitLaunch.environment().MARKER).toBe('after') + + const inheritedLaunch = resolveDshLaunch({ dshBin: '/bin/dsh' }) + process.env.DSH_SDK_LATE_ENV_TEST = 'late' + try { + expect(inheritedLaunch.environment().DSH_SDK_LATE_ENV_TEST).toBe('late') + } finally { + delete process.env.DSH_SDK_LATE_ENV_TEST + } + }) + + it.each([2, '2.0.0'])( + 'rejects a dsh version that differs from the client (%j)', + (version) => { + const pair = manifestPair({ version, bin: 'bin.js' }, { version: '1.0.0' }) + expect(() => resolveDshBinFromManifests(pair.dshUrl, pair.clientUrl)) + .toThrow(`requires the same dsh version, got ${String(version)}`) + }, + ) + + it('accepts the string npm bin form', () => { + const pair = manifestPair({ version: '1.0.0', bin: './bin.js' }, { version: '1.0.0' }) + expect(resolveDshBinFromManifests(pair.dshUrl, pair.clientUrl)).toBe(join(pair.root, 'bin.js')) + }) + + it.each([null, {}, ''])( + 'rejects a manifest without a usable dsh executable (%j)', + (bin) => { + const pair = manifestPair({ version: '1.0.0', bin }, { version: '1.0.0' }) + expect(() => resolveDshBinFromManifests(pair.dshUrl, pair.clientUrl)) + .toThrow('declares no dsh executable') + }, + ) +}) diff --git a/packages/sdk/client/tests/sdk-client.spec.ts b/packages/sdk/client/tests/sdk-client.spec.ts index a710a44c1d..2ff8a90ed6 100644 --- a/packages/sdk/client/tests/sdk-client.spec.ts +++ b/packages/sdk/client/tests/sdk-client.spec.ts @@ -20,7 +20,9 @@ import { TransportClosedError, type HarnessNotification, } from '../src/index.ts' -import { finalResponse, normalizeInput } from '../src/api.ts' +import { createProcessDeepSeekHarness, finalResponse, normalizeInput } from '../src/api.ts' +import { createProcessHarnessClient } from '../src/client.ts' +import type { RuntimeProcessOptions } from '../src/launch.ts' const fakeRuntime = fileURLToPath(new URL('./fake-runtime.ts', import.meta.url)) @@ -29,20 +31,26 @@ afterEach(async () => { for (const cleanup of cleanups.splice(0)) await cleanup() }) -type LaunchOverrides = Partial[0]> +type LaunchOverrides = Partial /** Launch options running the fake runtime on the current node (type stripping). */ -function fakeLaunch(env: Record = {}, extra: LaunchOverrides = {}) { +function fakeLaunch(env: Record = {}, extra: LaunchOverrides = {}): RuntimeProcessOptions { return { command: process.execPath, args: [fakeRuntime], - env: { ...process.env as Record, ...env }, + environment: () => ({ ...process.env as Record, ...env }), + description: 'scripted fake runtime', + initializeTimeoutMs: 5_000, ...extra, } } +function processClient(options: RuntimeProcessOptions): HarnessClient { + return createProcessHarnessClient(options) +} + function harnessWith(env: Record = {}, extra: LaunchOverrides = {}): DeepSeekHarness { - const harness = new DeepSeekHarness({ launch: fakeLaunch(env, extra) }) + const harness = createProcessDeepSeekHarness(fakeLaunch(env, extra)) cleanups.push(() => harness.close()) return harness } @@ -150,8 +158,7 @@ describe('DeepSeekHarness', () => { it('sends the configured cwd/provider/model/maxTokens in the handshake exactly once', async () => { const dir = await tempDir('sdk-client-init-') const recordFile = join(dir, 'init.jsonl') - const harness = new DeepSeekHarness({ - launch: fakeLaunch({ FAKE_RECORD_INIT: recordFile }), + const harness = createProcessDeepSeekHarness(fakeLaunch({ FAKE_RECORD_INIT: recordFile }), { cwd: dir, provider: 'custom-provider', model: 'custom-model', @@ -180,9 +187,9 @@ describe('DeepSeekHarness', () => { await mkdir(inner) const relativeCwd = relative(process.cwd(), inner) expect(isAbsolute(relativeCwd)).toBe(false) - const harness = new DeepSeekHarness({ - launch: fakeLaunch({ FAKE_RECORD_INIT: recordFile, FAKE_ECHO_CWD_IN_INIT: '1' }, { cwd: relativeCwd }), - }) + const harness = createProcessDeepSeekHarness( + fakeLaunch({ FAKE_RECORD_INIT: recordFile, FAKE_ECHO_CWD_IN_INIT: '1' }, { cwd: relativeCwd }), + ) cleanups.push(() => harness.close()) await harness.start() const identity = await harness.client.initialize({ cwd: inner, provider: 'p', model: 'm' }) @@ -232,7 +239,7 @@ describe('DeepSeekHarness', () => { it('supports await using disposal', async () => { let captured: DeepSeekHarness { - await using harness = new DeepSeekHarness({ launch: fakeLaunch() }) + await using harness = createProcessDeepSeekHarness(fakeLaunch()) captured = harness const result = await harness.run('scoped') expect(result.finalResponse).toBe('hello from fake runtime') @@ -240,20 +247,45 @@ describe('DeepSeekHarness', () => { // After scope exit the runtime is closed: reuse fails loudly. await expect(captured.run('after')).rejects.toThrow(TransportClosedError) }) + + it('constructs the public dsh-backed client lazily', async () => { + const harness = new DeepSeekHarness() + expect(harness.client).toBeInstanceOf(HarnessClient) + await harness.close() + }) }) describe('HarnessClient', () => { + it('bounds profile initialization and names the selected profile in its diagnostic', async () => { + const client = processClient(fakeLaunch( + { FAKE_HANG_INIT: '1' }, + { + description: 'dsh profile "profile-without-sdk-server"', + initializeTimeoutMs: 50, + disposeEofGraceMs: 100, + disposeGraceMs: 100, + }, + )) + cleanups.push(() => client.close()) + await expect(client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' })) + .rejects.toThrow(/initialize timed out after 50ms waiting for dsh profile "profile-without-sdk-server"/) + await client.close() + }) + it('times out a hung request at the per-call bound', async () => { - const client = new HarnessClient(fakeLaunch({ FAKE_HANG_PROMPT: '1' })) + const client = processClient(fakeLaunch({ + FAKE_HANG_PROMPT: '1', + FAKE_STDERR: 'runtime accepted initialize but hung the prompt', + })) cleanups.push(() => client.close()) await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }) await expect(client.request('session/prompt', { sessionId: 's', contentBlocks: normalizeInput('hi') }, 200)) - .rejects.toThrow(RequestTimeoutError) + .rejects.toThrow(/session\/prompt timed out.*stderr tail:\nruntime accepted initialize but hung the prompt/s) await client.close() }) it('a timed-out request leaves no pending transport state', async () => { - const client = new HarnessClient(fakeLaunch({ FAKE_HANG_PROMPT: '1' })) + const client = processClient(fakeLaunch({ FAKE_HANG_PROMPT: '1' })) cleanups.push(() => client.close()) await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }) for (let round = 0; round < 3; round++) { @@ -269,7 +301,7 @@ describe('HarnessClient', () => { }) it('applies the client-wide request timeout when no per-call bound is given', async () => { - const client = new HarnessClient(fakeLaunch({ FAKE_HANG_PROMPT: '1' }, { requestTimeoutMs: 400 })) + const client = processClient(fakeLaunch({ FAKE_HANG_PROMPT: '1' }, { requestTimeoutMs: 400 })) cleanups.push(() => client.close()) // The bound applies from send, so it holds regardless of runtime boot time. await expect(client.prompt('s', normalizeInput('hi'))).rejects.toThrow(RequestTimeoutError) @@ -277,14 +309,14 @@ describe('HarnessClient', () => { }) it('rejects a malformed prompt acceptance as a protocol error', async () => { - const client = new HarnessClient(fakeLaunch({ FAKE_MALFORMED: '1' })) + const client = processClient(fakeLaunch({ FAKE_MALFORMED: '1' })) cleanups.push(() => client.close()) await expect(client.prompt('s', normalizeInput('hi'))).rejects.toThrow(SdkProtocolError) await client.close() }) it('fails pending requests with exit code and stderr tail when the runtime dies', async () => { - const client = new HarnessClient(fakeLaunch({ FAKE_EXIT_BEFORE_INIT: '1', FAKE_STDERR: 'fatal: scripted death' })) + const client = processClient(fakeLaunch({ FAKE_EXIT_BEFORE_INIT: '1', FAKE_STDERR: 'fatal: scripted death' })) cleanups.push(() => client.close()) const failure = await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }).then( () => { throw new Error('initialize unexpectedly succeeded') }, @@ -298,7 +330,7 @@ describe('HarnessClient', () => { }) it('flushes an unterminated stderr line into the tail at close', async () => { - const client = new HarnessClient(fakeLaunch({ FAKE_STDERR_NO_NEWLINE: 'no trailing newline', FAKE_EXIT_BEFORE_INIT: '1' })) + const client = processClient(fakeLaunch({ FAKE_STDERR_NO_NEWLINE: 'no trailing newline', FAKE_EXIT_BEFORE_INIT: '1' })) cleanups.push(() => client.close()) const failure = await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }).then( () => { throw new Error('initialize unexpectedly succeeded') }, @@ -307,27 +339,37 @@ describe('HarnessClient', () => { expect(String(failure)).toContain('no trailing newline') }) - it('fails fast when the command does not exist', async () => { - const client = new HarnessClient({ command: join(tmpdir(), 'dsh-no-such-runtime-bin') }) + it('fails when the configured dsh CLI module does not exist', async () => { + const client = new HarnessClient({ dshBin: join(tmpdir(), 'dsh-no-such-runtime-bin') }) cleanups.push(() => client.close()) await expect(client.request('initialize', {}, 1_000)).rejects.toThrow(TransportClosedError) }) + it('reports a generic process spawn failure to internal transports', async () => { + const client = processClient(fakeLaunch({}, { + command: join(tmpdir(), 'dsh-no-such-process-command'), + args: [], + })) + cleanups.push(() => client.close()) + await expect(client.request('initialize', {}, 1_000)) + .rejects.toThrow(/spawn error:.*ENOENT/s) + }) + it('close() is idempotent, reaps the child, and fails later use', async () => { - const client = new HarnessClient(fakeLaunch()) + const client = processClient(fakeLaunch()) await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }) await Promise.all([client.close(), client.close()]) expect(() => { client.start() }).toThrow(TransportClosedError) await expect(client.request('anything')).rejects.toThrow(TransportClosedError) // Close with no child ever spawned is a no-op. - const untouched = new HarnessClient(fakeLaunch()) + const untouched = processClient(fakeLaunch()) await untouched.close() }) it('escalates through SIGTERM when the runtime ignores EOF', async () => { const dir = await tempDir('sdk-client-ladder-') const sigtermFile = join(dir, 'sigterm.txt') - const client = new HarnessClient(fakeLaunch( + const client = processClient(fakeLaunch( { FAKE_IGNORE_EOF: '1', FAKE_SIGTERM_FILE: sigtermFile }, { shutdownTimeoutMs: 100, disposeEofGraceMs: 100, disposeGraceMs: 1_000 }, )) @@ -341,7 +383,7 @@ describe('HarnessClient', () => { }) it('escalates to SIGKILL when the runtime traps SIGTERM too', async () => { - const client = new HarnessClient(fakeLaunch( + const client = processClient(fakeLaunch( { FAKE_IGNORE_EOF: '1', FAKE_TRAP_SIGTERM: '1' }, { shutdownTimeoutMs: 100, disposeEofGraceMs: 100, disposeGraceMs: 300 }, )) @@ -351,7 +393,7 @@ describe('HarnessClient', () => { }) it('delivers notifications to unfiltered and filtered subscriptions in wire order', async () => { - const client = new HarnessClient(fakeLaunch()) + const client = processClient(fakeLaunch()) cleanups.push(() => client.close()) await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }) @@ -385,7 +427,7 @@ describe('HarnessClient', () => { }) it('contains a throwing filter to its own subscription', async () => { - const client = new HarnessClient(fakeLaunch()) + const client = processClient(fakeLaunch()) cleanups.push(() => client.close()) await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }) @@ -405,7 +447,7 @@ describe('HarnessClient', () => { }) it('close() drops queued notifications; runtime death keeps them drainable', async () => { - const client = new HarnessClient(fakeLaunch()) + const client = processClient(fakeLaunch()) await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }) const closed = client.subscribe() const drainable = client.subscribe() @@ -422,20 +464,20 @@ describe('HarnessClient', () => { }) it('subscriptions created after termination are born failed', async () => { - const client = new HarnessClient(fakeLaunch()) + const client = processClient(fakeLaunch()) await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }) await client.close() // No producer can ever feed this subscription; next() must not park forever. await expect(client.subscribe().next()).rejects.toThrow(TransportClosedError) - const dead = new HarnessClient(fakeLaunch({ FAKE_EXIT_BEFORE_INIT: '1' })) + const dead = processClient(fakeLaunch({ FAKE_EXIT_BEFORE_INIT: '1' })) cleanups.push(() => dead.close()) await dead.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }).catch(() => {}) await expect(dead.subscribe().next()).rejects.toThrow(TransportClosedError) }) it('closes subscriptions with the runtime and rejects parked waiters', async () => { - const client = new HarnessClient(fakeLaunch()) + const client = processClient(fakeLaunch()) await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }) const subscription = client.subscribe() const parked = subscription.next() @@ -444,7 +486,7 @@ describe('HarnessClient', () => { }) it('scopes the session tree across multi-hop lineage and ignores foreign sessions', async () => { - const client = new HarnessClient(fakeLaunch()) + const client = processClient(fakeLaunch()) cleanups.push(() => client.close()) await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }) @@ -496,7 +538,7 @@ describe('wire payload validation', () => { describe('stderr tail bound', () => { it('keeps only the newest lines up to the limit', async () => { const manyLines = Array.from({ length: 450 }, (_, i) => `line-${i}`).join('\n') - const client = new HarnessClient(fakeLaunch({ FAKE_STDERR: manyLines, FAKE_EXIT_BEFORE_INIT: '1' })) + const client = processClient(fakeLaunch({ FAKE_STDERR: manyLines, FAKE_EXIT_BEFORE_INIT: '1' })) cleanups.push(() => client.close()) const failure = await client.initialize({ cwd: process.cwd(), provider: 'p', model: 'm' }).then( () => { throw new Error('initialize unexpectedly succeeded') }, diff --git a/packages/sdk/protocol/README.i18n.yaml b/packages/sdk/protocol/README.i18n.yaml index 262a608875..63a7e665ac 100644 --- a/packages/sdk/protocol/README.i18n.yaml +++ b/packages/sdk/protocol/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/sdk/protocol/README.md README.md: 9024f9ca34a5467aff1b83cb9cd864c1ec06e56b -README.zh.md: ffd67552a770561320f919eec95679681fadbe0d +README.zh.md: 28372bf3dcd57a817225a2769ef969d6f0c10936 diff --git a/packages/sdk/protocol/README.zh.md b/packages/sdk/protocol/README.zh.md index ffd67552a7..28372bf3dc 100644 --- a/packages/sdk/protocol/README.zh.md +++ b/packages/sdk/protocol/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -DeepSeek Harness SDK 运行时的共享协议格式(wire format):一个按换行分帧的 JSON-RPC 2.0 传输类,加上协议两端共同使用的具名请求、结果与通知类型。包根枚举协议消费方接口;源模块不支持深层导入。服务端是 [`dsh-sdk-jsonrpc-server`](../server/README.md) 插件;客户端是 [`dsh-sdk-client`](../client/README.md)(TypeScript)与 [Python SDK](../../../python/README.md)(后者复现这些结构但不导入它们)。纯库——无插件、无 Config、无注册。 +DeepSeek Harness SDK 运行时的共享协议格式(wire format):一个按换行分帧的 JSON-RPC 2.0 传输类,加上协议两端共同使用的具名请求、结果与通知类型。包根枚举协议消费方接口;源模块不支持深层导入。服务端是 [`dsh-sdk-jsonrpc-server`](../server/README.zh.md) 插件;客户端是 [`dsh-sdk-client`](../client/README.zh.md)(TypeScript)与 [Python SDK](../../../python/README.zh.md)(后者复现这些结构但不导入它们)。纯库——无插件、无 Config、无注册。 ## 传输 @@ -26,7 +26,7 @@ DeepSeek Harness SDK 运行时的共享协议格式(wire format):一个按 ## 模型体验 -无,因为此包定义面向客户端的协议格式;模型可见接口属于组合在对外服务入口 [`dsh-sdk-jsonrpc-server`](../server/README.md) 后方的运行时插件。 +无,因为此包定义面向客户端的协议格式;模型可见接口属于组合在对外服务入口 [`dsh-sdk-jsonrpc-server`](../server/README.zh.md) 后方的运行时插件。 #### KV Cache 影响 @@ -35,5 +35,5 @@ DeepSeek Harness SDK 运行时的共享协议格式(wire format):一个按 ## 已知限制与暂缓事项 - **无协议版本协商**——握手只携带 `serverInfo.version`(`0.0.1`,客户端不校验);处于预发布阶段,无兼容承诺。 -- **无取消与会话关闭方法**——客户端放弃轮次的方式是关闭运行时进程;见 [`dsh-sdk-jsonrpc-server` README](../server/README.md)。 +- **无取消与会话关闭方法**——客户端放弃轮次的方式是关闭运行时进程;见 [`dsh-sdk-jsonrpc-server` README](../server/README.zh.md)。 - **server→client 请求是未使用的功能**——传输层支持,但服务器从不发送;Python SDK 的应答接口为未来审批流程预留。 diff --git a/packages/sdk/protocol/package.json b/packages/sdk/protocol/package.json index 966d35d538..c5bac4b7e7 100644 --- a/packages/sdk/protocol/package.json +++ b/packages/sdk/protocol/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-sdk-protocol", "description": "Shared wire protocol for the DeepSeek Harness SDK runtime: the newline-delimited JSON-RPC stdio transport and the named request, result, and notification types spoken between the runtime server and SDK clients", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/sdk/python-runtime/README.i18n.yaml b/packages/sdk/python-runtime/README.i18n.yaml new file mode 100644 index 0000000000..d2b2859c7c --- /dev/null +++ b/packages/sdk/python-runtime/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/sdk/python-runtime/README.md +README.md: 291ad3edaff8182007079de8e41f33e91546e5d7 +README.zh.md: 54001ccfffb356db290d7c7b38db41070d8dd661 diff --git a/packages/sdk/python-runtime/README.md b/packages/sdk/python-runtime/README.md new file mode 100644 index 0000000000..291ad3edaf --- /dev/null +++ b/packages/sdk/python-runtime/README.md @@ -0,0 +1,34 @@ +# @deepseek-ai/dsh-sdk-python-runtime + +English | [中文](README.zh.md) + +Private direct-config carrier for the temporarily unchanged Python SDK runtime. Its [`jsonrpc`](../server/README.md) entry serves SDK clients over newline-delimited stdio, while an external `cordis.yml` composes the spine, backends, and serving plugin. This npm package exposes no public bin and is not published; the Python SDK's existing `dsh-jsonrpc-agent-pkg--` [single-executable runtime](../../../.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md) packages `lib/packaged-bin.js` from the closed deploy tree. Bare plugins resolve from that tree, while relative plugins remain configuration-relative. + +## Config discovery + +The first non-empty channel wins: `$DSH_CORDIS_CONFIG`, then positional `argv[2]`. If neither names an existing file, the packaged entry prints one-line usage to stderr and exits 1; there is no working-directory or built-in fallback. [`dsh-app-boot`](../../boot/app-boot/README.md) makes plugin load failures fatal. This protocol does not use `DSH_SNAPSHOT`. + +A config without `dsh-sdk-jsonrpc-server` is valid and serves nothing; the carrier does not designate a server plugin. + +## Exit lifecycle + +stdin EOF and `SIGTERM` dispose the root to quiescence and exit 0; `SIGINT` exits 130 after the same disposal. EOF may cut off an in-flight turn as documented in the [distribution Agent Note](../../../.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md). The `jsonrpc` plugin owns response-before-exit protocol shutdown; both paths are idempotent and safe to race. + +## stdout is the protocol + +stdout carries only JSON-RPC frames. The carrier and boot guards diagnose on stderr, and the config must omit stdout loggers. + +## Model Experience + +Indirectly, through the plugins loaded from the external `cordis.yml`, which own every model-bound prompt, schema, message, and result; this carrier adds none of its own. + +#### KV Cache effect + +No direct invalidation; the named consumer owns any request-prefix changes. + +## Known Limitations and Deferred Work + +- **Temporary direct-config exception** — this private carrier remains outside `dsh --profile sdk` only to preserve the current Python executable and wheel behavior; the later Python runtime migration deletes it and then renames the executable family. +- **The carrier cannot prove that the config serves JSON-RPC** — a valid config with no `dsh-sdk-jsonrpc-server` entry boots successfully and serves nothing. +- **No built-in or default config exists** — every launch must provide `DSH_CORDIS_CONFIG` or a positional path, and deployment owns the complete plugin tree and stdout discipline. +- **stdin EOF cuts off in-flight work** — client disappearance disposes the root immediately; callers that need orderly completion use the protocol-level `shutdown` request. diff --git a/packages/sdk/python-runtime/README.zh.md b/packages/sdk/python-runtime/README.zh.md new file mode 100644 index 0000000000..54001ccfff --- /dev/null +++ b/packages/sdk/python-runtime/README.zh.md @@ -0,0 +1,34 @@ +# @deepseek-ai/dsh-sdk-python-runtime + +[English](README.md) | 中文 + +这是为暂时保持不变的 Python SDK 运行时提供的私有直读配置载体。其 [`jsonrpc`](../server/README.zh.md) 入口通过按换行分隔的 stdio 为 SDK 客户端提供服务,外部 `cordis.yml` 则负责组合主干、后端和服务插件。该 npm 包不公开 bin,也不会发布;Python SDK 既有的 `dsh-jsonrpc-agent-pkg--` [单文件可执行运行时](../../../.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md)从封闭部署树打包 `lib/packaged-bin.js`。裸插件从该树解析,相对插件仍以配置目录为基准。 + +## 配置发现 + +第一个非空通道生效:先 `$DSH_CORDIS_CONFIG`,再位置参数 `argv[2]`。如果二者都没有指向现有文件,打包入口会向 stderr 打印单行用法并以 1 退出;没有工作目录回退或内置回退。[`dsh-app-boot`](../../boot/app-boot/README.zh.md) 会使插件加载失败成为致命错误。此协议不使用 `DSH_SNAPSHOT`。 + +不含 `dsh-sdk-jsonrpc-server` 的配置仍然有效,只是不提供任何服务;该载体不会指定服务器插件。 + +## 退出生命周期 + +stdin EOF 和 `SIGTERM` 会 dispose(释放资源)根上下文,等待完全停稳后以 0 退出;`SIGINT` 完成同样的 dispose 后以 130 退出。EOF 可能按[分发 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md) 所述截断正在处理的轮次。`jsonrpc` 插件拥有先响应再退出的协议关闭流程;两条路径均幂等,即使发生竞态也安全。 + +## stdout 是协议 + +stdout 只承载 JSON-RPC 帧。该载体和启动守卫在 stderr 上输出诊断,配置必须省略 stdout logger。 + +## 模型体验 + +模型体验由外部 `cordis.yml` 加载的插件间接提供;这些插件负责所有面向模型的提示词、schema、消息和结果,该载体不添加任何内容。 + +#### KV Cache 影响 + +不会直接失效;由上述消费方负责请求前缀的任何变更。 + +## 已知限制与暂缓事项 + +- **临时直读配置例外**:为了保持当前 Python 可执行文件与 wheel 包行为不变,该私有载体暂时不经过 `dsh --profile sdk`;后续 Python 运行时迁移会删除它,之后再重命名可执行文件族。 +- **载体无法证明配置提供 JSON-RPC 服务**:不含 `dsh-sdk-jsonrpc-server` 条目的有效配置也能成功启动,但不会提供任何服务。 +- **不存在内置或默认配置**:每次启动都必须提供 `DSH_CORDIS_CONFIG` 或位置路径;部署方负责完整的插件树和 stdout 纪律。 +- **stdin EOF 会截断正在处理的工作**:客户端消失时立即释放根上下文;需要有序完成的调用方应使用协议级 `shutdown` 请求。 diff --git a/packages/examples/jsonrpc-demo/package.json b/packages/sdk/python-runtime/package.json similarity index 70% rename from packages/examples/jsonrpc-demo/package.json rename to packages/sdk/python-runtime/package.json index f756389787..cdae65348f 100644 --- a/packages/examples/jsonrpc-demo/package.json +++ b/packages/sdk/python-runtime/package.json @@ -1,21 +1,16 @@ { - "name": "@deepseek-ai/dsh-sdk-jsonrpc-demo", - "description": "Bin that boots an external Cordis config for the stdio JSON-RPC SDK runtime", - "version": "0.0.1-rc.5", - "publishConfig": { - "access": "restricted" - }, + "name": "@deepseek-ai/dsh-sdk-python-runtime", + "description": "Private direct-config runtime carrier for the temporarily unchanged Python SDK", + "version": "0.1.1-rc.2", + "private": true, "repository": { "type": "git", "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", - "directory": "packages/examples/jsonrpc-demo" + "directory": "packages/sdk/python-runtime" }, "type": "module", "main": "lib/index.js", "types": "lib/types/index.d.ts", - "bin": { - "dsh-jsonrpc-agent": "lib/bin.js" - }, "exports": { ".": { "types": "./lib/types/index.d.ts", @@ -25,10 +20,6 @@ "types": "./lib/types/invariant.d.ts", "default": "./lib/invariant.js" }, - "./bin": { - "types": "./lib/types/bin.d.ts", - "default": "./lib/bin.js" - }, "./packaged-bin": { "types": "./lib/types/packaged-bin.d.ts", "default": "./lib/packaged-bin.js" @@ -39,7 +30,6 @@ "files": [ "lib/index.js", "lib/invariant.js", - "lib/bin.js", "lib/packaged-bin.js", "lib/types/**/*.d.ts" ], diff --git a/packages/sdk/python-runtime/src/index.ts b/packages/sdk/python-runtime/src/index.ts new file mode 100644 index 0000000000..74dabd87bf --- /dev/null +++ b/packages/sdk/python-runtime/src/index.ts @@ -0,0 +1,10 @@ +/** + * Private Python SDK runtime carrier: its packaged entry discovers an external + * `cordis.yml` and owns process exit. This module exports no composition plugin; + * the config chooses whether to load the + * {@link @deepseek-ai/dsh-sdk-jsonrpc-server} serving plugin. + * + * @module @deepseek-ai/dsh-sdk-python-runtime + */ + +export {} diff --git a/packages/examples/acp-demo/src/invariant.ts b/packages/sdk/python-runtime/src/invariant.ts similarity index 77% rename from packages/examples/acp-demo/src/invariant.ts rename to packages/sdk/python-runtime/src/invariant.ts index 106ba974d5..79bb5ec1f6 100644 --- a/packages/examples/acp-demo/src/invariant.ts +++ b/packages/sdk/python-runtime/src/invariant.ts @@ -1,16 +1,16 @@ /** - * Package-owned invariant companion for `@deepseek-ai/dsh-acp-demo`. - * @module @deepseek-ai/dsh-acp-demo/invariant + * Package-owned invariant companion for `@deepseek-ai/dsh-sdk-python-runtime`. + * @module @deepseek-ai/dsh-sdk-python-runtime/invariant */ /* jscpd:ignore-start */ import type { Context } from '@deepseek-ai/cordis' import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' -const PACKAGE_NAME = '@deepseek-ai/dsh-acp-demo' +const PACKAGE_NAME = '@deepseek-ai/dsh-sdk-python-runtime' /** Cordis companion plugin name. */ -export const name = 'acp-demo-invariant' +export const name = 'sdk-python-runtime-invariant' /** Service required before the companion can reserve package ownership. */ export const inject = ['invariants'] diff --git a/packages/examples/jsonrpc-demo/src/packaged-bin.ts b/packages/sdk/python-runtime/src/packaged-bin.ts similarity index 62% rename from packages/examples/jsonrpc-demo/src/packaged-bin.ts rename to packages/sdk/python-runtime/src/packaged-bin.ts index 4ca60cde4c..1c5e5c3e92 100644 --- a/packages/examples/jsonrpc-demo/src/packaged-bin.ts +++ b/packages/sdk/python-runtime/src/packaged-bin.ts @@ -3,10 +3,10 @@ * Closed-runtime JSON-RPC agent bin. Bare plugins resolve from the installed * runtime closure while relative plugins remain configuration-relative. * - * @module @deepseek-ai/dsh-sdk-jsonrpc-demo/packaged-bin + * @module @deepseek-ai/dsh-sdk-python-runtime/packaged-bin */ -import { runJsonrpcAgent } from './runner.ts' +import { runPythonSdkRuntime } from './runner.ts' /* v8 ignore next -- exercised through the built Python runtime carriers */ -await runJsonrpcAgent(import.meta.url) +await runPythonSdkRuntime(import.meta.url) diff --git a/packages/examples/jsonrpc-demo/src/runner.ts b/packages/sdk/python-runtime/src/runner.ts similarity index 82% rename from packages/examples/jsonrpc-demo/src/runner.ts rename to packages/sdk/python-runtime/src/runner.ts index dbc12adf92..9850c64a0b 100644 --- a/packages/examples/jsonrpc-demo/src/runner.ts +++ b/packages/sdk/python-runtime/src/runner.ts @@ -1,7 +1,7 @@ /** - * Shared process lifecycle for the generic and closed-runtime JSON-RPC bins. + * Process lifecycle for the Python SDK's closed direct-config runtime. * - * @module @deepseek-ai/dsh-sdk-jsonrpc-demo/runner + * @module @deepseek-ai/dsh-sdk-python-runtime/runner */ import { existsSync } from 'node:fs' @@ -12,12 +12,11 @@ const NAME = 'dsh-jsonrpc-agent' /** * Boot the explicitly selected external configuration and own process exit. - * @param bareModuleBaseUrl - optional installed-runtime base for bare plugins; - * omit it when the configuration project owns its plugin packages. + * @param bareModuleBaseUrl - installed-runtime base for bare plugins. * @returns after process handlers are installed; process lifetime then belongs * to stdin and signal events. */ -export async function runJsonrpcAgent(bareModuleBaseUrl?: string): Promise { +export async function runPythonSdkRuntime(bareModuleBaseUrl: string): Promise { installFailLoud(NAME) loadEnv(NAME) diff --git a/packages/examples/jsonrpc-demo/tsconfig.json b/packages/sdk/python-runtime/tsconfig.json similarity index 100% rename from packages/examples/jsonrpc-demo/tsconfig.json rename to packages/sdk/python-runtime/tsconfig.json diff --git a/packages/examples/jsonrpc-demo/tsdown.config.ts b/packages/sdk/python-runtime/tsdown.config.ts similarity index 79% rename from packages/examples/jsonrpc-demo/tsdown.config.ts rename to packages/sdk/python-runtime/tsdown.config.ts index 3609ebbc93..9b6d5d33a7 100644 --- a/packages/examples/jsonrpc-demo/tsdown.config.ts +++ b/packages/sdk/python-runtime/tsdown.config.ts @@ -10,10 +10,6 @@ export default defineConfig([ entry: ['lib/types/invariant.js'], outDir: 'lib', format: ['esm'], platform: 'node', target: 'es2024', fixedExtension: false, outputOptions: { codeSplitting: false }, dts: false, clean: false, }, - { - entry: ['lib/types/bin.js'], outDir: 'lib', format: ['esm'], platform: 'node', target: 'es2024', - fixedExtension: false, outputOptions: { codeSplitting: false }, dts: false, clean: false, - }, { entry: ['lib/types/packaged-bin.js'], outDir: 'lib', format: ['esm'], platform: 'node', target: 'es2024', fixedExtension: false, outputOptions: { codeSplitting: false }, dts: false, clean: false, diff --git a/packages/sdk/server/README.i18n.yaml b/packages/sdk/server/README.i18n.yaml index 98523811ba..adf84f76c4 100644 --- a/packages/sdk/server/README.i18n.yaml +++ b/packages/sdk/server/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/sdk/server/README.md -README.md: 5377b4fcf425cc5e10497e9d40fdddc075d52a10 -README.zh.md: dcca65a7175bb2774460bf265d98e41439df6a01 +README.md: e3deaa5288eb94d64a7f5a8425a829e3eed8896e +README.zh.md: a7273e9b4efbfae8e849205ea86b4c0c90588f7c diff --git a/packages/sdk/server/README.md b/packages/sdk/server/README.md index 5377b4fcf4..e3deaa5288 100644 --- a/packages/sdk/server/README.md +++ b/packages/sdk/server/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The `jsonrpc` plugin serves newline-delimited JSON-RPC over stdio so out-of-process SDK clients can drive harness agents. [`HarnessSdkJsonRpcServer`](src/server.ts) owns the protocol methods and notifications; the transport and the named wire types live in [`dsh-sdk-protocol`](../protocol/README.md), shared with the client SDKs; [`jsonrpc-demo`](../../examples/jsonrpc-demo/README.md) supplies the surrounding `cordis.yml` application. +The `jsonrpc` plugin serves newline-delimited JSON-RPC over stdio so out-of-process SDK clients can drive harness agents. [`HarnessSdkJsonRpcServer`](src/server.ts) owns the protocol methods and notifications; the transport and the named wire types live in [`dsh-sdk-protocol`](../protocol/README.md), shared with the client SDKs. The TypeScript client receives this server through `dsh --profile sdk`; the private [Python runtime carrier](../python-runtime/README.md) temporarily supplies a direct-config application around it. ## Wiring @@ -22,7 +22,7 @@ The plugin answers `shutdown`, flushes the response, disposes the root context s ## Wire notes -`initialize.serverInfo.name` is the wire-stable `deepseek-harness-sdk-runtime`. An optional positive `initialize.maxTokens` becomes the request output cap of each SDK-created agent and its in-process descendants; invalid values reject initialization, while omission sends no SDK cap and allows the selected adapter or provider route default to apply. `session/prompt` queues one identified user message and immediately returns `{ messageId }`. The server streams every durable fact as `session.event` and every whole-agent lifecycle transition as `session.status`; it does not assign an assistant message or `turn/end` to that prompt. Independent requests may enqueue more work on the same session. Persistence roots and persona come from `cordis.yml`. +`initialize` is the runtime-readiness boundary: when the server is mounted by a Loader composition, it waits for the current plugin tree to settle before replying, so async sibling capabilities such as initial MCP tool discovery are visible to the first prompt. Hand-built contexts without Loader remain immediately usable. `initialize.serverInfo.name` is the wire-stable `deepseek-harness-sdk-runtime`. An optional positive `initialize.maxTokens` becomes the request output cap of each SDK-created agent and its in-process descendants; invalid values reject initialization, while omission sends no SDK cap and allows the selected adapter or provider route default to apply. `session/prompt` queues one identified user message and immediately returns `{ messageId }`. The server streams every durable fact as `session.event` and every whole-agent lifecycle transition as `session.status`; it does not assign an assistant message or `turn/end` to that prompt. Independent requests may enqueue more work on the same session. Persistence roots and persona come from `cordis.yml`. ## Model Experience diff --git a/packages/sdk/server/README.zh.md b/packages/sdk/server/README.zh.md index dcca65a717..a7273e9b4e 100644 --- a/packages/sdk/server/README.zh.md +++ b/packages/sdk/server/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -`jsonrpc` 插件通过 stdio 提供以换行符分隔的 JSON-RPC,使进程外 SDK 客户端能够驱动 harness agent(智能体)。[`HarnessSdkJsonRpcServer`](src/server.ts) 负责协议方法和通知;传输与具名协议类型位于 [`dsh-sdk-protocol`](../protocol/README.md),与客户端 SDK 共享;[`jsonrpc-demo`](../../examples/jsonrpc-demo/README.md) 提供外围的 `cordis.yml` 应用。 +`jsonrpc` 插件通过 stdio 提供以换行符分隔的 JSON-RPC,使进程外 SDK 客户端能够驱动 harness agent(智能体)。[`HarnessSdkJsonRpcServer`](src/server.ts) 负责协议方法和通知;传输与具名协议类型位于 [`dsh-sdk-protocol`](../protocol/README.zh.md),与客户端 SDK 共享。TypeScript 客户端通过 `dsh --profile sdk` 获得该服务器;私有 [Python 运行时载体](../python-runtime/README.zh.md)暂时为其提供直读配置应用。 ## 组装 @@ -22,7 +22,7 @@ Stdout 只承载 JSON-RPC 帧。部署不得组合 stdout logger;诊断应写 ## 协议说明 -`initialize.serverInfo.name` 的协议稳定值为 `deepseek-harness-sdk-runtime`。可选的正整数 `initialize.maxTokens` 会成为每个 SDK 创建的 agent 及其进程内后代的请求输出上限;非法值会使初始化失败,省略时则不发送 SDK 上限,并应用所选适配器或提供方路由的默认值。`session/prompt` 将一条带标识的用户消息排入队列,并立即返回 `{ messageId }`。服务器将每个持久事实作为 `session.event` 流式发出,并将整个 agent 生命周期的每次状态转换作为 `session.status` 发出;它不会把某条助手消息或 `turn/end` 归属于该提示词。同一会话上的独立请求可以继续排入更多工作。持久化根目录和 persona 由 `cordis.yml` 提供。 +`initialize` 是运行时就绪边界:服务器由 Loader 组合挂载时,会等待当前插件树完成所有加载任务后再响应,因此首次提示词能够看到 MCP 初始工具发现等异步同级能力。没有 Loader 的手工组装上下文仍可立即使用。`initialize.serverInfo.name` 的协议稳定值为 `deepseek-harness-sdk-runtime`。可选的正整数 `initialize.maxTokens` 会成为每个 SDK 创建的 agent 及其进程内后代的请求输出上限;非法值会使初始化失败,省略时则不发送 SDK 上限,并应用所选适配器或提供方路由的默认值。`session/prompt` 将一条带标识的用户消息排入队列,并立即返回 `{ messageId }`。服务器将每个持久事实作为 `session.event` 流式发出,并将整个 agent 生命周期的每次状态转换作为 `session.status` 发出;它不会把某条助手消息或 `turn/end` 归属于该提示词。同一会话上的独立请求可以继续排入更多工作。持久化根目录和 persona 由 `cordis.yml` 提供。 ## 模型体验 diff --git a/packages/sdk/server/package.json b/packages/sdk/server/package.json index aad971d714..f32b4a0147 100644 --- a/packages/sdk/server/package.json +++ b/packages/sdk/server/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-sdk-jsonrpc-server", "description": "Stdio JSON-RPC server plugin for out-of-process DeepSeek Harness SDK clients", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/sdk/server/src/index.ts b/packages/sdk/server/src/index.ts index 20cb892b53..963b4fb3bd 100644 --- a/packages/sdk/server/src/index.ts +++ b/packages/sdk/server/src/index.ts @@ -1,6 +1,6 @@ /** - * SDK-facing JSON-RPC plugin over stdio. An external `cordis.yml` decides - * whether to load it; see the single-executable Agent Note and package README. + * SDK-facing JSON-RPC plugin over stdio. The selected dsh profile decides + * whether to load it; see the single-launch Agent Note and package README. * Stdout is reserved for protocol frames, so the tree must not load a stdout logger. * This plugin answers `shutdown`, disposes the complete root runtime, and exits 0; the app bin * owns EOF and signal exits. Keep named plugin exports with no default export so @@ -74,6 +74,16 @@ export function apply(ctx: Context, config: JsonRpcConfig): void { } transport.onRequest(async (method, params) => { + // `initialize` is the SDK's readiness boundary. This plugin can activate + // before async sibling Loader entries (for example an MCP client's initial + // tool discovery), so do not advertise a ready runtime until the complete + // current tree has settled. Loader settlement joins entry imports, fiber + // lifecycle work, and synchronous effect registration; no scheduler delay + // is part of readiness. A hand-built context without Loader remains + // immediately usable. + if (method === 'initialize') { + await ctx.get('loader')?.await() + } const result = await server.handleRequest(method, params) if (method === 'shutdown') { // Run after the handler result is written; the task then flushes, disposes, and exits. diff --git a/packages/sdk/server/src/server.ts b/packages/sdk/server/src/server.ts index 195caa908d..ccc3b6192d 100644 --- a/packages/sdk/server/src/server.ts +++ b/packages/sdk/server/src/server.ts @@ -133,7 +133,7 @@ export class HarnessSdkJsonRpcServer { const rec = await this.getOrCreateSession(params.sessionId) // An agent-loop-only reload disposes the loop's agents while this record // survives; a retained agent accepts followup() silently, so validate the - // record against the live registry before delivery (as the ACP bridge does). + // record against the live registry before delivery. if (this.ctx.agents.get(rec.handle.agent.id) !== rec.handle.agent) { throw new Error(`session agent was disposed outside the server: ${params.sessionId}`) } diff --git a/packages/sdk/server/tests/plugin-apply.spec.ts b/packages/sdk/server/tests/plugin-apply.spec.ts index 320a61ac63..28c89c19fe 100644 --- a/packages/sdk/server/tests/plugin-apply.spec.ts +++ b/packages/sdk/server/tests/plugin-apply.spec.ts @@ -6,7 +6,10 @@ import { tmpdir } from 'node:os' import { PassThrough, Writable } from 'node:stream' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' +import Loader from '@deepseek-ai/cordis-plugin-loader' import * as agentCore from '@deepseek-ai/dsh-agent-spine-demo' +import { LlmAdapter } from '@deepseek-ai/dsh-llm' +import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl' import * as jsonrpc from '../src/index.ts' @@ -38,6 +41,13 @@ interface ApplyHarness { dispose(): Promise } +/** Adapter whose route registration is the delayed Loader entry's readiness fact. */ +class DelayedAdapter extends LlmAdapter { + async * stream(_options: GenerateOptions): AsyncIterable { + throw new Error('not exercised') + } +} + /** Poll asynchronous output for up to five seconds. */ async function waitFor(get: () => T | undefined, description: string): Promise { const deadline = Date.now() + 5000 @@ -57,12 +67,17 @@ async function settle(): Promise { /** Mount the real plugin on a minimal harness with in-memory stdio and exit. */ async function mountPlugin( storageDir: string, - options: { writeDelayMs?: number; failFlush?: boolean } = {}, + options: { + writeDelayMs?: number + failFlush?: boolean + beforeServer?: (ctx: Context) => Promise | void + } = {}, ): Promise { const ctx = new Context() await ctx.plugin(agentCore, { workspaceContext: false }) await ctx.plugin(JsonlSessionPersistence, { root: storageDir }) await new Promise(resolve => setTimeout(resolve, 50)) + await options.beforeServer?.(ctx) const input = new PassThrough() const events: WireEvent[] = [] @@ -170,6 +185,61 @@ describe('dsh-sdk-jsonrpc-server plugin apply', () => { } }) + it('waits for Loader-owned adapter registration before initialize', async () => { + const storageDir = await mkdtemp(join(tmpdir(), 'dsh-jsonrpc-apply-readiness-')) + vi.stubEnv('DEEPSEEK_API_KEY', 'test-key') + let markStarted!: () => void + let release!: () => void + const started = new Promise((resolve) => { markStarted = resolve }) + const ready = new Promise((resolve) => { release = resolve }) + let delayedEntry: Promise | undefined + const harness = await mountPlugin(storageDir, { + beforeServer: async (ctx) => { + await ctx.plugin(Loader) + ctx.loader.builtins['delayed-readiness'] = { + inject: ['llm'], + async apply(entryCtx: Context) { + markStarted() + await ready + entryCtx.llm.registerAdapter(['delayed-private'], new DelayedAdapter()) + }, + } + delayedEntry = ctx.loader.create({ name: 'cordis:delayed-readiness' }) + await started + }, + }) + try { + const initialize = { + jsonrpc: '2.0', + id: 'init-delayed', + method: 'initialize', + params: { cwd: storageDir, provider: 'delayed-private', model: 'apply-model' }, + } + const probe = { jsonrpc: '2.0', id: 'probe-during-delay', method: 'nope/unknown' } + harness.sendRaw(`${JSON.stringify(initialize)}\n${JSON.stringify(probe)}\n`) + + // The transport processes independent requests concurrently. Receiving + // this later probe proves the preceding initialize handler has reached + // its Loader wait, without relying on a scheduler delay. + await harness.waitForFrame(frame => frame.id === 'probe-during-delay', 'probe while initialize waits') + expect(harness.frames().some(frame => frame.id === 'init-delayed')).toBe(false) + + release() + await delayedEntry + const response = await harness.waitForFrame(frame => frame.id === 'init-delayed', 'initialize response after Loader settlement') + expect(response).toMatchObject({ + id: 'init-delayed', + result: { serverInfo: { name: 'deepseek-harness-sdk-runtime' } }, + }) + expect(harness.ctx.llm.listProviders()).toContainEqual({ id: 'delayed-private', name: 'delayed-private' }) + } finally { + release() + await Promise.allSettled(delayedEntry === undefined ? [] : [delayedEntry]) + await harness.dispose() + await rm(storageDir, { recursive: true, force: true }) + } + }) + it('drives a session/prompt turn end-to-end and forwards session notifications as output frames', async () => { const storageDir = await mkdtemp(join(tmpdir(), 'dsh-jsonrpc-apply-prompt-')) const llmServer = await mockCompletionServer() diff --git a/packages/session-query/README.i18n.yaml b/packages/session-query/README.i18n.yaml index 544fef197b..ba11f33c04 100644 --- a/packages/session-query/README.i18n.yaml +++ b/packages/session-query/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session-query/README.md README.md: b955a7f6e19f5a49e48e564523cc343e32816dd2 -README.zh.md: d1e6c645050f4e58f9409e328552b5f9209bc3a5 +README.zh.md: ebf3bd5c5f18e00b0beec4855fb99907019664ed diff --git a/packages/session-query/README.zh.md b/packages/session-query/README.zh.md index d1e6c64505..ebf3bd5c5f 100644 --- a/packages/session-query/README.zh.md +++ b/packages/session-query/README.zh.md @@ -6,9 +6,9 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`session-query/`](session-query/README.md) | 定义可信读取、关系查询和搜索操作 | `ctx.sessionQuery` | -| [`session-query-sqlite/`](session-query-sqlite/README.md) | 使用 SQLite 全文搜索实现会话查询 | `ctx.sessionQuery` | -| [`session-log-export/`](session-log-export/README.md) | 在 Host ZIP 端点之上增加 Web `/export` 命令、共享浏览器下载状态和结果弹窗 | `ctx.sessionLogDownload` | -| [`tool-session-query/`](tool-session-query/README.md) | 向模型公开经过工作区授权的会话查询 | 注册到 `ctx.tools` | +| [`session-query/`](session-query/README.zh.md) | 定义可信读取、关系查询和搜索操作 | `ctx.sessionQuery` | +| [`session-query-sqlite/`](session-query-sqlite/README.zh.md) | 使用 SQLite 全文搜索实现会话查询 | `ctx.sessionQuery` | +| [`session-log-export/`](session-log-export/README.zh.md) | 在 Host ZIP 端点之上增加 Web `/export` 命令、共享浏览器下载状态和结果弹窗 | `ctx.sessionLogDownload` | +| [`tool-session-query/`](tool-session-query/README.zh.md) | 向模型公开经过工作区授权的会话查询 | 注册到 `ctx.tools` | -子系统参考——逻辑记录、有界读取、追踪、筛选器、结果页——见 [docs/subsystems/session-query.md](../../docs/subsystems/session-query.md)。 +子系统参考——逻辑记录、有界读取、追踪、筛选器、结果页——见 [docs/subsystems/session-query.md](../../docs/subsystems/session-query.zh.md)。 diff --git a/packages/session-query/session-log-export/README.i18n.yaml b/packages/session-query/session-log-export/README.i18n.yaml index d333d5ed90..0d158164ee 100644 --- a/packages/session-query/session-log-export/README.i18n.yaml +++ b/packages/session-query/session-log-export/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session-query/session-log-export/README.md README.md: f3625338e85a38ab279c5fa4700a7a9e1124cfb3 -README.zh.md: 5a2fa86d126f6bda808d3036bddc0e26fd310690 +README.zh.md: 101f0bb41c7296916b72a82186dae74b1b576e7f diff --git a/packages/session-query/session-log-export/README.zh.md b/packages/session-query/session-log-export/README.zh.md index 5a2fa86d12..101f0bb41c 100644 --- a/packages/session-query/session-log-export/README.zh.md +++ b/packages/session-query/session-log-export/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -Web Session 日志下载控制,使用 `dsh-host-apiproxy` 拥有的 Host 流式 ZIP 端点。Host 半包注册 `/export`;浏览器半包在 Session Header 中提供 111×32 的 `Session log` 操作,以及一个供该按钮与斜杠命令共用的下载控制器和弹窗。ZIP 生成、原始 JSONL/zstd 读取、子 Session、附件、背压和 HTTP 错误语义仍由 [ApiProxy 下载实现](../../host/apiproxy/README.md)负责。 +Web Session 日志下载控制,使用 `dsh-host-apiproxy` 拥有的 Host 流式 ZIP 端点。Host 半包注册 `/export`;浏览器半包在 Session Header 中提供 111×32 的 `Session log` 操作,以及一个供该按钮与斜杠命令共用的下载控制器和弹窗。ZIP 生成、原始 JSONL/zstd 读取、子 Session、附件、背压和 HTTP 错误语义仍由 [ApiProxy 下载实现](../../host/apiproxy/README.zh.md)负责。 ## 命令约定 diff --git a/packages/session-query/session-log-export/package.json b/packages/session-query/session-log-export/package.json index b020ce301b..0d5988c0e1 100644 --- a/packages/session-query/session-log-export/package.json +++ b/packages/session-query/session-log-export/package.json @@ -1,8 +1,8 @@ { "name": "@deepseek-ai/dsh-session-log-export", "description": "Web Session-log export command and shared download dialog", - "version": "0.0.1-rc.5", - "publishConfig": { "access": "restricted" }, + "version": "0.1.1-rc.2", + "publishConfig": { "access": "public" }, "repository": { "type": "git", "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", @@ -27,11 +27,8 @@ "@deepseek-ai/dsh-client-runtime": "workspace:^", "@deepseek-ai/dsh-client-ui-commands": "workspace:^", "@deepseek-ai/dsh-client-ui-conversation": "workspace:^", - "@deepseek-ai/dsh-client-ui-primitives": "workspace:^", - "@deepseek-ai/dsh-client-ui-slots": "workspace:^", "@deepseek-ai/dsh-commands": "workspace:^", - "@deepseek-ai/dsh-invariants": "workspace:^", - "react": "^18.2.0" + "@deepseek-ai/dsh-invariants": "workspace:^" }, "devDependencies": { "@deepseek-ai/cordis": "workspace:^", diff --git a/packages/session-query/session-log-export/tests/loader-composition.client.spec.ts b/packages/session-query/session-log-export/tests/loader-composition.client.spec.ts index facd7e13fd..2a993f52ef 100644 --- a/packages/session-query/session-log-export/tests/loader-composition.client.spec.ts +++ b/packages/session-query/session-log-export/tests/loader-composition.client.spec.ts @@ -60,7 +60,7 @@ describe('session-log-download real Loader composition', () => { expect(context.commands.list(agent)).toContainEqual({ name: 'export', description: 'Download this Session log as a ZIP archive', }) - const execution = await context.commands.execute(agent, '/export', new AbortController().signal) + const execution = await context.commands.execute(agent, '/export', [], new AbortController().signal) expect(execution?.result).toEqual({ kind: 'success', text: 'Session log download requested.' }) expect(session.events.map(event => event.type)).toEqual(['command/run', 'command/done']) expect(session.deriveMessages()).toEqual([]) diff --git a/packages/session-query/session-query-sqlite/package.json b/packages/session-query/session-query-sqlite/package.json index 1644203ecd..01bd2da609 100644 --- a/packages/session-query/session-query-sqlite/package.json +++ b/packages/session-query/session-query-sqlite/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-query-sqlite", "description": "Concrete ctx.sessionQuery backend with SQLite FTS5 search", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session-query/session-query/README.i18n.yaml b/packages/session-query/session-query/README.i18n.yaml index 783b6d9e6f..b668d26d4b 100644 --- a/packages/session-query/session-query/README.i18n.yaml +++ b/packages/session-query/session-query/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session-query/session-query/README.md README.md: 02640bee4b3eb7f6877756a2ce84b26f698c9f58 -README.zh.md: 794c0e6e86606d92ed82ec286844647487df3295 +README.zh.md: dbac518b2c295825459475dba5a9c6c38f53afc8 diff --git a/packages/session-query/session-query/README.zh.md b/packages/session-query/session-query/README.zh.md index 794c0e6e86..dbac518b2c 100644 --- a/packages/session-query/session-query/README.zh.md +++ b/packages/session-query/session-query/README.zh.md @@ -29,7 +29,7 @@ `SessionQueryEngine.searchSessions(request, exec?)` 按匹配最强的事件对逻辑语料库分组;`searchEvents(request, exec?)` 搜索一个逻辑会话。这两个是服务仅有的抽象方法。两者都返回分页结果,其延续信息是由服务持有的带品牌 `SessionSearchCursor`;接受可选取消,并在不使用提供方专用数值分数的情况下提供摘录。事件搜索分页结果还携带来自与命中相同索引世代的克隆目标 header,使授权消费方可将策略绑定到此次载荷观察。搜索请求只接受事件元数据过滤器,因为字面文本过滤使用上文所述扫描路径。 -该包没有提供方协调器、回退实现或独立具体插件。具体服务后端继承已实现的读取、过滤和跟踪,同时负责全文观察、对账、排名、游标世代和查询执行;第一个实现是 [`@deepseek-ai/dsh-session-query-sqlite`](../session-query-sqlite/README.md)。 +该包没有提供方协调器、回退实现或独立具体插件。具体服务后端继承已实现的读取、过滤和跟踪,同时负责全文观察、对账、排名、游标世代和查询执行;第一个实现是 [`@deepseek-ai/dsh-session-query-sqlite`](../session-query-sqlite/README.zh.md)。 `SessionQueryError.code` 是一个封闭联合,覆盖请求验证、缺失目标、格式错误的表层、来源冲突、持久化/索引失败、取消,以及无效或陈旧游标;精确字面值在 [`src/config.ts`](src/config.ts) 中定义。 @@ -53,4 +53,4 @@ ## 已知限制与暂缓事项 - **无调用方授权**:这是上下文范围内的可信基础设施;未来的模型工具或 UI 必须限制调用方可检查的会话。 -- **无注册表或面向模型工具**:尚未提供提取器和搜索提供方注册表、递归遍历所引用的源事件的能力,以及面向模型的工具。[跟踪决策](../../../.agents/notes/implemented/feature/2026-07-13-session-query-tracing.md) 负责关系语义;SQLite 归属和 tokenizer 决策位于[已实现搜索记录](../../../.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.md)。 +- **无注册表或面向模型工具**:尚未提供提取器和搜索提供方注册表、递归遍历所引用的源事件的能力,以及面向模型的工具。[跟踪决策](../../../.agents/notes/implemented/feature/2026-07-13-session-query-tracing.zh.md) 负责关系语义;SQLite 归属和 tokenizer 决策位于[已实现搜索记录](../../../.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.zh.md)。 diff --git a/packages/session-query/session-query/package.json b/packages/session-query/session-query/package.json index ce88a9665b..9d4f7bd1f2 100644 --- a/packages/session-query/session-query/package.json +++ b/packages/session-query/session-query/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-query", "description": "Combined session query service contract with concrete reads, traces, and filters", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -37,6 +37,7 @@ "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-session-title": "workspace:^", + "@deepseek-ai/dsh-tool-todo": "workspace:^", "@deepseek-ai/dsh-session-persistence": "workspace:^", "@deepseek-ai/cordis": "workspace:^" }, @@ -51,6 +52,7 @@ "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-session-title": "workspace:^", + "@deepseek-ai/dsh-tool-todo": "workspace:^", "@deepseek-ai/dsh-session-persistence": "workspace:^", "@deepseek-ai/cordis": "workspace:^" } diff --git a/packages/session-query/session-query/src/corpus.ts b/packages/session-query/session-query/src/corpus.ts index 717b2b5190..69fbc8a1a9 100644 --- a/packages/session-query/session-query/src/corpus.ts +++ b/packages/session-query/session-query/src/corpus.ts @@ -2,7 +2,7 @@ import type { Context, Fiber } from '@deepseek-ai/cordis' import type { Session, SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session' -import SessionPersistence, { SessionPersistenceCorruptionError } from '@deepseek-ai/dsh-session-persistence' +import type SessionPersistence from '@deepseek-ai/dsh-session-persistence' import type { SessionRecord } from './types.ts' import { SessionQueryError } from './config.ts' import { assertSessionHeadersCompatible } from './sources.ts' @@ -274,7 +274,7 @@ async function inspectPersisted( return await persistence.inspect(sessionId, signal) } catch (error: unknown) { if (signal?.aborted) signal.throwIfAborted() - if (error instanceof SessionPersistenceCorruptionError) { + if (error instanceof Error && error.name === 'SessionPersistenceCorruptionError') { throw new SessionQueryError( `stored session "${sessionId}" is corrupt: ${errorMessage(error)}`, 'SESSION_QUERY_CORRUPT_SESSION', diff --git a/packages/session-query/session-query/src/extraction.ts b/packages/session-query/session-query/src/extraction.ts index d01ccb7aa3..1efc83f251 100644 --- a/packages/session-query/session-query/src/extraction.ts +++ b/packages/session-query/session-query/src/extraction.ts @@ -1,6 +1,8 @@ /** First-party semantic text extraction for session-query consumers. */ import type { SessionEvent } from '@deepseek-ai/dsh-session' +// Type-only: includes the first-party todo event consumed below. +import type {} from '@deepseek-ai/dsh-tool-todo' /** * Extract searchable semantic text from one first-party session event. diff --git a/packages/session-query/session-query/tsconfig.json b/packages/session-query/session-query/tsconfig.json index b4764a9b48..408f51652b 100644 --- a/packages/session-query/session-query/tsconfig.json +++ b/packages/session-query/session-query/tsconfig.json @@ -26,6 +26,9 @@ { "path": "../../session/session-title" }, + { + "path": "../../todo/tool-todo" + }, { "path": "../../session/session-persistence" }, diff --git a/packages/session-query/tool-session-query/README.i18n.yaml b/packages/session-query/tool-session-query/README.i18n.yaml index 4bc40ec6a2..9434605a7a 100644 --- a/packages/session-query/tool-session-query/README.i18n.yaml +++ b/packages/session-query/tool-session-query/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session-query/tool-session-query/README.md -README.md: d973daf1124c4be05f7335b18661d431d45be39f -README.zh.md: 18ff18520ba8bb9d3d3dba5091d6281a0a47b78a +README.md: baa650dae9a0267024bdd0f6917d78d55d95f605 +README.zh.md: 16aab0be1407453a4eab42e42850c2740f8a561d diff --git a/packages/session-query/tool-session-query/README.md b/packages/session-query/tool-session-query/README.md index d973daf112..baa650dae9 100644 --- a/packages/session-query/tool-session-query/README.md +++ b/packages/session-query/tool-session-query/README.md @@ -11,11 +11,11 @@ Workspace-authorized model tools over `ctx.sessionQuery`. The opt-in package dep | `maxSearchResults` | `100` | Maximum authorized non-self hits collected across internal provider pages | | `searchTimeoutMs` | `30000` | Cooperative deadline attached to both full-text search tools | -The caller comes exclusively from `ToolExecution.exec.agent`. Cross-session access requires exact equality between the target and caller session `cwd` values; a caller without `cwd` can inspect only itself. Search never exposes provider cursors, offsets, page sizes, or a model-controlled limit. Because one search consumes generation-bound provider cursors internally, both search tools execute exclusively with sibling tool calls; the three exact trace/read tools opt into parallel execution. Every exact executor passes its unchanged execution signal through authorization and the service trace/read, so cancellation waits for cooperative persistence cleanup and retains the signal's exact reason. Timestamps at the tool boundary require an explicit `Z` or numeric offset and become inclusive epoch-millisecond filters. +The tools derive caller authority from `ToolExecution.exec.agent`. Cross-session access requires exact equality between the target and caller session `cwd` values; a caller without `cwd` can inspect only itself. Search never exposes paging controls or a model-controlled result limit. Cancellation retains the caller's reason, and timestamp filters require an explicit `Z` or numeric offset and include both endpoints. `session_search` always omits the caller session. Requested parent ids are deduplicated and checked against caller-workspace authority before FTS; only authorized ids reach the provider, while missing and cross-workspace guesses behave identically and the root marker remains independently ORed. A current-session `session_event_search` stops immediately before the step that invoked it, so the active assistant output and logged tool call cannot match themselves. Direct targets are authorized before trace, event, or title reads. Lineage output replaces unauthorized ancestor and descendant boundaries with markers that contain no hidden session id. -Every trusted `ctx.sessionQuery` call crosses one model-boundary sanitizer. Caller cancellation is checked first and preserved exactly. Available corpus and provider diagnostics, including safely inspectable nested causes, are logged internally on a best-effort basis; unprintable failures use a fixed log placeholder. Diagnostic formatting and error classification are independently guarded, so an unprintable cause cannot escape or prevent a safely classified outer error, while unsafe classification or logging falls back to the fixed `SESSION_QUERY_TOOL_FAILED` code and message. Local argument-validation and authorization errors retain their precise tool-owned messages. +Provider failures log available diagnostics but return the fixed `SESSION_QUERY_TOOL_FAILED` code and message to the model. Local argument-validation and authorization errors retain their precise tool-owned messages. The package deliberately performs no byte or character truncation and does not import a spill backend. Deployments that need bounded inline output mount `@deepseek-ai/dsh-spill-policy`, which can replace the rendered text after execution while retaining the complete result. diff --git a/packages/session-query/tool-session-query/README.zh.md b/packages/session-query/tool-session-query/README.zh.md index 18ff18520b..16aab0be14 100644 --- a/packages/session-query/tool-session-query/README.zh.md +++ b/packages/session-query/tool-session-query/README.zh.md @@ -11,11 +11,11 @@ | `maxSearchResults` | `100` | 在内部提供方分页中收集的最大已授权非自身命中数 | | `searchTimeoutMs` | `30000` | 附加到两个全文搜索工具的协作式截止时间 | -调用方只能来自 `ToolExecution.exec.agent`。跨会话访问要求目标和调用方会话的 `cwd` 值严格相等;没有 `cwd` 的调用方只能检查自己。搜索绝不公开提供方游标、偏移、分页大小或模型可控上限。由于一次搜索会在内部消费与世代绑定的提供方游标,两个搜索工具都与同级工具调用排他执行;三个精确跟踪/读取工具选择并行执行。每个精确执行器都将未更改的执行信号传递给授权和服务跟踪/读取,因此取消会等待协作式持久化清理,并保留信号的精确原因。工具边界上的时间戳要求显式 `Z` 或数字偏移,并转换为包含端点的 epoch 毫秒过滤器。 +工具根据 `ToolExecution.exec.agent` 派生调用方权限。跨会话访问要求目标和调用方会话的 `cwd` 值严格相等;没有 `cwd` 的调用方只能检查自己。搜索绝不公开分页控制项或模型可控的结果上限。取消会保留调用方原因,时间戳过滤器要求显式 `Z` 或数字偏移,并包含两个端点。 `session_search` 始终省略调用方会话。请求的父 id 会被去重,并在 FTS 前根据调用方工作区权限检查;只有已授权 id 会到达提供方,而缺失猜测和跨工作区猜测的行为完全相同,root 标记仍独立使用 OR。当前会话中的 `session_event_search` 会在调用它的步骤之前立即停止,因此当前 assistant 输出和已记录工具调用无法匹配自身。直接目标在跟踪、事件或标题读取前完成授权。血缘输出会用不含隐藏会话 id 的标记替换未授权祖先和后代边界。 -每个可信 `ctx.sessionQuery` 调用都会经过一个模型边界净化器。首先检查调用方取消,并精确保留。可获取的语料库诊断信息和提供方诊断信息(包括可安全检查的嵌套原因)会尽力记录到内部日志;不可打印的失败使用固定日志占位符。诊断格式化和错误分类各自独立受保护,因此不可打印的原因无法逃逸,也无法阻止已安全分类的外层错误;不安全的分类或日志记录则回退到固定 `SESSION_QUERY_TOOL_FAILED` 代码和消息。本地参数验证和授权错误保留精确的工具自有消息。 +提供方失败会记录可用的诊断信息,但向模型返回固定的 `SESSION_QUERY_TOOL_FAILED` 代码和消息。本地参数校验和授权错误保留精确的工具自有消息。 该包刻意不执行字节或字符截断,也不导入 spill 后端。需要限制内联输出的部署应挂载 `@deepseek-ai/dsh-spill-policy`,它可在执行后替换已渲染文本,同时保留完整结果。 @@ -45,7 +45,7 @@ Use session_search to find relevant work from prior sessions, or session_event_s #### 模型看到的内容 -模型会看到生成的 [`session_search`、`session_event_search`、`session_trace`、`session_event_trace` 和 `session_event_read` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-session-query)。搜索过滤器会增加固定 schema token,而游标、工作区路径、输出分页和模型可控结果上限仍不存在。 +模型会看到生成的 [`session_search`、`session_event_search`、`session_trace`、`session_event_trace` 和 `session_event_read` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-session-query)。搜索过滤器会增加固定 schema token,而游标、工作区路径、输出分页和模型可控结果上限仍不存在。 #### Token 影响 diff --git a/packages/session-query/tool-session-query/package.json b/packages/session-query/tool-session-query/package.json index 58bd6ee7ef..99ae13a209 100644 --- a/packages/session-query/tool-session-query/package.json +++ b/packages/session-query/tool-session-query/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-session-query", "description": "Workspace-authorized model-facing session history search, trace, and event read tools", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/README.i18n.yaml b/packages/session/README.i18n.yaml index 2939d73342..69cbe17e10 100644 --- a/packages/session/README.i18n.yaml +++ b/packages/session/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/README.md -README.md: 2680bc3c6aea519427da2f31ab2526b5115cac96 -README.zh.md: b446a6a1d17da39c2a3d29c10fbb8dddf573bf80 +README.md: 78fbc0ab6f0a3b9a77c9e3b1a538dd227c532296 +README.zh.md: e1c04337931a42a1751edb2d3d616b4b53d2af1b diff --git a/packages/session/README.md b/packages/session/README.md index 2680bc3c6a..78fbc0ab6f 100644 --- a/packages/session/README.md +++ b/packages/session/README.md @@ -13,7 +13,7 @@ Durable session persistence, semantic checkpoint policy, and the shipped storage | [`session-persistence/`](session-persistence/README.md) | Defines the persistence service and shared write coordination | `ctx.sessionPersistence` | | [`session-checkpoint-policy/`](session-checkpoint-policy/README.md) | Applies semantic durability checkpoints | wraps `ctx.llm` and `ctx.tools` | | [`session-persistence-jsonl/`](session-persistence-jsonl/README.md) | Persists sessions in JSONL files | registers on `ctx.sessionPersistence` | -| [`session-persistence-sqlite/`](session-persistence-sqlite/README.md) | Persists sessions in SQLite | registers on `ctx.sessionPersistence` | +| [`session-persistence-sqlite/`](session-persistence-sqlite/README.md) | Opt-in SQLite backend with packed physical chunk rows | registers on `ctx.sessionPersistence` | The [session-persistence decision](../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md) records the persistence design. diff --git a/packages/session/README.zh.md b/packages/session/README.zh.md index b446a6a1d1..e1c0433793 100644 --- a/packages/session/README.zh.md +++ b/packages/session/README.zh.md @@ -10,12 +10,12 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`session-persistence/`](session-persistence/README.md) | 定义持久化服务和共享写入协调机制 | `ctx.sessionPersistence` | -| [`session-checkpoint-policy/`](session-checkpoint-policy/README.md) | 应用语义持久性检查点 | 包装 `ctx.llm` 和 `ctx.tools` | -| [`session-persistence-jsonl/`](session-persistence-jsonl/README.md) | 将会话持久化到 JSONL 文件 | 注册到 `ctx.sessionPersistence` | -| [`session-persistence-sqlite/`](session-persistence-sqlite/README.md) | 将会话持久化到 SQLite | 注册到 `ctx.sessionPersistence` | +| [`session-persistence/`](session-persistence/README.zh.md) | 定义持久化服务和共享写入协调机制 | `ctx.sessionPersistence` | +| [`session-checkpoint-policy/`](session-checkpoint-policy/README.zh.md) | 应用语义持久性检查点 | 包装 `ctx.llm` 和 `ctx.tools` | +| [`session-persistence-jsonl/`](session-persistence-jsonl/README.zh.md) | 将会话持久化到 JSONL 文件 | 注册到 `ctx.sessionPersistence` | +| [`session-persistence-sqlite/`](session-persistence-sqlite/README.zh.md) | 使用物理分片打包行的可选 SQLite 后端 | 注册到 `ctx.sessionPersistence` | -[会话持久化决策](../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md)记录了持久化设计。 +[会话持久化决策](../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md)记录了持久化设计。 ## 投影 @@ -23,9 +23,9 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`session-projection/`](session-projection/README.md) | 定义并驱动会话投影单元 | `ctx.sessionProjections` | -| [`session-projection-cache/`](session-projection-cache/README.md) | 持久化并恢复投影检查点 | `ctx.sessionProjectionCache` | -| [`session-stats/`](session-stats/README.md) | 提供全日志会话计数与墙钟时间(`sessionStats` 单元) | 注册到 `ctx.sessionProjections` | +| [`session-projection/`](session-projection/README.zh.md) | 定义并驱动会话投影单元 | `ctx.sessionProjections` | +| [`session-projection-cache/`](session-projection-cache/README.zh.md) | 持久化并恢复投影检查点 | `ctx.sessionProjectionCache` | +| [`session-stats/`](session-stats/README.zh.md) | 提供全日志会话计数与墙钟时间(`sessionStats` 单元) | 注册到 `ctx.sessionProjections` | ## 标题 @@ -33,20 +33,20 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`session-title/`](session-title/README.md) | 负责标题状态、回退行为、提供方注册与刷新 | `ctx.sessionTitle` | -| [`session-title-llm/`](session-title-llm/README.md) | 提供共享的模型标题生成能力 | — | -| [`session-title-first-prompt-llm/`](session-title-first-prompt-llm/README.md) | 根据第一条合格的人类消息生成会话标题 | 注册到 `ctx.sessionTitle` | -| [`session-title-all-prompts-llm/`](session-title-all-prompts-llm/README.md) | 根据所有合格的人类消息生成会话标题 | 注册到 `ctx.sessionTitle` | +| [`session-title/`](session-title/README.zh.md) | 负责标题状态、回退行为、提供方注册与刷新 | `ctx.sessionTitle` | +| [`session-title-llm/`](session-title-llm/README.zh.md) | 提供共享的模型标题生成能力 | — | +| [`session-title-first-prompt-llm/`](session-title-first-prompt-llm/README.zh.md) | 根据第一条合格的人类消息生成会话标题 | 注册到 `ctx.sessionTitle` | +| [`session-title-all-prompts-llm/`](session-title-all-prompts-llm/README.zh.md) | 根据所有合格的人类消息生成会话标题 | 注册到 `ctx.sessionTitle` | 部署可以注册一个模型驱动提供方;未注册时,服务仍保留确定性回退机制。 ## 遥测 -将会话活动投影为外发遥测,并将投递委派给配置的上报后端。[遥测决策](../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md)记录上报边界;[模式决策](../../.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.md)记录即时、反馈门控与禁用投递。 +将会话活动投影为外发遥测,并将投递委派给配置的上报后端。[遥测决策](../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md)记录上报边界;[模式决策](../../.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.zh.md)记录即时、反馈门控与禁用投递。 | 包 | 职责 | |---|---| -| [`session-telemetry/`](session-telemetry/README.md) | 定义捕获、脱敏、投影,以及实时或按需后端投递。 | -| [`session-telemetry-otel/`](session-telemetry-otel/README.md) | 通过 OpenTelemetry 日志以 `FULL`、`FEEDBACK_ONLY` 或 `DISABLED` 模式投递遥测。 | +| [`session-telemetry/`](session-telemetry/README.zh.md) | 定义捕获、脱敏、投影,以及实时或按需后端投递。 | +| [`session-telemetry-otel/`](session-telemetry-otel/README.zh.md) | 通过 OpenTelemetry 日志以 `FULL`、`FEEDBACK_ONLY` 或 `DISABLED` 模式投递遥测。 | -子系统参考:[persistence.md](../../docs/subsystems/persistence.md)、[session-projection.md](../../docs/subsystems/session-projection.md)、[session-title.md](../../docs/subsystems/session-title.md) 与 [session-telemetry.md](../../docs/subsystems/session-telemetry.md)。同一时间只允许一个标题提供方注册;demo 主干挂载回退服务,两个模型提供方都留在默认组合之外。 +子系统参考:[persistence.md](../../docs/subsystems/persistence.zh.md)、[session-projection.md](../../docs/subsystems/session-projection.zh.md)、[session-title.md](../../docs/subsystems/session-title.zh.md) 与 [session-telemetry.md](../../docs/subsystems/session-telemetry.zh.md)。同一时间只允许一个标题提供方注册;demo 主干挂载回退服务,两个模型提供方都留在默认组合之外。 diff --git a/packages/session/session-checkpoint-policy/package.json b/packages/session/session-checkpoint-policy/package.json index a2b275ed29..563ddd47d1 100644 --- a/packages/session/session-checkpoint-policy/package.json +++ b/packages/session/session-checkpoint-policy/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-checkpoint-policy", "description": "Semantic session durability checkpoints before model requests and tool side effects", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-log-deepseek/README.i18n.yaml b/packages/session/session-log-deepseek/README.i18n.yaml new file mode 100644 index 0000000000..d9650b10d8 --- /dev/null +++ b/packages/session/session-log-deepseek/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/session/session-log-deepseek/README.md +README.md: 24cff61f77dab76caf086375931edd5ee45e3f04 +README.zh.md: 0411fcd298acae86aee93da7efe812ab6d589c29 diff --git a/packages/session/session-log-deepseek/README.md b/packages/session/session-log-deepseek/README.md new file mode 100644 index 0000000000..24cff61f77 --- /dev/null +++ b/packages/session/session-log-deepseek/README.md @@ -0,0 +1,47 @@ +# @deepseek-ai/dsh-session-log-deepseek + +English | [中文](README.zh.md) + +Incremental canonical session-log upload for official DeepSeek LLM API requests. This function plugin injects `ctx.sessions` and `ctx.deepseekLlmApiExtensions`, then owns the `dsh_session_log` request field and the durable `session-log-deepseek/delivery-accepted` event from which it derives the acceptance watermark. + +## Configuration + +| Key | Default | Meaning | +|---|---:|---| +| `enabled` | `false` | Register the `dsh_session_log` contribution. Set it to `true` to opt into Session-log upload. | + +Shipped profiles mount the plugin so an overlay can enable it, but the default configuration registers no request field and appends no acceptance watermark. + +## Request field + +For a request carrying a live `sessionId`, the plugin folds the greatest accepted watermark for that exact Session identity, snapshots `Session.events`, and sends the contiguous suffix after the watermark. A process-local fold scans each event once and consumes later appends incrementally; restart and HMR rebuild it from the durable log. The version-1 field contains the immutable `SessionHeader`, `afterSeq`, `throughSeq`, and every complete canonical `SessionEvent` in that range as a direct array element. Forked sessions ignore inherited parent watermarks because each watermark records the Session id sent on the accepted request. + +## Acceptance and retry + +The DeepSeek adapter calls the prepared contribution's `accept()` after HTTP 2xx, before it consumes the SSE body. Acceptance appends `session-log-deepseek/delivery-accepted` with the uploaded `throughSeq`; the next request uploads that event as part of its new suffix. Transport and non-2xx failures append no acceptance record, so later requests resend the uncertain range. Concurrent deliveries may be accepted out of order; folding the maximum matching `throughSeq` prevents cursor regression. + +A crash after server acceptance but before the watermark reaches persistence can replay an accepted range after restart. This is the at-least-once failure direction: uncertainty creates duplicates, never a skipped sequence. The ordinary session checkpoint policy persists the watermark at the next semantic checkpoint; this plugin performs no independent I/O. + +Direct requests without a live Session omit `dsh_session_log`. Normal agent, compaction, and session-title calls carry their live Session id. + +## Model Experience + +### Session-log metadata + +#### What the model sees + +Nothing. `dsh_session_log` is a sibling of the DeepSeek request's model-input fields and is not inserted into `messages`, the system prompt, or tool schemas. + +#### Token effect + +Zero model-input tokens; the field only increases HTTP request bytes. + +#### KV Cache effect + +None; the model-visible request prefix remains unchanged. + +## Known Limitations and Deferred Work + +- **Crash-window duplicates** — a 2xx followed by process loss before the acceptance watermark persists causes conservative replay on resume. +- **No live Session means no field** — direct or stale-session calls have no canonical log to snapshot; explicit absence semantics remain deferred. +- **No independent request-size cap** — complete delivery is fail-closed; provider rejection leaves the cursor unchanged instead of truncating the log. diff --git a/packages/session/session-log-deepseek/README.zh.md b/packages/session/session-log-deepseek/README.zh.md new file mode 100644 index 0000000000..0411fcd298 --- /dev/null +++ b/packages/session/session-log-deepseek/README.zh.md @@ -0,0 +1,47 @@ +# @deepseek-ai/dsh-session-log-deepseek + +[English](README.md) | 中文 + +用于 DeepSeek 官方 LLM API 请求的增量权威会话日志上传。该函数插件注入 `ctx.sessions` 与 `ctx.deepseekLlmApiExtensions`,并拥有 `dsh_session_log` 请求字段以及用于派生接受水位的持久 `session-log-deepseek/delivery-accepted` 事件。 + +## 配置 + +| 配置键 | 默认值 | 含义 | +|---|---:|---| +| `enabled` | `false` | 注册 `dsh_session_log` 贡献。将其设为 `true` 可选择启用会话日志上传。 | + +随附 profile 会挂载该插件,让 overlay 可以启用它;默认配置不会注册请求字段,也不会追加接受水位。 + +## 请求字段 + +对于携带存活 `sessionId` 的请求,插件会折叠该确切会话身份的最大已接受水位,对 `Session.events` 取快照,并发送水位之后的连续后缀。进程内 fold 会让每条事件只被扫描一次并增量消费后续追加;重启与 HMR 会从持久日志重建它。版本 1 字段包含不可变的 `SessionHeader`、`afterSeq`、`throughSeq`,以及作为数组直接元素的该范围内每个完整权威 `SessionEvent`。每个水位都会记录已接受请求发送的会话 id,因此 fork 会话会忽略从父会话继承的水位。 + +## 接受与重试 + +DeepSeek 适配器会在 HTTP 2xx 后、消费 SSE(Server-Sent Events)正文前调用已准备贡献的 `accept()`。接受操作会追加 `session-log-deepseek/delivery-accepted` 及已上传的 `throughSeq`;下一次请求再把该事件作为新后缀的一部分上传。传输失败与非 2xx 失败不会追加接受记录,因此后续请求会重发不确定范围。并发交付可能乱序得到接受;折叠匹配记录中最大的 `throughSeq` 可以防止游标回退。 + +服务端接受后、持久化水位前发生崩溃,可能让恢复后的进程重放已经接受的范围。这是至少一次交付的失败方向:不确定性会制造重复,绝不会跳过序列。普通会话检查点策略会在下一个语义检查点持久化水位;本插件不执行独立 I/O。 + +缺少存活会话的直接请求会省略 `dsh_session_log`。普通 agent(智能体)、压缩(compaction)与会话标题调用都会携带存活会话 id。 + +## 模型体验 + +### 会话日志元数据 + +#### 模型看到的内容 + +无。`dsh_session_log` 是 DeepSeek 请求中模型输入字段的同级字段,不会插入 `messages`、系统提示词或工具 schema。 + +#### Token 影响 + +模型输入 token 为零;该字段只会增加 HTTP 请求字节数。 + +#### KV Cache 影响 + +无;模型可见请求前缀保持不变。 + +## 已知限制与暂缓事项 + +- **崩溃窗口重复**——2xx 后、接受水位持久化前进程丢失,会在恢复时触发保守重放。 +- **缺少存活会话就没有字段**——直接调用或陈旧会话调用没有可供快照的权威日志;显式缺失语义仍暂缓处理。 +- **没有独立请求大小上限**——完整交付会快速失败;提供方拒绝会保持游标不变,而非截断日志。 diff --git a/packages/session/session-log-deepseek/package.json b/packages/session/session-log-deepseek/package.json new file mode 100644 index 0000000000..901bfc6180 --- /dev/null +++ b/packages/session/session-log-deepseek/package.json @@ -0,0 +1,54 @@ +{ + "name": "@deepseek-ai/dsh-session-log-deepseek", + "description": "Incremental lossless session-log request extension for the official DeepSeek LLM API", + "version": "0.1.1-rc.2", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/session/session-log-deepseek" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./types": { + "types": "./lib/types/types.d.ts", + "default": "./lib/types/types.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.js", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "dependencies": { + "@deepseek-ai/schemastery": "workspace:^" + }, + "peerDependencies": { + "@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + }, + "devDependencies": { + "@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + } +} diff --git a/packages/session/session-log-deepseek/src/index.ts b/packages/session/session-log-deepseek/src/index.ts new file mode 100644 index 0000000000..b66c21706c --- /dev/null +++ b/packages/session/session-log-deepseek/src/index.ts @@ -0,0 +1,99 @@ +/** + * Incremental session-log contribution for official DeepSeek LLM API requests. + * Accepted sequence watermarks live in the canonical log, so restart recovery + * can conservatively resend uncertain tails without maintaining another store. + * @module @deepseek-ai/dsh-session-log-deepseek + */ + +import type { Context } from '@deepseek-ai/cordis' +import z from '@deepseek-ai/schemastery' +import type {} from '@deepseek-ai/dsh-deepseek-llm-api-extensions' +import { SessionId, type Session, type SessionEvent } from '@deepseek-ai/dsh-session' +import type { DeepSeekSessionLogExtension } from './types.ts' + +export type * from './types.ts' + +/** Cordis plugin name. */ +export const name = 'session-log-deepseek' +/** Services required to resolve sessions and contribute the provider request field. */ +export const inject = ['deepseekLlmApiExtensions', 'sessions'] + +/** Session-log request contribution configuration. */ +export interface Config { + /** Contribute `dsh_session_log` to official DeepSeek requests. Defaults to `false`. */ + enabled?: boolean +} + +/** Validated Session-log request contribution configuration. */ +export const Config: z = z.object({ + enabled: z.boolean().default(false), +}) + +interface AcceptanceFold { + readonly scannedEvents: number + readonly throughSeq: number +} + +const acceptanceFolds = new WeakMap() + +/** + * Highest confirmed sequence for this exact session identity. + * @param session - canonical log whose matching acceptance events are folded. + * @returns greatest accepted sequence, or `-1` before any accepted request. + */ +export function acceptedThrough(session: Session): number { + const previous = acceptanceFolds.get(session) + let throughSeq = previous?.throughSeq ?? -1 + const events = session.events + const start = previous?.scannedEvents ?? 0 + for (let index = start; index < events.length; index++) { + const event = events[index] as SessionEvent + if (event.type !== 'session-log-deepseek/delivery-accepted') continue + if (typeof event.data.sessionId !== 'string' || event.data.sessionId.length === 0 + || !Number.isSafeInteger(event.data.throughSeq) || event.data.throughSeq < 0 + || event.data.throughSeq >= event.seq) { + throw new Error(`session-log-deepseek: malformed acceptance watermark at seq ${event.seq}`) + } + if (event.data.sessionId !== session.id) continue + throughSeq = Math.max(throughSeq, event.data.throughSeq) + } + acceptanceFolds.set(session, { scannedEvents: events.length, throughSeq }) + return throughSeq +} + +/** + * Register the incremental `dsh_session_log` request contribution when enabled. + * @param ctx - plugin context carrying Sessions and the DeepSeek request-extension registry. + * @param config - validated opt-in configuration. + */ +export function apply(ctx: Context, config: Config): void { + if (config.enabled !== true) return + ctx.deepseekLlmApiExtensions.register('dsh_session_log', { + prepare: (request) => { + // TODO: Define an explicit wire result for direct or stale-session calls if they become a supported product path. + if (request.sessionId === undefined) return undefined + const session = ctx.sessions.get(SessionId(request.sessionId)) + if (session === undefined) return undefined + + const afterSeq = acceptedThrough(session) + const snapshot = session.events + const throughSeq = snapshot.length - 1 + if (throughSeq < 0) return undefined + const suffix = snapshot.slice(afterSeq + 1) + const value: DeepSeekSessionLogExtension = { + version: 1, + session: session.header, + afterSeq, + throughSeq, + events: suffix, + } + return { + value, + accept: () => { + session.append('session-log-deepseek/delivery-accepted', { sessionId: session.id, throughSeq }) + // TODO: Add an immediate lightweight checkpoint if duplicate replay after a 2xx crash window becomes unacceptable. + }, + } + }, + }) +} diff --git a/packages/session/session-log-deepseek/src/invariant.ts b/packages/session/session-log-deepseek/src/invariant.ts new file mode 100644 index 0000000000..e739126ab6 --- /dev/null +++ b/packages/session/session-log-deepseek/src/invariant.ts @@ -0,0 +1,58 @@ +/** Package-owned invariants for DeepSeek session-log acceptance watermarks. */ + +import type { Context } from '@deepseek-ai/cordis' +import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' +import type { InvariantFailure, InvariantInstaller } from '@deepseek-ai/dsh-invariants' +import type {} from './types.ts' + +const PACKAGE_NAME = '@deepseek-ai/dsh-session-log-deepseek' + +/** Cordis companion plugin name. */ +export const name = 'session-log-deepseek-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** Validate one acceptance watermark against its containing event and session. */ +function validateDeliveryAccepted(session: Session, event: SessionEvent<'session-log-deepseek/delivery-accepted'>, fail: InvariantFailure): void { + const { sessionId, throughSeq } = event.data + const inherited = session.header.parentSession !== undefined + && session.header.seedLength !== undefined + && event.seq < session.header.seedLength + if (sessionId !== session.id && !inherited) { + fail('a non-inherited session-log-deepseek/delivery-accepted event must name its containing session') + } + if (!Number.isSafeInteger(throughSeq) || throughSeq < 0 || throughSeq >= event.seq) { + fail(`session-log-deepseek/delivery-accepted throughSeq must identify an earlier event, got ${throughSeq} at seq ${event.seq}`) + } +} + +/** Validate acceptance watermarks already present in one Session. */ +function validateSession(session: Session, fail: InvariantFailure): void { + for (const event of session.events) { + if (event.type === 'session-log-deepseek/delivery-accepted') validateDeliveryAccepted(session, event, fail) + } +} + +/** Validate one live session-event dispatch. */ +function validateDispatched(args: unknown[], fail: InvariantFailure): void { + const [session, event] = args as [Session, SessionEvent] + if (event.type === 'session-log-deepseek/delivery-accepted') validateDeliveryAccepted(session, event, fail) +} + +/** Install validation for restored, newly created, and newly appended watermarks. */ +const install: InvariantInstaller = Object.assign((ctx: Context, fail: InvariantFailure) => { + const validateExisting = (session: Session): void => { validateSession(session, fail) } + ctx.sessions.list().forEach(validateExisting) + ctx.on('session/created', validateExisting, { global: true }) + ctx.on('internal/dispatch', (_mode, eventName, args) => { + if (eventName === 'session/event') validateDispatched(args, fail) + }, { global: true }) +}, { inject: ['sessions'] }) + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) diff --git a/packages/session/session-log-deepseek/src/types.ts b/packages/session/session-log-deepseek/src/types.ts new file mode 100644 index 0000000000..d3281bf327 --- /dev/null +++ b/packages/session/session-log-deepseek/src/types.ts @@ -0,0 +1,33 @@ +/** Wire types for lossless incremental DeepSeek session-log upload. */ + +import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' + +/** Versioned incremental session-log field carried by an official DeepSeek request. */ +export interface DeepSeekSessionLogExtension { + readonly version: 1 + readonly session: SessionHeader + /** Highest sequence durably recorded as accepted before this request, or `-1`. */ + readonly afterSeq: number + /** Highest sequence represented by {@link events}. */ + readonly throughSeq: number + /** Complete canonical event envelopes for every sequence from `afterSeq + 1` through `throughSeq`. */ + readonly events: readonly SessionEvent[] +} + +declare module '@deepseek-ai/dsh-deepseek-llm-api-extensions/types' { + interface DeepSeekLlmApiExtensionMap { + dsh_session_log: DeepSeekSessionLogExtension + } +} + +declare module '@deepseek-ai/dsh-session/types' { + interface SessionEventMap { + /** Records that the configured endpoint accepted one delivery through `throughSeq`. */ + 'session-log-deepseek/delivery-accepted': { + /** Session identity the accepted delivery carried; inherited fork markers retain the parent's id. */ + sessionId: import('@deepseek-ai/dsh-session/types').SessionId + /** Last canonical event included in the accepted request. */ + throughSeq: number + } + } +} diff --git a/packages/session/session-log-deepseek/tests/invariant.spec.ts b/packages/session/session-log-deepseek/tests/invariant.spec.ts new file mode 100644 index 0000000000..f39c4aec45 --- /dev/null +++ b/packages/session/session-log-deepseek/tests/invariant.spec.ts @@ -0,0 +1,95 @@ +import { afterEach, describe, expect, it } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import InvariantRegistry, { InvariantError } from '@deepseek-ai/dsh-invariants' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import * as SessionLogInvariant from '../src/invariant.ts' +import type {} from '../src/types.ts' + +const contexts: Context[] = [] + +afterEach(async () => { + await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose())) +}) + +async function setup(): Promise { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(SessionStore) + await ctx.plugin(InvariantRegistry, { enabled: true }) + await ctx.plugin(SessionLogInvariant) + return ctx +} + +describe('DeepSeek session-log acceptance invariant', () => { + it('accepts a watermark naming an earlier event in its containing Session', async () => { + const ctx = await setup() + const session = ctx.sessions.create(SessionId('valid')) + session.append('turn/start', { turn: 1 }) + expect(() => session.append('session-log-deepseek/delivery-accepted', { sessionId: session.id, throughSeq: 0 })) + .not.toThrow() + }) + + it('rejects a live watermark for another Session or a non-earlier sequence', async () => { + const ctx = await setup() + const wrongId = ctx.sessions.create(SessionId('wrong-id')) + wrongId.append('turn/start', { turn: 1 }) + expect(() => wrongId.append('session-log-deepseek/delivery-accepted', { + sessionId: SessionId('other'), + throughSeq: 0, + })).toThrow(expect.objectContaining>({ + code: 'INVARIANT', + packageName: '@deepseek-ai/dsh-session-log-deepseek', + })) + + const wrongSeq = ctx.sessions.create(SessionId('wrong-seq')) + wrongSeq.append('turn/start', { turn: 1 }) + expect(() => wrongSeq.append('session-log-deepseek/delivery-accepted', { + sessionId: wrongSeq.id, + throughSeq: 1, + })).toThrow(expect.objectContaining>({ + code: 'INVARIANT', + packageName: '@deepseek-ai/dsh-session-log-deepseek', + })) + }) + + it('validates existing history when the invariant loads after the Session', async () => { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(SessionStore) + await ctx.plugin(InvariantRegistry, { enabled: true }) + const id = SessionId('late-invalid') + ctx.sessions.create(id, { seed: [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'session-log-deepseek/delivery-accepted', seq: 1, time: 2, data: { sessionId: id, throughSeq: 1 } }, + ] }) + + let failure: unknown + try { + await ctx.plugin(SessionLogInvariant) + } catch (error) { + failure = error + } + expect(failure).toMatchObject>({ + code: 'INVARIANT', + packageName: '@deepseek-ai/dsh-session-log-deepseek', + }) + }) + + it('allows an inherited parent watermark inside a fork seed', async () => { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(SessionStore) + await ctx.plugin(InvariantRegistry, { enabled: true }) + const parentId = SessionId('fork-parent') + const childId = SessionId('fork-child') + ctx.sessions.create(childId, { + seed: [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'session-log-deepseek/delivery-accepted', seq: 1, time: 2, data: { sessionId: parentId, throughSeq: 0 } }, + ], + meta: { parentSession: parentId, seedLength: 2 }, + }) + + await expect(ctx.plugin(SessionLogInvariant)).resolves.toBeDefined() + }) +}) diff --git a/packages/session/session-log-deepseek/tests/upload.spec.ts b/packages/session/session-log-deepseek/tests/upload.spec.ts new file mode 100644 index 0000000000..d473cc7de9 --- /dev/null +++ b/packages/session/session-log-deepseek/tests/upload.spec.ts @@ -0,0 +1,181 @@ +import { afterEach, describe, expect, it } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import SessionStore, { Session, SessionId, type CreateSessionOptions, type SessionEvent } from '@deepseek-ai/dsh-session' +import DeepSeekLlmApiExtensionRegistry from '@deepseek-ai/dsh-deepseek-llm-api-extensions' +import * as SessionLogDeepSeek from '../src/index.ts' + +const contexts: Context[] = [] +const SIGNAL = new AbortController().signal + +afterEach(async () => { + await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose())) +}) + +async function harness(id: string, seed?: readonly SessionEvent[], meta?: CreateSessionOptions['meta']): Promise<{ + ctx: Context + session: Session + disposeUpload: () => Promise +}> { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(SessionStore) + await ctx.plugin(DeepSeekLlmApiExtensionRegistry) + const upload = ctx.plugin(SessionLogDeepSeek, { enabled: true }) + await upload + const options = seed === undefined + ? undefined + : { seed, ...meta === undefined ? {} : { meta } } + const session = ctx.sessions.create(SessionId(id), options) + return { ctx, session, disposeUpload: () => upload.dispose() } +} + +function body(text = 'x'.repeat(300)) { + return { messages: [{ role: 'user', content: text }] } +} + +describe('incremental DeepSeek session-log upload', () => { + it('does not contribute the session log under its default configuration', async () => { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(SessionStore) + await ctx.plugin(DeepSeekLlmApiExtensionRegistry) + await ctx.plugin(SessionLogDeepSeek) + const session = ctx.sessions.create(SessionId('default-off')) + session.append('turn/start', { turn: 1 }) + + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ + body: body(), signal: SIGNAL, sessionId: session.id, + }) + expect(prepared.fields).not.toHaveProperty('dsh_session_log') + }) + + it('uploads the full first prefix, records acceptance, then sends only the appended suffix', async () => { + const { ctx, session } = await harness('incremental') + session.append('turn/start', { turn: 1 }) + session.append('step/start', { turn: 1, step: 1 }) + + const first = await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id }) + const firstPayload = first.fields.dsh_session_log + expect(firstPayload).toMatchObject({ afterSeq: -1, throughSeq: 1 }) + expect(firstPayload?.events).toHaveLength(2) + await first.accept() + expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(1) + + session.append('step/end', { turn: 1, step: 1 }) + const second = await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id }) + expect(second.fields.dsh_session_log).toMatchObject({ afterSeq: 1, throughSeq: 3 }) + expect(second.fields.dsh_session_log?.events).toHaveLength(2) + expect(second.fields.dsh_session_log?.events[0]).toMatchObject({ + type: 'session-log-deepseek/delivery-accepted', + seq: 2, + }) + }) + + it('reconstructs a persisted cursor and ignores an inherited parent watermark in a fork', async () => { + const first = await harness('parent') + first.session.append('turn/start', { turn: 1 }) + const prepared = await first.ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: first.session.id }) + await prepared.accept() + const seed = first.session.events + + const resumed = await harness('parent', seed) + expect(SessionLogDeepSeek.acceptedThrough(resumed.session)).toBe(0) + const resumedPayload = await resumed.ctx.deepseekLlmApiExtensions.prepare({ + body: body(), signal: SIGNAL, sessionId: resumed.session.id, + }) + expect(resumedPayload.fields.dsh_session_log?.afterSeq).toBe(0) + + const fork = await harness('child', seed, { parentSession: first.session.id, seedLength: seed.length }) + expect(SessionLogDeepSeek.acceptedThrough(fork.session)).toBe(-1) + const forkPayload = await fork.ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: fork.session.id }) + expect(forkPayload.fields.dsh_session_log).toMatchObject({ afterSeq: -1, throughSeq: fork.session.seq - 1 }) + }) + + it('takes the maximum watermark when concurrent acceptances settle out of order', async () => { + const { ctx, session } = await harness('concurrent') + session.append('turn/start', { turn: 1 }) + const earlier = await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id }) + session.append('step/start', { turn: 1, step: 1 }) + const later = await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id }) + + await later.accept() + await earlier.accept() + expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(1) + }) + + it('folds only events appended after the cached acceptance scan', () => { + const id = SessionId('incremental-fold') + const events: SessionEvent[] = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'session-log-deepseek/delivery-accepted', seq: 1, time: 2, data: { sessionId: id, throughSeq: 0 } }, + ] + let reads = 0 + const observed = new Proxy(events, { + get(target, property, receiver) { + if (typeof property === 'string' && /^\d+$/.test(property)) reads++ + return Reflect.get(target, property, receiver) as unknown + }, + }) + const session = { id, get events() { return observed } } as unknown as Session + + expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(0) + expect(reads).toBe(2) + reads = 0 + expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(0) + expect(reads).toBe(0) + + events.push( + { type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } }, + { type: 'session-log-deepseek/delivery-accepted', seq: 3, time: 4, data: { sessionId: id, throughSeq: 2 } }, + ) + expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(2) + expect(reads).toBe(2) + }) + + it('omits the field for direct or stale requests and uploads the prior acceptance marker next', async () => { + const { ctx, session } = await harness('edges') + await expect(ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL })) + .resolves.toMatchObject({ fields: {} }) + await expect(ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: 'missing' })) + .resolves.toMatchObject({ fields: {} }) + await expect(ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id })) + .resolves.toMatchObject({ fields: {} }) + session.append('turn/start', { turn: 1 }) + const first = await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id }) + await first.accept() + const current = await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id }) + expect(current.fields.dsh_session_log).toMatchObject({ + afterSeq: 0, + throughSeq: 1, + events: [{ type: 'session-log-deepseek/delivery-accepted' }], + }) + }) + + it('contributes complete events without reading request messages', async () => { + const { ctx, session } = await harness('direct-events') + session.append('turn/start', { turn: 1 }) + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: {}, signal: SIGNAL, sessionId: session.id }) + expect(prepared.fields.dsh_session_log?.events).toEqual(session.events) + }) + + it('fails closed on a malformed persisted acceptance watermark', async () => { + const malformed = [{ + type: 'session-log-deepseek/delivery-accepted', + seq: 0, + time: 1, + data: { sessionId: 'malformed', throughSeq: 0 }, + }] as unknown as SessionEvent[] + const session = Session.create(SessionId('malformed'), malformed) + expect(() => SessionLogDeepSeek.acceptedThrough(session)).toThrow(/malformed acceptance watermark/) + }) + + it('withdraws its request field when the contributing plugin reloads', async () => { + const { ctx, session, disposeUpload } = await harness('hmr') + session.append('turn/start', { turn: 1 }) + expect((await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id })).fields) + .toHaveProperty('dsh_session_log') + await disposeUpload() + expect((await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id })).fields) + .not.toHaveProperty('dsh_session_log') + }) +}) diff --git a/packages/session/session-log-deepseek/tsconfig.json b/packages/session/session-log-deepseek/tsconfig.json new file mode 100644 index 0000000000..eb85ebeddf --- /dev/null +++ b/packages/session/session-log-deepseek/tsconfig.json @@ -0,0 +1,30 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../core/session" + }, + { + "path": "../../llm/deepseek-llm-api-extensions" + }, + { + "path": "../../runtime-diagnostics/invariants" + } + ] +} diff --git a/packages/session/session-persistence-jsonl/README.i18n.yaml b/packages/session/session-persistence-jsonl/README.i18n.yaml index 2184f491c1..099e407149 100644 --- a/packages/session/session-persistence-jsonl/README.i18n.yaml +++ b/packages/session/session-persistence-jsonl/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-persistence-jsonl/README.md -README.md: 4cff3215cdb083d2fdb7c4a8f1b60e8c4028ba84 -README.zh.md: 38b41cb9f7fa0f144923feced06152a185254c0a +README.md: 0301691acbe42c7973274e717ee9ae6f405ebea1 +README.zh.md: c05c380166b65a9826b6cb7f31729a51628ab49b diff --git a/packages/session/session-persistence-jsonl/README.md b/packages/session/session-persistence-jsonl/README.md index 4cff3215cd..0301691acb 100644 --- a/packages/session/session-persistence-jsonl/README.md +++ b/packages/session/session-persistence-jsonl/README.md @@ -40,7 +40,7 @@ A root belongs to one encoding. Startup discovery and targeted lookup reject the ## Durability and crash semantics - **Bound storage identity.** Lookup requires one matching session directory across the readable project directories, then verifies that the header id equals the requested id and that the header's id/cwd derive the selected transcript path. Listing applies the same path check and rejects duplicate ids. Identity failures occur before repair or append. -- **Lazy materialization.** `create(meta)` writes nothing; on the first `append`, the backend writes and `fsync`s the encoded header and first batch in a temporary file. POSIX publishes it without overwrite via a hard link and `fsync`s the parent directory. Windows publishes it without overwrite via `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` and creates missing directories through the same write-through pattern. A created-but-never-appended session leaves nothing on disk and is absent from `list`. +- **Lazy materialization.** `create(meta)` writes nothing; on the first `append`, the backend writes and `fsync`s the encoded header and first batch in a temporary file. POSIX publishes it without overwrite via a hard link and `fsync`s the parent directory. Windows publishes it without overwrite via `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` and creates missing directories through the same write-through pattern. A created-but-never-appended session leaves nothing on disk and is absent from `list` unless a lifecycle consumer explicitly calls `ensureMaterialized`, which publishes one header frame without adding an event. - **Append-only.** Flushed events are never rewritten. Subsequent raw batches append lines; compressed batches append one frame. Both paths `fsync`, and a caught write or sync failure rolls the file back to its prior byte length. - **Crash recovery — preserve valid tail work.** `load` validates every complete compressed frame and scans their decompressed JSONL. If the last frame is structurally incomplete, the reader keeps its complete decoded records, truncates from that frame's start, and re-encodes those records with the synthetic tool, step, and turn closers required by the shared [persistence contract](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md). Raw mode truncates from its first incomplete line. An existing compressed artifact with no complete header frame, a checksum/decompression failure in a complete frame, or a defect at or before the last committed `turn/end` is corruption and rejects. - **Non-mutating inspection.** `inspect()` returns an immutable balanced logical view and may synthesize recovery closers in memory, without truncating an incomplete tail or changing the lightweight revision. diff --git a/packages/session/session-persistence-jsonl/README.zh.md b/packages/session/session-persistence-jsonl/README.zh.md index 38b41cb9f7..c05c380166 100644 --- a/packages/session/session-persistence-jsonl/README.zh.md +++ b/packages/session/session-persistence-jsonl/README.zh.md @@ -16,7 +16,7 @@ JSONL 持久会话存储后端:`SessionPersistence` 的一个具体实现(`d - 第一个逻辑行是不可变的 `SessionHeader`,标记为 `{ type: 'session', version, id, cwd?, createdAt, parentSession?, seedLength?, origin?, delegationDepth, agentPreset? }`。`delegationDepth` 在磁盘上必需,顶层会话为 `0`;缺失或无效值会拒绝日志。`agentPreset` 必须持久化,因为它决定了被恢复会话的工具与提示词——恢复成另一套组装,就会回放模型已无法据以行动的历史。后续每个逻辑行是一条存储记录;`assistant/chunk` 事件绝不丢弃,且 `seq` 在解码日志中保持连续(`events[i].seq === i`)。 - 存储记录是原样 `SessionEvent` JSON,或在 `packChunks` 已启用且连续段符合条件时写入的**打包分片行**(`text-chunks` / `reasoning-chunks` / `tool-call-chunks`;像 header 的 `session` 一样不带斜杠,因此行 tag 不会与事件类型混淆):一行保存至少 3 个连续同 block `assistant/chunk` delta 事件,`seq0`/`time0` 和各成员的 `dt` 间隔精确重建每个成员的 `seq`/`time`。无损 codec 位于 `@deepseek-ai/dsh-session`(`packChunkRuns`/`decodeStorageRecord`),并使用精确形态 allowlist:任何未识别内容原样存储。读取与布局无关:`load` 始终解码行,因此打包、非打包和混合文件加载结果一致。 -- 项目目录保留规范化 cwd 的可读形式,便于导航,并限制在文件系统组件上限内。分隔符替换和截断刻意有损,因此规范化相同的 cwd 字符串共享项目目录;会话 id 仍选择不同会话目录。在不区分大小写的文件系统上,只有文件系统规范化将两种写法解析到同一 transcript(文本记录)时,身份验证才接受备选路径写法。配置根仍由部署控制:可以是项目本地、共享、临时或集中式。[项目会话目录决策](../../../.agents/notes/implemented/architecture/2026-07-24-project-session-directories.md) 记录这项取舍。 +- 项目目录保留规范化 cwd 的可读形式,便于导航,并限制在文件系统组件上限内。分隔符替换和截断刻意有损,因此规范化相同的 cwd 字符串共享项目目录;会话 id 仍选择不同会话目录。在不区分大小写的文件系统上,只有文件系统规范化将两种写法解析到同一 transcript(文本记录)时,身份验证才接受备选路径写法。配置根仍由部署控制:可以是项目本地、共享、临时或集中式。[项目会话目录决策](../../../.agents/notes/implemented/architecture/2026-07-24-project-session-directories.zh.md) 记录这项取舍。 - 会话 id 是未验证的带品牌类型的字符串,因此在使用前单射转义为一个安全路径段(无遍历、无冲突)。结果目录保留给其他会话自有产物;发现只读取固定 transcript 文件名。 ## 配置 @@ -33,16 +33,16 @@ JSONL 持久会话存储后端:`SessionPersistence` 的一个具体实现(`d ## 物理编码 -默认产物是独立 [Zstandard frame](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md) 的标准拼接:一个仅包含 header 行的带 checksum frame,后跟每个持久 append 批次一个带 checksum frame。后端使用 Node 内置 Zstandard API 和默认压缩级别,不提供级别开关。列表只读取并验证 header frame。`compression: 'none'` 在原始表示中保留相同逻辑行。 +默认产物是独立 [Zstandard frame](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md) 的标准拼接:一个仅包含 header 行的带 checksum frame,后跟每个持久 append 批次一个带 checksum frame。后端使用 Node 内置 Zstandard API 和默认压缩级别,不提供级别开关。列表只读取并验证 header frame。`compression: 'none'` 在原始表示中保留相同逻辑行。 一个根只属于一种编码。启动发现和定向查找会拒绝相反 suffix,错误会命名不兼容产物,并指示调用方选择匹配 mode 或独立根。平铺 `/.jsonl*` 产物也会被拒绝,而不是忽略。不提供迁移、混合根回退或双写。 ## 持久性与崩溃语义 - **绑定存储身份。** 查找要求可读项目目录中只有一个匹配会话目录,然后验证 header id 等于请求 id,且 header id/cwd 派生所选 transcript 路径。列表应用同一路径检查,并拒绝重复 id。身份失败发生在修复或 append 前。 -- **延迟实体化。**`create(meta)` 不写入;第一次 `append` 将编码 header 和第一批写入临时文件并执行 `fsync`。POSIX 通过硬链接无覆盖发布,并对父目录 `fsync`。Windows 通过 `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` 无覆盖发布,并通过同一 write-through pattern 创建缺失目录。已创建但从未 append 的会话不留下磁盘内容,不在 `list` 中。 +- **延迟实体化。**`create(meta)` 不写入;第一次 `append` 将编码 header 和第一批写入临时文件并执行 `fsync`。POSIX 通过硬链接无覆盖发布,并对父目录 `fsync`。Windows 通过 `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` 无覆盖发布,并通过同一 write-through pattern 创建缺失目录。已创建但从未 append 的会话不留下磁盘内容,不在 `list` 中;生命周期 consumer 显式调用 `ensureMaterialized` 时除外,此时会发布一个 header frame,但不添加事件。 - **仅追加。** 已 flush 事件绝不重写。后续原始批次 append 行;压缩批次 append 一个 frame。两条路径都执行 `fsync`,并在捕获到写入或同步失败时回滚到之前字节长度。 -- **崩溃恢复:保留有效尾部工作。**`load` 验证每个完整压缩 frame,并扫描解压 JSONL。最后 frame 结构不完整时,读取器保留其完整解码记录,从 frame 开头截断,并使用共享[持久化约定](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md) 需要的合成工具、步骤和轮次 closer 重新编码这些记录。原始 mode 从第一个不完整行截断。已经存在却没有完整 header frame 的压缩工件、完整 frame 中的 checksum/解压失败,或位于最后已提交的 `turn/end` 处或之前的缺陷都属于损坏,会被拒绝。 +- **崩溃恢复:保留有效尾部工作。**`load` 验证每个完整压缩 frame,并扫描解压 JSONL。最后 frame 结构不完整时,读取器保留其完整解码记录,从 frame 开头截断,并使用共享[持久化约定](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md) 需要的合成工具、步骤和轮次 closer 重新编码这些记录。原始 mode 从第一个不完整行截断。已经存在却没有完整 header frame 的压缩工件、完整 frame 中的 checksum/解压失败,或位于最后已提交的 `turn/end` 处或之前的缺陷都属于损坏,会被拒绝。 - **非修改式检查。**`inspect()` 返回不可变、平衡的逻辑视图,并可在内存中合成恢复 closer,但不会截断不完整尾部或更改轻量修订。 - **连续 seq。**`append` 拒绝第一个 `seq` 不继续已存储日志的批次,并拒绝无法 JSON 序列化的 `event.data`,同时命名违规事件类型。 - **轻量修订。**`listSnapshots(signal?)` 使用 device、inode、size 和纳秒时间戳标识日志,避免解析完整日志;该标识会在 append、修复、替换或存储变更后改变。完整前缀读取要求读取字节前后的身份一致,`readStoredRevision()` 使用同一身份校验保留的 preparation,而不加载日志。快照列表通过产物发现原样转发该信号,并在每个 `stat` 前后检查取消;由于文件系统 `stat` 不可中断,取消会等待活动调用完成,然后在不启动另一次调用的情况下拒绝。 diff --git a/packages/session/session-persistence-jsonl/package.json b/packages/session/session-persistence-jsonl/package.json index cd48bbae0b..190de373b5 100644 --- a/packages/session/session-persistence-jsonl/package.json +++ b/packages/session/session-persistence-jsonl/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-persistence-jsonl", "description": "JSONL durable session persistence backend for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-persistence-jsonl/src/format.ts b/packages/session/session-persistence-jsonl/src/format.ts index 2923b9e09b..8092991eef 100644 --- a/packages/session/session-persistence-jsonl/src/format.ts +++ b/packages/session/session-persistence-jsonl/src/format.ts @@ -233,8 +233,8 @@ interface SessionLogScan { /** * Refuse a header carrying a format version this build does not read BEFORE * validating the current header shape or decoding any event row: a future - * format need not satisfy today's structural checks at all, and its user must - * see "upgrade the harness", never "corrupt session log". + * format need not satisfy this build's structural checks at all, and its user + * must see "upgrade the harness", never "corrupt session log". * @param parsed - the JSON-parsed first line of a session artifact. */ function refuseForeignFormatVersion(parsed: unknown): void { diff --git a/packages/session/session-persistence-jsonl/src/index.ts b/packages/session/session-persistence-jsonl/src/index.ts index 5113746fec..2b0d5878b1 100644 --- a/packages/session/session-persistence-jsonl/src/index.ts +++ b/packages/session/session-persistence-jsonl/src/index.ts @@ -21,7 +21,7 @@ import { type SessionInspection, type SessionPersistenceRevision as PersistenceRevision, type SessionRawArtifact, type StoredPrefix, } from '@deepseek-ai/dsh-session-persistence' -import type { SessionEvent, SessionId, SessionHeader, SessionPreparation } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionId, SessionHeader, SessionPreparation } from '@deepseek-ai/dsh-session' import { encodeSegment, eventLines, logPath, logSuffix, parseHeaderMeta, projectDir, scanLog, sessionDir, SessionLogScanner, toHeaderLine, @@ -177,6 +177,10 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi return this.coordinator.create(meta) } + override ensureMaterialized(session: Session): Promise { + return this.coordinator.ensureMaterialized(session) + } + append(id: SessionId, events: readonly SessionEvent[]): Promise { return this.coordinator.append(id, events) } @@ -428,6 +432,11 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi } } + /** Materialize a header-only JSONL artifact for an explicitly durable empty session. */ + async materializeHeader(meta: SessionHeader): Promise { + await this.materialize(meta, []) + } + /** * Make a crash repair durable: truncate a torn tail, restore complete events * decoded from it, then append synthetic closers. Two fsync'd steps — the seam @@ -559,7 +568,7 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi // parent directory's metadata is synced. await this.syncDirPosix(dir) // Best-effort temp cleanup: the log is already published and durable, so a - // failure to remove the (now-redundant) temp hard link must NOT reject the + // failure to remove the redundant temp hard link must NOT reject the // append. Swallow only the rm failure; nothing else of consequence runs here. try { await rm(tmp, { force: true }) @@ -618,6 +627,9 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi /** Encode the header and first batch without combining their frame boundaries. */ private async encodeMaterialization(meta: SessionHeader, events: readonly SessionEvent[]): Promise { const header = JSON.stringify(toHeaderLine(meta)) + '\n' + if (events.length === 0) { + return this.compression === 'none' ? header : compressZstdFrame(header) + } const body = eventLines(events, this.packChunks) + '\n' if (this.compression === 'none') return header + body const headerFrame = await compressZstdFrame(header) diff --git a/packages/session/session-persistence-jsonl/tests/jsonl.spec.ts b/packages/session/session-persistence-jsonl/tests/jsonl.spec.ts index 70ae6d1ac5..eea36d5089 100644 --- a/packages/session/session-persistence-jsonl/tests/jsonl.spec.ts +++ b/packages/session/session-persistence-jsonl/tests/jsonl.spec.ts @@ -99,7 +99,6 @@ function appendClosedTurn(session: Session): void { session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) } -// Run the shared backend contract against the real JSONL backend. runPersistenceContract('jsonl-none', async () => { const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-')) const ctx = new Context() @@ -192,7 +191,7 @@ describe('JsonlSessionPersistence: format helpers', () => { const ctx = new Context() await ctx.plugin(SessionStore) const fiber = await ctx.plugin(JsonlSessionPersistence, { root: absoluteRoot, compression: 'none' }) - // A future format need not satisfy today's header shape at all (no + // A future format need not satisfy this build's header shape at all (no // createdAt, unknown fields): the version must be refused before shape // validation, so the user sees the upgrade direction. const id = SessionId('future-shape') @@ -287,6 +286,27 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { expect((await ctx.sessionPersistence.list()).map(h => h.id)).toContain(m.id) }) + it('materializes an explicitly durable empty live session without an event row', async () => { + const id = SessionId('durable-empty') + const session = ctx.sessions.create(id, { meta: { cwd: '/work' } }) + + await ctx.sessionPersistence.ensureMaterialized(session) + + expect(await readFile(rawLogPath(root, '/work', id), 'utf8')).toBe(`${JSON.stringify(toHeaderLine(session.header))}\n`) + await expect(ctx.sessionPersistence.load(id)).resolves.toEqual({ meta: session.header, events: [] }) + }) + + it('delegates direct preparation through the JSONL provider', async () => { + const m = meta('direct-prepare', '/work') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + + const preparation = await ctx.sessionPersistence.prepare(m.id) + + expect(preparation.session.header).toMatchObject(m) + preparation[Symbol.dispose]() + }) + it('readRaw returns the stored artifact text verbatim with its original filename', async () => { const m = meta('raw-read', '/work') await ctx.sessionPersistence.create(m) diff --git a/packages/session/session-persistence-jsonl/tests/zstd.spec.ts b/packages/session/session-persistence-jsonl/tests/zstd.spec.ts index b9cced0087..7741a8751d 100644 --- a/packages/session/session-persistence-jsonl/tests/zstd.spec.ts +++ b/packages/session/session-persistence-jsonl/tests/zstd.spec.ts @@ -332,6 +332,19 @@ describe('Zstandard frame structure', () => { }) describe('JsonlSessionPersistence: default Zstandard encoding', () => { + it('materializes an explicitly durable empty session as one header frame', async () => { + const root = await freshRoot() + const ctx = await mount(root) + const session = ctx.sessions.create(SessionId('empty-zstd'), { meta: { cwd: '/work' } }) + + await ctx.sessionPersistence.ensureMaterialized(session) + + const buffer = await readFile(logPath(root, '/work', session.id, 'zstd')) + expect(scanZstdFrames(buffer).frames).toHaveLength(1) + expect((await decodeCompleteFrames(buffer)).toString()).toBe(`${JSON.stringify(toHeaderLine(session.header))}\n`) + await expect(ctx.sessionPersistence.load(session.id)).resolves.toEqual({ meta: session.header, events: [] }) + }) + it('writes .jsonl.zstd by default with one header frame and one first-batch frame', async () => { const root = await freshRoot() const ctx = await mount(root) diff --git a/packages/session/session-persistence-sqlite/README.i18n.yaml b/packages/session/session-persistence-sqlite/README.i18n.yaml index 8fd46fa101..db1e569112 100644 --- a/packages/session/session-persistence-sqlite/README.i18n.yaml +++ b/packages/session/session-persistence-sqlite/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-persistence-sqlite/README.md -README.md: ec42419a132a26c1f23ab99ab3da1db97a5483b0 -README.zh.md: 67c6bcfbec92f8f5b91150fb108c65a287906db9 +README.md: e105b01361f8129350bf820d50ef0acd1f8ee410 +README.zh.md: 579eb43e430668a1dbb5b9c2473e2fd7dd40a567 diff --git a/packages/session/session-persistence-sqlite/README.md b/packages/session/session-persistence-sqlite/README.md index ec42419a13..e105b01361 100644 --- a/packages/session/session-persistence-sqlite/README.md +++ b/packages/session/session-persistence-sqlite/README.md @@ -2,40 +2,39 @@ English | [中文](README.zh.md) -A SQLite durable session-persistence backend — a second `SessionPersistence` provider ([session persistence](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md)) satisfying the same contract as `dsh-session-persistence-jsonl` (append-only, contiguous-seq, lazy materialization, interrupted-turn close on load), expressed over `node:sqlite` rows instead of file bytes. +An opt-in SQLite `SessionPersistence` provider. It stores eligible `assistant/chunk` runs in packed physical rows, selectively Zstandard-compresses large payloads, and delta-encodes provenance sequences while restoring the exact logical `SessionEvent[]`. No shipped composition selects it; deployments mount this package explicitly and provide its database path. -`locate(meta)` returns `undefined`: all sessions share one database, so there is no honest independent per-session transcript path. +`locate(meta)` returns `undefined` because every session shares one database. The provider exposes no per-session raw artifact. ## Storage model -Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data, source_event_seqs, surface_op)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). The two `TEXT` columns `source_event_seqs` and `surface_op` are nullable; they store the event's optional surface-metadata fields (see [session surface](../../../.agents/notes/implemented/architecture/2026-06-18-session-surface.md)). Out-of-log metadata (`SessionHeader`), a per-materialization incarnation id, and a monotonic per-log revision live in a `sessions` row; `createdAt` is a non-negative safe integer stored in a strict `INTEGER` column. A singleton state row carries the immutable store id. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`list` reports exactly the sessions that have a row). +Schema 17 keeps ordinary ROWID tables and the composite `events(session_id, seq)` primary-key index. Scalar rows store one logical event. Packed rows use `text-chunks`, `reasoning-chunks`, or `tool-call-chunks` as the physical `type`; `seq` and `time` identify the first represented event, and `data` holds the shared packed-chunk payload. Packed rows set `ignorable=0` as a physical discriminator and leave `source_event_seqs` and `surface_op` as `NULL`; scalar rows use `ignorable=1` only for logical ignorable events and `NULL` otherwise. A future ignorable logical event may therefore reuse a storage-tag name without being decoded as a packed row. These tags are storage records, not `SessionEventMap` members. -The repository's Node range supports unflagged `node:sqlite`. The database enables foreign keys and uses the configured journal mode (`wal` by default; use a rollback mode where WAL shared-memory files are unsuitable). `PRAGMA application_id` identifies the canonical persistence database, and `PRAGMA user_version` stores its layout version. A fresh database must have no application identity or user-defined schema objects; initialization creates every table and stamps both pragmas in one transaction. Non-pristine unversioned databases, foreign application identities, and every non-current version reject before journal-mode mutation because this unreleased format has no migrations. +Schema 17 owns its codec locally rather than importing another persistence format's mutable implementation. Only exact, consecutive same-block text, reasoning, or tool-call delta forms pack. Unknown fields, surface metadata, sequence gaps, incompatible block/call identity, and unsafe timestamps remain scalar. A packed row represents at most 1,024 events and at most 1 MiB of uncompressed UTF-8 `data`; longer runs are partitioned without changing logical events. Reads reconstruct every original sequence number, timestamp, token boundary, argument fragment, and payload before returning data to the persistence coordinator. -On filesystems with POSIX modes, the backend requests mode `0700` for missing directories and exclusively creates a missing database with mode `0600` before SQLite opens it; the process umask may further restrict both. New WAL, shared-memory, and persistent rollback-journal sidecars receive the database's resulting owner-only mode. Existing directories, database files, and sidecars keep their modes; filesystem setup errors other than an existing database fail initialization. These defaults prevent incidental exposure through a permissive process umask, but do not protect database confidentiality or integrity when another principal can replace the database entry in its parent directory. +Serialized `data` smaller than 4 KiB stays as SQLite `TEXT`. At or above that threshold, the writer uses Zstandard level 3 and stores a `BLOB` only when the frame is smaller than the original text; the reader decompresses it before UTF-8 validation and JSON parsing. `source_event_seqs` remains the complete ordered provenance array. Its first sequence is an unsigned varint and each subsequent sequence is a signed delta encoded with ZigZag varints, stored as a `BLOB`; no source is omitted or converted to a range. -## Contract semantics over rows +Each append holds `BEGIN IMMEDIATE`, validates the bounded physical tail, packs only the new durable batch, inserts those records, and increments the session revision once. Normal appends never delete or replace an earlier event row. The default 200 ms write-behind window therefore compresses high-frequency streams while the physical write volume stays proportional to newly durable batches rather than repeatedly rewriting a growing packed value. A storage-level logical-tail check rejects a stale writer before mutation. -- **Append = a transaction.** `append` runs `BEGIN`/`COMMIT` around the batch: it materializes the `sessions` row (if still lazy) and INSERTs every event, asserting the contiguous-seq contract first (the first event's `seq` must equal the stored next-seq). A mid-batch failure (a UNIQUE violation on a duplicated seq) rolls back entirely, so the stored log and the in-memory cursor stay consistent. (`load()` already balanced the stored log, so `append` never has to repair a crash tail.) -- **Lazy materialization.** `create()` records intent in memory only — no row is written until the first `append`. A created-but-never-appended session has no `sessions` row, so it is absent from `list()` (which reports exactly the sessions that have a row). -- **Interrupted-turn close on load.** `load()` implements the shared [crash-recovery contract](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md): preserve the valid interrupted turn, append its synthetic closing events in one transaction, and remove only a torn tail row. Committed parse errors or sequence gaps make the session unloadable. Because recovery mutates stored rows, the next append starts from a balanced log and accurate cursor. -- **Non-mutating inspection.** `inspect()` returns an immutable balanced logical view and may synthesize recovery closers in memory, without deleting a torn tail row, appending recovery rows, or changing the lightweight revision. -- **Lightweight revisions.** `listSnapshots(signal?)` combines the immutable store and database-file identity, a per-materialization incarnation id, and a per-session counter incremented in each mutating transaction. A full-prefix read captures that revision and its event rows in one read transaction, while `readStoredRevision()` queries only the session row to validate retained preparations. This keeps unchanged observations stable without parsing event rows and distinguishes independent stores and recreated same-id logs. It checks cancellation before and after shared readiness and the synchronous metadata query; the query itself is non-preemptible. +Full reads scan physical rows in first-logical-sequence order. A reverse pass finds the last valid `turn/end` without retaining decoded copies of every physical row; the forward pass decodes and validates one physical row at a time into the returned logical event array. `readFrom(id, fromSeq)` examines packed predecessors only within the maximum row span and anchors the suffix at the earliest one that may contain `fromSeq`; this includes an event range that starts inside a packed row, detects overlapping physical corruption, and does not parse unrelated earlier scalar rows. A malformed packed row is all-or-nothing: committed corruption rejects, while a torn final row is deleted from its physical base during mutating recovery. Repair re-reads the tail under the write lock and rejects a stale marker before deleting anything. Packed `data` that exceeds the schema byte limit rejects before JSON parsing. + +## Schema compatibility + +A pristine database initializes directly at schema 17. Older schemas, foreign application identities, non-pristine unversioned databases, and incompatible schema objects reject; this pre-release provider supplies no migration. Every statement and fixed pragma lives in a packaged `.sql` resource; values use SQLite parameters and runtime code never assembles query text. ## Configuration (schemastery) ```ts interface Config { - path: string // SQLite database file path, or ':memory:' for an in-process DB - journalMode?: 'wal' | 'delete' | 'truncate' | 'persist' // journal_mode pragma; default 'wal' - preparedSessionCacheSize?: number // positive integer; default 5 - writeBatchMaxDelayMs?: number // positive integer; default 200; maximum 2_147_483_647 + path: string + journalMode?: 'wal' | 'delete' | 'truncate' | 'persist' + busyTimeoutMs?: number + preparedSessionCacheSize?: number + writeBatchMaxDelayMs?: number } ``` -## Write path - -Like the JSONL backend, the plugin copies each frozen `session/event` into one controller per live session. The first pending event starts the configured fixed batching window, and later events join without resetting it. Expiry starts one transaction; events admitted during that write form a separately bounded follow-up batch. `session/flush` cancels the wait and drains current and pending batches. The controller persists a fork's seed once, keeps a write cursor so resume never re-appends stored events, and seeds live sessions on apply because HMR does not replay `session/created`. Dispose drains every retained controller before closing the database. Every event remains a separate SQLite row; batching only groups more INSERTs into one transaction and revision increment. +`journalMode` defaults to `wal`, `busyTimeoutMs` defaults to `5,000`, `preparedSessionCacheSize` defaults to `5`, and `writeBatchMaxDelayMs` defaults to `200`. The timeout bounds each synchronous SQLite lock wait. Because SQLite may return `SQLITE_BUSY` immediately while changing journal mode, cold open yields between attempts and starts no further attempt after an open-relative retry cutoff. An in-progress synchronous SQLite call may finish after that cutoff. The provider disables trusted schemas and memory-mapped I/O on every connection, then reads both settings back. The selected journal mode is also read back and must match; in-memory databases explicitly accept SQLite's `memory` result. After selecting the journal, the provider pins `synchronous=FULL` and verifies it so SQLite build defaults cannot weaken committed-append durability. On POSIX, the database parent and file must be owned by the current user, the parent must not be group/world-writable, and the file must have no group/world permissions. Symbolic links and non-regular files reject. Windows also rejects symbolic links and non-regular files, but deployments remain responsible for restricting the directory and file ACLs to the harness user. Path and ownership failures reject plugin initialization. Node SQLite loads lazily on the first persistence operation; the import suppresses only Node 22's exact SQLite `ExperimentalWarning`. Store-identity and schema failures reject that operation before data is exposed or mutated. `ensureMaterialized` inserts a metadata row with zero event rows, while ordinary `create` remains lazy until the first append. ## Model Experience @@ -43,20 +42,22 @@ Like the JSONL backend, the plugin copies each frozen `session/event` into one c #### What the model sees -SQLite storage contributes no live prompt or schema. Loading restores the same surface history as JSONL and preserves prior headers for reconstruction; the new loop composes its current envelope. Recovery balances an assistant request without a durable call with `TOOL_NOT_STARTED`; a durable call without a result becomes `TOOL_OUTCOME_UNKNOWN`, which tells the model to retry only read-only or idempotent work and to verify possible side effects or ask the user. Row metadata and raw chunks are not messages. +Nothing specific to SQLite. Resume restores the same logical events and derived messages as JSONL; physical packed tags never reach prompts, tools, replay, or live `session/event` delivery. #### Token effect -Zero live-request tokens. Resume restores retained history and pays the current envelope, plus the quoted repair result for each interrupted call. +Zero live-request tokens. Resume pays only for the retained logical history and current request envelope. #### KV Cache effect -SQLite storage does not mutate live request prefixes. A resumed loop can reuse provider cache only when its reconstructed history, current envelope, and model route match; crash-repair results append. +Physical packing does not mutate request prefixes. Provider cache reuse depends on the reconstructed history, current envelope, and model route exactly as with other persistence backends. ## Known Limitations and Deferred Work -- **`DatabaseSync` is synchronous** — every append transaction blocks the event loop for its duration; acceptable for local stores, a throughput ceiling for busy multi-session servers. -- **Write contention has no wait or retry policy** — the backend sets no busy timeout and retries no locked-database error, so another connection holding a write transaction makes the operation reject immediately. -- **Only a pristine new database or the current owned `SCHEMA_VERSION` opens** — unversioned schema objects, foreign application identities, and every other schema version are rejected rather than migrated (unreleased software; no persisted user data to preserve). -- **Nothing deletes stored sessions** — rows accumulate until removed externally (the seam has no deletion API; `ON DELETE CASCADE` is wired for such out-of-band cleanup). -- **TODO:** this backend talks to `node:sqlite` directly. If a cordis database service (`cordis/db` / a `@cordisjs` SQL driver plugin) is adopted, route through that instead of holding a raw `DatabaseSync` here — the contract surface (`SessionPersistence`) would not change, only the storage driver. +- **Interim SQLite-specific design** — This efficiency-focused implementation is informed by [morlay/session-persistence-rdb](https://github.com/morlay/session-persistence-rdb). A unified relational-database design with multiple backends and configurable schemas is deferred; neither schema stability nor migration support is guaranteed during pre-release development. +- **Packing follows durable batch boundaries** — compatible runs split by the write-behind window or an explicit flush remain separate physical records; this avoids rewriting prior rows at the cost of a timing-dependent packing ratio. +- **Synchronous compression** — Node's SQLite and Zstandard calls block the JavaScript thread; the 4 KiB threshold limits per-frame work for small records. +- **`DatabaseSync` blocks the event loop** — physical row reduction does not make SQLite operations asynchronous. +- **Busy waits block the event loop** — SQLite waits inside synchronous `DatabaseSync` calls; only a busy journal-mode transition yields between attempts, and the open-relative cutoff prevents another attempt rather than interrupting an active call. +- **External SQL readers must understand physical tags** — supported consumers read through this provider rather than treating every `events.type` as a logical event type. +- **No deletion or background historical compaction** — normal appends are insert-only. diff --git a/packages/session/session-persistence-sqlite/README.zh.md b/packages/session/session-persistence-sqlite/README.zh.md index 67c6bcfbec..579eb43e43 100644 --- a/packages/session/session-persistence-sqlite/README.zh.md +++ b/packages/session/session-persistence-sqlite/README.zh.md @@ -2,61 +2,62 @@ [English](README.md) | 中文 -SQLite 持久会话存储后端:第二个 `SessionPersistence` 提供方(见[会话持久化](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md)),满足与 `dsh-session-persistence-jsonl` 相同的约定(仅追加、连续 seq、延迟实体化、在 load 时关闭中断轮次),但用 `node:sqlite` 行而非文件字节表达。 +一个可选启用的 SQLite `SessionPersistence` 提供方。它将符合条件的 `assistant/chunk` 连续段存入打包后的物理行,对大型 payload 选择性应用 Zstandard 压缩,并对来源序列进行 delta 编码,同时恢复完全一致的逻辑 `SessionEvent[]`。随产品交付的组合均不选择它;部署方需显式挂载本包并提供数据库路径。 -`locate(meta)` 返回 `undefined`:所有会话共享一个数据库,因此不存在真实、独立的逐会话 transcript(文本记录)路径。 +`locate(meta)` 返回 `undefined`,因为所有会话共享同一个数据库。该提供方不暴露逐会话原始产物。 ## 存储模型 -每个 `SessionEvent` 1:1 映射到 `events` 表中的一行 `(session_id, seq, type, time, data, source_event_seqs, surface_op)`;`data` 是作为 JSON 文本的事件 payload,因此行结构就是原始事件本身(包括 `assistant/chunk`,保持 `seq` 连续)。两个 `TEXT` 列 `source_event_seqs` 和 `surface_op` 可为空,存储事件可选接口元数据字段(见[会话接口](../../../.agents/notes/implemented/architecture/2026-06-18-session-surface.md))。日志外元数据(`SessionHeader`)、每实体化 incarnation id 和每日志单调修订位于 `sessions` 行;`createdAt` 是存储在 strict `INTEGER` 列中的非负安全整数。单例状态行携带不可变存储 id。`sessions` 行只由第一次 `append` 写入,其存在性是延迟实体化信号(`list` 精确报告有行的会话)。 +Schema 17 保留普通 ROWID 表以及复合主键索引 `events(session_id, seq)`。标量行存储一个逻辑事件。打包行把 `text-chunks`、`reasoning-chunks` 或 `tool-call-chunks` 用作物理 `type`;`seq` 与 `time` 标识所表示的第一个事件,`data` 保存共享的分片打包 payload。打包行把 `ignorable=0` 用作物理判别值,并让 `source_event_seqs` 与 `surface_op` 保持 `NULL`;标量行仅在逻辑事件可忽略时使用 `ignorable=1`,否则使用 `NULL`。因此,未来的可忽略逻辑事件即使复用了某个存储标签名称,也不会被解码为打包行。这些标签属于存储记录,而不是 `SessionEventMap` 成员。 -仓库支持的 Node 范围可不加 flag 使用 `node:sqlite`。数据库启用外键,并使用已配置 journal mode(默认 `wal`;WAL 共享内存文件不适用时使用 rollback mode)。`PRAGMA application_id` 标识规范持久化数据库,`PRAGMA user_version` 存储布局版本。新数据库必须没有 application identity 或用户定义 schema 对象;初始化在一个事务中创建全部表并盖上两个 pragma。非 pristine 无版本数据库、外部 application identity 和所有非当前版本在 journal-mode 变更前均会被拒绝,因为该未发布格式无迁移。 +Schema 17 在本包内拥有 codec,不导入其他持久化格式中可变的实现。只有字段完全匹配、连续且属于同一分片块的文本、推理或工具调用 delta 才会打包。未知字段、surface 元数据、序列缺口、不兼容的块/调用身份以及不安全时间戳仍以标量行存储。一个打包行最多表示 1,024 个事件,未压缩 UTF-8 `data` 最多 1 MiB;更长的连续段会在不改变逻辑事件的前提下分割。读取会在向持久化协调器返回数据前,重建每个原始序列号、时间戳、token 边界、参数片段和 payload。 -在具有 POSIX mode 的文件系统上,后端为缺失目录请求 mode `0700`,并在 SQLite 打开前以 mode `0600` 排他创建缺失数据库;进程 umask 可进一步限制两者。新 WAL、共享内存和持久 rollback-journal sidecar 获得数据库最终的仅所有者 mode。现有目录、数据库文件和 sidecar 保留原 mode;除已存在数据库外的文件系统设置错误会使初始化失败。这些默认值防止宽松进程 umask 造成的意外暴露,但当其他 principal 能替换父目录中的数据库条目时,不保护数据库机密性或完整性。 +序列化后的 `data` 小于 4 KiB 时保持为 SQLite `TEXT`。达到或超过该阈值时,写入方会使用 Zstandard level 3,并且只在 frame 小于原文本的情况下存储 `BLOB`;读取方会先解压,再执行 UTF-8 校验和 JSON 解析。`source_event_seqs` 仍是完整且有序的来源数组。第一个序列使用无符号 varint,后续序列使用 ZigZag varint 编码的有符号差值,并存为 `BLOB`;不会省略任何来源,也不会把数组转换成范围。 -## 行上的约定语义 +每次追加持有 `BEGIN IMMEDIATE`,验证有界物理尾部,只打包新的持久批次,插入这些记录,并把会话 revision 递增一次。普通追加绝不删除或替换既有事件行。默认 200 毫秒写后缓冲窗口因此仍能压缩高频流,而物理写入量与新增持久批次成正比,不会反复改写不断增长的打包值。存储层逻辑尾部检查会在陈旧写入方执行变更前拒绝该写入。 -- **Append = 事务。**`append` 围绕批次运行 `BEGIN`/`COMMIT`:它实体化 `sessions` 行(如果仍未实体化),并 INSERT 每个事件,首先断言连续 seq 约定(第一个事件 `seq` 必须等于已存储 next-seq)。批次中失败(重复 seq 上的 UNIQUE 违规)会完全回滚,使已存储日志和内存游标保持一致。(`load()` 已平衡已存储日志,因此 `append` 不必修复崩溃尾部。) -- **延迟实体化。**`create()` 只在内存记录意图,第一次 `append` 前不写行。已创建但从未 append 的会话没有 `sessions` 行,因此不在 `list()` 中(它精确报告有行的会话)。 -- **在 load 时关闭中断轮次。**`load()` 实现共享[崩溃恢复约定](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md):保留有效中断轮次,在一个事务中追加合成关闭事件,并只移除撕裂尾部行。已提交解析错误或序列缺口使会话无法加载。恢复会变更已存储行,因此下一次 append 从平衡日志和准确游标开始。 -- **非修改式检查。**`inspect()` 返回不可变、平衡的逻辑视图,并可在内存中合成恢复 closer,但不会删除撕裂尾部行、追加恢复行或更改轻量修订。 -- **轻量修订。**`listSnapshots(signal?)` 组合不可变存储与数据库文件身份、每实体化 incarnation id,以及在每个变更事务中递增的每会话计数器。完整前缀读取在同一个读事务中捕获该 revision 及其事件行,`readStoredRevision()` 则只查询 session 行来校验保留的 preparation。它在不解析事件行的情况下保持未变观察稳定,并区分独立存储和重建的同 id 日志。它在共享就绪和同步元数据查询前后检查取消;查询本身不可抢占。 +完整读取按首个逻辑序列号的顺序扫描物理行。反向扫描会定位最后一个有效 `turn/end`,但不会保留每个物理行的解码副本;正向扫描则逐行解码并校验,写入最终返回的逻辑事件数组。`readFrom(id, fromSeq)` 只检查最大行跨度内的打包前驱,并把后缀锚定在可能包含 `fromSeq` 的最早前驱;这样既可包含从打包行内部开始的事件范围,也能检测相互重叠的物理损坏,而不会解析无关的更早标量行。畸形打包行按全有或全无处理:已提交区域中的损坏会拒绝读取,最终撕裂行则在可变恢复期间从其物理起点删除。修复会在持有写锁时重新读取尾部,并在删除任何数据前拒绝陈旧 marker。打包 `data` 超出 schema 字节上限时,会在解析 JSON 前拒绝。 + +## Schema 兼容性 + +全新数据库直接初始化为 schema 17。旧 schema、外部 application identity、非空未版本化数据库以及不兼容 schema 对象都会被拒绝;这个预发布提供方不提供迁移。每条语句和固定 pragma 都位于随包发布的 `.sql` 资源中;值使用 SQLite 参数,运行时代码不会拼装查询文本。 ## 配置(schemastery) ```ts interface Config { - path: string // SQLite database file path, or ':memory:' for an in-process DB - journalMode?: 'wal' | 'delete' | 'truncate' | 'persist' // journal_mode pragma; default 'wal' - preparedSessionCacheSize?: number // positive integer; default 5 - writeBatchMaxDelayMs?: number // positive integer; default 200; maximum 2_147_483_647 + path: string + journalMode?: 'wal' | 'delete' | 'truncate' | 'persist' + busyTimeoutMs?: number + preparedSessionCacheSize?: number + writeBatchMaxDelayMs?: number } ``` -## 写入路径 - -与 JSONL 后端一样,插件将每个冻结的 `session/event` 复制到对应活动会话的 controller 中,每个活动会话各有一个 controller。第一个待处理事件会开启配置的固定批处理窗口,后续事件会加入但不会重置截止时间。窗口到期后会启动一个事务;该次写入期间接纳的事件会形成另一个独立有界的后续批次。`session/flush` 会取消等待并排空当前与待处理批次。Controller 会持久化一次 fork 种子,并保留写入游标,使恢复操作绝不重新 append 已存储事件;它还会在 apply 时为活动会话设置初始状态,因为 HMR(热模块替换)不回放 `session/created`。dispose(资源释放)会在关闭数据库前排空每个保留的 controller。每个事件仍各占一行 SQLite 记录;批处理只把更多 INSERT 归入同一个事务和同一次修订版本递增。 +`journalMode` 默认为 `wal`,`busyTimeoutMs` 默认为 `5,000`,`preparedSessionCacheSize` 默认为 `5`,`writeBatchMaxDelayMs` 默认为 `200`。该超时限制每次同步 SQLite 锁等待的时长。SQLite 在切换 journal mode 时可能立即返回 `SQLITE_BUSY`,因此冷打开会在尝试之间让出执行,并在从打开时开始计算的重试截止点后不再发起新尝试。正在执行的同步 SQLite 调用可能在该截止点之后才完成。提供方会在每个连接上禁用可信 schema 与内存映射 I/O,然后读回这两项设置。提供方还会读回所选 journal mode 并要求它匹配;内存数据库显式接受 SQLite 返回的 `memory`。选择 journal 后,提供方会把 `synchronous` 固定为 `FULL` 并验证该设置,避免 SQLite 构建默认值削弱已提交追加的持久性。在 POSIX 上,数据库父目录和文件必须归当前用户所有,父目录不得允许组或其他用户写入,文件不得授予组或其他用户任何权限。符号链接和非普通文件会被拒绝。Windows 同样拒绝符号链接与非普通文件,但部署方仍负责把目录和文件 ACL 限制给 harness 用户。路径与所有权错误会拒绝插件初始化。Node SQLite 在第一次持久化操作时才加载;导入时只抑制 Node 22 精确的 SQLite `ExperimentalWarning`。存储身份与 schema 错误会在暴露或变更数据前拒绝该操作。`ensureMaterialized` 会插入零事件行的元数据行;普通 `create` 仍保持惰性,直到第一次 append。 ## 模型体验 ### 恢复的对话历史 -#### 模型看到的内容 +#### 模型看到什么 -SQLite 存储不会向当前请求提供提示词或 schema。加载会恢复与 JSONL 相同的呈现历史,并保留之前的 header 用于重建;新 loop 组合当前 envelope。恢复会用 `TOOL_NOT_STARTED` 平衡没有已持久化调用的 assistant 请求;已有持久化调用但无结果时则变为 `TOOL_OUTCOME_UNKNOWN`,它要求模型只重试只读或幂等工作,并验证可能的副作用或询问用户。行元数据和原始分片不会成为消息。 +没有 SQLite 特有内容。恢复得到与 JSONL 相同的逻辑事件和派生消息;物理打包标签绝不会进入 prompt、工具、回放或实时 `session/event` 投递。 #### Token 影响 -SQLite 存储不会增加当前请求的 token 用量。恢复会还原已保留的历史,并产生当前 envelope 以及每个中断调用所附、以引用形式呈现的修复结果文本所产生的 token 开销。 +实时请求增加零 token。恢复只为保留的逻辑历史和当前请求 envelope 付出 token。 #### KV Cache 影响 -SQLite 存储不修改当前请求前缀。只有重建历史、当前 envelope 和模型路由匹配时,恢复 loop 才能重用提供方缓存;崩溃修复结果会追加到末尾。 +物理打包不会改变请求前缀。与其他持久化后端相同,提供方 cache 复用取决于重建历史、当前 envelope 和模型路由。 -## 已知限制与暂缓事项 +## 已知限制与延期工作 -- **`DatabaseSync` 是同步的**:每个 append 事务在整个期间阻塞事件循环;对本地存储可接受,对繁忙多会话服务器是吞吐上限。 -- **写入争用无等待或重试策略**:后端不设置 busy timeout,也不重试 locked-database 错误,因此其他连接持有写事务时操作立即拒绝。 -- **只有 pristine 新数据库或当前自有 `SCHEMA_VERSION` 才能打开**:无版本 schema 对象、外部 application identity 和所有其他 schema 版本被拒绝,而不是迁移(未发布软件,无持久用户数据需要保留)。 -- **不删除已存储会话**:行会累积,直到外部移除(seam 无删除接口;`ON DELETE CASCADE` 已为这种带外清理配置)。 -- **TODO:** 该后端直接调用 `node:sqlite`。如果采用 Cordis 数据库服务(`cordis/db` / `@cordisjs` SQL driver 插件),应改为通过该服务路由,而不在此直接持有 `DatabaseSync`;约定接口(`SessionPersistence`)不会变,只更换存储驱动。 +- **过渡性的 SQLite 专用设计**——这一以效率为重点的实现参考了 [morlay/session-persistence-rdb](https://github.com/morlay/session-persistence-rdb)。支持多种后端与可配置 schema 的统一关系数据库设计尚待后续完善;预发布开发阶段不保证 schema 稳定性或迁移支持。 +- **打包服从持久批次边界**——被写后缓冲窗口或显式 flush 分开的兼容连续段会保留为不同物理记录;这以打包率受时序影响为代价,避免改写既有行。 +- **同步压缩**——Node 的 SQLite 与 Zstandard 调用都会阻塞 JavaScript 线程;4 KiB 阈值限制了小型记录的逐 frame 工作。 +- **`DatabaseSync` 会阻塞事件循环**——减少物理行不会使 SQLite 操作变为异步。 +- **繁忙等待会阻塞事件循环**——SQLite 会在同步 `DatabaseSync` 调用内等待;只有繁忙的 journal-mode 切换会在两次尝试之间让出执行,而且从打开时计算的截止点只阻止新尝试,不会中断正在执行的调用。 +- **外部 SQL 读取方必须理解物理标签**——受支持的消费方通过本提供方读取,而不是把每个 `events.type` 都当作逻辑事件类型。 +- **没有删除或后台历史压缩**——普通追加只做插入。 diff --git a/packages/session/session-persistence-sqlite/package.json b/packages/session/session-persistence-sqlite/package.json index f824b9b3a8..a4495d43a9 100644 --- a/packages/session/session-persistence-sqlite/package.json +++ b/packages/session/session-persistence-sqlite/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-persistence-sqlite", - "description": "SQLite durable session persistence backend for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "description": "SQLite durable session persistence with physical chunk-row packing", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -28,11 +28,13 @@ "files": [ "lib/index.js", "lib/invariant.js", + "resources/sql/**/*.sql", "lib/types/**/*.d.ts" ], "license": "MIT", "peerDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-session-persistence": "workspace:^", "@deepseek-ai/cordis": "workspace:^" @@ -41,9 +43,13 @@ "@deepseek-ai/schemastery": "workspace:^" }, "devDependencies": { + "@deepseek-ai/cordis-plugin-include": "workspace:^", + "@deepseek-ai/cordis-plugin-loader": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-session-persistence": "workspace:^", - "@deepseek-ai/cordis": "workspace:^" + "@deepseek-ai/cordis": "workspace:^", + "typescript": "^6.0.3" } } diff --git a/packages/session/session-persistence-sqlite/resources/sql/begin-immediate.sql b/packages/session/session-persistence-sqlite/resources/sql/begin-immediate.sql new file mode 100644 index 0000000000..67edb1dd06 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/begin-immediate.sql @@ -0,0 +1 @@ +BEGIN IMMEDIATE; diff --git a/packages/session/session-persistence-sqlite/resources/sql/begin.sql b/packages/session/session-persistence-sqlite/resources/sql/begin.sql new file mode 100644 index 0000000000..1775571fa7 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/begin.sql @@ -0,0 +1 @@ +BEGIN; diff --git a/packages/session/session-persistence-sqlite/resources/sql/commit.sql b/packages/session/session-persistence-sqlite/resources/sql/commit.sql new file mode 100644 index 0000000000..87ef767444 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/commit.sql @@ -0,0 +1 @@ +COMMIT; diff --git a/packages/session/session-persistence-sqlite/resources/sql/delete-events-from.sql b/packages/session/session-persistence-sqlite/resources/sql/delete-events-from.sql new file mode 100644 index 0000000000..ab5f81d89a --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/delete-events-from.sql @@ -0,0 +1,2 @@ +DELETE FROM events +WHERE session_id = ? AND seq >= ?; diff --git a/packages/session/session-persistence-sqlite/resources/sql/foreign-keys-on.sql b/packages/session/session-persistence-sqlite/resources/sql/foreign-keys-on.sql new file mode 100644 index 0000000000..c8ccb1392a --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/foreign-keys-on.sql @@ -0,0 +1 @@ +PRAGMA foreign_keys = ON; diff --git a/packages/session/session-persistence-sqlite/resources/sql/insert-event.sql b/packages/session/session-persistence-sqlite/resources/sql/insert-event.sql new file mode 100644 index 0000000000..92b4de310d --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/insert-event.sql @@ -0,0 +1,3 @@ +INSERT INTO events + (session_id, seq, type, time, data, source_event_seqs, surface_op, ignorable) +VALUES (?, ?, ?, ?, ?, ?, ?, ?); diff --git a/packages/session/session-persistence-sqlite/resources/sql/insert-persistence-state.sql b/packages/session/session-persistence-sqlite/resources/sql/insert-persistence-state.sql new file mode 100644 index 0000000000..f516014d93 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/insert-persistence-state.sql @@ -0,0 +1,2 @@ +INSERT INTO persistence_state (singleton, store_id) +VALUES (1, ?); diff --git a/packages/session/session-persistence-sqlite/resources/sql/journal-mode-delete.sql b/packages/session/session-persistence-sqlite/resources/sql/journal-mode-delete.sql new file mode 100644 index 0000000000..0f4c37efca --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/journal-mode-delete.sql @@ -0,0 +1 @@ +PRAGMA journal_mode = DELETE; diff --git a/packages/session/session-persistence-sqlite/resources/sql/journal-mode-persist.sql b/packages/session/session-persistence-sqlite/resources/sql/journal-mode-persist.sql new file mode 100644 index 0000000000..02445b260b --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/journal-mode-persist.sql @@ -0,0 +1 @@ +PRAGMA journal_mode = PERSIST; diff --git a/packages/session/session-persistence-sqlite/resources/sql/journal-mode-truncate.sql b/packages/session/session-persistence-sqlite/resources/sql/journal-mode-truncate.sql new file mode 100644 index 0000000000..d119c32bec --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/journal-mode-truncate.sql @@ -0,0 +1 @@ +PRAGMA journal_mode = TRUNCATE; diff --git a/packages/session/session-persistence-sqlite/resources/sql/journal-mode-wal.sql b/packages/session/session-persistence-sqlite/resources/sql/journal-mode-wal.sql new file mode 100644 index 0000000000..2d30d8af9f --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/journal-mode-wal.sql @@ -0,0 +1 @@ +PRAGMA journal_mode = WAL; diff --git a/packages/session/session-persistence-sqlite/resources/sql/mmap-off.sql b/packages/session/session-persistence-sqlite/resources/sql/mmap-off.sql new file mode 100644 index 0000000000..22bcd0ea34 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/mmap-off.sql @@ -0,0 +1 @@ +PRAGMA mmap_size = 0; diff --git a/packages/session/session-persistence-sqlite/resources/sql/rollback.sql b/packages/session/session-persistence-sqlite/resources/sql/rollback.sql new file mode 100644 index 0000000000..3b18e77376 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/rollback.sql @@ -0,0 +1 @@ +ROLLBACK; diff --git a/packages/session/session-persistence-sqlite/resources/sql/schema.sql b/packages/session/session-persistence-sqlite/resources/sql/schema.sql new file mode 100644 index 0000000000..41d247dba7 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/schema.sql @@ -0,0 +1,30 @@ +CREATE TABLE persistence_state ( + singleton INTEGER PRIMARY KEY CHECK (singleton = 1), + store_id TEXT NOT NULL +) STRICT; + +CREATE TABLE sessions ( + id TEXT PRIMARY KEY, + version INTEGER NOT NULL, + created_at INTEGER NOT NULL, + cwd TEXT, + parent_session TEXT, + seed_length INTEGER, + origin TEXT, + delegation_depth INTEGER, + agent_preset TEXT, + incarnation TEXT NOT NULL, + revision INTEGER NOT NULL +) STRICT; + +CREATE TABLE events ( + session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE, + seq INTEGER NOT NULL, + type TEXT NOT NULL, + time INTEGER NOT NULL, + data ANY NOT NULL, + source_event_seqs ANY, + surface_op TEXT, + ignorable INTEGER CHECK (ignorable IS NULL OR ignorable IN (0, 1)), + PRIMARY KEY (session_id, seq) +) STRICT; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-application-id.sql b/packages/session/session-persistence-sqlite/resources/sql/select-application-id.sql new file mode 100644 index 0000000000..3de4abbb88 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-application-id.sql @@ -0,0 +1 @@ +PRAGMA application_id; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-events-from.sql b/packages/session/session-persistence-sqlite/resources/sql/select-events-from.sql new file mode 100644 index 0000000000..a5748dd974 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-events-from.sql @@ -0,0 +1,4 @@ +SELECT seq, type, time, data, source_event_seqs, surface_op, ignorable +FROM events +WHERE session_id = ? AND seq >= ? +ORDER BY seq; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-events.sql b/packages/session/session-persistence-sqlite/resources/sql/select-events.sql new file mode 100644 index 0000000000..76437f8eaf --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-events.sql @@ -0,0 +1,4 @@ +SELECT seq, type, time, data, source_event_seqs, surface_op, ignorable +FROM events +WHERE session_id = ? +ORDER BY seq; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-mmap-size.sql b/packages/session/session-persistence-sqlite/resources/sql/select-mmap-size.sql new file mode 100644 index 0000000000..58e55155f4 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-mmap-size.sql @@ -0,0 +1 @@ +PRAGMA mmap_size; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-packed-predecessors.sql b/packages/session/session-persistence-sqlite/resources/sql/select-packed-predecessors.sql new file mode 100644 index 0000000000..54a52180fc --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-packed-predecessors.sql @@ -0,0 +1,6 @@ +SELECT seq, type, time, data, source_event_seqs, surface_op, ignorable +FROM events +WHERE session_id = ? AND seq >= ? AND seq < ? + AND type IN ('text-chunks', 'reasoning-chunks', 'tool-call-chunks') + AND ignorable = 0 +ORDER BY seq; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-schema-objects.sql b/packages/session/session-persistence-sqlite/resources/sql/select-schema-objects.sql new file mode 100644 index 0000000000..216925f35a --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-schema-objects.sql @@ -0,0 +1,4 @@ +SELECT type, name, tbl_name, sql +FROM sqlite_schema +WHERE name NOT GLOB 'sqlite_*' +ORDER BY type, name; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-session.sql b/packages/session/session-persistence-sqlite/resources/sql/select-session.sql new file mode 100644 index 0000000000..456cba91c3 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-session.sql @@ -0,0 +1,4 @@ +SELECT id, version, created_at, cwd, parent_session, seed_length, origin, + delegation_depth, agent_preset, incarnation, revision +FROM sessions +WHERE id = ?; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-sessions.sql b/packages/session/session-persistence-sqlite/resources/sql/select-sessions.sql new file mode 100644 index 0000000000..37a1f22e36 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-sessions.sql @@ -0,0 +1,3 @@ +SELECT id, version, created_at, cwd, parent_session, seed_length, origin, + delegation_depth, agent_preset, incarnation, revision +FROM sessions; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-store-id.sql b/packages/session/session-persistence-sqlite/resources/sql/select-store-id.sql new file mode 100644 index 0000000000..168e09cb80 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-store-id.sql @@ -0,0 +1,3 @@ +SELECT store_id +FROM persistence_state +WHERE singleton = 1; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-synchronous.sql b/packages/session/session-persistence-sqlite/resources/sql/select-synchronous.sql new file mode 100644 index 0000000000..b41be01714 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-synchronous.sql @@ -0,0 +1 @@ +PRAGMA synchronous; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-tail-events.sql b/packages/session/session-persistence-sqlite/resources/sql/select-tail-events.sql new file mode 100644 index 0000000000..2d958de7c3 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-tail-events.sql @@ -0,0 +1,5 @@ +SELECT seq, type, time, data, source_event_seqs, surface_op, ignorable +FROM events +WHERE session_id = ? +ORDER BY seq DESC +LIMIT ?; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-trusted-schema.sql b/packages/session/session-persistence-sqlite/resources/sql/select-trusted-schema.sql new file mode 100644 index 0000000000..d304b1f850 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-trusted-schema.sql @@ -0,0 +1 @@ +PRAGMA trusted_schema; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-user-object-count.sql b/packages/session/session-persistence-sqlite/resources/sql/select-user-object-count.sql new file mode 100644 index 0000000000..60665bb66e --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-user-object-count.sql @@ -0,0 +1,3 @@ +SELECT COUNT(*) AS count +FROM sqlite_schema +WHERE name NOT GLOB 'sqlite_*'; diff --git a/packages/session/session-persistence-sqlite/resources/sql/select-user-version.sql b/packages/session/session-persistence-sqlite/resources/sql/select-user-version.sql new file mode 100644 index 0000000000..4edeca1a4d --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/select-user-version.sql @@ -0,0 +1 @@ +PRAGMA user_version; diff --git a/packages/session/session-persistence-sqlite/resources/sql/set-application-id.sql b/packages/session/session-persistence-sqlite/resources/sql/set-application-id.sql new file mode 100644 index 0000000000..617d10cab3 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/set-application-id.sql @@ -0,0 +1 @@ +PRAGMA application_id = 1146308688; diff --git a/packages/session/session-persistence-sqlite/resources/sql/set-user-version-17.sql b/packages/session/session-persistence-sqlite/resources/sql/set-user-version-17.sql new file mode 100644 index 0000000000..5aac576e8c --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/set-user-version-17.sql @@ -0,0 +1 @@ +PRAGMA user_version = 17; diff --git a/packages/session/session-persistence-sqlite/resources/sql/synchronous-full.sql b/packages/session/session-persistence-sqlite/resources/sql/synchronous-full.sql new file mode 100644 index 0000000000..b0380b1197 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/synchronous-full.sql @@ -0,0 +1 @@ +PRAGMA synchronous = FULL; diff --git a/packages/session/session-persistence-sqlite/resources/sql/trusted-schema-off.sql b/packages/session/session-persistence-sqlite/resources/sql/trusted-schema-off.sql new file mode 100644 index 0000000000..973c6d1def --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/trusted-schema-off.sql @@ -0,0 +1 @@ +PRAGMA trusted_schema = OFF; diff --git a/packages/session/session-persistence-sqlite/resources/sql/update-session-revision.sql b/packages/session/session-persistence-sqlite/resources/sql/update-session-revision.sql new file mode 100644 index 0000000000..2cfbcb2b82 --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/update-session-revision.sql @@ -0,0 +1,3 @@ +UPDATE sessions +SET revision = revision + 1 +WHERE id = ?; diff --git a/packages/session/session-persistence-sqlite/resources/sql/upsert-session.sql b/packages/session/session-persistence-sqlite/resources/sql/upsert-session.sql new file mode 100644 index 0000000000..c1000e4daa --- /dev/null +++ b/packages/session/session-persistence-sqlite/resources/sql/upsert-session.sql @@ -0,0 +1,13 @@ +INSERT INTO sessions + (id, version, created_at, cwd, parent_session, seed_length, origin, + delegation_depth, agent_preset, incarnation, revision) +VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0) +ON CONFLICT(id) DO UPDATE SET + version = excluded.version, + created_at = excluded.created_at, + cwd = excluded.cwd, + parent_session = excluded.parent_session, + seed_length = excluded.seed_length, + origin = excluded.origin, + delegation_depth = excluded.delegation_depth, + agent_preset = excluded.agent_preset; diff --git a/packages/session/session-persistence-sqlite/src/codec.ts b/packages/session/session-persistence-sqlite/src/codec.ts new file mode 100644 index 0000000000..5ee2126c6e --- /dev/null +++ b/packages/session/session-persistence-sqlite/src/codec.ts @@ -0,0 +1,343 @@ +/** + * Schema-17 physical chunk-row codec. This package owns the durable tags, + * validation, and row-size limits independently from other persistence formats. + * @module @deepseek-ai/dsh-session-persistence-sqlite/codec + */ + +import type { StreamChunk } from '@deepseek-ai/dsh-llm' +import type { SessionEvent } from '@deepseek-ai/dsh-session' + +/* jscpd:ignore-start -- schema 17 deliberately owns a frozen physical codec; + * importing or sharing the JSONL codec would let that format mutate this database interpreter. */ +type DeltaKind = 'text-delta' | 'reasoning-delta' | 'tool-call-delta' +type DeltaEvent = SessionEvent<'assistant/chunk'> + +interface RunDataBase { + readonly turn: number + readonly step: number + readonly index: number + readonly dt: number[] +} + +interface TextRunData extends RunDataBase { + readonly texts: string[] +} + +interface ToolCallRunData extends RunDataBase { + readonly id: Extract['id'] + readonly name?: string + readonly args: string[] +} + +/** One schema-17 packed physical record. */ +export type ChunkRow = + | { readonly type: 'text-chunks'; readonly seq0: number; readonly time0: number; readonly data: TextRunData } + | { readonly type: 'reasoning-chunks'; readonly seq0: number; readonly time0: number; readonly data: TextRunData } + | { readonly type: 'tool-call-chunks'; readonly seq0: number; readonly time0: number; readonly data: ToolCallRunData } + +/** One scalar event or schema-17 packed physical record. */ +export type StorageRecord = SessionEvent | ChunkRow + +/** Minimum eligible members in a packed physical record. */ +export const MIN_PACKED_ROW_MEMBERS = 3 +/** Maximum logical members represented by one packed physical record. */ +export const MAX_PACKED_ROW_MEMBERS = 1_024 +/** Maximum UTF-8 bytes in one packed physical record's data column. */ +export const MAX_PACKED_DATA_BYTES = 1_048_576 + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null +} + +function hasExactKeys(value: object, keys: readonly string[]): boolean { + return Object.keys(value).length === keys.length && keys.every(key => Object.hasOwn(value, key)) +} + +function classify(event: SessionEvent): DeltaKind | undefined { + if (event.type !== 'assistant/chunk') return undefined + if (!hasExactKeys(event, ['type', 'seq', 'time', 'data'])) return undefined + if (!Number.isSafeInteger(event.seq) || event.seq < 0 || !Number.isSafeInteger(event.time)) return undefined + const data: unknown = event.data + if (!isRecord(data) || !hasExactKeys(data, ['turn', 'step', 'chunk'])) return undefined + if (typeof data.turn !== 'number' || typeof data.step !== 'number') return undefined + const chunk = data.chunk + if (!isRecord(chunk) || typeof chunk.index !== 'number') return undefined + switch (chunk.type) { + case 'text-delta': + case 'reasoning-delta': + return hasExactKeys(chunk, ['type', 'index', 'text']) && typeof chunk.text === 'string' + ? chunk.type + : undefined + case 'tool-call-delta': { + const validKeys = hasExactKeys(chunk, ['type', 'index', 'id', 'argumentsDelta']) + || (hasExactKeys(chunk, ['type', 'index', 'id', 'name', 'argumentsDelta']) + && typeof chunk.name === 'string') + return validKeys && typeof chunk.id === 'string' && typeof chunk.argumentsDelta === 'string' + ? chunk.type + : undefined + } + default: + return undefined + } +} + +function toolCallOf(event: DeltaEvent): { readonly id: string; readonly name?: string } { + return event.data.chunk as { readonly id: string; readonly name?: string } +} + +function indexOf(event: DeltaEvent): number { + return (event.data.chunk as { readonly index: number }).index +} + +function continues(previous: DeltaEvent, next: DeltaEvent, kind: DeltaKind): boolean { + if (next.seq !== previous.seq + 1 || !Number.isSafeInteger(next.time - previous.time)) return false + if (next.data.turn !== previous.data.turn || next.data.step !== previous.data.step) return false + if (indexOf(next) !== indexOf(previous)) return false + if (kind !== 'tool-call-delta') return true + const left = toolCallOf(previous) + const right = toolCallOf(next) + return left.id === right.id + && Object.hasOwn(left, 'name') === Object.hasOwn(right, 'name') + && left.name === right.name +} + +function buildRow(kind: DeltaKind, run: readonly DeltaEvent[]): ChunkRow { + const first = run[0] as DeltaEvent + const base = { + turn: first.data.turn, + step: first.data.step, + index: indexOf(first), + dt: run.slice(1).map((event, index) => event.time - (run[index] as DeltaEvent).time), + } + const envelope = { seq0: first.seq, time0: first.time } + if (kind === 'tool-call-delta') { + const call = toolCallOf(first) + return { + type: 'tool-call-chunks', + ...envelope, + data: { + ...base, + id: call.id as Extract['id'], + ...Object.hasOwn(call, 'name') ? { name: call.name as string } : {}, + args: run.map(event => (event.data.chunk as { readonly argumentsDelta: string }).argumentsDelta), + }, + } + } + const data = { + ...base, + texts: run.map(event => (event.data.chunk as { readonly text: string }).text), + } + return kind === 'text-delta' + ? { type: 'text-chunks', ...envelope, data } + : { type: 'reasoning-chunks', ...envelope, data } +} + +function packedDataBytes(row: ChunkRow): number { + return Buffer.byteLength(JSON.stringify(row.data)) +} + +function emitBoundedRun(out: StorageRecord[], kind: DeltaKind, completeRun: readonly DeltaEvent[]): void { + let offset = 0 + while (completeRun.length - offset >= MIN_PACKED_ROW_MEMBERS) { + let low = MIN_PACKED_ROW_MEMBERS + let high = Math.min(completeRun.length - offset, MAX_PACKED_ROW_MEMBERS) + const largest = buildRow(kind, completeRun.slice(offset, offset + high)) + if (packedDataBytes(largest) <= MAX_PACKED_DATA_BYTES) { + out.push(largest) + offset += high + continue + } + high -= 1 + let accepted = 0 + let acceptedRow: ChunkRow | undefined + while (low <= high) { + const middle = Math.floor((low + high) / 2) + const candidate = buildRow(kind, completeRun.slice(offset, offset + middle)) + if (packedDataBytes(candidate) <= MAX_PACKED_DATA_BYTES) { + accepted = middle + acceptedRow = candidate + low = middle + 1 + } else { + high = middle - 1 + } + } + if (accepted === 0) { + out.push(completeRun[offset] as DeltaEvent) + offset += 1 + continue + } + /* v8 ignore next -- accepted is set only with its same-branch candidate. */ + out.push(acceptedRow ?? malformed(kind, 'bounded encoder lost its accepted row')) + offset += accepted + } + out.push(...completeRun.slice(offset)) +} + +/** + * Pack eligible logical chunk runs into bounded schema-17 records. + * @param events - logical events in sequence order. + * @returns scalar and packed physical records in equivalent order. + */ +export function packChunkRuns(events: readonly SessionEvent[]): StorageRecord[] { + const out: StorageRecord[] = [] + let kind: DeltaKind | undefined + let run: DeltaEvent[] = [] + const flush = (): void => { + if (kind === undefined) out.push(...run) + else emitBoundedRun(out, kind, run) + kind = undefined + run = [] + } + for (const event of events) { + const nextKind = classify(event) + if (nextKind === undefined) { + flush() + out.push(event) + continue + } + const delta = event as DeltaEvent + const previous = run.at(-1) + if (nextKind === kind && previous !== undefined && continues(previous, delta, nextKind)) { + run.push(delta) + continue + } + flush() + kind = nextKind + run = [delta] + } + flush() + return out +} + +function malformed(tag: string, reason: string): never { + throw new Error(`malformed ${tag} storage row: ${reason}`) +} + +function validateRunData( + tag: string, + data: Record, + payloadKey: 'texts' | 'args', + serializedBytes?: number, +): string[] { + if (typeof data.turn !== 'number' || typeof data.step !== 'number' || typeof data.index !== 'number') { + malformed(tag, 'turn/step/index must be numbers') + } + const payload = data[payloadKey] + if (!Array.isArray(payload) + || payload.length < MIN_PACKED_ROW_MEMBERS + || payload.length > MAX_PACKED_ROW_MEMBERS + || payload.some(member => typeof member !== 'string')) { + malformed(tag, `${payloadKey} must contain ${MIN_PACKED_ROW_MEMBERS}..${MAX_PACKED_ROW_MEMBERS} strings`) + } + const gaps = data.dt + if (!Array.isArray(gaps) || gaps.some(gap => !Number.isSafeInteger(gap))) { + malformed(tag, 'dt must be an array of safe integers') + } + if (gaps.length !== payload.length - 1) malformed(tag, 'dt length must match the member count') + if ((serializedBytes ?? Buffer.byteLength(JSON.stringify(data))) > MAX_PACKED_DATA_BYTES) { + malformed(tag, `data exceeds ${MAX_PACKED_DATA_BYTES} UTF-8 bytes`) + } + return payload as string[] +} + +function validateRow( + value: Record, + tag: ChunkRow['type'], + serializedBytes?: number, +): ChunkRow { + if (!hasExactKeys(value, ['type', 'seq0', 'time0', 'data'])) malformed(tag, 'invalid envelope fields') + if (!Number.isSafeInteger(value.seq0) || (value.seq0 as number) < 0) malformed(tag, 'seq0 must be non-negative') + if (!Number.isSafeInteger(value.time0)) malformed(tag, 'time0 must be a safe integer') + const data = value.data + if (!isRecord(data)) malformed(tag, 'data must be an object') + let payload: string[] + if (tag === 'tool-call-chunks') { + const withName = hasExactKeys(data, ['turn', 'step', 'index', 'id', 'name', 'dt', 'args']) + if (!withName && !hasExactKeys(data, ['turn', 'step', 'index', 'id', 'dt', 'args'])) { + malformed(tag, 'invalid tool-call data fields') + } + if (typeof data.id !== 'string' || (withName && typeof data.name !== 'string')) { + malformed(tag, 'id and optional name must be strings') + } + payload = validateRunData(tag, data, 'args', serializedBytes) + } else { + if (!hasExactKeys(data, ['turn', 'step', 'index', 'dt', 'texts'])) malformed(tag, 'invalid text data fields') + payload = validateRunData(tag, data, 'texts', serializedBytes) + } + if (!Number.isSafeInteger((value.seq0 as number) + payload.length - 1)) malformed(tag, 'member seqs exceed safe integers') + let time = value.time0 as number + for (const gap of data.dt as number[]) { + time += gap + if (!Number.isSafeInteger(time)) malformed(tag, 'member times exceed safe integers') + } + return value as unknown as ChunkRow +} + +function expandRow(row: ChunkRow): SessionEvent[] { + const members = row.type === 'tool-call-chunks' ? row.data.args : row.data.texts + const events: SessionEvent[] = [] + let time = row.time0 + for (let index = 0; index < members.length; index += 1) { + if (index > 0) time += row.data.dt[index - 1] as number + let chunk: StreamChunk + switch (row.type) { + case 'text-chunks': + chunk = { type: 'text-delta', index: row.data.index, text: members[index] as string } + break + case 'reasoning-chunks': + chunk = { type: 'reasoning-delta', index: row.data.index, text: members[index] as string } + break + case 'tool-call-chunks': + chunk = { + type: 'tool-call-delta', + index: row.data.index, + id: row.data.id, + ...Object.hasOwn(row.data, 'name') ? { name: row.data.name as string } : {}, + argumentsDelta: members[index] as string, + } + break + } + events.push({ + type: 'assistant/chunk', + seq: row.seq0 + index, + time, + data: { turn: row.data.turn, step: row.data.step, chunk }, + }) + } + return events +} + +/** + * Decode one scalar or packed schema-17 record. + * @param value - parsed physical-record value. + * @returns the represented logical events. + */ +export function decodeStorageRecord(value: unknown): SessionEvent[] { + if (!isRecord(value)) return [value as SessionEvent] + const tag = value.type + if (tag !== 'text-chunks' && tag !== 'reasoning-chunks' && tag !== 'tool-call-chunks') { + return [value as SessionEvent] + } + return expandRow(validateRow(value, tag)) +} + +/** + * Decode one packed row from its exact uncompressed data value. The byte bound + * rejects oversized input before JSON parsing and avoids serializing it again. + * @param tag - validated packed physical type. + * @param seq0 - first represented logical sequence number. + * @param time0 - first represented logical timestamp. + * @param serializedData - decoded SQLite data-column text. + * @returns the represented logical events. + */ +export function decodeSerializedChunkRow( + tag: ChunkRow['type'], + seq0: number, + time0: number, + serializedData: string, +): SessionEvent[] { + const bytes = Buffer.byteLength(serializedData) + if (bytes > MAX_PACKED_DATA_BYTES) malformed(tag, `data exceeds ${MAX_PACKED_DATA_BYTES} UTF-8 bytes`) + return expandRow(validateRow({ type: tag, seq0, time0, data: JSON.parse(serializedData) as unknown }, tag, bytes)) +} +/* jscpd:ignore-end */ diff --git a/packages/session/session-persistence-sqlite/src/compression.ts b/packages/session/session-persistence-sqlite/src/compression.ts new file mode 100644 index 0000000000..272b099a37 --- /dev/null +++ b/packages/session/session-persistence-sqlite/src/compression.ts @@ -0,0 +1,276 @@ +/** + * Fixed physical-record compression for SQLite. Schema-owned functions + * encode logical events and decode tagged rows before persistence consumers + * observe them. + * @module @deepseek-ai/dsh-session-persistence-sqlite/compression + */ + +import { TextDecoder } from 'node:util' +import { constants, zstdCompressSync, zstdDecompressSync } from 'node:zlib' +import type { SessionEvent, SurfaceEventType } from '@deepseek-ai/dsh-session' +import { + decodeSerializedChunkRow, + type ChunkRow, + MAX_PACKED_DATA_BYTES, + type StorageRecord, +} from './codec.ts' +import type { EventRow } from './schema.ts' + +/** One physical row ready for SQLite parameter binding. */ +export interface BoundRecord { + readonly seq: number + readonly type: string + readonly time: number + readonly data: string | Uint8Array + readonly sourceEventSeqs: Uint8Array | null + readonly surfaceOp: string | null + readonly ignorable: number | null +} + +/** Small values stay as SQLite text to avoid per-frame CPU and byte overhead. */ +export const ZSTD_DATA_THRESHOLD_BYTES = 4_096 + +const MAX_SAFE_INTEGER = BigInt(Number.MAX_SAFE_INTEGER) +const MAX_ZIGZAG_INTEGER = MAX_SAFE_INTEGER * 2n +const UTF8_DECODER = new TextDecoder('utf-8', { fatal: true }) +const ZSTD_COMPRESSION_LEVEL = 3 +const PACKED_ROW_SENTINEL = 0 + +const CHUNK_TAGS = ['text-chunks', 'reasoning-chunks', 'tool-call-chunks'] as const +type ChunkTag = typeof CHUNK_TAGS[number] + +function isChunkTag(value: string): value is ChunkTag { + return (CHUNK_TAGS as readonly string[]).includes(value) +} + +/** + * Decode one physical SQLite row into its complete logical event span. + * @param row - detached SQLite event row. + * @returns every logical event represented by the row. + */ +export function decodeRow(row: EventRow): SessionEvent[] { + if (row.ignorable !== PACKED_ROW_SENTINEL) return [decodeScalarRow(row)] + if (!isChunkTag(row.type)) { + throw new Error(`malformed ${row.type} storage row: packed discriminator requires a chunk tag`) + } + if (row.source_event_seqs !== null || row.surface_op !== null) { + throw new Error(`malformed ${row.type} storage row: packed surface fields must be null`) + } + return decodeSerializedChunkRow( + row.type, + row.seq, + row.time, + decodeData(row.data, MAX_PACKED_DATA_BYTES), + ) +} + +/** + * Convert a storage record to SQLite column values. + * @param record - scalar event or packed chunk record. + * @returns column values for one physical insert. + */ +export function bindRecord(record: StorageRecord): BoundRecord { + if (isChunkRow(record)) { + return { + seq: record.seq0, + type: record.type, + time: record.time0, + data: encodeData(JSON.stringify(record.data)), + sourceEventSeqs: null, + surfaceOp: null, + ignorable: PACKED_ROW_SENTINEL, + } + } + const event = record + const surface = event as SessionEvent + return { + seq: event.seq, + type: event.type, + time: event.time, + data: encodeData(JSON.stringify(event.data)), + sourceEventSeqs: surface.sourceEventSeqs === undefined + ? null + : encodeSourceEventSeqs(surface.sourceEventSeqs), + surfaceOp: surface.surfaceOp === undefined ? null : JSON.stringify(surface.surfaceOp), + ignorable: event.ignorable === true ? 1 : null, + } +} + +function encodeData(serialized: string): string | Uint8Array { + const bytes = Buffer.from(serialized) + if (bytes.length < ZSTD_DATA_THRESHOLD_BYTES) return serialized + const compressed = zstdCompressSync(bytes, { + params: { [constants.ZSTD_c_compressionLevel]: ZSTD_COMPRESSION_LEVEL }, + }) + return compressed.length < bytes.length ? compressed : serialized +} + +function decodeData(value: string | Uint8Array, maxOutputLength?: number): string { + if (typeof value === 'string') return value + const decoded = maxOutputLength === undefined + ? zstdDecompressSync(value) + : zstdDecompressSync(value, { maxOutputLength }) + return UTF8_DECODER.decode(decoded) +} + +function encodeSourceEventSeqs(values: readonly number[]): Uint8Array { + const bytes: number[] = [] + let previous = 0n + for (let index = 0; index < values.length; index += 1) { + const sourceSeq = values[index] as number + if (!Number.isSafeInteger(sourceSeq) || sourceSeq < 0) { + throw new TypeError('sourceEventSeqs must contain non-negative safe integers') + } + const value = BigInt(sourceSeq) + const encoded = index === 0 + ? value + : value >= previous + ? (value - previous) * 2n + : ((previous - value) * 2n) - 1n + appendVarint(bytes, encoded) + previous = value + } + return Buffer.from(bytes) +} + +function appendVarint(bytes: number[], value: bigint): void { + let remaining = value + while (remaining >= 0x80n) { + bytes.push(Number(remaining & 0x7fn) | 0x80) + remaining >>= 7n + } + bytes.push(Number(remaining)) +} + +function decodeSourceEventSeqs(bytes: Uint8Array): number[] { + const values: number[] = [] + let previous = 0n + let offset = 0 + while (offset < bytes.length) { + const first = values.length === 0 + const decoded = readVarint(bytes, offset, first ? MAX_SAFE_INTEGER : MAX_ZIGZAG_INTEGER) + offset = decoded.offset + const delta = first + ? decoded.value + : (decoded.value & 1n) === 0n + ? decoded.value / 2n + : -((decoded.value + 1n) / 2n) + const value = first ? delta : previous + delta + if (value < 0n || value > MAX_SAFE_INTEGER) { + throw new Error('malformed source_event_seqs storage value: decoded seq is out of range') + } + values.push(Number(value)) + previous = value + } + return values +} + +function readVarint( + bytes: Uint8Array, + offset: number, + limit: bigint, +): { readonly value: bigint; readonly offset: number } { + let value = 0n + let shift = 0n + while (offset < bytes.length) { + const byte = bytes[offset] as number + offset += 1 + value |= BigInt(byte & 0x7f) << shift + if ((byte & 0x80) === 0) { + if (shift > 0n && (byte & 0x7f) === 0) { + throw new Error('malformed source_event_seqs storage value: non-canonical varint') + } + if (value > limit) { + throw new Error('malformed source_event_seqs storage value: varint is out of range') + } + return { value, offset } + } + shift += 7n + if (shift > 56n) { + throw new Error('malformed source_event_seqs storage value: varint is out of range') + } + } + throw new Error('malformed source_event_seqs storage value: truncated varint') +} + +function isChunkRow(record: StorageRecord): record is ChunkRow { + return isChunkTag(record.type) && 'seq0' in record && !('seq' in record) +} + +function decodeScalarRow(row: EventRow): SessionEvent { + const surfaceFields = { + ...row.source_event_seqs === null + ? {} + : { sourceEventSeqs: decodeSourceEventSeqs(row.source_event_seqs) }, + ...row.surface_op === null + ? {} + : { surfaceOp: JSON.parse(row.surface_op) as SessionEvent['surfaceOp'] }, + } + return { + type: row.type as SessionEvent['type'], + seq: row.seq, + time: row.time, + data: JSON.parse(decodeData(row.data)) as SessionEvent['data'], + ...surfaceFields, + ...row.ignorable === 1 ? { ignorable: true as const } : {}, + } as SessionEvent +} + +/** + * Validate and flatten physical rows into their logical prefix. A malformed + * row or logical gap is committed corruption when a later valid turn end + * exists; otherwise it starts a removable physical tail. + * @param rows - physical rows ordered by their first logical sequence. + * @param base - logical sequence expected from the first selected row. + * @returns the contiguous logical prefix and optional physical deletion base. + */ +export function scanRows( + rows: readonly EventRow[], + base = 0, +): { preserved: SessionEvent[]; tornFrom?: number } { + let lastTurnEndRow = -1 + for (let index = rows.length - 1; index >= 0; index -= 1) { + try { + if (decodeRow(rows[index] as EventRow).some(event => event.type === 'turn/end')) { + lastTurnEndRow = index + break + } + } catch { + // A malformed row cannot prove that an earlier physical prefix committed. + } + } + + const preserved: SessionEvent[] = [] + let expected = base + for (let rowIndex = 0; rowIndex < rows.length; rowIndex += 1) { + const physical = rows[rowIndex] as EventRow + let logicalEvents: SessionEvent[] | undefined + try { + logicalEvents = decodeRow(physical) + } catch { + // The committed-prefix rule below owns whether this invalid row is fatal or repairable. + } + if (logicalEvents === undefined) { + if (rowIndex <= lastTurnEndRow) { + throw new Error(`corrupt session log: invalid committed physical row at seq ${physical.seq}`) + } + return { preserved, tornFrom: physical.seq } + } + let contiguous = true + for (const event of logicalEvents) { + if (event.seq !== expected) { + contiguous = false + break + } + expected += 1 + } + if (!contiguous) { + if (rowIndex <= lastTurnEndRow) { + throw new Error(`corrupt session log: invalid committed physical row at seq ${physical.seq}`) + } + return { preserved, tornFrom: physical.seq } + } + preserved.push(...logicalEvents) + } + return { preserved } +} diff --git a/packages/session/session-persistence-sqlite/src/index.ts b/packages/session/session-persistence-sqlite/src/index.ts index 969f1f5fac..50204be7ab 100644 --- a/packages/session/session-persistence-sqlite/src/index.ts +++ b/packages/session/session-persistence-sqlite/src/index.ts @@ -1,90 +1,46 @@ /** - * SQLite durable session-persistence backend. It maps each session header and - * event to rows, and delegates write-path orchestration to - * {@link PersistenceCoordinator}. It has no independent per-session artifact, - * so its locator returns `undefined`. + * Opt-in SQLite persistence provider. Logical sessions remain unchanged; + * the physical backend packs eligible chunk runs into schema-17 rows. * @module @deepseek-ai/dsh-session-persistence-sqlite */ -import { Context } from '@deepseek-ai/cordis' +import { Context, Service } from '@deepseek-ai/cordis' import z from '@deepseek-ai/schemastery' -import { randomUUID } from 'node:crypto' -import { statSync } from 'node:fs' -import { DatabaseSync } from 'node:sqlite' -import { mkdir, open } from 'node:fs/promises' -import { dirname, resolve } from 'node:path' +import type { + Session, + SessionEvent, + SessionHeader, + SessionId, + SessionPreparation, +} from '@deepseek-ai/dsh-session' import { - DEFAULT_PREPARED_SESSION_CACHE_SIZE, DEFAULT_WRITE_BATCH_MAX_DELAY_MS, MAX_WRITE_BATCH_DELAY_MS, - SessionPersistence, SessionPersistenceRevision, PersistenceCoordinator, - type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot, - type SessionInspection, type SessionPersistenceRevision as PersistenceRevision, - type StoredPrefix, type StoredSuffix, + DEFAULT_PREPARED_SESSION_CACHE_SIZE, + DEFAULT_WRITE_BATCH_MAX_DELAY_MS, + MAX_WRITE_BATCH_DELAY_MS, + PersistenceCoordinator, + SessionPersistence, + type SessionInspection, + type SessionLocation, + type SessionPersistenceSnapshot, } from '@deepseek-ai/dsh-session-persistence' -import type { SessionEvent, SurfaceEventType, SessionId, SessionHeader, SessionPreparation } from '@deepseek-ai/dsh-session' -import { - type JournalMode, openDatabase, rowToMeta, scanRows, type EventRow, type SessionRow, -} from './schema.ts' +import type { JournalMode } from './schema.ts' +import { SqliteStore } from './store.ts' export { SCHEMA_VERSION } from './schema.ts' -/** - * Serialize an event's optional envelope fields for SQL binding. The surface - * fields are nullable TEXT columns — null when the event has no surface - * metadata (non-surface events, events written before surface support); the - * ignorable marker is a nullable INTEGER column — `1` iff the envelope carries - * `ignorable: true`. - */ -function envelopeBindings(event: SessionEvent): [string | null, string | null, number | null] { - const se = event as SessionEvent - return [ - se.sourceEventSeqs ? JSON.stringify(se.sourceEventSeqs) : null, - se.surfaceOp !== undefined ? JSON.stringify(se.surfaceOp) : null, - event.ignorable === true ? 1 : null, - ] -} - -/** Build the source-qualified revision shared by full and lightweight reads. */ -function sqliteRevision(storeIdentity: string, row: SessionRow): PersistenceRevision { - return SessionPersistenceRevision( - `${storeIdentity}:incarnation:${row.incarnation}:revision:${row.revision}`, - ) -} - -/** - * Exclusively create a missing database file with owner-only permissions. - * Existing files retain their modes, and errors other than `EEXIST` propagate. - * `DatabaseSync` reopens by path, so this does not protect confidentiality or - * integrity when another principal can replace the database entry in its parent - * directory. - */ -async function createDatabaseFile(path: string): Promise { - try { - const handle = await open(path, 'wx', 0o600) - await handle.close() - } catch (error) { - if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error - } -} +/** Default wait for another SQLite connection's write reservation. */ +export const DEFAULT_BUSY_TIMEOUT_MS = 5_000 +/** Largest busy timeout accepted by SQLite's signed millisecond interface. */ +export const MAX_BUSY_TIMEOUT_MS = 2_147_483_647 /** Plugin configuration. */ export interface Config { - /** - * Filesystem path to the SQLite database file. The special value `:memory:` - * opens an in-process database (tests). On filesystems with POSIX modes, - * missing directories and databases are created owner-only; existing path - * modes are preserved. Filesystem setup errors other than an existing database - * fail initialization. The backend does not protect confidentiality or - * integrity when another principal can replace the database entry in its - * parent directory. - */ + /** SQLite database path, or `:memory:` for an in-process database. */ path: string - /** - * SQLite `journal_mode` pragma. `wal` (the default) is the recorded - * durability model; pick a rollback-journal mode (`delete`/`truncate`/ - * `persist`) on filesystems where WAL's shared-memory files do not work - * (network mounts). See {@link JournalMode}. - */ + /** Durable SQLite journal mode; defaults to `wal`. */ journalMode?: JournalMode + /** Maximum wait for another SQLite connection's lock; defaults to 5,000 ms. */ + busyTimeoutMs?: number /** Maximum cold Session preparations retained for history-to-resume reuse. */ preparedSessionCacheSize?: number /** Fixed live-event coalescing window; not a backend completion deadline. */ @@ -92,84 +48,49 @@ export interface Config { } /** - * The SQLite persistence backend. Load as a plugin; it registers as - * `ctx.sessionPersistence` and (via the coordinator) installs the write-path - * listeners. Its torn-tail marker is the seq to delete from. + * SQLite `SessionPersistence` provider with a schema-owned physical codec. */ -export class SqliteSessionPersistence extends SessionPersistence implements PersistenceBackend { +export class SqliteSessionPersistence extends SessionPersistence { override readonly supportsRawArtifacts = false + override readonly name = 'session-persistence-sqlite' static inject = ['sessions'] static Config: z = z.object({ path: z.string().required(), journalMode: z.union(['wal', 'delete', 'truncate', 'persist'] as const).default('wal'), + busyTimeoutMs: z.number().step(1).min(0).max(MAX_BUSY_TIMEOUT_MS).default(DEFAULT_BUSY_TIMEOUT_MS), preparedSessionCacheSize: z.number().step(1).min(1).default(DEFAULT_PREPARED_SESSION_CACHE_SIZE), writeBatchMaxDelayMs: z.number().step(1).min(1).max(MAX_WRITE_BATCH_DELAY_MS) .default(DEFAULT_WRITE_BATCH_MAX_DELAY_MS), }) - /** - * Backend label for the coordinator's dispose diagnostics. Intentionally - * shadows cordis `Service.name` (set to `'sessionPersistence'` by the base); - * see the JSONL backend for why this does not affect service resolution. - */ - override readonly name = 'session-persistence-sqlite' - - private db!: DatabaseSync - private storeIdentity!: string - private ready: Promise - private coordinator: PersistenceCoordinator + private readonly store: SqliteStore + private readonly coordinator: PersistenceCoordinator constructor(ctx: Context, public config: Config) { super(ctx) - // Programmatic wrappers may construct the backend without Schemastery normalization. const preparedSessionCacheSize = config.preparedSessionCacheSize ?? DEFAULT_PREPARED_SESSION_CACHE_SIZE const writeBatchMaxDelayMs = config.writeBatchMaxDelayMs ?? DEFAULT_WRITE_BATCH_MAX_DELAY_MS - // Open asynchronously so directory creation does not block plugin apply; - // every storage hook awaits the same readiness promise. - this.ready = this.openDb(config.path, (config as Required).journalMode) - this.coordinator = new PersistenceCoordinator(this.ctx, this, { + this.store = new SqliteStore({ + path: config.path, + journalMode: config.journalMode ?? 'wal', + busyTimeoutMs: config.busyTimeoutMs ?? DEFAULT_BUSY_TIMEOUT_MS, + }) + this.coordinator = new PersistenceCoordinator(this.ctx, this.store, { preparedSessionCacheSize, writeBatchMaxDelayMs, }) } - private async openDb(path: string, journalMode: JournalMode): Promise { - const actual = path === ':memory:' ? path : resolve(path) - if (actual !== ':memory:') { - await mkdir(dirname(actual), { recursive: true, mode: 0o700 }) - await createDatabaseFile(actual) - } - this.db = openDatabase(actual, journalMode) - try { - const row = this.db.prepare( - 'SELECT store_id FROM persistence_state WHERE singleton = 1', - ).get() as { store_id: string } | undefined - /* v8 ignore next -- openDatabase inserts the singleton before returning. */ - if (row === undefined) { - throw new Error(`session database at "${actual}" has no store identity`) - } - if (row.store_id.length === 0) { - throw new Error(`session database at "${actual}" has no valid store identity`) - } - if (actual !== ':memory:') { - const identity = statSync(actual, { bigint: true }) - this.storeIdentity = `file:${identity.dev}:${identity.ino}:${identity.birthtimeNs}:store:${row.store_id}` - } else { - this.storeIdentity = `memory:store:${row.store_id}` - } - } catch (error: unknown) { - this.db.close() - throw error - } + /** Reject self-contained path and ownership failures without loading Node SQLite. */ + protected async [Service.init](): Promise { + await this.store.validatePath() } - // --- SessionPersistence service API (delegated to the coordinator) --- - - /** SQLite has one database, not an independent local artifact per session. */ + /** SQLite has one database, not an independent per-session artifact. */ locate(_meta: SessionHeader): SessionLocation | undefined { return undefined } @@ -178,6 +99,10 @@ export class SqliteSessionPersistence extends SessionPersistence implements Pers return this.coordinator.create(meta) } + override ensureMaterialized(session: Session): Promise { + return this.coordinator.ensureMaterialized(session) + } + append(id: SessionId, events: readonly SessionEvent[]): Promise { return this.coordinator.append(id, events) } @@ -194,220 +119,20 @@ export class SqliteSessionPersistence extends SessionPersistence implements Pers return this.coordinator.inspect(id, signal) } - readFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + readFrom( + id: SessionId, + fromSeq: number, + signal?: AbortSignal, + ): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { return this.coordinator.readFrom(id, fromSeq, signal) } - // One method serves both public `list` and the backend hook; delegating it to - // the coordinator would call this hook recursively. - - // --- PersistenceBackend hooks (the SQLite storage primitives) --- - - /** Read a stored prefix by id (ids are globally unique — no scope to scan). */ - loadStored(id: SessionId, signal?: AbortSignal): Promise | undefined> { - return this.readPrefix(id, signal) + list(signal?: AbortSignal): Promise { + return this.store.list(signal) } - /** Read one row's revision without loading its events. */ - async readStoredRevision(id: SessionId, signal?: AbortSignal): Promise { - signal?.throwIfAborted() - await this.ready - signal?.throwIfAborted() - const row = this.rowFor(id) - return row === undefined ? undefined : sqliteRevision(this.storeIdentity, row) - } - - /** - * Seek-capable suffix read: SQL selects `seq >= fromSeq` directly, so the - * read scales with the suffix, not the log. Torn rows past the preserved - * region are dropped, never repaired (non-mutating read). - */ - async loadStoredFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise { - signal?.throwIfAborted() - await this.ready - signal?.throwIfAborted() - const row = this.rowFor(id) - if (row === undefined) return undefined - const meta = rowToMeta(row) - const eventRows = this.db - .prepare('SELECT seq, type, time, data, source_event_seqs, surface_op, ignorable FROM events WHERE session_id = ? AND seq >= ? ORDER BY seq') - .all(id, fromSeq) as unknown as EventRow[] - signal?.throwIfAborted() - const { preserved } = scanRows(eventRows, fromSeq) - return { meta, events: preserved } - } - - /** - * Read a session's row + ordered events into a {@link StoredPrefix}. The - * torn-tail marker is the seq from which a never-committed tail must be deleted - * (`scanRows` already returns it as `number | undefined`). - */ - private async readPrefix(id: SessionId, signal?: AbortSignal): Promise | undefined> { - signal?.throwIfAborted() - await this.ready - signal?.throwIfAborted() - this.db.exec('BEGIN') - let snapshot: { row: SessionRow; eventRows: EventRow[] } | undefined - try { - const row = this.rowFor(id) - if (row !== undefined) { - const eventRows = this.db - .prepare('SELECT seq, type, time, data, source_event_seqs, surface_op, ignorable FROM events WHERE session_id = ? ORDER BY seq') - .all(id) as unknown as EventRow[] - snapshot = { row, eventRows } - } - this.db.exec('COMMIT') - } catch (error: unknown) { - /* v8 ignore start -- synchronous read failures only need transaction cleanup before propagation. */ - this.db.exec('ROLLBACK') - throw error - /* v8 ignore stop */ - } - signal?.throwIfAborted() - if (snapshot === undefined) return undefined - const { row, eventRows } = snapshot - const { preserved, tornFrom } = scanRows(eventRows) - return { - meta: rowToMeta(row), - events: preserved, - revision: sqliteRevision(this.storeIdentity, row), - ...tornFrom !== undefined ? { tornMarker: tornFrom } : {}, - } - } - - /** - * Durably append a batch in ONE transaction: materialize the sessions row (if - * lazy) and INSERT every event, or roll back entirely. The transaction is the - * atomicity + durability boundary, so a mid-batch failure (a UNIQUE violation - * on a duplicated seq) leaves the stored log untouched. - */ - async appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise { - await this.ready - const insertEvent = this.db.prepare( - 'INSERT INTO events (session_id, seq, type, time, data, source_event_seqs, surface_op, ignorable) VALUES (?, ?, ?, ?, ?, ?, ?, ?)', - ) - this.db.exec('BEGIN') - try { - if (!isMaterialized) this.writeRow(meta) - for (const event of events) { - const [surfaceSeqs, surfaceOp, ignorable] = envelopeBindings(event) - insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data), surfaceSeqs, surfaceOp, ignorable) - } - this.db.prepare('UPDATE sessions SET revision = revision + 1 WHERE id = ?').run(meta.id) - this.db.exec('COMMIT') - } catch (error) { - this.db.exec('ROLLBACK') - throw error - } - } - - /** - * Make a crash repair durable in ONE transaction: DELETE the torn tail (from - * `tornMarker`) and INSERT the synthetic `closers`. After COMMIT the stored rows - * == the balanced log. - */ - async commitRepair(meta: SessionHeader, tornMarker: number | undefined, closers: readonly SessionEvent[]): Promise { - await this.ready - this.db.exec('BEGIN') - try { - if (tornMarker !== undefined) { - this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(meta.id, tornMarker) - } - if (closers.length > 0) { - const insertEvent = this.db.prepare( - 'INSERT INTO events (session_id, seq, type, time, data, source_event_seqs, surface_op, ignorable) VALUES (?, ?, ?, ?, ?, ?, ?, ?)', - ) - for (const event of closers) { - const [surfaceSeqs, surfaceOp, ignorable] = envelopeBindings(event) - insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data), surfaceSeqs, surfaceOp, ignorable) - } - } - if (tornMarker !== undefined || closers.length > 0) { - this.db.prepare('UPDATE sessions SET revision = revision + 1 WHERE id = ?').run(meta.id) - } - this.db.exec('COMMIT') - } catch (error) { - // The DELETE+INSERT cannot collide (a row at a closer's seq is preserved or - // deleted as torn first); this rolls back a DB-level failure (disk full, - // etc.), unreachable in test. - /* v8 ignore start */ - this.db.exec('ROLLBACK') - throw error - /* v8 ignore stop */ - } - } - - /** List all materialized sessions' metadata (every row is a materialized session). */ - async list(signal?: AbortSignal): Promise { - signal?.throwIfAborted() - await this.ready - signal?.throwIfAborted() - const rows = this.db - .prepare('SELECT * FROM sessions') - .all() as unknown as SessionRow[] - signal?.throwIfAborted() - return rows.map(rowToMeta) - } - - /** List metadata with a source-qualified monotonic revision per session. */ - async listSnapshots(signal?: AbortSignal): Promise { - signal?.throwIfAborted() - await this.ready - signal?.throwIfAborted() - const rows = this.db.prepare('SELECT * FROM sessions').all() as unknown as SessionRow[] - signal?.throwIfAborted() - return rows.map(row => ({ - header: rowToMeta(row), - revision: SessionPersistenceRevision( - `${this.storeIdentity}:incarnation:${row.incarnation}:revision:${row.revision}`, - ), - })) - } - - /** Close the database handle (awaited by the coordinator's dispose, post-drain). */ - async close(): Promise { - await this.ready - this.db.close() - } - - // --- row helpers --- - - /** Fetch a session's row, or undefined if absent. */ - private rowFor(id: SessionId): SessionRow | undefined { - return this.db.prepare('SELECT * FROM sessions WHERE id = ?').get(id) as unknown as SessionRow | undefined - } - - /** - * Insert-or-replace a session's metadata row. The only caller is the first - * materializing `appendBatch`, so writing the row IS the materialization (its - * existence is the signal `list` reads). - */ - private writeRow(meta: SessionHeader): void { - this.db.prepare(` - INSERT INTO sessions - (id, version, created_at, cwd, parent_session, seed_length, origin, delegation_depth, agent_preset, incarnation, revision) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0) - ON CONFLICT(id) DO UPDATE SET - version = excluded.version, - created_at = excluded.created_at, - cwd = excluded.cwd, - parent_session = excluded.parent_session, - seed_length = excluded.seed_length, - origin = excluded.origin, - delegation_depth = excluded.delegation_depth, - agent_preset = excluded.agent_preset - `).run( - meta.id, - meta.version, - meta.createdAt, - meta.cwd ?? null, - meta.parentSession ?? null, - meta.seedLength ?? null, - meta.origin ?? null, - meta.delegationDepth ?? null, - meta.agentPreset ?? null, - randomUUID(), - ) + listSnapshots(signal?: AbortSignal): Promise { + return this.store.listSnapshots(signal) } } diff --git a/packages/session/session-persistence-sqlite/src/invariant.ts b/packages/session/session-persistence-sqlite/src/invariant.ts index 7a5e905e30..8436d5fbe5 100644 --- a/packages/session/session-persistence-sqlite/src/invariant.ts +++ b/packages/session/session-persistence-sqlite/src/invariant.ts @@ -15,8 +15,8 @@ export const name = 'session-persistence-sqlite-invariant' export const inject = ['invariants'] /** - * No runtime invariant: persistence correctness requires backend round-trip and crash-tail tests; - * this package exposes no continuously observable in-process relation. + * No runtime invariant: physical packing is observable only by database + * round-trip and row-count checks, not a continuous in-process relation. */ const install: InvariantInstaller = () => {} diff --git a/packages/session/session-persistence-sqlite/src/schema.ts b/packages/session/session-persistence-sqlite/src/schema.ts index c7402d7d4b..39fc00504b 100644 --- a/packages/session/session-persistence-sqlite/src/schema.ts +++ b/packages/session/session-persistence-sqlite/src/schema.ts @@ -1,87 +1,86 @@ /** - * Schema + load-time helpers for the SQLite session-persistence backend: the - * DDL (a store-identity row, `sessions` metadata, and a 1:1 `events` row per - * `SessionEvent`), the database open/configure step, and the last-`turn/end` - * cut that gives the SQLite backend the SAME crash-tail-on-load semantics as - * the JSONL backend. - * - * @module dsh-session-persistence-sqlite/schema + * SQLite schema ownership and durable-row validation. + * @module @deepseek-ai/dsh-session-persistence-sqlite/schema */ import { randomUUID } from 'node:crypto' -import { DatabaseSync } from 'node:sqlite' -import type { SessionEvent, SessionId, SessionHeader, SurfaceOp } from '@deepseek-ai/dsh-session' +import { isAbsolute } from 'node:path' +import { performance } from 'node:perf_hooks' +import type { DatabaseSync } from 'node:sqlite' +import { setTimeout as delay } from 'node:timers/promises' +import { + SessionId, + type SessionHeader, +} from '@deepseek-ai/dsh-session' +import { sql } from './sql.ts' -/** - * The on-disk schema version. Bumped only on a breaking change to the table - * layout; orthogonal to a session's own `version` (which versions the EVENT - * vocabulary, stored per session in the `sessions` row). - */ -export const SCHEMA_VERSION = 15 - -/** SQLite application id protecting unrelated databases from persistence writes. */ +/** Current physical-record schema with packed and compressed event rows. */ +export const SCHEMA_VERSION = 17 +/** Application id reserved for DeepSeek Harness SQLite session databases. */ export const SESSION_PERSISTENCE_SQLITE_APPLICATION_ID = 0x44534850 -/** - * A row of the `sessions` table — the out-of-log metadata ({@link SessionHeader}). - * The row's EXISTENCE is the materialization signal: it is written only by the - * first `append` (lazy materialization), so a created-but-never-appended - * session has no row and is absent from `list`, mirroring the JSONL - * backend's "no file until first append". - */ +/** A materialized session's metadata and monotonic revision. */ export interface SessionRow { - id: string - version: number - created_at: number - cwd: string | null - parent_session: string | null - seed_length: number | null - origin: 'subagent' | null - /** Stable identity assigned when this log is materialized. */ - incarnation: string - /** Monotonic log-change token incremented in each mutating transaction. */ - revision: number - delegation_depth: number | null - agent_preset: string | null + readonly id: string + readonly version: number + readonly created_at: number + readonly cwd: string | null + readonly parent_session: string | null + readonly seed_length: number | null + readonly origin: 'subagent' | null + readonly incarnation: string + readonly revision: number + readonly delegation_depth: number | null + readonly agent_preset: string | null } -/** An `events` table row: one `SessionEvent` mapped 1:1 (`data` is JSON text). */ +/** One physical event row; packed rows may represent multiple logical events. */ export interface EventRow { - seq: number - type: string - time: number - data: string - /** JSON-encoded `number[]` — the event's sourceEventSeqs, or null. */ - source_event_seqs: string | null - /** JSON-encoded `SurfaceOp` — how the event entered the surface, or null. */ - surface_op: string | null - /** `1` iff the event carries the envelope's `ignorable: true` marker, else null. */ - ignorable: number | null + readonly seq: number + readonly type: string + readonly time: number + readonly data: string | Uint8Array + readonly source_event_seqs: Uint8Array | null + readonly surface_op: string | null + readonly ignorable: number | null } -/** - * Journal modes the backend will run under. `wal` is the default and the - * durability model the persistence ADR records; the rollback-journal modes - * (`delete`/`truncate`/`persist`) exist for filesystems where WAL's - * shared-memory files do not work (network mounts). `memory`/`off` are - * excluded: dropping journal durability silently contradicts what this - * backend promises. - */ +/** Durable journal modes accepted by the backend. */ export type JournalMode = 'wal' | 'delete' | 'truncate' | 'persist' +interface SchemaObjectRow { + readonly type: string + readonly name: string + readonly tbl_name: string + readonly sql: string +} + +const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/iu +const JOURNAL_BUSY_RETRY_INTERVAL_MS = 10 +type DatabaseSyncConstructor = typeof import('node:sqlite')['DatabaseSync'] + /** - * Open the database and apply its schema and pragmas. An empty database with a - * zero `user_version` is initialized at {@link SCHEMA_VERSION}; a nonempty - * unversioned database and every other non-current version reject rather than - * being migrated in place. - * @param path - the SQLite database file to open (created when absent). + * Open and validate a SQLite session database. + * @param Database - lazily imported Node SQLite constructor. + * @param path - SQLite path, including `:memory:`. * @param journalMode - validated journal pragma. - * @returns the open handle with pragmas applied and all three tables ensured. + * @param busyTimeoutMs - validated maximum wait for a competing SQLite lock. + * @returns the configured database handle. + * @throws when connection settings, schema ownership, or SQLite setup cannot be validated. */ -export function openDatabase(path: string, journalMode: JournalMode): DatabaseSync { - const db = new DatabaseSync(path) +export async function openDatabase( + Database: DatabaseSyncConstructor, + path: string, + journalMode: JournalMode, + busyTimeoutMs: number, +): Promise { + const deadline = performance.now() + busyTimeoutMs + const db = new Database(path, { timeout: busyTimeoutMs }) try { - configureDatabase(db, path, journalMode) + configureConnectionSecurity(db, path) + configureDatabase(Database, db, path) + await selectJournalMode(db, path, journalMode, deadline) + configureDurability(db, path) return db } catch (error: unknown) { db.close() @@ -89,182 +88,337 @@ export function openDatabase(path: string, journalMode: JournalMode): DatabaseSy } } -function configureDatabase(db: DatabaseSync, path: string, journalMode: JournalMode): void { - db.exec('PRAGMA foreign_keys = ON') +function configureConnectionSecurity(db: DatabaseSync, path: string): void { + db.exec(sql('trusted-schema-off')) + const trustedSchema = integerField(db.prepare(sql('select-trusted-schema')).get(), 'trusted_schema') + /* v8 ignore next 3 -- supported SQLite versions return the fixed setting. */ + if (trustedSchema !== 0) { + throw new Error(`session database at "${path}" retained trusted_schema=${trustedSchema}, expected 0`) + } + db.exec(sql('mmap-off')) + if (path === ':memory:') return + const mmapSize = integerField(db.prepare(sql('select-mmap-size')).get(), 'mmap_size') + /* v8 ignore next 3 -- supported file-backed SQLite connections return the fixed setting. */ + if (mmapSize !== 0) { + throw new Error(`session database at "${path}" retained mmap_size=${mmapSize}, expected 0`) + } +} + +function configureDatabase( + Database: DatabaseSyncConstructor, + db: DatabaseSync, + path: string, +): void { + db.exec(sql('foreign-keys-on')) let began = false try { - db.exec('BEGIN IMMEDIATE') + db.exec(sql('begin-immediate')) began = true - // Validate while holding the write lock so no other connection can change - // schema ownership between inspection and initialization. - const { user_version: onDisk } = db.prepare('PRAGMA user_version').get() as { user_version: number } - const { application_id: applicationId } = db.prepare('PRAGMA application_id').get() as { application_id: number } - const { count: userObjectCount } = db.prepare( - "SELECT COUNT(*) AS count FROM sqlite_schema WHERE name NOT GLOB 'sqlite_*'", - ).get() as { count: number } + const onDisk = integerField(db.prepare(sql('select-user-version')).get(), 'user_version') + const applicationId = integerField(db.prepare(sql('select-application-id')).get(), 'application_id') + const userObjectCount = integerField(db.prepare(sql('select-user-object-count')).get(), 'count') if (onDisk === 0 && (applicationId !== 0 || userObjectCount > 0)) { throw new Error(`session database at "${path}" has an unversioned schema or application identity`) } if (onDisk !== 0 && onDisk !== SCHEMA_VERSION) { - throw new Error(`session database at "${path}" has schema version ${onDisk}, incompatible with this build (${SCHEMA_VERSION})`) + throw new Error( + `session database at "${path}" has schema version ${onDisk}, incompatible with this build (${SCHEMA_VERSION})`, + ) } - if (onDisk === SCHEMA_VERSION && applicationId !== SESSION_PERSISTENCE_SQLITE_APPLICATION_ID) { + if (onDisk !== 0 && applicationId !== SESSION_PERSISTENCE_SQLITE_APPLICATION_ID) { throw new Error( `session database at "${path}" has application id ${applicationId}, expected ${SESSION_PERSISTENCE_SQLITE_APPLICATION_ID}`, ) } - db.exec(` - CREATE TABLE IF NOT EXISTS persistence_state ( - singleton INTEGER PRIMARY KEY CHECK (singleton = 1), - store_id TEXT NOT NULL - ) STRICT; - - CREATE TABLE IF NOT EXISTS sessions ( - id TEXT PRIMARY KEY, - version INTEGER NOT NULL, - created_at INTEGER NOT NULL, - cwd TEXT, - parent_session TEXT, - seed_length INTEGER, - origin TEXT, - delegation_depth INTEGER, - agent_preset TEXT, - incarnation TEXT NOT NULL, - revision INTEGER NOT NULL - ) STRICT; - - CREATE TABLE IF NOT EXISTS events ( - session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE, - seq INTEGER NOT NULL, - type TEXT NOT NULL, - time INTEGER NOT NULL, - data TEXT NOT NULL, - source_event_seqs TEXT, - surface_op TEXT, - ignorable INTEGER, - PRIMARY KEY (session_id, seq) - ) STRICT - `) - db.prepare( - 'INSERT OR IGNORE INTO persistence_state (singleton, store_id) VALUES (1, ?)', - ).run(randomUUID()) - if (onDisk === 0) { - db.exec(`PRAGMA application_id = ${SESSION_PERSISTENCE_SQLITE_APPLICATION_ID}`) - db.exec(`PRAGMA user_version = ${SCHEMA_VERSION}`) - } - db.exec('COMMIT') + if (onDisk === 0) initializeDatabase(db) + validateRequiredSchema(Database, db, path) + db.exec(sql('commit')) began = false } catch (error: unknown) { - /* v8 ignore next -- a BEGIN failure leaves no transaction to roll back. */ + /* v8 ignore else -- a failed begin leaves no transaction to roll back. */ if (began) { - /* v8 ignore next 5 -- preserve the original schema failure if SQLite also refuses rollback. */ + /* v8 ignore next 5 -- retain the original ownership failure if rollback fails too. */ try { - db.exec('ROLLBACK') + db.exec(sql('rollback')) } catch { - // The original SQLite failure remains the actionable cause. + // The original database-ownership failure remains actionable. } } throw error } - // The validated union is safe to interpolate into a non-bindable PRAGMA. - // Apply it only after ownership validation and initialization commit. - db.exec(`PRAGMA journal_mode = ${journalMode.toUpperCase()}`) } -/** - * Reconstruct the {@link SessionHeader} from a `sessions` row. - * @param row - the `sessions` table row. - * @returns the header, `NULL` columns mapped to omitted optional fields. - */ -export function rowToMeta(row: SessionRow): SessionHeader { - if (!Number.isSafeInteger(row.created_at) || row.created_at < 0) { - throw new Error('stored session createdAt must be a non-negative safe integer') - } - return { - version: row.version, - id: row.id as SessionId, - createdAt: row.created_at, - ...row.cwd !== null ? { cwd: row.cwd } : {}, - ...row.parent_session !== null ? { parentSession: row.parent_session as SessionId } : {}, - ...row.seed_length !== null ? { seedLength: row.seed_length } : {}, - ...row.origin !== null ? { origin: row.origin } : {}, - ...row.delegation_depth !== null ? { delegationDepth: row.delegation_depth } : {}, - ...row.agent_preset !== null ? { agentPreset: row.agent_preset } : {}, - } -} - -/** - * Reconstruct a {@link SessionEvent} from an `events` row (parses `data`). - * @param row - the `events` table row; `data` and the surface columns hold JSON text. - * @returns the reconstructed event; throws when a JSON column fails to parse - * ({@link scanRows} treats that as a hole, not corruption, in the tail). - */ -export function rowToEvent(row: EventRow): SessionEvent { - // Surface-metadata fields are conditional on the event type in the type - // system; spread them so each variant gets only the fields it declares. - const surfaceFields = { - ...row.source_event_seqs !== null ? { sourceEventSeqs: JSON.parse(row.source_event_seqs) as number[] } : {}, - ...row.surface_op !== null ? { surfaceOp: JSON.parse(row.surface_op) as SurfaceOp } : {}, - } - const ignorableField = row.ignorable === 1 ? { ignorable: true as const } : {} - return { - type: row.type as SessionEvent['type'], - seq: row.seq, - time: row.time, - data: JSON.parse(row.data) as SessionEvent['data'], - ...surfaceFields, - ...ignorableField, - } as SessionEvent -} - -/** - * Find the preserved prefix of ordered event rows. Fully written rows in an - * interrupted final turn remain in the prefix. The first unparsable row or seq - * gap after the last `turn/end` marks a tolerated torn tail; the same hole in - * the committed region rejects. - * - * @param rows - one session's event rows, ordered by seq ascending. - * @param base - the seq the first row is expected to carry; `0` for a whole - * log, the requested `fromSeq` for a suffix read (`loadStoredFrom`). - * @returns the preserved event prefix, plus `tornFrom` — the seq the physical - * delete starts at — when a torn tail exists. - */ -export function scanRows(rows: readonly EventRow[], base = 0): { preserved: SessionEvent[]; tornFrom?: number } { - // Pass 1: parse each row's data; a row whose data is not valid JSON is a hole. - // (The seq/type COLUMNS are always present even when `data` is corrupt.) - interface Parsed { ok: boolean; event?: SessionEvent } - const parsed: Parsed[] = rows.map((row) => { +async function selectJournalMode( + db: DatabaseSync, + path: string, + journalMode: JournalMode, + deadline: number, +): Promise { + let result: unknown + while (true) { try { - return { ok: true, event: rowToEvent(row) } - } catch { - return { ok: false } + result = db.prepare(sql(journalResource(journalMode))).get() + break + } catch (error: unknown) { + const remainingMs = Math.max(0, Math.ceil(deadline - performance.now())) + if (!isSqliteBusy(error) || remainingMs === 0) throw error + await delay(Math.min(JOURNAL_BUSY_RETRY_INTERVAL_MS, remainingMs)) + if (performance.now() >= deadline) throw error + } + } + const selected = stringField(result, 'journal_mode').toLowerCase() + const expected = path === ':memory:' ? 'memory' : journalMode + /* v8 ignore next 3 -- SQLite returns the selected mode from these fixed, valid pragmas. */ + if (selected !== expected) { + throw new Error(`session database at "${path}" selected journal mode ${selected}, expected ${expected}`) + } +} + +function configureDurability(db: DatabaseSync, path: string): void { + db.exec(sql('synchronous-full')) + const synchronous = integerField(db.prepare(sql('select-synchronous')).get(), 'synchronous') + /* v8 ignore next 3 -- supported SQLite versions return the fixed setting. */ + if (synchronous !== 2) { + throw new Error(`session database at "${path}" retained synchronous=${synchronous}, expected FULL (2)`) + } +} + +function isSqliteBusy(error: unknown): boolean { + return typeof error === 'object' + && error !== null + && Reflect.get(error, 'errcode') === 5 +} + +function journalResource(mode: JournalMode): + | 'journal-mode-wal' + | 'journal-mode-delete' + | 'journal-mode-truncate' + | 'journal-mode-persist' { + switch (mode) { + case 'wal': return 'journal-mode-wal' + case 'delete': return 'journal-mode-delete' + case 'truncate': return 'journal-mode-truncate' + case 'persist': return 'journal-mode-persist' + } +} + +function initializeDatabase(db: DatabaseSync): void { + db.exec(sql('schema')) + db.prepare(sql('insert-persistence-state')).run(randomUUID()) + db.exec(sql('set-application-id')) + db.exec(sql('set-user-version-17')) +} + +let canonicalSchema: readonly SchemaObjectRow[] | undefined + +function expectedSchema(Database: DatabaseSyncConstructor): readonly SchemaObjectRow[] { + if (canonicalSchema !== undefined) return canonicalSchema + const reference = new Database(':memory:') + try { + reference.exec(sql('foreign-keys-on')) + reference.exec(sql('schema')) + canonicalSchema = schemaObjects(reference) + return canonicalSchema + } finally { + reference.close() + } +} + +function schemaObjects(db: DatabaseSync): SchemaObjectRow[] { + return db.prepare(sql('select-schema-objects')).all().map((value) => { + const row = record(value, 'schema object') + return { + type: stringField(row, 'type'), + name: stringField(row, 'name'), + tbl_name: stringField(row, 'tbl_name'), + sql: normalizeSql(stringField(row, 'sql')), } }) +} - // The last index that is a valid `turn/end` — holes through a closed turn - // are always committed corruption. - let lastTurnEnd = -1 - for (let i = parsed.length - 1; i >= 0; i--) { - if (parsed[i]?.ok && rows[i]?.type === 'turn/end') { lastTurnEnd = i; break } +function normalizeSql(value: string): string { + return value.replaceAll(/\s+/gu, ' ').trim() +} + +function validateRequiredSchema( + Database: DatabaseSyncConstructor, + db: DatabaseSync, + path: string, +): void { + if (JSON.stringify(schemaObjects(db)) !== JSON.stringify(expectedSchema(Database))) { + throw new Error(`session database at "${path}" does not contain the required schema objects`) + } +} + +/** + * Recheck schema ownership inside the caller's mutation transaction. + * @param Database - constructor used to validate the canonical schema. + * @param db - open owned database with an active immediate transaction. + * @param path - database location used in ownership diagnostics. + * @throws when another writer changed the application identity, schema, or version. + */ +export function validateSchemaForMutation( + Database: DatabaseSyncConstructor, + db: DatabaseSync, + path: string, +): void { + const version = integerField(db.prepare(sql('select-user-version')).get(), 'user_version') + const applicationId = integerField(db.prepare(sql('select-application-id')).get(), 'application_id') + if (applicationId !== SESSION_PERSISTENCE_SQLITE_APPLICATION_ID) { + throw new Error( + `session database application id changed before mutation (expected ${SESSION_PERSISTENCE_SQLITE_APPLICATION_ID}, got ${applicationId})`, + ) + } + validateRequiredSchema(Database, db, path) + if (version !== SCHEMA_VERSION) { + throw new Error(`session database schema changed before mutation (expected ${SCHEMA_VERSION}, got ${version})`) + } +} + +/** + * Decode and validate one durable session row. + * @param value - value returned by SQLite. + * @returns a validated session row. + */ +export function decodeSessionRow(value: unknown): SessionRow { + const row = record(value, 'stored session metadata') + const id = nonemptyStringField(row, 'id') + const version = safeIntegerField(row, 'version') + const cwd = nullableStringField(row, 'cwd') + if (cwd !== null && !isAbsolute(cwd)) throw new Error('stored session cwd must be absolute') + const parent = nullableStringField(row, 'parent_session') + const origin = nullableStringField(row, 'origin') + if (origin !== null && origin !== 'subagent') throw new Error('stored session origin must be subagent or null') + const incarnation = nonemptyStringField(row, 'incarnation') + if (!UUID.test(incarnation)) throw new Error('stored session incarnation must be a UUID') + return { + id, + version, + created_at: nonnegativeSafeIntegerField(row, 'created_at'), + cwd, + parent_session: parent, + seed_length: nullableNonnegativeSafeIntegerField(row, 'seed_length'), + origin, + delegation_depth: nullableNonnegativeSafeIntegerField(row, 'delegation_depth'), + agent_preset: nullableStringField(row, 'agent_preset'), + incarnation, + revision: nonnegativeSafeIntegerField(row, 'revision'), + } +} + +/** + * Decode and validate one durable event row before JSON interpretation. + * @param value - value returned by SQLite. + * @returns a validated physical event row. + */ +export function decodeEventRow(value: unknown): EventRow { + const row = record(value, 'stored event') + const ignorable = nullableSafeIntegerField(row, 'ignorable') + if (ignorable !== null && ignorable !== 0 && ignorable !== 1) { + throw new Error('stored event ignorable must be 0, 1, or null') } + return { + seq: nonnegativeSafeIntegerField(row, 'seq'), + type: nonemptyStringField(row, 'type'), + time: safeIntegerField(row, 'time'), + data: stringOrBlobField(row, 'data'), + source_event_seqs: nullableBlobField(row, 'source_event_seqs'), + surface_op: nullableStringField(row, 'surface_op'), + ignorable, + } +} + +/** + * Validate the singleton identity read from durable storage. + * @param value - value returned by SQLite. + * @returns the UUID store identity. + */ +export function decodeStoreIdentity(value: unknown): string { + const identity = nonemptyStringField(value, 'store_id') + if (!UUID.test(identity)) throw new Error('stored store_id must be a UUID') + return identity +} - // Preserve the contiguous prefix, including a complete interrupted turn; - // holes through the last committed boundary throw, while later holes stop. - const preserved: SessionEvent[] = [] - for (let i = 0; i < rows.length; i++) { - const p = parsed[i] - if (!p?.ok || p.event === undefined) { - if (i <= lastTurnEnd) throw new Error(`corrupt session log: unparsable committed event at seq ${rows[i]?.seq}`) - break // torn tail fragment after the last turn/end — stop, tolerate - } - if (p.event.seq !== base + i) { - if (i <= lastTurnEnd) throw new Error(`corrupt session log: seq gap in committed region (expected ${base + i}, got ${p.event.seq})`) - break // gap after the last turn/end — torn tail, stop - } - preserved.push(p.event) +/** + * Reconstruct an immutable session header from a validated metadata row. + * @param row - validated stored metadata row. + * @returns the session header. + */ +export function rowToMeta(row: SessionRow): SessionHeader { + return { + version: row.version, + id: SessionId(row.id), + createdAt: row.created_at, + ...row.cwd === null ? {} : { cwd: row.cwd }, + ...row.parent_session === null ? {} : { parentSession: SessionId(row.parent_session) }, + ...row.seed_length === null ? {} : { seedLength: row.seed_length }, + ...row.origin === null ? {} : { origin: row.origin }, + ...row.delegation_depth === null ? {} : { delegationDepth: row.delegation_depth }, + ...row.agent_preset === null ? {} : { agentPreset: row.agent_preset }, } +} + +function record(value: unknown, label: string): Record { + if (typeof value !== 'object' || value === null) throw new Error(`${label} must be an object`) + return value as Record +} + +function stringField(value: unknown, key: string): string { + const field = record(value, 'SQLite row')[key] + if (typeof field !== 'string') throw new Error(`stored ${key} must be a string`) + return field +} + +function nonemptyStringField(value: unknown, key: string): string { + const field = stringField(value, key) + if (field.length === 0) throw new Error(`stored ${key} must not be empty`) + return field +} + +function nullableStringField(value: unknown, key: string): string | null { + const field = record(value, 'SQLite row')[key] + if (field === null) return null + if (typeof field !== 'string') throw new Error(`stored ${key} must be a string or null`) + return field +} + +function stringOrBlobField(value: unknown, key: string): string | Uint8Array { + const field = record(value, 'SQLite row')[key] + if (typeof field === 'string' || field instanceof Uint8Array) return field + throw new Error(`stored ${key} must be a string or blob`) +} + +function nullableBlobField(value: unknown, key: string): Uint8Array | null { + const field = record(value, 'SQLite row')[key] + if (field === null || field instanceof Uint8Array) return field + throw new Error(`stored ${key} must be a blob or null`) +} + +function integerField(value: unknown, key: string): number { + const field = record(value, 'SQLite row')[key] + if (!Number.isSafeInteger(field)) throw new Error(`stored ${key} must be a safe integer`) + return field as number +} + +function safeIntegerField(value: unknown, key: string): number { + return integerField(value, key) +} + +function nonnegativeSafeIntegerField(value: unknown, key: string): number { + const field = integerField(value, key) + if (field < 0) throw new Error(`stored ${key} must be non-negative`) + return field +} + +function nullableSafeIntegerField(value: unknown, key: string): number | null { + const field = record(value, 'SQLite row')[key] + if (field === null) return null + if (!Number.isSafeInteger(field)) throw new Error(`stored ${key} must be a safe integer or null`) + return field as number +} - // Any rows past the preserved prefix are a never-committed torn tail; their - // first seq is the deletion point for load's physical repair. - return preserved.length < rows.length ? { preserved, tornFrom: base + preserved.length } : { preserved } +function nullableNonnegativeSafeIntegerField(value: unknown, key: string): number | null { + const field = nullableSafeIntegerField(value, key) + if (field !== null && field < 0) throw new Error(`stored ${key} must be non-negative or null`) + return field } diff --git a/packages/session/session-persistence-sqlite/src/sql.ts b/packages/session/session-persistence-sqlite/src/sql.ts new file mode 100644 index 0000000000..5d14855723 --- /dev/null +++ b/packages/session/session-persistence-sqlite/src/sql.ts @@ -0,0 +1,65 @@ +/** + * Closed, package-owned SQL resource loading for SQLite. + * @module @deepseek-ai/dsh-session-persistence-sqlite/sql + */ + +import { readFileSync } from 'node:fs' +import { fileURLToPath } from 'node:url' + +const SQL_RESOURCES = [ + 'begin', + 'begin-immediate', + 'commit', + 'delete-events-from', + 'foreign-keys-on', + 'insert-event', + 'insert-persistence-state', + 'journal-mode-delete', + 'journal-mode-persist', + 'journal-mode-truncate', + 'journal-mode-wal', + 'mmap-off', + 'rollback', + 'schema', + 'select-application-id', + 'select-events', + 'select-events-from', + 'select-mmap-size', + 'select-packed-predecessors', + 'select-schema-objects', + 'select-session', + 'select-sessions', + 'select-store-id', + 'select-synchronous', + 'select-tail-events', + 'select-trusted-schema', + 'select-user-object-count', + 'select-user-version', + 'set-application-id', + 'set-user-version-17', + 'synchronous-full', + 'trusted-schema-off', + 'update-session-revision', + 'upsert-session', +] as const + +/** A resource basename selected exclusively by package code. */ +export type SqlResourceName = typeof SQL_RESOURCES[number] + +const cache = new Map() + +/** + * Load an immutable SQL statement by closed resource name. + * @param name - package-owned resource basename. + * @returns the resource text. + */ +export function sql(name: SqlResourceName): string { + const cached = cache.get(name) + if (cached !== undefined) return cached + const statement = readFileSync( + fileURLToPath(new URL(`../resources/sql/${name}.sql`, import.meta.url)), + 'utf8', + ) + cache.set(name, statement) + return statement +} diff --git a/packages/session/session-persistence-sqlite/src/store.ts b/packages/session/session-persistence-sqlite/src/store.ts new file mode 100644 index 0000000000..c28a9e2fa7 --- /dev/null +++ b/packages/session/session-persistence-sqlite/src/store.ts @@ -0,0 +1,483 @@ +/** + * SQLite storage primitives: transactional append-batch packing, physical + * reads, schema validation, revisions, repair, and lifecycle closure. + * @module @deepseek-ai/dsh-session-persistence-sqlite/store + */ + +import { randomUUID } from 'node:crypto' +import { statSync } from 'node:fs' +import { lstat, mkdir, open } from 'node:fs/promises' +import { dirname, resolve } from 'node:path' +import type { DatabaseSync, StatementSync } from 'node:sqlite' +import { + type SessionEvent, + type SessionHeader, + type SessionId, +} from '@deepseek-ai/dsh-session' +import { + SessionPersistenceRevision, + type PersistenceBackend, + type SessionPersistenceRevision as PersistenceRevision, + type SessionPersistenceSnapshot, + type StoredPrefix, + type StoredSuffix, +} from '@deepseek-ai/dsh-session-persistence' +import { + MAX_PACKED_ROW_MEMBERS, + packChunkRuns, +} from './codec.ts' +import { + bindRecord, + decodeRow, + scanRows, + type BoundRecord, +} from './compression.ts' +import { + type EventRow, + type JournalMode, + decodeEventRow, + decodeSessionRow, + decodeStoreIdentity, + openDatabase, + validateSchemaForMutation, + rowToMeta, + type SessionRow, +} from './schema.ts' +import { sql } from './sql.ts' + +/** Storage options resolved by the service provider. */ +export interface SqliteStoreOptions { + readonly path: string + readonly journalMode: JournalMode + readonly busyTimeoutMs: number +} + +/** SQLite implementation of the coordinator's physical backend hooks. */ +export class SqliteStore implements PersistenceBackend { + readonly name = 'session-persistence-sqlite' + private db!: DatabaseSync + private databaseConstructor!: typeof import('node:sqlite')['DatabaseSync'] + private storeIdentity!: string + private databasePath!: string + private opened = false + private pathReady: Promise | undefined + private ready: Promise | undefined + + constructor(private readonly options: SqliteStoreOptions) {} + + /** + * Validate filesystem ownership without importing or opening Node SQLite. + * @returns settlement of the store's one path-validation operation. + */ + validatePath(): Promise { + this.pathReady ??= this.preparePath(this.options.path) + return this.pathReady + } + + /** + * Lazily open and validate the database on first persistence use. + * @returns settlement of the store's one database-open operation. + */ + open(): Promise { + this.ready ??= this.openDb() + return this.ready + } + + private async preparePath(path: string): Promise { + const actual = path === ':memory:' ? path : resolve(path) + if (actual !== ':memory:') { + await mkdir(dirname(actual), { recursive: true, mode: 0o700 }) + await validateParentDirectory(dirname(actual)) + await validateDatabaseFileIfPresent(actual) + } + this.databasePath = actual + } + + private async openDb(): Promise { + await this.validatePath() + if (this.databasePath !== ':memory:') { + await createDatabaseFile(this.databasePath) + await validateDatabaseFile(this.databasePath) + } + const { DatabaseSync } = await loadNodeSqlite() + this.databaseConstructor = DatabaseSync + this.db = await openDatabase( + DatabaseSync, + this.databasePath, + this.options.journalMode, + this.options.busyTimeoutMs, + ) + try { + const row = this.db.prepare(sql('select-store-id')).get() + if (row === undefined) { + throw new Error(`session database at "${this.databasePath}" has no valid store identity`) + } + let storeId: string + try { + storeId = decodeStoreIdentity(row) + } catch (error: unknown) { + throw new Error(`session database at "${this.databasePath}" has no valid store identity`, { cause: error }) + } + if (this.databasePath === ':memory:') { + this.storeIdentity = `memory:store:${storeId}` + } else { + const identity = statSync(this.databasePath, { bigint: true }) + this.storeIdentity = `file:${identity.dev}:${identity.ino}:${identity.birthtimeNs}:store:${storeId}` + } + this.opened = true + } catch (error: unknown) { + this.db.close() + throw error + } + } + + async loadStored(id: SessionId, signal?: AbortSignal): Promise | undefined> { + await this.observe(signal) + const snapshot = this.readTransaction(() => { + const row = this.rowFor(id) + if (row === undefined) return undefined + const eventRows = this.db.prepare(sql('select-events')).all(id).map(decodeEventRow) + return { row, eventRows } + }) + signal?.throwIfAborted() + if (snapshot === undefined) return undefined + const scanned = scanRows(snapshot.eventRows) + return { + meta: rowToMeta(snapshot.row), + events: scanned.preserved, + revision: sqliteRevision(this.storeIdentity, snapshot.row), + ...scanned.tornFrom === undefined ? {} : { tornMarker: scanned.tornFrom }, + } + } + + async readStoredRevision(id: SessionId, signal?: AbortSignal): Promise { + await this.observe(signal) + const row = this.rowFor(id) + signal?.throwIfAborted() + return row === undefined ? undefined : sqliteRevision(this.storeIdentity, row) + } + + async loadStoredFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise { + await this.observe(signal) + const snapshot = this.readTransaction(() => { + const row = this.rowFor(id) + if (row === undefined) return undefined + return { row, ...this.physicalSpanFrom(id, fromSeq) } + }) + signal?.throwIfAborted() + if (snapshot === undefined) return undefined + const { preserved } = scanRows(snapshot.eventRows, snapshot.base) + return { meta: rowToMeta(snapshot.row), events: preserved.filter(event => event.seq >= fromSeq) } + } + + async appendBatch( + meta: SessionHeader, + events: readonly SessionEvent[], + isMaterialized: boolean, + ): Promise { + await this.open() + if (events.length === 0) return + this.db.exec(sql('begin-immediate')) + try { + validateSchemaForMutation(this.databaseConstructor, this.db, this.databasePath) + const tailRows = this.tailRows(meta.id) + const currentLast = this.logicalLastEvent(meta.id, tailRows) + const expected = currentLast === undefined ? 0 : currentLast.seq + 1 + const first = events[0] as SessionEvent + if (first.seq !== expected) { + throw new Error(`session ${meta.id} append starts at seq ${first.seq}, stored next seq is ${expected}`) + } + if (!isMaterialized) this.writeRow(meta) + + const insert = this.insertStatement() + for (const record of packChunkRuns(events)) this.insertRecord(insert, meta.id, bindRecord(record)) + this.incrementRevision(meta.id) + this.db.exec(sql('commit')) + } catch (error: unknown) { + this.rollback(error, 'append') + } + } + + async materializeHeader(meta: SessionHeader): Promise { + await this.open() + this.db.exec(sql('begin-immediate')) + try { + validateSchemaForMutation(this.databaseConstructor, this.db, this.databasePath) + this.writeRow(meta) + this.db.exec(sql('commit')) + } catch (error: unknown) { + /* v8 ignore next -- validate/write failure uses the same transaction rollback path covered by append and repair. */ + this.rollback(error, 'materialize empty session') + } + } + + async commitRepair( + meta: SessionHeader, + tornMarker: number | undefined, + closers: readonly SessionEvent[], + ): Promise { + await this.open() + if (tornMarker === undefined && closers.length === 0) return + this.db.exec(sql('begin-immediate')) + try { + validateSchemaForMutation(this.databaseConstructor, this.db, this.databasePath) + const row = this.rowFor(meta.id) + if (row === undefined) throw new Error(`session ${meta.id} metadata row is missing`) + const currentRows = this.db.prepare(sql('select-events')).all(meta.id).map(decodeEventRow) + const current = scanRows(currentRows) + if (tornMarker !== undefined) { + if (current.tornFrom !== tornMarker) { + throw new Error(`session ${meta.id} repair is stale: physical tail no longer starts at seq ${tornMarker}`) + } + this.db.prepare(sql('delete-events-from')) + .run(meta.id, tornMarker) + } else if (current.tornFrom !== undefined) { + throw new Error(`session ${meta.id} repair omitted current torn tail at seq ${current.tornFrom}`) + } + if (closers.length > 0) { + const expected = current.preserved.at(-1)?.seq === undefined + ? 0 + : (current.preserved.at(-1) as SessionEvent).seq + 1 + if (closers[0]?.seq !== expected) { + throw new Error(`session ${meta.id} repair is stale: closer starts at seq ${closers[0]?.seq}, stored next seq is ${expected}`) + } + const insert = this.insertStatement() + for (const closer of closers) this.insertRecord(insert, meta.id, bindRecord(closer)) + } + this.incrementRevision(meta.id) + this.db.exec(sql('commit')) + } catch (error: unknown) { + this.rollback(error, 'repair') + } + } + + async list(signal?: AbortSignal): Promise { + await this.observe(signal) + const rows = this.sessionRows() + signal?.throwIfAborted() + return rows.map(rowToMeta) + } + + /** + * Return every materialized header with its source-qualified revision. + * @param signal - optional cancellation before or after the metadata query. + * @returns stored headers and revisions without loading event rows. + */ + async listSnapshots(signal?: AbortSignal): Promise { + await this.observe(signal) + const rows = this.sessionRows() + signal?.throwIfAborted() + return rows.map(row => ({ + header: rowToMeta(row), + revision: sqliteRevision(this.storeIdentity, row), + })) + } + + async close(): Promise { + if (this.ready === undefined) { + if (this.pathReady !== undefined) await Promise.allSettled([this.pathReady]) + return + } + await Promise.allSettled([this.ready]) + if (!this.opened) return + this.opened = false + this.db.close() + } + + private rowFor(id: SessionId): SessionRow | undefined { + const value = this.db.prepare(sql('select-session')).get(id) + return value === undefined ? undefined : decodeSessionRow(value) + } + + private async observe(signal: AbortSignal | undefined): Promise { + signal?.throwIfAborted() + await this.open() + signal?.throwIfAborted() + } + + private readTransaction(read: () => T): T { + this.db.exec(sql('begin')) + try { + const value = read() + this.db.exec(sql('commit')) + return value + } catch (error: unknown) { + this.rollback(error, 'read') + } + } + + private sessionRows(): SessionRow[] { + return this.db.prepare(sql('select-sessions')).all().map(decodeSessionRow) + } + + private rollback(error: unknown, operation: string): never { + try { + this.db.exec(sql('rollback')) + } catch (rollbackError: unknown) { + /* v8 ignore next -- requires SQLite to fail both an operation and its immediate rollback. */ + throw new AggregateError([error, rollbackError], `${this.name} ${operation} failed and rollback also failed`) + } + throw error + } + + private incrementRevision(id: SessionId): void { + const updated = this.db.prepare(sql('update-session-revision')) + .run(id) + /* v8 ignore next -- materialized writes follow coordinator create(); other writes upsert in this transaction. */ + if (Number(updated.changes) !== 1) throw new Error(`session ${id} metadata row is missing`) + } + + private tailRows(id: SessionId): EventRow[] { + const tail = this.db.prepare(sql('select-tail-events')).all(id, 2).map(decodeEventRow).reverse() + if (tail.length === 0) return [] + return this.physicalSpanFrom(id, (tail[0] as EventRow).seq).eventRows + } + + /** Select the bounded physical span that may represent `fromSeq`. */ + private physicalSpanFrom( + id: SessionId, + fromSeq: number, + ): { readonly base: number; readonly eventRows: EventRow[] } { + const packedFloor = Math.max(0, fromSeq - MAX_PACKED_ROW_MEMBERS + 1) + const packedPredecessors = this.db.prepare(sql('select-packed-predecessors')) + .all(id, packedFloor, fromSeq) + .map(decodeEventRow) + let base = fromSeq + for (const predecessor of packedPredecessors) { + try { + const last = decodeRow(predecessor).at(-1) + if (last !== undefined && last.seq >= fromSeq) base = Math.min(base, predecessor.seq) + } catch { + // A malformed bounded predecessor may cover fromSeq; include it so the scanner fails closed. + base = Math.min(base, predecessor.seq) + } + } + const eventRows = this.db.prepare(sql('select-events-from')).all(id, base).map(decodeEventRow) + return { base, eventRows } + } + + private logicalLastEvent(id: SessionId, tailRows: readonly EventRow[]): SessionEvent | undefined { + if (tailRows.length === 0) return undefined + const { preserved, tornFrom } = scanRows(tailRows, (tailRows[0] as EventRow).seq) + if (tornFrom !== undefined) throw new Error(`session ${id} has an invalid physical tail at seq ${tornFrom}`) + return preserved.at(-1) + } + + private insertStatement(): StatementSync { + return this.db.prepare(sql('insert-event')) + } + + private insertRecord(insert: StatementSync, id: SessionId, record: BoundRecord): void { + insert.run( + id, + record.seq, + record.type, + record.time, + record.data, + record.sourceEventSeqs, + record.surfaceOp, + record.ignorable, + ) + } + + private writeRow(meta: SessionHeader): void { + this.db.prepare(sql('upsert-session')).run( + meta.id, + meta.version, + meta.createdAt, + meta.cwd ?? null, + meta.parentSession ?? null, + meta.seedLength ?? null, + meta.origin ?? null, + meta.delegationDepth ?? null, + meta.agentPreset ?? null, + randomUUID(), + ) + } +} + +function sqliteRevision(storeIdentity: string, row: SessionRow): PersistenceRevision { + return SessionPersistenceRevision( + `${storeIdentity}:incarnation:${row.incarnation}:revision:${row.revision}`, + ) +} + +async function createDatabaseFile(path: string): Promise { + try { + const handle = await open(path, 'wx', 0o600) + await handle.close() + } catch (error: unknown) { + if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error + } +} + +async function validateParentDirectory(path: string): Promise { + const parent = await lstat(path) + if (parent.isSymbolicLink() || !parent.isDirectory()) { + throw new Error(`session database parent "${path}" must be a real directory`) + } + const uid = process.getuid?.() + /* v8 ignore start -- Windows exposes neither process.getuid nor meaningful + * uid/mode bits; POSIX tests cover owner and mode rejection. */ + if (uid !== undefined && (parent.uid !== uid || (parent.mode & 0o022) !== 0)) { + throw new Error(`session database parent "${path}" must be owned by the current user and not group/world-writable`) + } + /* v8 ignore stop */ +} + +async function validateDatabaseFile(path: string): Promise { + const file = await lstat(path) + if (file.isSymbolicLink() || !file.isFile()) { + throw new Error(`session database "${path}" must be a regular file, not a symbolic link`) + } + const uid = process.getuid?.() + /* v8 ignore start -- Windows exposes neither process.getuid nor meaningful + * uid/mode bits; POSIX tests cover owner and mode rejection. */ + if (uid !== undefined && (file.uid !== uid || (file.mode & 0o077) !== 0)) { + throw new Error(`session database "${path}" must be owned by the current user and accessible only by that user`) + } + /* v8 ignore stop */ +} + +async function validateDatabaseFileIfPresent(path: string): Promise { + try { + await validateDatabaseFile(path) + } catch (error: unknown) { + if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error + } +} + +let nodeSqlite: Promise | undefined + +/** Load Node SQLite once so concurrent stores share one warning-filter lifetime. */ +function loadNodeSqlite(): Promise { + nodeSqlite ??= importNodeSqlite() + return nodeSqlite +} + +/** Import Node 22's SQLite dependency without its process-wide experimental warning. */ +async function importNodeSqlite(): Promise { + const emitWarning = Reflect.get(process, 'emitWarning') + /* v8 ignore start -- Node 22 alone emits this warning; primary coverage runs on Node 24. */ + const filteredEmitWarning = (warning: string | Error, ...args: unknown[]): void => { + const message = warning instanceof Error ? warning.message : warning + const first = args[0] + const type = warning instanceof Error + ? warning.name + : typeof first === 'string' + ? first + : typeof first === 'object' && first !== null && 'type' in first + ? first.type + : undefined + if (message === 'SQLite is an experimental feature and might change at any time' + && type === 'ExperimentalWarning') return + Reflect.apply(emitWarning, process, [warning, ...args]) + } + Reflect.set(process, 'emitWarning', filteredEmitWarning) + try { + return await import('node:sqlite') + } finally { + Reflect.set(process, 'emitWarning', emitWarning) + } + /* v8 ignore stop */ +} diff --git a/packages/session/session-persistence-sqlite/tests/built-package.spec.ts b/packages/session/session-persistence-sqlite/tests/built-package.spec.ts new file mode 100644 index 0000000000..ebbe2fc691 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/built-package.spec.ts @@ -0,0 +1,36 @@ +import { execFile } from 'node:child_process' +import { existsSync } from 'node:fs' +import { fileURLToPath } from 'node:url' +import { promisify } from 'node:util' +import { describe, expect, it } from 'vitest' + +const repoRoot = fileURLToPath(new URL('../../../../', import.meta.url)) +const builtBundle = fileURLToPath(new URL('../lib/index.js', import.meta.url)) +const execFileAsync = promisify(execFile) + +const probe = String.raw` +import { resolve } from 'node:path'; +import { pathToFileURL } from 'node:url'; +const load = path => import(pathToFileURL(resolve(path)).href); +const [{ Context }, { default: SessionStore }, { default: Sqlite }] = await Promise.all([ + load('vendor/cordis/lib/index.js'), + load('packages/core/session/lib/index.js'), + load('packages/session/session-persistence-sqlite/lib/index.js'), +]); +const ctx = new Context(); +await ctx.plugin(SessionStore); +await ctx.plugin(Sqlite, { path: ':memory:' }); +console.log(JSON.stringify(await ctx.sessionPersistence.list())); +await ctx.fiber.dispose(); +` + +describe.skipIf(!existsSync(builtBundle))('SQLite built package', () => { + it('loads packaged SQL resources from the published entry', async () => { + const { stdout, stderr } = await execFileAsync(process.execPath, ['--input-type=module', '-e', probe], { + cwd: repoRoot, + timeout: 15_000, + }) + expect(stderr).toBe('') + expect(JSON.parse(stdout) as unknown).toEqual([]) + }) +}) diff --git a/packages/session/session-persistence-sqlite/tests/compression-unprofitable.spec.ts b/packages/session/session-persistence-sqlite/tests/compression-unprofitable.spec.ts new file mode 100644 index 0000000000..f0fc765393 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/compression-unprofitable.spec.ts @@ -0,0 +1,25 @@ +import { describe, expect, it, vi } from 'vitest' +import type { SessionEvent } from '@deepseek-ai/dsh-session' + +vi.mock('node:zlib', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + zstdCompressSync: (input: ArrayBufferView) => Buffer.alloc(input.byteLength + 1), + } +}) + +import { bindRecord, ZSTD_DATA_THRESHOLD_BYTES } from '../src/compression.ts' + +describe('SQLite compression fallback', () => { + it('keeps large data as text when its Zstandard frame is not smaller', () => { + const event = { + type: 'assistant/message', + seq: 0, + time: 1, + data: { text: 'x'.repeat(ZSTD_DATA_THRESHOLD_BYTES) }, + } as unknown as SessionEvent + + expect(typeof bindRecord(event).data).toBe('string') + }) +}) diff --git a/packages/session/session-persistence-sqlite/tests/compression.spec.ts b/packages/session/session-persistence-sqlite/tests/compression.spec.ts new file mode 100644 index 0000000000..93f4da380f --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/compression.spec.ts @@ -0,0 +1,357 @@ +import { describe, expect, it } from 'vitest' +import { zstdCompressSync } from 'node:zlib' +import type { SessionEvent } from '@deepseek-ai/dsh-session' +import { CallId, type StreamChunk } from '@deepseek-ai/dsh-llm' +import { + decodeStorageRecord, + MAX_PACKED_DATA_BYTES, + MAX_PACKED_ROW_MEMBERS, + packChunkRuns, + type StorageRecord, +} from '../src/codec.ts' +import { + bindRecord, + decodeRow, + scanRows, + ZSTD_DATA_THRESHOLD_BYTES, +} from '../src/compression.ts' +import type { EventRow } from '../src/schema.ts' + +function chunk(seq: number, text = `token-${seq}`): SessionEvent { + return { + type: 'assistant/chunk', + seq, + time: 1_000 + seq, + data: { + turn: 1, + step: 1, + chunk: { type: 'text-delta', index: 0, text }, + }, + } +} + +function event(seq: number, time: number, value: StreamChunk, turn = 1, step = 1): SessionEvent { + return { type: 'assistant/chunk', seq, time, data: { turn, step, chunk: value } } +} + +function row(record: StorageRecord): EventRow { + const bound = bindRecord(record) + return { + seq: bound.seq, + type: bound.type, + time: bound.time, + data: bound.data, + source_event_seqs: bound.sourceEventSeqs, + surface_op: bound.surfaceOp, + ignorable: bound.ignorable, + } +} + +describe('SQLite compression', () => { + it('stores a 100-member run in one row and restores every logical event', () => { + const events = Array.from({ length: 100 }, (_, index) => chunk(index)) + const records = packChunkRuns(events) + expect(records).toHaveLength(1) + expect(records[0]?.type).toBe('text-chunks') + expect(scanRows(records.map(row)).preserved).toEqual(events) + }) + + it('partitions long and large runs within schema-owned row limits', () => { + const long = Array.from({ length: MAX_PACKED_ROW_MEMBERS + 3 }, (_, index) => chunk(index)) + const longRecords = packChunkRuns(long) + expect(longRecords).toHaveLength(2) + expect(scanRows(longRecords.map(row)).preserved).toEqual(long) + + const large = Array.from({ length: 4 }, (_, index) => chunk(index, 'x'.repeat(300_000))) + const largeRecords = packChunkRuns(large) + expect(largeRecords).toHaveLength(2) + for (const record of largeRecords) { + if (record.type.endsWith('-chunks')) { + expect(Buffer.byteLength(JSON.stringify(record.data))).toBeLessThanOrEqual(MAX_PACKED_DATA_BYTES) + } + } + expect(scanRows(largeRecords.map(row)).preserved).toEqual(large) + + const individuallyLarge = Array.from({ length: 3 }, (_, index) => chunk(index, 'x'.repeat(400_000))) + expect(packChunkRuns(individuallyLarge)).toEqual(individuallyLarge) + + const byteBound = Array.from({ length: 10 }, (_, index) => chunk(index, 'x'.repeat(150_000))) + const byteBoundRecords = packChunkRuns(byteBound) + expect(byteBoundRecords.length).toBeGreaterThan(1) + expect(scanRows(byteBoundRecords.map(row)).preserved).toEqual(byteBound) + }) + + it('packs every owned kind and preserves optional tool-call names', () => { + const events = [ + ...[0, 1, 2].map(seq => event(seq, seq, { type: 'reasoning-delta', index: 1, text: `${seq}` })), + ...[3, 4, 5].map(seq => event(seq, seq, { + type: 'tool-call-delta', index: 2, id: CallId('named'), name: 'write', argumentsDelta: `${seq}`, + })), + ...[6, 7, 8].map(seq => event(seq, seq, { + type: 'tool-call-delta', index: 3, id: CallId('unnamed'), argumentsDelta: `${seq}`, + })), + ] + const records = packChunkRuns(events) + expect(records.map(record => record.type)).toEqual([ + 'reasoning-chunks', 'tool-call-chunks', 'tool-call-chunks', + ]) + expect(records.flatMap(decodeStorageRecord)).toEqual(events) + }) + + it('keeps every off-format delta scalar and splits incompatible runs', () => { + const malformed = (seq: number, data: unknown): SessionEvent => ({ + type: 'assistant/chunk', seq, time: 10 + seq, data, + } as SessionEvent) + const values: SessionEvent[] = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { ...chunk(1), extra: true } as unknown as SessionEvent, + { ...chunk(-1), seq: -1 }, + { ...chunk(3), time: 1.5 }, + malformed(4, 'data'), + malformed(5, { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'x' }, extra: 1 }), + malformed(6, { turn: '1', step: 1, chunk: { type: 'text-delta', index: 0, text: 'x' } }), + malformed(7, { turn: 1, step: 1, chunk: 'chunk' }), + malformed(8, { turn: 1, step: 1, chunk: { type: 'text-delta', index: '0', text: 'x' } }), + malformed(9, { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 1 } }), + malformed(10, { turn: 1, step: 1, chunk: { type: 'tool-call-delta', index: 0, id: 1, argumentsDelta: 'x' } }), + malformed(11, { turn: 1, step: 1, chunk: { type: 'tool-call-delta', index: 0, id: 'id', name: 1, argumentsDelta: 'x' } }), + malformed(12, { turn: 1, step: 1, chunk: { type: 'usage', index: 0, totalTokens: 1 } }), + ] + expect(packChunkRuns(values)).toEqual(values) + + const gap = [chunk(0), chunk(1), chunk(3)] + const step = [chunk(0), chunk(1), event(2, 2, { type: 'text-delta', index: 0, text: 'x' }, 1, 2)] + const block = [chunk(0), chunk(1), event(2, 2, { type: 'text-delta', index: 1, text: 'x' })] + const unsafeTime = [ + event(0, Number.MIN_SAFE_INTEGER, { type: 'text-delta', index: 0, text: 'a' }), + event(1, Number.MAX_SAFE_INTEGER, { type: 'text-delta', index: 0, text: 'b' }), + event(2, Number.MAX_SAFE_INTEGER, { type: 'text-delta', index: 0, text: 'c' }), + ] + const toolName = [0, 1, 2].map(seq => event(seq, seq, { + type: 'tool-call-delta', index: 0, id: CallId('id'), + ...seq === 2 ? {} : { name: 'write' }, argumentsDelta: 'x', + })) + for (const events of [gap, step, block, unsafeTime, toolName]) { + expect(packChunkRuns(events)).toEqual(events) + } + }) + + it.each([ + ['extra envelope field', { type: 'text-chunks', seq0: 0, time0: 1, data: {}, extra: true }], + ['negative sequence', { type: 'text-chunks', seq0: -1, time0: 1, data: {} }], + ['fractional time', { type: 'text-chunks', seq0: 0, time0: 1.5, data: {} }], + ['primitive data', { type: 'text-chunks', seq0: 0, time0: 1, data: 'bad' }], + ['text fields', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [], args: [] } }], + ['non-numeric placement', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: '1', step: 1, index: 0, dt: [0, 0], texts: ['a', 'b', 'c'] } }], + ['non-array members', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [0, 0], texts: 'abc' } }], + ['too few members', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [0], texts: ['a', 'b'] } }], + ['too many members', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: Array(1_024).fill(0), texts: Array(1_025).fill('a') } }], + ['non-string member', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [0, 0], texts: ['a', 1, 'c'] } }], + ['invalid gaps', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [0, 0.5], texts: ['a', 'b', 'c'] } }], + ['non-array gaps', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: '00', texts: ['a', 'b', 'c'] } }], + ['gap arity', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [0], texts: ['a', 'b', 'c'] } }], + ['oversized data', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [0, 0], texts: ['x'.repeat(400_000), 'x'.repeat(400_000), 'x'.repeat(400_000)] } }], + ['sequence overflow', { type: 'text-chunks', seq0: Number.MAX_SAFE_INTEGER, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [0, 0], texts: ['a', 'b', 'c'] } }], + ['time overflow', { type: 'text-chunks', seq0: 0, time0: Number.MAX_SAFE_INTEGER, data: { turn: 1, step: 1, index: 0, dt: [1, 0], texts: ['a', 'b', 'c'] } }], + ['tool fields', { type: 'tool-call-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [0, 0], args: ['a', 'b', 'c'] } }], + ['tool id', { type: 'tool-call-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, id: 1, dt: [0, 0], args: ['a', 'b', 'c'] } }], + ['tool name', { type: 'tool-call-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, id: 'id', name: 1, dt: [0, 0], args: ['a', 'b', 'c'] } }], + ])('rejects malformed packed data: %s', (_label, record) => { + expect(() => decodeStorageRecord(record)).toThrow(/malformed .* storage row/) + }) + + it('decodes the schema-17 row vocabulary without another package codec', () => { + const fixture: EventRow = { + seq: 7, + type: 'text-chunks', + time: 90, + data: JSON.stringify({ turn: 2, step: 3, index: 1, dt: [2, -1], texts: ['a', 'b', 'c'] }), + source_event_seqs: null, + surface_op: null, + ignorable: 0, + } + expect(decodeRow(fixture)).toEqual([ + { ...chunk(7, 'a'), time: 90, data: { turn: 2, step: 3, chunk: { type: 'text-delta', index: 1, text: 'a' } } }, + { ...chunk(8, 'b'), time: 92, data: { turn: 2, step: 3, chunk: { type: 'text-delta', index: 1, text: 'b' } } }, + { ...chunk(9, 'c'), time: 91, data: { turn: 2, step: 3, chunk: { type: 'text-delta', index: 1, text: 'c' } } }, + ]) + expect(decodeStorageRecord('scalar')).toEqual(['scalar']) + expect(decodeStorageRecord(chunk(0))).toEqual([chunk(0)]) + }) + + it('rejects surface columns on packed rows', () => { + const packed = row(packChunkRuns([chunk(0), chunk(1), chunk(2)])[0]!) + const invalid: EventRow[] = [ + { ...packed, source_event_seqs: Buffer.alloc(0) }, + { ...packed, surface_op: '"append"' }, + ] + for (const candidate of invalid) { + expect(() => decodeRow(candidate)).toThrow(/surface fields must be null/) + } + }) + + it('rejects the packed discriminator on a scalar event type', () => { + const scalar = row({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }) + expect(() => decodeRow({ ...scalar, ignorable: 0 })) + .toThrow(/packed discriminator requires a chunk tag/) + }) + + it.each(['text-chunks', 'reasoning-chunks', 'tool-call-chunks'])( + 'preserves an ignorable logical event named %s as a scalar row', + (type) => { + const logical = { + type, + seq: 0, + time: 1, + data: { future: true }, + ignorable: true, + } as unknown as SessionEvent + const physical = row(logical) + expect(physical.ignorable).toBe(1) + expect(decodeRow(physical)).toEqual([logical]) + }, + ) + + it('compresses large data and delta-encodes complete provenance arrays', () => { + const sources = Array.from({ length: 2_000 }, (_, index) => index + 10) + const event = { + type: 'assistant/message', + seq: sources.at(-1)! + 1, + time: 1, + data: { text: 'x'.repeat(ZSTD_DATA_THRESHOLD_BYTES * 2) }, + sourceEventSeqs: sources, + surfaceOp: 'append', + } as unknown as SessionEvent + const bound = bindRecord(event) + expect(bound.data).toBeInstanceOf(Uint8Array) + expect(bound.sourceEventSeqs).toBeInstanceOf(Uint8Array) + expect(bound.sourceEventSeqs?.byteLength).toBeLessThan(Buffer.byteLength(JSON.stringify(sources))) + expect(decodeRow(row(event))).toEqual([event]) + + const small = bindRecord({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }) + expect(typeof small.data).toBe('string') + }) + + it('round-trips empty, descending, and maximum-safe provenance deltas', () => { + for (const sources of [ + [], + [Number.MAX_SAFE_INTEGER - 1, 0, Number.MAX_SAFE_INTEGER - 2], + ]) { + const event = { + type: 'assistant/message', + seq: Number.MAX_SAFE_INTEGER, + time: 1, + data: {}, + sourceEventSeqs: sources, + surfaceOp: 'append', + } as unknown as SessionEvent + expect(decodeRow(row(event))).toEqual([event]) + } + }) + + it.each([-1, 0.5])('rejects invalid provenance sequence %s before encoding', (sourceSeq) => { + const event = { + type: 'assistant/message', + seq: 1, + time: 1, + data: {}, + sourceEventSeqs: [sourceSeq], + surfaceOp: 'append', + } as unknown as SessionEvent + expect(() => bindRecord(event)).toThrow(/non-negative safe integers/) + }) + + it('rejects malformed compressed and delta-encoded values', () => { + const scalar = row({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }) + expect(() => decodeRow({ ...scalar, data: Buffer.from('not zstd') })).toThrow() + expect(() => decodeRow({ ...scalar, source_event_seqs: Buffer.from([0x80]) })) + .toThrow(/truncated varint/) + expect(() => decodeRow({ ...scalar, source_event_seqs: Buffer.from([0x80, 0x00]) })) + .toThrow(/non-canonical varint/) + expect(() => decodeRow({ ...scalar, source_event_seqs: Buffer.from([0x00, 0x01]) })) + .toThrow(/decoded seq is out of range/) + expect(() => decodeRow({ ...scalar, source_event_seqs: Buffer.from([ + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x0f, 0x02, + ]) })).toThrow(/decoded seq is out of range/) + expect(() => decodeRow({ ...scalar, source_event_seqs: Buffer.from([ + 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x10, + ]) })).toThrow(/varint is out of range/) + expect(() => decodeRow({ ...scalar, source_event_seqs: Buffer.alloc(9, 0x80) })) + .toThrow(/varint is out of range/) + }) + + it('rejects an oversized packed data column before JSON decoding', () => { + const oversized: EventRow = { + seq: 0, + type: 'text-chunks', + time: 1, + data: ' '.repeat(MAX_PACKED_DATA_BYTES + 1), + source_event_seqs: null, + surface_op: null, + ignorable: 0, + } + expect(() => decodeRow(oversized)).toThrow(/data exceeds/) + }) + + it('bounds packed data while decompressing', () => { + const serialized = JSON.stringify({ + turn: 1, + step: 1, + index: 0, + dt: [0, 0], + texts: ['x'.repeat(MAX_PACKED_DATA_BYTES), 'b', 'c'], + }) + const oversized: EventRow = { + seq: 0, + type: 'text-chunks', + time: 1, + data: zstdCompressSync(serialized), + source_event_seqs: null, + surface_op: null, + ignorable: 0, + } + expect(() => decodeRow(oversized)).toThrow(/Buffer larger than/) + }) + + it('distinguishes removable and committed physical corruption', () => { + const start = row({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }) + const skipped = row({ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }) + expect(scanRows([start, skipped])).toEqual({ preserved: [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + ], tornFrom: 2 }) + + const end = row({ + type: 'turn/end', + seq: 3, + time: 3, + data: { turn: 1, reason: { kind: 'completed' } }, + }) + expect(() => scanRows([start, skipped, end])).toThrow(/invalid committed physical row at seq 2/) + + const malformed = { + ...row(packChunkRuns([chunk(0), chunk(1), chunk(2)])[0]!), + data: '{not json', + } + const committedEnd = row({ + type: 'turn/end', + seq: 1, + time: 4, + data: { turn: 1, reason: { kind: 'completed' } }, + }) + expect(() => scanRows([malformed, committedEnd])) + .toThrow(/invalid committed physical row at seq 0/) + }) + + it('treats a malformed packed tail as one removable physical row', () => { + const malformed: EventRow = { + seq: 0, + type: 'text-chunks', + time: 1, + data: JSON.stringify({ turn: 1, step: 1, index: 0, dt: [], texts: ['a', 'b'] }), + source_event_seqs: null, + surface_op: null, + ignorable: 0, + } + expect(scanRows([malformed])).toEqual({ preserved: [], tornFrom: 0 }) + }) +}) diff --git a/packages/session/session-persistence-sqlite/tests/differential.spec.ts b/packages/session/session-persistence-sqlite/tests/differential.spec.ts new file mode 100644 index 0000000000..45ec734992 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/differential.spec.ts @@ -0,0 +1,273 @@ +import { afterEach, describe, expect, it } from 'vitest' +import fc from 'fast-check' +import { Context } from '@deepseek-ai/cordis' +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { DatabaseSync } from 'node:sqlite' +import { CallId, type StreamChunk } from '@deepseek-ai/dsh-llm' +import SessionStore, { type SessionEvent } from '@deepseek-ai/dsh-session' +import type { SessionPersistence } from '@deepseek-ai/dsh-session-persistence' +import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' +import SessionPersistenceSqlite from '@deepseek-ai/dsh-session-persistence-sqlite' +import { meta } from '../../session-persistence/tests/contract.ts' +import { testSql } from './test-sql.ts' + +type BackendName = 'jsonl-zstd' | 'sqlite' + +interface MountedBackend { + readonly persistence: SessionPersistence + dispose(): Promise +} + +const directories: string[] = [] +afterEach(async () => { + for (const directory of directories.splice(0)) { + await rm(directory, { recursive: true, force: true }) + } +}) + +async function freshDirectory(prefix: string): Promise { + const directory = await mkdtemp(join(tmpdir(), prefix)) + directories.push(directory) + return directory +} + +async function mount(name: BackendName, root: string): Promise { + const ctx = new Context() + await ctx.plugin(SessionStore) + switch (name) { + case 'jsonl-zstd': { + const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: join(root, 'jsonl') }) + return { persistence: ctx.sessionPersistence, dispose: async () => { await fiber.dispose() } } + } + case 'sqlite': { + const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: join(root, 'sessions.db') }) + return { persistence: ctx.sessionPersistence, dispose: async () => { await fiber.dispose() } } + } + } +} + +function closedChunkLog( + entries: readonly { readonly chunk: StreamChunk; readonly time: number; readonly ignorable?: true }[], +): SessionEvent[] { + const chunks = entries.map(({ chunk, time, ignorable }, index): SessionEvent => ({ + type: 'assistant/chunk', + seq: index + 2, + time, + data: { turn: 1, step: 1, chunk }, + ...ignorable === true ? { ignorable } : {}, + })) + return [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + ...chunks, + { type: 'step/end', seq: chunks.length + 2, time: 3, data: { turn: 1, step: 1 } }, + { + type: 'turn/end', + seq: chunks.length + 3, + time: 4, + data: { turn: 1, reason: { kind: 'completed' } }, + }, + ] +} + +function packingMatrixLog(): SessionEvent[] { + const entries: { chunk: StreamChunk; time: number; ignorable?: true }[] = [ + ...Array.from({ length: 5 }, (_, index) => ({ + chunk: { type: 'text-delta' as const, index: 0, text: `text-${index}` }, + time: 1_000 + index, + })), + ...Array.from({ length: 4 }, (_, index) => ({ + chunk: { type: 'reasoning-delta' as const, index: 1, text: `reason-${index}` }, + time: 990 - index, + })), + ...Array.from({ length: 4 }, (_, index) => ({ + chunk: { + type: 'tool-call-delta' as const, + index: 2, + id: CallId('named-call'), + name: 'write', + argumentsDelta: `{${index}`, + }, + time: 2_000 + index, + })), + ...Array.from({ length: 3 }, (_, index) => ({ + chunk: { + type: 'tool-call-delta' as const, + index: 3, + id: CallId('unnamed-call'), + argumentsDelta: `${index}}`, + }, + time: 3_000 + index, + })), + { chunk: { type: 'block-start', index: 4, blockType: 'text' }, time: 4_000 }, + { chunk: { type: 'text-delta', index: 4, text: 'short-a' }, time: 4_001 }, + { chunk: { type: 'text-delta', index: 4, text: 'short-b' }, time: 4_002 }, + { chunk: { type: 'text-delta', index: 5, text: 'scalar-envelope' }, time: 4_003, ignorable: true }, + { chunk: { type: 'finish', reason: { kind: 'stop' } }, time: 4_004 }, + ] + return closedChunkLog(entries) +} + +function storageTagCollisionLog(): SessionEvent[] { + return [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + ...['text-chunks', 'reasoning-chunks', 'tool-call-chunks'].map((type, index) => ({ + type, + seq: index + 1, + time: index + 2, + data: { future: true }, + ignorable: true as const, + }) as unknown as SessionEvent), + { type: 'turn/end', seq: 4, time: 5, data: { turn: 1, reason: { kind: 'completed' } } }, + ] +} + +function batches(events: readonly SessionEvent[], sizes: readonly number[]): SessionEvent[][] { + const result: SessionEvent[][] = [] + let offset = 0 + let index = 0 + while (offset < events.length) { + const size = sizes[index % sizes.length] as number + result.push(events.slice(offset, offset + size)) + offset += size + index += 1 + } + return result +} + +async function verifyBackend( + name: BackendName, + root: string, + events: readonly SessionEvent[], + sizes: readonly number[], +): Promise { + const header = { ...meta('differential', '/work'), delegationDepth: 0 } + let mounted = await mount(name, root) + try { + await mounted.persistence.create(header) + for (const batch of batches(events, sizes)) { + await mounted.persistence.append(header.id, batch) + } + expect(await mounted.persistence.inspect(header.id), name).toEqual({ meta: header, events }) + expect(await mounted.persistence.list(), name).toEqual([header]) + const revision = (await mounted.persistence.listSnapshots())[0]?.revision + for (let fromSeq = 0; fromSeq <= events.length + 1; fromSeq += 1) { + expect((await mounted.persistence.readFrom(header.id, fromSeq)).events, `${name} seq ${fromSeq}`) + .toEqual(events.slice(fromSeq)) + } + expect((await mounted.persistence.listSnapshots())[0]?.revision, name).toBe(revision) + } finally { + await mounted.dispose() + } + + mounted = await mount(name, root) + try { + expect(await mounted.persistence.inspect(header.id), `${name} reopen`).toEqual({ meta: header, events }) + } finally { + await mounted.dispose() + } +} + +const streamChunkArbitrary: fc.Arbitrary = fc.oneof( + fc.record({ type: fc.constant<'text-delta'>('text-delta'), index: fc.nat(2), text: fc.string() }), + fc.record({ type: fc.constant<'reasoning-delta'>('reasoning-delta'), index: fc.nat(2), text: fc.string() }), + fc.record({ + type: fc.constant<'tool-call-delta'>('tool-call-delta'), + index: fc.nat(2), + id: fc.constantFrom(CallId('call-1'), CallId('call-2')), + argumentsDelta: fc.string(), + }), + fc.record({ + type: fc.constant<'tool-call-delta'>('tool-call-delta'), + index: fc.nat(2), + id: fc.constantFrom(CallId('call-1'), CallId('call-2')), + name: fc.constantFrom('read', 'write'), + argumentsDelta: fc.string(), + }), + fc.record({ + type: fc.constant<'block-start'>('block-start'), + index: fc.nat(2), + blockType: fc.constant<'text'>('text'), + }), + fc.record({ type: fc.constant<'finish'>('finish'), reason: fc.constant({ kind: 'stop' as const }) }), +) + +const randomWorkload = fc.record({ + entries: fc.array(fc.record({ + chunk: streamChunkArbitrary, + time: fc.oneof( + { weight: 4, arbitrary: fc.integer({ min: 0, max: 10_000 }) }, + { weight: 1, arbitrary: fc.integer({ min: Number.MIN_SAFE_INTEGER, max: Number.MAX_SAFE_INTEGER }) }, + ), + ignorable: fc.option(fc.constant(true), { nil: undefined }), + }), { maxLength: 30 }), + batchSizes: fc.array(fc.integer({ min: 1, max: 8 }), { minLength: 1, maxLength: 8 }), +}).map(({ entries, batchSizes }) => ({ + events: JSON.parse(JSON.stringify(closedChunkLog(entries.map(({ chunk, time, ignorable }) => ({ + chunk, + time, + ...ignorable === true ? { ignorable } : {}, + }))))) as SessionEvent[], + batchSizes, +})) + +describe('SQLite cross-backend differential behavior', () => { + it('preserves ignorable logical events whose names match physical storage tags', async () => { + const events = storageTagCollisionLog() + const directory = await freshDirectory('dsh-sqlite-storage-tag-collision-') + const root = join(directory, 'sqlite') + await verifyBackend('sqlite', root, events, [2, 1]) + const db = new DatabaseSync(join(root, 'sessions.db'), { readOnly: true }) + try { + expect(db.prepare(testSql('count-physical-types')).all()).toEqual([]) + expect(db.prepare(testSql('count-ignorable-events')).get()).toEqual({ count: 3 }) + } finally { + db.close() + } + }) + + it('matches JSONL/Zstandard for every packed kind, scalar fallback, suffix, partition, and reopen', async () => { + const events = packingMatrixLog() + for (const [partitionIndex, sizes] of [[events.length], [1], [2, 1, 5, 3]].entries()) { + const directory = await freshDirectory(`dsh-sqlite-matrix-${partitionIndex}-`) + for (const name of ['jsonl-zstd', 'sqlite'] as const) { + const root = join(directory, name) + await verifyBackend(name, root, events, sizes) + if (name === 'sqlite') { + const db = new DatabaseSync(join(root, 'sessions.db'), { readOnly: true }) + try { + expect(db.prepare(testSql('count-physical-types')).all()).toEqual([ + [ + { type: 'reasoning-chunks', count: 1 }, + { type: 'text-chunks', count: 1 }, + { type: 'tool-call-chunks', count: 2 }, + ], + [], + [ + { type: 'reasoning-chunks', count: 1 }, + { type: 'text-chunks', count: 1 }, + { type: 'tool-call-chunks', count: 1 }, + ], + ][partitionIndex]) + expect(db.prepare(testSql('count-ignorable-events')).get()) + .toEqual({ count: 1 }) + } finally { + db.close() + } + } + } + } + }, 30_000) + + it('matches JSONL/Zstandard across randomized logical logs and append partitions', async () => { + await fc.assert(fc.asyncProperty(randomWorkload, async ({ events, batchSizes }) => { + const directory = await freshDirectory('dsh-sqlite-property-') + for (const name of ['jsonl-zstd', 'sqlite'] as const) { + await verifyBackend(name, join(directory, name), events, batchSizes) + } + }), { numRuns: 100, seed: 0x5A17E }) + }, 60_000) + +}) diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/add-unexpected-column.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/add-unexpected-column.sql new file mode 100644 index 0000000000..bc0d60887b --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/add-unexpected-column.sql @@ -0,0 +1 @@ +ALTER TABLE events ADD COLUMN unexpected TEXT; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/count-events.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/count-events.sql new file mode 100644 index 0000000000..b335590faa --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/count-events.sql @@ -0,0 +1,2 @@ +SELECT COUNT(*) AS count +FROM events; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/count-ignorable-events.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/count-ignorable-events.sql new file mode 100644 index 0000000000..4de44570ec --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/count-ignorable-events.sql @@ -0,0 +1,3 @@ +SELECT COUNT(*) AS count +FROM events +WHERE ignorable = 1; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/count-packed-events.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/count-packed-events.sql new file mode 100644 index 0000000000..1c11b3dd0b --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/count-packed-events.sql @@ -0,0 +1,3 @@ +SELECT COUNT(*) AS count +FROM events +WHERE type = 'text-chunks' AND ignorable = 0; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/count-physical-types.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/count-physical-types.sql new file mode 100644 index 0000000000..ba5e7f9d72 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/count-physical-types.sql @@ -0,0 +1,6 @@ +SELECT type, COUNT(*) AS count +FROM events +WHERE type IN ('text-chunks', 'reasoning-chunks', 'tool-call-chunks') + AND ignorable = 0 +GROUP BY type +ORDER BY type; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/create-loose-schema.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/create-loose-schema.sql new file mode 100644 index 0000000000..d63c09c540 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/create-loose-schema.sql @@ -0,0 +1,14 @@ +CREATE TABLE persistence_state (singleton ANY, store_id ANY); +CREATE TABLE sessions ( + id ANY, version ANY, created_at ANY, cwd ANY, parent_session ANY, + seed_length ANY, origin ANY, delegation_depth ANY, agent_preset ANY, + incarnation ANY, revision ANY +); +CREATE TABLE events ( + session_id ANY, seq ANY, type ANY, time ANY, data ANY, + source_event_seqs ANY, surface_op ANY, ignorable ANY +); +INSERT INTO persistence_state (singleton, store_id) +VALUES (1, '00000000-0000-4000-8000-000000000000'); +PRAGMA application_id = 1146308688; +PRAGMA user_version = 17; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/create-unrelated-table.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/create-unrelated-table.sql new file mode 100644 index 0000000000..23c813df9a --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/create-unrelated-table.sql @@ -0,0 +1 @@ +CREATE TABLE unrelated (value TEXT); diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/delete-persistence-state.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/delete-persistence-state.sql new file mode 100644 index 0000000000..c337450451 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/delete-persistence-state.sql @@ -0,0 +1 @@ +DELETE FROM persistence_state; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/delete-session-events.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/delete-session-events.sql new file mode 100644 index 0000000000..c8f86c2f8f --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/delete-session-events.sql @@ -0,0 +1,2 @@ +DELETE FROM events +WHERE session_id = ?; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/empty-store-id.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/empty-store-id.sql new file mode 100644 index 0000000000..5d3a64c1e5 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/empty-store-id.sql @@ -0,0 +1,3 @@ +UPDATE persistence_state +SET store_id = '' +WHERE singleton = 1; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/insert-corrupt-event.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/insert-corrupt-event.sql new file mode 100644 index 0000000000..82eb0af005 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/insert-corrupt-event.sql @@ -0,0 +1,2 @@ +INSERT INTO events (session_id, seq, type, time, data, ignorable) +VALUES (?, ?, ?, ?, ?, ?); diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/measure-write-traffic.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/measure-write-traffic.sql new file mode 100644 index 0000000000..3ca0e6bd0c --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/measure-write-traffic.sql @@ -0,0 +1,3 @@ +SELECT COUNT(*) AS rows, + COALESCE(MAX(length(CAST(data AS BLOB))), 0) AS largest +FROM events; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/replace-events-with-nonstrict-table.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/replace-events-with-nonstrict-table.sql new file mode 100644 index 0000000000..e39da0a3f4 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/replace-events-with-nonstrict-table.sql @@ -0,0 +1,14 @@ +PRAGMA foreign_keys = OFF; +ALTER TABLE events RENAME TO strict_events; +CREATE TABLE events ( + session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE, + seq INTEGER NOT NULL, + type TEXT NOT NULL, + time INTEGER NOT NULL, + data TEXT NOT NULL, + source_event_seqs TEXT, + surface_op TEXT, + ignorable INTEGER, + PRIMARY KEY (session_id, seq) +); +DROP TABLE strict_events; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/select-event-rowids.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/select-event-rowids.sql new file mode 100644 index 0000000000..91c3167273 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/select-event-rowids.sql @@ -0,0 +1,3 @@ +SELECT seq, rowid +FROM events +ORDER BY seq; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/select-event-rows.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/select-event-rows.sql new file mode 100644 index 0000000000..037c34641a --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/select-event-rows.sql @@ -0,0 +1,4 @@ +SELECT rowid, seq, type, time, data, source_event_seqs, surface_op, ignorable +FROM events +WHERE session_id = ? +ORDER BY seq; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/select-last-event.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/select-last-event.sql new file mode 100644 index 0000000000..14b6e17742 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/select-last-event.sql @@ -0,0 +1,5 @@ +SELECT seq, type, data +FROM events +WHERE session_id = ? +ORDER BY seq DESC +LIMIT 1; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/select-user-version.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/select-user-version.sql new file mode 100644 index 0000000000..4edeca1a4d --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/select-user-version.sql @@ -0,0 +1 @@ +PRAGMA user_version; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/set-application-id-12345.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/set-application-id-12345.sql new file mode 100644 index 0000000000..79d31a3a57 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/set-application-id-12345.sql @@ -0,0 +1 @@ +PRAGMA application_id = 12345; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/set-user-version-15.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/set-user-version-15.sql new file mode 100644 index 0000000000..fa5f49e3b2 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/set-user-version-15.sql @@ -0,0 +1 @@ +PRAGMA user_version = 15; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/set-user-version-16.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/set-user-version-16.sql new file mode 100644 index 0000000000..0750749350 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/set-user-version-16.sql @@ -0,0 +1 @@ +PRAGMA user_version = 16; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/set-user-version-17.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/set-user-version-17.sql new file mode 100644 index 0000000000..5aac576e8c --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/set-user-version-17.sql @@ -0,0 +1 @@ +PRAGMA user_version = 17; diff --git a/packages/session/session-persistence-sqlite/tests/resources/sql/update-invalid-session-metadata.sql b/packages/session/session-persistence-sqlite/tests/resources/sql/update-invalid-session-metadata.sql new file mode 100644 index 0000000000..a4d6b530a2 --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/resources/sql/update-invalid-session-metadata.sql @@ -0,0 +1,3 @@ +UPDATE sessions +SET origin = 'external', delegation_depth = -1, seed_length = -1 +WHERE id = ?; diff --git a/packages/session/session-persistence-sqlite/tests/sql-resource-boundary.spec.ts b/packages/session/session-persistence-sqlite/tests/sql-resource-boundary.spec.ts new file mode 100644 index 0000000000..8574d1c97f --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/sql-resource-boundary.spec.ts @@ -0,0 +1,102 @@ +import { readdir, readFile } from 'node:fs/promises' +import { fileURLToPath } from 'node:url' +import ts from 'typescript' +import { describe, expect, it } from 'vitest' + +const PACKAGE_ROOT = fileURLToPath(new URL('../', import.meta.url)) +const SQL_LITERAL = /^\s*(?:ALTER|ATTACH|BEGIN|COMMIT|CREATE|DELETE|DETACH|DROP|INSERT|PRAGMA|REINDEX|RELEASE|ROLLBACK|SAVEPOINT|SELECT|UPDATE|VACUUM|WITH)\s/iu // eslint-disable-line @stylistic/max-len + +async function filesUnder(path: string): Promise { + const entries = await readdir(path, { withFileTypes: true }) + return (await Promise.all(entries.map(async entry => entry.isDirectory() + ? filesUnder(`${path}/${entry.name}`) + : [`${path}/${entry.name}`]))).flat() +} + +function sqlLiteralText(node: ts.Node): string | undefined { + if (ts.isStringLiteral(node) || ts.isNoSubstitutionTemplateLiteral(node)) return node.text + if (node.kind === ts.SyntaxKind.TemplateHead) { + return (node as ts.Node & { readonly text: string }).text + } + return undefined +} + +function isOwnedSqlSource(node: ts.Expression | undefined, source: ts.SourceFile): boolean { + if (node === undefined) return false + if (ts.isCallExpression(node) + && ts.isIdentifier(node.expression) + && (node.expression.text === 'sql' || node.expression.text === 'testSql')) return true + if (!ts.isIdentifier(node) || node.text !== 'source') return false + const call = node.parent + if (!ts.isCallExpression(call) + || call.arguments.length !== 1 + || call.arguments[0] !== node + || !ts.isPropertyAccessExpression(call.expression) + || call.expression.expression.kind !== ts.SyntaxKind.SuperKeyword + || call.expression.name.text !== 'prepare') return false + let method: ts.Node | undefined = node.parent + while (method !== undefined && !ts.isMethodDeclaration(method)) method = method.parent + if (method === undefined + || method.name.getText(source) !== 'prepare' + || method.parameters.length !== 1 + || method.parameters[0]?.name.getText(source) !== 'source') return false + let classNode: ts.Node | undefined = method.parent + while (classNode !== undefined && !ts.isClassExpression(classNode)) classNode = classNode.parent + if (classNode === undefined || classNode.name?.text !== 'JournalFailureDatabase') return false + const guard = method.body?.statements[0] + if (guard === undefined + || !ts.isIfStatement(guard) + || !ts.isBinaryExpression(guard.expression) + || guard.expression.operatorToken.kind !== ts.SyntaxKind.ExclamationEqualsEqualsToken + || guard.expression.left.getText(source) !== 'source' + || guard.expression.right.getText(source) !== "sql('journal-mode-wal')") return false + return ts.isReturnStatement(guard.thenStatement) + && guard.thenStatement.expression === call +} + +describe('SQLite SQL resource boundary', () => { + it('keeps statements and query assembly out of TypeScript files', async () => { + const files = (await Promise.all([ + filesUnder(`${PACKAGE_ROOT}/src`), + filesUnder(`${PACKAGE_ROOT}/tests`), + ])).flat().filter(path => path.endsWith('.ts')) + const violations: string[] = [] + for (const path of files) { + const source = ts.createSourceFile(path, await readFile(path, 'utf8'), ts.ScriptTarget.Latest, true) + const usesNodeSqlite = source.statements.some(statement => ts.isImportDeclaration(statement) + && ts.isStringLiteral(statement.moduleSpecifier) + && statement.moduleSpecifier.text === 'node:sqlite') + const visit = (node: ts.Node): void => { + const literal = sqlLiteralText(node) + if (literal !== undefined && SQL_LITERAL.test(literal)) { + violations.push(`${path}:${source.getLineAndCharacterOfPosition(node.getStart()).line + 1}: SQL literal`) + } + // Awaited prepare() is SessionPersistence; DatabaseSync.prepare() is synchronous. + if (usesNodeSqlite + && ts.isCallExpression(node) + && ts.isPropertyAccessExpression(node.expression) + && (node.expression.name.text === 'exec' + || (node.expression.name.text === 'prepare' && !ts.isAwaitExpression(node.parent)))) { + const argument = node.arguments[0] + if (!isOwnedSqlSource(argument, source)) { + violations.push(`${path}:${source.getLineAndCharacterOfPosition(node.getStart()).line + 1}: unowned query source`) + } + } + ts.forEachChild(node, visit) + } + visit(source) + } + expect(violations).toEqual([]) + }) + + it('keeps resource text static instead of interpolated', async () => { + const files = (await Promise.all([ + filesUnder(`${PACKAGE_ROOT}/resources/sql`), + filesUnder(`${PACKAGE_ROOT}/tests/resources/sql`), + ])).flat() + for (const path of files) { + expect(path.endsWith('.sql')).toBe(true) + expect(await readFile(path, 'utf8')).not.toContain('${') + } + }) +}) diff --git a/packages/session/session-persistence-sqlite/tests/sqlite.spec.ts b/packages/session/session-persistence-sqlite/tests/sqlite.spec.ts index 8f0c49e71f..c05d9aa779 100644 --- a/packages/session/session-persistence-sqlite/tests/sqlite.spec.ts +++ b/packages/session/session-persistence-sqlite/tests/sqlite.spec.ts @@ -1,949 +1,847 @@ -import { createUserMessage, createMessage } from '@deepseek-ai/dsh-llm' -import { afterEach, describe, expect, it } from 'vitest' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { spawn } from 'node:child_process' import { Context } from '@deepseek-ai/cordis' -import { existsSync } from 'node:fs' -import { chmod, mkdtemp, rm, stat, symlink, writeFile } from 'node:fs/promises' +import { once } from 'node:events' +import { chmod, mkdir, mkdtemp, rm, stat, symlink, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' -import { dirname, join } from 'node:path' +import { join } from 'node:path' +import { performance } from 'node:perf_hooks' +import { pathToFileURL } from 'node:url' import { DatabaseSync } from 'node:sqlite' -import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SurfaceEvent, SurfaceEventType } from '@deepseek-ai/dsh-session' -import SqliteSessionPersistence, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite' +import Loader from '@deepseek-ai/cordis-plugin-loader' +import Include from '@deepseek-ai/cordis-plugin-include' +import SessionStore, { SessionId, type SessionEvent } from '@deepseek-ai/dsh-session' +import SessionPersistenceSqlite, { + DEFAULT_BUSY_TIMEOUT_MS, + SCHEMA_VERSION, +} from '@deepseek-ai/dsh-session-persistence-sqlite' import { + runCoordinatorContract, + type CoordinatorFixture, +} from '../../session-persistence/tests/coordinator-contract.ts' +import { + meta, + runPersistenceContract, +} from '../../session-persistence/tests/contract.ts' +import { MAX_PACKED_DATA_BYTES } from '../src/codec.ts' +import { + decodeEventRow, + decodeSessionRow, + decodeStoreIdentity, openDatabase, - rowToEvent, + validateSchemaForMutation, rowToMeta, - scanRows, SESSION_PERSISTENCE_SQLITE_APPLICATION_ID, - type EventRow, + type SessionRow, } from '../src/schema.ts' -import { runPersistenceContract, meta, oneTurnLog, appendLog } from '../../session-persistence/tests/contract.ts' -import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts' +import { SqliteStore } from '../src/store.ts' +import { sql } from '../src/sql.ts' +import { testSql } from './test-sql.ts' const dirs: string[] = [] -afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) }) +afterEach(async () => { + for (const directory of dirs.splice(0)) await rm(directory, { recursive: true, force: true }) +}) -async function expectFlushError(promise: Promise, message: RegExp): Promise { - try { - await promise - } catch (error) { - expect(error).toBeInstanceOf(Error) - expect((error as Error).message).toMatch(message) - return - } - throw new Error('expected flush to reject') +async function freshDbPath(prefix = 'dsh-sqlite-'): Promise { + const directory = await mkdtemp(join(tmpdir(), prefix)) + dirs.push(directory) + return join(directory, 'sessions.db') } -async function freshDbPath(): Promise { - const dir = await mkdtemp(join(tmpdir(), 'dsh-sqlite-')) - dirs.push(dir) - return join(dir, 'sessions.db') -} - -/** A context with the session store + SQLite backend, plus a teardown. */ -async function backend(path = ':memory:'): Promise<{ ctx: Context; dispose: () => Promise }> { +async function backendFailure(path: string): Promise { const ctx = new Context() await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SqliteSessionPersistence, { path }) - return { ctx, dispose: () => fiber.dispose() } + try { + await ctx.plugin(SessionPersistenceSqlite, { path }) + await ctx.sessionPersistence.list() + return undefined + } catch (error: unknown) { + return error + } finally { + await ctx.fiber.dispose() + } +} + +function errorMessage(error: unknown): string { + return error instanceof Error ? error.message : String(error) +} + +function databaseWithJournalFailure( + nextFailure: () => Error | undefined, +): typeof DatabaseSync { + return class JournalFailureDatabase extends DatabaseSync { + override prepare(source: string) { + if (source !== sql('journal-mode-wal')) return super.prepare(source) + const statement = super.prepare(sql('journal-mode-wal')) + const get = statement.get.bind(statement) + Object.defineProperty(statement, 'get', { + value: () => { + const failure = nextFailure() + if (failure !== undefined) throw failure + return get() + }, + }) + return statement + } + } +} + +function chunk(seq: number, text = `token-${seq}`): SessionEvent { + return { + type: 'assistant/chunk', + seq, + time: 1_000 + seq, + data: { + turn: 1, + step: 1, + chunk: { type: 'text-delta', index: 0, text }, + }, + } +} + +function chunkLog(count: number): SessionEvent[] { + return [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + ...Array.from({ length: count }, (_, index) => chunk(index + 2)), + { type: 'step/end', seq: count + 2, time: count + 3, data: { turn: 1, step: 1 } }, + { + type: 'turn/end', + seq: count + 3, + time: count + 4, + data: { turn: 1, reason: { kind: 'completed' } }, + }, + ] +} + +async function measureWriteTraffic( + path: string, + events: readonly SessionEvent[], +): Promise<{ + readonly walBytes: number + readonly idleWalBytes: number + readonly rows: number + readonly largest: number + readonly inserted: number + readonly changed: number + readonly removed: number +}> { + interface PhysicalRow { + readonly rowid: number + readonly seq: number + readonly type: string + readonly time: number + readonly data: string | Uint8Array + readonly source_event_seqs: Uint8Array | null + readonly surface_op: string | null + readonly ignorable: number | null + } + const sameValue = (left: string | Uint8Array | null, right: string | Uint8Array | null): boolean => ( + typeof left === 'string' || left === null + ? left === right + : right instanceof Uint8Array && Buffer.from(left).equals(Buffer.from(right)) + ) + const sameRow = (left: PhysicalRow, right: PhysicalRow): boolean => ( + left.rowid === right.rowid + && left.seq === right.seq + && left.type === right.type + && left.time === right.time + && sameValue(left.data, right.data) + && sameValue(left.source_event_seqs, right.source_event_seqs) + && left.surface_op === right.surface_op + && left.ignorable === right.ignorable + ) + const ctx = new Context() + await ctx.plugin(SessionStore) + await ctx.plugin(SessionPersistenceSqlite, { path, writeBatchMaxDelayMs: 200 }) + try { + const header = meta('traffic') + await ctx.sessionPersistence.create(header) + let previous = new Map() + let inserted = 0 + let changed = 0 + let removed = 0 + const probe = new DatabaseSync(path, { readOnly: true }) + try { + const selectRows = probe.prepare(testSql('select-event-rows')) + for (let offset = 0; offset < events.length; offset += 40) { + await ctx.sessionPersistence.append(header.id, events.slice(offset, offset + 40)) + const current = new Map((selectRows.all(header.id) as unknown as PhysicalRow[]) + .map(row => [row.seq, row])) + for (const [seq, row] of current) { + const old = previous.get(seq) + if (old === undefined) inserted += 1 + else if (!sameRow(old, row)) changed += 1 + } + for (const seq of previous.keys()) if (!current.has(seq)) removed += 1 + previous = current + } + } finally { + probe.close() + } + const db = new DatabaseSync(path, { readOnly: true }) + const measured = db.prepare(testSql('measure-write-traffic')).get() as { rows: number; largest: number } + db.close() + const walBytes = (await stat(`${path}-wal`)).size + await new Promise(resolve => setTimeout(resolve, 250)) + return { + walBytes, + idleWalBytes: (await stat(`${path}-wal`)).size, + rows: measured.rows, + largest: measured.largest, + inserted, + changed, + removed, + } + } finally { + await ctx.fiber.dispose() + } } -// Run the same backend-agnostic contract as JSONL to pin identical semantics. runPersistenceContract('sqlite', async () => { const ctx = new Context() await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SqliteSessionPersistence, { path: ':memory:' }) + const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) return { persistence: ctx.sessionPersistence, dispose: async () => { await fiber.dispose() }, } }) -// A file-backed database lets two mounts share rows across reload. `corruptTail` inserts invalid -// JSON past the committed seq, exercising coordinator repair against real database rows. runCoordinatorContract('sqlite', async (): Promise => { - const dir = await mkdtemp(join(tmpdir(), 'dsh-sqlite-coord-')) - const path = join(dir, 'sessions.db') + const directory = await mkdtemp(join(tmpdir(), 'dsh-sqlite-coord-')) + const path = join(directory, 'sessions.db') return { - mount: async ctx => ctx.plugin(SqliteSessionPersistence, { path }), + mount: async ctx => ctx.plugin(SessionPersistenceSqlite, { path }), corruptTail: async (id) => { - // A row past the committed region whose `data` does not parse: scanRows - // bounds the preserved prefix at it and returns its seq as tornFrom, which - // the backend surfaces to the coordinator as the tornMarker to delete from. - const db = openDatabase(path, 'wal') - const next = (db.prepare('SELECT COALESCE(MAX(seq), -1) + 1 AS n FROM events WHERE session_id = ?') - .get(id) as { n: number }).n - db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') - .run(id, next, 'assistant/chunk', 99, '{not valid json') + const db = new DatabaseSync(path) + const last = db.prepare(testSql('select-last-event')) + .get(id) as { seq: number; type: string; data: string } + const logicalLength = last.type === 'text-chunks' + ? (JSON.parse(last.data) as { texts: string[] }).texts.length + : 1 + const next = last.seq + logicalLength + db.prepare(testSql('insert-corrupt-event')) + .run(id, next, 'assistant/chunk', 99, '{not valid json', null) db.close() }, - cleanup: async () => { await rm(dir, { recursive: true, force: true }) }, + cleanup: async () => { await rm(directory, { recursive: true, force: true }) }, } }) -describe('scanRows', () => { - // scanRows works off EventRows (data is a JSON string column); build them from SessionEvents - // so the unit tests read in terms of the event vocabulary. Surface metadata is serialized to - // its nullable columns so the conversion remains faithful. - const rows = (events: SessionEvent[]): EventRow[] => - events.map((e) => { - const se = e as SessionEvent - return { - seq: e.seq, type: e.type, time: e.time, data: JSON.stringify(e.data), - source_event_seqs: se.sourceEventSeqs !== undefined ? JSON.stringify(se.sourceEventSeqs) : null, - surface_op: se.surfaceOp !== undefined ? JSON.stringify(se.surfaceOp) : null, - ignorable: e.ignorable === true ? 1 : null, - } +describe('SessionPersistenceSqlite physical packing', () => { + it('loads from cordis.yml and packs through the assembled service', async () => { + const path = await freshDbPath('dsh-sqlite-loader-') + const configPath = join(path, '..', 'cordis.yml') + await writeFile(configPath, [ + "- name: '@deepseek-ai/dsh-session'", + "- name: '@deepseek-ai/dsh-session-persistence-sqlite'", + ' config:', + ` path: ${JSON.stringify(path)}`, + '', + ].join('\n')) + + const ctx = new Context() + ctx.baseUrl = pathToFileURL(join(path, '..')).href + '/' + await ctx.plugin(Loader) + ctx.loader.builtins.include = Include + ctx.loader.internal = { + version: 'sqlite', + async import(specifier: string) { + if (specifier === '@deepseek-ai/dsh-session') return SessionStore + if (specifier === '@deepseek-ai/dsh-session-persistence-sqlite') { + return SessionPersistenceSqlite + } + throw new Error(`unexpected Loader import: ${specifier}`) + }, + } as unknown as NonNullable + await ctx.loader.create({ + name: 'cordis:include', + config: { path: pathToFileURL(configPath).href }, }) + await ctx.loader.await() - it('preserves the full log when it ends exactly on a turn/end (no torn tail)', () => { - const { preserved, tornFrom } = scanRows(rows(oneTurnLog())) - expect(preserved).toEqual(oneTurnLog()) - expect(tornFrom).toBeUndefined() - }) - - it('PRESERVES the real events of an interrupted turn after the last turn/end', () => { - // turn 1 committed (0..5) + a crashed turn 2 (turn/start 6, step/start 7, no - // close): all 8 rows are intact, so the whole prefix is preserved and there - // is no torn fragment to delete. (load() then synthesizes the closers.) - const withOpenTurn: SessionEvent[] = [ - ...oneTurnLog(), - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2 } }, - { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } }, - ] - const { preserved, tornFrom } = scanRows(rows(withOpenTurn)) - expect(preserved.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - expect(tornFrom).toBeUndefined() - }) - - it('preserves the contiguous prefix and flags a torn tail at a seq gap', () => { - // A gap after seq 0 (no committed turn/end): seq 0 is the preserved - // interrupted-turn event; the gap bounds it and marks the torn fragment. - const gapped: SessionEvent[] = [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, - { type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // seq 1 missing - ] - const { preserved, tornFrom } = scanRows(rows(gapped)) - expect(preserved.map(e => e.seq)).toEqual([0]) - expect(tornFrom).toBe(1) - }) - - it('an empty log preserves nothing and has no torn tail', () => { - expect(scanRows([])).toEqual({ preserved: [] }) - }) - - it('throws on a seq gap inside the committed region (before the last turn/end)', () => { - const gapped: SessionEvent[] = [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, - { type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // seq 1 missing - { type: 'turn/end', seq: 3, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }, - ] - expect(() => scanRows(rows(gapped))).toThrow(/seq gap in committed region/) - }) - - it('throws on an unparsable row inside the committed region', () => { - const withCorruptCommitted: EventRow[] = [ - { seq: 0, type: 'turn/start', time: 1, data: '{not json', source_event_seqs: null, surface_op: null, ignorable: null }, // corrupt, sits before a turn/end - { seq: 1, type: 'turn/end', time: 2, data: JSON.stringify({ turn: 1, reason: { kind: 'completed' } }), source_event_seqs: null, surface_op: null, ignorable: null }, - ] - expect(() => scanRows(withCorruptCommitted)).toThrow(/unparsable committed event/) - }) - - it('tolerates an unparsable torn-tail row after the last turn/end', () => { - const withCorruptTail: EventRow[] = [ - ...rows(oneTurnLog()), - { seq: 6, type: 'turn/start', time: 7, data: '{not json', source_event_seqs: null, surface_op: null, ignorable: null }, // torn fragment, no committed turn/end after - ] - const { preserved, tornFrom } = scanRows(withCorruptTail) - expect(preserved).toEqual(oneTurnLog()) - expect(tornFrom).toBe(6) - }) -}) - -describe('rowToMeta', () => { - it('restores optional origin metadata', () => { - expect(rowToMeta({ - id: 'with-origin', - version: 0, - created_at: 1, - cwd: null, - parent_session: null, - seed_length: null, - origin: 'subagent', - incarnation: 'with-origin', - revision: 1, - delegation_depth: null, - agent_preset: null, - })).toMatchObject({ id: 'with-origin', origin: 'subagent' }) - }) - - it('rejects fractional stored creation metadata', () => { - expect(() => rowToMeta({ - id: 'fractional', - version: 0, - created_at: 1.5, - cwd: null, - parent_session: null, - seed_length: null, - origin: null, - incarnation: 'fractional', - revision: 1, - delegation_depth: null, - agent_preset: null, - })).toThrow('stored session createdAt must be a non-negative safe integer') - }) - - it('restores the agent preset a session was composed from', () => { - // The preset decides the resumed session's tools and prompt; a row that - // dropped it would rebuild a composition the stored history contradicts. - expect(rowToMeta({ - id: 'composed', - version: 0, - created_at: 1, - cwd: null, - parent_session: null, - seed_length: null, - origin: null, - incarnation: 'composed', - revision: 1, - delegation_depth: null, - agent_preset: 'minimal', - })).toMatchObject({ agentPreset: 'minimal' }) - }) -}) - -describe('SqliteSessionPersistence: durability and crash semantics', () => { - it('rejects a stored v0 log containing a legacy request/header-delta event', async () => { - const path = await freshDbPath() - const m = meta('legacy-header-delta', '/legacy') - const db = openDatabase(path, 'wal') - db.prepare('INSERT INTO sessions (id, version, created_at, cwd, parent_session, seed_length, delegation_depth, incarnation, revision) VALUES (?, ?, ?, ?, NULL, NULL, NULL, ?, 1)') - .run(m.id, m.version, m.createdAt, m.cwd ?? null, 'legacy-header-delta') - const insert = db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') - insert.run(m.id, 0, 'turn/start', 1, JSON.stringify({ turn: 1 })) - insert.run(m.id, 1, 'request/header-delta', 2, JSON.stringify({ config: { model: 'legacy' } })) - insert.run(m.id, 2, 'turn/end', 3, JSON.stringify({ turn: 1, reason: { kind: 'completed' } })) - db.close() - - const mounted = await backend(path) - await expect(mounted.ctx.sessionPersistence.load(m.id)).rejects.toThrow(/unsupported legacy request\/header-delta event at seq 1/) - await mounted.dispose() - }) - - it('rejects a stored v0 full header carrying the legacy fallback reason', async () => { - const path = await freshDbPath() - const m = meta('legacy-header-fallback', '/legacy') - const db = openDatabase(path, 'wal') - db.prepare('INSERT INTO sessions (id, version, created_at, cwd, parent_session, seed_length, delegation_depth, incarnation, revision) VALUES (?, ?, ?, ?, NULL, NULL, NULL, ?, 1)') - .run(m.id, m.version, m.createdAt, m.cwd ?? null, 'legacy-header-fallback') - db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') - .run(m.id, 0, 'request/header', 1, JSON.stringify({ - header: { config: { model: 'legacy' } }, - reason: 'fallback', - })) - db.close() - - const mounted = await backend(path) - await expect(mounted.ctx.sessionPersistence.load(m.id)) - .rejects.toThrow(/unsupported legacy request\/header reason "fallback" at seq 0/) - await mounted.dispose() - }) - - it('has no independent per-session log location', async () => { - const { ctx, dispose } = await backend() - expect(ctx.sessionPersistence.locate(meta('sqlite-location'))).toBeUndefined() - await dispose() - }) - - it('an interrupted turn (rows after the last turn/end) is PRESERVED and closed during load', async () => { - const path = await freshDbPath() - const m = meta('crash') - // Run 1: persist a complete turn, then a half-written second turn (no turn/end). - const ctx1 = new Context() - await ctx1.plugin(SessionStore) - const fiber1 = await ctx1.plugin(SqliteSessionPersistence, { path }) - await ctx1.sessionPersistence.create(m) - await ctx1.sessionPersistence.append(m.id, oneTurnLog()) - await ctx1.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2 } }, - { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } }, - ]) - await fiber1.dispose() - - // Run 2: load PRESERVES the interrupted turn's real events (a turn can be huge - // — never truncated) and closes the orphaned turn with synthetic boundary - // events: step/end (the step was open) then turn/end {interrupted}. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SqliteSessionPersistence, { path }) - const loaded = await ctx2.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.type)).toEqual([ - 'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1 - 'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real events + synthetic closers - ]) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) - const last = loaded.events.at(-1)! - expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' }) - - // load durably closed the turn, so the next append continues at the balanced - // length (seq 10) and a reload round-trips identically. - await ctx2.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 10, time: 9, data: { turn: 3 } }, - { type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } }, - ]) - const reloaded = await ctx2.sessionPersistence.load(m.id) - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) - await fiber2.dispose() - }) - - it('load() durably closes the interrupted turn: the synthetic closers are on disk after load', async () => { - const path = await freshDbPath() - const m = meta('load-closes') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5 - await b1.dispose() - // Hand-write an interrupted turn (turn/start seq 6, no turn/end). - const db = openDatabase(path, 'wal') - db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, 6, ?, 7, ?)') - .run(m.id, 'turn/start', JSON.stringify({ turn: 2 })) - db.close() - - const b2 = await backend(path) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - // turn 2's real turn/start (seq 6) is preserved + a synthetic turn/end (seq 7). - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - expect(loaded.events.at(-1)!.type).toBe('turn/end') - // load() is mutating: the synthetic turn/end MUST be on disk so the stored log - // is balanced and the cursor is truthful (contract: load closes, not defers). - const probe = openDatabase(path, 'wal') - const stored = probe.prepare('SELECT seq, type FROM events WHERE session_id = ? ORDER BY seq').all(m.id) as { seq: number; type: string }[] - probe.close() - expect(stored.map(r => r.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - expect(stored.at(-1)!.type).toBe('turn/end') - await b2.dispose() - }) - - it('all-tail load: a session whose only turn never closed is preserved and closed on load', async () => { - const path = await freshDbPath() - const m = meta('all-tail') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - // A first turn that NEVER completed: turn/start + user/message, no turn/end. - await b1.ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, - { type: 'user/message', seq: 1, time: 2, data: createUserMessage({ - content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' }, - }), surfaceOp: 'append' }, - ]) - await b1.dispose() - - // A fresh backend loads it: the interrupted (only) turn's real events are - // preserved and closed with a synthetic turn/end {interrupted} — NOT - // truncated. The session was materialized, so list() reports it present. - const b2 = await backend(path) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'user/message', 'turn/end']) - expect(loaded.events.at(-1)!.type === 'turn/end' && loaded.events.at(-1)!.data).toMatchObject({ reason: { kind: 'interrupted' } }) - expect((await b2.ctx.sessionPersistence.list()).map(x => x.id)).toContain(m.id) - await b2.dispose() - }) - - it('rejects opening a database whose schema version is not the current build (newer OR older)', async () => { - const path = await freshDbPath() - openDatabase(path, 'wal').close() // stamp user_version = SCHEMA_VERSION - // Bump user_version past what this build supports. - const dbNewer = openDatabase(path, 'wal') - dbNewer.exec(`PRAGMA user_version = ${SCHEMA_VERSION + 1}`) - dbNewer.close() - expect(() => openDatabase(path, 'wal')).toThrow(/incompatible with this build/) - - // The immediately preceding layout lacks the required store identity and is - // rejected rather than migrated (unreleased software, no backward-compat). - const olderPath = await freshDbPath() - openDatabase(olderPath, 'wal').close() - const dbOlder = openDatabase(olderPath, 'wal') - dbOlder.exec(`PRAGMA user_version = ${SCHEMA_VERSION - 1}`) - dbOlder.close() - expect(() => openDatabase(olderPath, 'wal')).toThrow(/incompatible with this build/) - }) - - it('rejects a table-backed unversioned database before stamping or changing journal mode', async () => { - const path = await freshDbPath() - const legacy = new DatabaseSync(path) - legacy.exec('CREATE TABLE sessions (id TEXT PRIMARY KEY)') - legacy.close() - - expect(() => openDatabase(path, 'wal')).toThrow(/unversioned schema or application identity/) - - const unchanged = new DatabaseSync(path) - expect(unchanged.prepare('PRAGMA user_version').get()).toEqual({ user_version: 0 }) - expect(unchanged.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' }) - expect(unchanged.prepare( - "SELECT name FROM sqlite_schema WHERE type = 'table' AND name = 'sessions'", - ).get()).toEqual({ name: 'sessions' }) - unchanged.close() - }) - - it('counts a sqliteX table as user-owned instead of mistaking it for SQLite metadata', async () => { - const path = await freshDbPath() - const unrelated = new DatabaseSync(path) - unrelated.exec('CREATE TABLE sqliteX (value TEXT)') - unrelated.exec("INSERT INTO sqliteX VALUES ('safe')") - unrelated.close() - - expect(() => openDatabase(path, 'wal')).toThrow(/unversioned schema or application identity/) - - const unchanged = new DatabaseSync(path) - expect(unchanged.prepare('SELECT value FROM sqliteX').get()).toEqual({ value: 'safe' }) - expect(unchanged.prepare('PRAGMA application_id').get()).toEqual({ application_id: 0 }) - expect(unchanged.prepare('PRAGMA user_version').get()).toEqual({ user_version: 0 }) - expect(unchanged.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' }) - unchanged.close() - }) - - it('rejects view-only and foreign-application unversioned databases without mutation', async () => { - const viewPath = await freshDbPath() - const viewOnly = new DatabaseSync(viewPath) - viewOnly.exec('CREATE VIEW foreign_view AS SELECT 1 AS value') - viewOnly.close() - - expect(() => openDatabase(viewPath, 'wal')).toThrow(/unversioned schema or application identity/) - const unchangedView = new DatabaseSync(viewPath) - expect(unchangedView.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' }) - expect(unchangedView.prepare( - "SELECT type FROM sqlite_schema WHERE name = 'foreign_view'", - ).get()).toEqual({ type: 'view' }) - unchangedView.close() - - const applicationPath = await freshDbPath() - const foreignApplication = new DatabaseSync(applicationPath) - foreignApplication.exec('PRAGMA application_id = 12345') - foreignApplication.close() - - expect(() => openDatabase(applicationPath, 'wal')).toThrow(/unversioned schema or application identity/) - const unchangedApplication = new DatabaseSync(applicationPath) - expect(unchangedApplication.prepare('PRAGMA application_id').get()).toEqual({ application_id: 12345 }) - expect(unchangedApplication.prepare('PRAGMA user_version').get()).toEqual({ user_version: 0 }) - expect(unchangedApplication.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' }) - unchangedApplication.close() - }) - - it('rejects a current-version database with a foreign application identity', async () => { - const path = await freshDbPath() - const foreign = new DatabaseSync(path) - foreign.exec('PRAGMA application_id = 12345') - foreign.exec(`PRAGMA user_version = ${SCHEMA_VERSION}`) - foreign.close() - - expect(() => openDatabase(path, 'wal')).toThrow(/has application id 12345/) - - const unchanged = new DatabaseSync(path) - expect(unchanged.prepare('PRAGMA application_id').get()).toEqual({ application_id: 12345 }) - expect(unchanged.prepare('PRAGMA user_version').get()).toEqual({ user_version: SCHEMA_VERSION }) - expect(unchanged.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' }) - unchanged.close() - }) - - it('rolls back schema objects and identity stamps when initialization fails', async () => { - const path = await freshDbPath() - const conflicting = new DatabaseSync(path) - conflicting.exec(`PRAGMA application_id = ${SESSION_PERSISTENCE_SQLITE_APPLICATION_ID}`) - conflicting.exec(`PRAGMA user_version = ${SCHEMA_VERSION}`) - conflicting.exec("CREATE VIEW persistence_state AS SELECT 1 AS singleton, 'foreign' AS store_id") - conflicting.close() - - expect(() => openDatabase(path, 'wal')).toThrow() - - const unchanged = new DatabaseSync(path) - expect(unchanged.prepare( - "SELECT type FROM sqlite_schema WHERE name = 'persistence_state'", - ).get()).toEqual({ type: 'view' }) - expect(unchanged.prepare( - "SELECT type FROM sqlite_schema WHERE name = 'sessions'", - ).get()).toBeUndefined() - expect(unchanged.prepare( - "SELECT type FROM sqlite_schema WHERE name = 'events'", - ).get()).toBeUndefined() - expect(unchanged.prepare('PRAGMA application_id').get()) - .toEqual({ application_id: SESSION_PERSISTENCE_SQLITE_APPLICATION_ID }) - expect(unchanged.prepare('PRAGMA user_version').get()).toEqual({ user_version: SCHEMA_VERSION }) - expect(unchanged.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' }) - unchanged.close() - }) - - it('stamps the persistence application identity with the schema version', async () => { - const path = await freshDbPath() - openDatabase(path, 'wal').close() + const header = meta('loader') + const events = chunkLog(4) + await ctx.sessionPersistence.create(header) + await ctx.sessionPersistence.append(header.id, events) + expect((await ctx.sessionPersistence.inspect(header.id)).events).toEqual(events) + await ctx.fiber.dispose() const db = new DatabaseSync(path) - expect(db.prepare('PRAGMA application_id').get()) - .toEqual({ application_id: SESSION_PERSISTENCE_SQLITE_APPLICATION_ID }) - expect(db.prepare('PRAGMA user_version').get()).toEqual({ user_version: SCHEMA_VERSION }) + expect(db.prepare(testSql('count-packed-events')).get()) + .toEqual({ count: 1 }) db.close() }) - it('rejects a sibling v3 database (the merge-collided version) rather than opening it against missing columns', async () => { - // Version 3 identified two incompatible sibling layouts, so it is always rejected. + it('packs each append once without rewriting earlier rows and seeks inside packed rows', async () => { const path = await freshDbPath() - openDatabase(path, 'wal').close() // creates + stamps user_version = SCHEMA_VERSION - const db = openDatabase(path, 'wal') - db.exec('PRAGMA user_version = 3') - db.close() - expect(() => openDatabase(path, 'wal')).toThrow(/schema version 3, incompatible with this build/) - }) - - it('a corrupt-JSON row in the uncommitted tail is discarded on load, not unloadable', async () => { - const path = await freshDbPath() - const m = meta('corrupt-tail') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) // committed: seqs 0..5 - await b1.dispose() - - // A torn row after the last committed turn has invalid JSON. `scanRows` locates the boundary - // from seq/type columns without parsing the tail, preserves the committed prefix, and load - // deletes the row; invalid JSON inside the committed region would remain fatal. - const db = openDatabase(path, 'wal') - db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, 6, ?, 7, ?)') - .run(m.id, 'turn/start', '{not valid json') - db.close() - - const b2 = await backend(path) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.events).toEqual(oneTurnLog()) // torn tail discarded, committed intact (turn 1 already balanced → no closers) - // load physically deleted the corrupt tail row, so a fresh append continues. - await b2.ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 8, data: { turn: 2 } }, - { type: 'turn/end', seq: 7, time: 9, data: { turn: 2, reason: { kind: 'completed' } } }, - ]) - const reloaded = await b2.ctx.sessionPersistence.load(m.id) - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await b2.dispose() - }) - - it('append rolls back the whole batch on a mid-batch seq collision (transaction)', async () => { const ctx = new Context() await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SqliteSessionPersistence, { path: ':memory:' }) - const m = meta('rollback') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5 + const fiber = await ctx.plugin(SessionPersistenceSqlite, { path }) + const header = meta('packed') + const events = chunkLog(100) + await ctx.sessionPersistence.create(header) + await ctx.sessionPersistence.append(header.id, events.slice(0, 3)) + await ctx.sessionPersistence.append(header.id, events.slice(3, 4)) + const before = new DatabaseSync(path, { readOnly: true }) + const originalRows = before.prepare(testSql('select-event-rowids')).all() + before.close() + await ctx.sessionPersistence.append(header.id, events.slice(4)) - // A batch that re-states an already-stored seq must be rejected and leave - // the stored log unchanged (the UNIQUE (session_id, seq) constraint fires - // inside the transaction → ROLLBACK). - await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).rejects.toThrow() - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events).toEqual(oneTurnLog()) // unchanged + const inspected = await ctx.sessionPersistence.inspect(header.id) + expect(inspected.events).toEqual(events) + for (const fromSeq of [0, 2, 25, 101, 104, 105]) { + expect((await ctx.sessionPersistence.readFrom(header.id, fromSeq)).events) + .toEqual(events.filter(event => event.seq >= fromSeq)) + } await fiber.dispose() + + const db = new DatabaseSync(path) + expect(db.prepare(testSql('select-user-version')).get()).toEqual({ user_version: SCHEMA_VERSION }) + expect(db.prepare(testSql('count-events')).get()).toEqual({ count: 7 }) + expect(db.prepare(testSql('count-packed-events')).get()) + .toEqual({ count: 1 }) + expect(db.prepare(testSql('select-event-rowids')).all().slice(0, originalRows.length)) + .toEqual(originalRows) + db.close() }) - it('persists across separate backend instances over the same file', async () => { - const path = await freshDbPath() - const m = meta('persist', '/proj') - const ctx1 = new Context() - await ctx1.plugin(SessionStore) - const fiber1 = await ctx1.plugin(SqliteSessionPersistence, { path }) - await ctx1.sessionPersistence.create(m) - await ctx1.sessionPersistence.append(m.id, oneTurnLog()) - await fiber1.dispose() + it.runIf(process.platform !== 'win32')('bounds paced-stream WAL extent without rewriting committed rows', async () => { + const events = chunkLog(1_000) + const measured = await measureWriteTraffic(await freshDbPath('dsh-sqlite-traffic-'), events) - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SqliteSessionPersistence, { path }) - expect((await ctx2.sessionPersistence.list()).map(x => x.id)).toContain(m.id) - const loaded = await ctx2.sessionPersistence.load(m.id) - expect(loaded.meta).toMatchObject({ id: m.id, cwd: '/proj' }) - expect(loaded.events).toEqual(oneTurnLog()) - await fiber2.dispose() + expect(measured).toMatchObject({ rows: 31, inserted: 31, changed: 0, removed: 0 }) + expect(measured.inserted).toBe(measured.rows) + expect(measured.largest).toBeLessThanOrEqual(MAX_PACKED_DATA_BYTES) + expect(measured.idleWalBytes).toBe(measured.walBytes) }) - it('source-qualifies revisions across stores while preserving same-file reopen identity', async () => { - const pathA = await freshDbPath() - const pathB = await freshDbPath() - const m = meta('revision-source') - const a = await backend(pathA) - await a.ctx.sessionPersistence.create(m) - await a.ctx.sessionPersistence.append(m.id, oneTurnLog()) - const revisionA = (await a.ctx.sessionPersistence.listSnapshots())[0]?.revision - await a.dispose() + it('includes a packed predecessor when an overlapping scalar tail hides it', async () => { + const path = await freshDbPath('dsh-sqlite-overlap-') + const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS }) + const header = meta('overlap') + await store.appendBatch(header, [chunk(0), chunk(1), chunk(2)], false) - const probeA = openDatabase(pathA, 'wal') - const storeIdA = (probeA.prepare( - 'SELECT store_id FROM persistence_state WHERE singleton = 1', - ).get() as { store_id: string }).store_id - probeA.close() + const db = new DatabaseSync(path) + db.prepare(testSql('insert-corrupt-event')) + .run(header.id, 1, 'assistant/chunk', 2, JSON.stringify(chunk(1).data), null) + db.close() - const aliasA = `${pathA}.alias` - await symlink(pathA, aliasA) - const reopenedA = await backend(aliasA) - expect((await reopenedA.ctx.sessionPersistence.listSnapshots())[0]?.revision).toBe(revisionA) - await reopenedA.dispose() + expect((await store.loadStoredFrom(header.id, 2))?.events).toEqual([chunk(2)]) - const b = await backend(pathB) - await b.ctx.sessionPersistence.create(m) - await b.ctx.sessionPersistence.append(m.id, oneTurnLog()) - const revisionB = (await b.ctx.sessionPersistence.listSnapshots())[0]?.revision - const probeB = openDatabase(pathB, 'wal') - const storeIdB = (probeB.prepare( - 'SELECT store_id FROM persistence_state WHERE singleton = 1', - ).get() as { store_id: string }).store_id - probeB.close() - expect(storeIdB).not.toBe(storeIdA) - expect(revisionB).not.toBe(revisionA) - expect(String(revisionA)).toMatch(/:revision:1$/) - expect(String(revisionB)).toMatch(/:revision:1$/) - await b.dispose() + const malformed = new DatabaseSync(path) + malformed.prepare(testSql('delete-session-events')).run(header.id) + malformed.prepare(testSql('insert-corrupt-event')) + .run(header.id, 0, 'text-chunks', 1, '{not json', 0) + malformed.close() + expect((await store.loadStoredFrom(header.id, 2))?.events).toEqual([]) + await store.close() }) - it('binds a full stored prefix to the same revision as a lightweight read', async () => { - const b = await backend() - const m = meta('stored-prefix-revision') - await b.ctx.sessionPersistence.create(m) - await b.ctx.sessionPersistence.append(m.id, oneTurnLog()) - const persistence = b.ctx.sessionPersistence as SqliteSessionPersistence + it('waits for a competing process within the configured busy timeout', async () => { + const path = await freshDbPath('dsh-sqlite-busy-') + const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: 1_000 }) + const header = meta('busy') + await store.appendBatch(header, [chunk(0)], false) - const stored = await persistence.loadStored(m.id) - expect(stored?.revision).toBe(await persistence.readStoredRevision(m.id)) - expect(await persistence.readStoredRevision(SessionId('missing-revision'))).toBeUndefined() - await b.dispose() - }) - - it('changes revisions when a deleted session id is materialized again in the same database', async () => { - const path = await freshDbPath() - const m = meta('recreated-revision') - const first = await backend(path) - await first.ctx.sessionPersistence.create(m) - await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) - const before = (await first.ctx.sessionPersistence.listSnapshots())[0]?.revision - await first.dispose() - - const cleanup = openDatabase(path, 'wal') - cleanup.prepare('DELETE FROM sessions WHERE id = ?').run(m.id) - cleanup.close() - - const second = await backend(path) - await second.ctx.sessionPersistence.create(m) - await second.ctx.sessionPersistence.append(m.id, oneTurnLog()) - const after = (await second.ctx.sessionPersistence.listSnapshots())[0]?.revision - expect(after).not.toBe(before) - expect(String(before)).toMatch(/:revision:1$/) - expect(String(after)).toMatch(/:revision:1$/) - await second.dispose() - }) - - it('awaits in-flight readiness before surfacing snapshot-list cancellation', async () => { - const b = await backend() - const internals = b.ctx.sessionPersistence as unknown as { ready: Promise } - const originalReady = internals.ready - const readiness = Promise.withResolvers() - internals.ready = readiness.promise - const reason = new Error('SQLite snapshot readiness cancelled') - const controller = new AbortController() - const pending = b.ctx.sessionPersistence.listSnapshots(controller.signal) - let settled = false - void pending.then( - () => { settled = true }, - () => { settled = true }, - ) - - controller.abort(reason) - await Promise.resolve() - expect(settled).toBe(false) - - readiness.resolve(undefined) - await expect(pending).rejects.toBe(reason) - internals.ready = originalReady - await b.dispose() - }) - - it('exposes the schema version constant', () => { - expect(SCHEMA_VERSION).toBe(15) - }) - - it('keeps the revision stable for an empty repair hook', async () => { - const b = await backend() - const m = meta('empty-repair') - await b.ctx.sessionPersistence.create(m) - await b.ctx.sessionPersistence.append(m.id, oneTurnLog()) - const before = await b.ctx.sessionPersistence.listSnapshots() - await (b.ctx.sessionPersistence as SqliteSessionPersistence).commitRepair(m, undefined, []) - expect(await b.ctx.sessionPersistence.listSnapshots()).toEqual(before) - await b.dispose() - }) -}) - -describe('SqliteSessionPersistence: edge cases', () => { - it('resolves the preparation-cache default without schema normalization', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - let persistence!: SqliteSessionPersistence - await ctx.plugin(Object.assign((inner: Context) => { - persistence = new SqliteSessionPersistence(inner, { - path: ':memory:', - journalMode: 'wal', - }) - }, { inject: ['sessions'] })) - - expect(await persistence.list()).toEqual([]) - await ctx.fiber.dispose() - }) - - it('uses the configured preparation cache through the public service', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SqliteSessionPersistence, { - path: ':memory:', - preparedSessionCacheSize: 1, - writeBatchMaxDelayMs: 1, + const holder = spawn(process.execPath, ['--input-type=module', '-e', String.raw` + import { DatabaseSync } from 'node:sqlite'; + const db = new DatabaseSync(process.argv[1]); + db.exec('BEGIN IMMEDIATE'); + process.stdout.write('locked\n'); + setTimeout(() => { db.exec('COMMIT'); db.close(); }, 100); + `, path], { stdio: ['ignore', 'pipe', 'pipe'] }) + const exited = new Promise((resolve, reject) => { + holder.once('error', reject) + holder.once('exit', resolve) }) - const m = meta('sqlite-preparation-cache') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - - const preparation = await ctx.sessionPersistence.prepare(m.id) - expect(preparation.session.header).toEqual(m) - preparation[Symbol.dispose]() - await fiber.dispose() + try { + await once(holder.stdout, 'data') + await expect(store.appendBatch(header, [chunk(1)], true)).resolves.toBeUndefined() + const code = await exited + expect(code).toBe(0) + expect((await store.loadStored(header.id))?.events).toEqual([chunk(0), chunk(1)]) + } finally { + if (holder.exitCode === null) holder.kill() + await store.close() + } }) - it('rejects and closes a current-schema database with an invalid store identity', async () => { - const path = await freshDbPath() - const db = openDatabase(path, 'wal') - db.exec("UPDATE persistence_state SET store_id = '' WHERE singleton = 1") + it('rejects an older SQLite physical schema', async () => { + const path = await freshDbPath('dsh-sqlite-old-schema-') + const seed = await openDatabase(DatabaseSync, path, 'wal', DEFAULT_BUSY_TIMEOUT_MS) + seed.exec(testSql('set-user-version-16')) + seed.close() + await chmod(path, 0o600) + await expect(openDatabase(DatabaseSync, path, 'wal', DEFAULT_BUSY_TIMEOUT_MS)) + .rejects.toThrow(/schema version 16.*incompatible/) + }) + + it('rejects a stale physical append without replacing the winning tail', async () => { + const path = await freshDbPath('dsh-sqlite-stale-') + const first = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS }) + const second = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS }) + const header = meta(SessionId('stale')) + await first.appendBatch(header, [chunk(0)], false) + await second.appendBatch(header, [chunk(1)], true) + await expect(first.appendBatch(header, [chunk(1)], true)).rejects.toThrow(/stored next seq is 2/) + expect((await first.loadStored(header.id))?.events).toEqual([chunk(0), chunk(1)]) + await first.close() + await second.close() + }) + + it('rejects a stale repair without deleting a newer winning tail', async () => { + const path = await freshDbPath('dsh-sqlite-stale-repair-') + const stale = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS }) + const winner = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS }) + const header = meta(SessionId('stale-repair')) + await stale.appendBatch(header, [chunk(0)], false) + const db = new DatabaseSync(path) + db.prepare(testSql('insert-corrupt-event')).run(header.id, 1, 'assistant/chunk', 2, '{not json', null) db.close() + expect((await stale.loadStored(header.id))?.tornMarker).toBe(1) + await winner.commitRepair(header, 1, []) + await winner.appendBatch(header, [chunk(1), chunk(2)], true) + await expect(stale.commitRepair(header, 1, [])).rejects.toThrow(/repair is stale/) + expect((await stale.loadStored(header.id))?.events).toEqual([chunk(0), chunk(1), chunk(2)]) + await stale.close() + await winner.close() + }) +}) - const b = await backend(path) - await expect(b.ctx.sessionPersistence.listSnapshots()).rejects.toThrow(/no valid store identity/) - await expect(b.dispose()).resolves.toBeUndefined() +describe('SessionPersistenceSqlite schema ownership', () => { + it('accepts every configured journal mode and SQLite memory mode result', async () => { + const resources = { + wal: 'journal-mode-wal', + delete: 'journal-mode-delete', + truncate: 'journal-mode-truncate', + persist: 'journal-mode-persist', + } as const + for (const mode of ['wal', 'delete', 'truncate', 'persist'] as const) { + ;(await openDatabase(DatabaseSync, ':memory:', mode, DEFAULT_BUSY_TIMEOUT_MS)).close() + const path = await freshDbPath(`dsh-sqlite-journal-${mode}-`) + const db = await openDatabase(DatabaseSync, path, mode, DEFAULT_BUSY_TIMEOUT_MS) + expect(db.prepare(sql(resources[mode])).get()).toEqual({ journal_mode: mode }) + expect(db.prepare(sql('select-trusted-schema')).get()).toEqual({ trusted_schema: 0 }) + expect(db.prepare(sql('select-mmap-size')).get()).toEqual({ mmap_size: 0 }) + expect(db.prepare(sql('select-synchronous')).get()).toEqual({ synchronous: 2 }) + db.close() + } }) - it('creates a new database and WAL sidecars with owner-only modes without changing its parent mode', async () => { - if (process.platform === 'win32') return - const path = await freshDbPath() - const dir = dirname(path) - await chmod(dir, 0o755) + it('retries a busy journal-mode transition within its retry budget', async () => { + const path = await freshDbPath('dsh-sqlite-journal-busy-') + let attempts = 0 + const BusyOnceDatabase = databaseWithJournalFailure(() => { + attempts += 1 + return attempts === 1 + ? Object.assign(new Error('database is locked'), { + code: 'ERR_SQLITE_ERROR', + errcode: 5, + errstr: 'database is locked', + }) + : undefined + }) - const b = await backend(path) - await b.ctx.sessionPersistence.list() - - expect((await stat(dir)).mode & 0o777).toBe(0o755) - expect((await stat(path)).mode & 0o777).toBe(0o600) - expect((await stat(`${path}-wal`)).mode & 0o777).toBe(0o600) - expect((await stat(`${path}-shm`)).mode & 0o777).toBe(0o600) - await b.dispose() - }) - - it('creates a persistent rollback journal with owner-only mode', async () => { - if (process.platform === 'win32') return - const path = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SqliteSessionPersistence, { path, journalMode: 'persist' }) - const m = meta('persist-permissions') - - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - - expect((await stat(path)).mode & 0o777).toBe(0o600) - expect((await stat(`${path}-journal`)).mode & 0o777).toBe(0o600) - await fiber.dispose() - }) - - it('preserves the mode of an existing database file', async () => { - if (process.platform === 'win32') return - const path = await freshDbPath() - await writeFile(path, '', { mode: 0o644 }) - await chmod(path, 0o644) - - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SqliteSessionPersistence, { path, journalMode: 'delete' }) - await ctx.sessionPersistence.list() - - expect((await stat(path)).mode & 0o777).toBe(0o644) - await fiber.dispose() - }) - - it('surfaces an invalid database path during pre-creation', async () => { - const path = await freshDbPath() - const b = await backend(`${path}\0`) - - await expect(b.ctx.sessionPersistence.list()).rejects.toMatchObject({ code: 'ERR_INVALID_ARG_VALUE' }) - await b.dispose() - }) - - it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => { - const path = await freshDbPath() - const m = meta('rollback-insert') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - - // A SECOND backend over the same file loads the session first, so it adopts - // cursor 6 (the committed length) into its OWN in-memory state. - const b2 = await backend(path) - await b2.ctx.sessionPersistence.load(m.id) // cursor 6 in b2 - const turn2: SessionEvent[] = [ - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2 } }, - { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, - ] - // b1 commits seq 6..7 first. - await b1.ctx.sessionPersistence.append(m.id, turn2) - // b2 still thinks its cursor is 6, so this batch passes the contiguity check - // but its INSERT of seq 6 hits the UNIQUE (session_id, seq) constraint - // mid-transaction → ROLLBACK + rethrow. - await expect(b2.ctx.sessionPersistence.append(m.id, turn2)).rejects.toThrow(/UNIQUE/) - // b1's turn is intact; b2's rolled-back attempt left nothing extra. - const loaded = await b1.ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await b1.dispose() - await b2.dispose() - }) - - it('journalMode config reaches the database (default wal, rollback modes selectable)', async () => { - // :memory: databases always report journal_mode=memory, so probe file DBs. - const walPath = await freshDbPath() - const bWal = await backend(walPath) - await bWal.ctx.sessionPersistence.create(meta('jm-wal')) - const probe = openDatabase(walPath, 'wal') - expect((probe.prepare('PRAGMA journal_mode').get() as { journal_mode: string }).journal_mode).toBe('wal') - probe.close() - await bWal.dispose() - - const deletePath = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SqliteSessionPersistence, { path: deletePath, journalMode: 'delete' }) - await ctx.sessionPersistence.create(meta('jm-delete')) - // Probe through a second connection: journal_mode=delete is a per-database - // property only insofar as no WAL files exist — assert the world, not the - // backend's self-report (no -wal sidecar after writes in delete mode). - const db = openDatabase(deletePath, 'delete') - expect((db.prepare('PRAGMA journal_mode').get() as { journal_mode: string }).journal_mode).toBe('delete') + const db = await openDatabase(BusyOnceDatabase, path, 'wal', 100) + expect(attempts).toBe(2) + expect(db.prepare(sql('journal-mode-wal')).get()).toEqual({ journal_mode: 'wal' }) + expect(db.prepare(sql('select-trusted-schema')).get()).toEqual({ trusted_schema: 0 }) + expect(db.prepare(sql('select-mmap-size')).get()).toEqual({ mmap_size: 0 }) + expect(db.prepare(sql('select-synchronous')).get()).toEqual({ synchronous: 2 }) db.close() - expect(existsSync(`${deletePath}-wal`)).toBe(false) - await fiber.dispose() }) - it('HMR: a DIFFERENT session colliding with a materialized on-disk id is rejected', async () => { - const path = await freshDbPath() - // Instance 1 materializes a session and disposes. - const b1 = await backend(path) - const s1 = b1.ctx.sessions.create(SessionId('hmr-collide')) - appendLog(s1, oneTurnLog()) - await b1.ctx.sessions.flush(s1) - await b1.dispose() + it('does not retry journal failures outside the available busy budget', async () => { + for (const { errcode, timeout } of [ + { errcode: 5, timeout: 0 }, + { errcode: 6, timeout: 100 }, + ]) { + let attempts = 0 + const FailingDatabase = databaseWithJournalFailure(() => { + attempts += 1 + return Object.assign(new Error(`SQLite error ${errcode}`), { errcode }) + }) + await expect(openDatabase( + FailingDatabase, + await freshDbPath(`dsh-sqlite-journal-failure-${errcode}-`), + 'wal', + timeout, + )).rejects.toThrow(`SQLite error ${errcode}`) + expect(attempts).toBe(1) + } + }) - // A fresh context with an UNRELATED live session reusing the id meets a - // materialized row that is NOT a prefix of its events → reject. + it('starts no journal retry after its open-relative cutoff', async () => { + let attempts = 0 + const BusyDatabase = databaseWithJournalFailure(() => { + attempts += 1 + return Object.assign(new Error('database is locked'), { errcode: 5 }) + }) + const clock = vi.spyOn(performance, 'now') + .mockReturnValueOnce(0) + .mockReturnValueOnce(50) + .mockReturnValueOnce(100) + try { + await expect(openDatabase( + BusyDatabase, + await freshDbPath('dsh-sqlite-journal-cutoff-'), + 'wal', + 100, + )).rejects.toThrow('database is locked') + } finally { + clock.mockRestore() + } + expect(attempts).toBe(1) + }) + + it('paces repeated busy journal-mode attempts', async () => { + let attempts = 0 + const BusyDatabase = databaseWithJournalFailure(() => { + attempts += 1 + return Object.assign(new Error('database is locked'), { errcode: 5 }) + }) + await expect(openDatabase( + BusyDatabase, + await freshDbPath('dsh-sqlite-journal-paced-'), + 'wal', + 50, + )).rejects.toThrow('database is locked') + expect(attempts).toBeGreaterThan(1) + expect(attempts).toBeLessThanOrEqual(6) + }) + + it('rejects unversioned, incompatible, and foreign-application databases', async () => { + const unversionedPath = await freshDbPath('dsh-sqlite-unversioned-') + const unversioned = new DatabaseSync(unversionedPath) + unversioned.exec(testSql('create-unrelated-table')) + unversioned.close() + await expect(openDatabase(DatabaseSync, unversionedPath, 'wal', DEFAULT_BUSY_TIMEOUT_MS)).rejects.toThrow(/unversioned schema/) + + const incompatiblePath = await freshDbPath('dsh-sqlite-incompatible-') + const incompatible = new DatabaseSync(incompatiblePath) + incompatible.exec(testSql('set-user-version-16')) + incompatible.close() + await expect(openDatabase(DatabaseSync, incompatiblePath, 'wal', DEFAULT_BUSY_TIMEOUT_MS)).rejects.toThrow(/incompatible with this build/) + + const foreignPath = await freshDbPath('dsh-sqlite-foreign-') + const foreign = new DatabaseSync(foreignPath) + foreign.exec(testSql('set-user-version-17')) + foreign.exec(testSql('set-application-id-12345')) + foreign.close() + await expect(openDatabase(DatabaseSync, foreignPath, 'wal', DEFAULT_BUSY_TIMEOUT_MS)).rejects.toThrow(/has application id 12345/) + }) + + it('rejects changed columns and non-strict owned tables', async () => { + const changedPath = await freshDbPath('dsh-sqlite-columns-') + ;(await openDatabase(DatabaseSync, changedPath, 'wal', DEFAULT_BUSY_TIMEOUT_MS)).close() + const changed = new DatabaseSync(changedPath) + changed.exec(testSql('add-unexpected-column')) + changed.close() + await expect(openDatabase(DatabaseSync, changedPath, 'wal', DEFAULT_BUSY_TIMEOUT_MS)).rejects.toThrow(/required schema objects/) + + const nonStrictPath = await freshDbPath('dsh-sqlite-nonstrict-') + ;(await openDatabase(DatabaseSync, nonStrictPath, 'wal', DEFAULT_BUSY_TIMEOUT_MS)).close() + const nonStrict = new DatabaseSync(nonStrictPath) + nonStrict.exec(testSql('replace-events-with-nonstrict-table')) + nonStrict.close() + await expect(openDatabase(DatabaseSync, nonStrictPath, 'wal', DEFAULT_BUSY_TIMEOUT_MS)).rejects.toThrow(/required schema objects/) + + const loosePath = await freshDbPath('dsh-sqlite-loose-') + const loose = new DatabaseSync(loosePath) + loose.exec(testSql('create-loose-schema')) + loose.close() + await expect(openDatabase(DatabaseSync, loosePath, 'wal', DEFAULT_BUSY_TIMEOUT_MS)).rejects.toThrow(/required schema objects/) + }) + + it('rejects schema ownership changes observed at mutation time', async () => { + const changedVersion = await openDatabase(DatabaseSync, ':memory:', 'wal', DEFAULT_BUSY_TIMEOUT_MS) + changedVersion.exec(testSql('set-user-version-16')) + expect(() => { validateSchemaForMutation(DatabaseSync, changedVersion, ':memory:') }) + .toThrow(/schema changed before mutation/) + changedVersion.close() + + const changedApplication = await openDatabase(DatabaseSync, ':memory:', 'wal', DEFAULT_BUSY_TIMEOUT_MS) + changedApplication.exec(testSql('set-application-id-12345')) + expect(() => { validateSchemaForMutation(DatabaseSync, changedApplication, ':memory:') }) + .toThrow(/application id changed before mutation/) + changedApplication.close() + }) + + it('validates creation time and restores every optional header field', () => { + const base: SessionRow = { + id: 'stored-header', + version: 0, + created_at: 1, + cwd: '/project', + parent_session: 'parent', + seed_length: 4, + origin: 'subagent', + incarnation: '00000000-0000-4000-8000-000000000000', + revision: 1, + delegation_depth: 2, + agent_preset: 'minimal', + } + expect(rowToMeta(decodeSessionRow(base))).toMatchObject({ + cwd: '/project', + parentSession: 'parent', + seedLength: 4, + origin: 'subagent', + delegationDepth: 2, + agentPreset: 'minimal', + }) + expect(() => decodeSessionRow({ ...base, created_at: -1 })).toThrow(/created_at/) + expect(() => decodeSessionRow({ ...base, origin: 'external' })).toThrow(/origin/) + expect(() => decodeSessionRow({ ...base, delegation_depth: -1 })).toThrow(/delegation_depth/) + }) + + it('rejects malformed SQLite row primitives generically', () => { + const base: SessionRow = { + id: 'stored-header', + version: 0, + created_at: 1, + cwd: '/project', + parent_session: null, + seed_length: null, + origin: null, + incarnation: '00000000-0000-4000-8000-000000000000', + revision: 1, + delegation_depth: null, + agent_preset: null, + } + for (const [value, message] of [ + [null, /object/], + [{ ...base, id: 1 }, /id.*string/], + [{ ...base, id: '' }, /id.*empty/], + [{ ...base, version: '0' }, /version.*safe integer/], + [{ ...base, cwd: 'relative' }, /cwd.*absolute/], + [{ ...base, cwd: 1 }, /cwd.*string or null/], + [{ ...base, incarnation: 'invalid' }, /incarnation.*UUID/], + [{ ...base, seed_length: '1' }, /seed_length.*safe integer or null/], + [{ ...base, agent_preset: 1 }, /agent_preset.*string or null/], + ] as const) { + expect(() => decodeSessionRow(value)).toThrow(message) + } + + const eventRow = { + seq: 0, type: 'turn/start', time: 1, data: '{}', + source_event_seqs: null, surface_op: null, ignorable: null, + } + for (const [value, message] of [ + [null, /object/], + [{ ...eventRow, seq: '0' }, /seq.*safe integer/], + [{ ...eventRow, type: '' }, /type.*empty/], + [{ ...eventRow, time: '1' }, /time.*safe integer/], + [{ ...eventRow, data: 1 }, /data.*string or blob/], + [{ ...eventRow, source_event_seqs: 1 }, /source_event_seqs.*blob or null/], + [{ ...eventRow, ignorable: 2 }, /ignorable.*0, 1, or null/], + ] as const) { + expect(() => decodeEventRow(value)).toThrow(message) + } + expect(() => decodeStoreIdentity({ store_id: 'invalid' })).toThrow(/store_id.*UUID/) + }) + + it('rejects invalid durable metadata before exposing a session header', async () => { + const path = await freshDbPath('dsh-sqlite-metadata-') + const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS }) + const header = meta('invalid-metadata') + await store.appendBatch(header, [chunk(0)], false) + const db = new DatabaseSync(path) + db.prepare(testSql('update-invalid-session-metadata')).run(header.id) + db.close() + await expect(store.list()).rejects.toThrow(/seed_length|origin|delegation_depth/) + await expect(store.loadStored(header.id)).rejects.toThrow(/seed_length|origin|delegation_depth/) + await store.close() + }) + + it('uses the shared persistence application identity', () => { + expect(SESSION_PERSISTENCE_SQLITE_APPLICATION_ID).toBe(0x44534850) + }) +}) + +describe('SessionPersistenceSqlite edge behavior', () => { + it('materializes an explicitly durable empty live session', async () => { + const path = await freshDbPath('dsh-sqlite-empty-') const ctx = new Context() await ctx.plugin(SessionStore) - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create(SessionId('hmr-collide')) - }, { inject: ['sessions'] })) - session.append('turn/start', { turn: 1 }) - await ctx.plugin(SqliteSessionPersistence, { path }) - await expectFlushError(ctx.sessions.flush(session), /id collision/) + await ctx.plugin(SessionPersistenceSqlite, { path }) + const session = ctx.sessions.create(SessionId('empty'), { meta: { cwd: '/workspace' } }) + + await ctx.sessionPersistence.ensureMaterialized(session) + + await expect(ctx.sessionPersistence.list()).resolves.toEqual([session.header]) + await expect(ctx.sessionPersistence.load(session.id)).resolves.toEqual({ meta: session.header, events: [] }) await ctx.fiber.dispose() }) -}) -describe('surface field round-trip', () => { - it('rowToEvent parses surface fields from EventRow columns', () => { - const row: EventRow = { - seq: 0, type: 'assistant/message', time: 1, - data: JSON.stringify({ turn: 1, step: 1, content: [] }), - source_event_seqs: JSON.stringify([3, 5]), - surface_op: JSON.stringify('append'), - ignorable: null, - } - const event = rowToEvent(row) - expect((event as SurfaceEvent).sourceEventSeqs).toEqual([3, 5]) - expect((event as SurfaceEvent).surfaceOp).toBe('append') - }) - - it('rowToEvent handles replace surfaceOp object', () => { - const row: EventRow = { - seq: 0, type: 'assistant/message', time: 1, - data: JSON.stringify({ turn: 1, step: 1, content: [] }), - source_event_seqs: JSON.stringify([0, 1]), - surface_op: JSON.stringify({ op: 'replace', start: 0, end: 1 }), - ignorable: null, - } - const event = rowToEvent(row) - expect((event as SurfaceEvent).sourceEventSeqs).toEqual([0, 1]) - expect((event as SurfaceEvent).surfaceOp).toEqual({ op: 'replace', start: 0, end: 1 }) - }) - - it('scanRows with surface columns reconstructs events with surface fields', () => { - const rows: EventRow[] = [ - { seq: 0, type: 'user/message', time: 1, - data: JSON.stringify({ content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }), - source_event_seqs: null, surface_op: '{"op":"replace","start":0,"end":0}', ignorable: null }, - { seq: 1, type: 'turn/end', time: 2, - data: JSON.stringify({ turn: 1, reason: { kind: 'completed' } }), - source_event_seqs: null, surface_op: null, ignorable: 1 }, - ] - const { preserved } = scanRows(rows) - expect(preserved).toHaveLength(2) - expect((preserved[0]! as SurfaceEvent).surfaceOp).toEqual({ op: 'replace', start: 0, end: 0 }) - expect((preserved[0]! as SurfaceEvent).sourceEventSeqs).toBeUndefined() - expect((preserved[1] as SessionEvent).surfaceOp).toBeUndefined() - }) - - it('append and load round-trips surface fields through SQLite', async () => { + it('keeps a fresh database unopened until the first persistence operation', async () => { + const path = await freshDbPath('dsh-sqlite-lazy-') const ctx = new Context() await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SqliteSessionPersistence, { path: ':memory:' }) - const session = ctx.sessions.create(SessionId('roundtrip-surface')) - session.append('turn/start', { turn: 1 }) - session.append('step/start', { turn: 1, step: 1 }) - session.append('user/message', createUserMessage({ - content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' }, - }), { surfaceOp: 'append' }) - session.append('assistant/message', { - turn: 1, step: 1, - message: createMessage({ - role: 'assistant', - content: [], - source: { - kind: 'model', - ...{ provider: 'mock', model: 'mock' }, - }, - }), - }, { surfaceOp: 'append', sourceEventSeqs: [2] }) - session.append('step/end', { turn: 1, step: 1 }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.sessions.flush(session) - const loaded = await ctx.sessionPersistence.load(SessionId('roundtrip-surface')) - expect(loaded.events).toHaveLength(6) - const um = loaded.events[2]! - expect((um as SurfaceEvent).surfaceOp).toBe('append') - expect((um as SurfaceEvent).sourceEventSeqs).toBeUndefined() - const am = loaded.events[3]! - expect((am as SurfaceEvent).surfaceOp).toBe('append') - expect((am as SurfaceEvent).sourceEventSeqs).toEqual([2]) - await fiber.dispose() + await ctx.plugin(SessionPersistenceSqlite, { path }) + await expect(stat(path)).rejects.toMatchObject({ code: 'ENOENT' }) + const emitWarning = Reflect.get(process, 'emitWarning') + expect(await ctx.sessionPersistence.list()).toEqual([]) + expect(Reflect.get(process, 'emitWarning')).toBe(emitWarning) + expect(typeof (await stat(path)).size).toBe('number') + await ctx.fiber.dispose() }) - it('persists events with surfaceOp but no sourceEventSeqs (covers null branch in surfaceBindings)', async () => { + it('disposes after path validation without opening the database', async () => { + const path = await freshDbPath('dsh-sqlite-unused-') const ctx = new Context() await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SqliteSessionPersistence, { path: ':memory:' }) - const session = ctx.sessions.create(SessionId('surface-noseq')) - session.append('turn/start', { turn: 1 }) - session.append('user/message', createUserMessage({ - content: [], - source: { kind: 'user' }, - }), { surfaceOp: 'append' }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.sessions.flush(session) - const loaded = await ctx.sessionPersistence.load(SessionId('surface-noseq')) - expect((loaded.events[1]! as SurfaceEvent).surfaceOp).toBe('append') - expect((loaded.events[1]! as SurfaceEvent).sourceEventSeqs).toBeUndefined() - await fiber.dispose() + await ctx.plugin(SessionPersistenceSqlite, { path }) + await ctx.fiber.dispose() + await expect(stat(path)).rejects.toMatchObject({ code: 'ENOENT' }) + + const untouchedPath = await freshDbPath('dsh-sqlite-never-validated-') + const untouched = new SqliteStore({ + path: untouchedPath, + journalMode: 'wal', + busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS, + }) + await untouched.close() + await expect(stat(untouchedPath)).rejects.toMatchObject({ code: 'ENOENT' }) + }) + + it('uses constructor defaults and exposes locate and prepare directly', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + let persistence!: SessionPersistenceSqlite + await ctx.plugin(Object.assign((inner: Context) => { + persistence = new SessionPersistenceSqlite(inner, { path: ':memory:' }) + }, { inject: ['sessions'] })) + + const header = meta('direct-provider') + const events = chunkLog(3) + expect(persistence.locate(header)).toBeUndefined() + await persistence.create(header) + await persistence.append(header.id, events) + const preparation = await persistence.prepare(header.id) + expect(preparation.session.header).toEqual(header) + preparation[Symbol.dispose]() + await ctx.fiber.dispose() + }) + + it('keeps empty mutations inert and rolls back a repair without metadata', async () => { + const store = new SqliteStore({ + path: ':memory:', + journalMode: 'wal', + busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS, + }) + const header = meta('empty-store') + await store.appendBatch(header, [], false) + await store.commitRepair(header, undefined, []) + expect(await store.readStoredRevision(header.id)).toBeUndefined() + await expect(store.commitRepair(header, 0, [])).rejects.toThrow(/metadata row is missing/) + await store.close() + }) + + it('rejects omitted torn markers and stale closer positions', async () => { + const path = await freshDbPath('dsh-sqlite-repair-validation-') + const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS }) + const header = meta('repair-validation') + await store.appendBatch(header, [chunk(0)], false) + const db = new DatabaseSync(path) + db.prepare(testSql('insert-corrupt-event')).run(header.id, 1, 'assistant/chunk', 2, '{not json', null) + db.close() + await expect(store.commitRepair(header, undefined, [chunk(1)])).rejects.toThrow(/omitted current torn tail/) + await store.commitRepair(header, 1, []) + await expect(store.commitRepair(header, undefined, [chunk(2)])).rejects.toThrow(/closer starts at seq 2/) + + const cleared = new DatabaseSync(path) + cleared.prepare(testSql('delete-session-events')).run(header.id) + cleared.close() + await store.commitRepair(header, undefined, [chunk(0)]) + expect((await store.loadStored(header.id))?.events).toEqual([chunk(0)]) + await store.close() + }) + + it('rejects malformed physical tail rows before appending', async () => { + const path = await freshDbPath('dsh-sqlite-tail-') + const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS }) + const header = meta('invalid-tail') + await store.appendBatch(header, [chunk(0)], false) + const db = new DatabaseSync(path) + db.prepare(testSql('insert-corrupt-event')) + .run(header.id, 1, 'assistant/chunk', 2, '{not json', null) + db.close() + + await expect(store.appendBatch(header, [chunk(2)], true)).rejects.toThrow(/invalid physical tail/) + await store.close() + }) + + it('rejects missing and empty store identities', async () => { + for (const mode of ['missing', 'empty'] as const) { + const path = await freshDbPath(`dsh-sqlite-identity-${mode}-`) + const db = await openDatabase(DatabaseSync, path, 'wal', DEFAULT_BUSY_TIMEOUT_MS) + if (mode === 'missing') db.exec(testSql('delete-persistence-state')) + else db.exec(testSql('empty-store-id')) + db.close() + await chmod(path, 0o600) + + expect(errorMessage(await backendFailure(path))).toMatch(/no valid store identity/) + } + }) + + it('rejects invalid paths during service initialization', async () => { + const path = await freshDbPath('dsh-sqlite-invalid-path-') + const ctx = new Context() + await ctx.plugin(SessionStore) + await expect(ctx.plugin(SessionPersistenceSqlite, { path: `${path}\0` })).rejects.toMatchObject({ + code: 'ERR_INVALID_ARG_VALUE', + }) + await ctx.fiber.dispose() + }) + + it('rejects non-files and symbolic links', async () => { + const directoryPath = await freshDbPath('dsh-sqlite-directory-') + await mkdir(directoryPath) + expect(errorMessage(await backendFailure(directoryPath))) + .toMatch(/must be a regular file/) + + const linkPath = await freshDbPath('dsh-sqlite-link-') + const target = join(linkPath, '..', 'target.db') + await writeFile(target, '') + await symlink(target, linkPath) + expect(errorMessage(await backendFailure(linkPath))) + .toMatch(/not a symbolic link/) + + const parentLinkPath = await freshDbPath('dsh-sqlite-parent-link-') + const realParent = join(parentLinkPath, '..', 'real-parent') + const linkedParent = join(parentLinkPath, '..', 'linked-parent') + await mkdir(realParent, { mode: 0o700 }) + await symlink(realParent, linkedParent) + expect(errorMessage(await backendFailure(join(linkedParent, 'sessions.db')))) + .toMatch(/must be a real directory/) + }) + + it.runIf( + process.getuid !== undefined && process.getuid() !== 0, + )('rejects permissive files and writable parents', async () => { + const permissivePath = await freshDbPath('dsh-sqlite-permissive-') + await writeFile(permissivePath, '') + await chmod(permissivePath, 0o644) + expect(errorMessage(await backendFailure(permissivePath))) + .toMatch(/accessible only by that user/) + + const writableParentPath = await freshDbPath('dsh-sqlite-parent-') + await chmod(join(writableParentPath, '..'), 0o770) + expect(errorMessage(await backendFailure(writableParentPath))) + .toMatch(/not group\/world-writable/) + }) + + it('surfaces database creation failures after path validation', async () => { + const path = await freshDbPath('dsh-sqlite-create-failure-') + const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS }) + await store.validatePath() + const parent = join(path, '..') + await rm(parent, { recursive: true }) + await writeFile(parent, 'not a directory') + await expect(store.open()).rejects.toThrow(/ENOENT|ENOTDIR/) + await store.close() }) }) diff --git a/packages/session/session-persistence-sqlite/tests/test-sql.ts b/packages/session/session-persistence-sqlite/tests/test-sql.ts new file mode 100644 index 0000000000..77b53a404e --- /dev/null +++ b/packages/session/session-persistence-sqlite/tests/test-sql.ts @@ -0,0 +1,32 @@ +/** Test-only loader for fixed SQLite fixtures. */ + +import { readFileSync } from 'node:fs' + +export type TestSqlName = + | 'add-unexpected-column' + | 'count-events' + | 'count-ignorable-events' + | 'count-packed-events' + | 'count-physical-types' + | 'create-loose-schema' + | 'create-unrelated-table' + | 'delete-persistence-state' + | 'delete-session-events' + | 'empty-store-id' + | 'insert-corrupt-event' + | 'measure-write-traffic' + | 'replace-events-with-nonstrict-table' + | 'select-last-event' + | 'select-event-rowids' + | 'select-event-rows' + | 'select-user-version' + | 'set-application-id-12345' + | 'set-user-version-15' + | 'set-user-version-16' + | 'set-user-version-17' + | 'update-invalid-session-metadata' + +/** Load one fixed test SQL resource. */ +export function testSql(name: TestSqlName): string { + return readFileSync(new URL(`./resources/sql/${name}.sql`, import.meta.url), 'utf8') +} diff --git a/packages/session/session-persistence-sqlite/tsconfig.json b/packages/session/session-persistence-sqlite/tsconfig.json index 2bb79c5919..8865e04321 100644 --- a/packages/session/session-persistence-sqlite/tsconfig.json +++ b/packages/session/session-persistence-sqlite/tsconfig.json @@ -20,6 +20,9 @@ { "path": "../../core/session" }, + { + "path": "../../llm/llm" + }, { "path": "../session-persistence" }, diff --git a/packages/session/session-persistence/README.i18n.yaml b/packages/session/session-persistence/README.i18n.yaml index 74b808b05e..9a9b20071e 100644 --- a/packages/session/session-persistence/README.i18n.yaml +++ b/packages/session/session-persistence/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-persistence/README.md -README.md: 335f40e7439fc4a49fa49bb4541f101250468d1b -README.zh.md: 5a50f051a2c53a5f27b009df11ad46f5952a32ac +README.md: 76df109936070e0dd7afb18e98c6c94855be4f21 +README.zh.md: 6c366bf8287e4c96052935e46410fd27d28714f5 diff --git a/packages/session/session-persistence/README.md b/packages/session/session-persistence/README.md index 335f40e743..76df109936 100644 --- a/packages/session/session-persistence/README.md +++ b/packages/session/session-persistence/README.md @@ -14,12 +14,13 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l | `supportsRawArtifacts: boolean` | State explicitly whether this backend exposes one verbatim artifact per session. Consumers check this capability before calling `readRaw`; `false` is not session absence. | | `readRaw(id, signal?): Promise` | Read a supported backend's own artifact text verbatim, decoded from its physical encoding but never reconstructed from events. `undefined` means only that the requested artifact is absent; an unsupported backend rejects. | | `create(meta): Promise` | Register a new session's metadata. MAY defer the physical write until the first `append` (lazy materialization). | +| `ensureMaterialized(session): Promise` | Explicitly make an exact live session durable even with zero events, without inventing an event. Lifecycle frontends use this only when the empty session itself is a resumable resource; ordinary creation remains lazy. | | `append(id, events): Promise` | Durably persist a batch. Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. | | `prepare(id, signal?): Promise` | Reserve the exact unpublished Session used by resume. A coordinator reuses an earlier inspection when available, commits pending recovery, and releases an unpublished reservation back to its bounded cache on disposal. | | `load(id): Promise<{ meta; events }>` | Return an immutable balanced logical log after converting supported older records from the same format version and committing cold recovery. A live load first flushes its snapshot and rejects while its turn is open; a cold load preserves an interrupted final turn and durably closes it with synthetic `tool/result`/`step/end?`/`turn/end {interrupted}` events. Only a torn tail fragment is dropped; committed corruption and malformed records reject as `SessionPersistenceCorruptionError`, while an unsupported format `version` or an event type unknown to this build (without the envelope's `ignorable` marker) refuses as `SessionFormatUnsupportedError`, naming the refusal direction and the raw log path when the backend keeps one artifact per session. | | `inspect(id, signal?): Promise<{ meta; events }>` | Return an upgraded, validated, deeply frozen logical view without committing recovery or publishing a Session. A cold view receives in-memory synthetic recovery closers while its physical torn tail remains untouched; an already-live view is its current immutable snapshot and may contain an open turn. Coordinator-backed implementations retain the exact cold unpublished Session in a bounded LRU for later `prepare`, but discard and reload it when the stored revision changes. Same-id inspections share an in-flight read. | | `readFrom(id, fromSeq, signal?): Promise<{ meta; events }>` | Return valid stored events with `seq >= fromSeq` without preparation caching, truncation, closers, or coordinator state. A `fromSeq` at or past the stored end returns an empty event list; a negative or non-safe-integer `fromSeq` rejects. Seek-capable backends (SQLite) read only the suffix unless converting a supported older record requires earlier records; sequential backends (JSONL) parse the whole artifact and skip forward. Unknown-type refusal follows that access pattern: a seek read checks only the returned suffix, while the sequential fallback also refuses on an unknown required event below the window. Intended for checkpoint consumers that apply only events after a stored sequence number. | -| `list(signal?): Promise` | Lightweight listing from metadata, no full-log parse. The optional signal cancels backend listing work. A zero-event lazily-materialized session is absent from `list`. | +| `list(signal?): Promise` | Lightweight listing from metadata, no full-log parse. The optional signal cancels backend listing work. A zero-event session is absent until a consumer explicitly materializes it. | | `listSnapshots(signal?): Promise` | Lightweight metadata plus an opaque branded per-log revision, without loading event logs. A revision stays equal while that log and its backing store are unchanged, changes after append or mutating load repair, and cannot collide solely because two stores use the same local counter. The optional signal requests cancellation of backend discovery work; first-party backends settle any started listing work before rejecting so an awaited call is quiescent. | ## Invariants every backend must honor @@ -52,6 +53,7 @@ The `PersistenceBackend` hooks (the only contract between the coordi | `readStoredRevision(id, signal?)` | Read the current source-qualified revision for one id without loading its event log. It uses the same revision representation as `loadStored` and returns `undefined` when the id is absent. | | `loadStoredFrom?(id, fromSeq, signal?)` | Optional seek-capable suffix read behind the service's `readFrom`: the header plus stored events with `seq >= fromSeq`, non-mutating, no torn marker. SQLite implements it (`WHERE seq >= ?`); a backend that omits it gets the coordinator's fallback — `loadStored` plus a forward skip. | | `appendBatch(meta, events, isMaterialized)` | Durably append a contiguous batch, lazily materializing ATOMICALLY when not yet materialized. | +| `materializeHeader?(meta)` | Durably create a header-only artifact for `ensureMaterialized`; required by providers that support durable empty sessions. | | `commitRepair(meta, tornMarker, closers)` | Make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined` — a marker may be falsy, e.g. seq/offset `0`) and append `closers`. NOT required to be atomic. Used by load (truncate + closers) and live-adoption (truncate only). | | `list(signal?)` | List all stored metadata, observing optional cancellation. | | `close?()` | Optional lifecycle teardown (e.g. close a db handle), awaited after the dispose drain. | diff --git a/packages/session/session-persistence/README.zh.md b/packages/session/session-persistence/README.zh.md index 5a50f051a2..6c366bf828 100644 --- a/packages/session/session-persistence/README.zh.md +++ b/packages/session/session-persistence/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -会话持久化是一项能力 seam。抽象的 `SessionPersistence` 服务(`ctx.sessionPersistence`)是其 Service Definition。它要求持久化后端持久存储、重新加载和列出会话,但不规定具体存储实现。该 seam 采用与 `dsh-shell` 相同的角色划分(见[能力 seam](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):本包负责 Service Definition,同级包负责 Service Provider,Consumer 注入该服务。 +会话持久化是一项能力 seam。抽象的 `SessionPersistence` 服务(`ctx.sessionPersistence`)是其 Service Definition。它要求持久化后端持久存储、重新加载和列出会话,但不规定具体存储实现。该 seam 采用与 `dsh-shell` 相同的角色划分(见[能力 seam](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.zh.md)):本包负责 Service Definition,同级包负责 Service Provider,Consumer 注入该服务。 持久化单元就是现有 `SessionEvent`(事件溯源模型:日志是唯一真源),因此不存在另一套并行的「持久消息」类型。不属于可回放对话状态的元数据(格式版本、cwd、血缘、种子边界、origin、委托深度)作为 `SessionHeader` 单独传输,该类型归 `dsh-session` 所有,并在此重新导出。 @@ -14,12 +14,13 @@ | `supportsRawArtifacts: boolean` | 明确说明该后端是否为每个会话暴露一份逐字工件。Consumer 在调用 `readRaw` 前检查此能力;`false` 并不表示会话缺失。 | | `readRaw(id, signal?): Promise` | 读取受支持后端自身的逐字工件文本;只解码物理编码,绝不从事件重建。`undefined` 仅表示所请求工件缺失;不支持的后端会拒绝。 | | `create(meta): Promise` | 注册新会话元数据。可以将物理写入延迟到第一次 `append`(延迟实体化)。 | +| `ensureMaterialized(session): Promise` | 在不虚构事件的情况下,显式使一个确切 live session 即使零事件也保持持久。只有当空会话本身是可恢复资源时,生命周期前端才使用它;普通创建仍保持延迟实体化。 | | `append(id, events): Promise` | 持久保存一个批次。仅追加;任何修复后,第一个事件 `seq` == 已存储 next-seq;非 JSON 可序列化数据会被拒绝,并命名违规类型。 | | `prepare(id, signal?): Promise` | 预留恢复所使用的那个未发布 Session。协调器会尽可能复用之前的检查结果、提交待处理恢复,并在 dispose(资源释放)时将未发布 reservation 释放回有界缓存。 | | `load(id): Promise<{ meta; events }>` | 转换同一格式版本中受支持的旧记录后,返回不可变、平衡的逻辑日志,并提交冷恢复。实时 load 先 flush 其快照,并在轮次开放时拒绝;冷 load 保留中断的最终轮次,并用合成 `tool/result`/`step/end?`/`turn/end {interrupted}` 事件持久关闭它。只丢弃撕裂尾部碎片;已提交损坏和格式错误的记录以 `SessionPersistenceCorruptionError` 拒绝,不支持的格式 `version` 或本构建不认识且信封未带 `ignorable` 标记的事件类型以 `SessionFormatUnsupportedError` 拒绝,消息说明拒绝方向,并在后端为每个会话保留独立文件时给出原始日志路径。 | | `inspect(id, signal?): Promise<{ meta; events }>` | 返回已经升级、验证和深度冻结的逻辑视图,但不提交恢复或发布 Session。冷视图会获得仅存在于内存的合成恢复 closer,物理撕裂尾部保持不变;实时状态下的视图则是当前不可变快照,可能包含开放的轮次。基于协调器的实现会在有界 LRU 中保留该冷状态下未发布的 Session 本身,供后续 `prepare` 使用,但已存储修订值变化后会丢弃并重新读取。同 id 检查共享进行中的读取。 | | `readFrom(id, fromSeq, signal?): Promise<{ meta; events }>` | 返回 `seq >= fromSeq` 的有效已存储事件,不进入 preparation 缓存、不截断、不合成 closer,也不发布协调器状态。`fromSeq` 达到或超过已存储末尾时返回空事件列表;负数或非安全整数 `fromSeq` 会被拒绝。可寻址后端(SQLite)只读后缀,除非转换受支持的旧记录需要读取更早的记录;顺序后端(JSONL)解析整个产物并向前跳过。未知类型拒绝遵循同一读取方式:寻址读取只检查返回的后缀,顺序回退路径还会拒绝窗口以下的未知必需事件。供 checkpoint 消费方只应用已存序号之后的事件。 | -| `list(signal?): Promise` | 从元数据轻量列出,不解析完整日志。可选信号取消后端列表工作。零事件延迟实体化会话不在 `list` 中。 | +| `list(signal?): Promise` | 从元数据轻量列出,不解析完整日志。可选信号取消后端列表工作。零事件会话在 consumer 显式实体化前不在 `list` 中。 | | `listSnapshots(signal?): Promise` | 返回轻量元数据和每份日志一个不透明、带品牌类型的修订值,不加载事件日志。日志及其后端存储不变时,修订保持相等;append 或变更性 load 修复后会改变;不会仅因两个存储使用相同本地计数器而冲突。可选信号请求取消后端发现工作;第一方后端会先等待所有已启动的列出工作结束,再予以拒绝,因此调用返回拒绝时,相关工作已完全停稳。 | ## 每个后端必须遵守的不变量 @@ -31,13 +32,13 @@ ## 写入协调器 -`PersistenceCoordinator` 负责每 id 状态和串行化、每个活动会话各自的有界写入 controller、延迟实体化、崩溃尾部修复、会话接管和完全停稳的 dispose。第一方后端组合一个协调器,实现小型 `PersistenceBackend` 存储钩子接口,并委托其有状态方法。因此 JSONL 和 SQLite 共享生命周期正确性,同时保留不同存储原语;见[协调器 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md)、[flush controller 简化](../../../.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.md)和[有界批处理决策](../../../.agents/notes/implemented/architecture/2026-08-08-bounded-session-persistence-write-batching.md)。 +`PersistenceCoordinator` 负责每 id 状态和串行化、每个活动会话各自的有界写入 controller、延迟实体化、崩溃尾部修复、会话接管和完全停稳的 dispose。第一方后端组合一个协调器,实现小型 `PersistenceBackend` 存储钩子接口,并委托其有状态方法。因此 JSONL 和 SQLite 共享生命周期正确性,同时保留不同存储原语;见[协调器 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md)、[flush controller 简化](../../../.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.zh.md)和[有界批处理决策](../../../.agents/notes/implemented/architecture/2026-08-08-bounded-session-persistence-write-batching.zh.md)。 每个 `session/event` 将事件复制到会话 controller。第一个待处理事件会开启固定批处理窗口;后续事件会加入该批次,但不会重置截止时间。配置的 `writeBatchMaxDelayMs` 只限制这段有意等待,而不限制事件循环、初始化、串行化操作或后端延迟。写入期间接纳的事件会形成一个新的有界批次。`session/flush` 会取消等待,并作为共享的完全停稳屏障,排空屏障运行期间接纳的事件。后台写入失败只记录一次日志,保留顺序不变的批次,并暂停自动重试;新事件会开启新的固定窗口,而显式 flush 或后端拆卸会立即重试,并在失败再次发生时向调用方暴露失败。 崩溃修复只适用于冷状态。对于已有活动会话的 id,`load(id)` 为权威内存日志制作快照,等待该快照持久,并只在平衡时返回;活动会话中开放的轮次会被拒绝,而不会收到合成中断 closer。对于冷 id,检查只读取、验证、冻结并构造一次未发布 Session;只有来源修订值仍然是当前值时,重复检查才会复用该对象图。`prepare(id)` 在修复前执行相同校验,预留该 Session 本身,提交任何待处理的撕裂尾部或中断轮次修复,并将其返回用于发布。HMR(热模块替换)接管通过 `loadStored` 读取,应用协调器 cwd 检查,并绝不关闭活动轮次。 -后端读取会在验证当前记录前,转换同一格式版本中明确受支持的旧记录。消息标识机制引入前的消息会获得确定性的 id `legacy-message::`;工具结果的内容替换会继承其目标导入后的 id。react-loop 引入前的 `turn/start` 会移除过时的 trigger,已移除的 steering(中途引导)事件 `steering/message` 会转换为同一条带标识的 `user/message`;旧版 `turn/end` 会映射终止原因,但不会虚构旧记录中没有记载的调用方。协调器对 `load`、`inspect`、`readFrom`、无所有者状态的认领和 HMR 前缀接管使用同一份转换后视图。存储仍然仅追加:读取不会重写旧记录,此后追加的事件使用当前格式。这些是[消息标识机制引入前的消息](../../../.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.md)与 [react-loop 引入前会话](../../../.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.md)决策所规定的范围受限的导入例外,并不构成通用的 v0 迁移承诺。 +后端读取会在验证当前记录前,转换同一格式版本中明确受支持的旧记录。消息标识机制引入前的消息会获得确定性的 id `legacy-message::`;工具结果的内容替换会继承其目标导入后的 id。react-loop 引入前的 `turn/start` 会移除过时的 trigger,已移除的 steering(中途引导)事件 `steering/message` 会转换为同一条带标识的 `user/message`;旧版 `turn/end` 会映射终止原因,但不会虚构旧记录中没有记载的调用方。协调器对 `load`、`inspect`、`readFrom`、无所有者状态的认领和 HMR 前缀接管使用同一份转换后视图。存储仍然仅追加:读取不会重写旧记录,此后追加的事件使用当前格式。这些是[消息标识机制引入前的消息](../../../.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md)与 [react-loop 引入前会话](../../../.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.zh.md)决策所规定的范围受限的导入例外,并不构成通用的 v0 迁移承诺。 活动会话发出 `session/disposed` 时,协调器等待其 controller,以串行方式执行最终 drain,然后释放该精确 `Session` 对象拥有的状态。失败退役会将 controller 保留在活动会话 map 中,使后端拆卸可重试。后端拆卸先停止事件接纳,flush 每个剩余 controller,等待每 id 操作,最后才关闭存储句柄。 @@ -52,11 +53,12 @@ | `readStoredRevision(id, signal?)` | 在不加载事件日志的情况下读取一个 id 当前的来源限定修订值。它使用与 `loadStored` 相同的修订值表示;id 不存在时返回 `undefined`。 | | `loadStoredFrom?(id, fromSeq, signal?)` | 服务 `readFrom` 背后的可选可寻址后缀读取:返回 header 和 `seq >= fromSeq` 的已存储事件,非修改式、无撕裂标记。SQLite 实现它(`WHERE seq >= ?`);不实现的后端使用协调器回退——`loadStored` 加向前跳过。 | | `appendBatch(meta, events, isMaterialized)` | 持久追加连续批次;尚未实体化时以原子方式延迟实体化。 | +| `materializeHeader?(meta)` | 为 `ensureMaterialized` 持久创建仅含 header 的 artifact;支持持久空会话的 provider 必须实现。 | | `commitRepair(meta, tornMarker, closers)` | 使崩溃修复持久:截断撕裂尾部(当且仅当 `tornMarker !== undefined`;标记可为 falsy,例如 seq/offset `0`),并追加 `closers`。不要求原子性。由 load(截断 + closer)和活动会话接管(仅截断)使用。 | | `list(signal?)` | 列出全部已存储元数据,并遵循可选的取消信号。 | | `close?()` | 可选生命周期拆卸(例如关闭 db 句柄),在 dispose drain 后等待其完成。 | -协调器断言已存储 id,并在修复或活动会话接管前比较已存储/活动会话 cwd。其 `inspect()` 路径取得新鲜后端值的所有权,只验证和冻结一次,并在不调用 `commitRepair` 的情况下最多保留配置数量的未发布 Session。只有保留源的修订值仍等于 `readStoredRevision` 时,系统才会复用或修复它;否则协调器会重新读取。该新鲜性校验不会增加跨进程写入排他。持久日志在一次读取与复核往返内保持不变时,修订值重试才能收敛;持续的外部写入可能延迟 `load`、`inspect` 或 `prepare`。`tornMarker` 完全不透明:协调器只测试 `!== undefined`,并将其原样往返给 `commitRepair`,绝不检查值(JSONL 后端使用待截断字节偏移,SQLite 后端使用待删除 seq)。第三方后端可以不用协调器直接实现抽象服务,但必须提供相同的非修改式检查和可信轻量快照修订。详见[写入协调器 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md)。 +协调器断言已存储 id,并在修复或活动会话接管前比较已存储/活动会话 cwd。其 `inspect()` 路径取得新鲜后端值的所有权,只验证和冻结一次,并在不调用 `commitRepair` 的情况下最多保留配置数量的未发布 Session。只有保留源的修订值仍等于 `readStoredRevision` 时,系统才会复用或修复它;否则协调器会重新读取。该新鲜性校验不会增加跨进程写入排他。持久日志在一次读取与复核往返内保持不变时,修订值重试才能收敛;持续的外部写入可能延迟 `load`、`inspect` 或 `prepare`。`tornMarker` 完全不透明:协调器只测试 `!== undefined`,并将其原样往返给 `commitRepair`,绝不检查值(JSONL 后端使用待截断字节偏移,SQLite 后端使用待删除 seq)。第三方后端可以不用协调器直接实现抽象服务,但必须提供相同的非修改式检查和可信轻量快照修订。详见[写入协调器 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md)。 ## 元数据与位置类型 diff --git a/packages/session/session-persistence/package.json b/packages/session/session-persistence/package.json index f42b496e6d..195bb39ef5 100644 --- a/packages/session/session-persistence/package.json +++ b/packages/session/session-persistence/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-persistence", "description": "Abstract durable session persistence seam (ctx.sessionPersistence) for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-persistence/src/coordinator.ts b/packages/session/session-persistence/src/coordinator.ts index eb5f9714c4..3b99afb723 100644 --- a/packages/session/session-persistence/src/coordinator.ts +++ b/packages/session/session-persistence/src/coordinator.ts @@ -68,7 +68,7 @@ export class SessionFormatUnsupportedError extends Error { * Direction-aware refusal text for a stored session whose format version this * build does not read. Shared by the coordinator's load-time check and by * backends that must refuse BEFORE decoding version-dependent structure (a - * future format may not satisfy today's structural checks at all, and the + * future format may not satisfy this build's structural checks at all, and the * user must see "upgrade the harness", never "corrupt"). * @param id - the stored session id, for message context. * @param version - the stored format version. @@ -175,6 +175,9 @@ export interface PersistenceBackend { */ loadStoredFrom?(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise + /** Durably create an empty header-only session artifact. */ + materializeHeader?(meta: SessionHeader): Promise + /** * Durably append a CONTIGUOUS batch, lazily materializing the session first * when `!isMaterialized`. The materialize-write and the first event batch MUST @@ -642,6 +645,26 @@ export class PersistenceCoordinator { return this.serialize(snapshot.id, () => this.createCore(snapshot)) } + /** + * Materialize one exact live session without inventing a session event. + * @param session - live session already registered through the write path. + */ + async ensureMaterialized(session: Session): Promise { + await this.flush(session) + await this.serialize(session.id, async () => { + const state = this.states.get(session.id) + /* v8 ignore next -- successful live flush always initializes the exact session state. */ + if (state === undefined) throw new Error(`session "${session.id}" is not registered for persistence`) + if (state.materialized) return + if (this.backend.materializeHeader === undefined) { + throw new Error('session persistence backend cannot materialize an empty session') + } + await this.backend.materializeHeader(state.meta) + state.materialized = true + this.preparations.invalidate(session.id) + }) + } + private async createCore(meta: SessionHeader): Promise { // Do NOT clobber an existing session: the SessionId IS the identity. if (this.states.has(meta.id) || this.preparations.has(meta.id)) { @@ -977,7 +1000,7 @@ export class PersistenceCoordinator { const state = this.states.get(session.id) /* v8 ignore next -- successful flush always publishes this live session's durable state */ if (state === undefined) throw new Error(`session "${session.id}" lost persistence state during load`) - if (events.length === 0) throw new Error(`session "${session.id}" not found`) + if (events.length === 0 && !state.materialized) throw new Error(`session "${session.id}" not found`) if (interruptedTurnClosers(events).length > 0) { throw new Error(`cannot load session "${session.id}" while its live turn is open; use the live Session or wait for the turn to close`) } @@ -1131,8 +1154,7 @@ export class PersistenceCoordinator { // Session disposal is observe-only, so retirement contains its own failure. ctx.on('session/disposed', (session) => { this.retire(session) }) - // HMR: a hot reload does not replay session/created, so seed existing live - // sessions (mirrors dsh-invariants). + // HMR does not replay session/created, so seed existing live sessions. for (const session of ctx.sessions.list()) void this.initFor(session) } @@ -1170,7 +1192,8 @@ export class PersistenceCoordinator { this.live.set(session, restored) return restored } - const seed = session.events.map(e => structuredClone(e)) + // Session owns this stable deep-frozen snapshot; backends only serialize it. + const seed = session.events const live: LiveSessionState = { init: Promise.resolve(), writes: this.createWriteBehind(session, () => live.init), diff --git a/packages/session/session-persistence/src/index.ts b/packages/session/session-persistence/src/index.ts index d579dc46d9..e9c75107bb 100644 --- a/packages/session/session-persistence/src/index.ts +++ b/packages/session/session-persistence/src/index.ts @@ -7,7 +7,7 @@ import { Context, Service } from '@deepseek-ai/cordis' import { SessionPreparation } from '@deepseek-ai/dsh-session' -import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' import type { SessionPersistenceRevision } from './revision.ts' // Re-export the metadata vocabulary so Consumers import it from the Service Definition. @@ -132,6 +132,16 @@ export abstract class SessionPersistence extends Service { */ abstract create(meta: SessionHeader): Promise + /** + * Ensure a live session has a durable header even when it has no events. + * Ordinary sessions remain lazily materialized; lifecycle frontends call + * this only when an empty session itself is a durable resumable resource. + * @param _session - exact live session whose registered header is materialized. + */ + ensureMaterialized(_session: Session): Promise { + return Promise.reject(new Error('this session persistence backend cannot materialize an empty session')) + } + /** * Durably persist a batch of events. Honors the append-only and contiguous- * seq contracts: the first event's `seq` MUST equal the stored next-seq diff --git a/packages/session/session-persistence/tests/persistence.spec.ts b/packages/session/session-persistence/tests/persistence.spec.ts index 50ad798e04..d82627ffe9 100644 --- a/packages/session/session-persistence/tests/persistence.spec.ts +++ b/packages/session/session-persistence/tests/persistence.spec.ts @@ -97,6 +97,10 @@ class MemoryPersistence extends SessionPersistence implements PersistenceBackend return this.coordinator.create(m) } + override ensureMaterialized(session: Session): Promise { + return this.coordinator.ensureMaterialized(session) + } + append(id: SessionId, events: readonly SessionEvent[]): Promise { return this.coordinator.append(id, events) } @@ -151,6 +155,11 @@ class MemoryPersistence extends SessionPersistence implements PersistenceBackend } } + materializeHeader(m: SessionHeader): Promise { + this.store.set(m.id, { meta: structuredClone(m), events: [] }) + return Promise.resolve() + } + async commitRepair(m: SessionHeader, _tornMarker: undefined, closers: readonly SessionEvent[]): Promise { // No torn tails in a Map store, so `_tornMarker` is always undefined; only the // synthetic closers are appended (the same DELETE+INSERT a DB backend does, @@ -180,6 +189,7 @@ class ControlledBackend implements PersistenceBackend { readonly name = 'session-persistence-controlled' readonly store: MemoryStore = new Map() readonly lifecycle: string[] = [] + lastAppendedBatch: readonly SessionEvent[] | undefined appendAttempts = 0 loadAttempts = 0 repairAttempts = 0 @@ -212,6 +222,7 @@ class ControlledBackend implements PersistenceBackend { } async appendBatch(m: SessionHeader, events: readonly SessionEvent[], _isMaterialized: boolean): Promise { + this.lastAppendedBatch = events const attempt = ++this.appendAttempts await this.beforeAppend?.(attempt) const entry = this.store.get(m.id) @@ -237,7 +248,6 @@ class ControlledBackend implements PersistenceBackend { } } -// Run the shared contract against the in-memory backend. runPersistenceContract('memory', async () => { const ctx = new Context() await ctx.plugin(SessionStore) @@ -279,6 +289,28 @@ runCoordinatorContract('memory', async (): Promise => { } }) +describe('PersistenceCoordinator seed ownership', () => { + it('retains the immutable session seed without cloning it', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + new PersistenceCoordinator(inner, backend) + }, { inject: ['sessions'] })) + + try { + const session = ctx.sessions.create(SessionId('shared-seed'), { seed: oneTurnLog() }) + const seed = session.events + await ctx.sessions.flush(session) + + expect(backend.lastAppendedBatch).toBe(seed) + } finally { + await fiber.dispose() + await ctx.fiber.dispose() + } + }) +}) + describe('PersistenceCoordinator bounded writes', () => { it('cancels the batching deadline when live initialization rejects', async () => { vi.useFakeTimers() @@ -1757,6 +1789,72 @@ describe('PersistenceCoordinator retirement', () => { }) describe('SessionPersistence service registration', () => { + it('materializes an explicitly durable live session without adding events', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + await ctx.plugin(MemoryPersistence) + const session = ctx.sessions.create(SessionId('durable-empty'), { meta: { cwd: '/workspace' } }) + + await ctx.sessionPersistence.ensureMaterialized(session) + await ctx.sessionPersistence.ensureMaterialized(session) + + await expect(ctx.sessionPersistence.list()).resolves.toEqual([session.header]) + await expect(ctx.sessionPersistence.load(session.id)).resolves.toEqual({ meta: session.header, events: [] }) + await ctx.fiber.dispose() + }) + + it('fails loud when a direct backend does not support empty materialization', async () => { + const session = Session.create(SessionId('unsupported-empty')) + await expect(SessionPersistence.prototype.ensureMaterialized.call({} as SessionPersistence, session)) + .rejects.toThrow(/cannot materialize an empty session/) + }) + + it('fails loud when a coordinator backend omits empty materialization', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + let coordinator!: PersistenceCoordinator + await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, new ControlledBackend()) + }, { inject: ['sessions'] })) + const session = ctx.sessions.create(SessionId('unsupported-coordinator')) + + await expect(coordinator.ensureMaterialized(session)).rejects.toThrow(/cannot materialize an empty session/) + await ctx.fiber.dispose() + }) + + it('accepts current aborted and error turn endings without legacy conversion', async () => { + const store: MemoryStore = new Map() + const ctx = new Context() + await ctx.plugin(SessionStore) + const endings: SessionEvent[] = [ + { + type: 'turn/end', seq: 5, time: 6, + data: { turn: 1, reason: { kind: 'aborted', reason: { kind: 'user' } } }, + }, + { + type: 'turn/end', seq: 5, time: 6, + data: { turn: 1, reason: { kind: 'error', error: { message: 'failed', code: 'UNKNOWN' } } }, + }, + ] + for (const [index, ending] of endings.entries()) { + const m = meta(`current-ending-${index}`) + store.set(m.id, { meta: m, events: [...oneTurnLog().slice(0, -1), ending] }) + } + await ctx.plugin(MemoryPersistence, { store }) + await Promise.all([...store.keys()].map(id => ctx.sessionPersistence.load(SessionId(id)))) + await ctx.fiber.dispose() + }) + + it('rejects preparing an id that already has a live Session', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + await ctx.plugin(MemoryPersistence) + const session = ctx.sessions.create(SessionId('live-prepare-conflict')) + + await expect(ctx.sessionPersistence.prepare(session.id)).rejects.toThrow(/while it is live/) + await ctx.fiber.dispose() + }) + it('provides a cancellation-aware default preparation for simple backends', async () => { const ctx = new Context() await ctx.plugin(SessionStore) diff --git a/packages/session/session-projection-cache/README.i18n.yaml b/packages/session/session-projection-cache/README.i18n.yaml index 618533d282..609f146f24 100644 --- a/packages/session/session-projection-cache/README.i18n.yaml +++ b/packages/session/session-projection-cache/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-projection-cache/README.md -README.md: 5d4ad07fab6648acdb40c6aa86d32cc78b4c016e -README.zh.md: 5e38ee98b04bc8f856538112d2922b64cadce4d5 +README.md: 33908578a5127f2b6bb78ed7467833aaaa2cf085 +README.zh.md: 0ca410f91562360d85faadf4cf64cb61ac467482 diff --git a/packages/session/session-projection-cache/README.md b/packages/session/session-projection-cache/README.md index 5d4ad07fab..33908578a5 100644 --- a/packages/session/session-projection-cache/README.md +++ b/packages/session/session-projection-cache/README.md @@ -2,12 +2,13 @@ English | [中文](README.zh.md) -The persisted projection cache (`ctx.sessionProjectionCache`): durable checkpoints of every registered projection unit's state, one record per session on the domain data form (`session_projcache` domain — the shipped json backend lands it beside `workspace.json` under the configured storage root). Design authority: the [session-projection RFC](../../../.agents/notes/proposed/architecture/2026-07-27-session-projection-and-command-log.md) (persisted projection cache section). +The persisted projection cache (`ctx.sessionProjectionCache`): durable checkpoints of every projection unit's state, one record per session on the domain data form (`session_projcache` domain — the shipped json backend lands it beside `workspace.json` under the configured storage root). Design authority: the [session-projection RFC](../../../.agents/notes/proposed/architecture/2026-07-27-session-projection-and-command-log.md) (persisted projection cache section). A stored row `(key → {ver, seq, val})` is a fold shortcut, never an authority: possibly stale (`seq` says exactly how stale) but never wrong. Consequences the implementation commits to: - **Every background write is fail-soft.** A failed durable write logs a warning and keeps the cache stale; the next write or cold read self-heals. A crash between writes costs a longer tail replay, never a wrong value. - **A `ver` mismatch against the live unit's `stateVersion` discards, never migrates.** A unit bump invalidates its rows at read time; the key refolds from the log. +- **A row must pass the live unit's `stateSchema`.** A malformed row is omitted from the zero-I/O view and rejected by restore so the cold-read ladder refolds it from the log. - **Whole-record writes.** Each write replaces the session's full checkpoint (the registry cut is always complete), snapshotted through the lossless-JSON boundary — a unit state violating the plain-JSON contract fails loud. - **Records are bound to a log lifecycle, not just an id.** Each record stores the header identity (`createdAt`, `cwd`) it was folded from; every read validates it (the live or stored header is the witness) before accepting a row, so a deleted-then-recreated id or a persistence store swapped under a surviving cache discards the unrelated record instead of seeding phantom values. - **The log leads, the cache follows.** A live checkpoint flushes the session's buffered events durably BEFORE the cache row lands, so a crash can leave the cache behind the log (a longer tail replay) but never ahead of it. @@ -27,7 +28,7 @@ Both `Config` fields are required (no defaults): flush cadence is a deployment c ## Listing read (`cachedSnapshot(meta)`) -The zero-I/O rung: whole values viewed straight from the identity-matching stored record (version-matching keys only), returned as a `{asOfSeq, values}` cut — `asOfSeq` is the lowest served-row watermark, so a client seeding its per-session value store under higher-seq-wins can never let a stale list block overwrite a newer push frame. `undefined` when no usable record exists (unknown id, unrelated lifecycle, or no version-matching rows); the api-proxy list carrier turns that into an absent column. +The zero-I/O rung: client values viewed straight from the identity-matching stored record (version- and state-schema-matching keys only), returned as a `{asOfSeq, values}` cut — `asOfSeq` is the lowest served-row watermark, so a client seeding its per-session value store under higher-seq-wins can never let a stale list block overwrite a newer push frame. Host-only rows are never returned. `undefined` when no usable client row exists (unknown id, unrelated lifecycle, or no usable rows); the api-proxy list carrier turns that into an absent column. ## Cold read (`coldSnapshot(id, signal?)`) diff --git a/packages/session/session-projection-cache/README.zh.md b/packages/session/session-projection-cache/README.zh.md index 5e38ee98b0..0ca410f915 100644 --- a/packages/session/session-projection-cache/README.zh.md +++ b/packages/session/session-projection-cache/README.zh.md @@ -2,12 +2,13 @@ [English](README.md) | 中文 -持久投影缓存(`ctx.sessionProjectionCache`):把每个已注册投影单元的状态持久化为检查点,基于域数据形态(domain data form)每会话一条记录(`session_projcache` 域——出厂 JSON 后端将其落在配置的存储根目录下、`workspace.json` 旁边)。设计权威:[session-projection RFC](../../../.agents/notes/proposed/architecture/2026-07-27-session-projection-and-command-log.md)(persisted projection cache 一节)。 +持久投影缓存(`ctx.sessionProjectionCache`):把每个投影单元的状态保存为检查点,基于域数据形态(domain data form)每会话一条记录(`session_projcache` 域——出厂 JSON 后端将其落在配置的存储根目录下、`workspace.json` 旁边)。设计权威:[session-projection RFC](../../../.agents/notes/proposed/architecture/2026-07-27-session-projection-and-command-log.zh.md)(persisted projection cache 一节)。 一条存储行 `(key → {ver, seq, val})` 是折叠捷径,绝不是权威:可能陈旧(`seq` 精确说明陈旧到哪),但绝不会错。实现据此承诺: - **每次后台写入都 fail-soft。** 持久写失败只记一条警告并保持缓存陈旧;下一次写入或冷读自愈。两次写之间崩溃的代价是更长的尾部回放,绝不是错误的值。 - **`ver` 与当前运行单元的 `stateVersion` 不匹配即丢弃,绝不迁移。** 单元递增版本会在读取时使其行失效;该 key 从日志重新折叠。 +- **存储行必须通过当前单元的 `stateSchema`。** 畸形行从零 I/O view 中省略,并被 restore 拒绝,使冷读阶梯从日志重新折叠。 - **整记录写入。** 每次写入替换该会话的完整检查点(注册表切面始终是完整的),并经无损 JSON 边界快照——违反纯 JSON 约定的单元状态会显式失败并报错。 - **记录绑定到日志生命周期,而不只是 id。** 每条记录存储其折叠来源的 header 身份(`createdAt`、`cwd`);每次读取先以活 header 或存储 header 为证验证它,再接受任何行——被删后重建的 id、或缓存幸存而持久化存储被换掉时,无关记录被整体丢弃,绝不播种幻影值。 - **日志领先,缓存跟随。** 活会话检查点先把缓冲事件持久 flush,缓存行才落地,因此崩溃只会让缓存落后于日志(更长的尾部回放),绝不领先于它。 @@ -27,7 +28,7 @@ ## 列表读(`cachedSnapshot(meta)`) -零 I/O 一档:从身份匹配的存储记录直接 view 全量值(仅版本匹配的 key),以 `{asOfSeq, values}` 切面返回——`asOfSeq` 取所服务行的最低水位,客户端在 higher-seq-wins 规则下播种值存储时,陈旧列表块永远压不过更新的推送帧。无可用记录(未知 id、无关生命周期、无版本匹配行)时返回 `undefined`;api-proxy 列表载体将其转为列缺席。 +零 I/O 一档:从身份匹配的存储记录直接 view 客户端值(仅版本与 state schema 均匹配的 key),以 `{asOfSeq, values}` 切面返回——`asOfSeq` 取所服务行的最低水位,客户端在 higher-seq-wins 规则下播种值存储时,陈旧列表块永远压不过更新的推送帧。host-only 行永不返回。无可用客户端行(未知 id、无关生命周期、无可用行)时返回 `undefined`;api-proxy 列表载体将其转为列缺席。 ## 冷读(`coldSnapshot(id, signal?)`) diff --git a/packages/session/session-projection-cache/package.json b/packages/session/session-projection-cache/package.json index 1697967808..826d33ab9d 100644 --- a/packages/session/session-projection-cache/package.json +++ b/packages/session/session-projection-cache/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-projection-cache", "description": "Persisted projection cache (ctx.sessionProjectionCache): durable per-session projection checkpoints over the domain data form, throttled write-behind, and the cold-read ladder (cache row + persistence tail replay)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-projection-cache/src/index.ts b/packages/session/session-projection-cache/src/index.ts index 66ad637402..f7bdb62a3a 100644 --- a/packages/session/session-projection-cache/src/index.ts +++ b/packages/session/session-projection-cache/src/index.ts @@ -1,6 +1,6 @@ /** * Persisted projection cache (`ctx.sessionProjectionCache`): durable - * checkpoints of every registered projection unit's state, one record per + * checkpoints of every client-visible or explicitly persisted projection unit's state, one record per * session on the domain data form (`session_projcache` domain — the shipped * json backend lands it beside `workspace.json`). The cache is a fold * shortcut, never an authority: a row is possibly stale (its `seq` @@ -185,10 +185,9 @@ export class SessionProjectionCache extends Service { if (!related) throw new Error('unrelated log identity') restored = this.ctx.sessionProjections.restore(cached, tail.events, floor) } catch { - // The recoverable restore failures: an unrelated record, or a row - // overreaching the stored log end (or predating the floor). Both imply - // floor > 0 (baseSeq-0 restores never throw and an unrelated record - // still carried a usable watermark), so the full log is a fresh read. + // Recoverable failures are an unrelated record, a row outside the + // supplied suffix or log end, and stateSchema rejection. The full read + // removes every checkpoint seed and lets each unit refold from init. const whole = await persistence.readFrom(id, 0, signal) restored = this.ctx.sessionProjections.restore({}, whole.events, 0) } diff --git a/packages/session/session-projection-cache/tests/cache.spec.ts b/packages/session/session-projection-cache/tests/cache.spec.ts index 6395ed7d0f..89154ec108 100644 --- a/packages/session/session-projection-cache/tests/cache.spec.ts +++ b/packages/session/session-projection-cache/tests/cache.spec.ts @@ -19,6 +19,10 @@ import { MemoryMediaPool, MemoryStorageBackend } from '../../../storage/storage- import SessionProjectionCache from '../src/index.ts' declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + 'cache-test/marks': MarksState + 'cache-test/marks2': Map + } interface SessionProjectionMap { 'cache-test/marks': { marks: string[] } } @@ -35,14 +39,17 @@ declare module '@deepseek-ai/dsh-session/types' { } type MarksState = { marks: string[] } | null -const marksUnit = (stateVersion = 1): ProjectionDefinition<'cache-test/marks', MarksState> => ({ +const marksUnit = (stateVersion = 1) => ({ key: 'cache-test/marks', - schema: z.object({ marks: z.array(z.string()) }), + stateSchema: z.object({ marks: z.array(z.string()) }).nullable(), init: () => null, apply: (state, event) => (event.type === 'cache-test/mark' ? (event).data : state), - view: state => state ?? { marks: [] }, + wire: { + viewSchema: z.object({ marks: z.array(z.string()) }), + view: state => state ?? { marks: [] }, + }, stateVersion, -}) +}) satisfies ProjectionDefinition<'cache-test/marks', MarksState> /** A persistence double serving readFrom over a fixed per-id stored log (headers stamp createdAt 0). */ function fakePersistence(logs: Map) { @@ -175,11 +182,10 @@ describe('SessionProjectionCache write policy', () => { expect(storedRows(pool, clean.id)?.['cache-test/marks']).toEqual({ ver: 1, seq: -1, val: null }) // A unit whose state violates the plain-JSON contract fails the write loud. ctx.sessionProjections.register({ - key: 'cache-test/marks2' as never, - schema: { parse: (value: unknown) => value } as never, + key: 'cache-test/marks2', + stateSchema: z.custom>(() => true), init: () => new Map(), - apply: (state: unknown) => state, - view: () => null as never, + apply: state => state, stateVersion: 1, }) await expect(ctx.sessionProjectionCache.write(clean)).rejects.toThrow('not losslessly JSON-serializable') @@ -286,6 +292,19 @@ describe('SessionProjectionCache cold read', () => { expect(persistence.readFrom).toHaveBeenNthCalledWith(2, SessionId('shrunk'), 0, undefined) }) + it('discards malformed persisted state and degrades to one full re-read', async () => { + const pool = new MemoryMediaPool() + const logs = new Map([['malformed', storedLog([['real']])]]) + seedRow(pool, 'malformed', { ver: 1, seq: 1, val: { marks: 'not-an-array' } }) + const { cache, persistence } = await harness({ pool, logs }) + + const snapshot = await cache.coldSnapshot(SessionId('malformed')) + + expect(snapshot.values['cache-test/marks']).toEqual({ marks: ['real'] }) + expect(persistence.readFrom).toHaveBeenNthCalledWith(1, SessionId('malformed'), 1, undefined) + expect(persistence.readFrom).toHaveBeenNthCalledWith(2, SessionId('malformed'), 0, undefined) + }) + it('write-back failure is contained: the snapshot is still served', async () => { const pool = new MemoryMediaPool() const logs = new Map([['soft', storedLog([['a']])]]) diff --git a/packages/session/session-projection/README.i18n.yaml b/packages/session/session-projection/README.i18n.yaml index fb26401df9..7ce339474e 100644 --- a/packages/session/session-projection/README.i18n.yaml +++ b/packages/session/session-projection/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-projection/README.md -README.md: 9018b133bb69ed4717fede14c9a2070a07c3fa62 -README.zh.md: 2a3af5620f84ff9697268101a6cfb894232b68b7 +README.md: 3b7ccecb7040b5340cd24da45d99bbfdc13fa15c +README.zh.md: bdf691761a6debf8f904924ab7198a5aed093b02 diff --git a/packages/session/session-projection/README.md b/packages/session/session-projection/README.md index 9018b133bb..3b7ccecb70 100644 --- a/packages/session/session-projection/README.md +++ b/packages/session/session-projection/README.md @@ -9,21 +9,23 @@ Session-projection Service Definition and drive registry. It owns `ctx.sessionPr ### Public API - `ctx.sessionProjections.register(definition): () => void` Register one domain's unit. Duplicate keys and invalid `stateVersion` throw; the registration is an effect on the calling fiber, so an unloaded domain plugin's key (with its cached cells) disappears from subsequent drives and snapshots — clients read that as capability absence. -- `ctx.sessionProjections.onChanged(listener): () => void` Subscribe to the change feed: one call per unit whose state reference changed, per committed event, carrying the schema-validated view and the causing seq. Effect-tied like `register`. -- `ctx.sessionProjections.snapshot(session): ProjectionSnapshot` One consistent synchronous cut over every registered unit — `{ asOfSeq, values }` with `asOfSeq` = the seq of the last event every value reflects (`-1` for an empty log). +- `ctx.sessionProjections.onChanged(listener): () => void` Subscribe to the change feed: one call per client-visible unit whose state reference changed, per committed event, carrying the schema-validated view and the causing seq. Effect-tied like `register`. +- `ctx.sessionProjections.stateOf(session, key)` Read one registered unit's current host state without computing unrelated views. The returned value is a live read-only reference; callers must not mutate it. +- `ctx.sessionProjections.snapshot(session): ProjectionSnapshot` One consistent synchronous cut over every registered client-visible unit — `{ asOfSeq, values }` with `asOfSeq` = the seq of the last event every value reflects (`-1` for an empty log). Host-only state is available only through `stateOf`. ### Key Types -- `SessionProjectionMap` — the single merge-extensible type table for the whole chain (host unit, wire block, React hook). Values are wire-JSON whole values; rendering belongs to the slot system, never this layer. -- `ProjectionDefinition` — `{ key, schema, init(), apply(state, event), view(state), stateVersion }`: a state-driven computation unit of three pure synchronous functions plus declarations, never an opaque getter. +- `SessionProjectionMap` — the merge-extensible client-view table shared by wire blocks and client hooks. Values are wire-JSON whole values; rendering belongs to the slot system, never this layer. +- `SessionProjectionStateMap` — the merge-extensible host fold-state table. Every client-visible key appears in both tables; host-only keys appear only here. +- `ProjectionDefinition` — `{ key, stateSchema, init(), apply(state, event), wire?, stateVersion }`: a synchronous state-driven computation unit. `wire` supplies `viewSchema` and `view`; omitting it makes the unit host-only. ## Contract - **The framework drives, the domain computes.** The registry subscribes to `session/event` once; every committed event passes every unit's `apply` eagerly. Domains hold no subscriptions. Cells (`{state, observedSeq}` per unit per session, WeakMap-keyed) build lazily — a unit registered after events flowed, or a read of a session predating the registration, folds `init` over the in-memory log on first touch. - **Same-reference means no work.** `apply` MUST return the same state reference for events that do not concern the unit; the drive gates the change feed on `Object.is`, so non-matching events cost one call and nothing downstream. - **Whole-value event rule (load-bearing).** A state-carrying log event MUST carry the complete post-change state, never a bare delta — it keeps every transition trivially cheap and every served value self-describing (last-wins for consumers). -- **Synchronous unit discipline.** `init`/`apply`/`view` MUST be synchronous; carriers read `snapshot()` in the same tick as their page slice, which is what makes `asOfSeq` one consistent cut. An accidentally-async `view` returns a Promise, which fails the boundary `schema.parse` loudly. -- **State is plain JSON, `stateVersion` is its invalidation anchor.** The persisted projection cache stores `(sessionId, key, ver, seq, val)` rows; bump `stateVersion` whenever the state shape or the fold semantics change so stale rows are discarded instead of forward-applied into garbage. +- **Synchronous unit discipline.** `init`/`apply`/`wire.view` MUST be synchronous; carriers read `snapshot()` in the same tick as their page slice, which is what makes `asOfSeq` one consistent cut. An accidentally async view returns a Promise, which fails `wire.viewSchema.parse`. +- **State is validated plain JSON, `stateVersion` is its invalidation anchor.** The persisted projection cache stores `(sessionId, key, ver, seq, val)` rows and validates `val` with `stateSchema` before use; bump `stateVersion` whenever the state fields or fold semantics change. Every unit's state is checkpointed — client-visible and host-only alike. - **No wire vocabulary here.** The registry exposes only the change feed and the snapshot read face; carriers (api-proxy) mint their own frames (`session/projection`) and blocks from them. - **Optional capability.** Domain plugins register under `ctx.inject(['sessionProjections'], …)` so headless assemblies without the registry stay unaffected; carriers use `ctx.get('sessionProjections')` and omit their block/frames entirely when the registry is absent. @@ -41,8 +43,8 @@ None; projections never assemble or send provider requests. ## Known Limitations and Deferred Work -- **Every tail page carries every registered key** — there is no per-key opt-out or lazy-key request shape yet; acceptable while values are UI-scale whole states (a todo list, a goal snapshot), revisit if a domain's value grows large. +- **Every tail page carries every client-visible key** — there is no per-key opt-out or lazy-key request shape yet; acceptable while values are UI-scale whole states (a todo list, a goal snapshot), revisit if a domain's value grows large. - **The unit table is process-wide, so key presence is not a per-session capability signal** — a key registered by ANY agent preset appears in every session's snapshot, including sessions whose own composition mounts nothing that produces it. A client must read the VALUE (`plan.active`, an empty todo list) rather than treat an absent key as absence of the feature; a unit whose empty value is indistinguishable from a real one belongs on the host plane instead, which is why `dsh-token-meter` sits there. - **Eager drive touches every unit per event** — cheap by construction (whole-value rule, same-reference gate), but a hot path would justify per-unit event-type prefilters, addable without contract change. - **Registry cells live in memory only** — a restart rebuilds by folding the log on first touch; compositions that mount `dsh-session-projection-cache` seed that fold from persisted rows instead. -- **Synchronous unit discipline is only partially mechanical** — the boundary `schema.parse` rejects a Promise-returning `view`, but an `apply` that blocks or reads torn non-session state is a review concern; the invariant companion documents why no runtime check exists. +- **Synchronous unit discipline is only partially mechanical** — `wire.viewSchema.parse` rejects a Promise-returning view, but an `apply` that blocks or reads torn non-session state is a review concern; the invariant companion documents why no runtime check exists. diff --git a/packages/session/session-projection/README.zh.md b/packages/session/session-projection/README.zh.md index 2a3af5620f..bdf691761a 100644 --- a/packages/session/session-projection/README.zh.md +++ b/packages/session/session-projection/README.zh.md @@ -2,28 +2,30 @@ [English](README.md) | 中文 -会话投影 Service Definition 与驱动注册表。它拥有 `ctx.sessionProjections`:该注册表在已提交的会话事件上驱动每个已注册的投影单元,并向载体提供完整的最终值,目前包括 api-proxy 历史尾页和 `session/projection` 推送帧。领域注册的只是纯数学;驱动权归框架。[session-projection RFC](../../../.agents/notes/proposed/architecture/2026-07-27-session-projection-and-command-log.md) 记录了设计理由。 +会话投影 Service Definition 与驱动注册表。它拥有 `ctx.sessionProjections`:该注册表在已提交的会话事件上驱动每个已注册的投影单元,并向载体提供完整的最终值,目前包括 api-proxy 历史尾页和 `session/projection` 推送帧。领域注册的只是纯数学;驱动权归框架。[session-projection RFC](../../../.agents/notes/proposed/architecture/2026-07-27-session-projection-and-command-log.zh.md) 记录了设计理由。 ## 服务:`SessionProjectionRegistry`(ctx 键:`sessionProjections`) ### 公开 API - `ctx.sessionProjections.register(definition): () => void` 注册一个领域的单元。key 重复或 `stateVersion` 非法都会 throw;注册是挂在调用方 fiber 上的 effect,领域插件卸载后其 key(连同缓存的 cell)从后续驱动与快照中消失——客户端将其读作能力缺失。 -- `ctx.sessionProjections.onChanged(listener): () => void` 订阅变更流:每个已提交事件、每个状态引用发生变化的单元各回调一次,携带经 schema 校验的 view 与致因 seq。与 `register` 一样绑定 effect。 -- `ctx.sessionProjections.snapshot(session): ProjectionSnapshot` 对全部已注册单元做一次一致的同步切面——`{ asOfSeq, values }`,其中 `asOfSeq` = 所有值共同反映到的最后一个事件的 seq(空日志为 `-1`)。 +- `ctx.sessionProjections.onChanged(listener): () => void` 订阅变更流:每个已提交事件、每个状态引用发生变化的客户端可见单元各回调一次,携带经 schema 校验的 view 与致因 seq。与 `register` 一样绑定 effect。 +- `ctx.sessionProjections.stateOf(session, key)` 读取一个已注册单元的当前 host 状态,不计算无关 view。返回值是活的只读引用;调用方不得修改。 +- `ctx.sessionProjections.snapshot(session): ProjectionSnapshot` 对全部已注册客户端可见单元做一次一致的同步切面——`{ asOfSeq, values }`,其中 `asOfSeq` = 所有值共同反映到的最后一个事件的 seq(空日志为 `-1`)。host-only 状态只能通过 `stateOf` 读取。 ### 关键类型 -- `SessionProjectionMap`——整条链路唯一的 merge-extensible 类型表(host 侧单元、协议块、React 钩子)。值是协议层 JSON 全量值;渲染归 slot 体系管,永远不归本层。 -- `ProjectionDefinition`——`{ key, schema, init(), apply(state, event), view(state), stateVersion }`:由三个纯同步函数外加若干声明构成的状态驱动计算单元(state-driven computation unit),绝不是一个不透明的 getter。 +- `SessionProjectionMap`——协议块与客户端钩子共享的 merge-extensible client view 表。值是协议层 JSON 全量值;渲染归 slot 体系管,永远不归本层。 +- `SessionProjectionStateMap`——merge-extensible host 折叠状态表。每个 client-visible key 同时出现在两个表中;host-only key 只出现在这里。 +- `ProjectionDefinition`——`{ key, stateSchema, init(), apply(state, event), wire?, stateVersion }`:同步的状态驱动计算单元。`wire` 提供 `viewSchema` 与 `view`;省略它即为 host-only 单元。 ## 约定 - **框架负责驱动,领域负责计算。** 注册表只订阅一次 `session/event`;每个已提交事件都会主动经过每个单元的 `apply`。领域不持有任何订阅。cell(每会话每单元一份 `{state, observedSeq}`,以 WeakMap 为键)惰性构建——在事件流过之后才注册的单元,或读取一个早于该注册的会话,都在首次触达时从 `init` 出发在内存日志上折叠。 - **同引用即无工作。** 对与单元无关的事件,`apply` 必须返回同一个状态引用;驱动以 `Object.is` 把守变更流,因此不匹配的事件只花一次调用,不产生任何下游工作。 - **全量值事件规则(承重)。** 携带状态的日志事件必须携带变更后的完整状态,绝不携带裸增量——这让每次状态转移始终足够廉价,也让每个被供给的值自描述(对消费方即 last-wins)。 -- **单元的同步纪律。**`init`/`apply`/`view` 必须是同步的;载体在切出页面切片的同一 tick 内读取 `snapshot()`,`asOfSeq` 之所以是一个一致切面正系于此。误写成异步的 `view` 会返回 Promise,让边界的 `schema.parse` 当场大声失败。 -- **状态是纯 JSON,`stateVersion` 是其失效锚点。** 持久投影缓存(persisted projection cache)存储 `(sessionId, key, ver, seq, val)` 行;状态形状或折叠语义一旦变化就递增 `stateVersion`,使陈旧行被丢弃,而不是被正向 apply 成垃圾。 +- **单元的同步纪律。**`init`/`apply`/`wire.view` 必须是同步的;载体在切出页面切片的同一 tick 内读取 `snapshot()`,`asOfSeq` 之所以是一个一致切面正系于此。误写成异步的 view 会返回 Promise,并被 `wire.viewSchema.parse` 拒绝。 +- **状态是经校验的纯 JSON,`stateVersion` 是其失效锚点。** 持久投影缓存存储 `(sessionId, key, ver, seq, val)` 行,并在使用前以 `stateSchema` 校验 `val`;状态字段或折叠语义一旦变化就递增 `stateVersion`。每个单元的状态都会被检查点化——client-visible 与 host-only 一视同仁。 - **本层没有协议词汇。** 注册表只暴露变更流与快照读取面;载体(api-proxy)据此自铸各自的帧(`session/projection`)与块。 - **可选能力。** 领域插件在 `ctx.inject(['sessionProjections'], …)` 下注册,因此不带注册表的 headless 组装完全不受影响;载体使用 `ctx.get('sessionProjections')`,注册表缺席时完全省略自己的块与帧。 @@ -41,8 +43,8 @@ ## 已知限制与暂缓事项 -- **每个尾页携带每个已注册的 key**——尚无逐 key 的 opt-out 或惰性 key 请求形状;在值都是 UI 量级的全量状态(一张 todo 清单、一份 goal 快照)时可以接受,若某领域的值变大再重议。 +- **每个尾页携带每个 client-visible key**——尚无逐 key 的 opt-out 或惰性 key 请求形状;在值都是 UI 量级的全量状态(一张 todo 清单、一份 goal 快照)时可以接受,若某领域的值变大再重议。 - **单元表是进程级的,因此 key 是否存在不能当作逐会话的能力信号**——只要**任何**一个 agent preset 注册了某个 key,它就出现在每个会话的快照里,包括自身组装完全不产出该值的会话。客户端必须读**值**(`plan.active`、空的 todo 列表),不能把 key 缺席当作功能缺席;如果某个单元的空值与真实值无法区分,它就该待在宿主平面——`dsh-token-meter` 正因如此留在那里。 - **主动驱动(eager drive)逐事件触达每个单元**——按构造开销很低(全量值规则、同引用闸门),但若出现热点路径,可加按单元的事件类型预过滤,约定不变。 - **注册表 cell 只活在内存里**——重启后首次触达时靠折叠日志重建;挂载了 `dsh-session-projection-cache` 的组合改由持久行播种该折叠。 -- **单元同步纪律只有部分可机械把关**——边界 `schema.parse` 能拒绝返回 Promise 的 `view`,但阻塞的 `apply`、或读取撕裂的非会话状态的 `apply`,只能靠评审把关;invariant 配套项记载了为何不存在运行时检查。 +- **单元同步纪律只有部分可机械把关**——`wire.viewSchema.parse` 能拒绝返回 Promise 的 view,但阻塞的 `apply`、或读取撕裂的非会话状态的 `apply`,只能靠评审把关;invariant 配套项记载了为何不存在运行时检查。 diff --git a/packages/session/session-projection/package.json b/packages/session/session-projection/package.json index cdf61ac3a3..ca0bbba5a1 100644 --- a/packages/session/session-projection/package.json +++ b/packages/session/session-projection/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-projection", "description": "Session-projection seam: the merge-extensible projection type table, the provider contract, and the ctx.sessionProjections registry serving whole current values of log-derived per-session state", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-projection/src/index.ts b/packages/session/session-projection/src/index.ts index 9f0c24e72e..68d95d36f6 100644 --- a/packages/session/session-projection/src/index.ts +++ b/packages/session/session-projection/src/index.ts @@ -1,9 +1,9 @@ /** - * Service Definition and drive registry for the session-projection capability seam: the merge-extensible `SessionProjectionMap` type - * table, the `ProjectionDefinition` state-driven computation unit contract, + * Service Definition and drive registry for the session-projection capability seam: the merge-extensible state and client-view type + * tables, the `ProjectionDefinition` state-driven computation unit contract, * and the `ctx.sessionProjections` registry that DRIVES every registered unit * forward eagerly over committed session events. Domain host plugins - * contribute pure mathematics (init/apply/view); the framework owns the + * contribute pure folds and optional client views; the framework owns the * subscription, the per-session watermark cache, and change notification; * carriers consume the snapshot read face and the change feed. Neither side * knows the other @@ -27,28 +27,31 @@ declare module '@deepseek-ai/cordis' { } } -import type { SessionProjectionMap } from './types.ts' +import type { SessionProjectionMap, SessionProjectionStateMap } from './types.ts' -export type { SessionProjectionMap } from './types.ts' +export type { SessionProjectionMap, SessionProjectionStateMap } from './types.ts' /** - * One domain's state-driven computation unit: three pure synchronous - * functions plus declarations — never an opaque getter. The framework drives + * One domain's state-driven computation unit: a pure synchronous fold plus + * declarations and an optional client view — never an opaque getter. The framework drives * `apply` on every committed session event; the domain holds no - * subscriptions and owns only the mathematics. All three functions MUST be - * synchronous (an async unit would tear the carriers' consistency cut) and + * subscriptions and owns only the computation. All functions MUST be + * synchronous (an async unit would tear the carriers' consistency cut), and * `state` MUST be plain JSON (the persisted-cache precondition). */ -export interface ProjectionDefinition { - /** The projection key this unit owns (its `SessionProjectionMap` entry). */ +export interface ProjectionDefinition< + K extends keyof SessionProjectionStateMap, + S extends SessionProjectionStateMap[K] = SessionProjectionStateMap[K], +> { + /** The projection key this unit owns (its `SessionProjectionStateMap` entry). */ key: K - /** Validates the wire payload (`view` output) before it leaves the host. */ - schema: ZodType + /** Validates persisted state before it seeds a fold. */ + stateSchema: ZodType /** * State for the empty log. * @returns the initial state. */ - init(): S + init(): NoInfer /** * Pure transition: previous state + one committed event → next state. A * unit uninterested in an event MUST return the same state reference — an @@ -57,13 +60,18 @@ export interface ProjectionDefinition { * @param event - the next committed session event. * @returns the next state (same reference when the event is not the unit's). */ - apply(state: S, event: SessionEvent): S - /** - * State → wire payload (the read-side projection). - * @param state - the current state. - * @returns the whole current value for this unit's key. - */ - view(state: S): SessionProjectionMap[K] + apply(state: NoInfer, event: SessionEvent): NoInfer + /** Client view. Omit for host-only units. */ + wire?: K extends keyof SessionProjectionMap ? { + /** Validates the wire payload before it leaves the host. */ + viewSchema: ZodType + /** + * State → wire payload (the read-side projection). + * @param state - the current state. + * @returns the whole current value for this unit's key. + */ + view(state: NoInfer): SessionProjectionMap[K] + } : never /** * Persisted-cache invalidation version: bump whenever the serialized state fields or the * fold semantics change, so persisted `(sessionId, key, ver, seq, val)` @@ -86,14 +94,14 @@ export type ProjectionChangeListener = ( ) => void /** - * One consistent read cut over every registered unit for one session. + * One consistent read cut over every registered client-visible unit for one session. * `asOfSeq` is the shared watermark — the seq of the last event every value * reflects (`-1` for an empty log, mirroring `session/subscribed.lastSeq`). */ export interface ProjectionSnapshot { /** Seq of the last event the values reflect; -1 for an empty log. */ asOfSeq: number - /** Whole current value per registered key. */ + /** Whole current client value per registered key. */ values: Partial } @@ -120,10 +128,10 @@ export type ProjectionCheckpoint = Record /** Type-erased unit view the drive machinery works with (the registration contract already proved the typed form). */ interface ErasedDefinition { key: string - schema: { parse(value: unknown): unknown } + stateSchema: { parse(value: unknown): unknown } init(): unknown apply(state: unknown, event: SessionEvent): unknown - view(state: unknown): unknown + wire: { viewSchema: { parse(value: unknown): unknown }; view(state: unknown): unknown } | undefined stateVersion: number } @@ -139,7 +147,7 @@ interface UnitCell { * once the last registrant releases it). * * `refs` exists because one unit definition already serves every session — the - * cells are keyed by `Session` — while the registrants are now per-session: + * cells are keyed by `Session` — while registrants are per-session: * an agent preset mounts the same tool package once per agent, so N sessions * on one preset register the same key N times. Without a count the first * registrant would own the disposer, and its session ending would strip the @@ -156,7 +164,8 @@ interface Registration { * `ctx.sessionProjections`: the projection unit table and its drive. The * service subscribes to `session/event` once; every committed event passes * every registered unit's `apply` (eager drive), and a changed state - * reference notifies the change feed with the schema-validated view. + * reference in a client-visible unit notifies the change feed with the + * schema-validated view. * Cells build lazily — a unit registered after events flowed, or a session * older than the registry, folds `init` over the in-memory log on first * touch (event or read). Registration is an effect (disposer rides the @@ -191,22 +200,54 @@ export class SessionProjectionRegistry extends Service { * @param definition - key, state schema, pure unit functions, and stateVersion. * @returns the exact disposer that unregisters this unit. */ - register(definition: ProjectionDefinition): () => void { + register< + K extends keyof SessionProjectionMap, + S extends SessionProjectionStateMap[K], + >( + definition: Omit, 'wire'> & { + wire: NonNullable['wire']> + }, + ): () => void + /** + * Register one host-only unit. Its state is omitted from client snapshots + * and always checkpointed like every other unit. + * @param definition - key, state schema, pure unit functions, and stateVersion. + * @returns the exact disposer that unregisters this unit. + */ + register< + K extends Exclude, + S extends SessionProjectionStateMap[K], + >( + definition: Omit, 'wire'>, + ): () => void + register( + definition: ProjectionDefinition, + ): () => void { + const wire = definition.wire as { + viewSchema: ZodType + view(state: S): unknown + } | undefined + const erased: ErasedDefinition = { + key: definition.key, + stateSchema: definition.stateSchema, + init: () => definition.init(), + apply: (state, event) => definition.apply(state as S, event), + wire: wire === undefined + ? undefined + : { viewSchema: wire.viewSchema, view: state => wire.view(state as S) }, + stateVersion: definition.stateVersion, + } if (!Number.isSafeInteger(definition.stateVersion) || definition.stateVersion < 0) { throw new Error(`session projection ${JSON.stringify(definition.key)} stateVersion must be a non-negative integer, got ${String(definition.stateVersion)}`) } const dispose = this.ctx.effect(function* (this: SessionProjectionRegistry) { - const key = definition.key as string + const key = erased.key const existing = this.registrations.get(key) if (existing === undefined) { - this.registrations.set(key, { def: definition, cells: new WeakMap(), refs: 1 }) + this.registrations.set(key, { def: erased, cells: new WeakMap(), refs: 1 }) } else { - // A differing `stateVersion` is the one incompatibility this can name: - // the versioned contract says the cached state shape differs, so the - // two registrants cannot share cells. Anything else about a definition - // is functions, which no runtime comparison can tell apart. - if (existing.def.stateVersion !== definition.stateVersion) { - throw new Error(`session projection key ${JSON.stringify(key)} is already registered at stateVersion ${String(existing.def.stateVersion)}; refusing to share it with stateVersion ${String(definition.stateVersion)}`) + if (existing.def.stateVersion !== erased.stateVersion) { + throw new Error(`session projection key ${JSON.stringify(key)} is already registered at stateVersion ${String(existing.def.stateVersion)}; refusing to share it with stateVersion ${String(erased.stateVersion)}`) } existing.refs += 1 } @@ -224,7 +265,7 @@ export class SessionProjectionRegistry extends Service { /** * Subscribe to the change feed. The registration is an effect on the * calling context's fiber. - * @param listener - called once per unit whose state reference changed, per committed event. + * @param listener - called once per client-visible unit whose state reference changed, per committed event. * @returns the exact disposer that unsubscribes. */ onChanged(listener: ProjectionChangeListener): () => void { @@ -238,24 +279,41 @@ export class SessionProjectionRegistry extends Service { } /** - * One consistent cut over every registered unit for one session, read from + * Read one unit's current host state without computing unrelated views. + * The returned value is live; callers must not mutate it. + * @param session - the session whose state is read. + * @param key - the registered unit key. + * @returns current state, or `undefined` when the key is not registered. + */ + stateOf( + session: Session, + key: K, + ): SessionProjectionStateMap[K] | undefined { + const registration = this.registrations.get(key) + if (registration === undefined) return undefined + return this.cellFor(registration, session).state as SessionProjectionStateMap[K] + } + + /** + * One consistent cut over every registered client-visible unit for one session, read from * the watermark cache (missing cells fold lazily over the in-memory log). * Fully synchronous — every value and `asOfSeq` reflect the same log - * position. Each value passes its unit's schema before leaving. + * position. Each value passes its unit's `viewSchema` before leaving. * @param session - the session whose projection values are read. - * @returns the snapshot; `values` is empty when no unit is registered. + * @returns the snapshot; `values` is empty when no client-visible unit is registered. */ snapshot(session: Session): ProjectionSnapshot { const values: Record = {} for (const registration of this.registrations.values()) { + if (registration.def.wire === undefined) continue const cell = this.cellFor(registration, session) - values[registration.def.key] = registration.def.schema.parse(registration.def.view(cell.state)) + values[registration.def.key] = registration.def.wire.viewSchema.parse(registration.def.wire.view(cell.state)) } - return { asOfSeq: session.seq - 1, values: values } + return { asOfSeq: session.seq - 1, values } } /** - * State-level checkpoint of every registered unit for one session, read + * State-level checkpoint of every persisted unit for one session, read * from the watermark cache (missing cells fold lazily over the in-memory * log). This is the write side of the persisted projection cache: the * returned rows are the `(key → {ver, seq, val})` part of the durable @@ -266,7 +324,7 @@ export class SessionProjectionRegistry extends Service { * every subsequent snapshot and frame through it (plain JSON by the unit * contract, so the clone is total). * @param session - the session whose unit states are checkpointed. - * @returns one row per registered key; empty when no unit is registered. + * @returns one row per registered key. */ checkpoint(session: Session): ProjectionCheckpoint { const rows: ProjectionCheckpoint = {} @@ -311,8 +369,8 @@ export class SessionProjectionRegistry extends Service { /** * View a checkpoint's rows without any log read: for every registered - * unit whose row's `ver` matches, serve the schema-validated - * `view` of the stored state; mismatched or absent rows leave their key + * client-visible unit whose row's `ver` matches, serve the schema-validated + * `view` of the schema-validated stored state; mismatched, malformed, or absent rows leave their key * absent (a cold or listing consumer treats it as not-yet-available and a * fuller read path refolds it). The zero-I/O rung of the read ladder — * values are as stale as their rows, never wrong. @@ -323,15 +381,22 @@ export class SessionProjectionRegistry extends Service { const values: Record = {} for (const registration of this.registrations.values()) { const def = registration.def + if (def.wire === undefined) continue const row = checkpoint[def.key] if (row === undefined || row.ver !== def.stateVersion) continue - values[def.key] = def.schema.parse(def.view(row.val)) + let state: unknown + try { + state = def.stateSchema.parse(row.val) + } catch { + continue + } + values[def.key] = def.wire.viewSchema.parse(def.wire.view(state)) } return values } /** - * Cold read: fold every registered unit over a stored log suffix, seeding + * Cold read: fold every persisted unit over a stored log suffix, seeding * each from its checkpoint row when usable — the one read recipe (cached * state + forward tail replay + `view`) applied without a live `Session`. * Call with the events returned by a persistence @@ -352,7 +417,11 @@ export class SessionProjectionRegistry extends Service { * supplied event's seq, `baseSeq - 1` for an empty tail) plus the * refreshed checkpoint rows at that cut, ready for a durable write-back. */ - restore(checkpoint: ProjectionCheckpoint, events: readonly SessionEvent[], baseSeq: number): + restore( + checkpoint: ProjectionCheckpoint, + events: readonly SessionEvent[], + baseSeq: number, + ): { snapshot: ProjectionSnapshot; checkpoint: ProjectionCheckpoint } { const endSeq = events.at(-1)?.seq ?? baseSeq - 1 const values: Record = {} @@ -370,12 +439,12 @@ export class SessionProjectionRegistry extends Service { + 'its checkpoint row is missing, version-mismatched, or beyond the supplied log end; re-read from seq 0', ) } - let state = usable ? row.val : def.init() + let state = usable ? def.stateSchema.parse(row.val) : def.init() const from = usable ? row.seq : baseSeq - 1 for (const event of events) { if (event.seq > from) state = def.apply(state, event) } - values[def.key] = def.schema.parse(def.view(state)) + if (def.wire !== undefined) values[def.key] = def.wire.viewSchema.parse(def.wire.view(state)) refreshed[def.key] = { ver: def.stateVersion, seq: endSeq, val: state } } return { @@ -415,8 +484,8 @@ export class SessionProjectionRegistry extends Service { const changed = !Object.is(next, cell.state) cell.state = next cell.observedSeq = event.seq - if (changed && this.listeners.size > 0) { - const value = registration.def.schema.parse(registration.def.view(next)) + if (changed && registration.def.wire !== undefined && this.listeners.size > 0) { + const value = registration.def.wire.viewSchema.parse(registration.def.wire.view(next)) for (const listener of this.listeners) { listener(session, registration.def.key as Extract, value, event.seq) } diff --git a/packages/session/session-projection/src/types.ts b/packages/session/session-projection/src/types.ts index 37d24b5b40..8373e4677b 100644 --- a/packages/session/session-projection/src/types.ts +++ b/packages/session/session-projection/src/types.ts @@ -9,9 +9,16 @@ */ /** - * The single projection type table for the whole chain (host provider, wire - * block, client cell, React hook). Domain packages merge their key here via - * declaration merging; values are wire-JSON whole values. How a value is - * rendered is the slot system's business, never this layer's. + * The merge-extensible client projection table shared by wire blocks, client + * cells, and React hooks. Domain packages merge their client-visible key here; + * values are wire-JSON whole values. How a value is rendered is the slot + * system's business, never this layer's. */ export interface SessionProjectionMap {} + +/** + * The merge-extensible host fold-state table. Each client-visible key also + * appears in {@link SessionProjectionMap}; host-only keys appear only here. + * Values must be plain JSON so the projection cache can persist them. + */ +export interface SessionProjectionStateMap {} diff --git a/packages/session/session-projection/tests/registry.spec.ts b/packages/session/session-projection/tests/registry.spec.ts index 4ae4a3face..3ee2491d4f 100644 --- a/packages/session/session-projection/tests/registry.spec.ts +++ b/packages/session/session-projection/tests/registry.spec.ts @@ -16,9 +16,13 @@ import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection' declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + 'test/marks': MarksState + 'test/count': number + } + interface SessionProjectionMap { 'test/marks': { marks: string[] } - 'test/count': number } } @@ -28,24 +32,27 @@ declare module '@deepseek-ai/dsh-session/types' { } } -/** Whole-value unit: latest test/mark event wins; unrelated events return the same reference. */ type MarksState = { marks: string[] } | null -const marksUnit = (): ProjectionDefinition<'test/marks', MarksState> => ({ +/** Whole-value unit: latest test/mark event wins; unrelated events return the same reference. */ +const marksUnit = (): Omit, 'wire'> + & { wire: NonNullable['wire']> } => ({ key: 'test/marks', - schema: z.object({ marks: z.array(z.string()) }), + stateSchema: z.object({ marks: z.array(z.string()) }).nullable(), init: () => null, apply: (state, event) => (event.type === 'test/mark' ? (event).data : state), - view: state => state ?? { marks: [] }, + wire: { + viewSchema: z.object({ marks: z.array(z.string()) }), + view: state => state ?? { marks: [] }, + }, stateVersion: 1, }) -/** Counting unit over every event — state changes on each apply. */ +/** Host-only counting unit over every event — state changes on each apply. */ const countUnit = (): ProjectionDefinition<'test/count', number> => ({ key: 'test/count', - schema: z.number().int().nonnegative(), + stateSchema: z.number().int().nonnegative(), init: () => 0, apply: state => state + 1, - view: state => state, stateVersion: 1, }) @@ -111,7 +118,7 @@ describe('SessionProjectionRegistry drive', () => { expect(ctx.sessionProjections.snapshot(other).values['test/marks']).toEqual({ marks: ['two'] }) }) - it('runs every registered unit — a changing unit notifies while a same-reference unit stays silent', async () => { + it('updates host-only units without publishing them to wire listeners', async () => { const { ctx, session } = await harness() ctx.sessionProjections.register(marksUnit()) ctx.sessionProjections.register(countUnit()) @@ -120,11 +127,9 @@ describe('SessionProjectionRegistry drive', () => { changedKeys.push(key) }) session.append('turn/start', { turn: 1 }) - // count applied (+1 change), marks returned the same reference. - expect(changedKeys).toEqual(['test/count']) - const snapshot = ctx.sessionProjections.snapshot(session) - expect(snapshot.values['test/count']).toBe(1) - expect(snapshot.values['test/marks']).toEqual({ marks: [] }) + expect(changedKeys).toEqual([]) + expect(ctx.sessionProjections.stateOf(session, 'test/count')).toBe(1) + expect(ctx.sessionProjections.snapshot(session).values).toEqual({ 'test/marks': { marks: [] } }) }) it('shares one unit between registrants of the same key', async () => { @@ -147,9 +152,9 @@ describe('SessionProjectionRegistry drive', () => { first() - // The regression this counts against: one session ending used to strip - // the projection from every other live session, because the first - // registrant owned the only disposer. + // The regression this counts against: without last-release semantics, one + // session ending strips the projection from every other live session, + // because the first registrant owns the only disposer. expect(ctx.sessionProjections.snapshot(session).values['test/marks']).toEqual({ marks: ['kept'] }) second() expect(ctx.sessionProjections.snapshot(session).values).toEqual({}) @@ -200,7 +205,19 @@ describe('SessionProjectionRegistry drive', () => { expect(ctx.sessionProjections.snapshot(session).values).toEqual({}) }) - it('checkpoints every registered unit with its stateVersion and per-cell watermark', async () => { + it('snapshot serves client views and excludes host-only state', async () => { + const { ctx, session } = await harness() + ctx.sessionProjections.register(marksUnit()) + ctx.sessionProjections.register(countUnit()) + mark(session, ['a', 'b']) + const values = ctx.sessionProjections.snapshot(session).values + expect(values['test/marks']).toEqual({ marks: ['a', 'b'] }) + expect('test/count' in values).toBe(false) + expect(ctx.sessionProjections.stateOf(session, 'test/count')).toBe(1) + expect('test/unregistered' in values).toBe(false) + }) + + it('checkpoints every persisted unit with its stateVersion and per-cell watermark', async () => { const { ctx, session } = await harness() ctx.sessionProjections.register(marksUnit()) ctx.sessionProjections.register({ ...countUnit(), stateVersion: 7 }) @@ -277,7 +294,7 @@ describe('SessionProjectionRegistry drive', () => { }, full, 0) expect(snapshot.asOfSeq).toBe(4) expect(snapshot.values['test/marks']).toEqual({ marks: ['new'] }) - expect(snapshot.values['test/count']).toBe(5) // refolded from init over all 5 events + expect('test/count' in snapshot.values).toBe(false) // The refreshed rows sit at the served cut, ready for a durable write-back. expect(checkpoint['test/marks']).toEqual({ ver: 1, seq: 4, val: { marks: ['new'] } }) expect(checkpoint['test/count']).toEqual({ ver: 1, seq: 4, val: 5 }) @@ -295,20 +312,22 @@ describe('SessionProjectionRegistry drive', () => { { type: 'turn/start', seq: 3, time: 3, data: { turn: 2 } }, { type: 'turn/end', seq: 4, time: 4, data: { turn: 2, reason: { kind: 'completed' } } }, ] - const { snapshot } = ctx.sessionProjections.restore(rows, tail, 3) + const { snapshot, checkpoint } = ctx.sessionProjections.restore(rows, tail, 3) expect(snapshot.asOfSeq).toBe(4) // marks already covers the tail (watermark 4): nothing re-applied. expect(snapshot.values['test/marks']).toEqual({ marks: ['done'] }) - // count folds exactly seqs 3 and 4 on top of its checkpoint. - expect(snapshot.values['test/count']).toBe(5) + // count folds exactly seqs 3 and 4 on top of its checkpoint, but remains host-only. + expect(checkpoint['test/count']).toEqual({ ver: 1, seq: 4, val: 5 }) + expect('test/count' in snapshot.values).toBe(false) // Empty tail (checkpoint is current): the cut sits at baseSeq - 1. - const { snapshot: current } = ctx.sessionProjections.restore({ + const { snapshot: current, checkpoint: currentCheckpoint } = ctx.sessionProjections.restore({ 'test/marks': { ver: 1, seq: 4, val: { marks: ['done'] } }, 'test/count': { ver: 1, seq: 4, val: 5 }, }, [], 5) expect(current.asOfSeq).toBe(4) - expect(current.values['test/count']).toBe(5) + expect('test/count' in current.values).toBe(false) + expect(currentCheckpoint['test/count']).toEqual({ ver: 1, seq: 4, val: 5 }) }) it('viewCheckpoint serves version-matching rows without any log and skips mismatched keys', async () => { @@ -324,6 +343,36 @@ describe('SessionProjectionRegistry drive', () => { expect(ctx.sessionProjections.viewCheckpoint({})).toEqual({}) }) + it('viewCheckpoint and restore exclude host-only state while retaining its checkpoint', async () => { + const { ctx } = await harness() + ctx.sessionProjections.register(marksUnit()) + ctx.sessionProjections.register(countUnit()) + const rows = { + 'test/marks': { ver: 1, seq: 4, val: { marks: ['stored'] } }, + 'test/count': { ver: 1, seq: 4, val: 5 }, + } + expect(ctx.sessionProjections.viewCheckpoint(rows)).toEqual({ + 'test/marks': { marks: ['stored'] }, + }) + + const restored = ctx.sessionProjections.restore(rows, [], 5) + expect(restored.snapshot.values).toEqual({ + 'test/marks': { marks: ['stored'] }, + }) + expect(restored.checkpoint['test/count']).toEqual(rows['test/count']) + }) + + it('rejects version-matching rows whose state no longer matches the registered schema', async () => { + const { ctx } = await harness() + ctx.sessionProjections.register(marksUnit()) + const drifted = { + 'test/marks': { ver: 1, seq: 2, val: { marks: 'not-an-array' } }, + } + + expect(ctx.sessionProjections.viewCheckpoint(drifted)).toEqual({}) + expect(() => ctx.sessionProjections.restore(drifted, [], 3)).toThrow() + }) + it('restore rejects a row claiming events past the supplied log end (shrunk log ⇒ re-read)', async () => { const { ctx } = await harness() ctx.sessionProjections.register(countUnit()) @@ -334,7 +383,9 @@ describe('SessionProjectionRegistry drive', () => { expect(floor).toBe(9) // …an intact log serves the anchor event and the checkpoint stands as-is. const anchor: SessionEvent = { type: 'turn/end', seq: 9, time: 9, data: { turn: 2, reason: { kind: 'completed' } } } - expect(ctx.sessionProjections.restore(rows, [anchor], 9).snapshot.values['test/count']).toBe(10) + const anchored = ctx.sessionProjections.restore(rows, [anchor], 9) + expect(anchored.snapshot.values).toEqual({}) + expect(anchored.checkpoint['test/count']).toEqual({ ver: 1, seq: 9, val: 10 }) // …while a log crash-repaired down to fewer events returns an empty tail: // the row overreaches the proven end and a tail read cannot fix this key. expect(() => ctx.sessionProjections.restore(rows, [], 9)).toThrow(/re-read from seq 0/) @@ -343,21 +394,25 @@ describe('SessionProjectionRegistry drive', () => { { type: 'turn/start', seq: 0, time: 0, data: { turn: 1 } }, { type: 'turn/end', seq: 1, time: 1, data: { turn: 1, reason: { kind: 'completed' } } }, ] - const { snapshot } = ctx.sessionProjections.restore(rows, events, 0) + const { snapshot, checkpoint } = ctx.sessionProjections.restore(rows, events, 0) expect(snapshot.asOfSeq).toBe(1) - expect(snapshot.values['test/count']).toBe(2) + expect(snapshot.values).toEqual({}) + expect(checkpoint['test/count']).toEqual({ ver: 1, seq: 1, val: 2 }) }) it('fails loud when a unit view violates its own schema (async unit output is unrepresentable)', async () => { const { ctx, session } = await harness() ctx.sessionProjections.register({ key: 'test/marks', - schema: z.object({ marks: z.array(z.string()) }), + stateSchema: z.object({ marks: z.array(z.string()) }).nullable(), init: () => null as MarksState, apply: state => state, - // A Promise (what an accidentally-async view would return) is not the - // declared shape: the boundary parse rejects it before it leaves. - view: () => Promise.resolve({ marks: [] }) as never, + wire: { + viewSchema: z.object({ marks: z.array(z.string()) }), + // A Promise (what an accidentally-async view would return) is not the + // declared shape: the boundary parse rejects it before it leaves. + view: () => Promise.resolve({ marks: [] }) as never, + }, stateVersion: 1, }) expect(() => ctx.sessionProjections.snapshot(session)).toThrow() diff --git a/packages/session/session-stats/package.json b/packages/session/session-stats/package.json index c7006a5061..4627864028 100644 --- a/packages/session/session-stats/package.json +++ b/packages/session/session-stats/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-stats", "description": "Whole-log conversation counts and wall times projection (sessionStats) for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-stats/src/projection.ts b/packages/session/session-stats/src/projection.ts index a000300873..4ab43036c4 100644 --- a/packages/session/session-stats/src/projection.ts +++ b/packages/session/session-stats/src/projection.ts @@ -62,6 +62,12 @@ interface SessionStatsState extends SessionStatsTotals { pendingCalls: Record } +declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + sessionStats: SessionStatsState + } +} + const sessionStatsSchema = z.object({ turns: z.number().int().nonnegative(), steps: z.number().int().nonnegative(), @@ -73,6 +79,23 @@ const sessionStatsSchema = z.object({ decodeTokens: z.number().nonnegative(), }).strict() +/** + * The fold state's shape (totals plus in-flight boundaries), validated on + * persisted-cache rows after their `ver` gate — the unit's input boundary. + * The view is a strict subset of the state, so this schema extends + * `sessionStatsSchema` (the wire output boundary) with the boundary fields. + */ +const sessionStatsStateSchema = sessionStatsSchema.extend({ + lastTurn: z.number().int().nonnegative().nullable(), + openStep: z.object({ + turn: z.number().int().nonnegative(), + step: z.number().int().nonnegative(), + startTime: z.number().nonnegative(), + firstTokenTime: z.number().nonnegative().nullable(), + }).nullable(), + pendingCalls: z.record(z.string(), z.number().nonnegative()), +}) + /** * Provider-reported completion tokens, guarded the way the window fold guards * node usage. @@ -86,9 +109,10 @@ function usageOutputTokens(usage: unknown): number | null { } /** The `sessionStats` unit registered on `ctx.sessionProjections` (exported for the unit spec). */ -export const sessionStatsProjectionDefinition: ProjectionDefinition<'sessionStats', SessionStatsState> = { +export const sessionStatsProjectionDefinition = { key: 'sessionStats', - schema: sessionStatsSchema, + stateVersion: 1, + stateSchema: sessionStatsStateSchema, init: () => ({ turns: 0, steps: 0, @@ -169,15 +193,17 @@ export const sessionStatsProjectionDefinition: ProjectionDefinition<'sessionStat return state } }, - view: state => ({ - turns: state.turns, - steps: state.steps, - llmMs: state.llmMs, - toolMs: state.toolMs, - ttftMs: state.ttftMs, - ttftSteps: state.ttftSteps, - decodeMs: state.decodeMs, - decodeTokens: state.decodeTokens, - }), - stateVersion: 1, -} + wire: { + viewSchema: sessionStatsSchema, + view: state => ({ + turns: state.turns, + steps: state.steps, + llmMs: state.llmMs, + toolMs: state.toolMs, + ttftMs: state.ttftMs, + ttftSteps: state.ttftSteps, + decodeMs: state.decodeMs, + decodeTokens: state.decodeTokens, + }), + }, +} satisfies ProjectionDefinition<'sessionStats', SessionStatsState> diff --git a/packages/session/session-stats/tests/projection.spec.ts b/packages/session/session-stats/tests/projection.spec.ts index ebe728181b..63688ce436 100644 --- a/packages/session/session-stats/tests/projection.spec.ts +++ b/packages/session/session-stats/tests/projection.spec.ts @@ -154,11 +154,11 @@ function at(time: number, type: string, data: unknown): SessionEvent { /** Fold a synthetic event list through the definition and view the result. */ function fold(events: readonly SessionEvent[]): SessionStatsProjection { - const state = events.reduce( + const state = events.reduce[0]>( (folded, event) => sessionStatsProjectionDefinition.apply(folded, event), sessionStatsProjectionDefinition.init(), ) - return sessionStatsProjectionDefinition.view(state) + return sessionStatsProjectionDefinition.wire.view(state) } describe('sessionStats wall-time fold (controlled timestamps)', () => { @@ -269,7 +269,7 @@ describe('sessionStats wall-time fold (controlled timestamps)', () => { .toEqual(totals({ turns: 1, steps: 1, llmMs: 1_000, ttftMs: 400, ttftSteps: 1 })) // The first message closed the step boundary; a defensive duplicate finds // no open step and folds to the same reference. - const state = events.reduce( + const state = events.reduce[0]>( (folded, event) => sessionStatsProjectionDefinition.apply(folded, event), sessionStatsProjectionDefinition.init(), ) diff --git a/packages/session/session-telemetry-otel/README.i18n.yaml b/packages/session/session-telemetry-otel/README.i18n.yaml index c119707a56..31fa16233c 100644 --- a/packages/session/session-telemetry-otel/README.i18n.yaml +++ b/packages/session/session-telemetry-otel/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-telemetry-otel/README.md README.md: 351690b5941bfe33cfbbf591269683f706701978 -README.zh.md: e7031ff971cc8c9f2d1ce436513fbef1b9f27149 +README.zh.md: ae33e9df046e87d0c9726b8710525270cdd324e2 diff --git a/packages/session/session-telemetry-otel/README.zh.md b/packages/session/session-telemetry-otel/README.zh.md index e7031ff971..ae33e9df04 100644 --- a/packages/session/session-telemetry-otel/README.zh.md +++ b/packages/session/session-telemetry-otel/README.zh.md @@ -29,13 +29,13 @@ 上传授权采用显式许可,且为 fail-closed。通过直接构造传入未知模式时,会在读取传输配置前失败。只有 `FULL` 接受对 `ctx.sessionTelemetry.emit()` 的直接调用。`FEEDBACK_ONLY` 向其按需协调器提供私有后端能力,并且仅在 `feedback/record` 对象已经存储于 `session.events[event.seq]` 且对象身份完全相同时,才将其视为同意;独立发出的总线值会被忽略。即使存在导出器选项,`DISABLED` 也绝不会构造 SDK 流水线。 -已挂载的服务通过 seam 的 [`SessionTelemetrySharingStatus`](../session-telemetry/README.md#the-sharing-disclosure) `sharing` 属性披露解析后的模式(`full` / `feedback-only` / `disabled`),因此 `/feedback` 的确认文本可以报告会话是否以及如何被共享。该披露在构造函数中设置,与采集相互独立:即使 `DISABLED` 也会披露 `disabled`。 +已挂载的服务通过 seam 的 [`SessionTelemetrySharingStatus`](../session-telemetry/README.zh.md#the-sharing-disclosure) `sharing` 属性披露解析后的模式(`full` / `feedback-only` / `disabled`),因此 `/feedback` 的确认文本可以报告会话是否以及如何被共享。该披露在构造函数中设置,与采集相互独立:即使 `DISABLED` 也会披露 `disabled`。 `exporter.url` 在 `FULL` 与 `FEEDBACK_ONLY` 中必填,无默认值,且必须能解析为 `http(s)`;在 `DISABLED` 中可省略且不使用。在上传模式中,`shutdownTimeoutMillis` 是由 DSH 管理的有限正数外层截止时间,默认值为 3000 ms;`processor.maxExportBatchSize` 不是正整数时也会在插件加载时失败,因为 SDK 会接受该值,随后却在关闭时挂起。两个 SDK 配置块都整体透传(passthrough):`OTLPExporterNodeConfigBase` 的每个字段(`headers`、`timeoutMillis`、`compression`、`keepAlive` 等)都会到达导出器;批处理、导出节奏(`scheduledDelayMillis`)、重试、队列上限,以及持续失败下的丢失策略,都是通过 `processor` 调节的 SDK 行为。该后端不实现 `flush()`:常规 flush 由批处理器负责。关闭期间,OTel 会先等待 `exporter.forceFlush()`,再等待受处理器 `exportTimeoutMillis` 限制的完成 promise;如果该传输 promise 始终不结算,本包会在 `shutdownTimeoutMillis` 到期时放弃等待,通过协调器记录已隔离的关闭失败,并让应用继续拆卸。该截止时间无法取消 SDK 传输,因此届时仍待处理的记录可能在进程退出时丢失。 ## 哪些数据会离开本机 -在上传模式中,记录携带完整的 `event.data`,内容以 seam 的 `sessionTelemetry/record` waterfall(瀑布式事件)返回的结果为准:用户与 assistant 消息内容、工具参数与工具结果(命令输出、文件内容)、完整的系统提示词与工具 schema(`request/header`)、todo 文本、压缩(compaction)摘要、钩子的 `stderrSummary`、反馈文本,以及会话 `cwd`(一个本地路径)。seam 不带任何脱敏规则:未挂载 `sessionTelemetry/record` 监听器时,导出的就是捕获原样的副本,因此向可信边界之外导出的部署方要挂载自己的规则(见 [seam README](../session-telemetry/README.md#the-redact-waterfall))。`FULL` 在追加时运行脱敏;`FEEDBACK_ONLY` 不保留遥测副本,而是在反馈触发权威日志回放时运行当时挂载的规则。无论如何,提供方凭据都不会出现:适配器的 API key 是构造函数参数而非会话事件,因此它们在结构上就不存在于日志中,也就不存在于遥测中。`DISABLED` 不会构造 SDK 流水线,也不会将任何捕获内容交给后端。 +在上传模式中,记录携带完整的 `event.data`,内容以 seam 的 `sessionTelemetry/record` waterfall(瀑布式事件)返回的结果为准:用户与 assistant 消息内容、工具参数与工具结果(命令输出、文件内容)、完整的系统提示词与工具 schema(`request/header`)、todo 文本、压缩(compaction)摘要、钩子的 `stderrSummary`、反馈文本,以及会话 `cwd`(一个本地路径)。seam 不带任何脱敏规则:未挂载 `sessionTelemetry/record` 监听器时,导出的就是捕获原样的副本,因此向可信边界之外导出的部署方要挂载自己的规则(见 [seam README](../session-telemetry/README.zh.md#the-redact-waterfall))。`FULL` 在追加时运行脱敏;`FEEDBACK_ONLY` 不保留遥测副本,而是在反馈触发权威日志回放时运行当时挂载的规则。无论如何,提供方凭据都不会出现:适配器的 API key 是构造函数参数而非会话事件,因此它们在结构上就不存在于日志中,也就不存在于遥测中。`DISABLED` 不会构造 SDK 流水线,也不会将任何捕获内容交给后端。 ## 字段映射 diff --git a/packages/session/session-telemetry-otel/package.json b/packages/session/session-telemetry-otel/package.json index 37b5893cce..f5d4b60d88 100644 --- a/packages/session/session-telemetry-otel/package.json +++ b/packages/session/session-telemetry-otel/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-telemetry-otel", "description": "OpenTelemetry backend for the DeepSeek Harness telemetry seam: hands captured session records to the OTel JS SDK's log pipeline", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-telemetry/README.i18n.yaml b/packages/session/session-telemetry/README.i18n.yaml index efeccd9894..99cd673d4d 100644 --- a/packages/session/session-telemetry/README.i18n.yaml +++ b/packages/session/session-telemetry/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-telemetry/README.md README.md: e4ebee1324bc1228d6d97d891159685b05f94e4f -README.zh.md: 963d5f6d1c5c0f60f5bf14956f3b490d60bd6574 +README.zh.md: 718f07db3fb8d8f8a21236dcb917a33ed3e0cad8 diff --git a/packages/session/session-telemetry/README.zh.md b/packages/session/session-telemetry/README.zh.md index 963d5f6d1c..718f07db3f 100644 --- a/packages/session/session-telemetry/README.zh.md +++ b/packages/session/session-telemetry/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -遥测(telemetry)Service Definition 声明 `SessionTelemetrySink` 后端约定,捕获协调器把会话记录传给实现该约定的任意上报 SDK 后端。捕获侧可跟随实时会话事件,也可按需回放权威会话日志前缀。本包调用 `emit()` 后就停止处理:批处理、重试、排队与丢失策略都属于后端自身的 SDK,本包既不规定也不包装。设计依据与被否决的替代方案见[复活 Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md)、[反馈门控投递](../../../.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.md)与[无缓冲反馈回放](../../../.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.md)。 +遥测(telemetry)Service Definition 声明 `SessionTelemetrySink` 后端约定,捕获协调器把会话记录传给实现该约定的任意上报 SDK 后端。捕获侧可跟随实时会话事件,也可按需回放权威会话日志前缀。本包调用 `emit()` 后就停止处理:批处理、重试、排队与丢失策略都属于后端自身的 SDK,本包既不规定也不包装。设计依据与被否决的替代方案见[复活 Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md)、[反馈门控投递](../../../.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.zh.md)与[无缓冲反馈回放](../../../.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.zh.md)。 ## 后端约定 @@ -20,6 +20,8 @@ 在 `live` 模式中,协调器的全部注册都经由组合方 fiber 的 effect 完成:`session/created`(收养:记录 header,并经投影从构造边界起回读日志;来自 fork 或恢复的构造函数种子绝不会在 firehose 上再次发出,也绝不会再次导出)、`session/event`(投影、深拷贝、脱敏,再交接;零 I/O)、`session/flush`(转发可选的 `flush()` 提示并返回 void;循环所等待的并行任务绝不能等待遥测)、`session/disposed`(在会话自身的终止边缘捕获该会话的 `shutdown` 运维记录,然后将其退役)、`agent/error`(唯一的实时总线转发;会话事件词汇有意不包含运维错误记录)、一个 dispose effect(捕获每个仍存活会话的 shutdown,再等待后端的 `shutdown()`;失败只发出警告而不抛出),以及对 `ctx.sessions.list()` 的收养扫描(热重载不会重放 `session/created`)。在 `on-demand` 模式中,协调器只注册 dispose effect:`captureSession()` 读取权威日志,直至可选的序列号边界(含边界);flush 提示与运维事件留在本地。 + + ## 脱敏 waterfall(瀑布式事件) 每条记录在投影后立即经过 `sessionTelemetry/record` waterfall,这是 Service Definition 的脱敏扩展点。本包自身不带任何规则:最内层的 `next()` 原样透传记录,因此未挂载监听器时,记录以捕获时的原样到达后端;导出数据能干净到什么程度,恰恰取决于部署方挂载了什么规则。监听器通过变换 `next()` 的返回值来堆叠;不调用 `next()` 就返回,即替换其下方的全部逻辑;抛出异常的监听器会在协调器的隔离范围内以 fail-closed 方式拦下这一条记录。实时捕获在追加时运行 waterfall;按需捕获则在回放权威日志时使用当时挂载的规则运行 waterfall。脱敏只作用于外发副本;权威会话日志永不改写。 @@ -46,6 +48,6 @@ ## 已知限制与暂缓事项 -- **尽力而为的投递**:游标标记的是已交接而非已投递;在重载窗口内被拆除的会话无法重新收养;崩溃时留在后端队列中的内容会丢失。持久化 outbox(spool、每 sink 游标、at-least-once)推迟到有部署方提出明确的崩溃丢失要求时再实现;见[复活 Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md)。 +- **尽力而为的投递**:游标标记的是已交接而非已投递;在重载窗口内被拆除的会话无法重新收养;崩溃时留在后端队列中的内容会丢失。持久化 outbox(spool、每 sink 游标、at-least-once)推迟到有部署方提出明确的崩溃丢失要求时再实现;见[复活 Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md)。 - **不内置脱敏规则**:未挂载 `sessionTelemetry/record` 监听器时,记录以捕获时的原样离开进程,包括文件内容或命令输出中内嵌的任何凭据;向共享 collector 导出的部署方自行负责其规则集。 - **按需脱敏使用当前状态**:未捕获的事件只存在于权威会话日志中。后续的 `captureSession()` 会使用当时挂载的策略,深拷贝并脱敏其当前值;不存在捕获时的遥测快照或持久化的捕获前 spool。 diff --git a/packages/session/session-telemetry/package.json b/packages/session/session-telemetry/package.json index 132545f502..f802989e6d 100644 --- a/packages/session/session-telemetry/package.json +++ b/packages/session/session-telemetry/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-telemetry", "description": "SessionTelemetryBackend seam for the DeepSeek Harness: session-event capture, projection, redaction, and handoff to a reporting backend", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-telemetry/src/coordinator.ts b/packages/session/session-telemetry/src/coordinator.ts index f5efb0cb35..ff0417f168 100644 --- a/packages/session/session-telemetry/src/coordinator.ts +++ b/packages/session/session-telemetry/src/coordinator.ts @@ -159,8 +159,8 @@ export class SessionTelemetryCoordinator { * at or below the start still feed the projection state (first-chunk * tracking) without being re-handed, so a resumed fiber drops mid-step * chunk continuations exactly like the fiber that saw the step begin. The - * cost, accepted with the capture contract's at-most-once stance: a resume no longer - * backfills records a previous process failed to deliver. + * cost, accepted with the capture contract's at-most-once stance: a resume + * does not backfill records a previous process failed to deliver. * @param session - the live session to adopt; a second adoption is a no-op. */ private adopt(session: Session): void { diff --git a/packages/session/session-telemetry/src/index.ts b/packages/session/session-telemetry/src/index.ts index 077da9929d..0715b073f3 100644 --- a/packages/session/session-telemetry/src/index.ts +++ b/packages/session/session-telemetry/src/index.ts @@ -133,9 +133,8 @@ export interface SessionTelemetrySink { /** * Deployment-selected session-sharing policy disclosed by a mounted * {@link SessionTelemetryBackend} backend to human-facing acknowledgement surfaces (the - * `/feedback` command's confirmation text). The seam owns the vocabulary so - * any backend can disclose a policy without depending on the OTel package; - * the values mirror the OTel backend's serialized `SessionTelemetryMode` choices. + * `/feedback` command's confirmation text). The Service Definition owns the + * vocabulary so consumers and backends do not depend on a specific provider. */ export type SessionTelemetrySharingStatus = 'full' | 'feedback-only' | 'disabled' diff --git a/packages/session/session-title-all-prompts-llm/README.i18n.yaml b/packages/session/session-title-all-prompts-llm/README.i18n.yaml index 2d337c5a7c..06c98ab145 100644 --- a/packages/session/session-title-all-prompts-llm/README.i18n.yaml +++ b/packages/session/session-title-all-prompts-llm/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-title-all-prompts-llm/README.md README.md: cdda77f4bd92b927a156db093d690062ea10d531 -README.zh.md: 25af029ce1002ecffcd40e453213899d8f0b8f31 +README.zh.md: 8cf028ecc545f7ea2a494f0c9554d13e48b3adbf diff --git a/packages/session/session-title-all-prompts-llm/README.zh.md b/packages/session/session-title-all-prompts-llm/README.zh.md index 25af029ce1..8cf028ecc5 100644 --- a/packages/session/session-title-all-prompts-llm/README.zh.md +++ b/packages/session/session-title-all-prompts-llm/README.zh.md @@ -4,7 +4,7 @@ 可选的 `ctx.sessionTitle` 提供方,通过 `ctx.llm` 总结所有符合条件的用户消息。它注册 `all-prompts` 节奏,并在每条新用户提示词后启动新 revision,同时使用预置历史与子会话提示词。较新的 revision 会中止并取代旧工作;即使提供方忽略取消,也无法提交陈旧输出。 -该插件使用完整且必填的[共享 LLM(大语言模型)配置](../session-title-llm/README.md#configuration)。同时省略 `provider` 与 `model` 时,会继承每个当前已记录主请求的确切路由;也可以同时设置二者,使标题生成使用独立路由。如果最终封装的聚合提示词超过 `maxInputBytes`,请求会失败而不是截断历史;自动使用时会发出警告并保留先前标题。 +该插件使用完整且必填的[共享 LLM(大语言模型)配置](../session-title-llm/README.zh.md#configuration)。同时省略 `provider` 与 `model` 时,会继承每个当前已记录主请求的确切路由;也可以同时设置二者,使标题生成使用独立路由。如果最终封装的聚合提示词超过 `maxInputBytes`,请求会失败而不是截断历史;自动使用时会发出警告并保留先前标题。 ## 模型体验 diff --git a/packages/session/session-title-all-prompts-llm/package.json b/packages/session/session-title-all-prompts-llm/package.json index c165327beb..de26cc91db 100644 --- a/packages/session/session-title-all-prompts-llm/package.json +++ b/packages/session/session-title-all-prompts-llm/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-title-all-prompts-llm", "description": "All-user-messages LLM provider plugin for DeepSeek Harness session titles", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-title-first-prompt-llm/README.i18n.yaml b/packages/session/session-title-first-prompt-llm/README.i18n.yaml index cb3350a925..b5a4ed3952 100644 --- a/packages/session/session-title-first-prompt-llm/README.i18n.yaml +++ b/packages/session/session-title-first-prompt-llm/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-title-first-prompt-llm/README.md README.md: fef0651e1c4ba0032e6a1854590d5ac023bd71d8 -README.zh.md: 377779467a091b2c68d91a8be1903702a0808d10 +README.zh.md: 19998df3ab799fe77e18e207468bce92faf03ec8 diff --git a/packages/session/session-title-first-prompt-llm/README.zh.md b/packages/session/session-title-first-prompt-llm/README.zh.md index 377779467a..19998df3ab 100644 --- a/packages/session/session-title-first-prompt-llm/README.zh.md +++ b/packages/session/session-title-first-prompt-llm/README.zh.md @@ -4,7 +4,7 @@ 可选的 `ctx.sessionTitle` 提供方,通过 `ctx.llm` 总结第一条符合条件的用户消息。它注册 `first-prompt` 节奏,只在全新非 fork 会话首次创建回退时自动运行,并将结果归因于该消息的确切 seq。自动失败会保留回退,之后只能通过 `ctx.sessionTitle.refresh()` 重试。 -该插件使用完整且必填的[共享 LLM(大语言模型)配置](../session-title-llm/README.md#configuration)。同时省略 `provider` 与 `model` 时,会继承当前已记录主请求的确切路由;也可以同时设置二者,使标题生成使用独立路由。 +该插件使用完整且必填的[共享 LLM(大语言模型)配置](../session-title-llm/README.zh.md#configuration)。同时省略 `provider` 与 `model` 时,会继承当前已记录主请求的确切路由;也可以同时设置二者,使标题生成使用独立路由。 ## 模型体验 diff --git a/packages/session/session-title-first-prompt-llm/package.json b/packages/session/session-title-first-prompt-llm/package.json index 0e53d07e7f..86c3ebd34d 100644 --- a/packages/session/session-title-first-prompt-llm/package.json +++ b/packages/session/session-title-first-prompt-llm/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-title-first-prompt-llm", "description": "First-message LLM provider plugin for DeepSeek Harness session titles", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-title-llm/README.i18n.yaml b/packages/session/session-title-llm/README.i18n.yaml index 6d84c3559c..f3be01d147 100644 --- a/packages/session/session-title-llm/README.i18n.yaml +++ b/packages/session/session-title-llm/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-title-llm/README.md -README.md: db03205cabf9f723ee9bf6ad122026d75cb92113 -README.zh.md: f325cbd9344c15e91f3193204c27c40d2e3b62a6 +README.md: fd8d4b1166bd33ea9768c9abca74a219fb6725e5 +README.zh.md: 5508f110f1f8a9eb58a56b5b3efd7be4e8a25262 diff --git a/packages/session/session-title-llm/README.md b/packages/session/session-title-llm/README.md index db03205cab..fd8d4b1166 100644 --- a/packages/session/session-title-llm/README.md +++ b/packages/session/session-title-llm/README.md @@ -10,7 +10,7 @@ This package is a library, not a Cordis plugin. The provider plugins call `regis `provider` and `model` overrides are optional but must be supplied together as non-empty strings. Without that pair, the helper uses the exact provider/model route captured from the current session's logged `request/header`; an explicit refresh before any route exists therefore needs overrides. The helper measures the final JSON-framed user prompt, including seq fields, wrappers, and JSON escaping, against `maxInputBytes` before logging or dispatch instead of truncating it. Timeout and caller cancellation are rechecked while consuming the stream and after it completes, so a late successful result cannot be accepted even if an interceptor or adapter ignores abort. Malformed or empty output, tool calls, and non-stop finish reasons also reject; the session-title service decides whether that rejection is an automatic warning or an explicit caller failure. -After route and input validation, the helper appends a log-only `session/title-llm-request` event directly through `Session` before model dispatch. It contains the title-provider id, exact source seqs, route, system prompt, message list, and output-token cap used by the call. Persistence observes the record eagerly; the append does not need a title-specific marker, cast, settlement queue, or flush. The dispatched envelope is deep-frozen, carries `purpose: 'session-title'`, and deliberately lacks dsh-agent-loop's process-local request identity. Interceptors stay aligned with the record while loop-only reconstruction observers do not compare it with the conversation header. The DeepSeek adapter maps that purpose to thinking-disabled so the small output budget is reserved for visible title text; other adapters own their purpose-specific behavior. A later model failure leaves the request record intact; validation failures that never become dispatchable requests do not create one. The event stays outside derived model history. +After route and input validation, the helper records the exact dispatchable request as a log-only `session/title-llm-request` event before model dispatch. The event stays outside derived model history and remains recorded if the model call later fails; validation failures create no event. The request purpose disables thinking for DeepSeek title calls, while other adapters own their purpose-specific behavior. ## Configuration diff --git a/packages/session/session-title-llm/README.zh.md b/packages/session/session-title-llm/README.zh.md index f325cbd934..5508f110f1 100644 --- a/packages/session/session-title-llm/README.zh.md +++ b/packages/session/session-title-llm/README.zh.md @@ -10,7 +10,9 @@ `provider` 和 `model` 覆盖项都是可选的,但必须同时作为非空字符串提供。如果没有这一对取值,辅助模块会使用当前会话已记录 `request/header` 中捕获的确切提供方/模型路由;因此,在任何路由出现前显式刷新时必须提供覆盖项。辅助模块在记录或分发前,依据 `maxInputBytes` 检查最终 JSON 封装用户提示词的大小,包括 seq 字段、包装层与 JSON 转义,而不是将其截断。消费流期间和流完成后都会重新检查超时与调用方取消,因此即使拦截器或适配器忽略 abort,也不能接受迟到的成功结果。格式错误或空输出、工具调用和非 stop 结束原因同样会导致调用被拒绝;会话标题服务决定该拒绝属于自动警告还是显式调用方失败。 -路由与输入验证完成后,辅助模块会在模型分发前直接通过 `Session` 追加仅写入日志的 `session/title-llm-request` 事件。它包含标题提供方 id、确切来源 seq、路由、系统提示词、消息列表,以及该调用使用的输出 token 上限。持久化会立即观察到该记录;追加不需要标题专属标记、类型断言、结算队列或刷写。分发的请求封套会深度冻结,携带 `purpose: 'session-title'`,且有意不包含 dsh-agent-loop 的进程本地请求身份。拦截器会与记录保持一致,而循环专用重建观察者不会把它与对话请求头比较。DeepSeek 适配器会根据该用途禁用思考,使少量输出预算全部用于可见标题文本;其他适配器负责自身用途专用行为。后续模型失败会保留请求记录;从未成为可分发请求的验证失败不会创建记录。该事件始终位于派生模型历史之外。 +路由与输入校验完成后,辅助模块会在模型分发前将确切的可分发请求记录为仅写入日志的 `session/title-llm-request` 事件。该事件不进入派生模型历史;后续模型调用失败时仍会保留,而校验失败不会创建事件。请求用途会使 DeepSeek 标题调用禁用思考,其他适配器则负责自身的用途专用行为。 + + ## 配置 diff --git a/packages/session/session-title-llm/package.json b/packages/session/session-title-llm/package.json index 10d63c1d89..7c33b3fde6 100644 --- a/packages/session/session-title-llm/package.json +++ b/packages/session/session-title-llm/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-title-llm", "description": "Shared LLM generation policy for DeepSeek Harness session-title providers", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-title/README.i18n.yaml b/packages/session/session-title/README.i18n.yaml index bbe1cfc651..e28149f030 100644 --- a/packages/session/session-title/README.i18n.yaml +++ b/packages/session/session-title/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-title/README.md README.md: e923bd9700180214f235c4971d4b020565bfee43 -README.zh.md: 8e84ac27d9f1509c8530b838eeb5eb2216d87199 +README.zh.md: 6b1eef269dea65c66c1e9a70ce9469c66d192e84 diff --git a/packages/session/session-title/README.zh.md b/packages/session/session-title/README.zh.md index 8e84ac27d9..6b1eef269d 100644 --- a/packages/session/session-title/README.zh.md +++ b/packages/session/session-title/README.zh.md @@ -31,7 +31,7 @@ fork 出的会话会原样继承种子中的标题事件。首消息节奏不会 提供方会提供带品牌类型的稳定 id、自动模式(`first-prompt` 或 `all-prompts`)和 `generate(request)`。请求携带活跃会话、截至一次固定修订的所有符合条件消息、可用时当前已记录的主请求路由,以及取消信号。结果包含非空标题、该请求中互不重复且有序的来源消息 seq,以及生成该标题时使用的可选提供方/模型路由。服务会在结果持久保存前进行规范化和验证。 -参见[会话标题数据结构](../../../docs/subsystems/session-title.md)与[已实现决策](../../../.agents/notes/implemented/feature/2026-07-21-log-backed-session-titles.md)。 +参见[会话标题数据结构](../../../docs/subsystems/session-title.zh.md)与[已实现决策](../../../.agents/notes/implemented/feature/2026-07-21-log-backed-session-titles.zh.md)。 ## 模型体验 diff --git a/packages/session/session-title/package.json b/packages/session/session-title/package.json index 8a014863fe..57b3a03244 100644 --- a/packages/session/session-title/package.json +++ b/packages/session/session-title/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-session-title", "description": "Log-backed session title service and provider registry for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/session/session-title/src/index.ts b/packages/session/session-title/src/index.ts index 6ab14e7263..5ed0f1ca4d 100644 --- a/packages/session/session-title/src/index.ts +++ b/packages/session/session-title/src/index.ts @@ -306,12 +306,13 @@ export class SessionTitleService extends Service { // string clients list rows read. The unit child activates only when a // projection registry is composed (headless assemblies stay unaffected). ctx.inject(['sessionProjections'], (projectionCtx) => { + const titleSchema = zod.union([zod.string().min(1), zod.null()]) projectionCtx.sessionProjections.register<'title', string | null>({ key: 'title', - schema: zod.union([zod.string().min(1), zod.null()]), + stateSchema: titleSchema, init: () => null, apply: (state, event) => (event.type === 'session/title' ? event.data.title : state), - view: state => state, + wire: { viewSchema: titleSchema, view: state => state }, stateVersion: 1, }) }) diff --git a/packages/session/session-title/src/types.ts b/packages/session/session-title/src/types.ts index 76b27dc9b1..ad93926202 100644 --- a/packages/session/session-title/src/types.ts +++ b/packages/session/session-title/src/types.ts @@ -13,6 +13,9 @@ export {} declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + title: string | null + } interface SessionProjectionMap { /** * The session's current normalized title — the latest `session/title` diff --git a/packages/session/session-title/tests/session-title.spec.ts b/packages/session/session-title/tests/session-title.spec.ts index bb8ce4625d..2715db4fd4 100644 --- a/packages/session/session-title/tests/session-title.spec.ts +++ b/packages/session/session-title/tests/session-title.spec.ts @@ -72,11 +72,20 @@ describe('SessionTitleService', () => { expect(session.surface.nodes).toEqual([message.seq]) }) - it('derives a fallback title from the direct prompt instead of baked prefix context', async () => { + it('derives a fallback title from the direct prompt instead of injected context', async () => { const ctx = new Context() await ctx.plugin(SessionStore) await ctx.plugin(SessionTitleService, CONFIG) const session = ctx.sessions.create(SessionId('prefixed-title')) + session.append('user/message', createUserMessage({ + content: [{ type: 'text', text: 'Referenced session snapshot' }], + source: { + kind: 'session-reference', + form: 'recall', + version: 1, + references: [], + }, + }), { surfaceOp: 'append' }) session.append('turn/start', { turn: 1, }) diff --git a/packages/settings/README.i18n.yaml b/packages/settings/README.i18n.yaml index 0e9a8956bc..3523bfd4c7 100644 --- a/packages/settings/README.i18n.yaml +++ b/packages/settings/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/settings/README.md README.md: 9aad9391160ba6232d2de5005010ea99d8760abe -README.zh.md: 1ffc4f5184ac044d9605b256e9b57dcceecfa562 +README.zh.md: 30b9fab712dd2802e41f03fbf715b5f541519f8e diff --git a/packages/settings/README.zh.md b/packages/settings/README.zh.md index 1ffc4f5184..30b9fab712 100644 --- a/packages/settings/README.zh.md +++ b/packages/settings/README.zh.md @@ -6,7 +6,7 @@ | 包 | 职责 | ctx key | |---|---|---| -| [`settings/`](settings/README.md) | 定义命名空间注册、分层解析与提交 | `ctx.settings` | -| [`settings-file/`](settings-file/README.md) | 在本地文件中存储设置并观察外部编辑 | 注册到 `ctx.settings` | +| [`settings/`](settings/README.zh.md) | 定义命名空间注册、分层解析与提交 | `ctx.settings` | +| [`settings-file/`](settings-file/README.zh.md) | 在本地文件中存储设置并观察外部编辑 | 注册到 `ctx.settings` | -子系统参考——命名空间、owner scope、解析顺序、热提交——见 [docs/subsystems/settings.md](../../docs/subsystems/settings.md)。 +子系统参考——命名空间、owner scope、解析顺序、热提交——见 [docs/subsystems/settings.md](../../docs/subsystems/settings.zh.md)。 diff --git a/packages/settings/settings-file/package.json b/packages/settings/settings-file/package.json index 0cdbdbef35..3d0d2e460a 100644 --- a/packages/settings/settings-file/package.json +++ b/packages/settings/settings-file/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-settings-file", "description": "File-backed settings provider (settings.yaml) for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/settings/settings/package.json b/packages/settings/settings/package.json index 4208b479d4..5d3f0b8838 100644 --- a/packages/settings/settings/package.json +++ b/packages/settings/settings/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-settings", "description": "Abstract user-settings seam (ctx.settings) for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/README.i18n.yaml b/packages/shell/README.i18n.yaml index 65372e643b..3f17b799d3 100644 --- a/packages/shell/README.i18n.yaml +++ b/packages/shell/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/README.md README.md: f075db4fbe57c6052fea89b30e35126117f75a7c -README.zh.md: 2fdd92aac8d768024b6b4a9d6bd96bb6f5dafb05 +README.zh.md: 21425afbaf237887134ed29152e46ad699c9f639 diff --git a/packages/shell/README.zh.md b/packages/shell/README.zh.md index 2fdd92aac8..21425afbaf 100644 --- a/packages/shell/README.zh.md +++ b/packages/shell/README.zh.md @@ -6,14 +6,14 @@ | 包 | 职责 | ctx key | |---|---|---| -| [`shell/`](shell/README.md) | 定义 Service Provider 与 Consumer 共享的执行器约定。 | `ctx.shell` | -| [`bash-local/`](bash-local/README.md) | 通过本地 [`subprocess`](../subprocess/README.md) 服务执行命令。 | (注册 `ctx.shell`) | -| [`bash-sandbox/`](bash-sandbox/README.md) | 在本地执行前应用已配置的 [`sandbox`](../sandbox/README.md) 后端。 | (注册 `ctx.shell`) | -| [`pwsh-local/`](pwsh-local/README.md) | 采用 Windows 特有的进程行为执行 PowerShell 命令。 | (注册 `ctx.shell`) | -| [`shell-env/`](shell-env/README.md) | 提供 shell 工具共享的托管 `DSH_*` 环境。 | `ctx.shellEnv` | -| [`tool-bash/`](tool-bash/README.md) | 向模型公开 Bash 执行和后台任务集成。 | (注册到 `ctx.tools`) | -| [`tool-pwsh/`](tool-pwsh/README.md) | 向模型公开 PowerShell 执行。 | (注册到 `ctx.tools`) | +| [`shell/`](shell/README.zh.md) | 定义 Service Provider 与 Consumer 共享的执行器约定。 | `ctx.shell` | +| [`bash-local/`](bash-local/README.zh.md) | 通过本地 [`subprocess`](../subprocess/README.zh.md) 服务执行命令。 | (注册 `ctx.shell`) | +| [`bash-sandbox/`](bash-sandbox/README.zh.md) | 在本地执行前应用已配置的 [`sandbox`](../sandbox/README.zh.md) 后端。 | (注册 `ctx.shell`) | +| [`pwsh-local/`](pwsh-local/README.zh.md) | 采用 Windows 特有的进程行为执行 PowerShell 命令。 | (注册 `ctx.shell`) | +| [`shell-env/`](shell-env/README.zh.md) | 提供 shell 工具共享的托管 `DSH_*` 环境。 | `ctx.shellEnv` | +| [`tool-bash/`](tool-bash/README.zh.md) | 向模型公开 Bash 执行和后台任务集成。 | (注册到 `ctx.tools`) | +| [`tool-pwsh/`](tool-pwsh/README.zh.md) | 向模型公开 PowerShell 执行。 | (注册到 `ctx.tools`) | 叶节点 `cordis.yml` 选择一个执行器实现和所需的面向模型工具。沙箱化组合还会选择一个 `ctx.sandbox` 提供方;[ACP(Agent Client Protocol)示例](../../examples/acp-agent/)展示一套完整接线。 -子系统参考——请求/spec 词汇、结果、后台进程、服务与事件——见 [docs/subsystems/shell.md](../../docs/subsystems/shell.md)。 +子系统参考——请求/spec 词汇、结果、后台进程、服务与事件——见 [docs/subsystems/shell.md](../../docs/subsystems/shell.zh.md)。 diff --git a/packages/shell/bash-local/README.i18n.yaml b/packages/shell/bash-local/README.i18n.yaml index 9e8a0eaf85..82a7387566 100644 --- a/packages/shell/bash-local/README.i18n.yaml +++ b/packages/shell/bash-local/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/bash-local/README.md README.md: 5e62ea24676f3bedf32b1b48f8618d703d5fb462 -README.zh.md: 6f236f3dd78bd444eb57443ba068a0271ac5d8df +README.zh.md: 92f842a777aebec2e2cb6a8c54966e46202531f1 diff --git a/packages/shell/bash-local/README.zh.md b/packages/shell/bash-local/README.zh.md index 6f236f3dd7..92f842a777 100644 --- a/packages/shell/bash-local/README.zh.md +++ b/packages/shell/bash-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -`@deepseek-ai/dsh-shell` 执行器 seam 的本地 Service Provider,构建在 [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) 服务之上:`LocalBashExecutor` 每次调用都通过 `ctx.subprocess` 把 `bash -c ` 作为受管进程组 spawn,并负责所有 Bash 层职责(命令默认值补全与上限、超时与取消分类、适合模型的终端环境,以及后台读取时面向模型的 stdout/stderr 合并)。以 spill 文件兜底的有界输出、凭据清除、kill 升级和 dispose(资源释放)等进程组机制则由 subprocess 服务负责。 +`@deepseek-ai/dsh-shell` 执行器 seam 的本地 Service Provider,构建在 [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.zh.md) 服务之上:`LocalBashExecutor` 每次调用都通过 `ctx.subprocess` 把 `bash -c ` 作为受管进程组 spawn,并负责所有 Bash 层职责(命令默认值补全与上限、超时与取消分类、适合模型的终端环境,以及后台读取时面向模型的 stdout/stderr 合并)。以 spill 文件兜底的有界输出、凭据清除、kill 升级和 dispose(资源释放)等进程组机制则由 subprocess 服务负责。 包根目录导出默认与具名的 `LocalBashExecutor` 插件及其 `Config`。 @@ -23,11 +23,11 @@ ## 行为 - **每次调用都 spawn,不保留 shell 状态**:每次调用都启动新的非登录 `bash -c`,且不读取 rc 文件。 -- **组装条目是一层,而不是最终值**:当组装中存在 settings 提供方时,本执行器以上面的条目为 base 注册该能力的 [`bash` 命名空间](../shell/README.md),因此 `settings.yaml` 中的用户段会叠加其上,下一条命令即按新预算运行。schema 无法判定的值(正有限、`graceMs` 的定时器上界)会在写入时被拒绝,运行中的执行器保持它最后一份可用的段;没有提供方、或提供方脱离之后,运行的就是组装条目。 -- **在受管进程组之上应用配置预算**:`resolve()` 从配置补全 `workdir`/`timeoutMs`/`stdoutMaxBytes`,每次 spawn 都向服务传入显式的字节上限、spill 上限与 `graceMs`。该宽限期须为正有限值,且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md),这样 Node 就能用一个定时器表示它。进程组终止、退出后管道排空、尾部保留与有界 spill 文件是 [`dsh-subprocess-local`](../../subprocess/subprocess-local/README.md) 的机制。前台 `ShellExecRequest.stdoutMaxBytes` 可为某个受信任调用方提高单次 stdout 捕获预算;stderr 和后台运行仍使用 `maxOutputBytes`。 -- **超时与取消分类**:`run()` 通过同一个 deadline 把经配置钳位的超时与调用方的信号融合;只有执行器自身的超时报告 `timedOut`,上游取消报告 `aborted`,自身因信号终止的命令两者皆不报告(见[超时库 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md))。 -- **适合模型的终端环境**:`NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat` 防止分页器与 ANSI 颜色破坏结果。这些值作为普通 env 合并,遵循服务的凭据清除与 `DSH_*` 通道规则;调用方的显式条目依旧优先。详见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.md) 与 [受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 -- **后台进程**:`start()` 会立即返回活动的 `ShellProcess` 句柄且不应用超时;`readOutput()` 把基于偏移量的 stdout/stderr 读取合并为一条消费式增量,并在存在 stderr 时将其置于 `[stderr]` 标记下。运行中的进程属于 subprocess 服务,可在执行器重载后存活,并在服务 dispose 时被终止且等待退出。job id、所有权、轮询和通知属于通用 [`ctx.jobs` 运行时](../../jobs/jobs/README.md),工具层会在其中注册该句柄。 +- **组装条目是一层,而不是最终值**:当组装中存在 settings 提供方时,本执行器以上面的条目为 base 注册该能力的 [`bash` 命名空间](../shell/README.zh.md),因此 `settings.yaml` 中的用户段会叠加其上,下一条命令即按新预算运行。schema 无法判定的值(正有限、`graceMs` 的定时器上界)会在写入时被拒绝,运行中的执行器保持它最后一份可用的段;没有提供方、或提供方脱离之后,运行的就是组装条目。 +- **在受管进程组之上应用配置预算**:`resolve()` 从配置补全 `workdir`/`timeoutMs`/`stdoutMaxBytes`,每次 spawn 都向服务传入显式的字节上限、spill 上限与 `graceMs`。该宽限期须为正有限值,且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.zh.md),这样 Node 就能用一个定时器表示它。进程组终止、退出后管道排空、尾部保留与有界 spill 文件是 [`dsh-subprocess-local`](../../subprocess/subprocess-local/README.zh.md) 的机制。前台 `ShellExecRequest.stdoutMaxBytes` 可为某个受信任调用方提高单次 stdout 捕获预算;stderr 和后台运行仍使用 `maxOutputBytes`。 +- **超时与取消分类**:`run()` 通过同一个 deadline 把经配置钳位的超时与调用方的信号融合;只有执行器自身的超时报告 `timedOut`,上游取消报告 `aborted`,自身因信号终止的命令两者皆不报告(见[超时库 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.zh.md))。 +- **适合模型的终端环境**:`NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat` 防止分页器与 ANSI 颜色破坏结果。这些值作为普通 env 合并,遵循服务的凭据清除与 `DSH_*` 通道规则;调用方的显式条目依旧优先。详见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md) 与 [受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.zh.md)。 +- **后台进程**:`start()` 会立即返回活动的 `ShellProcess` 句柄且不应用超时;`readOutput()` 把基于偏移量的 stdout/stderr 读取合并为一条消费式增量,并在存在 stderr 时将其置于 `[stderr]` 标记下。运行中的进程属于 subprocess 服务,可在执行器重载后存活,并在服务 dispose 时被终止且等待退出。job id、所有权、轮询和通知属于通用 [`ctx.jobs` 运行时](../../jobs/jobs/README.zh.md),工具层会在其中注册该句柄。 ## 模型体验 @@ -39,9 +39,9 @@ ## 已知限制与暂缓事项 -- **自身不提供隔离**:此执行器始终以 harness 进程的权限运行命令;需要隔离的部署可以组合 [`dsh-bash-sandbox`](../bash-sandbox/README.md),每次调用的 allow/deny/ask 策略则属于 `tools/pre-execute`。 +- **自身不提供隔离**:此执行器始终以 harness 进程的权限运行命令;需要隔离的部署可以组合 [`dsh-bash-sandbox`](../bash-sandbox/README.zh.md),每次调用的 allow/deny/ask 策略则属于 `tools/pre-execute`。 - **没有持久 shell 或 PTY**:每次调用都启动新的非登录 `bash -c`;仅持久化 cwd 与交互式终端会话均继续暂缓,直到真实工作流需要它们。 - **仅支持 POSIX**:`bash` 二进制已硬编码,底层服务的进程组语义也是 POSIX 的;不支持 Windows。 - **后台 spawn 失败提示只交付一次**:subprocess 服务不会为从未真正运行的进程缓冲任何输出,因此执行器把 `spawn failed: …` 注入恰好一个 `readOutput()` 增量;丢弃了该增量的读取方无法再恢复它。 -凭据清除启发式规则与 spill 保留的注意事项随 [`dsh-subprocess-local`](../../subprocess/subprocess-local/README.md) 记录;这些机制归它所有。 +凭据清除启发式规则与 spill 保留的注意事项随 [`dsh-subprocess-local`](../../subprocess/subprocess-local/README.zh.md) 记录;这些机制归它所有。 diff --git a/packages/shell/bash-local/package.json b/packages/shell/bash-local/package.json index db1c919311..f26125d911 100644 --- a/packages/shell/bash-local/package.json +++ b/packages/shell/bash-local/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-bash-local", "description": "Local-subprocess implementation of the DeepSeek Harness bash executor seam", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/bash-sandbox/README.i18n.yaml b/packages/shell/bash-sandbox/README.i18n.yaml index 6792ffa955..ce3625a642 100644 --- a/packages/shell/bash-sandbox/README.i18n.yaml +++ b/packages/shell/bash-sandbox/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/bash-sandbox/README.md -README.md: fe9a0b1a891bd2b9102357f48141822c70465ddf -README.zh.md: 648264615dc476f9442b9bb9ae3a332196f4e104 +README.md: dd0f6dd80394ee8bf0349f706e8fd77cd3c14189 +README.zh.md: 00b146ea88ffb449c7eaca1de6185657592fe774 diff --git a/packages/shell/bash-sandbox/README.md b/packages/shell/bash-sandbox/README.md index fe9a0b1a89..dd0f6dd803 100644 --- a/packages/shell/bash-sandbox/README.md +++ b/packages/shell/bash-sandbox/README.md @@ -19,7 +19,7 @@ Semantics: - **Denials are result facts.** A failed run whose stderr carries the selected backend's own denial dialect — the signatures the provider stamps on every wrap (EROFS text under bwrap, EACCES under Landlock, EPERM under Seatbelt) — is reported as `ShellRunResult.sandbox.denied: true` (conservative classification, read from the collected stderr tail); every CONFINED run also carries the mode it executed under (`result.sandbox.mode`) and the provider's enforcement completeness (`result.sandbox.enforcement`: `full`, or `partial` on an older Landlock ABI). - **The runner path or syscall must match.** Before a process starts, a rejection is attributed to the runner only when the caller-owned workdir is independently usable and Node reports `ENOENT` or `EACCES` with either an `error.path` equal to provider argv[0] or, when `error.path` is absent, an exact `syscall: 'spawn '`. A present path also requires `syscall: 'spawn'` or the exact `spawn `. This covers a missing runner, a non-executable runner, or an executable script whose shebang interpreter is unavailable. A bare `syscall: 'spawn'` without an exact error path, any other code, an invalid or unusable workdir, a resource failure, an unrelated syscall, or an unstructured rejection retains the local executor's command-start failure semantics. Foreground execution throws `SANDBOX_UNAVAILABLE` with the original spawn detail, while asynchronous background settlement stamps `runnerFailed: true` and `denied: false`. If a `SubprocessRuntime` synchronously throws the same runner-identifying `ENOENT`/`EACCES` shape, background start throws `SANDBOX_UNAVAILABLE`; other synchronous errors propagate unchanged. After a process starts, a rule's optional exit-code check and a remaining fatal stderr line must both match after exact informational-line exclusions. A match takes priority over denial; foreground execution throws `SANDBOX_UNAVAILABLE` with the matched fatal line, while a settled background process stamps `process.sandbox.runnerFailed`, which the bash producer renders through generic `job_output`. Confined background handles retain their mode/enforcement facts and release per-process accounting in either path. - **Deployment fallback, per-call policy.** [`ctx.sandboxPolicy`](../../sandbox/sandbox-policy/) resolves a complete `SandboxExecutionPolicy` for every tool call: the calling session supplies its mode override and immutable cwd root, while deployment config supplies the fallbacks for agentless calls. An approved escalation changes only that policy's mode; its session root stays attached. `resolve()` carries the policy onto the spec, so overlapping commands from different projects run, classify, and report under their own roots and modes. The capability fact `ctx.shell.sandboxMode` reports the configured default so the tool layer advertises escalation only when this executor is mounted; the static bash tool description separately owns denial and escalation guidance. -- **File effects only.** Network and process visibility are deliberately not restricted — the mode vocabulary does not pretend to cover what the backend does not enforce. +- **File effects only.** The mode vocabulary claims only file effects. Network stays unrestricted; process visibility is backend-specific and documented by [`dsh-sandbox-local`](../../sandbox/sandbox-local/). - Process mechanics (spawn, process-group kills, output collection/spill, background handles, credential scrub) are inherited from [`dsh-bash-local`](../bash-local/); runner selection lives in [`dsh-sandbox-local`](../../sandbox/sandbox-local/). Deny-only at the seam: a denial is a reported fact, and this executor never negotiates permissions itself — the approval question lives in the tool layer (`dsh-tool-bash`), which drives the override this package honors. @@ -82,7 +82,7 @@ Append-only; newly visible content follows the reusable request prefix and does ## Known Limitations and Deferred Work -- **Confinement covers file effects only** — network access and process visibility are unchanged, so the modes are not a general-purpose security sandbox. +- **Confinement covers file effects only** — network restriction and a uniform process-visibility guarantee are absent, so the modes are not a general-purpose security sandbox. - **Denials are inferred from failed-command stderr** — backend signatures make the inference portable, but a matching application error can be classified as a denial and a denial omitted from the retained tail can be missed. - **An asynchronously observed background runner failure has no immediate error channel** — it is recorded on the settled process and surfaces when the caller reads the generic task with `job_output`; a synchronous `SubprocessRuntime` throw that names the runner path instead fails `start()` immediately. - **`danger-full-access` deliberately bypasses `ctx.sandbox`** — it is an explicit unconfined mode, not a wider sandbox profile. diff --git a/packages/shell/bash-sandbox/README.zh.md b/packages/shell/bash-sandbox/README.zh.md index 648264615d..00b146ea88 100644 --- a/packages/shell/bash-sandbox/README.zh.md +++ b/packages/shell/bash-sandbox/README.zh.md @@ -19,7 +19,7 @@ - **拒绝是结果事实。** 如果一次失败运行的 stderr 包含所选后端自身的拒绝方言,即提供方在每次包装时加上的特征(bwrap 下的 EROFS 文本、Landlock 下的 EACCES、Seatbelt 下的 EPERM),则结果报告 `ShellRunResult.sandbox.denied: true`(从已收集的 stderr 尾部进行保守分类)。每次受限制运行还会携带执行时模式(`result.sandbox.mode`)与提供方强制执行完整性(`result.sandbox.enforcement`:`full`,或在较旧 Landlock ABI 上为 `partial`)。 - **Runner 路径或 syscall 必须匹配。** 进程启动前,调用方拥有的 workdir 必须经独立验证可用,Node 必须报告 `ENOENT` 或 `EACCES`,并且错误必须符合以下一种形态:`error.path` 等于提供方返回的 `argv[0]`,同时 `syscall` 为 `'spawn'` 或精确的 `'spawn '`;或者 `error.path` 不存在,同时 `syscall` 为精确的 `'spawn '`。这样可以识别缺失的 runner、不可执行的 runner,或 shebang 解释器不可用的可执行脚本。没有精确错误路径的裸 `syscall: 'spawn'`、任何其他错误码、无效或不可用的 workdir、资源失败、无关 syscall 或无结构拒绝仍保留本地执行器的命令启动失败语义。前台执行会抛出 `SANDBOX_UNAVAILABLE` 并附带原始 spawn 错误详情,异步后台结算则会标记 `runnerFailed: true` 和 `denied: false`。如果 `SubprocessRuntime` 同步抛出同样能指明 runner 的 `ENOENT`/`EACCES` 形态,后台启动会抛出 `SANDBOX_UNAVAILABLE`;其他同步错误原样传播。进程启动后,先按整行精确匹配排除信息性行,随后规则的可选退出码检查和余下 stderr 中的一行致命诊断必须同时匹配。匹配结果优先于拒绝;前台执行会抛出 `SANDBOX_UNAVAILABLE` 并附带匹配到的致命行,已结算的后台进程则会标记 `process.sandbox.runnerFailed`,Bash 结果生成方通过通用 `job_output` 渲染它。无论走哪条路径,受限制的后台句柄都会保留自身的模式/强制执行事实,并释放每进程计数。 - **部署回退,每次调用策略。** [`ctx.sandboxPolicy`](../../sandbox/sandbox-policy/) 为每次工具调用解析完整的 `SandboxExecutionPolicy`:调用会话提供自身的模式覆盖与不可变 cwd 根目录,部署配置则为无 agent(智能体)调用提供回退。已批准的升权只更改该策略的模式,会话根目录仍然附着其上。`resolve()` 把策略带入 spec,因此来自不同项目的重叠命令会在各自的根目录与模式下运行、分类和报告。能力事实 `ctx.shell.sandboxMode` 报告已配置的默认值,因此工具层只在装载该执行器时才公布升权;静态 bash 工具描述则单独负责拒绝与升权引导。 -- **只限制文件影响。** 设计上不限制网络与进程可见性:模式词汇不会声称覆盖后端未强制执行的范围。 +- **只限制文件影响。** 模式词汇只声称文件影响。网络仍不受限制;进程可见性因后端而异,具体见 [`dsh-sandbox-local`](../../sandbox/sandbox-local/)。 - 进程机制(spawn、进程组终止、输出收集/spill、后台句柄、凭证清理)继承自 [`dsh-bash-local`](../bash-local/);runner 选择位于 [`dsh-sandbox-local`](../../sandbox/sandbox-local/)。 该 seam 只报告拒绝:拒绝是一项结果事实,本执行器绝不自行协商权限。批准问题位于工具层(`dsh-tool-bash`),由它设置本包所遵守的模式覆盖值。 @@ -42,7 +42,7 @@ #### 模型看到的内容 -基线是生成的 [`dsh-tool-bash` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-bash)。通过公布表明启用隔离的 `sandboxMode` 能力,此后端会为 `bash` 增加 `sandbox_permissions`,其 enum 为 `workspace-write` | `danger-full-access`,并增加 `justification`。策略归属方会另行贡献当前且不区分具体能力的 `sandbox:policy` 上下文。 +基线是生成的 [`dsh-tool-bash` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-bash)。通过公布表明启用隔离的 `sandboxMode` 能力,此后端会为 `bash` 增加 `sandbox_permissions`,其 enum 为 `workspace-write` | `danger-full-access`,并增加 `justification`。策略归属方会另行贡献当前且不区分具体能力的 `sandbox:policy` 上下文。 #### Token 影响 @@ -70,7 +70,7 @@ #### 模型看到的内容 -如果没有 runner 能强制执行受限模式,前台调用会传播 [`SANDBOX_UNAVAILABLE` 错误](../../sandbox/sandbox/README.md#confinement-error-indirectly);该错误由 `dsh-sandbox` 定义。判定为 runner 失败的 spawn 错误会以原始 spawn 错误作为详细信息;如果拒绝没有通过 `ENOENT`/`EACCES` 的 `path` 或 `syscall` 证据指明 `argv[0]`,它仍是普通的命令启动错误。已结算的 runner 失败则以匹配到的致命 stderr 行作为详细信息,并保留原始 stderr 收集结果。如果追加了 `Runner failure: `,它就是权威诊断;前面的后端安装文本只是通用的 `SANDBOX_UNAVAILABLE` 前缀。 +如果没有 runner 能强制执行受限模式,前台调用会传播 [`SANDBOX_UNAVAILABLE` 错误](../../sandbox/sandbox/README.zh.md#confinement-error-indirectly);该错误由 `dsh-sandbox` 定义。判定为 runner 失败的 spawn 错误会以原始 spawn 错误作为详细信息;如果拒绝没有通过 `ENOENT`/`EACCES` 的 `path` 或 `syscall` 证据指明 `argv[0]`,它仍是普通的命令启动错误。已结算的 runner 失败则以匹配到的致命 stderr 行作为详细信息,并保留原始 stderr 收集结果。如果追加了 `Runner failure: `,它就是权威诊断;前面的后端安装文本只是通用的 `SANDBOX_UNAVAILABLE` 前缀。 #### Token 影响 @@ -82,7 +82,7 @@ ## 已知限制与暂缓事项 -- **限制只覆盖文件影响**:网络访问与进程可见性不变,因此这些模式不是通用安全沙箱。 +- **限制只覆盖文件影响**:不提供网络限制和统一的进程可见性保证,因此这些模式不是通用安全沙箱。 - **拒绝从失败命令的 stderr 推断**:后端特征使该推断可跨平台使用,但包含相同后端特征的应用错误可能被分类为拒绝,也可能遗漏未出现在保留尾部中的拒绝。 - **异步观测到的后台 runner 失败没有即时错误通道**:它记录在已结算进程上,并在调用方使用 `job_output` 读取通用任务时呈现;`SubprocessRuntime` 同步抛出的错误包含 runner 路径时,则会使 `start()` 立即失败。 - **`danger-full-access` 有意绕过 `ctx.sandbox`**:它是显式无约束模式,不是更宽的沙箱 profile。 diff --git a/packages/shell/bash-sandbox/package.json b/packages/shell/bash-sandbox/package.json index bb68dc855b..4464609f97 100644 --- a/packages/shell/bash-sandbox/package.json +++ b/packages/shell/bash-sandbox/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-bash-sandbox", "description": "Sandbox-consuming implementation of the DeepSeek Harness bash executor seam (confines every command via ctx.sandbox, reports denial/enforcement result facts)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/pwsh-local/README.i18n.yaml b/packages/shell/pwsh-local/README.i18n.yaml index c9ce9f1d6b..a7712c1df6 100644 --- a/packages/shell/pwsh-local/README.i18n.yaml +++ b/packages/shell/pwsh-local/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/pwsh-local/README.md README.md: 2eccc59b919d1f729eef52a42581f4da0f1d9e60 -README.zh.md: a03f80343711a1471bc96017e23476a3253ec46c +README.zh.md: b7072ebd9be4e148cedd4edd6c20b12536814856 diff --git a/packages/shell/pwsh-local/README.zh.md b/packages/shell/pwsh-local/README.zh.md index a03f803437..b7072ebd9b 100644 --- a/packages/shell/pwsh-local/README.zh.md +++ b/packages/shell/pwsh-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -`@deepseek-ai/dsh-shell` 执行器 seam 的本地 PowerShell Service Provider,基于 [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) 服务:`PwshLocalExecutor` 每次调用以受管进程的方式通过 `ctx.subprocess` spawn `pwsh -NoLogo -NoProfile -NonInteractive -Command `,并负责所有 PowerShell 相关事项——可执行文件解析、命令默认化与上限、超时/取消分类、面向模型的终端环境,以及后台读取的 stdout/stderr 合并。进程组机制(有界 spill 输出、凭据清理、终止升级、dispose(资源释放))属于 subprocess 服务。 +`@deepseek-ai/dsh-shell` 执行器 seam 的本地 PowerShell Service Provider,基于 [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.zh.md) 服务:`PwshLocalExecutor` 每次调用以受管进程的方式通过 `ctx.subprocess` spawn `pwsh -NoLogo -NoProfile -NonInteractive -Command `,并负责所有 PowerShell 相关事项——可执行文件解析、命令默认化与上限、超时/取消分类、面向模型的终端环境,以及后台读取的 stdout/stderr 合并。进程组机制(有界 spill 输出、凭据清理、终止升级、dispose(资源释放))属于 subprocess 服务。 命令字符串作为单个 argv 元素传给 `-Command`:由 PowerShell 自己解析文本,不存在中间 shell,因此没有需要转义的 shell 引号层(这里不存在与 `bash -c` 字符串域对应的层)。原生 Win32 路径(`C:\...`)原样通过。 @@ -28,13 +28,13 @@ 这是 `dsh-bash-local` 的 Windows 对应实现,有意逐次调用保持语义一致: - **每次调用新建进程,无 shell 状态**——每次调用都是全新的非交互 `pwsh -Command`(确定性;不加载 profile 文件)。`-NoLogo -NoProfile -NonInteractive` 关闭启动横幅、profile 加载与会干扰工具输出的提示符。 -- **组装条目是一层,而不是最终值**——当组装中存在 settings 提供方时,本执行器以上面的条目为 base 注册该能力的 [`bash` 命名空间](../shell/README.md),因此 `settings.yaml` 中的用户段会叠加其上,下一条命令即按新预算运行。该命名空间与 POSIX 家族共用,因为一个宿主只组装一个 `ctx.shell` 提供方;在任一平台写下的文档在另一平台仍能解析。schema 无法判定的值(正有限、`graceMs` 的定时器上界)会在写入时被拒绝,运行中的执行器保持它最后一份可用的段。 +- **组装条目是一层,而不是最终值**——当组装中存在 settings 提供方时,本执行器以上面的条目为 base 注册该能力的 [`bash` 命名空间](../shell/README.zh.md),因此 `settings.yaml` 中的用户段会叠加其上,下一条命令即按新预算运行。该命名空间与 POSIX 家族共用,因为一个宿主只组装一个 `ctx.shell` 提供方;在任一平台写下的文档在另一平台仍能解析。schema 无法判定的值(正有限、`graceMs` 的定时器上界)会在写入时被拒绝,运行中的执行器保持它最后一份可用的段。 - **UTF-8 输出固定**——每条命令都先以 UTF-8 设置 `[Console]::OutputEncoding` 与 `$OutputEncoding`,因此 Windows PowerShell 5.1 兜底(或任何控制台代码页非 UTF-8 的主机)不会破坏非 ASCII 输出:subprocess 收集器以 UTF-8 解码字节。输入编码保持宿主默认;pwsh 7 默认为 UTF-8,不受影响。 - **可执行文件解析**——`resolvePwshPath` 优先显式 `pwshPath`,然后在 Windows 上依次探测 PowerShell 7 安装位置、每个 PATH 条目(Microsoft Store 安装;剥离两端引号)以及作为遗留兜底的 Windows PowerShell 5.1,逐一用 lstat 探测检查(接受真实文件或链接形态的重解析点:Store 的 app execution alias 对其目标 stat 会因 ACL 失败,但 lstat 能看到别名本身);其他平台回退为通过 PATH 解析的裸 `pwsh`。解析是 `(configured, env, platform)` 的纯函数;它在构造时执行,此后仅当存储的 `pwshPath` 与当前可执行文件所依据的值不同才再次执行,因此无关的设置变更绝不会重新探测文件系统。 -- **受管进程组之上的配置预算**——`resolve()` 从配置填充 `workdir`/`timeoutMs`/`stdoutMaxBytes`,每次 spawn 都向服务提供显式字节上限、spill 上限与 `graceMs`。该宽限期须为正有限值,且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md),这样 Node 就能用一个定时器表示它。进程树终止(Windows 用 taskkill,POSIX 用进程组信号)、退出后管道排空宽限、保尾截断与有界 spill 文件是 [`dsh-subprocess-local`](../../subprocess/subprocess-local/README.md) 的机制。前台 `ShellExecRequest.stdoutMaxBytes` 可为单个受信调用方提高 stdout 捕获预算;stderr 与后台运行仍使用 `maxOutputBytes`。 -- **超时与取消分类**——`run()` 通过一个 deadline 融合按配置上限截取的超时与调用方信号;只有执行器自身超时报告 `timedOut`,上游取消报告 `aborted`,自我终止的命令两者都不报告(见 [timeout 库 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md))。Windows 将强制终止报告为退出码 1 且无信号,因此带信号标记的事实(`signal`、`killed` 状态)在那里仅限 POSIX;超时/取消分类与平台无关。 +- **受管进程组之上的配置预算**——`resolve()` 从配置填充 `workdir`/`timeoutMs`/`stdoutMaxBytes`,每次 spawn 都向服务提供显式字节上限、spill 上限与 `graceMs`。该宽限期须为正有限值,且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.zh.md),这样 Node 就能用一个定时器表示它。进程树终止(Windows 用 taskkill,POSIX 用进程组信号)、退出后管道排空宽限、保尾截断与有界 spill 文件是 [`dsh-subprocess-local`](../../subprocess/subprocess-local/README.zh.md) 的机制。前台 `ShellExecRequest.stdoutMaxBytes` 可为单个受信调用方提高 stdout 捕获预算;stderr 与后台运行仍使用 `maxOutputBytes`。 +- **超时与取消分类**——`run()` 通过一个 deadline 融合按配置上限截取的超时与调用方信号;只有执行器自身超时报告 `timedOut`,上游取消报告 `aborted`,自我终止的命令两者都不报告(见 [timeout 库 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.zh.md))。Windows 将强制终止报告为退出码 1 且无信号,因此带信号标记的事实(`signal`、`killed` 状态)在那里仅限 POSIX;超时/取消分类与平台无关。 - **面向模型的终端环境**——`NO_COLOR=1 PAGER=cat GIT_PAGER=cat`(没有 `TERM=dumb`:那是 POSIX 概念;现代 PowerShell 渲染器遵循 `NO_COLOR`),作为普通 env 在服务的凭据清理与 `DSH_*` 通道规则之下合并;显式调用方条目仍然优先。 -- **后台进程**——`start()` 立即返回存活的 `ShellProcess` 句柄,不设超时;句柄的 `readOutput()` 把服务基于偏移的 stdout/stderr 读取合并为一条按分段标记、通过消费游标推进的增量。仍在运行的进程属于 subprocess 服务,因此它跨执行器重载存活,并随服务 dispose(被终止并 join)。一切任务相关职责(job id、所有权、轮询、通知)都在通用 [`ctx.jobs` 运行时](../../jobs/jobs/README.md) 中,由工具层把句柄注册进去——本执行器从不接触会话或注册表。 +- **后台进程**——`start()` 立即返回存活的 `ShellProcess` 句柄,不设超时;句柄的 `readOutput()` 把服务基于偏移的 stdout/stderr 读取合并为一条按分段标记、通过消费游标推进的增量。仍在运行的进程属于 subprocess 服务,因此它跨执行器重载存活,并随服务 dispose(被终止并 join)。一切任务相关职责(job id、所有权、轮询、通知)都在通用 [`ctx.jobs` 运行时](../../jobs/jobs/README.zh.md) 中,由工具层把句柄注册进去——本执行器从不接触会话或注册表。 ## 模型体验 @@ -54,4 +54,4 @@ - **编码 preamble 位于命令之前**——PowerShell 要求 `param(...)`、`#requires` 与 `using namespace`/`using assembly` 语句位于脚本最顶部,因此以其中一种开头的命令无法在 UTF-8 输出 preamble 下运行。`param(...)` 脚本可包进 `& { … }`(param 块可以合法地位于脚本块开头);`using` 语句与 `#requires` 在命令内没有变通办法(`#requires` 在 `-Command` 中无论位置如何都不生效)——此类脚本请改从文件运行。 - **Windows PowerShell 5.1 下的非 ASCII stdin 可能被错误解码**——preamble 只固定输出编码;`[Console]::InputEncoding` 保持主机默认,因为在重定向 stdin 下设置它会抛出异常。pwsh 7 默认 UTF-8,不受影响。 -清理启发式与 spill 保留的注意事项见 [`dsh-subprocess-local`](../../subprocess/subprocess-local/README.md),相关机制由其负责。 +清理启发式与 spill 保留的注意事项见 [`dsh-subprocess-local`](../../subprocess/subprocess-local/README.zh.md),相关机制由其负责。 diff --git a/packages/shell/pwsh-local/package.json b/packages/shell/pwsh-local/package.json index f6882f5c3e..f76007bd96 100644 --- a/packages/shell/pwsh-local/package.json +++ b/packages/shell/pwsh-local/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-pwsh-local", "description": "Local PowerShell implementation of the DeepSeek Harness bash executor seam", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/pwsh-sandbox/README.i18n.yaml b/packages/shell/pwsh-sandbox/README.i18n.yaml index 27d1e0d7c6..3a64425236 100644 --- a/packages/shell/pwsh-sandbox/README.i18n.yaml +++ b/packages/shell/pwsh-sandbox/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/pwsh-sandbox/README.md README.md: 533b89b533084843a17153f3f3da39f21d271b04 -README.zh.md: 370cc6fd4a3961d6f79c5c8829ac5c42e093c2a5 +README.zh.md: 8607554d10562bc519f379030ccd9518cb63b3b6 diff --git a/packages/shell/pwsh-sandbox/README.zh.md b/packages/shell/pwsh-sandbox/README.zh.md index 370cc6fd4a..8607554d10 100644 --- a/packages/shell/pwsh-sandbox/README.zh.md +++ b/packages/shell/pwsh-sandbox/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -沙盒消费型的 [`ctx.shell` 执行器 seam](../shell/) 的 PowerShell 实现:每条命令以 `pwsh -NoLogo -NoProfile -NonInteractive -Command ` 运行,**经 `ctx.sandbox` 隔离**,选定模式、强制完整性、拒绝事实都盖在每次结算的结果上。它是 [`@deepseek-ai/dsh-bash-sandbox`](../bash-sandbox/) 的 pwsh 孪生,按 [pwsh 执行器与工具决策](../../../.agents/notes/implemented/feature/2026-08-01-pwsh-tool-and-executor.md) 逐调用镜像——隔离实体本身是平台无关的:Windows 上沙盒 seam 解析到 ACL 受限令牌 runner 链([`@deepseek-ai/dsh-sandbox-windows-acl`](../../sandbox/sandbox-windows-acl/)),Linux/macOS 上解析到 bwrap/Landlock/Seatbelt。 +沙盒消费型的 [`ctx.shell` 执行器 seam](../shell/) 的 PowerShell 实现:每条命令以 `pwsh -NoLogo -NoProfile -NonInteractive -Command ` 运行,**经 `ctx.sandbox` 隔离**,选定模式、强制完整性、拒绝事实都盖在每次结算的结果上。它是 [`@deepseek-ai/dsh-bash-sandbox`](../bash-sandbox/) 的 pwsh 孪生,按 [pwsh 执行器与工具决策](../../../.agents/notes/implemented/feature/2026-08-01-pwsh-tool-and-executor.zh.md) 逐调用镜像——隔离实体本身是平台无关的:Windows 上沙盒 seam 解析到 ACL 受限令牌 runner 链([`@deepseek-ai/dsh-sandbox-windows-acl`](../../sandbox/sandbox-windows-acl/)),Linux/macOS 上解析到 bwrap/Landlock/Seatbelt。 执行器继承 [`@deepseek-ai/dsh-pwsh-local`](../pwsh-local/) 的进程机制,并消费其 argv 级 seam(`argv()` / `runArgv()` / `startArgv()` / `onProcessDone()`)把精确的 pwsh 调用经 provider 包装。沙盒策略(模式 + 工作区根目录)不是本包的配置:每次调用由 `ctx.sandboxPolicy` 随行(工具层传调用会话解析后的策略;直接调用回退到部署策略)。 diff --git a/packages/shell/pwsh-sandbox/package.json b/packages/shell/pwsh-sandbox/package.json index 74f1c5b3c1..267a89c4e6 100644 --- a/packages/shell/pwsh-sandbox/package.json +++ b/packages/shell/pwsh-sandbox/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-pwsh-sandbox", "description": "Sandbox-consuming implementation of the DeepSeek Harness PowerShell executor seam (confines every command via ctx.sandbox, reports denial/enforcement result facts)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/shell-env/README.i18n.yaml b/packages/shell/shell-env/README.i18n.yaml index 44eff1b922..a8eb16c5b9 100644 --- a/packages/shell/shell-env/README.i18n.yaml +++ b/packages/shell/shell-env/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/shell-env/README.md README.md: b056289bd58e960e9fd36ee84ed20579e7bde42e -README.zh.md: 3ced6d63525db8a15f9202311c9ca9e32335cab8 +README.zh.md: 76d1af497b603d4df41fa1b65eaeddd38c0ae9b3 diff --git a/packages/shell/shell-env/README.zh.md b/packages/shell/shell-env/README.zh.md index 3ced6d6352..76d1af497b 100644 --- a/packages/shell/shell-env/README.zh.md +++ b/packages/shell/shell-env/README.zh.md @@ -17,7 +17,7 @@ ## Managed environment -每次前台与后台模型 shell 调用都会收到一份新收集的受信任 `DSH_*` 环境。`DSH_HOME` 是由 [`@deepseek-ai/dsh-home-paths`](../../util/home-paths/README.md) 解析的 Harness 主目录绝对路径(`dshHome` 配置,然后环境变量 `$DSH_HOME`,然后 `~/.dsh`),`DSH_SHELL=1` 标识受管理的子进程。带 agent(智能体)的调用额外收到 `DSH_SESSION_ID=agent.session.header.id`;当活动的持久化 seam 定位到 JSONL 工件时,它们还会收到 `DSH_SESSION_JSONL=<绝对目标路径>`。JSONL 路径只是位置提示:首次 flush 之前它可能不存在,也不一定包含当前缓冲中的轮次,并且它不是授权凭据。 +每次前台与后台模型 shell 调用都会收到一份新收集的受信任 `DSH_*` 环境。`DSH_HOME` 是由 [`@deepseek-ai/dsh-home-paths`](../../util/home-paths/README.zh.md) 解析的 Harness 主目录绝对路径(`dshHome` 配置,然后环境变量 `$DSH_HOME`,然后 `~/.dsh`),`DSH_SHELL=1` 标识受管理的子进程。带 agent(智能体)的调用额外收到 `DSH_SESSION_ID=agent.session.header.id`;当活动的持久化 seam 定位到 JSONL 工件时,它们还会收到 `DSH_SESSION_JSONL=<绝对目标路径>`。JSONL 路径只是位置提示:首次 flush 之前它可能不存在,也不一定包含当前缓冲中的轮次,并且它不是授权凭据。 `ctx.shellEnv` 负责收集。其他插件可以注册一个受 effect 作用域约束的 contributor,带有稳定名称、已声明的键/描述以及 `resolve(execution: ToolExecution)`;重复所有权与未声明的运行时键会响亮失败,而 `list()` 只枚举声明、不执行 provider。Harness 内置键保留 `DSH_HOME`、`DSH_SHELL` 与 `DSH_SESSION_ID`;本插件的持久化翻译器通过读取与后端无关的 `sessionPersistence.locate()` seam 拥有 `DSH_SESSION_JSONL`。 diff --git a/packages/shell/shell-env/package.json b/packages/shell/shell-env/package.json index f5ef9d869a..bd796263d6 100644 --- a/packages/shell/shell-env/package.json +++ b/packages/shell/shell-env/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-shell-env", "description": "Tool-independent managed DSH_* shell environment registry", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/shell/README.i18n.yaml b/packages/shell/shell/README.i18n.yaml index a4e67f3760..5317414e55 100644 --- a/packages/shell/shell/README.i18n.yaml +++ b/packages/shell/shell/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/shell/README.md README.md: a15980ead797a73c18dd6d8825a4b38cf845df60 -README.zh.md: 87a7dffdfb28fcf8a58e6254bf450e513dd458f4 +README.zh.md: 008b19835697d5d9514cf5472696852cfc684acd diff --git a/packages/shell/shell/README.zh.md b/packages/shell/shell/README.zh.md index 87a7dffdfb..008b198356 100644 --- a/packages/shell/shell/README.zh.md +++ b/packages/shell/shell/README.zh.md @@ -13,7 +13,7 @@ | `@deepseek-ai/dsh-bash-sandbox` | Service Provider:沿用 `dsh-bash-local` 的机制,但通过 [`ctx.sandbox`](../../sandbox/sandbox/) 限制每次 spawn,并将拒绝报告为结果事实 | | `@deepseek-ai/dsh-tool-bash` | 基于 `ctx.shell`、面向模型的工具 schema | -该拆分是一个标准的能力 seam([capability-seams Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):`dsh-bash-sandbox` 是位于同一 Service Definition 之后的沙箱执行器——Consumer 检测其 `sandboxMode` 能力并添加升权字段,无需导入提供方——容器化或远程执行器也可以同样接入。 +该拆分是一个标准的能力 seam([capability-seams Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.zh.md)):`dsh-bash-sandbox` 是位于同一 Service Definition 之后的沙箱执行器——Consumer 检测其 `sandboxMode` 能力并添加升权字段,无需导入提供方——容器化或远程执行器也可以同样接入。 ## 服务 API(`ctx.shell`) @@ -33,9 +33,9 @@ `ShellExecRequest`(command、workdir?、timeoutMs?、stdoutMaxBytes?、signal?、stdin?、env?、dshEnv?、sandboxPolicy?)在执行前解析为 `ShellExecSpec`(command、workdir、timeoutMs、stdoutMaxBytes、signal?、stdin?、env?、dshEnv?、sandboxPolicy)。`stdoutMaxBytes` 是受信任前台运行的捕获预算,用于必须解析完整有界 stdout 的消费方;面向模型的 bash 工具不公开该字段。`sandboxPolicy` 在请求上可选,在已解析 spec 上必填但可为 null:它携带完整的每次调用模式与工作区根目录。沙箱工具路径通过 `ctx.sandboxPolicy` 从调用会话解析它;沙箱执行器的直接调用方回退到部署策略,非沙箱执行器则携带该字段但不作限制。 -每会话沙箱模式覆盖词汇(`'sandbox/mode'` 事件、`effectiveSandboxMode(events)` fold 以及 `setSandboxMode(session, mode)` 写入路径)不位于此处。它是所有强制执行家族共享的策略状态,属于 [`@deepseek-ai/dsh-sandbox-policy`](../../sandbox/sandbox-policy/)。`run()` 返回 `ShellRunResult`;`start()` 返回 `ShellProcess`,其增量读取与终止方法由 `dsh-tool-bash` 适配为通用任务注册。沙箱执行器会在前台结果与已结算进程句柄上标记 `ShellSandboxInfo`。详见 `src/types.ts` 与 [subsystems/shell.md](../../../docs/subsystems/shell.md)。 +每会话沙箱模式覆盖词汇(`'sandbox/mode'` 事件、`effectiveSandboxMode(events)` fold 以及 `setSandboxMode(session, mode)` 写入路径)不位于此处。它是所有强制执行家族共享的策略状态,属于 [`@deepseek-ai/dsh-sandbox-policy`](../../sandbox/sandbox-policy/)。`run()` 返回 `ShellRunResult`;`start()` 返回 `ShellProcess`,其增量读取与终止方法由 `dsh-tool-bash` 适配为通用任务注册。沙箱执行器会在前台结果与已结算进程句柄上标记 `ShellSandboxInfo`。详见 `src/types.ts` 与 [subsystems/shell.md](../../../docs/subsystems/shell.zh.md)。 -`stdin` 与普通 `env` 由同进程插件(hooks 桥接、原生插件)设置,用于向 hook 命令提供其 JSON payload 和 `CLAUDE_PROJECT_DIR`/`CLAUDE_PLUGIN_ROOT` 值。`dshEnv` 是受类型限制、仅允许受管 key 的独立受信任 overlay;导出的 `DSH_ENV_PREFIX` 是该 namespace、其 `DshEnvironmentKey` 模板类型、执行器清理、注册表验证、派生内置名称与模型指引的统一来源。模型 bash 使用 `ctx.shellEnv` 收集的当前快照。实现会移除继承的受管 key,再在普通 `env` 之后合并 `dshEnv`,因此省略的当前事实不会回退到陈旧环境状态,`env` 条目也无法顶掉受管值。面向模型的工具不将这三者中的任何一个公开为参数。这三者在已解析 spec 上仍然可选;缺失表示没有输入/overlay。详见 [bash-stdin-env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.md) 与 [会话环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 +`stdin` 与普通 `env` 由同进程插件(hooks 桥接、原生插件)设置,用于向 hook 命令提供其 JSON payload 和 `CLAUDE_PROJECT_DIR`/`CLAUDE_PLUGIN_ROOT` 值。`dshEnv` 是受类型限制、仅允许受管 key 的独立受信任 overlay;导出的 `DSH_ENV_PREFIX` 是该 namespace、其 `DshEnvironmentKey` 模板类型、执行器清理、注册表验证、派生内置名称与模型指引的统一来源。模型 bash 使用 `ctx.shellEnv` 收集的当前快照。实现会移除继承的受管 key,再在普通 `env` 之后合并 `dshEnv`,因此省略的当前事实不会回退到陈旧环境状态,`env` 条目也无法顶掉受管值。面向模型的工具不将这三者中的任何一个公开为参数。这三者在已解析 spec 上仍然可选;缺失表示没有输入/overlay。详见 [bash-stdin-env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md) 与 [会话环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.zh.md)。 导出的 `parseExitStatus`(连同 `ParsedExitStatus`)是 shell 工具共享渲染约定的另一半:`dsh-tool-bash` 的 `renderResult` 与 `dsh-tool-pwsh` 的 `renderPwshResult` 追加的 `[exit code: N]`/`[killed by signal: X]` marker 的逆解析。两个工具的 `presentResult` 都用它把渲染文本拆成 terminal 卡的输出正文与其退出状态 pill;它放在 Service Definition 中,两个工具便永远不会在 marker 约定上漂移。 @@ -50,4 +50,4 @@ ## 已知限制与暂缓事项 - **没有交互式输入词汇**:`stdin` 只会在 spawn 时写入一次并关闭;seam 不提供向运行中任务继续输入的通道,也没有 PTY 会话概念。 -- **前台超时始终由执行器负责**:seam 上由调用方负责 deadline 的模式已由 [工具调用超时策略 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.md) 明确暂缓。 +- **前台超时始终由执行器负责**:seam 上由调用方负责 deadline 的模式已由 [工具调用超时策略 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.zh.md) 明确暂缓。 diff --git a/packages/shell/shell/package.json b/packages/shell/shell/package.json index ab8a31dd7d..02d4792c8a 100644 --- a/packages/shell/shell/package.json +++ b/packages/shell/shell/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-shell", "description": "Abstract bash executor seam (ctx.shell) for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/tool-bash-persistent/README.i18n.yaml b/packages/shell/tool-bash-persistent/README.i18n.yaml index a9503a57a9..e512e891ce 100644 --- a/packages/shell/tool-bash-persistent/README.i18n.yaml +++ b/packages/shell/tool-bash-persistent/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/tool-bash-persistent/README.md -README.md: 37c259443cba566350d8ec7963857d5f6ce7396c -README.zh.md: 8f3cf4e4d20aad1537b48644cee1a9d5706f4b3c +README.md: 606920d087b42344f34b70103e167b0046d3cfd5 +README.zh.md: 85652bac23be02bd2c72753070734d91fb43ee98 diff --git a/packages/shell/tool-bash-persistent/README.md b/packages/shell/tool-bash-persistent/README.md index 37c259443c..606920d087 100644 --- a/packages/shell/tool-bash-persistent/README.md +++ b/packages/shell/tool-bash-persistent/README.md @@ -33,7 +33,7 @@ Prefix-stable while the configured description and schema remain unchanged. #### What the model sees -Commands share one shell per Agent, so cwd, exported variables, activated environments, functions, and background jobs persist across calls. Results exclude private completion markers and the shell prompt. A nonzero wrapped command appends `[exit code: N]`; a shell that exits before reporting that status instead appends `[shell exited: code N]`, `[shell killed by signal: SIG]`, or `[shell exited]` when the backend supplies neither, then resets and tells the model that the next call starts fresh. Long output keeps the earliest retained prefix plus a clipping notice. If the PTY has already dropped that prefix, the result says so explicitly instead of presenting a tail as complete output. Timeout returns bounded partial output, closes the uncertain shell, and reports the reset. +Commands share one shell per Agent, so cwd, exported variables, activated environments, functions, and background jobs persist across calls. Results exclude private completion markers. When the shell reads stdin again without having printed the completion marker — after `exec`, an interrupt, or an interactive foreground child whose stdin wait the provider proves — the call returns the captured partial output, which can end with the backend's own prompt text. A nonzero wrapped command appends `[exit code: N]`; a shell that exits before reporting that status instead appends `[shell exited: code N]`, `[shell killed by signal: SIG]`, or `[shell exited]` when the backend supplies neither, then resets and tells the model that the next call starts fresh. Long output keeps the earliest retained prefix plus a clipping notice. If the PTY has already dropped that prefix, the result says so explicitly instead of presenting a tail as complete output. Timeout returns bounded partial output, closes the uncertain shell, and reports the reset. #### Token effect @@ -46,5 +46,6 @@ Append-only tool results follow the reusable request prefix. ## Known Limitations and Deferred Work - The tool requires an owning Agent and a real PTY backend. +- An interactive foreground child (for example a REPL) returns early with partial output only where the subprocess provider proves its stdin wait; elsewhere the call runs to `timeoutMs`. - Explicit `exit` and timeout discard shell state. Cancellation also resets and discards the result, even when a complete status marker is already observable; the next call starts a fresh shell. - Environment facts such as network access and package mirrors belong in the configured `description`, not this package's default. diff --git a/packages/shell/tool-bash-persistent/README.zh.md b/packages/shell/tool-bash-persistent/README.zh.md index 8f3cf4e4d2..85652bac23 100644 --- a/packages/shell/tool-bash-persistent/README.zh.md +++ b/packages/shell/tool-bash-persistent/README.zh.md @@ -19,7 +19,7 @@ #### 模型所见 -生成的 [`bash` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-bash-persistent),其中包含配置的 `description`。本插件不贡献独立系统提示词段;persona 与环境指导由部署负责。 +生成的 [`bash` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-bash-persistent),其中包含配置的 `description`。本插件不贡献独立系统提示词段;persona 与环境指导由部署负责。 #### Token 影响 @@ -33,7 +33,7 @@ #### 模型所见 -每个 Agent 的命令共享一个 shell,因此 cwd、导出的环境变量、已激活环境、函数和后台任务会跨调用保留。结果不包含私有完成标记和 shell 提示符。经封装的命令以非零状态结束时,结果会追加 `[exit code: N]`;若 shell 在报告该状态前退出,则改为追加 `[shell exited: code N]`、`[shell killed by signal: SIG]`,或在后端既未提供退出码也未提供信号时追加 `[shell exited]`;随后重置 shell,并告知模型下次调用从新 shell 开始。长输出保留仍可读取的最早前缀并追加截断提示;若 PTY 已丢弃真正的开头,结果会明确说明,而不是把尾部伪装成完整输出。超时返回有界的部分输出、关闭状态不确定的 shell,并报告该重置。 +每个 Agent 的命令共享一个 shell,因此 cwd、导出的环境变量、已激活环境、函数和后台任务会跨调用保留。结果不包含私有完成标记。当 shell 在未打印完成标记的情况下再次读取 stdin 时——例如 `exec`、中断,或提供方能证明其 stdin 等待的交互式前台子进程——调用返回已捕获的部分输出,其末尾可能带有后端自己的提示符文本。经封装的命令以非零状态结束时,结果会追加 `[exit code: N]`;若 shell 在报告该状态前退出,则改为追加 `[shell exited: code N]`、`[shell killed by signal: SIG]`,或在后端既未提供退出码也未提供信号时追加 `[shell exited]`;随后重置 shell,并告知模型下次调用从新 shell 开始。长输出保留仍可读取的最早前缀并追加截断提示;若 PTY 已丢弃真正的开头,结果会明确说明,而不是把尾部伪装成完整输出。超时返回有界的部分输出、关闭状态不确定的 shell,并报告该重置。 #### Token 影响 @@ -46,5 +46,6 @@ ## 已知限制与延后工作 - 工具需要拥有它的 Agent 和真实 PTY 后端。 +- 交互式前台子进程(例如 REPL)只有在进程管理提供方能证明其 stdin 等待时才会提前返回部分输出;否则调用会一直运行到 `timeoutMs`。 - 显式 `exit` 与超时会丢弃 shell 状态。取消同样会重置 shell 并丢弃结果,即使已经能观察到完整状态标记也是如此;下次调用创建新 shell。 - 网络访问、软件包镜像等环境事实应写入配置的 `description`,而非包默认描述。 diff --git a/packages/shell/tool-bash-persistent/package.json b/packages/shell/tool-bash-persistent/package.json index 14648a2231..c66bef18e3 100644 --- a/packages/shell/tool-bash-persistent/package.json +++ b/packages/shell/tool-bash-persistent/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-bash-persistent", "description": "Model-facing owner-scoped persistent Bash tool backed by the Harness PTY service", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/tool-bash-persistent/src/index.ts b/packages/shell/tool-bash-persistent/src/index.ts index 16d127bbe2..f9cf8d764a 100644 --- a/packages/shell/tool-bash-persistent/src/index.ts +++ b/packages/shell/tool-bash-persistent/src/index.ts @@ -7,7 +7,7 @@ import { randomUUID } from 'node:crypto' import type { Context } from '@deepseek-ai/cordis' import z from '@deepseek-ai/schemastery' import type { Agent } from '@deepseek-ai/dsh-agent' -import type { TerminalReadResult, TerminalSendResult, TerminalSessionId } from '@deepseek-ai/dsh-terminal' +import type { TerminalReadResult, TerminalSessionId } from '@deepseek-ai/dsh-terminal' import { deadline, timeoutOf } from '@deepseek-ai/dsh-timeout' import { defineTool } from '@deepseek-ai/dsh-tools' @@ -15,7 +15,6 @@ import { defineTool } from '@deepseek-ai/dsh-tools' const TRUNCATED_MESSAGE = 'To save on context only part of this file has been shown to you. You should retry this tool after you have searched inside the file with `grep -n` in order to find the line numbers of what you are looking for.' const LOST_PREFIX_MESSAGE = 'The beginning of this command output was dropped by the terminal scrollback limit. The following text is the earliest retained output.\n' const SHELL_RESET_MESSAGE = 'The persistent bash shell was reset; the next bash call starts from the workspace with a fresh current directory and environment.' -const SHELL_PROMPT = '__DSH_PERSISTENT_BASH_PROMPT__ ' const TIMEOUT_CODE = 'PERSISTENT_BASH_TIMEOUT' // One page is enough to find a just-emitted completion marker; the full // scrollback is assembled only when a command settles or needs partial output. @@ -82,12 +81,8 @@ function wrapCommand(command: string, marker: CommandMarkers): string { return `printf '%s\\n' ${quoteForBash(marker.start)}; eval -- ${quoteForBash(command)}; __dsh_persistent_bash_status=$?; printf '%s%s\\n' ${quoteForBash(marker.end)} "$__dsh_persistent_bash_status"` } -function stripPrompt(text: string): string { - let result = text.replace(/\r?\n$/, '') - while (result.endsWith(SHELL_PROMPT)) { - result = result.slice(0, -SHELL_PROMPT.length) - } - return result.endsWith('\n') ? result.slice(0, -1) : result +function trimTrailingNewline(text: string): string { + return text.replace(/\r?\n$/, '') } function commandOutput( @@ -101,18 +96,12 @@ function commandOutput( const startMarker = text.lastIndexOf(marker.start, end) const start = startMarker < 0 ? 0 : startMarker + marker.start.length return { - text: stripPrompt(text.slice(start, end).replace(/^\r?\n/, '')), + text: trimTrailingNewline(text.slice(start, end).replace(/^\r?\n/, '')), incomplete: startMarker < 0, exitCode: Number(status), } } -function promptCompleted(result: TerminalSendResult): boolean { - return result.viewport.endsWith(SHELL_PROMPT) - || result.viewport.endsWith(`${SHELL_PROMPT}\r\n`) - || result.viewport.endsWith(`${SHELL_PROMPT}\n`) -} - function partialOutput( snapshot: RetainedOutput, marker: CommandMarkers, @@ -122,7 +111,7 @@ function partialOutput( const startMarker = snapshot.text.lastIndexOf(marker.start) if (startMarker >= 0) { return { - text: stripPrompt(snapshot.text.slice(startMarker + marker.start.length).replace(/^\r?\n/, '')), + text: trimTrailingNewline(snapshot.text.slice(startMarker + marker.start.length).replace(/^\r?\n/, '')), incomplete: false, } } @@ -133,7 +122,7 @@ function partialOutput( const fallbackEnd = afterStart.lastIndexOf(marker.end) const beforeEnd = fallbackEnd < 0 ? afterStart : afterStart.slice(0, fallbackEnd) return { - text: stripPrompt(beforeEnd.replaceAll(SHELL_PROMPT, '')), + text: trimTrailingNewline(beforeEnd), incomplete: fallbackTruncated || fallbackStart < 0, } } @@ -197,6 +186,36 @@ function renderShellExitStatus( return appendStatusMarker(content, marker) } +/** + * Render the exited-session result, reset the owner's shell, and reset the + * message that tells the model the next call starts fresh. + * @param shells - the owner-scoped registry to reset. + * @param status - the exited session status (exit code and signal). + * @returns the complete model-facing result. + */ +async function respondToSessionExit( + ctx: Context, + shells: PersistentShells, + owner: Agent, + id: TerminalSessionId, + status: { exitCode: number | null; signal: NodeJS.Signals | null }, + marker: CommandMarkers, + fallback: string, + fallbackTruncated: boolean, + config: ResolvedConfig, +): Promise { + const snapshot = retainedScrollback(ctx, owner, id) + await shells.reset(owner, 'persistent bash shell exited') + return [ + renderShellExitStatus( + renderCaptured(partialOutput(snapshot, marker, fallback, fallbackTruncated), config.maxOutputChars), + status.exitCode, + status.signal, + ), + SHELL_RESET_MESSAGE, + ].filter(part => part.length > 0).join('\n') +} + function persistentShells(ctx: Context, config: ResolvedConfig): PersistentShells { const pending = new WeakMap>() const live = new Map() @@ -243,8 +262,10 @@ function persistentShells(ctx: Context, config: ResolvedConfig): PersistentShell live.delete(owner) }, 'tool-bash-persistent owner cache cleanup') } + // Echo suppression only: the prompt stays the backend's own, so the + // backend's prompt-based readiness detection keeps working. const setup = ctx.terminals.startSend(owner, spawned.sessionId, { - text: `stty -echo; PS1=${quoteForBash(SHELL_PROMPT)}`, + text: 'stty -echo', submit: true, signal: combinedSignal, }) @@ -286,6 +307,15 @@ async function executeCommand( let fallbackTruncated = false while (true) { + // The shell may flip to exited between iterations (a fast `exit` can + // settle the previous send while its exit event is still in flight); + // re-observing status before the next send closes that gap. + const status = ctx.terminals.list(owner).find(session => session.sessionId === id)?.status + if (status?.kind === 'exited') { + return await respondToSessionExit( + ctx, shells, owner, id, status, marker, fallback, fallbackTruncated, config, + ) + } let operation let result try { @@ -328,18 +358,15 @@ async function executeCommand( if (complete !== undefined) return renderCaptured(complete, config.maxOutputChars) } if (result.sessionStatus.kind === 'exited') { - const snapshot = retainedScrollback(ctx, owner, id, latest) - await shells.reset(owner, 'persistent bash shell exited') - return [ - renderShellExitStatus( - renderCaptured(partialOutput(snapshot, marker, fallback, fallbackTruncated), config.maxOutputChars), - result.sessionStatus.exitCode, - result.sessionStatus.signal, - ), - SHELL_RESET_MESSAGE, - ].filter(part => part.length > 0).join('\n') + return await respondToSessionExit( + ctx, shells, owner, id, result.sessionStatus, marker, fallback, fallbackTruncated, config, + ) } - if (promptCompleted(result)) { + // The shell reads stdin again (its prompt, or a foreground child's own + // read) without having printed the end marker — e.g. `exec`, an interrupt, + // or an interactive child. Return what was captured instead of spinning + // until the command deadline. + if (result.waitReason === 'stdin_read') { const snapshot = retainedScrollback(ctx, owner, id, latest) return renderCaptured( partialOutput(snapshot, marker, fallback, fallbackTruncated), diff --git a/packages/shell/tool-bash-persistent/tests/loader-composition.spec.ts b/packages/shell/tool-bash-persistent/tests/loader-composition.spec.ts index 55e89bc9f4..6d6affdb44 100644 --- a/packages/shell/tool-bash-persistent/tests/loader-composition.spec.ts +++ b/packages/shell/tool-bash-persistent/tests/loader-composition.spec.ts @@ -84,7 +84,10 @@ suite('persistent Bash through a real cordis.yml Loader composition', () => { ' config:', ' pollIntervalMs: 10', ' exactProbeAfterMs: 20', - ' idleSilenceMs: 100', + // The silence tier is pushed beyond the send bound, so no send below can + // settle as inferred_idle: every case proves the controlled-prompt fast + // path that the production defaults (3.5s silence) would otherwise mask. + ' idleSilenceMs: 30000', ' handoffGraceMs: 100', ' scrollbackLines: 20000', ' timeoutMs: 2000', @@ -154,6 +157,12 @@ suite('persistent Bash through a real cordis.yml Loader composition', () => { expect(large).toContain('') expect(large).not.toContain('beginning of this command output was dropped') + // `exec` replaces the wrapper before its end marker prints; the seam's + // stdin_read readiness is what returns the replacement shell's prompt + // instead of spinning until the tool deadline. + const execed = text(await execute('exec-replacement', 'exec bash --noprofile --norc -i')) + expect(execed).toBe('dsh> ') + const exited = text(await execute('exit', 'exit')) expect(exited).toContain('next bash call starts from the workspace') expect(text(await execute('after-exit', 'printf "%s\\n" "$PWD"'))).toBe(root) diff --git a/packages/shell/tool-bash-persistent/tests/tools.spec.ts b/packages/shell/tool-bash-persistent/tests/tools.spec.ts index b3de46643c..4cdebbbdce 100644 --- a/packages/shell/tool-bash-persistent/tests/tools.spec.ts +++ b/packages/shell/tool-bash-persistent/tests/tools.spec.ts @@ -98,9 +98,10 @@ type StubMode = | 'incremental-fallback' | 'empty-page-after-latest' | 'paged-scrollback' + | 'exit-after-send' class StubPtySession implements TerminalBackendSession { - readonly motd = '__DSH_PERSISTENT_BASH_PROMPT__ ' + readonly motd = 'stub> ' readonly pid = 123 statusValue: TerminalSessionStatus = { kind: 'running' } scrollback = this.motd @@ -109,6 +110,7 @@ class StubPtySession implements TerminalBackendSession { sends = 0 pendingText = '' historyTruncated = false + throwOnSend = false constructor(mode: StubMode) { this.mode = mode @@ -127,6 +129,7 @@ class StubPtySession implements TerminalBackendSession { return this.operation(Promise.resolve(this.result(this.motd, 'stdin_read'))) } if (this.mode === 'send-error') throw new Error('stub send failed') + if (this.throwOnSend) throw new Error('PTY session has exited') if (this.mode === 'wait-for-abort' || this.mode === 'end-on-abort') { const done = new Promise>((resolve) => { request.signal?.addEventListener('abort', () => { @@ -171,6 +174,18 @@ class StubPtySession implements TerminalBackendSession { const incremental = `${start ?? ''}\nincrement\n${this.motd}` return this.operation(Promise.resolve(this.result(this.motd, 'stdin_read')), incremental) } + if (this.mode === 'exit-after-send') { + // A fast `exit` settles the send while the exit event is still in + // flight; the shell flips to exited before the tool's next poll, + // exactly like the real backend. The tool must re-observe status + // instead of sending. + const output = `${start ?? ''}\n` + this.scrollback += output + const settled = this.result(output, 'inferred_idle') + this.statusValue = { kind: 'exited', exitCode: 9, signal: null } + this.throwOnSend = true + return this.operation(Promise.resolve(settled)) + } if (this.mode === 'torn-status') { const output = `${start ?? ''}\nhello from stub\n${end ?? ''}` this.scrollback += output @@ -325,7 +340,7 @@ describe('tool-bash-persistent', () => { expect(ctx.tools.get('bash')).toBeUndefined() }) - it('handles inferred idle, prompt fallback, shell exit, clipping, and cleanup', async () => { + it('handles inferred idle, stdin_read fallback, shell exit, clipping, and cleanup', async () => { const { ctx, owner, stub, fiber } = await setup({ backendType: 'stub', maxOutputChars: 10, @@ -338,18 +353,16 @@ describe('tool-bash-persistent', () => { session.mode = 'incremental-fallback' session.scrollback = '' - expect(text(await call(ctx, owner, 'incremental fallback'))).toBe('increment') + expect(text(await call(ctx, owner, 'incremental fallback'))).toContain('increment') session.mode = 'prompt-only' const promptFallback = text(await call(ctx, owner, 'bad {')) expect(promptFallback).toContain('bash: synt') - expect(promptFallback).not.toContain('DSH_PERSISTENT_BASH_PROMPT') session.mode = 'prompt-crlf' session.scrollback = '' const crlfPromptFallback = text(await call(ctx, owner, 'bad {')) expect(crlfPromptFallback).toContain('bash: synt') - expect(crlfPromptFallback).not.toContain('DSH_PERSISTENT_BASH_PROMPT') session.mode = 'end-only' session.scrollback = '' @@ -397,6 +410,21 @@ describe('tool-bash-persistent', () => { expect(text(await call(ctx, owner, 'torn status'))).toBe('hello from stub\n[exit code: 7]') }) + it('reports the exit path when the shell exits between send settlement and the next poll', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub' }) + await call(ctx, owner, 'warm up') + const session = stub.sessions[0]! + session.mode = 'exit-after-send' + + const result = text(await call(ctx, owner, 'exit')) + expect(result).toContain('[shell exited: code 9]') + expect(result).toContain('next bash call starts from the workspace') + expect(session.closed).toContain('persistent bash shell exited') + + expect(text(await call(ctx, owner, 'echo "$PWD"'))).toBe('hello from stub') + expect(stub.sessions).toHaveLength(2) + }) + it('reports a shell exit when the backend has no code or signal', async () => { const { ctx, owner, stub } = await setup({ backendType: 'stub' }) await call(ctx, owner, 'warm up') @@ -435,7 +463,7 @@ describe('tool-bash-persistent', () => { expect(text(await call(ctx, owner, 'paged output'))).toBe('hello from stub') }) - it('sanitizes a prompt fallback reached after multiple polling rounds', async () => { + it('returns a stdin_read fallback reached after multiple polling rounds', async () => { const { ctx, owner, stub } = await setup({ backendType: 'stub', maxOutputChars: 1_000 }) await call(ctx, owner, 'warm up') const session = stub.sessions[0]! @@ -444,7 +472,8 @@ describe('tool-bash-persistent', () => { const result = text(await call(ctx, owner, 'bad {')) expect(result).toContain('partial syntax output') expect(result).toContain('bash: syntax error') - expect(result).not.toContain('DSH_PERSISTENT_BASH_PROMPT') + // The backend owns the prompt text, so the fallback retains it verbatim. + expect(result.endsWith('stub> ')).toBe(true) expect(result).not.toContain('DSH_PERSISTENT_BASH_START') }) diff --git a/packages/shell/tool-bash/README.i18n.yaml b/packages/shell/tool-bash/README.i18n.yaml index ccdd7fe9cd..f3acda2aca 100644 --- a/packages/shell/tool-bash/README.i18n.yaml +++ b/packages/shell/tool-bash/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/tool-bash/README.md README.md: 21749bf0a1cd3cdc46257fb2a02ba6b4ab1f5ee6 -README.zh.md: e79190ced58abbd73d85127f3a4362f4e6b292cd +README.zh.md: 860db1a4c347d1aefe04f70e14a65e4b511ff7f6 diff --git a/packages/shell/tool-bash/README.zh.md b/packages/shell/tool-bash/README.zh.md index e79190ced5..860db1a4c3 100644 --- a/packages/shell/tool-bash/README.zh.md +++ b/packages/shell/tool-bash/README.zh.md @@ -4,7 +4,7 @@ 模型侧 `bash` 工具,注册在 `ctx.shell` 执行器 seam 上。前台执行始终位于该 seam 之后;后台进程句柄会注册到通用 `ctx.jobs` 运行时,并通过 `job_output`、`job_list` 和 `job_kill` 控制;这些工具由 `@deepseek-ai/dsh-tool-jobs` 提供。 -需要加载执行器 Service Provider(例如 `@deepseek-ai/dsh-bash-local`)与 [`@deepseek-ai/dsh-shell-env`](../shell-env/README.md) 注册表;在每个注入服务就绪之前,插件会保持等待状态(`inject: ['tools', 'bash', 'systemPrompt', 'bashEnv']`)。工具约定是 bash 方言——请挂载能解析 bash 的执行器。 +需要加载执行器 Service Provider(例如 `@deepseek-ai/dsh-bash-local`)与 [`@deepseek-ai/dsh-shell-env`](../shell-env/README.zh.md) 注册表;在每个注入服务就绪之前,插件会保持等待状态(`inject: ['tools', 'bash', 'systemPrompt', 'bashEnv']`)。工具约定是 bash 方言——请挂载能解析 bash 的执行器。 包根只公开 Cordis 插件约定(`name`、`inject`、`Config`、`apply`);结果渲染和后台进程适配仍保留在包内部。 @@ -28,7 +28,7 @@ ### 托管 shell 环境 -每次模型发起的前台或后台 bash 调用都会通过共享的 [`dsh-shell-env`](../shell-env/README.md) 注册表收到新收集的一组可信 `DSH_*` 环境变量:`DSH_HOME`(Harness home 绝对路径)、`DSH_SHELL=1`、agent 的 `DSH_SESSION_ID`,以及当活跃持久化后端能定位时的 `DSH_SESSION_JSONL`。注册表约定——贡献方注册、重复键/未声明键的显式报错机制、内置项保留与贡献方示例——载于该包的 README。快照通过专用的 `ShellExecRequest.dshEnv` 通道传递;本地执行器会先删除继承的所有 `DSH_*` 再合并,因此嵌套 harness 和并发的父/子 agent 不会泄漏陈旧身份,且绝不修改 `process.env`。工具说明只教授通用 `$DSH_*` 约定,不会点名持久化专用变量,也不会添加永久的系统提示词段落。 +每次模型发起的前台或后台 bash 调用都会通过共享的 [`dsh-shell-env`](../shell-env/README.zh.md) 注册表收到新收集的一组可信 `DSH_*` 环境变量:`DSH_HOME`(Harness home 绝对路径)、`DSH_SHELL=1`、agent 的 `DSH_SESSION_ID`,以及当活跃持久化后端能定位时的 `DSH_SESSION_JSONL`。注册表约定——贡献方注册、重复键/未声明键的显式报错机制、内置项保留与贡献方示例——载于该包的 README。快照通过专用的 `ShellExecRequest.dshEnv` 通道传递;本地执行器会先删除继承的所有 `DSH_*` 再合并,因此嵌套 harness 和并发的父/子 agent 不会泄漏陈旧身份,且绝不修改 `process.env`。工具说明只教授通用 `$DSH_*` 约定,不会点名持久化专用变量,也不会添加永久的系统提示词段落。 结果文本依次包含 stdout、可选的 `[stderr]` 段落和适用的沙箱拒绝、超时、信号、退出代码及截断标记。超时与最终退出状态分别报告;非零退出仍是由模型解释的结果,不会成为 `isError`。截断结果会链接安全的完整 spill 文件,或报告文件不可用。只有 spawn 错误和中止等基础设施故障才会产生 `isError`。 @@ -42,17 +42,17 @@ ## 工具仅使用具名参数构建请求 -`ShellExecRequest` 携带可选的 `stdoutMaxBytes`、`stdin`、普通 `env` 和托管 `dshEnv`,供可信进程内插件及此工具的环境注册表使用。模型侧工具不公开 `stdoutMaxBytes`、`stdin` 或 `env`:它使用具名的命令/工作目录/超时/信号/沙箱字段,加上从注册表收集的 `dshEnv` 来构建请求。额外模型键会被忽略,无法替换托管值。Shell 语法可以提供等价的命令级行为,而本地执行器会清除环境中的凭据和陈旧 `DSH_*` 值。参见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.md)。 +`ShellExecRequest` 携带可选的 `stdoutMaxBytes`、`stdin`、普通 `env` 和托管 `dshEnv`,供可信进程内插件及此工具的环境注册表使用。模型侧工具不公开 `stdoutMaxBytes`、`stdin` 或 `env`:它使用具名的命令/工作目录/超时/信号/沙箱字段,加上从注册表收集的 `dshEnv` 来构建请求。额外模型键会被忽略,无法替换托管值。Shell 语法可以提供等价的命令级行为,而本地执行器会清除环境中的凭据和陈旧 `DSH_*` 值。参见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md)。 ## 权限与升权 除非启用沙箱的执行器([`dsh-bash-sandbox`](../bash-sandbox/))限制命令,否则命令以执行器的完整权限运行。仅拒绝型沙箱会把拒绝作为结果事实报告,并在此渲染为拒绝标记;逐调用的允许/拒绝/询问策略由 `tools/pre-execute` waterfall(瀑布式事件)负责(参见 docs/architecture.md)。 -需要升权的 bash 调用会在执行前解析 `ctx.approval`。`allowed-once` 只对该次调用应用请求模式;审批被拒、取消、不可用或缺少审批上下文时,命令完全不会执行,并返回不同的错误。发生真实拒绝后,模型可以在同一轮次中使用满足需要的最窄模式和理由重试同一命令一次;审批提示本身就是征求同意的步骤。升权绝不能预先推测,禁用或拒绝审批即为最终结果。其理由见 [沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md)。 +需要升权的 bash 调用会在执行前解析 `ctx.approval`。`allowed-once` 只对该次调用应用请求模式;审批被拒、取消、不可用或缺少审批上下文时,命令完全不会执行,并返回不同的错误。发生真实拒绝后,模型可以在同一轮次中使用满足需要的最窄模式和理由重试同一命令一次;审批提示本身就是征求同意的步骤。升权绝不能预先推测,禁用或拒绝审批即为最终结果。其理由见 [沙箱 Agent Note](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.zh.md)。 ## 逐会话模式切换 -对于启用沙箱的执行器,每次调用依次按单次升权、会话覆盖、执行器默认值解析模式。未启用沙箱以及没有 agent 的调用不携带会话覆盖。策略归属方贡献当前且不区分具体能力的常驻模式;拒绝结果仍负责特定于该操作的有效模式与重试引导。参见 [`dsh-shell` 折叠计算](../shell/README.md)和[沙箱切换约定](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.md)。 +对于启用沙箱的执行器,每次调用依次按单次升权、会话覆盖、执行器默认值解析模式。未启用沙箱以及没有 agent 的调用不携带会话覆盖。策略归属方贡献当前且不区分具体能力的常驻模式;拒绝结果仍负责特定于该操作的有效模式与重试引导。参见 [`dsh-shell` 折叠计算](../shell/README.zh.md)和[沙箱切换约定](../../../.agents/notes/implemented/feature/2026-07-06-sandbox.zh.md)。 ## 模型体验 @@ -80,7 +80,7 @@ Check the [exit code: N] marker on every bash result; investigate failures befor #### 模型看到的内容 -模型会看到生成的 [`bash` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-bash)。仅当此生产方启用 `run_in_background` 时,该字段才会出现;仅当已挂载执行器声明支持沙箱时,`sandbox_permissions` 和 `justification` 才会出现。Agent 作用域的工具限制可以移除该 agent 的定义。 +模型会看到生成的 [`bash` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-bash)。仅当此生产方启用 `run_in_background` 时,该字段才会出现;仅当已挂载执行器声明支持沙箱时,`sandbox_permissions` 和 `justification` 才会出现。Agent 作用域的工具限制可以移除该 agent 的定义。 #### Token 影响 @@ -94,7 +94,7 @@ Check the [exit code: N] marker on every bash result; investigate failures befor #### 模型看到的内容 -renderer 先输出依数据而定的 stdout 尾部,再输出可选的 `[stderr]` 和 stderr 尾部。没有输出时,它会精确输出 `(no output)`。条件行精确为 `[output truncated; full output: ]`、`[sandbox: file access denied under mode]`、`[timed out after ms]`、`[killed by signal: ]` 和 `[exit code: ]`;沙箱升权与 runner 故障行原文列于 [`dsh-bash-sandbox`](../bash-sandbox/README.md)。 +renderer 先输出依数据而定的 stdout 尾部,再输出可选的 `[stderr]` 和 stderr 尾部。没有输出时,它会精确输出 `(no output)`。条件行精确为 `[output truncated; full output: ]`、`[sandbox: file access denied under mode]`、`[timed out after ms]`、`[killed by signal: ]` 和 `[exit code: ]`;沙箱升权与 runner 故障行原文列于 [`dsh-bash-sandbox`](../bash-sandbox/README.zh.md)。 #### Token 影响 @@ -108,7 +108,7 @@ renderer 先输出依数据而定的 stdout 尾部,再输出可选的 `[stderr #### 模型看到的内容 -启动会精确返回 `started background job `。此生产方会向通用任务运行时提供增量进程输出、可选的 `[some output was dropped from memory; full output: ]`、沙箱事实,以及 `exit code: ` 或 `signal: ` 等终止详情。[`dsh-tool-jobs`](../../jobs/tool-jobs/README.md) 负责模型可见的状态行、完成通知、列表和取消响应。 +启动会精确返回 `started background job `。此生产方会向通用任务运行时提供增量进程输出、可选的 `[some output was dropped from memory; full output: ]`、沙箱事实,以及 `exit code: ` 或 `signal: ` 等终止详情。[`dsh-tool-jobs`](../../jobs/tool-jobs/README.zh.md) 负责模型可见的状态行、完成通知、列表和取消响应。 #### Token 影响 @@ -135,5 +135,5 @@ renderer 先输出依数据而定的 stdout 尾部,再输出可选的 `[stderr ## 已知限制与延期工作 - **回放退出状态 pill 从结果文本解析**:如果输出最后一行恰好精确为 `[exit code: N]` / `[killed by signal: …]`,会话回放将显示错误的 pill,并且该行会从卡片正文中丢失,因为解析会把它当作自己消耗的标记;这是仅影响展示的已知残留问题。 -- **`bash` 工具不采用 `timeout-policy` 预算**:根据[工具调用 timeout-policy Agent Note](../../../.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.md),它保留由执行器持有的 `BASH_TIMEOUT` 路径。 +- **`bash` 工具不采用 `timeout-policy` 预算**:根据[工具调用 timeout-policy Agent Note](../../../.agents/notes/implemented/architecture/2026-07-07-tool-call-timeout-policy.zh.md),它保留由执行器持有的 `BASH_TIMEOUT` 路径。 - **后台进程没有执行器超时**:工作不再需要时,调用方必须使用 `job_kill`,或依赖持有者/服务的 dispose。 diff --git a/packages/shell/tool-bash/package.json b/packages/shell/tool-bash/package.json index e762320465..321e2822d2 100644 --- a/packages/shell/tool-bash/package.json +++ b/packages/shell/tool-bash/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-bash", "description": "Model-facing bash tool with optional generic background-job and sandbox-escalation support", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/tool-pwsh-persistent/README.i18n.yaml b/packages/shell/tool-pwsh-persistent/README.i18n.yaml new file mode 100644 index 0000000000..cad7c52a6d --- /dev/null +++ b/packages/shell/tool-pwsh-persistent/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/shell/tool-pwsh-persistent/README.md +README.md: a57c940801606c2eef450f0e54fecb434c62a406 +README.zh.md: 75f6babfaffc79120f96441bdd5e08895b8e452f diff --git a/packages/shell/tool-pwsh-persistent/README.md b/packages/shell/tool-pwsh-persistent/README.md new file mode 100644 index 0000000000..a57c940801 --- /dev/null +++ b/packages/shell/tool-pwsh-persistent/README.md @@ -0,0 +1,55 @@ +# @deepseek-ai/dsh-tool-pwsh-persistent + +English | [中文](README.zh.md) + +Model-facing `pwsh(command)` backed by one owner-scoped `ctx.terminals` shell. The package owns the tool contract and shell reuse; deployments select the terminal backend (a `terminal-bash` instance configured with `shellDialect: pwsh`) and sandbox policy. It is the Windows counterpart of `tool-bash-persistent`: same persistent-state contract, PowerShell dialect. + +## Config + +| Key | Default | Meaning | +|---|---:|---| +| `backendType` | `shell` | Registered terminal backend used for each Agent shell. | +| `timeoutMs` | `300000` | Wall-clock limit for one command; timeout closes the shell. | +| `maxOutputChars` | `16000` | Maximum retained command-output characters; fixed diagnostics are added afterward. | +| `description` | Persistent-shell description | Model-facing environment contract. | + +## Model Experience + +### Tool schema + +#### What the model sees + +The generated [`pwsh` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-pwsh-persistent), including the configured `description`. The plugin contributes no standalone system-prompt section; the deployment owns persona and environment guidance. + +#### Token effect + +Fixed schema cost while `pwsh` is visible. + +#### KV Cache effect + +Prefix-stable while the configured description and schema remain unchanged. + +### Tool results + +#### What the model sees + +Commands share one shell per Agent, so cwd, `$env:` variables, functions, and background jobs persist across calls. Results exclude private completion markers, the shell prompt, and the echoed input line (PSReadLine renders submitted input back into the stream; the marker-anchored extraction and the wrapper-source strip remove it). A nonzero wrapped command appends `[exit code: N]` — the exact native exit code when the command ran a native program, `1` for a terminating PowerShell error. A shell that exits before reporting that status instead appends `[shell exited: code N]`, `[shell killed by signal: SIG]`, or `[shell exited]` when the backend supplies neither (Windows forced termination reports exit 1 without a signal), then resets and tells the model that the next call starts fresh. Long output keeps the earliest retained prefix plus a clipping notice; if the terminal has already dropped that prefix, the result says so explicitly. Timeout returns bounded partial output, closes the uncertain shell, and reports the reset. + +#### Token effect + +Data-dependent. `maxOutputChars` bounds retained command output; fixed clipping, lost-prefix, status, timeout, and reset diagnostics can extend the result. + +#### KV Cache effect + +Append-only tool results follow the reusable request prefix. + +## Known Limitations and Deferred Work + +- The tool requires an owning Agent and a real terminal backend with a pwsh dialect (Windows ConPTY or a POSIX pwsh). +- **Input echo is unavoidable**: PowerShell's PSReadLine renders submitted input back into the terminal stream, and there is no `stty -echo` equivalent. The marker-anchored extraction excludes the echo in complete results; the wrapper-source strip covers fallback paths, but a wrapper that wraps across the terminal width may leave a partial echo in partial-output results, bounded by `maxOutputChars`. +- Raw ESC characters inside model commands are unsupported: PSReadLine consumes them before execution. The wrapper escapes the control bytes it needs (`[char]27`-built OSC markers, backtick escapes for the body). +- A model redefinition of the `prompt` function removes the readiness marker; the shell then settles on the silence tier instead of the marker fast path. +- There is no interactive stdin during a command: a foreground command that reads input blocks until the readiness timeout, which resets the shell. +- SIGTSTP/SIGHUP are unavailable on Windows (backend-rejected); SIGINT is delivered as a console-wide Ctrl-C input write, which at a prompt cancels the pending line instead of signalling a process. +- Under the Windows ACL sandbox's read-only mode, pwsh starts in ConstrainedLanguage, which may deny the bootstrap's `[Console]::` encoding pin and prompt marker. Commands can still settle through the printable prompt and silence tier, but non-ASCII output may follow the host code page. +- The BEL-terminated OSC marker remains a readiness signal only; a BEL event channel to the model stays deferred, aligned with the current implementation. diff --git a/packages/shell/tool-pwsh-persistent/README.zh.md b/packages/shell/tool-pwsh-persistent/README.zh.md new file mode 100644 index 0000000000..75f6babfaf --- /dev/null +++ b/packages/shell/tool-pwsh-persistent/README.zh.md @@ -0,0 +1,55 @@ +# @deepseek-ai/dsh-tool-pwsh-persistent + +[English](README.md) | 中文 + +模型侧 `pwsh(command)`,由一个 owner 作用域的 `ctx.terminals` shell 支撑。本包拥有工具契约与 shell 复用;部署方选择 terminal backend(配置 `shellDialect: pwsh` 的 `terminal-bash` 实例)与沙箱策略。它是 `tool-bash-persistent` 的 Windows 对应物:相同的持久状态契约,PowerShell 方言。 + +## 配置 + +| 键 | 默认值 | 含义 | +|---|---:|---| +| `backendType` | `shell` | 每个 Agent shell 使用的已注册 terminal backend。 | +| `timeoutMs` | `300000` | 单条命令的墙钟上限;超时关闭 shell。 | +| `maxOutputChars` | `16000` | 保留的命令输出字符上限;固定诊断文本在其后追加。 | +| `description` | 持久 shell 描述 | 模型可见的环境契约。 | + +## 模型体验 + +### 工具 schema + +#### 模型看到什么 + +生成的 [`pwsh` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-pwsh-persistent),含配置的 `description`。本插件不贡献独立的 system-prompt 段落;persona 与环境指引由部署方负责。 + +#### Token 影响 + +`pwsh` 可见期间每个请求有固定的 schema 成本。 + +#### KV Cache 影响 + +配置的 description 与 schema 不变时前缀稳定。 + +### 工具结果 + +#### 模型看到什么 + +命令共享每个 Agent 的一个 shell,因此 cwd、`$env:` 变量、函数和后台任务跨调用保留。结果排除私有完成标记、shell 提示符与回显的输入行(PSReadLine 会把提交的输入渲染回输出流;marker 锚定提取与包装器原文剥离将其移除)。非零包装命令追加 `[exit code: N]` —— 命令运行原生程序时是精确的原生退出码,PowerShell 终止性错误为 `1`。shell 在报告状态前退出的,改为追加 `[shell exited: code N]`、`[shell killed by signal: SIG]` 或 `[shell exited]`(backend 两者都没有时;Windows 强杀按无 signal 的 exit 1 报告),然后重置并告知模型下一次调用从全新 shell 开始。长输出保留最早的前缀并附裁剪提示;若 PTY 已丢弃该前缀,结果会明确说明。超时返回有界的部分输出、关闭不确定的 shell 并报告重置。 + +#### Token 影响 + +数据相关。`maxOutputChars` 限制保留的命令输出;固定裁剪、前缀丢失、状态、超时与重置诊断可能扩展结果。 + +#### KV Cache 影响 + +追加式工具结果跟随可复用的请求前缀。 + +## 已知限制与延后工作 + +- 工具需要拥有 Agent 与一个真实支持 pwsh 方言的 terminal backend(Windows ConPTY 或 POSIX 上的 pwsh)。 +- **输入回显不可避免**:PowerShell 的 PSReadLine 会把提交的输入渲染回终端流,且没有 `stty -echo` 的对应物。完整结果中 marker 锚定提取排除回显;包装器原文剥离覆盖回退路径,但跨越终端宽度的包装器折行可能在部分输出结果中残留片段回显,受 `maxOutputChars` 约束。 +- 模型命令中的裸 ESC 字符不受支持:PSReadLine 会在执行前吞掉它们。包装器转义它需要的控制字节(`[char]27` 构造的 OSC 标记、body 的反引号转义)。 +- 模型重定义 `prompt` 函数会移除就绪标记;shell 随后退化为静默档而非 marker 快路径。 +- 命令执行期间没有交互 stdin:读取输入的前台命令会阻塞到就绪超时,随后重置 shell。 +- SIGTSTP/SIGHUP 在 Windows 不可用(backend 拒绝);SIGINT 以控制台级 Ctrl-C 输入写入投递,在提示符处取消当前行而非向进程发信号。 +- 在 Windows ACL 沙箱的只读模式下,pwsh 以 ConstrainedLanguage 启动,可能拒绝引导代码通过 `[Console]::` 固定编码并写入 prompt marker。命令仍可通过可打印提示符和静默档结算,但非 ASCII 输出可能沿用宿主代码页。 +- BEL 终结的 OSC 标记仍只是就绪信号;面向模型的 BEL 事件通道保持延后,与当前实现对齐。 diff --git a/packages/shell/tool-pwsh-persistent/package.json b/packages/shell/tool-pwsh-persistent/package.json new file mode 100644 index 0000000000..b722f12a05 --- /dev/null +++ b/packages/shell/tool-pwsh-persistent/package.json @@ -0,0 +1,62 @@ +{ + "name": "@deepseek-ai/dsh-tool-pwsh-persistent", + "description": "Model-facing owner-scoped persistent PowerShell tool backed by the Harness PTY service", + "version": "0.1.1-rc.2", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/shell/tool-pwsh-persistent" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "peerDependencies": { + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-terminal": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", + "@deepseek-ai/dsh-tools": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + }, + "dependencies": { + "@deepseek-ai/schemastery": "workspace:^" + }, + "devDependencies": { + "@deepseek-ai/cordis-plugin-include": "workspace:^", + "@deepseek-ai/cordis-plugin-loader": "workspace:^", + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-terminal": "workspace:^", + "@deepseek-ai/dsh-terminal-bash": "workspace:^", + "@deepseek-ai/dsh-pwsh-local": "workspace:^", + "@deepseek-ai/dsh-sandbox": "workspace:^", + "@deepseek-ai/dsh-sandbox-policy": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", + "@deepseek-ai/dsh-system-prompt": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", + "@deepseek-ai/dsh-tools": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + } +} diff --git a/packages/shell/tool-pwsh-persistent/src/index.ts b/packages/shell/tool-pwsh-persistent/src/index.ts new file mode 100644 index 0000000000..f0a575363e --- /dev/null +++ b/packages/shell/tool-pwsh-persistent/src/index.ts @@ -0,0 +1,514 @@ +/* jscpd:ignore-start -- deliberate mirror of tool-bash-persistent (persistent-pty note 2026-08-11-pwsh-persistent-pty): + the PowerShell counterpart shares the session registry, polling loop, and reset contract by design. */ +/** + * Model-facing persistent `pwsh` tool over the owner-scoped PTY seam. + * @module @deepseek-ai/dsh-tool-pwsh-persistent + */ + +import { randomUUID } from 'node:crypto' +import type { Context } from '@deepseek-ai/cordis' +import z from '@deepseek-ai/schemastery' +import type { Agent } from '@deepseek-ai/dsh-agent' +import type { TerminalReadResult, TerminalSendResult, TerminalSessionId } from '@deepseek-ai/dsh-terminal' +import { deadline, timeoutOf } from '@deepseek-ai/dsh-timeout' +import { defineTool } from '@deepseek-ai/dsh-tools' + +// TODO: Replace the file-search advice; arbitrary command output need not come from a searchable file. +const TRUNCATED_MESSAGE = 'To save on context only part of this file has been shown to you. You should retry this tool after you have searched inside the file with Select-String in order to find the line numbers of what you are looking for.' +const LOST_PREFIX_MESSAGE = 'The beginning of this command output was dropped by the terminal scrollback limit. The following text is the earliest retained output.\n' +const SHELL_RESET_MESSAGE = 'The persistent pwsh shell was reset; the next pwsh call starts from the workspace with a fresh current directory and environment.' +const SHELL_PROMPT = '__DSH_PERSISTENT_PWSH_PROMPT__ ' +const TIMEOUT_CODE = 'PERSISTENT_PWSH_TIMEOUT' +// One page is enough to find a just-emitted completion marker; the full +// scrollback is assembled only when a command settles or needs partial output. +const SCROLLBACK_PAGE_LINES = 1_000 +const POLL_INTERVAL_MS = 25 + +const DEFAULT_DESCRIPTION = 'Run commands in a persistent PowerShell shell. State, including the current directory and exported environment variables, persists across calls for this agent.' + +interface ResolvedConfig { + backendType: string + timeoutMs: number + maxOutputChars: number + description: string +} + +interface CommandMarkers { + start: string + end: string +} + +interface RetainedOutput { + text: string + truncated: boolean +} + +interface CapturedOutput { + text: string + incomplete: boolean + exitCode?: number +} + +interface PersistentShells { + get(owner: Agent, signal: AbortSignal): Promise + reset(owner: Agent, reason: string): Promise +} + +function maybeTruncate(content: string, maxOutputChars: number, incomplete = false): string { + if (content.length <= maxOutputChars && !incomplete) return content + return content.length <= maxOutputChars + ? content + TRUNCATED_MESSAGE + : content.slice(0, maxOutputChars) + TRUNCATED_MESSAGE +} + +function markers(): CommandMarkers { + const nonce = randomUUID() + return { + start: `__DSH_PERSISTENT_PWSH_START_${nonce}__`, + end: `__DSH_PERSISTENT_PWSH_END_${nonce}:`, + } +} + +/** + * Escape a command body for embedding in the wrapper's double-quoted string. + * Backtick escapes keep every character literal: backtick first so the + * escapes this function inserts are never re-escaped, `$` so no expansion + * happens at wrapper construction, and `\r\n`/ESC so multi-line commands and + * raw control bytes ride one physical input line without PSReadLine mangling. + * @param value - the model's PowerShell command text. + * @returns the escaped double-quoted-string body. + */ +function quoteForPwsh(value: string): string { + return value + .replaceAll('`', '``') + .replaceAll('"', '`"') + .replaceAll('$', '`$') + .replaceAll('\r', '') + .replaceAll('\n', '`n') + .replaceAll('\x1b', '`e') +} + +function wrapCommand(command: string, marker: CommandMarkers): string { + // Keep the wrapper on one physical line: PSReadLine renders the echoed + // input, and a wrapped line would split the echo the extraction strips. + // The echoed END nonce can never fabricate completion because the status + // regex needs digits immediately after it and the echo continues with + // quote characters. + const body = quoteForPwsh(command) + return `Write-Output '${marker.start}'; $LASTEXITCODE = $null; $__s = 1; try { Invoke-Expression "${body}"; $__ok = $? } catch { $__ok = $false }; if ($null -ne $LASTEXITCODE) { $__s = [int]$LASTEXITCODE } else { $__s = if ($__ok) { 0 } else { 1 } }; Write-Output ('${marker.end}' + $__s)` +} + +function stripPrompt(text: string): string { + let result = text.replace(/\r?\n$/, '') + while (result.endsWith(SHELL_PROMPT)) { + result = result.slice(0, -SHELL_PROMPT.length) + } + return result.endsWith('\n') ? result.slice(0, -1) : result +} + +function commandOutput( + snapshot: RetainedOutput, + marker: CommandMarkers, + wrapper: string, +): CapturedOutput | undefined { + const text = snapshot.text + const end = text.lastIndexOf(marker.end) + const status = /^(\d+)\r?\n/.exec(text.slice(end + marker.end.length))?.[1] + if (status === undefined) return undefined + const startMarker = text.lastIndexOf(marker.start, end) + const start = startMarker < 0 ? 0 : startMarker + marker.start.length + let captured = text.slice(start, end) + // The PSReadLine echo carries the wrapper source (including both marker + // nonces) before the real markers; anchor on the real markers excludes it, + // and stripping the wrapper covers the rare case where the real START + // scrolled out and extraction fell back to the echoed copy. + captured = captured.replaceAll(wrapper, '') + return { + text: captured.replace(/^\r?\n/, '').replace(/\r?\n$/, ''), + incomplete: startMarker < 0, + exitCode: Number(status), + } +} + +function promptCompleted(result: TerminalSendResult): boolean { + return result.viewport.endsWith(SHELL_PROMPT) + || result.viewport.endsWith(`${SHELL_PROMPT}\r\n`) + || result.viewport.endsWith(`${SHELL_PROMPT}\n`) +} + +function partialOutput( + snapshot: RetainedOutput, + marker: CommandMarkers, + wrapper: string, + fallback: string, + fallbackTruncated = false, +): CapturedOutput { + const startMarker = snapshot.text.lastIndexOf(marker.start) + if (startMarker >= 0) { + return { + text: stripPrompt(snapshot.text.slice(startMarker + marker.start.length).replace(/^\r?\n/, '')), + incomplete: false, + } + } + const fallbackStart = fallback.lastIndexOf(marker.start) + const afterStart = fallbackStart < 0 + ? fallback + : fallback.slice(fallbackStart + marker.start.length).replace(/^\r?\n/, '') + const fallbackEnd = afterStart.lastIndexOf(marker.end) + const beforeEnd = fallbackEnd < 0 ? afterStart : afterStart.slice(0, fallbackEnd) + return { + text: stripPrompt(beforeEnd.replaceAll(SHELL_PROMPT, '').replaceAll(wrapper, '')), + incomplete: fallbackTruncated || fallbackStart < 0, + } +} + +async function pause(): Promise { + await new Promise(resolve => setTimeout(resolve, POLL_INTERVAL_MS)) +} + +function nextScrollbackOffset(page: TerminalReadResult, offset: number): number | undefined { + if (page.text.length === 0 || page.lineEnd <= offset) return undefined + return page.lineEnd +} + +function retainedScrollback( + ctx: Context, + owner: Agent, + id: TerminalSessionId, + latest = ctx.terminals.read(owner, id, { offset: 0, count: SCROLLBACK_PAGE_LINES }), +): RetainedOutput { + const pages: string[] = latest.text.length === 0 ? [] : [latest.text] + let offset = latest.lineEnd + let truncated = latest.truncated + while (true) { + if (offset >= latest.totalLines) break + const page = ctx.terminals.read(owner, id, { offset, count: SCROLLBACK_PAGE_LINES }) + truncated ||= page.truncated + if (page.text.length > 0) pages.unshift(page.text) + const next = nextScrollbackOffset(page, offset) + if (next === undefined || next >= page.totalLines) break + offset = next + } + return { text: pages.join('\n'), truncated } +} + +function renderCaptured(output: CapturedOutput, maxOutputChars: number): string { + const rendered = maybeTruncate(output.text, maxOutputChars, output.incomplete) + const withPrefix = output.incomplete && output.text.length > 0 + ? LOST_PREFIX_MESSAGE + rendered + : rendered + const marker = output.exitCode !== undefined && output.exitCode !== 0 + ? `[exit code: ${output.exitCode}]` + : undefined + return appendStatusMarker(withPrefix, marker) +} + +function appendStatusMarker(content: string, marker: string | undefined): string { + if (marker === undefined) return content + return content.length === 0 ? marker : `${content}\n${marker}` +} + +function renderShellExitStatus( + content: string, + exitCode: number | null, + signal: NodeJS.Signals | null, +): string { + const marker = signal !== null + ? `[shell killed by signal: ${signal}]` + : exitCode !== null + ? `[shell exited: code ${exitCode}]` + : '[shell exited]' + return appendStatusMarker(content, marker) +} + +/** + * Render the exited-session result, reset the owner's shell, and reset the + * message that tells the model the next call starts fresh. + * @param shells - the owner-scoped registry to reset. + * @param status - the exited session status (exit code and signal). + * @returns the complete model-facing result. + */ +async function respondToSessionExit( + ctx: Context, + shells: PersistentShells, + owner: Agent, + id: TerminalSessionId, + status: { exitCode: number | null; signal: NodeJS.Signals | null }, + marker: CommandMarkers, + wrapped: string, + fallback: string, + fallbackTruncated: boolean, + config: ResolvedConfig, +): Promise { + const snapshot = retainedScrollback(ctx, owner, id) + await shells.reset(owner, 'persistent pwsh shell exited') + return [ + renderShellExitStatus( + renderCaptured(partialOutput(snapshot, marker, wrapped, fallback, fallbackTruncated), config.maxOutputChars), + status.exitCode, + status.signal, + ), + SHELL_RESET_MESSAGE, + ].filter(part => part.length > 0).join('\n') +} + +/** + * The pwsh prompt function that overrides the backend bootstrap value with + * this tool's own prompt. `[char]27`/`[char]7` build the OSC bytes at runtime + * because raw ESC characters in submitted input are unreliable under + * PSReadLine. + */ +const PWSH_PROMPT_SETUP = + "function prompt { [Console]::Write([char]27 + ']133;D;' + [int]$LASTEXITCODE + [char]7); '" + SHELL_PROMPT + "' }" + +function persistentShells(ctx: Context, config: ResolvedConfig): PersistentShells { + const pending = new WeakMap>() + const live = new Map() + const creating = new Set>() + const ownerCleanupInstalled = new WeakSet() + const lifecycle = new AbortController() + + const close = async (owner: Agent, id: TerminalSessionId, reason: string): Promise => { + if (!ctx.terminals.list(owner).some(snapshot => snapshot.sessionId === id)) return + await ctx.terminals.kill(owner, id, reason) + } + + ctx.effect(() => async () => { + lifecycle.abort(new Error('tool-pwsh-persistent disposed during shell creation')) + await Promise.allSettled([...creating]) + const closing = [...live].map(async ([owner, id]) => { await close(owner, id, 'tool-pwsh-persistent disposed') }) + await Promise.all(closing) + live.clear() + }, 'tool-pwsh-persistent shell cleanup') + + const reset = async (owner: Agent, reason: string): Promise => { + pending.delete(owner) + const id = live.get(owner) + live.delete(owner) + if (id !== undefined) await close(owner, id, reason) + } + + const get = (owner: Agent, signal: AbortSignal): Promise => { + const existing = pending.get(owner) + if (existing !== undefined) return existing + const combinedSignal = AbortSignal.any([signal, lifecycle.signal]) + const creation = (async () => { + try { + const cwd = owner.session.header.cwd + const spawned = await ctx.terminals.spawn(owner, { + type: config.backendType, + ...cwd === undefined ? {} : { cwd }, + }, combinedSignal) + live.set(owner, spawned.sessionId) + if (!ownerCleanupInstalled.has(owner)) { + ownerCleanupInstalled.add(owner) + owner.ctx.effect(() => () => { + pending.delete(owner) + live.delete(owner) + }, 'tool-pwsh-persistent owner cache cleanup') + } + const setup = ctx.terminals.startSend(owner, spawned.sessionId, { + text: PWSH_PROMPT_SETUP, + submit: true, + signal: combinedSignal, + }) + const result = await setup.done + if (result.sessionStatus.kind === 'exited' || result.waitReason === 'timeout') { + throw new Error('persistent pwsh shell did not accept initialization') + } + return spawned.sessionId + } catch (error: unknown) { + await reset(owner, 'persistent pwsh initialization failed') + throw error + } + })() + const tracked = creation.finally(() => { + creating.delete(tracked) + }) + creating.add(tracked) + pending.set(owner, tracked) + return tracked + } + + return { get, reset } +} + +async function executeCommand( + ctx: Context, + shells: PersistentShells, + owner: Agent, + command: string, + config: ResolvedConfig, + upstream: AbortSignal, +): Promise { + using commandDeadline = deadline(upstream, config.timeoutMs, TIMEOUT_CODE) + const id = await shells.get(owner, commandDeadline.signal) + const marker = markers() + const wrapped = wrapCommand(command, marker) + let first = true + let fallback = '' + let fallbackTruncated = false + + while (true) { + // The shell may flip to exited between iterations (a fast `exit` can + // settle the previous send while its exit event is still in flight, and + // the echoed wrapper can then carry a marker end without status digits); + // re-observing status before the next send closes that gap. + const status = ctx.terminals.list(owner).find(session => session.sessionId === id)?.status + if (status?.kind === 'exited') { + return await respondToSessionExit( + ctx, shells, owner, id, status, marker, wrapped, fallback, fallbackTruncated, config, + ) + } + let operation + let result + try { + operation = ctx.terminals.startSend(owner, id, { + text: first ? wrapped : '', + submit: first, + signal: commandDeadline.signal, + }) + first = false + result = await operation.done + } catch (error: unknown) { + await shells.reset(owner, 'persistent pwsh send failed') + throw error + } + const incremental = operation.readOutput() + fallback = incremental.delta.length > 0 ? fallback + incremental.delta : result.viewport + fallbackTruncated ||= incremental.truncated || result.truncated + const latest = ctx.terminals.read(owner, id, { offset: 0, count: SCROLLBACK_PAGE_LINES }) + const timedOut = timeoutOf(commandDeadline.signal, TIMEOUT_CODE) + if (timedOut !== undefined) { + const snapshot = retainedScrollback(ctx, owner, id, latest) + const partial = renderCaptured( + partialOutput(snapshot, marker, wrapped, fallback, fallbackTruncated), + config.maxOutputChars, + ) + await shells.reset(owner, 'persistent pwsh command timed out') + return [ + // TODO: Report a timeout only; this signal does not establish an OOM. + `Your command timed out after ${Math.round(timedOut.timeoutMs / 1000)} seconds or experienced an OOM error. Below is partial output:`, + partial, + SHELL_RESET_MESSAGE, + ].join('\n') + } + if (commandDeadline.signal.aborted) { + await shells.reset(owner, 'persistent pwsh command aborted') + commandDeadline.signal.throwIfAborted() + } + if (latest.text.includes(marker.end)) { + const complete = commandOutput(retainedScrollback(ctx, owner, id, latest), marker, wrapped) + if (complete !== undefined) return renderCaptured(complete, config.maxOutputChars) + } + if (result.sessionStatus.kind === 'exited') { + return await respondToSessionExit( + ctx, shells, owner, id, result.sessionStatus, marker, wrapped, fallback, fallbackTruncated, config, + ) + } + if (promptCompleted(result)) { + const snapshot = retainedScrollback(ctx, owner, id, latest) + return renderCaptured( + partialOutput(snapshot, marker, wrapped, fallback, fallbackTruncated), + config.maxOutputChars, + ) + } + await pause() + } +} + +/** + * Register the model-facing persistent `pwsh` tool. + * @param ctx - plugin context carrying tools and the owner-scoped PTY service. + * @param config - selected PTY backend and command deadline. + */ +function registerPersistentPwsh(ctx: Context, config: ResolvedConfig): void { + const shells = persistentShells(ctx, config) + const queues = new WeakMap>() + + const serialized = async (owner: Agent, operation: () => Promise): Promise => { + const prior = queues.get(owner) ?? Promise.resolve() + const run = prior.then(operation, operation) + const tail = run.then(() => undefined, () => undefined) + queues.set(owner, tail) + try { + return await run + } finally { + if (queues.get(owner) === tail) queues.delete(owner) + } + } + + ctx.tools.register(defineTool({ + name: 'pwsh', + description: config.description, + parameters: { + command: { + type: 'string', + required: true, + description: 'The PowerShell command to run. Relative path is preferred in the command.', + }, + }, + output: { + schema: { type: 'string' }, + render: (_args, value) => [{ type: 'text', text: value }], + }, + async execute(args, exec) { + if (args.command.trim().length === 0) throw new Error('command must be a non-empty string') + const owner = exec.agent + if (owner === undefined) throw new Error('pwsh requires an owning agent session') + return serialized(owner, async () => { + exec.signal.throwIfAborted() + return executeCommand(ctx, shells, owner, args.command, config, exec.signal) + }) + }, + presentCall: args => ({ card: 'terminal', title: args.command }), + })) +} + +export const name = 'tool-pwsh-persistent' +export const inject = ['tools', 'terminals'] + +/** Configuration for the persistent pwsh tool. */ +export interface Config { + /** PTY backend used for each owner-isolated persistent shell (default `shell`). */ + backendType?: string + /** Wall-clock limit for one command (default 300000). */ + timeoutMs?: number + /** Maximum returned command-output characters before clipping (default 16000). */ + maxOutputChars?: number + /** Model-facing tool description; deployments may describe their environment. */ + description?: string +} + +/** Runtime configuration schema for the persistent pwsh tool. */ +export const Config: z = z.object({ + backendType: z.string().default('shell'), + timeoutMs: z.number().default(300_000), + maxOutputChars: z.number().default(16_000), + description: z.string().default(DEFAULT_DESCRIPTION), +}) + +/** Register one owner-scoped persistent `pwsh` tool. */ +export function apply(ctx: Context, config: Config): void { + const resolved: ResolvedConfig = { + backendType: config.backendType ?? 'shell', + timeoutMs: config.timeoutMs ?? 300_000, + maxOutputChars: config.maxOutputChars ?? 16_000, + description: config.description ?? DEFAULT_DESCRIPTION, + } + if (resolved.backendType.trim().length === 0) { + throw new Error('tool-pwsh-persistent: backendType must be non-empty') + } + if (!Number.isSafeInteger(resolved.timeoutMs) || resolved.timeoutMs <= 0) { + throw new Error('tool-pwsh-persistent: timeoutMs must be a positive safe integer') + } + if (!Number.isSafeInteger(resolved.maxOutputChars) || resolved.maxOutputChars <= 0) { + throw new Error('tool-pwsh-persistent: maxOutputChars must be a positive safe integer') + } + if (resolved.description.trim().length === 0) { + throw new Error('tool-pwsh-persistent: description must be non-empty') + } + registerPersistentPwsh(ctx, resolved) +} + +/* jscpd:ignore-end */ diff --git a/packages/shell/tool-pwsh-persistent/src/invariant.ts b/packages/shell/tool-pwsh-persistent/src/invariant.ts new file mode 100644 index 0000000000..6f436019f2 --- /dev/null +++ b/packages/shell/tool-pwsh-persistent/src/invariant.ts @@ -0,0 +1,31 @@ +/** + * Package-owned invariant companion for `@deepseek-ai/dsh-tool-pwsh-persistent`. + * @module @deepseek-ai/dsh-tool-pwsh-persistent/invariant + */ + +/* jscpd:ignore-start */ +import type { Context } from '@deepseek-ai/cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-tool-pwsh-persistent' + +/** Cordis companion plugin name. */ +export const name = 'tool-pwsh-persistent-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: the adapter's private owner-to-shell cache has no + * observable event or data relation. Lifecycle tests prove its cleanup without + * adding a public API solely for an invariant. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/shell/tool-pwsh-persistent/tests/loader-composition.spec.ts b/packages/shell/tool-pwsh-persistent/tests/loader-composition.spec.ts new file mode 100644 index 0000000000..1a95d7fe23 --- /dev/null +++ b/packages/shell/tool-pwsh-persistent/tests/loader-composition.spec.ts @@ -0,0 +1,167 @@ +import { spawnSync } from 'node:child_process' +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { pathToFileURL } from 'node:url' +import { afterEach, describe, expect, it } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import Loader from '@deepseek-ai/cordis-plugin-loader' +import Include from '@deepseek-ai/cordis-plugin-include' +import { CallId } from '@deepseek-ai/dsh-llm' +import { Session, SessionId } from '@deepseek-ai/dsh-session' +import AgentRegistry, { Inbox } from '@deepseek-ai/dsh-agent' +import type { Agent } from '@deepseek-ai/dsh-agent' +import TerminalSessionService from '@deepseek-ai/dsh-terminal' +import * as TerminalBash from '@deepseek-ai/dsh-terminal-bash' +import SandboxProvider from '@deepseek-ai/dsh-sandbox' +import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' +import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import { resolvePwshPath } from '@deepseek-ai/dsh-pwsh-local/src/resolve.ts' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry from '@deepseek-ai/dsh-tools' +import * as ToolPwshPersistent from '@deepseek-ai/dsh-tool-pwsh-persistent' + +const hasPwsh = spawnSync( + resolvePwshPath(), ['-NoLogo', '-NoProfile', '-NonInteractive', '-Command', '$true'], + { encoding: 'utf8' }, +).status === 0 + +let root: string | undefined +let context: Context | undefined + +afterEach(async () => { + await context?.fiber.dispose() + context = undefined + if (root !== undefined) await rm(root, { recursive: true, force: true }) + root = undefined +}) + +class PassthroughSandbox extends SandboxProvider { + confine(argv: readonly string[], _policy: SandboxPolicy): ConfinedArgv { + return { argv: [...argv], enforcement: 'full', denialSignatures: [], runnerFailureRules: [] } + } +} + +function agent(ctx: Context, cwd: string): Agent { + const id = SessionId('persistent-pwsh-loader-agent') + const scope = ctx.plugin(() => {}) + const session = Session.create(id, [], { version: 0, id, createdAt: 0, cwd }) + const value: Agent = { + id, + options: {}, + session, + inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), + status: 'idle', + ctx: scope.ctx, + send: () => {}, + followup: () => {}, + steer: () => ({ outcome: Promise.resolve({ status: 'rejected' as const }) }), + inject: () => {}, + cancel() {}, + runMaintenance: task => task(new AbortController().signal), + whenIdle: () => Promise.resolve(), + } + ctx.agents.register(value) + return value +} + +function text(result: { content: { type: string; text?: string }[] }): string { + return result.content.filter(block => block.type === 'text').map(block => block.text).join('') +} + +describe.skipIf(!hasPwsh)('persistent pwsh through a real cordis.yml Loader composition', () => { + it('preserves cwd and environment across calls', async () => { + root = await mkdtemp(join(tmpdir(), 'dsh-persistent-pwsh-loader-')) + const configPath = join(root, 'cordis.yml') + await writeFile(configPath, [ + "- name: '@deepseek-ai/dsh-agent'", + "- name: '@deepseek-ai/dsh-system-prompt'", + "- name: '@deepseek-ai/dsh-tools'", + "- name: '@deepseek-ai/dsh-terminal'", + "- name: '@deepseek-ai/dsh-test-sandbox'", + "- name: '@deepseek-ai/dsh-sandbox-policy'", + ' config:', + ' mode: danger-full-access', + ` workspaceRoot: ${JSON.stringify(root)}`, + "- name: '@deepseek-ai/dsh-subprocess-local'", + "- name: '@deepseek-ai/dsh-terminal-bash'", + ' config:', + ' shellDialect: pwsh', + ' pollIntervalMs: 10', + ' exactProbeAfterMs: 20', + ' idleSilenceMs: 300', + ' handoffGraceMs: 300', + ' scrollbackLines: 20000', + ' timeoutMs: 8000', + ' disposeGraceMs: 500', + "- name: '@deepseek-ai/dsh-tool-pwsh-persistent'", + ' config:', + ' timeoutMs: 20000', + '', + ].join('\n')) + + context = new Context() + context.baseUrl = pathToFileURL(root).href + '/' + await context.plugin(Loader) + context.loader.builtins.include = Include + const modules = new Map([ + ['@deepseek-ai/dsh-agent', AgentRegistry], + ['@deepseek-ai/dsh-system-prompt', SystemPrompt], + ['@deepseek-ai/dsh-tools', ToolRegistry], + ['@deepseek-ai/dsh-terminal', TerminalSessionService], + ['@deepseek-ai/dsh-test-sandbox', PassthroughSandbox], + ['@deepseek-ai/dsh-sandbox-policy', SandboxPolicyService], + ['@deepseek-ai/dsh-subprocess-local', LocalSubprocessService], + ['@deepseek-ai/dsh-terminal-bash', TerminalBash], + ['@deepseek-ai/dsh-tool-pwsh-persistent', ToolPwshPersistent], + ]) + context.loader.internal = { + version: 'v2', + async import(specifier: string) { + if (!modules.has(specifier)) throw new Error(`unexpected Loader import: ${specifier}`) + return modules.get(specifier) + }, + } as unknown as NonNullable + await context.loader.create({ name: 'cordis:include', config: { path: pathToFileURL(configPath).href } }) + await context.loader.await() + + const owner = agent(context, root) + const signal = new AbortController().signal + const execute = (id: string, command: string) => context!.tools.execute({ + signal, + callId: CallId(id), + name: 'pwsh', + arguments: { command }, + agent: owner, + }) + + expect(context.tools.schemas().map(schema => schema.name)).toEqual(['pwsh']) + await execute('state', '$env:KEEP = "loader"; New-Item -ItemType Directory -Force -Path nested | Out-Null; Set-Location nested') + const observed = text(await execute('observe', 'Write-Output "cwd=$PWD keep=$env:KEEP"')) + expect(observed).toContain(`cwd=${join(root, 'nested')} keep=loader`) + expect(observed).not.toContain('DSH_PERSISTENT_PWSH') + + const multiline = text(await execute( + 'multiline', + '$value = "line one"\nWrite-Output "${value}:it\'s fine"', + )) + expect(multiline).toBe("line one:it's fine") + expect(multiline).not.toContain('DSH_PERSISTENT_PWSH') + + const hereString = text(await execute( + 'here-string', + "$h = @'\nalpha\nbeta\n'@\nWrite-Output $h", + )) + expect(hereString).toBe('alpha\nbeta') + + const large = text(await execute('large-output', '1..12050 | ForEach-Object { $_ }')) + expect(large.startsWith('1\n2\n3\n')).toBe(true) + expect(large).toContain('') + expect(large).not.toContain('beginning of this command output was dropped') + + const exited = text(await execute('exit', 'exit')) + expect(exited).toContain('next pwsh call starts from the workspace') + expect(text(await execute('after-exit', 'Write-Output "$PWD"'))).toBe(root) + }, 60_000) +}) diff --git a/packages/shell/tool-pwsh-persistent/tests/tools.spec.ts b/packages/shell/tool-pwsh-persistent/tests/tools.spec.ts new file mode 100644 index 0000000000..8d76abd3a6 --- /dev/null +++ b/packages/shell/tool-pwsh-persistent/tests/tools.spec.ts @@ -0,0 +1,636 @@ +import { afterEach, describe, expect, it } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import { CallId } from '@deepseek-ai/dsh-llm' +import { Session, SessionId } from '@deepseek-ai/dsh-session' +import AgentRegistry, { Inbox } from '@deepseek-ai/dsh-agent' +import type { Agent } from '@deepseek-ai/dsh-agent' +import TerminalSessionService from '@deepseek-ai/dsh-terminal' +import type { + TerminalBackend, + TerminalBackendSession, + TerminalReadRequest, + TerminalSendOperation, + TerminalSendRequest, + TerminalSessionStatus, + TerminalSignal, + TerminalWaitReason, +} from '@deepseek-ai/dsh-terminal' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry from '@deepseek-ai/dsh-tools' +import * as ToolPwshPersistent from '@deepseek-ai/dsh-tool-pwsh-persistent' + +const contexts: Context[] = [] +let callNumber = 0 + +afterEach(async () => { + for (const ctx of contexts.splice(0)) await ctx.fiber.dispose() +}) + +function agent(ctx: Context, cwd: string | undefined): Agent { + const id = SessionId(`persistent-pwsh-owner-${callNumber}`) + const scope = ctx.plugin(() => {}) + const session = Session.create(id, [], { + version: 0, + id, + createdAt: 0, + ...cwd === undefined ? {} : { cwd }, + }) + const value: Agent = { + id, + options: {}, + session, + inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), + status: 'idle', + ctx: scope.ctx, + send: () => {}, + followup: () => {}, + steer: () => ({ outcome: Promise.resolve({ status: 'rejected' as const }) }), + inject: () => {}, + cancel() {}, + runMaintenance: task => task(new AbortController().signal), + whenIdle: () => Promise.resolve(), + } + ctx.agents.register(value) + return value +} + +function text(result: { content: { type: string; text?: string }[] }): string { + return result.content.filter(block => block.type === 'text').map(block => block.text).join('') +} + +function call( + ctx: Context, + owner: Agent | undefined, + command: string, + signal = new AbortController().signal, +) { + return ctx.tools.execute({ + signal, + callId: CallId(`persistent-pwsh-${++callNumber}`), + name: 'pwsh', + arguments: { command }, + ...owner === undefined ? {} : { agent: owner }, + }) +} + +type StubMode = + | 'normal' + | 'prompt-only' + | 'prompt-crlf' + | 'empty-read' + | 'stalled-read' + | 'exit' + | 'signal-exit' + | 'unknown-exit' + | 'wait-for-abort' + | 'end-on-abort' + | 'idle-then-normal' + | 'large' + | 'nonzero' + | 'torn-status' + | 'finish-torn-status' + | 'end-only' + | 'init-exit' + | 'init-timeout' + | 'spawn-error' + | 'send-error' + | 'prompt-after-idle' + | 'incremental-fallback' + | 'empty-page-after-latest' + | 'paged-scrollback' + | 'with-echo' + | 'exit-after-send' + | 'prompt-collision' + +const START_PATTERN = /__DSH_PERSISTENT_PWSH_START_[^_]+(?:-[^_]+)*__/ +const END_PATTERN = /__DSH_PERSISTENT_PWSH_END_[^:]+:/ + +class StubTerminalSession implements TerminalBackendSession { + readonly motd = '__DSH_PERSISTENT_PWSH_PROMPT__ ' + readonly pid = 123 + statusValue: TerminalSessionStatus = { kind: 'running' } + scrollback = this.motd + closed: string[] = [] + mode: StubMode + sends = 0 + pendingText = '' + historyTruncated = false + throwOnSend = false + + constructor(mode: StubMode) { + this.mode = mode + } + + startSend(request: TerminalSendRequest): TerminalSendOperation { + this.sends += 1 + if (request.text.startsWith('function prompt')) { + if (this.mode === 'init-exit') { + this.statusValue = { kind: 'exited', exitCode: 1, signal: null } + return this.operation(Promise.resolve(this.result('', 'session_exit'))) + } + if (this.mode === 'init-timeout') { + return this.operation(Promise.resolve(this.result('', 'timeout'))) + } + return this.operation(Promise.resolve(this.result(this.motd, 'stdin_read'))) + } + if (this.mode === 'send-error') throw new Error('stub send failed') + if (this.throwOnSend) throw new Error('PTY session has exited') + if (this.mode === 'wait-for-abort' || this.mode === 'end-on-abort') { + const done = new Promise>((resolve) => { + request.signal?.addEventListener('abort', () => { + const start = START_PATTERN.exec(request.text)?.[0] + const end = END_PATTERN.exec(request.text)?.[0] + const output = this.mode === 'end-on-abort' + ? `${start ?? ''}\ninterrupted\n${end ?? ''}130\n${this.motd}` + : 'partial output' + this.scrollback += output + resolve(this.result(output, 'stdin_read')) + }, { once: true }) + }) + return this.operation(done) + } + if (this.mode === 'idle-then-normal') { + this.mode = 'normal' + this.pendingText = request.text + return this.operation(Promise.resolve(this.result('', 'inferred_idle'))) + } + if (this.mode === 'prompt-after-idle') { + if (request.text.length > 0) { + const start = START_PATTERN.exec(request.text)?.[0] + const output = `${start ?? ''}\npartial syntax output\n` + this.scrollback += output + return this.operation(Promise.resolve(this.result(output, 'inferred_idle'))) + } + const output = `pwsh: syntax error\n${this.motd}` + this.scrollback += output + return this.operation(Promise.resolve(this.result(output, 'stdin_read'))) + } + if (this.mode === 'prompt-only' || this.mode === 'prompt-crlf') { + const newline = this.mode === 'prompt-crlf' ? '\r\n' : '\n' + const output = `pwsh: syntax error${newline}${this.motd}${newline}` + this.scrollback += output + return this.operation(Promise.resolve(this.result(output, 'stdin_read'))) + } + const sent = request.text.length > 0 ? request.text : this.pendingText + this.pendingText = '' + const start = START_PATTERN.exec(sent)?.[0] + const end = END_PATTERN.exec(sent)?.[0] + if (this.mode === 'with-echo') { + // The PSReadLine echo renders the submitted wrapper before the real + // markers; the tool must strip it from the captured result. + const output = `${sent}\n${start ?? ''}\nhello from stub\n${end ?? ''}0\n${this.motd}` + this.scrollback += output + return this.operation(Promise.resolve(this.result(output, 'stdin_read'))) + } + if (this.mode === 'exit-after-send') { + // A fast `exit` settles the send with an echoed wrapper (marker end, + // no status digits) while the exit event is still in flight; the shell + // flips to exited before the tool's next poll, exactly like the real + // ConPTY backend. The tool must re-observe status instead of sending. + const output = `${sent}\n${start ?? ''}\n` + this.scrollback += output + const settled = this.result(output, 'inferred_idle') + this.statusValue = { kind: 'exited', exitCode: 9, signal: null } + this.throwOnSend = true + return this.operation(Promise.resolve(settled)) + } + if (this.mode === 'incremental-fallback') { + const incremental = `${start ?? ''}\nincrement\n${this.motd}` + return this.operation(Promise.resolve(this.result(this.motd, 'stdin_read')), incremental) + } + if (this.mode === 'torn-status') { + const output = `${start ?? ''}\nhello from stub\n${end ?? ''}` + this.scrollback += output + this.mode = 'finish-torn-status' + return this.operation(Promise.resolve(this.result(output, 'inferred_idle'))) + } + if (this.mode === 'finish-torn-status') { + const output = `7\n${this.motd}` + this.scrollback += output + return this.operation(Promise.resolve(this.result(output, 'stdin_read'))) + } + if (this.mode === 'end-only') { + const output = `recovered output\n${end ?? ''}0\n${this.motd}` + this.scrollback += output + return this.operation(Promise.resolve(this.result(output, 'stdin_read'))) + } + const commandOutput = this.mode === 'large' + ? 'x'.repeat(100) + : this.mode === 'nonzero' ? '' + : this.mode === 'prompt-collision' ? this.motd + : 'hello from stub' + const exitCode = this.mode === 'nonzero' ? 7 : 0 + const output = `${start ?? ''}\n${commandOutput}\n${end ?? ''}${exitCode}\n${this.motd}` + this.scrollback += output + if (this.mode === 'exit' || this.mode === 'signal-exit' || this.mode === 'unknown-exit') { + const exitedOutput = `${start ?? ''}\nhello from stub\n` + this.scrollback = this.scrollback.slice(0, -output.length) + exitedOutput + this.statusValue = this.mode === 'signal-exit' + ? { kind: 'exited', exitCode: null, signal: 'SIGTERM' } + : this.mode === 'exit' + ? { kind: 'exited', exitCode: 9, signal: null } + : { kind: 'exited', exitCode: null, signal: null } + return this.operation(Promise.resolve(this.result(exitedOutput, 'session_exit'))) + } + return this.operation(Promise.resolve(this.result(output, 'stdin_read'))) + } + + read(request: TerminalReadRequest) { + if (this.mode === 'empty-read') { + return { text: '', totalLines: 0, lineBegin: 0, lineEnd: 0, truncated: false } + } + if (this.mode === 'stalled-read') { + return { text: 'stalled', totalLines: 1, lineBegin: 0, lineEnd: 0, truncated: false } + } + if (this.mode === 'empty-page-after-latest' && (request.offset ?? 0) > 0) { + return { text: '', totalLines: 2, lineBegin: 1, lineEnd: 1, truncated: false } + } + const lines = this.scrollback.split('\n') + if (this.mode === 'paged-scrollback') { + const offset = request.offset ?? 0 + const end = lines.length - offset + const start = Math.max(0, end - 3) + const returnedLines = end - start + return { + text: lines.slice(start, end).join('\n'), + totalLines: lines.length, + lineBegin: offset, + lineEnd: offset + returnedLines, + truncated: this.historyTruncated, + } + } + return { + text: this.scrollback, + totalLines: this.mode === 'empty-page-after-latest' ? lines.length + 1 : lines.length, + lineBegin: 0, + lineEnd: this.mode === 'empty-page-after-latest' ? 1 : lines.length, + truncated: this.historyTruncated, + } + } + + signal(_signal: TerminalSignal) { + return Promise.resolve({ delivered: true as const, targetPgid: 123 }) + } + + status() { + return this.statusValue + } + + async close(reason: string) { + this.closed.push(reason) + this.statusValue = { kind: 'exited', exitCode: 0, signal: null } + } + + private result(viewport: string, waitReason: TerminalWaitReason) { + return { viewport, waitReason, sessionStatus: this.statusValue, truncated: false } + } + + private operation(done: Promise>, delta = ''): TerminalSendOperation { + return { + done, + readOutput: () => ({ delta, truncated: false }), + cancel: () => false, + } + } +} + +function stubBackend(initialMode: StubMode = 'normal') { + const sessions: StubTerminalSession[] = [] + const backend: TerminalBackend = { + type: 'stub', + async spawn() { + if (initialMode === 'spawn-error') throw new Error('stub spawn failed') + const session = new StubTerminalSession(initialMode) + sessions.push(session) + return session + }, + } + return { backend, sessions } +} + +async function setup( + config: ToolPwshPersistent.Config = { backendType: 'stub' }, + initialMode: StubMode = 'normal', +) { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(TerminalSessionService) + const stub = stubBackend(initialMode) + ctx.terminals.registerBackend(stub.backend) + const fiber = await ctx.plugin(ToolPwshPersistent, config) + return { ctx, stub, fiber, owner: agent(ctx, '/workspace') } +} + +describe('tool-pwsh-persistent', () => { + it('registers a configurable schema and reuses one owner shell', async () => { + const { ctx, owner, stub, fiber } = await setup({ + backendType: 'stub', + description: 'deployment-specific persistent shell', + }) + const schema = ctx.tools.schemas()[0] + expect(ctx.tools.schemas().map(item => item.name)).toEqual(['pwsh']) + expect(schema?.description).toBe('deployment-specific persistent shell') + expect(schema?.parameters).toMatchObject({ + required: ['command'], + properties: { command: { type: 'string' } }, + }) + expect(ctx.tools.get('pwsh')?.presentCall?.({ command: 'pwd' })) + .toEqual({ card: 'terminal', title: 'pwd' }) + + expect(text(await call(ctx, owner, 'Write-Output one'))).toBe('hello from stub') + expect(text(await call(ctx, owner, 'Write-Output two'))).toBe('hello from stub') + expect(stub.sessions).toHaveLength(1) + expect(stub.sessions[0]?.sends).toBe(3) + + const ownerWithoutCwd = agent(ctx, undefined) + expect(text(await call(ctx, ownerWithoutCwd, 'pwd'))).toBe('hello from stub') + expect(stub.sessions).toHaveLength(2) + + await fiber.dispose() + expect(ctx.tools.schemas()).toEqual([]) + expect(ctx.tools.get('pwsh')).toBeUndefined() + }) + + it('strips the echoed wrapper from captured output', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub' }) + await call(ctx, owner, 'warm up') + stub.sessions[0]!.mode = 'with-echo' + const result = text(await call(ctx, owner, 'Write-Output hi')) + expect(result).toBe('hello from stub') + expect(result).not.toContain('__DSH_PERSISTENT_PWSH_START_') + expect(result).not.toContain('__DSH_PERSISTENT_PWSH_END_') + expect(result).not.toContain('Invoke-Expression') + }) + + it('preserves command output that equals the private shell prompt', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub' }) + await call(ctx, owner, 'warm up') + const session = stub.sessions[0]! + + session.mode = 'prompt-collision' + expect(text(await call(ctx, owner, 'complete prompt collision'))).toBe(session.motd) + }) + + it('reports the exit path when the shell exits between send settlement and the next poll', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub' }) + await call(ctx, owner, 'warm up') + const session = stub.sessions[0]! + session.mode = 'exit-after-send' + + const result = text(await call(ctx, owner, 'exit')) + expect(result).toContain('[shell exited: code 9]') + expect(result).toContain('next pwsh call starts from the workspace') + expect(session.closed).toContain('persistent pwsh shell exited') + + expect(text(await call(ctx, owner, 'Write-Output "$PWD"'))).toBe('hello from stub') + expect(stub.sessions).toHaveLength(2) + }) + + it('handles inferred idle, prompt fallback, shell exit, clipping, and cleanup', async () => { + const { ctx, owner, stub, fiber } = await setup({ + backendType: 'stub', + maxOutputChars: 10, + }) + await call(ctx, owner, 'warm up') + const session = stub.sessions[0]! + + session.mode = 'idle-then-normal' + expect(text(await call(ctx, owner, 'silent then complete'))).toContain('hello from') + + session.mode = 'incremental-fallback' + session.scrollback = '' + expect(text(await call(ctx, owner, 'incremental fallback'))).toBe('increment') + + session.mode = 'prompt-only' + const promptFallback = text(await call(ctx, owner, 'bad {')) + expect(promptFallback).toContain('pwsh: synt') + expect(promptFallback).not.toContain('DSH_PERSISTENT_PWSH_PROMPT') + + session.mode = 'prompt-crlf' + session.scrollback = '' + const crlfPromptFallback = text(await call(ctx, owner, 'bad {')) + expect(crlfPromptFallback).toContain('pwsh: synt') + expect(crlfPromptFallback).not.toContain('DSH_PERSISTENT_PWSH_PROMPT') + + session.mode = 'end-only' + session.scrollback = '' + const missingStart = text(await call(ctx, owner, 'recover marker')) + expect(missingStart).toContain('recovered') + expect(missingStart).toContain('beginning of this command output was dropped') + expect(missingStart).toContain('') + + session.mode = 'large' + expect(text(await call(ctx, owner, 'large'))).toContain('') + + session.mode = 'nonzero' + expect(text(await call(ctx, owner, 'false'))).toBe('[exit code: 7]') + + session.mode = 'exit' + const exited = text(await call(ctx, owner, 'exit')) + expect(exited).toContain('hello from') + expect(exited).toContain('[shell exited: code 9]') + expect(exited).not.toContain('[exit code: 9]') + expect(exited).toContain('next pwsh call starts from the workspace') + expect(session.closed).toContain('persistent pwsh shell exited') + + await call(ctx, owner, 'new shell') + expect(stub.sessions).toHaveLength(2) + const replacement = stub.sessions[1]! + replacement.mode = 'signal-exit' + expect(text(await call(ctx, owner, 'kill shell'))) + .toContain('[shell killed by signal: SIGTERM]') + + await call(ctx, owner, 'another shell') + expect(stub.sessions).toHaveLength(3) + const externallyClosed = ctx.terminals.list(owner)[0]?.sessionId + expect(externallyClosed).toBeDefined() + await ctx.terminals.kill(owner, externallyClosed!, 'external cleanup') + await fiber.dispose() + expect(stub.sessions[2]?.closed).toEqual(['external cleanup']) + }) + + it('waits for status digits after a torn completion marker', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub', maxOutputChars: 1_000 }) + await call(ctx, owner, 'warm up') + stub.sessions[0]!.mode = 'torn-status' + stub.sessions[0]!.scrollback = '' + + expect(text(await call(ctx, owner, 'torn status'))).toBe('hello from stub\n[exit code: 7]') + }) + + it('reports a shell exit when the backend has no code or signal', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub' }) + await call(ctx, owner, 'warm up') + stub.sessions[0]!.mode = 'unknown-exit' + + expect(text(await call(ctx, owner, 'exit without status'))).toContain('[shell exited]') + }) + + it('marks a short missing-prefix result and tolerates exhausted scrollback pages', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub', maxOutputChars: 1_000 }) + await call(ctx, owner, 'warm up') + const session = stub.sessions[0]! + + session.mode = 'end-only' + session.scrollback = '' + expect(text(await call(ctx, owner, 'missing start'))) + .toContain('beginning of this command output was dropped') + + session.mode = 'empty-read' + expect(text(await call(ctx, owner, 'empty page'))).toContain('hello from stub') + + session.mode = 'stalled-read' + expect(text(await call(ctx, owner, 'stalled page'))).toContain('hello from stub') + + session.mode = 'empty-page-after-latest' + expect(text(await call(ctx, owner, 'empty continuation page'))).toContain('hello from stub') + }) + + it('assembles retained output across backward scrollback pages', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub', maxOutputChars: 1_000 }) + await call(ctx, owner, 'warm up') + const session = stub.sessions[0]! + session.mode = 'paged-scrollback' + session.scrollback = 'older one\nolder two\nolder three\nolder four\n' + + expect(text(await call(ctx, owner, 'paged output'))).toBe('hello from stub') + }) + + it('sanitizes a prompt fallback reached after multiple polling rounds', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub', maxOutputChars: 1_000 }) + await call(ctx, owner, 'warm up') + const session = stub.sessions[0]! + session.mode = 'prompt-after-idle' + session.scrollback = '' + const result = text(await call(ctx, owner, 'bad {')) + expect(result).toContain('partial syntax output') + expect(result).toContain('pwsh: syntax error') + expect(result).not.toContain('DSH_PERSISTENT_PWSH_PROMPT') + expect(result).not.toContain('DSH_PERSISTENT_PWSH_START') + }) + + it('does not attribute old scrollback truncation to a complete current command', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub', maxOutputChars: 1_000 }) + await call(ctx, owner, 'warm up') + stub.sessions[0]!.historyTruncated = true + const result = text(await call(ctx, owner, 'short command')) + expect(result).toBe('hello from stub') + expect(result).not.toContain('') + expect(result).not.toContain('beginning of this command output was dropped') + }) + + it('closes a timed-out shell and reports bounded partial output', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub', timeoutMs: 10 }) + await call(ctx, owner, 'warm up') + stub.sessions[0]!.mode = 'wait-for-abort' + const result = await call(ctx, owner, 'hang') + expect(text(result)).toContain('timed out after 0 seconds or experienced an OOM error') + expect(text(result)).toContain('partial output') + expect(text(result)).toContain('next pwsh call starts from the workspace') + expect(stub.sessions[0]?.closed).toContain('persistent pwsh command timed out') + }) + + it.each(['wait-for-abort', 'end-on-abort'] as const)( + 'cancels %s work, resets the shell, and releases a queued call', + async (mode) => { + const { ctx, owner, stub } = await setup({ backendType: 'stub', timeoutMs: 5_000 }) + await call(ctx, owner, 'warm up') + stub.sessions[0]!.mode = mode + const controller = new AbortController() + const cancelled = call(ctx, owner, 'hang', controller.signal) + const queued = call(ctx, owner, 'after cancellation') + setTimeout(() => { + controller.abort(new Error('caller stopped')) + }, 5) + + expect((await cancelled).isError).toBe(true) + expect(text(await queued)).toBe('hello from stub') + expect(stub.sessions[0]?.closed).toContain('persistent pwsh command aborted') + expect(stub.sessions).toHaveLength(2) + }, + ) + + it.each(['init-exit', 'init-timeout'] as const)( + 'fails initialization and closes the unusable shell for %s', + async (mode) => { + const { ctx, owner, stub } = await setup({ backendType: 'stub' }, mode) + expect((await call(ctx, owner, 'pwd')).isError).toBe(true) + expect(stub.sessions[0]?.closed).toContain('persistent pwsh initialization failed') + }, + ) + + it('clears a failed spawn without trying to close an unpublished shell', async () => { + const { ctx, owner, stub } = await setup({ backendType: 'stub' }, 'spawn-error') + expect((await call(ctx, owner, 'pwd')).isError).toBe(true) + expect(stub.sessions).toHaveLength(0) + }) + + it('resets a cached shell after startSend fails', async () => { + const { ctx, owner, stub } = await setup() + await call(ctx, owner, 'warm up') + stub.sessions[0]!.mode = 'send-error' + expect((await call(ctx, owner, 'fails')).isError).toBe(true) + expect(stub.sessions[0]?.closed).toContain('persistent pwsh send failed') + expect(text(await call(ctx, owner, 'recovers'))).toBe('hello from stub') + expect(stub.sessions).toHaveLength(2) + }) + + it('cancels and awaits a pending shell spawn when the plugin is disposed', async () => { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(TerminalSessionService) + const spawnStarted = Promise.withResolvers() + const spawnAborted = Promise.withResolvers() + ctx.terminals.registerBackend({ + type: 'slow', + spawn: spec => new Promise((_resolve, reject) => { + spawnStarted.resolve(undefined) + spec.signal?.addEventListener('abort', () => { + spawnAborted.resolve(undefined) + const reason: unknown = spec.signal?.reason + reject(reason instanceof Error + ? reason + : new Error('slow PTY spawn aborted', { cause: reason })) + }, { once: true }) + }), + }) + const fiber = await ctx.plugin(ToolPwshPersistent, { backendType: 'slow' }) + const owner = agent(ctx, '/workspace') + const running = call(ctx, owner, 'pwd') + await spawnStarted.promise + await fiber.dispose() + await spawnAborted.promise + expect((await running).isError).toBe(true) + expect(ctx.terminals.list(owner)).toEqual([]) + }) + + it('rejects invalid config and invalid calls', async () => { + const { ctx, owner, stub } = await setup() + expect((await call(ctx, undefined, 'pwd')).isError).toBe(true) + expect(text(await call(ctx, owner, ' '))).toContain('command must be a non-empty string') + + const controller = new AbortController() + controller.abort(new Error('caller stopped')) + expect((await call(ctx, owner, 'pwd', controller.signal)).isError).toBe(true) + expect(stub.sessions).toHaveLength(0) + + expect(() => { + ToolPwshPersistent.apply(new Context(), { backendType: '' }) + }).toThrow('backendType must be non-empty') + expect(() => { + ToolPwshPersistent.apply(new Context(), { timeoutMs: 0 }) + }).toThrow('timeoutMs must be a positive safe integer') + expect(() => { + ToolPwshPersistent.apply(new Context(), { maxOutputChars: 0 }) + }).toThrow('maxOutputChars must be a positive safe integer') + expect(() => { + ToolPwshPersistent.apply(new Context(), { description: ' ' }) + }).toThrow('description must be non-empty') + }) +}) diff --git a/packages/shell/tool-pwsh-persistent/tsconfig.json b/packages/shell/tool-pwsh-persistent/tsconfig.json new file mode 100644 index 0000000000..42ce584d6a --- /dev/null +++ b/packages/shell/tool-pwsh-persistent/tsconfig.json @@ -0,0 +1,17 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { "path": "../../../vendor/cordis" }, + { "path": "../../../vendor/schemastery" }, + { "path": "../../core/agent" }, + { "path": "../../core/tools" }, + { "path": "../../terminal/terminal" }, + { "path": "../../runtime-diagnostics/invariants" }, + { "path": "../../util/timeout" } + ] +} diff --git a/packages/shell/tool-pwsh/README.i18n.yaml b/packages/shell/tool-pwsh/README.i18n.yaml index 23f927d45f..c5dcb85005 100644 --- a/packages/shell/tool-pwsh/README.i18n.yaml +++ b/packages/shell/tool-pwsh/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/shell/tool-pwsh/README.md -README.md: af1de3d84c8815f9875faaa8e2fd6a79dff018c2 -README.zh.md: 1f662094e5d423299ae704efddc6a0b27bdfc128 +README.md: e862fcf0ca85d0ecb0a5fe6cff3ee3c7a8153716 +README.zh.md: 0a66b119ac2bf0386c0686e9e22ddfe1c3bb87b1 diff --git a/packages/shell/tool-pwsh/README.md b/packages/shell/tool-pwsh/README.md index af1de3d84c..e862fcf0ca 100644 --- a/packages/shell/tool-pwsh/README.md +++ b/packages/shell/tool-pwsh/README.md @@ -121,6 +121,6 @@ Append-only; newly visible content follows the reusable request prefix and does ## Known Limitations and Deferred Work - **Language mode and named-pipe capture under the Windows sandbox** — under the [Windows ACL sandbox](../../sandbox/sandbox-windows-acl/README.md), read-only pwsh starts in ConstrainedLanguage because its temp write denial makes PowerShell's AppLocker probe fail closed: `Add-Type`, non-core .NET statics (`[System.IO.*]::`, `[math]::`), COM objects, and reflection fail with "only core types" errors, and the mode cannot be lifted from inside. Workspace-write's private temp lets the probe complete, so it stays in FullLanguage unless host policy says otherwise. Both confined modes deny named-pipe opens, so a piped-stdio spawn inside a confined command fails with EPERM. The tool description teaches both contracts to the model; the backend README owns the full limitations. -- **No persistent shell or PTY** — every call starts a fresh `pwsh -Command`; the PTY backends are Linux/macOS-only today, and a Windows ConPTY persistent shell is roadmap work. +- **No persistent shell** — every call starts a fresh `pwsh -Command`; the persistent-shell counterpart is [`@deepseek-ai/dsh-tool-pwsh-persistent`](../tool-pwsh-persistent/README.md), which keeps one owner-scoped pwsh alive across calls on Windows (ConPTY) and POSIX hosts with pwsh. - **PowerShell-dialect contract** — the model must write PowerShell (native paths, `$env:` variables), not bash; there is no dialect translation. - **Session-cwd identity is not canonicalized** — the workdir base is the session header cwd as-is, unlike the bash tool's sandbox-root-canonicalized identity. Under a confining executor the policy's workspace root IS canonicalized (by the shared policy service), so the workdir and the confinement root can diverge when the raw session cwd differs from its canonical form — a parity gap deferred to the shared shell-tool base extraction. diff --git a/packages/shell/tool-pwsh/README.zh.md b/packages/shell/tool-pwsh/README.zh.md index 1f662094e5..0a66b119ac 100644 --- a/packages/shell/tool-pwsh/README.zh.md +++ b/packages/shell/tool-pwsh/README.zh.md @@ -66,7 +66,7 @@ Non-zero exits are reported as `[exit code: N]` markers; investigate failures be #### 模型看到的内容 -模型看到生成的 [`pwsh` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-pwsh)。按 agent 作用域的工具限制可以移除该 agent 的定义。 +模型看到生成的 [`pwsh` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-pwsh)。按 agent 作用域的工具限制可以移除该 agent 的定义。 #### Token 影响 @@ -120,7 +120,7 @@ ack 是固定短行;任务输出按读取有界。 ## 已知限制与暂缓事项 -- **Windows 沙箱下的语言模式与 named-pipe 捕获** — 在 [Windows ACL 沙箱](../../sandbox/sandbox-windows-acl/README.md) 下,read-only pwsh 会以 ConstrainedLanguage 启动,因为临时目录写入被拒绝,导致 PowerShell 的 AppLocker 探针失败并按 fail-closed 处理:`Add-Type`、非核心 .NET 静态调用(`[System.IO.*]::`、`[math]::`)、COM 对象与反射都会以“only core types”错误失败,且该模式无法从内部解除。workspace-write 的私有临时目录使探针得以完成,因此除非主机策略另有规定,否则它保持 FullLanguage。两种受限模式都拒绝 named-pipe 打开,因此受限命令内的管道 stdio spawn 以 EPERM 失败。工具描述把这两个约定教给模型;后端 README 负责完整的限制说明。 -- **无持久 shell 或 PTY** — 每次调用都启动全新的 `pwsh -Command`;PTY 后端目前仅限 Linux/macOS,Windows ConPTY 持久 shell 属于路线图工作。 +- **Windows 沙箱下的语言模式与 named-pipe 捕获** — 在 [Windows ACL 沙箱](../../sandbox/sandbox-windows-acl/README.zh.md) 下,read-only pwsh 会以 ConstrainedLanguage 启动,因为临时目录写入被拒绝,导致 PowerShell 的 AppLocker 探针失败并按 fail-closed 处理:`Add-Type`、非核心 .NET 静态调用(`[System.IO.*]::`、`[math]::`)、COM 对象与反射都会以“only core types”错误失败,且该模式无法从内部解除。workspace-write 的私有临时目录使探针得以完成,因此除非主机策略另有规定,否则它保持 FullLanguage。两种受限模式都拒绝 named-pipe 打开,因此受限命令内的管道 stdio spawn 以 EPERM 失败。工具描述把这两个约定教给模型;后端 README 负责完整的限制说明。 +- **无持久 shell** — 每次调用都启动全新的 `pwsh -Command`;持久 shell 对应物是 [`@deepseek-ai/dsh-tool-pwsh-persistent`](../tool-pwsh-persistent/README.zh.md),它在 Windows(ConPTY)以及装有 pwsh 的 POSIX 主机上跨调用保持一个 owner 作用域的 pwsh 存活。 - **PowerShell 方言约定** — 模型必须写 PowerShell(原生路径、`$env:` 变量),而不是 bash;没有方言翻译。 - **会话 cwd 身份不做规范化** — workdir 基座直接取会话头 cwd 原值,不同于 bash 工具经 sandbox-root 规范化的身份。在隔离执行器下,策略的工作区根**会**被规范化(由共享的策略服务完成),因此当原始会话 cwd 与其规范化形态不同时,workdir 与隔离根可能不一致——这一 parity 差距留待共享 shell 工具基座提取时解决。 diff --git a/packages/shell/tool-pwsh/package.json b/packages/shell/tool-pwsh/package.json index 195940e82b..2294a955a3 100644 --- a/packages/shell/tool-pwsh/package.json +++ b/packages/shell/tool-pwsh/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-pwsh", "description": "Model-facing pwsh tool over the bash executor seam", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/shell/tool-pwsh/src/index.ts b/packages/shell/tool-pwsh/src/index.ts index 9cf8d4f3f6..603322e381 100644 --- a/packages/shell/tool-pwsh/src/index.ts +++ b/packages/shell/tool-pwsh/src/index.ts @@ -116,11 +116,10 @@ function pwshDescription(backgroundEnabled: boolean, escalationModes: readonly S if (escalationModes.length === 0) return base // The language-mode and named-pipe contracts below are Windows-restricted-token // behavior, but the gate is 'any confining executor is mounted' - // (escalationModes non-empty). The conflation is safe today because every - // shipped composition pairing tool-pwsh with a confining executor is - // win32-only; a future POSIX pwsh-sandbox composition must gate both - // sentences on the platform instead (tracked in the pwsh-tool-and-executor - // Agent Note). + // (escalationModes non-empty). Every shipped composition pairing tool-pwsh + // with a confining executor is win32-only, so the gate is equivalent. A POSIX + // pwsh-sandbox composition must gate both sentences on the platform instead + // (tracked in the pwsh-tool-and-executor Agent Note). return base + ' Under the Windows sandbox, read-only pwsh runs in PowerShell ConstrainedLanguage mode, while ' + 'workspace-write stays in FullLanguage unless host policy says otherwise. In read-only, prefer cmdlets and core types (`[string]`, `[datetime]`, `[regex]`, `[guid]`); ' + '.NET static calls (`[System.IO.*]::`, `[math]::`), `Add-Type`, COM objects, and reflection fail ' diff --git a/packages/skill/README.i18n.yaml b/packages/skill/README.i18n.yaml index e935ba1c20..4e2383be6c 100644 --- a/packages/skill/README.i18n.yaml +++ b/packages/skill/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/skill/README.md README.md: a9fc04cbd4e81ddcf4b9cfebb4a4d8b9e8888227 -README.zh.md: 84bfa69e133962e27ecb792d6b989ca446eee787 +README.zh.md: 4b5f4e4ffdef0ea45797a5861840e13ec4405ab5 diff --git a/packages/skill/README.zh.md b/packages/skill/README.zh.md index 84bfa69e13..4b5f4e4ffd 100644 --- a/packages/skill/README.zh.md +++ b/packages/skill/README.zh.md @@ -6,11 +6,11 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`skill/`](skill/README.md) | 定义 skill 提供方注册和查找 | `ctx.skills` | -| [`skill-badge/`](skill-badge/README.md) | 贡献可选的内置 dsh 徽章 skill | 注册到 `ctx.skills` | -| [`skill-filesystem/`](skill-filesystem/README.md) | 从本地文件系统发现 skill | 注册到 `ctx.skills` | -| [`tool-skill/`](tool-skill/README.md) | 发布 skill 目录和面向模型的 loader | 注册到 `ctx.tools` | +| [`skill/`](skill/README.zh.md) | 定义 skill 提供方注册和查找 | `ctx.skills` | +| [`skill-badge/`](skill-badge/README.zh.md) | 贡献可选的内置 dsh 徽章 skill | 注册到 `ctx.skills` | +| [`skill-filesystem/`](skill-filesystem/README.zh.md) | 从本地文件系统发现 skill | 注册到 `ctx.skills` | +| [`tool-skill/`](tool-skill/README.zh.md) | 发布 skill 目录和面向模型的 loader | 注册到 `ctx.tools` | 该能力位于核心控制主干之外,可以使用本地、嵌入式或远程提供方,而无需更改面向模型的约定。 -子系统参考——发现优先级、目录快照、`skill` 加载器——见 [docs/subsystems/skills.md](../../docs/subsystems/skills.md)。 +子系统参考——发现优先级、目录快照、`skill` 加载器——见 [docs/subsystems/skills.md](../../docs/subsystems/skills.zh.md)。 diff --git a/packages/skill/skill-badge/package.json b/packages/skill/skill-badge/package.json index d012804e96..0f01470e6d 100644 --- a/packages/skill/skill-badge/package.json +++ b/packages/skill/skill-badge/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-skill-badge", "description": "Bundled dsh badge skill provider for DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/skill/skill-filesystem/README.i18n.yaml b/packages/skill/skill-filesystem/README.i18n.yaml index ea3b9892b8..ea8f3ec05e 100644 --- a/packages/skill/skill-filesystem/README.i18n.yaml +++ b/packages/skill/skill-filesystem/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/skill/skill-filesystem/README.md README.md: 33fb550a9ca804c4f0f380232c2bfd94cc58a67e -README.zh.md: 4c09111fc2d31bf5467b57f90f65cf68e298da09 +README.zh.md: c1d24f590c7fc525fbf15356f7a8bee9c4d1308b diff --git a/packages/skill/skill-filesystem/README.zh.md b/packages/skill/skill-filesystem/README.zh.md index 4c09111fc2..c1d24f590c 100644 --- a/packages/skill/skill-filesystem/README.zh.md +++ b/packages/skill/skill-filesystem/README.zh.md @@ -16,7 +16,7 @@ |---|---|---| | `providerName` | `filesystem` | 在 `ctx.skills` 上注册该提供方时使用的唯一名称。 | | `includeDefaultRoots` | `true` | 在 `customSkillDirs` 周围包含项目根和用户根;设为 false 时仅使用隔离的自定义根。 | -| `dshHome` | `$DSH_HOME` 或 `~/.dsh` | 由 [`@deepseek-ai/dsh-home-paths`](../../util/home-paths/README.md) 解析的 DeepSeek Harness 配置根目录;扫描该目录下的 `skills`。 | +| `dshHome` | `$DSH_HOME` 或 `~/.dsh` | 由 [`@deepseek-ai/dsh-home-paths`](../../util/home-paths/README.zh.md) 解析的 DeepSeek Harness 配置根目录;扫描该目录下的 `skills`。 | | `agentsHome` | `$DSH_AGENTS_HOME` 或 `~/.agents` | 为兼容 skill 扫描的共享 agent(智能体)配置根目录。 | | `customSkillDirs` | `[]` | 在项目根目录之后、用户根目录之前扫描的其他本地 skill 根目录。 | | `watch` | `true` | 监视宿主本地根,并在目录成员或 frontmatter 可能发生变化时使本地提供方失效。 | diff --git a/packages/skill/skill-filesystem/package.json b/packages/skill/skill-filesystem/package.json index 954a843c2b..4cd30a85c9 100644 --- a/packages/skill/skill-filesystem/package.json +++ b/packages/skill/skill-filesystem/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-skill-filesystem", "description": "Local filesystem skill provider for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/skill/skill/package.json b/packages/skill/skill/package.json index 4688a93064..04fa4dcd92 100644 --- a/packages/skill/skill/package.json +++ b/packages/skill/skill/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-skill", "description": "Agent skill provider registry for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/skill/tool-skill/README.i18n.yaml b/packages/skill/tool-skill/README.i18n.yaml index 918a10d01b..b1c6702524 100644 --- a/packages/skill/tool-skill/README.i18n.yaml +++ b/packages/skill/tool-skill/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/skill/tool-skill/README.md README.md: 4cfd8f22b3cfaf22b6f9f6cdc48f0db994858906 -README.zh.md: 9651e6d3209af77d3a2bbabb1bf0ec93647d3e90 +README.zh.md: 4b4fc6382a7f3e00592b6ebbf0b4887748cddf70 diff --git a/packages/skill/tool-skill/README.zh.md b/packages/skill/tool-skill/README.zh.md index 9651e6d320..4b4fc6382a 100644 --- a/packages/skill/tool-skill/README.zh.md +++ b/packages/skill/tool-skill/README.zh.md @@ -14,7 +14,7 @@ 如果最初没有模型可调用 skill,则省略目录;如果该 agent(智能体)的工具视图排除了随附的 `skill` 工具,或解析出同名的作用域内遮蔽项,也会省略目录。身份比对针对本插件所注册的那个定义,而非按自身名字回查,因此本插件既可全局挂载,也可挂在单个 agent 的组装内——在后者中 `register()` 只注册到该 agent 的层中。可见性变更参与 digest 计算,使提示词指引、模型可见 schema 和可执行分派保持对齐。 -`catalogDescriptionMaxLength` 控制规范化后的目录描述,渲染时会对其执行 XML 转义。其默认值是 `500`,且必须是不小于 `3` 的整数,以便为截断省略号保留空间。[skill 目录热刷新 Agent Note](../../../.agents/notes/implemented/feature/2026-07-27-skill-catalog-hot-refresh.md) 负责定义持久初始目录和替换目录的生命周期。 +`catalogDescriptionMaxLength` 控制规范化后的目录描述,渲染时会对其执行 XML 转义。其默认值是 `500`,且必须是不小于 `3` 的整数,以便为截断省略号保留空间。[skill 目录热刷新 Agent Note](../../../.agents/notes/implemented/feature/2026-07-27-skill-catalog-hot-refresh.zh.md) 负责定义持久初始目录和替换目录的生命周期。 ## 工具:`skill` @@ -65,7 +65,7 @@ A user may also invoke a skill directly; its block then appears #### 模型看到的内容 -模型会看到生成的 [`skill` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-skill)。 +模型会看到生成的 [`skill` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-skill)。 #### Token 影响 diff --git a/packages/skill/tool-skill/package.json b/packages/skill/tool-skill/package.json index 19b10f8bb6..798c09ca05 100644 --- a/packages/skill/tool-skill/package.json +++ b/packages/skill/tool-skill/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-skill", "description": "Model-facing skill loading tool for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/skill/tool-skill/src/index.ts b/packages/skill/tool-skill/src/index.ts index e2a0cc2dca..596876b5a7 100644 --- a/packages/skill/tool-skill/src/index.ts +++ b/packages/skill/tool-skill/src/index.ts @@ -387,7 +387,6 @@ function catalogMessage( return undefined } -/** Normalized, length-bounded description exactly as the catalog publishes it (unescaped). */ function catalogDescription(value: string, maxLength: number): string { const normalized = value.replaceAll(/\s+/g, ' ').trim() return normalized.length <= maxLength ? normalized : `${normalized.slice(0, maxLength - 3)}...` diff --git a/packages/spill/README.i18n.yaml b/packages/spill/README.i18n.yaml index 1d1ace946c..4dfc9aac96 100644 --- a/packages/spill/README.i18n.yaml +++ b/packages/spill/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/spill/README.md README.md: 74fd4837cb23767754174df168f2f4ba71c2b938 -README.zh.md: 2417bb793dca1cb10ead6208cb6ad896c18cdf03 +README.zh.md: 9a1f3495220142f7a34f806548c5ca5cbfab8734 diff --git a/packages/spill/README.zh.md b/packages/spill/README.zh.md index 2417bb793d..9a1f349522 100644 --- a/packages/spill/README.zh.md +++ b/packages/spill/README.zh.md @@ -6,10 +6,10 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`spill/`](spill/README.md) | 定义 spill 存储 | `ctx.spillStore` | -| [`spill-local/`](spill-local/README.md) | 在会话范围的本地文件中存储 spill 文本 | 注册到 `ctx.spillStore` | -| [`spill-policy/`](spill-policy/README.md) | 应用执行后 spill 策略 | 监听 `ctx.tools` | +| [`spill/`](spill/README.zh.md) | 定义 spill 存储 | `ctx.spillStore` | +| [`spill-local/`](spill-local/README.zh.md) | 在会话范围的本地文件中存储 spill 文本 | 注册到 `ctx.spillStore` | +| [`spill-policy/`](spill-policy/README.zh.md) | 应用执行后 spill 策略 | 监听 `ctx.tools` | -参见[工具输出 spill 决策](../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md),其中记录了存储、保留和工具自有输出处理之间的边界。 +参见[工具输出 spill 决策](../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md),其中记录了存储、保留和工具自有输出处理之间的边界。 -子系统参考——`SaveTextSpill`、所有者/来源、品牌化定位符——见 [docs/subsystems/spill.md](../../docs/subsystems/spill.md);依据见[工具输出 spill Agent Note](../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md)。 +子系统参考——`SaveTextSpill`、所有者/来源、品牌化定位符——见 [docs/subsystems/spill.md](../../docs/subsystems/spill.zh.md);依据见[工具输出 spill Agent Note](../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md)。 diff --git a/packages/spill/spill-local/README.i18n.yaml b/packages/spill/spill-local/README.i18n.yaml index 0a0ab3823d..37659c3aac 100644 --- a/packages/spill/spill-local/README.i18n.yaml +++ b/packages/spill/spill-local/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/spill/spill-local/README.md README.md: 2270a65d9270e1549a9e48d6a36b821e48c29070 -README.zh.md: 907a2b5c98f3ee07e0b896b8827e6a00fa1827df +README.zh.md: b3e4999d8f2d982ef01637199299f79d06e12c4b diff --git a/packages/spill/spill-local/README.zh.md b/packages/spill/spill-local/README.zh.md index 907a2b5c98..b3e4999d8f 100644 --- a/packages/spill/spill-local/README.zh.md +++ b/packages/spill/spill-local/README.zh.md @@ -18,7 +18,7 @@ |---|---|---| | `root` | 私有 0700 临时目录 | spill 文件的根目录。设置后可将这些文件保存在已知位置。 | -`saveText` 在发生真实存储故障(权限、ENOSPC)时返回拒绝;spill 策略会按尽力而为原则处理该拒绝,并保留内联结果。词汇见 seam README,设计见[工具输出 spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md)。 +`saveText` 在发生真实存储故障(权限、ENOSPC)时返回拒绝;spill 策略会按尽力而为原则处理该拒绝,并保留内联结果。词汇见 seam README,设计见[工具输出 spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md)。 ## 模型体验 diff --git a/packages/spill/spill-local/package.json b/packages/spill/spill-local/package.json index 07c2134591..44ca42effd 100644 --- a/packages/spill/spill-local/package.json +++ b/packages/spill/spill-local/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-spill-local", "description": "Local-filesystem implementation of the DeepSeek Harness spill storage seam (private session-scoped files)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/spill/spill-local/src/store.ts b/packages/spill/spill-local/src/store.ts index e44418767a..e4451d5268 100644 --- a/packages/spill/spill-local/src/store.ts +++ b/packages/spill/spill-local/src/store.ts @@ -1,9 +1,7 @@ /** * Cordis-free storage mechanics for the local spill backend: private * session-scoped directory selection, safe-name derivation, path-traversal - * protection, and the exclusive owner-only write. Kept out of the service class - * (like `dsh-bash-local`'s `run.ts`) so the filesystem behavior is unit-testable - * without a `ctx` and without the OS temp dir. + * protection, and the exclusive owner-only write. * * @module @deepseek-ai/dsh-spill-local/store */ @@ -29,8 +27,7 @@ export function privateRoot(): string { return defaultRoot } -// Deliberately mirrors the JSONL path encoder, but keeps spill's empty-name -// policy (`""` -> `"~"`) local so storage backends stay decoupled. +// Spill keeps its empty-name policy local so storage backends stay decoupled. /* jscpd:ignore-start */ /** * Encode an arbitrary string as one safe path segment, injectively over ALL JS @@ -40,7 +37,6 @@ export function privateRoot(): string { * as `~XXXX`; `~` is itself escaped, so the mapping is reversible and distinct * inputs never collide. The whole-segment tokens `.`/`..` are escaped so they * can never traverse. An empty string encodes to `~` (never an empty segment). - * (Mirrors the JSONL persistence backend's `encodeSegment`.) * * @param raw The untrusted string to encode as one safe path segment. * @returns An injective, filesystem-safe single path segment. diff --git a/packages/spill/spill-policy/README.i18n.yaml b/packages/spill/spill-policy/README.i18n.yaml index f98bf76167..e6fe69d9e3 100644 --- a/packages/spill/spill-policy/README.i18n.yaml +++ b/packages/spill/spill-policy/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/spill/spill-policy/README.md README.md: 715d10bee6d32b0753e0f6afcc18d83fe0261a60 -README.zh.md: 89635717e82f078055f6594b2aa8a49715435060 +README.zh.md: 11fe4aada6a5138dd1f7bb537cd8ad1463d72b05 diff --git a/packages/spill/spill-policy/README.zh.md b/packages/spill/spill-policy/README.zh.md index 89635717e8..11fe4aada6 100644 --- a/packages/spill/spill-policy/README.zh.md +++ b/packages/spill/spill-policy/README.zh.md @@ -30,11 +30,11 @@ **尽力而为**:没有会话所有者、没有 `ctx.spillStore` 后端,或 `saveText` 返回拒绝 ⇒ 策略记录警告并返回原始结果。spill 失败绝不会将成功调用变为 `isError`,也不会隐藏内联结果。成功替换时只会更改 `content`;规范的程序化值保持不变。 -**dispatch-log 分支:**注册在 `tools/code-dispatch-log` 上的第二个监听器,把同一套上限、替换流水线与尽力而为的回退应用到每个 `run_code` 子调用结果的持久化副本上(产物标签为 `dispatch`,按子调用 id 归档)。程序的值不受影响,因为它早已完整跨过 worker 边界;`read` 子调用同样设界:日志副本不是模型上下文,因此不会发生 read-again 循环,而 `read` 恰恰是最容易产生巨型日志的工具([原理](../../../.agents/notes/implemented/feature/2026-07-26-code-dispatch-log-spill.md))。 +**dispatch-log 分支:**注册在 `tools/code-dispatch-log` 上的第二个监听器,把同一套上限、替换流水线与尽力而为的回退应用到每个 `run_code` 子调用结果的持久化副本上(产物标签为 `dispatch`,按子调用 id 归档)。程序的值不受影响,因为它早已完整跨过 worker 边界;`read` 子调用同样设界:日志副本不是模型上下文,因此不会发生 read-again 循环,而 `read` 恰恰是最容易产生巨型日志的工具([原理](../../../.agents/notes/implemented/feature/2026-07-26-code-dispatch-log-spill.zh.md))。 ## 范围 -该策略只能看到最终格式化的呈现结果,看不到工具的内部资源或规范值。如果提供方已经截断内容(例如 `web-fetch-http.maxBodyChars`),spill 产物保存的是工具返回的完整格式化结果,而非完整原始源。提供方/资源上限仍然是必需的,并且与该策略相互独立。`glob`/`grep` 负责对项级呈现结果执行 spill,因为渲染前仍然存在完整的已获取值;bash 流负责在获取时 spill。通用策略预先注册自己的 waterfall(瀑布式事件)监听器,然后再委托,因此无论插件加载顺序如何,普通工具自身的异步投影都会在通用字节限制之前完成。详见[工具输出 spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md)。 +该策略只能看到最终格式化的呈现结果,看不到工具的内部资源或规范值。如果提供方已经截断内容(例如 `web-fetch-http.maxBodyChars`),spill 产物保存的是工具返回的完整格式化结果,而非完整原始源。提供方/资源上限仍然是必需的,并且与该策略相互独立。`glob`/`grep` 负责对项级呈现结果执行 spill,因为渲染前仍然存在完整的已获取值;bash 流负责在获取时 spill。通用策略预先注册自己的 waterfall(瀑布式事件)监听器,然后再委托,因此无论插件加载顺序如何,普通工具自身的异步投影都会在通用字节限制之前完成。详见[工具输出 spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md)。 ## 模型体验 diff --git a/packages/spill/spill-policy/package.json b/packages/spill/spill-policy/package.json index f74a5c4135..fec25041f6 100644 --- a/packages/spill/spill-policy/package.json +++ b/packages/spill/spill-policy/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-spill-policy", "description": "Tool-result spill policy for the DeepSeek Harness — replaces oversized plain-text tool results with a retained preview plus a spill-file path (no service API)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/spill/spill/README.i18n.yaml b/packages/spill/spill/README.i18n.yaml index 0a84c591d5..695457a9af 100644 --- a/packages/spill/spill/README.i18n.yaml +++ b/packages/spill/spill/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/spill/spill/README.md README.md: f36dccfa1e6f1f21e80c0b0aa97ee150a0f5fd45 -README.zh.md: 268e390923e30150a8d36b719b7dad75827f99fe +README.zh.md: 64f0a99bd1cdb10af2bbe0007de802d8ce658b7a diff --git a/packages/spill/spill/README.zh.md b/packages/spill/spill/README.zh.md index 268e390923..64f0a99bd1 100644 --- a/packages/spill/spill/README.zh.md +++ b/packages/spill/spill/README.zh.md @@ -26,7 +26,7 @@ `SaveTextSpill`(owner、source、suggestedName、content)是请求;`SpillRef`(locator、bytes、retrievalHint)是结果。`SpillLocator` 是[带品牌类型](../../util/brand)的值,并以不透明字符串的形式呈现给模型;对 `dsh-spill-local` 而言它是本地路径,但未来的后端可以返回 URI、键或命令 token,无需修改策略/工具消费方。`SpillOwner.sessionId` 是保存时存储命名空间:fork 后的会话会从种子日志继承现有定位信息,无需复制或更改其归属;fork 后新产生的 spill 使用子会话 id。`SpillSource` 记录产生该 spill 的 `toolName`、`callId` 和 `label`,供后端命名和检查使用,不用于访问控制。完整约定见 `src/types.ts`。 -设计原理见[工具输出 spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md),其中说明了为什么创建操作应由运行时 spill seam 而非面向模型的 `write` 工具承担。 +设计原理见[工具输出 spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md),其中说明了为什么创建操作应由运行时 spill seam 而非面向模型的 `write` 工具承担。 ## 模型体验 diff --git a/packages/spill/spill/package.json b/packages/spill/spill/package.json index 3d0f444b1d..4dc57a5b51 100644 --- a/packages/spill/spill/package.json +++ b/packages/spill/spill/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-spill", "description": "Abstract spill storage seam (ctx.spillStore) for the DeepSeek Harness — save oversized tool text and return a retrieval locator", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/storage/README.i18n.yaml b/packages/storage/README.i18n.yaml index 2d6db691b7..d0c1293124 100644 --- a/packages/storage/README.i18n.yaml +++ b/packages/storage/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/storage/README.md README.md: dd9eb5ac97529c75df5986c56b714331dca0b6f1 -README.zh.md: a69414826b3ffd613c432836ea506b664ceb8da5 +README.zh.md: 615212e46b358c55792e2cbc550797c7b0c8ca91 diff --git a/packages/storage/README.zh.md b/packages/storage/README.zh.md index a69414826b..615212e46b 100644 --- a/packages/storage/README.zh.md +++ b/packages/storage/README.zh.md @@ -6,11 +6,11 @@ | 包 | 职责 | ctx key | |---|---|---| -| [`storage/`](storage/README.md) | 将已注册后端与类型化数据形式连接起来 | `ctx.storage` | -| [`storage-json/`](storage-json/README.md) | 在 JSON 文件中存储数据 | 注册后端 `json` | -| [`storage-sqlite/`](storage-sqlite/README.md) | 在 SQLite 中存储数据 | 注册后端 `sqlite` | -| [`storage-domain/`](storage-domain/README.md) | 提供经过验证的领域记录存储 | `ctx.storageDomain` | +| [`storage/`](storage/README.zh.md) | 将已注册后端与类型化数据形式连接起来 | `ctx.storage` | +| [`storage-json/`](storage-json/README.zh.md) | 在 JSON 文件中存储数据 | 注册后端 `json` | +| [`storage-sqlite/`](storage-sqlite/README.zh.md) | 在 SQLite 中存储数据 | 注册后端 `sqlite` | +| [`storage-domain/`](storage-domain/README.zh.md) | 提供经过验证的领域记录存储 | `ctx.storageDomain` | -消费方使用数据形式,而不是直接访问后端。[领域存储决策](../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.md)记录了该家族的设计。 +消费方使用数据形式,而不是直接访问后端。[领域存储决策](../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md)记录了该家族的设计。 -子系统参考——后端约定、`StorageForms`、`DomainSpec`/`Domain`、`domain/changed`——见 [docs/subsystems/storage.md](../../docs/subsystems/storage.md)。 +子系统参考——后端约定、`StorageForms`、`DomainSpec`/`Domain`、`domain/changed`——见 [docs/subsystems/storage.md](../../docs/subsystems/storage.zh.md)。 diff --git a/packages/storage/storage-domain/README.i18n.yaml b/packages/storage/storage-domain/README.i18n.yaml index 80ccac9a5f..9c4c1c8b4c 100644 --- a/packages/storage/storage-domain/README.i18n.yaml +++ b/packages/storage/storage-domain/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/storage/storage-domain/README.md -README.md: 7c058ab557cfb128f54c15e715685d59f6720696 +README.md: 59f5590867d35169ad7636075da524ebbf162489 README.zh.md: 10eb33711e147bc5ac6b7770a196bfea4fe66306 diff --git a/packages/storage/storage-domain/README.md b/packages/storage/storage-domain/README.md index 7c058ab557..59f5590867 100644 --- a/packages/storage/storage-domain/README.md +++ b/packages/storage/storage-domain/README.md @@ -4,7 +4,7 @@ English | [中文](README.zh.md) Domain data form for the DeepSeek Harness storage hub: exposes the injectable `ctx.storageDomain` service and the matching `ctx.storage.domain` projection after every configured backend is registered. A domain is declared once with `defineDomain` (zod record schemas, `z.infer`-derived types), opened through `DomainFacility.open`, and served from authoritative in-memory state — reads are synchronous, writes serialize on one per-domain chain, reach durability on the routed backend first, then update memory and emit `domain/changed`. The opening consumer owns the handle's lifecycle and releases it with `Domain.close()` (idempotent; typically its own `ctx.effect` disposer); domains still open when the plugin unmounts are closed by the facility. -Design rationale, open semantics, and the storage/domain layer split live in the [Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md). +Design rationale, open semantics, and the storage/domain layer split live in the [Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.md). ## Configuration diff --git a/packages/storage/storage-domain/package.json b/packages/storage/storage-domain/package.json index aa8983a712..3f4984bce2 100644 --- a/packages/storage/storage-domain/package.json +++ b/packages/storage/storage-domain/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-storage-domain", "description": "Domain data form (ctx.storage.domain): schema-validated, event-emitting KV domains over storage backends for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/storage/storage-json/README.i18n.yaml b/packages/storage/storage-json/README.i18n.yaml index 0ca8f2d918..109bafd8f5 100644 --- a/packages/storage/storage-json/README.i18n.yaml +++ b/packages/storage/storage-json/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/storage/storage-json/README.md -README.md: c3417846a70f8a227a19a5abf834156d530e3597 -README.zh.md: e66c711dca838620eb9c90a3edacd2639970505e +README.md: 3a079da3f516af4a3f7bc13dcf0115589fa42ff3 +README.zh.md: f9b2ffb7444893a27a2da1d3dcd324c73452cc92 diff --git a/packages/storage/storage-json/README.md b/packages/storage/storage-json/README.md index c3417846a7..3a079da3f5 100644 --- a/packages/storage/storage-json/README.md +++ b/packages/storage/storage-json/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -JSON backend for the [storage hub](../storage/README.md): one human-readable `.json` file per unit under a configured root, registered as backend `json`. Design: [domain KV storage Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md). +JSON backend for the [storage hub](../storage/README.md): one human-readable `.json` file per unit under a configured root, registered as backend `json`. Design: [domain KV storage Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.md). ## Model diff --git a/packages/storage/storage-json/README.zh.md b/packages/storage/storage-json/README.zh.md index e66c711dca..f9b2ffb744 100644 --- a/packages/storage/storage-json/README.zh.md +++ b/packages/storage/storage-json/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[存储中心](../storage/README.md)的 JSON 后端:配置根目录下每个单元使用一个人类可读的 `.json` 文件,注册为后端 `json`。设计见[领域 KV 存储 Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md)。 +[存储中心](../storage/README.zh.md)的 JSON 后端:配置根目录下每个单元使用一个人类可读的 `.json` 文件,注册为后端 `json`。设计见[领域 KV 存储 Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md)。 ## 模型 diff --git a/packages/storage/storage-json/package.json b/packages/storage/storage-json/package.json index 5c04e79096..b147083c59 100644 --- a/packages/storage/storage-json/package.json +++ b/packages/storage/storage-json/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-storage-json", "description": "JSON file KV storage backend for the DeepSeek Harness storage hub", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/storage/storage-sqlite/README.i18n.yaml b/packages/storage/storage-sqlite/README.i18n.yaml index 2215e835e2..b1b8b1def0 100644 --- a/packages/storage/storage-sqlite/README.i18n.yaml +++ b/packages/storage/storage-sqlite/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/storage/storage-sqlite/README.md -README.md: 48aae14aa8613886d69652c39194878c190259cf -README.zh.md: d25591ac67a96bce2ea74b2686e2fbb8375af959 +README.md: 654ae004986240ae6c7c8f74f80e45089270a0ab +README.zh.md: 9f31c55a3281a02ece793d2810480de4e9d8f339 diff --git a/packages/storage/storage-sqlite/README.md b/packages/storage/storage-sqlite/README.md index 48aae14aa8..654ae00498 100644 --- a/packages/storage/storage-sqlite/README.md +++ b/packages/storage/storage-sqlite/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -SQLite backend for the [storage hub](../storage/README.md): registers as backend `sqlite`, serving the `kv` facet over one `node:sqlite` database file (or `:memory:`). Design and trade-offs: [domain KV storage Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md). +SQLite backend for the [storage hub](../storage/README.md): registers as backend `sqlite`, serving the `kv` facet over one `node:sqlite` database file (or `:memory:`). Design and trade-offs: [domain KV storage Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.md). ## Storage model diff --git a/packages/storage/storage-sqlite/README.zh.md b/packages/storage/storage-sqlite/README.zh.md index d25591ac67..9f31c55a32 100644 --- a/packages/storage/storage-sqlite/README.zh.md +++ b/packages/storage/storage-sqlite/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[存储中心](../storage/README.md)的 SQLite 后端:注册为后端 `sqlite`,通过一个数据库提供 `kv` facet;该数据库由 `node:sqlite` 操作,可以是单个文件,也可以是 `:memory:`。设计与取舍见[领域 KV 存储 Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md)。 +[存储中心](../storage/README.zh.md)的 SQLite 后端:注册为后端 `sqlite`,通过一个数据库提供 `kv` facet;该数据库由 `node:sqlite` 操作,可以是单个文件,也可以是 `:memory:`。设计与取舍见[领域 KV 存储 Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md)。 ## 存储模型 diff --git a/packages/storage/storage-sqlite/package.json b/packages/storage/storage-sqlite/package.json index 88d5958419..2b96120223 100644 --- a/packages/storage/storage-sqlite/package.json +++ b/packages/storage/storage-sqlite/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-storage-sqlite", "description": "SQLite storage backend (kv facet) for the DeepSeek Harness storage hub", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/storage/storage/README.i18n.yaml b/packages/storage/storage/README.i18n.yaml index 09fa051bac..74b71a0f88 100644 --- a/packages/storage/storage/README.i18n.yaml +++ b/packages/storage/storage/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/storage/storage/README.md README.md: bf827220afe0d8b1cbc53b3a12e6d0034004e1d2 -README.zh.md: 86b6994d74a727275e0bcaea2a83bd51f93b6ca1 +README.zh.md: fb52c96e38c17b76ef0e6b70a79177911a088fb7 diff --git a/packages/storage/storage/README.zh.md b/packages/storage/storage/README.zh.md index 86b6994d74..fb52c96e38 100644 --- a/packages/storage/storage/README.zh.md +++ b/packages/storage/storage/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -非会话数据的存储中心(`ctx.storage`):具名后端注册表加已挂载的数据形式设施。中心自身不执行 IO:后端拥有介质,数据形式拥有语义。[存储家族概述](../README.md)列出了这些包;[领域 KV 存储 Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.md)记录了设计理由。 +非会话数据的存储中心(`ctx.storage`):具名后端注册表加已挂载的数据形式设施。中心自身不执行 IO:后端拥有介质,数据形式拥有语义。[存储家族概述](../README.zh.md)列出了这些包;[领域 KV 存储 Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md)记录了设计理由。 ## 结构 diff --git a/packages/storage/storage/package.json b/packages/storage/storage/package.json index f4f39039b0..a4e1135596 100644 --- a/packages/storage/storage/package.json +++ b/packages/storage/storage/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-storage", "description": "Storage hub (ctx.storage): named backend registry plus mounted data-form facilities for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subagent/README.i18n.yaml b/packages/subagent/README.i18n.yaml index 5a5dc2c181..fdff53c59c 100644 --- a/packages/subagent/README.i18n.yaml +++ b/packages/subagent/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/README.md -README.md: a863ed3f5ef864b6eb6eb9a7a0c1ee2f40f247d6 -README.zh.md: 1c9bf8ba0814a74c5774d81a34a3266daa9c375b +README.md: 1b3ccb6afba226e0b6bcf9aacd0c9fa752fdfdbb +README.zh.md: 245b187b71b06f0c291966fb30a65941ed18aefb diff --git a/packages/subagent/README.md b/packages/subagent/README.md index a863ed3f5e..1b3ccb6afb 100644 --- a/packages/subagent/README.md +++ b/packages/subagent/README.md @@ -18,6 +18,8 @@ This family lets an agent delegate work to child agents. Multiple named provider | [`tool-subagent-control/`](tool-subagent-control/README.md) | Exposes child messaging and listing to the model | registers on `ctx.tools` | | [`tool-subagent-report/`](tool-subagent-report/README.md) | Provides the child-to-parent report channel | registers in child scopes | +The Codex and Claude Code packages are independent optional Profile Bundles. Install either or both with `dsh plugin --profile add @deepseek-ai/dsh-subagent-codex @deepseek-ai/dsh-subagent-claude-code`, then restart that Profile; each package registers only its dormant Host provider. To grant a tool, copy a complete Agent Preset, remove `disabled` from each matching tool row, and start a new Session. Removing one package withdraws only that provider and its private runtime closure on the next Profile start. + See the decisions for the [capability family](../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.md), [continuable children](../../.agents/notes/implemented/feature/2026-07-21-continuable-background-subagents.md), and [control tools](../../.agents/notes/implemented/simplification/2026-07-26-merge-subagent-control-service.md). The subsystem reference — start requests, results, live runs, the provider contract, continuable background children — is [docs/subsystems/subagent.md](../../docs/subsystems/subagent.md); design rationale in the [subagent capability seam](../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.md), [continuable background subagents](../../.agents/notes/implemented/feature/2026-07-21-continuable-background-subagents.md), and [merged subagent control service](../../.agents/notes/implemented/simplification/2026-07-26-merge-subagent-control-service.md) Agent Notes. diff --git a/packages/subagent/README.zh.md b/packages/subagent/README.zh.md index 1c9bf8ba08..245b187b71 100644 --- a/packages/subagent/README.zh.md +++ b/packages/subagent/README.zh.md @@ -6,18 +6,20 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`subagent/`](subagent/README.md) | 定义提供方注册、委派和继续执行 | `ctx.subagents` | -| [`subagent-inprocess/`](subagent-in-process-driver/README.md) | 提供共享的进程内运行驱动器 | 无 | -| [`subagent-spawn-in-process/`](subagent-spawn-in-process/README.md) | 启动全新的进程内子 agent | 注册到 `ctx.subagents` | -| [`subagent-fork-in-process/`](subagent-fork-in-process/README.md) | 从父 agent 已完成的历史记录启动进程内子 agent | 注册到 `ctx.subagents` | -| [`subagent-acp/`](subagent-acp/README.md) | 通过 ACP(Agent Client Protocol)启动进程外子 agent | 注册到 `ctx.subagents` | -| [`subagent-codex/`](subagent-codex/README.md) | 启动真实的 Codex app-server 子 agent | 注册到 `ctx.subagents` | -| [`subagent-claude-code/`](subagent-claude-code/README.md) | 通过官方 Claude Agent SDK 启动真实的 Claude Code 子 agent | 注册到 `ctx.subagents` | -| [`subagent-dsh-sdk/`](subagent-dsh-sdk/README.md) | 通过 TypeScript SDK 启动进程外 Harness 子 agent | 注册到 `ctx.subagents` | -| [`tool-subagent/`](tool-subagent/README.md) | 向模型公开委派操作 | 注册到 `ctx.tools` | -| [`tool-subagent-control/`](tool-subagent-control/README.md) | 向模型公开子级消息发送和列举操作 | 注册到 `ctx.tools` | -| [`tool-subagent-report/`](tool-subagent-report/README.md) | 提供从子级到父级的报告通道 | 注册到子级作用域 | +| [`subagent/`](subagent/README.zh.md) | 定义提供方注册、委派和继续执行 | `ctx.subagents` | +| [`subagent-inprocess/`](subagent-in-process-driver/README.zh.md) | 提供共享的进程内运行驱动器 | 无 | +| [`subagent-spawn-in-process/`](subagent-spawn-in-process/README.zh.md) | 启动全新的进程内子 agent | 注册到 `ctx.subagents` | +| [`subagent-fork-in-process/`](subagent-fork-in-process/README.zh.md) | 从父 agent 已完成的历史记录启动进程内子 agent | 注册到 `ctx.subagents` | +| [`subagent-acp/`](subagent-acp/README.zh.md) | 通过 ACP(Agent Client Protocol)启动进程外子 agent | 注册到 `ctx.subagents` | +| [`subagent-codex/`](subagent-codex/README.zh.md) | 启动真实的 Codex app-server 子 agent | 注册到 `ctx.subagents` | +| [`subagent-claude-code/`](subagent-claude-code/README.zh.md) | 通过官方 Claude Agent SDK 启动真实的 Claude Code 子 agent | 注册到 `ctx.subagents` | +| [`subagent-dsh-sdk/`](subagent-dsh-sdk/README.zh.md) | 通过 TypeScript SDK 启动进程外 Harness 子 agent | 注册到 `ctx.subagents` | +| [`tool-subagent/`](tool-subagent/README.zh.md) | 向模型公开委派操作 | 注册到 `ctx.tools` | +| [`tool-subagent-control/`](tool-subagent-control/README.zh.md) | 向模型公开子级消息发送和列举操作 | 注册到 `ctx.tools` | +| [`tool-subagent-report/`](tool-subagent-report/README.zh.md) | 提供从子级到父级的报告通道 | 注册到子级作用域 | -参见有关[能力家族](../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.md)、[可继续执行的子级](../../.agents/notes/implemented/feature/2026-07-21-continuable-background-subagents.md)和[控制工具](../../.agents/notes/implemented/simplification/2026-07-26-merge-subagent-control-service.md)的决策。 +Codex 与 Claude Code 包是彼此独立的可选 Profile Bundle。使用 `dsh plugin --profile add @deepseek-ai/dsh-subagent-codex @deepseek-ai/dsh-subagent-claude-code` 安装其中一个或两个包,再重启该 Profile;每个包只注册自己的休眠 Host provider。要授予工具,请复制一份完整 Agent Preset,删除各对应工具行的 `disabled`,再启动新 Session。移除其中一个包后,下一次 Profile 启动只会撤回对应 provider 及其私有运行时闭包。 -子系统参考——启动请求、结果、实时运行、提供方约定、可续跑后台子 agent——见 [docs/subsystems/subagent.md](../../docs/subsystems/subagent.md);设计依据见 [subagent 能力 seam](../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.md)、[可续跑后台 subagent](../../.agents/notes/implemented/feature/2026-07-21-continuable-background-subagents.md)与[合并 subagent 控制服务](../../.agents/notes/implemented/simplification/2026-07-26-merge-subagent-control-service.md) Agent Note。 +参见有关[能力家族](../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.zh.md)、[可继续执行的子级](../../.agents/notes/implemented/feature/2026-07-21-continuable-background-subagents.zh.md)和[控制工具](../../.agents/notes/implemented/simplification/2026-07-26-merge-subagent-control-service.zh.md)的决策。 + +子系统参考——启动请求、结果、实时运行、提供方约定、可续跑后台子 agent——见 [docs/subsystems/subagent.md](../../docs/subsystems/subagent.zh.md);设计依据见 [subagent 能力 seam](../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.zh.md)、[可续跑后台 subagent](../../.agents/notes/implemented/feature/2026-07-21-continuable-background-subagents.zh.md)与[合并 subagent 控制服务](../../.agents/notes/implemented/simplification/2026-07-26-merge-subagent-control-service.zh.md) Agent Note。 diff --git a/packages/subagent/subagent-acp/README.i18n.yaml b/packages/subagent/subagent-acp/README.i18n.yaml index 61b76620ae..fa56c715a5 100644 --- a/packages/subagent/subagent-acp/README.i18n.yaml +++ b/packages/subagent/subagent-acp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/subagent-acp/README.md -README.md: 3bccddbca021bed1f8bf5766b9575f3bd7441669 -README.zh.md: 38c882f240dcdd02213acecf266f5a2424b37cce +README.md: 0201f9bacca5c59031d2fc2eec7fe7210489dd09 +README.zh.md: c987e6251e035f0cf94ac41b570f3be10f0118f5 diff --git a/packages/subagent/subagent-acp/README.md b/packages/subagent/subagent-acp/README.md index 3bccddbca0..0201f9bacc 100644 --- a/packages/subagent/subagent-acp/README.md +++ b/packages/subagent/subagent-acp/README.md @@ -33,15 +33,18 @@ ACP advertises no start-time capabilities because this process cannot enforce th | `disposeEofGraceMs` | `6000` | Positive grace after stdin EOF before platform termination; it cannot exceed [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md). | | `disposeGraceMs` | `3000` | Positive POSIX grace after SIGTERM before SIGKILL (Windows force-terminates directly); it cannot exceed [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md). | +A DeepSeek Harness child uses the product launcher and an explicit absolute `DSH_HOME`. The isolated home prevents a nested runtime from discovering the launching person's profiles or credentials; the generic ACP provider does not impose this requirement on non-DSH agents. + ```yaml - id: subagent-acp name: '@deepseek-ai/dsh-subagent-acp' config: providerName: acp - command: node - args: ['--import', 'tsx', './packages/examples/acp-demo/src/bin.ts', '--config', './examples/acp-agent/cordis.yml'] + command: dsh + args: ['--profile', 'acp', '--patch', '/absolute/path/to/acp.patch.yml'] permission: reject env: + DSH_HOME: /absolute/path/to/isolated-child-home DEEPSEEK_API_KEY: !!js process.env.DEEPSEEK_API_KEY ``` diff --git a/packages/subagent/subagent-acp/README.zh.md b/packages/subagent/subagent-acp/README.zh.md index 38c882f240..c987e6251e 100644 --- a/packages/subagent/subagent-acp/README.zh.md +++ b/packages/subagent/subagent-acp/README.zh.md @@ -30,18 +30,21 @@ ACP 不声明任何启动时能力,因为当前进程无法强制执行远程 | `cwd` | 父会话 cwd | 子进程及其 ACP 会话的工作目录覆盖值;不得为空。相对值会在加载时以 harness 启动目录为基准解析,结果必须指向 harness 可以进入的目录。 | | `permission` | `reject` | 自动回答权限请求:拒绝,或选择第一个 `allow_once` 或 `allow_always` 选项。 | | `env` | `{}` | 显式子进程环境,叠加到已清理凭据的父进程环境之上。 | -| `disposeEofGraceMs` | `6000` | stdin EOF 之后、平台终止之前的宽限时间须为正值,且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md)。 | -| `disposeGraceMs` | `3000` | POSIX 在 SIGTERM 后、SIGKILL 前的宽限时间(Windows 直接强制终止),须为正值且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md)。 | +| `disposeEofGraceMs` | `6000` | stdin EOF 之后、平台终止之前的宽限时间须为正值,且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.zh.md)。 | +| `disposeGraceMs` | `3000` | POSIX 在 SIGTERM 后、SIGKILL 前的宽限时间(Windows 直接强制终止),须为正值且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.zh.md)。 | + +DeepSeek Harness 子进程使用产品启动器和一个显式的绝对路径 `DSH_HOME`。隔离 home 可防止嵌套 runtime 发现启动者个人的 profile 或凭据;通用 ACP provider 不会把这一要求强加给非 DSH agent。 ```yaml - id: subagent-acp name: '@deepseek-ai/dsh-subagent-acp' config: providerName: acp - command: node - args: ['--import', 'tsx', './packages/examples/acp-demo/src/bin.ts', '--config', './examples/acp-agent/cordis.yml'] + command: dsh + args: ['--profile', 'acp', '--patch', '/absolute/path/to/acp.patch.yml'] permission: reject env: + DSH_HOME: /absolute/path/to/isolated-child-home DEEPSEEK_API_KEY: !!js process.env.DEEPSEEK_API_KEY ``` @@ -57,9 +60,9 @@ ACP 不声明任何启动时能力,因为当前进程无法强制执行远程 ## 进程边界 -子进程经由 [`dsh-subprocess`](../../subprocess/subprocess/README.md) seam spawn:共享的凭据清除先移除疑似凭据的环境变量和环境中已有的 `DSH_*` 名称,显式 `config.env` 值在清除之后合并(有意转发的 `DEEPSEEK_API_KEY` 会保留下来,`DSH_PERMISSION_MODE` 这类 `DSH_*` 部署事实也以同样的方式到达子进程——清除只丢弃其陈旧的同名环境值),stderr 会继承到父进程自身的流,dispose 则先应用本插件的 EOF 时间窗,再由子进程责任方执行 SIGTERM→SIGKILL 升级并等待整棵进程树退出。ACP 协议格式(wire format)是真正的序列化边界;同进程 subagent 值不会为防御目的而克隆。 +子进程经由 [`dsh-subprocess`](../../subprocess/subprocess/README.zh.md) seam spawn:共享的凭据清除先移除疑似凭据的环境变量和环境中已有的 `DSH_*` 名称,显式 `config.env` 值在清除之后合并(有意转发的 `DEEPSEEK_API_KEY` 会保留下来,`DSH_PERMISSION_MODE` 这类 `DSH_*` 部署事实也以同样的方式到达子进程——清除只丢弃其陈旧的同名环境值),stderr 会继承到父进程自身的流,dispose 则先应用本插件的 EOF 时间窗,再由子进程责任方执行 SIGTERM→SIGKILL 升级并等待整棵进程树退出。ACP 协议格式(wire format)是真正的序列化边界;同进程 subagent 值不会为防御目的而克隆。 -本包没有默认导出。否则 Cordis loader 的解包会隐藏具名 `inject` 元数据;见[事故复盘(postmortem)0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md)。 +本包没有默认导出。否则 Cordis loader 的解包会隐藏具名 `inject` 元数据;见[事故复盘(postmortem)0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.zh.md)。 ## 模型体验 @@ -93,7 +96,7 @@ ACP 不声明任何启动时能力,因为当前进程无法强制执行远程 ## 已知限制与暂缓事项 -- **每次运行使用全新进程**:持久进程池属于后续优化(见 [seam Agent Note](../../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.md))。 +- **每次运行使用全新进程**:持久进程池属于后续优化(见 [seam Agent Note](../../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.zh.md))。 - **仅支持本地工作区**:解析后的 cwd 是交给同一台机器上子进程的本地路径;远程 ACP agent 的工作区映射需要独立的后端能力,此处尚未设计这种能力。 - **不支持可选启动时能力**:该提供方无法在远程进程内应用本地 harness 的 `outputSchema`、深度上限、工具过滤器或 persona,因此不会声明这些能力;服务会拒绝需要它们的请求。 - **只收集已提交的 `agent_message_chunk` 文本**:自动化服务器把推理(reasoning)、工具活动、计划和其他 trace 数据保留在子 agent 会话日志中,不通过 ACP 发出。 diff --git a/packages/subagent/subagent-acp/package.json b/packages/subagent/subagent-acp/package.json index 72f725f040..ff50f40a87 100644 --- a/packages/subagent/subagent-acp/package.json +++ b/packages/subagent/subagent-acp/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subagent-acp", "description": "Out-of-process ACP subagent backend: drives a child agent in a spawned subprocess over the Agent Client Protocol", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -42,7 +42,7 @@ "@deepseek-ai/cordis": "workspace:^" }, "dependencies": { - "@agentclientprotocol/sdk": "0.25.1", + "@agentclientprotocol/sdk": "1.4.0", "@deepseek-ai/schemastery": "workspace:^" }, "devDependencies": { diff --git a/packages/subagent/subagent-acp/src/run.ts b/packages/subagent/subagent-acp/src/run.ts index 5ba7bc1718..445e7d0de9 100644 --- a/packages/subagent/subagent-acp/src/run.ts +++ b/packages/subagent/subagent-acp/src/run.ts @@ -11,15 +11,11 @@ import { randomUUID } from 'node:crypto' import { Readable as NodeReadable, Writable as NodeWritable } from 'node:stream' import { - ClientSideConnection, + client as createAcpClientApp, + methods, ndJsonStream, PROTOCOL_VERSION, - type Agent as AcpAgent, - type Client, type ContentBlock as AcpContentBlock, - type RequestPermissionRequest, - type RequestPermissionResponse, - type SessionNotification, type StopReason, } from '@agentclientprotocol/sdk' import type { ContentBlock } from '@deepseek-ai/dsh-llm' @@ -239,8 +235,8 @@ export async function startAcpRun(request: SubagentStartRequest, spec: AcpRunSpe // Shared mutable state keeps cancellation visible across async closures. const flags = { cancelled: false } - const makeClient = (_agent: AcpAgent): Client => ({ - sessionUpdate(params: SessionNotification): Promise { + const clientApp = createAcpClientApp({ name: 'deepseek-harness-subagent-acp' }) + .onNotification(methods.client.session.update, ({ params }) => { const update = params.update if (update.sessionUpdate === 'agent_message_chunk') { fold.pushText(acpContentText(update.content)) @@ -248,8 +244,8 @@ export async function startAcpRun(request: SubagentStartRequest, spec: AcpRunSpe // Other updates (thoughts, tool calls, plans) are consumed but not // surfaced — the subagent returns only its final answer. return Promise.resolve() - }, - requestPermission(params: RequestPermissionRequest): Promise { + }) + .onRequest(methods.client.session.requestPermission, ({ params }) => { // Auto-answer by the configured policy. `allow` selects the first option // whose kind is `allow_once` or `allow_always`; if the child offered none (or we // reject), answer `cancelled` so the child does not proceed. @@ -260,16 +256,13 @@ export async function startAcpRun(request: SubagentStartRequest, spec: AcpRunSpe } } return Promise.resolve({ outcome: { outcome: 'cancelled' } }) - }, - }) + }) - const conn = new ClientSideConnection( - makeClient, - ndJsonStream( - NodeWritable.toWeb(child.stdin) as WritableStream, - NodeReadable.toWeb(child.stdout) as ReadableStream, - ), - ) + const connection = clientApp.connect(ndJsonStream( + NodeWritable.toWeb(child.stdin) as WritableStream, + NodeReadable.toWeb(child.stdout) as ReadableStream, + )) + const agent = connection.agent let sessionId: string | undefined // Cancellation settles the result without waiting for a cooperative child. @@ -281,7 +274,9 @@ export async function startAcpRun(request: SubagentStartRequest, spec: AcpRunSpe signalCancelSettled() // Best-effort ACP cancel; process teardown remains authoritative. /* v8 ignore next */ - if (sessionId !== undefined) void conn.cancel({ sessionId }).catch(() => { /* child gone / no session */ }) + if (sessionId !== undefined) { + void agent.notify(methods.agent.session.cancel, { sessionId }).catch(() => { /* child gone / no session */ }) + } } const onAbort = (): void => { requestCancel() } request.signal.addEventListener('abort', onAbort, { once: true }) @@ -294,16 +289,17 @@ export async function startAcpRun(request: SubagentStartRequest, spec: AcpRunSpe try { await Promise.race([ (async (): Promise => { - await conn.initialize({ + await agent.request(methods.agent.initialize, { protocolVersion: PROTOCOL_VERSION, // Advertise NO optional client capabilities (no fs, no terminal): the // child self-serves in its own process. clientCapabilities: {}, }) - const session = await conn.newSession({ cwd: spec.cwd, mcpServers: [] }) + const session = await agent.request(methods.agent.session.new, { cwd: spec.cwd, mcpServers: [] }) const returnedSessionId: unknown = Reflect.get(session, 'sessionId') if (typeof returnedSessionId !== 'string') throw new Error('ACP child published without a session id') sessionId = returnedSessionId + /* v8 ignore next -- cancelSettled wins the startup race before this post-response guard can settle it. */ if (flags.cancelled) throw new Error('subagent cancelled before the ACP session started') })(), spawnFailed, @@ -326,7 +322,10 @@ export async function startAcpRun(request: SubagentStartRequest, spec: AcpRunSpe // Race the remote turn against local cancellation. const prompt = async (): Promise => { // The startup phase cannot fulfill without assigning the session id. - const promptResult = await conn.prompt({ sessionId: remoteSessionId, prompt: toAcpPrompt(request.prompt) }) + const promptResult = await agent.request(methods.agent.session.prompt, { + sessionId: remoteSessionId, + prompt: toAcpPrompt(request.prompt), + }) return { output: collectOutput(), stopReason: acpStopReason(promptResult.stopReason) } } return await Promise.race([ diff --git a/packages/subagent/subagent-acp/tests/mock-acp-server.ts b/packages/subagent/subagent-acp/tests/mock-acp-server.ts index de5900906a..9e36353931 100644 --- a/packages/subagent/subagent-acp/tests/mock-acp-server.ts +++ b/packages/subagent/subagent-acp/tests/mock-acp-server.ts @@ -55,10 +55,11 @@ import { randomUUID } from 'node:crypto' import { existsSync, writeFileSync } from 'node:fs' import { Readable, Writable } from 'node:stream' import { - AgentSideConnection, + agent as createAcpAgentApp, + methods, ndJsonStream, PROTOCOL_VERSION, - type Agent, + type AgentContext, type CancelNotification, type AuthenticateRequest, type InitializeRequest, @@ -67,6 +68,7 @@ import { type NewSessionResponse, type PromptRequest, type PromptResponse, + type RequestPermissionResponse, type StopReason, } from '@agentclientprotocol/sdk' @@ -93,7 +95,7 @@ const NEWSESSION_GATE = process.env.MOCK_NEWSESSION_READY !== undefined && proce ? { ready: process.env.MOCK_NEWSESSION_READY, go: process.env.MOCK_NEWSESSION_GO } : undefined -function makeAgent(conn: AgentSideConnection): Agent { +function makeAgent() { // Pending cancel resolver for the HANG path: a `session/cancel` resolves the // prompt with `cancelled`. let resolveCancel: ((reason: StopReason) => void) | undefined @@ -104,7 +106,7 @@ function makeAgent(conn: AgentSideConnection): Agent { initialize(_params: InitializeRequest): Promise { return Promise.resolve({ protocolVersion: PROTOCOL_VERSION, - agentCapabilities: { loadSession: false, promptCapabilities: { image: false, audio: false, embeddedContext: false } }, + agentCapabilities: { promptCapabilities: { image: false, audio: false, embeddedContext: false } }, authMethods: [], }) }, @@ -124,7 +126,7 @@ function makeAgent(conn: AgentSideConnection): Agent { // No auth methods advertised; nothing to do. return Promise.resolve() }, - async prompt(params: PromptRequest): Promise { + async prompt(params: PromptRequest, conn: AgentContext): Promise { if (CRASH_ON_PROMPT) process.exit(1) if (WANT_PERMISSION) { // Ask the client to approve before answering; honor its decision. Under @@ -136,11 +138,11 @@ function makeAgent(conn: AgentSideConnection): Agent { { optionId: 'yes', name: 'Allow', kind: 'allow_once' as const }, { optionId: 'no', name: 'Reject', kind: 'reject_once' as const }, ] - const decision = await conn.requestPermission({ + const decision = await conn.request(methods.client.session.requestPermission, { sessionId: params.sessionId, toolCall: { toolCallId: 'mock-call', title: 'mock side effect' }, options, - }) + }) as RequestPermissionResponse if (decision.outcome.outcome === 'cancelled') { return { stopReason: 'cancelled' } } @@ -148,14 +150,14 @@ function makeAgent(conn: AgentSideConnection): Agent { // Optionally emit a NON-message update first (a thought), so the client's // sessionUpdate sees an update it must consume-but-not-accumulate. if (THOUGHT) { - await conn.sessionUpdate({ + await conn.notify(methods.client.session.update, { sessionId: params.sessionId, update: { sessionUpdate: 'agent_thought_chunk', content: { type: 'text', text: 'thinking…' } }, }) } // Stream the canned assistant text as one chunk (or, under MOCK_ECHO_CWD, // the observable process cwd + announced session cwd). - await conn.sessionUpdate({ + await conn.notify(methods.client.session.update, { sessionId: params.sessionId, update: { sessionUpdate: 'agent_message_chunk', @@ -195,13 +197,20 @@ function makeAgent(conn: AgentSideConnection): Agent { } } -new AgentSideConnection( - makeAgent, - ndJsonStream( +const implementation = makeAgent() +createAcpAgentApp({ name: 'dsh-subagent-acp-test-agent' }) + .onRequest(methods.agent.initialize, ({ params }) => implementation.initialize(params)) + .onRequest(methods.agent.authenticate, async ({ params }) => { + await implementation.authenticate(params) + return {} + }) + .onRequest(methods.agent.session.new, ({ params }) => implementation.newSession(params)) + .onRequest(methods.agent.session.prompt, ({ params, client }) => implementation.prompt(params, client)) + .onNotification(methods.agent.session.cancel, ({ params }) => implementation.cancel(params)) + .connect(ndJsonStream( Writable.toWeb(process.stdout) as WritableStream, Readable.toWeb(process.stdin) as ReadableStream, - ), -) + )) // Under MOCK_TRAP_SIGTERM, ignore SIGTERM and keep stdin open so the process // neither quiesces on EOF nor dies on the graceful signal — exercising the diff --git a/packages/subagent/subagent-acp/tests/subagent-acp.e2e.ts b/packages/subagent/subagent-acp/tests/subagent-acp.e2e.ts index 9654df5feb..e54b50d647 100644 --- a/packages/subagent/subagent-acp/tests/subagent-acp.e2e.ts +++ b/packages/subagent/subagent-acp/tests/subagent-acp.e2e.ts @@ -11,29 +11,33 @@ import { resolveExampleLaunch } from '@deepseek-ai/dsh-loader-smoke' import * as acp from '../src/index.ts' /** - * With-key cross-process boundary proof: the backend spawns the real acp-agent example, speaks ACP over + * With-key cross-process boundary proof: the backend spawns the real dsh ACP profile, speaks ACP over * stdio, and returns its real model answer. This is the out-of-process counterpart to in-process * spawn coverage and self-skips without `DEEPSEEK_API_KEY`. */ -// The real acp-agent example: its bin + cordis.yml (the live DeepSeek config). -const binScript = fileURLToPath(new URL('../../../examples/acp-demo/src/bin.ts', import.meta.url)) +// The real ACP profile: dsh plus the example's live DeepSeek patch. +const binScript = fileURLToPath(new URL('../../../../apps/cli/src/bin.ts', import.meta.url)) const exampleConfig = fileURLToPath(new URL('../../../../examples/acp-agent/cordis.yml', import.meta.url)) const repoTsconfig = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) -// How to launch the child acp-agent (src via tsx / lib via plain node, per DSH_EXAMPLE_MODE). +// How to launch the child ACP profile (src via tsx / lib via plain node, per DSH_EXAMPLE_MODE). // The subprocess seam scrubs ambient creds while spec.env merges after it, so the model key is // forwarded explicitly; TSX_TSCONFIG_PATH is added by the resolver in src mode only. -const childLaunch = resolveExampleLaunch({ - srcBin: binScript, - configArgs: ['--config', exampleConfig], - tsconfigPath: repoTsconfig, - env: { - ...process.env.DEEPSEEK_API_KEY !== undefined ? { DEEPSEEK_API_KEY: process.env.DEEPSEEK_API_KEY } : {}, - ...process.env.DEEPSEEK_BASE_URL !== undefined ? { DEEPSEEK_BASE_URL: process.env.DEEPSEEK_BASE_URL } : {}, - DSH_PERMISSION_MODE: 'danger-full-access', - }, -}) +function resolveChildLaunch(dshHome: string) { + return resolveExampleLaunch({ + srcBin: binScript, + sourceImport: 'tsx/esm', + configArgs: ['--profile', 'acp', '--patch', exampleConfig], + tsconfigPath: repoTsconfig, + env: { + ...process.env.DEEPSEEK_API_KEY !== undefined ? { DEEPSEEK_API_KEY: process.env.DEEPSEEK_API_KEY } : {}, + ...process.env.DEEPSEEK_BASE_URL !== undefined ? { DEEPSEEK_BASE_URL: process.env.DEEPSEEK_BASE_URL } : {}, + DSH_HOME: dshHome, + DSH_PERMISSION_MODE: 'danger-full-access', + }, + }) +} /** The ACP backend ignores the parent, but the seam requires one. */ const fakeParent = { id: 'parent', session: { header: {} } } as unknown as Agent @@ -51,6 +55,7 @@ afterEach(async () => { describe.skipIf(!process.env.DEEPSEEK_API_KEY)('ACP backend with-key e2e (drive our own acp-agent)', () => { it('drives the real acp-agent example process to answer a prompt', async () => { workdir = await mkdtemp(join(tmpdir(), 'dsh-subagent-acp-e2e-')) + const childLaunch = resolveChildLaunch(join(workdir, '.dsh-child')) ctx = new Context() await ctx.plugin(SubagentRuntime) await ctx.plugin(LocalSubprocessRuntime) @@ -81,6 +86,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('ACP backend with-key e2e (drive it('drives the child to do real file work via its own bash tool', async () => { workdir = await mkdtemp(join(tmpdir(), 'dsh-subagent-acp-e2e-')) + const childLaunch = resolveChildLaunch(join(workdir, '.dsh-child')) ctx = new Context() await ctx.plugin(SubagentRuntime) await ctx.plugin(LocalSubprocessRuntime) diff --git a/packages/subagent/subagent-claude-code/README.i18n.yaml b/packages/subagent/subagent-claude-code/README.i18n.yaml index b96d1751e0..ff129c60e2 100644 --- a/packages/subagent/subagent-claude-code/README.i18n.yaml +++ b/packages/subagent/subagent-claude-code/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/subagent-claude-code/README.md -README.md: 17b14e847baea3eadda7129b5e49f5e65b668cc8 -README.zh.md: 5c970f623c794e91ec2c4a6b2c71c6b28c887485 +README.md: 0260d9c82dee82541e5b60ff7fe2c323cf9b7331 +README.zh.md: 484241f1ca23f2b8ee843b6f412d0f779832d400 diff --git a/packages/subagent/subagent-claude-code/README.md b/packages/subagent/subagent-claude-code/README.md index 17b14e847b..0260d9c82d 100644 --- a/packages/subagent/subagent-claude-code/README.md +++ b/packages/subagent/subagent-claude-code/README.md @@ -2,21 +2,21 @@ English | [中文](README.zh.md) -This package registers the fixed `claude-code` subagent provider. Each accepted run invokes the official Claude Agent SDK in the delegating Session's workspace, resolves the native `claude` executable through the shared subprocess service, submits one self-contained text task, and returns only the final answer through the shared [`dsh-subagent`](../subagent/README.md) result contract. +This package registers a Profile-named Claude Code subagent provider whose default name is `claude-code`. Each accepted run invokes the official Claude Agent SDK in the delegating Session's workspace, lets the pinned SDK select its installed platform CLI, submits one self-contained text task, and returns either the strict final answer or a separate safe failure diagnostic through the shared [`dsh-subagent`](../subagent/README.md) result contract. ## Start and ownership `start(request)` accepts only a non-empty sequence of text blocks and derives the child cwd from the parent Session. It creates one private `AbortController`, calls the official SDK `query()`, and publishes the run only after the SDK's `spawnClaudeCodeProcess` hook has supplied a live CLI handle owned by [`dsh-subprocess`](../../subprocess/subprocess/README.md). A failure or cancellation before publication closes the query, terminates any acquired process tree, waits for it to exit, and rejects `start()`. -The SDK receives the exact concatenated text task. The provider iterates the complete SDK message stream and accepts only a `result` message with `subtype: "success"`, `is_error: false`, and a nonblank `result`, followed by normal iterator completion. Every SDK error subtype, an error-marked success, a missing answer, iterator failure, protocol failure, or process failure maps to `error`; the provider produces neither `max-tokens` nor `refusal`. +The SDK receives the exact concatenated text task. The provider iterates the complete SDK message stream and accepts only a `result` message with `subtype: "success"`, `is_error: false`, and a nonblank `result`, followed by normal iterator completion. Every failure still maps to `error`: the four error subtypes in Agent SDK 0.3.220 retain their exact category, an error-marked or blank success becomes `invalid-success`, a missing result becomes `missing-result`, an unclassified query failure becomes `unknown`, and an early CLI exit becomes `process-exit`. The diagnostic also names the current `query-start`, `query-run`, `process`, or `teardown` stage and independently includes an observed exit code and signal. The provider produces neither `max-tokens` nor `refusal`. -Local cancellation wins the result race and maps to `aborted`. `dispose()` is idempotent: it aborts the run, asks the SDK query to close, invokes the shared process-tree termination escalation, and waits for whole-tree exit. SDK graceful close expresses protocol intent; the subprocess handle remains the authority for process quiescence. Result failure and independent teardown failure remain separate. +Local cancellation wins the result race and maps to `aborted` without a failure diagnostic. `dispose()` is idempotent: it aborts the run, asks the SDK query to close, invokes the shared process-tree termination escalation, and waits for whole-tree exit. SDK graceful close expresses protocol intent; the subprocess handle remains the authority for process quiescence. Startup and teardown rejections expose the same fixed safe stage and process facts through their Error message, while the original product or Host error remains on the internal cause chain and in the Provider's Host log. Result failure and independent teardown failure remain separate. ## Native settings and interaction -The provider deliberately omits the SDK `settingSources` option. The official SDK therefore reads the host's normal user, project, and local Claude settings relative to the parent Session cwd, including native account state and product configuration. The provider neither copies nor filters those files and does not create or modify login state. +The provider deliberately omits the SDK `settingSources` option. The official SDK therefore reads the host's normal user, project, and local Claude settings relative to the parent Session cwd, including native account state and product configuration. The provider neither copies nor filters those files and does not create or modify login state. The Profile-selected `permissionMode` is the one query-level override: Claude Code still owns its settings and sandbox, while the selected native mode decides how this unattended query handles permission checks. -Each query sets `persistSession: false` and disables `AskUserQuestion`. It supplies no `canUseTool`, elicitation, or dialog callback, so unattended interactions fail through the SDK instead of waiting for a user interface this provider does not own. +Each query sets `persistSession: false` and disables `AskUserQuestion`. Except in bypass mode, `canUseTool` immediately denies requests that still require human approval. Plan mode also places `ExitPlanMode` in the SDK's `disallowedTools`, so native settings cannot pre-approve a transition back to execution and the model must return the completed plan as its final answer. MCP elicitation is declined, the known refusal fallback dialog is cancelled, and undeclared dialog kinds use the SDK's no-dialog failure behavior. These decisions never wait for a user interface. When both facts contribute to a failed run, `SubagentResult.diagnostic` contains the structured failure line first and the latest safe permission decision second; the shared result boundary limits the complete text to 4096 UTF-8 bytes. Successful and locally cancelled runs expose neither captured fact. ## Capabilities and context @@ -26,33 +26,83 @@ The provider advertises no optional start-time capabilities and reports `inherit | Key | Default | Meaning | |---|---|---| +| `providerName` | `claude-code` | Non-empty registry name on `ctx.subagents`; each mounted instance needs a unique value. | | `env` | `{}` | Explicit SDK/CLI environment layered over the shared credential-scrubbed parent environment. | +| `permissionMode` | `dontAsk` | Native non-interactive permission policy fixed for every run from this Provider instance. | | `disposeGraceMs` | `3000` | Positive finite grace in milliseconds, no greater than [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md), between the shared process-tree owner's termination tiers; disposal then waits for whole-tree exit. | -Production resolves `claude` from the subprocess execution world's credential-scrubbed `PATH`, with explicit `env` entries applied, and passes the resulting path to the SDK as `pathToClaudeCodeExecutable`. On Windows, a resolved `.cmd` or `.bat` path is carried as a quoted, per-spawn environment value that `cmd.exe /v:off` expands once, so valid path metacharacters remain data. The pinned SDK's fixed flags then occupy cmd's command tail and contain no cmd metacharacters; they are not ordinary Windows argv. Native settings and authentication remain authoritative. The plugin does not install another CLI, select a model, create a product home, log in, or probe an account. Credential-shaped ambient variables are removed before the explicit `env` overlay is applied, so an API key or token intended for the child must be supplied there. Non-credential endpoint variables such as `ANTHROPIC_BASE_URL`, along with ordinary ambient values such as `PATH` and `HOME`, remain inherited unless overridden. +| `permissionMode` value | Native behavior | +|---|---| +| `dontAsk` | Deny operations that are not already authorized instead of prompting. | +| `acceptEdits` | Accept file edits; any remaining permission prompt is denied by the unattended callback. | +| `auto` | Let Claude Code's native classifier allow or deny permission requests. | +| `plan` | Run in native planning mode, deny execution approval, and return the completed plan as the final answer. | +| `bypassPermissions` | Explicitly set the SDK's dangerous confirmation and bypass permission checks. | -Shipped profiles load this provider once on the host and start no Claude process until a tool call. Full Agent Presets carry the tool row below with `disabled: true`; copy a preset and remove that field to expose `subagent_claude_code` only to agents composed from the copy. A custom host composition can still use both rows directly. +Production omits `pathToClaudeCodeExecutable`, so Agent SDK 0.3.220 selects the matching native `claude` or `claude.exe` from its own platform package and passes that absolute command through the custom-spawn hook to `dsh-subprocess`. The provider does not inspect `PATH`, implement platform selection, or fall back to a host `claude`. Native settings and authentication remain authoritative, while `permissionMode` is the only query-level policy override. The plugin does not select a model, create a product home, log in, or probe an account. Credential-shaped ambient variables are removed before the explicit `env` overlay is applied, so an API key or token intended for the child must be supplied there. Non-credential endpoint variables such as `ANTHROPIC_BASE_URL`, along with ordinary ambient values such as `PATH` and `HOME`, remain inherited unless overridden; `PATH` does not choose the Claude executable. + +This package is an optional Profile Bundle. Install it into the target Profile, then restart that Profile; installation brings the pinned Agent SDK and one compatible platform CLI payload into that Profile, while the declared `cordis.patch.yml` layer registers only the dormant `claude-code` Host provider and starts no Claude process. Removing the package withdraws that provider and its private runtime closure on the next Profile start. + +```sh +dsh plugin --profile add @deepseek-ai/dsh-subagent-claude-code +dsh plugin --profile remove @deepseek-ai/dsh-subagent-claude-code +dsh --profile +``` + +Installation controls Host availability, not model permission. The Bundle supplies the dormant default `claude-code` row; the Profile may replace that row's complete config or mount additional rows with distinct `providerName`, `permissionMode`, and `env` values. Loading an instance starts no Claude process until a bound tool calls it. Each `dsh-tool-subagent` row names one provider and needs its own `toolName`, so the model sees static tools rather than a dynamic provider selector. Full Agent Presets carry a matching default product tool row with `disabled: true`; copy a preset and remove that field to expose `subagent_claude_code` only to agents composed from the copy. Its `one-shot` policy keeps omitted or `false` `run_in_background` calls in the foreground, while explicit `true` returns a parent-owned Job id for `job_output` or `job_kill`. The base host and full presets already provide the generic Job registry and controls. + +The standalone composition below shows the complete explicit capability. A Profile based on `@deepseek-ai/dsh-base` keeps its existing Job rows, adds the product provider and tool rows, and does not mount duplicate Job services. ```yaml -- id: subagent-claude-code +- id: subagent-claude-safe name: '@deepseek-ai/dsh-subagent-claude-code' config: + providerName: claude-safe + permissionMode: dontAsk env: ANTHROPIC_API_KEY: !!js process.env.ANTHROPIC_API_KEY -- id: tool-subagent-claude-code +- id: subagent-claude-bypass + name: '@deepseek-ai/dsh-subagent-claude-code' + config: + providerName: claude-bypass + permissionMode: bypassPermissions + env: + ANTHROPIC_API_KEY: !!js process.env.ANTHROPIC_API_KEY +``` + +```yaml +- id: jobs + name: '@deepseek-ai/dsh-jobs-local' + +- id: tool-jobs + name: '@deepseek-ai/dsh-tool-jobs' + +- id: tool-subagent-claude-safe name: '@deepseek-ai/dsh-tool-subagent' disabled: true config: - provider: claude-code - toolName: subagent_claude_code - enableRunInBackground: false + provider: claude-safe + toolName: subagent_claude_safe + backgroundMode: one-shot + maxDepth: provider-managed + +- id: tool-subagent-claude-bypass + name: '@deepseek-ai/dsh-tool-subagent' + config: + provider: claude-bypass + toolName: subagent_claude_bypass + backgroundMode: one-shot maxDepth: provider-managed ``` ## Product compatibility and evidence -The runtime dependency is pinned to `@anthropic-ai/claude-agent-sdk@0.3.220`. Production runs the native `claude` installation. The keyless real-product test uses the SDK-distributed Claude Code 2.1.220 CLI as a deterministic fixture, routed through the same native executable-resolution and Windows batch-shim path; it does not claim compatibility with every independently installed version. Loader composition proves that both product packages coexist without starting either product. +The runtime dependency is pinned to `@anthropic-ai/claude-agent-sdk@0.3.220`, whose eight platform packages carry Claude Code 2.1.220. A normal install selects one payload for the current OS, CPU, and Linux libc. For the current darwin-arm64 payload, `npm pack --dry-run --json` reports 74,858,812 packed bytes and 256,908,856 unpacked bytes; other platforms may differ, and these values are disclosure rather than an installation threshold. The keyless real-product test runs the SDK-selected CLI against a loopback Messages fixture and asserts that the shared subprocess argv begins with that platform package's native executable. Loader composition proves that installing the Bundle registers only the dormant Claude Code provider and starts no product process. + +Installing with optional dependencies omitted, using an unsupported platform, or losing the selected payload leaves provider registration dormant but makes the first delegation fail at the SDK startup boundary. The caller receives the safe `query-start` / `unknown` failure fact; the native payload error remains only on the internal cause chain and in the Provider's Host log. The provider neither probes a host CLI nor retries with one. + +Loader composition proves that the Bundle default, two additional named Claude instances, and the existing Codex package coexist without starting either product. The project owner's identity-scoped distribution authorization covers the official SDK and the official CLI/platform payloads declared by each SDK version. [`THIRD_PARTY_NOTICES.md`](../../../THIRD_PARTY_NOTICES.md) discloses the current optional payload closure without classifying its declared terms as permissive; unrelated non-permissive runtime dependencies continue to fail the notices gate. @@ -62,7 +112,7 @@ The project owner's identity-scoped distribution authorization covers the offici #### What the model sees -The Claude Code child receives the standalone text task as one fresh SDK query. Its workspace is the parent Session cwd, while its model, system instructions, tools, permissions, and authentication come from the host's native Claude settings and product installation. +The Claude Code child receives the standalone text task as one fresh SDK query. Its workspace is the parent Session cwd; its model, system instructions, tools, sandbox, and authentication come from native Claude settings, the selected Provider instance's Profile configuration fixes the query's environment and non-interactive permission mode, and the executable version comes from the Bundle's pinned SDK platform payload. #### Token effect @@ -72,27 +122,28 @@ The child pays for an independent Claude Code context and query. Child tokens do Independent of the parent request cache. Reuse depends only on Claude Code's own model, instructions, tools, native settings, and fresh query. -### Parent tool result, indirectly +### Parent scheduling and results, indirectly #### What the model sees -Through `dsh-tool-subagent`, the parent sees only the strict final Claude Code answer or the consumer's exact error for a non-completed result. Claude Code reasoning, tool activity, intermediate messages, stderr, workspace diffs, usage, and product ids are not copied into the parent Session. +Through `dsh-tool-subagent`, a foreground call gives the parent the strict final Claude Code answer or an error containing the stop reason and optional safe diagnostic for a non-completed result. That diagnostic can distinguish the fixed SDK error category, lifecycle stage, and observed process outcome without copying raw product text. A background call first returns a Job id; the generic job controls later deliver a completion notice, expose the same final answer or failed status detail through `job_output`, and let `job_kill` request cancellation. Claude Code reasoning, tool activity, intermediate messages, stderr, workspace diffs, usage, product ids, tool inputs, and raw protocol payloads are not copied into the parent Session. #### Token effect -Parent input grows only by the final answer or error retained in the tool result. This provider adds no parent tool schema by itself. +Foreground input grows by the retained final answer or error. Background input also includes the start acknowledgement, completion notice, and any `job_output`, `job_kill`, or later status results; child tokens still do not enter the parent context. This provider adds no parent tool schema by itself. #### KV Cache effect -Append-only: the new tool result follows the reusable parent request prefix. +Append-only: foreground adds one result after the reusable parent prefix, while background appends the Job acknowledgement, notice, and later control or collection results. Background scheduling can add a notice-driven turn, but none of these messages rewrites the earlier prefix. ## Known Limitations and Deferred Work - **One fresh query and process per run** — there is no continuation, resume, pooling, progress stream, or product-session persistence. +- **Static instance selection** — Profile rows fix provider names and tool bindings; calls cannot choose a provider dynamically, and every exposed tool needs a unique `toolName`. - **Host settings are intentionally authoritative** — project and user settings can change model, tools, and behavior; the provider does not provide a filtered or hermetic production mode. -- **Product installation and account state remain native** — a missing or incompatible `claude`, configuration error, or authentication failure is surfaced as a startup or run error; the plugin provides no installer or login flow. -- **The SDK platform CLI remains in the install closure** — production ignores it in favor of the host `claude`, but the current SDK optional dependency is still installed and supplies the keyless compatibility fixture. Removing that payload belongs to the separate product installation-closure follow-up. -- **No human interaction path** — `AskUserQuestion` is disabled and other interactive callbacks are absent, so tasks requiring new approval or input fail instead of suspending. -- **Final text only** — reasoning, intermediate messages, tool traffic, usage, stderr, and workspace diffs remain product-local. +- **Authentication and account state remain native** — the Bundle supplies the CLI but does not create an account, log in, or rewrite Claude settings; configuration and authentication failures surface with their lifecycle stage and the safe `unknown` fallback rather than a separate public classification. +- **The SDK platform payload is required at delegation time** — installs that omit optional dependencies, unsupported platforms, and missing or damaged payloads fail at the first query; there is no host-CLI fallback. +- **No human interaction path** — `AskUserQuestion` is disabled, permission prompts are denied, MCP elicitation is declined, and blocking dialogs fail closed instead of suspending. +- **Assistant payload is final text only** — a failed run may additionally expose the separate safe diagnostic; reasoning, intermediate messages, tool traffic, usage, stderr, and workspace diffs remain product-local, while generic Job ids, notices, and status come from the shared job runtime. - **No optional shared capabilities** — output schemas, child personas, tool filtering, and harness depth enforcement are rejected by the shared service for this provider. - **No wall-clock timeout or side-effect rollback** — the caller cancels long work, and files or external systems changed before cancellation are not restored. diff --git a/packages/subagent/subagent-claude-code/README.zh.md b/packages/subagent/subagent-claude-code/README.zh.md index 5c970f623c..484241f1ca 100644 --- a/packages/subagent/subagent-claude-code/README.zh.md +++ b/packages/subagent/subagent-claude-code/README.zh.md @@ -2,21 +2,21 @@ [English](README.md) | 中文 -本包(package)注册固定的 `claude-code` subagent 提供方。每次接受运行请求后,它都会在发起委托的会话工作区中调用官方 Claude Agent SDK,通过共享子进程服务解析原生 `claude` 可执行文件,提交一个自包含的文本任务,并通过共享的 [`dsh-subagent`](../subagent/README.md) 结果约定仅返回最终答案。 +本包(package)注册由 Profile 命名、默认名称为 `claude-code` 的 Claude Code subagent 提供方。每次接受运行请求后,它都会在发起委托的会话工作区中调用官方 Claude Agent SDK,让锁定版本的 SDK 选择随包安装的平台 CLI,提交一个自包含的文本任务,并通过共享的 [`dsh-subagent`](../subagent/README.zh.md) 结果约定返回严格的最终答案或独立的安全失败诊断。 ## 启动与所有权 -`start(request)` 只接受非空的文本块序列,并根据父会话确定子级 cwd。它会创建一个私有 `AbortController`,调用官方 SDK 的 `query()`,并仅在 SDK 的 `spawnClaudeCodeProcess` 钩子已经提供由 [`dsh-subprocess`](../../subprocess/subprocess/README.md) 管理的活动 CLI 句柄后发布此次运行。若在发布前发生失败或取消,它会关闭 query、终止所有已取得的进程树并等待其退出,然后拒绝 `start()` 调用。 +`start(request)` 只接受非空的文本块序列,并根据父会话确定子级 cwd。它会创建一个私有 `AbortController`,调用官方 SDK 的 `query()`,并仅在 SDK 的 `spawnClaudeCodeProcess` 钩子已经提供由 [`dsh-subprocess`](../../subprocess/subprocess/README.zh.md) 管理的活动 CLI 句柄后发布此次运行。若在发布前发生失败或取消,它会关闭 query、终止所有已取得的进程树并等待其退出,然后拒绝 `start()` 调用。 -SDK 接收由文本块原样拼接成的任务。提供方会完整迭代 SDK 消息流,而且只接受满足以下条件的 `result` 消息:其 `subtype: "success"`、`is_error: false` 且 `result` 非空白,之后迭代器还须正常结束。所有 SDK 错误子类型、标记为错误的成功消息、缺失答案、迭代器失败、协议失败或进程失败都映射为 `error`;该提供方不会产生 `max-tokens` 或 `refusal`。 +SDK 接收由文本块原样拼接成的任务。提供方会完整迭代 SDK 消息流,而且只接受满足以下条件的 `result` 消息:其 `subtype: "success"`、`is_error: false` 且 `result` 非空白,之后迭代器还须正常结束。所有失败仍映射为 `error`:Agent SDK 0.3.220 的四种错误子类型保留准确类别;标记为错误或内容空白的成功消息成为 `invalid-success`;缺失结果成为 `missing-result`;未分类的 query 失败成为 `unknown`;CLI 提前退出成为 `process-exit`。诊断还会注明当前 `query-start`、`query-run`、`process` 或 `teardown` 阶段,并分别保留已观测到的退出码与信号。该提供方不会产生 `max-tokens` 或 `refusal`。 -本地取消会在结果竞态中胜出并映射为 `aborted`。`dispose()`(资源释放)具有幂等性:它会中止此次运行、请求 SDK query 关闭、调用共享的进程树逐级终止机制,并等待整棵进程树退出。SDK 的优雅关闭只表达协议意图;进程是否完全停稳仍以子进程句柄为准。结果失败与独立的清理失败仍彼此分离。 +本地取消会在结果竞态中胜出并映射为 `aborted`,且不附带失败诊断。`dispose()`(资源释放)具有幂等性:它会中止此次运行、请求 SDK query 关闭、调用共享的进程树逐级终止机制,并等待整棵进程树退出。SDK 的优雅关闭只表达协议意图;进程是否完全停稳仍以子进程句柄为准。启动与清理拒绝会在 Error 消息中公开同样固定的安全阶段和进程事实,而原始产品或 Host 错误只保留在内部 cause 链与提供方的 Host 日志中。结果失败与独立的清理失败仍彼此分离。 ## 原生设置与交互 -提供方故意省略 SDK 的 `settingSources` 选项。因此,官方 SDK 会相对于父会话 cwd 读取宿主机常规的用户、项目和本地 Claude 设置,包括原生账户状态与产品配置。提供方既不复制也不过滤这些文件,也不会创建或修改登录状态。 +提供方故意省略 SDK 的 `settingSources` 选项。因此,官方 SDK 会相对于父会话 cwd 读取宿主机常规的用户、项目和本地 Claude 设置,包括原生账户状态与产品配置。提供方既不复制也不过滤这些文件,也不会创建或修改登录状态。Profile 选择的 `permissionMode` 是唯一的 query 级覆盖:Claude Code 仍拥有其设置与沙箱,而所选原生模式决定这个无人值守 query 如何处理权限检查。 -每次 query 都设置 `persistSession: false` 并禁用 `AskUserQuestion`。提供方不设置 `canUseTool`、elicitation 或对话回调,因此无人值守交互会经 SDK 失败,而不会等待本提供方不负责的用户界面。 +每次 query 都设置 `persistSession: false` 并禁用 `AskUserQuestion`。除 bypass 模式外,`canUseTool` 会立即拒绝仍需人工审批的请求。Plan 模式还会把 `ExitPlanMode` 放入 SDK 的 `disallowedTools`,因此原生 settings 无法预先放行回到执行模式的转换,模型必须把完整计划作为最终答案返回。MCP elicitation 会被拒绝,已知的拒绝回退对话会被取消,未声明的对话类型则使用 SDK 的无对话失败行为。这些决定都不会等待用户界面。当两类事实共同参与一次失败运行时,`SubagentResult.diagnostic` 会先写入结构化失败行,再写入最新的安全权限决定;共享结果边界会把完整文本限制在 4096 个 UTF-8 字节以内。成功运行与本地取消都不会公开已捕获的事实。 ## 能力与上下文 @@ -26,33 +26,83 @@ SDK 接收由文本块原样拼接成的任务。提供方会完整迭代 SDK | 配置键 | 默认值 | 含义 | |---|---|---| +| `providerName` | `claude-code` | `ctx.subagents` 中的非空注册名称;每个已挂载实例都需要唯一值。 | | `env` | `{}` | 显式指定的 SDK/CLI 环境,叠加在由共享机制清除凭证后的父环境之上。 | -| `disposeGraceMs` | `3000` | 共享进程树责任方各终止层级之间的宽限期,单位为毫秒且须为正有限值,并不得大于仓库共享的 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md);随后资源释放会等待整棵进程树退出。 | +| `permissionMode` | `dontAsk` | 为该提供方实例的每次运行固定原生非交互权限策略。 | +| `disposeGraceMs` | `3000` | 共享进程树责任方各终止层级之间的宽限期,单位为毫秒且须为正有限值,并不得大于仓库共享的 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.zh.md);随后资源释放会等待整棵进程树退出。 | -生产环境从子进程执行世界清除凭证后的 `PATH` 解析 `claude`,再应用显式 `env` 条目,并把所得路径作为 `pathToClaudeCodeExecutable` 交给 SDK。在 Windows 上,解析到的 `.cmd` 或 `.bat` 路径会作为带引号、仅供本次 spawn 使用的环境值交给 `cmd.exe /v:off` 展开一次,因此合法路径中的元字符仍只是数据。锁定版本的 SDK 随后把固定命令行选项放在 cmd 的命令尾部;这些选项不含 cmd 元字符,也并不是普通的 Windows argv。原生设置与身份验证继续是权威来源。本插件不安装另一份 CLI、不选择模型、不创建产品主目录、不执行登录,也不探测账户。具有凭证特征的环境变量会在显式 `env` 覆盖生效前被清除,因此供子进程使用的 API 密钥或 token 必须在该配置中显式提供。除非被覆盖,`ANTHROPIC_BASE_URL` 等非凭证端点变量以及 `PATH` 和 `HOME` 等普通环境变量仍会被继承。 +| `permissionMode` 值 | 原生行为 | +|---|---| +| `dontAsk` | 不弹出提示,直接拒绝尚未获授权的操作。 | +| `acceptEdits` | 接受文件编辑;其余权限提示由无人值守回调拒绝。 | +| `auto` | 由 Claude Code 原生分类器允许或拒绝权限请求。 | +| `plan` | 使用原生规划模式,拒绝执行审批,并把完整计划作为最终答案返回。 | +| `bypassPermissions` | 显式设置 SDK 的危险确认并跳过权限检查。 | -随附 profile 会在宿主上加载一次该提供方,而且在工具被调用前不会启动 Claude 进程。完整 Agent Preset 携带下列工具行并设置 `disabled: true`;复制一个 preset 后删除该字段,即可只向由该副本组装的 agent 暴露 `subagent_claude_code`。自定义宿主组装仍可直接使用两条配置行。 +生产环境会省略 `pathToClaudeCodeExecutable`,因此 Agent SDK 0.3.220 会从自己的平台包中选择匹配的原生 `claude` 或 `claude.exe`,再通过 custom-spawn 钩子把该绝对命令交给 `dsh-subprocess`。提供方不会检查 `PATH`、重复实现平台选择,也不会回退到宿主 `claude`。原生设置与身份验证继续是权威来源,而 `permissionMode` 是唯一的 query 级策略覆盖。本插件不选择模型、不创建产品主目录、不执行登录,也不探测账户。具有凭证特征的环境变量会在显式 `env` 覆盖生效前被清除,因此供子进程使用的 API 密钥或 token 必须在该配置中显式提供。除非被覆盖,`ANTHROPIC_BASE_URL` 等非凭证端点变量以及 `PATH` 和 `HOME` 等普通环境变量仍会被继承;`PATH` 不参与选择 Claude 可执行文件。 + +本包是可选的 Profile Bundle。将它安装进目标 Profile 后重启该 Profile;安装会把锁定的 Agent SDK 与一个兼容的平台 CLI 载荷带入该 Profile,而包所声明的 `cordis.patch.yml` 层只注册休眠的 `claude-code` Host provider,不会启动 Claude 进程。移除该包后,下一次 Profile 启动会撤回这一 provider 及其私有运行时闭包。 + +```sh +dsh plugin --profile add @deepseek-ai/dsh-subagent-claude-code +dsh plugin --profile remove @deepseek-ai/dsh-subagent-claude-code +dsh --profile +``` + +安装决定 Host 可用性,而不是模型权限。Bundle 会提供休眠的默认 `claude-code` 配置项;Profile 可以替换该配置项的完整 config,也可以挂载更多具有不同 `providerName`、`permissionMode` 与 `env` 的配置项。加载实例本身不会在绑定工具调用前启动 Claude 进程。每个 `dsh-tool-subagent` 配置项指定一个提供方,并需要独立的 `toolName`,因此模型看到的是静态工具,而不是动态提供方选择器。完整 Agent Preset 携带对应的默认产品工具行并设置 `disabled: true`;复制一个 preset 后删除该字段,即可只向由该副本组装的 agent 暴露 `subagent_claude_code`。其 `one-shot` 策略会让省略 `run_in_background` 或传入 `false` 的调用继续在前台等待,而显式传入 `true` 会返回由父 agent 拥有的 Job ID,供 `job_output` 或 `job_kill` 使用。base host(基础宿主)与完整 preset 已提供通用作业注册表和控制工具。 + +下列独立组装展示完整的显式能力。基于 `@deepseek-ai/dsh-base` 的 Profile 保留已有 Job 配置项,新增产品提供方与工具配置项,而且不重复挂载 Job 服务。 ```yaml -- id: subagent-claude-code +- id: subagent-claude-safe name: '@deepseek-ai/dsh-subagent-claude-code' config: + providerName: claude-safe + permissionMode: dontAsk env: ANTHROPIC_API_KEY: !!js process.env.ANTHROPIC_API_KEY -- id: tool-subagent-claude-code +- id: subagent-claude-bypass + name: '@deepseek-ai/dsh-subagent-claude-code' + config: + providerName: claude-bypass + permissionMode: bypassPermissions + env: + ANTHROPIC_API_KEY: !!js process.env.ANTHROPIC_API_KEY +``` + +```yaml +- id: jobs + name: '@deepseek-ai/dsh-jobs-local' + +- id: tool-jobs + name: '@deepseek-ai/dsh-tool-jobs' + +- id: tool-subagent-claude-safe name: '@deepseek-ai/dsh-tool-subagent' disabled: true config: - provider: claude-code - toolName: subagent_claude_code - enableRunInBackground: false + provider: claude-safe + toolName: subagent_claude_safe + backgroundMode: one-shot + maxDepth: provider-managed + +- id: tool-subagent-claude-bypass + name: '@deepseek-ai/dsh-tool-subagent' + config: + provider: claude-bypass + toolName: subagent_claude_bypass + backgroundMode: one-shot maxDepth: provider-managed ``` ## 产品兼容性与证据 -运行时依赖精确锁定为 `@anthropic-ai/claude-agent-sdk@0.3.220`。生产运行使用原生 `claude` 安装。无密钥真实产品测试使用由 SDK 分发的 Claude Code 2.1.220 CLI 作为确定性 fixture(测试前置数据),并通过同一套原生可执行文件解析路径与 Windows batch shim 路径运行;这项测试不声称兼容每个独立安装的版本。Loader 组合证明两个产品包能够共存且不会启动任一产品。 +运行时依赖精确锁定为 `@anthropic-ai/claude-agent-sdk@0.3.220`,其八个平台包都携带 Claude Code 2.1.220。普通安装会按当前操作系统、CPU 及 Linux libc 选择一个载荷。对于当前 darwin-arm64 载荷,`npm pack --dry-run --json` 报告压缩包为 74,858,812 字节、解包后为 256,908,856 字节;其他平台可能不同,这些数值只用于披露而不是安装阈值。无密钥真实产品测试会让 SDK 选择 CLI,通过回环 Messages fixture 运行它,并断言共享子进程 argv 的首项就是该平台包的原生可执行文件。Loader 组合证明安装该 Bundle 只会注册休眠的 Claude Code provider,不会启动产品进程。 + +如果安装时省略 optional dependencies、当前平台不受支持,或所选载荷缺失,提供方注册仍保持休眠,但第一次委派会在 SDK 启动边界失败。调用方只会收到安全的 `query-start` / `unknown` 失败事实;原生载荷错误只保留在内部 cause 链和提供方 Host 日志中。提供方既不会探测宿主 CLI,也不会用它重试。 + +Loader 组合证明 Bundle 默认实例、两个额外命名 Claude 实例与现有 Codex 包可以共存,而且不会启动任一产品。 限定于项目所有者身份的分发授权涵盖官方 SDK 及每个 SDK 版本声明的官方 CLI/平台载荷。[`THIRD_PARTY_NOTICES.md`](../../../THIRD_PARTY_NOTICES.md) 会披露当前可选载荷闭包,但不会认定其中声明的条款属于宽松许可;其他无关的非宽松运行时依赖仍会使第三方声明门禁失败。 @@ -62,7 +112,7 @@ SDK 接收由文本块原样拼接成的任务。提供方会完整迭代 SDK #### 模型看到的内容 -Claude Code 子级会在一个全新的 SDK query 中接收独立文本任务。它的工作区是父会话 cwd;其模型、系统指令、工具、权限和身份验证来自宿主机原生 Claude 设置与产品安装。 +Claude Code 子级会在一个全新的 SDK query 中接收独立文本任务。它的工作区是父会话 cwd;其模型、系统指令、工具、沙箱和身份验证来自原生 Claude 设置,所选提供方实例的 Profile 配置会固定该 query 的环境与非交互权限模式,而可执行版本来自 Bundle 锁定的 SDK 平台载荷。 #### 对 token 的影响 @@ -72,27 +122,28 @@ Claude Code 子级会在一个全新的 SDK query 中接收独立文本任务。 这与父请求缓存相互独立。能否复用只取决于 Claude Code 自身的模型、指令、工具、原生设置和全新 query。 -### 父级工具结果(间接) +### 父级调度与结果(间接) #### 模型看到的内容 -通过 `dsh-tool-subagent`,父级模型只会看到符合严格成功条件的 Claude Code 最终答案,或者在结果未完成时看到消费方给出的原样错误。Claude Code 的推理、工具活动、中间消息、stderr、工作区差异、用量信息和产品标识符均不会复制到父会话。 +通过 `dsh-tool-subagent`,前台调用会让父级模型看到符合严格成功条件的 Claude Code 最终答案;若结果未完成,错误中会包含终止原因和可选的安全诊断。该诊断可以区分固定 SDK 错误类别、生命周期阶段和已观测的进程结果,而不复制原始产品文本。后台调用会先返回 Job id;随后通用作业控制面会送达完成通知,通过 `job_output` 公开同一最终答案或失败状态 detail,并允许 `job_kill` 请求取消。Claude Code 的推理、工具活动、中间消息、stderr、工作区差异、用量信息、产品标识符、工具输入和原始协议载荷均不会复制到父会话。 #### 对 token 的影响 -父级输入只会增加工具结果中保留的最终答案或错误内容。本提供方自身不添加父级工具 schema。 +前台输入会增加工具结果中保留的最终答案或错误内容。后台输入还会包含启动确认、完成通知,以及 `job_output`、`job_kill` 或后续状态结果;子任务 token 仍不会进入父级上下文。本提供方自身不添加父级工具 schema。 #### 对 KV Cache 的影响 -仅追加:新的工具结果接在可复用的父请求前缀之后。 +仅追加:前台会在可复用的父请求前缀后增加一个结果,后台则会继续追加 Job 启动确认、通知以及后续控制或收集结果。后台调度可能增加一个由通知唤醒的轮次,但这些消息都不会改写更早的前缀。 ## 已知限制与后续工作 - **每次运行均新建一个 query 和一个进程**:不支持续接、恢复、池化、进度流或产品会话持久化。 +- **静态选择实例**:Profile 配置项固定提供方名称与工具绑定;调用无法动态选择提供方,而且每个公开工具都需要唯一的 `toolName`。 - **宿主设置有意保持权威**:项目和用户设置可以改变模型、工具与行为;本提供方不提供经过筛选或与宿主环境隔离的生产模式。 -- **产品安装与账户状态仍由原生机制管理**:`claude` 缺失或不兼容、配置错误或身份验证失败都会呈现为启动错误或运行错误;本插件不提供安装程序或登录流程。 -- **SDK 平台 CLI 仍在安装闭包内**:生产环境会忽略它,改用宿主提供的 `claude`,但当前 SDK 的可选依赖仍会安装,并提供无密钥兼容性 fixture。移除该载荷属于独立的产品安装闭包后续项。 -- **没有人工交互路径**:`AskUserQuestion` 被禁用,其他交互回调也不存在,因此需要新审批或输入的任务会失败而不会挂起。 -- **仅返回最终文本**:推理、中间消息、工具通信、用量信息、stderr 和工作区差异仍只保留在产品内部。 +- **身份验证与账户状态仍由原生机制管理**:Bundle 会提供 CLI,但不会创建账户、登录或改写 Claude 设置;配置与身份验证失败会公开其生命周期阶段与安全的 `unknown` 回退,而不会增加单独的公开分类。 +- **委派时必须存在 SDK 平台载荷**:省略 optional dependencies 的安装、不受支持的平台以及缺失或损坏的载荷都会在第一次 query 时失败;不会回退到宿主 CLI。 +- **没有人工交互路径**:`AskUserQuestion` 被禁用,权限提示会被拒绝,MCP elicitation 会被拒绝,阻塞对话会快速失败而不会挂起。 +- **assistant 载荷仅包含最终文本**:失败运行可以额外公开独立的安全诊断;推理、中间消息、工具通信、用量信息、stderr 和工作区差异仍只保留在产品内部,通用 Job id、通知与状态来自共享作业运行时。 - **没有可选的共享能力**:对于本提供方,共享服务会拒绝输出 schema、子任务角色设定、工具筛选和 harness 深度强制约束。 - **没有按实际经过时间触发的超时或副作用回滚**:长时间运行的工作由调用方取消,且取消前已更改的文件或外部系统不会恢复原状。 diff --git a/packages/subagent/subagent-claude-code/cordis.patch.yml b/packages/subagent/subagent-claude-code/cordis.patch.yml new file mode 100644 index 0000000000..63c0319626 --- /dev/null +++ b/packages/subagent/subagent-claude-code/cordis.patch.yml @@ -0,0 +1,6 @@ +# This optional Profile layer registers the dormant Claude Code provider. Agent +# presets separately decide whether one session receives its delegation tool. + +- insert: + - id: subagent-claude-code + name: '@deepseek-ai/dsh-subagent-claude-code' diff --git a/packages/subagent/subagent-claude-code/package.json b/packages/subagent/subagent-claude-code/package.json index ec5da9e8ac..20dcc8a4ba 100644 --- a/packages/subagent/subagent-claude-code/package.json +++ b/packages/subagent/subagent-claude-code/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subagent-claude-code", "description": "One-shot Claude Code subagent provider over the official Agent SDK", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -28,9 +28,15 @@ "files": [ "lib/index.js", "lib/invariant.js", + "cordis.patch.yml", "lib/types/**/*.d.ts" ], "license": "MIT", + "dsh": { + "bundle": { + "patch": "./cordis.patch.yml" + } + }, "peerDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", @@ -43,7 +49,9 @@ "dependencies": { "@anthropic-ai/sdk": "0.93.0", "@anthropic-ai/claude-agent-sdk": "0.3.220", - "@deepseek-ai/schemastery": "workspace:^" + "@deepseek-ai/schemastery": "workspace:^", + "@modelcontextprotocol/sdk": "^1.29.0", + "zod": "^4.4.3" }, "devDependencies": { "@deepseek-ai/dsh-agent": "workspace:^", diff --git a/packages/subagent/subagent-claude-code/src/index.ts b/packages/subagent/subagent-claude-code/src/index.ts index ccd150b746..741f93a6fb 100644 --- a/packages/subagent/subagent-claude-code/src/index.ts +++ b/packages/subagent/subagent-claude-code/src/index.ts @@ -1,7 +1,7 @@ /** - * Fixed Claude Code one-shot subagent provider. Every accepted run invokes - * the official Agent SDK in the delegating Session's workspace and places - * the SDK-spawned real CLI under the shared subprocess owner. + * Profile-named Claude Code one-shot subagent provider. Every accepted run + * invokes the official Agent SDK in the delegating Session's workspace and + * places the SDK-spawned real CLI under the shared subprocess owner. * * @module @deepseek-ai/dsh-subagent-claude-code */ @@ -18,29 +18,47 @@ import { type SubagentProvider, } from '@deepseek-ai/dsh-subagent' import { + CLAUDE_CODE_PERMISSION_MODES, + DEFAULT_CLAUDE_CODE_PERMISSION_MODE, DEFAULT_DISPOSE_GRACE_MS, + claudeCodeStartupFailure, startClaudeCodeRun, + type ClaudeCodePermissionMode, type ClaudeCodeRunSpec, } from './run.ts' export const name = 'subagent-claude-code' export const inject = ['subagents', 'subprocess'] -/* jscpd:ignore-start -- sibling product providers intentionally expose the - * same two deployment-owned fields without adding a shared config owner. */ -/** Deployment-owned environment and process-release bound. */ +const DEFAULT_PROVIDER_NAME = 'claude-code' + +/* jscpd:ignore-start -- sibling product providers intentionally expose + * overlapping deployment-owned fields without adding a shared config owner. */ +/** Deployment-owned permission, environment, and process-release settings. */ export interface Config { + /** Provider name on `ctx.subagents` (default `claude-code`). */ + providerName?: string /** * Explicit environment entries layered over the subprocess seam's * credential-scrubbed parent environment. */ env?: Record + /** + * Native non-interactive mode fixed for this Provider instance. Defaults to + * `dontAsk`; `acceptEdits` accepts edits, `auto` uses the native classifier, + * `plan` returns a plan without approving execution, and + * `bypassPermissions` explicitly skips permission checks. + */ + permissionMode?: ClaudeCodePermissionMode /** Grace in milliseconds for Claude Code process-tree termination. */ disposeGraceMs?: number } export const Config: z = z.object({ + providerName: z.string().min(1).default(DEFAULT_PROVIDER_NAME), env: z.dict(z.string()).default({}), + permissionMode: z.union([...CLAUDE_CODE_PERMISSION_MODES]) + .default(DEFAULT_CLAUDE_CODE_PERMISSION_MODE), disposeGraceMs: z.number().default(DEFAULT_DISPOSE_GRACE_MS), }) @@ -50,11 +68,11 @@ type ResolvedConfig = Required /* jscpd:ignore-start -- Cordis registration and shared-seam plumbing mirror * the Codex sibling; each product's lifecycle remains package-private. */ class ClaudeCodeProvider implements SubagentProvider { - readonly name = 'claude-code' readonly capabilities: SubagentCapabilities = NO_START_CAPABILITIES readonly inheritsParentContext = false constructor( + readonly name: string, private readonly ctx: Context, private readonly config: ResolvedConfig, ) {} @@ -66,24 +84,36 @@ class ClaudeCodeProvider implements SubagentProvider { 'subagent-claude-code: no working directory for the child — delegate from a parent session that has one', ) } - const executable = await this.ctx.subprocess.resolveExecutable( - 'claude', - this.config.env, - request.signal, - ) - const spec: ClaudeCodeRunSpec = { - cwd: resolveChildCwd( + let cwd: string + try { + cwd = resolveChildCwd( 'subagent-claude-code', undefined, parentCwd, - ), - executable, + ) + } catch (error: unknown) { + if (request.signal.aborted) { + throw new Error( + 'subagent-claude-code: request was aborted before SDK startup', + ) + } + const failure = claudeCodeStartupFailure(error) + this.ctx.logger.warn( + `subagent-claude-code "${this.name}": child start failed: %o`, + failure, + ) + throw failure + } + const spec: ClaudeCodeRunSpec = { + cwd, + permissionMode: this.config.permissionMode, env: this.config.env, disposeGraceMs: this.config.disposeGraceMs, spawn: spawnSpec => this.ctx.subprocess.spawn(spawnSpec), onError: (error, stopReason) => { this.ctx.logger.warn( - `subagent-claude-code: child run failed (${stopReason}): ${error.message}`, + `subagent-claude-code "${this.name}": child run failed (${stopReason}): %o`, + error, ) }, } @@ -92,12 +122,17 @@ class ClaudeCodeProvider implements SubagentProvider { } /** - * Register the fixed `claude-code` provider. + * Register one Profile-named Claude Code provider. * @param ctx - context carrying shared subagent and subprocess services. - * @param config - explicit child environment and disposal grace. + * @param config - registry name, permission mode, child environment, and disposal grace. */ export function apply(ctx: Context, config: Config): void { - const resolved = config as ResolvedConfig + const resolved: ResolvedConfig = { + providerName: config.providerName ?? DEFAULT_PROVIDER_NAME, + env: config.env as Record, + permissionMode: config.permissionMode ?? DEFAULT_CLAUDE_CODE_PERMISSION_MODE, + disposeGraceMs: config.disposeGraceMs as number, + } assertPositiveFinite( 'subagent-claude-code', 'disposeGraceMs', @@ -108,6 +143,10 @@ export function apply(ctx: Context, config: Config): void { `subagent-claude-code: disposeGraceMs must be no greater than ${MAX_TIMER_DELAY_MS}`, ) } - ctx.subagents.registerProvider(new ClaudeCodeProvider(ctx, resolved)) + ctx.subagents.registerProvider(new ClaudeCodeProvider( + resolved.providerName, + ctx, + resolved, + )) } /* jscpd:ignore-end */ diff --git a/packages/subagent/subagent-claude-code/src/process.ts b/packages/subagent/subagent-claude-code/src/process.ts index 1e2a259ca2..e9c3d81156 100644 --- a/packages/subagent/subagent-claude-code/src/process.ts +++ b/packages/subagent/subagent-claude-code/src/process.ts @@ -6,7 +6,6 @@ */ import { EventEmitter } from 'node:events' -import { extname } from 'node:path' import type { SpawnedProcess, SpawnOptions, @@ -14,11 +13,10 @@ import type { import { scrubbedParentEnv, type SubprocessHandle, + type SubprocessOutcome, type SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' -const WINDOWS_BATCH_EXECUTABLE_ENV = 'DSH_CLAUDE_CODE_EXECUTABLE' - function thrown(value: unknown): Error { /* v8 ignore next -- the subprocess seam rejects with Error. */ return value instanceof Error ? value : new Error(String(value)) @@ -43,33 +41,22 @@ export function sdkEnvironmentOverlay( * Translate one official SDK spawn request to the shared process owner. * @param options - command, arguments, workspace, environment, and forwarded signal from the SDK. * @param graceMs - process-tree termination grace. - * @param platform - host platform selecting the Windows batch-shim boundary. * @returns the fully explicit shared subprocess request. - * @remarks The batch-shim path quotes only the resolved executable. The pinned SDK - * supplies fixed flag arguments without cmd metacharacters; cmd reparses that tail. */ export function claudeSpawnSpec( options: SpawnOptions, graceMs: number, - platform: NodeJS.Platform = process.platform, ): SubprocessSpawnSpec { if (options.cwd === undefined || options.cwd.length === 0) { throw new Error('subagent-claude-code: SDK spawn request omitted its workspace') } - const extension = extname(options.command).toLowerCase() - const batchShim = platform === 'win32' && (extension === '.cmd' || extension === '.bat') - const env = sdkEnvironmentOverlay(options.env) - const argv = batchShim - ? ['cmd.exe', '/d', '/v:off', '/s', '/c', `%${WINDOWS_BATCH_EXECUTABLE_ENV}%`, ...options.args] - : [options.command, ...options.args] - if (batchShim) env[WINDOWS_BATCH_EXECUTABLE_ENV] = `"${options.command}"` return { - argv, + argv: [options.command, ...options.args], cwd: options.cwd, stdio: { stdin: 'pipe', stdout: 'pipe', stderr: 'inherit' }, graceMs, signal: options.signal, - env, + env: sdkEnvironmentOverlay(options.env), } } @@ -81,8 +68,7 @@ export class ManagedClaudeCodeProcess implements SpawnedProcess { readonly stdin readonly stdout private readonly events = new EventEmitter() - private exitCodeValue: number | null = null - private signalCodeValue: NodeJS.Signals | null = null + private outcomeValue: SubprocessOutcome | undefined private killRequested = false /** @@ -98,8 +84,7 @@ export class ManagedClaudeCodeProcess implements SpawnedProcess { this.events.on('error', () => {}) void child.done.then( (outcome) => { - this.exitCodeValue = outcome.exitCode - this.signalCodeValue = outcome.signal + this.outcomeValue = outcome this.events.emit('exit', outcome.exitCode, outcome.signal) }, (error: unknown) => { @@ -115,12 +100,17 @@ export class ManagedClaudeCodeProcess implements SpawnedProcess { /** Direct-child exit code, or null while running or after signal exit. */ get exitCode(): number | null { - return this.exitCodeValue + return this.outcomeValue?.exitCode ?? null } /** Direct-child terminating signal, if any. */ get signalCode(): NodeJS.Signals | null { - return this.signalCodeValue + return this.outcomeValue?.signal ?? null + } + + /** Exact managed-process outcome after exit, or undefined while running. */ + get outcome(): SubprocessOutcome | undefined { + return this.outcomeValue } /** @@ -131,8 +121,7 @@ export class ManagedClaudeCodeProcess implements SpawnedProcess { kill(_signal: NodeJS.Signals): boolean { if ( this.killRequested - || this.exitCodeValue !== null - || this.signalCodeValue !== null + || this.outcomeValue !== undefined ) { return false } diff --git a/packages/subagent/subagent-claude-code/src/run.ts b/packages/subagent/subagent-claude-code/src/run.ts index 6c1e0a8dbf..3b0a19073d 100644 --- a/packages/subagent/subagent-claude-code/src/run.ts +++ b/packages/subagent/subagent-claude-code/src/run.ts @@ -28,6 +28,7 @@ import { import { scrubbedParentEnv, type SubprocessHandle, + type SubprocessOutcome, type SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import { @@ -38,21 +39,126 @@ import { /** Default POSIX grace between subprocess termination tiers. */ export const DEFAULT_DISPOSE_GRACE_MS = 3_000 +/** Claude Code permission modes that cannot wait for a human response. */ +export const CLAUDE_CODE_PERMISSION_MODES = [ + 'dontAsk', + 'acceptEdits', + 'auto', + 'plan', + 'bypassPermissions', +] as const satisfies readonly NonNullable[] + +/** Profile-selectable non-interactive Claude Code permission mode. */ +export type ClaudeCodePermissionMode = typeof CLAUDE_CODE_PERMISSION_MODES[number] + +/** Safe default for unattended Claude Code runs. */ +export const DEFAULT_CLAUDE_CODE_PERMISSION_MODE: ClaudeCodePermissionMode = 'dontAsk' + +const SUPPORTED_UNATTENDED_DIALOG_KINDS = [ + 'refusal_fallback_prompt', +] satisfies NonNullable + +type ClaudeCodeErrorSubtype = Exclude + +type ClaudeCodeFailureStage = + | 'query-start' + | 'query-run' + | 'process' + | 'teardown' + +type ClaudeCodeFailureCategory = + | ClaudeCodeErrorSubtype + | 'invalid-success' + | 'missing-result' + | 'process-exit' + | 'unknown' + +interface ClaudeCodeFailureFacts { + readonly stage: ClaudeCodeFailureStage + readonly category: ClaudeCodeFailureCategory + readonly outcome?: SubprocessOutcome | undefined +} + +function failureDiagnostic(facts: ClaudeCodeFailureFacts): string { + const fields = [ + 'product: Claude Code', + `stage: ${facts.stage}`, + `category: ${facts.category}`, + ] + const exitCode = facts.outcome?.exitCode + if (exitCode !== null && exitCode !== undefined) { + fields.push(`exit code: ${exitCode}`) + } + const signal = facts.outcome?.signal + if (signal !== null && signal !== undefined) { + fields.push(`signal: ${signal}`) + } + return `Product subagent failure (${fields.join('; ')})` +} + +class ClaudeCodeFailure extends Error { + constructor( + readonly facts: ClaudeCodeFailureFacts, + cause?: unknown, + ) { + super( + `subagent-claude-code: ${failureDiagnostic(facts)}`, + cause === undefined ? undefined : { cause }, + ) + this.name = 'ClaudeCodeFailure' + } +} + +function sdkFailureCategory( + subtype: string, +): ClaudeCodeErrorSubtype | 'unknown' { + switch (subtype) { + case 'error_during_execution': + case 'error_max_turns': + case 'error_max_budget_usd': + case 'error_max_structured_output_retries': + return subtype + default: + return 'unknown' + } +} + +/** + * Hide an unpublished product startup failure behind fixed safe facts. + * @param cause - original host-side failure retained only on the Error cause chain. + * @returns a rejection safe to expose through the subagent start boundary. + */ +export function claudeCodeStartupFailure(cause: unknown): Error { + return new ClaudeCodeFailure({ + stage: 'query-start', + category: 'unknown', + }, cause) +} + +function unattendedDiagnostic( + mode: ClaudeCodePermissionMode, + request: 'tool permission' | 'MCP elicitation' | 'user dialog', + decision: 'denied' | 'declined' | 'cancelled', + reason: string, +): string { + return `Claude Code unattended decision (mode: ${mode}; request: ${request}; decision: ${decision}): ${reason}` +} + /* jscpd:ignore-start -- sibling providers intentionally keep product-private * run inputs and error normalization instead of adding a shared lifecycle owner. */ /** Fully resolved inputs for one official Claude Agent SDK query. */ export interface ClaudeCodeRunSpec { /** Parent Session workspace supplied to the SDK and real CLI. */ readonly cwd: string - /** Exact native Claude Code executable resolved from the host PATH. */ - readonly executable: string + /** Profile-selected native non-interactive permission mode. */ + readonly permissionMode: ClaudeCodePermissionMode /** Explicit deployment/test environment layered after shared scrubbing. */ readonly env: Record /** Subprocess termination grace passed to the shared process-tree owner. */ readonly disposeGraceMs: number /** Shared subprocess service spawn operation. */ readonly spawn: (spec: SubprocessSpawnSpec) => SubprocessHandle - /** Diagnostic sink for a post-publication error flattened into a result. */ + /** Host diagnostic sink for a product failure kept outside model-visible text. */ readonly onError?: (error: Error, stopReason: SubagentStopReason) => void } @@ -60,6 +166,12 @@ function thrown(value: unknown): Error { /* v8 ignore next -- typed SDK and subprocess failures reject with Error. */ return value instanceof Error ? value : new Error(String(value)) } + +/** Read live request cancellation across awaited startup cleanup. */ +function isAborted(signal: AbortSignal): boolean { + return signal.aborted +} + /* jscpd:ignore-end */ /** @@ -90,15 +202,23 @@ export function textTask(prompt: readonly ContentBlock[]): string { * @returns exact final text for a successful, non-error result. */ export function successfulResult(message: SDKResultMessage): string { - if ( - message.subtype !== 'success' - || message.is_error - || message.result.trim().length === 0 - ) { - const detail = message.subtype === 'success' - ? 'success result was marked as an error or contained no answer' - : message.errors.join('; ') || message.subtype - throw new Error(`subagent-claude-code: Claude Code failed: ${detail}`) + if (message.subtype !== 'success') { + const category = sdkFailureCategory(message.subtype) + const detail = category === 'unknown' + ? undefined + : message.errors.join('; ') + throw new ClaudeCodeFailure( + { stage: 'query-run', category }, + detail === undefined || detail.length === 0 + ? undefined + : new Error(detail), + ) + } + if (message.is_error || message.result.trim().length === 0) { + throw new ClaudeCodeFailure({ + stage: 'query-run', + category: 'invalid-success', + }) } return message.result } @@ -107,18 +227,30 @@ export function successfulResult(message: SDKResultMessage): string { * Consume the complete SDK stream and require one strict success plus normal * iterator completion. * @param query - published official SDK query. + * @param onPermissionDenied - records a safe fact when the SDK reports native denial. + * @param onResult - records that the SDK supplied a terminal result message. * @returns the completed shared result. */ export async function consumeClaudeQuery( query: AsyncIterable, + onPermissionDenied?: () => void, + onResult?: () => void, ): Promise { let answer: string | undefined for await (const message of query) { + if (message.type === 'system' && message.subtype === 'permission_denied') { + onPermissionDenied?.() + continue + } if (message.type !== 'result') continue + onResult?.() answer = successfulResult(message) } if (answer === undefined) { - throw new Error('subagent-claude-code: Claude Code ended without a result') + throw new ClaudeCodeFailure({ + stage: 'query-run', + category: 'missing-result', + }) } return { output: [{ type: 'text', text: answer }], @@ -130,7 +262,8 @@ export async function consumeClaudeQuery( * Close the official query, terminate the managed process tree, and wait for * the subprocess owner to prove it is gone. * @param query - official SDK query, when creation reached that point. - * @param child - shared-service handle that owns the CLI process tree. + * @param child - live shared-service handle that owns the CLI process tree; + * spawn-failed handles settle at the startup boundary instead. */ export async function disposeClaudeCodeChild( query: Pick | undefined, @@ -143,27 +276,25 @@ export async function disposeClaudeCodeChild( failures.push(thrown(error)) } - if (child.pid > 0) { - child.terminate() - try { - await child.waitForExit() - } catch (error: unknown) { - failures.push(thrown(error)) - } - } + child.terminate() try { - await child.done + await child.waitForExit() } catch (error: unknown) { failures.push(thrown(error)) } + const outcome = await child.done const firstFailure = failures[0] - if (failures.length === 1 && firstFailure !== undefined) throw firstFailure - if (failures.length > 1) { - throw new AggregateError( - failures, - 'subagent-claude-code: query and process cleanup failed', - ) + if (firstFailure !== undefined) { + const facts = { + stage: 'teardown', + category: 'unknown', + outcome, + } as const + const cause = failures.length === 1 + ? firstFailure + : new AggregateError(failures, 'Claude Code teardown failures') + throw new ClaudeCodeFailure(facts, cause) } } @@ -171,25 +302,68 @@ export async function disposeClaudeCodeChild( * Build the fixed official SDK options for one one-shot provider run. * @param spec - Workspace, environment, process service, and disposal policy. * @param controller - per-run cancellation owner. - * @param capture - receives the real managed child synchronously from the SDK hook. + * @param capture - receives the shared child and SDK-facing process synchronously. + * @param captureDiagnostic - receives safe facts from unattended interaction callbacks. * @returns options that inherit native settings while disabling persistence and user questions. */ export function claudeQueryOptions( spec: ClaudeCodeRunSpec, controller: AbortController, - capture: (child: SubprocessHandle) => void, + capture: ( + child: SubprocessHandle, + process: ManagedClaudeCodeProcess, + ) => void, + captureDiagnostic: (diagnostic: string) => void, ): Options { return { abortController: controller, cwd: spec.cwd, - pathToClaudeCodeExecutable: spec.executable, env: { ...scrubbedParentEnv(), ...spec.env }, persistSession: false, - disallowedTools: ['AskUserQuestion'], + disallowedTools: spec.permissionMode === 'plan' + ? ['AskUserQuestion', 'ExitPlanMode'] + : ['AskUserQuestion'], + permissionMode: spec.permissionMode, + ...spec.permissionMode === 'bypassPermissions' + ? { allowDangerouslySkipPermissions: true } + : { + canUseTool: () => { + captureDiagnostic(unattendedDiagnostic( + spec.permissionMode, + 'tool permission', + 'denied', + 'the provider does not request human approval', + )) + return Promise.resolve({ + behavior: 'deny' as const, + message: 'This unattended Claude Code subagent cannot request human approval.', + }) + }, + }, + onElicitation: () => { + captureDiagnostic(unattendedDiagnostic( + spec.permissionMode, + 'MCP elicitation', + 'declined', + 'the provider does not collect interactive MCP input', + )) + return Promise.resolve({ action: 'decline' }) + }, + onUserDialog: () => { + captureDiagnostic(unattendedDiagnostic( + spec.permissionMode, + 'user dialog', + 'cancelled', + 'the provider does not render blocking dialogs', + )) + return Promise.resolve({ behavior: 'cancelled' as const }) + }, + supportedDialogKinds: SUPPORTED_UNATTENDED_DIALOG_KINDS, spawnClaudeCodeProcess: (options: SpawnOptions) => { const child = spec.spawn(claudeSpawnSpec(options, spec.disposeGraceMs)) - capture(child) - return new ManagedClaudeCodeProcess(child) + const process = new ManagedClaudeCodeProcess(child) + capture(child, process) + return process }, } } @@ -217,15 +391,43 @@ export async function startClaudeCodeRun( } const onAbort = (): void => { requestCancel() } request.signal.addEventListener('abort', onAbort, { once: true }) + const reportFailure = (error: Error): void => { + try { + spec.onError?.(error, 'error') + } catch { + // Host diagnostic logging cannot replace the product failure. + } + } let child: SubprocessHandle | undefined let query: Query | undefined + let managedProcess: ManagedClaudeCodeProcess | undefined + let diagnostic: string | undefined + const capturePermissionDiagnostic = (value: string): void => { + diagnostic = value + } + const prependFailureDiagnostic = (facts: ClaudeCodeFailureFacts): void => { + const failure = failureDiagnostic(facts) + diagnostic = diagnostic === undefined + ? failure + : `${failure}\n${diagnostic}` + } + const captureChild = ( + captured: SubprocessHandle, + process: ManagedClaudeCodeProcess, + ): void => { + child = captured + managedProcess = process + } try { query = officialQuery({ prompt, - options: claudeQueryOptions(spec, controller, (captured) => { - child = captured - }), + options: claudeQueryOptions( + spec, + controller, + captureChild, + capturePermissionDiagnostic, + ), }) if (child === undefined || child.pid <= 0) { throw new Error( @@ -238,38 +440,135 @@ export async function startClaudeCodeRun( } catch (error: unknown) { request.signal.removeEventListener('abort', onAbort) const cancelledBeforeCleanup = controller.signal.aborted + // Let child.done publish a concurrently observed exit before classification. + await Promise.resolve() + const startupOutcome = managedProcess?.outcome + const startupFacts = { + stage: 'query-start', + category: 'unknown', + outcome: startupOutcome, + } as const + const startupFailure = (cause: unknown = error): ClaudeCodeFailure => new ClaudeCodeFailure( + startupFacts, + thrown(cause), + ) requestCancel() + if (child !== undefined && child.pid <= 0) { + let closeError: Error | undefined + try { + query?.close() + } catch (disposeError: unknown) { + closeError = thrown(disposeError) + } + + let spawnError = thrown(error) + try { + await child.done + } catch (childError: unknown) { + spawnError = thrown(childError) + } + + if (closeError !== undefined) { + const failure = startupFailure(spawnError) + const cleanupFailure = new ClaudeCodeFailure({ + stage: 'teardown', + category: 'unknown', + }, closeError) + const aggregate = new AggregateError( + [failure, cleanupFailure], + `${failure.message}; ${cleanupFailure.message}`, + ) + reportFailure(aggregate) + throw aggregate + } + if (cancelledBeforeCleanup || isAborted(request.signal)) { + throw new Error('subagent-claude-code: request was aborted before SDK startup') + } + const failure = startupFailure(spawnError) + reportFailure(failure) + throw failure + } if (child !== undefined) { try { await disposeClaudeCodeChild(query, child) } catch (disposeError: unknown) { - throw new AggregateError( - [thrown(error), thrown(disposeError)], - 'subagent-claude-code: startup failed and CLI cleanup also failed', + const failure = startupFailure() + const cleanupFailure = thrown(disposeError) + const aggregate = new AggregateError( + [failure, cleanupFailure], + `${failure.message}; ${cleanupFailure.message}`, ) + reportFailure(aggregate) + throw aggregate } } else if (query !== undefined) { try { query.close() } catch (disposeError: unknown) { - throw new AggregateError( - [thrown(error), thrown(disposeError)], - 'subagent-claude-code: startup failed and query cleanup also failed', + const failure = startupFailure() + const cleanupFailure = new ClaudeCodeFailure({ + stage: 'teardown', + category: 'unknown', + }, thrown(disposeError)) + const aggregate = new AggregateError( + [failure, cleanupFailure], + `${failure.message}; ${cleanupFailure.message}`, ) + reportFailure(aggregate) + throw aggregate } } - // oxlint-disable-next-line typescript/no-unnecessary-condition -- the request can abort while process cleanup is awaited. - if (cancelledBeforeCleanup || request.signal.aborted) { + if (cancelledBeforeCleanup || isAborted(request.signal)) { throw new Error('subagent-claude-code: request was aborted before SDK startup') } - throw thrown(error) + const failure = startupFailure() + reportFailure(failure) + throw failure } const publishedQuery = query const publishedChild = child + let receivedResult = false const result = settleRunResult({ - attempt: () => consumeClaudeQuery(publishedQuery), + attempt: async () => { + try { + return await consumeClaudeQuery(publishedQuery, () => { + capturePermissionDiagnostic(unattendedDiagnostic( + spec.permissionMode, + 'tool permission', + 'denied', + 'Claude Code denied the request before an interactive prompt', + )) + }, () => { + receivedResult = true + }) + } catch (error: unknown) { + const processOutcome = managedProcess?.outcome + let facts: ClaudeCodeFailureFacts + if (error instanceof ClaudeCodeFailure) { + facts = { ...error.facts, outcome: processOutcome } + } else if (processOutcome !== undefined && !receivedResult) { + facts = { + stage: 'process', + category: 'process-exit', + outcome: processOutcome, + } + } else { + facts = { + stage: 'query-run', + category: 'unknown', + outcome: processOutcome, + } + } + prependFailureDiagnostic(facts) + // Keep the SDK category and cause; the diagnostic adds later process facts. + throw error instanceof ClaudeCodeFailure + ? error + : new ClaudeCodeFailure(facts, thrown(error)) + } + }, collectOutput: () => [], + collectDiagnostic: () => diagnostic, cancelled: () => controller.signal.aborted, onError: spec.onError, signal: request.signal, @@ -282,9 +581,14 @@ export async function startClaudeCodeRun( signal: request.signal, onAbort, requestCancel, - teardown: () => disposeClaudeCodeChild( - publishedQuery, - publishedChild, - ), + teardown: async () => { + try { + await disposeClaudeCodeChild(publishedQuery, publishedChild) + } catch (error: unknown) { + const failure = thrown(error) + reportFailure(failure) + throw failure + } + }, }) } diff --git a/packages/subagent/subagent-claude-code/tests/loader-composition.e2e.ts b/packages/subagent/subagent-claude-code/tests/loader-composition.e2e.ts index 51a2ea0025..5f339ce22e 100644 --- a/packages/subagent/subagent-claude-code/tests/loader-composition.e2e.ts +++ b/packages/subagent/subagent-claude-code/tests/loader-composition.e2e.ts @@ -1,3 +1,4 @@ +import { readFileSync } from 'node:fs' import { join } from 'node:path' import { fileURLToPath } from 'node:url' import { describe, expect, it } from 'vitest' @@ -12,26 +13,34 @@ const fixtureDir = fileURLToPath(new URL( )) const driver = join(fixtureDir, 'driver.ts') const configPath = join(fixtureDir, 'cordis.yml') +const packageDir = fileURLToPath(new URL('..', import.meta.url)) +const manifest = JSON.parse(readFileSync(join(packageDir, 'package.json'), 'utf8')) as { + dsh?: { bundle?: { patch?: string } } +} +const bundlePatch = manifest.dsh?.bundle?.patch +if (bundlePatch === undefined) throw new Error('Claude Code package must declare a Bundle patch') +const bundlePatchPath = join(packageDir, bundlePatch) const repoTsconfig = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) describe('product-provider public Loader composition', () => { - it('loads both opt-in packages and foreground tools without starting either product', async () => { + it('loads the Bundle default, two named Claude instances, their tools, and Codex without starting either product', async () => { const { stdout, stderr } = await runLoaderSmoke({ label: 'product-provider Loader composition', tempDirPrefix: 'dsh-product-provider-loader-', binScript: driver, libBinScript: driver, configPath, + binArgs: [configPath, bundlePatchPath], tsconfigPath: repoTsconfig, env: { - // Loading either optional package must not probe or start its binary. + // Loading the optional package must not probe or start a Claude binary. PATH: '', }, }) expect(stderr).toBe('') expect(JSON.parse(stdout)).toEqual({ - registeredProviders: ['codex', 'claude-code'], + registeredProviders: ['codex', 'claude-primary', 'claude-secondary', 'claude-code'], providers: [ { name: 'codex', @@ -53,19 +62,50 @@ describe('product-provider public Loader composition', () => { }, inheritsParentContext: false, }, + { + name: 'claude-primary', + capabilities: { + outputSchema: false, + depthLimit: false, + toolFilter: false, + persona: false, + }, + inheritsParentContext: false, + }, + { + name: 'claude-secondary', + capabilities: { + outputSchema: false, + depthLimit: false, + toolFilter: false, + persona: false, + }, + inheritsParentContext: false, + }, ], tools: [ { name: 'subagent_codex', - parameterNames: ['description', 'prompt'], + parameterNames: ['description', 'prompt', 'run_in_background'], required: ['description', 'prompt'], }, { name: 'subagent_claude_code', - parameterNames: ['description', 'prompt'], + parameterNames: ['description', 'prompt', 'run_in_background'], + required: ['description', 'prompt'], + }, + { + name: 'subagent_claude_primary', + parameterNames: ['description', 'prompt', 'run_in_background'], + required: ['description', 'prompt'], + }, + { + name: 'subagent_claude_secondary', + parameterNames: ['description', 'prompt', 'run_in_background'], required: ['description', 'prompt'], }, ], + jobTools: ['job_kill', 'job_list', 'job_output'], starts: 0, }) }, LOADER_SMOKE_TEST_TIMEOUT_MS) diff --git a/packages/subagent/subagent-claude-code/tests/messages-fixture.ts b/packages/subagent/subagent-claude-code/tests/messages-fixture.ts index d8a04cf953..78d2f84176 100644 --- a/packages/subagent/subagent-claude-code/tests/messages-fixture.ts +++ b/packages/subagent/subagent-claude-code/tests/messages-fixture.ts @@ -4,6 +4,12 @@ import { createServer, type IncomingHttpHeaders, type ServerResponse } from 'nod export type MessagesBehavior = | { readonly kind: 'complete'; readonly text: string } | { readonly kind: 'hold' } + | { + readonly kind: 'tool-use' + readonly toolName: string + readonly input: Record + readonly finalText?: string + } /** One recorded Anthropic Messages request. */ interface RecordedMessagesRequest { @@ -81,6 +87,67 @@ function complete( response.end() } +function toolUse( + response: ServerResponse, + body: Record, + toolName: string, + input: Record, +): void { + const model = typeof body.model === 'string' ? body.model : 'fixture-model' + response.writeHead(200, { + 'content-type': 'text/event-stream', + 'cache-control': 'no-cache', + connection: 'keep-alive', + }) + event(response, 'message_start', { + type: 'message_start', + message: { + id: 'msg_dsh_fixture_tool_use', + type: 'message', + role: 'assistant', + model, + content: [], + stop_reason: null, + stop_sequence: null, + usage: { + input_tokens: 7, + output_tokens: 0, + cache_creation_input_tokens: 0, + cache_read_input_tokens: 0, + }, + }, + }) + event(response, 'content_block_start', { + type: 'content_block_start', + index: 0, + content_block: { + type: 'tool_use', + id: 'toolu_dsh_fixture', + name: toolName, + input: {}, + }, + }) + event(response, 'content_block_delta', { + type: 'content_block_delta', + index: 0, + delta: { + type: 'input_json_delta', + partial_json: JSON.stringify(input), + }, + }) + event(response, 'content_block_stop', { + type: 'content_block_stop', + index: 0, + }) + event(response, 'message_delta', { + type: 'message_delta', + delta: { stop_reason: 'tool_use', stop_sequence: null }, + usage: { output_tokens: 1 }, + }) + event(response, 'message_stop', { type: 'message_stop' }) + response.end() +} + /** * Start a loopback-only Anthropic Messages SSE fixture. * @param behavior - the single response behavior for this fixture. @@ -118,8 +185,15 @@ export async function startMessagesFixture( requestStartedResolve() if (behavior.kind === 'complete') { complete(response, body, behavior.text) + } else if (behavior.kind === 'tool-use' && requests.length === 1) { + toolUse(response, body, behavior.toolName, behavior.input) + } else if ( + behavior.kind === 'tool-use' + && behavior.finalText !== undefined + ) { + complete(response, body, behavior.finalText) } - // A hold deliberately leaves the response pending until client abort. + // A hold, or a tool-use without final text, waits for client abort. }) }) await new Promise((resolve, reject) => { diff --git a/packages/subagent/subagent-claude-code/tests/real-deepseek.e2e.ts b/packages/subagent/subagent-claude-code/tests/real-deepseek.e2e.ts index 89d08a1878..1881e14d33 100644 --- a/packages/subagent/subagent-claude-code/tests/real-deepseek.e2e.ts +++ b/packages/subagent/subagent-claude-code/tests/real-deepseek.e2e.ts @@ -7,7 +7,7 @@ import { rmSync, } from 'node:fs' import { tmpdir } from 'node:os' -import { delimiter, dirname, join, resolve } from 'node:path' +import { dirname, join, resolve } from 'node:path' import { fileURLToPath } from 'node:url' import { promisify } from 'node:util' import { Context } from '@deepseek-ai/cordis' @@ -87,7 +87,6 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)( ]) mkdirSync(directory) const env = { - PATH: `${dirname(claudeBin)}${delimiter}${process.env.PATH ?? ''}`, ANTHROPIC_AUTH_TOKEN: apiKey, ANTHROPIC_BASE_URL: `${deepSeekBaseUrl()}/anthropic`, ANTHROPIC_MODEL: 'deepseek-v4-pro[1m]', diff --git a/packages/subagent/subagent-claude-code/tests/real-product.spec.ts b/packages/subagent/subagent-claude-code/tests/real-product.spec.ts index f6767817c8..0141fb7311 100644 --- a/packages/subagent/subagent-claude-code/tests/real-product.spec.ts +++ b/packages/subagent/subagent-claude-code/tests/real-product.spec.ts @@ -1,14 +1,15 @@ import { execFile } from 'node:child_process' import { + existsSync, mkdirSync, mkdtempSync, readFileSync, - symlinkSync, + realpathSync, writeFileSync, } from 'node:fs' import { rm } from 'node:fs/promises' import { tmpdir } from 'node:os' -import { delimiter, dirname, join, resolve } from 'node:path' +import { dirname, join, resolve } from 'node:path' import { fileURLToPath } from 'node:url' import { promisify } from 'node:util' import type { @@ -20,9 +21,14 @@ import { Context } from '@deepseek-ai/cordis' import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from 'vitest' import type { Agent } from '@deepseek-ai/dsh-agent' import SubagentRuntime from '@deepseek-ai/dsh-subagent' -import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessHandle, + SubprocessOutcome, + SubprocessSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' import LocalSubprocessRuntime from '@deepseek-ai/dsh-subprocess-local' import * as claudeCode from '../src/index.ts' +import type { ClaudeCodePermissionMode } from '../src/run.ts' import { startMessagesFixture, type MessagesBehavior, @@ -30,6 +36,7 @@ import { } from './messages-fixture.ts' const observedSdkMessages = vi.hoisted((): SDKMessage[] => []) +const sdkTestOverrides = vi.hoisted((): { maxTurns?: number } => ({})) vi.mock('@anthropic-ai/claude-agent-sdk', async (importOriginal) => { const actual = await importOriginal< @@ -37,8 +44,13 @@ vi.mock('@anthropic-ai/claude-agent-sdk', async (importOriginal) => { >() return { ...actual, - query(options: Parameters[0]): Query { - const query = actual.query(options) + query(params: Parameters[0]): Query { + const query = actual.query(sdkTestOverrides.maxTurns === undefined + ? params + : { + ...params, + options: { ...params.options, maxTurns: sdkTestOverrides.maxTurns }, + }) // Observe the real SDK stream without replacing its protocol or CLI. return new Proxy(query, { get(target, property) { @@ -110,6 +122,7 @@ afterEach(async () => { await rm(root, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 }) } observedSdkMessages.length = 0 + delete sdkTestOverrides.maxTurns }) interface RealHarness { @@ -119,37 +132,39 @@ interface RealHarness { readonly parent: Agent readonly workspace: string readonly env: Record - readonly executable: string } -async function realHarness(behavior: MessagesBehavior): Promise<{ - readonly harness: RealHarness +interface RealInstanceFixture { readonly fixture: MessagesFixture -}> { + readonly workspace: string + readonly env: Record +} + +async function realInstanceFixture( + behavior: MessagesBehavior, + nativeAllow: readonly string[] = [], +): Promise { const root = mkdtempSync(join(tmpdir(), 'dsh-claude-code-real-')) roots.push(root) const workspace = join(root, 'workspace') const claudeConfig = join(root, 'claude-config') const xdgConfig = join(root, 'xdg') - const nativeBin = join(root, 'native&%literal%!bang!bin') mkdirSync(workspace) mkdirSync(claudeConfig) mkdirSync(xdgConfig) - mkdirSync(nativeBin) - const executable = join(nativeBin, process.platform === 'win32' ? 'claude.cmd' : 'claude') - if (process.platform === 'win32') { - writeFileSync(executable, `@echo off\r\n"${claudeBin}" %*\r\n`) - } else { - symlinkSync(claudeBin, executable) - } writeFileSync( join(claudeConfig, 'settings.json'), - `${JSON.stringify({ model: settingsModel }, null, 2)}\n`, + `${JSON.stringify({ + model: settingsModel, + permissions: { + defaultMode: 'default', + ...nativeAllow.length === 0 ? {} : { allow: nativeAllow }, + }, + }, null, 2)}\n`, ) const fixture = await startMessagesFixture(behavior) fixtures.push(fixture) const env = { - PATH: `${nativeBin}${delimiter}${process.env.PATH ?? ''}`, ANTHROPIC_API_KEY: fakeKey, ANTHROPIC_BASE_URL: fixture.baseUrl, CLAUDE_CONFIG_DIR: claudeConfig, @@ -164,6 +179,16 @@ async function realHarness(behavior: MessagesBehavior): Promise<{ ALL_PROXY: '', NO_PROXY: '127.0.0.1,localhost', } + return { fixture, workspace, env } +} + +interface RealRuntime { + readonly ctx: Context + readonly handles: SubprocessHandle[] + readonly spawnSpecs: SubprocessSpawnSpec[] +} + +async function realRuntime(): Promise { const ctx = new Context() contexts.push(ctx) await ctx.plugin(SubagentRuntime) @@ -177,14 +202,38 @@ async function realHarness(behavior: MessagesBehavior): Promise<{ handles.push(handle) return handle }) - await ctx.plugin(claudeCode, { env, disposeGraceMs: 3_000 }) + return { ctx, handles, spawnSpecs } +} + +async function realHarness( + behavior: MessagesBehavior, + permissionMode?: ClaudeCodePermissionMode, + nativeAllow: readonly string[] = [], +): Promise<{ + readonly harness: RealHarness + readonly fixture: MessagesFixture +}> { + const instance = await realInstanceFixture(behavior, nativeAllow) + const { ctx, handles, spawnSpecs } = await realRuntime() + await ctx.plugin(claudeCode, { + env: instance.env, + ...permissionMode === undefined ? {} : { permissionMode }, + disposeGraceMs: 3_000, + }) const parent = { id: 'real-parent', - session: { header: { cwd: workspace } }, + session: { header: { cwd: instance.workspace } }, } as unknown as Agent return { - harness: { ctx, handles, spawnSpecs, parent, workspace, env, executable }, - fixture, + harness: { + ctx, + handles, + spawnSpecs, + parent, + workspace: instance.workspace, + env: instance.env, + }, + fixture: instance.fixture, } } @@ -200,6 +249,29 @@ async function expectQuiescent( } } +function expectedFailure( + stage: 'query-run' | 'process', + category: 'error_during_execution' | 'process-exit', + outcome: SubprocessOutcome, +): string { + const fields = [ + 'product: Claude Code', + `stage: ${stage}`, + `category: ${category}`, + ] + if (outcome.exitCode !== null) fields.push(`exit code: ${outcome.exitCode}`) + if (outcome.signal !== null) fields.push(`signal: ${outcome.signal}`) + return `Product subagent failure (${fields.join('; ')})` +} + +function expectedObservedFailure(outcome: SubprocessOutcome): string { + return observedSdkMessages.some(message => + message.type === 'result' + && message.subtype === 'error_during_execution') + ? expectedFailure('query-run', 'error_during_execution', outcome) + : expectedFailure('process', 'process-exit', outcome) +} + function startRequest( harness: RealHarness, prompt: string, @@ -225,7 +297,7 @@ describe('real Claude Agent SDK 0.3.220 and its distributed Claude Code 2.1.220 expect(sdkPackage.version).toBe('0.3.220') expect(sdkPackage.claudeCodeVersion).toBe('2.1.220') expect(sdkPackage.optionalDependencies[platformPackage]).toBe('0.3.220') - const version = await execFileAsync(process.platform === 'win32' ? claudeBin : harness.executable, ['--version'], { + const version = await execFileAsync(claudeBin, ['--version'], { env: { ...process.env, ...harness.env }, }) expect(version.stdout.trim()).toBe('2.1.220 (Claude Code)') @@ -242,19 +314,14 @@ describe('real Claude Agent SDK 0.3.220 and its distributed Claude Code 2.1.220 message.type === 'system' && message.subtype === 'init', ) expect(initMessage?.claude_code_version).toBe('2.1.220') - if (process.platform === 'win32') { - expect(harness.spawnSpecs[0]?.argv.slice(0, 6)).toEqual([ - 'cmd.exe', '/d', '/v:off', '/s', '/c', '%DSH_CLAUDE_CODE_EXECUTABLE%', - ]) - const batchExecutable = harness.spawnSpecs[0]?.env?.DSH_CLAUDE_CODE_EXECUTABLE - expect(batchExecutable?.startsWith('"')).toBe(true) - expect(batchExecutable?.endsWith('"')).toBe(true) - expect(batchExecutable?.slice(1, -1).toLowerCase()) - .toBe(harness.executable.toLowerCase()) - } else { - expect(harness.spawnSpecs[0]?.argv[0]).toBe(harness.executable) - } - + const spawnedExecutable = harness.spawnSpecs[0]?.argv[0] + expect(spawnedExecutable).toBeDefined() + expect(process.platform === 'win32' + ? realpathSync(spawnedExecutable!).toLowerCase() + : realpathSync(spawnedExecutable!)) + .toBe(process.platform === 'win32' + ? realpathSync(claudeBin).toLowerCase() + : realpathSync(claudeBin)) expect(fixture.requests).toHaveLength(1) const recorded = fixture.requests[0]! expect(recorded.method).toBe('POST') @@ -278,22 +345,207 @@ describe('real Claude Agent SDK 0.3.220 and its distributed Claude Code 2.1.220 await expectQuiescent(harness.handles) }) - it('maps a real CLI process failure to error', async () => { + it('maps a real SDK max-turns result to safe query-run facts', async () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-claude-code-max-turns-')) + roots.push(root) + const target = join(root, 'max-turns.txt') + sdkTestOverrides.maxTurns = 1 + const { harness, fixture } = await realHarness({ + kind: 'tool-use', + toolName: 'Write', + input: { + file_path: target, + content: 'real-sdk-max-turns', + }, + }, 'bypassPermissions') + const run = await startRequest(harness, 'Exercise the SDK max-turns result.') + const result = await run.result + expect(observedSdkMessages + .filter(message => message.type === 'result') + .map(message => message.subtype)).toEqual(['error_max_turns']) + expect(result).toMatchObject({ + output: [], + stopReason: 'error', + }) + expect(result.diagnostic).toContain( + 'product: Claude Code; stage: query-run; category: error_max_turns', + ) + expect(readFileSync(target, 'utf8')).toBe('real-sdk-max-turns') + expect(result.diagnostic).not.toContain(target) + expect(result.diagnostic).not.toContain('real-sdk-max-turns') + await run.dispose() + expect(fixture.requests).toHaveLength(1) + await expectQuiescent(harness.handles) + }) + + it('runs two named instances concurrently and unloads one without revoking its run', async () => { + const safeInstance = await realInstanceFixture({ kind: 'hold' }) + const bypassInstance = await realInstanceFixture({ + kind: 'complete', + text: 'NAMED_BYPASS_RESULT', + }) + const { ctx, handles, spawnSpecs } = await realRuntime() + const safeFiber = await ctx.plugin(claudeCode, { + providerName: 'claude-safe', + env: safeInstance.env, + permissionMode: 'dontAsk', + disposeGraceMs: 3_000, + }) + const bypassFiber = await ctx.plugin(claudeCode, { + providerName: 'claude-bypass', + env: bypassInstance.env, + permissionMode: 'bypassPermissions', + disposeGraceMs: 3_000, + }) + const safeParent = { + id: 'safe-parent', + session: { header: { cwd: safeInstance.workspace } }, + } as unknown as Agent + const bypassParent = { + id: 'bypass-parent', + session: { header: { cwd: bypassInstance.workspace } }, + } as unknown as Agent + const safeController = new AbortController() + + const [safeRun, bypassRun] = await Promise.all([ + ctx.subagents.start('claude-safe', { + prompt: [{ type: 'text', text: 'Hold the safe instance.' }], + parent: safeParent, + signal: safeController.signal, + }), + ctx.subagents.start('claude-bypass', { + prompt: [{ type: 'text', text: 'Complete the bypass instance.' }], + parent: bypassParent, + signal: new AbortController().signal, + }), + ]) + await safeInstance.fixture.requestStarted + await safeFiber.dispose() + expect(ctx.subagents.list()).toEqual(['claude-bypass']) + await expect(ctx.subagents.start('claude-safe', { + prompt: [{ type: 'text', text: 'This start must fail.' }], + parent: safeParent, + signal: new AbortController().signal, + })).rejects.toMatchObject({ code: 'NO_PROVIDER' }) + + await expect(bypassRun.result).resolves.toEqual({ + output: [{ type: 'text', text: 'NAMED_BYPASS_RESULT' }], + stopReason: 'completed', + }) + safeController.abort(new Error('cancel only the published safe run')) + await expect(safeRun.result).resolves.toEqual({ + output: [], + stopReason: 'aborted', + }) + await Promise.all([safeRun.dispose(), bypassRun.dispose()]) + expect(safeInstance.fixture.requests).toHaveLength(1) + expect(bypassInstance.fixture.requests).toHaveLength(1) + expect(safeInstance.fixture.requests[0]?.body.messages) + .not.toEqual(bypassInstance.fixture.requests[0]?.body.messages) + expect(spawnSpecs.map(spec => spec.env?.CLAUDE_CONFIG_DIR).sort()) + .toEqual([ + safeInstance.env.CLAUDE_CONFIG_DIR, + bypassInstance.env.CLAUDE_CONFIG_DIR, + ].sort()) + await expectQuiescent(handles) + await bypassFiber.dispose() + expect(ctx.subagents.list()).toEqual([]) + }) + + it('maps a real CLI process failure to its exit outcome', async () => { const { harness, fixture } = await realHarness({ kind: 'hold' }) const run = await startRequest(harness, 'Exercise the failure path.') await fixture.requestStarted expect(harness.handles).toHaveLength(1) harness.handles[0]!.terminate() - await expect(run.result).resolves.toEqual({ - output: [], - stopReason: 'error', - }) + const outcome = await harness.handles[0]!.done + const result = await run.result + expect(result.output).toEqual([]) + expect(result.stopReason).toBe('error') + expect(result.diagnostic).toBe(expectedObservedFailure(outcome)) await run.dispose() expect(fixture.requests).toHaveLength(1) expect(fixture.requests[0]!.headers['x-api-key']).toBe(fakeKey) await expectQuiescent(harness.handles) }) + it('overrides interactive settings, denies a write, and returns a safe diagnostic', async () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-claude-code-denied-target-')) + roots.push(root) + const target = join(root, 'denied.txt') + const { harness } = await realHarness({ + kind: 'tool-use', + toolName: 'Write', + input: { + file_path: target, + content: 'SECRET_TOKEN must not reach the diagnostic', + }, + }) + const run = await startRequest(harness, 'Write the requested fixture file.') + await vi.waitFor(() => { + expect(observedSdkMessages.some(message => + message.type === 'system' + && message.subtype === 'permission_denied')).toBe(true) + }, { timeout: 30_000 }) + expect(existsSync(target)).toBe(false) + harness.handles[0]!.terminate() + const outcome = await harness.handles[0]!.done + const result = await run.result + expect(result.output).toEqual([]) + expect(result.stopReason).toBe('error') + const diagnosticLines = result.diagnostic?.split('\n') ?? [] + expect(diagnosticLines[0]).toBe(expectedObservedFailure(outcome)) + expect(diagnosticLines[1]).toBe( + 'Claude Code unattended decision (mode: dontAsk; request: tool permission; decision: denied): Claude Code denied the request before an interactive prompt', + ) + expect(result.diagnostic).not.toContain(target) + expect(result.diagnostic).not.toContain('SECRET_TOKEN') + await run.dispose() + await expectQuiescent(harness.handles) + }) + + it('runs an explicitly selected bypass write in the isolated workspace', async () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-claude-code-bypass-target-')) + roots.push(root) + const target = join(root, 'bypass.txt') + const { harness } = await realHarness({ + kind: 'tool-use', + toolName: 'Write', + input: { + file_path: target, + content: 'bypass write completed', + }, + finalText: 'write complete', + }, 'bypassPermissions') + const run = await startRequest(harness, 'Write the requested fixture file.') + await expect(run.result).resolves.toEqual({ + output: [{ type: 'text', text: 'write complete' }], + stopReason: 'completed', + }) + expect(readFileSync(target, 'utf8')).toBe('bypass write completed') + await run.dispose() + await expectQuiescent(harness.handles) + }) + + it('returns the completed plan without approving execution', async () => { + const { harness, fixture } = await realHarness({ + kind: 'tool-use', + toolName: 'ExitPlanMode', + input: {}, + finalText: 'PLAN_ONLY_RESULT', + }, 'plan', ['ExitPlanMode']) + const run = await startRequest(harness, 'Design the fixture change without implementing it.') + await expect(run.result).resolves.toEqual({ + output: [{ type: 'text', text: 'PLAN_ONLY_RESULT' }], + stopReason: 'completed', + }) + expect(fixture.requests).toHaveLength(2) + expect(JSON.stringify(fixture.requests[1]?.body.messages)) + .toContain('ExitPlanMode exists but is not enabled in this context') + await run.dispose() + await expectQuiescent(harness.handles) + }) + it('settles cancellation and leaves the real SDK-spawned CLI tree quiescent', async () => { const { harness, fixture } = await realHarness({ kind: 'hold' }) const controller = new AbortController() diff --git a/packages/subagent/subagent-claude-code/tests/subagent-claude-code.spec.ts b/packages/subagent/subagent-claude-code/tests/subagent-claude-code.spec.ts index a3df59a74f..3298e07037 100644 --- a/packages/subagent/subagent-claude-code/tests/subagent-claude-code.spec.ts +++ b/packages/subagent/subagent-claude-code/tests/subagent-claude-code.spec.ts @@ -1,13 +1,18 @@ +import { readFileSync } from 'node:fs' +import { dirname, resolve } from 'node:path' import { PassThrough } from 'node:stream' +import { fileURLToPath } from 'node:url' import type { Options, Query, SDKMessage, + SDKPermissionDeniedMessage, SDKResultMessage, SpawnOptions, } from '@anthropic-ai/claude-agent-sdk' import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' +import * as yaml from 'js-yaml' import { afterEach, beforeEach, @@ -36,6 +41,8 @@ import { sdkEnvironmentOverlay, } from '../src/process.ts' import { + CLAUDE_CODE_PERMISSION_MODES, + DEFAULT_CLAUDE_CODE_PERMISSION_MODE, claudeQueryOptions, consumeClaudeQuery, disposeClaudeCodeChild, @@ -52,6 +59,19 @@ type QueryFactory = (params: { const queryMock = vi.hoisted(() => vi.fn()) +const CLAUDE_AGENT_SDK_VERSION = '0.3.220' +const CLAUDE_CODE_VERSION = '2.1.220' +const CLAUDE_PLATFORM_PACKAGES = [ + '@anthropic-ai/claude-agent-sdk-darwin-arm64', + '@anthropic-ai/claude-agent-sdk-darwin-x64', + '@anthropic-ai/claude-agent-sdk-linux-arm64', + '@anthropic-ai/claude-agent-sdk-linux-arm64-musl', + '@anthropic-ai/claude-agent-sdk-linux-x64', + '@anthropic-ai/claude-agent-sdk-linux-x64-musl', + '@anthropic-ai/claude-agent-sdk-win32-arm64', + '@anthropic-ai/claude-agent-sdk-win32-x64', +] as const + vi.mock('@anthropic-ai/claude-agent-sdk', async importOriginal => ({ ...await importOriginal(), query: queryMock, @@ -73,6 +93,12 @@ async function nextTask(): Promise { await new Promise((resolve) => { setImmediate(resolve) }) } +function errorCause(value: unknown): Error | undefined { + return value instanceof Error && value.cause instanceof Error + ? value.cause + : undefined +} + interface FakeChildOptions { readonly pid?: number readonly exitOnTerminate?: boolean @@ -189,6 +215,39 @@ function failure( } as SDKResultMessage } +function expectedFailureDiagnostic( + stage: 'query-start' | 'query-run' | 'process' | 'teardown', + category: string, + outcome?: Partial, +): string { + const fields = [ + 'product: Claude Code', + `stage: ${stage}`, + `category: ${category}`, + ] + if (outcome?.exitCode !== null && outcome?.exitCode !== undefined) { + fields.push(`exit code: ${outcome.exitCode}`) + } + if (outcome?.signal !== null && outcome?.signal !== undefined) { + fields.push(`signal: ${outcome.signal}`) + } + return `Product subagent failure (${fields.join('; ')})` +} + +function permissionDenied(): SDKPermissionDeniedMessage { + return { + type: 'system', + subtype: 'permission_denied', + tool_name: 'Bash', + tool_use_id: 'tool-secret', + decision_reason_type: 'mode', + decision_reason: 'contains /private/secret.txt', + message: 'command with SECRET_TOKEN was denied', + uuid: '00000000-0000-4000-8000-000000000001', + session_id: 'session-secret', + } +} + function queryFrom( messages: readonly SDKMessage[], after?: Error, @@ -248,7 +307,7 @@ function fakeRun( const options: FakeRun['options'] = [] const spec: ClaudeCodeRunSpec = { cwd: '/workspace', - executable: '/native/claude', + permissionMode: DEFAULT_CLAUDE_CODE_PERMISSION_MODE, env: { ANTHROPIC_API_KEY: 'fake-key' }, disposeGraceMs: 5, spawn: (spawnSpec) => { @@ -282,6 +341,66 @@ afterEach(() => { }) describe('task admission and package contracts', () => { + it('ships one independently installable provider-only Bundle patch', () => { + const root = fileURLToPath(new URL('..', import.meta.url)) + const manifest = JSON.parse(readFileSync(resolve(root, 'package.json'), 'utf8')) as { + dependencies?: Record + files?: string[] + dsh?: { bundle?: { patch?: string } } + } + expect(manifest.dsh?.bundle?.patch).toBe('./cordis.patch.yml') + expect(manifest.files).toContain('cordis.patch.yml') + expect(manifest.dependencies).toHaveProperty( + '@anthropic-ai/claude-agent-sdk', + CLAUDE_AGENT_SDK_VERSION, + ) + expect(manifest.dependencies).toHaveProperty( + '@modelcontextprotocol/sdk', + '^1.29.0', + ) + expect(manifest.dependencies).toHaveProperty('zod', '^4.4.3') + expect(manifest.dependencies).not.toHaveProperty('@deepseek-ai/dsh-subagent-codex') + + const sdkRoot = dirname(fileURLToPath( + import.meta.resolve('@anthropic-ai/claude-agent-sdk'), + )) + const sdkManifest = JSON.parse(readFileSync( + resolve(sdkRoot, 'package.json'), + 'utf8', + )) as { + version: string + claudeCodeVersion: string + optionalDependencies: Record + } + expect(sdkManifest.version).toBe(CLAUDE_AGENT_SDK_VERSION) + expect(sdkManifest.claudeCodeVersion).toBe(CLAUDE_CODE_VERSION) + expect(sdkManifest.optionalDependencies).toEqual(Object.fromEntries( + CLAUDE_PLATFORM_PACKAGES.map(packageName => [ + packageName, + CLAUDE_AGENT_SDK_VERSION, + ]), + )) + const lockfile = readFileSync(resolve(root, '../../../pnpm-lock.yaml'), 'utf8') + for (const packageName of CLAUDE_PLATFORM_PACKAGES) { + expect(lockfile).toContain( + ` '${packageName}@${CLAUDE_AGENT_SDK_VERSION}':`, + ) + expect(lockfile).toContain( + ` '${packageName}': ${CLAUDE_AGENT_SDK_VERSION}`, + ) + } + + const parsed = yaml.load(readFileSync(resolve(root, manifest.dsh!.bundle!.patch!), 'utf8')) + const rows = Array.isArray(parsed) + ? (parsed as Array<{ insert?: Array<{ id?: string; name?: string }> }>).flatMap(entry => entry.insert ?? []) + : [] + expect(rows).toEqual([{ + id: 'subagent-claude-code', + name: '@deepseek-ai/dsh-subagent-claude-code', + }]) + expect(JSON.stringify(rows)).not.toContain('tool-subagent') + }) + it('preserves text sequences and rejects empty, blank, and non-text tasks', () => { expect(textTask([ { type: 'text', text: 'one' }, @@ -294,7 +413,7 @@ describe('task admission and package contracts', () => { .toThrow('must not be empty') }) - it('registers one fixed descriptor, validates config, and unregisters on HMR', async () => { + it('registers the default descriptor, validates config, and unregisters on HMR', async () => { const ctx = new Context() await ctx.plugin(SubagentRuntime) await ctx.plugin(LocalSubprocessRuntime) @@ -325,6 +444,143 @@ describe('task admission and package contracts', () => { await ctx.fiber.dispose() }) + it('keeps named instances, runs, and HMR ownership isolated', async () => { + const ctx = new Context() + await ctx.plugin(SubagentRuntime) + await ctx.plugin(LocalSubprocessRuntime) + const safeChild = fakeChild() + const bypassChild = fakeChild() + const spawnSpecs: SubprocessSpawnSpec[] = [] + vi.spyOn(ctx.subprocess, 'spawn').mockImplementation((spec) => { + spawnSpecs.push(spec) + return spec.env?.DSH_CLAUDE_INSTANCE === 'safe' + ? safeChild.handle + : bypassChild.handle + }) + const queryOptions: Options[] = [] + queryMock.mockImplementation(({ options }) => { + queryOptions.push(options) + options.spawnClaudeCodeProcess!(sdkSpawnOptions({ + cwd: options.cwd!, + env: options.env!, + signal: options.abortController!.signal, + })) + return options.permissionMode === 'dontAsk' + ? waitingQuery(options.abortController!.signal) + : queryFrom([success('bypass answer')]) + }) + + const added: string[] = [] + const started: string[] = [] + const ended: string[] = [] + const removed: string[] = [] + ctx.on('subagent/provider-added', provider => void added.push(provider.name)) + ctx.on('subagent/start', info => void started.push(info.provider)) + ctx.on('subagent/end', info => void ended.push(info.provider)) + ctx.on('subagent/provider-removed', providerName => void removed.push(providerName)) + const safeFiber = await ctx.plugin(claudeCode, { + providerName: 'claude-safe', + env: { DSH_CLAUDE_INSTANCE: 'safe' }, + permissionMode: 'dontAsk', + disposeGraceMs: 11, + }) + const bypassFiber = await ctx.plugin(claudeCode, { + providerName: 'claude-bypass', + env: { DSH_CLAUDE_INSTANCE: 'bypass' }, + permissionMode: 'bypassPermissions', + disposeGraceMs: 29, + }) + expect(ctx.subagents.list()).toEqual(['claude-safe', 'claude-bypass']) + expect(added).toEqual(['claude-safe', 'claude-bypass']) + + const safeController = new AbortController() + const [safeRun, bypassRun] = await Promise.all([ + ctx.subagents.start('claude-safe', request(undefined, safeController.signal)), + ctx.subagents.start('claude-bypass', request()), + ]) + await safeFiber.dispose() + expect(ctx.subagents.list()).toEqual(['claude-bypass']) + expect(removed).toEqual(['claude-safe']) + await expect(ctx.subagents.start('claude-safe', request())) + .rejects.toMatchObject({ code: 'NO_PROVIDER' }) + + await expect(bypassRun.result).resolves.toEqual({ + output: [{ type: 'text', text: 'bypass answer' }], + stopReason: 'completed', + }) + safeController.abort(new Error('stop only the safe instance')) + await expect(safeRun.result).resolves.toEqual({ + output: [], + stopReason: 'aborted', + }) + expect(queryOptions.map(options => ({ + instance: options.env?.DSH_CLAUDE_INSTANCE, + permissionMode: options.permissionMode, + }))).toEqual([ + { instance: 'safe', permissionMode: 'dontAsk' }, + { instance: 'bypass', permissionMode: 'bypassPermissions' }, + ]) + expect(spawnSpecs.map(spec => ({ + instance: spec.env?.DSH_CLAUDE_INSTANCE, + graceMs: spec.graceMs, + }))).toEqual([ + { instance: 'safe', graceMs: 11 }, + { instance: 'bypass', graceMs: 29 }, + ]) + + await Promise.all([safeRun.dispose(), bypassRun.dispose()]) + expect([...started].sort()).toEqual(['claude-bypass', 'claude-safe']) + expect([...ended].sort()).toEqual(['claude-bypass', 'claude-safe']) + expect(safeChild.terminate).toHaveBeenCalledOnce() + expect(bypassChild.terminate).toHaveBeenCalledOnce() + await bypassFiber.dispose() + expect(removed).toEqual(['claude-safe', 'claude-bypass']) + await ctx.fiber.dispose() + }) + + it('rejects duplicate provider names without replacing the first instance', async () => { + const ctx = new Context() + await ctx.plugin(SubagentRuntime) + await ctx.plugin(LocalSubprocessRuntime) + const firstFiber = await ctx.plugin(claudeCode, { + providerName: 'claude-duplicate', + }) + const first = ctx.subagents.getProvider('claude-duplicate') + await expect(ctx.plugin(claudeCode, { + providerName: 'claude-duplicate', + permissionMode: 'bypassPermissions', + })).rejects.toMatchObject({ code: 'DUPLICATE_PROVIDER' }) + expect(ctx.subagents.getProvider('claude-duplicate')).toBe(first) + expect(ctx.subagents.list()).toEqual(['claude-duplicate']) + await firstFiber.dispose() + await ctx.fiber.dispose() + }) + + it('accepts only the five fixed non-interactive permission modes', () => { + expect(claudeCode.Config({}).providerName).toBe('claude-code') + expect(claudeCode.Config({ providerName: 'claude-safe' }).providerName) + .toBe('claude-safe') + expect(() => claudeCode.Config({ providerName: '' })).toThrow() + expect(claudeCode.Config({}).permissionMode) + .toBe(DEFAULT_CLAUDE_CODE_PERMISSION_MODE) + for (const permissionMode of CLAUDE_CODE_PERMISSION_MODES) { + expect(claudeCode.Config({ permissionMode }).permissionMode) + .toBe(permissionMode) + } + for (const permissionMode of ['default', 'interactive', 'future-mode']) { + expect(() => claudeCode.Config({ permissionMode } as never)).toThrow() + } + }) + + it('resolves the safe permission default when apply is called directly', async () => { + const ctx = new Context() + await ctx.plugin(SubagentRuntime) + await ctx.plugin(LocalSubprocessRuntime) + claudeCode.apply(ctx, { env: {}, disposeGraceMs: 3_000 }) + expect(ctx.subagents.getProvider('claude-code')).toBeDefined() + await ctx.fiber.dispose() + }) + it('starts through the registered provider with its resolved config and diagnostics', async () => { const ctx = new Context() await ctx.plugin(SubagentRuntime) @@ -333,18 +589,20 @@ describe('task admission and package contracts', () => { const spawn = vi.spyOn(ctx.subprocess, 'spawn') .mockImplementation(() => child.handle) const resolveExecutable = vi.spyOn(ctx.subprocess, 'resolveExecutable') - .mockResolvedValue('/native/claude') + .mockResolvedValue('/host/bin/claude') const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {}) await ctx.plugin(claudeCode, { + providerName: 'claude-diagnostic', env: { ANTHROPIC_API_KEY: 'provider-fake-key', CLAUDE_CONFIG_DIR: '/private/tmp/dsh-claude-code-unit-config', HOME: '/private/tmp/dsh-claude-code-unit-home', }, + permissionMode: 'auto', disposeGraceMs: 29, }) - await expect(ctx.subagents.start('claude-code', { + await expect(ctx.subagents.start('claude-diagnostic', { ...request(), parent: { id: 'parent-without-cwd', @@ -355,28 +613,71 @@ describe('task admission and package contracts', () => { ) expect(queryMock).not.toHaveBeenCalled() - resolveExecutable.mockRejectedValueOnce(new Error('claude missing from PATH')) - await expect(ctx.subagents.start('claude-code', request())) - .rejects.toThrow('claude missing from PATH') - expect(queryMock).not.toHaveBeenCalled() + const invalidCwdParent = { + id: 'parent-with-invalid-cwd', + session: { header: { cwd: 'relative/SECRET_TOKEN' } }, + } as unknown as Agent + const invalidCwd = ctx.subagents.start('claude-diagnostic', { + ...request(), + parent: invalidCwdParent, + }) + await expect(invalidCwd) + .rejects.toThrow(expectedFailureDiagnostic('query-start', 'unknown')) + await expect(invalidCwd).rejects.not.toThrow('relative/SECRET_TOKEN') + expect(warn).toHaveBeenCalledWith( + 'subagent-claude-code "claude-diagnostic": child start failed: %o', + expect.any(Error), + ) + expect(errorCause(warn.mock.calls[0]?.[1] as unknown)?.message) + .toContain('relative/SECRET_TOKEN') - const run = await ctx.subagents.start('claude-code', request()) + const invalidCwdAbort = new AbortController() + invalidCwdAbort.abort(new Error('cancel invalid cwd startup')) + await expect(ctx.subagents.start('claude-diagnostic', { + ...request(undefined, invalidCwdAbort.signal), + parent: invalidCwdParent, + })).rejects.toThrow('aborted before SDK startup') + expect(queryMock).not.toHaveBeenCalled() + warn.mockClear() + + vi.stubEnv('PATH', '/host/bin') + queryMock.mockImplementationOnce(() => { + throw new Error( + 'Native CLI binary for fixture-platform not found. Reinstall @anthropic-ai/claude-agent-sdk without --omit=optional, or set options.pathToClaudeCodeExecutable.', + ) + }) + const missingPayload = ctx.subagents.start('claude-diagnostic', request()) + await expect(missingPayload) + .rejects.toThrow(expectedFailureDiagnostic('query-start', 'unknown')) + await expect(missingPayload).rejects.not.toThrow('Native CLI binary') + expect(warn).toHaveBeenCalledWith( + expect.stringContaining( + 'subagent-claude-code "claude-diagnostic": child run failed (error):', + ), + expect.any(Error), + ) + expect(errorCause(warn.mock.calls[0]?.[1] as unknown)?.message) + .toContain('Native CLI binary for fixture-platform not found') + expect(resolveExecutable).not.toHaveBeenCalled() + + const run = await ctx.subagents.start('claude-diagnostic', request()) child.settle({ exitCode: 9, signal: null }) child.stdout.end() await expect(run.result).resolves.toEqual({ output: [], + diagnostic: expectedFailureDiagnostic('query-run', 'missing-result'), stopReason: 'error', }) - expect(warn).toHaveBeenCalledWith(expect.stringContaining( - 'subagent-claude-code: child run failed (error):', - )) - expect(resolveExecutable).toHaveBeenCalledWith( - 'claude', - expect.objectContaining({ ANTHROPIC_API_KEY: 'provider-fake-key' }), - expect.any(AbortSignal), + expect(warn).toHaveBeenCalledWith( + expect.stringContaining( + 'subagent-claude-code "claude-diagnostic": child run failed (error):', + ), + expect.any(Error), ) - expect(queryMock.mock.calls[0]?.[0].options.pathToClaudeCodeExecutable) - .toBe('/native/claude') + expect(resolveExecutable).not.toHaveBeenCalled() + expect(queryMock.mock.calls[1]?.[0].options) + .not.toHaveProperty('pathToClaudeCodeExecutable') + expect(queryMock.mock.calls[1]?.[0].options.permissionMode).toBe('auto') expect(spawn).toHaveBeenCalledWith(expect.objectContaining({ cwd: process.cwd(), graceMs: 29, @@ -456,20 +757,16 @@ describe('official spawn projection', () => { )).toThrow('SDK spawn request omitted its workspace') }) - it.each(['cmd', 'bat'])('routes a Windows .%s shim through cmd.exe', (extension) => { - const command = String.raw`C:\Program Files\Claude\claude.${extension}` + it('forwards the SDK-selected Windows native executable without a batch shim', () => { + const command = String.raw`C:\Program Files\Claude\claude.exe` const spec = claudeSpawnSpec(sdkSpawnOptions({ command, args: ['--output-format', 'stream-json'], - }), 7, 'win32') + }), 7) expect(spec.argv).toEqual([ - 'cmd.exe', '/d', '/v:off', '/s', '/c', '%DSH_CLAUDE_CODE_EXECUTABLE%', - '--output-format', 'stream-json', + command, '--output-format', 'stream-json', ]) - expect(spec.env).toEqual(expect.objectContaining({ - DSH_CLAUDE_CODE_EXECUTABLE: `"${command}"`, - })) }) it('projects streams, exit facts, listeners, and idempotent tree termination', async () => { @@ -480,6 +777,7 @@ describe('official spawn projection', () => { expect(process.killed).toBe(false) expect(process.exitCode).toBeNull() expect(process.signalCode).toBeNull() + expect(process.outcome).toBeUndefined() const exit = vi.fn() const once = vi.fn() @@ -499,11 +797,12 @@ describe('official spawn projection', () => { expect(once).toHaveBeenCalledOnce() expect(removed).not.toHaveBeenCalled() expect(process.signalCode).toBe('SIGTERM') + expect(process.outcome).toEqual({ exitCode: null, signal: 'SIGTERM' }) expect(process.kill('SIGTERM')).toBe(false) }) it('emits spawn errors', async () => { - const child = fakeChild() + const child = fakeChild({ pid: -1 }) const process = new ManagedClaudeCodeProcess(child.handle) const errorListener = vi.fn() const removed = vi.fn() @@ -525,21 +824,23 @@ describe('official spawn projection', () => { await nextTask() expect(process.exitCode).toBe(7) expect(process.signalCode).toBeNull() + expect(process.outcome).toEqual({ exitCode: 7, signal: null }) expect(process.kill('SIGTERM')).toBe(false) }) }) describe('query options and result mapping', () => { - it('builds the fixed unattended options over the scrubbed environment', () => { + it('builds the fixed unattended options over the scrubbed environment', async () => { vi.stubEnv('HOST_VISIBLE', 'visible') vi.stubEnv('HOST_SECRET_TOKEN', 'must-not-leak') vi.stubEnv('DSH_INTERNAL', 'must-not-leak') const child = fakeChild() const spawn = vi.fn(() => child.handle) const captured: SubprocessHandle[] = [] + const diagnostics: string[] = [] const spec: ClaudeCodeRunSpec = { cwd: '/workspace', - executable: '/native/claude', + permissionMode: 'acceptEdits', env: { HOST_VISIBLE: 'overridden', ANTHROPIC_API_KEY: 'explicit-fake-key', @@ -548,32 +849,70 @@ describe('query options and result mapping', () => { spawn, } const controller = new AbortController() - const options = claudeQueryOptions(spec, controller, (value) => { - captured.push(value) - }) + const options = claudeQueryOptions( + spec, + controller, + (value) => { + captured.push(value) + }, + value => diagnostics.push(value), + ) expect(options).toMatchObject({ abortController: controller, cwd: '/workspace', - pathToClaudeCodeExecutable: '/native/claude', persistSession: false, disallowedTools: ['AskUserQuestion'], + permissionMode: 'acceptEdits', + supportedDialogKinds: ['refusal_fallback_prompt'], }) + expect(options).not.toHaveProperty('pathToClaudeCodeExecutable') + expect(options).not.toHaveProperty('allowDangerouslySkipPermissions') expect(options.env).toMatchObject({ HOST_VISIBLE: 'overridden', ANTHROPIC_API_KEY: 'explicit-fake-key', }) expect(options.env).not.toHaveProperty('HOST_SECRET_TOKEN') expect(options.env).not.toHaveProperty('DSH_INTERNAL') - for (const omitted of [ - 'settingSources', - 'canUseTool', - 'onElicitation', - 'onUserDialog', - 'supportedDialogKinds', - ]) { - expect(options).not.toHaveProperty(omitted) - } + expect(options).not.toHaveProperty('settingSources') + + const callbackSignal = new AbortController().signal + await expect(options.canUseTool!( + 'Bash', + { command: 'cat /private/secret.txt', token: 'SECRET_TOKEN' }, + { + signal: callbackSignal, + toolUseID: 'tool-1', + requestId: 'request-1', + blockedPath: '/private/secret.txt', + decisionReason: 'SECRET_TOKEN in /private/secret.txt', + }, + )).resolves.toEqual({ + behavior: 'deny', + message: 'This unattended Claude Code subagent cannot request human approval.', + }) + await expect(options.onElicitation!( + { + serverName: 'private-server', + message: 'enter SECRET_TOKEN', + requestedSchema: { secret: true }, + }, + { signal: callbackSignal }, + )).resolves.toEqual({ action: 'decline' }) + await expect(options.onUserDialog!( + { + dialogKind: 'refusal_fallback_prompt', + payload: { path: '/private/secret.txt', token: 'SECRET_TOKEN' }, + }, + { signal: callbackSignal }, + )).resolves.toEqual({ behavior: 'cancelled' }) + expect(diagnostics).toEqual([ + 'Claude Code unattended decision (mode: acceptEdits; request: tool permission; decision: denied): the provider does not request human approval', + 'Claude Code unattended decision (mode: acceptEdits; request: MCP elicitation; decision: declined): the provider does not collect interactive MCP input', + 'Claude Code unattended decision (mode: acceptEdits; request: user dialog; decision: cancelled): the provider does not render blocking dialogs', + ]) + expect(diagnostics.join('\n')).not.toContain('SECRET_TOKEN') + expect(diagnostics.join('\n')).not.toContain('/private/secret.txt') const spawned = options.spawnClaudeCodeProcess!(sdkSpawnOptions()) expect(spawned).toBeInstanceOf(ManagedClaudeCodeProcess) @@ -585,20 +924,77 @@ describe('query options and result mapping', () => { })) }) + it.each(CLAUDE_CODE_PERMISSION_MODES)( + 'maps the %s mode and only confirms the dangerous bypass', + (permissionMode) => { + const child = fakeChild() + const options = claudeQueryOptions({ + cwd: '/workspace', + permissionMode, + env: {}, + disposeGraceMs: 17, + spawn: () => child.handle, + }, new AbortController(), () => {}, () => {}) + expect(options.permissionMode).toBe(permissionMode) + expect(options.disallowedTools).toEqual(permissionMode === 'plan' + ? ['AskUserQuestion', 'ExitPlanMode'] + : ['AskUserQuestion']) + if (permissionMode === 'bypassPermissions') { + expect(options.allowDangerouslySkipPermissions).toBe(true) + expect(options).not.toHaveProperty('canUseTool') + } else { + expect(options).not.toHaveProperty('allowDangerouslySkipPermissions') + expect(options.canUseTool).toBeTypeOf('function') + } + }, + ) + + it('disallows ExitPlanMode before native plan-mode allow rules', () => { + const child = fakeChild() + const options = claudeQueryOptions({ + cwd: '/workspace', + permissionMode: 'plan', + env: {}, + disposeGraceMs: 17, + spawn: () => child.handle, + }, new AbortController(), () => {}, () => {}) + expect(options.disallowedTools).toEqual([ + 'AskUserQuestion', + 'ExitPlanMode', + ]) + }) + it('accepts only a non-error success with a non-blank final result', () => { expect(successfulResult(success('exact final'))).toBe('exact final') expect(() => successfulResult(success('answer', true))) - .toThrow('marked as an error') + .toThrow(expectedFailureDiagnostic('query-run', 'invalid-success')) expect(() => successfulResult(success(' \n '))) - .toThrow('contained no answer') - expect(() => successfulResult(failure( + .toThrow(expectedFailureDiagnostic('query-run', 'invalid-success')) + const sdkFailure = () => successfulResult(failure( 'error_during_execution', - ['first', 'second'], - ))).toThrow('first; second') + ['SECRET_TOKEN', '/private/secret.txt'], + )) + expect(sdkFailure).toThrow(expectedFailureDiagnostic( + 'query-run', + 'error_during_execution', + )) + expect(sdkFailure).not.toThrow('SECRET_TOKEN') + expect(sdkFailure).not.toThrow('/private/secret.txt') expect(() => successfulResult(failure( 'error_max_turns', [], - ))).toThrow('error_max_turns') + ))).toThrow(expectedFailureDiagnostic('query-run', 'error_max_turns')) + + const unknown = { + type: 'result', + subtype: 'future_failure', + is_error: true, + errors: ['SECRET_TOKEN'], + } as unknown as SDKResultMessage + expect(() => successfulResult(unknown)) + .toThrow(expectedFailureDiagnostic('query-run', 'unknown')) + expect(() => successfulResult(unknown)).not.toThrow('future_failure') + expect(() => successfulResult(unknown)).not.toThrow('SECRET_TOKEN') }) it('consumes the complete stream and keeps the latest strict success', async () => { @@ -613,7 +1009,17 @@ describe('query options and result mapping', () => { }) await expect(consumeClaudeQuery( queryFrom([{ type: 'system', subtype: 'init' } as SDKMessage]), - )).rejects.toThrow('ended without a result') + )).rejects.toThrow(expectedFailureDiagnostic('query-run', 'missing-result')) + + const onPermissionDenied = vi.fn() + await expect(consumeClaudeQuery(queryFrom([ + permissionDenied(), + success('after denial'), + ]), onPermissionDenied)).resolves.toEqual({ + output: [{ type: 'text', text: 'after denial' }], + stopReason: 'completed', + }) + expect(onPermissionDenied).toHaveBeenCalledOnce() }) }) @@ -657,6 +1063,7 @@ describe('run publication, cancellation, and settlement', () => { ) await expect(run.result).resolves.toEqual({ output: [], + diagnostic: expectedFailureDiagnostic('query-run', subtype), stopReason: 'error', }) expect(onError).toHaveBeenCalledWith( @@ -667,34 +1074,149 @@ describe('run publication, cancellation, and settlement', () => { } }) - it('fails closed when iteration rejects after a result', async () => { - const fixture = fakeRun( - [success('partial final')], - new Error('iterator boom'), - ) + it('attaches a safe diagnostic when a permission denial precedes failure', async () => { + const fixture = fakeRun([ + permissionDenied(), + failure('error_during_execution'), + ]) const run = await startClaudeCodeRun(request(), fixture.spec) + const result = await run.result + expect(result).toEqual({ + output: [], + diagnostic: `${expectedFailureDiagnostic('query-run', 'error_during_execution')}\nClaude Code unattended decision (mode: dontAsk; request: tool permission; decision: denied): Claude Code denied the request before an interactive prompt`, + stopReason: 'error', + }) + expect(result.diagnostic).not.toContain('SECRET_TOKEN') + expect(result.diagnostic).not.toContain('/private/secret.txt') + await run.dispose() + }) + + it('omits captured diagnostics on success and isolates concurrent runs', async () => { + const children = [fakeChild(), fakeChild()] + let childIndex = 0 + const spec: ClaudeCodeRunSpec = { + cwd: '/workspace', + permissionMode: 'dontAsk', + env: {}, + disposeGraceMs: 5, + spawn: () => children[childIndex++]!.handle, + } + queryMock.mockImplementation(({ prompt, options }) => { + options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + return prompt === 'denied then completed' + ? queryFrom([permissionDenied(), success('completed answer')]) + : queryFrom([failure('error_during_execution')]) + }) + + const [completed, failed] = await Promise.all([ + startClaudeCodeRun( + request([{ type: 'text', text: 'denied then completed' }]), + spec, + ), + startClaudeCodeRun( + request([{ type: 'text', text: 'unrelated failure' }]), + spec, + ), + ]) + await expect(completed.result).resolves.toEqual({ + output: [{ type: 'text', text: 'completed answer' }], + stopReason: 'completed', + }) + await expect(failed.result).resolves.toEqual({ + output: [], + diagnostic: expectedFailureDiagnostic( + 'query-run', + 'error_during_execution', + ), + stopReason: 'error', + }) + await Promise.all([completed.dispose(), failed.dispose()]) + }) + + it('fails closed when iteration rejects after a result', async () => { + const child = fakeChild() + const outcome = { exitCode: 31, signal: null } as const + async function* stream(): AsyncGenerator { + yield success('partial final') + child.settle(outcome) + await Promise.resolve() + throw new Error('iterator boom') + } + queryMock.mockImplementation(({ options }) => { + options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + return Object.assign(stream(), { close: vi.fn() }) as unknown as Query + }) + const run = await startClaudeCodeRun(request(), { + cwd: '/workspace', + permissionMode: DEFAULT_CLAUDE_CODE_PERMISSION_MODE, + env: {}, + disposeGraceMs: 5, + spawn: () => child.handle, + }) await expect(run.result).resolves.toEqual({ output: [], + diagnostic: expectedFailureDiagnostic('query-run', 'unknown', outcome), stopReason: 'error', }) await run.dispose() }) - it('maps invalid success and missing result to error', async () => { - for (const messages of [ - [success('answer', true)], - [success('')], - [{ type: 'system', subtype: 'init' } as SDKMessage], - ]) { + it('maps invalid success and missing result to fixed query-run facts', async () => { + for (const [messages, category] of [ + [[success('answer', true)], 'invalid-success'], + [[success('')], 'invalid-success'], + [[{ type: 'system', subtype: 'init' } as SDKMessage], 'missing-result'], + ] as const) { const fixture = fakeRun(messages) const run = await startClaudeCodeRun(request(), fixture.spec) - await expect(run.result).resolves.toMatchObject({ + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: expectedFailureDiagnostic('query-run', category), stopReason: 'error', }) await run.dispose() } }) + it('reports an early process exit with independent code and signal facts', async () => { + const outcomes: SubprocessOutcome[] = [ + { exitCode: 23, signal: null }, + { exitCode: null, signal: 'SIGABRT' }, + { exitCode: null, signal: null }, + ] + for (const outcome of outcomes) { + const child = fakeChild() + async function* stream(): AsyncGenerator { + child.settle(outcome) + await Promise.resolve() + throw new Error('SECRET_TOKEN from process transport') + } + queryMock.mockImplementation(({ options }) => { + options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + return Object.assign(stream(), { close: vi.fn() }) as unknown as Query + }) + const run = await startClaudeCodeRun(request(), { + cwd: '/workspace', + permissionMode: DEFAULT_CLAUDE_CODE_PERMISSION_MODE, + env: {}, + disposeGraceMs: 5, + spawn: () => child.handle, + }) + const result = await run.result + expect(result).toEqual({ + output: [], + diagnostic: expectedFailureDiagnostic( + 'process', + 'process-exit', + outcome, + ), + stopReason: 'error', + }) + expect(result.diagnostic).not.toContain('SECRET_TOKEN') + await run.dispose() + } + }) + it('gives local cancellation precedence and isolates overlapping controllers', async () => { const firstChild = fakeChild() const secondChild = fakeChild() @@ -703,7 +1225,7 @@ describe('run publication, cancellation, and settlement', () => { let index = 0 const spec: ClaudeCodeRunSpec = { cwd: '/workspace', - executable: '/native/claude', + permissionMode: 'dontAsk', env: {}, disposeGraceMs: 5, spawn: () => children[index++]!.handle, @@ -754,7 +1276,7 @@ describe('run publication, cancellation, and settlement', () => { request(undefined, parentAbort.signal), { cwd: '/workspace', - executable: '/native/claude', + permissionMode: DEFAULT_CLAUDE_CODE_PERMISSION_MODE, env: {}, disposeGraceMs: 5, spawn: () => child.handle, @@ -783,7 +1305,7 @@ describe('run publication, cancellation, and settlement', () => { ) await expect(startClaudeCodeRun(request(), { ...unused.spec, - })).rejects.toThrow('did not publish a controllable') + })).rejects.toThrow(expectedFailureDiagnostic('query-start', 'unknown')) expect(noChildClose).toHaveBeenCalledOnce() const closeFailure = vi.fn(() => { throw new Error('close boom') }) @@ -793,6 +1315,11 @@ describe('run publication, cancellation, and settlement', () => { const noChild = startClaudeCodeRun(request(), { ...unused.spec, }) + await expect(noChild) + .rejects.toThrow(expectedFailureDiagnostic('query-start', 'unknown')) + await expect(noChild).rejects.toThrow( + `${expectedFailureDiagnostic('query-start', 'unknown')}; subagent-claude-code: ${expectedFailureDiagnostic('teardown', 'unknown')}`, + ) await expect(noChild).rejects.toBeInstanceOf(AggregateError) const startupAbort = new AbortController() @@ -815,12 +1342,53 @@ describe('run publication, cancellation, and settlement', () => { expect(abortedClose).toHaveBeenCalledOnce() expect(abortedChild.terminate).toHaveBeenCalledOnce() + const cleanupAbort = new AbortController() + const cleanupFailedChild = fakeChild({ + waitForExitError: new Error('SECRET_TOKEN cleanup wait failure'), + }) + queryMock.mockImplementationOnce(({ options }) => { + options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + cleanupAbort.abort(new Error('startup cancelled')) + return queryFrom([]) + }) + const cancelledCleanupFailure = startClaudeCodeRun( + request(undefined, cleanupAbort.signal), + { + ...unused.spec, + spawn: () => cleanupFailedChild.handle, + }, + ) + await expect(cancelledCleanupFailure) + .rejects.toBeInstanceOf(AggregateError) + await expect(cancelledCleanupFailure) + .rejects.toThrow(expectedFailureDiagnostic('query-start', 'unknown')) + await expect(cancelledCleanupFailure).rejects.toThrow( + `${expectedFailureDiagnostic('query-start', 'unknown')}; subagent-claude-code: ${expectedFailureDiagnostic('teardown', 'unknown', { exitCode: 0, signal: null })}`, + ) + await expect(cancelledCleanupFailure) + .rejects.not.toThrow('SECRET_TOKEN') + queryMock.mockImplementationOnce(() => { throw new Error('query failed before resource creation') }) - await expect(startClaudeCodeRun(request(), { + const queryFailureOnError = vi.fn< + NonNullable + >() + const queryFailure = startClaudeCodeRun(request(), { ...unused.spec, - })).rejects.toThrow('query failed before resource creation') + onError: queryFailureOnError, + }) + await expect(queryFailure) + .rejects.toThrow(expectedFailureDiagnostic('query-start', 'unknown')) + await expect(queryFailure).rejects.not.toThrow( + 'query failed before resource creation', + ) + expect(queryFailureOnError).toHaveBeenCalledWith( + expect.any(Error), + 'error', + ) + expect(errorCause(queryFailureOnError.mock.calls[0]?.[0])?.message) + .toBe('query failed before resource creation') const spawned = fakeChild() const spawnSpecs: SubprocessSpawnSpec[] = [] @@ -828,6 +1396,7 @@ describe('run publication, cancellation, and settlement', () => { queryMock.mockImplementationOnce(({ options }) => { factoryController = options.abortController options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + spawned.settle({ exitCode: 17, signal: null }) throw new Error('query construction failed') }) const factoryFailure = startClaudeCodeRun(request(), { @@ -837,19 +1406,154 @@ describe('run publication, cancellation, and settlement', () => { return spawned.handle }, }) - await expect(factoryFailure).rejects.toThrow('query construction failed') + await expect(factoryFailure).rejects.toThrow(expectedFailureDiagnostic( + 'query-start', + 'unknown', + { exitCode: 17, signal: null }, + )) + await expect(factoryFailure).rejects.not.toThrow('query construction failed') expect(spawnSpecs).toHaveLength(1) expect(factoryController?.signal.aborted).toBe(true) expect(spawned.terminate).toHaveBeenCalledOnce() + const cleanupRaceAbort = new AbortController() + const cleanupRaceChild = fakeChild({ exitOnTerminate: false }) + queryMock.mockImplementationOnce(({ options }) => { + options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + throw new Error('query failed before cleanup wait') + }) + const cleanupRace = startClaudeCodeRun( + request(undefined, cleanupRaceAbort.signal), + { + ...unused.spec, + spawn: () => cleanupRaceChild.handle, + }, + ) + await nextTask() + cleanupRaceAbort.abort(new Error('cancelled during cleanup')) + cleanupRaceChild.settle() + await expect(cleanupRace).rejects.toThrow('aborted before SDK startup') + + const spawnError = Object.assign( + new Error('spawn /sdk/claude EACCES'), + { code: 'EACCES', path: '/sdk/claude' }, + ) const failedSpawn = fakeChild({ pid: -1, - doneError: new Error('spawn failed'), + doneError: spawnError, }) const failed = fakeRun([], undefined, failedSpawn) - await expect(startClaudeCodeRun(request(), failed.spec)) - .rejects.toBeInstanceOf(AggregateError) + const failedStartup = startClaudeCodeRun(request(), failed.spec) + await expect(failedStartup) + .rejects.toThrow(expectedFailureDiagnostic('query-start', 'unknown')) + await expect(failedStartup).rejects.not.toThrow('spawn /sdk/claude EACCES') + await expect(failedStartup).rejects.toMatchObject({ cause: spawnError }) expect(failed.close).toHaveBeenCalledOnce() + expect(failedSpawn.terminate).not.toHaveBeenCalled() + expect(failedSpawn.waitForExit).not.toHaveBeenCalled() + + const failedSpawnAbort = new AbortController() + const cancelledFailedSpawn = fakeChild({ + pid: -1, + doneError: spawnError, + }) + const cancelledFailedClose = vi.fn() + queryMock.mockImplementationOnce(({ options }) => { + options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + failedSpawnAbort.abort(new Error('startup cancelled')) + return queryFrom([], undefined, cancelledFailedClose) + }) + await expect(startClaudeCodeRun( + request(undefined, failedSpawnAbort.signal), + { ...unused.spec, spawn: () => cancelledFailedSpawn.handle }, + )).rejects.toThrow('aborted before SDK startup') + expect(cancelledFailedClose).toHaveBeenCalledOnce() + + const cancelledFailedSpawnCloseError = new Error('cancelled query close failed') + const cancelledFailedSpawnClose = vi.fn(() => { + throw cancelledFailedSpawnCloseError + }) + const cancelledFailedSpawnWithCloseFailure = fakeChild({ + pid: -1, + doneError: spawnError, + }) + const failedSpawnAbortWithCloseFailure = new AbortController() + queryMock.mockImplementationOnce(({ options }) => { + options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + failedSpawnAbortWithCloseFailure.abort(new Error('startup cancelled')) + return queryFrom([], undefined, cancelledFailedSpawnClose) + }) + const cancelledWithCloseFailure = startClaudeCodeRun( + request(undefined, failedSpawnAbortWithCloseFailure.signal), + { ...unused.spec, spawn: () => cancelledFailedSpawnWithCloseFailure.handle }, + ) + await expect(cancelledWithCloseFailure).rejects.toMatchObject({ + message: `subagent-claude-code: ${expectedFailureDiagnostic('query-start', 'unknown')}; subagent-claude-code: ${expectedFailureDiagnostic('teardown', 'unknown')}`, + errors: [ + expect.objectContaining({ + message: `subagent-claude-code: ${expectedFailureDiagnostic('query-start', 'unknown')}`, + cause: spawnError, + }), + expect.objectContaining({ + message: `subagent-claude-code: ${expectedFailureDiagnostic('teardown', 'unknown')}`, + cause: cancelledFailedSpawnCloseError, + }), + ], + }) + await expect(cancelledWithCloseFailure) + .rejects.not.toThrow('spawn /sdk/claude EACCES') + expect(cancelledFailedSpawnClose).toHaveBeenCalledOnce() + + const failedSpawnCloseError = new Error('query close failed') + const failedSpawnClose = vi.fn(() => { throw failedSpawnCloseError }) + const failedSpawnWithCloseFailure = fakeChild({ + pid: -1, + doneError: spawnError, + }) + queryMock.mockImplementationOnce(({ options }) => { + options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + return queryFrom([], undefined, failedSpawnClose) + }) + const failedWithCloseFailure = startClaudeCodeRun(request(), { + ...unused.spec, + spawn: () => failedSpawnWithCloseFailure.handle, + }) + await expect(failedWithCloseFailure) + .rejects.toThrow(expectedFailureDiagnostic('query-start', 'unknown')) + await expect(failedWithCloseFailure) + .rejects.not.toThrow('spawn /sdk/claude EACCES') + await expect(failedWithCloseFailure).rejects.toMatchObject({ + message: `subagent-claude-code: ${expectedFailureDiagnostic('query-start', 'unknown')}; subagent-claude-code: ${expectedFailureDiagnostic('teardown', 'unknown')}`, + errors: [ + expect.objectContaining({ cause: spawnError }), + expect.objectContaining({ cause: failedSpawnCloseError }), + ], + }) + + const cleanupError = new Error('live child cleanup failed') + const constructionError = new Error( + 'query construction failed with a live child', + ) + const liveChildCleanupFailure = fakeChild({ waitForExitError: cleanupError }) + queryMock.mockImplementationOnce(({ options }) => { + options.spawnClaudeCodeProcess!(sdkSpawnOptions()) + throw constructionError + }) + const liveCleanupFailure = startClaudeCodeRun(request(), { + ...unused.spec, + spawn: () => liveChildCleanupFailure.handle, + }) + await expect(liveCleanupFailure).rejects.toMatchObject({ + message: `subagent-claude-code: ${expectedFailureDiagnostic('query-start', 'unknown')}; subagent-claude-code: ${expectedFailureDiagnostic('teardown', 'unknown', { exitCode: 0, signal: null })}`, + errors: [ + expect.objectContaining({ cause: constructionError }), + expect.objectContaining({ cause: cleanupError }), + ], + }) + await expect(liveCleanupFailure) + .rejects.not.toThrow('query construction failed with a live child') + await expect(liveCleanupFailure) + .rejects.not.toThrow('live child cleanup failed') }) }) @@ -868,6 +1572,28 @@ describe('query and process disposal', () => { }) }) + it('reports a published teardown failure to the Host diagnostic sink', async () => { + const fixture = fakeRun([success('exact answer')]) + const onError = vi.fn>() + const run = await startClaudeCodeRun(request(), { + ...fixture.spec, + onError, + }) + await expect(run.result).resolves.toMatchObject({ stopReason: 'completed' }) + fixture.close.mockImplementationOnce(() => { + throw new Error('SECRET_TOKEN close failure') + }) + await expect(run.dispose()).rejects.toThrow( + expectedFailureDiagnostic('teardown', 'unknown', { + exitCode: 0, + signal: null, + }), + ) + expect(onError).toHaveBeenCalledWith(expect.any(Error), 'error') + expect(errorCause(onError.mock.calls[0]?.[0])?.message) + .toBe('SECRET_TOKEN close failure') + }) + it('does not finish disposal before the managed tree exits', async () => { const child = fakeChild({ exitOnTerminate: false }) let disposed = false @@ -884,33 +1610,30 @@ describe('query and process disposal', () => { expect(disposed).toBe(true) }) - it('reports wait, close, and direct-child failures without skipping cleanup', async () => { + it('reports close and tree-wait failures without skipping cleanup', async () => { const waitFailure = fakeChild({ waitForExitError: new Error('wait boom'), }) const closeFailure = vi.fn(() => { throw new Error('close boom') }) - await expect(disposeClaudeCodeChild( + const waitAndClose = disposeClaudeCodeChild( { close: closeFailure }, waitFailure.handle, - )).rejects.toBeInstanceOf(AggregateError) + ) + await expect(waitAndClose).rejects.toThrow(expectedFailureDiagnostic( + 'teardown', + 'unknown', + { exitCode: 0, signal: null }, + )) + const waitAndCloseError = await waitAndClose.then( + () => undefined, + (error: unknown) => error, + ) + const waitAndCloseCause = errorCause(waitAndCloseError) + expect(waitAndCloseCause).toBeInstanceOf(AggregateError) + expect((waitAndCloseCause as AggregateError).errors).toEqual([ + expect.objectContaining({ message: 'close boom' }), + expect.objectContaining({ message: 'wait boom' }), + ]) expect(waitFailure.terminate).toHaveBeenCalledOnce() - - const doneFailure = fakeChild({ - pid: -1, - doneError: new Error('spawn boom'), - }) - await expect(disposeClaudeCodeChild( - { close: vi.fn() }, - doneFailure.handle, - )).rejects.toThrow('spawn boom') - - const both = fakeChild({ - pid: -1, - doneError: new Error('spawn boom'), - }) - await expect(disposeClaudeCodeChild( - { close: () => { throw new Error('close boom') } }, - both.handle, - )).rejects.toBeInstanceOf(AggregateError) }) }) diff --git a/packages/subagent/subagent-codex/README.i18n.yaml b/packages/subagent/subagent-codex/README.i18n.yaml index 6f111f4ffc..0f8ed31eab 100644 --- a/packages/subagent/subagent-codex/README.i18n.yaml +++ b/packages/subagent/subagent-codex/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/subagent-codex/README.md -README.md: 3d59ca1eaf3db9dd9d9d2cd451692ebd2a956ef4 -README.zh.md: abff7b569e2ce8261366c004ab6d03ea53300fdb +README.md: 975f353b9f1bc6fab61a4c0eb40ebaf50c436623 +README.zh.md: 2ea256afb3bb8fdfe57fd555b6db10522a709a56 diff --git a/packages/subagent/subagent-codex/README.md b/packages/subagent/subagent-codex/README.md index 3d59ca1eaf..975f353b9f 100644 --- a/packages/subagent/subagent-codex/README.md +++ b/packages/subagent/subagent-codex/README.md @@ -2,17 +2,19 @@ English | [中文](README.zh.md) -This package registers the fixed `codex` subagent provider. Each accepted run starts the official `codex app-server --stdio` command in the delegating Session's workspace, creates one ephemeral Codex thread, submits one self-contained text task, and returns only the final answer through the shared [`dsh-subagent`](../subagent/README.md) result contract. +This package registers a Profile-named Codex subagent provider whose default name is `codex`. Each accepted run starts the official package-local Codex wrapper with `app-server --stdio` in the delegating Session's workspace, creates one ephemeral Codex thread, submits one self-contained text task, and returns either the selected final answer or a separate safe failure diagnostic through the shared [`dsh-subagent`](../subagent/README.md) result contract. ## Start and ownership -`start(request)` accepts only a non-empty sequence of text blocks and derives the child cwd from the parent Session. It then spawns the fixed command through [`dsh-subprocess`](../../subprocess/subprocess/README.md), performs `initialize` → `initialized` → `thread/start { cwd, ephemeral: true }`, and publishes the run only after Codex returns a valid ephemeral thread. A failure or cancellation before publication closes the wire, terminates the managed process tree, waits for it to exit, and rejects `start()`. +`start(request)` accepts only a non-empty sequence of text blocks and derives the child cwd from the parent Session. It then spawns the fixed command through [`dsh-subprocess`](../../subprocess/subprocess/README.md), performs `initialize` → `initialized`, maps the Profile-selected mode into official `thread/start` approval/reviewer/sandbox fields beside `{ cwd, ephemeral: true }`, and publishes the run only after Codex returns a valid ephemeral thread. A failure or cancellation before publication closes the wire, terminates the managed process tree, waits for it to exit, and rejects `start()`. Non-cancellation rejections expose only the fixed `initialize` or `thread-start` stage plus an already observed process outcome; raw product and Host errors remain on internal cause chains. The published `run.result` starts exactly one turn. It accepts only notifications for that run's thread and turn, then waits for the authoritative `turn/completed` terminal notification. The latest `agentMessage` with `phase: "final_answer"` wins; when Codex emits no explicit final phase, the latest message with `phase: null` is the compatibility fallback. Commentary never replaces either answer, and a successful turn with no nonblank answer settles as an error. -For command and file approvals, the unattended provider selects a non-approval decision offered by the request, preferring `cancel`; the stable 0.147.0 request shape without an offered-decision list falls back to `decline`. It answers permission requests with an empty turn-scoped permission set, answers user-input requests with no answers, and declines MCP elicitation. A request with no legal unattended response, or any unknown server request, fails the run. +For command and file approvals, the unattended provider selects a non-approval decision offered by the request, preferring `cancel`; the stable 0.147.0 request shape without an offered-decision list falls back to `decline`. It answers permission requests with an empty turn-scoped permission set, answers user-input requests with no answers, and declines MCP elicitation. A request with no legal unattended response, or any unknown server request, fails the run. The wire records only the effective mode, request category, decision, and fixed safe reason. It also recognizes declined command/file items and `sandboxError` terminals. Codex 0.147.0 writes some early `never` rejections and sandbox violations only to structured stderr, so the Provider pipes stderr, forwards it unchanged to the host, and matches two fixed signatures in a bounded per-run tail; raw stderr never enters the diagnostic. -Local cancellation wins the result race and maps to `aborted`. A failed turn whose `codexErrorInfo` is `contextWindowExceeded` maps to `max-tokens`; every other remote interrupted or failed turn maps to `error`, and the provider produces no `refusal`. `dispose()` is idempotent: it requests a best-effort `turn/interrupt` with both current ids when they are known, closes the JSON-RPC wire, ends stdin, invokes the shared process-tree termination escalation, and waits for whole-tree exit. Result failure and independent teardown failure remain separate. +Local cancellation wins the result race and maps to `aborted`. For failed turns, the diagnostic preserves all eleven string and five object variants in the Codex 0.147.0 `codexErrorInfo` union; the four connection/stream variants retain a numeric `httpStatusCode` when supplied, while `activeTurnNotSteerable` does not expose `turnKind`. The diagnostic also names `turn-start`, `turn`, or `process`, independently includes available exit code and signal, and uses `unknown` for unrecognized or malformed values without copying raw fields. `contextWindowExceeded` remains `max-tokens`; every other remote interruption or failure remains `error`, and the provider produces no `refusal`. A contributing permission decision follows the structured failure line. Successful and locally cancelled runs omit both facts. + +`dispose()` is idempotent: it requests a best-effort `turn/interrupt` with both current ids when they are known, closes the JSON-RPC wire, ends stdin, invokes the shared process-tree termination escalation, waits for whole-tree exit, and detaches the stderr observer. Independent cleanup rejection uses the fixed `teardown` stage and any available process outcome. When startup and rollback both fail, the top-level aggregate message preserves both safe stage lines while the raw failures remain internal. ## Capabilities and context @@ -22,33 +24,81 @@ The provider advertises no optional start-time capabilities and reports `inherit | Key | Default | Meaning | |---|---|---| +| `providerName` | `codex` | Non-empty registry name on `ctx.subagents`; each mounted instance needs a unique value. | | `env` | `{}` | Explicit child environment layered over the subprocess seam's credential-scrubbed parent environment. | +| `permissionMode` | `never` | Native non-interactive approval and sandbox mode fixed for every thread from this Provider instance. | | `disposeGraceMs` | `3000` | Positive finite grace in milliseconds, no greater than [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md), between the shared process-tree owner's termination tiers; disposal then waits for whole-tree exit. | -Production resolves `codex` from `PATH` and uses the host's native Codex configuration and authentication. The plugin does not install Codex, select a model, create `CODEX_HOME`, log in, or probe a version. Credential-shaped ambient variables are removed by the subprocess seam, so an API key intended for the child must be supplied explicitly in `env`; ordinary ambient values such as `PATH` and `HOME` remain available unless overridden. +| `permissionMode` value | `thread/start` fields | Native behavior | +|---|---|---| +| `never` | `approvalPolicy: never`; sandbox omitted | Never ask for approval; execution failures return to the model under the native sandbox. | +| `approve-for-me` | `approvalPolicy: on-request`, `approvalsReviewer: auto_review`, `sandbox: workspace-write` | Route permission requests through Codex automatic review without a human. | +| `dangerously-bypass-approvals-and-sandbox` | `approvalPolicy: never`, `sandbox: danger-full-access` | Skip approval and sandbox enforcement; this value must be selected explicitly. | -Shipped profiles load this provider once on the host and start no Codex process until a tool call. Full Agent Presets carry the tool row below with `disabled: true`; copy a preset and remove that field to expose `subagent_codex` only to agents composed from the copy. A custom host composition can still use both rows directly. +Production resolves the `codex` bin declared by its pinned `@openai/codex@0.147.0` dependency and launches that JavaScript wrapper with the current Node executable. The wrapper selects the matching native platform payload; the provider neither inspects nor falls back to a host `codex` on `PATH`. Native Codex configuration and authentication remain authoritative through the parent cwd, `HOME`, and `CODEX_HOME`, while the Provider overrides only the selected thread approval/reviewer/sandbox fields. All other project, model, provider, MCP, hook, skill, and account settings remain native. The plugin does not select a model, create `CODEX_HOME`, log in, or probe an account. Credential-shaped ambient variables are removed by the subprocess seam before the explicit `env` overlay is applied. + +This package is an optional Profile Bundle. Install it into the target Profile, then restart that Profile; installation brings the official wrapper and one compatible native platform payload into that Profile, while the declared `cordis.patch.yml` layer registers only the dormant `codex` Host provider and starts no Codex process. Removing the package withdraws that provider and its private runtime closure on the next Profile start. + +```sh +dsh plugin --profile add @deepseek-ai/dsh-subagent-codex +dsh plugin --profile remove @deepseek-ai/dsh-subagent-codex +dsh --profile +``` + +Installation controls Host availability, not model permission. The Bundle supplies the dormant default `codex` row; the Profile may replace that row's complete config or mount additional rows with distinct `providerName`, `permissionMode`, and `env` values. Loading an instance starts no Codex process until a bound tool calls it. Each `dsh-tool-subagent` row names one provider and needs its own `toolName`, so the model sees static tools rather than a dynamic provider selector. Full Agent Presets carry a matching default product tool row with `disabled: true`; copy a preset and remove that field to expose `subagent_codex` only to agents composed from the copy. Its `one-shot` policy keeps omitted or `false` `run_in_background` calls in the foreground, while explicit `true` returns a parent-owned Job id for `job_output` or `job_kill`. The base host and full presets already provide the generic Job registry and controls. + +The standalone composition below shows the complete explicit capability. A Profile based on `@deepseek-ai/dsh-base` keeps its existing Job rows, adds the product provider and tool rows, and does not mount duplicate Job services. ```yaml -- id: subagent-codex +- id: subagent-codex-safe name: '@deepseek-ai/dsh-subagent-codex' config: + providerName: codex-safe + permissionMode: never env: OPENAI_API_KEY: !!js process.env.OPENAI_API_KEY -- id: tool-subagent-codex +- id: subagent-codex-bypass + name: '@deepseek-ai/dsh-subagent-codex' + config: + providerName: codex-bypass + permissionMode: dangerously-bypass-approvals-and-sandbox + env: + OPENAI_API_KEY: !!js process.env.OPENAI_API_KEY +``` + +```yaml +- id: jobs + name: '@deepseek-ai/dsh-jobs-local' + +- id: tool-jobs + name: '@deepseek-ai/dsh-tool-jobs' + +- id: tool-subagent-codex-safe name: '@deepseek-ai/dsh-tool-subagent' disabled: true config: - provider: codex - toolName: subagent_codex - enableRunInBackground: false + provider: codex-safe + toolName: subagent_codex_safe + backgroundMode: one-shot + maxDepth: provider-managed + +- id: tool-subagent-codex-bypass + name: '@deepseek-ai/dsh-tool-subagent' + config: + provider: codex-bypass + toolName: subagent_codex_bypass + backgroundMode: one-shot maxDepth: provider-managed ``` ## Product compatibility and evidence -The production wire intentionally implements only the app-server methods required by this one-shot contract. Development evidence is pinned to `@openai/codex@0.147.0` / `codex-cli 0.147.0`; the npm package is a test-only dependency, and deployments still supply `codex` on `PATH`. +The production wire intentionally implements only the app-server methods required by this one-shot contract. The runtime dependency and all six optional-dependency aliases are pinned to `@openai/codex@0.147.0` / `codex-cli 0.147.0`. A normal install selects one payload for the current OS and CPU. For the current darwin-arm64 payload, `npm pack --dry-run --json @openai/codex@0.147.0-darwin-arm64` reports 111,199,052 packed bytes and 274,777,843 unpacked bytes. That package contains native `codex`, `codex-code-mode-host`, `rg`, and `zsh` resources; other platforms may differ, and these values are disclosure rather than an installation threshold. + +Generated schema evidence and package tests pin all sixteen error-info variants, HTTP-status locations, six lifecycle stages, process outcomes, stop-reason mapping, unknown fallback, sanitization, permission ordering, cancellation, concurrency, and cleanup aggregation. The keyless real-product test drives the package wrapper against a loopback Responses fixture and observes the package-local argv, exact Bearer key, original task, byte-exact final answer, thread-level `never` overriding ambient `on-request`, automatic-review startup, unattended rejection without file side effects, a real `internalServerError`, explicit dangerous-bypass writing in suite-owned temporary storage, process/protocol failure with safe exit facts, and wrapper/native quiescence. The same tier proves two named instances retain separate environments and native modes. + +Installing with optional dependencies omitted, using an unsupported platform, or losing the selected payload makes the first delegation fail at `initialize` with the safe `unknown` category and any observed process outcome. Raw wrapper text remains on Host stderr; the provider neither probes a host CLI nor retries with one. An isolated wrapper fixture separately proves the native payload failure and absence of host fallback. ## Model Experience @@ -56,7 +106,7 @@ The production wire intentionally implements only the app-server methods require #### What the model sees -The Codex child receives the standalone text blocks as one turn in a fresh ephemeral thread. Its workspace is the parent Session cwd, and its model, system instructions, tools, sandbox, and authentication come from the native Codex installation and configuration. +The Codex child receives the standalone text blocks as one turn in a fresh ephemeral thread. Its workspace is the parent Session cwd; its model, system instructions, tools, and authentication come from native Codex configuration, the selected Provider instance's Profile configuration fixes the thread's environment, non-interactive approval policy, and sandbox mode, and the executable version comes from the Bundle's pinned platform payload. #### Token effect @@ -66,26 +116,28 @@ The child pays for an independent Codex context and turn. Child tokens do not en Independent of the parent request cache. Reuse depends only on Codex's own provider, model, instructions, tools, and ephemeral-thread request. -### Parent tool result, indirectly +### Parent scheduling and results, indirectly #### What the model sees -Through `dsh-tool-subagent`, the parent sees only the selected final Codex answer or the consumer's exact error for a non-completed result. Codex commentary, reasoning, tool activity, stderr, workspace diffs, and product ids are not copied into the parent Session. +Through `dsh-tool-subagent`, a foreground call gives the parent the selected final Codex answer or an error containing the stop reason and optional safe diagnostic for a non-completed result. The diagnostic can distinguish the fixed error-info category, protocol stage, numeric HTTP status, and observed process outcome without copying product prose. A background call first returns a Job id; the generic job controls later deliver a completion notice, expose the same final answer or failed status detail through `job_output`, and let `job_kill` request cancellation. Codex commentary, reasoning, tool activity, raw stderr, workspace diffs, usage, product ids, commands, paths, and protocol payloads are not copied into the parent Session. #### Token effect -Parent input grows only by the final answer or error retained in the tool result. This provider adds no parent tool schema by itself. +Foreground input grows by the retained final answer or error. Background input also includes the start acknowledgement, completion notice, and any `job_output`, `job_kill`, or later status results; child tokens still do not enter the parent context. This provider adds no parent tool schema by itself. #### KV Cache effect -Append-only: the new tool result follows the reusable parent request prefix. +Append-only: foreground adds one result after the reusable parent prefix, while background appends the Job acknowledgement, notice, and later control or collection results. Background scheduling can add a notice-driven turn, but none of these messages rewrites the earlier prefix. ## Known Limitations and Deferred Work - **One fresh process, thread, and turn per run** — there is no continuation, resume, pooling, progress stream, or product-session persistence. -- **Host-managed product installation and account state** — a missing or incompatible `codex`, configuration error, or authentication failure is surfaced as a startup or run error; the plugin provides no installer, login flow, or runtime version gate. +- **Static instance selection** — Profile rows fix provider names and tool bindings; calls cannot choose a provider dynamically, and every exposed tool needs a unique `toolName`. +- **Authentication and account state remain native** — the Bundle supplies the CLI but does not create an account, log in, trust a project, or rewrite Codex settings; configuration and authentication failures surface with their lifecycle stage and the safe `unknown` fallback rather than a separate public taxonomy. +- **The native platform payload is required at delegation time** — installs that omit optional dependencies, unsupported platforms, and missing or damaged payloads fail at the first run; there is no host-CLI fallback. - **Compatibility is pinned by development evidence** — upgrading from the verified 0.147.0 protocol baseline requires regenerating upstream schema evidence and rerunning handshake, answer-selection, approval, cancellation, keyless real-product, and credentialed DeepSeek nonce tests. -- **No human approval path** — known unattended approval requests are denied and unknown server requests fail closed; deployments cannot configure an allow policy through this package. -- **Final text only** — reasoning, commentary, intermediate messages, tool traffic, usage, stderr, and workspace diffs remain product-local. +- **No human approval path** — known unattended approval requests are denied and unknown server requests fail closed; the three Profile modes never create a DSH interaction channel or per-call allow policy. +- **Assistant payload is final text only** — a failed run may additionally expose the separate safe diagnostic; reasoning, commentary, intermediate messages, tool traffic, usage, raw stderr, and workspace diffs remain outside the parent Session, while generic Job ids, notices, and status come from the shared job runtime. - **No optional shared capabilities** — output schemas, child personas, tool filtering, and harness depth enforcement are rejected by the shared service for this provider. - **No wall-clock timeout or side-effect rollback** — the caller cancels long work, and files or external systems changed before cancellation are not restored. diff --git a/packages/subagent/subagent-codex/README.zh.md b/packages/subagent/subagent-codex/README.zh.md index abff7b569e..2ea256afb3 100644 --- a/packages/subagent/subagent-codex/README.zh.md +++ b/packages/subagent/subagent-codex/README.zh.md @@ -2,17 +2,19 @@ [English](README.md) | 中文 -本包注册固定的 `codex` subagent 提供方。每次接受运行请求后,它都会在发起委托的会话工作区中启动官方 `codex app-server --stdio` 命令,创建一个临时 Codex 线程,提交一个自包含的文本任务,并通过共享的 [`dsh-subagent`](../subagent/README.md) 结果约定仅返回最终答案。 +本包注册由 Profile 命名、默认名称为 `codex` 的 Codex subagent 提供方。每次接受运行请求后,它都会在发起委托的会话工作区中使用 `app-server --stdio` 启动官方包内 Codex wrapper,创建一个临时 Codex 线程,提交一个自包含的文本任务,并通过共享的 [`dsh-subagent`](../subagent/README.zh.md) 结果约定返回选定的最终答案或独立的安全失败诊断。 ## 启动与所有权 -`start(request)` 只接受非空的文本块序列,并根据父会话确定子级 cwd。随后,它通过 [`dsh-subprocess`](../../subprocess/subprocess/README.md) spawn 固定命令,依次执行 `initialize` → `initialized` → `thread/start { cwd, ephemeral: true }`,且仅在 Codex 返回有效的临时线程后才发布此次运行。若在发布前发生失败或取消,它会关闭通信链路、终止受管进程树并等待其退出,然后拒绝 `start()` 调用。 +`start(request)` 只接受非空的文本块序列,并根据父会话确定子级 cwd。随后,它通过 [`dsh-subprocess`](../../subprocess/subprocess/README.zh.md) spawn 固定命令,依次执行 `initialize` → `initialized`,把 Profile 选择的模式映射为官方 `thread/start` approval/reviewer/sandbox 字段并与 `{ cwd, ephemeral: true }` 一起发送,且仅在 Codex 返回有效的临时线程后才发布此次运行。若在发布前发生失败或取消,它会关闭通信链路、终止受管进程树并等待其退出,然后拒绝 `start()` 调用。非取消拒绝只公开固定的 `initialize` 或 `thread-start` 阶段及已经观测到的进程结果;原始产品与 Host 错误只保留在内部 cause 链中。 已发布的 `run.result` 恰好启动一个轮次。它只接受与此次运行的线程和轮次匹配的通知,随后等待权威的终止通知 `turn/completed`。以最后一条 `phase: "final_answer"` 的 `agentMessage` 为准;若 Codex 没有发出明确的最终阶段,则以最后一条 `phase: null` 的消息作为兼容性回退。过程说明绝不会取代上述任一答案;成功完成的轮次若没有非空白答案,结果也会判为错误。 -对于命令与文件审批,无人值守的提供方会从请求给出的决策选项中选择一项不予批准的决策,并优先选择 `cancel`;稳定的 0.147.0 请求形态没有决策选项列表,因此回退到 `decline`。它对权限请求返回作用域限于当前轮次的空权限集,不向用户输入请求提供任何答案,并拒绝 MCP elicitation。若请求在无人值守模式下没有合法响应,或是未知服务器请求,此次运行就会失败。 +对于命令与文件审批,无人值守的提供方会从请求给出的决策选项中选择一项不予批准的决策,并优先选择 `cancel`;稳定的 0.147.0 请求形态没有决策选项列表,因此回退到 `decline`。它对权限请求返回作用域限于当前轮次的空权限集,不向用户输入请求提供任何答案,并拒绝 MCP elicitation。若请求在无人值守模式下没有合法响应,或是未知服务器请求,此次运行就会失败。wire 只记录有效模式、请求类别、决定与固定的安全原因,也会识别被拒绝的命令/文件 item 和 `sandboxError` 终态。Codex 0.147.0 的部分早期 `never` 拒绝和 sandbox violation 只写入结构化 stderr,因此提供方会 pipe stderr、原样转发给 Host,并在每次运行的有界尾缓冲中匹配两个固定签名;原始 stderr 不会进入诊断。 -本地取消会在结果竞态中胜出并映射为 `aborted`。失败轮次的 `codexErrorInfo` 若为 `contextWindowExceeded`,则映射为 `max-tokens`;其他任何远端中断或失败轮次都映射为 `error`,且该提供方不会产生 `refusal`。`dispose()`(资源释放)具有幂等性:如果当前的两个标识符均已知,它会尽力请求 `turn/interrupt`,关闭 JSON-RPC 通信链路,结束标准输入,调用共享的进程树逐级终止机制,并等待整棵进程树退出。结果失败与独立的清理失败仍彼此分离。 +本地取消会在结果竞态中胜出并映射为 `aborted`。对于失败轮次,诊断会保留 Codex 0.147.0 `codexErrorInfo` 联合中的全部十一种字符串与五种对象 variant;四种连接/stream variant 会在上游提供时保留数值 `httpStatusCode`,而 `activeTurnNotSteerable` 不公开 `turnKind`。诊断还会注明 `turn-start`、`turn` 或 `process`,分别包含可用的退出码与信号,并对无法识别或格式错误的值使用 `unknown`,且不复制原始字段。`contextWindowExceeded` 仍映射为 `max-tokens`;其他任何远端中断或失败仍映射为 `error`,且该提供方不会产生 `refusal`。参与失败的权限决定会跟在结构化失败行之后。成功与本地取消都不附带这两类事实。 + +`dispose()`(资源释放)具有幂等性:如果当前的两个标识符均已知,它会尽力请求 `turn/interrupt`,关闭 JSON-RPC 通信链路,结束标准输入,调用共享的进程树逐级终止机制,等待整棵进程树退出,并移除 stderr observer。独立清理拒绝使用固定的 `teardown` 阶段与可用进程结果。当启动与回滚同时失败时,顶层聚合消息会保留两条安全阶段说明,而原始失败仍只在内部可见。 ## 能力与上下文 @@ -22,33 +24,81 @@ | 配置键 | 默认值 | 含义 | |---|---|---| +| `providerName` | `codex` | `ctx.subagents` 中的非空注册名称;每个已挂载实例都需要唯一值。 | | `env` | `{}` | 显式指定的子进程环境,叠加在由子进程 seam 清除凭证后的父环境之上。 | -| `disposeGraceMs` | `3000` | 共享进程树责任方各终止层级之间的宽限期,单位为毫秒且须为正有限值,并不得大于仓库共享的 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md);随后资源释放会等待整棵进程树退出。 | +| `permissionMode` | `never` | 为该提供方实例的每个线程固定原生非交互审批与沙箱模式。 | +| `disposeGraceMs` | `3000` | 共享进程树责任方各终止层级之间的宽限期,单位为毫秒且须为正有限值,并不得大于仓库共享的 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.zh.md);随后资源释放会等待整棵进程树退出。 | -生产环境会从 `PATH` 中解析 `codex`,并使用宿主机原生的 Codex 配置与身份验证。本插件不安装 Codex、不选择模型、不创建 `CODEX_HOME`、不执行登录,也不探测版本。子进程 seam 会移除具有凭证特征的环境变量,因此供子进程使用的 API 密钥必须在 `env` 中显式提供;除非被覆盖,`PATH` 和 `HOME` 等普通环境变量值仍然可用。 +| `permissionMode` 值 | `thread/start` 字段 | 原生行为 | +|---|---|---| +| `never` | `approvalPolicy: never`;省略 sandbox | 永不请求审批;执行失败会在原生 sandbox 下返回模型。 | +| `approve-for-me` | `approvalPolicy: on-request`、`approvalsReviewer: auto_review`、`sandbox: workspace-write` | 由 Codex 自动评审权限请求,不等待人工。 | +| `dangerously-bypass-approvals-and-sandbox` | `approvalPolicy: never`、`sandbox: danger-full-access` | 跳过审批与 sandbox;必须显式选择该值。 | -随附 profile 会在宿主上加载一次该提供方,而且在工具被调用前不会启动 Codex 进程。完整 Agent Preset 携带下列工具行并设置 `disabled: true`;复制一个 preset 后删除该字段,即可只向由该副本组装的 agent 暴露 `subagent_codex`。自定义宿主组装仍可直接使用两条配置行。 +生产环境会解析锁定的 `@openai/codex@0.147.0` 依赖所声明的 `codex` bin,并使用当前 Node 可执行文件启动该 JavaScript wrapper。Wrapper 会选择匹配的原生平台载荷;提供方既不检查也不回退 `PATH` 中的宿主 `codex`。父会话 cwd、`HOME` 与 `CODEX_HOME` 继续让原生 Codex 配置和身份验证保持权威,而提供方只覆盖选定线程的 approval/reviewer/sandbox 字段。其他项目、模型、provider、MCP、hook、skill 与账户设置仍由原生机制负责。本插件不选择模型、不创建 `CODEX_HOME`、不执行登录,也不探测账户。子进程 seam 会先移除具有凭证特征的环境变量,再应用显式 `env` 覆盖。 + +本包是可选的 Profile Bundle。将它安装进目标 Profile 后重启该 Profile;安装会把官方 wrapper 与一个兼容的原生平台载荷带入该 Profile,而包所声明的 `cordis.patch.yml` 层只注册休眠的 `codex` Host provider,不会启动 Codex 进程。移除该包后,下一次 Profile 启动会撤回这一 provider 及其私有运行时闭包。 + +```sh +dsh plugin --profile add @deepseek-ai/dsh-subagent-codex +dsh plugin --profile remove @deepseek-ai/dsh-subagent-codex +dsh --profile +``` + +安装决定 Host 可用性,而不是模型权限。Bundle 会提供休眠的默认 `codex` 配置项;Profile 可以替换该配置项的完整 config,也可以挂载更多具有不同 `providerName`、`permissionMode` 与 `env` 的配置项。加载实例本身不会在绑定工具调用前启动 Codex 进程。每个 `dsh-tool-subagent` 配置项指定一个提供方,并需要独立的 `toolName`,因此模型看到的是静态工具,而不是动态提供方选择器。完整 Agent Preset 携带对应的默认产品工具行并设置 `disabled: true`;复制一个 preset 后删除该字段,即可只向由该副本组装的 agent 暴露 `subagent_codex`。其 `one-shot` 策略会让省略 `run_in_background` 或传入 `false` 的调用继续在前台等待,而显式传入 `true` 会返回由父 agent 拥有的 Job ID,供 `job_output` 或 `job_kill` 使用。base host(基础宿主)与完整 preset 已提供通用作业注册表和控制工具。 + +下列独立组装展示完整的显式能力。基于 `@deepseek-ai/dsh-base` 的 Profile 保留已有 Job 配置项,新增产品提供方与工具配置项,而且不重复挂载 Job 服务。 ```yaml -- id: subagent-codex +- id: subagent-codex-safe name: '@deepseek-ai/dsh-subagent-codex' config: + providerName: codex-safe + permissionMode: never env: OPENAI_API_KEY: !!js process.env.OPENAI_API_KEY -- id: tool-subagent-codex +- id: subagent-codex-bypass + name: '@deepseek-ai/dsh-subagent-codex' + config: + providerName: codex-bypass + permissionMode: dangerously-bypass-approvals-and-sandbox + env: + OPENAI_API_KEY: !!js process.env.OPENAI_API_KEY +``` + +```yaml +- id: jobs + name: '@deepseek-ai/dsh-jobs-local' + +- id: tool-jobs + name: '@deepseek-ai/dsh-tool-jobs' + +- id: tool-subagent-codex-safe name: '@deepseek-ai/dsh-tool-subagent' disabled: true config: - provider: codex - toolName: subagent_codex - enableRunInBackground: false + provider: codex-safe + toolName: subagent_codex_safe + backgroundMode: one-shot + maxDepth: provider-managed + +- id: tool-subagent-codex-bypass + name: '@deepseek-ai/dsh-tool-subagent' + config: + provider: codex-bypass + toolName: subagent_codex_bypass + backgroundMode: one-shot maxDepth: provider-managed ``` ## 产品兼容性与证据 -生产环境的协议层有意只实现这一单次执行约定所需的 app-server 方法。开发证据锁定在 `@openai/codex@0.147.0` / `codex-cli 0.147.0`;该 NPM 包仅作为测试依赖,部署环境仍需通过 `PATH` 提供 `codex`。 +生产环境的协议层有意只实现这一单次执行约定所需的 app-server 方法。运行时依赖与六个 optional-dependency alias 均锁定到 `@openai/codex@0.147.0` / `codex-cli 0.147.0`。普通安装会按当前操作系统与 CPU 选择一个载荷。对于当前 darwin-arm64 载荷,`npm pack --dry-run --json @openai/codex@0.147.0-darwin-arm64` 报告压缩包为 111,199,052 字节、解包后为 274,777,843 字节。该包包含原生 `codex`、`codex-code-mode-host`、`rg` 与 `zsh` 资源;其他平台可能不同,这些数值只用于披露而不是安装阈值。 + +生成的 schema 证据与包测试会固定全部十六种 error-info variant、HTTP status 所在位置、六个生命周期阶段、进程结果、终止原因映射、unknown 回退、脱敏、权限顺序、取消、并发与清理聚合。无密钥真实产品测试会驱动包内 wrapper 连接回环 Responses fixture,并观测包内 argv、确切的 Bearer 密钥、原始任务、逐字节完全一致的最终回答、线程级 `never` 对环境中 `on-request` 的覆盖、自动评审启动、不产生文件副作用的无人值守拒绝、真实 `internalServerError`、测试拥有临时存储中的显式危险绕过写入、携带安全退出事实的进程/协议失败,以及 wrapper/原生进程完全停稳。同一层级还会证明两个命名实例保留彼此独立的环境与原生模式。 + +如果安装时省略 optional dependencies、当前平台不受支持,或所选载荷缺失,第一次委派会在 `initialize` 阶段以安全 `unknown` 类别和已观测到的进程结果失败。原始 wrapper 文本只保留在 Host stderr;提供方既不会探测宿主 CLI,也不会用它重试。独立 wrapper fixture 会另行证明原生载荷失败与不存在宿主回退。 ## 模型体验 @@ -56,7 +106,7 @@ #### 模型看到的内容 -Codex 子级会在一个全新的临时线程中,以单个轮次接收这些独立文本块。它的工作区是父会话 cwd;其模型、系统指令、工具、沙箱和身份验证来自原生 Codex 安装与配置。 +Codex 子级会在一个全新的临时线程中,以单个轮次接收这些独立文本块。它的工作区是父会话 cwd;其模型、系统指令、工具和身份验证来自原生 Codex 配置,所选提供方实例的 Profile 配置会固定该线程的环境、非交互审批策略与沙箱模式,而可执行版本来自 Bundle 锁定的平台载荷。 #### 对 token 的影响 @@ -66,26 +116,28 @@ Codex 子级会在一个全新的临时线程中,以单个轮次接收这些 这与父请求缓存相互独立。能否复用只取决于 Codex 自身的提供方、模型、指令、工具和临时线程请求。 -### 父级工具结果(间接) +### 父级调度与结果(间接) #### 模型看到的内容 -通过 `dsh-tool-subagent`,父级模型只会看到选定的 Codex 最终答案,或者看到消费方针对未成功完成的结果给出的原样错误。Codex 的过程说明、推理(reasoning)、工具活动、stderr、工作区差异和产品标识符均不会复制到父会话。 +通过 `dsh-tool-subagent`,前台调用会让父级模型看到选定的 Codex 最终答案;若结果未完成,错误中会包含终止原因和可选的安全诊断。该诊断可以区分固定 error-info 类别、协议阶段、数值 HTTP status 和已观测的进程结果,而不复制产品正文。后台调用会先返回 Job id;随后通用作业控制面会送达完成通知,通过 `job_output` 公开同一最终答案或失败状态 detail,并允许 `job_kill` 请求取消。Codex 的过程说明、推理(reasoning)、工具活动、原始 stderr、工作区差异、用量信息、产品标识符、命令、路径和协议载荷均不会复制到父会话。 #### 对 token 的影响 -父级输入只会增加工具结果中保留的最终答案或错误内容。本提供方自身不添加父级工具 schema。 +前台输入会增加工具结果中保留的最终答案或错误内容。后台输入还会包含启动确认、完成通知,以及 `job_output`、`job_kill` 或后续状态结果;子任务 token 仍不会进入父级上下文。本提供方自身不添加父级工具 schema。 #### 对 KV Cache 的影响 -仅追加:新的工具结果接在可复用的父请求前缀之后。 +仅追加:前台会在可复用的父请求前缀后增加一个结果,后台则会继续追加 Job 启动确认、通知以及后续控制或收集结果。后台调度可能增加一个由通知唤醒的轮次,但这些消息都不会改写更早的前缀。 ## 已知限制与后续工作 - **每次运行均新建一个进程、一个线程和一个轮次**:不支持续接、恢复、池化、进度流或产品会话持久化。 -- **产品安装和账户状态由宿主管理**:`codex` 缺失或不兼容、配置错误或身份验证失败,都会呈现为启动错误或运行错误;本插件不提供安装程序、登录流程或运行时版本门禁。 +- **静态选择实例**:Profile 配置项固定提供方名称与工具绑定;调用无法动态选择提供方,而且每个公开工具都需要唯一的 `toolName`。 +- **身份验证与账户状态仍由原生机制管理**:Bundle 会提供 CLI,但不会创建账户、登录、信任项目或改写 Codex 设置;配置与身份验证失败会公开其生命周期阶段与安全的 `unknown` 回退,而不会增加单独的公开分类体系。 +- **委派时必须存在原生平台载荷**:省略 optional dependencies 的安装、不受支持的平台以及缺失或损坏的载荷都会在第一次运行时失败;不会回退到宿主 CLI。 - **兼容性由开发证据锁定**:若要从已验证的 0.147.0 协议基线升级,必须重新生成上游 schema 证据,并重新运行握手、答案选择、审批、取消、无密钥真实产品以及带密钥的 DeepSeek 随机数测试。 -- **没有人工审批路径**:已知的无人值守审批请求会被拒绝,未知服务器请求会以默认拒绝方式使运行失败;部署方无法通过本包配置允许策略。 -- **仅返回最终文本**:推理、过程说明、中间消息、工具通信、用量信息、stderr 和工作区差异仍只保留在产品内部。 +- **没有人工审批路径**:已知的无人值守审批请求会被拒绝,未知服务器请求会以默认拒绝方式使运行失败;三种 Profile 模式都不会创建 DSH 交互通道或逐次调用 allow 策略。 +- **assistant 载荷仅包含最终文本**:失败运行可以额外公开独立的安全诊断;推理、过程说明、中间消息、工具通信、用量信息、原始 stderr 和工作区差异不会进入父会话,通用 Job id、通知与状态来自共享作业运行时。 - **没有可选的共享能力**:对于本提供方,共享服务会拒绝输出 schema、子任务角色设定、工具筛选和 harness 深度强制约束。 - **没有按实际经过时间触发的超时或副作用回滚**:长时间运行的工作由调用方取消,且取消前已更改的文件或外部系统不会恢复原状。 diff --git a/packages/subagent/subagent-codex/cordis.patch.yml b/packages/subagent/subagent-codex/cordis.patch.yml new file mode 100644 index 0000000000..fb64fdcb9d --- /dev/null +++ b/packages/subagent/subagent-codex/cordis.patch.yml @@ -0,0 +1,5 @@ +# Optional Profile Bundle: register the Codex provider on the Host plane only. +# Agent Presets grant the model-facing tool independently. +- insert: + - id: subagent-codex + name: '@deepseek-ai/dsh-subagent-codex' diff --git a/packages/subagent/subagent-codex/package.json b/packages/subagent/subagent-codex/package.json index 269bf2e996..06cb026dfd 100644 --- a/packages/subagent/subagent-codex/package.json +++ b/packages/subagent/subagent-codex/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subagent-codex", "description": "One-shot Codex subagent provider over the official app-server protocol", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -28,13 +28,18 @@ "files": [ "lib/index.js", "lib/invariant.js", + "cordis.patch.yml", "lib/types/**/*.d.ts" ], "license": "MIT", + "dsh": { + "bundle": { + "patch": "./cordis.patch.yml" + } + }, "peerDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", - "@deepseek-ai/dsh-sdk-protocol": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-subagent": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", @@ -42,7 +47,9 @@ "@deepseek-ai/cordis": "workspace:^" }, "dependencies": { - "@deepseek-ai/schemastery": "workspace:^" + "@deepseek-ai/dsh-sdk-protocol": "workspace:^", + "@deepseek-ai/schemastery": "workspace:^", + "@openai/codex": "0.147.0" }, "devDependencies": { "@deepseek-ai/cordis-plugin-loader": "workspace:^", @@ -50,13 +57,11 @@ "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-loader-smoke": "workspace:^", - "@deepseek-ai/dsh-sdk-protocol": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-subagent": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", - "@openai/codex": "0.147.0", "@deepseek-ai/cordis": "workspace:^" } } diff --git a/packages/subagent/subagent-codex/src/index.ts b/packages/subagent/subagent-codex/src/index.ts index 3b1bbec799..79fbdab078 100644 --- a/packages/subagent/subagent-codex/src/index.ts +++ b/packages/subagent/subagent-codex/src/index.ts @@ -1,7 +1,7 @@ /** - * Fixed Codex one-shot subagent provider. Every accepted run starts a fresh - * official `codex app-server --stdio` process in the delegating Session's - * workspace and publishes only after an ephemeral thread exists. + * Profile-named Codex one-shot subagent provider. Every accepted run starts a + * fresh official package-local Codex wrapper with `app-server --stdio` in the + * delegating Session's workspace and publishes only after an ephemeral thread exists. * * @module @deepseek-ai/dsh-subagent-codex */ @@ -18,38 +18,51 @@ import { type SubagentProvider, } from '@deepseek-ai/dsh-subagent' import { + CODEX_PERMISSION_MODES, + DEFAULT_CODEX_PERMISSION_MODE, DEFAULT_DISPOSE_GRACE_MS, + codexStartupFailure, startCodexRun, + type CodexPermissionMode, type CodexRunSpec, } from './run.ts' export const name = 'subagent-codex' export const inject = ['subagents', 'subprocess'] -/** Deployment-owned environment and process-release bound. */ +const DEFAULT_PROVIDER_NAME = 'codex' + +/** Deployment-owned permission, environment, and process-release settings. */ export interface Config { + /** Provider name on `ctx.subagents` (default `codex`). */ + providerName?: string /** * Explicit environment entries layered over the subprocess seam's * credential-scrubbed parent environment. */ env?: Record + /** Native non-interactive permission mode fixed for this Provider instance. */ + permissionMode?: CodexPermissionMode /** Grace in milliseconds for app-server process-tree termination. */ disposeGraceMs?: number } export const Config: z = z.object({ + providerName: z.string().min(1).default(DEFAULT_PROVIDER_NAME), env: z.dict(z.string()).default({}), + permissionMode: z.union([...CODEX_PERMISSION_MODES]) + .default(DEFAULT_CODEX_PERMISSION_MODE), disposeGraceMs: z.number().default(DEFAULT_DISPOSE_GRACE_MS), }) type ResolvedConfig = Required class CodexProvider implements SubagentProvider { - readonly name = 'codex' readonly capabilities: SubagentCapabilities = NO_START_CAPABILITIES readonly inheritsParentContext = false constructor( + readonly name: string, private readonly ctx: Context, private readonly config: ResolvedConfig, ) {} @@ -61,18 +74,30 @@ class CodexProvider implements SubagentProvider { 'subagent-codex: no working directory for the child — delegate from a parent session that has one', ) } - const spec: CodexRunSpec = { - cwd: resolveChildCwd( + let cwd: string + try { + cwd = resolveChildCwd( 'subagent-codex', undefined, parentCwd, - ), + ) + } catch (error: unknown) { + if (request.signal.aborted) { + throw new Error( + 'subagent-codex: request was aborted before app-server startup', + ) + } + throw codexStartupFailure(error) + } + const spec: CodexRunSpec = { + cwd, + permissionMode: this.config.permissionMode, env: this.config.env, disposeGraceMs: this.config.disposeGraceMs, spawn: spawnSpec => this.ctx.subprocess.spawn(spawnSpec), onError: (error, stopReason) => { this.ctx.logger.warn( - `subagent-codex: child run failed (${stopReason}): ${error.message}`, + `subagent-codex "${this.name}": child run failed (${stopReason}): ${error.message}`, ) }, } @@ -81,12 +106,17 @@ class CodexProvider implements SubagentProvider { } /** - * Register the fixed `codex` provider. + * Register one Profile-named Codex provider. * @param ctx - context carrying shared subagent and subprocess services. - * @param config - explicit child environment and disposal grace. + * @param config - registry name, permission mode, child environment, and disposal grace. */ export function apply(ctx: Context, config: Config): void { - const resolved = config as ResolvedConfig + const resolved: ResolvedConfig = { + providerName: config.providerName ?? DEFAULT_PROVIDER_NAME, + env: config.env as Record, + permissionMode: config.permissionMode ?? DEFAULT_CODEX_PERMISSION_MODE, + disposeGraceMs: config.disposeGraceMs as number, + } assertPositiveFinite( 'subagent-codex', 'disposeGraceMs', @@ -97,5 +127,9 @@ export function apply(ctx: Context, config: Config): void { `subagent-codex: disposeGraceMs must be no greater than ${MAX_TIMER_DELAY_MS}`, ) } - ctx.subagents.registerProvider(new CodexProvider(ctx, resolved)) + ctx.subagents.registerProvider(new CodexProvider( + resolved.providerName, + ctx, + resolved, + )) } diff --git a/packages/subagent/subagent-codex/src/run.ts b/packages/subagent/subagent-codex/src/run.ts index ebce244f3b..9db5d7086c 100644 --- a/packages/subagent/subagent-codex/src/run.ts +++ b/packages/subagent/subagent-codex/src/run.ts @@ -8,6 +8,9 @@ */ import { randomUUID } from 'node:crypto' +import { readFileSync, writeFileSync } from 'node:fs' +import { createRequire } from 'node:module' +import { dirname, resolve } from 'node:path' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import { SessionId } from '@deepseek-ai/dsh-session' import { @@ -18,33 +21,124 @@ import { type SubagentStartRequest, type SubagentStopReason, } from '@deepseek-ai/dsh-subagent' -import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' -import { CodexAppServerWire } from './wire.ts' +import type { + SubprocessHandle, + SubprocessOutcome, + SubprocessSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import { + CodexAppServerWire, + type CodexWireFailureFacts, +} from './wire.ts' /** Default POSIX grace between subprocess termination tiers. */ export const DEFAULT_DISPOSE_GRACE_MS = 3_000 +interface CodexPackageManifest { + readonly bin: { + readonly codex: string + } +} + +const codexPackageJsonPath = createRequire(import.meta.url).resolve('@openai/codex/package.json') +const codexPackageManifest = JSON.parse( + readFileSync(codexPackageJsonPath, 'utf8'), +) as CodexPackageManifest + +/** Absolute package-local JavaScript wrapper selected by the package manifest. */ +const CODEX_PACKAGE_BIN = resolve( + dirname(codexPackageJsonPath), + codexPackageManifest.bin.codex, +) + +/** Profile-selectable non-interactive Codex permission mode. */ +export type CodexPermissionMode = + | 'never' + | 'approve-for-me' + | 'dangerously-bypass-approvals-and-sandbox' + +/** Native non-interactive Codex modes mapped to official `thread/start` fields. */ +export const CODEX_PERMISSION_MODES = [ + 'never', + 'approve-for-me', + 'dangerously-bypass-approvals-and-sandbox', +] as const satisfies readonly CodexPermissionMode[] + +/** Safe default for unattended Codex runs. */ +export const DEFAULT_CODEX_PERMISSION_MODE: CodexPermissionMode = 'never' + +type CodexFailureStage = + | 'initialize' + | 'thread-start' + | CodexWireFailureFacts['stage'] + | 'process' + | 'teardown' + +interface CodexFailureFacts { + readonly stage: CodexFailureStage + readonly category: string + readonly httpStatus?: number | undefined + readonly outcome?: SubprocessOutcome | undefined +} + +function failureDiagnostic(facts: CodexFailureFacts): string { + const fields = [ + 'product: Codex', + `stage: ${facts.stage}`, + `category: ${facts.category}`, + ] + if (facts.httpStatus !== undefined) { + fields.push(`HTTP status: ${facts.httpStatus}`) + } + const processFields = [ + ['exit code', facts.outcome?.exitCode], + ['signal', facts.outcome?.signal], + ] as const + for (const [label, value] of processFields) { + if (value !== null && value !== undefined) fields.push(`${label}: ${value}`) + } + return `Product subagent failure (${fields.join('; ')})` +} + +class CodexRunFailure extends Error { + constructor( + readonly facts: CodexFailureFacts, + cause?: unknown, + ) { + super( + `subagent-codex: ${failureDiagnostic(facts)}`, + cause === undefined ? undefined : { cause }, + ) + this.name = 'CodexRunFailure' + } +} + /** - * Resolve the fixed app-server command for a platform. - * - * Windows npm and pnpm installs expose `codex.cmd`, which requires `cmd.exe`; - * the argv is constant so no task or configuration text enters the - * shell boundary. - * @param platform - host platform used to select the executable boundary. - * @returns argv for the fixed Codex app-server command. + * Hide an unpublished Host failure behind fixed safe startup facts. + * @param cause Original Host failure retained for internal diagnostics. + * @returns A startup failure whose message contains only fixed safe facts. */ -export function codexAppServerArgv( - platform: NodeJS.Platform = process.platform, -): string[] { - return platform === 'win32' - ? ['cmd.exe', '/d', '/s', '/c', 'codex', 'app-server', '--stdio'] - : ['codex', 'app-server', '--stdio'] +export function codexStartupFailure(cause: unknown): Error { + return new CodexRunFailure({ + stage: 'initialize', + category: 'unknown', + }, cause) +} + +/** + * Fixed package-local app-server command, independent of the host `PATH`. + * @returns Node, the official wrapper, and the fixed app-server arguments. + */ +export function codexAppServerArgv(): string[] { + return [process.execPath, CODEX_PACKAGE_BIN, 'app-server', '--stdio'] } /** Fully resolved inputs for one Codex app-server run. */ export interface CodexRunSpec { /** Parent Session workspace, also supplied to `thread/start`. */ readonly cwd: string + /** Profile-selected native non-interactive permission mode. */ + readonly permissionMode: CodexPermissionMode /** Explicit deployment/test environment layered after the shared scrub. */ readonly env: Record /** Subprocess termination grace passed to the shared process-tree owner. */ @@ -93,18 +187,33 @@ export async function disposeCodexChild( child: SubprocessHandle, ): Promise { wire.close() - if (child.pid <= 0) { + + if (child.pid > 0) { + let outcome: SubprocessOutcome | undefined + void child.done.then( + (value) => { outcome = value }, + /* v8 ignore next -- a positive pid excludes spawn-level done rejection. */ + () => {}, + ) + try { + child.stdin?.end() + } catch { + // A concurrently closed stdin does not change tree ownership below. + } + child.terminate() + try { + await child.waitForExit() + } catch (error: unknown) { + throw new CodexRunFailure({ + stage: 'teardown', + category: 'unknown', + outcome, + }, thrown(error)) + } + await child.done + } else { await child.done.catch(() => {}) - return } - try { - child.stdin?.end() - } catch { - // A concurrently closed stdin does not change tree ownership below. - } - child.terminate() - await child.waitForExit() - await child.done } /** @@ -122,29 +231,76 @@ export async function startCodexRun( throw new Error('subagent-codex: request was aborted before app-server startup') } - const child = spec.spawn({ - argv: codexAppServerArgv(), - cwd: spec.cwd, - stdio: { stdin: 'pipe', stdout: 'pipe', stderr: 'inherit' }, - graceMs: spec.disposeGraceMs, - env: spec.env, - }) + let child: SubprocessHandle + try { + child = spec.spawn({ + argv: codexAppServerArgv(), + cwd: spec.cwd, + stdio: { stdin: 'pipe', stdout: 'pipe', stderr: 'pipe' }, + graceMs: spec.disposeGraceMs, + env: spec.env, + }) + } catch (error: unknown) { + throw new CodexRunFailure({ + stage: 'initialize', + category: 'unknown', + }, thrown(error)) + } const wire = new CodexAppServerWire( child.stdout as NonNullable, child.stdin as NonNullable, + spec.permissionMode, ) - const disposeProcess = (): Promise => disposeCodexChild(wire, child) + const onStderr = (chunk: Buffer | string): void => { + const bytes = typeof chunk === 'string' ? Buffer.from(chunk) : chunk + wire.observeStderr(bytes.toString()) + try { + // Synchronous fd forwarding preserves byte order without owning a + // backpressure queue. A slow host sink can block this event-loop turn. + writeFileSync(process.stderr.fd, bytes) + } catch { + // Host stderr is an observation sink, not a child-run failure authority. + } + } + const onStderrError = (): void => { + // Stderr observation is auxiliary. JSON-RPC and child.done remain the + // only terminal authorities if the diagnostic stream itself fails. + } + child.stderr?.on('data', onStderr) + child.stderr?.on('error', onStderrError) + const disposeProcess = async (): Promise => { + try { + await disposeCodexChild(wire, child) + // Let stderr already queued by the process close reach both bounded + // diagnostic consumers before their listeners are detached. + await new Promise((resolve) => { setImmediate(resolve) }) + } finally { + child.stderr?.off('data', onStderr) + child.stderr?.off('error', onStderrError) + } + } - const processFailure: Promise = child.done.then( - outcome => Promise.reject(new Error( - 'subagent-codex: app-server exited before the run settled ' - + `(code ${String(outcome.exitCode)}, signal ${String(outcome.signal)})`, - )), - (error: unknown) => Promise.reject(thrown(error)), + let processFailureFacts: CodexFailureFacts | undefined + const processFailure: Promise = child.done.then( + (outcome) => { + processFailureFacts = { + stage: 'process', + category: 'process-exit', + outcome, + } + throw new CodexRunFailure(processFailureFacts) + }, + (error: unknown) => { + processFailureFacts = { + stage: 'process', + category: 'unknown', + } + throw new CodexRunFailure(processFailureFacts, thrown(error)) + }, ) - // A normal post-result dispose also closes the process. Keep that expected - // late rejection observed after the result race has already settled. + // A normal post-result dispose also closes the process. Keep its expected + // late rejection observed when the terminal result settles first. processFailure.catch(() => {}) const runAbort = new AbortController() @@ -156,33 +312,116 @@ export async function startCodexRun( const onAbort = (): void => { requestCancel() } request.signal.addEventListener('abort', onAbort, { once: true }) + let startupStage: 'initialize' | 'thread-start' = 'initialize' try { wire.start() await Promise.race([wire.initialize(request.signal), processFailure]) + startupStage = 'thread-start' await Promise.race([wire.startThread(spec.cwd, request.signal), processFailure]) } catch (error: unknown) { request.signal.removeEventListener('abort', onAbort) + const cancelledBeforeCleanup = runAbort.signal.aborted + if (!(error instanceof CodexRunFailure) && !cancelledBeforeCleanup) { + // Node reports stdout EOF before the child close that owns its outcome. + // Let an already-exiting process publish those facts before rollback. + await new Promise((resolve) => { setImmediate(resolve) }) + } + const failure = new CodexRunFailure({ + stage: startupStage, + category: 'unknown', + outcome: error instanceof CodexRunFailure + ? error.facts.outcome + : processFailureFacts?.outcome, + }, thrown(error)) try { await disposeProcess() } catch (disposeError: unknown) { + const cleanupFailure = thrown(disposeError) throw new AggregateError( - [thrown(error), thrown(disposeError)], - 'subagent-codex: startup failed and app-server cleanup also failed', + [failure, cleanupFailure], + `${failure.message}; ${cleanupFailure.message}`, ) } - if (runAbort.signal.aborted) { + if (cancelledBeforeCleanup) { throw new Error('subagent-codex: request was aborted before run publication') } - throw thrown(error) + try { + request.signal.throwIfAborted() + } catch { + throw new Error('subagent-codex: request was aborted before run publication') + } + throw failure } const collectOutput = (): ContentBlock[] => wire.collectOutput() + let diagnostic: string | undefined + const recordFailureDiagnostic = (facts: CodexFailureFacts): string => { + const failure = failureDiagnostic(facts) + const permission = wire.collectDiagnostic() + diagnostic = permission === undefined + ? failure + : `${failure}\n${permission}` + return diagnostic + } + const withProcessOutcome = (facts: CodexFailureFacts): CodexFailureFacts => { + const outcome = processFailureFacts?.outcome + return outcome === undefined + ? facts + : { ...facts, outcome } + } + const publishedProcessFailure = processFailure.catch( + async (error: unknown): Promise => { + // Frames already queued by the exiting app-server remain authoritative. + // One I/O turn lets them settle before process exit ends the run. + await new Promise((resolve) => { setImmediate(resolve) }) + throw error + }, + ) const result: Promise = settleRunResult({ - attempt: () => Promise.race([ - wire.runTurn(texts, runAbort.signal), - processFailure, - ]), + attempt: async () => { + try { + const terminal = await Promise.race([ + wire.runTurn(texts, runAbort.signal), + publishedProcessFailure, + ]) + if (terminal.stopReason === 'completed') return terminal + // Let stderr already queued with the terminal frame contribute its + // fixed permission fact before the non-completed result is snapshotted. + await new Promise((resolve) => { setImmediate(resolve) }) + const facts = withProcessOutcome(wire.collectFailure()) + return { ...terminal, diagnostic: recordFailureDiagnostic(facts) } + } catch (error: unknown) { + // Give stderr data already queued in Node one turn to reach the wire + // before settlement snapshots the diagnostic. + await new Promise((resolve) => { setImmediate(resolve) }) + const endedBeforeTerminal = wire.endedBeforeTerminal() + if ( + endedBeforeTerminal + && processFailureFacts === undefined + && !runAbort.signal.aborted + ) { + try { + const exited = await child.waitForExit( + AbortSignal.timeout(Math.ceil(spec.disposeGraceMs)), + ) + if (exited) await child.done + } catch { + // The wire failure remains authoritative when exit observation fails. + } + } + const facts = error instanceof CodexRunFailure + ? error.facts + : endedBeforeTerminal && processFailureFacts !== undefined + ? processFailureFacts + : withProcessOutcome(wire.collectFailure()) + recordFailureDiagnostic(facts) + throw error instanceof CodexRunFailure + ? error + : new CodexRunFailure(facts, thrown(error)) + } + }, collectOutput, + collectDiagnostic: () => diagnostic, cancelled: () => runAbort.signal.aborted, onError: spec.onError, signal: request.signal, diff --git a/packages/subagent/subagent-codex/src/wire.ts b/packages/subagent/subagent-codex/src/wire.ts index bc00ff0acf..cbc0504e08 100644 --- a/packages/subagent/subagent-codex/src/wire.ts +++ b/packages/subagent/subagent-codex/src/wire.ts @@ -11,9 +11,64 @@ import type { Readable, Writable } from 'node:stream' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { SubagentResult } from '@deepseek-ai/dsh-subagent' import { JsonRpcLineTransport } from '@deepseek-ai/dsh-sdk-protocol' +import type { CodexPermissionMode } from './run.ts' type JsonObject = Record +/** Product facts owned by the Codex wire after publication. */ +export interface CodexWireFailureFacts { + readonly stage: 'turn-start' | 'turn' + readonly category: string + readonly httpStatus?: number | undefined +} + +const THREAD_PERMISSION_PARAMS: Readonly> = { + never: { approvalPolicy: 'never' }, + 'approve-for-me': { + approvalPolicy: 'on-request', + approvalsReviewer: 'auto_review', + sandbox: 'workspace-write', + }, + 'dangerously-bypass-approvals-and-sandbox': { + approvalPolicy: 'never', + sandbox: 'danger-full-access', + }, +} + +const STDERR_PERMISSION_SIGNATURES = [ + { + text: 'approval policy is Never; reject command', + request: 'command execution', + decision: 'denied', + reason: 'Codex rejected an escalation because the selected policy never asks for approval', + }, + { + text: 'recorded sandbox violation:', + request: 'sandbox execution', + decision: 'failed', + reason: 'Codex reported a sandbox violation', + }, +] as const + +const STDERR_SIGNATURE_TAIL_CHARS = Math.max( + ...STDERR_PERMISSION_SIGNATURES.map(signature => signature.text.length), +) - 1 + +function stderrSignatureTail(value: string): string { + for ( + let length = Math.min(STDERR_SIGNATURE_TAIL_CHARS, value.length) + ; length > 0 + ; length -= 1 + ) { + const tail = value.slice(-length) + if (STDERR_PERMISSION_SIGNATURES.some(signature => + tail.length < signature.text.length && signature.text.startsWith(tail))) { + return tail + } + } + return '' +} + function object(value: unknown, label: string): JsonObject { if (value === null || typeof value !== 'object' || Array.isArray(value)) { throw new Error(`subagent-codex: app-server returned invalid ${label}`) @@ -38,13 +93,87 @@ function unattendedDecision(params: JsonObject): 'cancel' | 'decline' { throw new Error('subagent-codex: app-server offered no unattended approval decision') } -function isContextWindowExceeded(turn: JsonObject): boolean { - if (turn.status !== 'failed') return false +function numericHttpStatus(value: unknown): number | undefined { + return typeof value === 'number' + && Number.isInteger(value) + && value >= 0 + && value <= 65_535 + ? value + : undefined +} + +function objectFailureInfo(value: JsonObject): { + readonly category: string + readonly httpStatus?: number | undefined +} { + const keys = Object.keys(value) + const category = keys[0] + if (keys.length !== 1 || category === undefined) { + return { category: 'unknown' } + } + const detail = value[category] + if (detail === null || typeof detail !== 'object' || Array.isArray(detail)) { + return { category: 'unknown' } + } + const fields = detail as JsonObject + switch (category) { + case 'httpConnectionFailed': + case 'responseStreamConnectionFailed': + case 'responseStreamDisconnected': + case 'responseTooManyFailedAttempts': + { + const httpStatus = numericHttpStatus(fields.httpStatusCode) + return httpStatus === undefined + ? { category } + : { category, httpStatus } + } + case 'activeTurnNotSteerable': + return { category } + default: + return { category: 'unknown' } + } +} + +function failureInfo(turn: JsonObject): { + readonly category: string + readonly httpStatus?: number | undefined +} { + if (turn.status !== 'failed') return { category: 'unknown' } const error = turn.error - return error !== null - && typeof error === 'object' - && !Array.isArray(error) - && (error as JsonObject).codexErrorInfo === 'contextWindowExceeded' + if (error === null || typeof error !== 'object' || Array.isArray(error)) { + return { category: 'unknown' } + } + const info = (error as JsonObject).codexErrorInfo + if (typeof info === 'string') { + switch (info) { + case 'contextWindowExceeded': + case 'sessionBudgetExceeded': + case 'usageLimitExceeded': + case 'serverOverloaded': + case 'cyberPolicy': + case 'internalServerError': + case 'unauthorized': + case 'badRequest': + case 'threadRollbackFailed': + case 'sandboxError': + case 'other': + return { category: info } + default: + return { category: 'unknown' } + } + } + return info !== null && typeof info === 'object' && !Array.isArray(info) + ? objectFailureInfo(info as JsonObject) + : { category: 'unknown' } +} + +function unattendedDiagnostic( + mode: CodexPermissionMode, + request: 'command approval' | 'file approval' | 'permission grant' | 'user input' | 'MCP elicitation' | 'command execution' | 'file change' | 'sandbox execution', + decision: 'cancelled' | 'declined' | 'denied' | 'empty response' | 'failed', + reason: string, +): string { + return `Codex unattended decision (mode: ${mode}; request: ${request}; decision: ${decision}): ${reason}` } function thrown(value: unknown): Error { @@ -86,18 +215,36 @@ export class CodexAppServerWire { private threadId: string | undefined private turnId: string | undefined private pendingTurnId: string | undefined - private turnCompleted: PromiseWithResolvers | undefined + private turnCompleted: PromiseWithResolvers<{ + readonly params: JsonObject + readonly order: number + }> | undefined private readonly earlyTurnNotifications: Array<{ readonly method: string readonly params: JsonObject + readonly order: number }> = [] private lastFinalAnswer: string | undefined private lastUnphasedAnswer: string | undefined + private diagnostic: string | undefined + private failure: CodexWireFailureFacts | undefined + private diagnosticOrder = 0 + private observationOrder = 0 + private pendingDiagnostic: { + readonly order: number + readonly request: Parameters[1] + readonly decision: Parameters[2] + readonly reason: string + } | undefined + private stderrTail = '' + private inputEnded = false + private terminalObserved = false private closed = false constructor( private readonly input: Readable, output: Writable, + private readonly permissionMode: CodexPermissionMode, ) { this.transport = new JsonRpcLineTransport(input, output) // Fatal protocol state can arrive after the current guarded operation has @@ -125,6 +272,14 @@ export class CodexAppServerWire { this.transport.start() } + /** + * Whether protocol output ended before a terminal turn notification. + * @returns `true` only for an early protocol close without a terminal turn. + */ + endedBeforeTerminal(): boolean { + return this.inputEnded && !this.terminalObserved + } + /** * Perform the required app-server initialize/initialized handshake. * @param signal - unpublished-start cancellation. @@ -154,6 +309,7 @@ export class CodexAppServerWire { const response = object(await this.guarded(this.transport.request('thread/start', { cwd, ephemeral: true, + ...THREAD_PERMISSION_PARAMS[this.permissionMode], }, signal), signal), 'thread/start response') const thread = object(response.thread, 'thread/start thread') const id = string(thread.id, 'thread/start thread id') @@ -174,30 +330,63 @@ export class CodexAppServerWire { texts: readonly string[], signal: AbortSignal, ): Promise { - const completion = Promise.withResolvers() + const completion = Promise.withResolvers<{ + readonly params: JsonObject + readonly order: number + }>() this.turnCompleted = completion const threadId = this.threadId as string - const response = object(await this.guarded(this.transport.request('turn/start', { - threadId, - input: texts.map(text => ({ type: 'text', text, text_elements: [] })), - }, signal), signal), 'turn/start response') - const turn = object(response.turn, 'turn/start turn') - this.commitTurnId(string(turn.id, 'turn/start turn id')) - - const completed = await this.guarded(completion.promise, signal) - const terminal = object(completed.turn, 'turn/completed turn') - const status = terminal.status - if (isContextWindowExceeded(terminal)) { - return { output: this.collectOutput(), stopReason: 'max-tokens' } + try { + const response = object(await this.guarded(this.transport.request('turn/start', { + threadId, + input: texts.map(text => ({ type: 'text', text, text_elements: [] })), + }, signal), signal), 'turn/start response') + const turn = object(response.turn, 'turn/start turn') + this.commitTurnId(string(turn.id, 'turn/start turn id')) + } catch (error: unknown) { + this.recordFailure({ stage: 'turn-start', category: 'unknown' }) + throw error } + + let completed: { + readonly params: JsonObject + readonly order: number + } + let terminal: JsonObject + try { + completed = await this.guarded(completion.promise, signal) + terminal = object(completed.params.turn, 'turn/completed turn') + } catch (error: unknown) { + this.recordFailure({ stage: 'turn', category: 'unknown' }) + throw error + } + const status = terminal.status if (status !== 'completed') { - const detail = status === 'failed' - ? `: ${JSON.stringify(terminal.error)}` - : '' + const parsed = failureInfo(terminal) + this.recordFailure(parsed.httpStatus === undefined + ? { stage: 'turn', category: parsed.category } + : { + stage: 'turn', + category: parsed.category, + httpStatus: parsed.httpStatus, + }) + if (parsed.category === 'sandboxError') { + this.recordDiagnostic( + 'sandbox execution', + 'failed', + 'Codex reported a sandbox failure', + completed.order, + ) + } + if (parsed.category === 'contextWindowExceeded') { + return { output: this.collectOutput(), stopReason: 'max-tokens' } + } + const detail = status === 'failed' ? `: ${parsed.category}` : '' throw new Error(`subagent-codex: Codex turn ended with status ${String(status)}${detail}`) } const output = this.collectOutput() if (output.length === 0) { + this.recordFailure({ stage: 'turn', category: 'unknown' }) throw new Error('subagent-codex: Codex completed without a final answer') } return { output, stopReason: 'completed' } @@ -226,6 +415,45 @@ export class CodexAppServerWire { : [] } + /** + * The latest safe unattended permission fact observed for this run. + * @returns provider-authored diagnostic text, when one was observed. + */ + collectDiagnostic(): string | undefined { + return this.diagnostic + } + + /** + * The structured failure fact observed for this published turn. + * Call only after a non-completed return or rejection from {@link runTurn}. + * @returns the fixed stage/category pair and optional HTTP status. + */ + collectFailure(): CodexWireFailureFacts { + return this.failure as CodexWireFailureFacts + } + + /** + * Observe product stderr while retaining only enough tail to recognize fixed + * permission signatures. The raw text is never copied into the diagnostic. + * @param chunk - one decoded stderr chunk already forwarded to the host. + */ + observeStderr(chunk: string): void { + const observed = `${this.stderrTail}${chunk}` + let latestIndex = -1 + let latest: (typeof STDERR_PERMISSION_SIGNATURES)[number] | undefined + for (const signature of STDERR_PERMISSION_SIGNATURES) { + const index = observed.lastIndexOf(signature.text) + if (index > latestIndex) { + latestIndex = index + latest = signature + } + } + if (latest !== undefined) { + this.recordDiagnostic(latest.request, latest.decision, latest.reason) + } + this.stderrTail = stderrSignatureTail(observed) + } + /** Detach JSON-RPC listeners and reject outstanding requests. Idempotent. */ close(): void { if (this.closed) return @@ -252,6 +480,7 @@ export class CodexAppServerWire { } private readonly onInputEnd = (): void => { + this.inputEnded = true this.fail(new Error('subagent-codex: app-server protocol stream closed')) } @@ -270,42 +499,166 @@ export class CodexAppServerWire { throw new Error('subagent-codex: turn/start response did not match the active turn') } this.turnId = id + const pendingDiagnostic = this.pendingDiagnostic + this.pendingDiagnostic = undefined + if (pendingDiagnostic !== undefined) { + this.recordDiagnostic( + pendingDiagnostic.request, + pendingDiagnostic.decision, + pendingDiagnostic.reason, + pendingDiagnostic.order, + ) + } const notifications = this.earlyTurnNotifications.splice(0) for (const notification of notifications) { - this.handleNotification(notification.method, notification.params) + this.handleNotification( + notification.method, + notification.params, + notification.order, + ) } } - private validateRunIds(params: JsonObject, nullableTurn = false): void { + /** + * Validate the request's thread and turn association. + * @returns `true` when the matching turn is still provisional, so the caller + * defers its diagnostic until `commitTurnId()`. + */ + private validateRunIds( + params: JsonObject, + nullableTurn = false, + ): boolean { if (params.threadId !== this.threadId) { throw new Error('subagent-codex: app-server request referenced another thread') } - if (nullableTurn && params.turnId === null) return + if (nullableTurn && params.turnId === null) return false const id = string(params.turnId, 'server request turn id') if (this.turnId === undefined) { this.observePendingTurnId(id) - return + return true } if (id !== this.turnId) { throw new Error('subagent-codex: app-server request referenced another turn') } + return false + } + + private recordRequestDiagnostic( + provisional: boolean, + request: Parameters[1], + decision: Parameters[2], + reason: string, + ): void { + const order = this.nextObservationOrder() + if (provisional) { + this.pendingDiagnostic = { + order, + request, + decision, + reason, + } + return + } + this.recordDiagnostic(request, decision, reason, order) + } + + private recordDiagnostic( + request: Parameters[1], + decision: Parameters[2], + reason: string, + order = this.nextObservationOrder(), + ): void { + if (order < this.diagnosticOrder) return + this.diagnosticOrder = order + this.diagnostic = unattendedDiagnostic( + this.permissionMode, + request, + decision, + reason, + ) + } + + private recordFailure(facts: CodexWireFailureFacts): void { + this.failure = facts + } + + private nextObservationOrder(): number { + this.observationOrder += 1 + return this.observationOrder + } + + private recordDeclinedItem(item: JsonObject, order?: number): boolean { + if (item.type === 'commandExecution' && item.status === 'declined') { + this.recordDiagnostic( + 'command execution', + 'declined', + 'Codex declined the command under the selected permission mode', + order, + ) + return true + } + if (item.type === 'fileChange' && item.status === 'declined') { + this.recordDiagnostic( + 'file change', + 'declined', + 'Codex declined the file change under the selected permission mode', + order, + ) + return true + } + return false } private handleServerRequest(method: string, params: JsonObject): Promise { try { switch (method) { case 'item/commandExecution/requestApproval': + { + const provisional = this.validateRunIds(params) + const decision = unattendedDecision(params) + this.recordRequestDiagnostic( + provisional, + 'command approval', + decision === 'cancel' ? 'cancelled' : 'declined', + 'the provider does not grant interactive approval', + ) + return Promise.resolve({ decision }) + } case 'item/fileChange/requestApproval': - this.validateRunIds(params) - return Promise.resolve({ decision: unattendedDecision(params) }) + { + const provisional = this.validateRunIds(params) + const decision = unattendedDecision(params) + this.recordRequestDiagnostic( + provisional, + 'file approval', + decision === 'cancel' ? 'cancelled' : 'declined', + 'the provider does not grant interactive approval', + ) + return Promise.resolve({ decision }) + } case 'item/permissions/requestApproval': - this.validateRunIds(params) + this.recordRequestDiagnostic( + this.validateRunIds(params), + 'permission grant', + 'denied', + 'the provider grants no additional turn permissions', + ) return Promise.resolve({ permissions: {}, scope: 'turn' }) case 'item/tool/requestUserInput': - this.validateRunIds(params) + this.recordRequestDiagnostic( + this.validateRunIds(params), + 'user input', + 'empty response', + 'the provider does not collect interactive answers', + ) return Promise.resolve({ answers: {} }) case 'mcpServer/elicitation/request': - this.validateRunIds(params, true) + this.recordRequestDiagnostic( + this.validateRunIds(params, true), + 'MCP elicitation', + 'declined', + 'the provider does not collect interactive MCP input', + ) return Promise.resolve({ action: 'decline', content: null, _meta: null }) default: throw new Error(`subagent-codex: unsupported app-server request ${JSON.stringify(method)}`) @@ -317,7 +670,11 @@ export class CodexAppServerWire { } } - private handleNotification(method: string, params: JsonObject): void { + private handleNotification( + method: string, + params: JsonObject, + order?: number, + ): void { if (method === 'turn/started') { const threadId = string(params.threadId, 'turn/started thread id') if (threadId !== this.threadId) return @@ -334,12 +691,17 @@ export class CodexAppServerWire { if (this.turnId === undefined) { if (this.turnCompleted !== undefined) { this.observePendingTurnId(id) - this.earlyTurnNotifications.push({ method, params }) + this.earlyTurnNotifications.push({ + method, + params, + order: this.nextObservationOrder(), + }) } return } if (id !== this.turnId) return const item = object(params.item, 'item/completed item') + if (this.recordDeclinedItem(item, order)) return if (item.type !== 'agentMessage') return const text = typeof item.text === 'string' ? item.text @@ -362,13 +724,21 @@ export class CodexAppServerWire { if (turnCompleted === undefined) return if (this.turnId === undefined) { this.observePendingTurnId(id) - this.earlyTurnNotifications.push({ method, params }) + this.earlyTurnNotifications.push({ + method, + params, + order: this.nextObservationOrder(), + }) return } if (id !== this.turnId) return + this.terminalObserved = true if (!['completed', 'interrupted', 'failed'].includes(String(turn.status))) { throw new Error(`subagent-codex: app-server returned invalid terminal turn status ${String(turn.status)}`) } - turnCompleted.resolve(params) + turnCompleted.resolve({ + params, + order: order ?? this.nextObservationOrder(), + }) } } diff --git a/packages/subagent/subagent-codex/tests/loader-composition.e2e.ts b/packages/subagent/subagent-codex/tests/loader-composition.e2e.ts index 6c4019f8c8..b55b819989 100644 --- a/packages/subagent/subagent-codex/tests/loader-composition.e2e.ts +++ b/packages/subagent/subagent-codex/tests/loader-composition.e2e.ts @@ -1,3 +1,4 @@ +import { readFileSync } from 'node:fs' import { join } from 'node:path' import { fileURLToPath } from 'node:url' import { describe, expect, it } from 'vitest' @@ -12,16 +13,24 @@ const fixtureDir = fileURLToPath(new URL( )) const driver = join(fixtureDir, 'driver.ts') const configPath = join(fixtureDir, 'cordis.yml') +const packageDir = fileURLToPath(new URL('..', import.meta.url)) +const manifest = JSON.parse(readFileSync(join(packageDir, 'package.json'), 'utf8')) as { + dsh?: { bundle?: { patch?: string } } +} +const bundlePatch = manifest.dsh?.bundle?.patch +if (bundlePatch === undefined) throw new Error('Codex package must declare a Bundle patch') +const bundlePatchPath = join(packageDir, bundlePatch) const repoTsconfig = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) describe('Codex provider public Loader composition', () => { - it('loads the opt-in package and foreground tool without starting Codex', async () => { + it('loads the Bundle default, two named instances, their tools, and job controls without starting Codex', async () => { const { stdout, stderr } = await runLoaderSmoke({ label: 'subagent-codex Loader composition', tempDirPrefix: 'dsh-subagent-codex-loader-', binScript: driver, libBinScript: driver, configPath, + binArgs: [configPath, bundlePatchPath], tsconfigPath: repoTsconfig, env: { // Loading the optional package must not probe or start a Codex binary. @@ -31,22 +40,57 @@ describe('Codex provider public Loader composition', () => { expect(stderr).toBe('') expect(JSON.parse(stdout)).toEqual({ - providers: ['codex'], - provider: { - name: 'codex', - capabilities: { - outputSchema: false, - depthLimit: false, - toolFilter: false, - persona: false, + providers: ['codex-primary', 'codex-secondary', 'codex'], + providerDetails: [ + { + name: 'codex', + capabilities: { + outputSchema: false, + depthLimit: false, + toolFilter: false, + persona: false, + }, + inheritsParentContext: false, }, - inheritsParentContext: false, - }, - tool: { - name: 'subagent_codex', - parameterNames: ['description', 'prompt'], - required: ['description', 'prompt'], - }, + { + name: 'codex-primary', + capabilities: { + outputSchema: false, + depthLimit: false, + toolFilter: false, + persona: false, + }, + inheritsParentContext: false, + }, + { + name: 'codex-secondary', + capabilities: { + outputSchema: false, + depthLimit: false, + toolFilter: false, + persona: false, + }, + inheritsParentContext: false, + }, + ], + tools: [ + { + name: 'subagent_codex', + parameterNames: ['description', 'prompt', 'run_in_background'], + required: ['description', 'prompt'], + }, + { + name: 'subagent_codex_primary', + parameterNames: ['description', 'prompt', 'run_in_background'], + required: ['description', 'prompt'], + }, + { + name: 'subagent_codex_secondary', + parameterNames: ['description', 'prompt', 'run_in_background'], + required: ['description', 'prompt'], + }, + ], + jobTools: ['job_kill', 'job_list', 'job_output'], starts: 0, }) }, LOADER_SMOKE_TEST_TIMEOUT_MS) diff --git a/packages/subagent/subagent-codex/tests/real-deepseek.e2e.ts b/packages/subagent/subagent-codex/tests/real-deepseek.e2e.ts index 075b6048f2..c12e96a306 100644 --- a/packages/subagent/subagent-codex/tests/real-deepseek.e2e.ts +++ b/packages/subagent/subagent-codex/tests/real-deepseek.e2e.ts @@ -7,9 +7,9 @@ import { rmSync, writeFileSync, } from 'node:fs' +import { createRequire } from 'node:module' import { tmpdir } from 'node:os' -import { delimiter, join, resolve } from 'node:path' -import { fileURLToPath } from 'node:url' +import { dirname, join, resolve } from 'node:path' import { promisify } from 'node:util' import { Context } from '@deepseek-ai/cordis' import { afterEach, describe, expect, it, vi } from 'vitest' @@ -24,12 +24,12 @@ import { } from './deepseek-responses-bridge.ts' const execFileAsync = promisify(execFile) -const packageRoot = resolve(fileURLToPath(new URL('..', import.meta.url))) -const codexBinDir = join(packageRoot, 'node_modules', '.bin') +const codexPackageJson = createRequire(import.meta.url).resolve('@openai/codex/package.json') const codexPackage = JSON.parse(readFileSync( - join(packageRoot, 'node_modules', '@openai', 'codex', 'package.json'), + codexPackageJson, 'utf8', -)) as { version: string } +)) as { version: string; bin: { codex: string } } +const codexEntry = resolve(dirname(codexPackageJson), codexPackage.bin.codex) const roots: string[] = [] const contexts: Context[] = [] @@ -88,7 +88,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)( CODEX_HOME: codexHome, HOME: root, XDG_CONFIG_HOME: join(root, 'xdg-config'), - PATH: `${codexBinDir}${delimiter}${process.env.PATH ?? ''}`, + PATH: root, HTTP_PROXY: '', HTTPS_PROXY: '', ALL_PROXY: '', @@ -106,7 +106,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)( return handle }) await ctx.plugin(codex, { env, disposeGraceMs: 2_000 }) - const version = await execFileAsync(join(codexBinDir, 'codex'), ['--version'], { + const version = await execFileAsync(process.execPath, [codexEntry, '--version'], { env: { ...process.env, ...env }, }) expect(codexPackage.version).toBe('0.147.0') diff --git a/packages/subagent/subagent-codex/tests/real-product.spec.ts b/packages/subagent/subagent-codex/tests/real-product.spec.ts index 551d6db765..bbebd46f10 100644 --- a/packages/subagent/subagent-codex/tests/real-product.spec.ts +++ b/packages/subagent/subagent-codex/tests/real-product.spec.ts @@ -1,5 +1,6 @@ import { execFile } from 'node:child_process' import { + cpSync, existsSync, mkdirSync, mkdtempSync, @@ -7,17 +8,23 @@ import { writeFileSync, } from 'node:fs' import { rm } from 'node:fs/promises' +import { createRequire } from 'node:module' import { tmpdir } from 'node:os' -import { delimiter, join, resolve } from 'node:path' +import { delimiter, dirname, join, resolve } from 'node:path' import { fileURLToPath } from 'node:url' import { promisify } from 'node:util' import { Context } from '@deepseek-ai/cordis' import { afterEach, describe, expect, it, vi } from 'vitest' import type { Agent } from '@deepseek-ai/dsh-agent' import SubagentRuntime from '@deepseek-ai/dsh-subagent' -import type { SubprocessHandle } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessHandle, + SubprocessOutcome, + SubprocessSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' import LocalSubprocessRuntime from '@deepseek-ai/dsh-subprocess-local' import * as codex from '../src/index.ts' +import type { CodexPermissionMode } from '../src/run.ts' import { startResponsesFixture, type ResponsesBehavior, @@ -27,11 +34,13 @@ import { const execFileAsync = promisify(execFile) const packageRoot = resolve(fileURLToPath(new URL('..', import.meta.url))) const codexBinDir = join(packageRoot, 'node_modules', '.bin') -const codexEntry = join(packageRoot, 'node_modules', '@openai', 'codex', 'bin', 'codex.js') +const codexPackageJson = createRequire(import.meta.url).resolve('@openai/codex/package.json') const codexPackage = JSON.parse(readFileSync( - join(packageRoot, 'node_modules', '@openai', 'codex', 'package.json'), + codexPackageJson, 'utf8', -)) as { version: string } +)) as { version: string; bin: { codex: string } } +const codexEntry = resolve(dirname(codexPackageJson), codexPackage.bin.codex) +const codexPackageRoot = dirname(dirname(codexEntry)) const roots: string[] = [] const fixtures: ResponsesFixture[] = [] @@ -48,23 +57,31 @@ afterEach(async () => { interface RealHarness { readonly ctx: Context readonly handles: SubprocessHandle[] + readonly spawnSpecs: SubprocessSpawnSpec[] readonly parent: Agent readonly env: Record readonly workspace: string } -async function realHarness(script: readonly ResponsesBehavior[]): Promise<{ - readonly harness: RealHarness +interface RealInstanceFixture { readonly fixture: ResponsesFixture -}> { + readonly env: Record + readonly workspace: string +} + +type ResponsesScript = readonly ResponsesBehavior[] | ((workspace: string) => readonly ResponsesBehavior[]) + +async function realInstanceFixture( + script: ResponsesScript, +): Promise { const root = mkdtempSync(join(tmpdir(), 'dsh-codex-real-')) roots.push(root) const workspace = join(root, 'workspace') const codexHome = join(root, 'codex-home') - const fixture = await startResponsesFixture(script) - fixtures.push(fixture) mkdirSync(workspace) mkdirSync(codexHome) + const fixture = await startResponsesFixture(typeof script === 'function' ? script(workspace) : script) + fixtures.push(fixture) writeFileSync(join(codexHome, 'config.toml'), [ 'model = "fixture-model"', 'model_provider = "fixture"', @@ -95,23 +112,61 @@ async function realHarness(script: readonly ResponsesBehavior[]): Promise<{ ALL_PROXY: '', NO_PROXY: '127.0.0.1,localhost', } + return { fixture, env, workspace } +} + +interface RealRuntime { + readonly ctx: Context + readonly handles: SubprocessHandle[] + readonly spawnSpecs: SubprocessSpawnSpec[] +} + +async function realRuntime(): Promise { const ctx = new Context() contexts.push(ctx) await ctx.plugin(SubagentRuntime) await ctx.plugin(LocalSubprocessRuntime) const handles: SubprocessHandle[] = [] + const spawnSpecs: SubprocessSpawnSpec[] = [] const spawn = ctx.subprocess.spawn.bind(ctx.subprocess) vi.spyOn(ctx.subprocess, 'spawn').mockImplementation((spec) => { + spawnSpecs.push(spec) const handle = spawn(spec) handles.push(handle) return handle }) - await ctx.plugin(codex, { env, disposeGraceMs: 2_000 }) + return { ctx, handles, spawnSpecs } +} + +async function realHarness( + script: ResponsesScript, + permissionMode?: CodexPermissionMode, +): Promise<{ + readonly harness: RealHarness + readonly fixture: ResponsesFixture +}> { + const instance = await realInstanceFixture(script) + const { ctx, handles, spawnSpecs } = await realRuntime() + await ctx.plugin(codex, { + env: instance.env, + ...permissionMode === undefined ? {} : { permissionMode }, + disposeGraceMs: 2_000, + }) const parent = { id: 'real-parent', - session: { header: { cwd: workspace } }, + session: { header: { cwd: instance.workspace } }, } as unknown as Agent - return { harness: { ctx, handles, parent, env, workspace }, fixture } + return { + harness: { + ctx, + handles, + spawnSpecs, + parent, + env: instance.env, + workspace: instance.workspace, + }, + fixture: instance.fixture, + } } async function expectQuiescent(handles: readonly SubprocessHandle[]): Promise { @@ -124,6 +179,26 @@ async function expectQuiescent(handles: readonly SubprocessHandle[]): Promise + readonly required?: string[] + readonly type?: string | string[] +} + function responseInputTexts(body: Record): string[] { if (!Array.isArray(body.input)) return [] return body.input.flatMap((item): string[] => { @@ -141,17 +216,67 @@ function responseInputTexts(body: Record): string[] { } describe('real @openai/codex 0.147.0 product', () => { - it('passes the exact task and fake authentication to local Responses and returns exact text', async () => { + it('starts approve-for-me through the real app-server and returns exact text', async () => { const sentinel = 'REAL_CODEX_SENTINEL_0_147_0' const task = 'Return the fixture sentinel exactly.' const { harness, fixture } = await realHarness([ { kind: 'complete', text: sentinel }, - ]) + ], 'approve-for-me') expect(codexPackage.version).toBe('0.147.0') const version = await execFileAsync(process.execPath, [codexEntry, '--version'], { env: { ...process.env, ...harness.env }, }) expect(version.stdout.trim()).toBe('codex-cli 0.147.0') + const schemaRoot = mkdtempSync(join(tmpdir(), 'dsh-codex-schema-')) + roots.push(schemaRoot) + await execFileAsync(process.execPath, [ + codexEntry, + 'app-server', + 'generate-json-schema', + '--out', + schemaRoot, + ], { env: { ...process.env, ...harness.env } }) + const schema = JSON.parse(readFileSync( + join(schemaRoot, 'ServerNotification.json'), + 'utf8', + )) as { + definitions: { + CodexErrorInfo: { + oneOf: JsonSchemaNode[] + } + } + } + expect(schema.definitions.CodexErrorInfo.oneOf[0]?.enum).toEqual([ + 'contextWindowExceeded', + 'sessionBudgetExceeded', + 'usageLimitExceeded', + 'serverOverloaded', + 'cyberPolicy', + 'internalServerError', + 'unauthorized', + 'badRequest', + 'threadRollbackFailed', + 'sandboxError', + 'other', + ]) + expect(schema.definitions.CodexErrorInfo.oneOf.slice(1).map(variant => + Object.keys(variant.properties ?? {})[0])).toEqual([ + 'httpConnectionFailed', + 'responseStreamConnectionFailed', + 'responseStreamDisconnected', + 'responseTooManyFailedAttempts', + 'activeTurnNotSteerable', + ]) + for (const variant of schema.definitions.CodexErrorInfo.oneOf.slice(1, 5)) { + const category = Object.keys(variant.properties ?? {})[0]! + const detail = variant.properties?.[category] + expect(detail?.required).toBeUndefined() + expect(detail?.properties?.httpStatusCode).toEqual({ + format: 'uint16', + minimum: 0, + type: ['integer', 'null'], + }) + } const run = await harness.ctx.subagents.start('codex', { prompt: [{ type: 'text', text: task }], @@ -164,6 +289,13 @@ describe('real @openai/codex 0.147.0 product', () => { }) await run.dispose() + expect(harness.spawnSpecs[0]?.argv).toEqual([ + process.execPath, + codexEntry, + 'app-server', + '--stdio', + ]) + expect(fixture.requests).toHaveLength(1) const recorded = fixture.requests[0]! expect(recorded.method).toBe('POST') @@ -173,7 +305,98 @@ describe('real @openai/codex 0.147.0 product', () => { await expectQuiescent(harness.handles) }, 60_000) - it('cancels a real app-server command approval without executing the command', async () => { + it('fails a missing platform payload without falling back to a host codex', async () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-codex-missing-payload-')) + roots.push(root) + const isolatedPackage = join(root, 'node_modules', '@openai', 'codex') + mkdirSync(dirname(isolatedPackage), { recursive: true }) + cpSync(codexPackageRoot, isolatedPackage, { recursive: true, dereference: true }) + const isolatedEntry = join(isolatedPackage, 'bin', 'codex.js') + + await expect(execFileAsync(process.execPath, [isolatedEntry, '--version'], { + env: { + PATH: codexBinDir, + ...process.platform === 'win32' && process.env.SystemRoot !== undefined + ? { SystemRoot: process.env.SystemRoot } + : {}, + }, + })).rejects.toThrow(/Missing optional dependency @openai\/codex-[a-z0-9-]+/) + }, 30_000) + + it('runs two named instances concurrently and unloads one without revoking its run', async () => { + const safeInstance = await realInstanceFixture([{ kind: 'hold' }]) + const bypassInstance = await realInstanceFixture([{ + kind: 'complete', + text: 'NAMED_CODEX_BYPASS_RESULT', + }]) + const { ctx, handles, spawnSpecs } = await realRuntime() + const safeFiber = await ctx.plugin(codex, { + providerName: 'codex-safe', + env: safeInstance.env, + permissionMode: 'never', + disposeGraceMs: 2_000, + }) + const bypassFiber = await ctx.plugin(codex, { + providerName: 'codex-bypass', + env: bypassInstance.env, + permissionMode: 'dangerously-bypass-approvals-and-sandbox', + disposeGraceMs: 2_000, + }) + const safeParent = { + id: 'safe-parent', + session: { header: { cwd: safeInstance.workspace } }, + } as unknown as Agent + const bypassParent = { + id: 'bypass-parent', + session: { header: { cwd: bypassInstance.workspace } }, + } as unknown as Agent + const safeController = new AbortController() + + const [safeRun, bypassRun] = await Promise.all([ + ctx.subagents.start('codex-safe', { + prompt: [{ type: 'text', text: 'Hold the safe instance.' }], + parent: safeParent, + signal: safeController.signal, + }), + ctx.subagents.start('codex-bypass', { + prompt: [{ type: 'text', text: 'Complete the bypass instance.' }], + parent: bypassParent, + signal: new AbortController().signal, + }), + ]) + await safeInstance.fixture.requestStarted + await safeFiber.dispose() + expect(ctx.subagents.list()).toEqual(['codex-bypass']) + await expect(ctx.subagents.start('codex-safe', { + prompt: [{ type: 'text', text: 'This start must fail.' }], + parent: safeParent, + signal: new AbortController().signal, + })).rejects.toMatchObject({ code: 'NO_PROVIDER' }) + + await expect(bypassRun.result).resolves.toEqual({ + output: [{ type: 'text', text: 'NAMED_CODEX_BYPASS_RESULT' }], + stopReason: 'completed', + }) + safeController.abort(new Error('cancel only the published safe run')) + await expect(safeRun.result).resolves.toEqual({ + output: [], + stopReason: 'aborted', + }) + await Promise.all([safeRun.dispose(), bypassRun.dispose()]) + expect(safeInstance.fixture.requests).toHaveLength(1) + expect(bypassInstance.fixture.requests).toHaveLength(1) + expect(safeInstance.fixture.requests[0]?.body.input) + .not.toEqual(bypassInstance.fixture.requests[0]?.body.input) + expect(spawnSpecs.map(spec => spec.env?.CODEX_HOME).sort()).toEqual([ + safeInstance.env.CODEX_HOME, + bypassInstance.env.CODEX_HOME, + ].sort()) + await expectQuiescent(handles) + await bypassFiber.dispose() + expect(ctx.subagents.list()).toEqual([]) + }, 60_000) + + it('overrides on-request with never and reports a denied command safely', async () => { const command = process.platform === 'win32' ? 'cmd /c type nul > approval-side-effect' : 'touch approval-side-effect' @@ -200,6 +423,11 @@ describe('real @openai/codex 0.147.0 product', () => { kind: 'advertisedFunctionCall', choices: commandCalls, }, + { + kind: 'error', + status: 400, + message: 'fixture terminal failure after permission denial', + }, ]) const sideEffect = join(harness.workspace, 'approval-side-effect') const run = await harness.ctx.subagents.start('codex', { @@ -207,14 +435,24 @@ describe('real @openai/codex 0.147.0 product', () => { parent: harness.parent, signal: new AbortController().signal, }) - await expect(run.result).resolves.toEqual({ - output: [], - stopReason: 'error', - }) + const result = await run.result + expect(result.output).toEqual([]) + expect(result.stopReason).toBe('error') + const diagnosticLines = result.diagnostic?.split('\n') ?? [] + expect(diagnosticLines[0]).toBe( + 'Product subagent failure (product: Codex; stage: turn; category: other)', + ) + expect([ + 'Codex unattended decision (mode: never; request: command approval; decision: cancelled): the provider does not grant interactive approval', + 'Codex unattended decision (mode: never; request: sandbox execution; decision: failed): Codex reported a sandbox failure', + 'Codex unattended decision (mode: never; request: command execution; decision: denied): Codex rejected an escalation because the selected policy never asks for approval', + ]).toContain(diagnosticLines[1]) + expect(result.diagnostic).not.toContain(command) + expect(result.diagnostic).not.toContain(harness.workspace) await run.dispose() expect(existsSync(sideEffect)).toBe(false) - expect(fixture.requests).toHaveLength(1) + expect(fixture.requests).toHaveLength(2) const tools = fixture.requests[0]!.body.tools as Array> expect(commandCalls.some(call => tools.some(tool => ( tool.type === 'function' && tool.name === call.name @@ -225,6 +463,91 @@ describe('real @openai/codex 0.147.0 product', () => { await expectQuiescent(harness.handles) }, 60_000) + it('reports a real service failure and an early app-server exit safely', async () => { + { + const { harness } = await realHarness([{ + kind: 'error', + status: 503, + message: 'SECRET_TOKEN in /private/secret.txt', + }]) + const run = await harness.ctx.subagents.start('codex', { + prompt: [{ type: 'text', text: 'Exercise the service failure path.' }], + parent: harness.parent, + signal: new AbortController().signal, + }) + const result = await run.result + expect(result).toMatchObject({ output: [], stopReason: 'error' }) + expect(result.diagnostic).toBe( + 'Product subagent failure (product: Codex; stage: turn; category: internalServerError)', + ) + expect(result.diagnostic).not.toContain('SECRET_TOKEN') + expect(result.diagnostic).not.toContain('/private/secret.txt') + await run.dispose() + await expectQuiescent(harness.handles) + } + { + const { harness, fixture } = await realHarness([{ kind: 'hold' }]) + const run = await harness.ctx.subagents.start('codex', { + prompt: [{ type: 'text', text: 'Exercise the process failure path.' }], + parent: harness.parent, + signal: new AbortController().signal, + }) + await fixture.requestStarted + expect(harness.handles).toHaveLength(1) + harness.handles[0]!.terminate() + const outcome = await harness.handles[0]!.done + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: expectedProcessExitDiagnostic(outcome), + stopReason: 'error', + }) + await run.dispose() + await expectQuiescent(harness.handles) + } + }, 60_000) + + it('executes an explicitly selected dangerous bypass write in the isolated workspace', async () => { + const sideEffect = 'bypass-side-effect' + const { harness, fixture } = await realHarness((workspace): readonly ResponsesBehavior[] => { + const target = join(workspace, sideEffect) + const command = process.platform === 'win32' + ? `powershell.exe -NoLogo -NoProfile -NonInteractive -Command "Set-Content -LiteralPath '${target.replaceAll("'", "''")}' -Value 'bypass' -NoNewline"` + : `printf bypass > ${JSON.stringify(target)}` + const commandCalls = [ + { + name: 'exec_command', + arguments: { + cmd: command, + }, + }, + { + name: 'shell_command', + arguments: { + command, + }, + }, + ] as const + return [ + { kind: 'advertisedFunctionCall', choices: commandCalls }, + { kind: 'complete', text: 'bypass complete' }, + ] + }, 'dangerously-bypass-approvals-and-sandbox') + const target = join(harness.workspace, sideEffect) + const run = await harness.ctx.subagents.start('codex', { + prompt: [{ type: 'text', text: 'Create the fixture side effect.' }], + parent: harness.parent, + signal: new AbortController().signal, + }) + await expect(run.result).resolves.toEqual({ + output: [{ type: 'text', text: 'bypass complete' }], + stopReason: 'completed', + }) + expect(existsSync(target), JSON.stringify(fixture.requests.at(-1)?.body.input)).toBe(true) + expect(readFileSync(target, 'utf8').trim()).toBe('bypass') + await run.dispose() + await expectQuiescent(harness.handles) + }, 60_000) + it('settles cancellation locally and leaves the real app-server tree quiescent', async () => { const { harness, fixture } = await realHarness([{ kind: 'hold' }]) const controller = new AbortController() diff --git a/packages/subagent/subagent-codex/tests/responses-fixture.ts b/packages/subagent/subagent-codex/tests/responses-fixture.ts index 2b6e5868ae..c2ef18d803 100644 --- a/packages/subagent/subagent-codex/tests/responses-fixture.ts +++ b/packages/subagent/subagent-codex/tests/responses-fixture.ts @@ -17,6 +17,7 @@ interface RecordedResponsesRequest { /** Behavior consumed by one Responses request. */ export type ResponsesBehavior = | { readonly kind: 'complete'; readonly text: string } + | { readonly kind: 'error'; readonly status: number; readonly message: string } | { readonly kind: 'functionCall' readonly name: string @@ -275,6 +276,11 @@ export async function startResponsesFixture( response.end(JSON.stringify({ error: { message: 'none of the fixture function calls was advertised' } })) return } + if (behavior.kind === 'error') { + response.writeHead(behavior.status, { 'content-type': 'application/json' }) + response.end(JSON.stringify({ error: { message: behavior.message } })) + return + } response.writeHead(200, { 'content-type': 'text/event-stream', 'cache-control': 'no-cache', diff --git a/packages/subagent/subagent-codex/tests/subagent-codex.spec.ts b/packages/subagent/subagent-codex/tests/subagent-codex.spec.ts index 37b2e9ff0b..296c8afa09 100644 --- a/packages/subagent/subagent-codex/tests/subagent-codex.spec.ts +++ b/packages/subagent/subagent-codex/tests/subagent-codex.spec.ts @@ -1,6 +1,10 @@ +import { readFileSync } from 'node:fs' +import { dirname, resolve } from 'node:path' import { PassThrough } from 'node:stream' +import { fileURLToPath } from 'node:url' import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' +import * as yaml from 'js-yaml' import { describe, expect, it, vi } from 'vitest' import type { Agent } from '@deepseek-ai/dsh-agent' import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' @@ -10,11 +14,14 @@ import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import type { SubprocessHandle, SubprocessOutcome, + SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import LocalSubprocessRuntime from '@deepseek-ai/dsh-subprocess-local' import * as codex from '../src/index.ts' import * as invariant from '../src/invariant.ts' import { + CODEX_PERMISSION_MODES, + DEFAULT_CODEX_PERMISSION_MODE, codexAppServerArgv, DEFAULT_DISPOSE_GRACE_MS, disposeCodexChild, @@ -24,8 +31,50 @@ import { } from '../src/run.ts' import { CodexAppServerWire } from '../src/wire.ts' +const { hostStderrWrite } = vi.hoisted(() => ({ + hostStderrWrite: { + capture: false, + failNext: false, + chunks: [] as Buffer[], + }, +})) + +vi.mock('node:fs', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + writeFileSync( + fd: number, + value: string | Uint8Array, + ): void { + if (fd === 2 && hostStderrWrite.capture) { + if (hostStderrWrite.failNext) { + hostStderrWrite.failNext = false + throw Object.assign(new Error('host stderr broke'), { code: 'EIO' }) + } + const bytes = typeof value === 'string' + ? Buffer.from(value) + : Buffer.from(value.buffer, value.byteOffset, value.byteLength) + hostStderrWrite.chunks.push(bytes) + return + } + actual.writeFileSync(fd, value) + }, + } +}) + type JsonObject = Record +const CODEX_VERSION = '0.147.0' +const CODEX_PLATFORM_PACKAGES = [ + '@openai/codex-darwin-arm64', + '@openai/codex-darwin-x64', + '@openai/codex-linux-arm64', + '@openai/codex-linux-x64', + '@openai/codex-win32-arm64', + '@openai/codex-win32-x64', +] as const + const fakeParent = { id: 'parent', session: { header: { cwd: process.cwd() } }, @@ -94,6 +143,7 @@ interface FakeChildOptions { readonly pid?: number readonly exitOnTerminate?: boolean readonly doneError?: Error + readonly waitForExitError?: Error } interface FakeChild { @@ -101,8 +151,10 @@ interface FakeChild { readonly peer: ProtocolPeer readonly fromChild: PassThrough readonly toChild: PassThrough + readonly stderr: PassThrough readonly settle: (outcome?: SubprocessOutcome) => void readonly fail: (error: Error) => void + readonly setStderr: (text: string) => void readonly terminate: () => void readonly waitForExit: (signal?: AbortSignal) => Promise } @@ -110,6 +162,7 @@ interface FakeChild { function fakeChild(options: FakeChildOptions = {}): FakeChild { const fromChild = new PassThrough() const toChild = new PassThrough() + const stderr = new PassThrough() const peer = new ProtocolPeer(toChild, fromChild) let exited = false let resolveDone!: (outcome: SubprocessOutcome) => void @@ -135,6 +188,9 @@ function fakeChild(options: FakeChildOptions = {}): FakeChild { if (options.exitOnTerminate !== false) settle() }) const waitForExit = vi.fn(async (signal?: AbortSignal) => { + if (options.waitForExitError !== undefined) { + throw options.waitForExitError + } if (exited) return true if (signal === undefined) { await done.catch(() => {}) @@ -159,7 +215,7 @@ function fakeChild(options: FakeChildOptions = {}): FakeChild { pid: options.pid ?? 1234, stdin: toChild, stdout: fromChild, - stderr: undefined, + stderr, collected: {}, done, terminate, @@ -170,19 +226,30 @@ function fakeChild(options: FakeChildOptions = {}): FakeChild { peer, fromChild, toChild, + stderr, settle, fail, + setStderr: (text: string): void => { stderr.write(text) }, terminate, waitForExit, } } +function defaultWire(child: FakeChild): CodexAppServerWire { + return new CodexAppServerWire( + child.handle.stdout!, + child.handle.stdin!, + DEFAULT_CODEX_PERMISSION_MODE, + ) +} + function runSpec( child: FakeChild, overrides: Partial = {}, ): CodexRunSpec { return { cwd: process.cwd(), + permissionMode: DEFAULT_CODEX_PERMISSION_MODE, env: {}, disposeGraceMs: DEFAULT_DISPOSE_GRACE_MS, spawn: () => child.handle, @@ -195,7 +262,7 @@ async function initializeWire(): Promise<{ readonly wire: CodexAppServerWire }> { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) wire.start() const initializing = wire.initialize(new AbortController().signal) const initialize = await child.peer.nextMethod('initialize') @@ -259,18 +326,93 @@ function turnCompleted( } } +function expectedFailureDiagnostic( + stage: 'initialize' | 'thread-start' | 'turn-start' | 'turn' | 'process' | 'teardown', + category: string, + options: { + readonly httpStatus?: number + readonly outcome?: Partial + } = {}, +): string { + const fields = [ + 'product: Codex', + `stage: ${stage}`, + `category: ${category}`, + ] + if (options.httpStatus !== undefined) { + fields.push(`HTTP status: ${options.httpStatus}`) + } + if ( + options.outcome?.exitCode !== null + && options.outcome?.exitCode !== undefined + ) { + fields.push(`exit code: ${options.outcome.exitCode}`) + } + if ( + options.outcome?.signal !== null + && options.outcome?.signal !== undefined + ) { + fields.push(`signal: ${options.outcome.signal}`) + } + return `Product subagent failure (${fields.join('; ')})` +} + describe('task admission and package contracts', () => { - it('resolves the fixed app-server command through the Windows npm shim boundary', () => { - expect(codexAppServerArgv('win32')).toEqual([ - 'cmd.exe', - '/d', - '/s', - '/c', - 'codex', + it('ships one independently installable provider-only Bundle patch', () => { + const root = fileURLToPath(new URL('..', import.meta.url)) + const manifest = JSON.parse(readFileSync(resolve(root, 'package.json'), 'utf8')) as { + dependencies?: Record + files?: string[] + dsh?: { bundle?: { patch?: string } } + } + expect(manifest.dsh?.bundle?.patch).toBe('./cordis.patch.yml') + expect(manifest.files).toContain('cordis.patch.yml') + expect(manifest.dependencies).toHaveProperty( + '@deepseek-ai/dsh-sdk-protocol', + 'workspace:^', + ) + expect(manifest.dependencies).toHaveProperty('@openai/codex', CODEX_VERSION) + expect(manifest.dependencies).not.toHaveProperty('@deepseek-ai/dsh-subagent-claude-code') + + const codexPackageJson = fileURLToPath(import.meta.resolve('@openai/codex/package.json')) + const codexManifest = JSON.parse(readFileSync(codexPackageJson, 'utf8')) as { + version: string + bin: { codex: string } + optionalDependencies: Record + } + expect(codexManifest.version).toBe(CODEX_VERSION) + expect(codexManifest.bin).toEqual({ codex: 'bin/codex.js' }) + expect(codexManifest.optionalDependencies).toEqual(Object.fromEntries( + CODEX_PLATFORM_PACKAGES.map(packageName => [ + packageName, + `npm:@openai/codex@${CODEX_VERSION}-${packageName.slice('@openai/codex-'.length)}`, + ]), + )) + expect(codexAppServerArgv()).toEqual([ + process.execPath, + resolve(dirname(codexPackageJson), codexManifest.bin.codex), 'app-server', '--stdio', ]) - expect(codexAppServerArgv('linux')).toEqual(['codex', 'app-server', '--stdio']) + + const lockfile = readFileSync(resolve(root, '../../../pnpm-lock.yaml'), 'utf8') + for (const packageName of CODEX_PLATFORM_PACKAGES) { + const suffix = packageName.slice('@openai/codex-'.length) + expect(lockfile).toContain(` '@openai/codex@${CODEX_VERSION}-${suffix}':`) + expect(lockfile).toContain( + ` '${packageName}': '@openai/codex@${CODEX_VERSION}-${suffix}'`, + ) + } + + const parsed = yaml.load(readFileSync(resolve(root, manifest.dsh!.bundle!.patch!), 'utf8')) + const rows = Array.isArray(parsed) + ? (parsed as Array<{ insert?: Array<{ id?: string; name?: string }> }>).flatMap(entry => entry.insert ?? []) + : [] + expect(rows).toEqual([{ + id: 'subagent-codex', + name: '@deepseek-ai/dsh-subagent-codex', + }]) + expect(JSON.stringify(rows)).not.toContain('tool-subagent') }) it('accepts one or more text blocks and rejects empty or non-text tasks', () => { @@ -285,7 +427,7 @@ describe('task admission and package contracts', () => { .toThrow('must not be empty') }) - it('registers one fixed descriptor, validates config, and unregisters on HMR', async () => { + it('registers the default descriptor, validates config, and unregisters on HMR', async () => { const ctx = new Context() await ctx.plugin(SubagentRuntime) await ctx.plugin(LocalSubprocessRuntime) @@ -314,6 +456,179 @@ describe('task admission and package contracts', () => { await ctx.fiber.dispose() }) + it('keeps named instances, runs, and HMR ownership isolated', async () => { + const ctx = new Context() + await ctx.plugin(SubagentRuntime) + await ctx.plugin(LocalSubprocessRuntime) + const safeChild = fakeChild() + const bypassChild = fakeChild() + const spawnSpecs: SubprocessSpawnSpec[] = [] + vi.spyOn(ctx.subprocess, 'spawn').mockImplementation((spec) => { + spawnSpecs.push(spec) + return spec.env?.DSH_CODEX_INSTANCE === 'safe' + ? safeChild.handle + : bypassChild.handle + }) + const added: string[] = [] + const started: string[] = [] + const ended: string[] = [] + const removed: string[] = [] + ctx.on('subagent/provider-added', provider => void added.push(provider.name)) + ctx.on('subagent/start', info => void started.push(info.provider)) + ctx.on('subagent/end', info => void ended.push(info.provider)) + ctx.on('subagent/provider-removed', providerName => void removed.push(providerName)) + const safeFiber = await ctx.plugin(codex, { + providerName: 'codex-safe', + env: { DSH_CODEX_INSTANCE: 'safe' }, + permissionMode: 'never', + disposeGraceMs: 11, + }) + const bypassFiber = await ctx.plugin(codex, { + providerName: 'codex-bypass', + env: { DSH_CODEX_INSTANCE: 'bypass' }, + permissionMode: 'dangerously-bypass-approvals-and-sandbox', + disposeGraceMs: 29, + }) + expect(ctx.subagents.list()).toEqual(['codex-safe', 'codex-bypass']) + expect(added).toEqual(['codex-safe', 'codex-bypass']) + + const safeController = new AbortController() + const safeStarting = ctx.subagents.start( + 'codex-safe', + request(undefined, safeController.signal), + ) + const bypassStarting = ctx.subagents.start('codex-bypass', request()) + for (const child of [safeChild, bypassChild]) { + const initialize = await child.peer.nextMethod('initialize') + child.peer.respond(initialize, { userAgent: 'codex-cli 0.147.0' }) + await child.peer.nextMethod('initialized') + const threadStart = await child.peer.nextMethod('thread/start') + child.peer.respond(threadStart, { + thread: { id: 'thread-1', ephemeral: true }, + }) + } + const [safeRun, bypassRun] = await Promise.all([ + safeStarting, + bypassStarting, + ]) + await safeFiber.dispose() + expect(ctx.subagents.list()).toEqual(['codex-bypass']) + expect(removed).toEqual(['codex-safe']) + await expect(ctx.subagents.start('codex-safe', request())) + .rejects.toMatchObject({ code: 'NO_PROVIDER' }) + + const safeTurn = await safeChild.peer.nextMethod('turn/start') + const bypassTurn = await bypassChild.peer.nextMethod('turn/start') + safeChild.peer.respond(safeTurn, { turn: { id: 'turn-safe' } }) + bypassChild.peer.send( + { id: bypassTurn.id, result: { turn: { id: 'turn-bypass' } } }, + agentMessage('bypass answer', 'final_answer', 'turn-bypass'), + turnCompleted('completed', 'turn-bypass'), + ) + await expect(bypassRun.result).resolves.toEqual({ + output: [{ type: 'text', text: 'bypass answer' }], + stopReason: 'completed', + }) + safeController.abort(new Error('stop only the safe instance')) + await expect(safeRun.result).resolves.toEqual({ + output: [], + stopReason: 'aborted', + }) + expect(spawnSpecs.map(spec => ({ + instance: spec.env?.DSH_CODEX_INSTANCE, + graceMs: spec.graceMs, + }))).toEqual([ + { instance: 'safe', graceMs: 11 }, + { instance: 'bypass', graceMs: 29 }, + ]) + + await Promise.all([safeRun.dispose(), bypassRun.dispose()]) + expect([...started].sort()).toEqual(['codex-bypass', 'codex-safe']) + expect([...ended].sort()).toEqual(['codex-bypass', 'codex-safe']) + expect(safeChild.terminate).toHaveBeenCalledOnce() + expect(bypassChild.terminate).toHaveBeenCalledOnce() + await bypassFiber.dispose() + expect(removed).toEqual(['codex-safe', 'codex-bypass']) + await ctx.fiber.dispose() + }) + + it('rejects duplicate provider names without replacing the first instance', async () => { + const ctx = new Context() + await ctx.plugin(SubagentRuntime) + await ctx.plugin(LocalSubprocessRuntime) + const firstFiber = await ctx.plugin(codex, { + providerName: 'codex-duplicate', + }) + const first = ctx.subagents.getProvider('codex-duplicate') + await expect(ctx.plugin(codex, { + providerName: 'codex-duplicate', + permissionMode: 'dangerously-bypass-approvals-and-sandbox', + })).rejects.toMatchObject({ code: 'DUPLICATE_PROVIDER' }) + expect(ctx.subagents.getProvider('codex-duplicate')).toBe(first) + expect(ctx.subagents.list()).toEqual(['codex-duplicate']) + await firstFiber.dispose() + await ctx.fiber.dispose() + }) + + it('accepts only the three fixed non-interactive permission modes', () => { + expect(codex.Config({}).providerName).toBe('codex') + expect(codex.Config({ providerName: 'codex-safe' }).providerName) + .toBe('codex-safe') + expect(() => codex.Config({ providerName: '' })).toThrow() + expect(codex.Config({}).permissionMode).toBe(DEFAULT_CODEX_PERMISSION_MODE) + for (const permissionMode of CODEX_PERMISSION_MODES) { + expect(codex.Config({ permissionMode }).permissionMode).toBe(permissionMode) + } + for (const permissionMode of ['on-request', 'untrusted', 'future-mode']) { + expect(() => codex.Config({ permissionMode } as never)).toThrow() + } + }) + + it('resolves the safe permission default when apply is called directly', async () => { + const ctx = new Context() + await ctx.plugin(SubagentRuntime) + await ctx.plugin(LocalSubprocessRuntime) + codex.apply(ctx, { env: {}, disposeGraceMs: 3_000 }) + expect(ctx.subagents.getProvider('codex')).toBeDefined() + await ctx.fiber.dispose() + }) + + it.each([ + ['never', { approvalPolicy: 'never' }], + ['approve-for-me', { + approvalPolicy: 'on-request', + approvalsReviewer: 'auto_review', + sandbox: 'workspace-write', + }], + ['dangerously-bypass-approvals-and-sandbox', { + approvalPolicy: 'never', + sandbox: 'danger-full-access', + }], + ] as const)('maps %s to the official thread/start fields', async (permissionMode, expected) => { + const child = fakeChild() + const wire = new CodexAppServerWire( + child.handle.stdout!, + child.handle.stdin!, + permissionMode, + ) + wire.start() + const initializing = wire.initialize(new AbortController().signal) + const initialize = await child.peer.nextMethod('initialize') + child.peer.respond(initialize, { userAgent: 'codex-cli 0.147.0' }) + await initializing + await child.peer.nextMethod('initialized') + const starting = wire.startThread('/workspace', new AbortController().signal) + const threadStart = await child.peer.nextMethod('thread/start') + expect(threadStart.params).toEqual({ + cwd: '/workspace', + ephemeral: true, + ...expected, + }) + child.peer.respond(threadStart, { thread: { id: 'thread-1', ephemeral: true } }) + await starting + wire.close() + }) + it('requires a parent session cwd without suggesting unsupported config', async () => { const ctx = new Context() await ctx.plugin(SubagentRuntime) @@ -363,7 +678,7 @@ describe('task admission and package contracts', () => { describe('CodexAppServerWire', () => { it('sends the fixed handshake, thread, and turn payloads and keeps final_answer', async () => { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) expect(wire.collectOutput()).toEqual([]) wire.start() @@ -386,7 +701,11 @@ describe('CodexAppServerWire', () => { const starting = wire.startThread('/workspace', new AbortController().signal) const threadStart = await child.peer.nextMethod('thread/start') - expect(threadStart.params).toEqual({ cwd: '/workspace', ephemeral: true }) + expect(threadStart.params).toEqual({ + cwd: '/workspace', + ephemeral: true, + approvalPolicy: 'never', + }) child.peer.respond(threadStart, { thread: { id: 'thread-1', ephemeral: true } }) await starting @@ -451,29 +770,109 @@ describe('CodexAppServerWire', () => { wire.close() }) - it('maps only an explicit context-window failure to max-tokens', async () => { - const { child, wire } = await initializeWire() - const result = wire.runTurn(['task'], new AbortController().signal) - const turnStart = await child.peer.nextMethod('turn/start') - child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) - child.peer.send( - agentMessage('partial answer', null), - turnCompleted('failed', 'turn-1', 'thread-1', { - message: 'too much context', - codexErrorInfo: 'contextWindowExceeded', - }), - ) - await expect(result).resolves.toEqual({ - output: [{ type: 'text', text: 'partial answer' }], - stopReason: 'max-tokens', - }) - wire.close() + it('maps the complete string error union without changing stop reasons', async () => { + const categories = [ + 'contextWindowExceeded', + 'sessionBudgetExceeded', + 'usageLimitExceeded', + 'serverOverloaded', + 'cyberPolicy', + 'internalServerError', + 'unauthorized', + 'badRequest', + 'threadRollbackFailed', + 'sandboxError', + 'other', + ] as const + for (const category of categories) { + const { child, wire } = await initializeWire() + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.peer.send( + agentMessage('partial answer', null), + turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'SECRET_TOKEN in /private/secret.txt', + codexErrorInfo: category, + }), + ) + if (category === 'contextWindowExceeded') { + await expect(result).resolves.toEqual({ + output: [{ type: 'text', text: 'partial answer' }], + stopReason: 'max-tokens', + }) + } else { + await expect(result).rejects.toThrow(`status failed: ${category}`) + } + expect(wire.collectFailure()).toEqual({ + stage: 'turn', + category, + }) + expect(JSON.stringify(wire.collectFailure())).not.toContain('SECRET_TOKEN') + expect(JSON.stringify(wire.collectFailure())).not.toContain('/private/secret.txt') + wire.close() + } + }) + + it('maps all object error variants and only numeric HTTP status', async () => { + const scenarios = [ + ['httpConnectionFailed', { httpStatusCode: 503 }, 503], + ['responseStreamConnectionFailed', { httpStatusCode: null }, undefined], + ['responseStreamDisconnected', {}, undefined], + ['responseTooManyFailedAttempts', { httpStatusCode: '503' }, undefined], + ['activeTurnNotSteerable', { turnKind: 'review' }, undefined], + ] as const + for (const [category, detail, httpStatus] of scenarios) { + const { child, wire } = await initializeWire() + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'SECRET_TOKEN in /private/secret.txt', + codexErrorInfo: { [category]: detail }, + })) + await expect(result).rejects.toThrow(`status failed: ${category}`) + expect(wire.collectFailure()).toEqual({ + stage: 'turn', + category, + ...(httpStatus === undefined ? {} : { httpStatus }), + }) + expect(JSON.stringify(wire.collectFailure())).not.toContain('turnKind') + wire.close() + } + }) + + it('uses unknown for version-external or malformed error info', async () => { + for (const codexErrorInfo of [ + 'futureError', + { futureVariant: { message: 'SECRET_TOKEN' } }, + { + httpConnectionFailed: { httpStatusCode: 503 }, + otherVariant: {}, + }, + { httpConnectionFailed: null }, + ]) { + const { child, wire } = await initializeWire() + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'SECRET_TOKEN in /private/secret.txt', + codexErrorInfo, + })) + await expect(result).rejects.toThrow('status failed: unknown') + expect(wire.collectFailure()).toEqual({ + stage: 'turn', + category: 'unknown', + }) + wire.close() + } }) it('rejects invalid handshake, thread, and turn response shapes', async () => { { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) wire.start() const pending = wire.initialize(new AbortController().signal) const frame = await child.peer.nextMethod('initialize') @@ -483,7 +882,7 @@ describe('CodexAppServerWire', () => { } { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) wire.start() const pending = wire.startThread('/workspace', new AbortController().signal) const frame = await child.peer.nextMethod('thread/start') @@ -497,6 +896,10 @@ describe('CodexAppServerWire', () => { const frame = await child.peer.nextMethod('turn/start') child.peer.respond(frame, { turn: { id: '' } }) await expect(pending).rejects.toThrow('turn/start turn id') + expect(wire.collectFailure()).toEqual({ + stage: 'turn-start', + category: 'unknown', + }) wire.close() } }) @@ -530,6 +933,10 @@ describe('CodexAppServerWire', () => { frames: [turnCompleted('failed', 'turn-1', 'thread-1', { message: 'no' })], message: 'status failed', }, + { + frames: [turnCompleted('failed', 'turn-1', 'thread-1', 'SECRET_TOKEN')], + message: 'status failed', + }, { frames: [turnCompleted('interrupted')], message: 'status interrupted', @@ -544,8 +951,13 @@ describe('CodexAppServerWire', () => { const result = wire.runTurn(['task'], new AbortController().signal) const turnStart = await child.peer.nextMethod('turn/start') child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + await nextTask() child.peer.send(...scenario.frames) await expect(result).rejects.toThrow(scenario.message) + expect(wire.collectFailure()).toEqual({ + stage: 'turn', + category: 'unknown', + }) wire.close() } }) @@ -586,15 +998,31 @@ describe('CodexAppServerWire', () => { threadId: 'thread-1', turnId: 'turn-1', availableDecisions: ['decline', 'cancel'], + command: 'cat /private/secret.txt', }, }) expect(await child.peer.nextResponse('command')).toMatchObject({ result: { decision: 'cancel' }, }) + expect(wire.collectDiagnostic()).toBeUndefined() child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) await nextTask() + expect(wire.collectDiagnostic()).toBe( + 'Codex unattended decision (mode: never; request: command approval; decision: cancelled): the provider does not grant interactive approval', + ) const requests = [ + { + id: 'command-decline', + method: 'item/commandExecution/requestApproval', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + availableDecisions: ['decline'], + }, + result: { decision: 'decline' }, + diagnostic: 'Codex unattended decision (mode: never; request: command approval; decision: declined): the provider does not grant interactive approval', + }, { id: 'file', method: 'item/fileChange/requestApproval', @@ -604,30 +1032,46 @@ describe('CodexAppServerWire', () => { availableDecisions: ['decline'], }, result: { decision: 'decline' }, + diagnostic: 'Codex unattended decision (mode: never; request: file approval; decision: declined): the provider does not grant interactive approval', + }, + { + id: 'file-cancel', + method: 'item/fileChange/requestApproval', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + availableDecisions: ['cancel'], + }, + result: { decision: 'cancel' }, + diagnostic: 'Codex unattended decision (mode: never; request: file approval; decision: cancelled): the provider does not grant interactive approval', }, { id: 'file-default', method: 'item/fileChange/requestApproval', params: { threadId: 'thread-1', turnId: 'turn-1' }, result: { decision: 'decline' }, + diagnostic: 'Codex unattended decision (mode: never; request: file approval; decision: declined): the provider does not grant interactive approval', }, { id: 'permissions', method: 'item/permissions/requestApproval', params: { threadId: 'thread-1', turnId: 'turn-1' }, result: { permissions: {}, scope: 'turn' }, + diagnostic: 'Codex unattended decision (mode: never; request: permission grant; decision: denied): the provider grants no additional turn permissions', }, { id: 'user-input', method: 'item/tool/requestUserInput', params: { threadId: 'thread-1', turnId: 'turn-1', questions: [] }, result: { answers: {} }, + diagnostic: 'Codex unattended decision (mode: never; request: user input; decision: empty response): the provider does not collect interactive answers', }, { id: 'mcp', method: 'mcpServer/elicitation/request', params: { threadId: 'thread-1', turnId: null }, result: { action: 'decline', content: null, _meta: null }, + diagnostic: 'Codex unattended decision (mode: never; request: MCP elicitation; decision: declined): the provider does not collect interactive MCP input', }, ] as const for (const serverRequest of requests) { @@ -635,13 +1079,205 @@ describe('CodexAppServerWire', () => { expect(await child.peer.nextResponse(serverRequest.id)).toMatchObject({ result: serverRequest.result, }) + expect(wire.collectDiagnostic()).toBe(serverRequest.diagnostic) } + expect(wire.collectDiagnostic()).not.toContain('/private/secret.txt') + child.peer.send(agentMessage('answer', 'final_answer'), turnCompleted('completed')) + await expect(result).resolves.toEqual({ + output: [{ type: 'text', text: 'answer' }], + stopReason: 'completed', + }) + wire.close() + }) + + it('records only a safe diagnostic for an explicit sandbox failure', async () => { + const { child, wire } = await initializeWire() + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'failed at /private/secret.txt with SECRET_TOKEN', + additionalDetails: 'raw command payload', + codexErrorInfo: 'sandboxError', + })) + await expect(result).rejects.toThrow('status failed') + expect(wire.collectDiagnostic()).toBe( + 'Codex unattended decision (mode: never; request: sandbox execution; decision: failed): Codex reported a sandbox failure', + ) + expect(wire.collectDiagnostic()).not.toContain('SECRET_TOKEN') + expect(wire.collectDiagnostic()).not.toContain('/private/secret.txt') + wire.close() + }) + + it('records declined command and file items without retaining their payloads', async () => { + const { child, wire } = await initializeWire() + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.peer.send({ + method: 'item/completed', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + item: { + type: 'commandExecution', + status: 'declined', + command: 'cat /private/secret.txt', + }, + }, + }) + await nextTask() + expect(wire.collectDiagnostic()).toBe( + 'Codex unattended decision (mode: never; request: command execution; decision: declined): Codex declined the command under the selected permission mode', + ) + expect(wire.collectDiagnostic()).not.toContain('/private/secret.txt') + + child.peer.send( + { + method: 'item/completed', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + item: { + type: 'fileChange', + status: 'declined', + patch: 'SECRET_TOKEN in /private/secret.txt', + }, + }, + }, + turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'SECRET_TOKEN in /private/secret.txt', + codexErrorInfo: 'other', + }), + ) + await expect(result).rejects.toThrow('status failed') + expect(wire.collectDiagnostic()).toBe( + 'Codex unattended decision (mode: never; request: file change; decision: declined): Codex declined the file change under the selected permission mode', + ) + expect(wire.collectDiagnostic()).not.toContain('SECRET_TOKEN') + expect(wire.collectDiagnostic()).not.toContain('/private/secret.txt') + wire.close() + }) + + it('recognizes large, split, and ordered stderr signatures without retaining raw text', () => { + const first = fakeChild() + const largeWire = new CodexAppServerWire( + first.handle.stdout!, + first.handle.stdin!, + 'never', + ) + largeWire.observeStderr( + `SECRET_TOKEN approval policy is Never; reject command${'x'.repeat(2_048)}`, + ) + expect(largeWire.collectDiagnostic()).toBe( + 'Codex unattended decision (mode: never; request: command execution; decision: denied): Codex rejected an escalation because the selected policy never asks for approval', + ) + expect(largeWire.collectDiagnostic()).not.toContain('SECRET_TOKEN') + + const second = fakeChild() + const splitWire = new CodexAppServerWire( + second.handle.stdout!, + second.handle.stdin!, + 'never', + ) + splitWire.observeStderr('SECRET_TOKEN approval policy is Ne') + splitWire.observeStderr('ver; reject command — /private/secret.txt') + expect(splitWire.collectDiagnostic()).toBe( + 'Codex unattended decision (mode: never; request: command execution; decision: denied): Codex rejected an escalation because the selected policy never asks for approval', + ) + expect(splitWire.collectDiagnostic()).not.toContain('SECRET_TOKEN') + expect(splitWire.collectDiagnostic()).not.toContain('/private/secret.txt') + + const third = fakeChild() + const orderedWire = new CodexAppServerWire( + third.handle.stdout!, + third.handle.stdin!, + 'dangerously-bypass-approvals-and-sandbox', + ) + orderedWire.observeStderr( + 'approval policy is Never; reject command; recorded sandbox violation: path=/private/secret.txt', + ) + expect(orderedWire.collectDiagnostic()).toBe( + 'Codex unattended decision (mode: dangerously-bypass-approvals-and-sandbox; request: sandbox execution; decision: failed): Codex reported a sandbox violation', + ) + expect(orderedWire.collectDiagnostic()).not.toContain('/private/secret.txt') + }) + + it('does not reapply an old stderr signature after a newer request diagnostic', async () => { + const { child, wire } = await initializeWire() + wire.observeStderr('recorded sandbox violation:') + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + await nextTask() + child.peer.send({ + id: 'file-approval', + method: 'item/fileChange/requestApproval', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + availableDecisions: ['decline'], + }, + }) + await child.peer.nextResponse('file-approval') + expect(wire.collectDiagnostic()).toContain('request: file approval') + wire.observeStderr('later benign stderr') + expect(wire.collectDiagnostic()).toContain('request: file approval') child.peer.send(agentMessage('answer', 'final_answer'), turnCompleted('completed')) await expect(result).resolves.toMatchObject({ stopReason: 'completed' }) wire.close() }) + it('keeps a newer request diagnostic after replaying an older early item', async () => { + const { child, wire } = await initializeWire() + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.send({ + method: 'item/completed', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + item: { type: 'fileChange', status: 'declined' }, + }, + }) + await nextTask() + child.peer.send({ + id: 'newer-command-request', + method: 'item/commandExecution/requestApproval', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + availableDecisions: ['cancel'], + }, + }) + await child.peer.nextResponse('newer-command-request') + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.peer.send(agentMessage('answer', 'final_answer'), turnCompleted('completed')) + await expect(result).resolves.toMatchObject({ stopReason: 'completed' }) + expect(wire.collectDiagnostic()).toContain('request: command approval') + wire.close() + }) + + it('keeps a newer stderr fact after replaying an older early terminal', async () => { + hostStderrWrite.capture = true + hostStderrWrite.chunks.length = 0 + const { child, wire } = await initializeWire() + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'sandbox failure', + codexErrorInfo: 'sandboxError', + })) + await nextTask() + wire.observeStderr('approval policy is Never; reject command') + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + await expect(result).rejects.toThrow('sandboxError') + expect(wire.collectDiagnostic()).toContain('request: command execution') + wire.close() + hostStderrWrite.capture = false + }) + it('fails the run on unknown requests or wrong request association', async () => { for (const serverRequest of [ { @@ -704,6 +1340,44 @@ describe('CodexAppServerWire', () => { wire.close() }) + it('does not retain a diagnostic from a mismatched early item', async () => { + const { child, wire } = await initializeWire() + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.send({ + method: 'item/completed', + params: { + threadId: 'thread-1', + turnId: 'turn-early', + item: { type: 'fileChange', status: 'declined' }, + }, + }) + child.peer.respond(turnStart, { turn: { id: 'turn-response' } }) + await expect(result).rejects.toThrow('did not match the active turn') + expect(wire.collectDiagnostic()).toBeUndefined() + wire.close() + }) + + it('does not retain a diagnostic from a mismatched provisional request', async () => { + const { child, wire } = await initializeWire() + const result = wire.runTurn(['task'], new AbortController().signal) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.send({ + id: 'provisional-approval', + method: 'item/commandExecution/requestApproval', + params: { + threadId: 'thread-1', + turnId: 'turn-early', + availableDecisions: ['cancel'], + }, + }) + await child.peer.nextResponse('provisional-approval') + child.peer.respond(turnStart, { turn: { id: 'turn-response' } }) + await expect(result).rejects.toThrow('did not match the active turn') + expect(wire.collectDiagnostic()).toBeUndefined() + wire.close() + }) + it('rejects conflicting early notifications and requests before turn/start', async () => { { const { child, wire } = await initializeWire() @@ -790,7 +1464,7 @@ describe('CodexAppServerWire', () => { it('rejects pending work on abort, EOF, and stream error', async () => { { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) wire.start() const controller = new AbortController() controller.abort('pre-aborted') @@ -800,7 +1474,7 @@ describe('CodexAppServerWire', () => { } { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) wire.start() const controller = new AbortController() const pending = wire.initialize(controller.signal) @@ -811,7 +1485,7 @@ describe('CodexAppServerWire', () => { } { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) wire.start() const pending = wire.initialize(new AbortController().signal) await child.peer.nextMethod('initialize') @@ -821,7 +1495,7 @@ describe('CodexAppServerWire', () => { } { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) wire.start() const pending = wire.initialize(new AbortController().signal) await child.peer.nextMethod('initialize') @@ -831,7 +1505,7 @@ describe('CodexAppServerWire', () => { } { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) wire.start() const pending = wire.initialize(new AbortController().signal) await child.peer.nextMethod('initialize') @@ -864,7 +1538,7 @@ describe('run lifecycle and quiescence', () => { expect(spawn).toHaveBeenCalledWith({ argv: codexAppServerArgv(), cwd: process.cwd(), - stdio: { stdin: 'pipe', stdout: 'pipe', stderr: 'inherit' }, + stdio: { stdin: 'pipe', stdout: 'pipe', stderr: 'pipe' }, graceMs: DEFAULT_DISPOSE_GRACE_MS, env: { OPENAI_API_KEY: 'fake' }, }) @@ -907,28 +1581,307 @@ describe('run lifecycle and quiescence', () => { await run.dispose() }) + it('reports turn-start failures and omits captured facts after success', async () => { + { + const { child, run, turnStart } = await publishRun() + child.peer.respond(turnStart, { turn: { id: '' } }) + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: expectedFailureDiagnostic('turn-start', 'unknown'), + stopReason: 'error', + }) + await run.dispose() + } + { + const { child, run, turnStart } = await publishRun() + child.peer.send({ + id: 'successful-approval', + method: 'item/commandExecution/requestApproval', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + availableDecisions: ['cancel'], + }, + }) + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + await child.peer.nextResponse('successful-approval') + child.peer.send( + agentMessage('answer', 'final_answer'), + turnCompleted('completed'), + ) + await expect(run.result).resolves.toEqual({ + output: [{ type: 'text', text: 'answer' }], + stopReason: 'completed', + }) + await run.dispose() + } + }) + + it('preserves representative terminal categories, HTTP status, and mapping', async () => { + const scenarios = [ + ['contextWindowExceeded', 'max-tokens', undefined], + ['sessionBudgetExceeded', 'error', undefined], + [{ httpConnectionFailed: { httpStatusCode: 503 } }, 'error', 503], + [{ activeTurnNotSteerable: { turnKind: 'review' } }, 'error', undefined], + ['futureError', 'error', undefined], + ] as const + for (const [codexErrorInfo, stopReason, httpStatus] of scenarios) { + const { child, run, turnStart } = await publishRun() + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.peer.send( + agentMessage('partial answer', null), + turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'SECRET_TOKEN in /private/secret.txt', + codexErrorInfo, + }), + ) + const category = typeof codexErrorInfo === 'string' + && codexErrorInfo !== 'futureError' + ? codexErrorInfo + : typeof codexErrorInfo === 'object' + ? Object.keys(codexErrorInfo)[0]! + : 'unknown' + const result = await run.result + expect(result).toEqual({ + output: [{ type: 'text', text: 'partial answer' }], + diagnostic: expectedFailureDiagnostic('turn', category, { + ...(httpStatus === undefined ? {} : { httpStatus }), + }), + stopReason, + }) + expect(result.diagnostic).not.toContain('SECRET_TOKEN') + expect(result.diagnostic).not.toContain('/private/secret.txt') + expect(result.diagnostic).not.toContain('turnKind') + await run.dispose() + } + }) + + it('includes a queued stderr permission fact in a max-token result', async () => { + const { child, run, turnStart } = await publishRun() + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + setImmediate(() => { + child.stderr.write('approval policy is Never; reject command') + child.peer.send( + agentMessage('partial answer', null), + turnCompleted('failed', 'turn-1', 'thread-1', { + codexErrorInfo: 'contextWindowExceeded', + }), + ) + }) + child.settle({ exitCode: 17, signal: null }) + await expect(run.result).resolves.toEqual({ + output: [{ type: 'text', text: 'partial answer' }], + diagnostic: `${expectedFailureDiagnostic('turn', 'contextWindowExceeded', { outcome: { exitCode: 17, signal: null } })}\nCodex unattended decision (mode: never; request: command execution; decision: denied): Codex rejected an escalation because the selected policy never asks for approval`, + stopReason: 'max-tokens', + }) + await run.dispose() + }) + it('flattens child exit and protocol failures after publication', async () => { const errors: string[] = [] - { + const outcomes: SubprocessOutcome[] = [ + { exitCode: 9, signal: null }, + { exitCode: null, signal: 'SIGABRT' }, + { exitCode: null, signal: null }, + ] + for (const outcome of outcomes) { const child = fakeChild({ exitOnTerminate: false }) const { run } = await publishRun(child, undefined, { onError: (error) => { errors.push(error.message) }, }) - child.settle({ exitCode: 9, signal: null }) - await expect(run.result).resolves.toEqual({ output: [], stopReason: 'error' }) - expect(errors.at(-1)).toContain('code 9') + child.settle(outcome) + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: expectedFailureDiagnostic('process', 'process-exit', { + outcome, + }), + stopReason: 'error', + }) + expect(errors.at(-1)).toBe( + `subagent-codex: ${expectedFailureDiagnostic('process', 'process-exit', { outcome })}`, + ) + await run.dispose().catch(() => {}) + } + { + const outcome = { exitCode: 17, signal: null } as const + const child = fakeChild({ exitOnTerminate: false }) + const { run, turnStart } = await publishRun(child, undefined, { + disposeGraceMs: 0.5, + }) + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + vi.spyOn(child.handle, 'waitForExit').mockImplementationOnce(async (signal?: AbortSignal) => { + expect(signal).toBeDefined() + child.settle(outcome) + return true + }) + child.fromChild.emit('end') + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: expectedFailureDiagnostic('process', 'process-exit', { + outcome, + }), + stopReason: 'error', + }) + await run.dispose().catch(() => {}) + } + { + const child = fakeChild({ exitOnTerminate: false }) + const { run, turnStart } = await publishRun(child) + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + setImmediate(() => { + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + codexErrorInfo: 'other', + })) + }) + child.settle({ exitCode: 17, signal: null }) + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: expectedFailureDiagnostic('turn', 'other', { + outcome: { exitCode: 17, signal: null }, + }), + stopReason: 'error', + }) + await run.dispose().catch(() => {}) + } + { + const child = fakeChild({ exitOnTerminate: false }) + const { run, turnStart } = await publishRun(child) + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.peer.send( + agentMessage('answer', 'final_answer'), + turnCompleted('completed'), + ) + child.fromChild.end() + child.settle({ exitCode: 17, signal: null }) + await expect(run.result).resolves.toEqual({ + output: [{ type: 'text', text: 'answer' }], + stopReason: 'completed', + }) await run.dispose().catch(() => {}) } { const child = fakeChild() const { run, turnStart } = await publishRun(child, undefined, { + disposeGraceMs: 10, onError: () => { throw new Error('diagnostic sink') }, }) child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) child.fromChild.end() - await expect(run.result).resolves.toEqual({ output: [], stopReason: 'error' }) + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: expectedFailureDiagnostic('turn', 'unknown'), + stopReason: 'error', + }) await run.dispose() } + { + const child = fakeChild() + const { run, turnStart } = await publishRun(child) + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.stderr.emit('error', new Error('stderr broke')) + child.peer.send(agentMessage('answer', 'final_answer'), turnCompleted('completed')) + await expect(run.result).resolves.toEqual({ + output: [{ type: 'text', text: 'answer' }], + stopReason: 'completed', + }) + await run.dispose() + expect(child.stderr.listenerCount('error')).toBe(0) + } + }) + + it('attaches a safe permission diagnostic when a published run fails', async () => { + const { child, run, turnStart } = await publishRun() + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + await nextTask() + child.peer.send({ + id: 'approval-diagnostic', + method: 'item/commandExecution/requestApproval', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + availableDecisions: ['cancel'], + command: 'cat /private/secret.txt', + }, + }) + expect(await child.peer.nextResponse('approval-diagnostic')).toMatchObject({ + result: { decision: 'cancel' }, + }) + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'SECRET_TOKEN in /private/secret.txt', + codexErrorInfo: 'other', + })) + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: `${expectedFailureDiagnostic('turn', 'other')}\nCodex unattended decision (mode: never; request: command approval; decision: cancelled): the provider does not grant interactive approval`, + stopReason: 'error', + }) + await run.dispose() + }) + + it('drains queued stderr before settling a failed published run', async () => { + hostStderrWrite.capture = true + hostStderrWrite.chunks.length = 0 + const { child, run, turnStart } = await publishRun() + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'fixture terminal failure', + codexErrorInfo: 'badRequest', + })) + setImmediate(() => { + child.stderr.write('approval policy is Never; reject command') + }) + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: `${expectedFailureDiagnostic('turn', 'badRequest')}\nCodex unattended decision (mode: never; request: command execution; decision: denied): Codex rejected an escalation because the selected policy never asks for approval`, + stopReason: 'error', + }) + await run.dispose() + hostStderrWrite.capture = false + }) + + it('forwards stderr while extracting only a fixed safe permission signature', async () => { + const child = fakeChild() + hostStderrWrite.capture = true + hostStderrWrite.chunks.length = 0 + const { run, turnStart } = await publishRun(child) + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.stderr.write('SECRET_TOKEN approval policy is Ne') + child.stderr.write('ver; reject command — /private/secret.txt') + child.stderr.emit('data', 'string stderr suffix') + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'fixture terminal failure', + codexErrorInfo: 'badRequest', + })) + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: `${expectedFailureDiagnostic('turn', 'badRequest')}\nCodex unattended decision (mode: never; request: command execution; decision: denied): Codex rejected an escalation because the selected policy never asks for approval`, + stopReason: 'error', + }) + expect(Buffer.concat(hostStderrWrite.chunks).toString()).toContain('SECRET_TOKEN') + expect(hostStderrWrite.chunks).toHaveLength(3) + await run.dispose() + expect(child.stderr.listenerCount('data')).toBe(0) + hostStderrWrite.capture = false + }) + + it('contains host stderr write failures without changing run settlement', async () => { + const child = fakeChild() + hostStderrWrite.capture = true + hostStderrWrite.failNext = true + const { run, turnStart } = await publishRun(child) + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + child.stderr.write('approval policy is Never; reject command') + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'fixture terminal failure', + codexErrorInfo: 'badRequest', + })) + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: `${expectedFailureDiagnostic('turn', 'badRequest')}\nCodex unattended decision (mode: never; request: command execution; decision: denied): Codex rejected an escalation because the selected policy never asks for approval`, + stopReason: 'error', + }) + await run.dispose() + hostStderrWrite.capture = false }) it('rejects before spawn when pre-aborted and rolls back startup failures', async () => { @@ -939,6 +1892,7 @@ describe('run lifecycle and quiescence', () => { request(undefined, controller.signal), { cwd: process.cwd(), + permissionMode: DEFAULT_CODEX_PERMISSION_MODE, env: {}, disposeGraceMs: 10, spawn, @@ -946,12 +1900,149 @@ describe('run lifecycle and quiescence', () => { )).rejects.toThrow('aborted before app-server startup') expect(spawn).not.toHaveBeenCalled() + const spawnFailure = startCodexRun(request(), { + cwd: process.cwd(), + permissionMode: DEFAULT_CODEX_PERMISSION_MODE, + env: {}, + disposeGraceMs: 10, + spawn: () => { throw new Error('SECRET_TOKEN spawn failure') }, + }) + await expect(spawnFailure) + .rejects.toThrow(expectedFailureDiagnostic('initialize', 'unknown')) + await expect(spawnFailure).rejects.not.toThrow('SECRET_TOKEN') + + const asyncSpawnFailureChild = fakeChild({ + pid: -1, + doneError: new Error('SECRET_TOKEN async spawn failure'), + }) + const asyncSpawnFailure = startCodexRun( + request(), + runSpec(asyncSpawnFailureChild), + ) + await expect(asyncSpawnFailure) + .rejects.toThrow(expectedFailureDiagnostic('initialize', 'unknown')) + await expect(asyncSpawnFailure).rejects.not.toThrow('SECRET_TOKEN') + expect(asyncSpawnFailureChild.terminate).not.toHaveBeenCalled() + const child = fakeChild() const starting = startCodexRun(request(), runSpec(child)) const initialize = await child.peer.nextMethod('initialize') child.peer.respond(initialize, null) - await expect(starting).rejects.toThrow('invalid initialize response') + await expect(starting) + .rejects.toThrow(expectedFailureDiagnostic('initialize', 'unknown')) + await expect(starting).rejects.not.toThrow('invalid initialize response') expect(child.terminate).toHaveBeenCalledTimes(1) + + const cleanupFailureChild = fakeChild({ + waitForExitError: new Error('SECRET_TOKEN wait failure'), + }) + const cleanupFailure = startCodexRun( + request(), + runSpec(cleanupFailureChild), + ) + const cleanupFailureInitialize = await cleanupFailureChild.peer + .nextMethod('initialize') + cleanupFailureChild.peer.respond(cleanupFailureInitialize, null) + const cleanupError: unknown = await cleanupFailure.then( + () => undefined, + (error: unknown) => error, + ) + expect(cleanupError).toBeInstanceOf(AggregateError) + expect(String(cleanupError)).toContain( + expectedFailureDiagnostic('initialize', 'unknown'), + ) + expect(String(cleanupError)).toContain(expectedFailureDiagnostic( + 'teardown', + 'unknown', + { outcome: { exitCode: 0, signal: null } }, + )) + expect(String(cleanupError)).not.toContain('SECRET_TOKEN') + + const cleanupRaceAbort = new AbortController() + const cleanupRaceChild = fakeChild({ exitOnTerminate: false }) + const cleanupRace = startCodexRun( + request(undefined, cleanupRaceAbort.signal), + runSpec(cleanupRaceChild), + ) + const cleanupRaceInitialize = await cleanupRaceChild.peer.nextMethod('initialize') + cleanupRaceChild.peer.respond(cleanupRaceInitialize, null) + await nextTask() + cleanupRaceAbort.abort(new Error('cancelled during cleanup')) + cleanupRaceChild.settle() + await expect(cleanupRace) + .rejects.toThrow('aborted before run publication') + + const threadChild = fakeChild() + const threadStarting = startCodexRun(request(), runSpec(threadChild)) + const threadInitialize = await threadChild.peer.nextMethod('initialize') + threadChild.peer.respond(threadInitialize, { userAgent: 'codex-cli 0.147.0' }) + await threadChild.peer.nextMethod('initialized') + const invalidThread = await threadChild.peer.nextMethod('thread/start') + threadChild.peer.respond(invalidThread, { thread: { id: '', ephemeral: true } }) + await expect(threadStarting) + .rejects.toThrow(expectedFailureDiagnostic('thread-start', 'unknown')) + await expect(threadStarting).rejects.not.toThrow('thread/start thread id') + + const exitedThreadChild = fakeChild({ exitOnTerminate: false }) + const exitedThreadStarting = startCodexRun( + request(), + runSpec(exitedThreadChild), + ) + const exitedThreadInitialize = await exitedThreadChild.peer.nextMethod('initialize') + exitedThreadChild.peer.respond(exitedThreadInitialize, { + userAgent: 'codex-cli 0.147.0', + }) + await exitedThreadChild.peer.nextMethod('initialized') + await exitedThreadChild.peer.nextMethod('thread/start') + exitedThreadChild.settle({ exitCode: null, signal: 'SIGABRT' }) + await expect(exitedThreadStarting).rejects.toThrow(expectedFailureDiagnostic( + 'thread-start', + 'unknown', + { outcome: { exitCode: null, signal: 'SIGABRT' } }, + )) + + const eofBeforeCloseChild = fakeChild({ exitOnTerminate: false }) + const eofBeforeCloseStarting = startCodexRun( + request(), + runSpec(eofBeforeCloseChild), + ) + const eofBeforeCloseInitialize = await eofBeforeCloseChild.peer + .nextMethod('initialize') + eofBeforeCloseChild.peer.respond(eofBeforeCloseInitialize, { + userAgent: 'codex-cli 0.147.0', + }) + await eofBeforeCloseChild.peer.nextMethod('initialized') + await eofBeforeCloseChild.peer.nextMethod('thread/start') + eofBeforeCloseChild.fromChild.emit('end') + setImmediate(() => { + eofBeforeCloseChild.settle({ exitCode: 23, signal: null }) + }) + await expect(eofBeforeCloseStarting).rejects.toThrow( + expectedFailureDiagnostic('thread-start', 'unknown', { + outcome: { exitCode: 23, signal: null }, + }), + ) + + const stderrChild = fakeChild() + const stderrStarting = startCodexRun(request(), runSpec(stderrChild)) + const stderrInitialize = await stderrChild.peer.nextMethod('initialize') + stderrChild.stderr.emit('error', new Error('startup stderr broke')) + stderrChild.peer.respond(stderrInitialize, { userAgent: 'codex-cli 0.147.0' }) + await stderrChild.peer.nextMethod('initialized') + const stderrThreadStart = await stderrChild.peer.nextMethod('thread/start') + stderrChild.peer.respond(stderrThreadStart, { + thread: { id: 'thread-1', ephemeral: true }, + }) + const stderrRun = await stderrStarting + const stderrTurnStart = await stderrChild.peer.nextMethod('turn/start') + stderrChild.peer.send( + { id: stderrTurnStart.id, result: { turn: { id: 'turn-1' } } }, + agentMessage('answer', 'final_answer'), + turnCompleted('completed'), + ) + await expect(stderrRun.result).resolves.toMatchObject({ stopReason: 'completed' }) + await stderrRun.dispose() + expect(stderrChild.stderr.listenerCount('error')).toBe(0) }) it('rolls back an abort that wins immediately after thread creation', async () => { @@ -965,36 +2056,33 @@ describe('run lifecycle and quiescence', () => { child.peer.respond(initialize, { userAgent: 'codex-cli 0.147.0' }) await child.peer.nextMethod('initialized') const threadStart = await child.peer.nextMethod('thread/start') + expect(threadStart.params).toEqual({ + cwd: process.cwd(), + ephemeral: true, + approvalPolicy: 'never', + }) child.peer.respond(threadStart, { thread: { id: 'thread-1', ephemeral: true } }) controller.abort('startup race') await expect(starting).rejects.toThrow('aborted before run publication') expect(child.terminate).toHaveBeenCalledTimes(1) }) - it('rolls back a subprocess done rejection during startup', async () => { - const child = fakeChild({ doneError: new Error('spawn observer failed') }) - const error: unknown = await startCodexRun(request(), runSpec(child)).then( - () => undefined, - (failure: unknown) => failure, - ) - expect(error).toBeInstanceOf(AggregateError) - if (!(error instanceof AggregateError)) { - throw new Error('expected startup and rollback failures') - } - expect(error.errors).toEqual([ - expect.objectContaining({ message: 'spawn observer failed' }), - expect.objectContaining({ message: 'spawn observer failed' }), - ]) - expect(child.terminate).toHaveBeenCalledTimes(1) - }) - it('keeps overlapping runs isolated', async () => { - const first = fakeChild() - const second = fakeChild() - const runs = await Promise.all([ - publishRun(first), - publishRun(second), - ]) + const initialStderrListeners = { + error: process.stderr.listenerCount('error'), + unpipe: process.stderr.listenerCount('unpipe'), + close: process.stderr.listenerCount('close'), + finish: process.stderr.listenerCount('finish'), + } + const runs = await Promise.all( + Array.from({ length: 6 }, () => publishRun(fakeChild())), + ) + expect({ + error: process.stderr.listenerCount('error'), + unpipe: process.stderr.listenerCount('unpipe'), + close: process.stderr.listenerCount('close'), + finish: process.stderr.listenerCount('finish'), + }).toEqual(initialStderrListeners) for (const [index, entry] of runs.entries()) { const id = `turn-${index + 1}` entry.child.peer.send( @@ -1004,14 +2092,64 @@ describe('run lifecycle and quiescence', () => { ) } const results = await Promise.all(runs.map(entry => entry.run.result)) - expect(results.map(result => result.output)).toEqual([ - [{ type: 'text', text: 'answer-1' }], - [{ type: 'text', text: 'answer-2' }], - ]) - expect(runs[0].run.id).not.toBe(runs[1].run.id) + expect(results.map(result => result.output)).toEqual( + Array.from({ length: 6 }, (_, index) => [ + { type: 'text', text: `answer-${index + 1}` }, + ]), + ) + expect(runs[0]!.run.id).not.toBe(runs[1]!.run.id) await Promise.all(runs.map(entry => entry.run.dispose())) }) + it('isolates permission modes and diagnostics across overlapping runs', async () => { + const first = await publishRun(fakeChild(), undefined, { + permissionMode: 'never', + }) + const second = await publishRun(fakeChild(), undefined, { + permissionMode: 'dangerously-bypass-approvals-and-sandbox', + }) + first.child.peer.respond(first.turnStart, { turn: { id: 'turn-never' } }) + second.child.peer.respond(second.turnStart, { turn: { id: 'turn-bypass' } }) + await nextTask() + first.child.peer.send({ + id: 'never-approval', + method: 'item/commandExecution/requestApproval', + params: { + threadId: 'thread-1', + turnId: 'turn-never', + availableDecisions: ['cancel'], + }, + }) + second.child.peer.send({ + id: 'bypass-elicitation', + method: 'mcpServer/elicitation/request', + params: { threadId: 'thread-1', turnId: null }, + }) + await Promise.all([ + first.child.peer.nextResponse('never-approval'), + second.child.peer.nextResponse('bypass-elicitation'), + ]) + first.child.peer.send(turnCompleted('failed', 'turn-never', 'thread-1', { + message: 'first failure', + codexErrorInfo: 'other', + })) + second.child.peer.send(turnCompleted('failed', 'turn-bypass', 'thread-1', { + message: 'second failure', + codexErrorInfo: 'other', + })) + await expect(first.run.result).resolves.toEqual({ + output: [], + diagnostic: `${expectedFailureDiagnostic('turn', 'other')}\nCodex unattended decision (mode: never; request: command approval; decision: cancelled): the provider does not grant interactive approval`, + stopReason: 'error', + }) + await expect(second.run.result).resolves.toEqual({ + output: [], + diagnostic: `${expectedFailureDiagnostic('turn', 'other')}\nCodex unattended decision (mode: dangerously-bypass-approvals-and-sandbox; request: MCP elicitation; decision: declined): the provider does not collect interactive MCP input`, + stopReason: 'error', + }) + await Promise.all([first.run.dispose(), second.run.dispose()]) + }) + it('uses the registered provider config and logs flattened errors', async () => { const ctx = new Context() await ctx.plugin(SubagentRuntime) @@ -1023,10 +2161,47 @@ describe('run lifecycle and quiescence', () => { warnings.push(String(message)) }) as typeof ctx.logger.warn await ctx.plugin(codex, { + providerName: 'codex-diagnostic', env: { OPENAI_API_KEY: 'fake' }, + permissionMode: 'approve-for-me', disposeGraceMs: 25, }) - const starting = ctx.subagents.start('codex', { + + const invalidCwdParent = { + id: 'parent-with-invalid-cwd', + session: { header: { cwd: 'relative/SECRET_TOKEN' } }, + } as unknown as Agent + const invalidCwdError: unknown = await ctx.subagents.start('codex-diagnostic', { + prompt: [{ type: 'text', text: 'task' }], + parent: invalidCwdParent, + signal: new AbortController().signal, + }).then( + () => undefined, + (error: unknown) => error, + ) + expect(invalidCwdError).toBeInstanceOf(Error) + if (!(invalidCwdError instanceof Error)) { + throw new Error('expected safe invalid-cwd failure') + } + expect(invalidCwdError.message).toContain( + expectedFailureDiagnostic('initialize', 'unknown'), + ) + expect(invalidCwdError.message).not.toContain('relative/SECRET_TOKEN') + expect(invalidCwdError.cause).toBeInstanceOf(Error) + expect((invalidCwdError.cause as Error).message) + .toContain('relative/SECRET_TOKEN') + expect(spawn).not.toHaveBeenCalled() + + const invalidCwdAbort = new AbortController() + invalidCwdAbort.abort(new Error('cancel invalid cwd startup')) + await expect(ctx.subagents.start('codex-diagnostic', { + prompt: [{ type: 'text', text: 'task' }], + parent: invalidCwdParent, + signal: invalidCwdAbort.signal, + })).rejects.toThrow('aborted before app-server startup') + expect(spawn).not.toHaveBeenCalled() + + const starting = ctx.subagents.start('codex-diagnostic', { prompt: [{ type: 'text', text: 'task' }], parent: fakeParent, signal: new AbortController().signal, @@ -1035,20 +2210,51 @@ describe('run lifecycle and quiescence', () => { child.peer.respond(initialize, { userAgent: 'codex-cli 0.147.0' }) await child.peer.nextMethod('initialized') const threadStart = await child.peer.nextMethod('thread/start') + expect(threadStart.params).toEqual({ + cwd: process.cwd(), + ephemeral: true, + approvalPolicy: 'on-request', + approvalsReviewer: 'auto_review', + sandbox: 'workspace-write', + }) child.peer.respond(threadStart, { thread: { id: 'thread-1', ephemeral: true } }) const run = await starting - await child.peer.nextMethod('turn/start') - child.settle({ exitCode: 1, signal: null }) - await expect(run.result).resolves.toMatchObject({ stopReason: 'error' }) + const turnStart = await child.peer.nextMethod('turn/start') + child.peer.respond(turnStart, { turn: { id: 'turn-1' } }) + await nextTask() + child.peer.send({ + id: 'provider-approval', + method: 'item/commandExecution/requestApproval', + params: { + threadId: 'thread-1', + turnId: 'turn-1', + availableDecisions: ['cancel'], + command: 'cat /private/secret.txt', + }, + }) + await child.peer.nextResponse('provider-approval') + child.peer.send(turnCompleted('failed', 'turn-1', 'thread-1', { + message: 'SECRET_TOKEN in /private/secret.txt', + codexErrorInfo: 'other', + })) + await expect(run.result).resolves.toEqual({ + output: [], + diagnostic: `${expectedFailureDiagnostic('turn', 'other')}\nCodex unattended decision (mode: approve-for-me; request: command approval; decision: cancelled): the provider does not grant interactive approval`, + stopReason: 'error', + }) expect(spawn).toHaveBeenCalledWith(expect.objectContaining({ env: { OPENAI_API_KEY: 'fake' }, graceMs: 25, cwd: process.cwd(), })) expect(warnings).toEqual([ - expect.stringContaining('subagent-codex: child run failed (error):'), + expect.stringContaining( + `subagent-codex "codex-diagnostic": child run failed (error): subagent-codex: ${expectedFailureDiagnostic('turn', 'other')}`, + ), ]) - await run.dispose().catch(() => {}) + expect(warnings.join('\n')).not.toContain('SECRET_TOKEN') + expect(warnings.join('\n')).not.toContain('/private/secret.txt') + await run.dispose() await ctx.fiber.dispose() }) }) @@ -1056,7 +2262,7 @@ describe('run lifecycle and quiescence', () => { describe('disposeCodexChild', () => { it('closes stdin, terminates, and waits for the managed tree', async () => { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) const end = vi.spyOn(child.toChild, 'end') await disposeCodexChild(wire, child.handle) expect(end).toHaveBeenCalled() @@ -1067,7 +2273,7 @@ describe('disposeCodexChild', () => { it('does not finish disposal before the managed tree exits', async () => { const child = fakeChild({ exitOnTerminate: false }) - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) let disposed = false const disposal = disposeCodexChild(wire, child.handle).then(() => { disposed = true @@ -1081,7 +2287,7 @@ describe('disposeCodexChild', () => { it('contains a concurrently closed stdin error', async () => { const child = fakeChild() - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) vi.spyOn(child.toChild, 'end').mockImplementation(() => { throw new Error('already closed') }) @@ -1094,27 +2300,44 @@ describe('disposeCodexChild', () => { pid: -1, doneError: new Error('spawn failed'), }) - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) + const wire = defaultWire(child) await expect(disposeCodexChild(wire, child.handle)) .resolves.toBeUndefined() expect(child.terminate).not.toHaveBeenCalled() expect(child.waitForExit).not.toHaveBeenCalled() }) - it('reports direct-child observer failure and accepts absent stdin', async () => { - { - const child = fakeChild({ - doneError: new Error('close observer failed'), - }) - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) - await expect(disposeCodexChild(wire, child.handle)) - .rejects.toThrow('close observer failed') - } - { - const child = fakeChild() - const handle = { ...child.handle, stdin: undefined } - const wire = new CodexAppServerWire(child.handle.stdout!, child.handle.stdin!) - await expect(disposeCodexChild(wire, handle)).resolves.toBeUndefined() - } + it('reports tree-wait failure with safe teardown facts', async () => { + const child = fakeChild({ + waitForExitError: new Error('SECRET_TOKEN wait failure'), + }) + const wire = defaultWire(child) + const disposal = disposeCodexChild(wire, child.handle) + await expect(disposal).rejects.toThrow(expectedFailureDiagnostic( + 'teardown', + 'unknown', + { outcome: { exitCode: 0, signal: null } }, + )) + await expect(disposal).rejects.not.toThrow('SECRET_TOKEN') + }) + + it('does not wait for a pending process outcome after tree observation fails', async () => { + const child = fakeChild({ + exitOnTerminate: false, + waitForExitError: new Error('SECRET_TOKEN wait failure'), + }) + const wire = defaultWire(child) + let disposalError: unknown + const disposal = disposeCodexChild(wire, child.handle).catch( + (error: unknown) => { disposalError = error }, + ) + await nextTask() + expect(disposalError).toBeInstanceOf(Error) + expect(String(disposalError)).toContain( + expectedFailureDiagnostic('teardown', 'unknown'), + ) + expect(String(disposalError)).not.toContain('SECRET_TOKEN') + child.settle() + await disposal }) }) diff --git a/packages/subagent/subagent-dsh-sdk/README.i18n.yaml b/packages/subagent/subagent-dsh-sdk/README.i18n.yaml index dc98bdc655..8946de4236 100644 --- a/packages/subagent/subagent-dsh-sdk/README.i18n.yaml +++ b/packages/subagent/subagent-dsh-sdk/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/subagent-dsh-sdk/README.md -README.md: 8d20b44dacd773345986e2a2fc0e6aca9470bf2c -README.zh.md: 4c9117cad610276ca9d12c914a44dcbaecbbf26e +README.md: 10f437f1618d006f58ad37701e6fde7944f5c1fb +README.zh.md: d3237d9a0b36ce63c9eacf85bfe7cff947acd9e0 diff --git a/packages/subagent/subagent-dsh-sdk/README.md b/packages/subagent/subagent-dsh-sdk/README.md index 8d20b44dac..10f437f161 100644 --- a/packages/subagent/subagent-dsh-sdk/README.md +++ b/packages/subagent/subagent-dsh-sdk/README.md @@ -2,13 +2,13 @@ English | [中文](README.zh.md) -The SDK provider runs each subagent as a complete DeepSeek Harness runtime in a fresh subprocess, driven over stdio JSON-RPC through the [TypeScript SDK client](../../sdk/client/README.md). It is the second out-of-process backend beside [`subagent-acp`](../subagent-acp/README.md), differing in the wire and the child contract: the ACP backend drives any Agent Client Protocol agent; this backend drives specifically a harness SDK runtime (`dsh-jsonrpc-agent` bin or packaged executable), so the child is a full peer harness — own `cordis.yml`-decided composition, session persistence, model route, and tools. +The SDK provider runs each subagent as a complete DeepSeek Harness runtime in a fresh subprocess, driven over stdio JSON-RPC through the [TypeScript SDK client](../../sdk/client/README.md). It is the second out-of-process backend beside [`subagent-acp`](../subagent-acp/README.md), differing in the wire and child application: the ACP backend drives any Agent Client Protocol agent; this backend launches the same-version `dsh --profile sdk` application, so the child has its own profile-and-patch composition, session persistence, model route, and tools. ## Start and ownership `start(request)` resolves the child's working directory, spawns the runtime through `DeepSeekHarness`, and completes the `initialize` handshake (with the configured `provider`/`model` route and optional `maxTokens` output cap) before it fulfills. Fulfillment therefore means the child runtime is ready and ownership has transferred to the caller. A spawn, handshake, or pre-publication cancellation failure rejects only after the subprocess has been reaped; a working-directory resolution failure rejects before anything is spawned. -The working directory resolves exactly like the ACP backend, through the seam's shared out-of-process helpers ([`dsh-subagent`](../subagent/README.md)): the configured `cwd` override when set (validated once at load), else the delegating parent session's cwd — never the server process's own cwd. The resolved path becomes the child process cwd and the workspace cwd of its SDK session. +The working directory resolves exactly like the ACP backend, through the seam's shared out-of-process helpers ([`dsh-subagent`](../subagent/README.md)): the configured `cwd` override when set (validated once at load), else the delegating parent session's cwd — never the server process's own cwd. The resolved path becomes the child process cwd and the workspace cwd of its SDK session. `dshHome` is separately required as an absolute path so a nested runtime cannot accidentally share its parent's profiles, plugin installation, or session storage. The returned run id is minted in the parent namespace; the child runtime's session id exists only inside the child process. After publication the provider owns one SDK activity and reads the child's answer from its session events: the last complete non-empty `assistant/message` (an empty-content message that records usage is skipped), or the accumulated `text-delta` stream when no such message exists. Partial output remains available after cancellation or an error. @@ -27,13 +27,15 @@ The provider advertises no start-time capabilities (`outputSchema`/`depthLimit`/ | Key | Default | Meaning | |---|---|---| | `providerName` | `dsh-sdk` | Registry name on `ctx.subagents`. | -| `command` | required | Executable spawned per run (the child runtime bin or packaged exe). | -| `args` | `[]` | Command arguments (typically the child's `cordis.yml` path). | +| `dshBin` | same-version SDK dependency | Explicit dsh CLI module override. A relative path resolves at plugin load and must name an existing file. Ordinary deployments omit it. | +| `profile` | `sdk` | Named dsh profile launched for each child. The profile must include the SDK app. | +| `patches` | `[]` | Ordered per-launch profile patch files. Relative paths resolve at plugin load, and every path must name an existing file. | +| `dshHome` | required | Absolute isolated dsh home for the nested runtime's profiles, installed plugins, and session data. | | `cwd` | parent session cwd | Working-directory override; same validation as [`subagent-acp`](../subagent-acp/README.md). | | `provider` | `deepseek-official` | Provider route sent in the child's `initialize`. | | `model` | `deepseek-v4-flash` | Model sent in the child's `initialize`. | | `maxTokens` | adapter/provider route default | Per-request output-token cap sent in the child's `initialize`; it applies to the child root agent and its in-process descendants. | -| `env` | `{}` | Explicit child environment layered over a credential-scrubbed parent environment (e.g. the child's own `DEEPSEEK_API_KEY`, or `DSH_CORDIS_CONFIG`). | +| `env` | `{}` | Explicit child environment layered over a credential-scrubbed parent environment, such as the child's own `DEEPSEEK_API_KEY`. | | `shutdownTimeoutMs` | `1000` | Bound on the protocol `shutdown` exchange during dispose. | | `disposeEofGraceMs` | `6000` | Grace after stdin EOF before platform termination. | | `disposeGraceMs` | `3000` | Exit-confirmation grace after termination; POSIX also waits this long after SIGTERM before SIGKILL. | @@ -43,8 +45,9 @@ The provider advertises no start-time capabilities (`outputSchema`/`depthLimit`/ name: '@deepseek-ai/dsh-subagent-dsh-sdk' config: providerName: dsh-sdk - command: node - args: ['./packages/examples/jsonrpc-demo/lib/bin.js', './examples/jsonrpc-agent/cordis.yml'] + profile: sdk + patches: ['./profiles/research-child.cordis.yml'] + dshHome: !!js dshHomePath('children') maxTokens: 49152 env: DEEPSEEK_API_KEY: !!js process.env.DEEPSEEK_API_KEY @@ -92,6 +95,6 @@ Append-only; newly visible content follows the reusable request prefix and does ## Known Limitations and Deferred Work - **A fresh runtime process per run** — no pooling; a harness runtime boots a full plugin tree, so per-run spawn cost is higher than the ACP backend's typical child. -- **No optional start-time capabilities** — the parent cannot enforce `outputSchema`, depth, tool filters, or persona inside the child process; configure the child's own `cordis.yml` instead. +- **No optional start-time capabilities** — the parent cannot enforce `outputSchema`, depth, tool filters, or persona inside the child process; configure the selected child profile and its ordered patches instead. - **The child's transcript stays in the child's own session root** — the parent log records only the delegation tool call/result (the seam's child-isolation rule); the streamed `session.event` channel is consumed for output extraction, not bridged into the parent log. - **Local child processes only** — the resolved cwd is a local path; a remote runtime would need its own backend. diff --git a/packages/subagent/subagent-dsh-sdk/README.zh.md b/packages/subagent/subagent-dsh-sdk/README.zh.md index 4c9117cad6..d3237d9a0b 100644 --- a/packages/subagent/subagent-dsh-sdk/README.zh.md +++ b/packages/subagent/subagent-dsh-sdk/README.zh.md @@ -2,13 +2,13 @@ [English](README.md) | 中文 -SDK 提供方会在全新的子进程中把每个 subagent 作为完整的 DeepSeek Harness 运行时运行,并经由 [TypeScript SDK 客户端](../../sdk/client/README.md) 通过 stdio JSON-RPC 驱动。它是 [`subagent-acp`](../subagent-acp/README.md) 之外的第二个进程外后端,差异在协议格式(wire format)和子进程约定:ACP(Agent Client Protocol)后端能驱动任何 Agent Client Protocol agent(智能体);本后端专门驱动 harness SDK 运行时(`dsh-jsonrpc-agent` bin 或打包后的可执行文件),因此子进程是一个完整的对等 harness,拥有由 `cordis.yml` 决定的组合、会话持久化、模型路由和工具。 +SDK 提供方会在全新的子进程中把每个 subagent 作为完整的 DeepSeek Harness 运行时运行,并经由 [TypeScript SDK 客户端](../../sdk/client/README.zh.md) 通过 stdio JSON-RPC 驱动。它是 [`subagent-acp`](../subagent-acp/README.zh.md) 之外的第二个进程外后端,差异在协议格式(wire format)和子应用:ACP(Agent Client Protocol)后端能驱动任何 Agent Client Protocol agent(智能体);本后端启动同版本的 `dsh --profile sdk` 应用,因此子进程拥有自己的 profile 与 patch 组合、会话持久化、模型路由和工具。 ## 启动与所有权 `start(request)` 先解析子进程工作目录,通过 `DeepSeekHarness` spawn 运行时,并在履行前完成 `initialize` 握手(携带配置的 `provider`/`model` 路由及可选的 `maxTokens` 输出上限)。因此,履行意味着子运行时已就绪、所有权已移交给调用方。spawn、握手或发布前取消失败时,只会在子进程被回收后拒绝;工作目录解析失败则会在尚未 spawn 任何内容时拒绝。 -工作目录的解析与 ACP 后端完全一致,并使用 seam 共享的进程外辅助工具([`dsh-subagent`](../subagent/README.md)):设置了 `cwd` 覆盖值时使用该值(加载时校验一次),否则使用发起委派的父会话 cwd,绝不使用服务器进程自身的 cwd。解析出的路径同时成为子进程 cwd 和其 SDK 会话的工作区 cwd。 +工作目录的解析与 ACP 后端完全一致,并使用 seam 共享的进程外辅助工具([`dsh-subagent`](../subagent/README.zh.md)):设置了 `cwd` 覆盖值时使用该值(加载时校验一次),否则使用发起委派的父会话 cwd,绝不使用服务器进程自身的 cwd。解析出的路径同时成为子进程 cwd 和其 SDK 会话的工作区 cwd。`dshHome` 必须另外指定为绝对路径,使嵌套运行时不会意外共享父运行时的 profile、插件安装或会话存储。 返回的 run id 在父级命名空间中生成;子运行时的会话 id 只存在于子进程内部。发布后,提供方拥有一段 SDK 活动,并从子会话事件中读取答案:最后一条完整且非空的 `assistant/message`(记录 usage 的空内容消息会被跳过);若没有这类消息,则取累积的 `text-delta` 流。取消或发生错误后,部分输出仍然可用。 @@ -27,13 +27,15 @@ Provider 不宣告任何启动期能力(`outputSchema`/`depthLimit`/`toolFilte | 键 | 默认 | 含义 | |---|---|---| | `providerName` | `dsh-sdk` | `ctx.subagents` 上的注册名。 | -| `command` | 必填 | 每次运行时 spawn 的可执行文件(子运行时 bin 或打包后的可执行文件)。 | -| `args` | `[]` | 命令参数(通常是子进程的 `cordis.yml` 路径)。 | -| `cwd` | 父会话 cwd | 工作目录覆盖;校验规则与 [`subagent-acp`](../subagent-acp/README.md) 相同。 | +| `dshBin` | SDK 同版本依赖 | 显式 dsh CLI 模块覆盖;相对路径在 plugin 加载时解析,且必须指向已有文件;普通部署应省略。 | +| `profile` | `sdk` | 为每个子进程启动的具名 dsh profile;该 profile 必须包含 SDK app。 | +| `patches` | `[]` | 有序的逐次启动 profile patch 文件;相对路径在 plugin 加载时解析,每个路径都必须指向已有文件。 | +| `dshHome` | 必填 | 嵌套运行时的绝对隔离 dsh home,用于 profile、已安装插件和会话数据。 | +| `cwd` | 父会话 cwd | 工作目录覆盖;校验规则与 [`subagent-acp`](../subagent-acp/README.zh.md) 相同。 | | `provider` | `deepseek-official` | 写入子进程 `initialize` 的提供方路由。 | | `model` | `deepseek-v4-flash` | 写入子进程 `initialize` 的模型。 | | `maxTokens` | 适配器/提供方路由默认值 | 写入子进程 `initialize` 的单次请求输出 token 上限;对子运行时的根 agent 及其进程内后代生效。 | -| `env` | `{}` | 在凭据擦除后的父环境之上叠加的显式子环境(例如子进程自己的 `DEEPSEEK_API_KEY`,或 `DSH_CORDIS_CONFIG`)。 | +| `env` | `{}` | 在凭据擦除后的父环境之上叠加的显式子环境,例如子进程自己的 `DEEPSEEK_API_KEY`。 | | `shutdownTimeoutMs` | `1000` | dispose 期间协议 `shutdown` 交换的时限。 | | `disposeEofGraceMs` | `6000` | stdin EOF 之后、平台终止之前的宽限。 | | `disposeGraceMs` | `3000` | 终止后的退出确认窗口;POSIX 在 SIGTERM 之后、SIGKILL 之前也等待同样时长。 | @@ -43,8 +45,9 @@ Provider 不宣告任何启动期能力(`outputSchema`/`depthLimit`/`toolFilte name: '@deepseek-ai/dsh-subagent-dsh-sdk' config: providerName: dsh-sdk - command: node - args: ['./packages/examples/jsonrpc-demo/lib/bin.js', './examples/jsonrpc-agent/cordis.yml'] + profile: sdk + patches: ['./profiles/research-child.cordis.yml'] + dshHome: !!js dshHomePath('children') maxTokens: 49152 env: DEEPSEEK_API_KEY: !!js process.env.DEEPSEEK_API_KEY @@ -55,9 +58,9 @@ Provider 不宣告任何启动期能力(`outputSchema`/`depthLimit`/`toolFilte ## 进程边界 -子进程环境以 [`dsh-subprocess`](../../subprocess/README.md) seam 的 `scrubbedParentEnv()` 为基础,先移除疑似凭据和名称为 `DSH_*` 的环境变量,再合并显式 `config.env` 值。子进程由 SDK 客户端 spawn,而不是经由 `ctx.subprocess` spawn(这是 subprocess README 中记录的 SDK 托管传输例外),因此本后端会自行执行环境清理。JSON-RPC 协议格式才是真正的序列化边界。 +子进程环境以 [`dsh-subprocess`](../../subprocess/README.zh.md) seam 的 `scrubbedParentEnv()` 为基础,先移除疑似凭据和名称为 `DSH_*` 的环境变量,再合并显式 `config.env` 值。子进程由 SDK 客户端 spawn,而不是经由 `ctx.subprocess` spawn(这是 subprocess README 中记录的 SDK 托管传输例外),因此本后端会自行执行环境清理。JSON-RPC 协议格式才是真正的序列化边界。 -本包没有默认导出。否则 Cordis loader 解包会隐藏具名 `inject` 元数据;见[事故复盘(postmortem)0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md)。 +本包没有默认导出。否则 Cordis loader 解包会隐藏具名 `inject` 元数据;见[事故复盘(postmortem)0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.zh.md)。 ## 模型体验 @@ -92,6 +95,6 @@ Provider 不宣告任何启动期能力(`outputSchema`/`depthLimit`/`toolFilte ## 已知限制与暂缓事项 - **每次运行都使用全新的运行时进程**:不使用进程池;harness 运行时需要启动完整的插件树,因此每次运行的 spawn 成本高于 ACP 后端通常使用的子进程。 -- **不支持可选的启动时能力**:父级无法在子进程内强制执行 `outputSchema`、深度限制、工具过滤或 persona;应改为配置子进程自身的 `cordis.yml`。 +- **不支持可选的启动时能力**:父级无法在子进程内强制执行 `outputSchema`、深度限制、工具过滤或 persona;应改为配置所选子 profile 及其有序 patch。 - **子进程的 transcript(文本记录)保留在其自身的会话根目录中**:父级日志只记录委派工具调用/结果(seam 的子级隔离规则);流式 `session.event` 通道只用于提取输出,不会桥接到父级日志中。 - **仅支持本地子进程**:解析出的 cwd 是本地路径;远程运行时需要独立的后端。 diff --git a/packages/subagent/subagent-dsh-sdk/package.json b/packages/subagent/subagent-dsh-sdk/package.json index f7eac52a8e..35617aa3b6 100644 --- a/packages/subagent/subagent-dsh-sdk/package.json +++ b/packages/subagent/subagent-dsh-sdk/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subagent-dsh-sdk", "description": "Out-of-process SDK subagent backend: drives a child DeepSeek Harness runtime subprocess over stdio JSON-RPC through the TypeScript SDK client", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subagent/subagent-dsh-sdk/src/index.ts b/packages/subagent/subagent-dsh-sdk/src/index.ts index fab89a36f0..7530e104d2 100644 --- a/packages/subagent/subagent-dsh-sdk/src/index.ts +++ b/packages/subagent/subagent-dsh-sdk/src/index.ts @@ -1,6 +1,6 @@ /** * Out-of-process SDK subagent backend. Each child is a complete DeepSeek - * Harness runtime in its own process — own `cordis.yml`-decided composition, + * Harness runtime in its own process — own named profile and patch composition, * session, model route, and tools — driven over stdio JSON-RPC through the * TypeScript SDK client, so it shares no Cordis context and advertises no * parent-enforced start capabilities; the ONE thing it reads off @@ -11,6 +11,8 @@ */ import type { Context } from '@deepseek-ai/cordis' +import { statSync } from 'node:fs' +import { isAbsolute, resolve } from 'node:path' import z from '@deepseek-ai/schemastery' import type { SubagentCapabilities, SubagentProvider, SubagentStartRequest } from '@deepseek-ai/dsh-subagent' import { assertPositiveFinite, NO_START_CAPABILITIES, resolveChildCwd, validateConfiguredCwd } from '@deepseek-ai/dsh-subagent' @@ -29,10 +31,14 @@ export const inject = ['subagents'] export interface Config { /** Provider name on `ctx.subagents` (default `dsh-sdk`). */ providerName: string - /** The executable to spawn for each run (the child runtime bin or packaged exe). */ - command: string - /** Arguments passed to {@link command} (typically the child's `cordis.yml` path). */ - args: string[] + /** Explicit dsh CLI module, resolved and checked at plugin load; omission uses the SDK dependency. */ + dshBin?: string + /** Named child profile (default `sdk`). */ + profile: string + /** Ordered per-launch profile patch files, resolved and checked at plugin load. */ + patches: string[] + /** Absolute isolated Harness home for every nested child process. */ + dshHome: string /** * Working directory override for the child process and its SDK session * workspace. Must be non-empty; a relative path resolves against the @@ -50,8 +56,7 @@ export interface Config { maxTokens?: number /** * Extra environment variables for the child process — e.g. the child - * runtime's own `DEEPSEEK_API_KEY`, or `DSH_CORDIS_CONFIG` naming its - * config. Forwarded on top of a credential-scrubbed copy of the parent + * runtime's own `DEEPSEEK_API_KEY`. Forwarded on top of a credential-scrubbed copy of the parent * env, so an explicit key here reaches the child while ambient secrets do * not leak implicitly. */ @@ -70,8 +75,10 @@ export interface Config { export const Config: z = z.object({ providerName: z.string().default('dsh-sdk'), - command: z.string().required(), - args: z.array(z.string()).default([]), + dshBin: z.string(), + profile: z.string().default('sdk'), + patches: z.array(z.string()).default([]), + dshHome: z.string().required(), cwd: z.string(), provider: z.string().default('deepseek-official'), model: z.string().default('deepseek-v4-flash'), @@ -83,7 +90,18 @@ export const Config: z = z.object({ }) /** The shape after schemastery applied the defaults (`cwd` and `maxTokens` have none). */ -type ResolvedConfig = Required> & Pick +type ResolvedConfig = Required> & Pick + +/** Resolve one configured runtime file against the harness launch directory and require a regular file. */ +function resolveConfiguredFile(field: string, value: string): string { + const path = resolve(value) + try { + if (statSync(path).isFile()) return path + } catch { + // The diagnostic below owns missing, inaccessible, and non-file paths uniformly. + } + throw new TypeError(`subagent-dsh-sdk ${field} must name an existing file: ${path}`) +} /** * The SDK provider. Advertises NO start-time capabilities: an out-of-process @@ -99,8 +117,10 @@ class SdkSubagentProvider implements SubagentProvider { start(request: SubagentStartRequest) { const spec: SdkRunSpec = { - command: this.config.command, - args: this.config.args, + ...this.config.dshBin === undefined ? {} : { dshBin: this.config.dshBin }, + profile: this.config.profile, + patches: this.config.patches, + dshHome: this.config.dshHome, cwd: resolveChildCwd('subagent-dsh-sdk', this.config.cwd, request.parent.session.header.cwd), provider: this.config.provider, model: this.config.model, @@ -128,11 +148,17 @@ export function apply(ctx: Context, config: Config): void { if (resolved.maxTokens !== undefined && (!Number.isSafeInteger(resolved.maxTokens) || resolved.maxTokens <= 0)) { throw new TypeError('subagent-dsh-sdk maxTokens must be a positive safe integer') } + if (!isAbsolute(resolved.dshHome)) throw new TypeError('subagent-dsh-sdk dshHome must be an absolute path') + const launchPaths: ResolvedConfig = { + ...resolved, + patches: resolved.patches.map((path, index) => resolveConfiguredFile(`patches[${String(index)}]`, path)), + ...resolved.dshBin === undefined ? {} : { dshBin: resolveConfiguredFile('dshBin', resolved.dshBin) }, + } // Interpret a relative configured cwd against the harness launch directory // ONCE, at load, and fail a misconfigured directory here — not per start. const configuredCwd = validateConfiguredCwd('subagent-dsh-sdk', resolved.cwd) const validated: ResolvedConfig = configuredCwd === undefined - ? resolved - : { ...resolved, cwd: configuredCwd } + ? launchPaths + : { ...launchPaths, cwd: configuredCwd } ctx.subagents.registerProvider(new SdkSubagentProvider(validated.providerName, ctx, validated)) } diff --git a/packages/subagent/subagent-dsh-sdk/src/run.ts b/packages/subagent/subagent-dsh-sdk/src/run.ts index 194ce3badf..4849460aa4 100644 --- a/packages/subagent/subagent-dsh-sdk/src/run.ts +++ b/packages/subagent/subagent-dsh-sdk/src/run.ts @@ -1,18 +1,17 @@ /** * Fresh-process SDK subagent client. Drives one child DeepSeek Harness * runtime over stdio JSON-RPC through `@deepseek-ai/dsh-sdk-client` and owns - * cancellation and quiescent disposal. Structure mirrors the ACP backend - * (`@deepseek-ai/dsh-subagent-acp`): publish after the child handshake, - * flatten child failures into stop reasons, tear down to quiescence. The - * child is spawned BY the SDK client rather than through `ctx.subprocess` — - * the subprocess seam's documented exception for SDK-managed transports — - * so this driver applies the seam's shared env scrub itself. + * cancellation and quiescent disposal. It publishes after the child + * handshake, maps child failures to stop reasons, and tears down to + * quiescence. The SDK client spawns the child rather than using + * `ctx.subprocess` — the subprocess seam's documented exception for + * SDK-managed transports — so this driver applies the seam's shared env scrub. * * @module @deepseek-ai/dsh-subagent-dsh-sdk/run */ import { randomUUID } from 'node:crypto' -import { DeepSeekHarness, type HarnessNotification } from '@deepseek-ai/dsh-sdk-client' +import { DeepSeekHarness, type DeepSeekHarnessOptions, type HarnessNotification } from '@deepseek-ai/dsh-sdk-client' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import { SessionId, type SessionEvent, type TurnEndReason } from '@deepseek-ai/dsh-session' import type { SubagentResult, SubagentRun, SubagentStartRequest, SubagentStopReason } from '@deepseek-ai/dsh-subagent' @@ -21,10 +20,14 @@ import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' /** Resolved spawn spec for an SDK runtime child process (no defaults — see Config). */ export interface SdkRunSpec { - /** The executable to spawn (the child runtime — a `dsh-jsonrpc-agent` bin or packaged exe). */ - command: string - /** Arguments passed to {@link command} (typically the child's `cordis.yml` path). */ - args: string[] + /** Explicit dsh CLI module; omission resolves the SDK client's same-version dependency. */ + dshBin?: string + /** Named child profile. */ + profile: string + /** Ordered per-launch profile patch files. */ + patches: string[] + /** Absolute isolated Harness home for the nested runtime. */ + dshHome: string /** * Absolute working directory for the child process AND the workspace cwd * of its SDK session. The provider resolves it before this spec exists: @@ -39,7 +42,7 @@ export interface SdkRunSpec { maxTokens?: number /** * Extra environment variables to ADD for the child (e.g. the child - * runtime's own `DEEPSEEK_API_KEY`, or `DSH_CORDIS_CONFIG`). Merged after + * runtime's own `DEEPSEEK_API_KEY`). Merged after * the seam's `scrubbedParentEnv()` base, so an explicit credential or * current `DSH_*` fact survives while ambient namesakes never leak. */ @@ -68,6 +71,11 @@ export const DEFAULT_DISPOSE_GRACE_MS = 3_000 /** Default bound on the protocol `shutdown` exchange during dispose. */ export const DEFAULT_SHUTDOWN_TIMEOUT_MS = 1_000 +/** Runtime constructor seam replaced only by package-local fake-runtime tests. */ +export const internals: { createHarness(options: DeepSeekHarnessOptions): DeepSeekHarness } = { + createHarness: options => new DeepSeekHarness(options), +} + /** * Map a child turn-end reason to a harness {@link SubagentStopReason}. * @param reason - the owned child run's final durable turn reason, or @@ -105,7 +113,7 @@ function toError(value: unknown): Error { * Child failures resolve through the run result; startup failures reject * after process reap. Disposal shuts the runtime down and reaps it. * @param request - the start request; its signal is the cancellation channel. - * @param spec - the resolved spawn spec: command/args/cwd, the child's + * @param spec - the resolved spawn spec: profile/patches/home/cwd, the child's * provider/model route, env, timeouts, and the optional error sink. * @returns the ready run handle for the child subprocess. */ @@ -115,16 +123,16 @@ export async function startSdkRun(request: SubagentStartRequest, spec: SdkRunSpe // (minted below, private to the wire) exists only inside the child process. const id = SessionId(randomUUID()) - const harness = new DeepSeekHarness({ - launch: { - command: spec.command, - args: spec.args, - cwd: spec.cwd, - env: { ...scrubbedParentEnv(), ...spec.env }, - shutdownTimeoutMs: spec.shutdownTimeoutMs, - disposeEofGraceMs: spec.disposeEofGraceMs, - disposeGraceMs: spec.disposeGraceMs, - }, + const harness = internals.createHarness({ + ...spec.dshBin === undefined ? {} : { dshBin: spec.dshBin }, + profile: spec.profile, + patches: spec.patches, + dshHome: spec.dshHome, + processCwd: spec.cwd, + env: { ...scrubbedParentEnv(), ...spec.env }, + shutdownTimeoutMs: spec.shutdownTimeoutMs, + disposeEofGraceMs: spec.disposeEofGraceMs, + disposeGraceMs: spec.disposeGraceMs, cwd: spec.cwd, provider: spec.provider, model: spec.model, @@ -152,7 +160,7 @@ export async function startSdkRun(request: SubagentStartRequest, spec: SdkRunSpe ]) // Defensive: an abort() is a macrotask and no user callback runs inside // the microtask drain between handshake fulfillment and this continuation, - // so the recheck is not schedulable today; it guards future reentrancy. + // so current callback ordering cannot schedule the recheck; it guards future reentrancy. /* v8 ignore next */ if (flags.cancelled) throw new Error('subagent cancelled before the SDK child initialized') } catch (error: unknown) { diff --git a/packages/subagent/subagent-dsh-sdk/tests/loader-composition.e2e.ts b/packages/subagent/subagent-dsh-sdk/tests/loader-composition.e2e.ts index 3c07d84247..6dc302902b 100644 --- a/packages/subagent/subagent-dsh-sdk/tests/loader-composition.e2e.ts +++ b/packages/subagent/subagent-dsh-sdk/tests/loader-composition.e2e.ts @@ -9,19 +9,20 @@ * keyless tier applies (the with-key tier lives in subagent-sdk.e2e.ts). */ -import { realpathSync } from 'node:fs' -import { readFile, readdir } from 'node:fs/promises' +import { existsSync, realpathSync } from 'node:fs' +import { mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' import { join } from 'node:path' -import { fileURLToPath } from 'node:url' +import { fileURLToPath, pathToFileURL } from 'node:url' import { describe, expect, it } from 'vitest' import { type SessionEvent } from '@deepseek-ai/dsh-session' -import { resolveExampleLaunch, runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' +import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' -const fixtureDir = new URL('../../../../examples/jsonrpc-agent/tests/fixtures/subagent/subagent-dsh-sdk/', import.meta.url) +const fixtureDir = new URL('../../../../examples/python-sdk-agent/tests/fixtures/subagent/subagent-dsh-sdk/', import.meta.url) const driver = fileURLToPath(new URL('driver.ts', fixtureDir)) const configPath = fileURLToPath(new URL('cordis.yml', fixtureDir)) const childConfigPath = fileURLToPath(new URL('child.cordis.yml', fixtureDir)) -const runtimeBin = fileURLToPath(new URL('../../../../packages/examples/jsonrpc-demo/src/bin.ts', import.meta.url)) +const childMockPath = fileURLToPath(new URL('child-mock-llm.ts', fixtureDir)) const repoTsconfig = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) async function jsonlFiles(dir: string): Promise { @@ -41,66 +42,67 @@ async function sessionEvents(log: string): Promise { describe('SDK subagent cwd inheritance through a real cordis.yml', () => { it('runs the child runtime in the parent session workspace', async () => { - // The child launch honors the same src/lib mode as the driving harness, - // per the shared example-launch resolver (testing policy forbids - // hand-written `--import tsx` argv for example subprocesses). - const childLaunch = resolveExampleLaunch({ - srcBin: runtimeBin, - configArgs: [childConfigPath], - tsconfigPath: repoTsconfig, - }) + const childHome = await mkdtemp(join(tmpdir(), 'dsh-sdk-subagent-home-')) + const childPatch = join(childHome, 'child.cordis.yml') + await writeFile(childPatch, (await readFile(childConfigPath, 'utf8')) + .replace("'./child-mock-llm.ts'", JSON.stringify(pathToFileURL(childMockPath).href))) let events: SessionEvent[] = [] let childEvents: SessionEvent[] = [] let workspace = '' - const { stderr } = await runLoaderSmoke({ - label: 'dsh-sdk-subagent cwd composition smoke', - tempDirPrefix: 'dsh-sdk-subagent-cwd-e2e-', - binScript: driver, - libBinScript: driver, - configPath, - tsconfigPath: repoTsconfig, - // Two complete harness runtimes boot in sequence (driver, then the SDK - // child); from-source tsx boots under load need more than the default - // 30s window. - processTimeoutMs: 120_000, - env: { - DSH_TEST_CHILD_COMMAND: childLaunch.command, - DSH_TEST_CHILD_ARGS: JSON.stringify(childLaunch.args), - DSH_TEST_CHILD_ENV: JSON.stringify({ - ...Object.fromEntries(Object.entries(childLaunch.env).filter(([, value]) => value !== undefined)), - }), - }, - inspect: async (cwd) => { - // The child reports realpaths; canonicalize the temp workspace to match. - workspace = realpathSync(cwd) - const parentLogs = await jsonlFiles(join(cwd, '.sessions')) - expect(parentLogs).toHaveLength(1) - events = await sessionEvents(parentLogs[0] as string) - // The child runtime persisted its own transcript in ITS cwd — which - // must be the parent session's workspace for the inheritance to hold. - const childLogs = await jsonlFiles(join(cwd, '.child-sessions')) - expect(childLogs).toHaveLength(1) - childEvents = await sessionEvents(childLogs[0] as string) - }, - }) - expect(stderr).not.toContain('UNHANDLED') + try { + const { stderr } = await runLoaderSmoke({ + label: 'dsh-sdk-subagent cwd composition smoke', + tempDirPrefix: 'dsh-sdk-subagent-cwd-e2e-', + binScript: driver, + libBinScript: driver, + configPath, + tsconfigPath: repoTsconfig, + // Two complete harness runtimes boot in sequence (driver, then the SDK + // child); from-source tsx boots under load need more than the default + // 30s window. + processTimeoutMs: 120_000, + env: { + DSH_TEST_CHILD_PATCHES: JSON.stringify([childPatch]), + DSH_TEST_CHILD_HOME: childHome, + }, + inspect: async (cwd) => { + // The child reports realpaths; canonicalize the temp workspace to match. + workspace = realpathSync(cwd) + const parentLogs = await jsonlFiles(join(cwd, '.sessions')) + expect(parentLogs).toHaveLength(1) + events = await sessionEvents(parentLogs[0] as string) + // The child runtime persists under its explicit isolated home. + const childSessions = join(childHome, 'sessions') + if (!existsSync(childSessions)) { + const result = events.find(event => event.type === 'tool/result') + throw new Error(`SDK child persisted no session; parent tool result: ${JSON.stringify(result?.data)}`) + } + const childLogs = await jsonlFiles(childSessions) + expect(childLogs).toHaveLength(1) + childEvents = await sessionEvents(childLogs[0] as string) + }, + }) + expect(stderr).not.toContain('UNHANDLED') - // The parent's tool result carries the child model's echo of its real - // process.cwd() — the parent session's workspace, never the harness - // process's launch directory. - const results = events.filter(event => event.type === 'tool/result') - expect(results).toHaveLength(1) - const resultText = results[0]!.data.message.content[0].content - .filter(block => block.type === 'text') - .map(block => block.text) - .join('') - expect(resultText).toBe(`child cwd: ${workspace}`) + // The parent's tool result carries the child model's echo of its real + // process.cwd() — the parent session's workspace, never the harness + // process's launch directory. + const results = events.filter(event => event.type === 'tool/result') + expect(results).toHaveLength(1) + const resultText = results[0]!.data.message.content[0].content + .filter(block => block.type === 'text') + .map(block => block.text) + .join('') + expect(resultText).toBe(`child cwd: ${workspace}`) - // The child ran a real turn of its own: user message in, assistant out. - expect(childEvents.some(event => event.type === 'user/message')).toBe(true) - const childAnswers = childEvents.filter(event => event.type === 'assistant/message') - expect(childAnswers.length).toBeGreaterThan(0) + // The child ran a real turn of its own: user message in, assistant out. + expect(childEvents.some(event => event.type === 'user/message')).toBe(true) + const childAnswers = childEvents.filter(event => event.type === 'assistant/message') + expect(childAnswers.length).toBeGreaterThan(0) + } finally { + await rm(childHome, { recursive: true, force: true }) + } // 15s of vitest headroom past the subprocess deadline, mirroring // LOADER_SMOKE_TEST_TIMEOUT_MS's margin over the default window. }, 135_000) diff --git a/packages/subagent/subagent-dsh-sdk/tests/subagent-dsh-sdk.spec.ts b/packages/subagent/subagent-dsh-sdk/tests/subagent-dsh-sdk.spec.ts index f4b5bd4267..2e67c76c24 100644 --- a/packages/subagent/subagent-dsh-sdk/tests/subagent-dsh-sdk.spec.ts +++ b/packages/subagent/subagent-dsh-sdk/tests/subagent-dsh-sdk.spec.ts @@ -6,14 +6,17 @@ * quiescent disposal are all exercised end to end. No model, no key. */ -import { describe, expect, it } from 'vitest' +import { afterEach, beforeEach, describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' import { existsSync, mkdtempSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' -import { join } from 'node:path' +import { join, relative } from 'node:path' import { fileURLToPath } from 'node:url' import SubagentRuntime from '@deepseek-ai/dsh-subagent' import type { Agent } from '@deepseek-ai/dsh-agent' +import { createProcessDeepSeekHarness } from '../../../sdk/client/src/api.ts' +import type { RuntimeProcessOptions } from '../../../sdk/client/src/launch.ts' +import type { DeepSeekHarnessOptions } from '@deepseek-ai/dsh-sdk-client' import * as sdk from '../src/index.ts' import { DEFAULT_DISPOSE_EOF_GRACE_MS, @@ -21,10 +24,41 @@ import { DEFAULT_SHUTDOWN_TIMEOUT_MS, sdkStopReason, startSdkRun, + internals as runInternals, type SdkRunSpec, } from '../src/run.ts' const fakeRuntime = fileURLToPath(new URL('../../../sdk/client/tests/fake-runtime.ts', import.meta.url)) +const existingPatch = fileURLToPath(new URL( + '../../../../examples/python-sdk-agent/tests/fixtures/subagent/subagent-dsh-sdk/child.cordis.yml', + import.meta.url, +)) +const defaultCreateHarness = runInternals.createHarness.bind(runInternals) +let createdHarnessOptions: DeepSeekHarnessOptions[] = [] + +beforeEach(() => { + createdHarnessOptions = [] + runInternals.createHarness = (options) => { + createdHarnessOptions.push(options) + const runtime: RuntimeProcessOptions = { + command: process.execPath, + args: [fakeRuntime], + ...options.processCwd === undefined ? {} : { cwd: options.processCwd }, + environment: () => options.env ?? process.env, + description: 'scripted SDK subagent runtime', + initializeTimeoutMs: options.initializeTimeoutMs ?? 5_000, + ...options.requestTimeoutMs === undefined ? {} : { requestTimeoutMs: options.requestTimeoutMs }, + ...options.shutdownTimeoutMs === undefined ? {} : { shutdownTimeoutMs: options.shutdownTimeoutMs }, + ...options.disposeEofGraceMs === undefined ? {} : { disposeEofGraceMs: options.disposeEofGraceMs }, + ...options.disposeGraceMs === undefined ? {} : { disposeGraceMs: options.disposeGraceMs }, + } + return createProcessDeepSeekHarness(runtime, options) + } +}) + +afterEach(() => { + runInternals.createHarness = defaultCreateHarness +}) /** A parent Agent stub. The SDK backend reads exactly one thing off it: the session header's cwd (the workspace its child inherits). */ const fakeParent = { id: 'parent', session: { header: { cwd: process.cwd() } } } as unknown as Agent @@ -42,8 +76,9 @@ async function setup(fakeEnv: Record = {}, config: Partial { }) describe('dsh-subagent-dsh-sdk provider', () => { + it('constructs the production dsh-backed harness lazily', async () => { + const harness = defaultCreateHarness({}) + expect(harness).toBeInstanceOf((await import('@deepseek-ai/dsh-sdk-client')).DeepSeekHarness) + await harness.close() + }) + it('runs a child turn end to end with a parent-unique run id', async () => { const ctx = await setup({ FAKE_TEXT: 'hello from sdk child' }) const run = await ctx.subagents.start('dsh-sdk', request('do X')) @@ -105,6 +146,21 @@ describe('dsh-subagent-dsh-sdk provider', () => { await ctx.fiber.dispose() }) + it('resolves relative launch files at load and forwards absolute paths', async () => { + const ctx = await setup({ FAKE_TEXT: 'explicit dsh child' }, { + dshBin: relative(process.cwd(), fakeRuntime), + patches: [relative(process.cwd(), existingPatch)], + }) + const run = await ctx.subagents.start('dsh-sdk', request()) + expect(text((await run.result).output)).toBe('explicit dsh child') + expect(createdHarnessOptions[0]).toMatchObject({ + dshBin: fakeRuntime, + patches: [existingPatch], + }) + await run.dispose() + await ctx.fiber.dispose() + }) + it('initializes the child with the configured provider/model/maxTokens and the parent cwd', async () => { const tmp = mkdtempSync(join(tmpdir(), 'subagent-dsh-sdk-init-')) const recordFile = join(tmp, 'init.jsonl') @@ -222,8 +278,9 @@ describe('dsh-subagent-dsh-sdk provider', () => { try { const controller = new AbortController() const spec: SdkRunSpec = { - command: process.execPath, - args: [fakeRuntime], + profile: 'sdk', + patches: [], + dshHome: process.cwd(), cwd: process.cwd(), provider: 'p', model: 'm', @@ -272,10 +329,10 @@ describe('dsh-subagent-dsh-sdk provider', () => { controller.abort() await expect(startSdkRun( request('p', controller.signal), - // `touch ` — runs only if the process is actually spawned. { - command: 'touch', - args: [sentinel], + profile: 'sdk', + patches: [], + dshHome: sentinel, cwd: tmp, provider: 'p', model: 'm', @@ -305,8 +362,9 @@ describe('dsh-subagent-dsh-sdk provider', () => { it('cancelling mid-handshake rejects start after reaping the child', async () => { const controller = new AbortController() const spec: SdkRunSpec = { - command: process.execPath, - args: [fakeRuntime], + profile: 'sdk', + patches: [], + dshHome: process.cwd(), cwd: process.cwd(), provider: 'p', model: 'm', @@ -323,8 +381,9 @@ describe('dsh-subagent-dsh-sdk provider', () => { it('routes a post-publication child failure through onError and settles error', async () => { const seen: string[] = [] const spec: SdkRunSpec = { - command: process.execPath, - args: [fakeRuntime], + profile: 'sdk', + patches: [], + dshHome: process.cwd(), cwd: process.cwd(), provider: 'p', model: 'm', @@ -364,8 +423,9 @@ describe('dsh-subagent-dsh-sdk provider', () => { await ctx.plugin(SubagentRuntime) const fiber = await ctx.plugin(sdk, { providerName: 'sdk-hmr', - command: process.execPath, - args: [fakeRuntime], + profile: 'sdk', + patches: [], + dshHome: process.cwd(), provider: 'p', model: 'm', env: {}, @@ -386,13 +446,48 @@ describe('dsh-subagent-dsh-sdk provider', () => { it('rejects non-positive timing bounds at load', async () => { const ctx = new Context() await ctx.plugin(SubagentRuntime) - const base = { providerName: 'sdk', command: 'true', args: [], provider: 'p', model: 'm', env: {} } + const base = { providerName: 'sdk', profile: 'sdk', patches: [], dshHome: process.cwd(), provider: 'p', model: 'm', env: {} } await expect(ctx.plugin(sdk, { ...base, shutdownTimeoutMs: 0 })).rejects.toThrow('shutdownTimeoutMs must be a positive finite number') await expect(ctx.plugin(sdk, { ...base, disposeEofGraceMs: -1 })).rejects.toThrow('disposeEofGraceMs must be a positive finite number') await expect(ctx.plugin(sdk, { ...base, disposeGraceMs: Number.NaN })).rejects.toThrow('disposeGraceMs must be a positive finite number') await ctx.fiber.dispose() }) + it('requires an explicit absolute Harness home for nested dsh runtimes', async () => { + const ctx = new Context() + await ctx.plugin(SubagentRuntime) + await expect(ctx.plugin(sdk, { + providerName: 'sdk', + profile: 'sdk', + patches: [], + dshHome: './personal-home', + provider: 'p', + model: 'm', + env: {}, + })).rejects.toThrow('dshHome must be an absolute path') + await ctx.fiber.dispose() + }) + + it.each([ + { field: 'dshBin', override: { dshBin: './missing-dsh-bin' } }, + { field: 'dshBin', override: { dshBin: '.' } }, + { field: 'patches[0]', override: { patches: ['./missing-child-patch.yml'] } }, + ])('rejects an invalid $field at load', async ({ field, override }) => { + const ctx = new Context() + await ctx.plugin(SubagentRuntime) + await expect(ctx.plugin(sdk, { + providerName: 'sdk', + profile: 'sdk', + patches: [], + dshHome: process.cwd(), + provider: 'p', + model: 'm', + env: {}, + ...override, + })).rejects.toThrow(`${field} must name an existing file`) + await ctx.fiber.dispose() + }) + it.each([0, -1, 1.5, Number.NaN, Number.MAX_SAFE_INTEGER + 1])( 'rejects invalid maxTokens %s at load', async (maxTokens) => { @@ -400,8 +495,9 @@ describe('dsh-subagent-dsh-sdk provider', () => { await ctx.plugin(SubagentRuntime) await expect(ctx.plugin(sdk, { providerName: 'sdk', - command: 'true', - args: [], + profile: 'sdk', + patches: [], + dshHome: process.cwd(), provider: 'p', model: 'm', maxTokens, @@ -418,8 +514,9 @@ describe('dsh-subagent-dsh-sdk provider', () => { await ctx.plugin(SubagentRuntime) expect(() => { sdk.apply(ctx, { providerName: 'sdk', - command: 'true', - args: [], + profile: 'sdk', + patches: [], + dshHome: process.cwd(), provider: 'p', model: 'm', maxTokens, @@ -437,8 +534,9 @@ describe('dsh-subagent-dsh-sdk provider', () => { await ctx.plugin(SubagentRuntime) await expect(ctx.plugin(sdk, { providerName: 'sdk', - command: 'true', - args: [], + profile: 'sdk', + patches: [], + dshHome: process.cwd(), cwd: '', provider: 'p', model: 'm', diff --git a/packages/subagent/subagent-fork-in-process/README.i18n.yaml b/packages/subagent/subagent-fork-in-process/README.i18n.yaml index b3c3bd3995..9cd2361d52 100644 --- a/packages/subagent/subagent-fork-in-process/README.i18n.yaml +++ b/packages/subagent/subagent-fork-in-process/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/subagent-fork-in-process/README.md README.md: 74c27ff10c76aa711ed3e954e806c00a27aacfa5 -README.zh.md: bca890769bab1f898abd0b71fc8aa321b66e93c8 +README.zh.md: 43e7ef489b33d52b674420d08f7edf8c89fb0e42 diff --git a/packages/subagent/subagent-fork-in-process/README.zh.md b/packages/subagent/subagent-fork-in-process/README.zh.md index bca890769b..43e7ef489b 100644 --- a/packages/subagent/subagent-fork-in-process/README.zh.md +++ b/packages/subagent/subagent-fork-in-process/README.zh.md @@ -14,7 +14,7 @@ subagent 启动时,父 agent 当前的工具调用轮次仍未结束:其日 ## 启动与能力 -`start(request)` 将已完成轮次的初始内容传给 [`startInProcessRun`](../subagent-in-process-driver/README.md),并等待子 agent 发布。共享驱动器负责取消、深度、定制、结果读取和 dispose(资源释放)。 +`start(request)` 将已完成轮次的初始内容传给 [`startInProcessRun`](../subagent-in-process-driver/README.zh.md),并等待子 agent 发布。共享驱动器负责取消、深度、定制、结果读取和 dispose(资源释放)。 fork 声明 `{ outputSchema: true, depthLimit: true, toolFilter: true, persona: true }`,与 spawn 相同。 @@ -23,7 +23,7 @@ fork 声明 `{ outputSchema: true, depthLimit: true, toolFilter: true, persona: | 键 | 含义 | |---|---| | `providerName` | `ctx.subagents` 上的注册表名称(默认 `fork`)。 | -运行生命周期、模型继承与深度跟踪均为共享行为,见 [`dsh-subagent-spawn-in-process`](../subagent-spawn-in-process/README.md)。 +运行生命周期、模型继承与深度跟踪均为共享行为,见 [`dsh-subagent-spawn-in-process`](../subagent-spawn-in-process/README.zh.md)。 ## 模型体验 @@ -39,7 +39,7 @@ fork 会把保留的已完成历史复制到独立的子 agent 请求中;随 #### KV Cache 影响 -在提供方和模型相同的前提下,子 agent 可以复用继承的逐字节相同前缀。persona、工具过滤、生成 SDK 或路由变化可能在继承历史之前使复用失效;后续子 agent 历史仅追加。因此随附组合把本提供方绑定为 `backgroundMode: one-shot`:可继续子 agent 还会额外携带作用域局部的 `report` 工具及其提示词 section,而这些增量位于继承历史之前,会使继承历史整体失效(见 [fork 保持 one-shot 的 Agent Note](../../../.agents/notes/implemented/architecture/2026-08-10-fork-children-stay-one-shot.md))。 +在提供方和模型相同的前提下,子 agent 可以复用继承的逐字节相同前缀。persona、工具过滤、生成 SDK 或路由变化可能在继承历史之前使复用失效;后续子 agent 历史仅追加。因此随附组合把本提供方绑定为 `backgroundMode: one-shot`:可继续子 agent 还会额外携带作用域局部的 `report` 工具及其提示词 section,而这些增量位于继承历史之前,会使继承历史整体失效(见 [fork 保持 one-shot 的 Agent Note](../../../.agents/notes/implemented/architecture/2026-08-10-fork-children-stay-one-shot.zh.md))。 ### 父 agent 工具结果(间接) @@ -58,4 +58,4 @@ fork 会把保留的已完成历史复制到独立的子 agent 请求中;随 ## 已知限制与暂缓事项 - **初始内容是一次性快照**:子 agent 只能看到 fork 时父 agent 已完成的轮次,看不到父 agent 此后记录的任何内容;不会实时共享上下文。 -- **没有任何随附组合会创建可继续的 fork 子 agent**:`prepareContinuable` 仍然实现完好,seam 也接受它,但每份随附的 `cordis.yml` 都在 fork 委派工具上设置 `backgroundMode: one-shot`,因此该提供方的可继续路径没有生产调用方。重新开放它需要子 agent 的系统提示词与工具 schema 与父 agent 逐字节一致,而这一点目前被 [`report` 返回通道](../tool-subagent-report/README.md)阻止。理由与重新开放条件见 [fork 保持 one-shot 的 Agent Note](../../../.agents/notes/implemented/architecture/2026-08-10-fork-children-stay-one-shot.md)。 +- **没有任何随附组合会创建可继续的 fork 子 agent**:`prepareContinuable` 仍然实现完好,seam 也接受它,但每份随附的 `cordis.yml` 都在 fork 委派工具上设置 `backgroundMode: one-shot`,因此该提供方的可继续路径没有生产调用方。重新开放它需要子 agent 的系统提示词与工具 schema 与父 agent 逐字节一致,而这一点目前被 [`report` 返回通道](../tool-subagent-report/README.zh.md)阻止。理由与重新开放条件见 [fork 保持 one-shot 的 Agent Note](../../../.agents/notes/implemented/architecture/2026-08-10-fork-children-stay-one-shot.zh.md)。 diff --git a/packages/subagent/subagent-fork-in-process/package.json b/packages/subagent/subagent-fork-in-process/package.json index 5ccbe0e465..b7383f45bf 100644 --- a/packages/subagent/subagent-fork-in-process/package.json +++ b/packages/subagent/subagent-fork-in-process/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subagent-fork-in-process", "description": "In-process fork subagent backend: runs a child agent seeded with a prefix of the parent's log", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subagent/subagent-fork-in-process/tests/subagent-fork-in-process.spec.ts b/packages/subagent/subagent-fork-in-process/tests/subagent-fork-in-process.spec.ts index 6d763b7c0a..8b69711233 100644 --- a/packages/subagent/subagent-fork-in-process/tests/subagent-fork-in-process.spec.ts +++ b/packages/subagent/subagent-fork-in-process/tests/subagent-fork-in-process.spec.ts @@ -173,7 +173,6 @@ describe('dsh-subagent-fork-in-process', () => { const result = await run.result expect(result.stopReason).toBe('completed') expect(result.structured).toEqual({ answer: 9 }) - // Run-scoped runtime: nothing stays registered after the settle. expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() await run.dispose() }) diff --git a/packages/subagent/subagent-in-process-driver/README.i18n.yaml b/packages/subagent/subagent-in-process-driver/README.i18n.yaml index 7c6f691e73..505c480cfe 100644 --- a/packages/subagent/subagent-in-process-driver/README.i18n.yaml +++ b/packages/subagent/subagent-in-process-driver/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/subagent-in-process-driver/README.md README.md: 47a5c09fc1c80c5dc3062be82e7355b874a627d3 -README.zh.md: bcd6a2cf31c351722dfefe55e8b8ff75d4252c40 +README.zh.md: b96399795a0fbac05ef1795888aa93c620687f30 diff --git a/packages/subagent/subagent-in-process-driver/README.zh.md b/packages/subagent/subagent-in-process-driver/README.zh.md index bcd6a2cf31..b96399795a 100644 --- a/packages/subagent/subagent-in-process-driver/README.zh.md +++ b/packages/subagent/subagent-in-process-driver/README.zh.md @@ -20,7 +20,7 @@ 该结果边界成立,是因为提供方拥有从发布到完全停稳的隔离子 agent 生命周期。在该生命周期内提交的 steering(中途引导)属于子运行;提供方不会声称输出只归初始 follow-up 所有。 -驱动器通过共享的子 agent 辅助函数应用该 seam 的[委派策略](../subagent/README.md#delegated-policy):它会在创建子 agent 前捕获父级的显式沙箱覆盖项与 `'never'` 审批钉定,并在未发布的设置阶段追加带来源标记的事件,使其位于所有 fork 历史之后、会话发布之前。参见[委派策略决策](../../../.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.md)。 +驱动器通过共享的子 agent 辅助函数应用该 seam 的[委派策略](../subagent/README.zh.md#delegated-policy):它会在创建子 agent 前捕获父级的显式沙箱覆盖项与 `'never'` 审批钉定,并在未发布的设置阶段追加带来源标记的事件,使其位于所有 fork 历史之后、会话发布之前。参见[委派策略决策](../../../.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.zh.md)。 ## 取消与所有权 @@ -66,7 +66,7 @@ #### 模型看到的内容 -结构化运行会添加下方的结构化输出指令。它还会添加子 agent 作用域的 `structured_output` 定义,其精确描述为 `Report your final structured result. Call this exactly once, when your answer is complete; the arguments must match this tool's parameter schema exactly.`,参数使用请求的 schema。该仅运行时存在的定义不在已生成并随产品发布的[工具包索引](../../../docs/tool-catalog.md#tool-package-map)中。其规范确认值是 `{ recorded: true }`,渲染为 `Structured output recorded.`;后续调用会变为 ``Error: structured output already recorded: the run is complete, so `` is not executed``。 +结构化运行会添加下方的结构化输出指令。它还会添加子 agent 作用域的 `structured_output` 定义,其精确描述为 `Report your final structured result. Call this exactly once, when your answer is complete; the arguments must match this tool's parameter schema exactly.`,参数使用请求的 schema。该仅运行时存在的定义不在已生成并随产品发布的[工具包索引](../../../docs/tool-catalog.zh.md#tool-package-map)中。其规范确认值是 `{ recorded: true }`,渲染为 `Structured output recorded.`;后续调用会变为 ``Error: structured output already recorded: the run is complete, so `` is not executed``。 ##### 结构化输出指令 diff --git a/packages/subagent/subagent-in-process-driver/package.json b/packages/subagent/subagent-in-process-driver/package.json index f3398a76e7..70ffbab72d 100644 --- a/packages/subagent/subagent-in-process-driver/package.json +++ b/packages/subagent/subagent-in-process-driver/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subagent-in-process-driver", "description": "Shared in-process subagent run driver: drives a child agent on ctx.agents (used by the spawn and fork backends)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subagent/subagent-spawn-in-process/README.i18n.yaml b/packages/subagent/subagent-spawn-in-process/README.i18n.yaml index 4286fabdd8..1246456cc0 100644 --- a/packages/subagent/subagent-spawn-in-process/README.i18n.yaml +++ b/packages/subagent/subagent-spawn-in-process/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/subagent-spawn-in-process/README.md README.md: f1fb96f2230359cb3ff55c630f29fd34345dbed7 -README.zh.md: 940701d6fac78ce75fda24519fa22b268085ea1a +README.zh.md: 95a3b5cdb7084eb75666f8d62001221c57ac676c diff --git a/packages/subagent/subagent-spawn-in-process/README.zh.md b/packages/subagent/subagent-spawn-in-process/README.zh.md index 940701d6fa..95a3b5cdb7 100644 --- a/packages/subagent/subagent-spawn-in-process/README.zh.md +++ b/packages/subagent/subagent-spawn-in-process/README.zh.md @@ -6,7 +6,7 @@ spawn 提供方会在当前进程中创建一个全新的子 `Agent`。子 agent ## 行为 -`start(request)` 不传入 seed,直接委托给 [`startInProcessRun`](../subagent-in-process-driver/README.md),并在子 agent 发布后才返回。子 agent 获得父 agent 的工作目录/会话谱系,并默认继承父 agent 模型(除非覆盖),但以空对话开始运行。 +`start(request)` 不传入 seed,直接委托给 [`startInProcessRun`](../subagent-in-process-driver/README.zh.md),并在子 agent 发布后才返回。子 agent 获得父 agent 的工作目录/会话谱系,并默认继承父 agent 模型(除非覆盖),但以空对话开始运行。 共享驱动器负责深度检查、persona 与工具过滤器设置、结构化输出、通过必需的信号执行取消、单次执行、结果读取和完全停稳后的 dispose(资源释放)。启动遭拒不会留下已发布的子 agent;启动调用兑现后卸载提供方,也不会撤销由持有方拥有的运行。 diff --git a/packages/subagent/subagent-spawn-in-process/package.json b/packages/subagent/subagent-spawn-in-process/package.json index 8c2c71c9c9..3ea456cf4e 100644 --- a/packages/subagent/subagent-spawn-in-process/package.json +++ b/packages/subagent/subagent-spawn-in-process/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subagent-spawn-in-process", "description": "In-process spawn subagent backend: runs a fresh child agent on ctx.agents", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subagent/subagent-spawn-in-process/tests/subagent-spawn-in-process.spec.ts b/packages/subagent/subagent-spawn-in-process/tests/subagent-spawn-in-process.spec.ts index 74aefb4ebd..ae60480c02 100644 --- a/packages/subagent/subagent-spawn-in-process/tests/subagent-spawn-in-process.spec.ts +++ b/packages/subagent/subagent-spawn-in-process/tests/subagent-spawn-in-process.spec.ts @@ -310,7 +310,6 @@ describe('dsh-subagent-spawn-in-process', () => { const result = await run.result expect(result.stopReason).toBe('completed') expect(result.structured).toEqual({ answer: 42 }) - // Run-scoped runtime: the settle released the last acquisition. expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() await run.dispose() }) diff --git a/packages/subagent/subagent/README.i18n.yaml b/packages/subagent/subagent/README.i18n.yaml index ace4708403..a645351bd8 100644 --- a/packages/subagent/subagent/README.i18n.yaml +++ b/packages/subagent/subagent/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/subagent/README.md -README.md: ed4a9123a2dfa5b2fa5abc67f4513547feb3d140 -README.zh.md: e117ac7a6078d5b63759560eef35381b43b00dea +README.md: e84a6b486253e81ccf7e7df12c4149e6df4ed9f2 +README.zh.md: e289863531c1686cedeccadfa76e2661dfa9bfc8 diff --git a/packages/subagent/subagent/README.md b/packages/subagent/subagent/README.md index ed4a9123a2..e84a6b4862 100644 --- a/packages/subagent/subagent/README.md +++ b/packages/subagent/subagent/README.md @@ -16,12 +16,13 @@ The [subagent family overview](../README.md) maps implementations and model-faci | `getProvider(name)` | Return the provider, or `undefined` when absent. | | `list()` | Return provider names in insertion order. | | `start(name, request)` | Validate an ordinary caller request, resolve its detached `one-shot` descriptor, then await the provider until a real one-shot child is published. Fulfillment returns a holder-owned `SubagentRun`; rejection means the provider has already cleaned every unpublished startup resource, while post-publication turn or infrastructure faults settle through the run. Continuable children never enter through this operation. | -| `startContinuable(spec)` | Establish one durable continuable child and deliver its initial prompt. Resolves with `{ childId, messageId }` when the child's inbox accepts that prompt, without waiting for the turn to start or for the message to reach the Session log; any earlier failure rejects with no ids and rolls the child back entirely. Requires `ctx.agents`, session persistence, and a provider with the `prepareContinuable` capability. | +| `startContinuable(spec)` | Establish one durable continuable child and deliver its initial prompt. Resolves with `{ childId, messageId }` when the child's inbox accepts that prompt, without waiting for the turn to start or for the message to reach the Session log; any earlier failure rejects with no ids and rolls the child back entirely. A caller-reserved `childId` is rejected when the live registries or configured persistence already own it. Requires `ctx.agents`, session persistence, and a provider with the `prepareContinuable` capability. | | `followup(parent, childId, content, { source, signal })` | Deliver one later message from the exact live direct parent as the child's next FIFO turn, matching `Agent.followup()` terminology, and return the accepted `MessageId`. A resident child's inbox accepts it directly (waking a waiting Activation); an absent one cold-resumes from its persisted Session. Requires `ctx.agents`; cold resume also requires session persistence. | | `interrupt(targetSessionId, authority)` | Interrupt one live continuable child's current turn under a human durable parent address (`{ kind: 'user', parentSessionId }`) or an exact live ancestor Agent (`{ kind: 'ancestor', agent }`). Admission is synchronous and the effect asynchronous: it issues `Agent.cancel(cause, { keepInbox: true })` and returns without waiting for the target to observe the signal. Unclaimed pending inbox work, the Activation, and published descendants are preserved; work already claimed into the interrupted turn is not requeued. An absent target is an accepted no-op; a wrong parent address or a stale, self-targeting, or non-ancestor caller rejects with `UNAUTHORIZED`. | -| `reportFrom(child, content, { delivery, signal })` | Deliver one selected message from the exact live continuable child to its exact live direct parent and return the accepted stable `MessageId`. Quiet delivery injects context; waking delivery submits one later parent turn. | +| `reportFrom(child, content, { delivery, signal })` | Deliver one selected message from the exact live continuable child to its exact live direct parent and return the accepted stable `MessageId`. Quiet delivery injects next-step context without waking; next-step delivery steers and wakes the parent. | | `registerContinuableSetup(contribution)` | Compose an optional deployment capability into each continuable child's unpublished scope, with immediate revocation from resident children. | | `drainContinuableDescendants(parents)` | Close admission below exact live host-owned parent Agents, stop only their visible continuable descendants, await materializations admitted below those roots through publication or rollback, then release the selected forests child-first. The cutoff lasts until each exact parent leaves the registry; unrelated parent forests and manager-wide admission remain live. | +| `drainContinuableChildren(parent, childIds)` | Release only the named resident continuable direct children of one exact live parent, recursively and child-first. It does not close admission or touch siblings, accepts absent ids as no-ops, and rejects a resident child owned by another parent. This is teardown, so unlike `interrupt()` it does not preserve pending inbox work. | | `listChildren(parentSessionId, signal?)` | List direct session-backed subagents with their `one-shot`/`continuable` mode, `running`/`inactive` activity, origin-classified one-level `hasChildren` hint, and per-child diagnostics, ordered by `createdAt` then id, without loading or resuming them. Reads the live session store and optional session persistence directly (live-only enumeration when persistence is absent) and requires the mounted `sessionProjections` registry; it does not require `ctx.agents`, the continuation manager, or any query service. | | `listDescendants(rootSessionId, signal?)` | Flatten the root's complete session tree in stable pre-order from the same live-preferred corpus, adding each subagent entry's durable `parentId` and root-relative `depth`. Ordinary sessions and one-shot children remain traversal nodes so continuable descendants below them are discovered. Identity, diagnostics, dependencies, and cancellation follow `listChildren()`. | @@ -64,7 +65,7 @@ Both in-process delegation paths fix the child's permission scope at the delegat `provider.start(request): Promise` is the ownership-transfer boundary; the delegation tool also uses it inside its one-shot Task-backed background path. Before fulfillment, the provider owns setup and must cancel, roll back, and quiesce unpublished resources on every failure. After fulfillment, the caller owns the run and must call `dispose()` on every path; remaining prompt and turn work belongs to `SubagentRun.result`. -`SubagentRun.result` resolves to `{ output, structured?, stopReason }`. Child-level failures resolve with a non-`completed` reason; only an infrastructure fault that the seam cannot represent may reject. `dispose()` is idempotent, cancels remaining work, and waits for both result settlement and child-resource quiescence. A result rejection remains on `result`; `dispose()` rejects only for an independent resource-release failure. `output` and the `subagent/end` event's `lastAssistantMessage` use the exported `AssistantOutputFold`/`finalAssistantOutput` helpers to select the child's last non-empty assistant message, or its accumulated assistant text when no such message exists. `output` is `[]` and the event field is absent when the child produced neither ([`SubagentResult.output`](../../../docs/subsystems/subagent.md#the-terminal-result-subagentresult) owns the result contract). +`SubagentRun.result` resolves to `{ output, structured?, diagnostic?, stopReason }`. Child-level failures resolve with a non-`completed` reason; only an infrastructure fault that the seam cannot represent may reject. A provider may add a safe `diagnostic` to a non-completed result after removing tool inputs, file contents, environment values, credentials, and raw protocol payloads and limiting the complete text to 4096 UTF-8 bytes. The common result type does not define provider categories or lifecycle stages: an out-of-process provider may derive fixed display text from its version-pinned structured product facts and an observed process outcome, while consumers render that text without parsing it. The field is not assistant output: consumers present it separately, and it does not enter `subagent/end.lastAssistantMessage`. `dispose()` is idempotent, cancels remaining work, and waits for both result settlement and child-resource quiescence. A result rejection remains on `result`; `dispose()` rejects only for an independent resource-release failure. `output` and the `subagent/end` event's `lastAssistantMessage` use the exported `AssistantOutputFold`/`finalAssistantOutput` helpers to select the child's last non-empty assistant message, or its accumulated assistant text when no such message exists. `output` is `[]` and the event field is absent when the child produced neither ([`SubagentResult`](../../../docs/subsystems/subagent.md#the-terminal-result-subagentresult) owns the terminal result contract). A local run publishes an ordinary child agent/session before `start()` fulfills, returns that shared session id as `SubagentRun.id`, exposes the exact child as `SubagentRun.localAgent`, records `request.parent.session.id` in the child's `parentSession` header, and appends the resolved descriptor inside its initial turn. Remote providers instead mint a parent-scoped lifecycle id and return `localAgent: undefined`; without a local child session, their one-shot runs are not part of trace-backed enumeration. @@ -146,7 +147,7 @@ Prefix-stable within a child: the statement never changes during the child's lif - **ACP children remain one-shot and are not trace-enumerable** — an ACP run has no local child session in the parent's session corpus. An ACP `prepareContinuable` requires persisting the remote session id in provider-specific descriptor data and a per-child continuation advertisement, since ACP `loadSession` support is negotiated per child rather than established by the method's presence. Remote providers also require a separate Activation ownership contract with equivalent authenticated control and child-first quiescence before they support continuable children. - **No host-user continuation** — `followup()` requires the exact live direct parent. Only `interrupt()` accepts a durable parent-address user authority, because stopping a turn is idempotent and delivers no content; a future host adapter needs a concrete authenticated interaction before the seam gains a user delivery capability. -- **No current-turn steering** — continuable messages and waking reports enqueue later turns; neither redirects an open turn. +- **Continuation messages never steer** — parent-to-child continuation messages enqueue later child turns. Child-to-parent reports are independent next-step input and may extend the parent's open turn. - **Wake gap during cancellation convergence** — a waking follow-up accepted after the interrupt signal is issued but before the active driver becomes idle remains queued until another waking send. Issue #1838 owns the agent-loop wake latch, which also affects ordinary session cancellation. - **Process-local residency** — the Activation inbox and ownership graph do not coordinate two harness processes; concurrent access to one persistence store still requires a durable mailbox and cross-process lease protocol. - **No replay of accepted-but-unlogged messages** — only messages written to the child Session log are reconstructable with the source that supplied them. A crash may lose an accepted initial prompt or follow-up that never reached the log; a later authorized message can cold-resume the child, but the lost message is not replayed automatically. diff --git a/packages/subagent/subagent/README.zh.md b/packages/subagent/subagent/README.zh.md index e117ac7a60..e289863531 100644 --- a/packages/subagent/subagent/README.zh.md +++ b/packages/subagent/subagent/README.zh.md @@ -2,9 +2,9 @@ [English](README.md) | 中文 -subagent seam 允许一个 agent(智能体)通过具名提供方把工作委派给子 agent。调用方使用统一的服务 API(`ctx.subagents`);提供方决定子 agent 在当前进程、另一进程还是未来的传输之上运行。 +subagent seam 允许一个 agent(智能体)通过具名提供方把工作委派给子 agent。调用方统一使用 `ctx.subagents` 服务 API;提供方决定子 agent 在当前进程、其他进程,还是通过未来的传输方式运行。 -[subagent 家族概述](../README.md)列出了实现和面向模型的消费方。本包负责提供方注册表、共享请求和结果约定、持久描述符以及可继续子级编排。多个具名提供方可以在该约定背后共存。 +[subagent 家族概述](../README.zh.md)列出了实现和面向模型的消费方。本包负责提供方注册表、共享请求和结果约定、持久描述符以及可继续子级编排。多个具名提供方可以在该约定背后共存。 ## 服务 API @@ -15,21 +15,22 @@ subagent seam 允许一个 agent(智能体)通过具名提供方把工作委 | `registerProvider(provider)` | 按名称注册一个可信的同进程实现。注册受 effect 作用域约束;移除注册会阻止新的启动,但不会撤销已返回给调用方的运行。重复名称会明确报错。 | | `getProvider(name)` | 返回提供方;不存在时返回 `undefined`。 | | `list()` | 按插入顺序返回提供方名称。 | -| `start(name, request)` | 校验普通调用方请求,解析其分离的 `one-shot` 描述符,然后等待提供方,直到真实的一次性子 agent 发布。兑现时返回由持有方拥有的 `SubagentRun`;拒绝表示提供方已清理所有未发布的启动资源,而发布后的轮次或基础设施故障会通过该 run 结算。可继续子 agent 绝不通过此操作进入。 | -| `startContinuable(spec)` | 建立一个持久化可继续子 agent,并投递其初始提示词。子 agent 的 inbox 接受该提示词时,兑现为 `{ childId, messageId }`,无需等待轮次开始或消息写入会话日志;此前任何失败都会以无 id 拒绝,并完全回滚该子 agent。要求 `ctx.agents`、会话持久化以及具备 `prepareContinuable` 能力的提供方。 | +| `start(name, request)` | 校验普通调用方请求,解析其已分离的 `one-shot` 描述符,然后等待提供方发布真正的一次性子 agent。兑现时返回由持有方拥有的 `SubagentRun`;如果调用被拒绝,提供方已经清理所有尚未发布的启动资源。发布后的轮次故障或基础设施故障则通过该 run 结算。可继续子 agent 绝不通过此操作进入。 | +| `startContinuable(spec)` | 建立一个持久化的可继续子 agent,并投递其初始提示词。子 agent 的 inbox 一接受该提示词,调用就会兑现为 `{ childId, messageId }`,无需等待轮次开始,也无需等待消息写入会话日志。在此之前发生的任何失败都会使调用被拒绝,不返回任何 id,并完全回滚该子 agent。如果在线注册表或已配置的持久化已经占用调用方预留的 `childId`,则拒绝该身份。要求 `ctx.agents`、会话持久化以及具备 `prepareContinuable` 能力的提供方。 | | `followup(parent, childId, content, { source, signal })` | 将来自确切在线直接父级的一条后续消息作为子 agent 的下一个 FIFO 轮次投递,术语与 `Agent.followup()` 一致,并返回被接受的 `MessageId`。驻留中的子 agent 由其 inbox 直接接受(唤醒处于 waiting 的 Activation);不驻留的则从其持久化会话冷恢复。要求 `ctx.agents`;冷恢复还要求会话持久化。 | -| `interrupt(targetSessionId, authority)` | 以人类持久化 parent 地址(`{ kind: 'user', parentSessionId }`)或确切在线 ancestor Agent(`{ kind: 'ancestor', agent }`)为授权,中断一个在线可继续 child 的当前轮次。准入同步完成、生效异步进行:它发出 `Agent.cancel(cause, { keepInbox: true })` 后立即返回,不等待目标观察到信号。尚未领取的待处理 inbox 工作、Activation 与已发布的后代均保持不变;已被领取进入中断轮次的工作不会重新入队。目标不存在时接受为 no-op;错误的 parent 地址以及陈旧、指向自身或非祖先调用方以 `UNAUTHORIZED` 拒绝。 | -| `reportFrom(child, content, { delivery, signal })` | 从确切在线可继续 child 向其确切在线直接 parent 投递一条选中消息,并返回已接受的稳定 `MessageId`。静默投递会注入上下文;唤醒投递会提交一个后续 parent 轮次。 | +| `interrupt(targetSessionId, authority)` | 凭人类出示的持久化父级地址 `{ kind: 'user', parentSessionId }`,或确切在线的祖先 Agent `{ kind: 'ancestor', agent }` 进行授权,中断一个在线可继续子级的当前轮次。准入判定同步完成,但取消异步生效:该操作发出 `Agent.cancel(cause, { keepInbox: true })` 后立即返回,不等待目标观察到信号。尚未领取的待处理 inbox 工作、Activation 和已发布的后代均会保留;已经领取到被中断轮次中的工作不会重新入队。目标不存在时视为已接受的空操作;错误的父级地址,或陈旧、指向自身、并非祖先的调用方,会以 `UNAUTHORIZED` 被拒绝。 | +| `reportFrom(child, content, { delivery, signal })` | 从确切在线可继续 child 向其确切在线直接 parent 投递一条选中消息,并返回已接受的稳定 `MessageId`。静默投递会注入不唤醒的 next-step 上下文;next-step 投递会 steering 并唤醒 parent。 | | `registerContinuableSetup(contribution)` | 把一项可选部署能力组合到每个可继续 child 尚未发布的作用域中,并支持从驻留 child 立即撤销。 | -| `drainContinuableDescendants(parents)` | 在由 host 确切拥有的在线 parent Agent 之下关闭准入,只停止其可见的可继续后代,等待在这些根之下已获准的物化过程完成发布或回滚,再按 child-first 顺序释放所选森林。该截止状态会持续到每个确切 parent 离开注册表;无关的 parent 森林和管理器全局准入保持在线。 | -| `listChildren(parentSessionId, signal?)` | 按 `createdAt` 再按 id 的顺序列出由会话支撑的直接 subagent,包括其 `one-shot`/`continuable` 模式、`running`/`inactive` 活动状态、基于 origin 分类的一层 `hasChildren` 提示与逐 child diagnostic,且不会加载或恢复它们。直接读取在线会话存储与可选的会话持久化(持久化缺席时仅枚举在线 child),并要求已挂载 `sessionProjections` 注册表;不要求 `ctx.agents`、继续执行管理器或任何查询服务。 | +| `drainContinuableDescendants(parents)` | 在由 host 拥有的确切在线父级 Agent 之下关闭准入,只停止这些父级可见的可继续后代;等待已在这些根节点下获准的物化过程完成发布或回滚后,再按子级优先顺序释放所选的各棵树。该截止状态会持续到每个确切父级离开注册表;无关的父级树仍在线,管理器全局准入仍保持开放。 | +| `drainContinuableChildren(parent, childIds)` | 只释放一个确切在线父级的具名驻留可继续直接子级,并递归保持子级优先顺序。它不关闭准入、不影响同级子级,缺失的 id 视为空操作;若驻留子级属于其他父级,则拒绝。这是拆卸操作,因此与 `interrupt()` 不同,它不会保留待处理的 inbox 工作。 | +| `listChildren(parentSessionId, signal?)` | 按 `createdAt`、再按 id 的顺序列出由会话支撑的直接 subagent,包括其 `one-shot`/`continuable` 模式、`running`/`inactive` 活动状态、根据 origin 分类得出的一层 `hasChildren` 提示,以及每个子级的诊断信息,且不会加载或恢复它们。该操作直接读取在线会话存储和可选的会话持久化(没有持久化时只枚举在线子级),并要求已挂载 `sessionProjections` 注册表;不要求 `ctx.agents`、继续执行管理器或任何查询服务。 | | `listDescendants(rootSessionId, signal?)` | 从同一份在线优先语料按稳定 pre-order 展平根的完整会话树,并为每个 subagent 条目附加持久 `parentId` 与相对根的 `depth`。普通会话与一次性 child 仍作为遍历节点,因此其下的可继续后代仍可发现。身份、diagnostic、依赖与取消约定均沿用 `listChildren()`。 | -`SubagentStartRequest.label` 是由会话支撑的一次性 child 所使用的可选简短持久化显示标签。面向模型的委派会提供其已有的 `description`;底层调用方无需凭空构造展示元数据。可继续启动始终携带自身的必填标签。`signal` 是必填项,也是一次性 `start` 的规范取消通道。发布前中止会使 `start()` 在回滚后拒绝;发布后中止会取消已返回 run 的剩余轮次工作,但不会隐藏其 id。请求还可以选择模型、要求结构化输出、限制委派深度、约束子 agent 工具或设置子 agent persona。对于可继续启动或后续操作,调用方信号只在 inbox 接受之前掌管查找、物化和准入;此后由管理器独立拥有 Activation,因此调用方后续取消既不会取消已接受的轮次,也不会 dispose(资源释放)子 agent。 +`SubagentStartRequest.label` 是由会话支撑的一次性 child 所使用的可选简短持久化显示标签。面向模型的委派会提供其已有的 `description`;底层调用方无需凭空构造展示元数据。可继续启动始终携带自身的必填标签。`signal` 是必填项,也是一次性 `start` 的规范取消通道。发布前中止会使 `start()` 在回滚后拒绝;发布后中止会取消已返回 run 的剩余轮次工作,但不会隐藏其 id。请求还可以选择模型、要求结构化输出、限制委派深度、约束子 agent 工具或设置子 agent persona。对于可继续启动或后续操作,调用方信号只负责 inbox 接受前的查找、物化和准入;此后,Activation 由管理器独立拥有,因此调用方取消既不会取消已接受的轮次,也不会 dispose(资源释放)子 agent。 后续操作的权限来自子 agent 持久化 header 中记录的确切在线直接父级。冷恢复会在重建前检查该权限,并在最终无 await 的 inbox 准入区间再次检查,因此在物化期间被注销或替换的 parent 无法授权投递。后续操作上的 `source` 记录谁提供了所投递的消息,不授予任何权限。 -同进程请求、描述符、结果和事件 payload 都是以不可变方式借用的可信类型值。服务不会克隆或冻结它们;序列化和不可信输入校验属于真实的进程、worker、持久化和模型边界。 +同进程请求、描述符、结果和事件 payload 都是可信的类型值,并按不可变约定借用。服务不会克隆或冻结它们;序列化和不可信输入校验属于真实的进程、worker、持久化和模型边界。 ## 能力 @@ -40,11 +41,11 @@ subagent seam 允许一个 agent(智能体)通过具名提供方把工作委 - `toolFilter`:应用请求的子 agent 工具限制; - `persona`:应用每个子 agent 独立的 persona。 -每个进程内子 agent 都由一次调用完成组装:`applyChildComposition(childCtx, parent, composition)` 先加入父方的 agent-preset 组装,再应用该子 agent 自己的 persona 与工具限制。加入组装正是子 agent 获得能力的途径:所有面向模型的行都在 agent 平面,没有加入任何组装的子 agent 抵达模型时工具注册表是空的(见 [`dsh-agent-presets`](../../preset/agent-presets/README.md))。把父方作为参数是刻意的——这让"组装一个子 agent 却不做该加入"在各调用点无法表达,而这正是这一次调用所要杜绝的缺陷。未组装 preset roster 的部署不加入任何组装、也不需要加入:它的面向模型的行位于宿主组装中,子 agent 已经能通过工具注册表的全局层解析到它们。 +每个进程内子 agent 都通过一次 `applyChildComposition(childCtx, parent, composition)` 调用完成组装:先加入父级的 agent-preset 组合,再应用子 agent 自己的 persona 和工具限制。加入父级组合正是子 agent 获得能力的途径:所有面向模型的行都位于 agent 平面,完全没有加入任何组合的子 agent 抵达模型时会看到空的工具注册表(见 [`dsh-agent-presets`](../../preset/agent-presets/README.zh.md))。将父级作为参数是刻意设计:这让“组装子 agent 却不做该加入”在各调用点无法表达,而这正是这一次调用所要杜绝的缺陷。未组装 preset roster 的部署不加入任何组合、也不需要加入;其面向模型的行位于宿主组合中,子 agent 已能通过工具注册表的全局层解析到它们。 `childSessionMeta()` 把所加入的 preset id 记在子 agent 的持久化 header 上,理由与顶层会话记录自己的那一个相同:preset 决定了模型所见的工具 schema 与提示段,因此冷读子 agent 的历史时必须重建那份组装,而不是部署默认值。该值从父方**活着的** scope 链读取,而不是从父方 header 读取,因为在空白期切换过 preset 的父方运行在更新的那份组装上,而它的 header 仍写着旧的那个。 -可继续创建对应可选的 `SubagentProvider.prepareContinuable?()` 方法:方法是否存在就是能力检查,因此服务会在没有该方法的提供方上拒绝已配置的可继续启动,而具备该方法的提供方仍可服务普通一次性委派。该方法只返回分离的 `ContinuableCreateSpec`(`{ seed? }`)——这是数据,绝非能力:它不携带任何 Agent、`AgentHandle`、提示词投递、结果、dispose 或恢复操作,因为准备之后,继续执行管理器拥有身份预留、组合、Agent 创建、提示词投递、冷恢复、所有权和 dispose。一次性 `SubagentRun` 表示一次可 dispose 的前台委派,只有一个结果,且没有冷恢复操作。服务可以针对不同的同级子 agent 并发调用同一提供方:每次启动或准备都拥有各自的可变状态和取消路径,一项操作的失败、结果或清理不得使另一项操作结算或释放。提供方可以在内部按自身容量排队,但不得改变这项独立性约定。 +可继续创建对应可选的 `SubagentProvider.prepareContinuable?()` 方法:方法是否存在就是能力检查,因此服务会在没有该方法的提供方上拒绝已配置的可继续启动,而具备该方法的提供方仍可服务普通一次性委派。该方法只返回已分离的 `ContinuableCreateSpec`(`{ seed? }`)。它只是数据,不携带任何能力:不包含 Agent、`AgentHandle`、提示词投递、结果、dispose 或恢复操作。准备完成后,身份预留、组合、Agent 创建、提示词投递、冷恢复、所有权和 dispose 均由继续执行管理器负责。一次性 `SubagentRun` 表示一次可 dispose 的前台委派,只有一个结果,且没有冷恢复操作。服务可以针对不同的同级子 agent 并发调用同一提供方:每次启动或准备都拥有各自的可变状态和取消路径,一项操作的失败、结果或清理不得使另一项操作结算或释放。提供方可以在内部按自身容量排队,但不得改变这项独立性约定。 ## 持久化描述符 @@ -52,43 +53,45 @@ subagent seam 允许一个 agent(智能体)通过具名提供方把工作委 ## 委派深度 -该 seam 拥有 Service Provider 和 Consumer 共享的深度词汇:`AgentOptions.subagentDepth` 声明、`assertSubagentMaxDepth` 和 `delegationDepthOf(agent)`。持久化的 `SessionHeader.delegationDepth` 具有权威性且单调:运行时选项可以加深计数,但绝不能降低它,因此恢复后的子 agent 不会被重新计为顶层。 +该 seam 拥有 Service Provider 和 Consumer 共享的深度词汇:`AgentOptions.subagentDepth` 声明、`assertSubagentMaxDepth` 和 `delegationDepthOf(agent)`。持久化的 `SessionHeader.delegationDepth` 具有权威性且单调:运行时选项可以增大委派深度,但绝不能将其降到这个下界以下,因此恢复后的子 agent 不会被重新计为顶层。 `inheritsParentContext` 只用于描述,不能强制执行。它仅说明子 agent 是否能看到父级已完成的对话历史(`fork` 可以;`spawn` 和各进程外一次性提供方不可以),不表示是否继承工具、服务或权限。 + + ## 委派策略 -两条进程内委派路径都会通过共享的子 agent 辅助函数,在委派边界固定子 agent 的权限范围。`captureDelegatedPolicyOverrides(parent)` 对父会话的显式沙箱覆盖项(`sandboxPolicy.overrideOf()`)获取快照,并在审批能力已组合时把子 agent 的审批策略钉定为 `'never'`——无论父级自身的策略是什么——因此被委派的子 agent 只在其继承的沙箱范围内行动,每次请求(例如一次 `sandbox_permissions` 升级)都被确定性拒绝,而不是等待一个无人在看的提示(这两个服务都是可选的 `ctx.get` 消费方)。`appendDelegatedPolicyOverrides()` 则在未发布的设置阶段、在任何 fork 种子之后,把每个值作为一条 `source: 'delegation'` 的 `sandbox/mode` 或 `approval/policy` 事件写入子 agent 自己的日志:因此新鲜策略压过陈旧的种子状态,而子 agent 的生效策略始终可以仅凭其日志重建。沙箱的部署默认值绝不复制:未切换的父级不会记录 `sandbox/mode`,其子 agent 会动态跟随部署默认值。可继续启动会在其第一次 await 之前捕获,并且只为新鲜的物化写入种子;冷恢复会重放已持久化的委派事件,而不是重新捕获父级,因此创建之后的父级切换绝不会追溯性地改变持久化子 agent。每个进程内子 agent 还会收到一条作用域内的运行时上下文声明(`subagent:delegation`),告知其权限范围已固定,需要更宽访问的任务应以上报限制收尾,而不是重试。参见[一次性](../../../.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.md)与[可继续](../../../.agents/notes/implemented/feature/2026-08-10-continuable-subagent-policy-inheritance.md)两篇委派策略 Agent Note。 +两条进程内委派路径都会通过共享的子 agent 辅助函数,在委派边界固定子 agent 的权限范围。`captureDelegatedPolicyOverrides(parent)` 会为父会话的显式沙箱覆盖项(`sandboxPolicy.overrideOf()`)创建快照,并在审批能力已组合时将子 agent 的审批策略固定为 `'never'`,无论父级自身采用何种策略。这样,被委派的子 agent 只能在继承的沙箱范围内行动,每次审批请求(例如 `sandbox_permissions` 升权)都会被确定性拒绝,而不会等待无人处理的提示(这两个服务都是可选的 `ctx.get` 消费方)。`appendDelegatedPolicyOverrides()` 则在未发布的设置阶段、在任何 fork 种子之后,把每个值作为一条 `source: 'delegation'` 的 `sandbox/mode` 或 `approval/policy` 事件写入子 agent 自己的日志。因此,新捕获的策略会覆盖种子中的陈旧状态,而子 agent 的生效策略始终可以仅凭其日志重建。沙箱的部署默认值绝不复制:未切换的父级不会记录 `sandbox/mode`,其子 agent 会动态跟随部署默认值。可继续启动会在第一次 await 前捕获策略,并且只为全新物化写入这些委派事件;冷恢复只会重放已持久化的委派事件,不会重新捕获父级策略,因此创建之后的父级切换绝不会追溯性地改变持久化子 agent。每个进程内子 agent 还会收到一条作用域内的运行时上下文声明(`subagent:delegation`),告知其权限范围已固定,需要更宽访问的任务应以上报限制收尾,而不是重试。参见[一次性](../../../.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.zh.md)与[可继续](../../../.agents/notes/implemented/feature/2026-08-10-continuable-subagent-policy-inheritance.zh.md)两篇委派策略 Agent Note。 ## 一次性所有权与生命周期 -`provider.start(request): Promise` 是所有权转移边界;委派工具也会在其由 Task 支撑的一次性后台路径中使用它。兑现前,提供方拥有设置过程,并且每次失败时都必须取消、回滚并使未发布资源完全停稳。兑现后,调用方拥有该运行,并且必须在每条路径上调用 `dispose()`;剩余提示词和轮次工作属于 `SubagentRun.result`。 +`provider.start(request): Promise` 是所有权转移边界;委派工具也会在其由 Task 支撑的一次性后台路径中使用它。兑现前,提供方拥有设置过程,并且在任何失败路径上都必须取消、回滚并使尚未发布的资源完全停稳。兑现后,run 的所有权转移给调用方;调用方必须在每条路径上调用 `dispose()`。剩余提示词和轮次工作属于 `SubagentRun.result`。 -`SubagentRun.result` 兑现为 `{ output, structured?, stopReason }`。子 agent 级失败会以非 `completed` 原因兑现;只有 seam 无法表示的基础设施故障才可以拒绝。`dispose()` 是幂等的,会取消剩余工作,并等待结果结算以及子 agent 资源完全停稳。`result` 的 rejection 仍归 `result` 通道;只有独立的资源释放失败会使 `dispose()` 拒绝。`output` 与 `subagent/end` 事件的 `lastAssistantMessage` 使用导出的 `AssistantOutputFold`/`finalAssistantOutput` 辅助函数选取子 agent 最后一条非空 assistant 消息;若没有这类消息,则选取其累积的 assistant 文本。子 agent 两种输出均未产生时,`output` 为 `[]`,该事件字段缺省(结果约定归 [`SubagentResult.output`](../../../docs/subsystems/subagent.md#the-terminal-result-subagentresult) 所有)。 +`SubagentRun.result` 兑现为 `{ output, structured?, diagnostic?, stopReason }`。子 agent 级失败会以非 `completed` 原因兑现;只有 seam 无法表示的基础设施故障才可以拒绝。提供方可以为非完成结果附加安全的 `diagnostic`:它会先排除工具输入、文件内容、环境值、凭证与原始协议载荷,并把完整文本限制在 4096 个 UTF-8 字节以内。共享结果类型不定义提供方类别或生命周期阶段:进程外提供方可以从锁定版本产品提供的结构化事实与已观测的进程结果派生固定展示文本,而消费方只负责原样呈现,不解析该文本。该字段不是 assistant 输出;消费方会将它分开呈现,它也不会进入 `subagent/end.lastAssistantMessage`。`dispose()` 是幂等的,会取消剩余工作,并等待结果结算以及子 agent 资源完全停稳。result 的拒绝只通过 `result` 本身报告;只有独立的资源释放失败,才会使 `dispose()` 被拒绝。`output` 与 `subagent/end` 事件的 `lastAssistantMessage` 使用导出的 `AssistantOutputFold`/`finalAssistantOutput` 辅助函数选取子 agent 最后一条非空 assistant 消息;若没有这类消息,则选取其累积的 assistant 文本。子 agent 两种输出均未产生时,`output` 为 `[]`,该事件字段缺省(终态结果约定归 [`SubagentResult`](../../../docs/subsystems/subagent.zh.md#the-terminal-result-subagentresult) 所有)。 本地运行会在 `start()` 兑现前发布普通的子 agent/会话,把该共享会话 id 作为 `SubagentRun.id` 返回,以 `SubagentRun.localAgent` 公开准确的子 agent,把 `request.parent.session.id` 记录到子 agent 的 `parentSession` header,并在其初始轮次内追加已解析的描述符。远程提供方则生成 parent 作用域的生命周期 id,并返回 `localAgent: undefined`;由于没有本地 child 会话,其一次性运行不会进入基于追踪的枚举结果。 ## 可继续子 agent 与 Activation -可继续子 agent 拥有一个持久化会话和至多一个进程内 **Activation**——即被重建的子 agent 的一个驻留时段,而不是请求、结果、取消或 Task 边界。Agent inbox 是唯一的轮次队列,因此继续执行管理器负责驻留,而 agent loop(智能体循环)负责所有轮次排序与执行。任何可继续路径都不会创建 Task 或中间的承载结果的包装层。 +每个可继续子 agent 都有一个持久化 Session,并且同一时刻至多有一个进程内 **Activation**。Activation 表示重建后的子 agent 的一次驻留时段,不是请求、结果、取消或 Task 的边界。Agent inbox 是唯一的轮次队列,因此驻留归继续执行管理器,所有轮次排序与执行归 agent loop(智能体循环)。任何可继续路径都不会创建 Task 或中间的承载结果的包装层。 -管理器根据 Agent 完全停稳状态和所拥有的子 agent 集合推导三个内部驻留条件,而非维护第二个状态机:running(存在活跃准入、进行中的轮次或唤醒型 inbox 工作)、waiting(已停稳但仍拥有至少一个未 dispose 的子 agent)、settled(已停稳且所有拥有的子 agent 都已 dispose,因此管理器 dispose `AgentHandle` 并移除 Activation)。每条后续消息都使用 `Agent.followup()` 并成为一个 FIFO 轮次,且不会对当前轮次进行 steering(中途引导)。路由只取决于驻留状态:running 入队、waiting 唤醒同一 Agent,无 Activation 时则冷恢复一个新的。 +管理器根据 Agent 的完全停稳状态和所拥有的子级集合推导三种内部驻留状态,而不维护第二套状态机:running 表示存在正在进行的准入、尚未结束的轮次,或会唤醒 Agent 的 inbox 工作;waiting 表示 Agent 已完全停稳,但仍拥有至少一个尚未 dispose 的子级;settled 表示 Agent 已完全停稳且所有拥有的子级均已 dispose,此时管理器会 dispose `AgentHandle` 并移除 Activation。每条后续消息都使用 `Agent.followup()` 并成为一个 FIFO 轮次,且不会对当前轮次进行 steering(中途引导)。路由只取决于驻留状态:running 入队、waiting 唤醒同一 Agent,无 Activation 时则冷恢复一个新的。 管理器预留子 agent 身份、解析持久化描述符,通过私有的 activation-owner 作用域调用 `ctx.agents.create()`(冷恢复时为 `ctx.agents.resume()`),把返回的 `AgentHandle` 安装到 Activation 中,建立任何可继续父级所有权,然后提交提示词。冷恢复绝不通过提供方分发,因为持久化会话已持有初始前缀,折叠后的描述符即是全部重建输入。 ### 结算投递 -当一个驻留 Activation 结算时,管理器会在父级自身的轮次流中告知该子级持久化的直接父级:这个子级已经产出它将产出的全部内容。对每个调用方真正拿到过 id 的子级,这条投递都是无条件的:它不考虑该子级是否调用过 `report`,因为最需要这条投递的结束方式——token 上限、模型失败、取消、拆卸——恰恰是子级根本没有机会选择的那些。在第一条消息被接受之前就回滚的物化保持静默,因为那位调用方已被告知该子级未建立。消息会携带该 epoch 的终止原因、它产出过的最终 assistant 内容,以及持久化来源 `{ kind: 'subagent-settled', form: 'notice', senderSessionId: }`——与子级自撰的 `subagent-report` 是不同的来源 kind,因此 transcript(文本记录)绝不会把运行时写下的话算到子级头上。 +当一个驻留 Activation 结算时,管理器会在父级自身的轮次流中告知该子级持久化的直接父级:这个子级已经产出它将产出的全部内容。对于每个已经向调用方返回过 id 的子级,管理器都会无条件投递结算通知,不考虑该子级是否调用过 `report`。最需要说明结局的终止情形,包括达到 token 上限、模型失败、取消或拆卸,恰恰是子级根本没有机会选择的那些情形。在第一条消息被接受之前就回滚的物化保持静默,因为那位调用方已被告知该子级未建立。消息会携带该 epoch 的终止原因、它产出过的最终 assistant 内容,以及持久化来源 `{ kind: 'subagent-settled', form: 'notice', senderSessionId: }`——与子级自撰的 `subagent-report` 是不同的来源 kind,因此 transcript(文本记录)绝不会把运行时写下的话算到子级头上。 -有两条顺序规则让这条投递可靠而非侥幸,它们也正是这件事属于管理器而非外部 `subagent/end` listener 的原因。第一,发送发生在子级所有权释放**之前**,此时父级仍然计入该子级,因此在结构上不可能被判定为已结算。第二,本身就是驻留 Activation 的父级会通过与 report 相同的唤醒准入记账接收该消息,因此同步发送与承认它的那个 microtask 之间的窗口不会被误判为静止——`Agent.status` 会把上下文维护折叠成 `idle`,而维护期间的唤醒发送只会预置一次延后唤醒。缺少其中任一条规则,父级都可能在通知仍留在 inbox 时被 dispose,而 `cancel()` 会清空该 inbox,于是通知被静默丢失。 +有两条顺序规则让这条投递可靠而非侥幸,它们也正是这件事属于管理器而非外部 `subagent/end` listener 的原因。第一,发送发生在子级所有权释放**之前**,此时父级仍然计入该子级,因此在结构上不可能被判定为已结算。第二,如果父级本身也是驻留 Activation,该消息会采用与 report 相同的唤醒准入记账。这样,从同步发送消息到负责准入该消息的 microtask 运行之间的窗口,不会被误判为完全停稳——`Agent.status` 会把上下文维护折叠成 `idle`,而维护期间的唤醒发送只会预置一次延后唤醒。缺少其中任一条规则,父级都可能在通知仍留在 inbox 时被 dispose,而 `cancel()` 会清空该 inbox,于是通知被静默丢失。 -空闲父级会以一个普通的后续轮次收到该通知。繁忙父级则被 steer 到其最近的 step 边界,因此同时结算的多个子级只消耗一个 step,而不是各自一个轮次;采用 steer 而非 inject 还意味着:即便驱动在状态读取与发送之间退出,该消息仍会被认领。若父级自身所在的谱系已在 draining,则该通知改为 inject 送达,完全不唤醒:对静息父级调用 `Agent.followup()` 会开启一个轮次,而 `cancel()` 不会对之后的轮次设防,因此在拆卸期间唤醒,会在宿主即将 dispose 的 Agent 上白花一次模型请求——而且每层树各一次,因为每层自己的通知又会唤醒它上面那层。被 inject 的消息会送达仍在读取自身 inbox 的父级,而无论如何日志都会记录这份记账;但它不会比该父级自身的 dispose 活得更久:`AgentHandle.dispose()` 是一次 `keepInbox: false` 的 cancel,会持久地取消尚未被认领的通知。因此 resume 后的父级没有待处理通知可读:`list_agents` 只告诉它有哪些子级、各自是在线还是仅存于存储;结局本身留在子级自己的 Session 里,一次 `send_message` 会通过 resume 该子级把它取回。已离开注册表的父级不算错误:通知被丢弃,子级自身的 Session 仍是持久记录。投递绝不会阻塞或使拆卸失败——发送被拒只会记录日志,因为为了重试一条通知而保留子级,会把它的整条祖先链永久钉在 `waiting` 上。 +空闲父级会以一个普通的后续轮次收到该通知。繁忙父级则被 steer 到其最近的 step 边界,因此同时结算的多个子级只消耗一个 step,而不是各自一个轮次;采用 steer 而非 inject 还意味着:即便驱动在状态读取与发送之间退出,该消息仍会被认领。如果父级所在的谱系已经开始排空,该通知会通过 inject 投递,且完全不会唤醒父级。对已经完全停稳的父级调用 `Agent.followup()` 会开启新轮次,而 `cancel()` 不会预先阻止之后开启的轮次;因此在拆卸期间唤醒父级,会让宿主即将 dispose 的 Agent 多执行一次模型请求,而且树的每一层各一次,因为每层自己的通知又会唤醒上一层。被 inject 的消息会送达仍在读取自身 inbox 的父级,而无论如何日志都会记录这份记账;但它不会比该父级自身的 dispose 活得更久:`AgentHandle.dispose()` 是一次 `keepInbox: false` 的 cancel,会持久地取消尚未被认领的通知。因此 resume 后的父级没有待处理通知可读:`list_agents` 只告诉它有哪些子级、各自是在线还是仅存于存储;结局本身留在子级自己的 Session 里,一次 `send_message` 会通过 resume 该子级把它取回。已离开注册表的父级不算错误:通知被丢弃,子级自身的 Session 仍是持久记录。投递绝不会阻塞或使拆卸失败——发送被拒只会记录日志,因为为了重试一条通知而保留子级,会把它的整条祖先链永久钉在 `waiting` 上。 -受继续执行管理的父级 Activation 会在子 agent 能够运行之前,把每个子 agent 的会话 id 记录到 `ownedChildren` 集合中,并且只有在每个所拥有的子 agent Activation 完成 `AgentHandle` dispose 之后才会 dispose(子先于父)。拆卸会先自顶向下传播 Agent 取消,再等待缓慢的后代,而 handle 释放仍保持 child-first。顶层及其他非继续执行的 Agent 没有 Activation,处于该等待图之外。最终结算会在 dispose handle 前等待 best-effort 的 `ctx.sessions.flush(child.session)`。listener rejection 会被记录,但不会使 Activation 失败,因为 listener 是否参与无法标识持久化后端;因此,恢复时持久化状态可能缺失或陈旧。 +受继续执行管理的父级 Activation 会在子 agent 能够运行之前,把每个子 agent 的会话 id 记录到 `ownedChildren` 集合中,并且只有在每个所拥有的子 agent Activation 完成 `AgentHandle` dispose 之后才会 dispose(子先于父)。拆卸会先自顶向下传播 Agent 取消,再等待缓慢的后代,而 handle 释放仍保持 child-first。顶层及其他非继续执行的 Agent 没有 Activation,处于该等待图之外。最终结算会在 dispose handle 前等待 best-effort 的 `ctx.sessions.flush(child.session)`。监听器拒绝会被记录,但不会使 Activation 失败,因为监听器参与本身不能标识持久化后端;因此恢复时的持久化状态仍可能缺失或陈旧。 ## 生命周期事件 -服务会为每次一次性运行以及每个已驻留的可继续 Activation 时段发出一对 `subagent/start`/`subagent/end`,因此可继续子 agent 可用与一次性运行相同的词汇观察,且不会暴露管理器是物化、唤醒还是冷恢复了它们。对于一次性启动,它会在同步的 `subagent/start` 之前附加结果观察器,因此即使子 agent 已经结算,也仍会先产生 `subagent/start`,再产生 `subagent/end`;在驻留前失败的可继续时段不会发出这对生命周期事件中的任何一个。这对事件共享服务生成的 `runId`;`local` 标志取自提供方准确 `localAgent` 的快照(可继续子 agent 恒为 true),因此观察器绝不会从可复用的提供方/会话名称推断运行身份或本地性。`provider` 字段包含子 agent 初次创建时记录的提供方名称,不表示该提供方当前仍在注册:已接受的一次性 run 可在提供方移除后才结算;冷恢复时段会从描述符读取初始提供方名称,不会调用或注册该提供方。 +服务会为每次一次性运行以及每个已驻留的可继续 Activation 时段发出一对 `subagent/start`/`subagent/end`,因此可继续子 agent 可用与一次性运行相同的词汇观察,且不会暴露管理器是物化、唤醒还是冷恢复了它们。对于一次性启动,它会在同步的 `subagent/start` 之前附加结果观察器,因此即使子 agent 已经结算,也仍会先产生 `subagent/start`,再产生 `subagent/end`;在驻留前失败的可继续时段不会发出这对生命周期事件中的任何一个。这对事件共享由服务生成的 `runId`;`local` 标志根据提供方返回的确切 `localAgent` 是否存在取得快照(可继续子级恒为 true),因此观察器不会根据可复用的提供方名称或会话名称推断运行身份或本地性。`provider` 字段包含子 agent 初次创建时记录的提供方名称,不表示该提供方当前仍在注册:已接受的一次性 run 可在提供方移除后才结算;冷恢复时段会从描述符读取初始提供方名称,不会调用或注册该提供方。 运行事件受执行委派的父级作用域约束。每个监听器都独立隔离:同步抛出或返回的 promise 被拒绝时,只会记录日志,不会阻塞同级监听器或改变运行。 @@ -102,12 +105,14 @@ subagent seam 允许一个 agent(智能体)通过具名提供方把工作委 ## 收集模型 -面向模型的工具默认同步收集:先等待子 agent 结果,再 dispose 运行,然后才返回。一次性后台委派会在工具中注册普通 Task,其通用状态、收集和取消工具负责后续交互,并将模型提供的 `description` 持久化为可选显示标签。可继续后台委派会调用 `ctx.subagents.startContinuable()`,只返回持久化子 agent id;子 agent 自 inbox 接受起就拥有自己的轮次,因此没有 Task、也没有结果 promise——调用方通过 `send_message` 后续操作工具发送后续工作,`interrupt()` 只停止当前轮次而不 dispose 子 agent,而持久化子 agent 会话仍是子 agent 详细输出的来源。只有 `ctx.agents` 可用时,继续执行管理器才会存在,而会话持久化按每项继续执行操作解析。与此独立,`listChildren()` 枚举在线会话存储与可选会话持久化的在线优先合并——持久化缺席时仅枚举在线 child,因为那时冷 child 本就无法恢复——并由已注册的 `subagent` 投影单元供给每个 child 的持久化模式与标签:在线 child 取注册表的水位快照;冷 child 先取可选投影缓存的持久化行,且仅当其 `seq` 门证明该值折叠自 child 自身后缀(fork 种子之后——自有描述符一经追加即不可变)才直接采用,否则经一次有界并发的持久化 inspect 再经注册表折叠,且 inspect 结果必须仍指向枚举时的生命周期(同 id 被重新发布的会话降级为 `corrupt` diagnostic)。缓存读取抛错不产生判决——缓存是派生数据——静默落到该权威重折。投影折叠是唯一的分类权威;列表自身不解析任何描述符。取得身份值即产出 child 行;已定局而折叠未产出身份的候选是 `corrupt` diagnostic,inspect 失败是瞬时的 `unavailable`(下次列表重试),运行中而暂无身份值的候选整行省略(描述符尚未追加的创建窗口)。它不查询继续执行管理器、Agent 注册信息、Activation 或提供方。每个 child 行都会根据合并结果中携带持久化 `origin: 'subagent'` 的 header 派生读取时的 `hasChildren` 提示;它不会读取后代事件日志,展开后仍以描述符支撑的 child 目录为权威依据。UI 等服务消费方可以保留两种模式,并为无标签的一次性 child 选择回退展示;面向模型的 `list_agents` 工具只投影 `continuable` 条目,通过在线 Agent 注册表细化状态,并把仅存于存储的状态映射为可恢复而非终态的 `ready`(`running`/`idle`/`ready`),并在 `descendants` scope 下遍历 `listDescendants()`。列表操作会把调用方的取消信号转发到每次持久化读取,在这些 await 前后检查取消,并将每次检测到的中止报告为 `SubagentError` 错误码 `CANCELLED`;投影注册表未挂载则以 `SUBAGENT_CONTROL_PROJECTIONS_UNAVAILABLE` 响亮失败,会话存储缺失则以 `SUBAGENT_CONTROL_SESSION_STORE_UNAVAILABLE` 响亮失败。完整约定见[后台 subagent 任务 Agent Note](../../../.agents/notes/implemented/feature/2026-07-08-background-subagent-tasks.md)、[可继续后台 subagent Agent Note](../../../.agents/notes/implemented/feature/2026-07-21-continuable-background-subagents.md)、[持久化目录 Agent Note](../../../.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.md)、[服务合并 Agent Note](../../../.agents/notes/implemented/simplification/2026-07-26-merge-subagent-control-service.md)、[能力 seam Agent Note](../../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.md)和 `src/types.ts`。 +面向模型的工具默认同步收集:先等待子 agent 结果,再 dispose 运行,然后才返回。一次性后台委派会在工具中注册普通 Task,其通用状态、收集和取消工具负责后续交互,并将模型提供的 `description` 持久化为可选显示标签。可继续后台委派会调用 `ctx.subagents.startContinuable()`,只返回持久化子 agent id;子 agent 自 inbox 接受起就拥有自己的轮次,因此没有 Task、也没有结果 promise——调用方通过 `send_message` 后续操作工具发送后续工作,`interrupt()` 只停止当前轮次而不 dispose 子 agent,而持久化子 agent 会话仍是子 agent 详细输出的来源。只有 `ctx.agents` 可用时,继续执行管理器才会存在,而会话持久化按每项继续执行操作解析。与此独立,`listChildren()` 枚举在线会话存储与可选会话持久化的在线优先合并——持久化缺席时仅枚举在线 child,因为那时冷 child 本就无法恢复——并由已注册的 `subagent` 投影单元供给每个 child 的持久化模式与标签:在线 child 取注册表的水位快照;冷 child 先取可选投影缓存的持久化行,且仅当其 `seq` 门证明该值折叠自 child 自身后缀(fork 种子之后——自有描述符一经追加即不可变)才直接采用,否则经一次有界并发的持久化 inspect 再经注册表折叠,且 inspect 结果必须仍指向枚举时的生命周期(同 id 被重新发布的会话降级为 `corrupt` diagnostic)。缓存读取抛出异常时,不会据此作出分类判断,因为缓存只是派生数据;静默落到该权威重折。分类结果完全以投影折叠为准;列表操作本身不解析描述符。取得身份值即产出 child 行;已定局而折叠未产出身份的候选是 `corrupt` diagnostic,inspect 失败是瞬时的 `unavailable`(下次列表重试),运行中而暂无身份值的候选整行省略(描述符尚未追加的创建窗口)。它不查询继续执行管理器、Agent 注册信息、Activation 或提供方。每个 child 行都会根据合并结果中携带持久化 `origin: 'subagent'` 的 header 派生读取时的 `hasChildren` 提示;它不会读取后代事件日志,展开后仍以描述符支撑的 child 目录为权威依据。UI 等服务消费方可以保留两种模式,并为无标签的一次性 child 选择回退展示;面向模型的 `list_agents` 工具只投影 `continuable` 条目,通过在线 Agent 注册表细化状态,并把仅存于存储的状态映射为可恢复而非终态的 `ready`(`running`/`idle`/`ready`),并在 `descendants` scope 下遍历 `listDescendants()`。列表操作会把调用方的取消信号转发到每次持久化读取,在这些 await 前后检查取消,并将每次检测到的中止报告为 `SubagentError` 错误码 `CANCELLED`;投影注册表未挂载则以 `SUBAGENT_CONTROL_PROJECTIONS_UNAVAILABLE` 响亮失败,会话存储缺失则以 `SUBAGENT_CONTROL_SESSION_STORE_UNAVAILABLE` 响亮失败。完整约定见[后台 subagent 任务 Agent Note](../../../.agents/notes/implemented/feature/2026-07-08-background-subagent-tasks.zh.md)、[可继续后台 subagent Agent Note](../../../.agents/notes/implemented/feature/2026-07-21-continuable-background-subagents.zh.md)、[持久化目录 Agent Note](../../../.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.zh.md)、[服务合并 Agent Note](../../../.agents/notes/implemented/simplification/2026-07-26-merge-subagent-control-service.zh.md)、[能力 seam Agent Note](../../../.agents/notes/implemented/feature/2026-06-21-subagent-capability-seam.zh.md)和 `src/types.ts`。 可继续 Activation 会等待 best-effort 的最终会话 flush,但不会把 listener 参与视为持久性确认。一次性运行保留尽力执行的会话检查点,因此已完成的一次性 child 只有在其会话确实进入持久化存储时,才可在 dispose 后继续被发现;如果该检查点缺失,服务不会根据 Task 历史虚构目录条目。 ## 模型体验 + + ### 结算通知 #### 模型看到的内容 @@ -116,7 +121,7 @@ subagent seam 允许一个 agent(智能体)通过具名提供方把工作委 #### Token 影响 -父级请求中,每个已结算的 Activation 一条通知,长度取决于子级的最终消息。既上报又结算的子级会让父级同时支付两份。 +父级请求中,每个已结算的 Activation 一条通知,长度取决于子级的最终消息。如果子级既上报又结算,父级请求会同时承担上报消息和结算通知两部分 token 开销。 #### KV Cache 影响 @@ -146,7 +151,7 @@ You are a delegated subagent: your permission scope was fixed when you were star - **ACP 子 agent 仍为一次性,且无法通过追踪枚举**:ACP 运行在 parent 会话语料中没有本地 child 会话。ACP 的 `prepareContinuable` 需要在提供方专用描述符数据中持久化远端会话 id,以及逐子 agent 的继续执行能力声明,因为 ACP 的 `loadSession` 支持按子 agent 协商,而不是通过方法是否存在来确定。远程提供方还需要一份独立的 Activation 所有权约定,具备等效的经认证控制和子先于父的完全停稳保证,才能支持可继续子 agent。 - **无 host-user 继续执行**:`followup()` 要求确切在线直接父级。只有 `interrupt()` 接受持久化 parent 地址形式的用户授权,因为停止一个轮次是幂等的且不投递任何内容;未来 host 适配器需要具体的经认证交互,才能让该 seam 获得用户投递能力。 -- **不对当前轮次进行 steering**:可继续消息和唤醒式 report 会排入后续轮次,均不会重定向正在进行的轮次。 +- **继续执行消息绝不 steering**:parent 到 child 的继续执行消息会排入后续 child 轮次。child 到 parent 的 report 是独立的 next-step 输入,可能延长 parent 已打开的轮次。 - **取消收敛期间存在唤醒缺口**:中断信号发出后、活动 driver 进入 idle 前被接受的唤醒型 follow-up 会保持排队,直到另一条唤醒发送到达。Issue #1838 负责 agent-loop 的唤醒锁存;普通会话取消也受此影响。 - **驻留仅限进程内**:Activation inbox 与所有权图不会在两个 harness 进程之间协调;对单个持久化存储的并发访问仍然需要持久化邮箱和跨进程租约协议。 - **不回放已接受但未记录的消息**:只有写入子 agent 会话日志的消息才能连同提供该消息的来源一起重建。崩溃可能丢失从未写入日志、已被接受的初始提示词或后续消息;此后一条经授权的消息可以冷恢复该子 agent,但丢失的消息不会自动回放。 diff --git a/packages/subagent/subagent/package.json b/packages/subagent/subagent/package.json index 87ec069782..269268dc6a 100644 --- a/packages/subagent/subagent/package.json +++ b/packages/subagent/subagent/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subagent", "description": "Abstract subagent seam (ctx.subagents): named-provider registry for delegating to child agents", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subagent/subagent/src/continuation.ts b/packages/subagent/subagent/src/continuation.ts index d826efbfdb..652a3ba6c8 100644 --- a/packages/subagent/subagent/src/continuation.ts +++ b/packages/subagent/subagent/src/continuation.ts @@ -98,7 +98,7 @@ declare module '@deepseek-ai/dsh-llm' { } /** Deployment scheduling policy for accepted child reports. */ -export type SubagentReportDelivery = 'quiet' | 'wakeup' +export type SubagentReportDelivery = 'quiet' | 'next-step' /** Options for one continuable child's report to its direct parent. */ export interface SubagentReportOptions { @@ -114,6 +114,12 @@ export interface ContinuableStartSpec { readonly provider: string /** The initial delegation's short `description`, persisted as the child's creation label. */ readonly label: string + /** + * Optional caller-reserved child identity. Omission preserves the manager's + * UUID allocation; supplying one lets a durable parent record provisioning + * before child materialization without a second identity handshake. + */ + readonly childId?: SessionId /** * The delegation request. The manager reserves the stable child id, resolves * the durable descriptor, and composes the child itself. @@ -404,9 +410,10 @@ export class SubagentContinuationManager { const request = spec.request const parent = request.parent this.assertAdmitting(parent) - this.requirePersistence() + const persistence = this.requirePersistence() assertSubagentMaxDepth(request.maxDepth) - const childId = SessionId(randomUUID()) + const childId = spec.childId ?? SessionId(randomUUID()) + this.assertChildIdAvailable(childId) const childDepth = resolveChildDepth(parent, request.maxDepth) // Snapshot before any await: invalid descriptor JSON rejects the call // before a child exists, and the detached value is what reaches the log. @@ -436,6 +443,18 @@ export class SubagentContinuationManager { const lineageSeedLength = prepared.seed?.length ?? 0 const seed = seedDescriptorTurn(childId, prepared.seed, descriptor) const messageId = await this.locks.run(childId, async () => { + spec.signal.throwIfAborted() + this.assertAdmitting(parent) + this.assertChildIdAvailable(childId) + if (spec.childId !== undefined) { + const persisted = await persistence.listSnapshots(spec.signal) + spec.signal.throwIfAborted() + this.assertAdmitting(parent) + this.assertChildIdAvailable(childId) + if (persisted.some(snapshot => snapshot.header.id === childId)) { + throw new SubagentError(`subagent "${childId}" already exists`, 'DUPLICATE_CHILD') + } + } const activation = await this.materialize({ childId, provider: spec.provider, @@ -456,6 +475,13 @@ export class SubagentContinuationManager { return { childId, messageId } } + /** Reject one child identity already owned by a live Agent or Session. */ + private assertChildIdAvailable(childId: SessionId): void { + if (this.ctx.agents.get(childId) !== undefined || this.ctx.get('sessions')?.get(childId) !== undefined) { + throw new SubagentError(`subagent "${childId}" already exists`, 'DUPLICATE_CHILD') + } + } + /** * Deliver one later message to a known continuable child as its next FIFO * turn. Routing depends only on Activation residency: a `running` Activation @@ -644,7 +670,7 @@ export class SubagentContinuationManager { senderSessionId: activation.childId, }, }) - if (delivery === 'wakeup') { + if (delivery === 'next-step') { this.sendWaking(parent, message, () => { this.sendReport(parent, message, delivery) }) } else { this.sendReport(parent, message, delivery) @@ -656,7 +682,7 @@ export class SubagentContinuationManager { * Perform one waking send to a parent, accounted against that parent's own * Activation when it has one. Registering the id before the send is what * keeps a continuation-managed parent from being judged quiescent in the - * window between `followup()` and the microtask that admits it. + * window between a waking send and the microtask that admits it. * @param parent - the exact live parent receiving the waking message. * @param message - the message whose id is accounted. * @param send - the synchronous waking send to perform. @@ -681,7 +707,7 @@ export class SubagentContinuationManager { delivery: SubagentReportDelivery, ): void { try { - if (delivery === 'wakeup') parent.followup(message) + if (delivery === 'next-step') parent.steer(message) else parent.inject(message) } catch (error: unknown) { throw new SubagentError( @@ -779,10 +805,46 @@ export class SubagentContinuationManager { await this.disposeRoots(targetRoots, 'scoped activation(s)') } + /** + * Release selected resident direct children of one exact live parent without + * closing admission for the parent's other continuable children. Owned + * descendants are released recursively through the same lifecycle. + * @param parent - exact live direct parent authorizing the selected release. + * @param childIds - durable direct-child ids to release when resident. + * @returns once every selected Activation released its handle. + * @throws {SubagentError} `UNAUTHORIZED` when a resident target is not the + * parent's direct continuable child or the parent identity is stale. + */ + async drainChildren(parent: Agent, childIds: readonly SessionId[]): Promise { + if (this.ctx.agents.get(parent.id) !== parent) { + throw new SubagentError('selected child teardown requires the exact live parent agent', 'UNAUTHORIZED') + } + const targets: Activation[] = [] + for (const childId of new Set(childIds)) { + const activation = this.activations.get(childId) + if (activation === undefined) continue + if (activation.parentSession !== parent.id || !activation.ancestry.has(parent)) { + throw new SubagentError( + `subagent "${childId}" is not a direct child of agent "${parent.id}"`, + 'UNAUTHORIZED', + ) + } + targets.push(activation) + } + + // Open every transaction before the first await so cancellation propagates + // across the selected roots in one synchronous span. + for (const activation of targets) { + const disposal = this.dispose(activation) + void disposal.catch(() => undefined) + } + await this.disposeRoots(targets, 'selected activation(s)') + } + /** Dispose independent roots and report every branch failure after all settle. */ private async disposeRoots( roots: readonly Activation[], - failureSubject: 'activation(s)' | 'scoped activation(s)', + failureSubject: 'activation(s)' | 'scoped activation(s)' | 'selected activation(s)', ): Promise { const failures = await Promise.all(roots.map(async (activation) => { try { @@ -1158,7 +1220,7 @@ export class SubagentContinuationManager { messageId: MessageId, send: () => void, ): MessageId { - // `Agent.followup()` publishes inbox events synchronously, so observers must + // Waking Agent sends publish inbox events synchronously, so observers must // see this Activation as busy before the call begins. activation.accepted.add(messageId) try { diff --git a/packages/subagent/subagent/src/index.ts b/packages/subagent/subagent/src/index.ts index fe270f1b4c..2f29e32010 100644 --- a/packages/subagent/subagent/src/index.ts +++ b/packages/subagent/subagent/src/index.ts @@ -4,10 +4,8 @@ * child before returning its run, so fulfillment is the single publication and * ownership-transfer boundary. * - * Unlike the bash seam (one executor per context, second load throws), MULTIPLE - * providers coexist here: each registers under a unique name and a caller picks - * one by name. The shape mirrors the LLM adapter registry - * (`LlmRuntime.registerAdapter`), not the single-service bash executor. + * Multiple providers coexist: each registers under a unique name and callers + * select one by name. * * This package owns the Service Definition role of the capability seam. Service Providers * (`@deepseek-ai/dsh-subagent-spawn-in-process`, `-fork`, `-acp`) and the model-facing @@ -284,7 +282,7 @@ export class SubagentRuntime extends Service { * @returns the exact Cordis effect disposer. */ registerContinuableSetup(contribution: ContinuableSetupContribution): () => void { - // oxlint-disable-next-line typescript/no-misused-promises -- synchronous cleanup; direct return preserves disposer identity + // oxlint-disable-next-line typescript/no-misused-promises -- synchronous disposer return this.ctx.effect( () => this.setupRegistry.register(contribution), 'subagents.registerContinuableSetup()', @@ -308,6 +306,22 @@ export class SubagentRuntime extends Service { await manager.drainDescendants(parents) } + /** + * Release selected resident continuable direct children of one exact live + * parent. Other children of the same parent remain admitted and resident. + * Absent targets and a manager-less composition are accepted no-ops. + * @param parent - exact live direct parent authorizing the selected release. + * @param childIds - durable direct-child ids to release when resident. + * @returns once every selected Activation released its `AgentHandle`. + * @throws {SubagentError} `UNAUTHORIZED` when a resident target belongs to a + * different parent or the supplied parent identity is stale. + */ + async drainContinuableChildren(parent: Agent, childIds: readonly SessionId[]): Promise { + const manager = this.continuations + if (manager === undefined) return + await manager.drainChildren(parent, childIds) + } + /** * Enumerate the parent's direct session-backed subagents without loading or * resuming an Agent and without any query service: the listing merges the live @@ -368,7 +382,7 @@ export class SubagentRuntime extends Service { */ registerProvider(provider: SubagentProvider): () => void { const name = provider.name - // oxlint-disable-next-line typescript/no-misused-promises -- synchronous cleanup; direct return preserves disposer identity + // oxlint-disable-next-line typescript/no-misused-promises -- synchronous disposer return this.ctx.effect(function* (this: SubagentRuntime) { if (this.providers.has(name)) { throw new SubagentError(`a subagent provider named "${name}" is already registered`, 'DUPLICATE_PROVIDER') diff --git a/packages/subagent/subagent/src/out-of-process.ts b/packages/subagent/subagent/src/out-of-process.ts index d049dba2be..abb6dd50e7 100644 --- a/packages/subagent/subagent/src/out-of-process.ts +++ b/packages/subagent/subagent/src/out-of-process.ts @@ -16,6 +16,31 @@ import { isAbsolute, resolve } from 'node:path' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { SubagentCapabilities, SubagentResult, SubagentRun, SubagentStopReason } from './types.ts' +/** Maximum UTF-8 size of {@link SubagentResult.diagnostic}. */ +const MAX_SUBAGENT_DIAGNOSTIC_BYTES = 4_096 + +const DIAGNOSTIC_TRUNCATION_SUFFIX = '\n[diagnostic truncated]' +const utf8Encoder = new TextEncoder() +const utf8Decoder = new TextDecoder() + +/** + * Limit provider-authored failure detail without splitting a UTF-8 sequence. + * @param diagnostic - safe diagnostic text produced by the provider. + * @returns the original text, or a visibly truncated value within the limit. + */ +function limitSubagentDiagnostic(diagnostic: string): string { + const bytes = utf8Encoder.encode(diagnostic) + if (bytes.byteLength <= MAX_SUBAGENT_DIAGNOSTIC_BYTES) return diagnostic + + const suffixBytes = utf8Encoder.encode(DIAGNOSTIC_TRUNCATION_SUFFIX).byteLength + let prefixBytes = MAX_SUBAGENT_DIAGNOSTIC_BYTES - suffixBytes + while (((bytes[prefixBytes] as number) & 0b1100_0000) === 0b1000_0000) { + prefixBytes -= 1 + } + return utf8Decoder.decode(bytes.subarray(0, prefixBytes)) + + DIAGNOSTIC_TRUNCATION_SUFFIX +} + /** * The capability advertisement of an out-of-process backend: NONE. A child in * another process cannot honor parent-enforced start features @@ -134,6 +159,8 @@ export interface RunResultSettlement { attempt: () => Promise /** Snapshot the provider exposes when cancellation or failure wins settlement. */ collectOutput: () => ContentBlock[] + /** Snapshot safe provider-authored detail when a failure wins settlement. */ + collectDiagnostic?: (() => string | undefined) | undefined /** Whether local cancellation settled before the attempt's outcome is observed. */ cancelled: () => boolean /** Diagnostic sink for a failure flattened to a stop reason; a throw from it is contained. */ @@ -168,7 +195,15 @@ export async function settleRunResult(parts: RunResultSettlement): Promise = z.object({ settledMs: z.number().int().nonnegative(), - active: z.object({ - since: z.number().int().nonnegative(), - through: z.number().int().nonnegative(), - }).strict().optional(), -}).strict() as unknown as z.ZodType + active: activeIntervalSchema.optional(), +}).strict().transform(({ settledMs, active }) => ({ + settledMs, + ...active === undefined ? {} : { active }, +})) + +const timingStateSchema: z.ZodType = z.object({ + settledMs: z.number().int().nonnegative(), + active: activeIntervalSchema.optional(), + pendingTurnStart: z.number().int().nonnegative().optional(), + descriptorSeen: z.boolean(), +}).strict() + +declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + subagentTiming: TimingState + subagent: IdentityState + } +} /** * Fold turn boundaries around the child's own durable descriptor. @@ -41,10 +59,9 @@ const projectionSchema = z.object({ * admits only a child with exactly one descriptor in its own suffix, making * the final reset the child's authoritative timing origin. */ -export const subagentTimingProjectionDefinition: -ProjectionDefinition<'subagentTiming', TimingState> = { +export const subagentTimingProjectionDefinition = { key: 'subagentTiming', - schema: projectionSchema, + stateSchema: timingStateSchema, init: () => ({ descriptorSeen: false, settledMs: 0 }), apply: (state, event) => { if (event.type === 'turn/start') { @@ -78,16 +95,19 @@ ProjectionDefinition<'subagentTiming', TimingState> = { if (state.active === undefined) return state return { ...state, active: { ...state.active, through: event.time } } }, - view: state => ({ - settledMs: state.settledMs, - ...(state.active === undefined ? {} : { active: state.active }), - }), + wire: { + viewSchema: projectionSchema, + view: state => ({ + settledMs: state.settledMs, + ...(state.active === undefined ? {} : { active: state.active }), + }), + }, stateVersion: 2, -} +} satisfies ProjectionDefinition<'subagentTiming', TimingState> interface IdentityState { /** Identity from the last valid descriptor; absent before one, and after an invalid one. */ - identity?: SubagentIdentityProjection + identity?: SubagentIdentityProjection | undefined } // The cast bridges only the optional-label arm: Zod's optional output @@ -95,7 +115,7 @@ interface IdentityState { // from the public interface. The no-value state itself is the serializable // `null` arm — never `undefined` — so every registry read and push frame // survives JSON.stringify losslessly. -const identitySchema = z.discriminatedUnion('mode', [ +const identityValueSchema = z.discriminatedUnion('mode', [ z.object({ mode: z.literal('one-shot'), label: z.string().optional(), @@ -106,7 +126,13 @@ const identitySchema = z.discriminatedUnion('mode', [ label: z.string(), seq: z.number().int().nonnegative(), }).strict(), -]).nullable() as unknown as z.ZodType +]) as unknown as z.ZodType + +const identitySchema = identityValueSchema.nullable() + +const identityStateSchema: z.ZodType = z.object({ + identity: identityValueSchema.optional(), +}).strict() /** Interpret one `subagent/descriptor` event's identity; no value when the payload cannot be trusted. */ function descriptorIdentity(event: SessionEvent): SubagentIdentityProjection | undefined { @@ -139,18 +165,17 @@ function descriptorIdentity(event: SessionEvent): SubagentIdentityProjection | u * holding the earlier identity replaces it instead of keeping it stale; * `null` ⟺ no valid descriptor, with the causes deliberately undistinguished. */ -export const subagentIdentityProjectionDefinition: -ProjectionDefinition<'subagent', IdentityState> = { +export const subagentIdentityProjectionDefinition = { key: 'subagent', - schema: identitySchema, + stateSchema: identityStateSchema, init: () => ({}), apply: (state, event) => { if (event.type !== 'subagent/descriptor') return state const identity = descriptorIdentity(event) return identity === undefined ? {} : { identity } }, - view: state => state.identity ?? null, + wire: { viewSchema: identitySchema, view: state => state.identity ?? null }, // Bumped when the identity gained its `seq` field: an older checkpoint row // would replay into a value the schema rejects, so it must refold instead. stateVersion: 2, -} +} satisfies ProjectionDefinition<'subagent', IdentityState> diff --git a/packages/subagent/subagent/src/run-settlement.ts b/packages/subagent/subagent/src/run-settlement.ts index 8824cca405..d17fdaad35 100644 --- a/packages/subagent/subagent/src/run-settlement.ts +++ b/packages/subagent/subagent/src/run-settlement.ts @@ -18,6 +18,14 @@ function finalText(blocks: ContentBlock[]): string { .join('') } +/** Render a failed stop reason with optional provider-authored detail. */ +function failureDetail(result: SubagentResult): string { + const stopReason = result.stopReason + return result.diagnostic === undefined + ? stopReason + : `${stopReason}; diagnostic: ${result.diagnostic}` +} + /** * Map a child result to the task outcome: completed carries final text, * aborted is killed, and every other reason is failed without partial output. @@ -33,10 +41,10 @@ function runOutcome(result: SubagentResult): JobOutcome { case 'error': case 'max-tokens': case 'refusal': - return { status: 'failed', detail: result.stopReason } - // Merge-extensible reasons remain failures with their raw detail. + return { status: 'failed', detail: failureDetail(result) } + // Merge-extensible reasons remain failures with provider-authored detail. default: - return { status: 'failed', detail: String(result.stopReason) } + return { status: 'failed', detail: failureDetail(result) } } } diff --git a/packages/subagent/subagent/src/types.ts b/packages/subagent/subagent/src/types.ts index 2c5209765f..17978550ab 100644 --- a/packages/subagent/subagent/src/types.ts +++ b/packages/subagent/subagent/src/types.ts @@ -233,6 +233,13 @@ export interface SubagentResult { * schema-agnostic. */ readonly structured?: unknown + /** + * Provider-authored, non-assistant failure detail for a non-`completed` + * result. Providers keep this text free of tool inputs, file contents, + * environment values, credentials, and raw protocol payloads, and limit it + * to 4096 UTF-8 bytes. Consumers present it separately from {@link output}. + */ + readonly diagnostic?: string /** Why the run ended. A non-`completed` reason means `output` may be partial. */ readonly stopReason: SubagentStopReason } diff --git a/packages/subagent/subagent/tests/continuation.spec.ts b/packages/subagent/subagent/tests/continuation.spec.ts index 07f497628d..6cf1aea5a8 100644 --- a/packages/subagent/subagent/tests/continuation.spec.ts +++ b/packages/subagent/subagent/tests/continuation.spec.ts @@ -51,9 +51,17 @@ class GatedAdapter extends LlmAdapter { } } -const roots: string[] = [] -afterEach(() => { - for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 }) +// Each persistence-backed temp root cleans up by closing its handle before +// removing the directory: Windows rmSync over a dir holding a still-open handle +// fails with EPERM. +const cleanups: Array<() => Promise> = [] +afterEach(async () => { + const errors: unknown[] = [] + for (const cleanup of cleanups.splice(0)) { + try { await cleanup() } catch (error) { errors.push(error) } + } + if (errors.length === 1) throw errors[0] + if (errors.length > 1) throw new AggregateError(errors, 'temp-root cleanup failed') }) /** Boot the full continuable stack: loop, persistence, providers, and subagents. */ @@ -64,9 +72,13 @@ async function setupWith(adapter: LlmAdapter, options: { persistence?: boolean } let root: string | undefined if (options.persistence !== false) { root = mkdtempSync(join(tmpdir(), 'dsh-subagent-continuation-')) - roots.push(root) + const persistedRoot = root const persistenceFiber = await ctx.plugin(JsonlSessionPersistence, { root }) disposePersistence = () => persistenceFiber.dispose() + cleanups.push(async () => { + await persistenceFiber.dispose() + rmSync(persistedRoot, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 }) + }) } await ctx.plugin(AgentLoop, { agents: [] }) await ctx.plugin(SubagentRuntime) @@ -190,6 +202,38 @@ describe('SubagentRuntime.startContinuable', () => { expect(hasUserText(loaded.events, 'child task')).toBe(true) }) + it('uses a caller-reserved child identity and rejects a duplicate reservation', async () => { + const release = Promise.withResolvers() + const adapter = new GatedAdapter([ + { chunks: textResponse('reserved answer'), gate: release.promise }, + ]) + const { ctx, parent } = await setupWith(adapter) + const reservedId = SessionId('00000000-0000-4000-8000-000000000123') + + const started = await ctx.subagents.startContinuable({ + ...startSpec(parent), + childId: reservedId, + }) + expect(started.childId).toBe(reservedId) + await vi.waitFor(() => { expect(adapter.requests).toHaveLength(1) }) + + await expect(ctx.subagents.startContinuable({ + ...startSpec(parent), + childId: reservedId, + })).rejects.toMatchObject({ code: 'DUPLICATE_CHILD' }) + + release.resolve(undefined) + await waitNoActivation(ctx, reservedId) + const loaded = await ctx.sessionPersistence.load(reservedId) + expect(loaded.meta.id).toBe(reservedId) + + await expect(ctx.subagents.startContinuable({ + ...startSpec(parent), + childId: reservedId, + })).rejects.toMatchObject({ code: 'DUPLICATE_CHILD' }) + expect(ctx.agents.get(reservedId)).toBeUndefined() + }) + it('rejects without ids when the provider has no prepareContinuable capability', async () => { const { ctx, parent } = await setup([]) const start = vi.fn(async () => { throw new Error('must not dispatch') }) @@ -372,7 +416,10 @@ describe('SubagentRuntime.startContinuable', () => { const fresh = new Context() await mountAgentLoopTestDependencies(fresh) - await fresh.plugin(JsonlSessionPersistence, { root: root! }) + const freshPersistence = await fresh.plugin(JsonlSessionPersistence, { root: root! }) + // This context opened a second handle on the same root; register it so + // afterEach closes it before removing the root (even on a failure path). + cleanups.push(async () => { await freshPersistence.dispose() }) await fresh.plugin(AgentLoop, { agents: [] }) await fresh.plugin(SubagentRuntime) await fresh.plugin(SubagentSpawn, { providerName: 'spawn' }) @@ -745,10 +792,10 @@ describe('continuable durability and teardown', () => { hold.resolve(undefined) + await waitNoActivation(ctx, started.childId) await vi.waitFor(() => { expect(warnings.some(warning => warning.includes('normal settlement cleanup failed'))).toBe(true) - }) - expect(ctx.agents.get(started.childId)).toBeUndefined() + }, { timeout: 5_000 }) }) it('disposes every live Activation forest child-first on manager teardown', async () => { @@ -873,6 +920,86 @@ describe('continuable durability and teardown', () => { await waitNoActivation(ctx, started.childId) }) + it('releases only selected direct children', async () => { + const releaseTarget = Promise.withResolvers() + const releaseSibling = Promise.withResolvers() + const releaseGrandchild = Promise.withResolvers() + const adapter = new GatedAdapter([ + { chunks: textResponse('target'), gate: releaseTarget.promise }, + { chunks: textResponse('sibling'), gate: releaseSibling.promise }, + { chunks: textResponse('grandchild'), gate: releaseGrandchild.promise }, + ]) + const { ctx, parent } = await setupWith(adapter) + const target = await ctx.subagents.startContinuable(startSpec(parent)) + const sibling = await ctx.subagents.startContinuable(startSpec(parent)) + await vi.waitFor(() => { expect(adapter.requests).toHaveLength(2) }) + const targetAgent = ctx.agents.get(target.childId)! + const siblingAgent = ctx.agents.get(sibling.childId)! + const grandchild = await ctx.subagents.startContinuable(startSpec(targetAgent)) + await vi.waitFor(() => { expect(adapter.requests).toHaveLength(3) }) + const cancel = vi.spyOn(targetAgent, 'cancel') + + const drained = ctx.subagents.drainContinuableChildren(parent, [target.childId, target.childId]) + + expect(cancel).toHaveBeenCalledWith({ kind: 'parent' }) + expect(ctx.agents.get(sibling.childId)).toBe(siblingAgent) + releaseTarget.resolve(undefined) + releaseGrandchild.resolve(undefined) + await drained + expect(ctx.agents.get(target.childId)).toBeUndefined() + expect(ctx.agents.get(grandchild.childId)).toBeUndefined() + expect(ctx.agents.get(sibling.childId)).toBe(siblingAgent) + releaseSibling.resolve(undefined) + await waitNoActivation(ctx, sibling.childId) + }) + + it('reports selected-child disposal failures after releasing the child', async () => { + const hold = Promise.withResolvers() + const adapter = new GatedAdapter([{ chunks: textResponse('target'), gate: hold.promise }]) + const { ctx, parent } = await setupWith(adapter) + const target = await ctx.subagents.startContinuable(startSpec(parent)) + await vi.waitFor(() => { expect(adapter.requests).toHaveLength(1) }) + const manager = (ctx.subagents as unknown as { + continuations: { activations: Map Promise } }> } + }).continuations + const activation = manager.activations.get(target.childId)! + const realDispose = activation.handle.dispose.bind(activation.handle) + activation.handle.dispose = async () => { + await realDispose() + throw new Error('selected cleanup failed') + } + + const drained = ctx.subagents.drainContinuableChildren(parent, [target.childId]) + hold.resolve(undefined) + + await expect(drained).rejects.toMatchObject({ code: 'ACTIVATION_TEARDOWN_FAILED' }) + expect(ctx.agents.get(target.childId)).toBeUndefined() + }) + + it('rejects selected-child teardown through another live parent', async () => { + const release = Promise.withResolvers() + const adapter = new GatedAdapter([{ chunks: textResponse('target'), gate: release.promise }]) + const { ctx, parent } = await setupWith(adapter) + const other = ctx.agentLoop.create(SessionId('other-parent'), { provider: 'mock', model: 'mock' }) + const target = await ctx.subagents.startContinuable(startSpec(parent)) + await vi.waitFor(() => { expect(adapter.requests).toHaveLength(1) }) + + await expect(ctx.subagents.drainContinuableChildren(other, [target.childId])) + .rejects.toMatchObject({ code: 'UNAUTHORIZED' }) + expect(ctx.agents.get(target.childId)).toBeDefined() + + release.resolve(undefined) + await waitNoActivation(ctx, target.childId) + }) + + it('rejects selected-child teardown through a stale parent identity', async () => { + const { ctx, parent } = await setup([]) + const stale = { ...parent, id: parent.id } as unknown as Agent + + await expect(ctx.subagents.drainContinuableChildren(stale, [])) + .rejects.toMatchObject({ code: 'UNAUTHORIZED' }) + }) + it('finds scoped descendants after an intermediate one-shot Agent leaves the registry', async () => { const releaseIntermediate = Promise.withResolvers() const releaseDescendant = Promise.withResolvers() @@ -1524,6 +1651,42 @@ function settlementNotices(agent: Agent): { sender: string; text: string; summar }) } +describe('continuable report delivery', () => { + it('wakes an idle parent for a next-step report', async () => { + const releaseChild = Promise.withResolvers() + const adapter = new GatedAdapter([ + { chunks: textResponse('child answer'), gate: releaseChild.promise }, + { chunks: textResponse('parent report ack') }, + { chunks: textResponse('parent settlement ack') }, + ]) + const { ctx, parent } = await setupWith(adapter) + const started = await ctx.subagents.startContinuable(startSpec(parent)) + await vi.waitFor(() => { + expect(adapter.requests.filter(request => request.sessionId === started.childId)).toHaveLength(1) + }) + const child = ctx.agents.get(started.childId) + expect(child).toBeDefined() + + const messageId = await ctx.subagents.reportFrom(child!, message('an explicit report'), { + delivery: 'next-step', + signal: testSignal, + }) + + await vi.waitFor(() => { + expect(adapter.requests.filter(request => request.sessionId === parent.id)).toHaveLength(1) + }) + const report = parent.session.events.flatMap(event => event.type === 'user/message' + && event.data.source.kind === 'subagent-report' ? [event.data] : [])[0] + expect(report?.id).toBe(messageId) + + releaseChild.resolve(undefined) + await waitNoActivation(ctx, started.childId) + await vi.waitFor(() => { + expect(adapter.requests.filter(request => request.sessionId === parent.id)).toHaveLength(2) + }) + }) +}) + describe('continuable settlement delivery', () => { it('tells the parent what the child finished with, without being asked', async () => { const { ctx, parent } = await setup([textResponse('the answer'), textResponse('parent ack')]) @@ -2284,8 +2447,11 @@ describe('continuable errors', () => { const ctx = new Context() await mountAgentLoopTestDependencies(ctx) const root = mkdtempSync(join(tmpdir(), 'dsh-subagent-continuation-')) - roots.push(root) - await ctx.plugin(JsonlSessionPersistence, { root }) + const persistenceFiber = await ctx.plugin(JsonlSessionPersistence, { root }) + cleanups.push(async () => { + await persistenceFiber.dispose() + rmSync(root, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 }) + }) await ctx.plugin(AgentLoop, { agents: [] }) const serviceFiber = await ctx.plugin(SubagentRuntime) await ctx.plugin(SubagentSpawn, { providerName: 'spawn' }) diff --git a/packages/subagent/subagent/tests/list-children.spec.ts b/packages/subagent/subagent/tests/list-children.spec.ts index 3fd303e7f6..e9a5633189 100644 --- a/packages/subagent/subagent/tests/list-children.spec.ts +++ b/packages/subagent/subagent/tests/list-children.spec.ts @@ -118,6 +118,9 @@ function descriptorPayload(label: string, version = SUBAGENT_DESCRIPTOR_VERSION) } declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + subagentListHostileProbe: { poisoned?: boolean | undefined } + } interface SessionProjectionMap { /** Test-only hostile probe proving per-child isolation of foreign unit failures. */ subagentListHostileProbe: null @@ -130,20 +133,23 @@ declare module '@deepseek-ai/dsh-session-projection/types' { * through it), while the poisoned state detonates only when a listing read * folds or serves this child through the registry. */ -const hostileProjectionDefinition: ProjectionDefinition<'subagentListHostileProbe', { poisoned?: boolean }> = { +const hostileProjectionDefinition = { key: 'subagentListHostileProbe', - schema: z.null(), + stateSchema: z.object({ poisoned: z.boolean().optional() }), init: () => ({}), apply: (state, event) => event.type === 'subagent/descriptor' && (event.data as { label?: string }).label === 'poison me' ? { poisoned: true } : state, - view: (state) => { - if (state.poisoned === true) throw new Error('hostile unit rejects the poisoned log') - return null + wire: { + viewSchema: z.null(), + view: (state) => { + if (state.poisoned === true) throw new Error('hostile unit rejects the poisoned log') + return null + }, }, stateVersion: 1, -} +} satisfies ProjectionDefinition<'subagentListHostileProbe', { poisoned?: boolean | undefined }> describe('SubagentRuntime.listChildren', () => { it('lists live children without persistence, query services, or the continuation runtime', async () => { diff --git a/packages/subagent/subagent/tests/run-settlement.spec.ts b/packages/subagent/subagent/tests/run-settlement.spec.ts index 9e13d55507..90070c8b5f 100644 --- a/packages/subagent/subagent/tests/run-settlement.spec.ts +++ b/packages/subagent/subagent/tests/run-settlement.spec.ts @@ -1,6 +1,11 @@ import { describe, expect, it } from 'vitest' import { SessionId } from '@deepseek-ai/dsh-session' -import { settleRun } from '../src/index.ts' +import { + settleRun, + settleRunResult, +} from '../src/index.ts' + +const MAX_SUBAGENT_DIAGNOSTIC_BYTES = 4_096 describe('outcome mapping helpers', () => { it.each([ @@ -62,4 +67,51 @@ describe('outcome mapping helpers', () => { detail: 'Error: result failed; dispose failed: Error: reap failed', }) }) + + it('keeps provider diagnostics separate in failed background outcomes', async () => { + await expect(settleRun({ + id: SessionId('child-diagnostic'), + localAgent: undefined, + result: Promise.resolve({ + output: [{ type: 'text', text: 'partial assistant text' }], + diagnostic: 'Claude Code denied a tool request', + stopReason: 'error', + }), + dispose: () => Promise.resolve(), + })).resolves.toEqual({ + status: 'failed', + detail: 'error; diagnostic: Claude Code denied a tool request', + }) + }) + + it('bounds multibyte diagnostics and marks truncation', async () => { + const exact = 'x'.repeat(MAX_SUBAGENT_DIAGNOSTIC_BYTES) + const oversized = '权限'.repeat(MAX_SUBAGENT_DIAGNOSTIC_BYTES) + const controller = new AbortController() + const exactResult = await settleRunResult({ + attempt: async () => { throw new Error('provider failed') }, + collectOutput: () => [], + collectDiagnostic: () => exact, + cancelled: () => false, + signal: controller.signal, + onAbort: () => {}, + }) + expect(exactResult.diagnostic).toBe(exact) + + const result = await settleRunResult({ + attempt: async () => { throw new Error('provider failed') }, + collectOutput: () => [], + collectDiagnostic: () => oversized, + cancelled: () => false, + signal: controller.signal, + onAbort: () => {}, + }) + const limited = result.diagnostic ?? '' + expect(Buffer.byteLength(limited, 'utf8')) + .toBeLessThanOrEqual(MAX_SUBAGENT_DIAGNOSTIC_BYTES) + expect(limited.endsWith('[diagnostic truncated]')).toBe(true) + expect(limited).not.toContain('\uFFFD') + expect(result.stopReason).toBe('error') + expect(result.diagnostic).toBe(limited) + }) }) diff --git a/packages/subagent/subagent/tests/service.spec.ts b/packages/subagent/subagent/tests/service.spec.ts index e387a758e8..05e9785611 100644 --- a/packages/subagent/subagent/tests/service.spec.ts +++ b/packages/subagent/subagent/tests/service.spec.ts @@ -127,11 +127,12 @@ describe('SubagentRuntime', () => { expect('resume' in provider).toBe(false) }) - it('does not expose manager teardown and treats a scoped drain as a no-op when no manager was bound', async () => { + it('does not expose manager teardown and treats public drains as no-ops when no manager was bound', async () => { const { subagents } = await service() // Without `ctx.agents` no manager exists, so nothing was ever materialized. expect('drainContinuable' in subagents).toBe(false) await expect(subagents.drainContinuableDescendants([])).resolves.toBeUndefined() + await expect(subagents.drainContinuableChildren(fakeParent(), [SessionId('child')])).resolves.toBeUndefined() }) it('treats interrupt as an accepted no-op when no manager was bound', async () => { diff --git a/packages/subagent/subagent/tests/timing-projection.spec.ts b/packages/subagent/subagent/tests/timing-projection.spec.ts index ab3ffac9a2..bfc42eae3b 100644 --- a/packages/subagent/subagent/tests/timing-projection.spec.ts +++ b/packages/subagent/subagent/tests/timing-projection.spec.ts @@ -4,16 +4,16 @@ import SessionStore from '@deepseek-ai/dsh-session' import type { SessionEvent } from '@deepseek-ai/dsh-session' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import SubagentRuntime from '../src/index.ts' -import { subagentTimingProjectionDefinition } from '../src/projection.ts' +import { subagentTimingProjectionDefinition, type TimingState } from '../src/projection.ts' function event(type: SessionEvent['type'], seq: number, time: number): SessionEvent { return { type, seq, time, data: {} } as SessionEvent } function fold(events: SessionEvent[]) { - let state = subagentTimingProjectionDefinition.init() + let state: TimingState = subagentTimingProjectionDefinition.init() for (const item of events) state = subagentTimingProjectionDefinition.apply(state, item) - return subagentTimingProjectionDefinition.view(state) + return subagentTimingProjectionDefinition.wire.view(state) } describe('subagent timing projection', () => { diff --git a/packages/subagent/tool-subagent-control/README.i18n.yaml b/packages/subagent/tool-subagent-control/README.i18n.yaml index 16b43e0b14..07ec431848 100644 --- a/packages/subagent/tool-subagent-control/README.i18n.yaml +++ b/packages/subagent/tool-subagent-control/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/tool-subagent-control/README.md README.md: 91f8d23ac092049e5315418070cdaae025054860 -README.zh.md: 66d4b5a446fc9e412c5fec7bccf27bdefd1f5f77 +README.zh.md: ca63383e541ccb157f5ee8a6a7a7de7be84b9f5a diff --git a/packages/subagent/tool-subagent-control/README.zh.md b/packages/subagent/tool-subagent-control/README.zh.md index 66d4b5a446..ca63383e54 100644 --- a/packages/subagent/tool-subagent-control/README.zh.md +++ b/packages/subagent/tool-subagent-control/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -可选的全局具名 `send_message`、`interrupt_agent` 与 `list_agents` 工具是 `ctx.subagents` 之上的轻量适配器。绑定提供方的 `@deepseek-ai/dsh-tool-subagent` 实例会为每种传输注册不同的委派工具;这个单独加载的包只注册一次共享控制工具,因此多个委派工具绝不会重复注册全局控制工具。根插件注册 `send_message` 与 `interrupt_agent`,且只要求 `subagents`;可单独加载的 `./list-agents` 插件注册 `list_agents`,并将 `subagents` 与 `agents` 声明为加载时依赖。其目录读取在调用时还要求会话存储与投影注册表,但不要求任何查询服务。部署可保留根插件工具并省略列表工具。是否加载这些工具不会决定委派工具是否启动可继续工作。这些工具只负责父到子的方向;单独安装的 [`@deepseek-ai/dsh-tool-subagent-report`](../tool-subagent-report/README.md) 负责子到父的方向。 +可选的全局具名 `send_message`、`interrupt_agent` 与 `list_agents` 工具是 `ctx.subagents` 之上的轻量适配器。绑定提供方的 `@deepseek-ai/dsh-tool-subagent` 实例会为每种传输注册不同的委派工具;这个单独加载的包只注册一次共享控制工具,因此多个委派工具绝不会重复注册全局控制工具。根插件注册 `send_message` 与 `interrupt_agent`,且只要求 `subagents`;可单独加载的 `./list-agents` 插件注册 `list_agents`,并将 `subagents` 与 `agents` 声明为加载时依赖。其目录读取在调用时还要求会话存储与投影注册表,但不要求任何查询服务。部署可保留根插件工具并省略列表工具。是否加载这些工具不会决定委派工具是否启动可继续工作。这些工具只负责父到子的方向;单独安装的 [`@deepseek-ai/dsh-tool-subagent-report`](../tool-subagent-report/README.zh.md) 负责子到父的方向。 本工具不执行生命周期路由:驻留与冷恢复归 subagent 服务所有。它将 `exec.agent` 作为授权投递的确切在线父级传入,并把每条消息的来源记录为 `{ kind: 'coordinator', senderSessionId: parent.id }`;服务会保留该来源,但绝不将其视为权限。每条消息都会通过 `Agent.followup()` 成为 subagent 的下一个 FIFO 轮次:如果子 agent(智能体)仍在工作,该消息会等待其当前轮次结束,因此无法重定向已经在进行的工作。本工具会转发其执行信号,该信号只在 inbox 接受之前掌管准入;一旦子 agent 接受消息,已接受的轮次便无法再通过本工具取消。本次调用不会返回子 agent 的回复;通过该 id 查看其 transcript(文本记录),才是了解它完成了哪些工作的真源。拥有 `report` 的子 agent 会自行把内容作为一条单独的父级消息发回。投递失败会变为出错的工具结果,并明确说明消息未送达。 @@ -16,7 +16,7 @@ #### 模型看到的内容 -已生成的 [schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-subagent-control):`send_message` 包含 `subagent_id` 和 `message`,说明消息会成为 subagent 的下一个轮次、本次调用不会返回 subagent 的回答,以及失败即表示消息未送达;`interrupt_agent` 包含 `agent_id`,说明只有当前轮次会停止、已排队消息保持暂停、后代继续运行,以及接受先于实际停止;`list_agents` 包含可选的 `scope` 枚举。 +已生成的 [schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-subagent-control):`send_message` 包含 `subagent_id` 和 `message`,说明消息会成为 subagent 的下一个轮次、本次调用不会返回 subagent 的回答,以及失败即表示消息未送达;`interrupt_agent` 包含 `agent_id`,说明只有当前轮次会停止、已排队消息保持暂停、后代继续运行,以及接受先于实际停止;`list_agents` 包含可选的 `scope` 枚举。 #### Token 影响 diff --git a/packages/subagent/tool-subagent-control/package.json b/packages/subagent/tool-subagent-control/package.json index 6be2ae1d26..9d22d4a93b 100644 --- a/packages/subagent/tool-subagent-control/package.json +++ b/packages/subagent/tool-subagent-control/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-subagent-control", "description": "Globally named send_message, interrupt_agent, and list_agents tools over ctx.subagents continuations", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subagent/tool-subagent-report/README.i18n.yaml b/packages/subagent/tool-subagent-report/README.i18n.yaml index f96d1d5031..e3b06edfa9 100644 --- a/packages/subagent/tool-subagent-report/README.i18n.yaml +++ b/packages/subagent/tool-subagent-report/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/tool-subagent-report/README.md -README.md: b38b6b541d9c03174de1515df7a0babe5da055b4 -README.zh.md: d8557a43bca4b9a403aac31b24812425e0586fcf +README.md: ab2bcc5e4b829b66810da4089edb7146ebc1c791 +README.zh.md: 5bac66a637eeb36dcd2133d387fd304f0912be4a diff --git a/packages/subagent/tool-subagent-report/README.md b/packages/subagent/tool-subagent-report/README.md index b38b6b541d..ab2bcc5e4b 100644 --- a/packages/subagent/tool-subagent-report/README.md +++ b/packages/subagent/tool-subagent-report/README.md @@ -6,7 +6,7 @@ The optional child-scoped `report` tool is a thin adapter over `ctx.subagents.re The child-scoped `tool:report` prompt section instructs the child to call `report` once before finishing, with a self-contained answer, and earlier whenever a partial finding changes what the parent should do next. The instruction is guidance, not enforcement: the mechanism still accepts zero or many calls in one turn, and no runtime path rejects a child that never reports. A successful call neither concludes the turn, settles the Activation, nor prevents later parent follow-ups, and finishing a turn never reports automatically. The tool accepts no recipient: `exec.agent` is the sender's exact live Agent and the authority credential, and the service derives the sole recipient from that child's durable `parentSession`. Success returns the stable `MessageId` of the parent-accepted message, not a read receipt, an inbox-occurrence id, a parent-log acknowledgement, a turn-completion receipt, or a persistence flush. A parent absent from the registry fails the call with `direct parent is not live; report was not delivered` — registry presence governs parent resolution, and a registered parent already in host-owned disposal still accepts while its log admits appends. The service performs no injection, parent cold resume, or offline mailbox write; the durable child transcript remains the recovery source, and a failed tool call does not prove non-delivery (a later `tools/post-execute` veto can fail a call whose report was already accepted). -`reportDelivery` selects parent scheduling for every accepted report. `wakeup` (the default) uses `parent.followup()`, creating exactly one ordinary later parent turn and waking a parked parent driver; it never steers an open turn. It is the default because a parent that already parked has no other reason to look, so quiet delivery would leave an accepted report unread until something unrelated woke it. `quiet` uses `parent.inject()`, adding model-facing context without starting a parent model request: an idle parent's append completes before the call returns, while a report reaching an admitting or running parent stages for the next safe log position. This is deployment scheduling policy, so the model-facing schema cannot select or override it per call. +`reportDelivery` selects parent scheduling for every accepted report. `next-step` (the default) uses `parent.steer()`: a running parent receives the report at its nearest safe step boundary, while an idle parent starts a turn. Reports accepted in sequence share the next-step FIFO, including the later manager-authored settlement notice, so the parent cannot observe settlement before an earlier report; reports waiting together enter one claimed batch. `quiet` uses `parent.inject()`, adding the same next-step context without waking a parked parent. This is deployment scheduling policy, so the model-facing schema cannot select or override it per call. Scope-local registration deliberately survives the child's global `toolFilter`, so a delegation allow-list cannot remove the only return channel. A deployment that requires a child with no return channel omits this package. @@ -36,7 +36,7 @@ Prefix-stable within a child; neither the schema nor the section changes at runt #### Token effect -One short acknowledgement per call in the reporting child. The reported content is additionally billed to the parent: waking delivery makes it the sole ordinary message of one new parent turn, while quiet delivery adds it to the parent's next request. +One short acknowledgement per call in the reporting child. The reported content is additionally billed to the parent: next-step delivery joins the next request in an open parent turn or starts a turn for an idle parent, while quiet delivery waits for another input to wake the parent. #### KV Cache effect @@ -54,7 +54,7 @@ The child's complete `output` plus the one-line frame, uncapped by this package. #### KV Cache effect -Append-only; the report follows the parent's reusable request prefix. Waking delivery starts an independent parent model request, while quiet delivery does not. +Append-only; the report follows the parent's reusable request prefix. Next-step delivery wakes the parent and may extend its open turn, while quiet delivery does not wake it. ## Known Limitations and Deferred Work @@ -63,4 +63,4 @@ Append-only; the report follows the parent's reusable request prefix. Waking del - **A staged quiet report is not immediately reconstructable** — acceptance returns its stable `MessageId`, but the parent Session reconstructs the framed content only after pending context reaches its ordinary log boundary. - **Granting waits for the next Activation; revocation is immediate** — installing this package after a child becomes resident grants `report` and its guidance only on that child's next Activation, while removing the package revokes both from resident children immediately. - **Nested reporting reaches exactly one edge upward** — a grandchild reports to its direct child parent, never to the top-level coordinator, which must explicitly report a derived update later. -- **No rate limiting** — the default `wakeup` mode can amplify model work when nested children report frequently; a deployment that accepts unread reports over that amplification selects `quiet`. +- **No rate limiting** — the default `next-step` mode can amplify model work when nested children report frequently, although reports waiting together share one step; a deployment that accepts unread reports over that amplification selects `quiet`. diff --git a/packages/subagent/tool-subagent-report/README.zh.md b/packages/subagent/tool-subagent-report/README.zh.md index d8557a43bc..5bac66a637 100644 --- a/packages/subagent/tool-subagent-report/README.zh.md +++ b/packages/subagent/tool-subagent-report/README.zh.md @@ -2,11 +2,11 @@ [English](README.md) | 中文 -可选的子级作用域 `report` 工具是 `ctx.subagents.reportFrom()` 之上的轻量适配器。它为每个可继续的进程内子级提供一条返回通道,指向启动该子级的 Agent(智能体),并安装指示子级使用该通道的提示词 section。本包注册的是可继续子级设置贡献,而不是全局工具,因此该工具及其指引只存在于这些子级内部。根 Agent、一次性 subagent、远程 subagent 提供方、同级作用域以及不关联 Agent 的工具执行都不会提供或执行它。安装本包只授予这项子级作用域能力;父到子方向仍由独立的 [`@deepseek-ai/dsh-tool-subagent-control`](../tool-subagent-control/README.md) 负责,可继续模式不依赖这两个包中的任一个。 +可选的子级作用域 `report` 工具是 `ctx.subagents.reportFrom()` 之上的轻量适配器。它为每个可继续的进程内子级提供一条返回通道,指向启动该子级的 Agent(智能体),并安装指示子级使用该通道的提示词 section。本包注册的是可继续子级设置贡献,而不是全局工具,因此该工具及其指引只存在于这些子级内部。根 Agent、一次性 subagent、远程 subagent 提供方、同级作用域以及不关联 Agent 的工具执行都不会提供或执行它。安装本包只授予这项子级作用域能力;父到子方向仍由独立的 [`@deepseek-ai/dsh-tool-subagent-control`](../tool-subagent-control/README.zh.md) 负责,可继续模式不依赖这两个包中的任一个。 子级作用域的 `tool:report` 提示词 section 要求子级在结束前调用一次 `report` 并给出自足的答案,并在部分发现会改变父级下一步动作时提前上报。该指令是引导而非强制:机制本身仍接受一个轮次中调用零次或多次,也没有任何运行时路径会拒绝从不上报的子级。调用成功既不会结束轮次或结算 Activation,也不会阻止父级后续消息;轮次结束也绝不会自动上报。该工具不接受接收方参数:`exec.agent` 是发送方确切在线的 Agent,也是权限凭据;服务根据该子级持久化的 `parentSession` 推导唯一接收方。成功时返回父级已接受消息的稳定 `MessageId`,不表示已读回执、inbox 中该次出现的 id、父级日志确认、轮次完成回执或持久化刷盘。父级解析由注册表中的存在性决定:父级不在注册表时,调用失败并返回 `direct parent is not live; report was not delivered`;已开始由宿主管理的 dispose(资源释放)但仍在注册表中的父级在其日志仍接受追加时仍会接受。服务不会执行注入、父级冷恢复或离线 mailbox 写入;持久化子级 transcript(文本记录)仍是恢复依据,且工具调用失败不能证明未送达(后续 `tools/post-execute` 否决可能让报告已被接受的调用以失败结束)。 -`reportDelivery` 为每条已接受的报告选择父级调度方式。`wakeup`(默认值)使用 `parent.followup()`,恰好创建一个普通的后续父级轮次,并唤醒停驻的父级驱动;它绝不会对正在运行的轮次进行 steering(中途引导)。之所以作为默认值:已经停驻的父级没有别的理由再去查看,静默投递会让一条已被接受的报告一直无人阅读,直到别的事件把父级唤醒。`quiet` 使用 `parent.inject()`,在不启动父级模型请求的情况下添加面向模型的上下文:父级空闲时,追加操作会在调用返回前完成;报告到达正在准入或运行的父级时,则会暂存到下一个安全日志位置。这是部署调度策略,因此面向模型的 schema 不能在单次调用中选择或覆盖该策略。 +`reportDelivery` 为每条已接受的报告选择父级调度方式。`next-step`(默认值)使用 `parent.steer()`:运行中的父级会在最近的安全 step 边界收到报告,空闲父级则会启动一个轮次。按顺序接受的报告会共享 next-step FIFO,稍后由管理器撰写的结算通知也进入该队列,因此父级不会在更早的报告之前看到结算;一起等待的报告会进入同一个领取批次。`quiet` 使用 `parent.inject()`,添加相同的 next-step 上下文,但不会唤醒停驻的父级。这是部署调度策略,因此面向模型的 schema 不能在单次调用中选择或覆盖该策略。 作用域局部注册有意不受子级全局 `toolFilter` 影响,因此委派允许列表无法移除唯一的返回通道。需要子级不具备返回通道的部署应省略本包。 @@ -18,7 +18,7 @@ #### 模型看到的内容 -已生成的 [`report` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-subagent-report):包含一个必填 `output` 字符串。其描述说明子级必须在结束前上报一次,上报只会到达启动该子级的 Agent,并且不会结束轮次。它不包含接收方或投递模式参数。独立的 `tool:report` 提示词 section 在 schema 之外重申该义务,使忽略工具描述的子级仍能读到。 +已生成的 [`report` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-subagent-report):包含一个必填 `output` 字符串。其描述说明子级必须在结束前上报一次,上报只会到达启动该子级的 Agent,并且不会结束轮次。它不包含接收方或投递模式参数。独立的 `tool:report` 提示词 section 在 schema 之外重申该义务,使忽略工具描述的子级仍能读到。 #### Token 影响 @@ -36,7 +36,7 @@ #### Token 影响 -每次调用都会在执行上报的子级中产生一条简短确认消息。父级还会为上报内容支付 token 成本:唤醒投递会使该内容成为一个新父级轮次中唯一的普通消息,静默投递则把内容加入父级的下一次请求。 +每次调用都会在执行上报的子级中产生一条简短确认消息。父级还会为上报内容支付 token 成本:next-step 投递会加入父级已打开轮次的下一次请求,或为空闲父级启动一个轮次;静默投递则等待其他输入唤醒父级。 #### KV Cache 影响 @@ -54,7 +54,7 @@ #### KV Cache 影响 -仅追加;报告位于父级可复用请求前缀之后。唤醒投递会启动一次独立的父级模型请求,静默投递则不会。 +仅追加;报告位于父级可复用请求前缀之后。next-step 投递会唤醒父级,并可能延长其已打开的轮次;静默投递则不会唤醒父级。 ## 已知限制与暂缓事项 @@ -63,4 +63,4 @@ - **暂存的静默报告无法立即重建**:接受时会返回其稳定 `MessageId`,但只有当待处理上下文到达普通日志边界后,父级会话才能重建带前缀的内容。 - **授权须等到下一个 Activation,撤销则立即生效**:子级驻留后再安装本包,只会在该子级的下一个 Activation 中授予 `report` 及其指引;移除本包则会立即从驻留子级撤销两者。 - **嵌套上报只向上到达一条直接边**:孙级只向作为其直接父级的子级上报,不会直接到达顶层协调器;该直接父级必须随后显式发出一条衍生更新。 -- **没有速率限制**:嵌套子级频繁上报时,默认的 `wakeup` 模式会放大模型工作量;宁可接受报告无人阅读也要避免这种放大的部署应选择 `quiet`。 +- **没有速率限制**:嵌套子级频繁上报时,默认的 `next-step` 模式会放大模型工作量,但一起等待的报告会共享一个 step;宁可接受报告无人阅读也要避免这种放大的部署应选择 `quiet`。 diff --git a/packages/subagent/tool-subagent-report/package.json b/packages/subagent/tool-subagent-report/package.json index b3c8083f8c..a0ff7ecede 100644 --- a/packages/subagent/tool-subagent-report/package.json +++ b/packages/subagent/tool-subagent-report/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-subagent-report", "description": "Child-scoped report tool over ctx.subagents continuations", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subagent/tool-subagent-report/src/index.ts b/packages/subagent/tool-subagent-report/src/index.ts index 6f2a6b3c6a..85b139e641 100644 --- a/packages/subagent/tool-subagent-report/src/index.ts +++ b/packages/subagent/tool-subagent-report/src/index.ts @@ -26,15 +26,15 @@ const REPORT_SECTION_ORDER = 117 /** Config: how accepted reports are scheduled on the parent. */ export interface Config { /** - * Parent scheduling (default `wakeup`). `wakeup` creates one ordinary later - * parent turn; `quiet` adds context without waking, so a parked parent learns - * of the report only when something else wakes it. + * Parent scheduling (default `next-step`). `next-step` wakes the parent and + * enters at its nearest step boundary; `quiet` adds the same context without + * waking, so a parked parent waits for another waking input. */ reportDelivery?: SubagentReportDelivery } export const Config: z = z.object({ - reportDelivery: z.union(['quiet', 'wakeup'] as const).default('wakeup'), + reportDelivery: z.union(['quiet', 'next-step'] as const).default('next-step'), }) /** diff --git a/packages/subagent/tool-subagent-report/tests/tool-subagent-report.spec.ts b/packages/subagent/tool-subagent-report/tests/tool-subagent-report.spec.ts index b71e8365db..28d6068757 100644 --- a/packages/subagent/tool-subagent-report/tests/tool-subagent-report.spec.ts +++ b/packages/subagent/tool-subagent-report/tests/tool-subagent-report.spec.ts @@ -7,7 +7,7 @@ import type { Agent } from '@deepseek-ai/dsh-agent' import { assembleContextFor } from '@deepseek-ai/dsh-agent' import AgentLoop from '@deepseek-ai/dsh-agent-loop' import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-testkit' -import { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm' +import { CallId, LlmAdapter, createUserMessage } from '@deepseek-ai/dsh-llm' import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' import { SessionId } from '@deepseek-ai/dsh-session' import type { SessionEvent } from '@deepseek-ai/dsh-session' @@ -20,22 +20,39 @@ import * as tool from '../src/index.ts' const testSignal = new AbortController().signal -/** Adapter that keeps child Activations resident until released. */ +/** Adapter that keeps selected Agent requests open until released. */ class HeldAdapter extends LlmAdapter { readonly requests: GenerateOptions[] = [] - private readonly gate = Promise.withResolvers() + private readonly gates = new Map>() + private readonly releasedSessions = new Set() + private released = false async * stream(options: GenerateOptions): AsyncIterable { this.requests.push(options) - await this.gate.promise + if (!this.released && !this.releasedSessions.has(options.sessionId)) { + let gate = this.gates.get(options.sessionId) + if (gate === undefined) { + gate = Promise.withResolvers() + this.gates.set(options.sessionId, gate) + } + await gate.promise + } for (const chunk of textResponse('held answer')) { if (options.signal?.aborted) throw new Error('aborted') yield chunk } } - release(): void { - this.gate.resolve(undefined) + release(sessionId?: SessionId): void { + if (sessionId !== undefined) { + this.releasedSessions.add(sessionId) + this.gates.get(sessionId)?.resolve(undefined) + this.gates.delete(sessionId) + return + } + this.released = true + for (const gate of this.gates.values()) gate.resolve(undefined) + this.gates.clear() } } @@ -86,6 +103,17 @@ async function startChild(ctx: Context, parent: Agent, prompt = 'child task') { return { started, child } } +/** Start one parent request that remains open in the held adapter. */ +async function startHeldParentTurn(parent: Agent, adapter: HeldAdapter): Promise { + parent.followup(createUserMessage({ + content: [{ type: 'text', text: 'parent work' }], + source: { kind: 'user' }, + })) + await vi.waitFor(() => { + expect(adapter.requests.some(request => request.sessionId === parent.id)).toBe(true) + }) +} + let calls = 0 function callReport(ctx: Context, child: Agent, output: string, signal = testSignal) { return ctx.tools.execute({ @@ -204,8 +232,8 @@ describe('dsh-tool-subagent-report', () => { expect(adapter.requests.filter(request => request.sessionId === parent.id)).toHaveLength(parentRequests) }) - it('queues wakeup reports as one later parent turn', async () => { - const { ctx, parent, adapter } = await setup({ config: { reportDelivery: 'wakeup' } }) + it('delivers next-step reports through waking steering', async () => { + const { ctx, parent, adapter } = await setup({ config: { reportDelivery: 'next-step' } }) const { child } = await startChild(ctx, parent) const enqueues: string[] = [] ctx.on('agent/inbox/inserted', ({ agent, message }) => { @@ -216,19 +244,37 @@ describe('dsh-tool-subagent-report', () => { const result = await callReport(ctx, child, 'WAKE_UP') expect(result.isError).toBe(false) - expect(enqueues).toEqual(['queued']) + expect(enqueues).toEqual(['steering']) await vi.waitFor(() => { expect(adapter.requests.some(request => request.sessionId === parent.id)).toBe(true) }) }) - it('preserves accepted order across repeated reports', async () => { - const { ctx, parent } = await setup() + it('batches repeated next-step reports in accepted order', async () => { + const { ctx, parent, adapter } = await setup({ config: { reportDelivery: 'next-step' } }) + await startHeldParentTurn(parent, adapter) const { child } = await startChild(ctx, parent) expect((await callReport(ctx, child, 'FIRST')).isError).toBe(false) expect((await callReport(ctx, child, 'SECOND')).isError).toBe(false) expect(reports(parent).map(report => report.text.split('\n').at(-1))).toEqual(['FIRST', 'SECOND']) + expect(parent.inbox.nextStep).toHaveLength(2) + }) + + it('keeps a report before the child settlement in one busy-parent batch', async () => { + const { ctx, parent, adapter } = await setup({ config: { reportDelivery: 'next-step' } }) + await startHeldParentTurn(parent, adapter) + const { started, child } = await startChild(ctx, parent) + + expect((await callReport(ctx, child, 'ORDERED_REPORT')).isError).toBe(false) + adapter.release(started.childId) + await vi.waitFor(() => { expect(ctx.agents.get(started.childId)).toBeUndefined() }) + + expect(parent.inbox.nextStep.map(message => message.source.kind)).toEqual([ + 'subagent-report', + 'subagent-settled', + ]) + expect(parent.inbox.nextTurn).toHaveLength(0) }) it('keeps an accepted report after the child settles', async () => { @@ -261,8 +307,8 @@ describe('dsh-tool-subagent-report', () => { expect(reports(child)[0]?.text).toContain('FROM_GRANDCHILD') }) - it('accounts wakeup reports delivered to a resident continuable parent', async () => { - const { ctx, parent, adapter } = await setup({ config: { reportDelivery: 'wakeup' } }) + it('accounts next-step reports delivered to a resident continuable parent', async () => { + const { ctx, parent, adapter } = await setup({ config: { reportDelivery: 'next-step' } }) const { child } = await startChild(ctx, parent, 'outer task') const { started: grandchildStart, child: grandchild } = await startChild(ctx, child, 'inner task') @@ -509,9 +555,10 @@ describe('dsh-tool-subagent-report', () => { expect('default' in tool).toBe(false) expect(tool.name).toBe('tool-subagent-report') expect(tool.inject).toEqual(['subagents', 'tools', 'systemPrompt']) - // Waking is the default because a report that never wakes its parent - // cannot deliver a result to an agent that already parked. - expect(tool.Config({}).reportDelivery).toBe('wakeup') + // Next-step delivery wakes a parked parent and lets a running parent act at + // its nearest safe boundary. + expect(tool.Config({}).reportDelivery).toBe('next-step') + expect(() => tool.Config({ reportDelivery: 'wakeup' } as never)).toThrow() expect(() => tool.Config({ reportDelivery: 'shout' } as never)).toThrow() }) @@ -526,7 +573,7 @@ describe('dsh-tool-subagent-report', () => { }) expect((await callReport(ctx, child, 'DEFAULT_WAKES')).isError).toBe(false) - expect(enqueues).toEqual(['queued']) + expect(enqueues).toEqual(['steering']) await vi.waitFor(() => { expect(adapter.requests.some(request => request.sessionId === parent.id)).toBe(true) }) diff --git a/packages/subagent/tool-subagent/README.i18n.yaml b/packages/subagent/tool-subagent/README.i18n.yaml index 41f712ba34..29d65cf9da 100644 --- a/packages/subagent/tool-subagent/README.i18n.yaml +++ b/packages/subagent/tool-subagent/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/tool-subagent/README.md -README.md: 9d7ed2e364f6a9dff26a1c9006535f898bdaabcc -README.zh.md: 8650ee35588c2615e4d6c016cb672030ee2e8194 +README.md: 28e6213b903ffffa7934e244b2a74ada519b32b2 +README.zh.md: 74e8896a152c787abd0aebf055d6e13f6158bcd3 diff --git a/packages/subagent/tool-subagent/README.md b/packages/subagent/tool-subagent/README.md index 9d7ed2e364..28e6213b90 100644 --- a/packages/subagent/tool-subagent/README.md +++ b/packages/subagent/tool-subagent/README.md @@ -8,9 +8,9 @@ The model-facing delegation tool over one configured `ctx.subagents` provider. C Each plugin instance binds one `provider` to one `toolName`; the model receives no provider selector. Load another distinctly named instance to expose another transport. The tool registers only while its provider exists, avoiding sibling load-order and provider-reload dependencies. Its description follows `provider.inheritsParentContext`: fresh children require standalone prompts, while forked children already see completed parent turns. -A foreground call passes the execution signal through startup and execution, awaits `run.result`, and always awaits `run.dispose()` before returning. Only `completed` returns the canonical `{ kind: 'foreground', runId, output: JsonValue[] }`, rendered as the same final text; abort, refusal, token limit, and other failures become errored tool results whose message appends the child's preserved partial text (the `SubagentResult.output` selection) after the stop-reason headline, so a truncated answer is never reported as success yet never silently lost. If result collection and disposal both reject, the errored result preserves both diagnostics. +A foreground call passes the execution signal through startup and execution, awaits `run.result`, and always awaits `run.dispose()` before returning. Only `completed` returns the canonical `{ kind: 'foreground', runId, output: JsonValue[] }`, rendered as the same final text. Abort, refusal, token limit, and other failures become errored tool results whose message contains the stop-reason headline, an optional provider-authored `SubagentResult.diagnostic`, and then any preserved partial assistant text. The diagnostic remains separate from `SubagentResult.output`, so a truncated answer is never reported as success or confused with infrastructure detail. If result collection and disposal both reject, the errored result preserves both failures. -`backgroundMode` selects both the background route and the omitted `run_in_background` default. `one-shot` waits in the foreground by default; an explicit `true` registers a plain parent-owned Task and returns canonical `{ kind: 'background', jobId }`, rendered as `started background subagent job `, even when the provider supports continuable children. Generic task tools own its later status, collection, cancellation, and notices. `continuable` runs in the background when the argument is omitted or `true`; an explicit `false` waits for the result in the foreground. Its background route requires a provider with the `prepareContinuable` capability, calls `ctx.subagents.startContinuable()`, and returns `{ kind: 'continuable', subagentId }`, rendered as `started subagent `. The route resolves at inbox acceptance: the child owns its own turns from there, so this call neither waits for nor collects a result. The child's transcript by that id remains the source of its detailed output, and the optional global `send_message` tool sends it more work. The continuation service delivers one settlement notice whenever the child's Activation ends, containing its outcome and any final assistant message independently of `report`. Starting continuable work does not require `send_message` to be loaded. See the [background subagent Agent Note](../../../.agents/notes/implemented/feature/2026-07-08-background-subagent-tasks.md), the [continuable subagents Agent Note](../../../.agents/notes/implemented/feature/2026-07-28-continuable-subagent-conversations.md), and the [background-first delegation Agent Note](../../../.agents/notes/implemented/feature/2026-08-11-background-first-continuable-delegation.md). +`backgroundMode` selects both the background route and the omitted `run_in_background` default. `one-shot` waits in the foreground by default; an explicit `true` registers a plain parent-owned Task and returns canonical `{ kind: 'background', jobId }`, rendered as `started background subagent job `, even when the provider supports continuable children. Generic task tools own its later status, collection, cancellation, and notices; a failed Task keeps the stop reason and the same optional provider diagnostic in its detail. `continuable` runs in the background when the argument is omitted or `true`; an explicit `false` waits for the result in the foreground. Its background route requires a provider with the `prepareContinuable` capability, calls `ctx.subagents.startContinuable()`, and returns `{ kind: 'continuable', subagentId }`, rendered as `started subagent `. The route resolves at inbox acceptance: the child owns its own turns from there, so this call neither waits for nor collects a result. The child's transcript by that id remains the source of its detailed output, and the optional global `send_message` tool sends it more work. The continuation service delivers one settlement notice whenever the child's Activation ends, containing its outcome and any final assistant message independently of `report`. Starting continuable work does not require `send_message` to be loaded. See the [background subagent Agent Note](../../../.agents/notes/implemented/feature/2026-07-08-background-subagent-tasks.md), the [continuable subagents Agent Note](../../../.agents/notes/implemented/feature/2026-07-28-continuable-subagent-conversations.md), and the [background-first delegation Agent Note](../../../.agents/notes/implemented/feature/2026-08-11-background-first-continuable-delegation.md). `toolFilter` changes the child's global tool layer but is not a parent-derived authority ceiling. See the [agent-scope security non-goal](../../../.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.md#security-and-authority-are-non-goals). @@ -51,7 +51,7 @@ Prefix-stable while provider instances, names, descriptions, and schemas are unc #### What the model sees -The call retains the description and prompt. Success contains only the child's final text; other outcomes become `Error: `. Intermediate child steps stay out of the parent. +The call retains the description and prompt. Success contains only the child's final text; other outcomes become `Error: `, followed by a safe provider diagnostic when present and then any partial assistant text. Intermediate child steps stay out of the parent. #### Token effect @@ -65,7 +65,7 @@ Append-only; newly visible content follows the reusable request prefix and does #### What the model sees -Start returns exactly `started subagent ` in configured continuable mode, or `started background subagent job ` in configured one-shot mode. In one-shot mode the generic task surface provides later status, final output, cancellation responses, and notices. In continuable mode this tool returns no result of its own; the child's settlement reaches the parent as a [service-owned notice](../subagent/README.md#settlement-notice), an independently loaded `send_message` tool delivers follow-ups, and the child's transcript by its id is the source of its detailed output. +Start returns exactly `started subagent ` in configured continuable mode, or `started background subagent job ` in configured one-shot mode. In one-shot mode the generic task surface provides later status, final output, cancellation responses, and notices; failed status detail includes the provider diagnostic when the result supplied one. In continuable mode this tool returns no result of its own; the child's settlement reaches the parent as a [service-owned notice](../subagent/README.md#settlement-notice), an independently loaded `send_message` tool delivers follow-ups, and the child's transcript by its id is the source of its detailed output. #### Token effect diff --git a/packages/subagent/tool-subagent/README.zh.md b/packages/subagent/tool-subagent/README.zh.md index 8650ee3558..74e8896a15 100644 --- a/packages/subagent/tool-subagent/README.zh.md +++ b/packages/subagent/tool-subagent/README.zh.md @@ -8,11 +8,11 @@ 每个插件实例把一个 `provider` 绑定到一个 `toolName`;模型不会收到提供方选择器。如需公开另一种传输,请加载另一个名称不同的实例。工具只在其提供方存在时注册,从而避免对同级加载顺序和提供方重新加载的依赖。工具描述遵循 `provider.inheritsParentContext`:新建子 agent(智能体)需要独立提示词,而 fork 子 agent 已能看到父级已完成轮次。 -前台调用会让执行信号贯穿启动和执行,等待 `run.result`,并且在返回前总会等待 `run.dispose()`。只有 `completed` 会返回规范值 `{ kind: 'foreground', runId, output: JsonValue[] }`,并渲染为相同的最终文本;中止、拒绝、token 上限和其他失败都会变成出错的工具结果,其消息在终止原因标题之后附带子 agent 保留下来的部分文本(即 `SubagentResult.output` 的选取结果)——被截断的回答不会被报告为成功,也绝不会被悄悄丢弃。如果结果收集与 dispose(资源释放)都 reject,出错的结果会保留两项诊断信息。 +前台调用会让执行信号贯穿启动和执行,等待 `run.result`,并且在返回前总会等待 `run.dispose()`。只有 `completed` 会返回规范值 `{ kind: 'foreground', runId, output: JsonValue[] }`,并渲染为相同的最终文本。中止、拒绝、token 上限和其他失败都会变成出错的工具结果,其消息依次包含终止原因标题、可选的提供方 `SubagentResult.diagnostic`,以及子 agent 保留下来的部分 assistant 文本。诊断与 `SubagentResult.output` 保持分离,因此被截断的回答不会被报告为成功,也不会与基础设施说明混淆。如果结果收集与 dispose(资源释放)都 reject,出错结果会保留两项失败。 -`backgroundMode` 同时选择后台路由与省略 `run_in_background` 时的默认行为。`one-shot` 默认在前台等待;显式传入 `true` 时,它会注册一个归父级所有的普通 Task,并返回规范值 `{ kind: 'background', jobId }`,渲染为 `started background subagent job `,即使提供方支持可继续子 agent 也不例外。通用 Task 工具负责其后续状态、收集、取消和通知。`continuable` 在参数省略或为 `true` 时于后台运行;显式传入 `false` 时则在前台等待结果。其后台路由要求提供方具备 `prepareContinuable` 能力,调用 `ctx.subagents.startContinuable()`,并返回 `{ kind: 'continuable', subagentId }`,渲染为 `started subagent `。该路由在 inbox 接受时结算:子 agent 自此拥有自己的轮次,因此该调用既不等待也不收集结果。通过该 id 查看其 transcript(文本记录)仍是其详细输出的来源,可选的全局 `send_message` 工具则向其发送更多工作。每当子 agent 的 Activation 结束,继续执行服务都会投递一条结算通知,其中包含结束结果及可能存在的最终 assistant 消息,且这项投递不依赖 `report`。启动可继续工作不要求加载 `send_message`。见[后台 subagent Agent Note](../../../.agents/notes/implemented/feature/2026-07-08-background-subagent-tasks.md)、[可继续的 subagent Agent Note](../../../.agents/notes/implemented/feature/2026-07-28-continuable-subagent-conversations.md)和[后台优先委派 Agent Note](../../../.agents/notes/implemented/feature/2026-08-11-background-first-continuable-delegation.md)。 +`backgroundMode` 同时选择后台路由与省略 `run_in_background` 时的默认行为。`one-shot` 默认在前台等待;显式传入 `true` 时,它会注册一个归父级所有的普通 Task,并返回规范值 `{ kind: 'background', jobId }`,渲染为 `started background subagent job `,即使提供方支持可继续子 agent 也不例外。通用 Task 工具负责其后续状态、收集、取消和通知;失败 Task 的 detail 会保留终止原因与同一份可选提供方诊断。`continuable` 在参数省略或为 `true` 时于后台运行;显式传入 `false` 时则在前台等待结果。其后台路由要求提供方具备 `prepareContinuable` 能力,调用 `ctx.subagents.startContinuable()`,并返回 `{ kind: 'continuable', subagentId }`,渲染为 `started subagent `。该路由在 inbox 接受时结算:子 agent 自此拥有自己的轮次,因此该调用既不等待也不收集结果。通过该 id 查看其 transcript(文本记录)仍是其详细输出的来源,可选的全局 `send_message` 工具则向其发送更多工作。每当子 agent 的 Activation 结束,继续执行服务都会投递一条结算通知,其中包含结束结果及可能存在的最终 assistant 消息,且这项投递不依赖 `report`。启动可继续工作不要求加载 `send_message`。见[后台 subagent Agent Note](../../../.agents/notes/implemented/feature/2026-07-08-background-subagent-tasks.zh.md)、[可继续的 subagent Agent Note](../../../.agents/notes/implemented/feature/2026-07-28-continuable-subagent-conversations.zh.md)和[后台优先委派 Agent Note](../../../.agents/notes/implemented/feature/2026-08-11-background-first-continuable-delegation.zh.md)。 -`toolFilter` 会改变子 agent 的全局工具层,但不是从父级派生的权限上限。见 [agent 作用域的安全非目标](../../../.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.md#security-and-authority-are-non-goals)。 +`toolFilter` 会改变子 agent 的全局工具层,但不是从父级派生的权限上限。见 [agent 作用域的安全非目标](../../../.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.zh.md#security-and-authority-are-non-goals)。 ## 配置 @@ -29,7 +29,7 @@ ## 并发 -前台调用和后台调用均并发安全:同一条 assistant 消息中的同级委派会在循环的滚动池(`maxParallelToolCalls`)下重叠执行,结果仍按模型顺序提交。子 agent 在各自的会话中工作,一次运行绝不变更父会话;一次性后台形态对父级拥有状态的唯一写入是注册一个 Task——这是一次同步、可交换、能容忍并发分发的插入,因此重叠的后台调用按分发竞态顺序获得各自的 job id。协调同级工作区效果由模型负责,正如模型已经对后台和可继续子 agent 所承担的那样。见 [并行 subagent Agent Note](../../../.agents/notes/implemented/feature/2026-08-09-parallel-subagent-delegations.md) 和 [并行工具调用 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-parallel-tool-call-execution.md)。 +前台调用和后台调用均并发安全:同一条 assistant 消息中的同级委派会在循环的滚动池(`maxParallelToolCalls`)下重叠执行,结果仍按模型顺序提交。子 agent 在各自的会话中工作,一次运行绝不变更父会话;一次性后台形态对父级拥有状态的唯一写入是注册一个 Task——这是一次同步、可交换、能容忍并发分发的插入,因此重叠的后台调用按分发竞态顺序获得各自的 job id。协调同级工作区效果由模型负责,正如模型已经对后台和可继续子 agent 所承担的那样。见 [并行 subagent Agent Note](../../../.agents/notes/implemented/feature/2026-08-09-parallel-subagent-delegations.zh.md) 和 [并行工具调用 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-parallel-tool-call-execution.zh.md)。 ## 模型体验 @@ -37,7 +37,7 @@ #### 模型看到的内容 -当提供方存在时,以当前实例配置的名称公开已生成的默认 [`subagent` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-subagent)。提供方是否继承上下文会改变工具描述和提示词描述。启用后台模式会添加 `run_in_background`:可继续模式会记录其默认值为 `true`、运行时结算通知与显式前台覆盖;一次性模式会记录其默认值为 `false`,以及用 `job_output` 收集或用 `job_kill` 停止的 job id。当工具在本次组装的作用域中可见时,一个 `tool:` 系统提示词 section 会指示模型同时启动相互独立的可继续委派、在它们运行时继续工作,并且仅当下一步动作依赖结果时选择前台;工具限制会同时移除其 schema 和这段指引。 +当提供方存在时,以当前实例配置的名称公开已生成的默认 [`subagent` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-subagent)。提供方是否继承上下文会改变工具描述和提示词描述。启用后台模式会添加 `run_in_background`:可继续模式会记录其默认值为 `true`、运行时结算通知与显式前台覆盖;一次性模式会记录其默认值为 `false`,以及用 `job_output` 收集或用 `job_kill` 停止的 job id。当工具在本次组装的作用域中可见时,一个 `tool:` 系统提示词 section 会指示模型同时启动相互独立的可继续委派、在它们运行时继续工作,并且仅当下一步动作依赖结果时选择前台;工具限制会同时移除其 schema 和这段指引。 #### Token 影响 @@ -51,7 +51,7 @@ #### 模型看到的内容 -调用会保留描述和提示词。成功时只包含子 agent 的最终文本;其他结果变为 `Error: `。子 agent 中间步骤不会进入父级。 +调用会保留描述和提示词。成功时只包含子 agent 的最终文本;其他结果会变为 `Error: <终止原因>`,随后在存在时附上安全的提供方诊断,再附上任何部分 assistant 文本。子 agent 中间步骤不会进入父级。 #### Token 影响 @@ -65,7 +65,7 @@ #### 模型看到的内容 -在配置的可继续模式下,启动时返回内容恰为 `started subagent `;在配置的一次性模式下,则返回 `started background subagent job `。一次性模式下,通用 Task 接口提供后续状态、最终输出、取消响应和通知。可继续模式下,本工具不返回自己的结果;子 agent 的结算会以[服务负责的通知](../subagent/README.md#settlement-notice)到达父级,独立加载的 `send_message` 工具会投递后续消息,而通过其 id 查看子 agent 的 transcript 即是其详细输出来源。 +在配置的可继续模式下,启动时返回内容恰为 `started subagent `;在配置的一次性模式下,则返回 `started background subagent job `。一次性模式下,通用 Task 接口提供后续状态、最终输出、取消响应和通知;若结果携带提供方诊断,失败状态的 detail 会包含它。可继续模式下,本工具不返回自己的结果;子 agent 的结算会以[服务负责的通知](../subagent/README.zh.md#settlement-notice)到达父级,独立加载的 `send_message` 工具会投递后续消息,而通过其 id 查看子 agent 的 transcript 即是其详细输出来源。 #### Token 影响 diff --git a/packages/subagent/tool-subagent/package.json b/packages/subagent/tool-subagent/package.json index b1a73d0e8c..9ddab59996 100644 --- a/packages/subagent/tool-subagent/package.json +++ b/packages/subagent/tool-subagent/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-subagent", "description": "Model-facing subagent delegation tool over the ctx.subagents seam", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subagent/tool-subagent/src/index.ts b/packages/subagent/tool-subagent/src/index.ts index e23aae4157..ef0e9941eb 100644 --- a/packages/subagent/tool-subagent/src/index.ts +++ b/packages/subagent/tool-subagent/src/index.ts @@ -113,7 +113,9 @@ async function settleStart(start: Promise, signal: AbortSignal): Pr try { return await settleRun(await start) } catch (error: unknown) { - return signal.aborted + // Product providers aggregate startup and rollback failures. Cancellation + // must not turn a failed cleanup into a cleanly killed Job. + return signal.aborted && !(error instanceof AggregateError) ? { status: 'killed' } : { status: 'failed', detail: String(error) } } @@ -140,18 +142,25 @@ function stopReasonError(result: SubagentResult): string | undefined { } /** - * Append the child's preserved partial answer to a stop-reason error so a - * truncated or cancelled child's real text still reaches the parent model. + * Append provider-authored failure detail and the child's preserved partial + * answer to a stop-reason error, keeping diagnostic text separate from the + * child's assistant output. * @param error - the stop-reason headline. - * @param output - the child's selected output (`SubagentResult.output`). - * @returns the headline, extended with the partial text when any exists. + * @param result - the child's terminal result. + * @returns the headline, diagnostic, and partial text that are present. */ -function withPartialText(error: string, output: ContentBlock[]): string { - const text = output +function withDiagnosticAndPartialText(error: string, result: SubagentResult): string { + const diagnostic = result.diagnostic === undefined + ? '' + : `\nDiagnostic: ${result.diagnostic}` + const text = result.output .filter((block): block is Extract => block.type === 'text') .map(block => block.text) .join('') - return text.length === 0 ? error : `${error}\nPartial output before the run ended:\n${text}` + const partial = text.length === 0 + ? '' + : `\nPartial output before the run ended:\n${text}` + return `${error}${diagnostic}${partial}` } type ForegroundToolResult = { @@ -171,7 +180,7 @@ async function settleForegroundRun(run: SubagentRun): Promise void) | undefined const mount = (provider: SubagentProvider): void => { // A numeric cap the provider cannot enforce is a misconfiguration — fail at @@ -357,7 +366,7 @@ export function apply(ctx: Context, config: Config): void { render: (_args, value) => [{ type: 'text', text: value.kind === 'background' - ? `started background subagent task ${value.jobId}` + ? `started background subagent job ${value.jobId}` : value.kind === 'continuable' ? `started subagent ${value.subagentId}` : outputValueText(value.output), diff --git a/packages/subagent/tool-subagent/tests/scripted-provider.ts b/packages/subagent/tool-subagent/tests/scripted-provider.ts index 0be724a8bb..c0da403cd4 100644 --- a/packages/subagent/tool-subagent/tests/scripted-provider.ts +++ b/packages/subagent/tool-subagent/tests/scripted-provider.ts @@ -27,6 +27,8 @@ export interface Config { reply?: string /** Terminal result reason. */ stopReason?: SubagentStopReason + /** Safe non-assistant detail for a non-completed result. */ + diagnostic?: string /** Start-time features advertised by the provider. */ capabilities?: Partial /** Whether tool descriptions say the child inherits completed turns. */ @@ -65,11 +67,17 @@ class ScriptedSubagentProvider implements SubagentProvider { throw new Error('scripted subagent start aborted before publication') } - const resultFor = (): SubagentResult => ({ - output, - ...wantsStructured ? { structured: this.config.structured ?? { reply } } : {}, - stopReason: state.cancelled ? 'aborted' : stopReason, - }) + const resultFor = (): SubagentResult => { + const terminal = state.cancelled ? 'aborted' : stopReason + return { + output, + ...wantsStructured ? { structured: this.config.structured ?? { reply } } : {}, + ...this.config.diagnostic !== undefined && terminal !== 'completed' + ? { diagnostic: this.config.diagnostic } + : {}, + stopReason: terminal, + } + } const gate = Promise.resolve(this.config.onStart?.(request)) const result = gate.then(() => new Promise((resolve) => { setTimeout(() => { resolve(resultFor()) }, 0) diff --git a/packages/subagent/tool-subagent/tests/tool-subagent.spec.ts b/packages/subagent/tool-subagent/tests/tool-subagent.spec.ts index be9f3a0062..1ee5e40228 100644 --- a/packages/subagent/tool-subagent/tests/tool-subagent.spec.ts +++ b/packages/subagent/tool-subagent/tests/tool-subagent.spec.ts @@ -85,7 +85,11 @@ describe('dsh-tool-subagent', () => { it('registers a `subagent` tool that delegates to the configured provider and returns its output', async () => { const ctx = await setup({ provider: 'mock' }, { reply: 'child says hi' }) - const result = await callSubagent(ctx, { description: 'do a thing', prompt: 'go research X' }) + const result = await callSubagent(ctx, { + description: 'do a thing', + prompt: 'go research X', + run_in_background: false, + }) expect(result.isError).toBe(false) if (result.isError) throw new Error('expected subagent success') expect(result.value).toEqual({ @@ -181,6 +185,22 @@ describe('dsh-tool-subagent', () => { expect(text(result)).toContain('scripted subagent reply') }) + it('renders provider diagnostics before preserved partial assistant output', async () => { + const ctx = await setup({ provider: 'mock' }, { + reply: 'partial assistant text', + diagnostic: 'Claude Code denied a tool request', + stopReason: 'error', + }) + + const result = await callSubagent(ctx, { description: 'd', prompt: 'p' }) + expect(result.isError).toBe(true) + expect(text(result)).toBe( + 'Error: subagent run failed\n' + + 'Diagnostic: Claude Code denied a tool request\n' + + 'Partial output before the run ended:\npartial assistant text', + ) + }) + it('registers under a configurable toolName so multiple providers can coexist', async () => { // The defining multi-provider use case: two loads, two distinct tool names, // each bound to a different provider — the tool registry rejects duplicate @@ -814,7 +834,7 @@ describe('dsh-tool-subagent background mode', () => { agent: parent, }) - expect(text(started)).toBe('started background subagent task subagent-1') + expect(text(started)).toBe('started background subagent job subagent-1') expect(prepareCalls).toBe(0) }) @@ -826,7 +846,7 @@ describe('dsh-tool-subagent background mode', () => { expect(start.isError).toBe(false) if (start.isError) throw new Error('expected background subagent success') expect(start.value).toEqual({ kind: 'background', jobId: 'subagent-1' }) - expect(text(start)).toBe('started background subagent task subagent-1') + expect(text(start)).toBe('started background subagent job subagent-1') const collected = await ctx.tools.execute({ signal: testToolSignal, @@ -848,6 +868,36 @@ describe('dsh-tool-subagent background mode', () => { expect(text(again)).toBe('background answer\n[status: completed]') }) + it('preserves provider diagnostics in one-shot background failure detail', async () => { + const ctx = await backgroundSetup({ provider: 'mock' }, { + reply: 'not background output', + diagnostic: 'Claude Code cancelled an unattended dialog', + stopReason: 'error', + }) + const parent = ownerAgent(ctx, 'sess-parent') + + const started = await ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('diagnostic-background-start'), + name: 'subagent', + arguments: { description: 'd', prompt: 'p', run_in_background: true }, + agent: parent, + }) + expect(text(started)).toBe('started background subagent job subagent-1') + + const output = await ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('diagnostic-background-output'), + name: 'job_output', + arguments: { job_id: 'subagent-1', wait: true }, + agent: parent, + }) + expect(text(output)).toBe( + '(no new output)\n' + + '[status: failed, error; diagnostic: Claude Code cancelled an unattended dialog]', + ) + }) + it('fails loud when the tasks runtime is not loaded', async () => { const ctx = await setup({ provider: 'mock' }) const result = await callSubagent(ctx, { description: 'd', prompt: 'p', run_in_background: true }) @@ -887,7 +937,7 @@ describe('dsh-tool-subagent background mode', () => { arguments: { description: 'broken', prompt: 'p', run_in_background: true }, agent: parent, }) - expect(text(started)).toBe('started background subagent task subagent-1') + expect(text(started)).toBe('started background subagent job subagent-1') const output = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('broken-output'), @@ -935,6 +985,48 @@ describe('dsh-tool-subagent background mode', () => { expect(text(output)).toBe('(no new output)\n[status: killed]') }) + it('reports startup rollback failure after cancellation as a failed job', async () => { + const ctx = await backgroundSetup({ provider: 'mock' }) + const parent = ownerAgent(ctx, 'sess-parent') + ctx.subagents.registerProvider({ + name: 'broken-start-rollback', + capabilities: { outputSchema: false, depthLimit: false, toolFilter: false, persona: false }, + inheritsParentContext: false, + start: request => new Promise((_resolve, reject) => { + request.signal.addEventListener('abort', () => { + reject(new AggregateError( + [new Error('startup aborted'), new Error('cleanup failed')], + 'startup failed and cleanup also failed', + )) + }, { once: true }) + }), + }) + tool.apply(ctx, { provider: 'broken-start-rollback', toolName: 'subagent_broken_rollback' }) + + await ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('broken-rollback-start'), + name: 'subagent_broken_rollback', + arguments: { description: 'broken rollback', prompt: 'p', run_in_background: true }, + agent: parent, + }) + await ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('broken-rollback-kill'), + name: 'job_kill', + arguments: { job_id: 'subagent-1' }, + agent: parent, + }) + const output = await ctx.tools.execute({ + signal: testToolSignal, + callId: CallId('broken-rollback-output'), + name: 'job_output', + arguments: { job_id: 'subagent-1', wait: true }, + agent: parent, + }) + expect(text(output)).toContain('[status: failed, AggregateError: startup failed and cleanup also failed]') + }) + it('forwards job_kill reasons through the run signal (and defaults one when absent)', async () => { // Use a provider that remains live until its signal is aborted. const ctx = await backgroundSetup({ provider: 'mock', agentOptions: { model: 'child-model' } }) @@ -966,8 +1058,8 @@ describe('dsh-tool-subagent background mode', () => { const startOne = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('h1'), name: 'subagent_hang', arguments: { description: 'one', prompt: 'p', run_in_background: true }, agent: parent }) const startTwo = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('h2'), name: 'subagent_hang', arguments: { description: 'two', prompt: 'p', run_in_background: true }, agent: parent }) - expect(text(startOne)).toBe('started background subagent task subagent-1') - expect(text(startTwo)).toBe('started background subagent task subagent-2') + expect(text(startOne)).toBe('started background subagent job subagent-1') + expect(text(startTwo)).toBe('started background subagent job subagent-2') const withReason = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('k1'), name: 'job_kill', arguments: { job_id: 'subagent-1', reason: 'superseded' }, agent: parent }) const withoutReason = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('k2'), name: 'job_kill', arguments: { job_id: 'subagent-2' }, agent: parent }) diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index b294c97415..871f3c0769 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: 72a30775f45a8140935a013c63e9427e9cbd94d6 -README.zh.md: 35c801a1f4a5e8ad161b8c4ff57ad6ffa07e699d +README.md: 790db2c3c82fc9e359cae6a0ff1eab156b2776b5 +README.zh.md: 4a01d23b4e409edb6d6ad914351c7a7b95b3c744 diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index 72a30775f4..790db2c3c8 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -8,6 +8,7 @@ The shared process substrate for one execution world: executable lookup, fully-s |---|---|---| | [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | Service Definition: executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | | [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | Local Service Provider: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, and terminate-and-join disposal | +| [`win32-process`](win32-process/README.md) (`@deepseek-ai/dsh-win32-process`) | — | Windows-only low-level library: the single Koffi owner for restricted process creation, inherited/anonymous-pipe stdio, suspended Job assignment, waits, and handle cleanup | The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index 35c801a1f4..4a01d23b4e 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -2,13 +2,14 @@ [English](README.md) | 中文 -这里集中提供一个执行世界的共享进程基底:可执行文件查找、具有原始或收集式 stdio 的完全明确指定的受管子进程树,以及一项底层终端进程原语,负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方:[bash 执行器](../shell/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../terminal/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见 [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +这里集中提供一个执行世界的共享进程基底:可执行文件查找、具有原始或收集式 stdio 的完全明确指定的受管子进程树,以及一项底层终端进程原语,负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方:[bash 执行器](../shell/README.zh.md)、[LSP 主机](../lsp/README.zh.md)、[PTY shell 后端](../terminal/README.zh.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.zh.md)。参见 [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md)。 | 包 | ctx 键 | 角色 | |---|---|---| -| [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | Service Definition:可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期,以及共享的环境/输出词汇 | -| [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地 Service Provider:detached 进程树、有界收集/spill、`node-pty`、前台/会话检查、进程树信号发送,以及先终止再等待退出的 dispose(资源释放) | +| [`subprocess`](subprocess/README.zh.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | Service Definition:可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期,以及共享的环境/输出词汇 | +| [`subprocess-local`](subprocess-local/README.zh.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地 Service Provider:detached 进程树、有界收集/spill、`node-pty`、前台/会话检查、进程树信号发送,以及先终止再等待退出的 dispose(资源释放) | +| [`win32-process`](win32-process/README.zh.md)(`@deepseek-ai/dsh-win32-process`) | 无 | 仅限 Windows 的底层库:restricted process creation、继承/匿名管道 stdio、suspended Job 分配、wait 与句柄清理的唯一 Koffi owner | 即使消费方重载,进程生命周期仍由服务负责管理;消费方负责定义进程的含义(一条 bash 命令、未来的非 shell 运行器),以及决定塑造该进程的每一项默认值。 -子系统参考——spawn spec、输出读取器、结果、`DSH_*` 环境——见 [docs/subsystems/subprocess.md](../../docs/subsystems/subprocess.md);seam 决定见 [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +子系统参考——spawn spec、输出读取器、结果、`DSH_*` 环境——见 [docs/subsystems/subprocess.md](../../docs/subsystems/subprocess.zh.md);seam 决定见 [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md)。 diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index 10c7c1cd62..2a77e8192c 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: a88eab6776c7259bb8ee8fdad2d6002d8f57e133 -README.zh.md: ce088557875daceb5643f13bef78d6ea9a61e12f +README.md: e77aa36d8e6dc4ac999261f3a7d0c21a81ed08ff +README.zh.md: 25f7fed19898c81ececb5308b67e8d8a1140e3af diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index a88eab6776..e77aa36d8e 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -11,13 +11,13 @@ Local Service Provider for the [`@deepseek-ai/dsh-subprocess`](../subprocess/REA - **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. - **Executable lookup** — `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions; relative paths containing separators are rejected at the seam, and relative PATH entries resolve from the host process cwd. -- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal text, inspects and signals the current foreground process group, and exposes one awaited termination operation that sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. +- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal text, inspects and signals the current foreground process group, and exposes one awaited termination operation that sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. On Windows the koffi-backed inspector enumerates the process table through Toolhelp32, combines GetProcessTimes start identities with zero-time process-handle waits for liveness, reports the shell pid as the pseudo foreground group (Windows has no POSIX groups), and teardown verifies the shell's termination because externally taskkilled shells may never fire node-pty's exit notification. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles so its own disposal can escalate every running tree and await its exit; quiescent and spawn-failed handles leave the live set after whole-tree or terminal-session cleanup finishes. - **Synchronous host-exit finalization** — while the service effect is active, a Node `exit` listener force-terminates every ordinary tree and observable terminal session still in the same live sets. The local-only operations send POSIX SIGKILL to the managed group, run Windows `taskkill /T /F`, and synchronously signal captured/current terminal identities around the PTY root kill; they create no promise or timer, preserve the host's exit code and diagnostic, contain each target's failure, and do not claim quiescence. Normal disposal keeps the awaited graceful path above. See the [host-exit cleanup decision](../../../.agents/notes/implemented/bug-fix/2026-08-11-synchronous-subprocess-exit-cleanup.md). ## Model Experience -Indirectly, through Consumers (today the bash executor family behind `dsh-tool-bash`), which own all model-facing rendering of process output and lifecycle. +Indirectly, through Consumers (the bash executor family behind `dsh-tool-bash`), which own all model-facing rendering of process output and lifecycle. #### KV Cache effect @@ -26,7 +26,7 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work - **Windows tree support is best-effort** — termination routes through `taskkill /PID /T /F` with all outcomes contained (absent tree, races, missing binary), and liveness falls back to the direct-child boundary. -- **Terminal process inspection is Linux/macOS only** — the terminal primitive fails when its inspector has no supported platform implementation; Linux exact probes cover x64 and arm64, while macOS uses `ps` snapshots. +- **Windows terminal signalling is console-wide** — SIGINT is delivered as a `\x03` Ctrl-C input write that conhost turns into a console-wide CTRL_C event; SIGTSTP and SIGHUP are rejected as unavailable; a `taskkill` without `/F` does not terminate console processes, so the teardown TERM tier is a grace wait before the `/F` escalation. Windows readiness has no exact stdin-wait tier: the prompt-marker fast path compares the shell pid as the pseudo foreground group, and silence/timing tiers cover the rest. - **A daemonized terminal descendant can still escape the observable boundary** — on macOS, a child that reparents before any foreground-inspection snapshot is no longer discoverable from the `node-pty` root; on Linux, a child that calls `setsid` leaves both the tree and owned terminal session. The local provider does not add a continuous process-table monitor. - **In-process cleanup requires a JavaScript-observable exit** — direct `process.exit()`, default uncaught exceptions, and default unhandled rejections emit Node's synchronous `exit` event. The default OS disposition for an unhandled `SIGTERM`, `SIGINT`, or `SIGHUP` bypasses that event; an application covers those signals only by installing a handler that performs normal disposal or calls `process.exit()`. `SIGKILL`, fatal OOM, `process.abort()`, native crashes, power loss, and any failure that cannot run JavaScript require an external supervisor, container init, or equivalent OS owner. - **The credential scrub is a name heuristic** — `*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSPHRASE*`) pass through, and a whitelist for over-scrubbed vars is noted future work. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index ce08855787..25f7fed198 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -2,22 +2,22 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地 Service Provider。`LocalSubprocessRuntime` 解析本地可执行文件,以显式 stdio spawn 普通 detached 进程树,并通过 `node-pty` 加平台进程检查实现终端进程。该实现没有任何配置:每项处置方式、限制、终端尺寸、宽限期与目录都来自调用方能力 seam([`dsh-bash-local`](../../shell/bash-local/README.md)、[`dsh-lsp-stdio`](../../lsp/lsp-stdio/README.md) 和 [`dsh-terminal-bash`](../../terminal/terminal-bash/README.md))。 +[`@deepseek-ai/dsh-subprocess`](../subprocess/README.zh.md) seam 的本地 Service Provider。`LocalSubprocessRuntime` 解析本地可执行文件,以显式 stdio spawn 普通 detached 进程树,并通过 `node-pty` 加平台进程检查实现终端进程。该实现没有任何配置:每项处置方式、限制、终端尺寸、宽限期与目录都来自调用方能力 seam([`dsh-bash-local`](../../shell/bash-local/README.zh.md)、[`dsh-lsp-stdio`](../../lsp/lsp-stdio/README.zh.md) 和 [`dsh-terminal-bash`](../../terminal/terminal-bash/README.zh.md))。 ## 行为 - **以适合平台的方式发送信号的 detached 进程树**:POSIX 子进程使用 `detached` spawn(拥有独立进程组),信号以负 pgid 发送并以直接子进程作为回退;Windows 通过 `taskkill /PID /T /F` 终止进程树。`terminate()`(句柄唯一的终止操作)先发送 SIGTERM,经过 spec 的宽限期后再发送 SIGKILL(沿用 OpenCode 的升级策略;流水线与子 shell 会随父进程一起结束),进程树消亡后为空操作;`waitForExit()` 轮询整棵进程树的存活状态,使消费方的拆卸能确认真正的完全停稳。组长进程退出后,仍然打开的管道也只获得同样有界的排空宽限期,因此存活的后代进程无法无限期地拖住结果不结算。系统会容忍 ESRCH;重新指定父进程并脱离该组的 daemon 仍可能存活。 - **按流划分的处置方式**:`'pipe'` 把原始流原样交给调用方(协议分帧仍归消费方所有);`'inherit'` 直通父进程的描述符;收集模式(collect)在输出超过上限后于内存中保留尾部(错误与结果通常聚集在末尾,沿用 pi/OpenCode 的理由),并在配置了 spill 上限时把完整流追加到一个私有临时文件;省略 `spill` 则只保留用于诊断的尾部。某条流大于 spill 上限时,会丢弃已不完整的 spill,仅返回带截断标记的尾部;spill 文件描述符在结算时封存,最终关闭失败时则不公布路径,以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机,位于按需创建、权限为 `0700` 的每进程目录之下。 -- **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 +- **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-api.zh.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.zh.md)。 - **基于偏移量的读取**:收集模式的读取器按完整流的字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 - **可执行文件查找**:`resolveExecutable` 检查绝对文件,或根据平台可执行文件扩展名在清理后的有效 PATH 中搜索;含分隔符的相对路径在该 seam 处被拒绝,相对 PATH 条目从宿主进程 cwd 解析。 -- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端文本,检查当前前台进程组并向其发送信号,还会公开一项须等待的终止操作,在终止顶层 shell 前后清理后代进程。每次前台检查都会保留根进程树中的精确身份;Linux 还会在 POSIX 会话 leader 退出后枚举该会话。因此,之前观察到的 macOS 后代以及同会话 Linux 成员在重新设定父进程后仍受围栏保护,pid/start 身份则防止清理跟随 PID 复用。上层 PTY 后端负责提示符就绪、缓冲区与面向模型的操作。 +- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端文本,检查当前前台进程组并向其发送信号,还会公开一项须等待的终止操作,在终止顶层 shell 前后清理后代进程。每次前台检查都会保留根进程树中的精确身份;Linux 还会在 POSIX 会话 leader 退出后枚举该会话。因此,之前观察到的 macOS 后代以及同会话 Linux 成员在重新设定父进程后仍受围栏保护,pid/start 身份则防止清理跟随 PID 复用。在 Windows 上,基于 koffi 的检查器通过 Toolhelp32 枚举进程表,把 GetProcessTimes 启动身份与进程句柄零时等待结合起来判断存活状态,并把 shell pid 作为伪前台进程组(Windows 没有 POSIX 进程组)。拆卸会验证 shell 已终止,因为被外部 taskkill 的 shell 可能永远不会触发 node-pty 的退出通知。上层 PTY 后端负责提示符就绪、缓冲区与面向模型的操作。 - **先终止再等待退出的 dispose(资源释放)**:服务保留存活句柄,使自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;完全停稳与 spawn 失败的句柄会在整棵进程树或 terminal session 清理完成后离开存活集合。 -- **同步宿主退出最终清理**:服务 effect 仍有效时,Node `exit` listener 会强制终止同一组存活集合中仍存在的每棵普通进程树和可观察 terminal session。这些仅供本地实现使用的操作会向受管 POSIX 进程组发送 SIGKILL、在 Windows 运行 `taskkill /T /F`,并在终止 PTY root 前后同步向已捕获及当前可观察的 terminal 身份发送信号;它们不会创建 Promise 或 timer,不改变宿主退出码与诊断,会分别包含每个目标的失败,也不会声称已经完全停稳。正常 dispose 仍使用上面的须等待温和路径。参见[宿主退出清理决策](../../../.agents/notes/implemented/bug-fix/2026-08-11-synchronous-subprocess-exit-cleanup.md)。 +- **同步宿主退出最终清理**:服务 effect 仍有效时,Node `exit` listener 会强制终止同一组存活集合中仍存在的每棵普通进程树和可观察 terminal session。这些仅供本地实现使用的操作会向受管 POSIX 进程组发送 SIGKILL、在 Windows 运行 `taskkill /T /F`,并在终止 PTY root 前后同步向已捕获及当前可观察的 terminal 身份发送信号;它们不会创建 Promise 或 timer,不改变宿主退出码与诊断,会分别包含每个目标的失败,也不会声称已经完全停稳。正常 dispose 仍使用上面的须等待温和路径。参见[宿主退出清理决策](../../../.agents/notes/implemented/bug-fix/2026-08-11-synchronous-subprocess-exit-cleanup.zh.md)。 ## 模型体验 -通过 Consumer 间接影响(目前是 `dsh-tool-bash` 背后的 bash 执行器家族);进程输出与生命周期面向模型的全部渲染归 Consumer 所有。 +通过 Consumer 间接影响(即 `dsh-tool-bash` 背后的 bash 执行器家族);进程输出与生命周期面向模型的全部渲染归 Consumer 所有。 #### KV Cache 影响 @@ -26,7 +26,7 @@ ## 已知限制与暂缓事项 - **Windows 进程树支持仅为尽力而为**:终止经由 `taskkill /PID /T /F` 完成,所有结果都被就地吸收,不向外抛出(进程树已不存在、竞态、二进制缺失),存活探测则回退到直接子进程边界。 -- **终端进程检查仅支持 Linux/macOS**:检查器没有受支持的平台实现时,终端原语会失败;Linux 精确探针覆盖 x64 与 arm64,macOS 则使用 `ps` 快照。 +- **Windows 终端信号是控制台级的**:SIGINT 以 `\x03` Ctrl-C 输入写入投递,由 conhost 转为控制台级 CTRL_C 事件;SIGTSTP 与 SIGHUP 被拒绝(不可用);不带 `/F` 的 `taskkill` 无法终止控制台进程,因此拆卸的 TERM 档是 `/F` 升级前的宽限等待。Windows 就绪没有精确的 stdin-wait 档:prompt-marker 快路径把 shell pid 作为伪前台进程组比较,其余由静默/计时档覆盖。 - **守护化的终端后代仍可能逃出可观察边界**:在 macOS 上,子进程如果在任何前台检查快照之前重新设定父进程,将无法再从 `node-pty` 根进程发现;在 Linux 上,调用 `setsid` 的子进程会同时离开进程树与自有终端会话。本地提供方不会新增持续进程表监视器。 - **进程内清理要求退出阶段仍能执行 JavaScript**:直接 `process.exit()`、默认未捕获异常和默认未处理 rejection 会发出 Node 同步 `exit` 事件。未安装 handler 时,`SIGTERM`、`SIGINT` 或 `SIGHUP` 的默认 OS 处置不会发出该事件;应用只有安装执行正常 dispose 或调用 `process.exit()` 的 handler 才能覆盖这些信号。`SIGKILL`、fatal OOM、`process.abort()`、native crash、断电,以及任何无法运行 JavaScript 的故障,都需要外部 supervisor、容器 init 或等价的 OS 所有者负责。 - **凭据清除依赖名称启发式规则**:只匹配 `*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`;名称不同的 secret(例如 `*PASSPHRASE*`)会继续传递,对误删变量引入白名单属于已记录的后续工作。 diff --git a/packages/subprocess/subprocess-local/package.json b/packages/subprocess/subprocess-local/package.json index 7581f54183..8dbb8acb6a 100644 --- a/packages/subprocess/subprocess-local/package.json +++ b/packages/subprocess/subprocess-local/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subprocess-local", "description": "Local-subprocess implementation of the DeepSeek Harness subprocess seam", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -42,7 +42,8 @@ "@deepseek-ai/cordis": "workspace:^" }, "dependencies": { - "node-pty": "^1.1.0" + "koffi": "^3.1.0", + "node-pty": "1.2.0-beta.15" }, "devDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", diff --git a/packages/subprocess/subprocess-local/src/process-inspector.ts b/packages/subprocess/subprocess-local/src/process-inspector.ts index f31de010de..89effc0082 100644 --- a/packages/subprocess/subprocess-local/src/process-inspector.ts +++ b/packages/subprocess/subprocess-local/src/process-inspector.ts @@ -3,6 +3,7 @@ import { closeSync, openSync, readFileSync, readdirSync, readSync } from 'node:fs' import { execFileSync } from 'node:child_process' import type { SubprocessTerminalSignal } from '@deepseek-ai/dsh-subprocess' +import { createWindowsProcessInspector } from './windows-inspector.ts' /** PID plus start identity, preventing teardown escalation after PID reuse. */ export interface ProcessIdentity { @@ -370,5 +371,6 @@ export function createProcessInspector( ): ProcessInspector { if (platform === 'linux') return new LinuxProcessInspector(arch, internals) if (platform === 'darwin') return new MacProcessInspector(internals) + if (platform === 'win32') return createWindowsProcessInspector() throw new Error(`subprocess-local: terminal inspection is unsupported on platform ${platform}`) } diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts index 6d818c8a7f..0a51287d24 100644 --- a/packages/subprocess/subprocess-local/src/terminal.ts +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -50,11 +50,13 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { * @param terminal - allocated node-pty process. * @param inspector - platform process/session operations. * @param graceMs - TERM-to-KILL and exit-wait grace. + * @param platform - host platform; defaults to the running platform, injectable for deterministic tests. */ constructor( private readonly terminal: IPty, private readonly inspector: ProcessInspector, private readonly graceMs: number, + private readonly platform: NodeJS.Platform = process.platform, ) { this.pid = terminal.pid this.rootIdentity = inspector.processTree(this.pid).find(member => member.pid === this.pid) @@ -98,6 +100,19 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') } + if (this.platform === 'win32') { + if (signal === 'SIGINT') { + // Windows has no process-group signalling: a `\x03` input write is the + // Ctrl-C delivery path conhost turns into a console-wide CTRL_C event + // for attached processes. node-pty's signal kills throw on Windows, so + // no signal ever reaches the inspector. + this.terminal.write('\x03') + return foreground.processGroupId + } + if (signal === 'SIGTSTP' || signal === 'SIGHUP') { + throw new Error(`signal ${signal} is unsupported on Windows; only SIGINT, SIGTERM, and SIGKILL are available`) + } + } this.inspector.signalGroup(foreground.processGroupId, signal) return foreground.processGroupId } @@ -214,6 +229,10 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { } private async stopShell(): Promise { + if (this.platform === 'win32') { + await this.stopShellWindows() + return + } if (!this.exited) { try { this.terminal.kill('SIGTERM') @@ -233,6 +252,44 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { if (!this.exited) throw new Error(`terminal cleanup failed; surviving pid: ${this.pid}`) } + private async stopShellWindows(): Promise { + // node-pty's Windows kill(signal) throws ("Signals not supported on + // windows"), and its bare kill() delegates to a console-list agent that + // fails when the parent has no console. taskkill tree escalation is the + // teardown path, fenced on the shell's start identity like every + // descendant; a root identity miss falls back to the bare kill. taskkill + // termination also does not reliably fire node-pty's exit notification + // (the same console-list agent), so the tiers verify the shell's absence + // through the inspector instead of waiting on `done` alone. + const shellGone = (): boolean => + this.exited || (this.rootIdentity !== undefined && !this.inspector.isAlive(this.rootIdentity)) + if (!shellGone() && this.rootIdentity !== undefined) { + this.inspector.signalProcess(this.rootIdentity, 'SIGTERM') + await this.waitForWindowsShellExit() + } + if (!shellGone() && this.rootIdentity === undefined) { + try { + this.terminal.kill() + } catch (_topLevelAlreadyExitedDuringKill) { + // The exit callback is authoritative. + } + await Promise.race([this.done.then(() => undefined), delay(this.graceMs)]) + } + if (!shellGone() && this.rootIdentity !== undefined) { + this.inspector.signalProcess(this.rootIdentity, 'SIGKILL') + await this.waitForWindowsShellExit() + } + if (!shellGone()) throw new Error(`terminal cleanup failed; surviving pid: ${this.pid}`) + } + + private async waitForWindowsShellExit(): Promise { + const until = Date.now() + this.graceMs + while (!this.exited && Date.now() < until) { + if (this.rootIdentity !== undefined && !this.inspector.isAlive(this.rootIdentity)) return + await delay(Math.min(25, Math.max(1, until - Date.now()))) + } + } + private async closeOnce(): Promise { let survivors = await this.stopDescendants() if (survivors.length > 0) { @@ -243,7 +300,24 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { if (survivors.length > 0) { throw new Error(`terminal cleanup failed; surviving pids: ${survivors.map(member => member.pid).join(', ')}`) } + this.settleExitIfGone() this.dataDisposable.dispose() this.exitDisposable.dispose() } + + private settleExitIfGone(): void { + // An externally taskkilled Windows shell may never fire node-pty's exit + // notification (its console-list agent fails without a parent console), + // which would leave `done` — and every consumer awaiting it — unsettled + // forever. Teardown has just verified the shell's absence through the + // inspector, so a missing exit event is itself the outcome. + if (this.platform !== 'win32') return + if (this.exited) return + /* v8 ignore next -- stopShellWindows() verified the shell is gone or threw; + the identity re-check is a defensive fence for a future caller. */ + if (this.rootIdentity !== undefined && this.inspector.isAlive(this.rootIdentity)) return + this.exited = true + this.output.end() + this.outcome.resolve({ exitCode: null, signal: null }) + } } diff --git a/packages/subprocess/subprocess-local/src/windows-inspector.ts b/packages/subprocess/subprocess-local/src/windows-inspector.ts new file mode 100644 index 0000000000..7280cbb9ee --- /dev/null +++ b/packages/subprocess/subprocess-local/src/windows-inspector.ts @@ -0,0 +1,325 @@ +/** + * Windows process-table operations for terminal readiness, signalling, and + * teardown: Toolhelp32 snapshot enumeration with GetProcessTimes creation-time + * identity and process-handle wait-state liveness, the shell pid as a pseudo + * process group (Windows has no POSIX groups), and taskkill tree signalling. + * The koffi bindings load lazily so + * non-Windows processes never touch Win32 libraries; all decision logic takes + * an injectable internals boundary so suites can pin it on any host. + * @module dsh-subprocess-local/windows-inspector + */ + +import { spawnSync } from 'node:child_process' +import koffi from 'koffi' +import type { SubprocessTerminalSignal } from '@deepseek-ai/dsh-subprocess' +import type { ProcessIdentity, ProcessInspector } from './process-inspector.ts' + +/** One Toolhelp32 process-table row. */ +export interface ProcessEntry { + pid: number + parentPid: number +} + +/** Creation identity plus the process object's current wait state. */ +export interface WindowsProcessState { + /** GetProcessTimes creation identity used to fence PID reuse. */ + started: string + /** Whether a zero-time process-handle wait reports the process still running. */ + active: boolean +} + +/** Injectable Windows process operations used by one local PTY session. */ +export interface WindowsProcessInspectorInternals { + /** Enumerate the current process table (pid/parent pairs). */ + snapshot(): ProcessEntry[] + /** Return one process's creation identity and wait state, or undefined when unreadable. */ + processState(pid: number): WindowsProcessState | undefined + /** Terminate one process tree; `force` maps to taskkill `/F`. */ + taskkill(pid: number, force: boolean): void +} + +/** + * Walk a process table from one root in children-first order, retaining only + * members whose start identity is readable (unreadable members are detector + * misses, exactly like an unreadable `/proc` entry on Linux). + * @param entries - the process table snapshot. + * @param rootPid - the tree root to descend from. + * @param started - creation-time identity resolver for one member. + * @returns the root and its current transitive descendants, children first. + */ +/* jscpd:ignore-start -- the Windows inspector deliberately mirrors process-inspector.ts: + the decision logic (tree walk, identity fencing, group signalling) is the same contract over + Win32 primitives, per the persistent-pty note 2026-08-11-pwsh-persistent-pty. */ +export function windowsProcessTree( + entries: ProcessEntry[], + rootPid: number, + started: (pid: number) => string | undefined, +): ProcessIdentity[] { + const byPid = new Map(entries.map(entry => [entry.pid, entry])) + const root = byPid.get(rootPid) + if (root === undefined) return [] + const byParent = new Map() + for (const entry of entries) { + const children = byParent.get(entry.parentPid) ?? [] + children.push(entry) + byParent.set(entry.parentPid, children) + } + const visited = new Set() + const result: ProcessIdentity[] = [] + const visit = (entry: ProcessEntry): void => { + if (visited.has(entry.pid)) return + visited.add(entry.pid) + for (const child of byParent.get(entry.pid) ?? []) visit(child) + const identity = started(entry.pid) + if (identity !== undefined) result.push({ pid: entry.pid, started: identity }) + } + visit(root) + return result +} + +/** + * Windows {@link ProcessInspector}. The shell pid stands in for a foreground + * process group: it is a stable pseudo-group that lets the prompt-marker + * readiness path compare foreground identities, while every actual signal + * targets the console-wide tree through taskkill (SIGINT is delivered by the + * terminal handle as a `\x03` input write and never reaches this layer). + */ +export class WindowsProcessInspector implements ProcessInspector { + constructor( + private readonly internals: WindowsProcessInspectorInternals = defaultWindowsProcessInternals(), + ) {} + + foregroundPgid(shellPid: number): number { + return shellPid + } + + isStdinWaiting(_pgid: number): boolean { + return false + } + + processTree(rootPid: number): ProcessIdentity[] { + return windowsProcessTree(this.internals.snapshot(), rootPid, pid => this.internals.processState(pid)?.started) + } + + processSession(_sessionId: number): ProcessIdentity[] { + return [] + } + + isAlive(identity: ProcessIdentity): boolean { + const state = this.internals.processState(identity.pid) + return state?.active === true && state.started === identity.started + } + + signalGroup(pgid: number, signal: SubprocessTerminalSignal): void { + this.internals.taskkill(pgid, signal === 'SIGKILL') + } + + signalProcess(identity: ProcessIdentity, signal: 'SIGTERM' | 'SIGKILL'): void { + if (this.isAlive(identity)) this.internals.taskkill(identity.pid, signal === 'SIGKILL') + } +} +/* jscpd:ignore-end */ + +/** + * Create the Windows process inspector. + * @param internals - injectable process operations; defaults to the koffi-backed table. + * @returns the Windows inspector. + */ +export function createWindowsProcessInspector( + internals: WindowsProcessInspectorInternals = defaultWindowsProcessInternals(), +): WindowsProcessInspector { + return new WindowsProcessInspector(internals) +} + +/** Terminate one Windows process tree with taskkill, contained like POSIX group signalling. */ +function taskkillTree(pid: number, force: boolean): void { + if (pid <= 0) return + // Outcome deliberately unchecked: an already-absent tree, exit races, and a + // missing taskkill binary are as tolerable here as ESRCH is for POSIX. + spawnSync('taskkill', ['/PID', String(pid), '/T', ...(force ? ['/F'] : [])], { stdio: 'ignore' }) +} + +declare const nativePtr: unique symbol +/** Koffi 3 native pointer (a BigInt address), branded so it cannot silently enter numeric contexts. */ +export type NativePtr = bigint & { readonly [nativePtr]: true } + +/** + * True for NULL and INVALID_HANDLE_VALUE returns from Win32 handle APIs. + * @param value - a handle as koffi may hand it back (pointer, null, or 0n). + * @returns whether the value signals an invalid handle. + */ +export function isInvalidHandle(value: NativePtr | null | undefined): boolean { + if (value === null || value === undefined) return true + const asBigInt = value as bigint + return asBigInt === 0n || asBigInt === 0xFFFFFFFFFFFFFFFFn || asBigInt === -1n +} + +/** The lazy koffi binding table: every Win32 call the Windows inspector uses. */ +interface Win32Bindings { + createToolhelp32Snapshot(flags: number, processId: number): NativePtr + process32FirstW(snapshot: NativePtr, entry: NativePtr): number + process32NextW(snapshot: NativePtr, entry: NativePtr): number + openProcess(desiredAccess: number, inheritHandle: number, pid: number): NativePtr + getProcessTimes( + process: NativePtr, + creation: NativePtr, + exit: NativePtr, + kernel: NativePtr, + user: NativePtr, + ): number + waitForSingleObject(handle: NativePtr, milliseconds: number): number + closeHandle(handle: NativePtr): number +} + +const PVOID: ReturnType = koffi.pointer('void') + +/** + * Resolve the koffi Win32 struct types once. Registration is lazy and cached + * because koffi's type registry is global per process: test runners that + * re-evaluate this module (a hoisted `vi.mock` re-imports the graph) must not + * re-register the names. + */ +function win32Structs(): { PROCESSENTRY32W: ReturnType; FILETIME: ReturnType } { + if (cachedStructs !== undefined) return cachedStructs + // koffi PROCESSENTRY32W layout (tlhelp32.h); the size assert pins the x64 layout. + const PROCESSENTRY32W = koffi.struct('PROCESSENTRY32W', { + dwSize: 'uint32', + cntUsage: 'uint32', + th32ProcessID: 'uint32', + th32DefaultHeapID: PVOID, + th32ModuleID: 'uint32', + cCntThreads: 'uint32', + th32ParentProcessID: 'uint32', + pcPriClassBase: 'int32', + dwFlags: 'uint32', + szExeFile: koffi.array('char16', 260), + }) + // koffi FILETIME layout (minwinbase.h): two 32-bit halves of the 64-bit timestamp. + const FILETIME = koffi.struct('FILETIME', { + dwLowDateTime: 'uint32', + dwHighDateTime: 'uint32', + }) + /* v8 ignore start -- a layout-mismatch guard fires only on ABI breakage; the windows-native suites exercise the real struct. */ + if (PROCESSENTRY32W.size !== 568) { + throw new Error(`PROCESSENTRY32W layout mismatch: koffi computed ${PROCESSENTRY32W.size}, Windows headers say 568`) + } + /* v8 ignore stop */ + cachedStructs = { PROCESSENTRY32W, FILETIME } + return cachedStructs +} + +let cachedStructs: ReturnType | undefined + +const TH32CS_SNAPPROCESS = 0x2 +const PROCESS_QUERY_LIMITED_INFORMATION = 0x1000 +const SYNCHRONIZE = 0x00100000 +const WAIT_OBJECT_0 = 0 +const WAIT_TIMEOUT = 0x102 + +let cachedBindings: Win32Bindings | undefined + +/** + * Resolve the lazy Win32 bindings (throws the first binding failure, fail-closed). + * @returns the cached binding table. + */ +function win32Bindings(): Win32Bindings { + if (cachedBindings !== undefined) return cachedBindings + const { PROCESSENTRY32W, FILETIME } = win32Structs() + const kernel32 = koffi.load('kernel32.dll') + const bind = ( + name: string, + result: ReturnType | string, + args: Array | string>, + ): unknown => kernel32.func('__stdcall', name, result, args) + cachedBindings = { + createToolhelp32Snapshot: bind('CreateToolhelp32Snapshot', PVOID, ['uint32', 'uint32']), + process32FirstW: bind('Process32FirstW', 'int', [PVOID, koffi.pointer(PROCESSENTRY32W)]), + process32NextW: bind('Process32NextW', 'int', [PVOID, koffi.pointer(PROCESSENTRY32W)]), + openProcess: bind('OpenProcess', PVOID, ['uint32', 'int', 'uint32']), + getProcessTimes: bind('GetProcessTimes', 'int', [ + PVOID, + koffi.pointer(FILETIME), + koffi.pointer(FILETIME), + koffi.pointer(FILETIME), + koffi.pointer(FILETIME), + ]), + waitForSingleObject: bind('WaitForSingleObject', 'uint32', [PVOID, 'uint32']), + closeHandle: bind('CloseHandle', 'int', [PVOID]), + } as unknown as Win32Bindings + return cachedBindings +} + +/** + * Allocate koffi memory as a branded {@link NativePtr}; koffi's TS types are + * `any`, so the cast goes through `unknown` to keep the unsafe surface here. + * @param type - the koffi type to allocate. + * @param count - element count. + * @returns the branded allocation pointer. + */ +function allocNative(type: Parameters[0], count: number): NativePtr { + const value: unknown = koffi.alloc(type, count) + return value as NativePtr +} + +/** Enumerate the current process table through Toolhelp32. */ +function snapshotWindowsProcesses(bindings: Win32Bindings): ProcessEntry[] { + const { PROCESSENTRY32W } = win32Structs() + const snapshot = bindings.createToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0) + /* v8 ignore next -- an invalid snapshot for the process flag is not producible through the public API; + the guard mirrors POSIX's unreadable-proc tolerance and isInvalidHandle is unit-tested. */ + if (isInvalidHandle(snapshot)) return [] + const entries: ProcessEntry[] = [] + try { + const entry = allocNative(PROCESSENTRY32W, 1) + koffi.encode(entry, 'uint32', PROCESSENTRY32W.size) + let ok = bindings.process32FirstW(snapshot, entry) + while (ok !== 0) { + const record = koffi.decode(entry, PROCESSENTRY32W) as { + th32ProcessID: number + th32ParentProcessID: number + } + entries.push({ pid: record.th32ProcessID, parentPid: record.th32ParentProcessID }) + ok = bindings.process32NextW(snapshot, entry) + } + } finally { + bindings.closeHandle(snapshot) + } + return entries +} + +/** Read one process's creation identity and current wait state. */ +function windowsProcessState(bindings: Win32Bindings, pid: number): WindowsProcessState | undefined { + const { FILETIME } = win32Structs() + const handle = bindings.openProcess(PROCESS_QUERY_LIMITED_INFORMATION | SYNCHRONIZE, 0, pid) + if (isInvalidHandle(handle)) return undefined + try { + const creation = allocNative(FILETIME, 1) + const exit = allocNative(FILETIME, 1) + const kernel = allocNative(FILETIME, 1) + const user = allocNative(FILETIME, 1) + /* v8 ignore next -- a GetProcessTimes failure after a successful open races process exit and + cannot be staged deterministically; the absent-process path is covered and the caller + treats undefined as a detector miss. */ + if (bindings.getProcessTimes(handle, creation, exit, kernel, user) === 0) return undefined + const record = koffi.decode(creation, FILETIME) as { dwLowDateTime: number; dwHighDateTime: number } + const wait = bindings.waitForSingleObject(handle, 0) + /* v8 ignore next -- an opened process handle has exactly one of these two + zero-time wait states; an unexpected Win32 failure is an unreadable process. */ + if (wait !== WAIT_OBJECT_0 && wait !== WAIT_TIMEOUT) return undefined + return { + started: `${record.dwHighDateTime}:${record.dwLowDateTime}`, + active: wait === WAIT_TIMEOUT, + } + } finally { + bindings.closeHandle(handle) + } +} + +/** The koffi-backed default internals; bindings resolve lazily on first use. */ +function defaultWindowsProcessInternals(): WindowsProcessInspectorInternals { + return { + snapshot: () => snapshotWindowsProcesses(win32Bindings()), + processState: pid => windowsProcessState(win32Bindings(), pid), + taskkill: taskkillTree, + } +} diff --git a/packages/subprocess/subprocess-local/tests/fixtures/managed-tree.ts b/packages/subprocess/subprocess-local/tests/fixtures/managed-tree.ts index 31d26b9e39..a949e5fbfa 100644 --- a/packages/subprocess/subprocess-local/tests/fixtures/managed-tree.ts +++ b/packages/subprocess/subprocess-local/tests/fixtures/managed-tree.ts @@ -1,5 +1,5 @@ import { spawn } from 'node:child_process' -import { writeFile } from 'node:fs/promises' +import { rename, writeFile } from 'node:fs/promises' const [statePath] = process.argv.slice(2) if (statePath === undefined) throw new Error('usage: managed-tree.ts ') @@ -12,5 +12,7 @@ const descendant = spawn(process.execPath, [ ], { stdio: 'ignore' }) if (descendant.pid === undefined) throw new Error('managed descendant did not publish a pid') -await writeFile(statePath, JSON.stringify({ root: process.pid, descendant: descendant.pid })) +const pendingStatePath = `${statePath}.pending-${process.pid}` +await writeFile(pendingStatePath, JSON.stringify({ root: process.pid, descendant: descendant.pid })) +await rename(pendingStatePath, statePath) setInterval(() => {}, 60_000) diff --git a/packages/subprocess/subprocess-local/tests/fixtures/process-exit-host.ts b/packages/subprocess/subprocess-local/tests/fixtures/process-exit-host.ts index e59289be09..721c5134c3 100644 --- a/packages/subprocess/subprocess-local/tests/fixtures/process-exit-host.ts +++ b/packages/subprocess/subprocess-local/tests/fixtures/process-exit-host.ts @@ -13,7 +13,6 @@ if ((kind !== 'ordinary' && kind !== 'terminal') } const treeState = join(root, 'tree.json') -const ready = join(root, 'ready') const proceed = join(root, 'proceed') const managedTree = fileURLToPath(new URL('./managed-tree.ts', import.meta.url)) @@ -58,7 +57,6 @@ const published = JSON.parse(await readFile(treeState, 'utf8')) as { root?: unkn if (!Number.isSafeInteger(published.root) || !Number.isSafeInteger(published.descendant)) { throw new Error('managed tree published invalid process ids') } -await writeFile(ready, 'ready') await waitForFile(proceed) if (trigger === 'dispose') { diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index c41d4025d6..41e6b48bc8 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -7,8 +7,16 @@ import type { SubprocessSpawnSpec, SubprocessTerminalHandle, SubprocessTerminalS import { childEnv } from '../src/spawn.ts' function spec(command: string, overrides: Partial = {}): SubprocessSpawnSpec { + // Windows has no bash; the suite's simple commands translate to node one-liners. + const argv = process.platform === 'win32' + ? [process.execPath, '-e', { + 'echo managed': 'console.log("managed")', + 'sleep 60': 'setTimeout(() => {}, 60000)', + 'true': '', + }[command] ?? command] + : ['bash', '-c', command] return { - argv: ['bash', '-c', command], + argv, cwd: process.cwd(), stdio: { stdin: 'ignore', @@ -148,9 +156,9 @@ describe('LocalSubprocessRuntime', () => { const explicit = childEnv({ Path: '/bin', PathExt: '.EXE;.CMD' }) expect(Object.keys(explicit).filter(key => key.toUpperCase() === 'PATH')).toEqual(['Path']) expect(Object.keys(explicit).filter(key => key.toUpperCase() === 'PATHEXT')).toEqual(['PathExt']) - expect(candidates('tool', explicit)).toEqual(['/bin/tool.EXE', '/bin/tool.CMD']) + expect(candidates('tool', explicit)).toEqual([resolve('/bin', 'tool.EXE'), resolve('/bin', 'tool.CMD')]) expect(candidates('tool', { Path: '/ambient', PATH: '/explicit', PATHEXT: '.EXE' })) - .toEqual(['/explicit/tool.EXE']) + .toEqual([resolve('/explicit', 'tool.EXE')]) expect(candidates('tool.exe', {})).toEqual([resolve(process.cwd(), 'tool.exe')]) expect(candidates('tool', { PATH: '/bin' })).toHaveLength(4) await expect(ctx.subprocess.resolveExecutable(String.raw`bin\server.exe`)) @@ -398,7 +406,8 @@ describe('LocalSubprocessRuntime', () => { const handle = ctx.subprocess.spawn(spec('sleep 60')) await fiber.dispose() const outcome = await handle.done - expect(outcome.signal).toBe('SIGTERM') + // Windows teardown terminates through taskkill, which reports no signal. + expect(outcome.signal).toBe(process.platform === 'win32' ? null : 'SIGTERM') }) it('a settled process leaves the live set (disposal does not re-kill it)', async () => { diff --git a/packages/subprocess/subprocess-local/tests/process-exit.spec.ts b/packages/subprocess/subprocess-local/tests/process-exit.spec.ts index 217338fa1e..cfea99f12a 100644 --- a/packages/subprocess/subprocess-local/tests/process-exit.spec.ts +++ b/packages/subprocess/subprocess-local/tests/process-exit.spec.ts @@ -106,11 +106,9 @@ async function runScenario(kind: ManagedKind, trigger: ExitTrigger) { let settled = false let treeGone = false try { + // The host validates tree.json before waiting for proceed, so observing it + // is sufficient readiness; a second marker only adds a redundant Windows poll. state = await readTree(join(root, 'tree.json')) - await vi.waitFor(() => readFile(join(root, 'ready'), 'utf8'), { - interval: 10, - timeout: scenarioTimeoutMs, - }) if (process.platform !== 'win32') identities = await captureIdentities(createProcessInspector(), state) await writeFile(join(root, 'proceed'), 'proceed') const outcome = await child diff --git a/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts index c90a7b3490..aadf2e1388 100644 --- a/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts +++ b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts @@ -5,6 +5,7 @@ import { parseProcStat, } from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' import type { ProcessInspectorInternals } from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' +import { WindowsProcessInspector } from '@deepseek-ai/dsh-subprocess-local/src/windows-inspector.ts' function stat(pid: number, pgrp: number, session: number, tpgid: number, started: string, parentPid = 1, state = 'S'): string { const rest = [state, String(parentPid), String(pgrp), String(session), '99', String(tpgid)] @@ -237,12 +238,13 @@ describe('macOS process inspector', () => { ]) }) - it('returns undefined for missing or invalid foreground groups and rejects unsupported platforms', () => { + it('returns undefined for missing or invalid foreground groups and dispatches platform inspectors', () => { const fake = fakeInternals() fake.setTpgid('-1') expect(createProcessInspector('darwin', 'arm64', fake.internals).foregroundPgid(1)).toBeUndefined() fake.internals.exec = () => { throw new Error('gone') } expect(createProcessInspector('darwin', 'arm64', fake.internals).foregroundPgid(1)).toBeUndefined() - expect(() => createProcessInspector('win32', 'x64', fake.internals)).toThrow('unsupported on platform win32') + expect(createProcessInspector('win32', 'x64', fake.internals)).toBeInstanceOf(WindowsProcessInspector) + expect(() => createProcessInspector('freebsd', 'x64', fake.internals)).toThrow('unsupported on platform freebsd') }) }) diff --git a/packages/subprocess/subprocess-local/tests/spawn.spec.ts b/packages/subprocess/subprocess-local/tests/spawn.spec.ts index 1f87fd67cc..1f21dfaaa1 100644 --- a/packages/subprocess/subprocess-local/tests/spawn.spec.ts +++ b/packages/subprocess/subprocess-local/tests/spawn.spec.ts @@ -3,6 +3,7 @@ import { tmpdir } from 'node:os' import { dirname, join } from 'node:path' import { describe, expect, it, vi } from 'vitest' import { + childEnv, killGroup, OutputCollector, spawnSubprocess, @@ -11,6 +12,49 @@ import { import type { SubprocessHandle, SubprocessOutputReader } from '@deepseek-ai/dsh-subprocess' import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' +/** + * Translate the suite's POSIX command strings into node one-liners on Windows, + * where no bash exists; the translated commands keep the same observable + * stdout/stderr/exit-code contract the bash originals pin on POSIX. + * @param command - the bash `-c` command string used by the test. + * @returns the argv to spawn. + */ +function shellArgv(command: string): string[] { + if (process.platform !== 'win32') return ['bash', '-c', command] + const node = (script: string): string[] => [process.execPath, '-e', script] + switch (command) { + case 'true': return node('') + case 'echo hello': return node('console.log("hello")') + case 'echo hi': return node('console.log("hi")') + case 'echo oops >&2': return node('console.error("oops")') + case 'echo err >&2': return node('console.error("err")') + case 'echo out; echo err >&2': return node('console.log("out"); console.error("err")') + case 'echo out; echo to-parent >&2': return node('console.log("out"); console.error("to-parent")') + case 'echo to-parent; echo err >&2': return node('console.log("to-parent"); console.error("err")') + case 'exit 42': return node('process.exit(42)') + case 'exit 7': return node('process.exit(7)') + case 'pwd': return node('console.log(process.cwd())') + case 'sleep 60': return node('setTimeout(() => {}, 60000)') + case 'cat': return node('process.stdin.pipe(process.stdout)') + case 'unused': return node('') + case 'echo "${TERM:-unset}"': return node('console.log(process.env.TERM ?? "unset")') + case 'echo "$EXTRA_ONE/$EXTRA_TWO"': return node('console.log(process.env.EXTRA_ONE + "/" + process.env.EXTRA_TWO)') + case 'echo "$EXPLICIT_OVERRIDE_PASSWORD"': return node('console.log(process.env.EXPLICIT_OVERRIDE_PASSWORD)') + case 'echo "${SUBPROCESS_TOMBSTONE_PROBE:-absent}"': return node('console.log(process.env.SUBPROCESS_TOMBSTONE_PROBE ?? "absent")') + case 'echo "[${DSH_STALE:-absent}|$DSH_SHELL|$DSH_SESSION_ID]"': + return node('console.log("[" + [process.env.DSH_STALE ?? "absent", process.env.DSH_SHELL, process.env.DSH_SESSION_ID].join("|") + "]")') + case 'echo "[${DSH_TEST_API_KEY:-absent}|${DSH_TEST_TOKEN:-absent}|${SUBPROCESS_TEST_PASSWORD:-absent}|${DSH_TEST_PLAIN:-absent}]"': + return node('console.log("[" + [process.env.DSH_TEST_API_KEY ?? "absent", process.env.DSH_TEST_TOKEN ?? "absent", process.env.SUBPROCESS_TEST_PASSWORD ?? "absent", process.env.DSH_TEST_PLAIN ?? "absent"].join("|") + "]")') + case 'printf "%.0sx" $(seq 1 500)': return node('process.stdout.write("x".repeat(500))') + case 'printf "%.0sx" $(seq 1 500); printf "%.0se" $(seq 1 500) >&2': + return node('process.stdout.write("x".repeat(500)); process.stderr.write("e".repeat(500))') + case 'for i in $(seq 1 200); do printf "line-%04d\\n" $i; done': + return node('for (let i = 1; i <= 200; i++) console.log("line-" + String(i).padStart(4, "0"))') + default: + throw new Error(`spawn.spec: no win32 node translation for ${JSON.stringify(command)}`) + } +} + const { failNextClose, failNextUnlink } = vi.hoisted(() => ({ failNextClose: { value: false }, failNextUnlink: { value: false }, @@ -48,7 +92,7 @@ type SpecOverrides = Partial[0]> & { function spec(command: string, overrides: SpecOverrides = {}) { const { stdoutMaxBytes = 64_000, stderrMaxBytes = 64_000, maxSpillBytes = 64 * 1024 * 1024, stdin, ...rest } = overrides return { - argv: ['bash', '-c', command], + argv: shellArgv(command), cwd: process.cwd(), stdio: { stdin: stdin !== undefined ? { data: stdin } : 'ignore' as const, @@ -161,7 +205,7 @@ describe('spawnSubprocess', () => { expect(result.stdout.text).toBe('callers-choice\n') }) - it('runs in the requested cwd', async () => { + it.skipIf(process.platform === 'win32')('runs in the requested cwd', async () => { const result = await finish(spawnSubprocess(spec('pwd', { cwd: '/tmp' }))) expect(result.stdout.text.trim()).toMatch(/\/tmp$/) }) @@ -176,11 +220,12 @@ describe('spawnSubprocess', () => { setTimeout(() => { controller.abort('deadline') }, 100) const result = await running.done expect(Date.now() - start).toBeLessThan(5_000) - expect(result.signal).toBe('SIGTERM') - expect(result.exitCode).toBeNull() + // Windows teardown terminates through taskkill, which reports no signal. + expect(result.signal).toBe(process.platform === 'win32' ? null : 'SIGTERM') + expect(result.exitCode).toBe(process.platform === 'win32' ? 1 : null) }) - it('terminate() escalates to SIGKILL when SIGTERM is trapped', async () => { + it.skipIf(process.platform === 'win32')('terminate() escalates to SIGKILL when SIGTERM is trapped', async () => { const running = spawnSubprocess(spec('trap \'\' TERM; echo ready; while :; do sleep 60 & wait $!; done', { graceMs: 200 })) await waitForStdout(running, 'ready\n') running.terminate() @@ -231,12 +276,16 @@ describe('spawnSubprocess', () => { expect(forceSignals).toBe(0) } finally { killSpy.mockRestore() - process.kill(helper, 'SIGKILL') + try { + process.kill(helper, 'SIGKILL') + } catch { + // taskkill already took the helper down on Windows. + } await waitGone(helper) } }) - it('terminates the whole process group (grandchildren die too)', async () => { + it.skipIf(process.platform === 'win32')('terminates the whole process group (grandchildren die too)', async () => { // The subshell writes the sleep's pid then waits on it; terminating the // group must take the sleep down with bash. const pidFile = join(spillDir, `grandchild-${Date.now()}.pid`) @@ -255,7 +304,7 @@ describe('spawnSubprocess', () => { const running = spawnSubprocess(spec('sleep 60', { signal: controller.signal })) setTimeout(() => { controller.abort('user cancelled') }, 50) const result = await running.done - expect(result.signal).toBe('SIGTERM') + expect(result.signal).toBe(process.platform === 'win32' ? null : 'SIGTERM') }) it('throws when the signal is already aborted before spawn', () => { @@ -275,10 +324,10 @@ describe('spawnSubprocess', () => { running.terminate() running.terminate() const result = await running.done - expect(result.signal).toBe('SIGTERM') + expect(result.signal).toBe(process.platform === 'win32' ? null : 'SIGTERM') }) - it('does not wait for a Linux group that has only zombie members', async () => { + it.skipIf(process.platform === 'win32')('does not wait for a Linux group that has only zombie members', async () => { const pidFile = join(spillDir, `zombie-group-${Date.now()}.pid`) const running = spawnSubprocess(spec(`sleep 60 & echo $! > ${pidFile}; echo leader-done`, { graceMs: 100 }), { platform: 'linux', @@ -296,7 +345,7 @@ describe('spawnSubprocess', () => { } }) - it('bounds inherited-pipe draining after the shell exits', async () => { + it.skipIf(process.platform === 'win32')('bounds inherited-pipe draining after the shell exits', async () => { const pidFile = join(spillDir, `pipe-holder-${Date.now()}.pid`) const started = Date.now() const running = spawnSubprocess(spec(`sleep 60 & echo $! > ${pidFile}; echo shell-done`, { graceMs: 100 })) @@ -328,7 +377,7 @@ describe('stdin and extra env (set by in-process plugins)', () => { expect(result.stdout.text).toBe('') }) - it('gives fd 0 the exact pre-seam type: /dev/null when no stdin, a pipe when supplied', async () => { + it.skipIf(process.platform === 'win32')('gives fd 0 the exact pre-seam type: /dev/null when no stdin, a pipe when supplied', async () => { // With no bytes, fd 0 remains the pre-spawn `ignore` default (/dev/null, a character device). // Supplied bytes use Node's spawn pipe, which is an AF_UNIX socket rather than a FIFO. const none = await finish(spawnSubprocess(spec('test -c /dev/stdin && echo char || echo other'))) @@ -619,7 +668,7 @@ describe('windows tree semantics (injected platform)', () => { running.terminate() const outcome = await running.done expect(killed).toContain(running.pid) - expect(outcome.signal).toBe('SIGKILL') + expect(outcome.signal).toBe(process.platform === 'win32' ? null : 'SIGKILL') }) it('waitForExit falls back to direct-child liveness where groups do not exist', async () => { @@ -630,7 +679,7 @@ describe('windows tree semantics (injected platform)', () => { }) describe('waitForExit', () => { - it('waits for the whole detached tree, not just the shell', async () => { + it.skipIf(process.platform === 'win32')('waits for the whole detached tree, not just the shell', async () => { const pidFile = join(spillDir, `tree-wait-${Date.now()}.pid`) const running = spawnSubprocess(spec(`sleep 60 & echo $! > ${pidFile}; wait`)) const grandchild = await waitForPidFile(pidFile) @@ -650,7 +699,7 @@ describe('waitForExit', () => { }) }) -describe('synchronous host-exit termination', () => { +describe.skipIf(process.platform === 'win32')('synchronous host-exit termination', () => { it('force-kills the current process tree without waiting for the normal grace', async () => { const running = spawnSubprocess(spec('trap "" TERM; sleep 60', { graceMs: 60_000 })) running.terminateForHostExit() @@ -667,7 +716,7 @@ describe('synchronous host-exit termination', () => { }) }) -describe('tree-survivor escalation (terminate and bounded waits reach helpers the leader left behind)', () => { +describe.skipIf(process.platform === 'win32')('tree-survivor escalation (terminate and bounded waits reach helpers the leader left behind)', () => { it('terminate() SIGKILLs a TERM-trapping descendant after the direct child settles', async () => { // The leader spawns a TERM-trapping helper with all stdio detached from // the collected pipes, then exits: the helper holds the GROUP alive while @@ -733,6 +782,97 @@ describe('coverage seams', () => { expect(() => { taskkillProcessTree(2 ** 30) }).not.toThrow() }) + it('covers the injected POSIX group paths on any host', async () => { + // Windows has no POSIX groups, so the tree-liveness probe, group + // signalling, and the SIGKILL escalation timer only run here through the + // injected platform; the mock keeps the group alive through TERM and + // terminates the direct child when the escalation tier delivers SIGKILL. + const running = spawnSubprocess(spec('sleep 60', { graceMs: 100 }), { + platform: 'linux', + linuxProcessGroupHasLiveMembers: () => false, + }) + const realKill = process.kill.bind(process) + const killSpy = vi.spyOn(process, 'kill').mockImplementation((target, signal) => { + if (typeof target === 'number' && target < 0) { + if (signal === 0) return true + if (signal === 'SIGKILL') realKill(running.pid, 'SIGKILL') + return true + } + return realKill(target, signal) + }) + try { + running.terminate() + await running.done + await expect(running.waitForExit()).resolves.toBe(true) + } finally { + killSpy.mockRestore() + } + }) + + it('treats a vanished group probe as quiescent without signalling', async () => { + const running = spawnSubprocess(spec('sleep 60'), { platform: 'linux' }) + const realKill = process.kill.bind(process) + const killSpy = vi.spyOn(process, 'kill').mockImplementation((target, signal) => { + if (typeof target === 'number' && target < 0) { + throw Object.assign(new Error('simulated absent group'), { code: 'ESRCH' }) + } + return realKill(target, signal) + }) + try { + running.terminate() + await new Promise(resolve => setTimeout(resolve, 20)) + realKill(running.pid, 'SIGKILL') + await running.done + await expect(running.waitForExit()).resolves.toBe(true) + } finally { + killSpy.mockRestore() + } + }) + + it('childEnv keeps the POSIX spread on non-Windows hosts', () => { + const platform = vi.spyOn(process, 'platform', 'get').mockReturnValue('linux') + try { + expect(childEnv({ DSH_X: '1' }).DSH_X).toBe('1') + } finally { + platform.mockRestore() + } + }) + + it('settles through the pipe-drain timer when a descendant holds a collected pipe', async () => { + // The leader spawns a detached grandchild inheriting the collected stdout + // pipe, then exits: `close` cannot settle while the grandchild holds the + // pipe, so the bounded pipe-drain timer must settle the outcome. + const pidFile = join(spillDir, `pipe-drain-${Date.now()}.pid`) + const childScript = ` + const { spawn } = require('node:child_process') + const { writeFileSync } = require('node:fs') + const helper = spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { + detached: true, + stdio: ['ignore', 1, 2], + }) + writeFileSync(${JSON.stringify(pidFile)}, String(helper.pid)) + helper.unref() + ` + const running = spawnSubprocess({ + ...spec('unused', { graceMs: 100 }), + argv: [process.execPath, '-e', childScript], + }) + // The drain timer starts when the child's stdio closes, which can precede + // the pid file becoming visible; measure from before that wait so the + // lower bound cannot be eroded by the pid-file handoff. + const started = Date.now() + const helper = await waitForPidFile(pidFile) + const outcome = await running.done + expect(outcome.exitCode).toBe(0) + expect(Date.now() - started).toBeGreaterThanOrEqual(90) + try { + process.kill(helper, 'SIGKILL') + } catch { + // Already gone; the drain bound is the point under test. + } + await waitGone(helper) + }) + it('a spawn-failed handle rejects done while waitForExit reports gone', async () => { const running = spawnSubprocess(spec('true', { cwd: '/nonexistent-dir-dsh-dispose-test' })) await expect(running.done).rejects.toThrow() @@ -869,7 +1009,7 @@ describe('argv validation', () => { expect(() => spawnSubprocess({ ...spec('true'), argv: [''] })).toThrow(/non-empty program name/) }) - it('spawns argv verbatim without shell interpretation', async () => { + it.skipIf(process.platform === 'win32')('spawns argv verbatim without shell interpretation', async () => { const result = await finish(spawnSubprocess({ ...spec('unused'), argv: ['printf', '%s', '$HOME'] })) expect(result.stdout.text).toBe('$HOME') }) @@ -889,7 +1029,7 @@ describe('abort edge cases', () => { .toThrow(/aborted before spawn: aborted/) }) - it('reports the terminating signal of an externally self-killed command', async () => { + it.skipIf(process.platform === 'win32')('reports the terminating signal of an externally self-killed command', async () => { // spawnSubprocess reports the raw signal; whether it counts as timeout/cancel is the // executor's classification (a self-kill is neither) — see executor.spec.ts. const result = await finish(spawnSubprocess(spec('kill -TERM $$'))) @@ -930,7 +1070,7 @@ describe('environment and spill-file hardening', () => { } }) - it('creates spill files with owner-only permissions and random names', async () => { + it.skipIf(process.platform === 'win32')('creates spill files with owner-only permissions and random names', async () => { const result = await finish(spawnSubprocess( spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { stdoutMaxBytes: 500, stderrMaxBytes: 500 }), { spillDir }, @@ -941,7 +1081,7 @@ describe('environment and spill-file hardening', () => { expect(mode).toBe(0o600) }) - it('defaults spills into a private per-process directory', async () => { + it.skipIf(process.platform === 'win32')('defaults spills into a private per-process directory', async () => { const result = await finish(spawnSubprocess( spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { stdoutMaxBytes: 500, stderrMaxBytes: 500 }), )) @@ -967,6 +1107,6 @@ describe('environment and spill-file hardening', () => { const running = spawnSubprocess(spec('sleep 60', { signal: controller.signal })) setTimeout(() => { controller.abort() }, 50) const result = await running.done - expect(result.signal).toBe('SIGTERM') + expect(result.signal).toBe(process.platform === 'win32' ? null : 'SIGTERM') }) }) diff --git a/packages/subprocess/subprocess-local/tests/terminal.spec.ts b/packages/subprocess/subprocess-local/tests/terminal.spec.ts index 4bfd9f1025..c2aca88085 100644 --- a/packages/subprocess/subprocess-local/tests/terminal.spec.ts +++ b/packages/subprocess/subprocess-local/tests/terminal.spec.ts @@ -73,6 +73,8 @@ class FakeInspector implements ProcessInspector { this.groups.push([pgid, signal]) } signalProcess(identity: ProcessIdentity, signal: 'SIGTERM' | 'SIGKILL') { + // Mirrors the real inspectors' alive-gated signalling. + if (!this.alive.has(identity.pid)) return if (this.throwProcess) throw new Error('process raced') if (!this.isAlive(identity)) return this.processes.push([identity.pid, signal]) @@ -82,6 +84,12 @@ class FakeInspector implements ProcessInspector { afterEach(() => { vi.useRealTimers() }) +function makeHandle(pty: FakePty, inspector: ProcessInspector, graceMs: number): LocalTerminalHandle { + // The suite pins POSIX signalling semantics deterministically on every host; + // the win32 branches get their own platform-explicit tests below. + return new LocalTerminalHandle(pty.asPty(), inspector, graceMs, 'linux') +} + describe('LocalTerminalHandle', () => { it('force-kills descendants around the shell during synchronous host exit', () => { const pty = new FakePty() @@ -166,7 +174,7 @@ describe('LocalTerminalHandle', () => { const pty = new FakePty() const inspector = new FakeInspector() inspector.waiting = true - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const handle = makeHandle(pty, inspector, 10) const chunks: Buffer[] = [] handle.output.on('data', (chunk: Buffer) => { chunks.push(chunk) }) @@ -187,7 +195,7 @@ describe('LocalTerminalHandle', () => { it('rejects unsafe foreground signals and writes after exit', async () => { const pty = new FakePty() const inspector = new FakeInspector() - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const handle = makeHandle(pty, inspector, 10) inspector.pgid = handle.pid await expect(handle.signalForeground('SIGKILL')).rejects.toThrow('terminate the terminal session') inspector.pgid = undefined @@ -207,7 +215,7 @@ describe('LocalTerminalHandle', () => { inspector.members = [{ pid: 124, started: 'child' }] inspector.alive.add(124) inspector.removeOnSignal = false - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + const handle = makeHandle(pty, inspector, 20) const quiescent = handle.terminate() expect(handle.terminate()).toBe(quiescent) @@ -228,7 +236,7 @@ describe('LocalTerminalHandle', () => { inspector.members = [{ pid: 124, started: 'child' }] inspector.alive.add(124) inspector.removeOnSignal = false - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + const handle = makeHandle(pty, inspector, 20) pty.emitExit() const waiting = handle.terminate() let settled = false @@ -247,7 +255,7 @@ describe('LocalTerminalHandle', () => { const disowned = { pid: 124, started: 'disowned' } inspector.processSession = () => inspector.alive.has(disowned.pid) ? [disowned] : [] inspector.alive.add(124) - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + const handle = makeHandle(pty, inspector, 20) pty.emitExit() @@ -261,7 +269,7 @@ describe('LocalTerminalHandle', () => { const descendant = { pid: 124, started: 'observed' } inspector.members = [descendant] inspector.alive.add(descendant.pid) - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + const handle = makeHandle(pty, inspector, 20) await handle.inspectForeground() inspector.members = [] @@ -274,7 +282,7 @@ describe('LocalTerminalHandle', () => { it('does not adopt the children of a recycled shell pid', async () => { const pty = new FakePty() const inspector = new FakeInspector() - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const handle = makeHandle(pty, inspector, 10) pty.emitExit() const imposterChild = { pid: 999, started: 'imposter-child' } @@ -293,7 +301,7 @@ describe('LocalTerminalHandle', () => { const orphan = { pid: 321, started: 'unverifiable' } inspector.members = [orphan] inspector.alive.add(orphan.pid) - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const handle = makeHandle(pty, inspector, 10) await handle.terminate() expect(inspector.processes).toEqual([]) @@ -318,7 +326,7 @@ describe('LocalTerminalHandle', () => { } return [] } - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const handle = makeHandle(pty, inspector, 10) await handle.terminate() expect(inspector.processes).toEqual([[124, 'SIGTERM'], [125, 'SIGKILL']]) expect(pty.kills).toEqual(['SIGTERM']) @@ -332,7 +340,7 @@ describe('LocalTerminalHandle', () => { inspector.sessionMembers = [late] inspector.alive.add(late.pid) } - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const handle = makeHandle(pty, inspector, 10) await handle.terminate() @@ -350,7 +358,7 @@ describe('LocalTerminalHandle', () => { inspector.sessionMembers = [late] inspector.alive.add(late.pid) } - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const handle = makeHandle(pty, inspector, 10) const first = handle.terminate() const failed = expect(first).rejects.toThrow('surviving pids: 124') @@ -377,7 +385,7 @@ describe('LocalTerminalHandle', () => { inspector.processes.push([identity.pid, signal]) if (signal === 'SIGKILL') inspector.alive.delete(identity.pid) } - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + const handle = makeHandle(pty, inspector, 20) const quiescent = handle.terminate() await vi.advanceTimersByTimeAsync(25) await quiescent @@ -388,7 +396,7 @@ describe('LocalTerminalHandle', () => { vi.useFakeTimers() const pty = new FakePty() pty.autoExitOnKill = false - const handle = new LocalTerminalHandle(pty.asPty(), new FakeInspector(), 10) + const handle = makeHandle(pty, new FakeInspector(), 10) const failed = expect(handle.terminate()).rejects.toThrow('surviving pid: 123') await vi.advanceTimersByTimeAsync(25) await failed @@ -406,7 +414,98 @@ describe('LocalTerminalHandle', () => { inspector.members = [{ pid: 124, started: 'child' }] inspector.alive.add(124) inspector.throwProcess = true - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 1) + const handle = makeHandle(pty, inspector, 1) await expect(handle.terminate()).rejects.toThrow('surviving pids: 124') }) }) + +describe('LocalTerminalHandle on Windows', () => { + const win32 = 'win32' as NodeJS.Platform + + it('delivers SIGINT as a Ctrl-C input write without inspector signalling', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10, win32) + await expect(handle.signalForeground('SIGINT')).resolves.toBe(456) + expect(pty.writes).toEqual(['\x03']) + expect(inspector.groups).toEqual([]) + }) + + it('rejects SIGTSTP and SIGHUP as unavailable on Windows', async () => { + const handle = new LocalTerminalHandle(new FakePty().asPty(), new FakeInspector(), 10, win32) + await expect(handle.signalForeground('SIGTSTP')).rejects.toThrow('unsupported on Windows') + await expect(handle.signalForeground('SIGHUP')).rejects.toThrow('unsupported on Windows') + }) + + it('routes SIGTERM through the inspector tree with the pseudo foreground group', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10, win32) + await expect(handle.signalForeground('SIGTERM')).resolves.toBe(456) + expect(inspector.groups).toEqual([[456, 'SIGTERM']]) + expect(pty.writes).toEqual([]) + }) + + it('still refuses to SIGKILL the terminal shell on Windows', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10, win32) + inspector.pgid = handle.pid + await expect(handle.signalForeground('SIGKILL')).rejects.toThrow('terminate the terminal session') + }) + + it('escalates the shell through taskkill tiers instead of node-pty signal kills', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.alive.add(123) + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10, win32) + const quiescent = handle.terminate() + await vi.advanceTimersByTimeAsync(5) + expect(inspector.processes).toEqual([[123, 'SIGTERM']]) + expect(pty.kills).toEqual([]) + + pty.emitExit() + await quiescent + expect(inspector.processes).toEqual([[123, 'SIGTERM']]) + expect(pty.kills).toEqual([]) + }) + + it('reports a shell that survives both taskkill tiers', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.alive.add(123) + inspector.removeOnSignal = false + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10, win32) + const failed = expect(handle.terminate()).rejects.toThrow('surviving pid: 123') + await vi.advanceTimersByTimeAsync(25) + await failed + expect(inspector.processes).toEqual([[123, 'SIGTERM'], [123, 'SIGKILL']]) + expect(pty.kills).toEqual([]) + + pty.emitExit() + await handle.terminate() + }) + + it('skips taskkill escalation entirely when the shell already exited', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.alive.add(123) + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10, win32) + pty.emitExit() + await handle.terminate() + expect(inspector.processes).toEqual([]) + expect(pty.kills).toEqual([]) + }) + + it('falls back to the bare node-pty kill when the shell identity was never observable', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.root = undefined + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10, win32) + await handle.terminate() + expect(pty.kills).toHaveLength(1) + expect(inspector.processes).toEqual([]) + }) +}) diff --git a/packages/subprocess/subprocess-local/tests/windows-inspector.spec.ts b/packages/subprocess/subprocess-local/tests/windows-inspector.spec.ts new file mode 100644 index 0000000000..e00bdeb9e2 --- /dev/null +++ b/packages/subprocess/subprocess-local/tests/windows-inspector.spec.ts @@ -0,0 +1,148 @@ +import { describe, expect, it } from 'vitest' +import { + createWindowsProcessInspector, + isInvalidHandle, + windowsProcessTree, + WindowsProcessInspector, +} from '@deepseek-ai/dsh-subprocess-local/src/windows-inspector.ts' +import type { + NativePtr, + ProcessEntry, + WindowsProcessInspectorInternals, + WindowsProcessState, +} from '@deepseek-ai/dsh-subprocess-local/src/windows-inspector.ts' + +function fakeInternals() { + const entries: ProcessEntry[] = [] + const states = new Map() + const kills: Array<[number, boolean]> = [] + return { + internals: { + snapshot: () => [...entries], + processState: pid => states.get(pid), + taskkill: (pid: number, force: boolean) => { kills.push([pid, force]) }, + } satisfies WindowsProcessInspectorInternals, + add(entry: ProcessEntry, started?: string, active = true): void { + entries.push(entry) + if (started !== undefined) states.set(entry.pid, { started, active }) + }, + kills, + } +} + +describe('windowsProcessTree', () => { + it('walks a table children-first with readable identities only', () => { + const started = (pid: number): string | undefined => pid === 12 ? undefined : `t${pid}` + expect(windowsProcessTree([ + { pid: 10, parentPid: 0 }, + { pid: 11, parentPid: 10 }, + { pid: 12, parentPid: 11 }, + { pid: 13, parentPid: 11 }, + { pid: 14, parentPid: 10 }, + ], 10, started)).toEqual([ + { pid: 13, started: 't13' }, + { pid: 11, started: 't11' }, + { pid: 14, started: 't14' }, + { pid: 10, started: 't10' }, + ]) + }) + + it('returns an empty walk for an absent root', () => { + expect(windowsProcessTree([{ pid: 10, parentPid: 0 }], 99, () => 't')).toEqual([]) + }) + + it('terminates on a parent cycle instead of recursing forever', () => { + const entries = [ + { pid: 10, parentPid: 11 }, + { pid: 11, parentPid: 10 }, + ] + expect(windowsProcessTree(entries, 10, () => 't')).toHaveLength(2) + }) +}) + +describe('WindowsProcessInspector (injected internals)', () => { + it('exposes the shell pid as the pseudo foreground group and never proves stdin waits', () => { + const fake = fakeInternals() + const inspector = new WindowsProcessInspector(fake.internals) + expect(inspector.foregroundPgid(77)).toBe(77) + expect(inspector.isStdinWaiting(77)).toBe(false) + expect(inspector.processSession(77)).toEqual([]) + }) + + it('delegates tree walks and identity checks to the internals', () => { + const fake = fakeInternals() + fake.add({ pid: 10, parentPid: 0 }, 't10') + fake.add({ pid: 11, parentPid: 10 }, 't11') + const inspector = new WindowsProcessInspector(fake.internals) + expect(inspector.processTree(10)).toEqual([ + { pid: 11, started: 't11' }, + { pid: 10, started: 't10' }, + ]) + expect(inspector.isAlive({ pid: 11, started: 't11' })).toBe(true) + expect(inspector.isAlive({ pid: 11, started: 'stale' })).toBe(false) + expect(inspector.isAlive({ pid: 99, started: 't99' })).toBe(false) + + fake.add({ pid: 12, parentPid: 10 }, 't12', false) + expect(inspector.isAlive({ pid: 12, started: 't12' })).toBe(false) + }) + + it('maps SIGKILL to a forced taskkill and other signals to the grace form', () => { + const fake = fakeInternals() + const inspector = new WindowsProcessInspector(fake.internals) + inspector.signalGroup(77, 'SIGKILL') + inspector.signalGroup(77, 'SIGTERM') + inspector.signalGroup(0, 'SIGKILL') + expect(fake.kills).toEqual([[77, true], [77, false], [0, true]]) + }) + + it('signals a process only while its start identity matches', () => { + const fake = fakeInternals() + fake.add({ pid: 10, parentPid: 0 }, 't10') + fake.add({ pid: 11, parentPid: 10 }, 't11', false) + const inspector = new WindowsProcessInspector(fake.internals) + inspector.signalProcess({ pid: 10, started: 't10' }, 'SIGKILL') + inspector.signalProcess({ pid: 11, started: 't11' }, 'SIGKILL') + inspector.signalProcess({ pid: 10, started: 'stale' }, 'SIGTERM') + expect(fake.kills).toEqual([[10, true]]) + }) + + it('accepts an injected internals factory through the creator', () => { + const fake = fakeInternals() + expect(createWindowsProcessInspector(fake.internals)).toBeInstanceOf(WindowsProcessInspector) + expect(createWindowsProcessInspector()).toBeInstanceOf(WindowsProcessInspector) + }) +}) + +describe('isInvalidHandle', () => { + it('rejects null, zero, and the all-ones INVALID_HANDLE_VALUE forms', () => { + const ptr = (value: bigint): NativePtr => value as NativePtr + expect(isInvalidHandle(null)).toBe(true) + expect(isInvalidHandle(undefined)).toBe(true) + expect(isInvalidHandle(ptr(0n))).toBe(true) + expect(isInvalidHandle(ptr(0xFFFFFFFFFFFFFFFFn))).toBe(true) + expect(isInvalidHandle(ptr(-1n))).toBe(true) + expect(isInvalidHandle(ptr(1234n))).toBe(false) + }) +}) + +const win32 = process.platform === 'win32' ? describe : describe.skip + +win32('WindowsProcessInspector over the real koffi bindings', () => { + it('walks the live process table from the test runner itself', () => { + const inspector = createWindowsProcessInspector() + const tree = inspector.processTree(process.pid) + const self = tree.find(member => member.pid === process.pid) + expect(self).toBeDefined() + expect(inspector.isAlive(self!)).toBe(true) + expect(inspector.foregroundPgid(process.pid)).toBe(process.pid) + }) + + it('reports unreadable identities for absent processes and no-ops tree signalling', () => { + const inspector = createWindowsProcessInspector() + expect(inspector.isAlive({ pid: 0x7FFFFFFF, started: 'absent' })).toBe(false) + expect(() => { inspector.signalGroup(0x7FFFFFFF, 'SIGKILL') }).not.toThrow() + expect(() => { inspector.signalGroup(0x7FFFFFFF, 'SIGTERM') }).not.toThrow() + expect(() => { inspector.signalGroup(0, 'SIGKILL') }).not.toThrow() + expect(() => { inspector.signalProcess({ pid: 0x7FFFFFFF, started: 'absent' }, 'SIGKILL') }).not.toThrow() + }) +}) diff --git a/packages/subprocess/subprocess/README.i18n.yaml b/packages/subprocess/subprocess/README.i18n.yaml index c53673c2cd..cf36674d4b 100644 --- a/packages/subprocess/subprocess/README.i18n.yaml +++ b/packages/subprocess/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess/README.md -README.md: 5e80bc7205c2f06b3528bbcfcb775941d2fa4545 -README.zh.md: 62a8b0704d73bc433888f841273505fc92748fa9 +README.md: 4d02faf53c38825dc3862df1326455e4d9677ea5 +README.zh.md: a325d0d30732c13488f373564ee87b1e02238ef2 diff --git a/packages/subprocess/subprocess/README.md b/packages/subprocess/subprocess/README.md index 5e80bc7205..4d02faf53c 100644 --- a/packages/subprocess/subprocess/README.md +++ b/packages/subprocess/subprocess/README.md @@ -19,7 +19,7 @@ See the [subprocess subsystem page](../../../docs/subsystems/subprocess.md) and ## Model Experience -Indirectly, through Consumers (today the bash executor family behind `dsh-tool-bash`), which own all model-facing rendering of process output and lifecycle. +Indirectly, through Consumers (the bash executor family behind `dsh-tool-bash`), which own all model-facing rendering of process output and lifecycle. #### KV Cache effect diff --git a/packages/subprocess/subprocess/README.zh.md b/packages/subprocess/subprocess/README.zh.md index 62a8b0704d..a325d0d307 100644 --- a/packages/subprocess/subprocess/README.zh.md +++ b/packages/subprocess/subprocess/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -子进程 seam(`ctx.subprocess`)是一个执行世界的进程部分。抽象的 `SubprocessRuntime` 公开可执行文件查找、普通受管 `spawn` 和一项终端进程原语;其词汇涵盖原始/收集式 stdio、进程与终端句柄、退出事实、进程树/会话清理,以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 +子进程 seam(`ctx.subprocess`)是一个执行世界的进程部分。抽象的 `SubprocessRuntime` 公开可执行文件查找、普通受管 `spawn` 和一项终端进程原语;其词汇涵盖原始/收集式 stdio、进程与终端句柄、退出事实、进程树/会话清理,以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.zh.md)。 ## 约定 @@ -15,11 +15,11 @@ - `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的环境清理定义:环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃,显式 `env` 在清除之后合并。本地的普通 spawn 与终端 spawn 都应用该定义;拥有自身 spawn 的 SDK 管理传输可直接导入它。 - 服务自身的 dispose(资源释放)会终止所有仍在运行的受管进程并等待其退出。 -参见[子进程子系统页面](../../../docs/subsystems/subprocess.md)与[seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +参见[子进程子系统页面](../../../docs/subsystems/subprocess.zh.md)与[seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md)。 ## 模型体验 -通过 Consumer 间接影响(目前是 `dsh-tool-bash` 背后的 bash 执行器家族);进程输出和生命周期的全部面向模型渲染均由 Consumer 负责。 +通过 Consumer 间接影响(即 `dsh-tool-bash` 背后的 bash 执行器家族);进程输出和生命周期的全部面向模型渲染均由 Consumer 负责。 #### KV Cache 影响 diff --git a/packages/subprocess/subprocess/package.json b/packages/subprocess/subprocess/package.json index 5d2d4fa09d..274ed88295 100644 --- a/packages/subprocess/subprocess/package.json +++ b/packages/subprocess/subprocess/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-subprocess", "description": "Subprocess seam (ctx.subprocess) for the DeepSeek Harness — managed process groups, bounded spill-backed output, and escalated kills behind one abstract service", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/subprocess/win32-process/README.i18n.yaml b/packages/subprocess/win32-process/README.i18n.yaml new file mode 100644 index 0000000000..d5743ba165 --- /dev/null +++ b/packages/subprocess/win32-process/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/subprocess/win32-process/README.md +README.md: 208a31741098c5c4d76d846b3a1345b24b6fc135 +README.zh.md: 3c403efa8cdf389539663e79d95b766fb8ba0fcf diff --git a/packages/subprocess/win32-process/README.md b/packages/subprocess/win32-process/README.md new file mode 100644 index 0000000000..208a317410 --- /dev/null +++ b/packages/subprocess/win32-process/README.md @@ -0,0 +1,50 @@ +# @deepseek-ai/dsh-win32-process + +English | [中文](README.zh.md) + +Low-level Win32 process library consumed by the Windows ACL sandbox. It owns the repository's one Koffi binding table for reusable restricted-process, stdio, and Job Object operations; it is not a Cordis service and does not choose sandbox policy or public child behavior. + +## Behavior + +- **One reusable ABI owner** — `abi.ts` owns the Win32 constants and x64 layout values consumed by the sandbox process paths. `ffi.ts` lazily loads `kernel32.dll` and `advapi32.dll`, verifies `STARTUPINFOW` and `PROCESS_INFORMATION`, exposes typed operations and error formatting, and lets sandbox policy bind its remaining APIs through the same loaded libraries. +- **Restricted-token creation** — `RestrictedProcessSpawnOptions` requires the sandbox's primary token and uses `CreateProcessAsUserW`. Piped and inherited-stdio paths share command-line quoting, cwd, the inherited environment block, checked return values, and handle cleanup. +- **Piped process primitive** — `spawnPipedProcess()` creates anonymous stdin/stdout/stderr pipes, closes stdin immediately, returns the two read ends, and leaves process waiting and pipe draining to the caller. Every partial failure closes the handles already owned by the operation, and every Koffi out-parameter or struct allocation is freed after its Win32 lifetime. +- **Inherited-stdio Job primitive** — `spawnInheritedJobProcess()` creates one kill-on-close Job, temporarily marks the current stdio handles inheritable, creates the restricted child suspended, assigns it to the Job, and then resumes its initial thread. Target code cannot run before Job assignment; controlled assignment or resume failures terminate the suspended child or close the assigned Job before releasing every owned handle. +- **Explicit settlement ownership** — `waitForProcessExit()` waits and closes the process handle. `drainPipe()` reuses one native count slot while draining, frees it, and closes the pipe read handle. The sandbox retains its existing scheduling, result composition, and caller-owned Job closure. + +The Windows ACL sandbox adds SID, DACL, grant, workspace, and public child policy above these primitives. + +## Header verification + +The process, stdio, and Job constants plus selected structure sizes and offsets are checked against the MinGW Windows headers by [`verify/abi-probe.cpp`](verify/abi-probe.cpp): + +```sh +g++ -std=c++20 -municode -O2 -o abi-probe.exe verify/abi-probe.cpp && ./abi-probe.exe +``` + +The Koffi `STARTUPINFOW` and `PROCESS_INFORMATION` definitions also assert their 64-bit sizes at module load. The probe remains the evidence for the other recorded offsets and constants. + +## Model Experience + +### Process primitives + +#### What the model sees + +Nothing directly. The package exposes `Win32ProcessBindings` and process primitives to the sandbox, which owns all model-visible tools, output, and diagnostics; this package contributes no prompt text or tool schema. + +#### Token effect + +None directly. Consumers decide whether process output enters a tool result or later model request. + +#### KV Cache effect + +The package contributes no stable request prefix, so it does not invalidate model KV caches. + +## Known Limitations and Deferred Work + +- **Windows-only native loading** — importing the generic types is portable, but resolving the binding table loads Windows DLLs and fails on other hosts. Cross-platform tests inject a binding table instead of loading native APIs. +- **No public process service** — the package intentionally does not wrap its primitives in Cordis or Node streams. A consumer must own its policy, async scheduling, output limits, cancellation, and final handle closure. +- **Inherited environment only** — process creation passes a null environment block. The sandbox establishes changes through `SetEnvironmentVariableW` first because passing an explicit block through Koffi makes `CreateProcessAsUserW` fail with `ERROR_INVALID_PARAMETER`. Other callers that need environment changes must establish them before invoking the primitive or use their own runner process. +- **Restricted-token consumer only** — ordinary `CreateProcessW`, exact `applicationName`, parent-stdio release, and whole-Job settlement are absent until an ordinary process consumer requires them. +- **Create-to-assignment interruption** — the target starts suspended and cannot execute before Job assignment, but an external termination of the runner in the narrow interval between process creation and assignment can leave the suspended target behind. The package does not claim atomic Job attachment. +- **Header evidence is architecture-specific** — the committed ABI probe and layout constants cover the repository's current 64-bit Windows targets. A new pointer width or incompatible Windows ABI requires updating the probe before support is claimed. diff --git a/packages/subprocess/win32-process/README.zh.md b/packages/subprocess/win32-process/README.zh.md new file mode 100644 index 0000000000..3c403efa8c --- /dev/null +++ b/packages/subprocess/win32-process/README.zh.md @@ -0,0 +1,52 @@ +# @deepseek-ai/dsh-win32-process + +[English](README.md) | 中文 + +供 Windows ACL 沙箱消费的底层 Win32 进程库。它唯一拥有仓库中可复用 restricted-process、stdio 与 Job Object 操作的 Koffi 绑定表;它不是 Cordis 服务,也不决定沙箱策略或公共 child 行为。 + +## Behavior + +- **唯一可复用 ABI owner** — `abi.ts` 拥有 sandbox process 路径消费的 Win32 常量与 x64 布局值。`ffi.ts` 懒加载 `kernel32.dll` 与 `advapi32.dll`,核验 `STARTUPINFOW` 和 `PROCESS_INFORMATION`,提供带类型的操作与错误格式化,并让 sandbox policy 通过同一组已加载库绑定剩余 API。 +- **restricted-token 创建** — `RestrictedProcessSpawnOptions` 要求 sandbox 的 primary token,并使用 `CreateProcessAsUserW`。pipe 与 inherited-stdio 路径共用命令行引用、cwd、继承环境块、返回值检查与句柄清理。 +- **管道进程原语** — `spawnPipedProcess()` 创建匿名 stdin/stdout/stderr 管道,立即关闭 stdin,并返回两个读取端;调用方负责等待进程与排空管道。任一局部失败都会关闭该操作已经拥有的句柄,并在各自 Win32 生命周期结束后释放每个 Koffi 输出槽与结构体分配。 +- **继承 stdio 的 Job 原语** — `spawnInheritedJobProcess()` 创建一个 kill-on-close Job,临时把当前 stdio 句柄设为可继承,以 suspended 状态创建 restricted child,把它分配给 Job,再恢复初始线程。目标代码不会在 Job 分配前运行;受控的分配或恢复失败会终止 suspended child,或在释放全部已拥有句柄前关闭已分配的 Job。 +- **显式结算归属** — `waitForProcessExit()` 等待并关闭进程句柄。`drainPipe()` 在排空期间复用一个 native count slot,释放该分配并关闭管道读取句柄。sandbox 保留既有调度、result 组合与调用方拥有的 Job 关闭行为。 + +Windows ACL 沙箱在这些原语上增加 SID、DACL、grant、workspace 与公共 child policy。 + + + +## 头部验证 + +process、stdio 与 Job 的常量以及选定结构体的大小和偏移由 [`verify/abi-probe.cpp`](verify/abi-probe.cpp) 对照 MinGW Windows 头文件检查: + +```sh +g++ -std=c++20 -municode -O2 -o abi-probe.exe verify/abi-probe.cpp && ./abi-probe.exe +``` + +Koffi 的 `STARTUPINFOW` 与 `PROCESS_INFORMATION` 定义还会在模块加载时断言各自的 64 位大小;其余已记录偏移和常量由该探针提供证据。 + +## Model Experience + +### 进程原语 + +#### 模型看到什么 + +没有直接内容。本包向 sandbox 提供 `Win32ProcessBindings` 与进程原语;sandbox 拥有全部模型可见工具、输出与诊断,本包不贡献提示词或工具 schema。 + +#### Token 影响 + +没有直接影响。消费方决定进程输出是否进入工具结果或后续模型请求。 + +#### KV Cache 影响 + +本包不贡献稳定请求前缀,因此不会使模型 KV Cache 失效。 + +## Known Limitations and Deferred Work + +- **仅在 Windows 原生加载** — 导入通用类型可跨平台进行,但解析绑定表会加载 Windows DLL,并在其他宿主失败。跨平台测试注入绑定表,不加载原生 API。 +- **没有公共进程服务** — 本包刻意不把原语包装成 Cordis 或 Node streams。消费方必须拥有自己的策略、异步调度、输出上限、取消与最终句柄关闭。 +- **只继承环境** — 进程创建传入空环境块。sandbox 会先通过 `SetEnvironmentVariableW` 建立改动,因为经 Koffi 传入显式环境块会使 `CreateProcessAsUserW` 以 `ERROR_INVALID_PARAMETER` 失败。其他需要改写环境的调用方必须在调用原语前建立环境,或使用自己的 runner 进程。 +- **只有 restricted-token 消费方** — ordinary `CreateProcessW`、精确 `applicationName`、parent-stdio release 与 whole-Job settlement 在 ordinary process 消费方出现前均不提供。 +- **创建到分配之间的中断** — 目标以 suspended 状态启动,不能在 Job 分配前执行,但 runner 若在进程创建到分配之间的极窄区间被外力终止,可能留下 suspended target。本包不声明原子 Job 附加保证。 +- **header 证据限定架构** — 已提交的 ABI probe 与布局常量覆盖仓库当前 64 位 Windows 目标。支持新的指针宽度或不兼容 Windows ABI 前,必须先更新 probe。 diff --git a/packages/client/schema-form/package.json b/packages/subprocess/win32-process/package.json similarity index 70% rename from packages/client/schema-form/package.json rename to packages/subprocess/win32-process/package.json index a6fd68893b..802287e09c 100644 --- a/packages/client/schema-form/package.json +++ b/packages/subprocess/win32-process/package.json @@ -1,14 +1,14 @@ { - "name": "@deepseek-ai/dsh-client-schema-form", - "description": "Schema/draft model layer for settings editors: rehydrates a serialized schemastery schema, validates drafts, and edits them immutably by path", - "version": "0.0.1-rc.5", + "name": "@deepseek-ai/dsh-win32-process", + "description": "Low-level Win32 process, stdio, and Job Object primitives for the DeepSeek Harness Windows sandbox", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", - "directory": "packages/client/schema-form" + "directory": "packages/subprocess/win32-process" }, "type": "module", "main": "lib/index.js", @@ -25,21 +25,21 @@ "./src/*": "./src/*", "./package.json": "./package.json" }, - "license": "MIT", - "dependencies": { - "@deepseek-ai/schemastery": "workspace:^" - }, - "peerDependencies": { - "@deepseek-ai/dsh-invariants": "workspace:^", - "@deepseek-ai/cordis": "workspace:^" - }, - "devDependencies": { - "@deepseek-ai/dsh-invariants": "workspace:^", - "@deepseek-ai/cordis": "workspace:^" - }, "files": [ "lib/index.js", "lib/invariant.js", "lib/types/**/*.d.ts" - ] + ], + "license": "MIT", + "peerDependencies": { + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + }, + "dependencies": { + "koffi": "^3.1.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + } } diff --git a/packages/subprocess/win32-process/src/abi.ts b/packages/subprocess/win32-process/src/abi.ts new file mode 100644 index 0000000000..fbdda9059f --- /dev/null +++ b/packages/subprocess/win32-process/src/abi.ts @@ -0,0 +1,38 @@ +/** Generic Win32 process, stdio, and Job Object constants verified on x64. */ + +/** STARTUPINFOW uses the standard input, output, and error handles. */ +export const STARTF_USESTDHANDLES = 0x00000100 +/** HandleInformation flag that permits child inheritance. */ +export const HANDLE_FLAG_INHERIT = 0x1 +/** Infinite WaitForSingleObject timeout. */ +export const INFINITE = 0xFFFFFFFF +/** CreateProcess flag that prevents user code from running before resume. */ +export const CREATE_SUSPENDED = 0x4 +/** GetStdHandle selector for standard input. */ +export const STD_INPUT_HANDLE = -10 +/** GetStdHandle selector for standard output. */ +export const STD_OUTPUT_HANDLE = -11 +/** GetStdHandle selector for standard error. */ +export const STD_ERROR_HANDLE = -12 +/** FormatMessage reads the operating system message table. */ +export const FORMAT_MESSAGE_FROM_SYSTEM = 0x00001000 +/** FormatMessage leaves insertion placeholders uninterpreted. */ +export const FORMAT_MESSAGE_IGNORE_INSERTS = 0x00000200 +/** Win32 code reporting a caller-provided buffer is too small. */ +export const ERROR_INSUFFICIENT_BUFFER = 122 +/** Win32 code reporting that the other pipe end closed. */ +export const ERROR_BROKEN_PIPE = 109 +/** Win32 code reporting that a pipe has no remaining data. */ +export const ERROR_NO_DATA = 232 +/** Job limit that terminates every member when the final Job handle closes. */ +export const JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE = 0x00002000 +/** SetInformationJobObject class for JOBOBJECT_EXTENDED_LIMIT_INFORMATION. */ +export const JobObjectExtendedLimitInformation = 9 +/** x64 JOBOBJECT_EXTENDED_LIMIT_INFORMATION byte size. */ +export const JOBOBJECT_EXTENDED_LIMIT_SIZE = 144 +/** Byte offset of BasicLimitInformation.LimitFlags in the extended Job record. */ +export const JOBOBJECT_EXTENDED_LIMIT_FLAGS_OFFSET = 16 +/** x64 STARTUPINFOW byte size verified by the native probe. */ +export const STARTUPINFOW_SIZE = 104 +/** x64 PROCESS_INFORMATION byte size verified by the native probe. */ +export const PROCESS_INFORMATION_SIZE = 24 diff --git a/packages/subprocess/win32-process/src/errors.ts b/packages/subprocess/win32-process/src/errors.ts new file mode 100644 index 0000000000..f6e90805a1 --- /dev/null +++ b/packages/subprocess/win32-process/src/errors.ts @@ -0,0 +1,14 @@ +/** Win32 call failure with the exact API name and error code. */ +export class Win32Error extends Error { + /** Win32 function whose checked result failed. */ + readonly api: string + /** Exact GetLastError value or direct Win32 API error code. */ + readonly win32Code: number + + constructor(api: string, win32Code: number, detail?: string) { + super(`${api} failed (Win32 ${win32Code})${detail === undefined ? '' : `: ${detail}`}`) + this.name = 'Win32Error' + this.api = api + this.win32Code = win32Code + } +} diff --git a/packages/subprocess/win32-process/src/ffi.ts b/packages/subprocess/win32-process/src/ffi.ts new file mode 100644 index 0000000000..a2171d0023 --- /dev/null +++ b/packages/subprocess/win32-process/src/ffi.ts @@ -0,0 +1,319 @@ +/** Lazy Koffi bindings for generic Win32 process, stdio, and Job operations. */ + +import koffi from 'koffi' +import * as abi from './abi.ts' +import { Win32Error } from './errors.ts' + +declare const nativePtr: unique symbol +/** Koffi native pointer branded against accidental numeric use. */ +export type NativePtr = bigint & { readonly [nativePtr]: true } + +type Ptr = ReturnType +const PVOID: Ptr = koffi.pointer('void') +const PPVOID: Ptr = koffi.pointer(PVOID) + +/** Loaded Win32 libraries and the shared stdcall binder used by process extensions. */ +export interface Win32BindingContext { + /** Kernel process, handle, pipe, and Job APIs. */ + readonly kernel32: ReturnType + /** Token and security APIs. */ + readonly advapi32: ReturnType + /** Bind one stdcall function from a loaded Win32 library. */ + readonly bind: ( + library: ReturnType, + name: string, + result: Ptr | string, + args: Array, + ) => unknown +} + +/** + * Return whether a Koffi pointer represents NULL. + * @param value - pointer value returned by Koffi or a Win32 call. + * @returns true for null, undefined, or address zero. + */ +export function isNullPtr(value: NativePtr | null | undefined): value is null | undefined { + return value === null || value === undefined || (value as bigint) === 0n +} + +/** STARTUPINFOW fields used by inherited or piped stdio launches. */ +export interface StartupInfoInput { + cb: number + dwFlags: number + hStdInput: NativePtr + hStdOutput: NativePtr + hStdError: NativePtr +} + +/** Decoded PROCESS_INFORMATION result. */ +export interface ProcessInfoOutput { + hProcess: NativePtr | null + hThread: NativePtr | null + dwProcessId: number + dwThreadId: number +} + +/** Generic Win32 calls consumed by restricted-token sandbox process operations. */ +export interface Win32ProcessBindings { + closeHandle(handle: NativePtr): number + getLastError(): number + formatMessageW( + flags: number, + source: null, + messageId: number, + languageId: number, + buffer: Buffer, + size: number, + args: null, + ): number + createPipe(readHandle: NativePtr, writeHandle: NativePtr, attributes: null, size: number): number + setHandleInformation(handle: NativePtr, mask: number, flags: number): number + createProcessAsUserW( + token: NativePtr, + applicationName: null, + commandLine: string, + processAttributes: null, + threadAttributes: null, + inheritHandles: number, + creationFlags: number, + environment: null, + currentDirectory: string | null, + startupInfo: NativePtr, + processInfo: NativePtr, + ): number + readFile(file: NativePtr, buffer: Buffer, count: number, bytesRead: NativePtr, overlapped: null): number + peekNamedPipe( + pipe: NativePtr, + buffer: null, + size: number, + bytesRead: NativePtr | null, + totalAvail: NativePtr, + leftThisMessage: NativePtr | null, + ): number + waitForSingleObject(handle: NativePtr, milliseconds: number): number + getExitCodeProcess(process: NativePtr, exitCode: NativePtr): number + createJobObjectW(attributes: null, name: null): NativePtr + setInformationJobObject(job: NativePtr, cls: number, information: Buffer, length: number): number + assignProcessToJobObject(job: NativePtr, process: NativePtr): number + resumeThread(thread: NativePtr): number + terminateProcess(process: NativePtr, exitCode: number): number + getStdHandle(stdHandle: number): NativePtr +} + +/** Koffi STARTUPINFOW layout. */ +export const STARTUPINFOW = koffi.struct('DSH_STARTUPINFOW', { + cb: 'uint32', + lpReserved: 'str16', + lpDesktop: 'str16', + lpTitle: 'str16', + dwX: 'uint32', + dwY: 'uint32', + dwXSize: 'uint32', + dwYSize: 'uint32', + dwXCountChars: 'uint32', + dwYCountChars: 'uint32', + dwFillAttribute: 'uint32', + dwFlags: 'uint32', + wShowWindow: 'uint16', + cbReserved2: 'uint16', + lpReserved2: koffi.pointer('uint8'), + hStdInput: PVOID, + hStdOutput: PVOID, + hStdError: PVOID, +}) + +/** Koffi PROCESS_INFORMATION layout. */ +export const PROCESS_INFORMATION = koffi.struct('DSH_PROCESS_INFORMATION', { + hProcess: PVOID, + hThread: PVOID, + dwProcessId: 'uint32', + dwThreadId: 'uint32', +}) + +/* v8 ignore start -- ABI guards are pinned by native header probes. */ +if (STARTUPINFOW.size !== abi.STARTUPINFOW_SIZE) { + throw new Error(`STARTUPINFOW layout mismatch: koffi computed ${STARTUPINFOW.size}, expected ${abi.STARTUPINFOW_SIZE}`) +} +if (PROCESS_INFORMATION.size !== abi.PROCESS_INFORMATION_SIZE) { + throw new Error(`PROCESS_INFORMATION layout mismatch: koffi computed ${PROCESS_INFORMATION.size}, expected ${abi.PROCESS_INFORMATION_SIZE}`) +} +/* v8 ignore stop */ + +/** + * Allocate a pointer-sized out-parameter slot. + * @returns allocated native slot. + */ +export function allocPtrSlot(): NativePtr { + return koffi.alloc(PVOID, 1) as NativePtr +} + +/** + * Allocate a uint32 out-parameter slot. + * @returns allocated native slot. + */ +export function allocUint32(): NativePtr { + return koffi.alloc('uint32', 1) as NativePtr +} + +/** + * Decode a pointer out-parameter. + * @param slot - pointer-sized slot filled by Win32. + * @returns decoded pointer, or null for address zero. + */ +export function decodePtr(slot: NativePtr): NativePtr | null { + const value = koffi.decode(slot, PVOID) as NativePtr | null + return isNullPtr(value) ? null : value +} + +/** + * Decode a uint32 out-parameter. + * @param slot - uint32 slot filled by Win32. + * @returns decoded unsigned value. + */ +export function decodeUint32(slot: NativePtr): number { + return koffi.decode(slot, 'uint32') as number +} + +/** + * Allocate a zeroed STARTUPINFOW. + * @returns allocated struct pointer. + */ +export function allocStartupInfo(): NativePtr { + return koffi.alloc(STARTUPINFOW, 1) as NativePtr +} + +/** + * Encode the stdio-bearing STARTUPINFOW fields. + * @param startupInfo - allocated STARTUPINFOW pointer. + * @param fields - fields required for inherited stdio. + */ +export function encodeStartupInfo(startupInfo: NativePtr, fields: StartupInfoInput): void { + koffi.encode(startupInfo, STARTUPINFOW, fields) +} + +/** + * Allocate a zeroed PROCESS_INFORMATION. + * @returns allocated struct pointer. + */ +export function allocProcessInfo(): NativePtr { + return koffi.alloc(PROCESS_INFORMATION, 1) as NativePtr +} + +/** + * Decode PROCESS_INFORMATION. + * @param processInfo - struct pointer filled by CreateProcess. + * @returns process/thread handles and ids. + */ +export function decodeProcessInfo(processInfo: NativePtr): ProcessInfoOutput { + return koffi.decode(processInfo, PROCESS_INFORMATION) as ProcessInfoOutput +} + +let cachedContext: Win32BindingContext | undefined +let cached: Win32ProcessBindings | undefined + +/* v8 ignore start -- exercised by native Windows ABI and sandbox jobs. */ +function bindingContext(): Win32BindingContext { + if (cachedContext !== undefined) return cachedContext + const kernel32 = koffi.load('kernel32.dll') + const advapi32 = koffi.load('advapi32.dll') + const bind = ( + lib: ReturnType, + name: string, + result: Ptr | string, + args: Array, + ): unknown => lib.func('__stdcall', name, result, args) + cachedContext = { kernel32, advapi32, bind } + return cachedContext +} + +function bindings(): Win32ProcessBindings { + if (cached !== undefined) return cached + const { kernel32, advapi32, bind } = bindingContext() + cached = { + closeHandle: bind(kernel32, 'CloseHandle', 'int', [PVOID]), + getLastError: bind(kernel32, 'GetLastError', 'uint32', []), + formatMessageW: bind(kernel32, 'FormatMessageW', 'uint32', [ + 'uint32', PVOID, 'uint32', 'uint32', PVOID, 'uint32', PVOID, + ]), + createPipe: bind(kernel32, 'CreatePipe', 'int', [PPVOID, PPVOID, PVOID, 'uint32']), + setHandleInformation: bind(kernel32, 'SetHandleInformation', 'int', [PVOID, 'uint32', 'uint32']), + createProcessAsUserW: bind(advapi32, 'CreateProcessAsUserW', 'int', [ + PVOID, 'str16', 'str16', PVOID, PVOID, 'int', 'uint32', PVOID, 'str16', + koffi.pointer(STARTUPINFOW), koffi.pointer(PROCESS_INFORMATION), + ]), + readFile: bind(kernel32, 'ReadFile', 'int', [PVOID, PVOID, 'uint32', koffi.pointer('uint32'), PVOID]), + peekNamedPipe: bind(kernel32, 'PeekNamedPipe', 'int', [ + PVOID, PVOID, 'uint32', koffi.pointer('uint32'), koffi.pointer('uint32'), koffi.pointer('uint32'), + ]), + waitForSingleObject: bind(kernel32, 'WaitForSingleObject', 'uint32', [PVOID, 'uint32']), + getExitCodeProcess: bind(kernel32, 'GetExitCodeProcess', 'int', [PVOID, koffi.pointer('uint32')]), + createJobObjectW: bind(kernel32, 'CreateJobObjectW', PVOID, [PVOID, 'str16']), + setInformationJobObject: bind(kernel32, 'SetInformationJobObject', 'int', [PVOID, 'int', PVOID, 'uint32']), + assignProcessToJobObject: bind(kernel32, 'AssignProcessToJobObject', 'int', [PVOID, PVOID]), + resumeThread: bind(kernel32, 'ResumeThread', 'uint32', [PVOID]), + terminateProcess: bind(kernel32, 'TerminateProcess', 'int', [PVOID, 'uint32']), + getStdHandle: bind(kernel32, 'GetStdHandle', PVOID, ['int']), + } as unknown as Win32ProcessBindings + return cached +} + +/** + * Extend the shared process table with caller-owned Win32 API families. + * @param create - binds only the caller-specific operations from the shared libraries. + * @returns generic process bindings combined with the caller-specific operations. + */ +export function extendWin32ProcessBindings( + create: (context: Win32BindingContext) => Extension, +): Win32ProcessBindings & Extension { + return { ...bindings(), ...create(bindingContext()) } +} +/* v8 ignore stop */ + +/** + * Format a Win32 error code through FormatMessageW. + * @param api - active binding table. + * @param win32Code - captured GetLastError value. + * @returns trimmed system message, or an empty string when unavailable. + */ +export function errorText(api: Win32ProcessBindings, win32Code: number): string { + const buffer = Buffer.alloc(1024) + const length = api.formatMessageW( + abi.FORMAT_MESSAGE_FROM_SYSTEM | abi.FORMAT_MESSAGE_IGNORE_INSERTS, + null, + win32Code, + 0, + buffer, + buffer.length / 2, + null, + ) + return length === 0 ? '' : buffer.subarray(0, length * 2).toString('utf16le').trim() +} + +/** + * Throw the current GetLastError value. + * @param api - active binding table. + * @param name - failing Win32 operation. + * @param detail - optional operation context. + * @returns never; always throws Win32Error. + */ +export function throwLastError(api: Win32ProcessBindings, name: string, detail?: string): never { + const win32Code = api.getLastError() + throw new Win32Error(name, win32Code, detail ?? errorText(api, win32Code)) +} + +/** + * Throw an explicitly captured Win32 error code. + * @param api - active binding table. + * @param name - failing Win32 operation. + * @param win32Code - error captured before cleanup. + * @param detail - optional operation context. + * @returns never; always throws Win32Error. + */ +export function throwWin32( + api: Win32ProcessBindings, + name: string, + win32Code: number, + detail?: string, +): never { + throw new Win32Error(name, win32Code, detail ?? errorText(api, win32Code)) +} diff --git a/packages/subprocess/win32-process/src/index.ts b/packages/subprocess/win32-process/src/index.ts new file mode 100644 index 0000000000..d6dee59d58 --- /dev/null +++ b/packages/subprocess/win32-process/src/index.ts @@ -0,0 +1,28 @@ +/** Low-level Win32 process, stdio, and Job Object primitives used by the Windows ACL sandbox. */ + +export { ERROR_INSUFFICIENT_BUFFER } from './abi.ts' +export * from './errors.ts' +export { + allocPtrSlot, + allocUint32, + decodePtr, + decodeUint32, + extendWin32ProcessBindings, + isNullPtr, + throwLastError, + throwWin32, +} from './ffi.ts' +export type { + NativePtr, + Win32ProcessBindings, +} from './ffi.ts' +export { + drainPipe, + spawnInheritedJobProcess, + spawnPipedProcess, + waitForProcessExit, +} from './process.ts' +export type { + SpawnedJobProcess, + SpawnedPipedProcess, +} from './process.ts' diff --git a/packages/subprocess/win32-process/src/invariant.ts b/packages/subprocess/win32-process/src/invariant.ts new file mode 100644 index 0000000000..bd29124923 --- /dev/null +++ b/packages/subprocess/win32-process/src/invariant.ts @@ -0,0 +1,17 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-win32-process`. */ + +/* jscpd:ignore-start */ +import type { Context } from '@deepseek-ai/cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-win32-process' + +export const name = 'win32-process-invariant' +export const inject = ['invariants'] + +/** No runtime invariant: operations own only call-local native handles. */ +const install: InvariantInstaller = () => {} + +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/subprocess/win32-process/src/process.ts b/packages/subprocess/win32-process/src/process.ts new file mode 100644 index 0000000000..7c676e1f7f --- /dev/null +++ b/packages/subprocess/win32-process/src/process.ts @@ -0,0 +1,429 @@ +/** Typed Win32 process operations over the shared binding table. */ + +import koffi from 'koffi' +import * as abi from './abi.ts' +import { + allocProcessInfo, + allocPtrSlot, + allocStartupInfo, + allocUint32, + decodeProcessInfo, + decodePtr, + decodeUint32, + encodeStartupInfo, + isNullPtr, + throwLastError, + throwWin32, +} from './ffi.ts' +import type { NativePtr, Win32ProcessBindings } from './ffi.ts' + +/** + * Quote one argument according to CommandLineToArgvW parsing. + * @param argument - one argv entry. + * @returns bare or quoted command-line segment. + */ +export function quoteArg(argument: string): string { + if (argument === '') return '""' + if (!/[\s"]/u.test(argument)) return argument + let quoted = '"' + for (let index = 0; index < argument.length; index++) { + let backslashes = 0 + while (index < argument.length && argument.charAt(index) === '\\') { + backslashes += 1 + index += 1 + } + if (index === argument.length) { + quoted += '\\'.repeat(backslashes * 2) + } else if (argument.charAt(index) === '"') { + quoted += '\\'.repeat(backslashes * 2 + 1) + '"' + } else { + quoted += '\\'.repeat(backslashes) + argument.charAt(index) + } + } + return quoted + '"' +} + +/** + * Build the mutable command line accepted by CreateProcessAsUserW. + * @param program - executable argv entry. + * @param args - remaining argv entries. + * @returns joined Win32 command line. + */ +export function buildCommandLine(program: string, args: readonly string[]): string { + return [program, ...args].map(quoteArg).join(' ') +} + +/** Restricted-token process creation inputs owned by the Windows ACL sandbox. */ +export interface RestrictedProcessSpawnOptions { + /** Executable argv entry passed through CreateProcessAsUserW. */ + command: string + /** Arguments excluding the executable. */ + args: readonly string[] + /** Existing child working directory. */ + cwd: string + /** Restricted primary token supplied by sandbox policy. */ + token: NativePtr +} + +/** Piped child resources whose process and read handles remain caller-owned. */ +export interface SpawnedPipedProcess { + /** Direct child process id. */ + pid: number + /** Process handle closed by waitForProcessExit. */ + process: NativePtr + /** Stdout pipe read end closed by drainPipe. */ + stdoutRead: NativePtr + /** Stderr pipe read end closed by drainPipe. */ + stderrRead: NativePtr +} + +/** Suspended child assigned to one caller-owned kill-on-close Job before resume. */ +export interface SpawnedJobProcess { + /** Direct child process id. */ + pid: number + /** Process handle closed by waitForProcessExit. */ + process: NativePtr + /** Job handle closed by the lifecycle owner. */ + job: NativePtr +} + +interface PipePair { + read: NativePtr + write: NativePtr +} + +function freeNative(pointer: NativePtr | undefined): void { + if (pointer !== undefined) koffi.free(pointer) +} + +function closeBestEffort(api: Win32ProcessBindings, handle: NativePtr | null | undefined): void { + if (!isNullPtr(handle)) api.closeHandle(handle) +} + +function createPipe(api: Win32ProcessBindings, owned: Set): PipePair { + const readSlot = allocPtrSlot() + let writeSlot: NativePtr | undefined + try { + writeSlot = allocPtrSlot() + if (api.createPipe(readSlot, writeSlot, null, 0) === 0) throwLastError(api, 'CreatePipe') + const read = decodePtr(readSlot) + const write = decodePtr(writeSlot) + if (read === null || write === null) { + closeBestEffort(api, read) + closeBestEffort(api, write) + throwLastError(api, 'CreatePipe', 'null pipe handle') + } + owned.add(read) + owned.add(write) + return { read, write } + } finally { + freeNative(writeSlot) + koffi.free(readSlot) + } +} + +function closeOwned(api: Win32ProcessBindings, owned: Set, handle: NativePtr): void { + /* v8 ignore next -- each successfully decoded pipe end is uniquely owned. */ + if (!owned.delete(handle)) return + api.closeHandle(handle) +} + +function closeAllOwned(api: Win32ProcessBindings, owned: Set): void { + for (const handle of owned) api.closeHandle(handle) + owned.clear() +} + +function createRestrictedProcess( + api: Win32ProcessBindings, + options: RestrictedProcessSpawnOptions, + commandLine: string, + creationFlags: number, + startupInfo: NativePtr, + processInfo: NativePtr, +): number { + // The sandbox mutates its process environment before this call. Passing an + // explicit block through Koffi makes CreateProcessAsUserW reject the request + // with ERROR_INVALID_PARAMETER, so lpEnvironment remains NULL. + return api.createProcessAsUserW( + options.token, + null, + commandLine, + null, + null, + 1, + creationFlags, + null, + options.cwd, + startupInfo, + processInfo, + ) +} + +/** + * Spawn a process with anonymous-pipe stdout/stderr and immediate stdin EOF. + * @param api - active binding table. + * @param options - command, cwd, args, and restricted primary token. + * @returns caller-owned process and pipe read handles. + */ +export function spawnPipedProcess( + api: Win32ProcessBindings, + options: RestrictedProcessSpawnOptions, +): SpawnedPipedProcess { + const owned = new Set() + let startupInfo: NativePtr | undefined + let processInfo: NativePtr | undefined + try { + const stdIn = createPipe(api, owned) + const stdOut = createPipe(api, owned) + const stdErr = createPipe(api, owned) + for (const [handle, label] of [ + [stdIn.read, 'stdin read end'], + [stdOut.write, 'stdout write end'], + [stdErr.write, 'stderr write end'], + ] as const) { + if (api.setHandleInformation(handle, abi.HANDLE_FLAG_INHERIT, abi.HANDLE_FLAG_INHERIT) === 0) { + throwLastError(api, 'SetHandleInformation', label) + } + } + startupInfo = allocStartupInfo() + encodeStartupInfo(startupInfo, { + cb: abi.STARTUPINFOW_SIZE, + dwFlags: abi.STARTF_USESTDHANDLES, + hStdInput: stdIn.read, + hStdOutput: stdOut.write, + hStdError: stdErr.write, + }) + processInfo = allocProcessInfo() + const created = createRestrictedProcess( + api, + options, + buildCommandLine(options.command, options.args), + 0, + startupInfo, + processInfo, + ) + if (created === 0) { + const win32Code = api.getLastError() + throwWin32(api, 'CreateProcessAsUserW', win32Code, `command: ${options.command}, cwd: ${options.cwd}`) + } + const info = decodeProcessInfo(processInfo) + if (info.hProcess === null || info.hThread === null) { + if (info.hProcess !== null) api.terminateProcess(info.hProcess, 1) + closeBestEffort(api, info.hThread) + closeBestEffort(api, info.hProcess) + throw new Error(`CreateProcessAsUserW succeeded but returned null process/thread handles (pid ${info.dwProcessId})`) + } + closeOwned(api, owned, stdIn.read) + closeOwned(api, owned, stdIn.write) + closeOwned(api, owned, stdOut.write) + closeOwned(api, owned, stdErr.write) + closeBestEffort(api, info.hThread) + owned.delete(stdOut.read) + owned.delete(stdErr.read) + return { + pid: info.dwProcessId, + process: info.hProcess, + stdoutRead: stdOut.read, + stderrRead: stdErr.read, + } + } catch (error) { + closeAllOwned(api, owned) + throw error + } finally { + freeNative(processInfo) + freeNative(startupInfo) + } +} + +/** + * Drain one anonymous pipe until the writer closes it. + * @param api - active binding table. + * @param handle - caller-owned pipe read end. + * @returns complete bytes read before EOF; the handle is always closed. + * @throws when a Win32 pipe operation fails. + */ +export async function drainPipe( + api: Win32ProcessBindings, + handle: NativePtr, +): Promise { + const chunks: Buffer[] = [] + let countSlot: NativePtr | undefined + try { + countSlot = allocUint32() + for (;;) { + const peeked = api.peekNamedPipe(handle, null, 0, null, countSlot, null) + if (peeked === 0) { + const win32Code = api.getLastError() + if (win32Code === abi.ERROR_BROKEN_PIPE || win32Code === abi.ERROR_NO_DATA) break + throwLastError(api, 'PeekNamedPipe', `drain failure after ${chunks.length} chunk(s)`) + } + const available = decodeUint32(countSlot) + if (available > 0) { + const chunk = Buffer.alloc(available) + if (api.readFile(handle, chunk, chunk.length, countSlot, null) === 0) { + throwLastError(api, 'ReadFile', `drain failure after ${chunks.length} chunk(s)`) + } + chunks.push(chunk.subarray(0, decodeUint32(countSlot))) + } + await new Promise(resolve => setTimeout(resolve, 1)) + } + return Buffer.concat(chunks) + } finally { + freeNative(countSlot) + api.closeHandle(handle) + } +} + +/** + * Wait for a process and always close its handle. + * @param api - active binding table. + * @param process - caller-owned process handle. + * @returns direct process exit code. + */ +export function waitForProcessExit(api: Win32ProcessBindings, process: NativePtr): number { + let exitCodeSlot: NativePtr | undefined + try { + if (api.waitForSingleObject(process, abi.INFINITE) === 0xFFFFFFFF) { + throwLastError(api, 'WaitForSingleObject') + } + exitCodeSlot = allocUint32() + if (api.getExitCodeProcess(process, exitCodeSlot) === 0) throwLastError(api, 'GetExitCodeProcess') + return decodeUint32(exitCodeSlot) + } finally { + freeNative(exitCodeSlot) + api.closeHandle(process) + } +} + +function createKillOnCloseJob(api: Win32ProcessBindings): NativePtr { + const job = api.createJobObjectW(null, null) + if (isNullPtr(job)) throwLastError(api, 'CreateJobObjectW') + const information = Buffer.alloc(abi.JOBOBJECT_EXTENDED_LIMIT_SIZE) + information.writeUInt32LE( + abi.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, + abi.JOBOBJECT_EXTENDED_LIMIT_FLAGS_OFFSET, + ) + if (api.setInformationJobObject( + job, + abi.JobObjectExtendedLimitInformation, + information, + information.length, + ) === 0) { + const win32Code = api.getLastError() + api.closeHandle(job) + throwWin32(api, 'SetInformationJobObject', win32Code) + } + return job +} + +/** + * Spawn suspended, assign the child to a kill-on-close Job, then resume it. + * @param api - active binding table. + * @param options - command, cwd, args, and restricted primary token. + * @returns caller-owned process and Job handles after successful resume. + * @remarks Node clears stdio handle inheritability at startup through + * uv_disable_stdio_inheritance. This operation temporarily restores the bits + * required by STARTF_USESTDHANDLES. Restoring them afterward is best-effort: + * failure must not replace the already-created child's outcome. + */ +export function spawnInheritedJobProcess( + api: Win32ProcessBindings, + options: RestrictedProcessSpawnOptions, +): SpawnedJobProcess { + const job = createKillOnCloseJob(api) + const getStdHandle = (selector: number, label: string): NativePtr => { + const handle = api.getStdHandle(selector) + if (!isNullPtr(handle)) return handle + const win32Code = api.getLastError() + api.closeHandle(job) + throwWin32(api, 'GetStdHandle', win32Code, `null ${label} handle`) + } + const stdIn = getStdHandle(abi.STD_INPUT_HANDLE, 'stdin') + const stdOut = getStdHandle(abi.STD_OUTPUT_HANDLE, 'stdout') + const stdErr = getStdHandle(abi.STD_ERROR_HANDLE, 'stderr') + const enabled: NativePtr[] = [] + let startupInfo: NativePtr | undefined + let processInfo: NativePtr | undefined + let created = 0 + let createFailureCode = 0 + try { + for (const [handle, label] of [ + [stdIn, 'stdin'], + [stdOut, 'stdout'], + [stdErr, 'stderr'], + ] as const) { + if (api.setHandleInformation(handle, abi.HANDLE_FLAG_INHERIT, abi.HANDLE_FLAG_INHERIT) === 0) { + throwLastError(api, 'SetHandleInformation', `${label} (enable inherit)`) + } + enabled.push(handle) + } + startupInfo = allocStartupInfo() + encodeStartupInfo(startupInfo, { + cb: abi.STARTUPINFOW_SIZE, + dwFlags: abi.STARTF_USESTDHANDLES, + hStdInput: stdIn, + hStdOutput: stdOut, + hStdError: stdErr, + }) + processInfo = allocProcessInfo() + created = createRestrictedProcess( + api, + options, + buildCommandLine(options.command, options.args), + abi.CREATE_SUSPENDED, + startupInfo, + processInfo, + ) + if (created === 0) createFailureCode = api.getLastError() + } catch (error) { + freeNative(processInfo) + api.closeHandle(job) + throw error + } finally { + freeNative(startupInfo) + for (const handle of enabled) { + // The runner spawns nothing else; cleanup failure must not mask the child. + api.setHandleInformation(handle, abi.HANDLE_FLAG_INHERIT, 0) + } + } + if (created === 0) { + freeNative(processInfo) + api.closeHandle(job) + throwWin32( + api, + 'CreateProcessAsUserW', + createFailureCode, + `command: ${options.command}, cwd: ${options.cwd}`, + ) + } + let info: ReturnType + try { + info = decodeProcessInfo(processInfo) + } finally { + freeNative(processInfo) + } + if (info.hProcess === null || info.hThread === null) { + if (info.hProcess !== null) api.terminateProcess(info.hProcess, 1) + api.closeHandle(job) + closeBestEffort(api, info.hThread) + closeBestEffort(api, info.hProcess) + throw new Error(`CreateProcessAsUserW succeeded but returned null process/thread handles (pid ${info.dwProcessId})`) + } + if (api.assignProcessToJobObject(job, info.hProcess) === 0) { + const win32Code = api.getLastError() + api.terminateProcess(info.hProcess, 1) + closeBestEffort(api, info.hThread) + closeBestEffort(api, info.hProcess) + api.closeHandle(job) + throwWin32(api, 'AssignProcessToJobObject', win32Code, `pid ${info.dwProcessId}`) + } + if (api.resumeThread(info.hThread) === 0xFFFFFFFF) { + const win32Code = api.getLastError() + closeBestEffort(api, info.hThread) + closeBestEffort(api, info.hProcess) + api.closeHandle(job) + throwWin32(api, 'ResumeThread', win32Code, `pid ${info.dwProcessId}`) + } + closeBestEffort(api, info.hThread) + return { pid: info.dwProcessId, process: info.hProcess, job } +} diff --git a/packages/subprocess/win32-process/tests/ffi.spec.ts b/packages/subprocess/win32-process/tests/ffi.spec.ts new file mode 100644 index 0000000000..dfe27537f3 --- /dev/null +++ b/packages/subprocess/win32-process/tests/ffi.spec.ts @@ -0,0 +1,45 @@ +import koffi from 'koffi' +import { describe, expect, it, vi } from 'vitest' +import { + Win32Error, + allocPtrSlot, + decodePtr, + isNullPtr, + throwLastError, +} from '../src/index.ts' +import { PROCESS_INFORMATION_SIZE, STARTUPINFOW_SIZE } from '../src/abi.ts' +import { PROCESS_INFORMATION, STARTUPINFOW, errorText } from '../src/ffi.ts' +import type { NativePtr, Win32ProcessBindings } from '../src/index.ts' + +describe('shared Win32 process ABI', () => { + it('matches the verified x64 structure sizes', () => { + expect(STARTUPINFOW.size).toBe(STARTUPINFOW_SIZE) + expect(PROCESS_INFORMATION.size).toBe(PROCESS_INFORMATION_SIZE) + }) + + it('handles NULL pointer out-parameters', () => { + const slot = allocPtrSlot() + expect(decodePtr(slot)).toBeNull() + expect(isNullPtr(0n as NativePtr)).toBe(true) + expect(isNullPtr(1n as NativePtr)).toBe(false) + }) + + it('formats and throws the exact Win32 error', () => { + const api = { + getLastError: vi.fn(() => 5), + formatMessageW: vi.fn((_flags, _source, _id, _language, buffer: Buffer) => { + buffer.write('access denied', 'utf16le') + return 'access denied'.length + }), + } as unknown as Win32ProcessBindings + expect(errorText(api, 5)).toBe('access denied') + expect(() => throwLastError(api, 'Probe')).toThrow(Win32Error) + expect(new Win32Error('CloseHandle', 6).message).toBe('CloseHandle failed (Win32 6)') + }) + + it('decodes a pointer stored by Koffi', () => { + const slot = allocPtrSlot() + koffi.encode(slot, koffi.pointer('void'), 42n) + expect(decodePtr(slot)).toBe(42n) + }) +}) diff --git a/packages/subprocess/win32-process/tests/invariant.spec.ts b/packages/subprocess/win32-process/tests/invariant.spec.ts new file mode 100644 index 0000000000..82078672d3 --- /dev/null +++ b/packages/subprocess/win32-process/tests/invariant.spec.ts @@ -0,0 +1,16 @@ +import { describe, expect, it, vi } from 'vitest' +import { apply, inject, name } from '../src/invariant.ts' + +describe('win32-process invariant companion', () => { + it('registers the package-owned empty invariant', async () => { + const dispose = vi.fn() + const register = vi.fn((_packageName: string, _installer: () => void) => dispose) + const ctx = { invariants: { register } } as never + await expect(apply(ctx)).resolves.toBe(dispose) + expect(name).toBe('win32-process-invariant') + expect(inject).toEqual(['invariants']) + expect(register).toHaveBeenCalledWith('@deepseek-ai/dsh-win32-process', expect.any(Function)) + const installer = register.mock.calls[0]![1] + installer() + }) +}) diff --git a/packages/subprocess/win32-process/tests/process-allocation-failure.spec.ts b/packages/subprocess/win32-process/tests/process-allocation-failure.spec.ts new file mode 100644 index 0000000000..fe3c603c02 --- /dev/null +++ b/packages/subprocess/win32-process/tests/process-allocation-failure.spec.ts @@ -0,0 +1,145 @@ +import koffi from 'koffi' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + drainPipe, + spawnInheritedJobProcess, + spawnPipedProcess, + waitForProcessExit, +} from '../src/index.ts' +import * as ffi from '../src/ffi.ts' +import { PROCESS_INFORMATION } from '../src/ffi.ts' +import type { NativePtr, Win32ProcessBindings } from '../src/ffi.ts' + +vi.mock('../src/ffi.ts', { spy: true }) + +const PVOID = koffi.pointer('void') + +afterEach(() => { + vi.restoreAllMocks() +}) + +describe('spawnInheritedJobProcess allocation cleanup', () => { + it('frees startup info when process-info allocation throws', () => { + const api = { + createJobObjectW: vi.fn(() => 50n), + setInformationJobObject: vi.fn(() => 1), + getStdHandle: vi.fn((selector: number) => BigInt(100 - selector)), + setHandleInformation: vi.fn(() => 1), + closeHandle: vi.fn(() => 1), + getLastError: vi.fn(() => 5), + formatMessageW: vi.fn(() => 0), + } as unknown as Win32ProcessBindings + const free = vi.spyOn(koffi, 'free') + vi.mocked(ffi.allocProcessInfo).mockImplementationOnce(() => { throw new Error('process-info allocation failed') }) + expect(() => spawnInheritedJobProcess(api, { + command: 'cmd.exe', + args: [], + cwd: 'C:\\', + token: 70n as NativePtr, + })).toThrow('process-info allocation failed') + expect(free).toHaveBeenCalledOnce() + }) + + it('frees process info after a successful inherited spawn', () => { + const api = { + createJobObjectW: vi.fn(() => 50n), + setInformationJobObject: vi.fn(() => 1), + getStdHandle: vi.fn((selector: number) => BigInt(100 - selector)), + setHandleInformation: vi.fn(() => 1), + createProcessAsUserW: vi.fn((_token, _app, _line, _pa, _ta, _inherit, _flags, _env, _cwd, _startup, info) => { + koffi.encode(info, PROCESS_INFORMATION, { + hProcess: 60n, + hThread: 61n, + dwProcessId: 1234, + dwThreadId: 5678, + }) + return 1 + }), + assignProcessToJobObject: vi.fn(() => 1), + resumeThread: vi.fn(() => 0), + closeHandle: vi.fn(() => 1), + getLastError: vi.fn(() => 5), + formatMessageW: vi.fn(() => 0), + } as unknown as Win32ProcessBindings + const free = vi.spyOn(koffi, 'free') + expect(spawnInheritedJobProcess(api, { + command: 'cmd.exe', + args: [], + cwd: 'C:\\', + token: 70n as NativePtr, + })).toEqual({ pid: 1234, process: 60n, job: 50n }) + expect(free).toHaveBeenCalledTimes(2) + }) +}) + +describe('shared process allocation cleanup', () => { + it('frees pipe slots and process structs after a successful piped spawn', () => { + let nextHandle = 10n + const api = { + createPipe: vi.fn((readSlot: NativePtr, writeSlot: NativePtr) => { + koffi.encode(readSlot, PVOID, nextHandle++) + koffi.encode(writeSlot, PVOID, nextHandle++) + return 1 + }), + setHandleInformation: vi.fn(() => 1), + createProcessAsUserW: vi.fn((_token, _app, _line, _pa, _ta, _inherit, _flags, _env, _cwd, _startup, info) => { + koffi.encode(info, PROCESS_INFORMATION, { + hProcess: 60n, + hThread: 61n, + dwProcessId: 1234, + dwThreadId: 5678, + }) + return 1 + }), + closeHandle: vi.fn(() => 1), + getLastError: vi.fn(() => 5), + formatMessageW: vi.fn(() => 0), + } as unknown as Win32ProcessBindings + const free = vi.spyOn(koffi, 'free') + expect(spawnPipedProcess(api, { + command: 'cmd.exe', + args: [], + cwd: 'C:\\', + token: 70n as NativePtr, + })).toMatchObject({ pid: 1234, process: 60n }) + expect(free).toHaveBeenCalledTimes(8) + }) + + it('reuses one drain count slot and frees it at EOF', async () => { + let peeks = 0 + const api = { + peekNamedPipe: vi.fn((_pipe, _buffer, _size, _read, totalAvail: NativePtr) => { + peeks += 1 + if (peeks > 1) return 0 + koffi.encode(totalAvail, 'uint32', 1) + return 1 + }), + readFile: vi.fn((_file, buffer: Buffer, _count, readSlot: NativePtr) => { + buffer[0] = 0x61 + koffi.encode(readSlot, 'uint32', 1) + return 1 + }), + getLastError: vi.fn(() => 109), + closeHandle: vi.fn(() => 1), + } as unknown as Win32ProcessBindings + const alloc = vi.spyOn(koffi, 'alloc') + const free = vi.spyOn(koffi, 'free') + await expect(drainPipe(api, 70n as NativePtr)).resolves.toEqual(Buffer.from('a')) + expect(alloc).toHaveBeenCalledOnce() + expect(free).toHaveBeenCalledOnce() + }) + + it('frees the exit-code slot after reading a process result', () => { + const api = { + waitForSingleObject: vi.fn(() => 0), + getExitCodeProcess: vi.fn((_process, exitCode: NativePtr) => { + koffi.encode(exitCode, 'uint32', 42) + return 1 + }), + closeHandle: vi.fn(() => 1), + } as unknown as Win32ProcessBindings + const free = vi.spyOn(koffi, 'free') + expect(waitForProcessExit(api, 60n as NativePtr)).toBe(42) + expect(free).toHaveBeenCalledOnce() + }) +}) diff --git a/packages/sandbox/sandbox-windows-acl/tests/failure-paths.spec.ts b/packages/subprocess/win32-process/tests/process-failure-paths.spec.ts similarity index 63% rename from packages/sandbox/sandbox-windows-acl/tests/failure-paths.spec.ts rename to packages/subprocess/win32-process/tests/process-failure-paths.spec.ts index a6bea87998..41ba438516 100644 --- a/packages/sandbox/sandbox-windows-acl/tests/failure-paths.spec.ts +++ b/packages/subprocess/win32-process/tests/process-failure-paths.spec.ts @@ -1,23 +1,28 @@ /** * Failure-path unit tests with minimal stub binding tables: the spawn * helpers must close every handle they created before throwing, and - * getTempPath must refuse to decode a buffer GetTempPathW never wrote. + * every generic process failure remains owned by the shared package. * Pure stubs — no real Win32 calls, so these run on every platform. */ import { describe, expect, it, vi } from 'vitest' import koffi from 'koffi' -import { PROCESS_INFORMATION, getTempPath } from '../src/ffi.ts' -import type { NativePtr, Win32Bindings } from '../src/ffi.ts' -import { Win32Error } from '../src/errors.ts' -import { drainPipe, spawnSandboxed, spawnSandboxedInherited, waitForExit } from '../src/spawn.ts' -import * as abi from '../src/win32-abi.ts' +import { + Win32Error, + drainPipe, + spawnInheritedJobProcess, + spawnPipedProcess, + waitForProcessExit, +} from '../src/index.ts' +import type { NativePtr, Win32ProcessBindings } from '../src/index.ts' +import * as abi from '../src/abi.ts' +import { PROCESS_INFORMATION } from '../src/ffi.ts' const PVOID = koffi.pointer('void') /** The stub the CreateProcessAsUserW failure branch needs: pipes "succeed", the spawn fails with Win32 5. */ -function pipeFailureApi(): { api: Win32Bindings; closed: bigint[]; closeHandle: ReturnType } { +function pipeFailureApi(): { api: Win32ProcessBindings; closed: bigint[]; closeHandle: ReturnType } { const closed: bigint[] = [] let next = 1n const closeHandle = vi.fn((handle: NativePtr) => { @@ -35,36 +40,7 @@ function pipeFailureApi(): { api: Win32Bindings; closed: bigint[]; closeHandle: getLastError: vi.fn(() => 5), // ERROR_ACCESS_DENIED: the failure the branch reports closeHandle, formatMessageW: vi.fn(() => 0), - } as unknown as Win32Bindings - return { api, closed, closeHandle } -} - -/** The stub the ResumeThread failure branch needs: everything succeeds until ResumeThread returns 0xFFFFFFFF. */ -function resumeFailureApi(): { api: Win32Bindings; closed: bigint[]; closeHandle: ReturnType } { - const closed: bigint[] = [] - let std = 50n - const closeHandle = vi.fn((handle: NativePtr) => { - closed.push(handle) - return 1 - }) - const api = { - createJobObjectW: vi.fn(() => 100n), - setInformationJobObject: vi.fn(() => 1), - getStdHandle: vi.fn(() => std++), - setHandleInformation: vi.fn(() => 1), - createProcessAsUserW: vi.fn(( - _token: unknown, _app: unknown, _cmd: unknown, _pa: unknown, _ta: unknown, - _inherit: unknown, _flags: unknown, _env: unknown, _cwd: unknown, _si: unknown, processInfo: NativePtr, - ) => { - koffi.encode(processInfo, PROCESS_INFORMATION, { hProcess: 200n, hThread: 201n, dwProcessId: 1234, dwThreadId: 5678 }) - return 1 - }), - assignProcessToJobObject: vi.fn(() => 1), - resumeThread: vi.fn(() => 0xFFFFFFFF), - getLastError: vi.fn(() => 5), - closeHandle, - formatMessageW: vi.fn(() => 0), - } as unknown as Win32Bindings + } as unknown as Win32ProcessBindings return { api, closed, closeHandle } } @@ -72,11 +48,11 @@ describe('spawn failure paths close their handles', () => { // A dummy token value; the stubbed spawn never reads it. const token = 1n as NativePtr - it('spawnSandboxed closes all six pipe handles before throwing when CreateProcessAsUserW fails', () => { + it('closes all six pipe handles before throwing when CreateProcessAsUserW fails', () => { const { api, closed, closeHandle } = pipeFailureApi() let caught: unknown try { - spawnSandboxed(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + spawnPipedProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) } catch (error) { caught = error } @@ -87,60 +63,11 @@ describe('spawn failure paths close their handles', () => { expect(closed).toEqual([1n, 2n, 3n, 4n, 5n, 6n]) }) - it('spawnSandboxedInherited closes thread, process, and kill-on-close job before throwing when ResumeThread fails', () => { - const { api, closed, closeHandle } = resumeFailureApi() - let caught: unknown - try { - spawnSandboxedInherited(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) - } catch (error) { - caught = error - } - expect(caught).toBeInstanceOf(Win32Error) - expect((caught as Win32Error).api).toBe('ResumeThread') - expect((caught as Win32Error).win32Code).toBe(5) - // thread, process, job — closing the job triggers kill-on-close so the - // suspended child dies instead of hanging until this process exits. - expect(closeHandle).toHaveBeenCalledTimes(3) - expect(closed).toEqual([201n, 200n, 100n]) - }) - - it('spawnSandboxedInherited TERMINATES the suspended child before closing handles when AssignProcessToJobObject fails', () => { - // The child is created suspended and is NOT in the kill-on-close job when - // the assignment fails: closing the job cannot kill it, so the failure - // branch must TerminateProcess first or every failure strands a hanging - // orphan forever. - const { api: baseApi, closeHandle } = resumeFailureApi() - type JobFailureApi = Win32Bindings & { - assignProcessToJobObject: ReturnType - terminateProcess: ReturnType - } - const api = baseApi as JobFailureApi - api.assignProcessToJobObject = vi.fn(() => 0) - api.terminateProcess = vi.fn(() => 1) - let caught: unknown - try { - spawnSandboxedInherited(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) - } catch (error) { - caught = error - } - expect(caught).toBeInstanceOf(Win32Error) - expect((caught as Win32Error).api).toBe('AssignProcessToJobObject') - expect(api.terminateProcess).toHaveBeenCalledExactlyOnceWith(200n, 1) - // thread, process, job — and the child is already dead before they close. - expect(closeHandle).toHaveBeenCalledTimes(3) - }) -}) - -describe('getTempPath buffer defense', () => { - it('throws a clear error instead of decoding a buffer GetTempPathW never wrote', () => { - const api = { getTempPathW: vi.fn(() => 300) } as unknown as Win32Bindings // 300 > the 261-char buffer - expect(() => getTempPath(api)).toThrow(/GetTempPathW failed \(Win32 122\): required 300/u) - }) }) /** The stub the pipe-happy path needs: CreatePipe fills both out slots with fresh handles. */ -function pipeOkApi(overrides: Partial = {}): { - api: Win32Bindings +function pipeOkApi(overrides: Partial = {}): { + api: Win32ProcessBindings closed: bigint[] closeHandle: ReturnType } { @@ -168,18 +95,22 @@ function pipeOkApi(overrides: Partial = {}): { closeHandle, formatMessageW: vi.fn(() => 0), ...overrides, - } as unknown as Win32Bindings + } as unknown as Win32ProcessBindings return { api, closed, closeHandle } } describe('spawn pipe failures close their handles', () => { const token = 1n as NativePtr - it('spawnSandboxed reports a CreatePipe failure', () => { - const api = { createPipe: vi.fn(() => 0), getLastError: vi.fn(() => 5), formatMessageW: vi.fn(() => 0) } as unknown as Win32Bindings + it('reports a CreatePipe failure', () => { + const api = { + createPipe: vi.fn(() => 0), + getLastError: vi.fn(() => 5), + formatMessageW: vi.fn(() => 0), + } as unknown as Win32ProcessBindings let caught: unknown try { - spawnSandboxed(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + spawnPipedProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) } catch (error) { caught = error } @@ -187,11 +118,15 @@ describe('spawn pipe failures close their handles', () => { expect((caught as Win32Error).api).toBe('CreatePipe') }) - it('spawnSandboxed reports a NULL pipe handle after CreatePipe succeeds', () => { - const api = { createPipe: vi.fn(() => 1), getLastError: vi.fn(() => 5), formatMessageW: vi.fn(() => 0) } as unknown as Win32Bindings + it('reports a NULL pipe handle after CreatePipe succeeds', () => { + const api = { + createPipe: vi.fn(() => 1), + getLastError: vi.fn(() => 5), + formatMessageW: vi.fn(() => 0), + } as unknown as Win32ProcessBindings let caught: unknown try { - spawnSandboxed(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + spawnPipedProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) } catch (error) { caught = error } @@ -199,11 +134,11 @@ describe('spawn pipe failures close their handles', () => { expect((caught as Win32Error).api).toBe('CreatePipe') }) - it('spawnSandboxed reports a SetHandleInformation failure', () => { + it('reports a SetHandleInformation failure', () => { const { api } = pipeOkApi({ setHandleInformation: vi.fn(() => 0) }) let caught: unknown try { - spawnSandboxed(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + spawnPipedProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) } catch (error) { caught = error } @@ -211,7 +146,7 @@ describe('spawn pipe failures close their handles', () => { expect((caught as Win32Error).api).toBe('SetHandleInformation') }) - it('spawnSandboxed rejects NULL process/thread handles after a successful spawn', () => { + it('rejects NULL process/thread handles after a successful spawn', () => { const { api } = pipeOkApi({ createProcessAsUserW: vi.fn(( _token: unknown, _app: unknown, _cmd: unknown, _pa: unknown, _ta: unknown, @@ -221,17 +156,17 @@ describe('spawn pipe failures close their handles', () => { return 1 }), }) - expect(() => spawnSandboxed(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' })) + expect(() => spawnPipedProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token })) .toThrow(/null process\/thread handles/u) }) }) -describe('spawnSandboxedInherited failure paths', () => { +describe('spawnInheritedJobProcess failure paths', () => { const token = 1n as NativePtr /** The stub the inherited-happy path needs; overrides flip one call per test. */ - function inheritedApi(overrides: Partial = {}): { - api: Win32Bindings + function inheritedApi(overrides: Partial = {}): { + api: Win32ProcessBindings closed: bigint[] closeHandle: ReturnType } { @@ -255,11 +190,12 @@ describe('spawnSandboxedInherited failure paths', () => { }), assignProcessToJobObject: vi.fn(() => 1), resumeThread: vi.fn(() => 0), + terminateProcess: vi.fn(() => 1), getLastError: vi.fn(() => 5), closeHandle, formatMessageW: vi.fn(() => 0), ...overrides, - } as unknown as Win32Bindings + } as unknown as Win32ProcessBindings return { api, closed, closeHandle } } @@ -267,7 +203,7 @@ describe('spawnSandboxedInherited failure paths', () => { const { api, closeHandle } = inheritedApi({ getStdHandle: vi.fn(() => 0n as NativePtr) }) let caught: unknown try { - spawnSandboxedInherited(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + spawnInheritedJobProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) } catch (error) { caught = error } @@ -280,7 +216,7 @@ describe('spawnSandboxedInherited failure paths', () => { const { api } = inheritedApi({ setHandleInformation: vi.fn(() => 0) }) let caught: unknown try { - spawnSandboxedInherited(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + spawnInheritedJobProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) } catch (error) { caught = error } @@ -292,7 +228,7 @@ describe('spawnSandboxedInherited failure paths', () => { const { api, closeHandle } = inheritedApi({ createProcessAsUserW: vi.fn(() => 0) }) let caught: unknown try { - spawnSandboxedInherited(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + spawnInheritedJobProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) } catch (error) { caught = error } @@ -311,16 +247,51 @@ describe('spawnSandboxedInherited failure paths', () => { return 1 }), }) - expect(() => spawnSandboxedInherited(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' })) + expect(() => spawnInheritedJobProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token })) .toThrow(/null process\/thread handles/u) expect(closeHandle).toHaveBeenCalledWith(100n) }) + it('terminates the suspended child before closing handles when Job assignment fails', () => { + const terminateProcess = vi.fn(() => 1) + const { api, closeHandle, closed } = inheritedApi({ + assignProcessToJobObject: vi.fn(() => 0), + terminateProcess, + }) + let caught: unknown + try { + spawnInheritedJobProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) + } catch (error) { + caught = error + } + expect(caught).toMatchObject({ api: 'AssignProcessToJobObject', win32Code: 5 }) + expect(terminateProcess).toHaveBeenCalledWith(200n, 1) + expect(closeHandle).toHaveBeenCalledWith(201n) + expect(closeHandle).toHaveBeenCalledWith(200n) + expect(closeHandle).toHaveBeenCalledWith(100n) + expect(closed).toEqual([201n, 200n, 100n]) + }) + + it('closes the assigned child and Job when ResumeThread fails', () => { + const { api, closeHandle, closed } = inheritedApi({ resumeThread: vi.fn(() => 0xFFFFFFFF) }) + let caught: unknown + try { + spawnInheritedJobProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) + } catch (error) { + caught = error + } + expect(caught).toMatchObject({ api: 'ResumeThread', win32Code: 5 }) + expect(closeHandle).toHaveBeenCalledWith(201n) + expect(closeHandle).toHaveBeenCalledWith(200n) + expect(closeHandle).toHaveBeenCalledWith(100n) + expect(closed).toEqual([201n, 200n, 100n]) + }) + it('closes the job and reports when SetInformationJobObject fails', () => { const { api, closeHandle } = inheritedApi({ setInformationJobObject: vi.fn(() => 0) }) let caught: unknown try { - spawnSandboxedInherited(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + spawnInheritedJobProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) } catch (error) { caught = error } @@ -333,7 +304,7 @@ describe('spawnSandboxedInherited failure paths', () => { const { api } = inheritedApi({ createJobObjectW: vi.fn(() => 0n as NativePtr) }) let caught: unknown try { - spawnSandboxedInherited(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + spawnInheritedJobProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) } catch (error) { caught = error } @@ -342,8 +313,10 @@ describe('spawnSandboxedInherited failure paths', () => { }) it('returns the pid, process handle, and kill-on-close job when every call succeeds', () => { - const { api, closeHandle } = inheritedApi() - const spawned = spawnSandboxedInherited(api, token, { command: 'probe.exe', args: [], cwd: 'C:\\' }) + const assignProcessToJobObject = vi.fn(() => 1) + const resumeThread = vi.fn(() => 0) + const { api, closeHandle } = inheritedApi({ assignProcessToJobObject, resumeThread }) + const spawned = spawnInheritedJobProcess(api, { command: 'probe.exe', args: [], cwd: 'C:\\', token }) expect(spawned.pid).toBe(1234) expect(spawned.process).toBe(200n) expect(spawned.job).toBe(100n) @@ -351,6 +324,8 @@ describe('spawnSandboxedInherited failure paths', () => { expect(closeHandle).toHaveBeenCalledWith(201n) expect(closeHandle).not.toHaveBeenCalledWith(200n) expect(closeHandle).not.toHaveBeenCalledWith(100n) + expect(assignProcessToJobObject).toHaveBeenCalledWith(100n, 200n) + expect(resumeThread).toHaveBeenCalledWith(201n) }) }) @@ -362,7 +337,7 @@ describe('drainPipe', () => { getLastError: vi.fn(() => abi.ERROR_NO_DATA), closeHandle, formatMessageW: vi.fn(() => 0), - } as unknown as Win32Bindings + } as unknown as Win32ProcessBindings return drainPipe(api, 30n as NativePtr).then((buffer) => { expect(buffer.length).toBe(0) expect(closeHandle).toHaveBeenCalledWith(30n) @@ -370,13 +345,15 @@ describe('drainPipe', () => { }) it('reports a PeekNamedPipe failure that is not a clean EOF', () => { + const closeHandle = vi.fn(() => 1) const api = { peekNamedPipe: vi.fn(() => 0), getLastError: vi.fn(() => 5), - closeHandle: vi.fn(() => 1), + closeHandle, formatMessageW: vi.fn(() => 0), - } as unknown as Win32Bindings + } as unknown as Win32ProcessBindings return expect(drainPipe(api, 30n as NativePtr)).rejects.toMatchObject({ api: 'PeekNamedPipe' }) + .then(() => { expect(closeHandle).toHaveBeenCalledWith(30n) }) }) it('reports a ReadFile failure after data was reported available', () => { @@ -389,7 +366,7 @@ describe('drainPipe', () => { getLastError: vi.fn(() => 5), closeHandle: vi.fn(() => 1), formatMessageW: vi.fn(() => 0), - } as unknown as Win32Bindings + } as unknown as Win32ProcessBindings return expect(drainPipe(api, 30n as NativePtr)).rejects.toMatchObject({ api: 'ReadFile' }) }) @@ -410,31 +387,37 @@ describe('drainPipe', () => { getLastError: vi.fn(() => abi.ERROR_BROKEN_PIPE), closeHandle: vi.fn(() => 1), formatMessageW: vi.fn(() => 0), - } as unknown as Win32Bindings + } as unknown as Win32ProcessBindings return drainPipe(api, 30n as NativePtr).then((buffer) => { expect(buffer.toString('utf8')).toBe('ab') }) }) }) -describe('waitForExit', () => { +describe('waitForProcessExit', () => { it('reports a WaitForSingleObject failure', () => { + const closeHandle = vi.fn(() => 1) const api = { waitForSingleObject: vi.fn(() => 0xFFFFFFFF), getLastError: vi.fn(() => 5), + closeHandle, formatMessageW: vi.fn(() => 0), - } as unknown as Win32Bindings - expect(() => waitForExit(api, 200n as NativePtr)).toThrow(Win32Error) + } as unknown as Win32ProcessBindings + expect(() => waitForProcessExit(api, 200n as NativePtr)).toThrow(Win32Error) + expect(closeHandle).toHaveBeenCalledWith(200n) }) it('reports a GetExitCodeProcess failure', () => { + const closeHandle = vi.fn(() => 1) const api = { waitForSingleObject: vi.fn(() => 0), getExitCodeProcess: vi.fn(() => 0), getLastError: vi.fn(() => 5), + closeHandle, formatMessageW: vi.fn(() => 0), - } as unknown as Win32Bindings - expect(() => waitForExit(api, 200n as NativePtr)).toThrow(Win32Error) + } as unknown as Win32ProcessBindings + expect(() => waitForProcessExit(api, 200n as NativePtr)).toThrow(Win32Error) + expect(closeHandle).toHaveBeenCalledWith(200n) }) it('returns the exit code and closes the process handle', () => { @@ -447,8 +430,8 @@ describe('waitForExit', () => { }), closeHandle, formatMessageW: vi.fn(() => 0), - } as unknown as Win32Bindings - expect(waitForExit(api, 200n as NativePtr)).toBe(42) + } as unknown as Win32ProcessBindings + expect(waitForProcessExit(api, 200n as NativePtr)).toBe(42) expect(closeHandle).toHaveBeenCalledWith(200n) }) }) diff --git a/packages/subprocess/win32-process/tests/process.spec.ts b/packages/subprocess/win32-process/tests/process.spec.ts new file mode 100644 index 0000000000..85835a659c --- /dev/null +++ b/packages/subprocess/win32-process/tests/process.spec.ts @@ -0,0 +1,244 @@ +import koffi from 'koffi' +import { describe, expect, it, vi } from 'vitest' +import { + Win32Error, + drainPipe, + spawnInheritedJobProcess, + spawnPipedProcess, +} from '../src/index.ts' +import { CREATE_SUSPENDED } from '../src/abi.ts' +import { PROCESS_INFORMATION } from '../src/ffi.ts' +import type { NativePtr, Win32ProcessBindings } from '../src/index.ts' + +const PVOID = koffi.pointer('void') + +function inheritedApi(overrides: Partial = {}): { + api: Win32ProcessBindings + events: string[] + createProcessAsUserW: ReturnType + assignProcessToJobObject: ReturnType + resumeThread: ReturnType +} { + const events: string[] = [] + const createProcessAsUserWImpl: Win32ProcessBindings['createProcessAsUserW'] = + overrides.createProcessAsUserW + ?? ((_token, _app, _line, _pa, _ta, _inherit, _flags, _env, _cwd, _startup, info) => { + events.push('create') + koffi.encode(info, PROCESS_INFORMATION, { + hProcess: 60n, + hThread: 61n, + dwProcessId: 1234, + dwThreadId: 5678, + }) + return 1 + }) + const createProcessAsUserW = vi.fn(createProcessAsUserWImpl) + const assignProcessToJobObject = vi.fn(() => { + events.push('assign') + return 1 + }) + const resumeThread = vi.fn(() => { + events.push('resume') + return 1 + }) + const api = { + createJobObjectW: vi.fn(() => 50n), + setInformationJobObject: vi.fn(() => 1), + getStdHandle: vi.fn((selector: number) => BigInt(100 - selector)), + setHandleInformation: vi.fn((_handle: NativePtr, _mask: number, flags: number) => { + events.push(flags === 0 ? 'restore' : 'inherit') + return 1 + }), + assignProcessToJobObject, + resumeThread, + terminateProcess: vi.fn(() => 1), + closeHandle: vi.fn((handle: NativePtr) => { events.push(`close:${handle}`); return 1 }), + getLastError: vi.fn(() => 5), + formatMessageW: vi.fn(() => 0), + ...overrides, + createProcessAsUserW, + } as unknown as Win32ProcessBindings + return { + api, + events, + createProcessAsUserW, + assignProcessToJobObject, + resumeThread, + } +} + +describe('spawnInheritedJobProcess', () => { + const token = 70n as NativePtr + + it('creates suspended, assigns the Job, then resumes the restricted child', () => { + const { + api, + events, + createProcessAsUserW, + assignProcessToJobObject, + resumeThread, + } = inheritedApi() + const child = spawnInheritedJobProcess(api, { + command: 'cmd.exe', + args: ['/c', 'exit', '0'], + cwd: 'C:\\work', + token, + }) + expect(child).toEqual({ pid: 1234, process: 60n, job: 50n }) + expect(events.indexOf('create')).toBeLessThan(events.indexOf('assign')) + expect(events.indexOf('assign')).toBeLessThan(events.indexOf('resume')) + expect(assignProcessToJobObject).toHaveBeenCalledWith(50n, 60n) + expect(resumeThread).toHaveBeenCalledWith(61n) + expect(createProcessAsUserW).toHaveBeenCalledWith( + token, + null, + 'cmd.exe /c exit 0', + null, + null, + 1, + CREATE_SUSPENDED, + null, + 'C:\\work', + expect.anything(), + expect.anything(), + ) + }) + + it('restores already-enabled stdio and closes the Job when inheritance setup fails', () => { + let calls = 0 + const closeHandle = vi.fn((_handle: NativePtr) => 1) + const setHandleInformation = vi.fn((_handle: NativePtr, _mask: number, flags: number) => { + if (flags === 0) return 1 + calls += 1 + return calls === 2 ? 0 : 1 + }) + const { api } = inheritedApi({ closeHandle, setHandleInformation }) + expect(() => spawnInheritedJobProcess(api, { + command: 'cmd.exe', + args: [], + cwd: 'C:\\work', + token, + })).toThrow(Win32Error) + expect(setHandleInformation).toHaveBeenCalledWith(expect.anything(), 1, 0) + expect(closeHandle).toHaveBeenCalledWith(50n) + }) + + it('captures a GetStdHandle error before Job cleanup changes last-error', () => { + let lastError = 123 + const { api } = inheritedApi({ + getStdHandle: vi.fn(() => 0n as NativePtr), + getLastError: vi.fn(() => lastError), + closeHandle: vi.fn(() => { lastError = 999; return 1 }), + }) + let caught: unknown + try { + spawnInheritedJobProcess(api, { command: 'cmd.exe', args: [], cwd: 'C:\\work', token }) + } catch (error) { + caught = error + } + expect(caught).toMatchObject({ api: 'GetStdHandle', win32Code: 123 }) + }) + + it('captures a CreateProcess error before inheritance restoration changes last-error', () => { + let lastError = 87 + const { api } = inheritedApi({ + createProcessAsUserW: vi.fn(() => 0), + getLastError: vi.fn(() => lastError), + setHandleInformation: vi.fn((_handle, _mask, flags) => { + if (flags === 0) lastError = 999 + return 1 + }), + closeHandle: vi.fn(() => { lastError = 998; return 1 }), + }) + let caught: unknown + try { + spawnInheritedJobProcess(api, { command: 'cmd.exe', args: [], cwd: 'C:\\work', token }) + } catch (error) { + caught = error + } + expect(caught).toMatchObject({ api: 'CreateProcessAsUserW', win32Code: 87 }) + }) + + it('terminates the suspended process and closes the Job when CreateProcessAsUserW returns a null thread handle', () => { + const closeHandle = vi.fn((_handle: NativePtr) => 1) + const terminateProcess = vi.fn(() => 1) + const { api } = inheritedApi({ + closeHandle, + terminateProcess, + createProcessAsUserW: vi.fn((_token, _app, _line, _pa, _ta, _inherit, _flags, _env, _cwd, _startup, info) => { + koffi.encode(info, PROCESS_INFORMATION, { + hProcess: 60n, + hThread: 0n, + dwProcessId: 1234, + dwThreadId: 0, + }) + return 1 + }), + }) + expect(() => spawnInheritedJobProcess(api, { + command: 'cmd.exe', + args: [], + cwd: 'C:\\work', + token, + })).toThrow('null process/thread handles') + expect(terminateProcess).toHaveBeenCalledWith(60n, 1) + expect(closeHandle).toHaveBeenCalledWith(50n) + expect(closeHandle).toHaveBeenCalledWith(60n) + }) + +}) + +describe('wait and pipe cleanup', () => { + const token = 70n as NativePtr + + it('waits when a pipe is temporarily empty before observing EOF', async () => { + const closeHandle = vi.fn(() => 1) + let peeks = 0 + const api = { + peekNamedPipe: vi.fn((_handle, _buffer, _size, _read, available) => { + peeks += 1 + if (peeks === 1) { + koffi.encode(available, 'uint32', 0) + return 1 + } + return 0 + }), + getLastError: vi.fn(() => 109), + closeHandle, + } as unknown as Win32ProcessBindings + await expect(drainPipe(api, 80n as NativePtr)).resolves.toEqual(Buffer.alloc(0)) + expect(closeHandle).toHaveBeenCalledWith(80n) + }) + + it('terminates a piped child when CreateProcess returns a null thread handle', () => { + let nextPipe = 10n + const terminateProcess = vi.fn(() => 1) + const closeHandle = vi.fn(() => 1) + const api = { + createPipe: vi.fn((readSlot, writeSlot) => { + koffi.encode(readSlot, PVOID, nextPipe++) + koffi.encode(writeSlot, PVOID, nextPipe++) + return 1 + }), + setHandleInformation: vi.fn(() => 1), + createProcessAsUserW: vi.fn((_token, _app, _line, _pa, _ta, _inherit, _flags, _env, _cwd, _startup, info) => { + koffi.encode(info, PROCESS_INFORMATION, { + hProcess: 60n, + hThread: 0n, + dwProcessId: 1234, + dwThreadId: 0, + }) + return 1 + }), + terminateProcess, + closeHandle, + } as unknown as Win32ProcessBindings + expect(() => spawnPipedProcess(api, { + command: 'cmd.exe', + args: [], + cwd: 'C:\\work', + token, + })).toThrow('null process/thread handles') + expect(terminateProcess).toHaveBeenCalledWith(60n, 1) + }) +}) diff --git a/packages/subprocess/win32-process/tests/quote.spec.ts b/packages/subprocess/win32-process/tests/quote.spec.ts new file mode 100644 index 0000000000..63dcc7bc77 --- /dev/null +++ b/packages/subprocess/win32-process/tests/quote.spec.ts @@ -0,0 +1,65 @@ +import { describe, expect, it } from 'vitest' +import { buildCommandLine, quoteArg } from '../src/process.ts' + +const isWin32 = process.platform === 'win32' + +const cases: Array<[string, string]> = [ + ['', '""'], + ['a', 'a'], + ['a b', '"a b"'], + ['a"b', '"a\\"b"'], + ['a\\b', 'a\\b'], + ['a b\\', '"a b\\\\"'], + ['a b\\\\', '"a b\\\\\\\\"'], + ['a\\\\"b', '"a\\\\\\\\\\"b"'], +] + +describe('quoteArg', () => { + it.each(cases)('quotes %j as %j', (input, expected) => { + expect(quoteArg(input)).toBe(expected) + }) + + it('builds one CreateProcess command line without shell interpretation', () => { + expect(buildCommandLine('C:\\Program Files\\tool.exe', ['a b', 'c'])).toBe( + '"C:\\Program Files\\tool.exe" "a b" c', + ) + }) +}) + +describe.skipIf(!isWin32)('CommandLineToArgvW round-trip', () => { + it('parses the shared command line back to the original argv', async () => { + const { default: koffi } = await import('koffi') + const PVOID = koffi.pointer('void') + const shell32 = koffi.load('shell32.dll') + const kernel32 = koffi.load('kernel32.dll') + const commandLineToArgvW = shell32.func( + '__stdcall', + 'CommandLineToArgvW', + PVOID, + ['str16', koffi.pointer('int')], + ) + const lstrcpynW = kernel32.func('__stdcall', 'lstrcpynW', PVOID, [PVOID, PVOID, 'int']) + const lstrlenW = kernel32.func('__stdcall', 'lstrlenW', 'int', [PVOID]) + const localFree = kernel32.func('__stdcall', 'LocalFree', PVOID, [PVOID]) + const parse = (commandLine: string): string[] => { + const countSlot = koffi.alloc('int', 1) as unknown + const argvBlock = commandLineToArgvW(commandLine, countSlot) as unknown + try { + if (argvBlock === null) throw new Error('CommandLineToArgvW returned NULL') + const count = koffi.decode(countSlot, 0, 'int') as number + const table = Buffer.from(koffi.view(argvBlock, count * 8)) + return Array.from({ length: count }, (_, index) => { + const stringAddress = table.readBigUInt64LE(index * 8) + const copied = Buffer.alloc(2048) + lstrcpynW(copied, stringAddress, copied.length / 2) + const length = lstrlenW(copied) as number + return copied.subarray(0, length * 2).toString('utf16le') + }) + } finally { + localFree(argvBlock) + } + } + const argv = ['', 'a', 'a b', 'a"b', 'a\\b', 'a b\\', 'a b\\\\', 'a\\\\"b'] + expect(parse(buildCommandLine('prog.exe', argv))).toEqual(['prog.exe', ...argv]) + }) +}) diff --git a/packages/subprocess/win32-process/tsconfig.json b/packages/subprocess/win32-process/tsconfig.json new file mode 100644 index 0000000000..730993dc97 --- /dev/null +++ b/packages/subprocess/win32-process/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../runtime-diagnostics/invariants" + } + ] +} diff --git a/packages/subprocess/win32-process/verify/abi-probe.cpp b/packages/subprocess/win32-process/verify/abi-probe.cpp new file mode 100644 index 0000000000..1c9480105d --- /dev/null +++ b/packages/subprocess/win32-process/verify/abi-probe.cpp @@ -0,0 +1,48 @@ +#include +#include +#include + +#define P(expr) printf("%-52s = %llu\n", #expr, (unsigned long long)(expr)) + +int wmain() +{ + P(sizeof(void*)); + P(sizeof(HANDLE)); + P(sizeof(STARTUPINFOW)); + P(offsetof(STARTUPINFOW, dwFlags)); + P(offsetof(STARTUPINFOW, hStdInput)); + P(offsetof(STARTUPINFOW, hStdOutput)); + P(offsetof(STARTUPINFOW, hStdError)); + P(sizeof(PROCESS_INFORMATION)); + P(offsetof(PROCESS_INFORMATION, hProcess)); + P(offsetof(PROCESS_INFORMATION, hThread)); + P(offsetof(PROCESS_INFORMATION, dwProcessId)); + P(CREATE_SUSPENDED); + P(STARTF_USESTDHANDLES); + P(HANDLE_FLAG_INHERIT); + P(INFINITE); + P(STD_INPUT_HANDLE); + P(STD_OUTPUT_HANDLE); + P(STD_ERROR_HANDLE); + P(FORMAT_MESSAGE_FROM_SYSTEM); + P(FORMAT_MESSAGE_IGNORE_INSERTS); + P(ERROR_INSUFFICIENT_BUFFER); + P(ERROR_BROKEN_PIPE); + P(ERROR_NO_DATA); + P(sizeof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION)); + P(offsetof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION, BasicLimitInformation) + offsetof(JOBOBJECT_BASIC_LIMIT_INFORMATION, LimitFlags)); + P((int)JobObjectExtendedLimitInformation); + P(JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE); + + static_assert(sizeof(STARTUPINFOW) == 104, "STARTUPINFOW size"); + static_assert(sizeof(PROCESS_INFORMATION) == 24, "PROCESS_INFORMATION size"); + static_assert(CREATE_SUSPENDED == 0x4, "suspended process flag"); + static_assert(STARTF_USESTDHANDLES == 0x100, "std handles flag"); + static_assert(HANDLE_FLAG_INHERIT == 0x1, "inherit flag"); + static_assert(sizeof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION) == 144, "job extended limit size"); + static_assert(offsetof(JOBOBJECT_EXTENDED_LIMIT_INFORMATION, BasicLimitInformation) + offsetof(JOBOBJECT_BASIC_LIMIT_INFORMATION, LimitFlags) == 16, "job LimitFlags offset"); + static_assert(JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE == 0x2000, "kill on job close flag"); + static_assert(JobObjectExtendedLimitInformation == 9, "extended limit class"); + printf("\nstatic_asserts passed\n"); + return 0; +} diff --git a/packages/terminal/README.i18n.yaml b/packages/terminal/README.i18n.yaml index 428fa2ac60..171ab04444 100644 --- a/packages/terminal/README.i18n.yaml +++ b/packages/terminal/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/terminal/README.md -README.md: 6204eb4de27bc2361f2a49849865f65729dce12c -README.zh.md: 8b7d704440cb8f9082228fc1265500ebe41e1b58 +README.md: faa391b5a61b1e752e40e94cf10cd0807a17afb0 +README.zh.md: be26fff469d3adf95455912d7e5285fe09ace5a6 diff --git a/packages/terminal/README.md b/packages/terminal/README.md index 6204eb4de2..faa391b5a6 100644 --- a/packages/terminal/README.md +++ b/packages/terminal/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -`PTY` stands for **Pseudo-Terminal**(伪终端). This capability provides persistent, owner-scoped terminal sessions for workflows that require state across tool calls or interactive stdin. PTY complements the one-shot bash and filesystem tools; it does not replace their stronger per-operation contracts. +`PTY` stands for **pseudo-terminal**. This capability provides persistent, owner-scoped terminal sessions for workflows that require state across tool calls or interactive stdin. PTY complements the one-shot bash and filesystem tools; it does not replace their stronger per-operation contracts. | Package | Role | ctx key | |---|---|---| @@ -10,6 +10,4 @@ English | [中文](README.zh.md) | `terminal-bash` (`@deepseek-ai/dsh-terminal-bash`) | Shell backend over `ctx.subprocess.spawnTerminal`: readiness detection, bounded terminal state, sandbox policy, and session operations | registers on `ctx.terminals` | | `tool-terminal` (`@deepseek-ai/dsh-tool-terminal`) | Six model-facing tools and generic task integration for background sends | registers on `ctx.tools` | -The design and deferred boundaries live in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md). - -The subsystem reference — ids, backend/session contracts, send readiness, bounded reads — is [docs/subsystems/terminal.md](../../docs/subsystems/terminal.md); design and deferred boundaries in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md). +The [terminal subsystem reference](../../docs/subsystems/terminal.md) owns ids, backend and session contracts, send readiness, and bounded reads. The [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) owns the rationale and deferred boundaries. diff --git a/packages/terminal/README.zh.md b/packages/terminal/README.zh.md index 8b7d704440..be26fff469 100644 --- a/packages/terminal/README.zh.md +++ b/packages/terminal/README.zh.md @@ -2,14 +2,12 @@ [English](README.md) | 中文 -`PTY` 的全称是 **Pseudo-Terminal(伪终端)**。这项能力提供持久且限定所有者范围的终端会话,适用于需要跨工具调用保留状态或使用交互式 stdin 的工作流。PTY 是单次 bash 与文件系统工具的补充,不会取代后两者更严格的逐操作约定。 +`PTY` 的全称是 **pseudo-terminal(伪终端)**。这项能力提供持久且限定所有者范围的终端会话,适用于需要跨工具调用保留状态或使用交互式 stdin 的工作流。PTY 是单次 bash 与文件系统工具的补充,不会取代后两者更严格的逐操作约定。 | 包 | 职责 | ctx 键 | |---|---|---| -| [`pty`](terminal/README.md)(`@deepseek-ai/dsh-terminal`) | 后端注册表、品牌化 id、精确的 Agent 所有权、会话操作与等待完成的清理 | `ctx.terminals` | +| [`pty`](terminal/README.zh.md)(`@deepseek-ai/dsh-terminal`) | 后端注册表、品牌化 id、精确的 Agent 所有权、会话操作与等待完成的清理 | `ctx.terminals` | | `terminal-bash`(`@deepseek-ai/dsh-terminal-bash`) | `ctx.subprocess.spawnTerminal` 之上的 shell 后端:就绪检测、有界终端状态、沙箱策略与会话操作 | 注册到 `ctx.terminals` | | `tool-terminal`(`@deepseek-ai/dsh-tool-terminal`) | 6 个面向模型的工具,并为后台发送集成通用任务 | 注册到 `ctx.tools` | -设计与暂缓边界记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中。 - -子系统参考——id、后端/会话约定、发送就绪、有界读取——见 [docs/subsystems/terminal.md](../../docs/subsystems/terminal.md);设计与暂缓边界见[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md)。 +[终端子系统参考](../../docs/subsystems/terminal.zh.md)负责记录 id、后端与会话约定、发送就绪条件和有界读取。[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md)负责记录设计理由与暂缓边界。 diff --git a/packages/terminal/terminal-bash/README.i18n.yaml b/packages/terminal/terminal-bash/README.i18n.yaml index 231dac9190..d6e544137a 100644 --- a/packages/terminal/terminal-bash/README.i18n.yaml +++ b/packages/terminal/terminal-bash/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/terminal/terminal-bash/README.md -README.md: 36e725fd4ce86be09755768a9e21ccb66d025251 -README.zh.md: 080f45eb03dfdeece91269a3253aeb3fb280864d +README.md: 2f3f59b1acb88ff9905e78e7fc8d0d9fbcdbf0ba +README.zh.md: f3daa0a3bc9c160236ad19b35589778d99d48b36 diff --git a/packages/terminal/terminal-bash/README.md b/packages/terminal/terminal-bash/README.md index 36e725fd4c..2f3f59b1ac 100644 --- a/packages/terminal/terminal-bash/README.md +++ b/packages/terminal/terminal-bash/README.md @@ -8,7 +8,9 @@ Persistent shell backend for `ctx.terminals` over `ctx.subprocess.spawnTerminal` The plugin injects `pty`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly without requiring a sandbox provider; confined modes require a same-world `ctx.sandbox` and wrap the exact shell argv through it, failing before spawn when none is mounted. At spawn, one `ctx.sandboxPolicy.resolve({ session })` call supplies both the effective mode and the session workspace root; the same root is the default shell cwd when the caller omits one. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following an earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `TerminalBackendCleanupError` separately preserves a cleanup failure. The caller's signal is forwarded for terminal allocation and readiness initialization; after publication the handle owns its lifetime. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; malformed UTF-8 terminal output uses replacement characters, and a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +`shellDialect` selects the shell stack (`bash` default, `pwsh`): it picks the default `shellPath`/`shellArgs` (bash `--noprofile --norc -i`; pwsh `-NoLogo -NoProfile` through the shared `dsh-pwsh-local` resolver) and the startup contract. The bash dialect installs its prompt through the environment (`PS1` plus an OSC `133;D;`-terminated `PROMPT_COMMAND`). pwsh cannot install a prompt from the environment, so the backend writes a `prompt` function through the session and waits until the controlled prompt is actually visible — looping over follow-up sends because the pwsh banner-to-prompt gap can outlast the silence bound — while its environment drops the bash-only markers and adds `NO_COLOR`. That first send also prefixes the shared `dsh-pwsh-local` encoding preamble, pinning `[Console]::OutputEncoding` and `$OutputEncoding` to UTF-8 before anything runs: the session decode path reads PTY bytes as UTF-8, and an un-pinned console writes its host code page for non-ASCII output. Both dialects emit the same BEL-terminated OSC marker, so the readiness machinery and consumers are dialect-agnostic. + +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following an earlier prompt therefore cannot settle the current send. The controlled `PROMPT_COMMAND` re-asserts that `PS1` before every prompt, so an in-shell prompt override cannot degrade later sends to silence readiness. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `TerminalBackendCleanupError` separately preserves a cleanup failure. The caller's signal is forwarded for terminal allocation and readiness initialization; after publication the handle owns its lifetime. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; malformed UTF-8 terminal output uses replacement characters, and a trailing carriage return is carried across callbacks so split CRLF becomes one newline. Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. If a provider write is already in flight, signalling waits for it to settle; a rejected write sends no signal. The canceled send retains its slot until the write and foreground signalling settle, so a successor cannot receive either late bytes or that signal. A provider write or signal that never settles therefore retains the slot indefinitely; closing the session (`terminal_close`) is the recovery. The absolute deadline remains armed while cancellation waits. A signal failure is a terminal transport failure and rejects the active send. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close rejects new public signals, stops readiness polling, and awaits the handle's provider-owned complete-session termination before settling the active send as `session_exit`. @@ -31,6 +33,7 @@ A standing-policy change appends an owner-rendered superseding runtime-context s ## Known Limitations and Deferred Work - Line-oriented output is normalized; full-screen alternate-buffer interaction is unsupported. -- Exact stdin-wait detection depends on the mounted subprocess provider; providers that cannot prove it use prompt-marker and silence/timeout readiness. +- Exact stdin-wait detection depends on the mounted subprocess provider; providers that cannot prove it use prompt-marker and silence/timeout readiness. Windows is such a provider: the shell pid is the pseudo foreground group and there is no exact stdin-wait tier, so a marker-less child settles on the silence bound. +- The pwsh bootstrap writes through `[Console]::` (the UTF-8 encoding pin and the prompt function), which the Windows ACL sandbox's read-only mode (ConstrainedLanguage) may deny. The shell can still settle through the controlled printable prompt and silence tier, but marker readiness is unavailable and non-ASCII output may follow the host code page. - Cleanup guarantees are those of `SubprocessTerminalHandle`; provider-specific gaps belong to that implementation's contract rather than this PTY consumer. - Sessions do not survive harness process exit. diff --git a/packages/terminal/terminal-bash/README.zh.md b/packages/terminal/terminal-bash/README.zh.md index 080f45eb03..f3daa0a3bc 100644 --- a/packages/terminal/terminal-bash/README.zh.md +++ b/packages/terminal/terminal-bash/README.zh.md @@ -8,7 +8,9 @@ 该插件注入 `pty`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 无需沙箱提供方即可直接启动 shell;受限模式要求同一执行世界中存在 `ctx.sandbox`,并通过它包装确切的 shell argv,未挂载时会在 spawn 前失败。spawn 时,一次 `ctx.sandboxPolicy.resolve({ session })` 调用会同时给出实际模式与会话工作区根目录;调用方省略 cwd 时,同一根目录也是 shell 的默认 cwd。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建完成并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最新自有标记之后的可打印尾部与受控 `PS1` 完全相等,标记才算就绪;即使 OSC 标记和提示符被拆到多个数据回调中也一样。因此,较早提示符之后的回显输入或输出无法使当前 send 完成。提供方写入前收集的提示符与静默证据,包括写入前前台检查仍在等待时收集的证据,都会在写入边界丢弃。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法一直抑制推断空闲就绪直至绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`TerminalBackendCleanupError` 会单独保留清理失败。调用方的 signal 会转发给终端分配与就绪初始化;发布后,句柄负责其生命周期。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。格式错误的 UTF-8 终端输出使用替换字符;末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +`shellDialect` 选择 shell 栈(默认 `bash`,或 `pwsh`):它决定默认的 `shellPath`/`shellArgs`(bash 为 `--noprofile --norc -i`;pwsh 经共享的 `dsh-pwsh-local` 解析器得到 `-NoLogo -NoProfile`)与启动契约。bash 方言通过环境安装提示符(`PS1` 加 OSC `133;D;` 终结的 `PROMPT_COMMAND`)。pwsh 无法从环境安装提示符,因此后端通过会话写入 `prompt` 函数,并等待受控提示符真正可见——因为 pwsh 从横幅到提示符的间隙可能超过静默上限,所以会在后续 send 上循环等待;同时其环境去掉 bash 专属标记并加 `NO_COLOR`。同一条首发送还会带上共享的 `dsh-pwsh-local` 编码前缀,在一切运行之前把 `[Console]::OutputEncoding` 与 `$OutputEncoding` 钉为 UTF-8:会话解码路径按 UTF-8 读取 PTY 字节,未钉住编码的控制台会以宿主代码页输出非 ASCII 内容。两种方言发出相同的 BEL 终结 OSC 标记,因此就绪机制与消费方与方言无关。 + +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最新自有标记之后的可打印尾部与受控 `PS1` 完全相等,标记才算就绪;即使 OSC 标记和提示符被拆到多个数据回调中也一样。因此,较早提示符之后的回显输入或输出无法使当前 send 完成。受控 `PROMPT_COMMAND` 会在每次输出提示符前重新设定该 `PS1`,因此在 shell 内覆盖提示符不会使后续 send 退化到静默就绪。提供方写入前收集的提示符与静默证据,包括写入前前台检查仍在等待时收集的证据,都会在写入边界丢弃。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法一直抑制推断空闲就绪直至绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`TerminalBackendCleanupError` 会单独保留清理失败。调用方的 signal 会转发给终端分配与就绪初始化;发布后,句柄负责其生命周期。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。格式错误的 UTF-8 终端输出使用替换字符;末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 取消发送时,系统会先把排队输入标记为已取消,再要求终端句柄向当前前台进程组发送真正的 `SIGINT`;异步写入前检查即使随后结算,也无法执行该输入。如果提供方写入已在途,信号发送会等待其结算;写入被拒绝时不会发送信号。已取消的 send 会保留其位置,直到写入与前台信号发送都结算,因此后继 send 不会收到延迟字节或该信号。因此,永不结算的提供方写入或信号会无限期保留该位置;恢复手段是关闭会话(`terminal_close`)。取消等待期间,绝对 deadline 仍保持启用。信号发送失败是终端传输失败,会拒绝活跃 send。取消绝不会通过写入 `\x03` 模拟中断,因此,即使程序运行在 raw 模式下,也仍可取消。关闭操作会拒绝新的公开信号、停止就绪轮询,并等待由句柄提供方负责的完整会话终止,然后才把活跃 send 结算为 `session_exit`。 @@ -31,6 +33,7 @@ ## 已知限制与暂缓事项 - 输出按行规范化;不支持全屏备用缓冲区交互。 -- 精确 stdin 等待检测取决于已挂载的进程管理提供方;无法证明该状态的提供方使用提示符标记和静默/超时就绪机制。 +- 精确 stdin 等待检测取决于已挂载的进程管理提供方;无法证明该状态的提供方使用提示符标记和静默/超时就绪机制。Windows 正是这样的提供方:shell pid 是伪前台进程组,没有精确的 stdin-wait 档,因此无标记的子进程按静默上限结算。 +- pwsh 引导(UTF-8 编码钉与 `prompt` 函数)通过 `[Console]::` 写入,Windows ACL 沙箱的只读模式(ConstrainedLanguage)可能拒绝它。shell 仍可通过受控可打印提示符和静默档结算,但无法使用 marker 就绪,非 ASCII 输出也可能沿用宿主代码页。 - 清理保证以 `SubprocessTerminalHandle` 的保证为准;提供方特定的缺口属于该实现的约定,而非这个 PTY 消费方。 - harness 进程退出后,会话无法继续存在。 diff --git a/packages/terminal/terminal-bash/package.json b/packages/terminal/terminal-bash/package.json index cad0c60fba..cb3dccce08 100644 --- a/packages/terminal/terminal-bash/package.json +++ b/packages/terminal/terminal-bash/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-terminal-bash", "description": "Persistent shell PTY backend over the DeepSeek Harness subprocess terminal primitive", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -42,6 +42,7 @@ "@deepseek-ai/cordis": "workspace:^" }, "dependencies": { + "@deepseek-ai/dsh-pwsh-local": "workspace:^", "@deepseek-ai/schemastery": "workspace:^" }, "devDependencies": { diff --git a/packages/terminal/terminal-bash/src/config.ts b/packages/terminal/terminal-bash/src/config.ts index b4faadbbe5..848fd8bf9a 100644 --- a/packages/terminal/terminal-bash/src/config.ts +++ b/packages/terminal/terminal-bash/src/config.ts @@ -1,14 +1,20 @@ /** Validated configuration for the local PTY backend. */ import z from '@deepseek-ai/schemastery' +import { resolvePwshPath } from '@deepseek-ai/dsh-pwsh-local' + +/** One supported interactive shell dialect. */ +export type ShellDialect = 'bash' | 'pwsh' /** Public plugin configuration. */ export interface Config { /** Backend registry type (default: `shell`). */ backendType?: string - /** Interactive shell executable (default: `/bin/bash`). */ + /** Interactive shell dialect (default: `bash`); selects the argv/env/startup defaults. */ + shellDialect?: ShellDialect + /** Interactive shell executable (default per dialect: `/bin/bash`, or the resolved pwsh). */ shellPath?: string - /** Shell arguments (default: `--noprofile --norc -i`). */ + /** Shell arguments (default per dialect: bash `--noprofile --norc -i`, pwsh `-NoLogo -NoProfile`). */ shellArgs?: string[] /** Terminal rows. */ rows?: number @@ -37,14 +43,49 @@ export interface Config { disposeGraceMs?: number } -/** Configuration after Schemastery defaults. */ -export type ResolvedConfig = Required +/** Configuration after Schemastery defaults and dialect resolution. */ +export type ResolvedConfig = Omit, 'shellDialect' | 'shellPath' | 'shellArgs'> & { + shellDialect: ShellDialect + shellPath: string + shellArgs: string[] +} + +/** Bash dialect default executable. */ +export const DEFAULT_BASH_SHELL = '/bin/bash' +/** Bash dialect default arguments (interactive, profile-free). */ +export const DEFAULT_BASH_ARGS = ['--noprofile', '--norc', '-i'] +/** Pwsh dialect default arguments (interactive host, profile-free). */ +export const DEFAULT_PWSH_ARGS = ['-NoLogo', '-NoProfile'] + +/** + * Resolve the effective per-dialect shell specification. Defaulting is this + * explicit step: an unset or empty `shellPath`/`shellArgs` selects the + * dialect's defaults, while a non-empty explicit value always wins. + * (Schemastery materializes an absent optional array as `[]`, so emptiness — + * not just `undefined` — means "dialect default".) + * @param config - Schemastery-resolved plugin configuration. + * @returns the fully resolved configuration. + */ +export function resolveConfig(config: Config): ResolvedConfig { + const shellDialect = config.shellDialect ?? 'bash' + return { + ...(config as Required), + shellDialect, + shellPath: config.shellPath !== undefined && config.shellPath.length > 0 + ? config.shellPath + : (shellDialect === 'pwsh' ? resolvePwshPath() : DEFAULT_BASH_SHELL), + shellArgs: config.shellArgs !== undefined && config.shellArgs.length > 0 + ? config.shellArgs + : (shellDialect === 'pwsh' ? DEFAULT_PWSH_ARGS : DEFAULT_BASH_ARGS), + } +} /** Schemastery config exposed by the plugin. */ export const Config: z = z.object({ backendType: z.string().default('shell'), - shellPath: z.string().default('/bin/bash'), - shellArgs: z.array(z.string()).default(['--noprofile', '--norc', '-i']), + shellDialect: z.union(['bash', 'pwsh'] as const).default('bash'), + shellPath: z.string().required(false), + shellArgs: z.array(z.string()).required(false), rows: z.number().default(40), cols: z.number().default(160), scrollbackLines: z.number().default(10_000), @@ -59,7 +100,7 @@ export const Config: z = z.object({ }) /** - * Assert every numeric config field is a positive safe integer and bounds compose. + * Assert every effective numeric config field is a positive safe integer and bounds compose. * @param config - Schemastery-resolved plugin configuration. * @returns Narrows the input to the fully resolved configuration. */ diff --git a/packages/terminal/terminal-bash/src/index.ts b/packages/terminal/terminal-bash/src/index.ts index 82e864e7ab..79296959ad 100644 --- a/packages/terminal/terminal-bash/src/index.ts +++ b/packages/terminal/terminal-bash/src/index.ts @@ -12,7 +12,8 @@ import type { TerminalBackend, TerminalBackendSpawnSpec } from '@deepseek-ai/dsh import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' import type { SandboxExecutionPolicy } from '@deepseek-ai/dsh-sandbox' import { effectiveSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' -import { type Config, type ResolvedConfig, validateConfig } from './config.ts' +import { ENCODING_PREAMBLE } from '@deepseek-ai/dsh-pwsh-local' +import { type Config, type ResolvedConfig, resolveConfig, type ShellDialect, validateConfig } from './config.ts' import { LocalPtySession } from './session.ts' import { CONTROLLED_PROMPT } from './sanitize.ts' @@ -52,22 +53,42 @@ function ensureSandboxModeFence(ctx: Context, owner: Agent): void { }, { global: true }) } -function childEnvironment(spec: TerminalBackendSpawnSpec): Record { +function childEnvironment(spec: TerminalBackendSpawnSpec, dialect: ShellDialect): Record { // The subprocess provider supplies its own scrubbed ambient base; these are // deliberate terminal-specific overrides layered after it. - return { + const common = { TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', - PS1: CONTROLLED_PROMPT, - PROMPT_COMMAND: 'printf "\\033]133;D;%s\\007" "$?"', - BASH_SILENCE_DEPRECATION_WARNING: '1', DSH_SHELL: '1', DSH_SESSION_ID: spec.owner.id, DSH_PTY_SESSION_ID: spec.sessionId, } + if (dialect === 'pwsh') { + // pwsh ignores PS1/PROMPT_COMMAND; its prompt is installed by the startup + // bootstrap instead, and NO_COLOR keeps the renderer quiet. + return { ...common, NO_COLOR: '1' } + } + return { + ...common, + PS1: CONTROLLED_PROMPT, + // Re-asserting PS1 after the marker keeps prompt readiness working when a + // command overwrote the shell variable: bash runs PROMPT_COMMAND before + // rendering each prompt, so an override never survives to the next prompt. + PROMPT_COMMAND: `printf "\\033]133;D;%s\\007" "$?"; PS1='${CONTROLLED_PROMPT}'`, + BASH_SILENCE_DEPRECATION_WARNING: '1', + } } +/** + * The pwsh prompt function that emits the shared OSC `133;D;` + BEL marker + * before every prompt, mirroring bash's PROMPT_COMMAND. `[char]27`/`[char]7` + * build the control bytes at runtime because raw ESC characters in submitted + * input are unreliable under PSReadLine. + */ +export const PWSH_PROMPT_SETUP = + "function prompt { [Console]::Write([char]27 + ']133;D;' + [int]$LASTEXITCODE + [char]7); '" + CONTROLLED_PROMPT + "' }" + function spawnArgv(ctx: Context, config: ResolvedConfig, policy: SandboxExecutionPolicy): string[] { const argv = [config.shellPath, ...config.shellArgs] if (policy.mode === 'danger-full-access') return argv @@ -82,9 +103,45 @@ function spawnArgv(ctx: Context, config: ResolvedConfig, policy: SandboxExecutio // TODO(pty-initialize-race-home): Fold this outer abort race into // LocalPtySession.initialize when the send-state consolidation lands; the // session already owns the send lifecycle the race protects. -async function initializeSession(session: LocalPtySession, signal?: AbortSignal): Promise { +async function startupSession( + session: LocalPtySession, + dialect: ShellDialect, + signal?: AbortSignal, +): Promise { + const start = async (): Promise => { + if (dialect === 'bash') { + await session.initialize(signal) + return + } + // pwsh cannot install its prompt from the environment: write the prompt + // function through the session and wait for the first marker prompt, + // which is also the readiness contract of the bash initialize path. The + // first send also pins UTF-8 output (the shared pwsh-local preamble) + // before anything runs: the session decode path treats PTY bytes as + // UTF-8, and an un-pinned console writes its host code page for + // non-ASCII output. The banner-to-prompt gap can outlast the silence + // bound, so the wait loops over follow-up sends until the controlled + // prompt is actually visible (in the viewport or the retained scrollback + // when it landed between sends), bounded by the send deadline. + let viewport = '' + for (;;) { + const first = viewport.length === 0 + const operation = session.startSend({ + text: first ? ENCODING_PREAMBLE + PWSH_PROMPT_SETUP : '', + submit: first, + ...signal !== undefined ? { signal } : {}, + }) + const result = await operation.done + if (result.waitReason === 'session_exit') throw new Error('PTY shell exited during startup') + if (result.waitReason === 'timeout') throw new Error('PTY shell did not reach readiness before startup timeout') + viewport = result.viewport + const scrollback = session.read({ offset: 0, count: 20 }).text + if (viewport.includes(CONTROLLED_PROMPT) || scrollback.includes(CONTROLLED_PROMPT)) break + } + session.motd = viewport + } if (signal === undefined) { - await session.initialize(signal) + await start() return } const aborted = Promise.withResolvers() @@ -92,7 +149,7 @@ async function initializeSession(session: LocalPtySession, signal?: AbortSignal) signal.addEventListener('abort', onAbort, { once: true }) try { signal.throwIfAborted() - await Promise.race([session.initialize(signal), aborted.promise]) + await Promise.race([start(), aborted.promise]) } finally { signal.removeEventListener('abort', onAbort) } @@ -125,7 +182,7 @@ export class BashTerminalBackend implements TerminalBackend { const terminal = await this.spawnTerminal({ argv, cwd: spec.cwd ?? policy.workspaceRoot, - env: childEnvironment(spec), + env: childEnvironment(spec, this.config.shellDialect), rows: this.config.rows, cols: this.config.cols, graceMs: this.config.disposeGraceMs, @@ -133,7 +190,7 @@ export class BashTerminalBackend implements TerminalBackend { }) const session = this.createSession(terminal, this.config) try { - await initializeSession(session, spec.signal) + await startupSession(session, this.config.shellDialect, spec.signal) return session } catch (error) { try { @@ -148,6 +205,7 @@ export class BashTerminalBackend implements TerminalBackend { /** Register the local PTY backend. */ export function apply(ctx: Context, config: Config): void { - validateConfig(config) - ctx.terminals.registerBackend(new BashTerminalBackend(ctx, config)) + const resolved = resolveConfig(config) + validateConfig(resolved) + ctx.terminals.registerBackend(new BashTerminalBackend(ctx, resolved)) } diff --git a/packages/terminal/terminal-bash/tests/config.spec.ts b/packages/terminal/terminal-bash/tests/config.spec.ts index 6c72da4363..d7557a2d90 100644 --- a/packages/terminal/terminal-bash/tests/config.spec.ts +++ b/packages/terminal/terminal-bash/tests/config.spec.ts @@ -1,10 +1,10 @@ import { describe, expect, it } from 'vitest' import type { Config } from '@deepseek-ai/dsh-terminal-bash/src/config.ts' -import { validateConfig } from '@deepseek-ai/dsh-terminal-bash/src/config.ts' +import { resolveConfig, validateConfig } from '@deepseek-ai/dsh-terminal-bash/src/config.ts' function config(overrides: Partial = {}): Config { return { - backendType: 'shell', shellPath: '/bin/bash', shellArgs: [], rows: 40, cols: 160, + backendType: 'shell', shellDialect: 'bash', shellPath: '/bin/bash', shellArgs: [], rows: 40, cols: 160, scrollbackLines: 100, scrollbackMaxBytes: 1024, maxReadBytes: 512, pollIntervalMs: 10, exactProbeAfterMs: 20, idleSilenceMs: 100, handoffGraceMs: 50, timeoutMs: 1000, disposeGraceMs: 100, @@ -30,3 +30,43 @@ describe('terminal-bash config', () => { expect(() => { validateConfig(config({ handoffGraceMs: 10, pollIntervalMs: 10 })) }).not.toThrow() }) }) + +describe('terminal-bash dialect resolution', () => { + it('defaults bash argv to the interactive profile-free form', () => { + const { shellPath, shellArgs, shellDialect } = resolveConfig({ backendType: 'shell', rows: 24, cols: 80 }) + expect(shellDialect).toBe('bash') + expect(shellPath).toBe('/bin/bash') + expect(shellArgs).toEqual(['--noprofile', '--norc', '-i']) + }) + + it('defaults pwsh argv to the interactive profile-free form and resolves the executable', () => { + const resolved = resolveConfig({ backendType: 'shell', shellDialect: 'pwsh', rows: 24, cols: 80 }) + expect(resolved.shellDialect).toBe('pwsh') + expect(resolved.shellPath.length).toBeGreaterThan(0) + expect(resolved.shellArgs).toEqual(['-NoLogo', '-NoProfile']) + }) + + it('lets an explicit shell specification win over the dialect defaults', () => { + const resolved = resolveConfig({ + backendType: 'shell', shellDialect: 'pwsh', shellPath: '/custom/pwsh', shellArgs: ['-NoProfile'], rows: 24, cols: 80, + }) + expect(resolved.shellPath).toBe('/custom/pwsh') + expect(resolved.shellArgs).toEqual(['-NoProfile']) + }) + + it('treats empty shell values as unset so Schemastery materialization cannot drop the dialect defaults', () => { + // Schemastery materializes an absent optional array as `[]`; the resolver + // must treat that shape like an unset value or a real bash spawn would + // start non-interactive without the controlled prompt. + const resolved = resolveConfig({ + backendType: 'shell', shellDialect: 'bash', shellPath: '', shellArgs: [], rows: 24, cols: 80, + }) + expect(resolved.shellPath).toBe('/bin/bash') + expect(resolved.shellArgs).toEqual(['--noprofile', '--norc', '-i']) + }) + + it('validates the effective shell path, not only the raw one', () => { + expect(() => { validateConfig(resolveConfig({ backendType: 'shell', shellDialect: 'bash', rows: 24, cols: 80 })) }).not.toThrow() + expect(() => { validateConfig(resolveConfig({ backendType: 'shell', shellDialect: 'pwsh', rows: 24, cols: 80 })) }).not.toThrow() + }) +}) diff --git a/packages/terminal/terminal-bash/tests/index.spec.ts b/packages/terminal/terminal-bash/tests/index.spec.ts index 9e2a3529eb..4f8c347221 100644 --- a/packages/terminal/terminal-bash/tests/index.spec.ts +++ b/packages/terminal/terminal-bash/tests/index.spec.ts @@ -1,5 +1,6 @@ import { describe, expect, it, vi } from 'vitest' import { PassThrough } from 'node:stream' +import { resolve } from 'node:path' import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' import SessionStore, { Session, SessionId } from '@deepseek-ai/dsh-session' @@ -8,7 +9,9 @@ import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService, { setSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' import TerminalSessionService, { TerminalBackendCleanupError, TerminalSessionId } from '@deepseek-ai/dsh-terminal' -import { BashTerminalBackend } from '@deepseek-ai/dsh-terminal-bash' +import type { TerminalSendRequest, TerminalWaitReason } from '@deepseek-ai/dsh-terminal' +import { BashTerminalBackend, PWSH_PROMPT_SETUP } from '@deepseek-ai/dsh-terminal-bash' +import { ENCODING_PREAMBLE } from '@deepseek-ai/dsh-pwsh-local' import * as ptyLocal from '@deepseek-ai/dsh-terminal-bash' import type { ResolvedConfig } from '@deepseek-ai/dsh-terminal-bash/src/config.ts' import type { LocalPtySession } from '@deepseek-ai/dsh-terminal-bash/src/session.ts' @@ -37,7 +40,7 @@ class RecordingSandbox extends SandboxProvider { function config(): ResolvedConfig { return { - backendType: 'shell', shellPath: '/bin/bash', shellArgs: [], rows: 24, cols: 80, + backendType: 'shell', shellDialect: 'bash', shellPath: '/bin/bash', shellArgs: [], rows: 24, cols: 80, scrollbackLines: 10, scrollbackMaxBytes: 100, maxReadBytes: 50, pollIntervalMs: 10, exactProbeAfterMs: 20, idleSilenceMs: 50, handoffGraceMs: 10, timeoutMs: 100, disposeGraceMs: 10, @@ -208,6 +211,7 @@ describe('BashTerminalBackend startup rollback', () => { graceMs: 10, env: { TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', BASH_SILENCE_DEPRECATION_WARNING: '1', + PROMPT_COMMAND: 'printf "\\033]133;D;%s\\007" "$?"; PS1=\'dsh> \'', DSH_SHELL: '1', DSH_SESSION_ID: 'agent', DSH_PTY_SESSION_ID: 'pty-1', }, }) @@ -215,7 +219,7 @@ describe('BashTerminalBackend startup rollback', () => { expect(initialized).toHaveBeenCalledWith(undefined) expect((ctx.sandbox as RecordingSandbox).calls).toEqual([{ argv: ['/bin/bash', '-i'], - policy: { mode: 'workspace-write', sessionId: 'agent', workspaceRoot: '/workspace' }, + policy: { mode: 'workspace-write', sessionId: 'agent', workspaceRoot: resolve('/workspace') }, }]) }) @@ -243,11 +247,11 @@ describe('BashTerminalBackend startup rollback', () => { expect(spawned).toMatchObject({ argv: ['/sandbox', '--', '/bin/bash', '-i'], - cwd: '/session-workspace', + cwd: resolve('/session-workspace'), }) expect((ctx.sandbox as RecordingSandbox).calls).toEqual([{ argv: ['/bin/bash', '-i'], - policy: { mode: 'workspace-write', sessionId: 'agent', workspaceRoot: '/session-workspace' }, + policy: { mode: 'workspace-write', sessionId: 'agent', workspaceRoot: resolve('/session-workspace') }, }]) }) @@ -337,6 +341,132 @@ describe('BashTerminalBackend startup rollback', () => { expect(session.motd).toBe('dsh> ') await session.close('test complete') }) + + it('bootstraps a pwsh dialect through the prompt function and scrubs bash-only env', async () => { + const ctx = new Context() + await ctx.plugin(EmptySandbox) + await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/workspace' }) + let spawned: SubprocessTerminalSpawnSpec | undefined + let sent: TerminalSendRequest | undefined + const session = { + motd: '', + startSend: (request: TerminalSendRequest) => { + sent = request + return { + done: Promise.resolve({ + viewport: 'setup-echo dsh> ', waitReason: 'stdin_read' as const, + sessionStatus: { kind: 'running' as const }, truncated: false, + }), + readOutput: () => ({ delta: '', truncated: false }), + cancel: () => false, + } + }, + read: () => ({ text: '', totalLines: 0, lineBegin: 0, lineEnd: 0, truncated: false }), + } as unknown as LocalPtySession + const backend = new BashTerminalBackend( + ctx, + { ...config(), shellDialect: 'pwsh', shellPath: 'pwsh' }, + async (spec) => { spawned = spec; return terminalHandle() }, + () => session, + ) + expect(await backend.spawn(spec(agent(ctx)))).toBe(session) + expect(sent).toMatchObject({ text: ENCODING_PREAMBLE + PWSH_PROMPT_SETUP, submit: true }) + expect(session.motd).toBe('setup-echo dsh> ') + expect(spawned?.env).toMatchObject({ + TERM: 'dumb', NO_COLOR: '1', DSH_SHELL: '1', DSH_SESSION_ID: 'agent', DSH_PTY_SESSION_ID: 'pty-1', + }) + expect(spawned?.env?.PS1).toBeUndefined() + expect(spawned?.env?.PROMPT_COMMAND).toBeUndefined() + }) + + it('keeps waiting for the marker prompt when the first send settles on silence', async () => { + const ctx = new Context() + await ctx.plugin(EmptySandbox) + await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/workspace' }) + const sends: TerminalSendRequest[] = [] + const session = { + motd: '', + startSend: (request: TerminalSendRequest) => { + sends.push(request) + const second = sends.length > 1 + return { + done: Promise.resolve({ + viewport: second ? 'dsh> ' : 'PowerShell 7.6.4\n', + waitReason: 'inferred_idle' as const, + sessionStatus: { kind: 'running' as const }, truncated: false, + }), + readOutput: () => ({ delta: '', truncated: false }), + cancel: () => false, + } + }, + read: () => ({ text: '', totalLines: 0, lineBegin: 0, lineEnd: 0, truncated: false }), + } as unknown as LocalPtySession + const backend = new BashTerminalBackend( + ctx, + { ...config(), shellDialect: 'pwsh', shellPath: 'pwsh' }, + async () => terminalHandle(), + () => session, + ) + await backend.spawn(spec(agent(ctx))) + expect(sends).toHaveLength(2) + expect(sends[1]).toMatchObject({ text: '', submit: false }) + expect(session.motd).toBe('dsh> ') + }) + + it('rejects a pwsh bootstrap whose shell exits or times out', async () => { + const ctx = new Context() + await ctx.plugin(EmptySandbox) + await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/workspace' }) + const sessionFor = (waitReason: TerminalWaitReason): LocalPtySession => ({ + startSend: () => ({ + done: Promise.resolve({ + viewport: 'no-prompt', waitReason, + sessionStatus: { kind: 'running' as const }, truncated: false, + }), + readOutput: () => ({ delta: '', truncated: false }), + cancel: () => false, + }), + read: () => ({ text: '', totalLines: 0, lineBegin: 0, lineEnd: 0, truncated: false }), + close: () => Promise.resolve(), + }) as unknown as LocalPtySession + const exited = new BashTerminalBackend(ctx, { ...config(), shellDialect: 'pwsh' }, async () => terminalHandle(), () => sessionFor('session_exit')) + await expect(exited.spawn(spec(agent(ctx)))).rejects.toThrow('PTY shell exited during startup') + const timedOut = new BashTerminalBackend(ctx, { ...config(), shellDialect: 'pwsh' }, async () => terminalHandle(), () => sessionFor('timeout')) + await expect(timedOut.spawn(spec(agent(ctx)))).rejects.toThrow('did not reach readiness before startup timeout') + }) + + it('forwards the spawn signal into the pwsh bootstrap sends', async () => { + const ctx = new Context() + await ctx.plugin(EmptySandbox) + await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/workspace' }) + const sends: TerminalSendRequest[] = [] + const session = { + motd: '', + startSend: (request: TerminalSendRequest) => { + sends.push(request) + return { + done: Promise.resolve({ + viewport: 'dsh> ', waitReason: 'stdin_read' as const, + sessionStatus: { kind: 'running' as const }, truncated: false, + }), + readOutput: () => ({ delta: '', truncated: false }), + cancel: () => false, + } + }, + read: () => ({ text: '', totalLines: 0, lineBegin: 0, lineEnd: 0, truncated: false }), + } as unknown as LocalPtySession + const backend = new BashTerminalBackend( + ctx, + { ...config(), shellDialect: 'pwsh', shellPath: 'pwsh' }, + async () => terminalHandle(), + () => session, + ) + const signal = new AbortController().signal + const spawned = await backend.spawn({ ...spec(agent(ctx)), signal }) + expect(spawned.motd).toBe('dsh> ') + expect(sends).toHaveLength(1) + expect(sends[0]?.signal).toBe(signal) + }) }) describe('terminal-bash plugin shape', () => { diff --git a/packages/terminal/terminal-bash/tests/local.spec.ts b/packages/terminal/terminal-bash/tests/local.spec.ts index d8c8fe6b59..ed1b6ab15d 100644 --- a/packages/terminal/terminal-bash/tests/local.spec.ts +++ b/packages/terminal/terminal-bash/tests/local.spec.ts @@ -1,4 +1,5 @@ import { existsSync, mkdtempSync, readFileSync, realpathSync, rmSync } from 'node:fs' +import { spawnSync } from 'node:child_process' import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, describe, expect, it } from 'vitest' @@ -12,6 +13,7 @@ import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' import LocalSubprocessRuntime from '@deepseek-ai/dsh-subprocess-local' +import { resolvePwshPath } from '@deepseek-ai/dsh-pwsh-local/src/resolve.ts' import * as ptyLocal from '@deepseek-ai/dsh-terminal-bash' const roots: string[] = [] @@ -49,6 +51,7 @@ function stubAgent(ctx: Context, rawId: string): Agent { async function harness( mode: 'danger-full-access' | 'workspace-write', timing: { idleSilenceMs?: number; handoffGraceMs?: number; timeoutMs?: number } = {}, + dialect: 'bash' | 'pwsh' = 'bash', ) { const root = mkdtempSync(join(tmpdir(), 'dsh-pty-local-')) roots.push(root) @@ -60,6 +63,7 @@ async function harness( await ctx.plugin(SandboxPolicyService, { mode, workspaceRoot: root }) await ctx.plugin(LocalSubprocessRuntime) const fiber = await ctx.plugin(ptyLocal, { + shellDialect: dialect, pollIntervalMs: 10, exactProbeAfterMs: 20, idleSilenceMs: timing.idleSilenceMs ?? 250, @@ -114,7 +118,9 @@ function processIsRunning(pid: number): boolean { } } -describe('terminal-bash real shell', () => { +// The real-shell suite drives a POSIX bash over the actual node-pty terminal; +// Windows has no bash, and its pwsh counterpart lives in the describe below. +describe.skipIf(process.platform === 'win32')('terminal-bash real shell', () => { it('persists cwd and environment across sends, scrubs secrets, and closes', async () => { const previous = process.env.DSH_TEST_SECRET process.env.DSH_TEST_SECRET = 'must-not-leak' @@ -137,6 +143,26 @@ describe('terminal-bash real shell', () => { } }, 10_000) + it('restores the controlled prompt after an in-shell PS1 override', async () => { + // The silence tier is pushed beyond every assertion below, so each settle + // proves prompt-based readiness survives the override rather than the + // inferred_idle fallback absorbing a broken prompt. + const { ctx, agent } = await harness('danger-full-access', { + idleSilenceMs: 5_000, + timeoutMs: 8_000, + }) + const created = await ctx.terminals.spawn(agent, { type: 'shell' }) + + const override = ctx.terminals.startSend(agent, created.sessionId, { text: 'PS1=broken-prompt', submit: true }) + expect((await override.done).waitReason).toBe('stdin_read') + + const after = ctx.terminals.startSend(agent, created.sessionId, { text: 'printf "healed=[%s]\\n" "$PS1"', submit: true }) + const result = await after.done + expect(result.waitReason).toBe('stdin_read') + expect(result.viewport).toContain('healed=[dsh> ]') + await ctx.terminals.kill(agent, created.sessionId) + }, 20_000) + it('wraps the exact shell argv under confined policy and unregisters on reload', async () => { const { ctx, root, agent, fiber, sandbox } = await harness('workspace-write') const created = await ctx.terminals.spawn(agent, { type: 'shell' }) @@ -247,3 +273,72 @@ describe('terminal-bash real shell', () => { await ctx.terminals.kill(agent, created.sessionId) }, 35_000) }) + +const hasPwsh = spawnSync( + resolvePwshPath(), ['-NoLogo', '-NoProfile', '-NonInteractive', '-Command', '$true'], + { encoding: 'utf8' }, +).status === 0 + +describe.skipIf(!hasPwsh)('terminal-bash pwsh real shell', () => { + it('bootstraps a persistent pwsh, persists state, and scrubs secrets', async () => { + const previous = process.env.DSH_TEST_SECRET + process.env.DSH_TEST_SECRET = 'must-not-leak' + try { + const { ctx, root, agent } = await harness('danger-full-access', { + idleSilenceMs: 300, + handoffGraceMs: 300, + timeoutMs: 8_000, + }, 'pwsh') + const created = await ctx.terminals.spawn(agent, { type: 'shell', name: 'main', cwd: root }) + expect(created.motd).toContain('dsh> ') + + const first = ctx.terminals.startSend(agent, created.sessionId, { + text: '$env:KEEP = "ok"; Set-Location /', + submit: true, + }) + expect((await first.done).waitReason).toBe('stdin_read') + const second = ctx.terminals.startSend(agent, created.sessionId, { + text: 'Write-Output "keep=$env:KEEP secret=$env:DSH_TEST_SECRET"', + submit: true, + }) + const result = await second.done + expect(result.viewport).toContain('keep=ok') + expect(result.viewport).toContain('secret=') + expect(result.viewport).not.toContain('must-not-leak') + + expect(ctx.terminals.read(agent, created.sessionId, { offset: 0, count: 40 }).text).toContain('keep=ok') + expect(await ctx.terminals.kill(agent, created.sessionId)).toBe(true) + expect(ctx.terminals.list(agent)).toEqual([]) + } finally { + if (previous === undefined) delete process.env.DSH_TEST_SECRET + else process.env.DSH_TEST_SECRET = previous + } + }, 30_000) + + it('pins UTF-8 output encoding so non-ASCII output survives the byte decode', async () => { + const { ctx, root, agent } = await harness('danger-full-access', { + idleSilenceMs: 300, + handoffGraceMs: 300, + timeoutMs: 8_000, + }, 'pwsh') + const created = await ctx.terminals.spawn(agent, { type: 'shell', name: 'main', cwd: root }) + // The bootstrap itself must have pinned both encodings: the session byte + // decode is UTF-8, so an un-pinned console writing its host code page + // garbles every non-ASCII byte that follows. + const pinned = ctx.terminals.startSend(agent, created.sessionId, { + text: '"console=" + [Console]::OutputEncoding.WebName + " out=" + $OutputEncoding.WebName', + submit: true, + }) + const pinnedResult = await pinned.done + expect(pinnedResult.viewport).toContain('console=utf-8 out=utf-8') + // Char codes keep the submitted line ASCII-only, so the assertion is a + // pure output-decode check. + const sent = ctx.terminals.startSend(agent, created.sessionId, { + text: "[Console]::Write([char]0x4E2D + [char]0x6587 + ' encoding-ok')", + submit: true, + }) + const result = await sent.done + expect(result.viewport).toContain('中文 encoding-ok') + await ctx.terminals.kill(agent, created.sessionId) + }, 30_000) +}) diff --git a/packages/terminal/terminal-bash/tests/session.spec.ts b/packages/terminal/terminal-bash/tests/session.spec.ts index ef9fd215f2..bf46317c5c 100644 --- a/packages/terminal/terminal-bash/tests/session.spec.ts +++ b/packages/terminal/terminal-bash/tests/session.spec.ts @@ -130,7 +130,7 @@ function makeSession( function config(overrides: Partial = {}): ResolvedConfig { return { - backendType: 'shell', shellPath: '/bin/bash', shellArgs: [], rows: 24, cols: 80, + backendType: 'shell', shellDialect: 'bash', shellPath: '/bin/bash', shellArgs: [], rows: 24, cols: 80, scrollbackLines: 10, scrollbackMaxBytes: 128, maxReadBytes: 64, pollIntervalMs: 10, exactProbeAfterMs: 20, idleSilenceMs: 50, handoffGraceMs: 10, timeoutMs: 100, disposeGraceMs: 20, diff --git a/packages/terminal/terminal-bash/tsconfig.json b/packages/terminal/terminal-bash/tsconfig.json index 40cb387342..b42e1015be 100644 --- a/packages/terminal/terminal-bash/tsconfig.json +++ b/packages/terminal/terminal-bash/tsconfig.json @@ -17,6 +17,9 @@ { "path": "../../../vendor/schemastery" }, + { + "path": "../../shell/pwsh-local" + }, { "path": "../../core/agent" }, diff --git a/packages/terminal/terminal/package.json b/packages/terminal/terminal/package.json index f0dc5e84c7..7faf010edf 100644 --- a/packages/terminal/terminal/package.json +++ b/packages/terminal/terminal/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-terminal", "description": "Persistent PTY session seam for the DeepSeek Harness — owner-scoped ids, backend registry, interactive sends, reads, signals, and awaited cleanup", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/terminal/tool-terminal/README.i18n.yaml b/packages/terminal/tool-terminal/README.i18n.yaml index c99082f62f..2b36b16d7c 100644 --- a/packages/terminal/tool-terminal/README.i18n.yaml +++ b/packages/terminal/tool-terminal/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/terminal/tool-terminal/README.md README.md: f6fd4020d423425d7b20ad6bc21b4e009cf25c85 -README.zh.md: 827d179515779f87665a80920e72a11be051023f +README.zh.md: dca0ce9280fc7a7d748e90a5c553c0e712e2453d diff --git a/packages/terminal/tool-terminal/README.zh.md b/packages/terminal/tool-terminal/README.zh.md index 827d179515..dca0ce9280 100644 --- a/packages/terminal/tool-terminal/README.zh.md +++ b/packages/terminal/tool-terminal/README.zh.md @@ -41,7 +41,7 @@ Use a terminal session only when work needs persistent terminal state or interac #### 模型看到的内容 -6 个生成的 schema 列在 [`dsh-tool-terminal` 目录章节](../../../docs/tool-catalog.md#deepseek-aidsh-tool-terminal)中。此插件活跃时,请求中会包含它们的固定 schema token;按 agent 范围过滤工具时可能隐藏这些 schema。 +6 个生成的 schema 列在 [`dsh-tool-terminal` 目录章节](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-terminal)中。此插件活跃时,请求中会包含它们的固定 schema token;按 agent 范围过滤工具时可能隐藏这些 schema。 #### Token 影响 diff --git a/packages/terminal/tool-terminal/package.json b/packages/terminal/tool-terminal/package.json index 6eebf5d33e..c4d4eefe7f 100644 --- a/packages/terminal/tool-terminal/package.json +++ b/packages/terminal/tool-terminal/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-terminal", "description": "Six model-facing persistent PTY tools with owner isolation and generic background-job integration", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/test-support/README.i18n.yaml b/packages/test-support/README.i18n.yaml index 484db0b32b..d229930962 100644 --- a/packages/test-support/README.i18n.yaml +++ b/packages/test-support/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/test-support/README.md README.md: 8f07d08828c2f4d69676487842373b600037e199 -README.zh.md: b56b082e0d58a32d6af39588fbe6f4ab422af9b7 +README.zh.md: 8793ebd9a4e6b456d34c6fe84983524dfc1d5fa2 diff --git a/packages/test-support/README.zh.md b/packages/test-support/README.zh.md index b56b082e0d..8793ebd9a4 100644 --- a/packages/test-support/README.zh.md +++ b/packages/test-support/README.zh.md @@ -6,13 +6,13 @@ | 包 | 职责 | |---|---| -| [`acp-snapshot/`](acp-snapshot/README.md) | 提供 ACP(Agent Client Protocol)快照测试工具包 | -| [`agent-loop-testkit/`](agent-loop-testkit/README.md) | 为 AgentLoop 测试挂载共享先决条件 | -| [`invariants/`](../runtime-diagnostics/invariants/README.md) | 运行开发期运行时约定断言 | -| [`loader-smoke/`](loader-smoke/README.md) | 启动由 Loader 组合的应用以执行冒烟测试 | -| [`llm-mock-server/`](llm-mock-server/README.md) | 提供确定性的 OpenAI 兼容故障服务器 | -| [`llm-replay/`](llm-replay/README.md) | 为无密钥测试和演示回放已记录的模型响应 | +| [`acp-snapshot/`](acp-snapshot/README.zh.md) | 提供 ACP(Agent Client Protocol)快照测试工具包 | +| [`agent-loop-testkit/`](agent-loop-testkit/README.zh.md) | 为 AgentLoop 测试挂载共享先决条件 | +| [`invariants/`](../runtime-diagnostics/invariants/README.zh.md) | 运行开发期运行时约定断言 | +| [`loader-smoke/`](loader-smoke/README.zh.md) | 启动由 Loader 组合的应用以执行冒烟测试 | +| [`llm-mock-server/`](llm-mock-server/README.zh.md) | 提供确定性的 OpenAI 兼容故障服务器 | +| [`llm-replay/`](llm-replay/README.zh.md) | 为无密钥测试和演示回放已记录的模型响应 | 当一个包获得产品约定和产品消费方时,它会移出 `test-support/`。 -不变式约定记录在 [docs/subsystems/invariants.md](../../docs/subsystems/invariants.md)。 +不变式约定记录在 [docs/subsystems/invariants.md](../../docs/subsystems/invariants.zh.md)。 diff --git a/packages/test-support/acp-snapshot/README.i18n.yaml b/packages/test-support/acp-snapshot/README.i18n.yaml index dedbd2eb71..921dfcc00c 100644 --- a/packages/test-support/acp-snapshot/README.i18n.yaml +++ b/packages/test-support/acp-snapshot/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/test-support/acp-snapshot/README.md -README.md: 06f1cb67cfcd954254db480ea696d10d81b37438 -README.zh.md: 1dfc023be843160f40b136382592d19f06e71d5c +README.md: 2d753cb0d05c35f78cd4effe3f410b021c16462f +README.zh.md: 9a5413297d0a4af1a506d846fa26128e84201e45 diff --git a/packages/test-support/acp-snapshot/README.md b/packages/test-support/acp-snapshot/README.md index 06f1cb67cf..2d753cb0d0 100644 --- a/packages/test-support/acp-snapshot/README.md +++ b/packages/test-support/acp-snapshot/README.md @@ -6,12 +6,12 @@ The ACP snapshot suite kit: the shared machinery behind the keyless snapshot tie Four layers, importable separately: -- **`launchAcpTestAgent` (launcher)** — boots a source agent under tsx or a built `lib` agent under plain Node from a supplied cwd, connects the SDK client over a raw-byte stdout tee, collects session updates and stderr, surfaces asynchronous spawn failures through startup, fails closed on unhandled permission requests, and owns graceful or signalled shutdown. Shutdown waits for process exit, inherited stdio closure, and ACP parser exhaustion before resolving or propagating a child error, so captures are complete and callers can remove owned paths after either outcome. When Windows accepts forced termination but publishes its exit marker asynchronously, shutdown gives that marker a bounded grace before treating fallback refusal as a second failure. Snapshot and ordinary e2e suites share this process boundary; a test supplies only agent paths, cwd, environment overrides, and any permission policy. +- **`launchAcpTestAgent` (launcher)** — boots a source entry under tsx or a built `lib` entry under plain Node from a supplied cwd, connects the SDK client over a raw-byte stdout tee, collects session updates and stderr, surfaces asynchronous spawn failures through startup, fails closed on unhandled permission requests, and owns graceful or signalled shutdown. Product suites name a `dsh` profile: the launcher passes the base and selected scenario patches through `--patch`, selects the scenario's sibling `*cordis.snapshot.yml` in replay, and materializes temporary copies whose relative plugin modules become absolute file URLs. Test-only fake bins may omit the profile and retain their own config grammar. Shutdown waits for process exit, inherited stdio closure, and ACP parser exhaustion before resolving or propagating a child error, so captures are complete and callers can remove owned paths after either outcome. - **`runScenario` (harness)** — drives ACP JSON-RPC stdio from a deterministic `input.json` script through the launcher, tees raw stdout for the expected-output and purity checks, and harvests every persisted raw JSONL session log (parent and subagent children, primary-first) after graceful stdin EOF. `AgentUnderTest` supplies absolute `binScript`, optional `libBinScript`, `configPath`, and `tsconfigPath` paths because the subprocess cwd is outside the repo; `workspaceParent` may move the generated child cwd from the platform temp directory when that grant is itself under test. Startup failures preserve captured agent stderr in the rejected diagnostic. -- **Normalizers** — pure functions turning captured surfaces into stable text or portable fixtures: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs and every native/JavaScript filesystem spelling of the generated cwd → tokens, longest-first; cwd-rooted separators selected as canonical `/` or host-native; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept, the same cwd-path policy), `tokenizeSessionFixtureCwd` (the generated workspace and its filesystem aliases → one canonical `{{cwd}}`, including an already-tokenized macOS `/private` alias; authored temp paths unchanged), `scrubSystemPrompts` (prompt text → `{{system}}`), `scrubToolSchemas` (schema bulk → `{{tools}}`), `scrubRequestHeaders` (all header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)), and `stabilizeFixtureMessageIds` (committed UUIDs carried into unchanged, mutually unique messages by structurally rewriting only complete surface and durable-inbox message ID fields across any recorder's fixture-ready parent/child logs). +- **Normalizers** — pure functions turning captured surfaces into stable text or portable fixtures: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs and every native/JavaScript filesystem spelling of the generated cwd → tokens, longest-first; cwd-rooted separators selected as canonical `/` or host-native; doubles as the stdout-purity check), `normalizeSessionLog` (sequence envelopes retained, times zeroed, the same cwd-path policy), `normalizeSessionSnapshot` (session-log normalization followed by request-header scrubbing and body-envelope projection), `tokenizeSessionFixtureCwd` (the generated workspace and its filesystem aliases → one canonical `{{cwd}}`, including an already-tokenized macOS `/private` alias; authored temp paths unchanged), `scrubSystemPrompts` (prompt text → `{{system}}`), `scrubToolSchemas` (schema bulk → `{{tools}}`), `scrubRequestHeaders` (all header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)), and `stabilizeFixtureMessageIds` (committed UUIDs carried into unchanged, mutually unique messages by structurally rewriting only complete surface and durable-inbox message ID fields across any recorder's fixture-ready parent/child logs). - **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario expected-output and re-persisted-log comparisons, record/refresh fixture write-back, rejection of structured `UNKNOWN_TOOL` results, a tokenized pin per header class composed with independently shared `system-prompt.expected.md` and `tool-schemas.expected.json` sidecars, and a live uniformity guard. Its fixture guards reject orphan scenario dirs, missing files, multiple pins for one class, duplicate sidecar content, noncanonical macOS-prefixed cwd tokens, unscrubbed JSONL headers, and malformed pinning headers. Before record or refresh writes fixtures, an unchanged complete message retains its committed UUID only when both its ID and identity-free fingerprint are unique across the scenario's fixture-ready parent/child logs; the session package's authoritative surface-type predicate selects surface carriers, correlated `agent/inbox/spliced` copies join the same mapping, and only validated `id` fields in those carriers are rewritten. New, changed, malformed, and graph-ambiguous messages keep fresh UUIDs. Refresh evaluates fresh leaves with the harvested run's ids, cwd, and every cwd alias, then reuses normalized-equivalent leaves only when the complete logical-record layout aligns and volatile string replacements form a bijection; complete message IDs in surface or inbox carriers are excluded because the later structural pass owns them, ambiguous logs keep fresh strings, and fresh semantic values remain authoritative. It also expands packed timing envelopes before aligning event times, so switching between packed and unpacked layouts cannot shift later records. A newly inserted `session/title` receives its preceding event's time so feature-driven insertions do not churn the remainder of a fixture. Each scenario directory's `session.jsonl` plus contiguous `session..jsonl` siblings are the ordered primary/child inventory; the scenario table does not duplicate their count. Must be called at vitest collection time. -Committed session fixtures use canonical packed rows; the [temporary repository migrator](../../../scripts/migrate-packed-session-fixtures.ts) (`pnpm run migrate:packed-session-fixtures`) rewrites older fixture layouts, and its [removal proposal](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) owns its deletion. +Committed session fixtures retain their complete session header and event payloads but omit ordinary `seq`/`time` and packed-row `seq0`/`time0` envelopes. Replay synthesizes those envelopes in memory; runtime persistence remains unchanged. Fixtures also use canonical packed rows; the [temporary repository migrator](../../../scripts/migrate-packed-session-fixtures.ts) (`pnpm run migrate:packed-session-fixtures`) rewrites older fixture layouts, and its [removal proposal](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) owns its deletion. A consuming `*.snapshot.ts` is the scenario table plus one factory call: @@ -41,8 +41,9 @@ const SCENARIOS: Scenario[] = [ defineAcpSnapshotSuite({ agent: { // absolute paths, resolved from the suite's own location - binScript: fileURLToPath(new URL('../../../packages/examples/acp-demo/src/bin.ts', import.meta.url)), + binScript: fileURLToPath(new URL('../../../apps/cli/src/bin.ts', import.meta.url)), configPath: fileURLToPath(new URL('../cordis.yml', import.meta.url)), + profile: 'acp', tsconfigPath: fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)), }, snapshotsDir: join(dirname(fileURLToPath(import.meta.url)), 'snapshots'), @@ -51,7 +52,7 @@ defineAcpSnapshotSuite({ }) ``` -A scenario booting a differently-composed tree sets its own `configPath` (an overlay whose basename still ends in `cordis.yml`, so the bin's replay swap finds the sibling `*cordis.snapshot.yml`) and, when that composition changes the request header, its own `headerClass` with its own pinning scenario — the acp-agent example's Code Mode and filesystem scenarios are templates. Default generated workspaces are stored in session fixtures as `{{cwd}}` so platform temp roots and random basenames do not affect recordings; `workspaceParent` moves the generated cwd outside the platform temp area when temporary-directory grants are themselves under test, keeps that explicit path in the fixture, and remains parent-owned while the harness removes only the generated child. A scenario's committed `workspace/` is copied into that child first, then `prepareWorkspace` runs against the generated cwd before the agent starts. Reserve this hook for fixtures Git cannot represent portably, keep ordinary seeds in `workspace/`, and pair it with `posixOnly` when the generated paths are invalid on Windows. +A scenario booting a differently composed profile sets its own `configPath` patch (its basename still ends in `cordis.yml`, so the launcher finds the sibling `*cordis.snapshot.yml`) and, when that composition changes the request header, its own `headerClass` with its own pinning scenario — the acp-agent example's Code Mode and filesystem scenarios are templates. Default generated workspaces are stored in session fixtures as `{{cwd}}` so platform temp roots and random basenames do not affect recordings; `workspaceParent` moves the generated cwd outside the platform temp area when temporary-directory grants are themselves under test, keeps that explicit path in the fixture, and remains parent-owned while the harness removes only the generated child. A scenario's committed `workspace/` is copied into that child first, then `prepareWorkspace` runs against the generated cwd before the agent starts. Reserve this hook for fixtures Git cannot represent portably, keep ordinary seeds in `workspace/`, and pair it with `posixOnly` when the generated paths are invalid on Windows. A pin owns its generated `system-prompt.expected.md` or `tool-schemas.expected.json` by default; `systemPromptSource` and `toolSchemasSource` name another pin when the complete corresponding sequence is identical, so each distinct version is committed once. The pin's `session.jsonl` stores `"system":"{{system}}","tools":"{{tools}}"` while retaining config, reason, and any model-visible prefix. A pin with legitimate mid-run header changes declares `expectedHeaderChanges`; a shared source must declare the same count, and record/refresh rejects claimants that generate different bytes. @@ -59,9 +60,9 @@ A child session whose own scope composes a different request declares it per fix Every scenario compares `stdout.expected.jsonl` with cwd-rooted separators canonicalized to `/`. On Windows, `pinsNativeWindowsStdout` additionally compares the complete `stdout.expected.windows.jsonl` after the shared expected output and requires that sidecar exactly when enabled. A scenario requiring a non-Windows host declares `posixOnly`, which skips its run test on Windows while the fixture guards keep covering its committed files everywhere; examples include POSIX process semantics (e.g. cancelling a live bash call kills a detached process group) and generated paths Windows cannot represent. A scenario whose composition needs a usable `pwsh` declares `pwshOnly`; the caller-supplied `hasPwsh` probe (the shipped acp-agent suite follows the executor's own resolution, so Program Files installs count) skips the run test when no usable `pwsh` resolves while the fixture guards keep covering its committed files everywhere. -The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. `pnpm run test:snapshot:record` calls the live LLM and rewrites the recorded scenarios' model fixtures; `pnpm run test:snapshot:refresh` stays keyless, runs the replay overlay, and rewrites stdout, comparable session-log expected outputs, and owned prompt and tool-schema sidecars from the committed model scripts. Fixture roles, record/replay/refresh semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md). +The example also ships a `cordis.snapshot.yml` replay patch next to its live patch. The launcher applies the live base patch and the selected replay sibling under `DSH_SNAPSHOT=replay` ([single-source replay config Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md)); [`dsh-llm-replay`](../llm-replay/README.md) serves fixtures named by the `DSH_SNAPSHOT_*` environment values. `pnpm run test:snapshot:record` calls the live LLM and rewrites the recorded scenarios' model fixtures; `pnpm run test:snapshot:refresh` stays keyless, runs the replay patch, and rewrites stdout, comparable session-log expected outputs, and owned prompt and tool-schema sidecars from the committed model scripts. Fixture roles, record/replay/refresh semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md). -Constraints: `suite.ts` and `harness.ts` import vitest (the harness polls its durable-boundary waits through `vi.waitFor`), so the package entry is importable only inside a vitest run (the launcher and normalizers have no such dependency but ship from the same entry). The launcher and suite factory are ACP-specific by design — the launcher speaks the SDK's `ClientSideConnection` — while the normalizers are transport-neutral session-log/text helpers also consumed by the JSON-RPC and Web snapshot recorders. Input scripts cover initialization, fresh-session creation, text prompting, cancellation, expected RPC failures, and durable turn-boundary waits. Permission round-trips are a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) mapped to the agent-issued `optionId`; an absent or exhausted queue answers `cancelled`, and an unoffered kind rejects the run. +Constraints: `suite.ts` and `harness.ts` import vitest (the harness polls its durable-boundary waits through `vi.waitFor`), so the package entry is importable only inside a vitest run (the launcher and normalizers have no such dependency but ship from the same entry). The launcher and suite factory are ACP-specific by design — the launcher speaks the SDK's `ClientSideConnection` — while the normalizers are transport-neutral session-log/text helpers also consumed by the JSON-RPC and Web snapshot recorders. Input scripts cover initialization, fresh-session creation, shorthand text prompts, exact structured ACP prompt blocks, cancellation, expected RPC failures, and durable turn-boundary waits. Permission round-trips are a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) mapped to the agent-issued `optionId`; an absent or exhausted queue answers `cancelled`, and an unoffered kind rejects the run. ## Model Experience diff --git a/packages/test-support/acp-snapshot/README.zh.md b/packages/test-support/acp-snapshot/README.zh.md index 1dfc023be8..9a5413297d 100644 --- a/packages/test-support/acp-snapshot/README.zh.md +++ b/packages/test-support/acp-snapshot/README.zh.md @@ -2,16 +2,16 @@ [English](README.md) | 中文 -ACP(Agent Client Protocol)快照套件工具包:无密钥快照层(`pnpm run test:snapshot`,见[测试策略](../../../docs/testing.md))背后的共享机制。示例只需场景表和 fixture(测试前置数据)目录就能获得完整快照套件;每项比较/保护机制都位于此处,受每文件覆盖率门禁约束,而不是在每个示例中复制。 +ACP(Agent Client Protocol)快照套件工具包:无密钥快照层(`pnpm run test:snapshot`,见[测试策略](../../../docs/testing.zh.md))背后的共享机制。示例只需场景表和 fixture(测试前置数据)目录就能获得完整快照套件;每项比较/保护机制都位于此处,受每文件覆盖率门禁约束,而不是在每个示例中复制。 四层可单独导入: -- **`launchAcpTestAgent`(启动器)**:从指定 cwd 在 tsx 下启动源 agent(智能体),或在普通 Node 下启动已构建 `lib` agent;通过原始字节 stdout tee 连接 SDK 客户端,收集会话更新和 stderr,在启动阶段报告异步 spawn 失败,默认拒绝未处理的权限请求,并负责优雅或带信号关闭。关闭会等待进程退出、继承 stdio 关闭和 ACP parser 耗尽,然后才完成关闭或传播子级错误,使捕获内容完整,且调用方可在任一结果后移除自有路径。当 Windows 接受强制终止但异步发布退出标记时,关闭会给该标记有界宽限,然后才将回退拒绝视为第二次失败。快照和普通 e2e 套件共享该进程边界;测试只需提供 agent 路径、cwd、环境覆盖和任何权限策略。 +- **`launchAcpTestAgent`(启动器)**:从指定 cwd 在 tsx 下启动源码入口,或在普通 Node 下启动已构建 `lib` 入口;通过原始字节 stdout tee 连接 SDK 客户端,收集会话更新和 stderr,在启动阶段报告异步 spawn 失败,默认拒绝未处理的权限请求,并负责优雅或带信号关闭。产品套件指定一个 `dsh` profile:启动器通过 `--patch` 传入基础 patch 与所选场景 patch,在 replay 时选择场景同级的 `*cordis.snapshot.yml`,并把相对插件模块改写成绝对 file URL 后物化为临时副本。测试专用 fake bin 可以省略 profile 并保留自己的配置语法。关闭会等待进程退出、继承 stdio 关闭和 ACP parser 耗尽,然后才完成关闭或传播子级错误,使捕获内容完整,且调用方可在任一结果后移除自有路径。 - **`runScenario`(harness)**:通过启动器从确定性 `input.json` 脚本驱动 ACP JSON-RPC stdio,将原始 stdout tee 给预期输出和纯度检查,并在优雅 stdin EOF 后收集每个持久化原始 JSONL 会话日志(父会话和 subagent 子会话,主会话优先)。`AgentUnderTest` 提供绝对 `binScript`、可选 `libBinScript`、`configPath` 和 `tsconfigPath` 路径,因为子进程 cwd 位于仓库外。当生成子级 cwd 的授权本身是测试对象时,`workspaceParent` 可以将它从平台临时目录移出。启动失败会在拒绝诊断中保留已捕获 agent stderr。 -- **规范化器**:将已捕获内容转换为稳定文本或可移植 fixture 的纯函数:`normalizeStdout`(JSON-RPC id → 首次出现序列;UUID 以及生成 cwd 的每种原生/JavaScript 文件系统写法 → token,按最长优先;根据 cwd 的分隔符选择规范 `/` 或宿主原生形式;同时作为 stdout 纯度检查)、`normalizeSessionLog`(时间归零、保留 `seq`、使用同一 cwd 路径策略)、`tokenizeSessionFixtureCwd`(生成的 workspace 及其文件系统别名,包括已进行 token 化的 macOS `/private` 别名 → 单一规范 `{{cwd}}`;手工编写的临时路径保持不变)、`scrubSystemPrompts`(提示词文本 → `{{system}}`)、`scrubToolSchemas`(schema bulk → `{{tools}}`)、`scrubRequestHeaders`(每个 pin 之外的所有 header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}`,保留结构;见[header 固定 Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md))和 `stabilizeFixtureMessageIds`(针对任意录制器已准备写入 fixture 的父级/子级日志,通过结构化方式仅改写 surface 和持久 inbox 中完整消息的 ID 字段,将已提交 UUID 带入未变化且双向唯一匹配的消息)。 +- **规范化器**:将已捕获内容转换为稳定文本或可移植 fixture 的纯函数:`normalizeStdout`(JSON-RPC id → 首次出现序列;UUID 以及生成 cwd 的每种原生/JavaScript 文件系统写法 → token,按最长优先;根据 cwd 的分隔符选择规范 `/` 或宿主原生形式;同时作为 stdout 纯度检查)、`normalizeSessionLog`(保留序号 envelope、将时间归零,并使用同一 cwd 路径策略)、`normalizeSessionSnapshot`(依次执行会话日志规范化、request header 清理和正文 envelope 投影)、`tokenizeSessionFixtureCwd`(生成的 workspace 及其文件系统别名,包括已进行 token 化的 macOS `/private` 别名 → 单一规范 `{{cwd}}`;手工编写的临时路径保持不变)、`scrubSystemPrompts`(提示词文本 → `{{system}}`)、`scrubToolSchemas`(schema bulk → `{{tools}}`)、`scrubRequestHeaders`(每个 pin 之外的所有 header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}`,保留结构;见[header 固定 Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md))和 `stabilizeFixtureMessageIds`(针对任意录制器已准备写入 fixture 的父级/子级日志,通过结构化方式仅改写 surface 和持久 inbox 中完整消息的 ID 字段,将已提交 UUID 带入未变化且双向唯一匹配的消息)。 - **`defineAcpSnapshotSuite`(工厂)**:为场景表注册完整 describe/it 树:每场景预期输出与重新持久化日志比较、录制/刷新 fixture 回写、拒绝结构化 `UNKNOWN_TOOL` 结果、每个 header 类别一个 token 化 pin(由可独立共享的 `system-prompt.expected.md` 和 `tool-schemas.expected.json` 伴随文件组合而成),以及实时一致性保护。其 fixture 保护会拒绝遗留场景目录、缺失文件、一个类别包含多个 pin、重复的伴随文件内容、带非规范 macOS 前缀的 cwd token、未擦除的 JSONL header,以及格式错误的 pin header。在录制或刷新模式写入 fixture 前,仅当一条未变化完整消息的 ID 及其去除身份后的指纹在场景可写入 fixture 的父级/子级日志中均唯一时,该消息才会保留已提交的 UUID;会话包的权威 surface 类型谓词负责选择 surface 载体,与其关联的 `agent/inbox/spliced` 副本也纳入同一映射,且仅改写这些载体中通过验证的 `id` 字段。新增、发生变化、格式错误以及图关系存在歧义的消息保留本次生成的 UUID。刷新会使用收集所得本次运行的 id、cwd 及全部 cwd 别名评估本次生成的叶值;只有完整逻辑记录布局对齐且易变字符串替换形成双射时,才会复用归一化后等价的叶值;surface 或 inbox 载体中的完整消息 ID 不参与此路径,因为后续结构化处理负责这些 ID;有歧义的日志保留本次生成的字符串,而本次生成的语义值仍为权威数据。它还会在对齐事件时间前展开打包时序 envelope,因此切换打包/非打包布局无法移动后续记录。新插入的 `session/title` 使用前一个事件的时间,因此功能驱动的插入不会扰动 fixture 余下部分。每个场景目录的 `session.jsonl` 和连续 `session..jsonl` 同级文件构成有序的主会话/子会话清单;场景表不重复其数量。必须在 vitest 收集时调用。 -签入仓库的会话 fixture 使用规范打包行;[临时仓库迁移器](../../../scripts/migrate-packed-session-fixtures.ts)(`pnpm run migrate:packed-session-fixtures`)会改写较旧的 fixture 布局,由其[移除提案](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md)负责删除该迁移器。 +签入仓库的会话 fixture 保留完整会话 header 和事件 payload,但省略普通行的 `seq`/`time` 与打包行的 `seq0`/`time0` envelope。回放只在内存中合成这些 envelope;运行时持久化不变。fixture 同时使用规范打包行;[临时仓库迁移器](../../../scripts/migrate-packed-session-fixtures.ts)(`pnpm run migrate:packed-session-fixtures`)会改写较旧的 fixture 布局,由其[移除提案](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.zh.md)负责删除该迁移器。 消费方 `*.snapshot.ts` 就是场景表加一次工厂调用: @@ -41,8 +41,9 @@ const SCENARIOS: Scenario[] = [ defineAcpSnapshotSuite({ agent: { // absolute paths, resolved from the suite's own location - binScript: fileURLToPath(new URL('../../../packages/examples/acp-demo/src/bin.ts', import.meta.url)), + binScript: fileURLToPath(new URL('../../../apps/cli/src/bin.ts', import.meta.url)), configPath: fileURLToPath(new URL('../cordis.yml', import.meta.url)), + profile: 'acp', tsconfigPath: fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)), }, snapshotsDir: join(dirname(fileURLToPath(import.meta.url)), 'snapshots'), @@ -51,7 +52,7 @@ defineAcpSnapshotSuite({ }) ``` -启动不同组合树的场景会设置自己的 `configPath`(一个 basename 仍以 `cordis.yml` 结尾的 overlay,使 bin 的回放交换可找到同级 `*cordis.snapshot.yml`);当该组合改变请求 header 时,还会设置自己的 `headerClass` 和 pin 场景,acp-agent 示例的 Code Mode 与文件系统场景是模板。默认生成的 workspace 在会话 fixture 中存储为 `{{cwd}}`,使平台临时根目录和随机 basename 不影响录制结果;当临时目录授权自身待测时,`workspaceParent` 将生成 cwd 移出平台临时区域,在 fixture 中保留该显式路径,并仍归父级所有,而 harness 只移除生成的子级。场景签入的 `workspace/` 会先复制到该子级,随后 `prepareWorkspace` 在 agent 启动前针对生成 cwd 运行。此 hook 仅用于 Git 无法跨平台表示的 fixture;普通种子应留在 `workspace/` 中,而生成路径在 Windows 上无效时还必须搭配 `posixOnly`。 +启动不同 profile 组合的场景会设置自己的 `configPath` patch(其 basename 仍以 `cordis.yml` 结尾,使启动器可找到同级 `*cordis.snapshot.yml`);当该组合改变请求 header 时,还会设置自己的 `headerClass` 和 pin 场景,acp-agent 示例的 Code Mode 与文件系统场景是模板。默认生成的 workspace 在会话 fixture 中存储为 `{{cwd}}`,使平台临时根目录和随机 basename 不影响录制结果;当临时目录授权自身待测时,`workspaceParent` 将生成 cwd 移出平台临时区域,在 fixture 中保留该显式路径,并仍归父级所有,而 harness 只移除生成的子级。场景签入的 `workspace/` 会先复制到该子级,随后 `prepareWorkspace` 在 agent 启动前针对生成 cwd 运行。此 hook 仅用于 Git 无法跨平台表示的 fixture;普通种子应留在 `workspace/` 中,而生成路径在 Windows 上无效时还必须搭配 `posixOnly`。 每个 pin 默认拥有其生成的 `system-prompt.expected.md` 或 `tool-schemas.expected.json`;当完整的对应序列相同时,`systemPromptSource` 和 `toolSchemasSource` 指定另一个 pin 作为来源,因此每个不同版本只提交一次。该 pin 的 `session.jsonl` 存储 `"system":"{{system}}","tools":"{{tools}}"`,同时保留配置、原因和任何模型可见前缀。具有合法运行中 header 变更的 pin 声明 `expectedHeaderChanges`;共享来源必须声明相同的 header 变更数量,录制/刷新会拒绝生成不同字节的共享引用方。 @@ -59,9 +60,9 @@ defineAcpSnapshotSuite({ 每个场景都比较 `stdout.expected.jsonl`,其中以 cwd 为根的分隔符规范化为 `/`。在 Windows 上,`pinsNativeWindowsStdout` 还会在共享预期输出之后比较完整 `stdout.expected.windows.jsonl`,并且仅在启用时要求存在该伴随文件。需要非 Windows 主机的场景声明 `posixOnly`,在 Windows 上跳过运行测试,但 fixture 保护仍在所有平台覆盖其已提交文件;示例包括 POSIX 进程语义(例如取消正在运行的 bash 调用会终止一个已脱离的进程组)和 Windows 无法表示的生成路径。组合需要可用 `pwsh` 的场景声明 `pwshOnly`;调用方提供的 `hasPwsh` 探测(随附的 acp-agent 套件遵循执行器自身的解析,因此 Program Files 安装也计入)在解析不到可用 `pwsh` 时跳过运行测试,而 fixture 保护仍处处覆盖其已提交文件。 -示例还发布 `cordis.snapshot.yml` 回放 overlay,位于 `cordis.yml` 旁边(bin 在 `DSH_SNAPSHOT=replay` 下交换它们,见[单源回放配置 Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md));回放 fixture 由 [`dsh-llm-replay`](../llm-replay/README.md) 提供,本包通过为子进程设置的 `DSH_SNAPSHOT_*` env var 指向它。`pnpm run test:snapshot:record` 调用在线 LLM(大语言模型),并重写已记录场景的模型 fixture;`pnpm run test:snapshot:refresh` 保持无密钥,运行回放 overlay,并从已提交模型脚本重写 stdout、可比较会话日志预期输出,以及各 pin 自有的提示词与工具 schema 伴随文件。Fixture 角色、录制/回放/刷新语义和场景表字段记录在 `Scenario` 以及[快照 Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md) 中。 +示例还在 live patch 旁提供 `cordis.snapshot.yml` replay patch。`DSH_SNAPSHOT=replay` 下,启动器应用 live 基础 patch 和所选场景的 replay 同级文件(见[单源回放配置 Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md));[`dsh-llm-replay`](../llm-replay/README.zh.md) 提供由 `DSH_SNAPSHOT_*` 环境值指向的 fixture。`pnpm run test:snapshot:record` 调用在线 LLM(大语言模型),并重写已记录场景的模型 fixture;`pnpm run test:snapshot:refresh` 保持无密钥,运行回放 overlay,并从已提交模型脚本重写 stdout、可比较会话日志预期输出,以及各 pin 自有的提示词与工具 schema 伴随文件。Fixture 角色、录制/回放/刷新语义和场景表字段记录在 `Scenario` 以及[快照 Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.zh.md) 中。 -约束:`suite.ts` 与 `harness.ts` 导入 vitest(harness 通过 `vi.waitFor` 轮询其持久边界等待),因此包入口只能在 vitest 运行中导入(启动器和规范化器没有此依赖,但从同一入口发布)。启动器和套件工厂按设计专用于 ACP,启动器使用 SDK 的 `ClientSideConnection`;规范化器是与传输无关的会话日志/文本辅助工具,还由 JSON-RPC 和 Web 快照录制器消费。输入脚本覆盖初始化、新建会话、文本提示、取消、预期 RPC 失败和持久轮次边界等待。权限往返是选项类别选择(`allow_once`、`reject_once` 等)的 FIFO 队列,映射到 agent 发出的 `optionId`;缺少或耗尽的队列回答 `cancelled`,未提供类别会拒绝运行。 +约束:`suite.ts` 与 `harness.ts` 导入 vitest(harness 通过 `vi.waitFor` 轮询其持久边界等待),因此包入口只能在 vitest 运行中导入(启动器和规范化器没有此依赖,但从同一入口发布)。启动器和套件工厂按设计专用于 ACP,启动器使用 SDK 的 `ClientSideConnection`;规范化器是与传输无关的会话日志/文本辅助工具,还由 JSON-RPC 和 Web 快照录制器消费。输入脚本覆盖初始化、新建会话、文本提示简写、精确结构化 ACP 提示词块、取消、预期 RPC 失败和持久轮次边界等待。权限往返是选项类别选择(`allow_once`、`reject_once` 等)的 FIFO 队列,映射到 agent 发出的 `optionId`;缺少或耗尽的队列回答 `cancelled`,未提供类别会拒绝运行。 ## 模型体验 @@ -75,4 +76,4 @@ defineAcpSnapshotSuite({ - **会话收集需要原始 JSONL mode**:`runScenario` 收集持久化 `.jsonl` 日志,因此快照配置使用 `persistenceCompression: 'none'`;压缩 JSONL 和 SQLite 组合没有快照收集路径。 - **构建 mode 需要当前产物**:先运行 `pnpm run build`,再选择 `DSH_EXAMPLE_MODE=lib`;源 mode 仍是零构建路径。 -- **后端覆盖仍使用 ACP 驱动器**:保留场景为何使用该传输,见[仅自动化 ACP 决策](../../../.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.md#snapshot-boundary)。 +- **后端覆盖仍使用 ACP 驱动器**:保留场景为何使用该传输,见[仅自动化 ACP 决策](../../../.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.zh.md#snapshot-boundary)。 diff --git a/packages/test-support/acp-snapshot/package.json b/packages/test-support/acp-snapshot/package.json index a0a3697896..3c6b58df14 100644 --- a/packages/test-support/acp-snapshot/package.json +++ b/packages/test-support/acp-snapshot/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-acp-snapshot", "description": "ACP test kit: shared subprocess launcher, snapshot scenario harness, expected-output normalizers, and suite factory", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -32,8 +32,10 @@ ], "license": "MIT", "dependencies": { - "@agentclientprotocol/sdk": "0.25.1", + "@agentclientprotocol/sdk": "1.4.0", + "@deepseek-ai/cordis-plugin-include": "workspace:*", "@deepseek-ai/dsh-loader-smoke": "workspace:*", + "js-yaml": "^4.2.0", "vitest": "^4.1.8" }, "peerDependencies": { @@ -44,6 +46,7 @@ "devDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", - "@deepseek-ai/cordis": "workspace:^" + "@deepseek-ai/cordis": "workspace:^", + "@types/js-yaml": "^4.0.9" } } diff --git a/packages/test-support/acp-snapshot/src/harness.ts b/packages/test-support/acp-snapshot/src/harness.ts index d37bf289b5..f98ff9892d 100644 --- a/packages/test-support/acp-snapshot/src/harness.ts +++ b/packages/test-support/acp-snapshot/src/harness.ts @@ -6,7 +6,7 @@ * It boots the REAL agent bin subprocess via the cordis Loader (so the * export-shape bug class stays guarded — see docs/postmortem/0001), drives it * over real ACP JSON-RPC stdio with a deterministic input script, tees raw - * stdout (for the expected-output and purity checks) into an SDK `ClientSideConnection`, + * stdout (for the expected-output and purity checks) into an SDK client app, * and — in record mode — harvests the persisted session JSONL after a graceful * shutdown flush. The pure normalizers in ./normalize.ts turn the captured * stdout frames and the session-log events into stable, snapshot-able text. @@ -23,13 +23,18 @@ import { tmpdir } from 'node:os' import { basename, dirname, join, delimiter } from 'node:path' import { vi } from 'vitest' import { - ClientSideConnection, PROTOCOL_VERSION, + type ContentBlock as AcpContentBlock, type RequestPermissionRequest, type RequestPermissionResponse, type SessionNotification, } from '@agentclientprotocol/sdk' -import { launchAcpTestAgent, type AgentUnderTest, type LaunchedAcpTestAgent } from './launcher.ts' +import { + launchAcpTestAgent, + type AcpTestClient, + type AgentUnderTest, + type LaunchedAcpTestAgent, +} from './launcher.ts' export type { AgentUnderTest } from './launcher.ts' @@ -69,6 +74,7 @@ export type InputStep = | { op: 'newSession' } | { op: 'newSessionExpectError'; additionalDirectories?: string[] } | { op: 'prompt'; text: string } + | { op: 'promptContent'; content: AcpContentBlock[] } | { op: 'promptAndWaitForAgentMessage'; text: string; waitForText: string } | { op: 'promptExpectError'; text: string } | { @@ -186,12 +192,9 @@ export interface RunOptions { */ workspaceParent?: string /** - * Alternate LIVE config path for the boot (absolute), overriding - * {@link AgentUnderTest.configPath} for this run. A scenario needing a - * differently-composed tree (the Code Mode scenarios) ships an overlay - * whose basename still ends in `cordis.yml`, so the bin's replay swap - * resolves the sibling `*cordis.snapshot.yml` the same way it does for - * the default. + * Alternate live profile patch (absolute), overriding + * {@link AgentUnderTest.configPath} for this run. Its basename still ends + * in `cordis.yml` so the launcher can select the replay sibling. */ configPath?: string } @@ -375,7 +378,7 @@ export async function runScenario(input: InputScript, opts: RunOptions): Promise /** Drive one input step over the client connection. */ async function runStep( - client: ClientSideConnection, + client: AcpTestClient, step: InputStep, cwd: string, waitForUpdate: (match: (u: SessionNotification['update']) => boolean) => Promise, @@ -422,6 +425,12 @@ async function runStep( await client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] }) return } + case 'promptContent': { + const sessionId = getSessionId() + if (sessionId === undefined) throw new Error('snapshot-harness: promptContent before newSession') + await client.prompt({ sessionId, prompt: step.content }) + return + } case 'promptAndWaitForAgentMessage': { const sessionId = getSessionId() if (sessionId === undefined) throw new Error('snapshot-harness: promptAndWaitForAgentMessage before newSession') diff --git a/packages/test-support/acp-snapshot/src/index.ts b/packages/test-support/acp-snapshot/src/index.ts index 6d8f5c0953..8eca980585 100644 --- a/packages/test-support/acp-snapshot/src/index.ts +++ b/packages/test-support/acp-snapshot/src/index.ts @@ -34,8 +34,10 @@ export { export { extractSnapshotSpillPaths, normalizeSessionLog, + normalizeSessionSnapshot, normalizeStdout, scrubRequestHeaders, + scrubSessionSnapshot, scrubSystemPrompts, scrubToolSchemas, tokenizeSessionFixtureCwd, diff --git a/packages/test-support/acp-snapshot/src/launcher.ts b/packages/test-support/acp-snapshot/src/launcher.ts index c4570c90d1..5b146f8ebe 100644 --- a/packages/test-support/acp-snapshot/src/launcher.ts +++ b/packages/test-support/acp-snapshot/src/launcher.ts @@ -8,29 +8,57 @@ */ import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process' -import { join } from 'node:path' +import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, symlinkSync, writeFileSync } from 'node:fs' +import { createRequire } from 'node:module' +import { basename, dirname, join, resolve } from 'node:path' import { Readable, Writable } from 'node:stream' +import { pathToFileURL } from 'node:url' +import * as yaml from 'js-yaml' import { - ClientSideConnection, + client as createAcpClientApp, + methods, ndJsonStream, - type Agent as AcpAgent, - type Client, + type CancelNotification, + type CloseSessionRequest, + type CloseSessionResponse, + type InitializeRequest, + type InitializeResponse, + type ListSessionsRequest, + type ListSessionsResponse, + type NewSessionRequest, + type NewSessionResponse, + type PromptRequest, + type PromptResponse, type RequestPermissionRequest, type RequestPermissionResponse, + type ResumeSessionRequest, + type ResumeSessionResponse, + type SetSessionConfigOptionRequest, + type SetSessionConfigOptionResponse, type SessionNotification, } from '@agentclientprotocol/sdk' +import { entryListSchema, type PatchOptions } from '@deepseek-ai/cordis-plugin-include' import { resolveExampleLaunch } from '@deepseek-ai/dsh-loader-smoke' const EXIT_MARKER_GRACE_MS = 250 -/** The source/built agent entry, leaf config, and workspace tsconfig an ACP test boots. */ +/** Loader fields needed while rebasing authored relative module names. */ +interface ProfilePatchEntry { + name: string + group?: boolean | null + config?: unknown +} + +/** The source/built entry, profile patch, and workspace tsconfig an ACP test boots. */ export interface AgentUnderTest { - /** The agent source bin entry (for example `packages/examples/acp-demo/src/bin.ts`). */ + /** The source bin entry; product suites use `apps/cli/src/bin.ts`. */ binScript: string /** Explicit built-mode entry for fixtures whose source path is not under `src/`. */ libBinScript?: string | undefined - /** The leaf `cordis.yml` loaded by the bin. */ + /** Base config or profile patch loaded by the bin. */ configPath: string + /** Named dsh profile; omitted only for test-only fake bins with their own config grammar. */ + profile?: string /** The repo tsconfig whose paths resolve unbuilt workspace imports. */ tsconfigPath: string } @@ -49,6 +77,19 @@ export interface AcpTestLaunchOptions { requestPermission?: (params: RequestPermissionRequest) => Promise } +/** Stable ACP methods used by the subprocess test harness. */ +export interface AcpTestClient { + readonly closed: Promise + initialize: (params: InitializeRequest) => Promise + newSession: (params: NewSessionRequest) => Promise + listSessions: (params: ListSessionsRequest) => Promise + resumeSession: (params: ResumeSessionRequest) => Promise + closeSession: (params: CloseSessionRequest) => Promise + setSessionConfigOption: (params: SetSessionConfigOptionRequest) => Promise + prompt: (params: PromptRequest) => Promise + cancel: (params: CancelNotification) => Promise +} + /** A running ACP test process and its captured client-side outputs. */ export interface LaunchedAcpTestAgent { /** The child process, exposed for process-level assertions. */ @@ -56,7 +97,7 @@ export interface LaunchedAcpTestAgent { /** Resolve when the OS spawns the child; reject with its asynchronous spawn failure. */ spawned: Promise /** The SDK connection backed by the child's stdio. */ - client: ClientSideConnection + client: AcpTestClient /** Session updates in receive order. */ updates: SessionNotification['update'][] /** Decode all stdout bytes captured so far. */ @@ -77,11 +118,15 @@ export interface LaunchedAcpTestAgent { */ export function launchAcpTestAgent(options: AcpTestLaunchOptions): LaunchedAcpTestAgent { const { agent, cwd } = options + const selectedConfig = options.configPath ?? agent.configPath const launch = resolveExampleLaunch({ srcBin: agent.binScript, libBin: agent.libBinScript, - configArgs: ['--config', options.configPath ?? agent.configPath], + configArgs: agent.profile === undefined + ? ['--config', selectedConfig] + : profileArgs(agent.profile, agent.configPath, selectedConfig, options.env?.DSH_SNAPSHOT, cwd), tsconfigPath: agent.tsconfigPath, + ...agent.profile === undefined ? {} : { sourceImport: 'tsx/esm' }, env: { ...options.env, DSH_HOME: join(cwd, '.dsh'), @@ -152,8 +197,8 @@ export function launchAcpTestAgent(options: AcpTestLaunchOptions): LaunchedAcpTe } const requestPermission = options.requestPermission ?? (() => Promise.resolve({ outcome: { outcome: 'cancelled' as const } })) - const makeClient = (_agent: AcpAgent): Client => ({ - sessionUpdate(params: SessionNotification): Promise { + const clientApp = createAcpClientApp({ name: 'deepseek-harness-acp-test-client' }) + .onNotification(methods.client.session.update, ({ params }) => { return trackClientCallback(() => { updates.push(params.update) for (let index = updateWaiters.length - 1; index >= 0; index--) { @@ -173,17 +218,34 @@ export function launchAcpTestAgent(options: AcpTestLaunchOptions): LaunchedAcpTe waiter.resolve(params.update) } }) - }, - requestPermission: params => trackClientCallback(() => requestPermission(params)), - }) - const client = new ClientSideConnection(makeClient, stream) + }) + .onRequest(methods.client.session.requestPermission, ({ params }) => ( + trackClientCallback(() => requestPermission(params)) + )) + const connection = clientApp.connect(stream) + const context = connection.agent + const client: AcpTestClient = { + closed: connection.closed, + initialize: params => context.request(methods.agent.initialize, params), + newSession: params => context.request(methods.agent.session.new, params), + /* v8 ignore next -- exercised by the real-process ACP control-surface conformance e2e. */ + listSessions: params => context.request(methods.agent.session.list, params), + /* v8 ignore next -- exercised by the real-process ACP control-surface conformance e2e. */ + resumeSession: params => context.request(methods.agent.session.resume, params), + /* v8 ignore next -- exercised by the real-process ACP control-surface conformance e2e. */ + closeSession: params => context.request(methods.agent.session.close, params), + /* v8 ignore next -- exercised by the real-process ACP control-surface conformance e2e. */ + setSessionConfigOption: params => context.request(methods.agent.session.setConfigOption, params), + prompt: params => context.request(methods.agent.session.prompt, params), + cancel: params => context.notify(methods.agent.session.cancel, params), + } // `exit` only reports the parent process's status. Descendants may retain // inherited stdout/stderr handles and buffered ACP frames may still be // crossing the SDK parser. Node's `close` follows stdio closure; the SDK's // `closed` follows parser exhaustion. Capture both eagerly so a caller that // invokes close after process exit still joins the complete drain boundary. const stdioClosed = new Promise(resolve => child.once('close', () => { resolve() })) - const drained = Promise.all([stdioClosed, client.closed]).then(async () => { + const drained = Promise.all([stdioClosed, connection.closed]).then(async () => { // The ACP SDK's readable loop dispatches client callbacks without awaiting // them. Once `closed` settles no new callbacks can start, but callbacks // already in flight still belong to this launch's teardown boundary. @@ -194,7 +256,7 @@ export function launchAcpTestAgent(options: AcpTestLaunchOptions): LaunchedAcpTe // A caller may await a pending update without calling close(). Make natural // stream exhaustion terminal for those waiters too, but only after the // parser has dispatched every buffered frame. - void client.closed.then(closeUpdateStream) + void connection.closed.then(closeUpdateStream) return { child, @@ -278,6 +340,99 @@ export function launchAcpTestAgent(options: AcpTestLaunchOptions): LaunchedAcpTe } } +/** Build one dsh profile invocation from the base and optional scenario patches. */ +function profileArgs( + profile: string, + basePatch: string, + selectedPatch: string, + snapshotMode: string | undefined, + cwd: string, +): string[] { + const base = resolve(cwd, basePatch) + const selected = resolve(cwd, selectedPatch) + // Replay siblings already contain the selected scenario's complete delta + // over the base patch; live scenario patches are not also applied. + const patches = snapshotMode === 'replay' + ? [base, replayPatchPath(selected)] + : [...new Set([base, selected])] + const materializedRoot = join(cwd, '.dsh-profile-patches') + mkdirSync(materializedRoot, { recursive: true }) + const materializedDir = mkdtempSync(join(materializedRoot, 'launch-')) + const materialized = patches.map((file, index) => materializeProfilePatch(file, cwd, materializedDir, index)) + return ['--profile', profile, ...materialized.flatMap(file => ['--patch', file])] +} + +/** Derive the replay-only sibling patch selected by the former app-bin swap. */ +function replayPatchPath(source: string): string { + const replayName = basename(source).replace(/cordis\.ya?ml$/, 'cordis.snapshot.yml') + return resolve(dirname(source), replayName) +} + +/** Parse a bare package or subpath specifier into its package name. */ +function barePackageName(specifier: string): string | undefined { + if (specifier.startsWith('.') || specifier.startsWith('/') || specifier.includes(':')) return undefined + const [first = '', second = ''] = specifier.split('/') + return first.startsWith('@') ? `${first}/${second}` : first +} + +/** Find a bare package's directory from the authored patch's module-resolution anchor. */ +function packageDirFromPatch(source: string, packageName: string): string | undefined { + for (const searchPath of createRequire(pathToFileURL(source)).resolve.paths(packageName) ?? []) { + const candidate = join(searchPath, packageName) + if (existsSync(join(candidate, 'package.json'))) return realpathSync(candidate) + } + return undefined +} + +/** + * Install an authored patch's resolvable bare package into the temporary + * profile fallback. This mirrors `dsh plugin` while retaining the bare entry + * name and package provenance used by request metadata. + */ +function linkProfilePackage(source: string, cwd: string, packageName: string): void { + const packageDir = packageDirFromPatch(source, packageName) + // The package may instead belong to the dsh installation; profile boot heals those links. + if (packageDir === undefined) return + const link = join(cwd, '.dsh', 'profiles', 'node_modules', packageName) + mkdirSync(dirname(link), { recursive: true }) + if (existsSync(link)) { + if (realpathSync(link) !== packageDir) { + throw new Error(`ACP profile package ${packageName} resolves to two directories`) + } + return + } + /* v8 ignore next -- Windows uses directory junctions; the platform lane owns that branch. */ + symlinkSync(packageDir, link, process.platform === 'win32' ? 'junction' : 'dir') +} + +/** Copy one authored patch into the launch cwd with relative plugin names made absolute. */ +function materializeProfilePatch(source: string, cwd: string, targetDir: string, index: number): string { + const parsed = yaml.load(readFileSync(source, 'utf8'), { schema: entryListSchema }) + if (!Array.isArray(parsed)) throw new Error(`ACP profile patch must be a top-level array: ${source}`) + const patches = parsed as PatchOptions[] + const baseDir = dirname(source) + const resolveName = (value: string): string => { + const packageName = barePackageName(value) + if (packageName !== undefined) linkProfilePackage(source, cwd, packageName) + return value.startsWith('./') || value.startsWith('../') + ? pathToFileURL(resolve(baseDir, value)).href + : value + } + const visitEntry = (entry: ProfilePatchEntry): void => { + entry.name = resolveName(entry.name) + if (entry.group === true && Array.isArray(entry.config)) { + for (const child of entry.config as ProfilePatchEntry[]) visitEntry(child) + } + } + for (const patch of patches) { + if (typeof patch.name === 'string') patch.name = resolveName(patch.name) + for (const entry of patch.insert ?? []) visitEntry(entry) + } + const target = join(targetDir, `${String(index)}-${basename(source)}`) + writeFileSync(target, yaml.dump(patches, { schema: entryListSchema, lineWidth: -1, noRefs: true })) + return target +} + /** Resolve once a running child exits. */ function waitForExit(child: ChildProcessWithoutNullStreams): Promise { return new Promise(resolve => child.once('exit', () => { resolve() })) diff --git a/packages/test-support/acp-snapshot/src/normalize.ts b/packages/test-support/acp-snapshot/src/normalize.ts index a3d5e66aa1..259f3ad8fe 100644 --- a/packages/test-support/acp-snapshot/src/normalize.ts +++ b/packages/test-support/acp-snapshot/src/normalize.ts @@ -1,17 +1,31 @@ /** * Pure ACP transcript and session-log normalizers. They scrub session ids, run cwd, RPC ids, - * timestamps, and hook duration while preserving deterministic event sequence numbers. + * timestamps and hook duration while preserving event payloads. * Request-header scrubbers stay composable so one scenario per header class can pin prompt and * tool-schema sidecars. * @module @deepseek-ai/dsh-acp-snapshot/normalize */ const SESSION_ID = '{{sessionId}}' +const MESSAGE_ID = '{{messageId}}' +const USED_TOKENS = '{{usedTokens}}' const CWD = '{{cwd}}' const SYSTEM = '{{system}}' const TOOLS = '{{tools}}' const EVENT_TIME = '{{eventTime}}' const EVENT_OMITTED_BYTES = '{{eventOmittedBytes}}' +const PACKED_CHUNK_ROW_TYPES = new Set(['text-chunks', 'reasoning-chunks', 'tool-call-chunks']) + +function isPackedFixtureRow(record: Record): boolean { + return typeof record.type === 'string' && PACKED_CHUNK_ROW_TYPES.has(record.type) +} + +function omitFixtureEnvelope(record: Record): void { + delete record.seq + delete record.time + delete record.seq0 + delete record.time0 +} /** A cwd-rooted path after volatile cwd replacement, through its last separator-delimited segment. */ const CWD_ROOTED_PATH_RE = /\{\{cwd\}\}(?:[\\/][^\s<>"'`]+)+/g @@ -172,6 +186,7 @@ function scrubString(value: string, ctx: NormalizeContext, cwdPathMode: CwdPathM /** Recursively scrub a parsed JSON value (strings replaced; structure kept). */ function scrubValue(value: unknown, ctx: NormalizeContext, cwdPathMode: CwdPathMode, key?: string): unknown { if (typeof value === 'string') { + if (key === 'messageId') return MESSAGE_ID const scrubbed = scrubString(value, ctx, cwdPathMode) return cwdPathMode === 'canonical' && key === 'path' ? scrubbed.replaceAll('\\', '/') : scrubbed } @@ -179,6 +194,10 @@ function scrubValue(value: unknown, ctx: NormalizeContext, cwdPathMode: CwdPathM if (value !== null && typeof value === 'object') { const out: Record = {} for (const [k, v] of Object.entries(value)) out[k] = scrubValue(v, ctx, cwdPathMode, k) + if ( + (value as { sessionUpdate?: unknown }).sessionUpdate === 'usage_update' + && typeof (value as { used?: unknown }).used === 'number' + ) out.used = USED_TOKENS return out } return value @@ -282,10 +301,10 @@ export function normalizeStdout( /** * Normalize a session JSONL log into a stable expected output: the header line's - * volatile fields (`createdAt`, `id`, `cwd`) and every event's `time` are - * zeroed/scrubbed, all volatile strings scrubbed, and `seq` is LEFT INTACT - * (deterministic by contract). A packed chunk row's timing (`time0`, the `dt` - * gaps) zeroes just like an event `time`; its `seq0` stays, like `seq`. + * volatile fields (`createdAt`, `id`, `cwd`) are zeroed/scrubbed, ordinary + * event `time` and packed-row `time0` values are zeroed, and all volatile + * strings are scrubbed. Projected inputs remain projected. Packed `data.dt` + * gaps are normalized even when the projected row omits its `time0` anchor. * Output is JSONL in the same shape as the input — one compact record per * line. * @@ -303,32 +322,44 @@ export function normalizeSessionLog( const lines = rawLog.split('\n').filter(line => line.trim().length > 0) const records = lines.map((line) => { const record = JSON.parse(line) as Record - // Header line: { type: 'session', createdAt, id, cwd, … }. if (record.type === 'session') { if ('createdAt' in record) record.createdAt = 0 - } else if ('time0' in record) { - // Packed chunk row: zero the anchor timestamp and every member gap. - record.time0 = 0 + } else if (isPackedFixtureRow(record)) { + if ('time0' in record) record.time0 = 0 const data = record.data if (data !== null && typeof data === 'object' && Array.isArray((data as { dt?: unknown }).dt)) { (data as { dt: unknown[] }).dt = (data as { dt: unknown[] }).dt.map(() => 0) } } else if ('time' in record) { - // Event line: zero the epoch-ms timestamp; keep seq (deterministic). record.time = 0 - // A hook/result carries the hook's wall-clock runtime (`data.durationMs`), - // which is run-to-run noise like `time` — zero it so the expected output reflects - // the hook's decision/exit, not how long the shell took. - if (record.type === 'hook/result' && record.data !== null && typeof record.data === 'object') { - const data = record.data as Record - if ('durationMs' in data) data.durationMs = 0 - } + } + if (record.type === 'hook/result' && record.data !== null && typeof record.data === 'object') { + const data = record.data as Record + if ('durationMs' in data) data.durationMs = 0 } return scrubValue(record, ctx, cwdPathMode) as Record }) return records.map(r => JSON.stringify(r)).join('\n') + '\n' } +/** + * Normalize and project persisted session JSONL for a committed fixture. + * This composes ordinary log normalization with request-header scrubbing and + * persistence-envelope projection. + * + * @param rawLog - persisted or already-projected session JSONL. + * @param ctx - the run's volatile values to scrub. + * @param options - separator output controls. + * @returns normalized committed session snapshot JSONL. + */ +export function normalizeSessionSnapshot( + rawLog: string, + ctx: NormalizeContext, + options: NormalizeOptions = {}, +): string { + return scrubSessionSnapshot(normalizeSessionLog(rawLog, ctx, options)) +} + /** * Replace system-prompt content in request headers with `{{system}}` tokens * while retaining field presence. @@ -373,6 +404,30 @@ export function scrubRequestHeaders(rawLog: string): string { return scrubHeaderContent(rawLog, { system: true, tools: true }) } +/** + * Project a persisted session log while tokenizing all request-header bulk. + * Each non-empty line is parsed at most once; the session header stays + * byte-identical. Body records omit their persistence-only envelopes, and + * request-header payloads are tokenized. + * + * @param rawLog - persisted or already-projected session JSONL. + * @returns committed snapshot JSONL with request headers tokenized. + */ +export function scrubSessionSnapshot(rawLog: string): string { + const scrubbed = scrubRequestHeaders(rawLog) + let recordIndex = 0 + return scrubbed.split('\n').map((line) => { + if (line.trim().length === 0) return line + const record = JSON.parse(line) as Record + if (recordIndex++ === 0) { + if (record.type !== 'session') throw new Error('session snapshot must start with a session header') + return line + } + omitFixtureEnvelope(record) + return JSON.stringify(record) + }).join('\n') +} + /** Which independent request-header payloads a scrubber replaces. */ interface HeaderScrubOptions { system?: boolean diff --git a/packages/test-support/acp-snapshot/src/suite.ts b/packages/test-support/acp-snapshot/src/suite.ts index 1728671c3a..3312089d92 100644 --- a/packages/test-support/acp-snapshot/src/suite.ts +++ b/packages/test-support/acp-snapshot/src/suite.ts @@ -30,6 +30,7 @@ import { normalizeSessionLog, normalizeStdout, scrubRequestHeaders, + scrubSessionSnapshot, scrubSystemPrompts, scrubToolSchemas, tokenizeSessionFixtureCwd, @@ -144,11 +145,10 @@ export interface Scenario { */ headerClass?: string /** - * Alternate LIVE config path (absolute) this scenario boots instead of - * {@link AgentUnderTest.configPath} — an overlay composing a different - * tree (its basename must still end in `cordis.yml` so the bin's replay - * swap finds the sibling `*cordis.snapshot.yml`). A scenario whose - * overlay changes the composed header also needs its own + * Alternate live profile patch (absolute) this scenario boots instead of + * {@link AgentUnderTest.configPath}. Its basename must still end in + * `cordis.yml` so the launcher finds the replay sibling. A scenario whose + * patch changes the composed header also needs its own * {@link headerClass}. */ configPath?: string @@ -691,8 +691,8 @@ export function stabilizeFixtureMessageIds(logs: readonly string[], fixtures: re /** One packed row's member times, or `undefined` for an ordinary record. */ function packedTimes(record: Record): number[] | undefined { if (!PACKED_CHUNK_ROW_TYPES.has(record.type as string)) return undefined - const row = record as unknown as { time0: number; data: { dt: number[] } } - const times = [row.time0] + const row = record as unknown as { time0?: number; data: { dt: number[] } } + const times = [row.time0 ?? 0] for (const gap of row.data.dt) times.push((times[times.length - 1] as number) + gap) return times } @@ -1013,8 +1013,8 @@ function normalizedStringMappings( * complete record layout aligns and volatile strings form a consistent * bijection. Complete durable-message ids are excluded because the later * fixture-ready structural pass owns them. Ambiguous layouts or mappings - * keep fresh strings. Packed timing envelopes expand for alignment, so - * packing does not shift later records; + * keep fresh strings. Packed timing gaps expand from zero when a projected + * fixture omits `time0`, so packing does not shift later records; * fresh semantic values and fragment arrays remain authoritative. * * @param fresh The newly harvested session JSONL. @@ -1193,8 +1193,8 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { ...existsSync(workspaceDir) ? { workspaceDir } : {}, ...scenario.prepareWorkspace !== undefined ? { prepareWorkspace: scenario.prepareWorkspace } : {}, ...scenario.workspaceParent !== undefined ? { workspaceParent: scenario.workspaceParent } : {}, - // A scenario booting an overlay tree passes its own live config; the - // bin's replay swap derives the sibling `*cordis.snapshot.yml` from it. + // A scenario passes its live profile patch; the launcher derives + // the sibling `*cordis.snapshot.yml` for replay. ...scenario.configPath !== undefined ? { configPath: scenario.configPath } : {}, }) @@ -1220,9 +1220,6 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { // Record writes live model fixtures; keyless refresh writes every comparable replayed // fixture. Pinning JSONL keeps prefixes but moves prompts and schemas into sidecars. - const scrub = scenario.pinsHeader === true - ? (log: string): string => scrubToolSchemas(scrubSystemPrompts(log)) - : scrubRequestHeaders const portableFixture = scenario.workspaceParent === undefined ? tokenizeSessionFixtureCwd : (log: string): string => log @@ -1246,13 +1243,13 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { ? refreshFixtureReplacements(result.sessionLogs, existingFixtures) : [] const freshFixtures = REFRESHING - ? result.sessionLogs.map((log, index) => scrub(portableFixture(stabilizeRefreshLog( + ? result.sessionLogs.map((log, index) => scrubSessionSnapshot(portableFixture(stabilizeRefreshLog( log.content, existingFixtures[index] as string, refreshReplacements, ctx, )))) - : result.sessionLogs.map(log => scrub(portableFixture(log.content))) + : result.sessionLogs.map(log => scrubSessionSnapshot(portableFixture(log.content))) const outputFixtures = stabilizeFixtureMessageIds(freshFixtures, existingFixtures) await Promise.all(outputFixtures.map((fixture, index) => writeFile(join(dir, outputFixtureFiles[index] as string), fixture))) @@ -1334,8 +1331,8 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { // The harvested logs (primary-first) must match their committed fixtures 1:1. expect(result.sessionLogs.length, 'this scenario must persist one log per session fixture').toBe(fixtureFiles.length) for (let i = 0; i < fixtureFiles.length; i++) { - const harvested = scrub((result.sessionLogs[i] as HarvestedLog).content) - const fixture = scrub(await readFile(join(dir, fixtureFiles[i] as string), 'utf8')) + const harvested = scrubSessionSnapshot((result.sessionLogs[i] as HarvestedLog).content) + const fixture = scrubSessionSnapshot(await readFile(join(dir, fixtureFiles[i] as string), 'utf8')) expect(normalizeSessionLog(harvested, ctx), `${fixtureFiles[i]} mismatch`) .toEqual(normalizeSessionLog(fixture, fixtureContext(fixture))) } diff --git a/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl index 384b3954cf..00292b57b8 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl @@ -1,3 +1,3 @@ {"type":"session","id":"abababab-cdcd-4efe-8ada-badabadabada","createdAt":800,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","parentSession":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88","delegationDepth":1} -{"type":"request/header","seq":0,"time":2,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} -{"type":"user/message","seq":1,"time":5,"data":{"role":"user","content":[{"type":"text","text":"same inherited message"}],"source":{"kind":"user"},"id":"22222222-2222-4222-8222-222222222222"},"surfaceOp":"append"} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"user/message","data":{"role":"user","content":[{"type":"text","text":"same inherited message"}],"source":{"kind":"user"},"id":"22222222-2222-4222-8222-222222222222"},"surfaceOp":"append"} diff --git a/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl index 0ffe7f9f5f..0e73443adf 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl @@ -1,3 +1,3 @@ {"type":"session","id":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88","createdAt":700,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","delegationDepth":0} -{"type":"request/header","seq":0,"time":3,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} -{"type":"user/message","seq":1,"time":5,"data":{"role":"user","content":[{"type":"text","text":"same inherited message"}],"source":{"kind":"user"},"id":"22222222-2222-4222-8222-222222222222"},"surfaceOp":"append"} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"user/message","data":{"role":"user","content":[{"type":"text","text":"same inherited message"}],"source":{"kind":"user"},"id":"22222222-2222-4222-8222-222222222222"},"surfaceOp":"append"} diff --git a/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.1.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.1.jsonl index 1caf2610b3..3022d0f070 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.1.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.1.jsonl @@ -1,2 +1,2 @@ {"type":"session","id":"abababab-cdcd-4efe-8ada-badabadabada","createdAt":800,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","parentSession":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88"} -{"type":"request/header","seq":0,"time":2,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} diff --git a/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl index 40bb4d37dc..3f3c253101 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl @@ -1,2 +1,2 @@ {"type":"session","id":"ccdc749f-56f3-4267-9750-598b5c60b7b2","createdAt":600,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-nOQ4Gy","delegationDepth":0} -{"type":"request/header","seq":0,"time":4,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} diff --git a/packages/test-support/acp-snapshot/tests/fixtures/suite/authored-error/session.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/suite/authored-error/session.jsonl index a202c7141d..7de48a3768 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/suite/authored-error/session.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/suite/authored-error/session.jsonl @@ -1,2 +1,2 @@ {"type":"session","id":"44444444-3333-4222-8111-000000000000","createdAt":17,"cwd":"/rec/authored-cwd","delegationDepth":0} -{"type":"turn/end","seq":1,"time":17,"data":{"error":"model exploded"}} +{"type":"turn/end","data":{"error":"model exploded"}} diff --git a/packages/test-support/acp-snapshot/tests/fixtures/suite/blocked-log/session.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/suite/blocked-log/session.jsonl index 75eb18fa71..a55b440570 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/suite/blocked-log/session.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/suite/blocked-log/session.jsonl @@ -1,2 +1,2 @@ {"type":"session","id":"99999999-8888-4777-8666-555555555555","createdAt":13,"cwd":"/rec/blocked-cwd","delegationDepth":0} -{"type":"hook/result","seq":1,"time":13,"data":{"decision":"block","durationMs":99}} +{"type":"hook/result","data":{"decision":"block","durationMs":99}} diff --git a/packages/test-support/acp-snapshot/tests/fixtures/suite/pin-turn/session.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/suite/pin-turn/session.jsonl index 693e728d2c..c5a48ce7eb 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/suite/pin-turn/session.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/suite/pin-turn/session.jsonl @@ -1,4 +1,4 @@ {"type":"session","id":"12121212-3434-4545-8686-787878787878","createdAt":7,"cwd":"/rec/pin-cwd","delegationDepth":0} -{"type":"request/header","seq":0,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} -{"type":"request/header","seq":1,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"change"}} -{"type":"turn/start","seq":2,"time":7,"data":{"turn":1}} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"change"}} +{"type":"turn/start","data":{"turn":1}} diff --git a/packages/test-support/acp-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl index 5fb01dbe8a..4a99404db1 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl @@ -1,2 +1,2 @@ {"type":"session","id":"eeeeeeee-1111-4222-8333-444444444444","createdAt":12,"cwd":"/rec/plain-cwd","parentSession":"56565656-7878-4989-8a9a-9b9b9b9b9b9b","delegationDepth":1} -{"type":"request/header","seq":0,"time":12,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} diff --git a/packages/test-support/acp-snapshot/tests/fixtures/suite/plain-turn/session.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/suite/plain-turn/session.jsonl index 6bdf784ce2..a8ef50bdd7 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/suite/plain-turn/session.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/suite/plain-turn/session.jsonl @@ -1,3 +1,3 @@ {"type":"session","id":"56565656-7878-4989-8a9a-9b9b9b9b9b9b","createdAt":11,"cwd":"/rec/plain-cwd","delegationDepth":0} -{"type":"request/header","seq":0,"time":11,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} -{"type":"assistant/chunk","seq":1,"time":11,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"hi"}}} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"hi"}}} diff --git a/packages/test-support/acp-snapshot/tests/fixtures/suite/shared-pin/session.jsonl b/packages/test-support/acp-snapshot/tests/fixtures/suite/shared-pin/session.jsonl index 8a9407cbc0..87329aa45b 100644 --- a/packages/test-support/acp-snapshot/tests/fixtures/suite/shared-pin/session.jsonl +++ b/packages/test-support/acp-snapshot/tests/fixtures/suite/shared-pin/session.jsonl @@ -1,4 +1,4 @@ {"type":"session","id":"13131313-3434-4545-8686-787878787878","createdAt":7,"cwd":"/rec/shared-pin-cwd","delegationDepth":0} -{"type":"request/header","seq":0,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} -{"type":"request/header","seq":1,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"change"}} -{"type":"turn/start","seq":2,"time":7,"data":{"turn":1}} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"change"}} +{"type":"turn/start","data":{"turn":1}} diff --git a/packages/test-support/acp-snapshot/tests/harness.spec.ts b/packages/test-support/acp-snapshot/tests/harness.spec.ts index 68dc58c770..9a2d70855a 100644 --- a/packages/test-support/acp-snapshot/tests/harness.spec.ts +++ b/packages/test-support/acp-snapshot/tests/harness.spec.ts @@ -1,8 +1,8 @@ -import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' +import { mkdir, mkdtemp, readFile, readdir, realpath, rm, symlink, writeFile } from 'node:fs/promises' import { once } from 'node:events' import { tmpdir } from 'node:os' import { delimiter, join, relative, sep } from 'node:path' -import { fileURLToPath } from 'node:url' +import { fileURLToPath, pathToFileURL } from 'node:url' import { afterAll, describe, expect, it, vi } from 'vitest' import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' import { runScenario, snapshotSpillRoot, type AgentUnderTest, type InputStep } from '../src/harness.ts' @@ -44,6 +44,13 @@ const AGENT: AgentUnderTest = { tsconfigPath: fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)), } +async function materializedPatch(root: string, filename: string): Promise { + const entries = await readdir(root, { recursive: true }) + const matches = entries.filter(entry => entry.endsWith(filename)) + expect(matches).toHaveLength(1) + return join(root, matches[0] as string) +} + /** Temp scenario dirs to drop after the suite. */ const tempDirs: string[] = [] afterAll(async () => { @@ -142,6 +149,110 @@ describe('runScenario', () => { expect(exited).toBe(true) }) + it('builds dsh profile argv and rebases relative modules in live and replay patches', async () => { + const { dir, fixtureFile } = await scenario({}) + const patchDir = join(dir, 'patches') + const basePatch = join(patchDir, 'base.cordis.yml') + const selectedPatch = join(patchDir, 'selected.cordis.yml') + const replayPatch = join(patchDir, 'selected.cordis.snapshot.yml') + const packageDir = join(dir, 'node_modules', 'example-package') + const scopedPackageDir = join(dir, 'node_modules', '@fixture', 'example-package') + await Promise.all([ + mkdir(patchDir, { recursive: true }), + mkdir(packageDir, { recursive: true }), + mkdir(scopedPackageDir, { recursive: true }), + ]) + await Promise.all([ + writeFile(basePatch, [ + '- id: asserted', + ' name: ./plugin.mjs', + ' disabled: true', + '- insert:', + ' - id: relative', + ' name: ./plugin.mjs', + ' - id: package', + ' name: example-package', + ' - id: scoped-package', + " name: '@fixture/example-package/subpath'", + ' - id: builtin', + ' name: fs', + ' - id: group', + ' name: cordis:group', + ' group: true', + ' config:', + ' - id: nested', + ' name: ../nested.mjs', + '', + ].join('\n')), + writeFile(selectedPatch, '[]\n'), + writeFile(replayPatch, '[]\n'), + writeFile(join(patchDir, 'plugin.mjs'), 'export function apply() {}\n'), + writeFile(join(packageDir, 'package.json'), '{"name":"example-package","version":"1.0.0"}\n'), + writeFile(join(scopedPackageDir, 'package.json'), '{"name":"@fixture/example-package","version":"1.0.0"}\n'), + ]) + const profileAgent: AgentUnderTest = { ...AGENT, configPath: basePatch, profile: 'acp' } + + const live = launchAcpTestAgent({ + agent: profileAgent, + cwd: dir, + configPath: selectedPatch, + env: { DSH_SNAPSHOT: 'record', DSH_SNAPSHOT_FILE: fixtureFile }, + }) + await live.spawned + await live.close() + const materializedRoot = join(dir, '.dsh-profile-patches') + const materialized = await readFile(await materializedPatch(materializedRoot, '0-base.cordis.yml'), 'utf8') + expect(materialized).toContain(pathToFileURL(join(patchDir, 'plugin.mjs')).href) + expect(materialized).toContain(pathToFileURL(join(dir, 'nested.mjs')).href) + expect(materialized).toContain('example-package') + expect(await realpath(join(dir, '.dsh', 'profiles', 'node_modules', 'example-package'))) + .toBe(await realpath(packageDir)) + expect(await realpath(join(dir, '.dsh', 'profiles', 'node_modules', '@fixture', 'example-package'))) + .toBe(await realpath(scopedPackageDir)) + expect(await readFile(await materializedPatch(materializedRoot, '1-selected.cordis.yml'), 'utf8')).toContain('[]') + + const replay = launchAcpTestAgent({ + agent: profileAgent, + cwd: dir, + configPath: selectedPatch, + env: { DSH_SNAPSHOT: 'replay', DSH_SNAPSHOT_FILE: fixtureFile }, + }) + await replay.spawned + await replay.close() + expect(await readFile(await materializedPatch(materializedRoot, '1-selected.cordis.snapshot.yml'), 'utf8')) + .toContain('[]') + expect((await readdir(materializedRoot, { withFileTypes: true })).filter(entry => entry.isDirectory())) + .toHaveLength(2) + + const conflictPatch = join(dir, 'conflict.cordis.yml') + const conflictPackage = join(dir, 'node_modules', 'conflict-package') + const otherPackage = join(dir, 'other-conflict-package') + const conflictLink = join(dir, '.dsh', 'profiles', 'node_modules', 'conflict-package') + await Promise.all([ + mkdir(conflictPackage, { recursive: true }), + mkdir(otherPackage, { recursive: true }), + writeFile(conflictPatch, '- id: conflict\n name: conflict-package\n'), + ]) + await Promise.all([ + writeFile(join(conflictPackage, 'package.json'), '{"name":"conflict-package","version":"1.0.0"}\n'), + writeFile(join(otherPackage, 'package.json'), '{"name":"conflict-package","version":"2.0.0"}\n'), + ]) + await symlink(otherPackage, conflictLink, process.platform === 'win32' ? 'junction' : 'dir') + expect(() => launchAcpTestAgent({ + agent: { ...profileAgent, configPath: conflictPatch }, + cwd: dir, + env: { DSH_SNAPSHOT: 'record', DSH_SNAPSHOT_FILE: fixtureFile }, + })).toThrow('ACP profile package conflict-package resolves to two directories') + + const invalidPatch = join(dir, 'invalid.cordis.yml') + await writeFile(invalidPatch, 'not: a-list\n') + expect(() => launchAcpTestAgent({ + agent: { ...profileAgent, configPath: invalidPatch }, + cwd: dir, + env: { DSH_SNAPSHOT: 'record', DSH_SNAPSHOT_FILE: fixtureFile }, + })).toThrow(`ACP profile patch must be a top-level array: ${invalidPatch}`) + }) + it('waits for inherited stdio and buffered ACP parsing after the parent exits', { timeout: 20_000 }, async () => { const { dir, fixtureFile } = await scenario({ lateInheritedOutput: true }) const launched = launchAcpTestAgent({ @@ -407,6 +518,24 @@ describe('runScenario', () => { expect((JSON.parse(sessionLine) as { cwd?: string }).cwd).toBe(result.cwd) }) + it('drives a structured prompt-content step without flattening its wire blocks', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({}) + const result = await runScenario( + { + steps: [...boot, { + op: 'promptContent', + content: [ + { type: 'text', text: 'before' }, + { type: 'image', data: 'AQ==', mimeType: 'image/png' }, + { type: 'text', text: 'after' }, + ], + }], + }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.rawStdout).toContain('"stopReason":"end_turn"') + }) + it('forwards override/child fixture paths into the child env and captures stderr', { timeout: 20_000 }, async () => { const { dir, fixtureFile } = await scenario({ echoEnv: true, stderrNote: 'fake bin booted' }) const childFiles = [join(dir, 'session.1.jsonl'), join(dir, 'session.2.jsonl')] @@ -687,9 +816,9 @@ describe('runScenario', () => { it('waitForTurnStart rejects missing, earlier, and malformed durable turns', { timeout: 20_000 }, async () => { const missing = await scenario({}) await expect(runScenario( - { steps: [...boot, { op: 'waitForTurnStart', timeoutMs: 20 }] }, + { steps: [...boot, { op: 'waitForTurnStart', timeoutMs: 200 }] }, { agent: AGENT, mode: 'replay', fixtureFile: missing.fixtureFile }, - )).rejects.toThrow(/did not persist turn\/start within 20ms/) + )).rejects.toThrow(/did not persist turn\/start within 200ms/) const earlier = await scenario({ prompt: 'hang-until-cancel', @@ -707,11 +836,11 @@ describe('runScenario', () => { steps: [ ...boot, { op: 'promptAndCancel', text: 'hang' }, - { op: 'waitForTurnStart', minimumTurn: 3, timeoutMs: 20 }, + { op: 'waitForTurnStart', minimumTurn: 3, timeoutMs: 200 }, ], }, { agent: AGENT, mode: 'replay', fixtureFile: earlier.fixtureFile }, - )).rejects.toThrow(/turn\/start at or beyond turn 3 within 20ms/) + )).rejects.toThrow(/turn\/start at or beyond turn 3 within 200ms/) const closed = await scenario({ prompt: 'hang-until-cancel', @@ -730,11 +859,11 @@ describe('runScenario', () => { steps: [ ...boot, { op: 'promptAndCancel', text: 'hang' }, - { op: 'waitForTurnStart', timeoutMs: 20 }, + { op: 'waitForTurnStart', timeoutMs: 200 }, ], }, { agent: AGENT, mode: 'replay', fixtureFile: closed.fixtureFile }, - )).rejects.toThrow(/did not persist turn\/start within 20ms/) + )).rejects.toThrow(/did not persist turn\/start within 200ms/) for (const turn of [undefined, 0]) { const malformed = await scenario({ @@ -1097,6 +1226,7 @@ describe('runScenario', () => { it.each([ [{ op: 'prompt', text: 'x' }, /prompt before newSession/], + [{ op: 'promptContent', content: [{ type: 'text', text: 'x' }] }, /promptContent before newSession/], [{ op: 'promptAndWaitForAgentMessage', text: 'x', waitForText: 'later' }, /promptAndWaitForAgentMessage before newSession/], [{ op: 'promptExpectError', text: 'x' }, /promptExpectError before newSession/], [{ op: 'promptAndCancel', text: 'x' }, /promptAndCancel before newSession/], diff --git a/packages/test-support/acp-snapshot/tests/normalize.spec.ts b/packages/test-support/acp-snapshot/tests/normalize.spec.ts index aea21eb747..86c6059a81 100644 --- a/packages/test-support/acp-snapshot/tests/normalize.spec.ts +++ b/packages/test-support/acp-snapshot/tests/normalize.spec.ts @@ -3,8 +3,10 @@ import { type NormalizeContext, extractSnapshotSpillPaths, normalizeSessionLog, + normalizeSessionSnapshot, normalizeStdout, scrubRequestHeaders, + scrubSessionSnapshot, scrubSystemPrompts, scrubToolSchemas, tokenizeSessionFixtureCwd, @@ -46,6 +48,47 @@ describe('normalizeStdout', () => { expect(out).not.toContain(ctx.sessionIds[0] as string) }) + it('keeps standard message identity distinct from session identity', () => { + const raw = JSON.stringify({ + jsonrpc: '2.0', + method: 'session/update', + params: { + sessionId: ctx.sessionIds[0], + update: { + sessionUpdate: 'agent_message_chunk', + messageId: 'aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee', + content: { type: 'text', text: 'done' }, + }, + }, + }) + + const out = normalizeStdout(raw, ctx) + + expect(out).toContain('"sessionId":"{{sessionId}}"') + expect(out).toContain('"messageId":"{{messageId}}"') + }) + + it('stabilizes path-dependent context occupancy without hiding capacity', () => { + const raw = JSON.stringify({ + jsonrpc: '2.0', + method: 'session/update', + params: { + sessionId: ctx.sessionIds[0], + update: { sessionUpdate: 'usage_update', used: 6_438, size: 1_000_000 }, + }, + }) + + const frame = JSON.parse(normalizeStdout(raw, ctx)) as { + params: { update: { used: string; size: number } } + } + + expect(frame.params.update).toEqual({ + sessionUpdate: 'usage_update', + used: '{{usedTokens}}', + size: 1_000_000, + }) + }) + it('scrubs cwd at file URI and chained-punctuation boundaries', () => { const raw = JSON.stringify({ jsonrpc: '2.0', @@ -218,13 +261,22 @@ describe('normalizeSessionLog', () => { expect(out).not.toContain('123') }) - it('zeroes each event time but keeps seq', () => { + it('preserves event sequence and zeroes event time', () => { const out = normalizeSessionLog(`${header({})}\n${event({ seq: 7, time: 999 })}\n`, ctx) expect(out).toContain('"time":0') - expect(out).toContain('"seq":7') // seq is deterministic — NOT scrubbed + expect(out).toContain('"seq":7') expect(out).not.toContain('999') }) + it('normalizes a projected event without adding a persistence envelope', () => { + const projected = JSON.stringify({ type: 'turn/start', data: { turn: 1 } }) + const out = normalizeSessionLog(`${header({})}\n${projected}\n`, ctx) + expect(JSON.parse(out.trimEnd().split('\n')[1] ?? '{}')).toStrictEqual({ + type: 'turn/start', + data: { turn: 1 }, + }) + }) + it('scrubs cwd and session id deep inside event data', () => { const ev = JSON.stringify({ type: 'tool/result', seq: 2, time: 5, @@ -370,7 +422,7 @@ describe('normalizeSessionLog', () => { expect(out).toContain('"decision":"block"') // the decision is the behavior — kept }) - it('zeroes a packed chunk row\'s time0 and dt gaps but keeps seq0 and payload', () => { + it('preserves a packed chunk row\'s sequence, zeroes time, and zeroes volatile dt gaps', () => { const row = JSON.stringify({ type: 'text-chunks', seq0: 7, time0: 999, data: { turn: 1, step: 1, index: 0, dt: [212, 27, 0], texts: ['a', 'b', 'c', 'd'] }, @@ -378,17 +430,17 @@ describe('normalizeSessionLog', () => { const out = normalizeSessionLog(`${header({})}\n${row}\n`, ctx) expect(out).toContain('"time0":0') expect(out).toContain('"dt":[0,0,0]') - expect(out).toContain('"seq0":7') // seq0 is deterministic, like seq — NOT scrubbed + expect(out).toContain('"seq0":7') expect(out).toContain('"texts":["a","b","c","d"]') expect(out).not.toContain('999') expect(out).not.toContain('212') }) - it('zeroes time0 even when a malformed row carries no dt array', () => { + it('normalizes a headerless packed-like stream record without decoding it', () => { const row = JSON.stringify({ type: 'text-chunks', seq0: 1, time0: 999, data: 'not-an-object' }) - const out = normalizeSessionLog(`${header({})}\n${row}\n`, ctx) + const out = normalizeSessionLog(`${row}\n`, ctx) + expect(out).toContain('"seq0":1') expect(out).toContain('"time0":0') - expect(out).not.toContain('999') }) it('leaves a non-hook event durationMs untouched (only hook/result is scrubbed)', () => { @@ -397,18 +449,50 @@ describe('normalizeSessionLog', () => { expect(out).toContain('"durationMs":88') }) - it('tolerates records missing the volatile fields it would zero', () => { + it('handles complete envelopes when optional normalized fields are absent', () => { const bareHeader = JSON.stringify({ type: 'session', id: 's' }) - const timeless = JSON.stringify({ type: 'note', seq: 1 }) const bareHook = JSON.stringify({ type: 'hook/result', seq: 2, time: 5, data: { decision: 'allow' } }) const nullDataHook = JSON.stringify({ type: 'hook/result', seq: 3, time: 6, data: null }) - const out = normalizeSessionLog(`${bareHeader}\n${timeless}\n${bareHook}\n${nullDataHook}\n`, ctx) - expect(out).toContain('"type":"note","seq":1') + const out = normalizeSessionLog(`${bareHeader}\n${bareHook}\n${nullDataHook}\n`, ctx) expect(out).toContain('"decision":"allow"') expect(out).not.toContain('durationMs') }) }) +describe('normalizeSessionSnapshot', () => { + it('normalizes, scrubs, and projects each parsed body record', () => { + const raw = [ + JSON.stringify({ type: 'session', version: 0, createdAt: 123, cwd: ctx.cwd }), + JSON.stringify({ + type: 'request/header', + seq: 7, + time: 999, + data: { header: { system: 'volatile', tools: [{ name: 'tool' }] } }, + }), + ].join('\n') + '\n' + expect(normalizeSessionSnapshot(raw, ctx)).toBe([ + JSON.stringify({ type: 'session', version: 0, createdAt: 0, cwd: '{{cwd}}' }), + JSON.stringify({ type: 'request/header', data: { header: { system: '{{system}}', tools: '{{tools}}' } } }), + ].join('\n') + '\n') + }) + + it('normalizes an already-projected packed row', () => { + const raw = [ + JSON.stringify({ type: 'session', version: 0 }), + JSON.stringify({ + type: 'text-chunks', + data: { turn: 1, step: 1, index: 0, dt: [9], texts: ['a', 'b'] }, + }), + ].join('\n') + '\n' + expect(normalizeSessionSnapshot(raw, ctx)).toContain('"dt":[0]') + }) + + it('rejects headerless input', () => { + expect(() => normalizeSessionSnapshot('{"type":"turn/start"}\n', ctx)) + .toThrow('session snapshot must start with a session header') + }) +}) + describe('tokenizeSessionFixtureCwd', () => { it.each([ { @@ -565,6 +649,32 @@ describe('scrubRequestHeaders', () => { }) }) +describe('scrubSessionSnapshot', () => { + it('preserves the header while projecting and scrubbing each body record', () => { + const header = ' {"type":"session","version":0,"id":"s","createdAt":7} ' + const request = JSON.stringify({ + type: 'request/header', seq: 0, time: 9, + data: { header: { system: 'secret', tools: [{ name: 'read' }] }, reason: 'initial' }, + }) + const event = JSON.stringify({ + type: 'turn/start', seq: 1, time: 10, + data: { turn: 1, seq: 41, time: 42 }, + }) + + expect(scrubSessionSnapshot(`${header}\n${request}\n${event}\n`)).toBe([ + header, + '{"type":"request/header","data":{"header":{"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}', + '{"type":"turn/start","data":{"turn":1,"seq":41,"time":42}}', + '', + ].join('\n')) + }) + + it('rejects headerless input', () => { + expect(() => scrubSessionSnapshot('{"type":"turn/start"}\n')) + .toThrow('session snapshot must start with a session header') + }) +}) + describe('scrubSystemPrompts', () => { it('scrubs only system prompt payloads while keeping tools verbatim', () => { const header = JSON.stringify({ diff --git a/packages/test-support/acp-snapshot/tests/suite.spec.ts b/packages/test-support/acp-snapshot/tests/suite.spec.ts index 8f072bc2e5..d8e7892b92 100644 --- a/packages/test-support/acp-snapshot/tests/suite.spec.ts +++ b/packages/test-support/acp-snapshot/tests/suite.spec.ts @@ -932,6 +932,25 @@ describe('stabilizeRefreshLog', () => { ].join('\n')) }) + it('reads projected packed fixtures while retaining their data gaps', () => { + const fresh = [ + '{"type":"session","id":"same","createdAt":200}', + '{"type":"text-chunks","seq0":2,"time0":200,"data":{"turn":1,"step":1,"index":0,"dt":[5,7],"texts":["new",""," split"]}}', + '', + ].join('\n') + const existing = [ + '{"type":"session","id":"same","createdAt":100}', + '{"type":"text-chunks","data":{"turn":1,"step":1,"index":0,"dt":[1,2],"texts":["old","chunk","shape"]}}', + '', + ].join('\n') + + expect(stabilize(fresh, existing)).toBe([ + '{"type":"session","id":"same","createdAt":100}', + '{"type":"text-chunks","seq0":2,"time0":0,"data":{"turn":1,"step":1,"index":0,"dt":[1,2],"texts":["new",""," split"]}}', + '', + ].join('\n')) + }) + it.each([ ['the old run is absent', [], 200], ['the old run is shorter', [100, 101], 100], @@ -1069,7 +1088,7 @@ describe('stabilizeRefreshLog', () => { const existing = [ '{"type":"session","id":"old-child","createdAt":100,"cwd":"/old","parentSession":"old-parent","seedLength":5}', '{"type":"hook/result","seq":1,"time":11,"data":{"decision":"stale","durationMs":99}}', - '{"type":"turn/end","seq":2,"data":{"error":"stale"}}', + '{"type":"turn/end","seq":2,"time":12,"data":{"error":"stale"}}', '{"type":"assistant/message","seq":3,"time":12,"data":{"text":"different type"}}', '{"type":"hook/result","seq":4,"time":13,"data":{"decision":"stale"}}', '', @@ -1082,7 +1101,7 @@ describe('stabilizeRefreshLog', () => { ])).toBe([ '{"type":"session","id":"old-child","createdAt":100,"cwd":"/old","parentSession":"old-parent","seedLength":1}', '{"type":"hook/result","seq":1,"time":11,"data":{"decision":"block","durationMs":99}}', - '{"type":"turn/end","seq":2,"time":33,"data":{"error":"fresh error"}}', + '{"type":"turn/end","seq":2,"time":12,"data":{"error":"fresh error"}}', '{"type":"tool/result","seq":3,"time":44,"data":{"text":"old-parent in /old"}}', '{"type":"hook/result","seq":4,"time":13,"data":{"decision":"allow","durationMs":5}}', '', diff --git a/packages/test-support/acp-snapshot/tsconfig.json b/packages/test-support/acp-snapshot/tsconfig.json index 5c4fa7d80a..b258ab7851 100644 --- a/packages/test-support/acp-snapshot/tsconfig.json +++ b/packages/test-support/acp-snapshot/tsconfig.json @@ -8,6 +8,9 @@ "src" ], "references": [ + { + "path": "../../../vendor/include" + }, { "path": "../loader-smoke" }, diff --git a/packages/test-support/agent-loop-testkit/package.json b/packages/test-support/agent-loop-testkit/package.json index 4e0bb8aa9c..cba0828f2b 100644 --- a/packages/test-support/agent-loop-testkit/package.json +++ b/packages/test-support/agent-loop-testkit/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-agent-loop-testkit", "description": "Shared prerequisite mounting for tests that exercise the concrete agent loop", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/test-support/client-runtime/README.i18n.yaml b/packages/test-support/client-runtime/README.i18n.yaml index 4d8edbf29c..d17ac65b2a 100644 --- a/packages/test-support/client-runtime/README.i18n.yaml +++ b/packages/test-support/client-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/test-support/client-runtime/README.md -README.md: 15646d05daae0a39a36639f335a71d6d208da13b -README.zh.md: 9624bc8ffc997ba786f3cb1bba670344d44ac60d +README.md: 443cadb63d2e290874d7248d7fb104c7340d55c7 +README.zh.md: 596f6a90f1223a6108fd09b66c1451539c8ce1fd diff --git a/packages/test-support/client-runtime/README.md b/packages/test-support/client-runtime/README.md index 15646d05da..443cadb63d 100644 --- a/packages/test-support/client-runtime/README.md +++ b/packages/test-support/client-runtime/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -jsdom slot test runtime for client feature specs: a real Cordis `Context`, the production `SlotRegistry` and web-react renderer, assembled around typed session/workspace doubles. Feature suites exercise declaration, registration, scope, store, inject, rendering, updates, and disposal without hand-building the machinery per suite — and without a second implementation of any production logic. +jsdom slot test runtime for client feature specs: a real Cordis `Context`, the production `SlotRegistry` and UI renderer, assembled around typed session/workspace doubles. Feature suites exercise declaration, registration, scope, store, inject, rendering, updates, and disposal without hand-building the machinery per suite — and without a second implementation of any production logic. The doubles implement the same outward faces features receive through ctx (`TestSessions implements ISessions`, `TestWorkspaces implements IWorkspaces`; each fixture session is a `FixtureSession implements SessionFace`; `stubSettingsScope` is a `SettingsScope` with test-driven publications and a write spy), so a production face change breaks the bench at compile time instead of silently drifting. Provide-bundle materialization runs the production `SessionProvideChannel` — the one implementation shared with `SessionRuntime`. Fixtures feed plain data: list rows, conversation snapshots (immer-patched via `updateSnapshot`), projection values, and `ISession`-typed behavior stubs that fail loud when a spec calls an unstubbed verb. The typed `provide()` constrains fakes for declared service names to `Partial` of that service's outward face. diff --git a/packages/test-support/client-runtime/README.zh.md b/packages/test-support/client-runtime/README.zh.md index 9624bc8ffc..596f6a90f1 100644 --- a/packages/test-support/client-runtime/README.zh.md +++ b/packages/test-support/client-runtime/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -面向客户端功能测试的 jsdom slot 测试运行时:真实 Cordis `Context`、生产 `SlotRegistry` 与 web-react 渲染器,围绕带类型的 session/workspace 测试替身组装。功能套件无需逐套件手搭机器即可测遍声明、注册、scope、store、inject、渲染、更新与销毁——且不存在任何生产逻辑的第二份实现。 +面向客户端功能测试的 jsdom slot 测试运行时:真实 Cordis `Context`、生产 `SlotRegistry` 与 UI 渲染器,围绕带类型的 session/workspace 测试替身组装。功能套件无需逐套件手搭机器即可测遍声明、注册、scope、store、inject、渲染、更新与销毁——且不存在任何生产逻辑的第二份实现。 替身实现的正是功能通过 ctx 获得的对外接口(`TestSessions implements ISessions`、`TestWorkspaces implements IWorkspaces`;每个 fixture session 是 `FixtureSession implements SessionFace`;`stubSettingsScope` 是发布由测试驱动、带写入 spy 的 `SettingsScope`),生产面一旦改形,测试台在编译期即断,而非静默漂移。provide bundle 材料化直接运行生产 `SessionProvideChannel`——与 `SessionRuntime` 共用同一份实现。fixture 灌入的是普通数据:列表行、会话快照(经 `updateSnapshot` 以 immer 补丁改写)、projection 值,以及按 `ISession` 取型的行为桩——spec 调用未打桩的动词时报错自明。带类型的 `provide()` 将已声明服务名的 fake 约束为该服务对外面的 `Partial` 子集。 diff --git a/packages/test-support/client-runtime/package.json b/packages/test-support/client-runtime/package.json index 9a6fb3a7a0..fb39c32177 100644 --- a/packages/test-support/client-runtime/package.json +++ b/packages/test-support/client-runtime/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-client-test-runtime", - "description": "jsdom slot test runtime: real Cordis Context + SlotRegistry + web-react renderer with test-owned session/workspace doubles for feature specs", - "version": "0.0.1-rc.5", + "description": "jsdom slot test runtime: real Cordis Context + SlotRegistry + UI renderer with test-owned session/workspace doubles for feature specs", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -34,7 +34,7 @@ "peerDependencies": { "@deepseek-ai/dsh-client-runtime": "workspace:^", "@deepseek-ai/dsh-client-ui-slots": "workspace:^", - "@deepseek-ai/dsh-client-web-react": "workspace:^", + "@deepseek-ai/dsh-client-ui-renderer": "workspace:^", "@deepseek-ai/dsh-host-apiproxy": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/cordis": "workspace:^", @@ -44,7 +44,7 @@ "devDependencies": { "@deepseek-ai/dsh-client-runtime": "workspace:^", "@deepseek-ai/dsh-client-ui-slots": "workspace:^", - "@deepseek-ai/dsh-client-web-react": "workspace:^", + "@deepseek-ai/dsh-client-ui-renderer": "workspace:^", "@deepseek-ai/dsh-host-apiproxy": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@types/react": "~18.3.1", diff --git a/packages/test-support/client-runtime/src/index.ts b/packages/test-support/client-runtime/src/index.ts index dc513e388b..37aebf66ee 100644 --- a/packages/test-support/client-runtime/src/index.ts +++ b/packages/test-support/client-runtime/src/index.ts @@ -1,6 +1,6 @@ /** * jsdom slot test runtime: a real small runtime — Cordis `Context`, the - * runtime `SlotRegistry`, and the web-react renderer — assembled around + * runtime `SlotRegistry`, and the UI renderer — assembled around * test-owned session/workspace doubles, so feature specs exercise * declaration, registration, scope, store, inject, rendering, updates, and * disposal without hand-building the machinery per suite. @@ -25,15 +25,18 @@ import type { BoundFunctions } from '@testing-library/dom' import { ConversationEventRegistry, ConversationViewRegistry, SlotRegistry, } from '@deepseek-ai/dsh-client-runtime/client' -import { createSlotRenderer } from '@deepseek-ai/dsh-client-web-react' +import { bindSnapshotSelector as bindRendererSnapshotSelector } from '@deepseek-ai/dsh-client-ui-renderer/src/client/bind.ts' +import { createSlotRenderer as createRenderer } from '@deepseek-ai/dsh-client-ui-renderer/src/client/scoped-slots.tsx' import type { - ChildrenDecl, ComposedProps, OwnerOf, SlotComponent, SlotMap, SlotRendererHost, StoreInstanceLike, + ChildrenDecl, ComposedProps, HostObservable, OwnerOf, SlotComponent, SlotMap, SlotRenderer, + SlotRendererHost, SnapshotSelectorHook, StoreInstanceLike, } from '@deepseek-ai/dsh-client-ui-slots' import { registerDomSnapshotSerializer } from './snapshot.ts' import { TestSessions } from './sessions.ts' import { TestWorkspaces } from './workspaces.ts' import type { Stabilizer } from './fixtures.ts' +export type { UseSession } from '@deepseek-ai/dsh-client-ui-renderer/client' export { domSnapshotSerializer, registerDomSnapshotSerializer } from './snapshot.ts' export { FixtureSession, TestSessions } from './sessions.ts' export { stubSettingsScope } from './settings-scope.ts' @@ -45,6 +48,23 @@ export type { SessionBehaviorOverrides, SessionFixture, Stabilizer } from './fix export { makeTranslate } from './translate.ts' export { usePinnedBrowserLanguages } from './locale-env.ts' +/** + * Bind an observable source to the production renderer's selector hook. + * @param source - Observable snapshot source. + * @returns Typed React selector hook. + */ +export function bindSnapshotSelector(source: HostObservable): SnapshotSelectorHook { + return bindRendererSnapshotSelector(source) +} + +/** + * Create the production slot renderer used by client feature tests. + * @returns Slot renderer instance. + */ +export function createSlotRenderer(): SlotRenderer { + return createRenderer() +} + /** Erased register face for the internal root call (the public declaration contract holds the typing). */ type ErasedRegister = (options: object, component: unknown) => () => void diff --git a/packages/test-support/client-runtime/tests/remote.client.spec.ts b/packages/test-support/client-runtime/tests/remote.client.spec.ts index 3f11c82323..90b1bde56f 100644 --- a/packages/test-support/client-runtime/tests/remote.client.spec.ts +++ b/packages/test-support/client-runtime/tests/remote.client.spec.ts @@ -30,7 +30,7 @@ describe('TestRemote', () => { new TestRemote(ctx) // No subscriber for this name: the emit must be inert rather than throwing, // because the wire carries whatever the Host allowlist selected. - expect(() => { ctx.remote.$dispatch('credentials/updated', ['DEEPSEEK_API_KEY']) }).not.toThrow() + expect(() => { ctx.remote.$dispatch('credentials/reference-updated', ['DEEPSEEK_API_KEY']) }).not.toThrow() await ctx.fiber.dispose() }) diff --git a/packages/test-support/client-runtime/tsconfig.json b/packages/test-support/client-runtime/tsconfig.json index 3b87e03c46..63e3a17eba 100644 --- a/packages/test-support/client-runtime/tsconfig.json +++ b/packages/test-support/client-runtime/tsconfig.json @@ -15,7 +15,7 @@ "path": "../../client/ui-slots" }, { - "path": "../../client/web-react" + "path": "../../client/ui-renderer" }, { "path": "../../client/runtime" diff --git a/packages/test-support/llm-mock-server/package.json b/packages/test-support/llm-mock-server/package.json index 2555c27275..eac77fc152 100644 --- a/packages/test-support/llm-mock-server/package.json +++ b/packages/test-support/llm-mock-server/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-llm-mock-server", "description": "Scriptable OpenAI-compatible HTTP/SSE fault server for LLM recovery tests", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/test-support/llm-replay/README.i18n.yaml b/packages/test-support/llm-replay/README.i18n.yaml index 8e84398842..3f61eb41f1 100644 --- a/packages/test-support/llm-replay/README.i18n.yaml +++ b/packages/test-support/llm-replay/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/test-support/llm-replay/README.md -README.md: 24ee2684ad5ff393b95db0075950fe40bce2e0dd -README.zh.md: 5adb16e08604081ebc8db4e07a8a22c031d02ae2 +README.md: e12d359950e1caf6e31b9d25035c50bb83c73747 +README.zh.md: 83964fed15c6a985df8a92327748918c1616a83e diff --git a/packages/test-support/llm-replay/README.md b/packages/test-support/llm-replay/README.md index 24ee2684ad..e12d359950 100644 --- a/packages/test-support/llm-replay/README.md +++ b/packages/test-support/llm-replay/README.md @@ -8,11 +8,13 @@ Its consumers are the ACP and headless `stream-json` snapshot suites plus the We ## How the fixture works -The fixture IS the persisted session log (`/session.jsonl`). Its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each agent-loop `stream()` call's chunk sequence. A successful compaction summarizer is logged differently: when `compaction/summary` carries `llmStreamCall: true` and its complete `rawOutput`, replay reconstructs a canonical successful stream at that event's position using one `block-start`/`block-end` pair per block, the recorded usage when present, and a terminal `stop`. Exact provider delta partitioning is not part of the durable compaction result. `rawOutput` without the marker does not imply a local LLM call because template and remote summarizers may retain complete output without using this context's adapter. +The fixture is a projection of a persisted session log (`/session.jsonl`): it keeps the header and every event payload, but omits body `seq`/`time` envelopes (`seq0`/`time0` for packed rows). Replay restores contiguous synthetic envelopes while parsing; one file must contain either projected body rows or complete persisted body rows, never both. Runtime persistence still writes the complete log. The fixture's `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each agent-loop `stream()` call's chunk sequence. A successful compaction summarizer is logged differently: when `compaction/summary` carries `llmStreamCall: true` and its complete `rawOutput`, replay reconstructs a canonical successful stream at that event's position using one `block-start`/`block-end` pair per block, the recorded usage when present, and a terminal `stop`. Exact provider delta partitioning is not part of the durable compaction result. `rawOutput` without the marker does not imply a local LLM call because template and remote summarizers may retain complete output without using this context's adapter. Recording is therefore "run the real agent once and harvest the `.jsonl`", done by the snapshot harness — this plugin does not record. A fixture may carry its `request/header` content tokenized to `{{system}}`/`{{tools}}` (the harness pins that content in one scenario and scrubs the rest); replay is indifferent — derivation reads only `assistant/chunk` and `compaction/summary` events plus the line-0 session header. -Two failure modes are not reconstructable from `assistant/chunk` alone — a pure throw before any chunk (e.g. an HTTP 401, where the log holds only a `turn/end {error}` and no chunks) and a cancel/hang (timing, not chunk content). A scenario that needs those supplies an optional sidecar (`/replay.override.json`) that either replaces the derived script (a bare `ReplayEntry[]`) or augments it (`{ patches: [{ at, entry }] }`: keep every JSONL-derived call and swap the named 0-based call indexes; `at` equal to the derived length appends the retry attempt after an injected transient throw). Patch indexes must be unique. The override document, each patch and entry, and every chunk discriminant are validated when the file loads. A `hang` entry may name `readyFile`; replay writes that empty marker after its prefix chunks reach the loop and before it waits for cancellation, so an external driver can cancel deterministically without observing a presentation update. +When replay serves `deepseek-official` in a composition carrying `ctx.deepseekLlmApiExtensions`, it prepares and accepts those fields after selecting a valid script entry and before yielding its first chunk. This mirrors the live adapter's post-2xx commit point, so durable acceptance watermarks and SDK event notifications remain identical between recording and replay. Replay supplies a synthetic `{ messages: [] }` base body: it proves acceptance side effects, not prepared field bytes. An extension whose acceptance action depends on the serialized provider body needs dedicated fixture support before this replay path can represent it. Other provider routes and compositions without the optional registry are unchanged. + +Two failure modes are not reconstructable from `assistant/chunk` alone — a pure throw before any chunk (e.g. an HTTP 401, where the log holds only a `turn/end {error}` and no chunks) and a cancel/hang (timing, not chunk content). A scenario that needs those supplies an optional sidecar (`/replay.override.json`) that either replaces the derived script (a bare `ReplayEntry[]`) or augments it (`{ patches: [{ at, entry }] }`: keep every JSONL-derived call and swap the named 0-based call indexes; `at` equal to the derived length appends the retry attempt after an injected transient throw). Patch indexes must be unique. A `throw` entry accepts DeepSeek request extensions when it has prefix chunks; a zero-chunk throw defaults to pre-2xx non-acceptance and can set `accepted: true` for a post-2xx failure without chunks. The override document, each patch and entry, and every chunk discriminant are validated when the file loads. A `hang` entry may name `readyFile`; replay writes that empty marker after its prefix chunks reach the loop and before it waits for cancellation, so an external driver can cancel deterministically without observing a presentation update. A scripted string may embed `{{fromRequest:}}` to fill a value no static sidecar can know — for example a randomly minted goal id the model must echo back into `update_goal`. At stream time every placeholder resolves against the live request: the corpus is every string leaf of the request messages joined by newlines, the pattern's LAST corpus match wins, and its first capture group (or the whole match without one) substitutes in place. A pattern that matches nothing, an invalid pattern, and an unterminated placeholder each fail loud. The last two braces of a consecutive `}` run terminate the placeholder, so a pattern may end with a brace quantifier (`[0-9a-f]{4}`) but cannot contain `}}` followed by further pattern content. Resolution applies to every scripted entry, including ones derived from the recorded JSONL — a recorded fixture whose text legitimately contains the literal marker must be expressed through a sidecar without it. @@ -59,7 +61,7 @@ Replay keys every call by its calling session id (`GenerateOptions.sessionId`, s - `installLlmReplay(ctx, config)` — install the configured replay adapter or catch-all `llm/stream` listener; returns a `ReplayHandle` (`dispose()` for HMR safety plus `assertConsumed()`, the teardown check that every recorded script bound to a live session and every bound cursor drained — turning a scenario that silently drove fewer model calls than recorded into a crisp diagnostic). Use this in tests to drive replay without the Loader or env vars. - `loadSessionScripts(config)` — resolve the ordered `SessionScript[]` (primary + children) for a scenario, ready to bind to live sessions in first-call order. - `loadReplayScript(config)` — resolve the `ReplayEntry[]` for the primary session only (validated sidecar replacement/patches if present, else derived from the JSONL; fail-loud if the fixture is missing). -- `deriveReplayScript(events)` / `parseSessionLog(text)` / `parseSessionHeader(text)` / `resolveScriptedEntry(entry, messages)` — the pure helpers that turn ordinary loop chunks and explicitly marked local compaction outputs in a recorded session log into a script, read its header `id`/`createdAt`, and resolve `{{fromRequest:...}}` placeholders against one live request. A derived assistant group must end in a `finish` chunk; a group without one is the fingerprint of a thrown `stream()` and must instead be expressed via an override sidecar. +- `deriveReplayScript(events)` / `parseSessionLog(text)` / `parseSessionHeader(text)` / `resolveScriptedEntry(entry, messages)` — the pure helpers that turn ordinary loop chunks and explicitly marked local compaction outputs in a persisted or projected session log into a script, read its header `id`/`createdAt`, and resolve `{{fromRequest:...}}` placeholders against one live request. A derived assistant group must end in a `finish` chunk; a group without one is the fingerprint of a thrown `stream()` and must instead be expressed via an override sidecar. - Types `ReplayEntry` / `ReplayOverrideDoc` / `ReplayOverridePatch` / `SessionScript` / `ReplayConfig` / `ReplayProviderConfig` / `ReplayModelConfig` / `ReplayHandle` / `Config`. ## Plugin export shape diff --git a/packages/test-support/llm-replay/README.zh.md b/packages/test-support/llm-replay/README.zh.md index 5adb16e086..83964fed15 100644 --- a/packages/test-support/llm-replay/README.zh.md +++ b/packages/test-support/llm-replay/README.zh.md @@ -8,11 +8,13 @@ ## fixture 的工作方式 -fixture 就是持久化的会话日志(`/session.jsonl`)。其 `assistant/chunk` 事件包含每个 `StreamChunk`,因此按 `(turn, step)` 分组即可重建每次 agent loop(智能体循环)的 `stream()` 调用的分片序列。压缩(compaction)摘要器成功时,日志记录方式有所不同:当 `compaction/summary` 携带 `llmStreamCall: true` 和完整的 `rawOutput` 时,回放会在该事件的位置重建一条规范成功流,其中每个块各使用一对 `block-start`/`block-end`,带上已记录的用量(如有),并以 `stop` 终止。提供方增量的精确切分不属于持久压缩结果。不带该标记的 `rawOutput` 并不意味着发生了本地 LLM 调用,因为模板摘要器和远程摘要器即使未使用此上下文的适配器,也可能保留完整输出。 +fixture 是持久化会话日志(`/session.jsonl`)的投影:它保留 header 与每个事件 payload,但省略正文的 `seq`/`time` envelope(打包行使用 `seq0`/`time0`)。回放在解析时恢复连续的 synthetic envelope;同一文件的正文必须全部采用投影行或完整持久化行,不能混用。运行时持久化仍写入完整日志。fixture 的 `assistant/chunk` 事件包含每个 `StreamChunk`,因此按 `(turn, step)` 分组即可重建每次 agent loop(智能体循环)的 `stream()` 调用的分片序列。压缩(compaction)摘要器成功时,日志记录方式有所不同:当 `compaction/summary` 携带 `llmStreamCall: true` 和完整的 `rawOutput` 时,回放会在该事件的位置重建一条规范成功流,其中每个块各使用一对 `block-start`/`block-end`,带上已记录的用量(如有),并以 `stop` 终止。提供方增量的精确切分不属于持久压缩结果。不带该标记的 `rawOutput` 并不意味着发生了本地 LLM 调用,因为模板摘要器和远程摘要器即使未使用此上下文的适配器,也可能保留完整输出。 因此,录制就是「运行一次真实 agent 并收集 `.jsonl`」,由快照 harness 完成;该插件本身不录制。fixture 的 `request/header` 内容可能被标记化为 `{{system}}`/`{{tools}}`(harness 会在一个场景中固定该内容,并清除其余场景中的内容);回放不受影响,因为派生过程只读取 `assistant/chunk` 和 `compaction/summary` 事件以及第 0 行的会话 header。 -有两种失败模式无法仅根据 `assistant/chunk` 重建:在产生任何分片前直接抛出异常(例如 HTTP 401,此时日志只有 `turn/end {error}` 而没有分片),以及取消或挂起(差异在时序,而非分片内容)。需要这些行为的场景可提供伴随文件(`/replay.override.json`):它可以替换派生脚本(裸 `ReplayEntry[]`),也可以增补派生脚本(`{ patches: [{ at, entry }] }`:保留所有从 JSONL 派生的调用,只替换指定的从 0 开始计数的调用索引;当 `at` 等于派生长度时,则在注入瞬态异常后的重试位置追加一次调用)。补丁索引不得重复。文件加载时会校验覆写文档、每个补丁和条目,以及每个分片的判别标签。`hang` 条目可以指定 `readyFile`;当前缀分片到达循环后、开始等待取消前,回放会写入这个空标记,使外部驱动程序无需观察展示层更新即可确定性地取消。 +当回放在带有 `ctx.deepseekLlmApiExtensions` 的组合中提供 `deepseek-official` 时,它会在选中有效脚本条目后、产出第一个分片前准备并接受这些字段。这会复现实时适配器的 2xx 后提交点,使持久接受水位与 SDK 事件通知在录制和回放之间保持一致。回放会提供合成的 `{ messages: [] }` 基础正文:它证明的是接受副作用,而不是已准备字段的字节内容。如果某个扩展的接受操作依赖序列化后的提供方正文,该扩展需要专用 fixture 支持,此回放路径才能表示它。其他提供方路由以及未挂载该可选注册表的组合不受影响。 + +有两种失败模式无法仅根据 `assistant/chunk` 重建:在产生任何分片前直接抛出异常(例如 HTTP 401,此时日志只有 `turn/end {error}` 而没有分片),以及取消或挂起(差异在时序,而非分片内容)。需要这些行为的场景可提供伴随文件(`/replay.override.json`):它可以替换派生脚本(裸 `ReplayEntry[]`),也可以增补派生脚本(`{ patches: [{ at, entry }] }`:保留所有从 JSONL 派生的调用,只替换指定的从 0 开始计数的调用索引;当 `at` 等于派生长度时,则在注入瞬态异常后的重试位置追加一次调用)。补丁索引不得重复。带前缀分片的 `throw` 条目会接受 DeepSeek 请求扩展;零分片 `throw` 默认为 2xx 前不接受,无分片的 2xx 后失败可显式设置 `accepted: true`。文件加载时会校验覆写文档、每个补丁和条目,以及每个分片的判别标签。`hang` 条目可以指定 `readyFile`;当前缀分片到达循环后、开始等待取消前,回放会写入这个空标记,使外部驱动程序无需观察展示层更新即可确定性地取消。 脚本字符串可以内嵌 `{{fromRequest:}}`,用来填入静态伴随文件不可能预知的值——例如模型必须原样回填到 `update_goal` 的随机生成 goal id。回放时每个占位符针对实时请求解析:语料是请求消息的所有字符串叶子按换行拼接的结果,取该模式在语料中的最后一次匹配,用其第一个捕获组(无捕获组时用整个匹配)原位替换。模式匹配不到内容、模式非法、占位符未闭合都会明确报错。连续右花括号串的最后两个花括号才是占位符结束符,因此模式可以以花括号量词收尾(如 `[0-9a-f]{4}`),但不能在 `}}` 之后还有后续模式内容。解析作用于所有脚本条目,包括从已记录 JSONL 派生的条目——若录制文本本身合法地含有该字面量标记,需改用不含标记的伴随文件表达。 @@ -59,12 +61,12 @@ fixture 就是持久化的会话日志(`/session.jsonl`)。其 `as - `installLlmReplay(ctx, config)`:安装已配置回放适配器或 catch-all `llm/stream` 监听器;返回 `ReplayHandle`(包含用于保证 HMR(热模块替换)安全的 `dispose()`,以及清理阶段执行的 `assertConsumed()` 检查;后者确保每个已记录脚本都绑定到实时会话,且每个已绑定游标都已耗尽,从而将场景静默驱动的模型调用少于记录数转换为明确诊断)。在测试中使用它,可以不通过 Loader 或 env var 驱动回放。 - `loadSessionScripts(config)`:解析场景中有序的 `SessionScript[]`(主会话 + 子会话),准备按首次调用顺序绑定到实时会话。 - `loadReplayScript(config)`:只解析主会话的 `ReplayEntry[]`(如果伴随文件存在,则使用经校验的替换或补丁;否则从 JSONL 派生;fixture 缺失时明确报错)。 -- `deriveReplayScript(events)` / `parseSessionLog(text)` / `parseSessionHeader(text)` / `resolveScriptedEntry(entry, messages)`:将已记录会话日志中的普通 loop 分片和显式标记的本地压缩输出转换为脚本、读取其 header `id`/`createdAt`、并针对单次实时请求解析 `{{fromRequest:...}}` 占位符的纯辅助工具。派生的 assistant 分组必须以 `finish` 分片结束;没有该分片的分组是 `stream()` 抛出异常的指纹,必须改用 override 伴随文件表达。 +- `deriveReplayScript(events)` / `parseSessionLog(text)` / `parseSessionHeader(text)` / `resolveScriptedEntry(entry, messages)`:将持久化或投影后会话日志中的普通 loop 分片和显式标记的本地压缩输出转换为脚本、读取其 header `id`/`createdAt`、并针对单次实时请求解析 `{{fromRequest:...}}` 占位符的纯辅助工具。派生的 assistant 分组必须以 `finish` 分片结束;没有该分片的分组是 `stream()` 抛出异常的指纹,必须改用 override 伴随文件表达。 - 类型 `ReplayEntry` / `ReplayOverrideDoc` / `ReplayOverridePatch` / `SessionScript` / `ReplayConfig` / `ReplayProviderConfig` / `ReplayModelConfig` / `ReplayHandle` / `Config`。 ## 插件导出形态 -命名导出 `name` / `inject` / `Config` / `apply`,且**没有默认导出**:Cordis Loader 的 `unwrapExports` 执行 `exports.default ?? exports`,因此意外的默认导出会将模块折叠为函数本身,并丢弃 `inject` 命名空间(见 [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md))。 +命名导出 `name` / `inject` / `Config` / `apply`,且**没有默认导出**:Cordis Loader 的 `unwrapExports` 执行 `exports.default ?? exports`,因此意外的默认导出会将模块折叠为函数本身,并丢弃 `inject` 命名空间(见 [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.zh.md))。 ## 模型体验 diff --git a/packages/test-support/llm-replay/package.json b/packages/test-support/llm-replay/package.json index 3d5d70010b..315a3b456b 100644 --- a/packages/test-support/llm-replay/package.json +++ b/packages/test-support/llm-replay/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-llm-replay", "description": "Replay LLM plugin: short-circuits llm/stream with model chunks reconstructed from a recorded session JSONL (keyless snapshot tests)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", @@ -33,13 +33,20 @@ "license": "MIT", "peerDependencies": { "@deepseek-ai/dsh-compaction": "workspace:^", + "@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/cordis": "workspace:^" }, + "peerDependenciesMeta": { + "@deepseek-ai/dsh-deepseek-llm-api-extensions": { + "optional": true + } + }, "devDependencies": { "@deepseek-ai/dsh-compaction": "workspace:^", + "@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", diff --git a/packages/test-support/llm-replay/src/index.ts b/packages/test-support/llm-replay/src/index.ts index 867e2f2b45..779963fa65 100644 --- a/packages/test-support/llm-replay/src/index.ts +++ b/packages/test-support/llm-replay/src/index.ts @@ -11,8 +11,8 @@ import { existsSync, readFileSync, writeFileSync } from 'node:fs' import { delimiter as pathDelimiter } from 'node:path' import type { Context } from '@deepseek-ai/cordis' import type {} from '@deepseek-ai/dsh-compaction' -import { decodeStorageRecord } from '@deepseek-ai/dsh-session' -import type { SessionEvent } from '@deepseek-ai/dsh-session' +import type {} from '@deepseek-ai/dsh-deepseek-llm-api-extensions' +import { decodeStorageRecord, type SessionEvent } from '@deepseek-ai/dsh-session' import type { ContentBlock, GenerateOptions, @@ -27,6 +27,8 @@ import type { } from '@deepseek-ai/dsh-llm' import { LlmAdapter, LlmError, ReasoningEffortId, assertNever, resolveRetryPolicy } from '@deepseek-ai/dsh-llm' +const PACKED_CHUNK_ROW_TYPES = new Set(['text-chunks', 'reasoning-chunks', 'tool-call-chunks']) + /** * One recorded model call. `throw` may replay prefix chunks before failing; * `hang` models cancellation. Derived chunk entries come from ordinary model @@ -35,7 +37,7 @@ import { LlmAdapter, LlmError, ReasoningEffortId, assertNever, resolveRetryPolic */ export type ReplayEntry = | { kind: 'chunks'; chunks: StreamChunk[] } - | { kind: 'throw'; chunks: StreamChunk[]; message: string; code: string } + | { kind: 'throw'; chunks: StreamChunk[]; message: string; code: string; accepted?: boolean } | { kind: 'hang' /** Optional marker written after the prefix chunks are consumed and before the stream waits for cancellation. */ @@ -114,7 +116,7 @@ export interface ReplayConfig { * this long before yielding, so a downstream transport (e.g. the web SSE * mux observed by a browser) sees genuinely incremental delivery. A realism * knob only — correctness must never depend on it. Absent or `0` keeps - * today's synchronous burst yield. Must be a non-negative finite integer; + * a synchronous burst yield. Must be a non-negative finite integer; * aborting mid-wait cancels the stream like any other abort. */ paceMs?: number @@ -165,11 +167,43 @@ export interface SessionScript { * @returns every event after the header, in log order. */ export function parseSessionLog(text: string): SessionEvent[] { - const lines = text.split('\n').filter(line => line.trim().length > 0) const events: SessionEvent[] = [] - // The JSONL backend guarantees line 0 is the session header. - for (let i = 1; i < lines.length; i++) { - events.push(...decodeStorageRecord(JSON.parse(lines[i] as string))) + let nextSeq = 0 + let headerSkipped = false + // The JSONL backend guarantees line 0 is the session header. Projected + // fixtures omit event envelopes; synthesize them while decoding so callers + // still receive complete SessionEvent values. + for (const [index, line] of text.split(/\r?\n/).entries()) { + if (line.trim().length === 0) continue + if (!headerSkipped) { + headerSkipped = true + continue + } + let value: unknown + try { + value = JSON.parse(line) as unknown + } catch (error) { + throw new Error(`session snapshot line ${index + 1} contains invalid JSON`, { cause: error }) + } + if (value === null || typeof value !== 'object' || Array.isArray(value)) { + throw new Error(`session snapshot line ${index + 1} must be a JSON object`) + } + const record = value as Record + const packed = PACKED_CHUNK_ROW_TYPES.has(record.type as string) + const seqKey = packed ? 'seq0' : 'seq' + const timeKey = packed ? 'time0' : 'time' + if (!Object.hasOwn(record, seqKey)) record[seqKey] = nextSeq + if (!Object.hasOwn(record, timeKey)) record[timeKey] = 0 + let decoded: SessionEvent[] + try { + decoded = decodeStorageRecord(record) + } catch (error) { + /* v8 ignore next -- decodeStorageRecord only throws Error instances; the String arm satisfies unknown narrowing. */ + const detail = error instanceof Error ? error.message : String(error) + throw new Error(`session snapshot line ${index + 1}: ${detail}`, { cause: error }) + } + events.push(...decoded) + nextSeq += decoded.length } return events } @@ -415,7 +449,11 @@ function readReplayEntry(value: unknown, file: string, location: string): Replay return { kind: 'chunks', chunks: readChunks(value['chunks'], file, location) } } case 'throw': { - if (!hasExactKeys(value, ['kind', 'chunks', 'message', 'code'])) { + const accepted = value['accepted'] + const keys = accepted === undefined + ? ['kind', 'chunks', 'message', 'code'] + : ['kind', 'chunks', 'message', 'code', 'accepted'] + if (!hasExactKeys(value, keys)) { invalidOverride(file, location, 'has invalid throw-entry fields') } if (typeof value['message'] !== 'string' || value['message'].length === 0) { @@ -424,11 +462,15 @@ function readReplayEntry(value: unknown, file: string, location: string): Replay if (typeof value['code'] !== 'string' || value['code'].length === 0) { invalidOverride(file, location, 'code must be a non-empty string') } + if (accepted !== undefined && typeof accepted !== 'boolean') { + invalidOverride(file, location, 'accepted must be a boolean') + } return { kind: 'throw', chunks: readChunks(value['chunks'], file, location), message: value['message'], code: value['code'], + ...(accepted === undefined ? {} : { accepted }), } } case 'hang': { @@ -683,6 +725,20 @@ async function* replayEntry(entry: ReplayEntry, signal: AbortSignal | undefined, } } +/** Whether the scripted provider call reached the live adapter's post-2xx commit point. */ +function providerAccepted(entry: ReplayEntry): boolean { + switch (entry.kind) { + case 'chunks': + case 'hang': + return true + case 'throw': + return entry.accepted ?? entry.chunks.length > 0 + /* v8 ignore next -- override parsing and derived entries close the local union before replay. */ + default: + return assertNever(entry, 'llm-replay acceptance entry') + } +} + /** * Install per-session positional replay. A newly seen live session takes the * next ordered recorded script, then advances its own cursor synchronously at @@ -742,7 +798,22 @@ export function installLlmReplay(ctx: Context, config: ReplayConfig): ReplayHand + `but its script has only ${boundState.entries.length}; re-record the scenario`, ) } - yield* replayEntry(resolveScriptedEntry(entry, options.messages), options.signal, paceMs) + const resolved = resolveScriptedEntry(entry, options.messages) + if (options.provider === 'deepseek-official' && providerAccepted(resolved)) { + const extensions = ctx.get('deepseekLlmApiExtensions') + if (extensions !== undefined) { + const signal = options.signal ?? new AbortController().signal + const prepared = await extensions.prepare({ + // Replay reproduces post-2xx side effects, not the provider wire body. + body: { messages: [] }, + signal, + ...options.sessionId === undefined ? {} : { sessionId: String(options.sessionId) }, + ...options.purpose === undefined ? {} : { purpose: options.purpose }, + }) + await prepared.accept() + } + } + yield* replayEntry(resolved, options.signal, paceMs) })() } const providers = config.providers ?? [] diff --git a/packages/test-support/llm-replay/tests/llm-replay.spec.ts b/packages/test-support/llm-replay/tests/llm-replay.spec.ts index 837f5d9af0..bffa976dc7 100644 --- a/packages/test-support/llm-replay/tests/llm-replay.spec.ts +++ b/packages/test-support/llm-replay/tests/llm-replay.spec.ts @@ -1,10 +1,11 @@ import { existsSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' -import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' import type { SessionEvent } from '@deepseek-ai/dsh-session' import { CompactionId } from '@deepseek-ai/dsh-compaction' +import DeepSeekLlmApiExtensionRegistry from '@deepseek-ai/dsh-deepseek-llm-api-extensions' import LlmRuntime, { CallId, createUserMessage, GenerateOptions, LlmAdapter, StreamChunk } from '@deepseek-ai/dsh-llm' import { type Config, @@ -22,6 +23,12 @@ import { resolveScriptedEntry, } from '../src/index.ts' +declare module '@deepseek-ai/dsh-deepseek-llm-api-extensions/types' { + interface DeepSeekLlmApiExtensionMap { + test_replay: { readonly version: 1 } + } +} + /** * Unit tests for the replay llm/stream plugin. These drive the listener through * the REAL LlmRuntime waterfall (not a hand-rolled stub) so they verify the @@ -96,6 +103,12 @@ describe('parseSessionLog', () => { expect(parseSessionLog(`${header}\n\n${JSON.stringify(ev)}\n\n`)).toEqual([ev]) }) + it('rejects non-object body rows with their source line', () => { + const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 }) + expect(() => parseSessionLog(`${header}\nnull\n`)) + .toThrow('session snapshot line 2 must be a JSON object') + }) + it('expands a packed chunk row into its events (a fixture recorded with packChunks on)', () => { const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 }) const row = JSON.stringify({ @@ -108,6 +121,20 @@ describe('parseSessionLog', () => { chunkEvent(3, 1, 1, { type: 'text-delta', index: 0, text: 'c' }), ]) }) + + it('synthesizes omitted ordinary and packed snapshot envelopes', () => { + const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 7 }) + const ordinary = JSON.stringify({ type: 'turn/start', data: { turn: 1 } }) + const packed = JSON.stringify({ + type: 'text-chunks', + data: { turn: 1, step: 1, index: 0, dt: [3], texts: ['a', 'b'] }, + }) + expect(parseSessionLog(`${header}\n${ordinary}\n${packed}\n`)).toEqual([ + { type: 'turn/start', seq: 0, time: 0, data: { turn: 1 } }, + chunkEvent(1, 1, 1, { type: 'text-delta', index: 0, text: 'a' }), + { ...chunkEvent(2, 1, 1, { type: 'text-delta', index: 0, text: 'b' }), time: 3 }, + ]) + }) }) describe('deriveReplayScript', () => { @@ -644,7 +671,7 @@ describe('installLlmReplay (through the real LlmRuntime)', () => { }) expect(ctx.llm.providerRetryPolicy('empty')).toMatchObject({ mode: 'normal', - maxRetries: 2, + maxRetries: 5, initialDelayMs: 500, maxDelayMs: 10_000, jitterRatio: 0.1, @@ -682,6 +709,90 @@ describe('installLlmReplay (through the real LlmRuntime)', () => { expect(await drain(ctx.llm.stream({ provider: 'm', model: 'm', messages: [] }))).toEqual(second) }) + it('settles official DeepSeek request extensions before replayed chunks', async () => { + writeLog(TEXT_CHUNKS, TEXT_CHUNKS, TEXT_CHUNKS) + const ctx = new Context() + await ctx.plugin(LlmRuntime) + await ctx.plugin(DeepSeekLlmApiExtensionRegistry) + const accepted = vi.fn() + ctx.deepseekLlmApiExtensions.register('test_replay', { + prepare: () => ({ value: { version: 1 }, accept: accepted }), + }) + installLlmReplay(ctx, { file }) + + const sessionId = 'deepseek-replay' as NonNullable + await drain(ctx.llm.stream({ provider: 'deepseek-official', model: 'm', messages: [], sessionId })) + expect(accepted).toHaveBeenCalledOnce() + await drain(ctx.llm.stream({ + provider: 'deepseek-official', + model: 'm', + messages: [], + sessionId, + signal: new AbortController().signal, + purpose: 'compaction', + })) + expect(accepted).toHaveBeenCalledTimes(2) + await drain(ctx.llm.stream({ provider: 'another-provider', model: 'm', messages: [], sessionId })) + expect(accepted).toHaveBeenCalledTimes(2) + }) + + it('accepts anonymous official extensions and tolerates an absent optional registry', async () => { + writeLog(TEXT_CHUNKS) + const withRegistry = new Context() + await withRegistry.plugin(LlmRuntime) + await withRegistry.plugin(DeepSeekLlmApiExtensionRegistry) + const accepted = vi.fn() + withRegistry.deepseekLlmApiExtensions.register('test_replay', { + prepare: () => ({ value: { version: 1 }, accept: accepted }), + }) + installLlmReplay(withRegistry, { file }) + await drain(withRegistry.llm.stream({ provider: 'deepseek-official', model: 'm', messages: [] })) + expect(accepted).toHaveBeenCalledOnce() + + const withoutRegistry = new Context() + await withoutRegistry.plugin(LlmRuntime) + installLlmReplay(withoutRegistry, { file }) + await expect(drain(withoutRegistry.llm.stream({ provider: 'deepseek-official', model: 'm', messages: [] }))) + .resolves.toEqual(TEXT_CHUNKS) + }) + + it('accepts only throw entries that reached the post-2xx point', async () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + writeFileSync(overrideFile, JSON.stringify([ + { kind: 'throw', chunks: [{ type: 'block-start', index: 0, blockType: 'text' }], message: 'partial', code: 'STREAM_CLOSED' }, + { kind: 'throw', chunks: [], message: 'unauthorized', code: 'AUTH' }, + { kind: 'throw', chunks: [], message: 'empty body', code: 'EMPTY_RESPONSE', accepted: true }, + ]), 'utf8') + const ctx = new Context() + await ctx.plugin(LlmRuntime) + await ctx.plugin(DeepSeekLlmApiExtensionRegistry) + const accepted = vi.fn() + ctx.deepseekLlmApiExtensions.register('test_replay', { + prepare: () => ({ value: { version: 1 }, accept: accepted }), + }) + installLlmReplay(ctx, { file, overrideFile }) + const request = { provider: 'deepseek-official', model: 'm', messages: [] } + + await expect(drain(ctx.llm.stream(request))).rejects.toThrow('partial') + expect(accepted).toHaveBeenCalledOnce() + await expect(drain(ctx.llm.stream(request))).rejects.toThrow('unauthorized') + expect(accepted).toHaveBeenCalledOnce() + await expect(drain(ctx.llm.stream(request))).rejects.toThrow('empty body') + expect(accepted).toHaveBeenCalledTimes(2) + }) + + it('rejects a non-boolean throw acceptance override', async () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + writeFileSync(overrideFile, JSON.stringify([ + { kind: 'throw', chunks: [], message: 'bad', code: 'X', accepted: 'yes' }, + ]), 'utf8') + const ctx = new Context() + await ctx.plugin(LlmRuntime) + expect(() => { installLlmReplay(ctx, { file, overrideFile }) }).toThrow(/accepted must be a boolean/) + }) + it('replays a sidecar throw-entry as an LlmError with its stable code, after its prefix chunks', async () => { writeFileSync(file, sessionJsonl([]), 'utf8') const overrideFile = join(dir, 'replay.override.json') diff --git a/packages/test-support/llm-replay/tsconfig.json b/packages/test-support/llm-replay/tsconfig.json index 6683cbc73b..5c84dff52e 100644 --- a/packages/test-support/llm-replay/tsconfig.json +++ b/packages/test-support/llm-replay/tsconfig.json @@ -20,6 +20,9 @@ { "path": "../../llm/llm" }, + { + "path": "../../llm/deepseek-llm-api-extensions" + }, { "path": "../../core/session" }, diff --git a/packages/test-support/loader-smoke/package.json b/packages/test-support/loader-smoke/package.json index 20e6c1a6d8..2772c11be6 100644 --- a/packages/test-support/loader-smoke/package.json +++ b/packages/test-support/loader-smoke/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-loader-smoke", "description": "Shared subprocess and direct-agent harness for keyless real-Loader example smoke tests", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/test-support/loader-smoke/src/index.ts b/packages/test-support/loader-smoke/src/index.ts index 33ec56ec0e..7f8b262ead 100644 --- a/packages/test-support/loader-smoke/src/index.ts +++ b/packages/test-support/loader-smoke/src/index.ts @@ -65,6 +65,8 @@ export interface ExampleLaunchOptions { readonly mode?: ExampleMode /** Absolute repo tsconfig whose `paths` map resolves unbuilt workspace imports. Required in `src` mode, ignored in `lib`. */ readonly tsconfigPath?: string + /** Select the ESM-only tsx hook instead of the generic loader. */ + readonly sourceImport?: 'tsx/esm' /** Extra environment entries the mode-specific ones layer over; the caller then merges the result over `process.env`. */ readonly env?: NodeJS.ProcessEnv } @@ -113,7 +115,9 @@ export function resolveExampleLaunch(options: ExampleLaunchOptions): ExampleLaun if (options.tsconfigPath === undefined) { throw new Error("resolveExampleLaunch: 'src' mode needs tsconfigPath for the workspace paths map.") } - const tsxLoader = import.meta.resolve('tsx') + const tsxLoader = options.sourceImport === 'tsx/esm' + ? import.meta.resolve('tsx/esm') + : import.meta.resolve('tsx') env.TSX_TSCONFIG_PATH = options.tsconfigPath return { command: process.execPath, args: ['--import', tsxLoader, options.srcBin, ...configArgs], env } } diff --git a/packages/test-support/loader-smoke/tests/example-launch.spec.ts b/packages/test-support/loader-smoke/tests/example-launch.spec.ts index 48bb58b09d..03b5e39ee4 100644 --- a/packages/test-support/loader-smoke/tests/example-launch.spec.ts +++ b/packages/test-support/loader-smoke/tests/example-launch.spec.ts @@ -5,7 +5,7 @@ import { resolveExampleMode, } from '@deepseek-ai/dsh-loader-smoke' -const SRC_BIN = '/repo/packages/examples/acp-demo/src/bin.ts' +const SRC_BIN = '/repo/apps/cli/src/bin.ts' const TSCONFIG = '/repo/tsconfig.json' const originalMode = process.env[EXAMPLE_MODE_ENV] @@ -57,6 +57,17 @@ describe('resolveExampleLaunch', () => { expect(() => resolveExampleLaunch({ srcBin: SRC_BIN, mode: 'src' })).toThrow(/needs tsconfigPath/) }) + it('src mode: resolves an app-specific tsx import hook', () => { + const { args } = resolveExampleLaunch({ + srcBin: SRC_BIN, + mode: 'src', + sourceImport: 'tsx/esm', + tsconfigPath: TSCONFIG, + }) + expect(args[0]).toBe('--import') + expect(args[1]).toContain('/tsx/dist/esm/index.mjs') + }) + it('lib mode: plain node on the derived lib bin, no tsx and no paths env', () => { const { args, env } = resolveExampleLaunch({ srcBin: SRC_BIN, @@ -65,7 +76,7 @@ describe('resolveExampleLaunch', () => { env: { DSH_HOME: '/tmp/home' }, }) expect(args).not.toContain('--import') - expect(args).toContain('/repo/packages/examples/acp-demo/lib/bin.js') + expect(args).toContain('/repo/apps/cli/lib/bin.js') expect(args.slice(-2)).toEqual(['--config', './cordis.yml']) expect(env.TSX_TSCONFIG_PATH).toBeUndefined() expect(env.DSH_HOME).toBe('/tmp/home') @@ -79,18 +90,18 @@ describe('resolveExampleLaunch', () => { it('lib mode: rewrites only the last /src/ segment', () => { const { args } = resolveExampleLaunch({ - srcBin: '/repo/src/packages/examples/acp-demo/src/bin.ts', + srcBin: '/repo/src/apps/cli/src/bin.ts', mode: 'lib', }) - expect(args).toContain('/repo/src/packages/examples/acp-demo/lib/bin.js') + expect(args).toContain('/repo/src/apps/cli/lib/bin.js') }) it('lib mode: derives the built bin from a Windows source path', () => { const { args } = resolveExampleLaunch({ - srcBin: String.raw`D:\repo\src\packages\examples\acp-demo\src\bin.ts`, + srcBin: String.raw`D:\repo\src\apps\cli\src\bin.ts`, mode: 'lib', }) - expect(args).toContain(String.raw`D:\repo\src\packages\examples\acp-demo\lib\bin.js`) + expect(args).toContain(String.raw`D:\repo\src\apps\cli\lib\bin.js`) }) it('lib mode: throws when the bin has no /src/ segment', () => { @@ -100,6 +111,6 @@ describe('resolveExampleLaunch', () => { it('defaults the mode from the environment', () => { process.env[EXAMPLE_MODE_ENV] = 'lib' const { args } = resolveExampleLaunch({ srcBin: SRC_BIN }) - expect(args).toContain('/repo/packages/examples/acp-demo/lib/bin.js') + expect(args).toContain('/repo/apps/cli/lib/bin.js') }) }) diff --git a/packages/todo/README.i18n.yaml b/packages/todo/README.i18n.yaml index 4df1b35bcb..825b502560 100644 --- a/packages/todo/README.i18n.yaml +++ b/packages/todo/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/todo/README.md -README.md: 6bbc9fe4f04cc4a82573c643c5c00bbecde069ed -README.zh.md: ae7de52eeded877e92c94dbc6c6e1addf76aa8b4 +README.md: 85499ac2382ec0f220a7fafda0acbbed369f9ca2 +README.zh.md: 9056208e82d4abc1db51bcae21205f4e5afd8bf9 diff --git a/packages/todo/README.md b/packages/todo/README.md index 6bbc9fe4f0..85499ac238 100644 --- a/packages/todo/README.md +++ b/packages/todo/README.md @@ -10,4 +10,4 @@ The model-facing todo capability. It is a single **product** package because one The child README owns the tool, persistence, and rendering contract. -The event payload is documented on [docs/subsystems/session.md](../../docs/subsystems/session.md). +The event payload is documented on [docs/subsystems/todo.md](../../docs/subsystems/todo.md). diff --git a/packages/todo/README.zh.md b/packages/todo/README.zh.md index ae7de52eed..9056208e82 100644 --- a/packages/todo/README.zh.md +++ b/packages/todo/README.zh.md @@ -6,8 +6,8 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`tool-todo/`](tool-todo/README.md) | 存储并公开会话的 todo 列表。 | (注册到 `ctx.tools`) | +| [`tool-todo/`](tool-todo/README.zh.md) | 存储并公开会话的 todo 列表。 | (注册到 `ctx.tools`) | 子级 README 负责工具、持久化和渲染约定。 -事件载荷记录在 [docs/subsystems/session.md](../../docs/subsystems/session.md)。 +事件载荷记录在 [docs/subsystems/todo.md](../../docs/subsystems/todo.zh.md)。 diff --git a/packages/todo/tool-todo/README.i18n.yaml b/packages/todo/tool-todo/README.i18n.yaml index 469cb71023..8e8176193c 100644 --- a/packages/todo/tool-todo/README.i18n.yaml +++ b/packages/todo/tool-todo/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/todo/tool-todo/README.md -README.md: 4848758dd8f2049221901744e5d09cef2dd31591 -README.zh.md: f3e9e50e1c548dc8c0d01b0bae5c91dd813b5899 +README.md: 498b817c3eacd1e56c6b1b72b4fb863018c2207b +README.zh.md: 6c77e3c78e858d270a7764bfd82df2212cb31a69 diff --git a/packages/todo/tool-todo/README.md b/packages/todo/tool-todo/README.md index 4848758dd8..498b817c3e 100644 --- a/packages/todo/tool-todo/README.md +++ b/packages/todo/tool-todo/README.md @@ -24,6 +24,8 @@ The flag moves the model-facing instruction and the accepted input together — Beyond the schema's type/required/enum checks, `execute` rejects an empty or duplicate `content`, and any item key beyond `content`/`status` — an extended item shape (ids, nesting) fails loud instead of silently flattening, keeping the logged snapshot equal to what the model believes it wrote. How many tasks may be `in_progress` at once is the deployment's call (§ Configuration): a composition that chooses `true` permits parallel work (concurrent subagents, background commands) to mark several tasks simultaneously. Ordering and the discipline of keeping the list current are left to the model via the tool description. +This package's invariant companion validates every durable `todo/write` payload and requires the event to occur inside an open turn. It validates existing and newly announced sessions once, then advances a committed per-session turn trace for live appends. Core session treats declaration-merged events generically; the producing package owns these todo-specific rules ([event ownership](../../../.agents/notes/implemented/architecture/2026-07-20-todo-event-ownership.md)). + ## Rendering The canonical result is `{ todos, counts: { pending, inProgress, completed } }`; its Native renderer returns the compact update acknowledgement. The tool also writes the full `todo/write` session event. UIs subscribe to the event stream and render that durable list themselves: the [web client](../../client/ui-conversation) shows a plan strip plus a dedicated tool row off the standing plan — latest `todo/write` with no later `turn/start` ([display](../../../.agents/notes/implemented/feature/2026-07-23-web-todo-display.md), [lifetime](../../../.agents/notes/implemented/feature/2026-07-28-todo-plan-clears-on-next-turn.md)). @@ -34,7 +36,7 @@ When the composition mounts `ctx.sessionProjections` ([`@deepseek-ai/dsh-session ## Export shape -A function/namespace plugin: it exports `name` / `inject` / `apply` and NO default. A stray `export default` would collapse the module via the Loader's `unwrapExports` and drop `inject` (see [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md)). +A function/namespace plugin: it exports `name` / `inject` / `apply` and NO default. Its type-only outlet declares `TodoItem` and the `todo/write` `SessionEventMap` member; the root and `/client` entrypoints both export `TodoItem`. A stray `export default` would collapse the module via the Loader's `unwrapExports` and drop `inject` (see [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md)). ## Model Experience diff --git a/packages/todo/tool-todo/README.zh.md b/packages/todo/tool-todo/README.zh.md index f3e9e50e1c..6c77e3c78e 100644 --- a/packages/todo/tool-todo/README.zh.md +++ b/packages/todo/tool-todo/README.zh.md @@ -24,17 +24,19 @@ 除 schema 的类型/必填/枚举检查外,`execute` 还会拒绝空或重复的 `content`,以及 `content`/`status` 之外的任何条目键——扩展条目形状(id、嵌套)会明确报错而不是被静默压平,保证落日志的快照与模型自认为写入的内容一致。同时可以有多少任务处于 `in_progress` 由部署决定(见 § 配置):选择 `true` 的组合允许并行工作(并发 subagent、后台命令)同时将多个任务标记为 `in_progress`。列表的顺序及及时更新由模型依照工具描述负责。 +本包的不变量配套插件会校验每个持久 `todo/write` payload,并要求事件位于开放轮次内。它会各自单次校验现有会话与新发布的会话,随后为实时追加推进逐会话的已提交轮次追踪状态。核心会话只会通用处理声明合并事件,todo 专属规则由生产该事件的包负责(见[事件所有权](../../../.agents/notes/implemented/architecture/2026-07-20-todo-event-ownership.zh.md))。 + ## 渲染 -规范结果为 `{ todos, counts: { pending, inProgress, completed } }`;其 Native 渲染器返回精简的更新确认。工具还会写入完整 `todo/write` 会话事件。UI 订阅事件流,并自行渲染该持久化列表:[web 客户端](../../client/ui-conversation)基于当前有效计划(其后没有更晚 `turn/start` 的最近一次 `todo/write`)显示计划条和专属工具行([展示](../../../.agents/notes/implemented/feature/2026-07-23-web-todo-display.md)、[生命周期](../../../.agents/notes/implemented/feature/2026-07-28-todo-plan-clears-on-next-turn.md))。 +规范结果为 `{ todos, counts: { pending, inProgress, completed } }`;其 Native 渲染器返回精简的更新确认。工具还会写入完整 `todo/write` 会话事件。UI 订阅事件流,并自行渲染该持久化列表:[web 客户端](../../client/ui-conversation)基于当前有效计划(其后没有更晚 `turn/start` 的最近一次 `todo/write`)显示计划条和专属工具行([展示](../../../.agents/notes/implemented/feature/2026-07-23-web-todo-display.zh.md)、[生命周期](../../../.agents/notes/implemented/feature/2026-07-28-todo-plan-clears-on-next-turn.zh.md))。 ## 会话投影 -当组合挂载了 `ctx.sessionProjections`([`@deepseek-ai/dsh-session-projection`](../../session/session-projection/README.md))时,本包在一个注入的子插件中注册 `todos` 投影单元:`init` = `null`(尚无写入)、`apply` = 从每个 `todo/write` 取整表,并在每个 `turn/start` 清为 `null`(当前有效计划;`turn/end` 保留刚完成的清单;其余事件都返回同一个状态引用)、`view` = 恒等、`stateVersion` = 2。该键在本包中合并进 `SessionProjectionMap`(经 Service Definition 包的 `/types` 出口);框架驱动该单元,载体通过历史尾页与 `session/projection` 推送帧提供该值。未挂载注册表的组合不受影响。生命周期理由见 [在下一轮次清空 todo 计划](../../../.agents/notes/implemented/feature/2026-07-28-todo-plan-clears-on-next-turn.md)。 +当组合挂载了 `ctx.sessionProjections`([`@deepseek-ai/dsh-session-projection`](../../session/session-projection/README.zh.md))时,本包在一个注入的子插件中注册 `todos` 投影单元:`init` = `null`(尚无写入)、`apply` = 从每个 `todo/write` 取整表,并在每个 `turn/start` 清为 `null`(当前有效计划;`turn/end` 保留刚完成的清单;其余事件都返回同一个状态引用)、`view` = 恒等、`stateVersion` = 2。该键在本包中合并进 `SessionProjectionMap`(经 Service Definition 包的 `/types` 出口);框架驱动该单元,载体通过历史尾页与 `session/projection` 推送帧提供该值。未挂载注册表的组合不受影响。生命周期理由见 [在下一轮次清空 todo 计划](../../../.agents/notes/implemented/feature/2026-07-28-todo-plan-clears-on-next-turn.zh.md)。 ## 导出形状 -函数/命名空间插件:导出 `name`/`inject`/`apply`,不提供默认导出。意外的 `export default` 会被 Loader 的 `unwrapExports` 折叠为默认导出,并导致 `inject` 丢失(参见 [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md))。 +函数/命名空间插件:导出 `name`/`inject`/`apply`,不提供默认导出。其仅类型出口声明 `TodoItem` 与 `todo/write` `SessionEventMap` 成员;包根入口和 `/client` 入口都导出 `TodoItem`。意外的 `export default` 会被 Loader 的 `unwrapExports` 折叠为默认导出,并导致 `inject` 丢失(参见 [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.zh.md))。 ## 模型体验 @@ -42,7 +44,7 @@ #### 模型看到的内容 -模型会看到生成的 [`todo_write` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-todo)。 +模型会看到生成的 [`todo_write` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-todo)。 #### Token 影响 diff --git a/packages/todo/tool-todo/package.json b/packages/todo/tool-todo/package.json index c14ac4358b..54c32c66fc 100644 --- a/packages/todo/tool-todo/package.json +++ b/packages/todo/tool-todo/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-todo", "description": "Model-facing todo_write tool over the DeepSeek Harness event-sourced session log", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/todo/tool-todo/src/index.ts b/packages/todo/tool-todo/src/index.ts index e92af70313..9ccbb1ab7a 100644 --- a/packages/todo/tool-todo/src/index.ts +++ b/packages/todo/tool-todo/src/index.ts @@ -10,7 +10,7 @@ import z from '@deepseek-ai/schemastery' import { z as zod } from 'zod' import type { ZodType } from 'zod' import { defineTool } from '@deepseek-ai/dsh-tools' -import type { TodoItem } from '@deepseek-ai/dsh-session' +import type { TodoItem } from './types.ts' // Type-only: resolves ctx.sessionProjections for the optional unit child. import type {} from '@deepseek-ai/dsh-session-projection' // The `todos` projection-key declaration lives in src/types.ts (its one home); @@ -135,14 +135,14 @@ export function apply(ctx: Context, config: Config): void { ctx.inject(['sessionProjections'], (projectionCtx) => { projectionCtx.sessionProjections.register<'todos', TodoItem[] | null>({ key: 'todos', - schema: todosProjectionSchema, + stateSchema: todosProjectionSchema, init: () => null, apply: (state, event) => { if (event.type === 'todo/write') return event.data.todos if (event.type === 'turn/start') return null return state }, - view: state => state, + wire: { viewSchema: todosProjectionSchema, view: state => state }, stateVersion: 2, }) }) diff --git a/packages/todo/tool-todo/src/invariant.ts b/packages/todo/tool-todo/src/invariant.ts index f1b8c63066..fee7088a40 100644 --- a/packages/todo/tool-todo/src/invariant.ts +++ b/packages/todo/tool-todo/src/invariant.ts @@ -39,20 +39,57 @@ function validateTodos(value: unknown, fail: InvariantFailure): void { } /* jscpd:ignore-start -- package companions share replay and dispatch plumbing */ -/** Validate the package-owned event fields and ignore unrelated events. */ -function validateEvent(event: SessionEvent, fail: InvariantFailure): void { - if (event.type === 'todo/write') validateTodos(event.data.todos, fail) +/** Incremental turn state for one committed session log. */ +interface TurnTrace { + open: boolean +} + +/** Advance the trace after one event has committed. */ +function advanceTrace(trace: TurnTrace, event: SessionEvent): void { + if (event.type === 'turn/start') trace.open = true + if (event.type === 'turn/end') trace.open = false +} + +/** Validate one package-owned event against the preceding committed trace. */ +function validateEvent(event: SessionEvent, trace: TurnTrace, fail: InvariantFailure): void { + if (event.type !== 'todo/write') return + validateTodos(event.data.todos, fail) + if (!trace.open) fail('todo/write appended outside any open turn') +} + +/** Validate one existing log in a single pass and return its tail trace. */ +function seedTrace(session: Session, fail: InvariantFailure): TurnTrace { + const trace: TurnTrace = { open: false } + for (const event of session.events) { + validateEvent(event, trace, fail) + advanceTrace(trace, event) + } + return trace } /** Install validation for loaded and newly appended whole-list todo snapshots. */ const install: InvariantInstaller = Object.assign((ctx: Context, fail: InvariantFailure) => { - for (const session of ctx.sessions.list()) { - for (const event of session.events) validateEvent(event, fail) + const traces = new WeakMap() + const seed = (session: Session): void => { + traces.set(session, seedTrace(session, fail)) } + const traceFor = (session: Session): TurnTrace => { + let trace = traces.get(session) + if (trace === undefined) { + trace = seedTrace(session, fail) + traces.set(session, trace) + } + return trace + } + for (const session of ctx.sessions.list()) seed(session) + ctx.on('session/created', (session) => { seed(session) }, { global: true }) ctx.on('internal/dispatch', (_mode, eventName, args) => { if (eventName !== 'session/event') return - const event = (args as [Session, SessionEvent])[1] - validateEvent(event, fail) + const [session, event] = args as [Session, SessionEvent] + validateEvent(event, traceFor(session), fail) + }, { global: true }) + ctx.on('session/event', (session, event) => { + advanceTrace(traceFor(session), event) }, { global: true }) }, { inject: ['sessions'] }) /* jscpd:ignore-end */ diff --git a/packages/todo/tool-todo/src/types.ts b/packages/todo/tool-todo/src/types.ts index fe37e65d55..68d5649bb0 100644 --- a/packages/todo/tool-todo/src/types.ts +++ b/packages/todo/tool-todo/src/types.ts @@ -8,11 +8,34 @@ * @module @deepseek-ai/dsh-tool-todo/types */ -import type { TodoItem } from '@deepseek-ai/dsh-session/types' +/** + * One entry in an agent's todo list — the unit of the `todo/write` + * whole-list snapshot declared by this package. + * + * Deliberately minimal: a human-readable `content` line and a three-state + * `status`. No id, priority, or `activeForm` — the list is replaced wholesale + * on every write (last-write-wins), so entries need no stable identity. The + * three statuses describe the complete portable lifecycle needed by model and + * UI consumers. + */ +export interface TodoItem { + /** What this task is — a short imperative line shown in the UI. */ + content: string + /** Lifecycle state. `in_progress` marks a task being worked now; parallel work may mark several. */ + status: 'pending' | 'in_progress' | 'completed' +} -export type { TodoItem } from '@deepseek-ai/dsh-session/types' +declare module '@deepseek-ai/dsh-session/types' { + interface SessionEventMap { + /** Whole-list snapshot; latest write wins on replay. Log-only UI state; never derived history. */ + 'todo/write': { todos: TodoItem[] } + } +} declare module '@deepseek-ai/dsh-session-projection/types' { + interface SessionProjectionStateMap { + todos: TodoItem[] | null + } interface SessionProjectionMap { /** * The agent's current whole todo list (the latest `todo/write` snapshot), diff --git a/packages/todo/tool-todo/tests/invariant.spec.ts b/packages/todo/tool-todo/tests/invariant.spec.ts index e533e11feb..7d53ad052b 100644 --- a/packages/todo/tool-todo/tests/invariant.spec.ts +++ b/packages/todo/tool-todo/tests/invariant.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import SessionStore, { type Session, type SessionEvent } from '@deepseek-ai/dsh-session' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import ToolRuntime from '@deepseek-ai/dsh-tools' import * as ToolTodo from '@deepseek-ai/dsh-tool-todo' import * as TodoInvariant from '@deepseek-ai/dsh-tool-todo/invariant' @@ -14,10 +14,6 @@ async function setup(): Promise { return ctx } -function event(todos: unknown): SessionEvent { - return { type: 'todo/write', seq: 0, time: 0, data: { todos } } as SessionEvent -} - describe('todo snapshot invariants', () => { it('accepts historical and live parallel snapshots under the single-active tool policy', async () => { const todos = [ @@ -30,11 +26,13 @@ describe('todo snapshot invariants', () => { await ctx.plugin(SessionStore) await ctx.plugin(ToolRuntime) await ctx.plugin(ToolTodo, { allowParallelInProgress: false }) - ctx.sessions.create().append('todo/write', { todos: [...todos] }) + const session = ctx.sessions.create() + session.append('turn/start', { turn: 1 }) + session.append('todo/write', { todos: [...todos] }) await ctx.plugin(InvariantRegistry, { enabled: true }) await expect(ctx.plugin(TodoInvariant).then(() => undefined)).resolves.toBeUndefined() - expect(() => { ctx.emit('session/event', {} as Session, event(todos)) }).not.toThrow() + expect(() => { session.append('todo/write', { todos: [...todos] }) }).not.toThrow() }) it.each([ @@ -49,23 +47,46 @@ describe('todo snapshot invariants', () => { [[{ content: 'task', status: 'paused' }], /unknown status/], ])('rejects an incoherent durable todo snapshot', async (todos, message) => { const ctx = await setup() - expect(() => { ctx.emit('session/event', {} as Session, event(todos)) }).toThrow(message) + const session = ctx.sessions.create() + session.append('turn/start', { turn: 1 }) + expect(() => { session.append('todo/write', { todos } as never) }).toThrow(message) }) it('ignores unrelated dispatches and session events', async () => { const ctx = await setup() + const session = ctx.sessions.create() expect(() => { ctx.emit('tools/change') - ctx.emit('session/event', {} as Session, { - type: 'turn/start', seq: 0, time: 0, data: { turn: 1 }, - }) + session.append('turn/start', { turn: 1 }) }).not.toThrow() }) + it('rejects a live snapshot outside an open turn before it enters the log', async () => { + const ctx = await setup() + const session = ctx.sessions.create() + session.append('turn/start', { turn: 1 }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + const before = [...session.events] + + expect(() => session.append('todo/write', { todos: [] })).toThrow(/outside any open turn/) + expect(session.events).toEqual(before) + }) + + it('rejects an existing snapshot outside an open turn on late registration', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + ctx.sessions.create().append('todo/write', { todos: [] }) + await ctx.plugin(InvariantRegistry, { enabled: true }) + + await expect(ctx.plugin(TodoInvariant).then(() => undefined)).rejects.toThrow(/outside any open turn/) + }) + it('rejects an invalid existing snapshot on late registration', async () => { const ctx = new Context() await ctx.plugin(SessionStore) - ctx.sessions.create().append('todo/write', { + const session = ctx.sessions.create() + session.append('turn/start', { turn: 1 }) + session.append('todo/write', { todos: [ { content: 'duplicate', status: 'pending' }, { content: 'duplicate', status: 'completed' }, @@ -75,4 +96,32 @@ describe('todo snapshot invariants', () => { await expect(ctx.plugin(TodoInvariant).then(() => undefined)).rejects.toThrow(/repeats content "duplicate"/) }) + + it('validates seeded sessions announced after companion installation', async () => { + const ctx = await setup() + const valid = ctx.sessions.create(SessionId('todo-seeded-valid'), { seed: [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'todo/write', seq: 1, time: 2, data: { todos: [] } }, + ] }) + expect(() => valid.append('todo/write', { todos: [] })).not.toThrow() + + expect(() => ctx.sessions.create(SessionId('todo-seeded-invalid'), { seed: [ + { type: 'todo/write', seq: 0, time: 1, data: { todos: [] } }, + ] })).toThrow(/outside any open turn/) + }) + + it('tracks events committed before a prepared session is announced', async () => { + const ctx = await setup() + const session = ctx.sessions.prepare(SessionId('todo-prepared')) + const detach = ctx.sessions.enter(session) + try { + session.append('turn/start', { turn: 1 }) + expect(() => session.append('todo/write', { todos: [] })).not.toThrow() + ctx.sessions.announce(session) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + expect(() => session.append('todo/write', { todos: [] })).toThrow(/outside any open turn/) + } finally { + detach() + } + }) }) diff --git a/packages/todo/tool-todo/tests/projection.spec.ts b/packages/todo/tool-todo/tests/projection.spec.ts index ff374e641e..dc2283a98c 100644 --- a/packages/todo/tool-todo/tests/projection.spec.ts +++ b/packages/todo/tool-todo/tests/projection.spec.ts @@ -13,7 +13,8 @@ import AgentRegistry from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import { createUserMessage } from '@deepseek-ai/dsh-llm' import SessionStore from '@deepseek-ai/dsh-session' -import type { Session, TodoItem } from '@deepseek-ai/dsh-session' +import type { Session } from '@deepseek-ai/dsh-session' +import type { TodoItem } from '@deepseek-ai/dsh-tool-todo' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRuntime from '@deepseek-ai/dsh-tools' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' @@ -83,6 +84,7 @@ describe('todos projection provider', () => { { content: 'a', status: 'completed' }, { content: 'b', status: 'in_progress' }, ] + session.append('turn/start', { turn: 1 }) session.append('todo/write', { todos: first }) session.append('todo/write', { todos: second }) const projections = await bench.tailProjections() @@ -96,10 +98,11 @@ describe('todos projection provider', () => { const session = bench.session seedMessage(session) const list: TodoItem[] = [{ content: 'done', status: 'completed' }] + session.append('turn/start', { turn: 1 }) session.append('todo/write', { todos: list }) session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) expect((await bench.tailProjections())?.values.todos).toEqual(list) - session.append('turn/start', { turn: 1 }) + session.append('turn/start', { turn: 2 }) const cleared = await bench.tailProjections() expect(cleared?.values.todos).toBeNull() expect(cleared?.asOfSeq).toBe(session.seq - 1) diff --git a/packages/todo/tool-todo/tests/tool-todo.spec.ts b/packages/todo/tool-todo/tests/tool-todo.spec.ts index dd959f2f9f..5d44a94679 100644 --- a/packages/todo/tool-todo/tests/tool-todo.spec.ts +++ b/packages/todo/tool-todo/tests/tool-todo.spec.ts @@ -1,11 +1,11 @@ import { describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' -import { CallId } from '@deepseek-ai/dsh-llm' +import { createUserMessage, CallId } from '@deepseek-ai/dsh-llm' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRuntime from '@deepseek-ai/dsh-tools' import { Session, SessionId } from '@deepseek-ai/dsh-session' -import type { TodoItem } from '@deepseek-ai/dsh-session' +import type { TodoItem } from '@deepseek-ai/dsh-tool-todo' import { type Agent } from '@deepseek-ai/dsh-agent' import * as tool from '../src/index.ts' @@ -233,3 +233,68 @@ describe('dsh-tool-todo', () => { expect(typeof unwrapped.apply).toBe('function') }) }) + +describe('todo/write event', () => { + it('appends the whole-list snapshot and isolates the log from later mutation', () => { + const session = Session.create(SessionId('t1')) + session.append('turn/start', { turn: 1 }) + const todos: TodoItem[] = [ + { content: 'plan the work', status: 'in_progress' }, + { content: 'write the code', status: 'pending' }, + ] + session.append('todo/write', { todos }) + + const event = session.events.findLast(e => e.type === 'todo/write')! + expect(event.type).toBe('todo/write') + expect(event.data.todos).toEqual(todos) + + todos.push({ content: 'sneak in', status: 'pending' }) + todos[0]!.status = 'completed' + expect(event.data.todos).toEqual([ + { content: 'plan the work', status: 'in_progress' }, + { content: 'write the code', status: 'pending' }, + ]) + }) + + it('is last-write-wins: the current list is the most recent todo/write', () => { + const session = Session.create(SessionId('t2')) + session.append('turn/start', { turn: 1 }) + session.append('todo/write', { todos: [{ content: 'first', status: 'pending' }] }) + session.append('todo/write', { todos: [ + { content: 'first', status: 'completed' }, + { content: 'second', status: 'in_progress' }, + ] }) + + const current = session.events.findLast(e => e.type === 'todo/write')!.data.todos + expect(current).toEqual([ + { content: 'first', status: 'completed' }, + { content: 'second', status: 'in_progress' }, + ]) + }) + + it('does not add a derived message or surface node', () => { + const session = Session.create(SessionId('t3')) + session.append('turn/start', { turn: 1 }) + session.append('user/message', createUserMessage({ + content: [{ type: 'text', text: 'q' }], source: { kind: 'user' }, + }), { surfaceOp: 'append' }) + const before = session.deriveMessages().length + session.append('todo/write', { todos: [{ content: 'a task', status: 'pending' }] }) + + expect(session.deriveMessages()).toHaveLength(before) + expect(session.surface.nodes).not.toContain(session.seq - 1) + }) + + it('round-trips through a seeded replay identically without surface metadata', () => { + const original = Session.create(SessionId('t4')) + original.append('turn/start', { turn: 1 }) + original.append('todo/write', { todos: [{ content: 'only', status: 'completed' }] }) + original.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + const replayed = Session.create(SessionId('t4-replay'), [...original.events]) + + expect(replayed.events.findLast(e => e.type === 'todo/write')!.data.todos) + .toEqual([{ content: 'only', status: 'completed' }]) + expect(replayed.events.slice(0, original.seq)).toEqual(original.events) + expect(replayed.firstLiveSeq).toBe(original.seq) + }) +}) diff --git a/packages/typert/README.i18n.yaml b/packages/typert/README.i18n.yaml index 3a061e43fd..35034e4c11 100644 --- a/packages/typert/README.i18n.yaml +++ b/packages/typert/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/typert/README.md README.md: ad9f843e48be0e3be85921ed8fd3ca4e2c327160 -README.zh.md: aae74bf89f657f50f4519b2f8a0628fd391f8de3 +README.zh.md: 9ebd00f71a509e24a0b6c674c56d87c411dd7794 diff --git a/packages/typert/README.zh.md b/packages/typert/README.zh.md index aae74bf89f..9ebd00f71a 100644 --- a/packages/typert/README.zh.md +++ b/packages/typert/README.zh.md @@ -6,6 +6,6 @@ Typert 将源代码分析、运行时存储和 Loader 发现机制分离。 | 包 | 职责 | Cordis 键 | |---|---|---| -| [`registry/`](registry/README.md) | 存储运行时包反射和 schema | `ctx.typert` | -| [`loader/`](loader/README.md) | 发现 Loader 条目并注册生成的宿主产物 | 使用 `ctx.loader`、`ctx.typert` | -| [`generator/`](generator/README.md) | 从源代码类型生成运行时产物 | 构建时库 | +| [`registry/`](registry/README.zh.md) | 存储运行时包反射和 schema | `ctx.typert` | +| [`loader/`](loader/README.zh.md) | 发现 Loader 条目并注册生成的宿主产物 | 使用 `ctx.loader`、`ctx.typert` | +| [`generator/`](generator/README.zh.md) | 从源代码类型生成运行时产物 | 构建时库 | diff --git a/packages/typert/generator/package.json b/packages/typert/generator/package.json index 39136c2877..78a18e510f 100644 --- a/packages/typert/generator/package.json +++ b/packages/typert/generator/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-typert-generator", "description": "TypeScript project analyzer and model-driven Typert artifact generator", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/typert/generator/src/cordis-catalog.ts b/packages/typert/generator/src/cordis-catalog.ts index c38b7fd9e1..8ce666b96c 100644 --- a/packages/typert/generator/src/cordis-catalog.ts +++ b/packages/typert/generator/src/cordis-catalog.ts @@ -951,6 +951,12 @@ function anchorFor(headingText: string): string[] { return [``, ''] } +/** Render a subsystem `file:line` source pointer as a file-only link. */ +function sourceLink(source: string): string { + const file = source.split(':')[0] + return `[\`${file}\`](../../${file})` +} + /** Render one harness event entry onto its owning page, nested under its scope heading. */ function renderEvent(e: EventEntry, onPage: string, linkedTypePages: Readonly>): string[] { const out = [...anchorFor(`${e.name} — ${e.mode}`), `#### \`${e.name}\` — ${e.mode}`, ''] @@ -958,7 +964,7 @@ function renderEvent(e: EventEntry, onPage: string, linkedTypePages: Readonly method.signature).join('\n'), onPage, linkedTypePages) if (links) out.push(links, '') } - out.push(`Source: [\`${s.source}\`](../../${s.source.split(':')[0]})`, '') + out.push(`Source: ${sourceLink(s.source)}`, '') return out } @@ -1010,7 +1016,7 @@ export function renderPageRegion(page: string, services: ServiceEntry[], events: '', '## Cordis API', '', - 'Generated from source by `scripts/gen-cordis-catalog.ts` (verified fresh by `pnpm run verify-cordis-catalog` in doc-sync; regenerate with `pnpm run gen-cordis-catalog`) — this section is byte-identical in both language sides of the page. Signature blocks use a `ts cordis-catalog` fence and keep the original source JSDoc; dispatch modes are defined in the [primer](../cordis-primer.md#dispatch-modes), and the framework-inherited `ctx` API lives in [cordis-api/inherited.md](../cordis-api/inherited.md).', + 'Generated from source by `scripts/gen-cordis-catalog.ts` (verified fresh by `pnpm run verify-cordis-catalog` in doc-sync; regenerate with `pnpm run gen-cordis-catalog`) — the language sides differ only in locale-specific paired document paths. Signature blocks use a `ts cordis-catalog` fence and keep the original source JSDoc; dispatch modes are defined in the [primer](../cordis-primer.md#dispatch-modes), and the framework-inherited `ctx` API lives in [cordis-api/inherited.md](../cordis-api/inherited.md).', '', ] for (const s of services) lines.push(...renderService(s, page, policy.linkedTypePages)) diff --git a/packages/typert/generator/tests/cordis-catalog.spec.ts b/packages/typert/generator/tests/cordis-catalog.spec.ts index 5cb9351999..ab62291aed 100644 --- a/packages/typert/generator/tests/cordis-catalog.spec.ts +++ b/packages/typert/generator/tests/cordis-catalog.spec.ts @@ -5,8 +5,16 @@ import { projectCordisCatalog, renderInheritedPage, renderPageRegion, + type CordisCatalogPolicy, } from '../src/cordis-catalog.ts' -import { CORDIS_CATALOG_POLICY, EVENT_SCOPE_PAGE, REGION_BEGIN, REGION_END, SERVICE_PAGE } from '../../../../scripts/gen-cordis-catalog.ts' +import { + CORDIS_CATALOG_POLICY, + EVENT_SCOPE_PAGE, + localizePageRegion, + REGION_BEGIN, + REGION_END, + SERVICE_PAGE, +} from '../../../../scripts/gen-cordis-catalog.ts' const workspaceRoot = resolve(import.meta.dirname, '../../../..') @@ -15,7 +23,40 @@ let cached: ReturnType | undefined const projection = (): ReturnType => (cached ??= projectCordisCatalog(workspaceRoot, CORDIS_CATALOG_POLICY)) +const SOURCE_LINK_POLICY: CordisCatalogPolicy = { + linkedTypePages: {}, + foundationTypeNames: new Set(), + typeLinkExemptions: {}, + inheritedEvents: [{ name: 'ready', summary: 'Ready.', source: 'vendor/cordis/src/events.ts:9' }], + inheritedServices: [{ name: 'ctx.root', summary: 'Root.', source: 'vendor/cordis/src/context.ts:12' }], +} + describe('Typert-backed Cordis catalog', () => { + it('omits subsystem source lines while preserving inherited Cordis source lines', () => { + const page = renderPageRegion('fixture.md', [{ + key: 'fixture', + type: 'Fixture', + abstract: false, + doc: 'Fixture.', + methods: [], + source: 'packages/fixture/service.ts:24', + }], [{ + name: 'fixture/ready', + scope: 'fixture', + signature: "'fixture/ready'(): void", + jsDoc: '/** Ready. */', + mode: 'emit', + doc: 'Ready.', + source: 'packages/fixture/events.ts:42', + }], SOURCE_LINK_POLICY) + const inherited = renderInheritedPage(SOURCE_LINK_POLICY) + + expect(page).toContain('Source: [`packages/fixture/events.ts`](../../packages/fixture/events.ts)') + expect(page).toContain('Source: [`packages/fixture/service.ts`](../../packages/fixture/service.ts)') + expect(inherited).toContain('([`vendor/cordis/src/events.ts:9`](../../vendor/cordis/src/events.ts))') + expect(inherited).toContain('([`vendor/cordis/src/context.ts:12`](../../vendor/cordis/src/context.ts))') + }) + it('reproduces every committed catalog artifact byte for byte', { timeout: 480_000 }, () => { const { projector, model } = projection() const expected = (path: string): string => readFileSync(join(workspaceRoot, path), 'utf8') @@ -29,11 +70,14 @@ describe('Typert-backed Cordis catalog', () => { CORDIS_CATALOG_POLICY, ) for (const side of [page, page.replace(/\.md$/, '.zh.md')]) { - const committed = expected(`docs/subsystems/${side}`) + const rel = `docs/subsystems/${side}` + const committed = expected(rel) const begin = committed.indexOf(REGION_BEGIN) const end = committed.indexOf(REGION_END) - expect(begin, `docs/subsystems/${side} carries the region`).toBeGreaterThanOrEqual(0) - expect(committed.slice(begin, end + REGION_END.length)).toBe(region) + expect(begin, `${rel} carries the region`).toBeGreaterThanOrEqual(0) + expect(committed.slice(begin, end + REGION_END.length)).toBe( + localizePageRegion(region, rel, workspaceRoot), + ) } } expect(projector.renderRuntimeApi(model)).toBe( diff --git a/packages/typert/loader/README.i18n.yaml b/packages/typert/loader/README.i18n.yaml index 03abaafe53..b85f70d24a 100644 --- a/packages/typert/loader/README.i18n.yaml +++ b/packages/typert/loader/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/typert/loader/README.md README.md: ab9293de1630fdbe8c560bb9e6d00c272cc34161 -README.zh.md: 4e962c05b73982f03accab938cf3282dfe333dbc +README.zh.md: d5f9d8e5de5d47ed424f4ed03a12c03cb1b53997 diff --git a/packages/typert/loader/README.zh.md b/packages/typert/loader/README.zh.md index 4e962c05b7..d5f9d8e5de 100644 --- a/packages/typert/loader/README.zh.md +++ b/packages/typert/loader/README.zh.md @@ -12,7 +12,7 @@ ## 模型体验 -无。loader 只向 [`ctx.typert`](../registry/README.md) 提供注册项;任何模型可见投影均由消费方负责。 +无。loader 只向 [`ctx.typert`](../registry/README.zh.md) 提供注册项;任何模型可见投影均由消费方负责。 #### KV Cache 影响 diff --git a/packages/typert/loader/package.json b/packages/typert/loader/package.json index 20a7172175..8433b81657 100644 --- a/packages/typert/loader/package.json +++ b/packages/typert/loader/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-typert-loader", "description": "Loader integration for generated Typert package contributions", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/typert/loader/src/index.ts b/packages/typert/loader/src/index.ts index 7cb73c446f..ce5aa0d54f 100644 --- a/packages/typert/loader/src/index.ts +++ b/packages/typert/loader/src/index.ts @@ -11,9 +11,9 @@ * fails loud — aggregated into this plugin's activation throw for existing * entries, contained to a logged error per package in steady state. * - * Scanning is incremental per entry name, mirroring the client-modules node - * half: every cordis `internal/plugin` emission marks the fiber's entry name - * dirty and a microtask flush reconciles each dirty name against the live + * Scanning is incremental per entry name. Every cordis `internal/plugin` + * emission marks the fiber's entry name dirty, and a microtask flush + * reconciles each dirty name against the live * loader entries; the activation pass seeds the same dirty set with all * current entries. Package verdicts and imported manifests are cached per * package name and never expire — plugin-set changes take effect on restart. diff --git a/packages/typert/loader/tests/loader.spec.ts b/packages/typert/loader/tests/loader.spec.ts index 6dccd2b5dd..2f4b4cae04 100644 --- a/packages/typert/loader/tests/loader.spec.ts +++ b/packages/typert/loader/tests/loader.spec.ts @@ -398,9 +398,10 @@ describe('typert loader', () => { await ctx.loader.create({ name: '@fixture/steady-failure' }) await ctx.loader.await() - await new Promise(resolve => setTimeout(resolve, 20)) - - expect(logged).toHaveBeenCalledWith(expect.objectContaining({ message: 'register failed' })) + // The failing contributor's error is reported on the post-await flush. + await vi.waitFor(() => { + expect(logged).toHaveBeenCalledWith(expect.objectContaining({ message: 'register failed' })) + }, { timeout: 10_000 }) expect(ctx.typert.getPackage('@fixture/steady-failure')).toBeUndefined() }) }) diff --git a/packages/typert/protocol/package.json b/packages/typert/protocol/package.json index 7b5c6b4d91..12f8283931 100644 --- a/packages/typert/protocol/package.json +++ b/packages/typert/protocol/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-typert-protocol", "description": "Compiler-independent Remote metadata and Typert provider protocols", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/typert/registry/README.i18n.yaml b/packages/typert/registry/README.i18n.yaml index 5730472079..b3c081b128 100644 --- a/packages/typert/registry/README.i18n.yaml +++ b/packages/typert/registry/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/typert/registry/README.md README.md: fa227b1c8faf1abd5a6492d4b8fe7d0c51ceeef1 -README.zh.md: b708e536acb7a5c4dc47e8c83180728e2ce150cd +README.zh.md: 62c5a2141de9ccdd564654658a464d721b77f97d diff --git a/packages/typert/registry/README.zh.md b/packages/typert/registry/README.zh.md index b708e536ac..62c5a2141d 100644 --- a/packages/typert/registry/README.zh.md +++ b/packages/typert/registry/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -生成的 Typert 产物所用的运行时注册表。每个注册项包含某个包在一个 face 上的业务反射信息,以及可选的运行时 Zod schema;`ctx.typert` 会以原子方式同时注册两者,并在发起调用的 Cordis fiber 释放时一并移除它们。TypeScript 分析和代码生成由 [`dsh-typert-generator`](../generator/README.md) 负责。 +生成的 Typert 产物所用的运行时注册表。每个注册项包含某个包在一个 face 上的业务反射信息,以及可选的运行时 Zod schema;`ctx.typert` 会以原子方式同时注册两者,并在发起调用的 Cordis fiber 释放时一并移除它们。TypeScript 分析和代码生成由 [`dsh-typert-generator`](../generator/README.zh.md) 负责。 包反射信息以 `#` 为键。schema 以 `#` 为键,并保留生成方的 Zod 实例。系统按需在消费方边界计算 JSON Schema。 @@ -17,7 +17,7 @@ - `toJSONSchema(key, params?)` 使用 `z.toJSONSchema()` 投影当前有效的 schema,且不缓存结果。 - `typertKey()` 和 `typertPackageKey()` 构造两种稳定的标识形式。 -`@deepseek-ai/dsh-typert-registry/types` 子路径包含注册项和记录的纯类型约定。[`dsh-typert-loader`](../loader/README.md) 会在 Loader 组合中发现并注册生成的宿主侧产物;其他组合所有者可以直接调用 `ctx.typert.register()`。 +`@deepseek-ai/dsh-typert-registry/types` 子路径包含注册项和记录的纯类型约定。[`dsh-typert-loader`](../loader/README.zh.md) 会在 Loader 组合中发现并注册生成的宿主侧产物;其他组合所有者可以直接调用 `ctx.typert.register()`。 ## 模型体验 diff --git a/packages/typert/registry/package.json b/packages/typert/registry/package.json index 7c34532970..b66a41ea7f 100644 --- a/packages/typert/registry/package.json +++ b/packages/typert/registry/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-typert-registry", "description": "Runtime registry for generated package reflection and Zod schemas", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/util/README.i18n.yaml b/packages/util/README.i18n.yaml index c041aed49e..04314b33ef 100644 --- a/packages/util/README.i18n.yaml +++ b/packages/util/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/util/README.md README.md: 7867f6c235dfd3cb063214381ac2f40ed0f26ce3 -README.zh.md: 65c48de0de038b4910583f79d9fa0c31ea2d67d0 +README.zh.md: 33ca9912f3670dfd3419b7a2a33f462cb4be95a3 diff --git a/packages/util/README.zh.md b/packages/util/README.zh.md index 65c48de0de..33ca9912f3 100644 --- a/packages/util/README.zh.md +++ b/packages/util/README.zh.md @@ -6,9 +6,9 @@ | 包 | 职责 | |---|---| -| [`brand/`](brand/README.md) | 提供带名义品牌的类型 | -| [`paths/`](home-paths/README.md) | 解析 Harness 数据根目录和共享路径 | -| [`timeout/`](timeout/README.md) | 提供截止时间和超时分类原语 | -| [`retention/`](output-retention/README.md) | 限制保留文本和项集合的大小 | -| [`atomic-write/`](atomic-write/README.md) | 以原子方式替换文件 | -| [`native-command/`](native-command/README.md) | 不经 shell 运行宿主原生命令 | +| [`brand/`](brand/README.zh.md) | 提供带名义品牌的类型 | +| [`paths/`](home-paths/README.zh.md) | 解析 Harness 数据根目录和共享路径 | +| [`timeout/`](timeout/README.zh.md) | 提供截止时间和超时分类原语 | +| [`retention/`](output-retention/README.zh.md) | 限制保留文本和项集合的大小 | +| [`atomic-write/`](atomic-write/README.zh.md) | 以原子方式替换文件 | +| [`native-command/`](native-command/README.zh.md) | 不经 shell 运行宿主原生命令 | diff --git a/packages/util/atomic-write/README.i18n.yaml b/packages/util/atomic-write/README.i18n.yaml index d293beb656..98e35ef83a 100644 --- a/packages/util/atomic-write/README.i18n.yaml +++ b/packages/util/atomic-write/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/util/atomic-write/README.md -README.md: a767f24064c368b60d85fed6fa1d88349cab9587 -README.zh.md: 6388e264898e0025fb6586acecab450be3eb9e55 +README.md: 0e6501c0ae35bf26d7df67f28ce0ae24c85ecca6 +README.zh.md: 377f85f5c4aab74e8c04068b616de32a8ba063d6 diff --git a/packages/util/atomic-write/README.md b/packages/util/atomic-write/README.md index a767f24064..0e6501c0ae 100644 --- a/packages/util/atomic-write/README.md +++ b/packages/util/atomic-write/README.md @@ -28,7 +28,9 @@ await withFileLock('/home/u/.dsh/settings.yaml', async () => { - **Same-directory sibling** keeps the rename on one filesystem, so the swap stays atomic. - Parent directories are created; on any failure the temp is removed and the failure rethrown; readers observe either the old or the new complete content. -`withFileLock` serializes the writers of one file across processes, for the read-render-commit cycles a bare atomic commit cannot make safe on its own. The lock is a `wx`-created `.lock` sibling, so readers never contend; waiters back off exponentially and fail with a timeout rather than block forever. A contender never removes the existing lock: age cannot distinguish a crashed owner from a paused live writer. +`withFileLock` serializes the writers of one file across processes, for the read-render-commit cycles a bare atomic commit cannot make safe on its own. The lock is a `wx`-created `.lock` sibling, so readers never contend; waiters back off exponentially and fail with a timeout rather than block forever. `EEXIST` identifies contention directly; `EPERM` does so only when a fresh `lstat` confirms that the lock path exists, covering Windows exclusive-create behavior without hiding an unrelated permission failure. A contender never removes the existing lock: age cannot distinguish a crashed owner from a paused live writer. + +How long a contender waits is a property of the operation the holder runs, so it is stated per call through `waitMs`. The default is sized for file work alone; a holder whose cycle includes a network round trip — a credential mutation that refreshes an expired token — states a longer one, because leaving the default would fail every other writer of that file for the duration. The retry cadence stays fixed: it governs how often a contender asks, which no caller has a reason to vary. ## Model Experience diff --git a/packages/util/atomic-write/README.zh.md b/packages/util/atomic-write/README.zh.md index 6388e26489..377f85f5c4 100644 --- a/packages/util/atomic-write/README.zh.md +++ b/packages/util/atomic-write/README.zh.md @@ -28,7 +28,9 @@ await withFileLock('/home/u/.dsh/settings.yaml', async () => { - **同目录兄弟文件**保证 rename 落在同一文件系统上,交换保持原子。 - 自动创建父目录;任何失败都会移除临时文件并重新抛出该失败;读取方只会观察到旧内容或完整的新内容。 -`withFileLock` 跨进程串行化同一文件的写入方,服务于单靠原子提交无法保证安全的读-渲染-提交循环。锁是以 `wx` 创建的同目录 `.lock`,因此读取方从不参与竞争;等待方按指数退避,超时即失败而非无限阻塞。竞争者绝不移除现有锁:锁龄无法区分已经崩溃的所有者与被暂停但仍存活的写入方。 +`withFileLock` 跨进程串行化同一文件的写入方,服务于单靠原子提交无法保证安全的读-渲染-提交循环。锁是以 `wx` 创建的同目录 `.lock`,因此读取方从不参与竞争;等待方按指数退避,超时即失败而非无限阻塞。`EEXIST` 直接表示竞争;只有一次新的 `lstat` 确认锁路径存在时,`EPERM` 才表示竞争,从而兼容 Windows 的独占创建行为,又不掩盖无关的权限故障。竞争者绝不移除现有锁:锁龄无法区分已经崩溃的所有者与被暂停但仍存活的写入方。 + +等待多久是持锁方所跑操作的属性,因此由每次调用经 `waitMs` 声明。默认值只按纯文件工作量级选定;若持锁方的循环包含一次网络往返——例如刷新过期 token 的凭据变更——就应声明更长的值,否则该文件的其他写入方会在这段时间内全部失败。退避节奏保持固定:它决定竞争者多久问一次,调用方没有理由改变它。 ## 模型体验 diff --git a/packages/util/atomic-write/package.json b/packages/util/atomic-write/package.json index 964fcba265..481ea9246b 100644 --- a/packages/util/atomic-write/package.json +++ b/packages/util/atomic-write/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-atomic-write", "description": "Zero-dependency atomic file replacement: exclusive-create random-suffix temp + rename carrying the caller-stated permissions (writeFileAtomic)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/util/atomic-write/src/index.ts b/packages/util/atomic-write/src/index.ts index 70af9fa40b..3e5764a329 100644 --- a/packages/util/atomic-write/src/index.ts +++ b/packages/util/atomic-write/src/index.ts @@ -11,7 +11,7 @@ */ import { randomBytes } from 'node:crypto' -import { mkdir, rename, rm, writeFile } from 'node:fs/promises' +import { lstat, mkdir, rename, rm, writeFile } from 'node:fs/promises' import { dirname } from 'node:path' /** @@ -63,46 +63,82 @@ export async function writeFileAtomic(filename: string, content: string, options } } -/** Whether an exclusive create failed because the path already exists. */ -function isEEXIST(error: unknown): boolean { - return (error as NodeJS.ErrnoException | null)?.code === 'EEXIST' +/** Whether an exclusive create found an existing lock. */ +async function isLockContention(error: unknown, lockPath: string): Promise { + const code = (error as NodeJS.ErrnoException | null)?.code + if (code === 'EEXIST') return true + if (code !== 'EPERM') return false + try { + await lstat(lockPath) + return true + } catch { + // Keep the original EPERM authoritative when lock existence is unproven. + return false + } } /** - * Writer-lock protocol constants. These are robustness invariants of the - * cross-process write protocol, not deployment tunables: contention normally - * resolves within the retry deadline, while expiry fails the contender without - * guessing whether the existing lock still has an owner. + * Retry cadence for a contended lock. These stay robustness invariants of the + * cross-process write protocol rather than deployment tunables: they govern how + * often a contender asks, which no caller has a reason to vary. */ const LOCK_RETRY_INITIAL_MS = 20 const LOCK_RETRY_MAX_MS = 200 -const LOCK_TIMEOUT_MS = 2_000 + +/** + * How long a contender waits when the caller states no limit — sized for the + * render-and-rename cycle every call site had when this package was written. + * Expiry fails the contender rather than guessing whether the existing lock + * still has an owner. How long is *worth* waiting is a property of the + * operation the lock holder runs, which is why {@link FileLockOptions.waitMs} + * exists; the value here is the floor for an operation that does file work + * alone. + */ +const DEFAULT_LOCK_WAIT_MS = 2_000 + +/** Options for one {@link withFileLock} acquisition. */ +export interface FileLockOptions { + /** + * Maximum time to wait for the lock, in milliseconds. State one when the + * holder's operation legitimately runs longer than file work — a credential + * mutation that refreshes a token performs a network round trip while + * holding the lock, and leaving the default in place would fail every other + * writer of the same file for the duration. Waiting is productive: a + * contender that acquires the lock afterwards re-reads the committed state. + */ + waitMs?: number +} /** * Hold the cross-process writer lock for `filename` around one operation. The * lock is a `wx`-created sibling (`.lock`); paired with the * rename-based commit of {@link writeFileAtomic}, readers stay lock-free and - * only writers contend. Contention backs off exponentially and fails with a - * timed-out error after the deadline. The contender never removes an existing - * lock because file age cannot prove that its owner stopped; orphan recovery - * is an operator action. The parent directory must exist. + * only writers contend. `EEXIST` is contention directly; an `EPERM` is + * contention only when a fresh `lstat` confirms the lock path exists, covering + * Windows exclusive-create behavior without hiding an unrelated permission + * failure. Contention backs off exponentially and fails with a timed-out error + * after the deadline. The contender never removes an existing lock because + * file age cannot prove that its owner stopped; orphan recovery is an operator + * action. The parent directory must exist. * @param filename - the file whose writers this lock serializes. * @param operation - the read-render-commit cycle to run while holding the lock. + * @param options - acquisition options; omitted waits {@link DEFAULT_LOCK_WAIT_MS}. * @returns the operation's result; the lock releases on both outcomes. */ export async function withFileLock( filename: string, operation: () => Promise, + options?: FileLockOptions, ): Promise { const lockPath = `${filename}.lock` - const deadline = Date.now() + LOCK_TIMEOUT_MS + const deadline = Date.now() + (options?.waitMs ?? DEFAULT_LOCK_WAIT_MS) let delay = LOCK_RETRY_INITIAL_MS for (;;) { try { await writeFile(lockPath, `${process.pid}\n`, { mode: 0o600, flag: 'wx' }) break } catch (error) { - if (!isEEXIST(error)) throw error + if (!await isLockContention(error, lockPath)) throw error } if (Date.now() >= deadline) { throw new Error(`atomic-write: timed out waiting for the writer lock at ${lockPath}`) diff --git a/packages/util/atomic-write/tests/atomic-write.spec.ts b/packages/util/atomic-write/tests/atomic-write.spec.ts index e71e5b7abd..683abe51bc 100644 --- a/packages/util/atomic-write/tests/atomic-write.spec.ts +++ b/packages/util/atomic-write/tests/atomic-write.spec.ts @@ -1,13 +1,45 @@ -import { lstat, mkdir, mkdtemp, readFile, readdir, stat, symlink, writeFile } from 'node:fs/promises' +import { lstat, mkdir, mkdtemp, readFile, readdir, rm, stat, symlink, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' -import { describe, expect, it } from 'vitest' +import { afterEach, describe, expect, it, vi } from 'vitest' import { withFileLock, writeFileAtomic } from '../src/index.ts' +const state = vi.hoisted(() => ({ failLockCreateWithEPERM: false })) + +vi.mock('node:fs/promises', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + writeFile: (async (path: unknown, ...rest: never[]) => { + if (state.failLockCreateWithEPERM && String(path).endsWith('.lock')) { + state.failLockCreateWithEPERM = false + throw Object.assign(new Error('EPERM: injected exclusive-create failure'), { code: 'EPERM' }) + } + return (actual.writeFile as (path: unknown, ...args: never[]) => Promise)(path, ...rest) + }) as typeof actual.writeFile, + } +}) + +afterEach(() => { + state.failLockCreateWithEPERM = false +}) + async function scratch(): Promise { return mkdtemp(join(tmpdir(), 'dsh-atomic-write-')) } +/** Resolve once the lockfile exists, so contention is measured against a held lock. */ +async function waitForLock(lockPath: string): Promise { + for (;;) { + try { + await stat(lockPath) + return + } catch { + await new Promise(resolve => setTimeout(resolve, 5)) + } + } +} + describe('writeFileAtomic', () => { it('creates the file and its parents with exactly the stated mode', async () => { const dir = await scratch() @@ -48,6 +80,32 @@ describe('writeFileAtomic', () => { }) describe('withFileLock', () => { + it('retries EPERM only when the lock path currently exists', async () => { + const dir = await scratch() + const target = join(dir, 'document') + const lockPath = `${target}.lock` + await writeFile(lockPath, 'holder\n') + const release = setTimeout(() => { void rm(lockPath, { force: true }) }, 50) + state.failLockCreateWithEPERM = true + let called = false + + try { + await withFileLock(target, async () => { called = true }) + } finally { + clearTimeout(release) + } + expect(called).toBe(true) + }) + + it('preserves EPERM when no lock path exists', async () => { + const dir = await scratch() + const operation = vi.fn(async () => {}) + state.failLockCreateWithEPERM = true + + await expect(withFileLock(join(dir, 'document'), operation)).rejects.toMatchObject({ code: 'EPERM' }) + expect(operation).not.toHaveBeenCalled() + }) + it('rejects an invalid parent hierarchy before running the operation', async () => { const dir = await scratch() const parent = join(dir, 'not-a-directory') @@ -59,4 +117,33 @@ describe('withFileLock', () => { })).rejects.toThrow(/ENOENT|ENOTDIR|not a directory/i) expect(called).toBe(false) }) + + it('waits for the caller-stated limit rather than the protocol default', async () => { + // An operation whose work includes a network round trip legitimately holds + // the lock far longer than the render-and-rename the default was sized + // for. The limit is per call so one such operation cannot fail every other + // writer of the same file, and a caller that states a short one still + // fails fast. + const dir = await scratch() + const target = join(dir, 'document') + let release = (): void => {} + const held = new Promise((resolve) => { release = resolve }) + const holder = withFileLock(target, () => held) + // The holder owns the lock once its lockfile exists; contending before + // that would measure nothing. + await waitForLock(`${target}.lock`) + + // Elapsed time is the assertion that distinguishes a honoured limit from + // the ignored argument: without it the contender simply waits out the + // protocol default and fails with the same message. + const startedAt = Date.now() + await expect(withFileLock(target, async () => 'impatient', { waitMs: 50 })) + .rejects.toThrow(/timed out waiting for the writer lock/) + expect(Date.now() - startedAt).toBeLessThan(1_000) + + const patient = withFileLock(target, async () => 'patient', { waitMs: 10_000 }) + release() + await holder + expect(await patient).toBe('patient') + }) }) diff --git a/packages/util/brand/package.json b/packages/util/brand/package.json index 63a9bd0ac8..fd383c1360 100644 --- a/packages/util/brand/package.json +++ b/packages/util/brand/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-brand", "description": "Type-only Branded nominal-typing primitive for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/util/crypto/README.i18n.yaml b/packages/util/crypto/README.i18n.yaml new file mode 100644 index 0000000000..50e8178e37 --- /dev/null +++ b/packages/util/crypto/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/util/crypto/README.md +README.md: 5c01ed2dbdc6d5639867d848b3a97ccb06b9d347 +README.zh.md: fe8ab5d81eee0faa30d5f09c84ca82e0475b1c91 diff --git a/packages/util/crypto/README.md b/packages/util/crypto/README.md new file mode 100644 index 0000000000..5c01ed2dbd --- /dev/null +++ b/packages/util/crypto/README.md @@ -0,0 +1,31 @@ +# dsh-util-crypto + +English | [中文](README.zh.md) + +Zero-dependency v4 UUID minting over `crypto.getRandomValues` — the one random primitive every shipped context provides. `crypto.randomUUID` is a secure-context Web API: a page or worker served over plain HTTP on a LAN address (the browser preview deployment) has no such method, so code that must run there cannot call it. The repository-wide `no-restricted-properties` lint rule points `crypto.randomUUID` callers here; Node-only code importing `randomUUID` from `node:crypto` stays as it is. + +It is a **library, not a service or plugin**: no `ctx`, registers nothing, holds no state. + +## API + +```ts +import { randomUUID, type Uuid } from '@deepseek-ai/dsh-util-crypto' +``` + +| Export | Role | +|---|---| +| `randomUUID()` | Random RFC 9562 v4 UUID string, minted from `crypto.getRandomValues`. Drop-in for `crypto.randomUUID()`. | +| `Uuid` | The five-group UUID string type, matching `crypto.randomUUID`'s declared return shape. | + +## Model Experience + +Indirectly, through consumers that mint request, session, and attachment identifiers with it, none of which enter prompts as semantic content. + +#### KV Cache effect + +No direct invalidation; identifier-minting consumers own any request changes. + +## Known Limitations and Deferred Work + +- **v4 only** — no other UUID versions, namespaces, or parsing; consumers needing more should take a real UUID dependency. +- **Uniqueness is probabilistic** — 122 random bits, the same guarantee `crypto.randomUUID` gives; nothing here detects collisions. diff --git a/packages/util/crypto/README.zh.md b/packages/util/crypto/README.zh.md new file mode 100644 index 0000000000..fe8ab5d81e --- /dev/null +++ b/packages/util/crypto/README.zh.md @@ -0,0 +1,31 @@ +# dsh-util-crypto + +[English](README.md) | 中文 + +零依赖的 v4 UUID 铸造,基于 `crypto.getRandomValues`——所有发布上下文都提供的那个随机原语。`crypto.randomUUID` 是安全上下文限定的 Web API:经普通 HTTP 在局域网地址上提供的页面或 worker(浏览器预览部署)根本没有这个方法,必须在那里运行的代码不能调它。全仓 `no-restricted-properties` lint 规则把 `crypto.randomUUID` 的调用者指到这里;只跑在 Node 的代码从 `node:crypto` 导入 `randomUUID` 维持原样。 + +它是**库,不是服务也不是插件**:无 `ctx`、不注册任何东西、不持有状态。 + +## API + +```ts +import { randomUUID, type Uuid } from '@deepseek-ai/dsh-util-crypto' +``` + +| 导出 | 角色 | +|---|---| +| `randomUUID()` | 随机 RFC 9562 v4 UUID 字符串,由 `crypto.getRandomValues` 铸造。可原位替换 `crypto.randomUUID()`。 | +| `Uuid` | 五段式 UUID 字符串类型,与 `crypto.randomUUID` 声明的返回形状一致。 | + +## Model Experience + +间接地,经由用它铸造请求、会话与附件标识符的消费方,这些标识符均不作为语义内容进入提示词。 + +#### KV Cache effect + +无直接失效;铸造标识符的消费方自行负责其请求变化。 + +## Known Limitations and Deferred Work + +- **仅 v4**——不提供其他 UUID 版本、命名空间或解析;需要更多能力的消费方应引入真正的 UUID 依赖。 +- **唯一性是概率性的**——122 位随机,与 `crypto.randomUUID` 同级保证;此处不做碰撞检测。 diff --git a/packages/util/crypto/package.json b/packages/util/crypto/package.json new file mode 100644 index 0000000000..88a3bca3ed --- /dev/null +++ b/packages/util/crypto/package.json @@ -0,0 +1,42 @@ +{ + "name": "@deepseek-ai/dsh-util-crypto", + "description": "Zero-dependency crypto-adjacent helpers usable in every context (browser, worker, Node) including insecure origins: v4 UUID minting over crypto.getRandomValues, with room for sibling helpers", + "version": "0.1.1-rc.2", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/util/crypto" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "peerDependencies": { + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + }, + "devDependencies": { + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/cordis": "workspace:^" + } +} diff --git a/packages/util/crypto/src/index.ts b/packages/util/crypto/src/index.ts new file mode 100644 index 0000000000..afde4fe67c --- /dev/null +++ b/packages/util/crypto/src/index.ts @@ -0,0 +1,27 @@ +/** + * UUID minting that works in every JavaScript context this repository ships + * to. `crypto.randomUUID` is a secure-context Web API — a page or worker + * served over plain HTTP on a LAN address has no such method — while + * `crypto.getRandomValues` is unrestricted everywhere (browsers, workers, + * Node ≥ 19). One implementation here replaces per-caller polyfills; the + * `no-restricted-properties` lint rule points `crypto.randomUUID` callers at + * this module. + * @module @deepseek-ai/dsh-util-crypto + */ + +/** RFC 9562 UUID string, the shape `crypto.randomUUID` declares. */ +export type Uuid = `${string}-${string}-${string}-${string}-${string}` + +/** + * Random v4 UUID, minted from `crypto.getRandomValues`. + * @returns the UUID string. + */ +export function randomUUID(): Uuid { + const bytes = globalThis.crypto.getRandomValues(new Uint8Array(16)) + // RFC 9562 §5.4: version 4 in the high nibble of byte 6, variant 10 in byte 8. + const hex = Array.from(bytes, (byte, index) => { + const pinned = index === 6 ? (byte & 0x0f) | 0x40 : index === 8 ? (byte & 0x3f) | 0x80 : byte + return pinned.toString(16).padStart(2, '0') + }).join('') + return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20)}` +} diff --git a/packages/util/crypto/src/invariant.ts b/packages/util/crypto/src/invariant.ts new file mode 100644 index 0000000000..e889ba1023 --- /dev/null +++ b/packages/util/crypto/src/invariant.ts @@ -0,0 +1,30 @@ +/** + * Package-owned invariant companion for `@deepseek-ai/dsh-util-crypto`. + * @module @deepseek-ai/dsh-util-crypto/invariant + */ + +/* jscpd:ignore-start */ +import type { Context } from '@deepseek-ai/cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-util-crypto' + +/** Cordis companion plugin name. */ +export const name = 'util-crypto-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: this pure utility owns no event stream or mutable runtime data; its value + * algebra is enforced by unit tests. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/util/crypto/tests/uuid.spec.ts b/packages/util/crypto/tests/uuid.spec.ts new file mode 100644 index 0000000000..9a2f261648 --- /dev/null +++ b/packages/util/crypto/tests/uuid.spec.ts @@ -0,0 +1,29 @@ +/** + * The minted UUID's contract: RFC 9562 v4 shape (version and variant bits + * pinned), uniqueness across calls, and no dependence on the secure-context + * `crypto.randomUUID` — the reason this package exists. + */ +import { describe, expect, it, vi } from 'vitest' +import { randomUUID } from '../src/index.ts' + +const V4_SHAPE = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/ + +describe('randomUUID', () => { + it('emits RFC 9562 v4 ids', () => { + for (let round = 0; round < 64; round += 1) expect(randomUUID()).toMatch(V4_SHAPE) + }) + + it('emits distinct ids across calls', () => { + expect(new Set(Array.from({ length: 64 }, () => randomUUID())).size).toBe(64) + }) + + it('mints without the platform randomUUID, the insecure-origin shape', () => { + const bare = { getRandomValues: globalThis.crypto.getRandomValues.bind(globalThis.crypto) } + vi.stubGlobal('crypto', bare) + try { + expect(randomUUID()).toMatch(V4_SHAPE) + } finally { + vi.unstubAllGlobals() + } + }) +}) diff --git a/packages/client/schema-form/tsconfig.json b/packages/util/crypto/tsconfig.json similarity index 64% rename from packages/client/schema-form/tsconfig.json rename to packages/util/crypto/tsconfig.json index 34abf11c47..779effc3cc 100644 --- a/packages/client/schema-form/tsconfig.json +++ b/packages/util/crypto/tsconfig.json @@ -1,5 +1,5 @@ { - "extends": "../../../tsconfig.base.client.json", + "extends": "../../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", "outDir": "lib/types" @@ -8,9 +8,6 @@ "src" ], "references": [ - { - "path": "../../../vendor/schemastery" - }, { "path": "../../runtime-diagnostics/invariants" } diff --git a/packages/util/home-paths/package.json b/packages/util/home-paths/package.json index 3bd0db8091..45ccf74b9a 100644 --- a/packages/util/home-paths/package.json +++ b/packages/util/home-paths/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-home-paths", "description": "Shared filesystem path helpers for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/util/launch-environment/README.i18n.yaml b/packages/util/launch-environment/README.i18n.yaml index b28e720234..53a8b13705 100644 --- a/packages/util/launch-environment/README.i18n.yaml +++ b/packages/util/launch-environment/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/util/launch-environment/README.md README.md: 80ec4d59c38ee296e836f8cff52e3a89b0700ff5 -README.zh.md: 5dc9bfd1fc70e5432ca9f62e81a7d7a23f4a48c9 +README.zh.md: 8fb8df559c1469b25543c15fe03ee7cf67492304 diff --git a/packages/util/launch-environment/README.zh.md b/packages/util/launch-environment/README.zh.md index 5dc9bfd1fc..8fb8df559c 100644 --- a/packages/util/launch-environment/README.zh.md +++ b/packages/util/launch-environment/README.zh.md @@ -32,5 +32,5 @@ const endpoint = launchEnvironmentOf(ctx).get('DEEPSEEK_BASE_URL')?.value ## 已知限制与暂缓事项 -- **快照不是子进程边界**:每一层同样会被物化进 `process.env`,因此项目里的普通变量会按 [`dsh-subprocess`](../../subprocess/subprocess/README.md) 的清洗规则抵达子进程。产品启动器的 [`.env` 约定](../../boot/app-boot/README.md#profiles) 会在物化之前拒绝 bootstrap 变量。 +- **快照不是子进程边界**:每一层同样会被物化进 `process.env`,因此项目里的普通变量会按 [`dsh-subprocess`](../../subprocess/subprocess/README.zh.md) 的清洗规则抵达子进程。产品启动器的 [`.env` 约定](../../boot/app-boot/README.zh.md#profiles) 会在物化之前拒绝 bootstrap 变量。 - **没有按工作区划分的层**:项目层是*调用*目录,在启动时固定。之后在 Web UI 中选择的工作区不贡献任何内容,这是刻意的:跟随它等于让模型自己的工作区在会话中途改变 harness 的环境。 diff --git a/packages/util/launch-environment/package.json b/packages/util/launch-environment/package.json index eeea9a70ee..18db056476 100644 --- a/packages/util/launch-environment/package.json +++ b/packages/util/launch-environment/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-launch-environment", "description": "Immutable DeepSeek Harness launch environment that records which layer supplied each value", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/util/native-command/README.i18n.yaml b/packages/util/native-command/README.i18n.yaml index e890eca65d..abc1f7ed2a 100644 --- a/packages/util/native-command/README.i18n.yaml +++ b/packages/util/native-command/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/util/native-command/README.md README.md: 8a552ac36fac150ae18b2a8adaab1dd489ecb476 -README.zh.md: 618e50afe7cd157b2dfc4763adfbf1d5d88d213e +README.zh.md: 7b5ea7fa1985d394b88fd5bcdbce9d9831eef8ee diff --git a/packages/util/native-command/README.zh.md b/packages/util/native-command/README.zh.md index 618e50afe7..7b5ea7fa19 100644 --- a/packages/util/native-command/README.zh.md +++ b/packages/util/native-command/README.zh.md @@ -4,7 +4,7 @@ 宿主原生 OS 集成共享的**零依赖免 shell `execFile` 运行器**:一次 `runNativeCommand(command, args, signal)` 调用直接 spawn 可执行文件(绝不拼 shell 字符串),以 utf8 捕获 stdout/stderr,把调用方的 abort 传播为子进程终止,并在 Windows 上隐藏瞬时控制台窗口。失败时,调用会以错误拒绝;该错误附带退出 `code` 与两路已捕获输出,调用方无需重跑即可分类(工具缺失、已取消、真实失败)。 -它的两个消费方都是宿主侧原生集成:[`directory-picker-native`](../../host/directory-picker-native/README.md) 后端的 OS 选择器命令,以及网关将路径交由默认应用打开的操作([`dsh-host-apiproxy`](../../host/apiproxy/README.md) 的 `host.openPath`)。`NativeCommandRunner` 类型是这些调用方的可注入命令边界。 +它的两个消费方都是宿主侧原生集成:[`directory-picker-native`](../../host/directory-picker-native/README.zh.md) 后端的 OS 选择器命令,以及网关将路径交由默认应用打开的操作([`dsh-host-apiproxy`](../../host/apiproxy/README.zh.md) 的 `host.openPath`)。`NativeCommandRunner` 类型是这些调用方的可注入命令边界。 它是**库,不是服务或插件**:没有 `ctx`、不注册任何东西、不持有状态、不发事件。 diff --git a/packages/util/native-command/package.json b/packages/util/native-command/package.json index 36547ffee9..285c468c95 100644 --- a/packages/util/native-command/package.json +++ b/packages/util/native-command/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-native-command", "description": "Zero-dependency no-shell execFile runner for host-native OS integrations: utf8 stdio capture, abort propagation, Windows hide", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/util/output-retention/README.i18n.yaml b/packages/util/output-retention/README.i18n.yaml index ede6c1260f..aa30e2695f 100644 --- a/packages/util/output-retention/README.i18n.yaml +++ b/packages/util/output-retention/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/util/output-retention/README.md README.md: e1d2c828fd4c6526cc391e7f93d7fe20df26e268 -README.zh.md: 36730ab83dc7f6a93376a8f0272160d29935e6e0 +README.zh.md: 51055dd33e2cce3b82a1d15d6994f9d9f37a0579 diff --git a/packages/util/output-retention/README.zh.md b/packages/util/output-retention/README.zh.md index 36730ab83d..51055dd33e 100644 --- a/packages/util/output-retention/README.zh.md +++ b/packages/util/output-retention/README.zh.md @@ -4,7 +4,7 @@ 一个轻依赖的**保留**库:为必须限制返回上下文量的工具提供有界的面向模型输出。调用方将项或文本分片送入有界对象,然后取回保留的内容和精确的省略元数据。 -该库**只**负责这个机制问题:*「我们保留了什么,又省略了什么?」*。工具专用代码保留其业务语义:文件分组、行号、退出码、提供方错误状态、每行预览截断、spill 文件以及面向模型的文案。这就是 [Agent Note](../../../.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.md) 划定的边界。 +该库**只**负责这个机制问题:*「我们保留了什么,又省略了什么?」*。工具专用代码保留其业务语义:文件分组、行号、退出码、提供方错误状态、每行预览截断、spill 文件以及面向模型的文案。这就是 [Agent Note](../../../.agents/notes/implemented/architecture/2026-07-06-tool-result-retention-library.zh.md) 划定的边界。 它是**库,而非服务或插件**:没有 `ctx`,不注册任何内容,不发出任何事件。状态只存在于每个 retainer(一次累积)中,绝不跨调用。工具包直接导入它。 diff --git a/packages/util/output-retention/package.json b/packages/util/output-retention/package.json index cdd47c905c..491bb5b49c 100644 --- a/packages/util/output-retention/package.json +++ b/packages/util/output-retention/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-output-retention", "description": "Zero-dependency bounded-retention primitive: ItemRetainer/TextRetainer + neutral notice helpers (what did we keep, what did we omit)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/util/timeout/README.i18n.yaml b/packages/util/timeout/README.i18n.yaml index c000fae463..43c4165a7c 100644 --- a/packages/util/timeout/README.i18n.yaml +++ b/packages/util/timeout/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/util/timeout/README.md README.md: c29f6a7661abf40eadafc7637c1bbca509a62a24 -README.zh.md: ca05fcaf0ac7f3379d746e11a123294e3febf436 +README.zh.md: 38bbebe1462e4351a207b28a831d1274d9578e58 diff --git a/packages/util/timeout/README.zh.md b/packages/util/timeout/README.zh.md index ca05fcaf0a..38bbebe146 100644 --- a/packages/util/timeout/README.zh.md +++ b/packages/util/timeout/README.zh.md @@ -4,7 +4,7 @@ 超时的**时序与分类**部分:一个零依赖纯函数库(无运行时 harness 依赖),由每个需要限制调用方超时提示、启动 deadline,并在之后区分「已超时」与「已取消」的能力共享。 -它**不负责终止**。它发出的信号只会*通知*;真正停止工作仍由各能力负责,因为机制各不相同:bash 对操作系统进程组发送 SIGKILL,web 关闭 `fetch` 套接字,没有任何共享层能够承担全部终止机制。[Agent Note](../../../.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md) 将边界划定为:共享时序/分类,将强制终止保留在本地。 +它**不负责终止**。它发出的信号只会*通知*;真正停止工作仍由各能力负责,因为机制各不相同:bash 对操作系统进程组发送 SIGKILL,web 关闭 `fetch` 套接字,没有任何共享层能够承担全部终止机制。[Agent Note](../../../.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.zh.md) 将边界划定为:共享时序/分类,将强制终止保留在本地。 它是**库,而非服务或插件**:没有 `ctx`,不注册任何内容,不持有状态,也不发出事件。「超时服务」必须了解如何停止每项能力的工作,这正是微内核要排除在共享层之外的知识。 @@ -52,7 +52,7 @@ export async function runWithDeadline(upstream: AbortSignal | undefined, timeout ## 哪些操作不设置超时 -本地文件 `read`/`write`/`edit` 不接受 `timeoutMs`:文件 IO 不设时限地运行,因为截止时间会中止操作系统仍会完成的工作。详见[文件系统子系统页面](../../../docs/subsystems/filesystem.md)。 +本地文件 `read`/`write`/`edit` 不接受 `timeoutMs`:文件 IO 不设时限地运行,因为截止时间会中止操作系统仍会完成的工作。详见[文件系统子系统页面](../../../docs/subsystems/filesystem.zh.md)。 ## 模型体验 diff --git a/packages/util/timeout/package.json b/packages/util/timeout/package.json index cbe16cfcaf..43f8a36507 100644 --- a/packages/util/timeout/package.json +++ b/packages/util/timeout/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-timeout", "description": "Zero-dependency timeout/deadline primitive: clampTimeout, deadline, timeoutOf, TimeoutReason (timing + classification only, no termination)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/web/README.i18n.yaml b/packages/web/README.i18n.yaml index 19da6ba386..06a41eba22 100644 --- a/packages/web/README.i18n.yaml +++ b/packages/web/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/web/README.md README.md: fc37d7cdead59138db149b5a86f0a0c031d40037 -README.zh.md: 53fe673ddaed235bbb938db4757e9cd240928d61 +README.zh.md: 40a64e09b85b0655739f73abe6388d6cc2b40a0d diff --git a/packages/web/README.zh.md b/packages/web/README.zh.md index 53fe673dda..40a64e09b8 100644 --- a/packages/web/README.zh.md +++ b/packages/web/README.zh.md @@ -6,13 +6,13 @@ | 包 | 职责 | ctx key | |---|---|---| -| [`web/`](web/README.md) | 定义 web 提供方注册、选择和共享错误 | `ctx.web` | -| [`web-search-exa/`](web-search-exa/README.md) | 通过 Exa 提供 web 搜索 | 注册到 `ctx.web` | -| [`web-search-perplexity/`](web-search-perplexity/README.md) | 通过 Perplexity 提供 web 搜索 | 注册到 `ctx.web` | -| [`web-search-deepseek/`](web-search-deepseek/README.md) | 提供 DeepSeek 原生 web 搜索 | 注册到 `ctx.web` | -| [`web-fetch-http/`](web-fetch-http/README.md) | 抓取公共 HTTP 和 HTTPS 资源 | 注册到 `ctx.web` | -| [`tool-web/`](tool-web/README.md) | 向模型公开 web 搜索和抓取 | 注册到 `ctx.tools` | +| [`web/`](web/README.zh.md) | 定义 web 提供方注册、选择和共享错误 | `ctx.web` | +| [`web-search-exa/`](web-search-exa/README.zh.md) | 通过 Exa 提供 web 搜索 | 注册到 `ctx.web` | +| [`web-search-perplexity/`](web-search-perplexity/README.zh.md) | 通过 Perplexity 提供 web 搜索 | 注册到 `ctx.web` | +| [`web-search-deepseek/`](web-search-deepseek/README.zh.md) | 提供 DeepSeek 原生 web 搜索 | 注册到 `ctx.web` | +| [`web-fetch-http/`](web-fetch-http/README.zh.md) | 抓取公共 HTTP 和 HTTPS 资源 | 注册到 `ctx.web` | +| [`tool-web/`](tool-web/README.zh.md) | 向模型公开 web 搜索和抓取 | 注册到 `ctx.tools` | -[web 能力决策](../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md)记录了搜索和抓取共用一项提供方选择服务的原因。 +[web 能力决策](../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md)记录了搜索和抓取共用一项提供方选择服务的原因。 -子系统参考——搜索/抓取请求与结果、可用性、`WebError`——见 [docs/subsystems/web.md](../../docs/subsystems/web.md);依据(含延后的 SSRF 防护)见 [web 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md)。 +子系统参考——搜索/抓取请求与结果、可用性、`WebError`——见 [docs/subsystems/web.md](../../docs/subsystems/web.zh.md);依据(含延后的 SSRF 防护)见 [web 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md)。 diff --git a/packages/web/tool-web/README.i18n.yaml b/packages/web/tool-web/README.i18n.yaml index 2e50c2ecb2..5af88ca380 100644 --- a/packages/web/tool-web/README.i18n.yaml +++ b/packages/web/tool-web/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/web/tool-web/README.md -README.md: 81dc4d3f3dbf9788e68b86513bfa09271fb99c84 -README.zh.md: 8e976f23f72347e12c6aea35f9b99c5d8ad31d85 +README.md: 787b70a5070f48a3bac6435d5d7e8b64c01e0341 +README.zh.md: f0185deffa8643317f5f01f7e1c3af7af1ce1194 diff --git a/packages/web/tool-web/README.md b/packages/web/tool-web/README.md index 81dc4d3f3d..787b70a507 100644 --- a/packages/web/tool-web/README.md +++ b/packages/web/tool-web/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The model-facing web tool suite — `web_search` and `web_fetch` — over the [web capability seam](../web/README.md) (`ctx.web`). It owns model-facing concerns only: tool names, JSON schemas, snake_case argument names, prompt sections, the result-count bound, result formatting, HTML→markdown presentation, and the UI presentation projection — `presentCall`, `presentResult` (a `card: 'web'` result card discriminated by `kind: 'search' | 'fetch'`), and the `output.presentationMeta` that carries the structured search sources or the fetch summary the lossy render text cannot (see the [web-result-card Agent Note](../../../.agents/notes/implemented/feature/2026-07-30-web-result-card.md)). All web access goes through `ctx.web`; this package never imports a concrete provider. Neither tool exposes a model-facing timeout — each tool's cooperative tool-call budget is declared here via config (`fetchTimeoutMs`/`searchTimeoutMs`, attached as `ToolDefinition.timeoutMs`) and enforced by [`@deepseek-ai/dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.md) (a `tools/execute` wrapper); each tool just forwards `exec.signal` to the seam. +The model-facing web tool suite — `web_search` and `web_fetch` — over the [web capability seam](../web/README.md) (`ctx.web`). It owns model-facing concerns only: tool names, JSON schemas, snake_case argument names, prompt sections, the result-count bound, result formatting, HTML→markdown presentation, and the UI presentation projection — `presentCall`, `presentResult` (a `card: 'web'` result card discriminated by `kind: 'search' | 'fetch'`), and the `output.presentationMeta` that carries the structured search sources or the fetch summary the lossy render text cannot (see the [web-result-card Agent Note](../../../.agents/notes/implemented/feature/2026-07-30-web-result-card.md)). All web access goes through `ctx.web`; this package never imports a concrete provider. Neither tool exposes a model-facing timeout — each tool's cooperative tool-call budget is declared here via config (`fetchTimeoutMs`/`searchTimeoutMs`, attached as `ToolDefinition.timeoutMs`) and enforced by [`@deepseek-ai/dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.md) (a `tools/execute` wrapper). Single operations forward `exec.signal`; a multi-query search fuses it with batch cancellation so a failed query aborts its siblings. Each tool is registered independently; a product that wants only one disables the other via config (`{ search: false }` / `{ fetch: false }`). Search guidance mentions `web_fetch` only when fetch is also config-enabled; a search-only composition instead tells the model to use returned snippets and cite their URLs. @@ -10,7 +10,7 @@ Each tool is registered independently; a product that wants only one disables th | Tool | Args | Behavior | |---|---|---| -| `web_search` | `query` (string) | Discovery. Returns an optional answer plus source URLs. `max_results` is **not** model-facing — the tool sets the bound (the `searchMaxResults` config, default 8) and passes it to the seam. | +| `web_search` | `queries` (required string[]) | Discovery. Returns an optional answer plus source URLs. It runs one to `searchMaxQueries` distinct searches concurrently and merges their sources in round-robin order before applying the combined `searchMaxResults` cap. A one-item array performs one search. Exact duplicate queries run once. Any failed search aborts the remaining batch, which settles before the call returns an error. Neither bound is model-facing. | | `web_fetch` | `url` (string) | Retrieves a specific URL. HTML bodies are rendered to markdown (turndown with GFM tables/strikethrough); text bodies pass through. A non-2xx status is reported, not an error. The tool-call timeout is deployment policy (`dsh-tool-call-timeout-policy`), not a model argument. | Both tools opt into concurrent scheduling because provider reads return content without mutating parent-agent state. @@ -23,12 +23,13 @@ The normalized service results are also the canonical tool values: `WebSearchRes |---|---|---| | `search` | `true` | Register `web_search`. | | `fetch` | `true` | Register `web_fetch`. | -| `searchMaxResults` | `8` | Upper bound on sources returned by one `web_search` call (the seam truncates a longer provider list and flags it). | +| `searchMaxResults` | `8` | Upper bound on sources returned by one `web_search` call (the seam truncates each provider list; the tool also caps a combined multi-query list). | +| `searchMaxQueries` | `4` | Upper bound on queries accepted by one `web_search` call. The configured value appears in its prompt guidance and schema descriptions. | | `fetchTimeoutMs` | `30000` | Cooperative tool-call timeout budget (ms) for `web_fetch`. | | `searchTimeoutMs` | `30000` | Cooperative tool-call timeout budget (ms) for `web_search`. | | `fetchMaxOutputChars` | `200000` | Cap on source characters converted synchronously and on one complete `web_fetch` output (header, rendered body, and footer); a cut body gets the truncation notice when it fits. | -`fetchTimeoutMs`/`searchTimeoutMs` declare each tool's cooperative timeout budget (attached as `ToolDefinition.timeoutMs`), enforced by [`@deepseek-ai/dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.md); the model-facing schema exposes no timeout argument. `fetchMaxOutputChars` bounds both synchronous conversion work and the complete rendered result: only that many source characters are converted, and the header, converted prefix, and truncation notice are then capped together. The default leaves headroom above the local provider's 100,000-character body cap, but rendered expansion can still make the final bound truncate the result. +`searchMaxQueries` bounds the accepted array before exact-string deduplication, provider fan-out, and combined provider-answer growth; validation rejects an oversized array before any search starts, then dispatch keeps the first occurrence of each query. Together with each provider's own controls such as `maxUses`, these independent settings are the product's search budgets; the generic seam does not expose provider-internal native-search accounting. `fetchTimeoutMs`/`searchTimeoutMs` declare each tool's cooperative timeout budget (attached as `ToolDefinition.timeoutMs`), enforced by [`@deepseek-ai/dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.md); the model-facing schema exposes no timeout argument. `fetchMaxOutputChars` bounds both synchronous conversion work and the complete rendered result: only that many source characters are converted, and the header, converted prefix, and truncation notice are then capped together. The default leaves headroom above the local provider's 100,000-character body cap, but rendered expansion can still make the final bound truncate the result. ```yaml - id: tool-web @@ -52,13 +53,13 @@ Search and fetch contribute the web-search and web-fetch guidance below. Search ##### Web search guidance with fetch enabled ```markdown -Use the web_search tool to discover current information on the web. It returns an optional answer plus a list of source URLs. Follow up with web_fetch when you need the full content of a specific result, and cite the relevant URLs as markdown links. +Use the web_search tool to discover current information on the web. The required queries array accepts 1–4 non-empty search queries; use a one-item array for a single search. It returns an optional answer plus a list of source URLs. Follow up with web_fetch when you need the full content of a specific result, and cite the relevant URLs as markdown links. ``` ##### Web search-only guidance ```markdown -Use the web_search tool to discover current information on the web. It returns an optional answer plus a list of source URLs. Use the returned source snippets when available, and cite the relevant URLs as markdown links. +Use the web_search tool to discover current information on the web. The required queries array accepts 1–4 non-empty search queries; use a one-item array for a single search. It returns an optional answer plus a list of source URLs. Use the returned source snippets when available, and cite the relevant URLs as markdown links. ``` ##### Web fetch guidance @@ -69,11 +70,11 @@ Use the web_fetch tool to retrieve the content of a specific HTTP(S) URL (for ex #### Token effect -Fixed guidance cost per request for each config-enabled tool, even when a restriction hides its schema. Toggling fetch changes the search guidance as well as registering or removing the fetch section. +Fixed guidance cost per request for each config-enabled tool, even when a restriction hides its schema. Toggling fetch or changing `searchMaxQueries` changes the search guidance; toggling fetch also registers or removes the fetch section. #### KV Cache effect -Prefix-stable while enabled tools, scope, and guidance text are unchanged. Config enablement—including toggling fetch's search-guidance branch—or plugin lifecycle may invalidate reuse from the first changed prompt section; scoped schema restrictions do not remove it. +Prefix-stable while enabled tools, scope, and guidance text are unchanged. Config enablement—including toggling fetch's search-guidance branch—changing `searchMaxQueries`, or plugin lifecycle may invalidate reuse from the first changed prompt section; scoped schema restrictions do not remove it. ### Tool schemas @@ -83,26 +84,40 @@ The model sees the generated [`web_search` and `web_fetch` schemas](../../../doc #### Token effect -Fixed schema cost per request; config disablement removes both schema and guidance, while a scoped restriction removes only the schema. +Fixed schema cost per request for a resolved `searchMaxQueries`; config disablement removes both schema and guidance, while a scoped restriction removes only the schema. #### KV Cache effect -Prefix-stable while definitions and visibility are unchanged. Config enablement, plugin lifecycle, or scoped restrictions may invalidate reuse from the first changed schema token. +Prefix-stable while definitions, resolved query cap, and visibility are unchanged. Config enablement, changing `searchMaxQueries`, plugin lifecycle, or scoped restrictions may invalidate reuse from the first changed schema token. ### Search result #### What the model sees -The optional provider-owned answer is followed by `Sources:` and data-dependent lines shaped exactly `- []()`, optionally suffixed ` — ()`. With neither answer nor sources the result says `No results found.` A capped list adds `(Showing the first sources. Refine the query for more.)`; every result ends `Cite the relevant URLs above as markdown links in your answer.` +The optional provider-owned answer is followed by `Sources:` and data-dependent lines shaped exactly `- []()`, optionally suffixed ` — ()`. A multi-query call runs each exact query string once, preserving its first position; it labels each provider answer with the originating query as a markdown heading, deduplicates sources by URL, and takes one source at each rank from every query before advancing to the next rank. With neither answer nor sources the result says `No results found.` A capped list adds `(Showing the first sources. Refine the query for more.)`; every result ends `Cite the relevant URLs above as markdown links in your answer.` #### Token effect -Data-dependent results are resent until compaction and sources are capped by `searchMaxResults`. +Data-dependent results are resent until compaction; query fan-out is capped by `searchMaxQueries`, and sources are capped by `searchMaxResults`. #### KV Cache effect Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. +### Search failure + +#### What the model sees + +If any query in a multi-query call fails, `web_search` aborts the other searches, waits for every started search to settle, discards successful results, and returns `Error: ` for the first failure. + +#### Token effect + +Only the retained error result adds tokens; discarded successful results do not enter model history. + +#### KV Cache effect + +Append-only; the error follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Fetch result #### What the model sees @@ -121,7 +136,7 @@ Append-only; newly visible content follows the reusable request prefix and does #### What the model sees -Blank inputs become exactly `Error: query must be a non-empty string` or `Error: url must be a non-empty string`. +Schema validation rejects an absent or non-array `queries` field and non-string array elements before execution. Value errors become exactly `Error: queries must contain at least one query`, `Error: queries must contain at most 1 query` when the configured cap is one, `Error: queries must contain at most queries` for larger caps, `Error: each query must be a non-empty string`, or `Error: url must be a non-empty string`. #### Token effect @@ -133,6 +148,7 @@ Append-only; newly visible content follows the reusable request prefix and does ## Known Limitations and Deferred Work +- **There is no batch-wide native-search counter** — `searchMaxQueries` bounds `ctx.web.search` calls, but a provider may perform several native searches inside each call. For example, a model-backed provider configured with `maxUses` can permit up to `searchMaxQueries × maxUses` native searches; `searchMaxResults` limits only the combined sources returned to the caller. Deployments control cost through these independent consumer and provider settings because the generic seam does not know provider-internal search units. - **HTML→markdown conversion degrades on inputs GFM cannot safely represent** — [turndown](https://github.com/mixmark-io/turndown) (with GFM tables/strikethrough) converts at most `fetchMaxOutputChars` source characters through a real DOM. A conservative 512-level lexical guard passes deeply or ambiguously nested bodies through as raw HTML, conversion exceptions do the same, and table `colspan` is ignored because GFM has no spanning-cell representation; these bounds avoid blocking the event loop or expanding output from an untrusted numeric attribute ([archived dependency decision](../../../.agents/notes/archived/simplification/2026-07-26-turndown-for-tool-web-html-markdown.md)). - **The model-facing API is minimal by design, with promotions deferred** — `max_results` stays a config bound (not a model argument), and `web_fetch` takes only `url` (no `format`/`prompt`/LLM-summarization mode); both are named later steps in [the seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md). - **No web-specific permission policy** — both tools execute without requesting `ctx.approval`; a deployment that needs confirmation must add a `tools/pre-execute` policy, and the package does not define persistent URL/domain grants. diff --git a/packages/web/tool-web/README.zh.md b/packages/web/tool-web/README.zh.md index 8e976f23f7..f0185deffa 100644 --- a/packages/web/tool-web/README.zh.md +++ b/packages/web/tool-web/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -面向模型的 web 工具套件 `web_search` 与 `web_fetch`,构建于 [web 能力 seam](../web/README.md)(`ctx.web`)之上。它只负责面向模型的事项:工具名称、JSON Schema、snake_case 参数名称、提示词区段、结果数量上限、结果格式、HTML→markdown 呈现,以及 UI 呈现投影——`presentCall`、`presentResult`(以 `kind: 'search' | 'fetch'` 区分的 `card: 'web'` 结果卡片),以及承载有损渲染文本无法携带的结构化搜索来源或抓取摘要的 `output.presentationMeta`(见 [web-result-card Agent Note](../../../.agents/notes/implemented/feature/2026-07-30-web-result-card.md))。所有 web 访问都通过 `ctx.web`;该包绝不导入具体提供方。两个工具都不公开面向模型的超时:每个工具的协作式工具调用超时预算通过配置在此声明(`fetchTimeoutMs`/`searchTimeoutMs`,附加为 `ToolDefinition.timeoutMs`),由 [`@deepseek-ai/dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.md)(`tools/execute` 包装层)强制执行;每个工具只把 `exec.signal` 转发给 seam。 +面向模型的 web 工具套件 `web_search` 与 `web_fetch`,构建于 [web 能力 seam](../web/README.zh.md)(`ctx.web`)之上。它只负责面向模型的事项:工具名称、JSON Schema、snake_case 参数名称、提示词区段、结果数量上限、结果格式、HTML→markdown 呈现,以及 UI 呈现投影——`presentCall`、`presentResult`(以 `kind: 'search' | 'fetch'` 区分的 `card: 'web'` 结果卡片),以及承载有损渲染文本无法携带的结构化搜索来源或抓取摘要的 `output.presentationMeta`(见 [web-result-card Agent Note](../../../.agents/notes/implemented/feature/2026-07-30-web-result-card.zh.md))。所有 web 访问都通过 `ctx.web`;该包绝不导入具体提供方。两个工具都不公开面向模型的超时:每个工具的协作式工具调用超时预算通过配置在此声明(`fetchTimeoutMs`/`searchTimeoutMs`,附加为 `ToolDefinition.timeoutMs`),由 [`@deepseek-ai/dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.zh.md)(`tools/execute` 包装层)强制执行。单项操作会转发 `exec.signal`;多查询搜索会把它与批次取消信号融合,使失败查询能够中止其余查询。 每个工具独立注册;只需要其中一个工具的产品可以通过配置禁用另一个(`{ search: false }`/`{ fetch: false }`)。仅当抓取也通过配置启用时,搜索指引才会提及 `web_fetch`;仅启用搜索的组合则会要求模型使用返回的 snippet 并引用其 URL。 @@ -10,7 +10,7 @@ | 工具 | 参数 | 行为 | |---|---|---| -| `web_search` | `query`(string) | 用于发现信息。返回可选答案与来源 URL。`max_results` **不**面向模型:工具设置上限(`searchMaxResults` 配置,默认 8)并传给 seam。 | +| `web_search` | `queries`(必填 string[]) | 用于发现信息。返回可选答案与来源 URL。它会并发执行 1 至 `searchMaxQueries` 个不同搜索,按轮询顺序合并来源,再应用组合后的 `searchMaxResults` 上限。单元素数组执行一次搜索。完全相同的查询只执行一次。任何搜索失败都会中止批次中的其余搜索;批次结算完毕后调用才返回错误。两个上限都不面向模型。 | | `web_fetch` | `url`(string) | 获取特定 URL。HTML 主体渲染为 markdown(turndown,带 GFM 表格/删除线);文本主体原样通过。非 2xx 状态会报告,而非报错。工具调用超时是部署策略(`dsh-tool-call-timeout-policy`),不是模型参数。 | 两个工具都选择并发调度,因为提供方读取会返回内容,不会修改父 agent(智能体)的状态。 @@ -23,12 +23,13 @@ |---|---|---| | `search` | `true` | 注册 `web_search`。 | | `fetch` | `true` | 注册 `web_fetch`。 | -| `searchMaxResults` | `8` | 一次 `web_search` 调用返回的来源数量上限(seam 截断更长的提供方列表并标记)。 | +| `searchMaxResults` | `8` | 一次 `web_search` 调用返回的来源数量上限(seam 截断各提供方列表;工具还会限制多查询组合列表)。 | +| `searchMaxQueries` | `4` | 一次 `web_search` 调用接受的查询数量上限。配置值会出现在提示词指引与 schema 描述中。 | | `fetchTimeoutMs` | `30000` | `web_fetch` 的协作式工具调用超时预算(ms)。 | | `searchTimeoutMs` | `30000` | `web_search` 的协作式工具调用超时预算(ms)。 | | `fetchMaxOutputChars` | `200000` | 同步转换的源字符数与单次完整 `web_fetch` 输出的上限(状态头、渲染后的主体与页脚合并计算);主体被截断时,在能容纳的情况下附带截断提示。 | -`fetchTimeoutMs`/`searchTimeoutMs` 声明每个工具的协作式超时预算(附加为 `ToolDefinition.timeoutMs`),由 [`@deepseek-ai/dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.md) 强制执行;面向模型的 schema 不公开超时参数。`fetchMaxOutputChars` 同时限制同步转换工作量和完整渲染结果:只转换至多该数量的源字符,随后对状态头、转换后的前缀和截断提示合并设限。默认值为本地提供方的 100,000 字符主体上限留出余量,但渲染膨胀仍可能使最终上限截断结果。 +`searchMaxQueries` 在完全相同的字符串去重前限制可接受数组、提供方请求扇出与组合后的提供方答案增长;校验会在任何搜索开始前拒绝超限数组,随后分发只保留每个查询第一次出现的位置。该设置与各提供方自己的 `maxUses` 等控制项共同构成产品的搜索预算;通用 seam 不公开提供方内部的原生搜索计数。`fetchTimeoutMs`/`searchTimeoutMs` 声明每个工具的协作式超时预算(附加为 `ToolDefinition.timeoutMs`),由 [`@deepseek-ai/dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.zh.md) 强制执行;面向模型的 schema 不公开超时参数。`fetchMaxOutputChars` 同时限制同步转换工作量和完整渲染结果:只转换至多该数量的源字符,随后对状态头、转换后的前缀和截断提示合并设限。默认值为本地提供方的 100,000 字符主体上限留出余量,但渲染膨胀仍可能使最终上限截断结果。 ```yaml - id: tool-web @@ -52,13 +53,13 @@ ##### 启用抓取时的 Web 搜索指引 ```markdown -Use the web_search tool to discover current information on the web. It returns an optional answer plus a list of source URLs. Follow up with web_fetch when you need the full content of a specific result, and cite the relevant URLs as markdown links. +Use the web_search tool to discover current information on the web. The required queries array accepts 1–4 non-empty search queries; use a one-item array for a single search. It returns an optional answer plus a list of source URLs. Follow up with web_fetch when you need the full content of a specific result, and cite the relevant URLs as markdown links. ``` ##### 仅搜索时的 Web 搜索指引 ```markdown -Use the web_search tool to discover current information on the web. It returns an optional answer plus a list of source URLs. Use the returned source snippets when available, and cite the relevant URLs as markdown links. +Use the web_search tool to discover current information on the web. The required queries array accepts 1–4 non-empty search queries; use a one-item array for a single search. It returns an optional answer plus a list of source URLs. Use the returned source snippets when available, and cite the relevant URLs as markdown links. ``` ##### Web 抓取指引 @@ -69,40 +70,54 @@ Use the web_fetch tool to retrieve the content of a specific HTTP(S) URL (for ex #### Token 影响 -每个通过配置启用的工具都会为每次请求增加固定的指引 token 开销,即使限制隐藏了其 schema。切换抓取状态不仅会注册或移除抓取区段,也会更改搜索指引。 +每个通过配置启用的工具都会为每次请求增加固定的指引 token 开销,即使限制隐藏了其 schema。切换抓取状态或更改 `searchMaxQueries` 会改变搜索指引;切换抓取状态还会注册或移除抓取区段。 #### KV Cache 影响 -只要启用工具、scope 与指引文本不变,前缀就保持稳定。配置启用状态(包括因切换抓取状态而改变搜索指引分支)或插件生命周期可能使从第一个变化的提示词区段起的复用失效;scope schema 限制不会移除该区段。 +只要启用工具、scope 与指引文本不变,前缀就保持稳定。配置启用状态(包括因切换抓取状态而改变搜索指引分支)、更改 `searchMaxQueries` 或插件生命周期可能使从第一个变化的提示词区段起的复用失效;scope schema 限制不会移除该区段。 ### 工具 schema #### 模型看到的内容 -模型会看到生成的 [`web_search` 与 `web_fetch` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-web)。结果数量与超时预算属于部署设置,不是模型参数。 +模型会看到生成的 [`web_search` 与 `web_fetch` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-web)。结果数量与超时预算属于部署设置,不是模型参数。 #### Token 影响 -每次请求都会产生固定的 schema token 开销;通过配置禁用会同时移除 schema 与指引,scope 限制只移除 schema。 +对于已解析的 `searchMaxQueries`,每次请求都会产生固定的 schema token 开销;通过配置禁用会同时移除 schema 与指引,scope 限制只移除 schema。 #### KV Cache 影响 -只要定义与可见性不变,前缀就保持稳定。配置启用状态、插件生命周期或 scope 限制可能使从第一个变化的 schema token 起的复用失效。 +只要定义、已解析查询上限与可见性不变,前缀就保持稳定。配置启用状态、更改 `searchMaxQueries`、插件生命周期或 scope 限制可能使从第一个变化的 schema token 起的复用失效。 ### 搜索结果 #### 模型看到的内容 -可选的提供方答案之后是 `Sources:`,再跟随内容取决于数据且格式严格为 `- []()` 的行,并可添加后缀 ` — ()`。既无答案也无来源时,结果显示 `No results found.`。列表被截断至上限时会添加 `(Showing the first sources. Refine the query for more.)`;每个结果都以 `Cite the relevant URLs above as markdown links in your answer.` 结尾。 +可选的提供方答案之后是 `Sources:`,再跟随内容取决于数据且格式严格为 `- []()` 的行,并可添加后缀 ` — ()`。多查询调用会让每个完全相同的查询字符串只执行一次,并保留它首次出现的位置;调用会用来源查询作为 markdown 标题标注每个提供方答案,按 URL 对来源去重,并从每个查询取得同一排名的一条来源后再推进至下一排名。既无答案也无来源时,结果显示 `No results found.`。列表被截断至上限时会添加 `(Showing the first sources. Refine the query for more.)`;每个结果都以 `Cite the relevant URLs above as markdown links in your answer.` 结尾。 #### Token 影响 -数据相关结果会重复发送直到压缩(compaction),来源数量由 `searchMaxResults` 限制。 +数据相关结果会重复发送直到压缩(compaction);查询请求扇出由 `searchMaxQueries` 限制,来源数量由 `searchMaxResults` 限制。 #### KV Cache 影响 仅追加;新可见内容位于可复用请求前缀之后,不会使现有 KV Cache 条目失效。 +### 搜索失败 + +#### 模型看到的内容 + +多查询调用中的任何查询失败时,`web_search` 会中止其余搜索,等待所有已启动搜索结算,丢弃成功结果,并针对首次失败返回 `Error: `。 + +#### Token 影响 + +只有保留的错误结果会增加 token;被丢弃的成功结果不会进入模型历史。 + +#### KV Cache 影响 + +仅追加;错误位于可复用请求前缀之后,不会使现有 KV Cache 条目失效。 + ### 抓取结果 #### 模型看到的内容 @@ -121,7 +136,7 @@ Use the web_fetch tool to retrieve the content of a specific HTTP(S) URL (for ex #### 模型看到的内容 -空输入精确地变为 `Error: query must be a non-empty string` 或 `Error: url must be a non-empty string`。 +schema 校验会在执行前拒绝缺失或非数组的 `queries` 字段以及非字符串数组元素。值错误精确地变为 `Error: queries must contain at least one query`、配置上限为 1 时的 `Error: queries must contain at most 1 query`、上限更大时的 `Error: queries must contain at most queries`、`Error: each query must be a non-empty string` 或 `Error: url must be a non-empty string`。 #### Token 影响 @@ -133,6 +148,7 @@ Use the web_fetch tool to retrieve the content of a specific HTTP(S) URL (for ex ## 已知限制与暂缓事项 +- **没有覆盖整个批次的原生搜索计数器**:`searchMaxQueries` 限制 `ctx.web.search` 调用数,但提供方可以在每次调用内执行多次原生搜索。例如,配置了 `maxUses` 的模型型提供方最多可以执行 `searchMaxQueries × maxUses` 次原生搜索;`searchMaxResults` 只限制返回给调用方的组合来源。部署通过这些独立的消费方与提供方设置控制成本,因为通用 seam 不知道提供方内部的搜索计量单位。 - **HTML→markdown 转换会在 GFM 无法安全表示的输入上降级**:[turndown](https://github.com/mixmark-io/turndown)(带 GFM 表格/删除线)通过真实 DOM 转换至多 `fetchMaxOutputChars` 个源字符。保守的 512 层词法守卫会将深层或嵌套有歧义的主体作为原始 HTML 直接透传,转换异常也会如此处理;表格的 `colspan` 会被忽略,因为 GFM 无法表示跨列单元格。这些限制可避免阻塞事件循环,也避免不受信任的数值属性使输出膨胀([已归档的依赖决策](../../../.agents/notes/archived/simplification/2026-07-26-turndown-for-tool-web-html-markdown.md))。 -- **面向模型的接口有意保持精简,后续扩展暂缓**:`max_results` 保持为配置上限(不是模型参数),`web_fetch` 只接受 `url`(没有 `format`/`prompt`/LLM(大语言模型)摘要模式);两项都列为 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md) 中的后续步骤。 +- **面向模型的接口有意保持精简,后续扩展暂缓**:`max_results` 保持为配置上限(不是模型参数),`web_fetch` 只接受 `url`(没有 `format`/`prompt`/LLM(大语言模型)摘要模式);两项都列为 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md) 中的后续步骤。 - **没有 web 专用权限策略**:两个工具都不会请求 `ctx.approval` 就直接执行;需要确认的部署必须添加 `tools/pre-execute` 策略,该包不定义持久化的 URL/域名授权。 diff --git a/packages/web/tool-web/package.json b/packages/web/tool-web/package.json index ecc7717b2b..751ae9e376 100644 --- a/packages/web/tool-web/package.json +++ b/packages/web/tool-web/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-web", "description": "Model-facing web tools (web_search, web_fetch) over the DeepSeek Harness web capability seam (ctx.web)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/web/tool-web/src/index.ts b/packages/web/tool-web/src/index.ts index a0dcfaa09d..4145fdeb0f 100644 --- a/packages/web/tool-web/src/index.ts +++ b/packages/web/tool-web/src/index.ts @@ -9,10 +9,10 @@ import type { Context } from '@deepseek-ai/cordis' import z from '@deepseek-ai/schemastery' import type {} from '@deepseek-ai/dsh-web' -import { applyWebSearchTool, WEB_SEARCH_MAX_RESULTS } from './search.ts' +import { applyWebSearchTool, WEB_SEARCH_MAX_QUERIES, WEB_SEARCH_MAX_RESULTS } from './search.ts' import { applyWebFetchTool } from './fetch.ts' -export { WEB_SEARCH_MAX_RESULTS, applyWebSearchTool, formatSearchOutput, parseSearchArgs, presentSearchCall, presentSearchResult, searchMetaFromValue, searchMetaFromResult } from './search.ts' +export { WEB_SEARCH_MAX_QUERIES, WEB_SEARCH_MAX_RESULTS, applyWebSearchTool, formatSearchOutput, presentSearchCall, presentSearchResult, searchMetaFromValue, searchMetaFromResult } from './search.ts' export type { WebSearchMeta } from './search.ts' export { applyWebFetchTool, formatFetchOutput, parseFetchArgs, presentFetchCall, presentFetchResult, fetchMetaFromValue, fetchMetaFromResult } from './fetch.ts' export type { WebFetchMeta } from './fetch.ts' @@ -33,7 +33,7 @@ export const DEFAULT_WEB_TOOL_TIMEOUT_MS = 30_000 */ export const DEFAULT_FETCH_MAX_OUTPUT_CHARS = 200_000 -/** Plugin config: which web tools to register, the source cap, per-tool budgets, and the fetch output cap. */ +/** Plugin config: which web tools to register, search bounds, per-tool budgets, and the fetch output cap. */ export interface Config { /** Register `web_search`. Defaults to true. */ search?: boolean @@ -41,6 +41,8 @@ export interface Config { fetch?: boolean /** Upper bound on sources returned by one `web_search` call. */ searchMaxResults?: number + /** Upper bound on queries accepted by one `web_search` call. */ + searchMaxQueries?: number /** Cooperative timeout budget (ms) for `web_fetch`. Defaults to 30000. */ fetchTimeoutMs?: number /** Cooperative timeout budget (ms) for `web_search`. Defaults to 30000. */ @@ -53,6 +55,7 @@ export const Config: z = z.object({ search: z.boolean().default(true), fetch: z.boolean().default(true), searchMaxResults: z.number().default(WEB_SEARCH_MAX_RESULTS), + searchMaxQueries: z.number().default(WEB_SEARCH_MAX_QUERIES), fetchTimeoutMs: z.number().default(DEFAULT_WEB_TOOL_TIMEOUT_MS), searchTimeoutMs: z.number().default(DEFAULT_WEB_TOOL_TIMEOUT_MS), fetchMaxOutputChars: z.number().default(DEFAULT_FETCH_MAX_OUTPUT_CHARS), @@ -81,11 +84,12 @@ export function apply(ctx: Context, config: Config): void { // schemastery (Config) has already filled every defaulted field. const resolved = config as ResolvedConfig assertPositiveInteger('searchMaxResults', resolved.searchMaxResults) + assertPositiveInteger('searchMaxQueries', resolved.searchMaxQueries) assertPositiveInteger('fetchTimeoutMs', resolved.fetchTimeoutMs) assertPositiveInteger('searchTimeoutMs', resolved.searchTimeoutMs) assertPositiveInteger('fetchMaxOutputChars', resolved.fetchMaxOutputChars) if (resolved.search) { - applyWebSearchTool(ctx, resolved.searchMaxResults, resolved.searchTimeoutMs, resolved.fetch) + applyWebSearchTool(ctx, resolved.searchMaxResults, resolved.searchMaxQueries, resolved.searchTimeoutMs, resolved.fetch) } if (resolved.fetch) applyWebFetchTool(ctx, resolved.fetchTimeoutMs, resolved.fetchMaxOutputChars) } diff --git a/packages/web/tool-web/src/search.ts b/packages/web/tool-web/src/search.ts index d3c4acc756..f382582172 100644 --- a/packages/web/tool-web/src/search.ts +++ b/packages/web/tool-web/src/search.ts @@ -13,22 +13,40 @@ import type {} from '@deepseek-ai/dsh-system-prompt' /** * Default upper bound on returned sources (the `searchMaxResults` config). - * Owned by the consumer (not the provider or model), mirroring `dsh-tool-fs`'s - * `READ_LIMIT`. The model just asks a question; the product controls how much - * context returns. The default `8` aligns with OpenCode's Exa default. + * The consumer owns the returned-context limit; providers and models do not. */ export const WEB_SEARCH_MAX_RESULTS = 8 +/** Default upper bound on concurrent searches in one tool call. */ +export const WEB_SEARCH_MAX_QUERIES = 4 + +/** Model-facing `web_search` arguments. */ +interface WebSearchArgs { + queries: string[] +} + /** - * Validate value constraints the schema DSL can't express: a non-blank - * `query`. Throws a plain `Error` otherwise. + * Validate value constraints the schema DSL can't express: `queries` is + * non-empty, contains only non-blank strings, and fits the deployment's + * query-count bound. Exact duplicate strings are collapsed after the bound + * check. Throws a plain `Error` otherwise. * * @param args - the schema-validated `web_search` arguments. - * @returns the accepted arguments, passed through unchanged. + * @param maxQueries - the deployment's upper bound on queries in one call. + * @returns the accepted queries in their first-occurrence order. */ -export function parseSearchArgs(args: { query: string }): { query: string } { - if (args.query.trim().length === 0) throw new Error('query must be a non-empty string') - return { query: args.query } +export function parseSearchArgs( + args: WebSearchArgs, + maxQueries: number, +): string[] { + const queries = args.queries + if (queries.length === 0) throw new Error('queries must contain at least one query') + if (queries.length > maxQueries) { + const noun = maxQueries === 1 ? 'query' : 'queries' + throw new Error(`queries must contain at most ${maxQueries} ${noun}`) + } + if (queries.some(query => query.trim().length === 0)) throw new Error('each query must be a non-empty string') + return [...new Set(queries)] } /** Display label for a source: its title, else its hostname. */ @@ -75,13 +93,14 @@ export function formatSearchOutput(result: WebSearchResult): string { } /** - * Pending-call presentation: a search card titled by the query. + * Pending-call presentation: a search card titled by the query list. * - * @param args - the raw tool arguments; only `query` feeds the view. + * @param args - the raw tool arguments; only the query text feeds the view. * @returns the generic card view (`kind: 'search'`) shown while the call runs. */ -export function presentSearchCall(args: { query: string }): GenericCallView { - return { card: 'generic', title: args.query, kind: 'search', rawInput: args.query } +export function presentSearchCall(args: WebSearchArgs): GenericCallView { + const title = args.queries.join(', ') + return { card: 'generic', title, kind: 'search', rawInput: title } } /** @@ -95,7 +114,7 @@ export function presentSearchCall(args: { query: string }): GenericCallView { export interface WebSearchMeta { /** The faithful structured sources, in result order. */ sources: WebSource[] - /** True when the seam cut the source list to honor the result cap. */ + /** True when the seam or multi-query merge cut the source list to honor the result cap. */ truncated: boolean /** The provider-generated answer text, when any. */ answer?: string @@ -175,26 +194,103 @@ export function searchMetaFromResult(meta: unknown): WebSearchMeta | undefined { * `web` capability falls back to the raw `tool/result` content, which is the * same text (see the web-result-card Agent Note). * - * @param args - the raw tool arguments; `query` becomes the result-state title so - * a window-truncated replay that dropped the call head still has one. + * @param args - the raw tool arguments; the queries become the result-state + * title so a window-truncated replay that dropped the call head still has one. * @param result - the final model-facing tool result; `meta` carries the sources. * @returns the search result view, or `undefined` (generic card) on failure or * malformed meta. */ -export function presentSearchResult(args: { query: string }, result: ToolResult): WebSearchResultView | undefined { +export function presentSearchResult(args: WebSearchArgs, result: ToolResult): WebSearchResultView | undefined { if (result.isError) return undefined const meta = searchMetaFromResult(result.meta) if (meta === undefined) return undefined return { card: 'web', kind: 'search', - title: args.query, + title: args.queries.join(', '), sources: meta.sources, truncated: meta.truncated, ...meta.answer !== undefined ? { answer: meta.answer } : {}, } } +/** + * Run one or more searches through the web seam. A single query keeps the + * provider's exact result; multiple queries run concurrently and are merged + * into one normalized result capped at `maxResults`. A failed search aborts + * its siblings, and this function waits for every search to settle before + * rethrowing the first failure. + * + * @param ctx - context whose `web` service performs the searches. + * @param queries - validated non-empty queries. + * @param maxResults - the deployment's source cap for the combined result. + * @param signal - cancellation signal forwarded to every search. + * @returns the combined search result. + */ +async function runSearchQueries( + ctx: Context, + queries: string[], + maxResults: number, + signal: AbortSignal, +): Promise { + if (queries.length === 1) { + return ctx.web.search({ query: queries[0] as string, maxResults }, signal) + } + const controller = new AbortController() + const batchSignal = AbortSignal.any([signal, controller.signal]) + let firstFailure: { error: unknown } | undefined + const results: WebSearchResult[] = [] + const searches = queries.map(async (query, index) => { + try { + results[index] = await ctx.web.search({ query, maxResults }, batchSignal) + } catch (error) { + if (firstFailure === undefined) firstFailure = { error } + controller.abort(error) + throw error + } + }) + await Promise.allSettled(searches) + if (firstFailure !== undefined) throw firstFailure.error + return mergeSearchResults(queries, results, maxResults) +} + +/** Merge per-query results into one deduplicated, round-robin, capped result. */ +function mergeSearchResults( + queries: string[], + results: WebSearchResult[], + maxResults: number, +): WebSearchResult { + const seen = new Set() + const sources: WebSearchSource[] = [] + let sourceRanks = 0 + for (const result of results) { + sourceRanks = Math.max(sourceRanks, result.sources.length) + } + let droppedSource = false + merge: for (let rank = 0; rank < sourceRanks; rank++) { + for (const result of results) { + const source = result.sources[rank] + if (source !== undefined && !seen.has(source.url)) { + seen.add(source.url) + if (sources.length === maxResults) { + droppedSource = true + break merge + } + sources.push(source) + } + } + } + const contents = results.flatMap((result, index) => { + if (result.content === undefined || result.content.length === 0) return [] + return [`### ${queries[index]}\n\n${result.content}`] + }) + return { + ...contents.length > 0 ? { content: contents.join('\n\n') } : {}, + sources, + truncated: results.some(result => result.truncated) || droppedSource, + } +} + /** * Register the `web_search` tool and its system-prompt guidance. * @@ -202,6 +298,7 @@ export function presentSearchResult(args: { query: string }, result: ToolResult) * registrations; both are effect-scoped and unregister on plugin dispose. * @param maxResults - the deployment's source cap, sent as every seam * request's `maxResults`. + * @param maxQueries - the deployment's query cap enforced before provider calls. * @param timeoutMs - the cooperative tool-call budget (ms) attached as the tool's * `ToolDefinition.timeoutMs` for `@deepseek-ai/dsh-tool-call-timeout-policy` to enforce. * @param fetchEnabled - whether the same composition exposes `web_fetch`, which @@ -210,6 +307,7 @@ export function presentSearchResult(args: { query: string }, result: ToolResult) export function applyWebSearchTool( ctx: Context, maxResults: number, + maxQueries: number, timeoutMs: number, fetchEnabled: boolean, ): void { @@ -217,15 +315,20 @@ export function applyWebSearchTool( name: 'tool:web_search', order: 110, text: fetchEnabled - ? 'Use the web_search tool to discover current information on the web. It returns an optional answer plus a list of source URLs. Follow up with web_fetch when you need the full content of a specific result, and cite the relevant URLs as markdown links.' - : 'Use the web_search tool to discover current information on the web. It returns an optional answer plus a list of source URLs. Use the returned source snippets when available, and cite the relevant URLs as markdown links.', + ? `Use the web_search tool to discover current information on the web. The required queries array accepts 1–${maxQueries} non-empty search queries; use a one-item array for a single search. It returns an optional answer plus a list of source URLs. Follow up with web_fetch when you need the full content of a specific result, and cite the relevant URLs as markdown links.` + : `Use the web_search tool to discover current information on the web. The required queries array accepts 1–${maxQueries} non-empty search queries; use a one-item array for a single search. It returns an optional answer plus a list of source URLs. Use the returned source snippets when available, and cite the relevant URLs as markdown links.`, }) ctx.tools.register(defineTool({ name: 'web_search', - description: 'Search the web for current information. Returns an optional summary answer and a list of source URLs.', + description: `Search the web for current information. Provide 1–${maxQueries} queries in the required queries array. Returns an optional summary answer and a list of source URLs.`, parameters: { - query: { type: 'string', required: true, description: 'The search query.' }, + queries: { + type: 'array', + required: true, + items: { type: 'string' }, + description: `Required search queries; accepts 1–${maxQueries} items and merges their results.`, + }, }, output: { schema: { @@ -257,11 +360,8 @@ export function applyWebSearchTool( // Provider reads do not mutate parent-agent state. isConcurrencySafe: () => true, async execute(args, exec) { - const input = parseSearchArgs(args) - const result = await ctx.web.search( - { query: input.query, maxResults }, - exec.signal, - ) + const queries = parseSearchArgs(args, maxQueries) + const result = await runSearchQueries(ctx, queries, maxResults, exec.signal) return { ...result.content !== undefined ? { content: result.content } : {}, sources: result.sources.map(projectSource), diff --git a/packages/web/tool-web/tests/integration.spec.ts b/packages/web/tool-web/tests/integration.spec.ts index 1ee7cb359c..1aa1fa6416 100644 --- a/packages/web/tool-web/tests/integration.spec.ts +++ b/packages/web/tool-web/tests/integration.spec.ts @@ -97,7 +97,7 @@ describe('web_search integration over the real Exa provider', () => { JSON.stringify({ results: [{ url: 'https://result.test', title: 'Result', highlights: ['a highlight'] }] }), { status: 200, headers: { 'content-type': 'application/json' } }, ))) - const out = await call('web_search', { query: 'deepseek-official' }) + const out = await call('web_search', { queries: ['deepseek-official'] }) expect(out.isError).toBe(false) expect(out.content.map(b => b.type === 'text' ? b.text : '').join('')).toContain('[Result](https://result.test)') }) @@ -107,10 +107,11 @@ describe('tool-call timeout policy over the migrated web tools', () => { it('neither model schema exposes a timeout parameter after the migration', () => { const byName = new Map(ctx.tools.schemas().map(s => [s.name, s])) const fetchParams = byName.get('web_fetch')!.parameters as { properties: Record } - const searchParams = byName.get('web_search')!.parameters as { properties: Record } + const searchParams = byName.get('web_search')!.parameters as { properties: Record; required?: string[] } expect(Object.keys(fetchParams.properties)).toEqual(['url']) expect('timeout_ms' in fetchParams.properties).toBe(false) - expect(Object.keys(searchParams.properties)).toEqual(['query']) + expect(Object.keys(searchParams.properties)).toEqual(['queries']) + expect(searchParams.required).toEqual(['queries']) }) }) diff --git a/packages/web/tool-web/tests/tool-web.spec.ts b/packages/web/tool-web/tests/tool-web.spec.ts index 5cfd43ba0b..2adad8b79c 100644 --- a/packages/web/tool-web/tests/tool-web.spec.ts +++ b/packages/web/tool-web/tests/tool-web.spec.ts @@ -10,7 +10,6 @@ import * as ToolWeb from '@deepseek-ai/dsh-tool-web' import { formatSearchOutput, formatFetchOutput, - parseSearchArgs, parseFetchArgs, presentSearchCall, presentFetchCall, @@ -20,10 +19,12 @@ import { searchMetaFromResult, fetchMetaFromValue, fetchMetaFromResult, + WEB_SEARCH_MAX_QUERIES, WEB_SEARCH_MAX_RESULTS, } from '@deepseek-ai/dsh-tool-web' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { ToolResult } from '@deepseek-ai/dsh-tools' +import { parseSearchArgs } from '../src/search.ts' const testToolSignal = new AbortController().signal @@ -84,9 +85,14 @@ describe('search formatting', () => { expect(out).toContain('Showing the first 1 sources') }) - it('validates the query', () => { - expect(() => parseSearchArgs({ query: ' ' })).toThrow('non-empty') - expect(parseSearchArgs({ query: 'hi' })).toEqual({ query: 'hi' }) + it('validates queries', () => { + expect(parseSearchArgs({ queries: ['hi'] }, WEB_SEARCH_MAX_QUERIES)).toEqual(['hi']) + expect(parseSearchArgs({ queries: ['one', 'one', ' two '] }, WEB_SEARCH_MAX_QUERIES)) + .toEqual(['one', ' two ']) + expect(() => parseSearchArgs({ queries: [] }, WEB_SEARCH_MAX_QUERIES)).toThrow('at least one query') + expect(() => parseSearchArgs({ queries: ['one', 'two'] }, 1)).toThrow('at most 1 query') + expect(() => parseSearchArgs({ queries: ['one', 'two', 'three'] }, 2)).toThrow('at most 2 queries') + expect(() => parseSearchArgs({ queries: ['ok', ' '] }, WEB_SEARCH_MAX_QUERIES)).toThrow('each query must be a non-empty string') }) it('falls back to the raw URL as a source label when the URL is unparseable', () => { @@ -94,8 +100,8 @@ describe('search formatting', () => { expect(out).toContain('[not a url](not a url)') }) - it('presents a search call as a search-kind card titled by the query', () => { - expect(presentSearchCall({ query: 'find me' })).toEqual({ card: 'generic', title: 'find me', kind: 'search', rawInput: 'find me' }) + it('presents a search call with a joined query title', () => { + expect(presentSearchCall({ queries: ['one', 'two'] })).toEqual({ card: 'generic', title: 'one, two', kind: 'search', rawInput: 'one, two' }) }) }) @@ -145,7 +151,7 @@ describe('web_search presentation meta and result view', () => { content: 'an answer', truncated: true, sources: [{ url: 'https://a.test', title: 'A', snippet: 'snip', publishedAt: '2026-07-20' }], }) - expect(presentSearchResult({ query: 'q' }, toolResult(meta, 'rendered'))).toEqual({ + expect(presentSearchResult({ queries: ['q'] }, toolResult(meta, 'rendered'))).toEqual({ card: 'web', kind: 'search', title: 'q', @@ -157,7 +163,7 @@ describe('web_search presentation meta and result view', () => { it('omits the answer from the view when meta carries none', () => { const meta = searchMetaFromValue({ truncated: false, sources: [{ url: 'https://a.test' }] }) - const view = presentSearchResult({ query: 'q' }, toolResult(meta)) + const view = presentSearchResult({ queries: ['q'] }, toolResult(meta)) expect(view).toBeDefined() expect(view && 'answer' in view).toBe(false) expect(view && 'content' in view).toBe(false) @@ -165,11 +171,11 @@ describe('web_search presentation meta and result view', () => { it('falls back to the generic card on an error result', () => { const meta = searchMetaFromValue({ truncated: false, sources: [{ url: 'https://a.test' }] }) - expect(presentSearchResult({ query: 'q' }, toolResult(meta, 'body', true))).toBeUndefined() + expect(presentSearchResult({ queries: ['q'] }, toolResult(meta, 'body', true))).toBeUndefined() }) it('falls back to the generic card on absent or malformed meta', () => { - expect(presentSearchResult({ query: 'q' }, toolResult(undefined))).toBeUndefined() + expect(presentSearchResult({ queries: ['q'] }, toolResult(undefined))).toBeUndefined() expect(searchMetaFromResult(undefined)).toBeUndefined() expect(searchMetaFromResult(null)).toBeUndefined() expect(searchMetaFromResult('nope')).toBeUndefined() @@ -445,7 +451,7 @@ describe('tool-web registration', () => { const names = ctx.tools.schemas().map(s => s.name) expect(names).toContain('web_search') expect(names).toContain('web_fetch') - expect(ctx.tools.executionMode({ signal: testToolSignal, callId: CallId('search-safe'), name: 'web_search', arguments: { query: 'q' } })) + expect(ctx.tools.executionMode({ signal: testToolSignal, callId: CallId('search-safe'), name: 'web_search', arguments: { queries: ['q'] } })) .toEqual({ kind: 'parallel' }) expect(ctx.tools.executionMode({ signal: testToolSignal, callId: CallId('fetch-safe'), name: 'web_fetch', arguments: { url: 'https://a.test' } })) .toEqual({ kind: 'parallel' }) @@ -474,7 +480,7 @@ describe('tool-web registration', () => { expect(ctx.tools.schemas().map(s => s.name)).toContain('web_search') // No provider is registered: the schema stays visible and execution reports // the structured unavailability instead. - const out = await call('web_search', { query: 'q' }) + const out = await call('web_search', { queries: ['q'] }) expect(out.error?.info?.code).toBe('WEB_PROVIDER_UNAVAILABLE') await fiber.dispose() }) @@ -483,7 +489,7 @@ describe('tool-web registration', () => { const { fiber, ctx } = await mountTools() const prompt = await ctx.systemPrompt.assemble() const text = prompt.sections.map(s => s.text).join('\n') - expect(text).toContain('Use the web_search tool to discover current information on the web. It returns an optional answer plus a list of source URLs. Follow up with web_fetch when you need the full content of a specific result, and cite the relevant URLs as markdown links.') + expect(text).toContain(`Use the web_search tool to discover current information on the web. The required queries array accepts 1–${WEB_SEARCH_MAX_QUERIES} non-empty search queries; use a one-item array for a single search. It returns an optional answer plus a list of source URLs. Follow up with web_fetch when you need the full content of a specific result, and cite the relevant URLs as markdown links.`) expect(text).toContain('Use the web_fetch tool to retrieve the content of a specific HTTP(S) URL') await fiber.dispose() }) @@ -505,25 +511,156 @@ describe('tool-web execution through the real registry', () => { sources: [{ url: 'https://a.test', title: 'A', snippet: 'snip', publishedAt: '2026-07-20' }], } const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: searchProvider(result) }) - const out = await call('web_search', { query: 'q' }) + const out = await call('web_search', { queries: ['q'] }) expect(out.isError).toBe(false) expect(out.value).toEqual(result) expect(out.content.map(b => b.type === 'text' ? b.text : '').join('')).toContain('[A](https://a.test)') await fiber.dispose() }) + it('executes web_search with multiple queries concurrently and merges results', async () => { + const seen: string[] = [] + let releaseFirst: (() => void) | undefined + const firstResult = new Promise((resolve) => { + releaseFirst = () => { + resolve({ + content: 'answer one', truncated: false, + sources: [ + { url: 'https://a.test', title: 'A' }, + { url: 'https://shared.test' }, + ], + }) + } + }) + const provider: WebSearchProvider = { + id: 'stub-search', + available: () => available, + search: (request) => { + seen.push(request.query) + if (request.query === 'one') return firstResult + return Promise.resolve({ + content: 'answer two', truncated: false, + sources: [ + { url: 'https://b.test', title: 'B' }, + { url: 'https://shared.test' }, + ], + }) + }, + } + const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: provider }) + const pending = call('web_search', { queries: ['one', 'one', 'two'] }) + try { + await vi.waitFor(() => { expect(seen).toEqual(['one', 'two']) }) + } finally { + releaseFirst?.() + } + const out = await pending + expect(out.isError).toBe(false) + expect(out.value).toEqual({ + content: '### one\n\nanswer one\n\n### two\n\nanswer two', + sources: [ + { url: 'https://a.test', title: 'A' }, + { url: 'https://b.test', title: 'B' }, + { url: 'https://shared.test' }, + ], + truncated: false, + }) + const body = out.content.map(b => b.type === 'text' ? b.text : '').join('') + expect(body).toContain('### one') + expect(body).toContain('### two') + await fiber.dispose() + }) + + it('continues round-robin merging after a shorter result is exhausted', async () => { + const provider: WebSearchProvider = { + id: 'stub-search', + available: () => available, + search: request => Promise.resolve(request.query === 'one' + ? { content: '', sources: [{ url: 'https://a.test' }], truncated: false } + : { sources: [{ url: 'https://b.test' }, { url: 'https://c.test' }], truncated: false }), + } + const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: provider }) + const out = await call('web_search', { queries: ['one', 'two'] }) + expect(out.isError).toBe(false) + expect(out.value).toEqual({ + sources: [ + { url: 'https://a.test' }, + { url: 'https://b.test' }, + { url: 'https://c.test' }, + ], + truncated: false, + }) + await fiber.dispose() + }) + + it('aborts sibling searches and waits for them to settle before reporting a batch failure', async () => { + let siblingAborted = false + let releaseSibling: (() => void) | undefined + const provider: WebSearchProvider = { + id: 'stub-search', + available: () => available, + search: (request, signal) => { + if (request.query === 'one') return Promise.reject(new Error('first search failed')) + return new Promise((_resolve, reject) => { + releaseSibling = () => { reject(new Error('sibling search stopped')) } + signal?.addEventListener('abort', () => { + siblingAborted = true + }, { once: true }) + }) + }, + } + const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: provider }) + const pending = call('web_search', { queries: ['one', 'two'] }) + let callSettled = false + void pending.then(() => { callSettled = true }) + try { + await vi.waitFor(() => { expect(siblingAborted).toBe(true) }) + await Promise.resolve() + expect(callSettled).toBe(false) + } finally { + releaseSibling?.() + } + const out = await pending + expect(out.isError).toBe(true) + expect(out.content).toEqual([{ type: 'text', text: 'Error: first search failed' }]) + await fiber.dispose() + }) + + it('caps combined multi-query results to searchMaxResults', async () => { + const provider: WebSearchProvider = { + id: 'stub-search', + available: () => available, + search: request => Promise.resolve({ + sources: request.query === 'one' + ? [{ url: 'https://a.test' }, { url: 'https://b.test' }] + : [{ url: 'https://c.test' }, { url: 'https://d.test' }], + truncated: false, + }), + } + const { fiber, call } = await mountTools({ config: { searchMaxResults: 2 }, webConfig: { searchProvider: 'stub-search' }, search: provider }) + const out = await call('web_search', { queries: ['one', 'two'] }) + expect(out.isError).toBe(false) + expect(out.value).toEqual({ + sources: [{ url: 'https://a.test' }, { url: 'https://c.test' }], + truncated: true, + }) + const body = out.content.map(b => b.type === 'text' ? b.text : '').join('') + expect(body).toContain('Showing the first 2 sources.') + await fiber.dispose() + }) + it('projects the search sources into the tool result meta and derives its web/search view', async () => { const result: WebSearchResult = { content: 'answer', truncated: true, sources: [{ url: 'https://a.test', title: 'A', snippet: 'snip', publishedAt: '2026-07-20' }], } const { ctx, fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: searchProvider(result) }) - const out = await call('web_search', { query: 'q' }) + const out = await call('web_search', { queries: ['q'] }) expect(out.meta).toEqual({ answer: 'answer', truncated: true, sources: [{ url: 'https://a.test', title: 'A', snippet: 'snip', publishedAt: '2026-07-20' }], }) - const view = ctx.tools.get('web_search')?.presentResult?.({ query: 'q' }, { content: out.content, isError: out.isError, ...out.meta !== undefined ? { meta: out.meta } : {} }) + const view = ctx.tools.get('web_search')?.presentResult?.({ queries: ['q'] }, { content: out.content, isError: out.isError, ...out.meta !== undefined ? { meta: out.meta } : {} }) expect(view).toMatchObject({ card: 'web', kind: 'search', truncated: true, answer: 'answer' }) await fiber.dispose() }) @@ -546,7 +683,7 @@ describe('tool-web execution through the real registry', () => { it('surfaces a structured WebError when no provider is available', async () => { const { fiber, call } = await mountTools() - const out = await call('web_search', { query: 'q' }) + const out = await call('web_search', { queries: ['q'] }) expect(out.isError).toBe(true) expect(out.error?.info?.code).toBe('WEB_PROVIDER_UNAVAILABLE') await fiber.dispose() @@ -555,15 +692,15 @@ describe('tool-web execution through the real registry', () => { it('surfaces WEB_PROVIDER_AMBIGUOUS for multiple unconfigured providers', async () => { const { ctx, fiber, call } = await mountTools({ search: searchProvider({ sources: [], truncated: false }) }) ctx.web.registerSearchProvider({ id: 'other', available: () => available, search: () => Promise.resolve({ sources: [], truncated: false }) }) - const out = await call('web_search', { query: 'q' }) + const out = await call('web_search', { queries: ['q'] }) expect(out.isError).toBe(true) expect(out.error?.info?.code).toBe('WEB_PROVIDER_AMBIGUOUS') await fiber.dispose() }) - it('rejects invalid arguments with a structured INVALID_ARGS error', async () => { + it.each([{}, { queries: [123] }])('rejects absent or wrongly typed queries with a structured INVALID_ARGS error', async (args) => { const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: searchProvider({ sources: [], truncated: false }) }) - const out = await call('web_search', { query: 123 }) + const out = await call('web_search', args) expect(out.isError).toBe(true) expect(out.error?.info?.code).toBe('INVALID_ARGS') await fiber.dispose() @@ -635,10 +772,34 @@ describe('tool-web execution through the real registry', () => { } const { ctx, fiber } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: provider }) const controller = new AbortController() - await ctx.tools.execute({ callId: CallId('search-1'), name: 'web_search', arguments: { query: 'q' }, signal: controller.signal }) + await ctx.tools.execute({ callId: CallId('search-1'), name: 'web_search', arguments: { queries: ['q'] }, signal: controller.signal }) expect(seen.signal).toBe(controller.signal) await fiber.dispose() }) + + it('cascades caller cancellation to every multi-query search', async () => { + const signals: (AbortSignal | undefined)[] = [] + const provider: WebSearchProvider = { + id: 'stub-search', + available: () => available, + search: (_request, signal) => { + signals.push(signal) + return new Promise((_resolve, reject) => { + signal?.addEventListener('abort', () => { reject(new Error('search aborted')) }, { once: true }) + }) + }, + } + const { ctx, fiber } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: provider }) + const controller = new AbortController() + const pending = ctx.tools.execute({ callId: CallId('search-multi-1'), name: 'web_search', arguments: { queries: ['one', 'two'] }, signal: controller.signal }) + await vi.waitFor(() => { expect(signals).toHaveLength(2) }) + expect(signals[0]).toBe(signals[1]) + expect(signals[0]).not.toBe(controller.signal) + controller.abort(new Error('caller cancelled')) + await pending + expect(signals.every(signal => signal?.aborted === true)).toBe(true) + await fiber.dispose() + }) }) describe('searchMaxResults is plugin config', () => { @@ -650,7 +811,7 @@ describe('searchMaxResults is plugin config', () => { search: (request) => { seen.maxResults = request.maxResults; return Promise.resolve({ sources: [], truncated: false }) }, } const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: provider }) - await call('web_search', { query: 'q' }) + await call('web_search', { queries: ['q'] }) expect(seen.maxResults).toBe(WEB_SEARCH_MAX_RESULTS) await fiber.dispose() }) @@ -663,7 +824,7 @@ describe('searchMaxResults is plugin config', () => { search: () => Promise.resolve({ sources, truncated: false }), } const { fiber, call } = await mountTools({ config: { searchMaxResults: 2 }, webConfig: { searchProvider: 'stub-search' }, search: provider }) - const out = await call('web_search', { query: 'q' }) + const out = await call('web_search', { queries: ['q'] }) expect(out.isError).toBe(false) const body = out.content.map(b => b.type === 'text' ? b.text : '').join('') expect(body).toContain('https://s1.test') @@ -686,6 +847,43 @@ describe('searchMaxResults is plugin config', () => { }) }) +describe('searchMaxQueries is plugin config', () => { + it('exposes the configured cap to the model and enforces it before provider calls', async () => { + const seen: string[] = [] + const provider: WebSearchProvider = { + id: 'stub-search', + available: () => available, + search: (request) => { + seen.push(request.query) + return Promise.resolve({ sources: [], truncated: false }) + }, + } + const { fiber, ctx, call } = await mountTools({ + config: { searchMaxQueries: 2 }, + webConfig: { searchProvider: 'stub-search' }, + search: provider, + }) + const schema = ctx.tools.schemas().find(item => item.name === 'web_search') + expect(schema?.description).toContain('1–2 queries') + const prompt = await ctx.systemPrompt.assemble() + expect(prompt.sections.map(section => section.text).join('\n')).toContain('accepts 1–2 non-empty search queries') + const out = await call('web_search', { queries: ['one', 'two', 'three'] }) + expect(out.isError).toBe(true) + expect(out.content).toEqual([{ type: 'text', text: 'Error: queries must contain at most 2 queries' }]) + expect(seen).toEqual([]) + await fiber.dispose() + }) + + it.each([0, -1, 1.5])('rejects an invalid searchMaxQueries value %s at load', async (value) => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRuntime) + await ctx.plugin(WebRuntime, {}) + await expect(ctx.plugin(ToolWeb, { searchMaxQueries: value })) + .rejects.toThrow(/tool-web: searchMaxQueries must be a positive integer/) + }) +}) + describe('tool-call timeout budget is plugin config', () => { it('attaches the default 30s budget to web_fetch and web_search', async () => { const { fiber, ctx } = await mountTools() diff --git a/packages/web/web-fetch-http/README.i18n.yaml b/packages/web/web-fetch-http/README.i18n.yaml index 8ba306ea66..078606e11b 100644 --- a/packages/web/web-fetch-http/README.i18n.yaml +++ b/packages/web/web-fetch-http/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/web/web-fetch-http/README.md README.md: 5589a8e8605a64ae9ef5f6d9978a9b63331d5b0d -README.zh.md: e9cf98feb9065947af1321c87562a2ae9ac21315 +README.zh.md: b0dff1d992f9f84cc8b9b9747544ef5e6c0fc3eb diff --git a/packages/web/web-fetch-http/README.zh.md b/packages/web/web-fetch-http/README.zh.md index e9cf98feb9..b0dff1d992 100644 --- a/packages/web/web-fetch-http/README.zh.md +++ b/packages/web/web-fetch-http/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -一个匿名公共 HTTP(S) `WebFetchProvider`,用于 harness [web 能力 seam](../web/README.md)(`ctx.web`)。它获取具体 URL,返回状态码和长度受限的解码内容。 +一个匿名公共 HTTP(S) `WebFetchProvider`,用于 harness [web 能力 seam](../web/README.zh.md)(`ctx.web`)。它获取具体 URL,返回状态码和长度受限的解码内容。 这是一个**实现**包:它向 `ctx.web` 注册提供方,不拥有该键,也不注册面向模型的工具。它是函数/命名空间插件(`inject: ['web']`)。 @@ -10,7 +10,7 @@ 提供方拥有**安全资源获取**:URL 验证、HTTP 传输、重定向策略、资源兜底超时、中止传播、字节上限、charset 解码、内容类型分类与二进制拒绝。`@deepseek-ai/dsh-tool-web` 拥有**呈现**(HTML→markdown、截断格式)。非 2xx HTTP 响应是*结果*(状态码 + 解码主体),不是错误;`WebError` 只用于无法安全获取或表示资源的失败。 -提供方的 `timeoutMs` 是直接 `ctx.web.fetch()` 调用方和配置有误的部署所用的资源兜底,不是面向模型的工具调用预算。[`dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.md) 拥有 `web_fetch` 工具调用预算,并让 `exec.signal` 在超时时触发,以强制执行该预算。 +提供方的 `timeoutMs` 是直接 `ctx.web.fetch()` 调用方和配置有误的部署所用的资源兜底,不是面向模型的工具调用预算。[`dsh-tool-call-timeout-policy`](../../guard/timeout-policy/README.zh.md) 拥有 `web_fetch` 工具调用预算,并让 `exec.signal` 在超时时触发,以强制执行该预算。 已交付的 web 工具部署会把提供方兜底设为高于工具预算,因此模型调用通常返回 `TOOL_TIMEOUT`。如果外层截止期限先于提供方的兜底超时触发,提供方会报告 `WEB_ABORTED`,外层策略再将其替换为 `TOOL_TIMEOUT`。因此,`WEB_FETCH_TIMEOUT` 表明直接服务调用方的提供方预算已经耗尽。 @@ -38,7 +38,7 @@ ## 模型体验 -通过 [`dsh-tool-web`](../tool-web/README.md) 间接影响;该工具把此提供方经 `maxBodyChars` 限制的解码文本或由 HTML 转换得到的 markdown 置于抓取结果包装层中,并保留提供方失败;重定向、标头与传输机制保持隐藏。 +通过 [`dsh-tool-web`](../tool-web/README.zh.md) 间接影响;该工具把此提供方经 `maxBodyChars` 限制的解码文本或由 HTML 转换得到的 markdown 置于抓取结果包装层中,并保留提供方失败;重定向、标头与传输机制保持隐藏。 #### KV Cache 影响 @@ -46,6 +46,6 @@ ## 已知限制与暂缓事项 -- **SSRF/私有网络防护暂缓**:不会阻止私有、loopback、link-local、multicast 或其他非公开目标,也不进行 DNS 解析后验证或逐跳重新验证(见 [web 能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md))。在此功能落地前,该提供方是 SSRF 原语;能够访问敏感内部网络目标的部署**禁止启用它**。 +- **SSRF/私有网络防护暂缓**:不会阻止私有、loopback、link-local、multicast 或其他非公开目标,也不进行 DNS 解析后验证或逐跳重新验证(见 [web 能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md))。在此功能落地前,该提供方是 SSRF 原语;能够访问敏感内部网络目标的部署**禁止启用它**。 - **只解码文本内容**:包括 html/xhtml 与 `text/*` 加 JSON/XML 家族;缺少 `Content-Type` 或任何二进制类型都会抛出 `WEB_UNSUPPORTED_CONTENT_TYPE`,可提取文本的 PDF 解码属于明确的暂缓工作。 - **charset 只来自 `Content-Type` 标头**(默认为 UTF-8):HTML `` 声明会被忽略;声明但无法识别的 charset 标签会抛出异常,而非回退。 diff --git a/packages/web/web-fetch-http/package.json b/packages/web/web-fetch-http/package.json index 0089033964..3dfee71b40 100644 --- a/packages/web/web-fetch-http/package.json +++ b/packages/web/web-fetch-http/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-web-fetch-http", "description": "Anonymous public HTTP(S) fetch provider for the DeepSeek Harness web capability seam (ctx.web)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/web/web-fetch-http/src/index.ts b/packages/web/web-fetch-http/src/index.ts index 3f8ff13898..a3ce03c9b2 100644 --- a/packages/web/web-fetch-http/src/index.ts +++ b/packages/web/web-fetch-http/src/index.ts @@ -1,8 +1,6 @@ /** - * `@deepseek-ai/dsh-web-fetch-http`: registers an anonymous public HTTP(S) - * `WebFetchProvider` with `ctx.web`. A function/namespace plugin (NOT a - * default-export service): it registers INTO the seam's fetch registry, like the - * search providers register into the search registry. + * Anonymous public HTTP(S) `WebFetchProvider` plugin. It contributes to the + * `ctx.web` registry without owning the service. * * @module @deepseek-ai/dsh-web-fetch-http */ diff --git a/packages/web/web-search-deepseek/README.i18n.yaml b/packages/web/web-search-deepseek/README.i18n.yaml index 653d2b654f..f4eea9bf3d 100644 --- a/packages/web/web-search-deepseek/README.i18n.yaml +++ b/packages/web/web-search-deepseek/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/web/web-search-deepseek/README.md README.md: 25162a3e82a05bb17f31a69bc65f6552e42906ad -README.zh.md: a7a2566ecb36558c581dd801531bd394eb63dea3 +README.zh.md: 65b3c9175357b44ab07f84bdd10a4a250f55f43f diff --git a/packages/web/web-search-deepseek/README.zh.md b/packages/web/web-search-deepseek/README.zh.md index a7a2566ecb..65b3c91753 100644 --- a/packages/web/web-search-deepseek/README.zh.md +++ b/packages/web/web-search-deepseek/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -由 [DeepSeek](https://deepseek.com) 支持的 `WebSearchProvider`,用于 harness [web 能力 seam](../web/README.md)(`ctx.web`)。它调用 DeepSeek 的 **Anthropic 兼容 Messages API**(`POST {baseURL}/messages`),启用原生 `web_search_20250305` 服务器工具,并把 DeepSeek 返回的结构化 `web_search_tool_result` 块映射为 seam 规范化的 `WebSearchResult`。 +由 [DeepSeek](https://deepseek.com) 支持的 `WebSearchProvider`,用于 harness [web 能力 seam](../web/README.zh.md)(`ctx.web`)。它调用 DeepSeek 的 **Anthropic 兼容 Messages API**(`POST {baseURL}/messages`),启用原生 `web_search_20250305` 服务器工具,并把 DeepSeek 返回的结构化 `web_search_tool_result` 块映射为 seam 规范化的 `WebSearchResult`。 这是一个**实现**包:它向 `ctx.web` 注册提供方,通过可选的 `ctx.credentials` seam 为每次搜索解析凭据,若存在发起请求的 agent(智能体)会话,还会在其中记录该辅助请求,且不注册面向模型的工具。与 `@deepseek-ai/dsh-llm-deepseek` 一样,它是函数/命名空间插件(`inject: ['web']`)。Anthropic 协议格式(wire format)是提供方私有细节,并**不**使该提供方依赖 `ctx.llm`。 @@ -68,7 +68,7 @@ DeepSeek 返回的提供方生成答案均不被该提供方信任为 `content` #### 模型看到的内容 -通过 [`dsh-tool-web`](../tool-web/README.md),会话模型会看到结构化搜索块中去重后的 URL、标题、日期与引用 snippet;提供方文本不会作为答案受到信任。该提供方的具体错误消息包括带有处理指引的凭据缺失消息、`DeepSeek search credential resolution failed: `、`DeepSeek search aborted`、`DeepSeek search request failed: `、`DeepSeek returned no web_search_tool_result blocks; the request may not have triggered native web search` 和 `DeepSeek returned an unprocessable response body: `;HTTP 失败保留提供方消息。错误包装属于消费方。 +通过 [`dsh-tool-web`](../tool-web/README.zh.md),会话模型会看到结构化搜索块中去重后的 URL、标题、日期与引用 snippet;提供方文本不会作为答案受到信任。该提供方的具体错误消息包括带有处理指引的凭据缺失消息、`DeepSeek search credential resolution failed: `、`DeepSeek search aborted`、`DeepSeek search request failed: `、`DeepSeek returned no web_search_tool_result blocks; the request may not have triggered native web search` 和 `DeepSeek returned an unprocessable response body: `;HTTP 失败保留提供方消息。错误包装属于消费方。 #### Token 影响 diff --git a/packages/web/web-search-deepseek/package.json b/packages/web/web-search-deepseek/package.json index 55bd7ecdf2..dc1a46bf0f 100644 --- a/packages/web/web-search-deepseek/package.json +++ b/packages/web/web-search-deepseek/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-web-search-deepseek", "description": "DeepSeek-backed search provider (native web_search via the Anthropic-compatible API) for the DeepSeek Harness web capability seam (ctx.web)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/web/web-search-exa/README.i18n.yaml b/packages/web/web-search-exa/README.i18n.yaml index 4afbbcbc5d..d94873e683 100644 --- a/packages/web/web-search-exa/README.i18n.yaml +++ b/packages/web/web-search-exa/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/web/web-search-exa/README.md README.md: 5b1212c1fb210a3aca7836875cc295fad87e7437 -README.zh.md: fc9afbe44dcb5dd55c7007a11b35048217143351 +README.zh.md: 0d236cb1cb42ab3742d86872a2329060f8ceb09f diff --git a/packages/web/web-search-exa/README.zh.md b/packages/web/web-search-exa/README.zh.md index fc9afbe44d..0d236cb1cb 100644 --- a/packages/web/web-search-exa/README.zh.md +++ b/packages/web/web-search-exa/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -由 [Exa](https://exa.ai) 支持的 `WebSearchProvider`,用于 harness [web 能力 seam](../web/README.md)(`ctx.web`)。它调用 Exa 的 `POST /search` 端点并请求高亮摘要内容,把扁平 `results[]` 映射为 seam 规范化的 `WebSearchResult`。 +由 [Exa](https://exa.ai) 支持的 `WebSearchProvider`,用于 harness [web 能力 seam](../web/README.zh.md)(`ctx.web`)。它调用 Exa 的 `POST /search` 端点并请求高亮摘要内容,把扁平 `results[]` 映射为 seam 规范化的 `WebSearchResult`。 这是一个**实现**包:它向 `ctx.web` 注册提供方,不拥有 `ctx.web` 键,也不注册面向模型的工具(后者属于 `@deepseek-ai/dsh-tool-web`)。与 `@deepseek-ai/dsh-llm-deepseek` 一样,它是函数/命名空间插件(`inject: ['web']`),负责注册后端,而非默认导出服务。 @@ -29,7 +29,7 @@ Exa 返回扁平 `results[]`,不返回生成答案,因此省略 `content`。 ## 模型体验 -通过 [`dsh-tool-web`](../tool-web/README.md) 间接影响;该工具保留此提供方经 `maxResults` 限制的 URL、标题、首条 highlight 与发布日期,或将确切的错误消息 `Exa search aborted`、`Exa search request failed: ` 和 `Exa returned an unprocessable response body: ` 置于消费方的错误包装层内;生成答案与提供方私有字段不进入上下文。 +通过 [`dsh-tool-web`](../tool-web/README.zh.md) 间接影响;该工具保留此提供方经 `maxResults` 限制的 URL、标题、首条 highlight 与发布日期,或将确切的错误消息 `Exa search aborted`、`Exa search request failed: ` 和 `Exa returned an unprocessable response body: ` 置于消费方的错误包装层内;生成答案与提供方私有字段不进入上下文。 #### KV Cache 影响 @@ -38,5 +38,5 @@ Exa 返回扁平 `results[]`,不返回生成答案,因此省略 `content`。 ## 已知限制与暂缓事项 - **没有非空白高亮摘要的结果会被整个丢弃**:没有可映射的可移植 snippet,因此返回源可能少于请求数量。 -- **只公开 `searchType`/`numResults`/`highlightsPerResult`**:Exa 的其他控制项(livecrawl、category、域名/日期过滤条件、全文内容)等待提供方无关的 Service Definition 字段(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md))。 +- **只公开 `searchType`/`numResults`/`highlightsPerResult`**:Exa 的其他控制项(livecrawl、category、域名/日期过滤条件、全文内容)等待提供方无关的 Service Definition 字段(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md))。 - **按错误形状分类中止**:只有 `DOMException` 且名为 `AbortError` 时才映射为 `WEB_ABORTED`;携带自定义原因的中止(例如 `dsh-timeout` 的 `TimeoutReason`)会呈现为 `WEB_PROVIDER_ERROR`。 diff --git a/packages/web/web-search-exa/package.json b/packages/web/web-search-exa/package.json index ae0aa563cf..5ff018d22a 100644 --- a/packages/web/web-search-exa/package.json +++ b/packages/web/web-search-exa/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-web-search-exa", "description": "Exa-backed search provider for the DeepSeek Harness web capability seam (ctx.web)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/web/web-search-exa/src/index.ts b/packages/web/web-search-exa/src/index.ts index 34a257cd27..a0a75e54ee 100644 --- a/packages/web/web-search-exa/src/index.ts +++ b/packages/web/web-search-exa/src/index.ts @@ -1,9 +1,6 @@ /** - * `@deepseek-ai/dsh-web-search-exa`: registers an Exa-backed `WebSearchProvider` - * with `ctx.web`. A function/namespace plugin (NOT a default-export service): - * a search provider does not own the `ctx.web` key — it registers INTO the - * seam's provider registry, exactly as `@deepseek-ai/dsh-llm-deepseek` - * registers an adapter into `ctx.llm`. The key is owned by `@deepseek-ai/dsh-web`. + * Exa-backed `WebSearchProvider` plugin. It contributes to the `ctx.web` + * registry without owning the service. * * @module @deepseek-ai/dsh-web-search-exa */ diff --git a/packages/web/web-search-perplexity/README.i18n.yaml b/packages/web/web-search-perplexity/README.i18n.yaml index e0af4f5cbb..0d597f7966 100644 --- a/packages/web/web-search-perplexity/README.i18n.yaml +++ b/packages/web/web-search-perplexity/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/web/web-search-perplexity/README.md README.md: 8800609a05e99d3fa8a61e12b2360e633f0fd330 -README.zh.md: 488c1463d28b7d36c34625a3ddbab1aa3417d70c +README.zh.md: ddf2ef2ef767d7ac2e361e873cefade70ea78115 diff --git a/packages/web/web-search-perplexity/README.zh.md b/packages/web/web-search-perplexity/README.zh.md index 488c1463d2..ddf2ef2ef7 100644 --- a/packages/web/web-search-perplexity/README.zh.md +++ b/packages/web/web-search-perplexity/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -由 [Perplexity](https://perplexity.ai) 支持的 `WebSearchProvider`,用于 harness [web 能力 seam](../web/README.md)(`ctx.web`)。它调用 Perplexity 的 OpenAI 兼容 `POST /chat/completions` 端点,把生成答案与引用映射为 seam 规范化的 `WebSearchResult`。 +由 [Perplexity](https://perplexity.ai) 支持的 `WebSearchProvider`,用于 harness [web 能力 seam](../web/README.zh.md)(`ctx.web`)。它调用 Perplexity 的 OpenAI 兼容 `POST /chat/completions` 端点,把生成答案与引用映射为 seam 规范化的 `WebSearchResult`。 这是一个**实现**包:它向 `ctx.web` 注册提供方,不拥有该键,也不注册面向模型的工具。与 `@deepseek-ai/dsh-llm-deepseek` 一样,它是函数/命名空间插件(`inject: ['web']`)。OpenAI 兼容协议格式(wire format)是提供方私有细节,并**不**使该提供方依赖 `ctx.llm`。 @@ -47,7 +47,7 @@ #### 模型看到的内容 -通过 [`dsh-tool-web`](../tool-web/README.md),会话模型会看到生成答案及结构化结果元数据,或只含 URL 的引用。该提供方确切的错误消息为 `Perplexity search aborted`、`Perplexity search request failed: ` 和 `Perplexity returned an unprocessable response body: `;HTTP 失败保留提供方消息。错误包装层属于消费方。 +通过 [`dsh-tool-web`](../tool-web/README.zh.md),会话模型会看到生成答案及结构化结果元数据,或只含 URL 的引用。该提供方确切的错误消息为 `Perplexity search aborted`、`Perplexity search request failed: ` 和 `Perplexity returned an unprocessable response body: `;HTTP 失败保留提供方消息。错误包装层属于消费方。 #### Token 影响 @@ -61,5 +61,5 @@ - **引用回退源只含 URL**:Perplexity 省略结构化 `search_results[]` 时,源不含 `title`/`snippet`/`publishedAt`,因此工具只渲染纯主机名标签。 - **超量返回的来源仍会增加 token 消耗和延迟**:协议没有结果数量控制,`maxResults` 只能由 seam 在事后截断。 -- **只公开 `model`/`maxTokens`/`searchRecency`**:Perplexity 的其他搜索控制项(域名过滤条件、`web_search_options` 上下文大小、图片)有待提供方无关的 Service Definition 字段支持(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md))。 +- **只公开 `model`/`maxTokens`/`searchRecency`**:Perplexity 的其他搜索控制项(域名过滤条件、`web_search_options` 上下文大小、图片)有待提供方无关的 Service Definition 字段支持(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md))。 - **按错误形状分类中止**:只有 `DOMException` 且名为 `AbortError` 时才映射为 `WEB_ABORTED`;携带自定义原因的中止(例如 `dsh-timeout` 的 `TimeoutReason`)会呈现为 `WEB_PROVIDER_ERROR`。 diff --git a/packages/web/web-search-perplexity/package.json b/packages/web/web-search-perplexity/package.json index 7f4aa73079..467f54299f 100644 --- a/packages/web/web-search-perplexity/package.json +++ b/packages/web/web-search-perplexity/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-web-search-perplexity", "description": "Perplexity-backed search provider for the DeepSeek Harness web capability seam (ctx.web)", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/web/web-search-perplexity/src/index.ts b/packages/web/web-search-perplexity/src/index.ts index de20f83747..5584658093 100644 --- a/packages/web/web-search-perplexity/src/index.ts +++ b/packages/web/web-search-perplexity/src/index.ts @@ -1,8 +1,6 @@ /** - * `@deepseek-ai/dsh-web-search-perplexity`: registers a Perplexity-backed - * `WebSearchProvider` with `ctx.web`. A function/namespace plugin (NOT a - * default-export service): it registers INTO the seam's provider registry, like - * `@deepseek-ai/dsh-llm-deepseek` registers an adapter into `ctx.llm`. + * Perplexity-backed `WebSearchProvider` plugin. It contributes to the + * `ctx.web` registry without owning the service. * * @module @deepseek-ai/dsh-web-search-perplexity */ diff --git a/packages/web/web/README.i18n.yaml b/packages/web/web/README.i18n.yaml index 0b2f806c2a..0b8f351822 100644 --- a/packages/web/web/README.i18n.yaml +++ b/packages/web/web/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/web/web/README.md README.md: 8dfc7f032e25e40207bb3880777b814e67175a20 -README.zh.md: 3354037bfcd77ca2b1a07da710b565065baf543b +README.zh.md: 98922ad83fedb7ded07b2e180f8a6e4261032112 diff --git a/packages/web/web/README.zh.md b/packages/web/web/README.zh.md index 3354037bfc..98922ad83f 100644 --- a/packages/web/web/README.zh.md +++ b/packages/web/web/README.zh.md @@ -56,6 +56,6 @@ ## 已知限制与暂缓事项 - **没有观测接口**:没有提供方变更事件或能力状态查询;可用性只能通过执行 `search()`/`fetch()` 并按抛出的 `WebError` code 路由来观测,无提供方失败是通用的 `WEB_PROVIDER_UNAVAILABLE`,不会枚举逐提供方原因(见 [Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md))。 -- **`WebSearchRequest` 只携带 `query` + `maxResults`**:提供方无关的控制项(新近程度、域名过滤条件、区域提示、搜索深度)暂缓至 Exa 与 Perplexity 都能诚实支持时(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md))。 +- **`WebSearchRequest` 只携带 `query` + `maxResults`**:提供方无关的控制项(新近程度、域名过滤条件、区域提示、搜索深度)暂缓至 Exa 与 Perplexity 都能诚实支持时(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.zh.md))。 - **`WebFetchBody` 没有 `pdf` 分支**:可提取文本的 PDF 支持属于明确的暂缓工作;封闭联合会使新增该分支成为三个 web 包中由编译强制执行的变更。 - **提供方支持的页面提取不属于 `fetch()` 范围**:Firecrawl/Tavily 风格的 `web_extract` 能力暂缓,而不会扩展抓取操作。 diff --git a/packages/web/web/package.json b/packages/web/web/package.json index 16db0b092c..c6c3dd5ff3 100644 --- a/packages/web/web/package.json +++ b/packages/web/web/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-web", "description": "Abstract web access capability seam (ctx.web) for the DeepSeek Harness — search/fetch provider registry, registration-order-independent selection, request/result vocabulary, and the WebError taxonomy", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/web/web/src/types.ts b/packages/web/web/src/types.ts index 2c7fcb5fcd..9e94bac46b 100644 --- a/packages/web/web/src/types.ts +++ b/packages/web/web/src/types.ts @@ -8,9 +8,10 @@ import { HarnessError } from '@deepseek-ai/dsh-llm' /** - * What one search-capable backend can return. The model-facing argument is just - * a query; `maxResults` is a `dsh-tool-web`-layer bound passed through unchanged - * and enforced on the way back by the seam (see {@link WebSearchResult}). + * What one search-capable backend is asked to search. Each request carries one + * query; a consumer may issue several requests. `maxResults` is a + * `dsh-tool-web`-layer bound passed through unchanged and enforced on the way + * back by the seam (see {@link WebSearchResult}). */ export interface WebSearchRequest { readonly query: string diff --git a/packages/webhook/README.i18n.yaml b/packages/webhook/README.i18n.yaml new file mode 100644 index 0000000000..df1c8a304d --- /dev/null +++ b/packages/webhook/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/webhook/README.md +README.md: 0917934d280c1619b3b304a43222418b36b95b07 +README.zh.md: 76c2de19e0a3cf12a6d6c7f6260aeecd14201d43 diff --git a/packages/webhook/README.md b/packages/webhook/README.md new file mode 100644 index 0000000000..0917934d28 --- /dev/null +++ b/packages/webhook/README.md @@ -0,0 +1,12 @@ +# webhook/ — verified external events to DSH Sessions + +English | [中文](README.zh.md) + +The Webhook family receives authenticated provider events, runs trusted programmatic rules, and optionally creates ordinary root Sessions inside Web Workspaces. Dispatch is process-local and fire-and-forget: the family owns no delivery database, queue, retry, deduplication, or Agent-completion state. + +| Package | Role | ctx key | +|---|---|---| +| [`webhook/`](webhook/README.md) | Rule registry, callback lifecycle, and Workspace-backed Session creation | `ctx.webhookRuntime` | +| [`webhook-github/`](webhook-github/README.md) | Signed GitHub HTTP adapter | consumes `ctx.webhookRuntime` and `ctx.webServer` | + +Provider adapters authenticate and normalize deliveries. Rules own arbitrary conditions and external calls, then return `null` or one Session request. The [Webhook subsystem reference](../../docs/subsystems/webhook.md) owns the shared types and timing guarantees. diff --git a/packages/webhook/README.zh.md b/packages/webhook/README.zh.md new file mode 100644 index 0000000000..76c2de19e0 --- /dev/null +++ b/packages/webhook/README.zh.md @@ -0,0 +1,12 @@ +# webhook/ — 从已验证外部事件到 DSH Session + +[English](README.md) | 中文 + +Webhook 系列接收通过身份验证的提供方事件,运行受信任的程序化规则,并可选择在 Web Workspace 中创建普通根 Session。分发仅存在于进程内并采用 fire-and-forget:本系列不拥有交付数据库、队列、重试、去重或 Agent 完成状态。 + +| 包 | 角色 | ctx key | +|---|---|---| +| [`webhook/`](webhook/README.zh.md) | 规则注册表、回调生命周期与基于 Workspace 的 Session 创建 | `ctx.webhookRuntime` | +| [`webhook-github/`](webhook-github/README.zh.md) | 签名 GitHub HTTP 适配器 | 消费 `ctx.webhookRuntime` 与 `ctx.webServer` | + +提供方适配器负责验证身份并规范化交付。规则拥有任意条件和外部调用,随后返回 `null` 或一个 Session 请求。[Webhook 子系统参考](../../docs/subsystems/webhook.zh.md)拥有共享类型与时序保证。 diff --git a/packages/webhook/webhook-github/README.i18n.yaml b/packages/webhook/webhook-github/README.i18n.yaml new file mode 100644 index 0000000000..335673b8f0 --- /dev/null +++ b/packages/webhook/webhook-github/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/webhook/webhook-github/README.md +README.md: e79ff29f72eee5c9c48fbd96882a65f26ab88ca3 +README.zh.md: 97fb162da3633dd971ba832cc603dd8390bc97ee diff --git a/packages/webhook/webhook-github/README.md b/packages/webhook/webhook-github/README.md new file mode 100644 index 0000000000..e79ff29f72 --- /dev/null +++ b/packages/webhook/webhook-github/README.md @@ -0,0 +1,51 @@ +# @deepseek-ai/dsh-webhook-github + +English | [中文](README.zh.md) + +`dsh-webhook-github` registers one exact HTTP route on the injected `ctx.webServer`. It bounds and verifies GitHub's raw JSON body, projects a provider-neutral delivery, calls `ctx.webhookRuntime.dispatch()`, and returns `202` without waiting for rules or Sessions. + +## Configuration + +| Key | Meaning | +|---|---| +| `source` | Non-empty adapter instance carried to rules, such as `primary-github`. | +| `path` | Exact non-root pathname without trailing slash, query, or fragment. | +| `secretEnv` | Credential reference containing the GitHub webhook secret. | +| `maxBodyBytes` | Positive safe-integer ceiling for the untouched request body. | + +All fields are required. The secret reference is resolved for every request, so rotation affects the next delivery without reloading the plugin. + +## HTTP contract + +Only `POST application/json` is accepted. The adapter reads a bounded UTF-8 body, requires `X-Hub-Signature-256`, `X-GitHub-Delivery`, and `X-GitHub-Event`, resolves the secret, verifies HMAC before JSON parsing, and requires a top-level lossless-JSON object. It never logs the secret, signature, or payload. + +| Status | Meaning | +|---|---| +| `202` | Verified JSON was dispatched in memory. | +| `400` | Required header, UTF-8, JSON, or top-level object was invalid. | +| `401` | Signature was missing or invalid. | +| `405` | Method was not `POST`. | +| `413` | Declared or streamed body exceeded `maxBodyBytes`. | +| `415` | Media type was not `application/json`. | +| `503` | Credential or webhook runtime was unavailable. | + +`202` does not state that any rule matched or that a Session was created. GitHub event-specific field validation belongs to each rule; the adapter guarantees only authenticated generic JSON. + +## Dedicated listener composition + +The normal Web profile already owns `ctx.webServer`. Mount another `dsh-host-webserver` and this adapter inside a group that isolates only `webServer`; the adapter still inherits credentials and `webhookRuntime`. The [GitHub review example](../../../examples/web-github-review/README.md) uses `127.0.0.1:3081/github` behind a TLS reverse proxy while the UI remains on port 3080. + +## Model Experience + +Indirectly, through `dsh-webhook`: this adapter contributes no prompt or tool schema; a matching rule owns the Session request and model-visible text. + +#### KV Cache effect + +Independent. Authentication and HTTP dispatch do not touch a model request; any new Session prefix belongs to the consuming rule and runtime. + +## Known Limitations and Deferred Work + +- **No TLS** — the injected development WebServer is normally loopback-only behind a TLS reverse proxy or tunnel. +- **Generic payload validation only** — rules own validation of the GitHub event fields they consume. +- **No provider acknowledgement of downstream work** — `202` precedes arbitrary rule calls and Session creation. +- **No form encoding** — GitHub must send `application/json`; `application/x-www-form-urlencoded` is rejected. diff --git a/packages/webhook/webhook-github/README.zh.md b/packages/webhook/webhook-github/README.zh.md new file mode 100644 index 0000000000..97fb162da3 --- /dev/null +++ b/packages/webhook/webhook-github/README.zh.md @@ -0,0 +1,51 @@ +# @deepseek-ai/dsh-webhook-github + +[English](README.md) | 中文 + +`dsh-webhook-github` 会在注入的 `ctx.webServer` 上注册一条精确 HTTP 路由。它限制并验证 GitHub 原始 JSON body,投影提供方无关的交付,调用 `ctx.webhookRuntime.dispatch()`,并在不等待规则或 Session 的情况下返回 `202`。 + +## 配置 + +| Key | 含义 | +|---|---| +| `source` | 携带给规则的非空适配器实例,例如 `primary-github`。 | +| `path` | 不带尾随斜杠、查询或片段的精确非根路径。 | +| `secretEnv` | 包含 GitHub webhook 密钥的凭据引用。 | +| `maxBodyBytes` | 未改动请求 body 的正安全整数上限。 | + +所有字段均为必填。每次请求都会重新解析密钥引用,因此轮换会在下一次交付生效,而无需重新加载插件。 + +## HTTP 约定 + +只接受 `POST application/json`。适配器读取有界 UTF-8 body,要求 `X-Hub-Signature-256`、`X-GitHub-Delivery` 与 `X-GitHub-Event`,解析密钥,在 JSON 解析前验证 HMAC,并要求顶层是无损 JSON 对象。它绝不记录密钥、签名或 payload。 + +| 状态 | 含义 | +|---|---| +| `202` | 已验证 JSON 已在内存中分发。 | +| `400` | 必需 header、UTF-8、JSON 或顶层对象无效。 | +| `401` | 签名缺失或无效。 | +| `405` | 方法不是 `POST`。 | +| `413` | 声明或流式 body 超过 `maxBodyBytes`。 | +| `415` | media type 不是 `application/json`。 | +| `503` | 凭据或 webhook runtime 不可用。 | + +`202` 不表示任何规则已经匹配,也不表示已创建 Session。GitHub 事件特定字段的验证属于各规则;适配器只保证通过身份验证的通用 JSON。 + +## 专用监听器组合 + +普通 Web profile 已经拥有 `ctx.webServer`。把另一个 `dsh-host-webserver` 和此适配器挂载到仅隔离 `webServer` 的 group 内;适配器仍会继承凭据与 `webhookRuntime`。[GitHub 评审示例](../../../examples/web-github-review/README.zh.md)在 TLS 反向代理后使用 `127.0.0.1:3081/github`,而 UI 继续位于端口 3080。 + +## Model Experience + +通过 `dsh-webhook` 间接产生影响:此适配器不贡献提示词或工具 schema;匹配规则拥有 Session 请求与模型可见文本。 + +#### KV Cache effect + +相互独立。身份验证与 HTTP 分发不触碰模型请求;任何新 Session 前缀都属于消费它的规则与 runtime。 + +## Known Limitations and Deferred Work + +- **无 TLS** — 注入的开发 WebServer 通常只监听 loopback,并位于 TLS 反向代理或 tunnel 后。 +- **仅通用 payload 验证** — 规则负责验证自己消费的 GitHub 事件字段。 +- **不向提供方确认下游工作** — `202` 先于任意规则调用与 Session 创建。 +- **不支持表单编码** — GitHub 必须发送 `application/json`;`application/x-www-form-urlencoded` 会被拒绝。 diff --git a/packages/webhook/webhook-github/package.json b/packages/webhook/webhook-github/package.json new file mode 100644 index 0000000000..a430a96a2e --- /dev/null +++ b/packages/webhook/webhook-github/package.json @@ -0,0 +1,61 @@ +{ + "name": "@deepseek-ai/dsh-webhook-github", + "description": "Signed GitHub HTTP webhook adapter for the DeepSeek Harness webhook runtime", + "version": "0.1.1-rc.2", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/webhook/webhook-github" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./types": { + "types": "./lib/types/types.d.ts", + "default": "./lib/types/types.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.js", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "peerDependencies": { + "@deepseek-ai/cordis": "workspace:^", + "@deepseek-ai/dsh-credentials": "workspace:^", + "@deepseek-ai/dsh-host-webserver": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-webhook": "workspace:^" + }, + "dependencies": { + "@deepseek-ai/schemastery": "workspace:^", + "@octokit/webhooks": "^14.2.0" + }, + "devDependencies": { + "@deepseek-ai/cordis": "workspace:^", + "@deepseek-ai/cordis-plugin-include": "workspace:^", + "@deepseek-ai/cordis-plugin-loader": "workspace:^", + "@deepseek-ai/dsh-credentials": "workspace:^", + "@deepseek-ai/dsh-host-webserver": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-webhook": "workspace:^" + } +} diff --git a/packages/webhook/webhook-github/src/body.ts b/packages/webhook/webhook-github/src/body.ts new file mode 100644 index 0000000000..4661b5761c --- /dev/null +++ b/packages/webhook/webhook-github/src/body.ts @@ -0,0 +1,69 @@ +/** Bounded raw HTTP body intake for GitHub signature verification. */ + +import type { IncomingMessage } from 'node:http' + +/** HTTP refusal whose message is safe to return without request data. */ +export class WebhookHttpError extends Error { + override readonly name = 'WebhookHttpError' + + constructor( + readonly status: 400 | 401 | 405 | 413 | 415 | 503, + message: string, + ) { + super(message) + } +} + +/** Parse a decimal Content-Length or reject an ambiguous header. */ +function contentLength(request: IncomingMessage): number | undefined { + const value = request.headers['content-length'] + if (value === undefined) return undefined + if (!/^(0|[1-9]\d*)$/.test(value)) { + throw new WebhookHttpError(400, 'invalid Content-Length') + } + const length = Number(value) + if (!Number.isSafeInteger(length)) throw new WebhookHttpError(413, 'request body is too large') + return length +} + +/** + * Read one request body as exact, bounded UTF-8 text. + * @param request - incoming request before any parser consumes it. + * @param maxBodyBytes - positive byte ceiling. + * @returns the decoded body after EOF. + * @throws {WebhookHttpError} for invalid length, excessive bytes, invalid UTF-8, or an aborted stream. + */ +export async function readBoundedUtf8Body( + request: IncomingMessage, + maxBodyBytes: number, +): Promise { + const declared = contentLength(request) + if (declared !== undefined && declared > maxBodyBytes) { + request.resume() + throw new WebhookHttpError(413, 'request body is too large') + } + + const chunks: Buffer[] = [] + let size = 0 + try { + for await (const raw of request) { + const chunk = Buffer.isBuffer(raw) ? raw : Buffer.from(raw as string) + size += chunk.byteLength + if (size > maxBodyBytes) { + request.resume() + throw new WebhookHttpError(413, 'request body is too large') + } + chunks.push(chunk) + } + } catch (error: unknown) { + if (error instanceof WebhookHttpError) throw error + throw new WebhookHttpError(400, 'request body was aborted') + } + if (!request.complete) throw new WebhookHttpError(400, 'request body was aborted') + try { + return new TextDecoder('utf-8', { fatal: true }).decode(Buffer.concat(chunks, size)) + } catch { + // TextDecoder is the only statement in the try; GitHub JSON must be valid UTF-8. + throw new WebhookHttpError(400, 'request body is not valid UTF-8') + } +} diff --git a/packages/webhook/webhook-github/src/handler.ts b/packages/webhook/webhook-github/src/handler.ts new file mode 100644 index 0000000000..8f1bf30c73 --- /dev/null +++ b/packages/webhook/webhook-github/src/handler.ts @@ -0,0 +1,130 @@ +/** GitHub HTTP authentication, parsing, and fire-and-forget dispatch. */ + +import type { Context } from '@deepseek-ai/cordis' +import type { IncomingMessage, ServerResponse } from 'node:http' +import { Webhooks } from '@octokit/webhooks' +import type { CredentialRef } from '@deepseek-ai/dsh-credentials' +import { snapshotJsonValue } from '@deepseek-ai/dsh-session' +import { + WebhookDeliveryId, + WebhookSourceId, + type VerifiedWebhookDelivery, +} from '@deepseek-ai/dsh-webhook' +import type { WebRoute } from '@deepseek-ai/dsh-host-webserver' +import { readBoundedUtf8Body, WebhookHttpError } from './body.ts' +import type { GitHubJsonObject } from './types.ts' + +/** Handler values validated once at plugin load. */ +export interface GitHubWebhookHandlerConfig { + readonly source: string + readonly secretEnv: CredentialRef + readonly maxBodyBytes: number +} + +/** Require one unambiguous non-empty request header. */ +function requiredHeader(request: IncomingMessage, name: string): string { + const values = request.headersDistinct[name] + const value = values?.[0] + if (values?.length !== 1 || value === undefined || value.trim() === '') { + throw new WebhookHttpError(400, `missing ${name} header`) + } + return value +} + +/** Whether Content-Type names JSON with at most one UTF-8 charset parameter. */ +function isJsonContentType(value: string | undefined): boolean { + if (value === undefined) return false + const parts = value.split(';').map(part => part.trim()) + const [mediaType, parameter, ...extra] = parts + if (mediaType?.toLowerCase() !== 'application/json') return false + if (parameter === undefined) return true + return extra.length === 0 && /^charset=(?:utf-8|"utf-8")$/i.test(parameter) +} + +/** Send one empty or plain-text response exactly once. */ +function respond(response: ServerResponse, status: number, message?: string): void { + if (message === undefined) { + response.writeHead(status) + response.end() + return + } + response.writeHead(status, { 'content-type': 'text/plain; charset=utf-8' }) + response.end(message) +} + +/** Convert a parsed value into the adapter's generic signed-object guarantee. */ +function parsePayload(body: string): GitHubJsonObject { + let parsed: unknown + try { + parsed = JSON.parse(body) + } catch { + // JSON.parse is the only statement in the try; no other failure is normalized. + throw new WebhookHttpError(400, 'request body is not valid JSON') + } + if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) { + throw new WebhookHttpError(400, 'GitHub webhook payload must be a JSON object') + } + const snapshot = snapshotJsonValue(parsed) + if (snapshot === undefined) throw new WebhookHttpError(400, 'GitHub webhook payload is not lossless JSON') + return snapshot as GitHubJsonObject +} + +/** + * Create one exact-route GitHub handler. + * @param ctx - adapter context carrying credentials and webhook runtime. + * @param config - validated source, credential reference, and body ceiling. + * @returns an HTTP handler that answers after in-memory dispatch, never rule settlement. + */ +export function createGitHubWebhookHandler( + ctx: Context, + config: GitHubWebhookHandlerConfig, +): WebRoute['handler'] { + return async (request, response) => { + try { + if (request.method !== 'POST') { + response.setHeader('allow', 'POST') + throw new WebhookHttpError(405, 'method not allowed') + } + if (!isJsonContentType(request.headers['content-type'])) { + throw new WebhookHttpError(415, 'content type must be application/json') + } + const body = await readBoundedUtf8Body(request, config.maxBodyBytes) + const signature = requiredHeader(request, 'x-hub-signature-256') + const deliveryId = requiredHeader(request, 'x-github-delivery') + const eventName = requiredHeader(request, 'x-github-event') + const credential = await ctx.credentials.resolve(config.secretEnv) + if (credential === undefined || credential.value === '') { + throw new WebhookHttpError(503, 'GitHub webhook secret is unavailable') + } + let verified = false + try { + verified = await new Webhooks({ secret: credential.value }).verify(body, signature) + } catch { + // Octokit verification errors carry no response detail safe or useful to the sender. + } + if (!verified) throw new WebhookHttpError(401, 'invalid webhook signature') + const payload = parsePayload(body) + const delivery: VerifiedWebhookDelivery<'github'> = { + kind: 'github', + source: WebhookSourceId(config.source), + deliveryId: WebhookDeliveryId(deliveryId), + event: { name: eventName, payload }, + receivedAt: Date.now(), + } + try { + ctx.webhookRuntime.dispatch(delivery) + } catch { + ctx.logger.warn('webhook-github: dispatch unavailable') + throw new WebhookHttpError(503, 'webhook runtime is unavailable') + } + respond(response, 202) + } catch (error: unknown) { + if (error instanceof WebhookHttpError) { + respond(response, error.status, error.message) + return + } + ctx.logger.warn('webhook-github: request failed') + respond(response, 503, 'webhook ingress is unavailable') + } + } +} diff --git a/packages/webhook/webhook-github/src/index.ts b/packages/webhook/webhook-github/src/index.ts new file mode 100644 index 0000000000..fd11ce4af3 --- /dev/null +++ b/packages/webhook/webhook-github/src/index.ts @@ -0,0 +1,62 @@ +/** Signed GitHub HTTP adapter for the provider-neutral webhook runtime. */ + +import type { Context } from '@deepseek-ai/cordis' +import { credentialRef } from '@deepseek-ai/dsh-credentials' +import type {} from '@deepseek-ai/dsh-host-webserver' +import z from '@deepseek-ai/schemastery' +import { createGitHubWebhookHandler } from './handler.ts' + +export type * from './types.ts' + +/** Cordis function-plugin name. */ +export const name = 'webhook-github' +/** Host services required before the exact route can register. */ +export const inject = ['webServer', 'webhookRuntime', 'credentials'] + +/** Required GitHub ingress configuration. */ +export interface Config { + /** Adapter instance name carried to rules. */ + readonly source: string + /** Exact absolute route path. */ + readonly path: string + /** Credential reference containing the shared webhook secret. */ + readonly secretEnv: string + /** Positive raw body ceiling in bytes. */ + readonly maxBodyBytes: number +} + +export const Config: z = z.object({ + source: z.string().required(), + path: z.string().required(), + secretEnv: z.string().role('credential-ref').required(), + maxBodyBytes: z.number().step(1).min(1).max(Number.MAX_SAFE_INTEGER).required(), +}) + +/** Validate route and source facts that Schemastery cannot express. */ +function assertConfig(config: Config): void { + if (config.source.trim() !== config.source || config.source === '') { + throw new Error('webhook-github source must be a non-empty trimmed string') + } + if (!config.path.startsWith('/') || config.path === '/' || config.path.endsWith('/') + || config.path.includes('?') || config.path.includes('#')) { + throw new Error('webhook-github path must be an absolute non-root pathname without a trailing slash, query, or fragment') + } +} + +/** Register one signed GitHub endpoint on the injected WebServer. */ +export function apply(ctx: Context, config: Config): void { + assertConfig(config) + const route = { + kind: 'exact' as const, + path: config.path, + handler: createGitHubWebhookHandler(ctx, { + source: config.source, + secretEnv: credentialRef(config.secretEnv), + maxBodyBytes: config.maxBodyBytes, + }), + } + ctx.effect( + () => ctx.webServer.register(route), + `webhook-github: ${config.path}`, + ) +} diff --git a/packages/webhook/webhook-github/src/invariant.ts b/packages/webhook/webhook-github/src/invariant.ts new file mode 100644 index 0000000000..cd4e42bd98 --- /dev/null +++ b/packages/webhook/webhook-github/src/invariant.ts @@ -0,0 +1,25 @@ +/** Package-owned invariant companion for the GitHub webhook adapter. */ + +import type { Context } from '@deepseek-ai/cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-webhook-github' + +/** Cordis invariant-companion plugin name. */ +export const name = 'webhook-github-invariant' +/** Registry required before reserving this package's invariant ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: authentication and input validation occur at the exact + * HTTP operation; dsh-host-webserver owns route/disposer symmetry. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's explained empty invariant. + * @param ctx - Cordis context carrying the invariant registry. + * @returns the invariant registration disposer. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) diff --git a/packages/webhook/webhook-github/src/types.ts b/packages/webhook/webhook-github/src/types.ts new file mode 100644 index 0000000000..e5c6373e51 --- /dev/null +++ b/packages/webhook/webhook-github/src/types.ts @@ -0,0 +1,22 @@ +/** GitHub event values projected after signature verification. */ + +import type { JsonValue } from '@deepseek-ai/dsh-session' + +/** Signed GitHub JSON object. Event-specific field validation belongs to each rule. */ +export type GitHubJsonObject = { readonly [key: string]: JsonValue } + +/** Provider event supplied to `WebhookRule<'github'>`. */ +export interface GitHubWebhookEvent { + /** Raw `X-GitHub-Event` name such as `pull_request`. */ + readonly name: string + /** Signed JSON object exactly as parsed from the request body. */ + readonly payload: GitHubJsonObject +} + +declare module '@deepseek-ai/dsh-webhook' { + interface WebhookEventMap { + github: GitHubWebhookEvent + } +} + +export type { EmitterWebhookEvent, EmitterWebhookEventName } from '@octokit/webhooks' diff --git a/packages/webhook/webhook-github/tests/body.spec.ts b/packages/webhook/webhook-github/tests/body.spec.ts new file mode 100644 index 0000000000..8f24d18618 --- /dev/null +++ b/packages/webhook/webhook-github/tests/body.spec.ts @@ -0,0 +1,57 @@ +import type { IncomingMessage } from 'node:http' +import { describe, expect, it, vi } from 'vitest' +import { readBoundedUtf8Body } from '../src/body.ts' + +/** Minimal async-iterable request for byte-level branches Node fetch cannot construct. */ +function request(options: { + chunks?: Array + contentLength?: string + complete?: boolean + error?: unknown +} = {}): IncomingMessage & { resume: ReturnType } { + const resume = vi.fn() + return { + headers: { + ...(options.contentLength === undefined ? {} : { 'content-length': options.contentLength }), + }, + complete: options.complete ?? true, + resume, + async * [Symbol.asyncIterator]() { + for (const chunk of options.chunks ?? []) yield chunk + if (options.error !== undefined) throw options.error + }, + } as unknown as IncomingMessage & { resume: ReturnType } +} + +describe('bounded webhook body intake', () => { + it('accepts an absent length and both Buffer and string chunks', async () => { + await expect(readBoundedUtf8Body(request({ chunks: [Buffer.from('{'), '}'] }), 2)).resolves.toBe('{}') + }) + + it('rejects malformed, unsafe, and oversized declared lengths', async () => { + await expect(readBoundedUtf8Body(request({ contentLength: '01' }), 10)).rejects.toMatchObject({ status: 400 }) + await expect(readBoundedUtf8Body(request({ contentLength: '999999999999999999999' }), Number.MAX_SAFE_INTEGER)) + .rejects.toMatchObject({ status: 413 }) + const oversized = request({ contentLength: '3' }) + await expect(readBoundedUtf8Body(oversized, 2)).rejects.toMatchObject({ status: 413 }) + expect(oversized.resume).toHaveBeenCalledOnce() + }) + + it('rejects a chunked body at the first byte beyond the cap', async () => { + const streamed = request({ chunks: [Buffer.from('ab'), Buffer.from('c')] }) + await expect(readBoundedUtf8Body(streamed, 2)).rejects.toMatchObject({ status: 413 }) + expect(streamed.resume).toHaveBeenCalledOnce() + }) + + it('normalizes stream failure and incomplete EOF as an aborted body', async () => { + await expect(readBoundedUtf8Body(request({ error: new Error('socket') }), 10)) + .rejects.toMatchObject({ status: 400, message: 'request body was aborted' }) + await expect(readBoundedUtf8Body(request({ complete: false }), 10)) + .rejects.toMatchObject({ status: 400, message: 'request body was aborted' }) + }) + + it('rejects invalid UTF-8 after a complete bounded read', async () => { + await expect(readBoundedUtf8Body(request({ chunks: [Buffer.from([0xff])] }), 1)) + .rejects.toMatchObject({ status: 400, message: 'request body is not valid UTF-8' }) + }) +}) diff --git a/packages/webhook/webhook-github/tests/config.spec.ts b/packages/webhook/webhook-github/tests/config.spec.ts new file mode 100644 index 0000000000..8ae0d726af --- /dev/null +++ b/packages/webhook/webhook-github/tests/config.spec.ts @@ -0,0 +1,53 @@ +import { Context } from '@deepseek-ai/cordis' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { apply, type Config } from '../src/index.ts' + +const contexts: Context[] = [] + +afterEach(async () => { + await Promise.allSettled(contexts.splice(0).map(ctx => ctx.fiber.dispose())) +}) + +/** Context with only the services direct apply reads. */ +function harness(): { ctx: Context; register: ReturnType; remove: ReturnType } { + const ctx = new Context() + contexts.push(ctx) + const remove = vi.fn() + const register = vi.fn(() => remove) + ctx.provide('webServer', { register } as never) + ctx.provide('webhookRuntime', {} as never) + ctx.provide('credentials', {} as never) + return { ctx, register, remove } +} + +const valid = { + source: 'primary', + path: '/github', + secretEnv: 'DSH_GITHUB_WEBHOOK_SECRET', + maxBodyBytes: 1024, +} satisfies Config + +describe('GitHub webhook plugin config', () => { + it('registers one exact route and removes it with the plugin fiber', async () => { + const test = harness() + apply(test.ctx, valid) + expect(test.register).toHaveBeenCalledWith(expect.objectContaining({ kind: 'exact', path: '/github' })) + await test.ctx.fiber.dispose() + expect(test.remove).toHaveBeenCalledOnce() + }) + + it.each([ + [{ ...valid, source: '' }, /source/], + [{ ...valid, source: ' primary' }, /source/], + [{ ...valid, path: 'github' }, /path/], + [{ ...valid, path: '/' }, /path/], + [{ ...valid, path: '/github/' }, /path/], + [{ ...valid, path: '/github?q=1' }, /path/], + [{ ...valid, path: '/github#x' }, /path/], + [{ ...valid, secretEnv: 'not valid' }, /credential ref/], + ] as const)('rejects invalid config %# before route registration', (config, message) => { + const test = harness() + expect(() => { apply(test.ctx, config) }).toThrow(message) + expect(test.register).not.toHaveBeenCalled() + }) +}) diff --git a/packages/webhook/webhook-github/tests/handler.spec.ts b/packages/webhook/webhook-github/tests/handler.spec.ts new file mode 100644 index 0000000000..25e6dda2f5 --- /dev/null +++ b/packages/webhook/webhook-github/tests/handler.spec.ts @@ -0,0 +1,258 @@ +import { createHmac } from 'node:crypto' +import { createServer, request as httpRequest, type IncomingMessage, type Server, type ServerResponse } from 'node:http' +import type { AddressInfo } from 'node:net' +import type { Context } from '@deepseek-ai/cordis' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { credentialRef } from '@deepseek-ai/dsh-credentials' +import { createGitHubWebhookHandler } from '../src/handler.ts' + +const servers: Server[] = [] + +afterEach(async () => { + await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(() => { resolve() })))) +}) + +/** One mutable fake for credential rotation and dispatch observation. */ +function fakeContext(secret = 'fixture-secret'): { + ctx: Context + dispatch: ReturnType + setSecret(value: string | undefined): void + warnings: ReturnType +} { + let current = secret as string | undefined + const dispatch = vi.fn() + const warnings = vi.fn() + return { + ctx: { + credentials: { + resolve: async () => current === undefined ? undefined : { value: current, source: 'environment' }, + }, + webhookRuntime: { dispatch }, + logger: { warn: warnings }, + } as unknown as Context, + dispatch, + setSecret(value) { current = value }, + warnings, + } +} + +/** Start a real Node server around the package-owned route handler. */ +async function serve(ctx: Context, maxBodyBytes = 1024): Promise { + const handler = createGitHubWebhookHandler(ctx, { + source: 'primary', + secretEnv: credentialRef('DSH_GITHUB_WEBHOOK_SECRET'), + maxBodyBytes, + }) + const server = createServer((request, response) => { void handler(request, response) }) + servers.push(server) + await new Promise(resolve => server.listen(0, '127.0.0.1', resolve)) + const port = (server.address() as AddressInfo).port + return `http://127.0.0.1:${String(port)}` +} + +/** HMAC header for one exact UTF-8 body. */ +function signature(secret: string, body: string): string { + return `sha256=${createHmac('sha256', secret).update(body).digest('hex')}` +} + +/** Send one GitHub-shaped request. */ +async function post( + base: string, + body: string, + options: { + secret?: string + signature?: string + event?: string + delivery?: string + contentType?: string + method?: string + } = {}, +): Promise { + const secret = options.secret ?? 'fixture-secret' + return await fetch(base, { + method: options.method ?? 'POST', + headers: { + 'content-type': options.contentType ?? 'application/json', + 'x-hub-signature-256': options.signature ?? signature(secret, body), + 'x-github-event': options.event ?? 'pull_request', + 'x-github-delivery': options.delivery ?? 'delivery-1', + }, + ...(options.method === 'GET' ? {} : { body }), + }) +} + +/** Send body chunks without Content-Length through a real Node client socket. */ +async function postChunked( + base: string, + chunks: readonly string[], + endDelayMs = 0, +): Promise<{ body: string; status: number }> { + return await new Promise((resolve, reject) => { + const request = httpRequest(base, { + method: 'POST', + headers: { + connection: 'close', + 'content-type': 'application/json', + 'transfer-encoding': 'chunked', + 'x-hub-signature-256': 'sha256=unused', + 'x-github-event': 'pull_request', + 'x-github-delivery': 'chunked-delivery', + }, + }, (response) => { + let body = '' + response.setEncoding('utf8') + response.on('data', (chunk: string) => { body += chunk }) + response.on('end', () => { resolve({ body, status: response.statusCode ?? 0 }) }) + }) + request.once('error', reject) + request.once('socket', (socket) => { socket.setNoDelay(true) }) + for (const chunk of chunks) request.write(chunk) + if (endDelayMs === 0) request.end() + else setTimeout(() => { request.end() }, endDelayMs) + }) +} + +describe('GitHub webhook HTTP handler', () => { + it('verifies, projects, dispatches, and answers 202', async () => { + const fake = fakeContext() + const base = await serve(fake.ctx) + const body = JSON.stringify({ action: 'ready_for_review', number: 1 }) + const response = await post(base, body, { contentType: 'application/json; charset=utf-8' }) + expect(response.status).toBe(202) + expect(await response.text()).toBe('') + expect(fake.dispatch).toHaveBeenCalledOnce() + const dispatched: unknown = fake.dispatch.mock.calls[0]?.[0] + expect(dispatched).toMatchObject({ + kind: 'github', + source: 'primary', + deliveryId: 'delivery-1', + event: { name: 'pull_request', payload: { action: 'ready_for_review', number: 1 } }, + }) + expect(typeof (dispatched as { receivedAt?: unknown }).receivedAt).toBe('number') + }) + + it('resolves the secret for each request so rotation takes effect immediately', async () => { + const fake = fakeContext('first') + const base = await serve(fake.ctx) + const body = JSON.stringify({ ping: true }) + expect((await post(base, body, { secret: 'first', delivery: 'first' })).status).toBe(202) + fake.setSecret('second') + expect((await post(base, body, { secret: 'first', delivery: 'stale' })).status).toBe(401) + expect((await post(base, body, { secret: 'second', delivery: 'second' })).status).toBe(202) + expect(fake.dispatch).toHaveBeenCalledTimes(2) + }) + + it.each([ + ['method', { method: 'GET' }, 405], + ['content type', { contentType: 'text/plain' }, 415], + ['content type parameter', { contentType: 'application/json; boundary=x' }, 415], + ['content type parameters', { contentType: 'application/json; charset=utf-8; boundary=x' }, 415], + ['signature', { signature: 'sha256=bad' }, 401], + ['event header', { event: '' }, 400], + ['delivery header', { delivery: '' }, 400], + ] as const)('rejects an invalid %s before dispatch', async (_label, options, status) => { + const fake = fakeContext() + const base = await serve(fake.ctx) + const response = await post(base, '{}', options) + expect(response.status).toBe(status) + if (status === 405) expect(response.headers.get('allow')).toBe('POST') + expect(fake.dispatch).not.toHaveBeenCalled() + }) + + it('rejects a missing Content-Type before body processing', async () => { + const fake = fakeContext() + const handler = createGitHubWebhookHandler(fake.ctx, { + source: 'primary', + secretEnv: credentialRef('DSH_GITHUB_WEBHOOK_SECRET'), + maxBodyBytes: 1024, + }) + const request = { method: 'POST', headers: {}, headersDistinct: {} } as unknown as IncomingMessage + const writeHead = vi.fn() + const response = { setHeader: vi.fn(), writeHead, end: vi.fn() } as unknown as ServerResponse + await handler(request, response) + expect(writeHead).toHaveBeenCalledWith(415, expect.any(Object)) + expect(fake.dispatch).not.toHaveBeenCalled() + }) + + it('rejects duplicate required headers', async () => { + const fake = fakeContext() + const handler = createGitHubWebhookHandler(fake.ctx, { + source: 'primary', + secretEnv: credentialRef('DSH_GITHUB_WEBHOOK_SECRET'), + maxBodyBytes: 1024, + }) + const request = { + method: 'POST', + headers: { 'content-type': 'application/json' }, + headersDistinct: { + 'x-hub-signature-256': ['sha256=unused'], + 'x-github-delivery': ['delivery-1'], + 'x-github-event': ['pull_request', 'ping'], + }, + complete: true, + async * [Symbol.asyncIterator]() { yield Buffer.from('{}') }, + } as unknown as IncomingMessage + const writeHead = vi.fn() + const response = { setHeader: vi.fn(), writeHead, end: vi.fn() } as unknown as ServerResponse + await handler(request, response) + expect(writeHead).toHaveBeenCalledWith(400, expect.any(Object)) + expect(fake.dispatch).not.toHaveBeenCalled() + }) + + it.each([ + ['not JSON', '{', 400], + ['array', '[]', 400], + ['non-lossless number', '{"value":1e400}', 400], + ] as const)('rejects a signed %s body', async (_label, body, status) => { + const fake = fakeContext() + const base = await serve(fake.ctx) + const response = await post(base, body) + expect(response.status).toBe(status) + expect(fake.dispatch).not.toHaveBeenCalled() + }) + + it('rejects a declared body over the configured cap', async () => { + const fake = fakeContext() + const base = await serve(fake.ctx, 2) + const response = await post(base, '{} ') + expect(response.status).toBe(413) + expect(fake.dispatch).not.toHaveBeenCalled() + }) + + it('answers 413 for a chunked body over the cap without resetting the connection', async () => { + const fake = fakeContext() + const base = await serve(fake.ctx, 2) + + await expect(postChunked(base, ['abc'], 50)).resolves.toEqual({ + body: 'request body is too large', + status: 413, + }) + expect(fake.dispatch).not.toHaveBeenCalled() + }) + + it('answers 503 when the credential or runtime is unavailable', async () => { + const missing = fakeContext() + missing.setSecret(undefined) + const missingBase = await serve(missing.ctx) + expect((await post(missingBase, '{}')).status).toBe(503) + + const closing = fakeContext() + closing.dispatch.mockImplementation(() => { throw new Error('closing') }) + const closingBase = await serve(closing.ctx) + expect((await post(closingBase, '{}')).status).toBe(503) + expect(closing.warnings).toHaveBeenCalledTimes(1) + }) + + it('does not leak the signed payload or secret in an infrastructure diagnostic', async () => { + const fake = fakeContext('super-secret') + ;(fake.ctx.credentials.resolve as ReturnType | undefined) = vi.fn(async () => { + throw new Error('credential store unavailable') + }) as never + const base = await serve(fake.ctx) + const body = JSON.stringify({ private: 'payload-secret' }) + expect((await post(base, body, { secret: 'super-secret' })).status).toBe(503) + const diagnostics = JSON.stringify(fake.warnings.mock.calls) + expect(diagnostics).not.toContain('super-secret') + expect(diagnostics).not.toContain('payload-secret') + }) +}) diff --git a/packages/webhook/webhook-github/tests/invariant.spec.ts b/packages/webhook/webhook-github/tests/invariant.spec.ts new file mode 100644 index 0000000000..dece3e9db7 --- /dev/null +++ b/packages/webhook/webhook-github/tests/invariant.spec.ts @@ -0,0 +1,13 @@ +import { Context } from '@deepseek-ai/cordis' +import InvariantRegistry from '@deepseek-ai/dsh-invariants' +import { describe, expect, it } from 'vitest' +import * as GitHubInvariant from '../src/invariant.ts' + +describe('GitHub webhook invariant companion', () => { + it('registers its explained empty installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantRegistry) + await expect(ctx.plugin(GitHubInvariant)).resolves.toBeDefined() + await ctx.fiber.dispose() + }) +}) diff --git a/packages/webhook/webhook-github/tests/loader-composition.spec.ts b/packages/webhook/webhook-github/tests/loader-composition.spec.ts new file mode 100644 index 0000000000..5316e9e35a --- /dev/null +++ b/packages/webhook/webhook-github/tests/loader-composition.spec.ts @@ -0,0 +1,90 @@ +import { createHmac } from 'node:crypto' +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { pathToFileURL } from 'node:url' +import { Context } from '@deepseek-ai/cordis' +import Include from '@deepseek-ai/cordis-plugin-include' +import Loader from '@deepseek-ai/cordis-plugin-loader' +import WebServer from '@deepseek-ai/dsh-host-webserver' +import { afterEach, describe, expect, it, vi } from 'vitest' +import * as GitHubAdapter from '../src/index.ts' + +let root: string | undefined +let context: Context | undefined + +afterEach(async () => { + await context?.fiber.dispose() + context = undefined + if (root !== undefined) await rm(root, { recursive: true, force: true }) + root = undefined +}) + +describe('real Loader composition', () => { + it('registers on a real WebServer and dispatches a signed request', { timeout: 60_000 }, async () => { + root = await mkdtemp(join(tmpdir(), 'dsh-webhook-github-loader-')) + const configPath = join(root, 'cordis.yml') + await writeFile(configPath, [ + '- name: fixture-dependencies', + "- name: '@deepseek-ai/dsh-host-webserver'", + ' config:', + " host: '127.0.0.1'", + ' port: 0', + "- name: '@deepseek-ai/dsh-webhook-github'", + ' config:', + ' source: loader', + ' path: /github', + ' secretEnv: DSH_GITHUB_WEBHOOK_SECRET', + ' maxBodyBytes: 1024', + '', + ].join('\n')) + + const dispatch = vi.fn() + const dependencies = { + name: 'fixture-dependencies', + apply(ctx: Context) { + ctx.provide('webhookRuntime', { dispatch } as never) + ctx.provide('credentials', { + resolve: async () => ({ value: 'loader-secret', source: 'environment' }), + } as never) + }, + } + context = new Context() + context.baseUrl = pathToFileURL(root).href + '/' + await context.plugin(Loader) + context.loader.builtins.include = Include + const modules = new Map([ + ['fixture-dependencies', dependencies], + ['@deepseek-ai/dsh-host-webserver', WebServer], + ['@deepseek-ai/dsh-webhook-github', GitHubAdapter], + ]) + context.loader.internal = { + version: 'v2', + async import(specifier: string) { + if (!modules.has(specifier)) throw new Error(`unexpected Loader import: ${specifier}`) + return modules.get(specifier) + }, + } as unknown as NonNullable + await context.loader.create({ + name: 'cordis:include', + config: { path: pathToFileURL(configPath).href }, + }) + await context.loader.await() + expect([...context.loader.entries()].filter(entry => entry.fiber === undefined && !entry.disabled)).toEqual([]) + + const body = JSON.stringify({ action: 'ready_for_review' }) + const signature = `sha256=${createHmac('sha256', 'loader-secret').update(body).digest('hex')}` + const response = await fetch(`http://127.0.0.1:${String(context.webServer.port)}/github`, { + method: 'POST', + headers: { + 'content-type': 'application/json', + 'x-hub-signature-256': signature, + 'x-github-event': 'pull_request', + 'x-github-delivery': 'loader-delivery', + }, + body, + }) + expect(response.status).toBe(202) + expect(dispatch).toHaveBeenCalledOnce() + }) +}) diff --git a/packages/webhook/webhook-github/tsconfig.json b/packages/webhook/webhook-github/tsconfig.json new file mode 100644 index 0000000000..88d2fe7ac4 --- /dev/null +++ b/packages/webhook/webhook-github/tsconfig.json @@ -0,0 +1,39 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/loader" + }, + { + "path": "../../../vendor/include" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../credentials/credentials" + }, + { + "path": "../../core/session" + }, + { + "path": "../../host/webserver" + }, + { + "path": "../webhook" + }, + { + "path": "../../runtime-diagnostics/invariants" + } + ] +} diff --git a/packages/webhook/webhook/README.i18n.yaml b/packages/webhook/webhook/README.i18n.yaml new file mode 100644 index 0000000000..5f6dca50de --- /dev/null +++ b/packages/webhook/webhook/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/webhook/webhook/README.md +README.md: 79337c7edb708f6862e0dc9b8ce98ebb491daf34 +README.zh.md: 90555a71eea8485766e2d96baccff07303daacf0 diff --git a/packages/webhook/webhook/README.md b/packages/webhook/webhook/README.md new file mode 100644 index 0000000000..79337c7edb --- /dev/null +++ b/packages/webhook/webhook/README.md @@ -0,0 +1,51 @@ +# @deepseek-ai/dsh-webhook + +English | [中文](README.zh.md) + +`dsh-webhook` provides the Host `ctx.webhookRuntime`: a registry for trusted programmatic webhook rules plus the one built-in action, creating an ordinary root Session inside a Web Workspace. The interface stays at `register(rule)` and `dispatch(delivery)`; provider authentication belongs to adapter packages. + +## Rule interface + +`WebhookRule` has a branded unique `id`, a provider `kind`, and `run(delivery, signal)`. A callback may execute arbitrary trusted code and returns either `null` or one `WebhookSessionRequest`. Rules of the same kind start independently, and one throw or rejection is logged without starving siblings. + +`VerifiedWebhookDelivery` carries provider kind, configured source id, provider delivery id, normalized lossless JSON, and receipt time. The runtime snapshots and freezes the complete value before sharing it. `deliveryId` is provenance only; repeated delivery runs the rules again. + +Registration is an effect. Its awaitable disposer first hides the rule, then aborts and drains active callbacks. Callbacks must observe the supplied signal; same-process code that ignores cancellation cannot be forcibly stopped safely. + +## Session request + +`WebhookSessionRequest` requires `workspacePath`, `title`, `prompt`, `agentPreset`, and `permissionPreset`; optional `model` names an explicit provider/model route plus an output-token cap. An explicit route uses its adapter's reasoning default. Omission snapshots the complete current deployment selection, including reasoning effort, until the first request records its durable header; later Web model changes retain the ordinary session behavior. + +The runtime validates presets before mutation, resolves or creates the canonical Workspace, creates an Agent with that Workspace path as `SessionHeader.cwd`, mounts the agent preset before publication, and attaches the Session before applying permissions, title, and prompt. Failed attachment disposes the unpublished action. A later pre-prompt failure detaches the Workspace and disposes the Agent on a best-effort rollback. + +Successful `Agent.followup()` is the webhook operation's commit point. The message uses `source.kind: "webhook"` with provider, source, delivery, and rule provenance. The runtime does not wait for idle, flush specially, inspect the reply, or publish completion state; ordinary Agent and Session behavior owns everything afterward. + +## Composition + +Load the runtime on the Web Host plane after Agents, model defaults, agent presets, permission presets, titles, and the Workspace registry. User-authored rule plugins inject `webhookRuntime` and yield the disposer returned by `register()` through their own effect. + +The runnable [GitHub review example](../../../examples/web-github-review/README.md) shows a rule module, dedicated ingress port, secret setup, and Workspace routing. + +## Model Experience + +### Rule-authored initial prompt + +#### What the model sees + +For each matching rule, the model sees exactly the non-empty text returned as `WebhookSessionRequest.prompt`. The generic runtime adds no private framing; a rule incorporating external text owns its trust labeling. The shipped GitHub example labels selected PR fields as untrusted JSON metadata. + +#### Token effect + +One data-dependent user-role message is retained in the new Session and contributes tokens until ordinary compaction replaces or removes that history. + +#### KV Cache effect + +The initial prompt begins a new Session, so it establishes rather than invalidates that Session's reusable request prefix. + +## Known Limitations and Deferred Work + +- **Process-local fire-and-forget only** — a crash loses rule calls that have not admitted a prompt; there is no queue, replay, or retry. +- **No built-in deduplication** — repeated provider deliveries may create repeated Sessions; rules that need idempotency own it. +- **No completion result** — HTTP acceptance and rule settlement do not report Agent success, idle, or output. +- **Trusted callbacks must cooperate with cancellation** — runtime teardown aborts and awaits them but cannot terminate arbitrary same-process code. +- **Workspace creation may outlive a failed Session attempt** — an empty Workspace is retained because another concurrent caller may already use it. diff --git a/packages/webhook/webhook/README.zh.md b/packages/webhook/webhook/README.zh.md new file mode 100644 index 0000000000..90555a71ee --- /dev/null +++ b/packages/webhook/webhook/README.zh.md @@ -0,0 +1,51 @@ +# @deepseek-ai/dsh-webhook + +[English](README.md) | 中文 + +`dsh-webhook` 提供 Host 侧的 `ctx.webhookRuntime`:它既是受信任程序化 webhook 规则的注册表,也拥有唯一内置动作——在 Web Workspace 中创建普通根 Session。接口只包含 `register(rule)` 和 `dispatch(delivery)`;提供方身份验证属于适配器包。 + +## 规则接口 + +`WebhookRule` 具有带品牌类型的唯一 `id`、提供方 `kind` 与 `run(delivery, signal)`。回调可以执行任意受信任代码,并返回 `null` 或一个 `WebhookSessionRequest`。同类规则彼此独立启动;某个规则抛出或拒绝只会记录日志,不会阻止同级规则。 + +`VerifiedWebhookDelivery` 携带提供方种类、已配置来源 id、提供方交付 id、规范化的无损 JSON 与接收时间。runtime 会在共享前快照并冻结完整值。`deliveryId` 仅是来源信息;重复交付会再次运行规则。 + +注册是一项 effect。它的可等待 disposer 会先隐藏规则,再中止并排空活动回调。回调必须观察所提供的 signal;忽略取消的同进程代码无法被安全强制停止。 + +## Session 请求 + +`WebhookSessionRequest` 要求 `workspacePath`、`title`、`prompt`、`agentPreset` 与 `permissionPreset`;可选 `model` 会指定明确的提供方/模型路由与输出 token 上限。明确路由使用其适配器的默认推理强度。省略时会快照包含推理强度的完整当前部署选择,直到首个请求记录持久 header;之后的 Web 模型变更保留普通 Session 行为。 + +runtime 会在变更状态前验证 preset,解析或创建规范 Workspace,以该 Workspace 路径作为 `SessionHeader.cwd` 创建 Agent,在发布前挂载 agent preset,并在应用权限、标题与提示词前附加 Session。附加失败会释放尚未提交动作的 Agent。之后若在提示词前失败,则以尽力而为方式脱离 Workspace 并释放 Agent。 + +成功的 `Agent.followup()` 是 webhook 操作的提交点。消息使用 `source.kind: "webhook"`,并携带提供方、来源、交付与规则来源信息。runtime 不等待 idle、不执行特殊 flush、不检查回复,也不发布完成状态;之后完全由普通 Agent 与 Session 行为接管。 + +## 组合 + +在 Web Host plane 上,于 Agents、模型默认值、agent presets、permission presets、标题与 Workspace 注册表之后加载 runtime。用户编写的规则插件注入 `webhookRuntime`,并通过自己的 effect 交出 `register()` 返回的 disposer。 + +可运行的 [GitHub 评审示例](../../../examples/web-github-review/README.zh.md)展示了规则模块、专用入口端口、密钥设置与 Workspace 路由。 + +## Model Experience + +### 规则编写的初始提示词 + +#### What the model sees + +每个匹配规则都会让模型看到 `WebhookSessionRequest.prompt` 返回的非空文本原文。通用 runtime 不增加私有框架;若规则包含外部文本,则由规则负责标明其信任属性。随附 GitHub 示例会把选定 PR 字段标为不受信任的 JSON 元数据。 + +#### Token effect + +一条依赖数据的 user-role 消息保留在新 Session 中,并持续贡献 token,直到普通 compaction 替换或移除该历史。 + +#### KV Cache effect + +初始提示词开启一个新 Session,因此它建立而不是使该 Session 的可复用请求前缀失效。 + +## Known Limitations and Deferred Work + +- **仅限进程内 fire-and-forget** — 崩溃会丢失尚未接纳提示词的规则调用;不存在队列、重放或重试。 +- **无内置去重** — 提供方重复交付可能创建重复 Session;需要幂等性的规则自行负责。 +- **无完成结果** — HTTP 接受与规则结算都不报告 Agent 成功、idle 或输出。 +- **受信任回调必须配合取消** — runtime teardown 会中止并等待回调,但无法终止任意同进程代码。 +- **Workspace 创建可能比失败的 Session 尝试更长寿** — 空 Workspace 会保留,因为另一个并发调用者可能已经使用它。 diff --git a/packages/webhook/webhook/package.json b/packages/webhook/webhook/package.json new file mode 100644 index 0000000000..c37e5f5356 --- /dev/null +++ b/packages/webhook/webhook/package.json @@ -0,0 +1,67 @@ +{ + "name": "@deepseek-ai/dsh-webhook", + "description": "Fire-and-forget webhook rule runtime that creates Workspace-backed DeepSeek Harness Sessions", + "version": "0.1.1-rc.2", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/webhook/webhook" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./types": { + "types": "./lib/types/types.d.ts", + "default": "./lib/types/types.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.js", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "peerDependencies": { + "@deepseek-ai/cordis": "workspace:^", + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-agent-default-model": "workspace:^", + "@deepseek-ai/dsh-agent-presets": "workspace:^", + "@deepseek-ai/dsh-brand": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-permission-presets": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-session-title": "workspace:^", + "@deepseek-ai/dsh-workspace": "workspace:^" + }, + "devDependencies": { + "@deepseek-ai/cordis": "workspace:^", + "@deepseek-ai/cordis-plugin-include": "workspace:^", + "@deepseek-ai/cordis-plugin-loader": "workspace:^", + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-agent-default-model": "workspace:^", + "@deepseek-ai/dsh-agent-presets": "workspace:^", + "@deepseek-ai/dsh-brand": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-permission-presets": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-session-title": "workspace:^", + "@deepseek-ai/dsh-workspace": "workspace:^" + } +} diff --git a/packages/webhook/webhook/src/brand.ts b/packages/webhook/webhook/src/brand.ts new file mode 100644 index 0000000000..695177232f --- /dev/null +++ b/packages/webhook/webhook/src/brand.ts @@ -0,0 +1,39 @@ +/** Opaque webhook identities shared by adapters, rules, and Session provenance. */ + +import type { Branded } from '@deepseek-ai/dsh-brand' + +/** Identifies one programmatic webhook rule. */ +export type WebhookRuleId = Branded<'WebhookRuleId'> + +/** Identifies one configured webhook adapter instance. */ +export type WebhookSourceId = Branded<'WebhookSourceId'> + +/** Identifies one provider delivery. The runtime assigns no deduplication semantics. */ +export type WebhookDeliveryId = Branded<'WebhookDeliveryId'> + +/** + * Brand a webhook rule id. + * @param value - non-empty rule identifier validated at registration. + * @returns the same string with its compile-time brand. + */ +export function WebhookRuleId(value: string): WebhookRuleId { + return value as WebhookRuleId +} + +/** + * Brand a configured webhook source id. + * @param value - non-empty adapter instance identifier validated by its adapter. + * @returns the same string with its compile-time brand. + */ +export function WebhookSourceId(value: string): WebhookSourceId { + return value as WebhookSourceId +} + +/** + * Brand a provider delivery id. + * @param value - non-empty provider identity validated by its adapter. + * @returns the same string with its compile-time brand. + */ +export function WebhookDeliveryId(value: string): WebhookDeliveryId { + return value as WebhookDeliveryId +} diff --git a/packages/webhook/webhook/src/index.ts b/packages/webhook/webhook/src/index.ts new file mode 100644 index 0000000000..96a35d95aa --- /dev/null +++ b/packages/webhook/webhook/src/index.ts @@ -0,0 +1,178 @@ +/** Fire-and-forget webhook rule registry and Workspace-backed Session runtime. */ + +import { Context, Service } from '@deepseek-ai/cordis' +import { deepFreeze, errorChain } from '@deepseek-ai/dsh-llm' +import { snapshotJsonValue } from '@deepseek-ai/dsh-session' +import type { WebhookRuleId } from './brand.ts' +import { createWebhookSession } from './session.ts' +import type { VerifiedWebhookDelivery, WebhookRule, WebhookSessionRequest } from './types.ts' + +export * from './brand.ts' +export type * from './types.ts' + +declare module '@deepseek-ai/cordis' { + interface Context { + webhookRuntime: WebhookRuntime + } +} + +/** Internal type erasure after public generic registration validates the provider kind. */ +interface AnyWebhookRule { + readonly id: WebhookRuleId + readonly kind: string + run( + delivery: Readonly, + signal: AbortSignal, + ): WebhookSessionRequest | null | Promise +} + +/** One effect-owned rule registration and the invocations that currently use it. */ +interface RuleRegistration { + readonly rule: AnyWebhookRule + readonly controller: AbortController + readonly active: Set> + closing: boolean + disposal?: Promise +} + +/** Validate and detach one delivery before sharing it across arbitrary rules. */ +function snapshotDelivery(delivery: VerifiedWebhookDelivery): VerifiedWebhookDelivery { + if (typeof delivery.kind !== 'string' || delivery.kind.trim() === '') { + throw new TypeError('webhook delivery kind must be a non-empty string') + } + if (typeof delivery.source !== 'string' || delivery.source.trim() === '') { + throw new TypeError('webhook delivery source must be a non-empty string') + } + if (typeof delivery.deliveryId !== 'string' || delivery.deliveryId.trim() === '') { + throw new TypeError('webhook delivery id must be a non-empty string') + } + if (!Number.isSafeInteger(delivery.receivedAt) || delivery.receivedAt < 0) { + throw new TypeError('webhook delivery receivedAt must be a non-negative safe integer') + } + const snapshot = snapshotJsonValue(delivery) + if (snapshot === undefined) throw new TypeError('webhook delivery must be lossless JSON') + return deepFreeze(snapshot) +} + +/** Fire-and-forget rule runtime. Session creation is the only built-in action. */ +export class WebhookRuntime extends Service { + static inject = [ + 'agents', + 'agentDefaultModel', + 'agentPresets', + 'permissionPresets', + 'sessionTitle', + 'workspaceRegistry', + ] + + private readonly rules = new Map() + private readonly selfCtx: Context + private closing = false + + constructor(ctx: Context) { + super(ctx, 'webhookRuntime') + this.selfCtx = ctx + ctx.effect(() => async () => { + this.closing = true + /* v8 ignore next -- caller-owned registration effects normally dispose first; this covers provider-first unload. */ + await Promise.all( + [...this.rules.values()].map(rule => this.disposeRegistration(rule)), + ) + }, 'webhookRuntime.lifecycle()') + } + + /** + * Register one trusted programmatic rule. + * @param rule - unique id, provider kind, and arbitrary callback. + * @returns awaitable effect disposer that aborts and drains this rule's active callbacks. + */ + register(rule: WebhookRule): () => Promise { + if (this.closing) throw new Error('webhook runtime is closing') + if (typeof rule.id !== 'string' || rule.id.trim() === '') { + throw new TypeError('webhook rule id must be a non-empty string') + } + if (typeof rule.kind !== 'string' || rule.kind.trim() === '') { + throw new TypeError(`webhook rule "${String(rule.id)}" kind must be a non-empty string`) + } + if (typeof rule.run !== 'function') { + throw new TypeError(`webhook rule "${String(rule.id)}" requires run()`) + } + + // The public generic preserves adapter-specific authoring types. The runtime + // stores one erased callback after validating the shared provider tag. + const erased = rule as unknown as AnyWebhookRule + let registration!: RuleRegistration + const disposeEffect = this.ctx.effect(() => { + /* v8 ignore next -- no await separates the public liveness check from this initializer. */ + if (this.closing) throw new Error('webhook runtime is closing') + if (this.rules.has(rule.id)) throw new Error(`webhook rule "${rule.id}" is already registered`) + registration = { + rule: erased, + controller: new AbortController(), + active: new Set(), + closing: false, + } + this.rules.set(rule.id, registration) + return () => this.disposeRegistration(registration) + }, `webhookRuntime.register(${rule.id})`) + return async () => { await disposeEffect() } + } + + /** + * Start every currently matching rule and return before any callback settles. + * @param delivery - authenticated provider data; snapshotted before dispatch. + * @throws synchronously when the runtime is closing or the delivery is malformed. + */ + dispatch(delivery: VerifiedWebhookDelivery): void { + if (this.closing) throw new Error('webhook runtime is closing') + const snapshot = snapshotDelivery(delivery) + for (const registration of [...this.rules.values()]) { + if (registration.closing || registration.rule.kind !== snapshot.kind) continue + this.startInvocation(registration, snapshot) + } + } + + /** Start one contained invocation and attach it to registration teardown. */ + private startInvocation(registration: RuleRegistration, delivery: VerifiedWebhookDelivery): void { + const tracked = Promise.resolve().then(async () => { + registration.controller.signal.throwIfAborted() + const request = await registration.rule.run(delivery, registration.controller.signal) + registration.controller.signal.throwIfAborted() + if (request !== null) { + await createWebhookSession( + this.selfCtx, + delivery, + registration.rule.id, + request, + registration.controller.signal, + ) + } + }).catch((error: unknown) => { + const invocation = `webhook: provider=${JSON.stringify(delivery.kind)} source=${JSON.stringify(delivery.source)} ` + + `delivery=${JSON.stringify(delivery.deliveryId)} rule=${JSON.stringify(registration.rule.id)}` + if (registration.controller.signal.aborted) { + this.selfCtx.logger.debug(`${invocation} stopped after disposal: ${errorChain(error)}`) + } else { + this.selfCtx.logger.warn(`${invocation} failed: ${errorChain(error)}`) + } + }).finally(() => { + registration.active.delete(tracked) + }) + registration.active.add(tracked) + } + + /** Memoized registration teardown: hide, abort, then drain. */ + private disposeRegistration(registration: RuleRegistration): Promise { + registration.disposal ??= (async () => { + registration.closing = true + this.rules.delete(registration.rule.id) + registration.controller.abort(new Error(`webhook rule "${registration.rule.id}" was disposed`)) + while (registration.active.size > 0) { + await Promise.allSettled([...registration.active]) + } + })() + return registration.disposal + } +} + +export default WebhookRuntime diff --git a/packages/webhook/webhook/src/invariant.ts b/packages/webhook/webhook/src/invariant.ts new file mode 100644 index 0000000000..604eee43bd --- /dev/null +++ b/packages/webhook/webhook/src/invariant.ts @@ -0,0 +1,48 @@ +/** Package-owned relationship invariant for webhook-origin prompt admission. */ + +import type { Context } from '@deepseek-ai/cordis' +import type {} from '@deepseek-ai/dsh-agent' +import type { InvariantFailure, InvariantInstaller } from '@deepseek-ai/dsh-invariants' +import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' +import type {} from '@deepseek-ai/dsh-workspace' +import type {} from './types.ts' + +const PACKAGE_NAME = '@deepseek-ai/dsh-webhook' + +/** Cordis invariant-companion plugin name. */ +export const name = 'webhook-invariant' +/** Registry required before reserving this package's invariant ownership. */ +export const inject = ['invariants'] + +/** Verify that one webhook-origin message already belongs to its cwd Workspace. */ +const install: InvariantInstaller = Object.assign(function installWebhookMessages( + ctx: Context, + fail: InvariantFailure, +): void { + ctx.on('internal/dispatch', (_mode, eventName, args) => { + if (eventName !== 'session/event') return + const [session, event] = args as [Session, SessionEvent] + if (event.type !== 'agent/inbox/spliced') return + const webhookMessages = event.data.inserted.filter(message => message.source.kind === 'webhook') + if (webhookMessages.length === 0) return + const cwd = session.header.cwd + if (cwd === undefined) return fail(`webhook Session "${session.id}" has no cwd`) + const owners = ctx.workspaceRegistry.list().filter(workspace => workspace.sessionIds.includes(session.id)) + if (owners.length !== 1) { + return fail(`webhook Session "${session.id}" belongs to ${owners.length} Workspaces at prompt admission`) + } + if (owners[0]?.path !== cwd) { + fail(`webhook Session "${session.id}" cwd ${JSON.stringify(cwd)} differs from its Workspace path`) + } + }, { global: true }) +}, { + inject: ['workspaceRegistry'], +}) + +/** + * Register this package's relationship invariant. + * @param ctx - Cordis context carrying the invariant registry. + * @returns the invariant registration disposer. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) diff --git a/packages/webhook/webhook/src/session.ts b/packages/webhook/webhook/src/session.ts new file mode 100644 index 0000000000..eef687ce4e --- /dev/null +++ b/packages/webhook/webhook/src/session.ts @@ -0,0 +1,181 @@ +/** Workspace-backed Session creation for one settled webhook rule result. */ + +import type { Context } from '@deepseek-ai/cordis' +import { randomUUID } from 'node:crypto' +import { isAbsolute } from 'node:path' +import type { ModelSelection } from '@deepseek-ai/dsh-agent' +import type {} from '@deepseek-ai/dsh-agent-default-model' +import type {} from '@deepseek-ai/dsh-agent-presets' +import { boundContextSummary, createUserMessage, errorChain, type LlmCallConfig } from '@deepseek-ai/dsh-llm' +import type {} from '@deepseek-ai/dsh-permission-presets' +import { SessionId } from '@deepseek-ai/dsh-session' +import type {} from '@deepseek-ai/dsh-session-title' +import type {} from '@deepseek-ai/dsh-workspace' +import type { WebhookRuleId } from './brand.ts' +import type { VerifiedWebhookDelivery, WebhookSessionRequest } from './types.ts' + +/** Detached values the creation transaction keeps across asynchronous preflight. */ +interface ResolvedWebhookSessionRequest { + readonly workspacePath: string + readonly title: string + readonly prompt: string + readonly agentPreset: string + readonly permissionPreset: string + readonly modelSelection: ModelSelection + readonly agentOptions: { + readonly provider: string + readonly model: string + readonly maxTokens?: number + } +} + +/** Require one non-empty string field from an untyped rule result. */ +function requiredString(record: Record, field: string): string { + const value = record[field] + if (typeof value !== 'string' || value.trim() === '') { + throw new TypeError(`webhook Session request ${field} must be a non-empty string`) + } + return value +} + +/** Snapshot and validate a same-process rule result before crossing awaits. */ +function resolveRequest(ctx: Context, input: WebhookSessionRequest): ResolvedWebhookSessionRequest { + const candidate: unknown = input + if (candidate === null || typeof candidate !== 'object' || Array.isArray(candidate)) { + throw new TypeError('webhook rule result must be null or a Session request object') + } + const record = candidate as Record + const workspacePath = requiredString(record, 'workspacePath') + if (!isAbsolute(workspacePath)) { + throw new TypeError(`webhook Session request workspacePath must be absolute, got ${JSON.stringify(workspacePath)}`) + } + const title = requiredString(record, 'title') + const prompt = requiredString(record, 'prompt') + const agentPreset = requiredString(record, 'agentPreset') + const permissionPreset = requiredString(record, 'permissionPreset') + const model = record['model'] + if (model !== undefined && (model === null || typeof model !== 'object' || Array.isArray(model))) { + throw new TypeError('webhook Session request model must be an object') + } + let agentOptions: ResolvedWebhookSessionRequest['agentOptions'] + let modelSelection: ModelSelection + if (model === undefined) { + const selected = ctx.agentDefaultModel.currentSelection() + agentOptions = { provider: selected.provider, model: selected.model } + modelSelection = { ...selected } + } else { + const modelRecord = model as Record + const provider = requiredString(modelRecord, 'provider') + const modelId = requiredString(modelRecord, 'model') + const maxTokens = modelRecord['maxTokens'] + if (maxTokens !== undefined + && (typeof maxTokens !== 'number' || !Number.isSafeInteger(maxTokens) || maxTokens <= 0)) { + throw new TypeError('webhook Session request model.maxTokens must be a positive safe integer') + } + agentOptions = { + provider, + model: modelId, + ...(maxTokens === undefined ? {} : { maxTokens }), + } + modelSelection = { provider, model: modelId } + } + return { workspacePath, title, prompt, agentPreset, permissionPreset, modelSelection, agentOptions } +} + +/** Log a rollback failure without replacing the operation's original failure. */ +function reportRollbackFailure(ctx: Context, subject: string, error: unknown): void { + ctx.logger.warn(`webhook: ${subject} rollback failed: ${errorChain(error)}`) +} + +/** Apply the creation-time selection until its first durable request header exists. */ +function installInitialModelSelection(agentCtx: Context, selection: ModelSelection): void { + agentCtx.on('agent/request', async (_payload, next): Promise => { + const resolved = await next() + const agent = agentCtx.agent + /* v8 ignore next -- AgentRegistry setup always provides the unpublished scoped Agent. */ + if (agent === undefined) throw new Error('webhook Session setup has no scoped Agent') + if (agent.session.requestHeader() !== undefined + || resolved.provider !== selection.provider + || resolved.model !== selection.model) return resolved + const { reasoningEffort: _inheritedEffort, ...withoutInheritedEffort } = resolved + return { + ...withoutInheritedEffort, + ...selection.reasoningEffort === undefined ? {} : { reasoningEffort: selection.reasoningEffort }, + } + }) +} + +/** + * Create, attach, title, configure, and prompt one ordinary root Session. + * Successful prompt admission ends webhook ownership of the operation; the + * Agent remains lifecycle-owned by `ctx` and follows normal Session behavior. + * + * @param ctx - untraced runtime context that owns the resulting Agent. + * @param delivery - exact verified provider delivery used for provenance. + * @param ruleId - rule that returned the request. + * @param request - same-process rule result. + * @param signal - registration lifetime cancellation through publication. + */ +export async function createWebhookSession( + ctx: Context, + delivery: VerifiedWebhookDelivery, + ruleId: WebhookRuleId, + request: WebhookSessionRequest, + signal: AbortSignal, +): Promise { + const resolved = resolveRequest(ctx, request) + ctx.permissionPresets.resolve(resolved.permissionPreset) + const preset = await ctx.agentPresets.resolve(resolved.agentPreset) + await ctx.agentPresets.standingKeyFor(preset.id) + signal.throwIfAborted() + + const workspace = await ctx.workspaceRegistry.create(resolved.workspacePath) + signal.throwIfAborted() + const sessionId = SessionId(`webhook-${randomUUID()}`) + const handle = await ctx.agents.create({ + sessionId, + signal, + meta: { cwd: workspace.path, agentPreset: preset.id }, + agentOptions: resolved.agentOptions, + setup: async (agentCtx) => { + await ctx.agentPresets.mount(agentCtx, preset.id) + installInitialModelSelection(agentCtx, resolved.modelSelection) + }, + }) + + let attached = false + try { + signal.throwIfAborted() + await workspace.attachSession(sessionId) + attached = true + signal.throwIfAborted() + ctx.permissionPresets.set(handle.agent.session, resolved.permissionPreset) + ctx.sessionTitle.rename(handle.agent.session, resolved.title) + handle.agent.followup(createUserMessage({ + content: [{ type: 'text', text: resolved.prompt }], + source: { + kind: 'webhook', + provider: delivery.kind, + source: delivery.source, + deliveryId: delivery.deliveryId, + ruleId, + form: 'notice', + summary: boundContextSummary(`${delivery.kind} webhook handled by ${ruleId}`), + }, + })) + } catch (error: unknown) { + if (attached) { + try { + await workspace.detachSession(sessionId) + } catch (rollbackError: unknown) { + reportRollbackFailure(ctx, `Workspace detach for Session "${sessionId}"`, rollbackError) + } + } + try { + await handle.dispose() + } catch (rollbackError: unknown) { + reportRollbackFailure(ctx, `Agent disposal for Session "${sessionId}"`, rollbackError) + } + throw error + } +} diff --git a/packages/webhook/webhook/src/types.ts b/packages/webhook/webhook/src/types.ts new file mode 100644 index 0000000000..158820b274 --- /dev/null +++ b/packages/webhook/webhook/src/types.ts @@ -0,0 +1,84 @@ +/** Provider-neutral webhook deliveries, rules, and Session requests. */ + +import type { JsonValue } from '@deepseek-ai/dsh-session' +import type { WebhookDeliveryId, WebhookRuleId, WebhookSourceId } from './brand.ts' + +/** Provider adapters add their normalized event type through declaration merging. */ +export interface WebhookEventMap {} + +/** Event value for a known provider kind, or generic lossless JSON for an out-of-tree kind. */ +export type WebhookEventOf = + K extends keyof WebhookEventMap ? WebhookEventMap[K] : JsonValue + +/** One authenticated and parsed provider delivery. */ +export interface VerifiedWebhookDelivery { + /** Provider family such as `github`. */ + readonly kind: K + /** Configured adapter instance such as `primary-github`. */ + readonly source: WebhookSourceId + /** Provider identity exposed as provenance, never as built-in deduplication state. */ + readonly deliveryId: WebhookDeliveryId + /** Provider-normalized lossless JSON. */ + readonly event: WebhookEventOf + /** Host receipt time in Unix epoch milliseconds. */ + readonly receivedAt: number +} + +/** Optional explicit model route and output cap for a webhook-created Agent. */ +export interface WebhookModelSelection { + /** Registered provider route. */ + readonly provider: string + /** Provider-owned model id. */ + readonly model: string + /** Optional positive output-token cap. */ + readonly maxTokens?: number +} + +/** The sole runtime action: create and prompt one root Session. */ +export interface WebhookSessionRequest { + /** Existing local directory to resolve or create as a Web Workspace. */ + readonly workspacePath: string + /** Explicit Session title. */ + readonly title: string + /** Non-empty initial text prompt. */ + readonly prompt: string + /** Agent composition mounted before publication. */ + readonly agentPreset: string + /** Sandbox and approval preset applied before prompt admission. */ + readonly permissionPreset: string + /** Optional explicit route; omission uses the complete current default, including reasoning effort. */ + readonly model?: WebhookModelSelection +} + +/** Trusted code that optionally creates one Session for a delivery. */ +export interface WebhookRule { + /** Globally unique diagnostic identity. */ + readonly id: WebhookRuleId + /** Provider kind this rule receives. */ + readonly kind: K + /** + * Run arbitrary trusted code and optionally request one Session. + * @param delivery - immutable authenticated provider data. + * @param signal - aborts when this registration or the runtime unloads. + * @returns one Session request, or `null` for no action. + */ + run( + delivery: Readonly>, + signal: AbortSignal, + ): WebhookSessionRequest | null | Promise +} + +declare module '@deepseek-ai/dsh-llm' { + interface MessageSourceMap { + /** Programmatic input admitted from one verified webhook rule. */ + webhook: { + readonly kind: 'webhook' + readonly provider: string + readonly source: WebhookSourceId + readonly deliveryId: WebhookDeliveryId + readonly ruleId: WebhookRuleId + readonly form: 'notice' + readonly summary: string + } + } +} diff --git a/packages/webhook/webhook/tests/invariant.spec.ts b/packages/webhook/webhook/tests/invariant.spec.ts new file mode 100644 index 0000000000..796b27d54e --- /dev/null +++ b/packages/webhook/webhook/tests/invariant.spec.ts @@ -0,0 +1,94 @@ +import { Context } from '@deepseek-ai/cordis' +import { createUserMessage } from '@deepseek-ai/dsh-llm' +import InvariantRegistry from '@deepseek-ai/dsh-invariants' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import { describe, expect, it } from 'vitest' +import { WebhookDeliveryId, WebhookRuleId, WebhookSourceId } from '../src/index.ts' +import * as WebhookInvariant from '../src/invariant.ts' + +/** Install the invariant over one mutable Workspace projection. */ +async function harness(): Promise<{ + ctx: Context + workspaces: { path: string; sessionIds: readonly SessionId[] }[] +}> { + const ctx = new Context() + const workspaces: { path: string; sessionIds: readonly SessionId[] }[] = [] + ctx.provide('workspaceRegistry', { list: () => workspaces } as never) + await ctx.plugin(SessionStore) + await ctx.plugin(InvariantRegistry) + await ctx.plugin(WebhookInvariant) + return { ctx, workspaces } +} + +/** Append one candidate webhook inbox insertion. */ +function insert(ctx: Context, id: SessionId, cwd?: string): void { + const session = ctx.sessions.create(id, { meta: { ...(cwd === undefined ? {} : { cwd }) } }) + session.append('agent/inbox/spliced', { + target: 'next-turn', + start: 0, + inserted: [createUserMessage({ + content: [{ type: 'text', text: 'review' }], + source: { + kind: 'webhook', + provider: 'github', + source: WebhookSourceId('primary'), + deliveryId: WebhookDeliveryId('delivery'), + ruleId: WebhookRuleId('review'), + form: 'notice', + summary: 'review', + }, + })], + }) +} + +describe('webhook prompt invariant', () => { + it('accepts prompt admission after matching Workspace attachment', async () => { + const { ctx, workspaces } = await harness() + const id = SessionId('attached') + workspaces.push({ path: '/workspace', sessionIds: [id] }) + expect(() => { insert(ctx, id, '/workspace') }).not.toThrow() + await ctx.fiber.dispose() + }) + + it('rejects a missing cwd, missing or duplicate Workspace, and path mismatch', async () => { + const missingCwd = await harness() + const noCwdId = SessionId('no-cwd') + missingCwd.workspaces.push({ path: '/workspace', sessionIds: [noCwdId] }) + expect(() => { insert(missingCwd.ctx, noCwdId) }).toThrow(/has no cwd/) + await missingCwd.ctx.fiber.dispose() + + const missing = await harness() + expect(() => { insert(missing.ctx, SessionId('missing'), '/workspace') }).toThrow(/belongs to 0 Workspaces/) + await missing.ctx.fiber.dispose() + + const duplicate = await harness() + const duplicateId = SessionId('duplicate') + duplicate.workspaces.push( + { path: '/workspace', sessionIds: [duplicateId] }, + { path: '/workspace', sessionIds: [duplicateId] }, + ) + expect(() => { insert(duplicate.ctx, duplicateId, '/workspace') }).toThrow(/belongs to 2 Workspaces/) + await duplicate.ctx.fiber.dispose() + + const mismatch = await harness() + const mismatchId = SessionId('mismatch') + mismatch.workspaces.push({ path: '/other', sessionIds: [mismatchId] }) + expect(() => { insert(mismatch.ctx, mismatchId, '/workspace') }).toThrow(/differs from its Workspace path/) + await mismatch.ctx.fiber.dispose() + }) + + it('ignores non-webhook inbox messages', async () => { + const { ctx } = await harness() + const session = ctx.sessions.create(SessionId('human'), { meta: { cwd: '/workspace' } }) + expect(() => session.append('agent/inbox/spliced', { + target: 'next-turn', + start: 0, + inserted: [createUserMessage({ + content: [{ type: 'text', text: 'hello' }], + source: { kind: 'user' }, + })], + })).not.toThrow() + expect(() => session.append('todo/write', { todos: [] })).not.toThrow() + await ctx.fiber.dispose() + }) +}) diff --git a/packages/webhook/webhook/tests/loader-composition.spec.ts b/packages/webhook/webhook/tests/loader-composition.spec.ts new file mode 100644 index 0000000000..1ddd743910 --- /dev/null +++ b/packages/webhook/webhook/tests/loader-composition.spec.ts @@ -0,0 +1,93 @@ +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { pathToFileURL } from 'node:url' +import { Context } from '@deepseek-ai/cordis' +import Include from '@deepseek-ai/cordis-plugin-include' +import Loader from '@deepseek-ai/cordis-plugin-loader' +import { afterEach, describe, expect, it } from 'vitest' +import WebhookRuntime, { + WebhookDeliveryId, + WebhookRuleId, + WebhookSourceId, +} from '../src/index.ts' + +let root: string | undefined +let context: Context | undefined + +afterEach(async () => { + await context?.fiber.dispose() + context = undefined + if (root !== undefined) await rm(root, { recursive: true, force: true }) + root = undefined +}) + +describe('real Loader composition', () => { + it('loads the default Service export and an effect-scoped rule', { timeout: 60_000 }, async () => { + root = await mkdtemp(join(tmpdir(), 'dsh-webhook-loader-')) + const configPath = join(root, 'cordis.yml') + await writeFile(configPath, [ + '- name: fixture-dependencies', + "- name: '@deepseek-ai/dsh-webhook'", + '- name: fixture-rule', + '', + ].join('\n')) + + const called = Promise.withResolvers() + const dependencies = { + name: 'fixture-dependencies', + apply(ctx: Context) { + for (const service of [ + 'agents', 'agentDefaultModel', 'agentPresets', 'permissionPresets', 'sessionTitle', 'workspaceRegistry', + ]) { + ctx.provide(service as never, {} as never) + } + }, + } + const rule = { + name: 'fixture-rule', + inject: ['webhookRuntime'], + apply(ctx: Context) { + ctx.webhookRuntime.register({ + id: WebhookRuleId('loader-rule'), + kind: 'fixture', + run() { + called.resolve(true) + return null + }, + }) + }, + } + + context = new Context() + context.baseUrl = pathToFileURL(root).href + '/' + await context.plugin(Loader) + context.loader.builtins.include = Include + const modules = new Map([ + ['fixture-dependencies', dependencies], + ['@deepseek-ai/dsh-webhook', WebhookRuntime], + ['fixture-rule', rule], + ]) + context.loader.internal = { + version: 'v2', + async import(specifier: string) { + if (!modules.has(specifier)) throw new Error(`unexpected Loader import: ${specifier}`) + return modules.get(specifier) + }, + } as unknown as NonNullable + await context.loader.create({ + name: 'cordis:include', + config: { path: pathToFileURL(configPath).href }, + }) + await context.loader.await() + expect([...context.loader.entries()].filter(entry => entry.fiber === undefined && !entry.disabled)).toEqual([]) + context.webhookRuntime.dispatch({ + kind: 'fixture', + source: WebhookSourceId('loader'), + deliveryId: WebhookDeliveryId('loader-delivery'), + event: {}, + receivedAt: 1, + }) + await called.promise + }) +}) diff --git a/packages/webhook/webhook/tests/runtime.spec.ts b/packages/webhook/webhook/tests/runtime.spec.ts new file mode 100644 index 0000000000..6356b2eed1 --- /dev/null +++ b/packages/webhook/webhook/tests/runtime.spec.ts @@ -0,0 +1,281 @@ +import { Context } from '@deepseek-ai/cordis' +import { readFileSync } from 'node:fs' +import { afterEach, describe, expect, it, vi } from 'vitest' +import WebhookRuntime, { + WebhookDeliveryId, + WebhookRuleId, + WebhookSourceId, + type VerifiedWebhookDelivery, +} from '../src/index.ts' + +const contexts: Context[] = [] + +afterEach(async () => { + await Promise.allSettled(contexts.splice(0).map(ctx => ctx.fiber.dispose())) +}) + +/** Construct the runtime directly so callback-only tests need no Agent stack. */ +function harness(): { ctx: Context; runtime: WebhookRuntime } { + const ctx = new Context() + contexts.push(ctx) + return { ctx, runtime: new WebhookRuntime(ctx) } +} + +/** One valid generic delivery. */ +function delivery(id = 'delivery-1'): VerifiedWebhookDelivery<'fixture'> { + return { + kind: 'fixture', + source: WebhookSourceId('fixture-source'), + deliveryId: WebhookDeliveryId(id), + event: { value: 1 }, + receivedAt: 1, + } +} + +describe('WebhookRuntime', () => { + it('dispatches a detached immutable snapshot and returns before the rule settles', async () => { + const { runtime } = harness() + const entered = Promise.withResolvers>>() + const release = Promise.withResolvers() + runtime.register({ + id: WebhookRuleId('fixture-rule'), + kind: 'fixture', + async run(input) { + entered.resolve(input) + await release.promise + return null + }, + }) + const original = delivery() + runtime.dispatch(original) + ;(original.event as { value: number }).value = 2 + const seen = await entered.promise + expect(seen).not.toBe(original) + expect(seen.event).toEqual({ value: 1 }) + expect(Object.isFrozen(seen)).toBe(true) + expect(Object.isFrozen(seen.event)).toBe(true) + release.resolve(true) + }) + + it('starts matching siblings independently and contains one failure', async () => { + const { runtime } = harness() + const started: string[] = [] + const both = Promise.withResolvers() + const maybeDone = (): void => { if (started.length === 2) both.resolve(true) } + runtime.register({ + id: WebhookRuleId('throws'), + kind: 'fixture', + run() { + started.push('throws') + maybeDone() + throw new Error('fixture failure') + }, + }) + runtime.register({ + id: WebhookRuleId('succeeds'), + kind: 'fixture', + run() { + started.push('succeeds') + maybeDone() + return null + }, + }) + runtime.register({ + id: WebhookRuleId('other-kind'), + kind: 'other', + run: vi.fn(() => null), + }) + runtime.dispatch(delivery()) + await both.promise + expect(started).toEqual(['throws', 'succeeds']) + }) + + it('rejects malformed registrations and duplicate ids', async () => { + const { runtime } = harness() + expect(() => runtime.register({ id: WebhookRuleId(''), kind: 'fixture', run: () => null })) + .toThrow(/id must be a non-empty string/) + expect(() => runtime.register({ id: WebhookRuleId('bad-kind'), kind: '', run: () => null })) + .toThrow(/kind must be a non-empty string/) + expect(() => runtime.register({ id: WebhookRuleId('bad-run'), kind: 'fixture', run: 1 as never })) + .toThrow(/requires run/) + const dispose = runtime.register({ id: WebhookRuleId('same'), kind: 'fixture', run: () => null }) + expect(() => runtime.register({ id: WebhookRuleId('same'), kind: 'fixture', run: () => null })) + .toThrow(/already registered/) + await dispose() + expect(() => runtime.register({ id: WebhookRuleId('same'), kind: 'fixture', run: () => null })) + .not.toThrow() + }) + + it('hides, aborts, and drains a registration before disposal resolves', async () => { + const { ctx, runtime } = harness() + const warnings = vi.spyOn(ctx.logger, 'warn') + const entered = Promise.withResolvers() + const finished = Promise.withResolvers() + let calls = 0 + const dispose = runtime.register({ + id: WebhookRuleId('draining'), + kind: 'fixture', + async run(_input, signal) { + calls++ + entered.resolve(signal) + await new Promise((resolve) => { + signal.addEventListener('abort', () => { resolve() }, { once: true }) + }) + finished.resolve(true) + return null + }, + }) + runtime.dispatch(delivery()) + const signal = await entered.promise + const draining = dispose() + expect(signal.aborted).toBe(true) + runtime.dispatch(delivery('after-dispose')) + await draining + await finished.promise + expect(calls).toBe(1) + await expect(dispose()).resolves.toBeUndefined() + expect(warnings).not.toHaveBeenCalled() + }) + + it('aborts active rules and refuses later work when the runtime disposes', async () => { + const { ctx, runtime } = harness() + const entered = Promise.withResolvers() + runtime.register({ + id: WebhookRuleId('runtime-disposal'), + kind: 'fixture', + async run(_input, signal) { + entered.resolve(signal) + await new Promise((resolve) => { + signal.addEventListener('abort', () => { resolve() }, { once: true }) + }) + return null + }, + }) + runtime.dispatch(delivery()) + const signal = await entered.promise + await ctx.fiber.dispose() + expect(signal.aborted).toBe(true) + expect(() => { runtime.dispatch(delivery()) }).toThrow(/closing/) + expect(() => runtime.register({ id: WebhookRuleId('late'), kind: 'fixture', run: () => null })) + .toThrow(/closing/) + }) + + it.each([ + [{ ...delivery(), kind: '' }, /kind/], + [{ ...delivery(), source: WebhookSourceId('') }, /source/], + [{ ...delivery(), deliveryId: WebhookDeliveryId('') }, /delivery id/], + [{ ...delivery(), receivedAt: -1 }, /receivedAt/], + [{ ...delivery(), event: { invalid: undefined } }, /lossless JSON/], + ] as const)('rejects malformed deliveries synchronously', (input, message) => { + const { runtime } = harness() + expect(() => { runtime.dispatch(input as never) }).toThrow(message) + }) + + it('intentionally invokes a rule again for a repeated delivery', async () => { + const { runtime } = harness() + const calledTwice = Promise.withResolvers() + let calls = 0 + runtime.register({ + id: WebhookRuleId('repeat'), + kind: 'fixture', + run() { + calls++ + if (calls === 2) calledTwice.resolve(true) + return null + }, + }) + runtime.dispatch(delivery()) + runtime.dispatch(delivery()) + await calledTwice.promise + expect(calls).toBe(2) + }) + + it('keeps execution-state, retry, dedupe, and completion machinery out of the runtime', () => { + const production = [ + '../src/brand.ts', + '../src/types.ts', + '../src/session.ts', + '../src/index.ts', + '../src/invariant.ts', + ].map(path => readFileSync(new URL(path, import.meta.url), 'utf8')).join('\n') + const forbidden: ReadonlyArray = [ + ['execution records', /\bWebhook(?:Execution|Status)\b/], + ['delivery storage domains', /@deepseek-ai\/dsh-storage|\bstorageDomain\b|\bDomainSpec\b/], + ['retry timers', /\bset(?:Timeout|Interval)\s*\(/], + ['delivery-id dedupe maps', /new Map<\s*WebhookDeliveryId/], + ['Agent idle waits', /\.whenIdle\s*\(/], + ['Agent status listeners', /\.on\(\s*['"]agent\/status/], + ['turn completion listeners', /\.on\(\s*['"]turn\/end/], + ['webhook completion events', /['"]webhook\/(?:completion|completed)['"]/], + ['webhook management Remotes', /@Remote\b|\bRemote\s*\(/], + ] + for (const [label, pattern] of forbidden) { + expect(production, label).not.toMatch(pattern) + } + }) + + it('creates one Session per matching repeated delivery', async () => { + const ctx = new Context() + contexts.push(ctx) + const followedTwice = Promise.withResolvers() + const messages: unknown[] = [] + const session = {} + const attachSession = vi.fn(async () => {}) + ctx.provide('agentDefaultModel', { + currentSelection: () => ({ provider: 'p', model: 'm' }), + } as never) + ctx.provide('permissionPresets', { + resolve: () => ({}), + set: () => {}, + } as never) + ctx.provide('agentPresets', { + resolve: async (id: string) => ({ id }), + standingKeyFor: async () => ({}), + mount: async (_agentCtx: unknown, id: string) => ({ id }), + } as never) + ctx.provide('workspaceRegistry', { + create: async () => ({ + path: '/workspace', + attachSession, + detachSession: async () => {}, + }), + } as never) + ctx.provide('sessionTitle', { rename: () => ({}) } as never) + ctx.provide('agents', { + create: async (options: { setup?: (agentCtx: unknown) => Promise }) => { + await options.setup?.({ on: () => () => {} }) + return { + agent: { + session, + followup: (message: unknown) => { + messages.push(message) + if (messages.length === 2) followedTwice.resolve(true) + }, + }, + dispose: async () => {}, + } + }, + } as never) + const runtime = new WebhookRuntime(ctx) + runtime.register({ + id: WebhookRuleId('creates'), + kind: 'fixture', + run: () => ({ + workspacePath: '/workspace', + title: 'Created', + prompt: 'Work', + agentPreset: 'standard', + permissionPreset: 'read-only', + }), + }) + runtime.dispatch(delivery()) + runtime.dispatch(delivery()) + await followedTwice.promise + expect(attachSession).toHaveBeenCalledTimes(2) + expect(messages).toHaveLength(2) + expect(messages[0]).toMatchObject({ + content: [{ type: 'text', text: 'Work' }], + source: { kind: 'webhook', ruleId: 'creates' }, + }) + }) +}) diff --git a/packages/webhook/webhook/tests/session.spec.ts b/packages/webhook/webhook/tests/session.spec.ts new file mode 100644 index 0000000000..bdb9201fa9 --- /dev/null +++ b/packages/webhook/webhook/tests/session.spec.ts @@ -0,0 +1,319 @@ +import type { Context } from '@deepseek-ai/cordis' +import { ReasoningEffortId, type LlmCallConfig } from '@deepseek-ai/dsh-llm' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + WebhookDeliveryId, + WebhookRuleId, + WebhookSourceId, + type VerifiedWebhookDelivery, + type WebhookSessionRequest, +} from '../src/index.ts' +import { createWebhookSession } from '../src/session.ts' + +interface HarnessOptions { + failAt?: 'permission-resolve' | 'preset-resolve' | 'standing' | 'workspace' | 'agent' | 'attach' | 'permission-set' | 'title' | 'followup' + failDetach?: boolean + failDispose?: boolean + abortAt?: 'workspace' | 'agent' +} + +interface SessionHarness { + readonly ctx: Context + readonly calls: string[] + readonly messages: unknown[] + readonly modelListeners: Map + markRequestHeader(): void + readonly controller: AbortController + readonly request: WebhookSessionRequest +} + +const active: SessionHarness[] = [] + +afterEach(() => { + active.length = 0 +}) + +/** Build a same-process fake around the private creation transaction. */ +function harness(options: HarnessOptions = {}): SessionHarness { + const calls: string[] = [] + const messages: unknown[] = [] + const modelListeners = new Map() + const controller = new AbortController() + let requestHeader: object | undefined + const session = { + id: 'webhook-session', + header: { cwd: '/workspace' }, + requestHeader: () => requestHeader, + } + const agent = { + id: 'webhook-session', + session, + followup(message: unknown) { + calls.push('followup') + if (options.failAt === 'followup') throw new Error('followup failed') + messages.push(message) + }, + } + const handle = { + agent, + async dispose() { + calls.push('dispose') + if (options.failDispose) throw new Error('dispose failed') + }, + } + const workspace = { + path: '/workspace', + async attachSession() { + calls.push('attach') + if (options.failAt === 'attach') throw new Error('attach failed') + }, + async detachSession() { + calls.push('detach') + if (options.failDetach) throw new Error('detach failed') + }, + } + const fake = { + logger: { warn: vi.fn() }, + permissionPresets: { + resolve(name: string) { + calls.push(`permission-resolve:${name}`) + if (options.failAt === 'permission-resolve') throw new Error('permission resolve failed') + return {} + }, + set(_session: unknown, name: string) { + calls.push(`permission-set:${name}`) + if (options.failAt === 'permission-set') throw new Error('permission set failed') + }, + }, + agentDefaultModel: { + currentSelection() { + calls.push('default-model') + return { provider: 'default-provider', model: 'default-model', reasoningEffort: 'high' } + }, + }, + agentPresets: { + async resolve(name: string) { + calls.push(`preset-resolve:${name}`) + if (options.failAt === 'preset-resolve') throw new Error('preset resolve failed') + return { id: name } + }, + async standingKeyFor(name: string) { + calls.push(`standing:${name}`) + if (options.failAt === 'standing') throw new Error('standing failed') + return {} + }, + async mount(_agentCtx: unknown, name: string) { + calls.push(`mount:${name}`) + return { id: name } + }, + }, + workspaceRegistry: { + async create(path: string) { + calls.push(`workspace:${path}`) + if (options.failAt === 'workspace') throw new Error('workspace failed') + if (options.abortAt === 'workspace') controller.abort(new Error('abort after workspace')) + return workspace + }, + }, + agents: { + async create(createOptions: { setup?: (ctx: unknown) => Promise }) { + calls.push('agent-create') + if (options.failAt === 'agent') throw new Error('agent failed') + await createOptions.setup?.({ + agent, + on(event: string, listener: unknown) { + modelListeners.set(event, listener) + return () => {} + }, + }) + if (options.abortAt === 'agent') controller.abort(new Error('abort after agent')) + return handle + }, + }, + sessionTitle: { + rename() { + calls.push('title') + if (options.failAt === 'title') throw new Error('title failed') + return {} + }, + }, + } + const result: SessionHarness = { + ctx: fake as unknown as Context, + calls, + messages, + modelListeners, + markRequestHeader() { requestHeader = {} }, + controller, + request: { + workspacePath: '/workspace', + title: 'Review PR', + prompt: 'Review it', + agentPreset: 'standard', + permissionPreset: 'read-only', + }, + } + active.push(result) + return result +} + +const delivery: VerifiedWebhookDelivery = { + kind: 'github', + source: WebhookSourceId('primary'), + deliveryId: WebhookDeliveryId('delivery'), + event: { action: 'ready_for_review' }, + receivedAt: 1, +} + +async function create(test: SessionHarness, request = test.request): Promise { + await createWebhookSession( + test.ctx, + delivery, + WebhookRuleId('review'), + request, + test.controller.signal, + ) +} + +/** Read the initial model-selection listener installed during Agent setup. */ +function modelRequestListener(test: SessionHarness): ( + payload: unknown, + next: () => Promise, +) => Promise { + const listener = test.modelListeners.get('agent/request') + if (typeof listener !== 'function') { + throw new Error('webhook Session did not install its initial model selection') + } + return listener as ( + payload: unknown, + next: () => Promise, + ) => Promise +} + +describe('webhook Session creation', () => { + it('preflights, mounts, attaches, configures, titles, and prompts in order', async () => { + const test = harness() + await create(test) + expect(test.calls).toEqual([ + 'default-model', + 'permission-resolve:read-only', + 'preset-resolve:standard', + 'standing:standard', + 'workspace:/workspace', + 'agent-create', + 'mount:standard', + 'attach', + 'permission-set:read-only', + 'title', + 'followup', + ]) + expect(test.messages).toHaveLength(1) + expect(test.messages[0]).toMatchObject({ + role: 'user', + content: [{ type: 'text', text: 'Review it' }], + source: { + kind: 'webhook', provider: 'github', source: 'primary', deliveryId: 'delivery', ruleId: 'review', + }, + }) + }) + + it('uses a complete explicit model without consulting the default', async () => { + const test = harness() + await create(test, { ...test.request, model: { provider: 'p', model: 'm', maxTokens: 10 } }) + expect(test.calls).not.toContain('default-model') + const withoutCap = harness() + await create(withoutCap, { ...withoutCap.request, model: { provider: 'p', model: 'm' } }) + expect(withoutCap.calls).not.toContain('default-model') + await expect(modelRequestListener(withoutCap)(undefined, async () => ({ + provider: 'p', model: 'm', reasoningEffort: ReasoningEffortId('inherited'), + }))).resolves.toEqual({ provider: 'p', model: 'm' }) + }) + + it('preserves default reasoning until the first request header is durable', async () => { + const test = harness() + await create(test) + const request = modelRequestListener(test) + + await expect(request(undefined, async () => ({ + provider: 'other-provider', + model: 'default-model', + reasoningEffort: ReasoningEffortId('other-provider-effort'), + }))).resolves.toMatchObject({ reasoningEffort: 'other-provider-effort' }) + await expect(request(undefined, async () => ({ + provider: 'default-provider', + model: 'other-model', + reasoningEffort: ReasoningEffortId('other-model-effort'), + }))).resolves.toMatchObject({ reasoningEffort: 'other-model-effort' }) + + const routed = await request(undefined, async () => ({ + provider: 'default-provider', + model: 'default-model', + reasoningEffort: ReasoningEffortId('inherited'), + })) as unknown + expect(routed).toEqual({ + provider: 'default-provider', + model: 'default-model', + reasoningEffort: 'high', + }) + + test.markRequestHeader() + await expect(request(undefined, async () => ({ + provider: 'later-provider', + model: 'later-model', + reasoningEffort: ReasoningEffortId('later'), + }))).resolves.toEqual({ + provider: 'later-provider', + model: 'later-model', + reasoningEffort: 'later', + }) + }) + + it.each([ + [null, /must be null or a Session request object/], + [{}, /workspacePath/], + [{ workspacePath: 'relative', title: 't', prompt: 'p', agentPreset: 'a', permissionPreset: 'x' }, /must be absolute/], + [{ workspacePath: '/w', title: ' ', prompt: 'p', agentPreset: 'a', permissionPreset: 'x' }, /title/], + [{ workspacePath: '/w', title: 't', prompt: '', agentPreset: 'a', permissionPreset: 'x' }, /prompt/], + [{ workspacePath: '/w', title: 't', prompt: 'p', agentPreset: '', permissionPreset: 'x' }, /agentPreset/], + [{ workspacePath: '/w', title: 't', prompt: 'p', agentPreset: 'a', permissionPreset: '' }, /permissionPreset/], + [{ workspacePath: '/w', title: 't', prompt: 'p', agentPreset: 'a', permissionPreset: 'x', model: null }, /model must be an object/], + [{ workspacePath: '/w', title: 't', prompt: 'p', agentPreset: 'a', permissionPreset: 'x', model: {} }, /provider/], + [{ workspacePath: '/w', title: 't', prompt: 'p', agentPreset: 'a', permissionPreset: 'x', model: { provider: 'p', model: 'm', maxTokens: 0 } }, /maxTokens/], + ] as const)('rejects malformed rule result %# before side effects', async (request, message) => { + const test = harness() + await expect(create(test, request as never)).rejects.toThrow(message) + expect(test.calls).toEqual([]) + }) + + it.each([ + 'permission-resolve', 'preset-resolve', 'standing', 'workspace', 'agent', 'attach', + ] as const)('contains a %s failure before prompt admission', async (failAt) => { + const test = harness({ failAt }) + await expect(create(test)).rejects.toThrow() + expect(test.calls).not.toContain('followup') + if (failAt === 'attach') expect(test.calls).toContain('dispose') + }) + + it.each(['permission-set', 'title', 'followup'] as const)( + 'detaches and disposes after a %s failure', + async (failAt) => { + const test = harness({ failAt }) + await expect(create(test)).rejects.toThrow() + expect(test.calls).toContain('detach') + expect(test.calls).toContain('dispose') + }, + ) + + it('preserves the original failure while reporting rollback failures', async () => { + const test = harness({ failAt: 'title', failDetach: true, failDispose: true }) + await expect(create(test)).rejects.toThrow('title failed') + expect((test.ctx.logger.warn as ReturnType)).toHaveBeenCalledTimes(2) + }) + + it.each(['workspace', 'agent'] as const)('honors cancellation after %s settlement', async (abortAt) => { + const test = harness({ abortAt }) + await expect(create(test)).rejects.toThrow(/abort after/) + expect(test.calls).not.toContain('followup') + if (abortAt === 'agent') expect(test.calls).toContain('dispose') + }) +}) diff --git a/packages/examples/acp-demo/tsconfig.json b/packages/webhook/webhook/tsconfig.json similarity index 55% rename from packages/examples/acp-demo/tsconfig.json rename to packages/webhook/webhook/tsconfig.json index 8511f21347..8a3f35ba70 100644 --- a/packages/examples/acp-demo/tsconfig.json +++ b/packages/webhook/webhook/tsconfig.json @@ -11,38 +11,38 @@ { "path": "../../../vendor/cordis" }, - { - "path": "../../../vendor/schemastery" - }, { "path": "../../../vendor/loader" }, { - "path": "../../boot/app-boot" + "path": "../../../vendor/include" }, { - "path": "../../acp/acp" + "path": "../../util/brand" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/session" }, { "path": "../../core/agent" }, { - "path": "../../session-query/session-query" + "path": "../../core/agent-default-model" }, { - "path": "../../session-query/session-query-sqlite" + "path": "../../preset/agent-presets" }, { - "path": "../agent-spine-demo" + "path": "../../interaction/permission-presets" }, { - "path": "../../context/agent-instructions" + "path": "../../session/session-title" }, { - "path": "../../session/session-checkpoint-policy" - }, - { - "path": "../../session/session-persistence-jsonl" + "path": "../../workspace/workspace" }, { "path": "../../runtime-diagnostics/invariants" diff --git a/packages/workflow/README.i18n.yaml b/packages/workflow/README.i18n.yaml index 1b39e5fc7a..ff979da66e 100644 --- a/packages/workflow/README.i18n.yaml +++ b/packages/workflow/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/workflow/README.md README.md: 156e9f0d84ced4eab8c25da91d29d239d18b0629 -README.zh.md: 24687f81d612c23786150c5b5a70bce352954663 +README.zh.md: ec65b32bfe2983ae163cb208842969f6f79b5f79 diff --git a/packages/workflow/README.zh.md b/packages/workflow/README.zh.md index 24687f81d6..ec65b32bfe 100644 --- a/packages/workflow/README.zh.md +++ b/packages/workflow/README.zh.md @@ -6,11 +6,11 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`workflow/`](workflow/README.md) | 定义工作流执行和生命周期事件 | `ctx.workflowEngine` | -| [`workflow-worker-thread/`](workflow-worker-thread/README.md) | 在线程中运行工作流脚本 | 注册到 `ctx.workflowEngine` | -| [`tool-workflow/`](tool-workflow/README.md) | 向模型公开通用工作流执行 | 注册到 `ctx.tools` | -| [`tool-ralph/`](tool-ralph/README.md) | 公开使用全新 agent(智能体)的固定 Ralph 工作流 | 注册到 `ctx.tools` | +| [`workflow/`](workflow/README.zh.md) | 定义工作流执行和生命周期事件 | `ctx.workflowEngine` | +| [`workflow-worker-thread/`](workflow-worker-thread/README.zh.md) | 在线程中运行工作流脚本 | 注册到 `ctx.workflowEngine` | +| [`tool-workflow/`](tool-workflow/README.zh.md) | 向模型公开通用工作流执行 | 注册到 `ctx.tools` | +| [`tool-ralph/`](tool-ralph/README.zh.md) | 公开使用全新 agent(智能体)的固定 Ralph 工作流 | 注册到 `ctx.tools` | -worker thread 将工作流执行与宿主事件循环隔离,但不构成安全边界。参见[动态工作流](../../.agents/notes/implemented/feature/2026-07-05-dynamic-workflows.md)和 [Ralph 工具](../../.agents/notes/implemented/feature/2026-07-19-fresh-agent-ralph-workflow-tool.md)决策。 +worker thread 将工作流执行与宿主事件循环隔离,但不构成安全边界。参见[动态工作流](../../.agents/notes/implemented/feature/2026-07-05-dynamic-workflows.zh.md)和 [Ralph 工具](../../.agents/notes/implemented/feature/2026-07-19-fresh-agent-ralph-workflow-tool.zh.md)决策。 -子系统参考——启动请求、`WorkflowMeta`、结果、实时运行、`workflow/*` 事件——见 [docs/subsystems/workflow.md](../../docs/subsystems/workflow.md);决策见[动态工作流](../../.agents/notes/implemented/feature/2026-07-05-dynamic-workflows.md)与 [Ralph 消费方](../../.agents/notes/implemented/feature/2026-07-19-fresh-agent-ralph-workflow-tool.md) Agent Note。 +子系统参考——启动请求、`WorkflowMeta`、结果、实时运行、`workflow/*` 事件——见 [docs/subsystems/workflow.md](../../docs/subsystems/workflow.zh.md);决策见[动态工作流](../../.agents/notes/implemented/feature/2026-07-05-dynamic-workflows.zh.md)与 [Ralph 消费方](../../.agents/notes/implemented/feature/2026-07-19-fresh-agent-ralph-workflow-tool.zh.md) Agent Note。 diff --git a/packages/workflow/tool-ralph/README.i18n.yaml b/packages/workflow/tool-ralph/README.i18n.yaml index 07a851efb9..d6a1526129 100644 --- a/packages/workflow/tool-ralph/README.i18n.yaml +++ b/packages/workflow/tool-ralph/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/workflow/tool-ralph/README.md README.md: ae9d7c80763f751860bc2073950dde331fdd5a8e -README.zh.md: 692ebb693b54ade2c2fbde18f818ced10e9d5863 +README.zh.md: 9e41b9e527f865a090eaa78ffef30bf6b890357b diff --git a/packages/workflow/tool-ralph/README.zh.md b/packages/workflow/tool-ralph/README.zh.md index 692ebb693b..9e41b9e527 100644 --- a/packages/workflow/tool-ralph/README.zh.md +++ b/packages/workflow/tool-ralph/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -面向模型的 `ralph` 工具运行固定的前台工作流,把一个不可变目标依次交给多个全新子 agent(智能体)。它展示如何把专用编排策略实现为基于 [`ctx.workflowEngine`](../workflow/README.md) 和 [`ctx.subagents`](../../subagent/subagent/README.md) 的普通插件:不会向 `agent-loop` 添加 Ralph 模式或全新 agent loop(智能体循环),同会话的[目标领域](../../goal/goal/README.md)也保持独立。策略和暂缓事项由 [Ralph Agent Note(agent 决策记录)](../../../.agents/notes/implemented/feature/2026-07-19-fresh-agent-ralph-workflow-tool.md)负责。 +面向模型的 `ralph` 工具运行固定的前台工作流,把一个不可变目标依次交给多个全新子 agent(智能体)。它展示如何把专用编排策略实现为基于 [`ctx.workflowEngine`](../workflow/README.zh.md) 和 [`ctx.subagents`](../../subagent/subagent/README.zh.md) 的普通插件:不会向 `agent-loop` 添加 Ralph 模式或全新 agent loop(智能体循环),同会话的[目标领域](../../goal/goal/README.zh.md)也保持独立。策略和暂缓事项由 [Ralph Agent Note(agent 决策记录)](../../../.agents/notes/implemented/feature/2026-07-19-fresh-agent-ralph-workflow-tool.zh.md)负责。 ## 契约 @@ -59,7 +59,7 @@ Use the ralph tool ONLY when the direct human explicitly asks for a Ralph loop o #### 模型看到的内容 -已生成的 [`ralph` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-ralph)公开一个必填 `objective` 字符串和一个可选 `maxRounds` 数字。提供方选择、交接大小、报告 schema、工作流脚本和编排行为均由部署侧控制,不在调用 schema 中。 +已生成的 [`ralph` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-ralph)公开一个必填 `objective` 字符串和一个可选 `maxRounds` 数字。提供方选择、交接大小、报告 schema、工作流脚本和编排行为均由部署侧控制,不在调用 schema 中。 #### Token 影响 diff --git a/packages/workflow/tool-ralph/package.json b/packages/workflow/tool-ralph/package.json index ca7b5f4889..980ed904a6 100644 --- a/packages/workflow/tool-ralph/package.json +++ b/packages/workflow/tool-ralph/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-ralph", "description": "Model-facing fresh-agent Ralph loop over the workflow and subagent seams", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/workflow/tool-workflow/README.i18n.yaml b/packages/workflow/tool-workflow/README.i18n.yaml index 197b47832a..f87a12407f 100644 --- a/packages/workflow/tool-workflow/README.i18n.yaml +++ b/packages/workflow/tool-workflow/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/workflow/tool-workflow/README.md README.md: 918fe5b9aff74b3b959107ec1eebfd7966afe6d1 -README.zh.md: f7e62560a1b581cf64d78dc379b821d17f29b3aa +README.zh.md: 2546a0fc92ef39d71949cc0061dcf363ecc2e22d diff --git a/packages/workflow/tool-workflow/README.zh.md b/packages/workflow/tool-workflow/README.zh.md index f7e62560a1..2546a0fc92 100644 --- a/packages/workflow/tool-workflow/README.zh.md +++ b/packages/workflow/tool-workflow/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -面向模型的 **`workflow` 工具**:运行一段扇出 subagent 的 JavaScript 编排脚本,并返回脚本的最终值。本包负责基于 [`ctx.workflowEngine`](../workflow/README.md) 定义面向模型的 schema 和运行生命周期;脚本解析、执行、上限与取消位于 seam 之后,消费方仍负责面向父级的 schema 和结果包络。 +面向模型的 **`workflow` 工具**:运行一段扇出 subagent 的 JavaScript 编排脚本,并返回脚本的最终值。本包负责基于 [`ctx.workflowEngine`](../workflow/README.zh.md) 定义面向模型的 schema 和运行生命周期;脚本解析、执行、上限与取消位于 seam 之后,消费方仍负责面向父级的 schema 和结果包络。 ## 模型看到的内容 @@ -10,7 +10,7 @@ ## 生命周期 -收集是同步的(类似 [`dsh-tool-subagent`](../../subagent/tool-subagent/README.md)):`execute` 启动运行并等待 `run.result`;这些操作位于 `try/finally` 中,该结构总会 dispose(资源释放)运行,使脚本及其子 agent(智能体)在每条路径上完全停稳。`exec.signal` 会桥接到 `run.cancel()`,包括启动前已经中止的情况。非 `completed` 结束原因会映射为报告原因的 `isError` 结果,绝不会把局部输出当作成功;`start()` 同步抛出的解析/meta 失败会变成模型可据以修正的 `isError`。完成时返回规范值 `{ runId, agentsStarted, result }`;Native 渲染器保留 meta 名称、agent 数量和 JSON 值,只会在 `maxResultChars` 处截断该投影。 +收集是同步的(类似 [`dsh-tool-subagent`](../../subagent/tool-subagent/README.zh.md)):`execute` 启动运行并等待 `run.result`;这些操作位于 `try/finally` 中,该结构总会 dispose(资源释放)运行,使脚本及其子 agent(智能体)在每条路径上完全停稳。`exec.signal` 会桥接到 `run.cancel()`,包括启动前已经中止的情况。非 `completed` 结束原因会映射为报告原因的 `isError` 结果,绝不会把局部输出当作成功;`start()` 同步抛出的解析/meta 失败会变成模型可据以修正的 `isError`。完成时返回规范值 `{ runId, agentsStarted, result }`;Native 渲染器保留 meta 名称、agent 数量和 JSON 值,只会在 `maxResultChars` 处截断该投影。 对于根 transport 执行(`exec.parent` 缺省),工具还会把运行投影到调用 Agent 的 Session:`start()` 返回后写 run-start,只记录 `run.id` 匹配的成员开始与结束,并且只在 `run.result` 已取得且 `dispose()` 完全停稳后写 run-end。嵌套 transport 调用照常执行,但不写工作流记录。任一次 Session append 首次失败后,本运行会停止后续记录并只告警一次,留下空记录或合法连续前缀,同时不改变工具结果和清理。 @@ -18,7 +18,7 @@ ## 渲染意图 -渲染意图预先确定(见[渲染意图 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.md)):使用一个 `generic` 卡片,标题为 `workflow: `,直接从 `args.meta.name` 读取(呈现是参数的纯函数,不要求引擎解析);脚本文本作为 `rawInput` 携带。结果继续使用 generic 卡片。 +渲染意图预先确定(见[渲染意图 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.zh.md)):使用一个 `generic` 卡片,标题为 `workflow: `,直接从 `args.meta.name` 读取(呈现是参数的纯函数,不要求引擎解析);脚本文本作为 `rawInput` 携带。结果继续使用 generic 卡片。 ## 配置 @@ -53,7 +53,7 @@ Use the tool ONLY when the user explicitly asks for a workflow or for #### 模型看到的内容 -工具可见时,已生成的默认 [`workflow` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-workflow) 包含完整的 JavaScript 钩子与元数据约定;`toolName` 可以重命名该定义,模型会提交脚本、元数据和可选 args。 +工具可见时,已生成的默认 [`workflow` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-workflow) 包含完整的 JavaScript 钩子与元数据约定;`toolName` 可以重命名该定义,模型会提交脚本、元数据和可选 args。 #### Token 影响 diff --git a/packages/workflow/tool-workflow/package.json b/packages/workflow/tool-workflow/package.json index 6abde88c22..455d2ee54c 100644 --- a/packages/workflow/tool-workflow/package.json +++ b/packages/workflow/tool-workflow/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-tool-workflow", "description": "Model-facing workflow tool: run a JavaScript orchestration script over ctx.workflowEngine", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/workflow/workflow-worker-thread/README.i18n.yaml b/packages/workflow/workflow-worker-thread/README.i18n.yaml index 19bb92d976..c761f76437 100644 --- a/packages/workflow/workflow-worker-thread/README.i18n.yaml +++ b/packages/workflow/workflow-worker-thread/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/workflow/workflow-worker-thread/README.md README.md: ba32c9dab4e870dd52c8f8acba3cfa627fa78000 -README.zh.md: 99107aec40a45c8460f10082f2b75b908e3e8692 +README.zh.md: b4c30eac3221b7091b2fc5d73149c244f5abf173 diff --git a/packages/workflow/workflow-worker-thread/README.zh.md b/packages/workflow/workflow-worker-thread/README.zh.md index 99107aec40..b4c30eac32 100644 --- a/packages/workflow/workflow-worker-thread/README.zh.md +++ b/packages/workflow/workflow-worker-thread/README.zh.md @@ -105,7 +105,7 @@ worker 错误、消息失败或提前退出会在清理前关闭消息接纳, #### 模型看到的内容 -通过 [`dsh-tool-workflow`](../tool-workflow/README.md),成功结果只会在该消费方的包装层中公开实体化的最终 JSON 值和子 agent 数量。本引擎提供稳定错误,包括 `workflow script does not parse: `、`invalid meta: `、`agent() requires a non-empty prompt string`、`agent() could not start a child: `、`child agent run failed: `,以及其精确的 `parallel()`、`pipeline()`、`phase()`、选项、schema 和 JSON 边界校验消息。中间子 agent 输出可供脚本使用,但不提供给父模型。 +通过 [`dsh-tool-workflow`](../tool-workflow/README.zh.md),成功结果只会在该消费方的包装层中公开实体化的最终 JSON 值和子 agent 数量。本引擎提供稳定错误,包括 `workflow script does not parse: `、`invalid meta: `、`agent() requires a non-empty prompt string`、`agent() could not start a child: `、`child agent run failed: `,以及其精确的 `parallel()`、`pipeline()`、`phase()`、选项、schema 和 JSON 边界校验消息。中间子 agent 输出可供脚本使用,但不提供给父模型。 #### Token 影响 diff --git a/packages/workflow/workflow-worker-thread/package.json b/packages/workflow/workflow-worker-thread/package.json index 4f32c42493..0a3713f7de 100644 --- a/packages/workflow/workflow-worker-thread/package.json +++ b/packages/workflow/workflow-worker-thread/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-workflow-worker-thread", "description": "worker-thread workflow engine: executes model-written orchestration scripts off the host event loop, bridging agent() calls back to ctx.subagents", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/workflow/workflow-worker-thread/tests/session.spec.ts b/packages/workflow/workflow-worker-thread/tests/session.spec.ts index b856a48785..5b323d1638 100644 --- a/packages/workflow/workflow-worker-thread/tests/session.spec.ts +++ b/packages/workflow/workflow-worker-thread/tests/session.spec.ts @@ -47,9 +47,7 @@ interface FakeHostOptions { /** * Drive runWorkerSession IN-PROCESS over a MessageChannel: this is where the * worker-side files earn their coverage — code inside a real Worker is - * invisible to main-process coverage. The fake host mirrors the real host's - * protocol discipline (one started/start-error per start; settled/disposed - * follow). + * invisible to main-process coverage. */ function fakeHost(options?: FakeHostOptions): FakeHost { const channel = new MessageChannel() @@ -238,7 +236,6 @@ describe('runWorkerSession over an in-process MessageChannel', () => { const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' }) host.send({ type: HostToWorkerType.Cancel, reason: 'stop everything' }) - // The real host settles the aborted child; mirror it. host.send({ type: HostToWorkerType.ChildSettled, callId, result: { output: [], stopReason: 'aborted' } }) const result = await host.result() expect(result.stopReason).toBe('cancelled') diff --git a/packages/workflow/workflow/README.i18n.yaml b/packages/workflow/workflow/README.i18n.yaml index e8f1900e7c..4ed7c36d13 100644 --- a/packages/workflow/workflow/README.i18n.yaml +++ b/packages/workflow/workflow/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/workflow/workflow/README.md README.md: cc2c24f62512273ea1657542d0d904b0b870e236 -README.zh.md: 22bda1bd33bc6350c8ff51f27538258bee872fdb +README.zh.md: 0b02cea896b3cb64559ce110c0da2478728f513c diff --git a/packages/workflow/workflow/README.zh.md b/packages/workflow/workflow/README.zh.md index 22bda1bd33..0b02cea896 100644 --- a/packages/workflow/workflow/README.zh.md +++ b/packages/workflow/workflow/README.zh.md @@ -58,4 +58,4 @@ - **没有 token 预算词汇**:引擎会限制并发、条目和子 agent,但请求与结果都不会统计跨子 agent 的模型 token。 - **运行由持有方负责,不由服务跟踪**:卸载引擎不会发现独立的活动句柄;每个消费方都必须 dispose 自己启动的运行。 -暂缓实现的工作流接口见[动态工作流 Agent Note(agent 决策记录)](../../../.agents/notes/implemented/feature/2026-07-05-dynamic-workflows.md)。 +暂缓实现的工作流接口见[动态工作流 Agent Note(agent 决策记录)](../../../.agents/notes/implemented/feature/2026-07-05-dynamic-workflows.zh.md)。 diff --git a/packages/workflow/workflow/package.json b/packages/workflow/workflow/package.json index 1b689da55a..17ff2d2959 100644 --- a/packages/workflow/workflow/package.json +++ b/packages/workflow/workflow/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-workflow", "description": "Workflow capability seam: ctx.workflowEngine service, run vocabulary, and workflow/* events", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/workspace/README.i18n.yaml b/packages/workspace/README.i18n.yaml index 42e824497a..b8e94e910c 100644 --- a/packages/workspace/README.i18n.yaml +++ b/packages/workspace/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/workspace/README.md README.md: d12eb9434ec092834a7c34e424edc1d003430b35 -README.zh.md: 796c48cdbd1916c06e9bb914c6a5149769a58b4e +README.zh.md: 220a18cfeb6ecb9569dfd2bce18bafe6c7aac879 diff --git a/packages/workspace/README.zh.md b/packages/workspace/README.zh.md index 796c48cdbd..220a18cfeb 100644 --- a/packages/workspace/README.zh.md +++ b/packages/workspace/README.zh.md @@ -6,8 +6,8 @@ | 包 | 职责 | ctx 键 | |---|---|---| -| [`workspace/`](workspace/README.md) | 注册 workspace 并记录其会话归属 | `ctx.workspaceRegistry` | +| [`workspace/`](workspace/README.zh.md) | 注册 workspace 并记录其会话归属 | `ctx.workspaceRegistry` | -[workspace 包参考](workspace/README.md)负责生命周期、持久化和删除语义。 +[workspace 包参考](workspace/README.zh.md)负责生命周期、持久化和删除语义。 -子系统参考——实体、realpath 规范、注册/解析——见 [docs/subsystems/workspace.md](../../docs/subsystems/workspace.md);存储设计见 [domain KV 存储 Agent Note](../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.md)。 +子系统参考——实体、realpath 规范、注册/解析——见 [docs/subsystems/workspace.md](../../docs/subsystems/workspace.zh.md);存储设计见 [domain KV 存储 Agent Note](../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md)。 diff --git a/packages/workspace/workspace/README.i18n.yaml b/packages/workspace/workspace/README.i18n.yaml index 20f5d3b867..2c2bd8d4ea 100644 --- a/packages/workspace/workspace/README.i18n.yaml +++ b/packages/workspace/workspace/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/workspace/workspace/README.md README.md: c1c4309efdc85f4bc7a6b0f128df6d9f4f810ae1 -README.zh.md: f44ad04be0c63061200a256c5ce8cb24a4d3c54b +README.zh.md: 4ba66cb5bb83a86cec00b33769f0ac85fef41190 diff --git a/packages/workspace/workspace/README.zh.md b/packages/workspace/workspace/README.zh.md index f44ad04be0..4ba66cb5bb 100644 --- a/packages/workspace/workspace/README.zh.md +++ b/packages/workspace/workspace/README.zh.md @@ -4,7 +4,7 @@ DeepSeek Harness 的 Workspace 实体注册表(`ctx.workspaceRegistry`):通过领域数据形式存储持久 workspace 记录、稳定 workspace 顺序和按新到旧排列的候选会话索引。消费方看到 `Workspace` 接口;实体实现保持包私有。 -实体/存储理由见[领域 Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.md);仅使用头部的引导初始化和 GUI 排序见 [Workspace UI 产品流 Agent Note](../../../.agents/notes/implemented/feature/2026-07-25-workspace-ui-product-flow.md)。 +实体/存储理由见[领域 Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md);仅使用头部的引导初始化和 GUI 排序见 [Workspace UI 产品流 Agent Note](../../../.agents/notes/implemented/feature/2026-07-25-workspace-ui-product-flow.zh.md)。 ## 结构 @@ -40,5 +40,5 @@ DeepSeek Harness 的 Workspace 实体注册表(`ctx.workspaceRegistry`): ## 已知限制与暂缓事项 -- 会话删除与破坏性的文件夹移除是彼此独立且尚未提供的功能;删除 Workspace 注册记录绝不能替代二者(参见[决策记录](../../../.agents/notes/implemented/feature/2026-07-27-workspace-registration-deletion.md))。 +- 会话删除与破坏性的文件夹移除是彼此独立且尚未提供的功能;删除 Workspace 注册记录绝不能替代二者(参见[决策记录](../../../.agents/notes/implemented/feature/2026-07-27-workspace-registration-deletion.zh.md))。 - 头部索引会在启动时刷新,也会在 attach 必须解析未缓存持久 id 时刷新;另一进程执行的删除或造成的 cwd 损坏会在下次刷新或重启后被发现。 diff --git a/packages/workspace/workspace/package.json b/packages/workspace/workspace/package.json index abcd7bc903..6c2fa2782e 100644 --- a/packages/workspace/workspace/package.json +++ b/packages/workspace/workspace/package.json @@ -1,9 +1,9 @@ { "name": "@deepseek-ai/dsh-workspace", "description": "Workspace entity registry (ctx.workspaceRegistry): durable workspace records with validated session attachment over the domain data form for the DeepSeek Harness", - "version": "0.0.1-rc.5", + "version": "0.1.1-rc.2", "publishConfig": { - "access": "restricted" + "access": "public" }, "repository": { "type": "git", diff --git a/packages/workspace/workspace/src/spec.ts b/packages/workspace/workspace/src/spec.ts index ba1b89e39d..abcd29da0b 100644 --- a/packages/workspace/workspace/src/spec.ts +++ b/packages/workspace/workspace/src/spec.ts @@ -1,7 +1,7 @@ /** * The workspace domain declaration: record schema and the `defineDomain` spec - * the registry opens. The zod schema is the durable-boundary validator today - * and the direct source of the RPC wire projection in a later phase. + * the registry opens. The zod schema validates the shipped format at the + * durability boundary and is the direct source of a future RPC wire projection. * @module @deepseek-ai/dsh-workspace/src/spec */ diff --git a/patches/node-pty@1.1.0.patch b/patches/node-pty@1.1.0.patch deleted file mode 100644 index 56892a3d58..0000000000 --- a/patches/node-pty@1.1.0.patch +++ /dev/null @@ -1,62 +0,0 @@ -diff --git a/lib/unixTerminal.js b/lib/unixTerminal.js -index 1ec12f796a822c78fba9ad7f6448c3987e325c23..5cd6b7d635f4752be5a6c5ff9cf9edf988cf94c5 100644 ---- a/lib/unixTerminal.js -+++ b/lib/unixTerminal.js -@@ -26,10 +26,23 @@ var terminal_1 = require("./terminal"); - var utils_1 = require("./utils"); - var native = utils_1.loadNativeModule('pty'); - var pty = native.module; --var helperPath = native.dir + '/spawn-helper'; --helperPath = path.resolve(__dirname, helperPath); --helperPath = helperPath.replace('app.asar', 'app.asar.unpacked'); --helperPath = helperPath.replace('node_modules.asar', 'node_modules.asar.unpacked'); -+// A current external embedded-runtime consumer supplies a non-sibling helper. -+var helperPath = process.env.DSH_NODE_PTY_SPAWN_HELPER; -+if (helperPath) { -+ helperPath = path.resolve(helperPath); -+} -+else { -+ var executableSibling = process.execPath + '-spawn-helper'; -+ if (fs.existsSync(executableSibling)) { -+ helperPath = executableSibling; -+ } -+ else { -+ helperPath = native.dir + '/spawn-helper'; -+ helperPath = path.resolve(__dirname, helperPath); -+ helperPath = helperPath.replace('app.asar', 'app.asar.unpacked'); -+ helperPath = helperPath.replace('node_modules.asar', 'node_modules.asar.unpacked'); -+ } -+} - var DEFAULT_FILE = 'sh'; - var DEFAULT_NAME = 'xterm'; - var DESTROY_SOCKET_TIMEOUT_MS = 200; -diff --git a/src/unixTerminal.ts b/src/unixTerminal.ts -index 98733dc0cd752b554bd94e45904ca341ad141bba..fa234291206617ae5a6d8605abf9771220392d17 100644 ---- a/src/unixTerminal.ts -+++ b/src/unixTerminal.ts -@@ -14,10 +14,21 @@ import { assign, loadNativeModule } from './utils'; - - const native = loadNativeModule('pty'); - const pty: IUnixNative = native.module; --let helperPath = native.dir + '/spawn-helper'; --helperPath = path.resolve(__dirname, helperPath); --helperPath = helperPath.replace('app.asar', 'app.asar.unpacked'); --helperPath = helperPath.replace('node_modules.asar', 'node_modules.asar.unpacked'); -+// A current external embedded-runtime consumer supplies a non-sibling helper. -+let helperPath = process.env.DSH_NODE_PTY_SPAWN_HELPER; -+if (helperPath) { -+ helperPath = path.resolve(helperPath); -+} else { -+ const executableSibling = process.execPath + '-spawn-helper'; -+ if (fs.existsSync(executableSibling)) { -+ helperPath = executableSibling; -+ } else { -+ helperPath = native.dir + '/spawn-helper'; -+ helperPath = path.resolve(__dirname, helperPath); -+ helperPath = helperPath.replace('app.asar', 'app.asar.unpacked'); -+ helperPath = helperPath.replace('node_modules.asar', 'node_modules.asar.unpacked'); -+ } -+} - - const DEFAULT_FILE = 'sh'; - const DEFAULT_NAME = 'xterm'; diff --git a/patches/node-pty@1.2.0-beta.15.patch b/patches/node-pty@1.2.0-beta.15.patch new file mode 100644 index 0000000000..74eecb16cd --- /dev/null +++ b/patches/node-pty@1.2.0-beta.15.patch @@ -0,0 +1,32 @@ +diff --git a/lib/unixTerminal.js b/lib/unixTerminal.js +index 6966d24..18f1d25 100644 +--- a/lib/unixTerminal.js ++++ b/lib/unixTerminal.js +@@ -28,10 +28,23 @@ var terminal_1 = require("./terminal"); + var utils_1 = require("./utils"); + var native = (0, utils_1.loadNativeModule)('pty'); + var pty = native.module; +-var helperPath = native.dir + '/spawn-helper'; +-helperPath = path.resolve(__dirname, helperPath); +-helperPath = helperPath.replace('app.asar', 'app.asar.unpacked'); +-helperPath = helperPath.replace('node_modules.asar', 'node_modules.asar.unpacked'); ++// A current external embedded-runtime consumer supplies a non-sibling helper. ++var helperPath = process.env.DSH_NODE_PTY_SPAWN_HELPER; ++if (helperPath) { ++ helperPath = path.resolve(helperPath); ++} ++else { ++ var executableSibling = process.execPath + '-spawn-helper'; ++ if (fs.existsSync(executableSibling)) { ++ helperPath = executableSibling; ++ } ++ else { ++ helperPath = native.dir + '/spawn-helper'; ++ helperPath = path.resolve(__dirname, helperPath); ++ helperPath = helperPath.replace('app.asar', 'app.asar.unpacked'); ++ helperPath = helperPath.replace('node_modules.asar', 'node_modules.asar.unpacked'); ++ } ++} + var DEFAULT_FILE = 'sh'; + var DEFAULT_NAME = 'xterm'; + var DESTROY_SOCKET_TIMEOUT_MS = 200; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 1490a0f9f7..ddeacead94 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -9,15 +9,12 @@ overrides: '@deepseek-ai/schemastery': link:vendor/schemastery patchedDependencies: - node-pty@1.1.0: 7a0c04f1f49d798a9ffe2f7f414c01064a44ca2489772d0c3e1235ab336755e6 + node-pty@1.2.0-beta.15: b40ae545608897914bd25fb009c97eeac478c34e8a910298ddcb01b746534bb0 importers: .: devDependencies: - '@agentclientprotocol/sdk': - specifier: 0.25.1 - version: 0.25.1(zod@4.4.3) '@deepseek-ai/dsh-tool-session-query': specifier: workspace:^ version: link:packages/session-query/tool-session-query @@ -71,7 +68,7 @@ importers: version: 5.0.12 jsdom: specifier: 29.1.1 - version: 29.1.1 + version: 29.1.1(@noble/hashes@2.3.0) knip: specifier: ^6.16.1 version: 6.16.1 @@ -122,7 +119,7 @@ importers: version: 6.1.1(typescript@6.0.3)(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) vitest: specifier: ^4.1.8 - version: 4.1.8(@opentelemetry/api@1.9.1)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) + version: 4.1.8(@opentelemetry/api@1.9.1)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1(@noble/hashes@2.3.0))(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) apps/cli: dependencies: @@ -141,6 +138,9 @@ importers: '@deepseek-ai/cordis-plugin-timer': specifier: workspace:^ version: link:../../vendor/timer + '@deepseek-ai/dsh-acp-app': + specifier: workspace:^ + version: link:../../packages/bundle/acp-app '@deepseek-ai/dsh-agent-instructions': specifier: workspace:^ version: link:../../packages/context/agent-instructions @@ -192,6 +192,12 @@ importers: '@deepseek-ai/dsh-home-paths': specifier: workspace:^ version: link:../../packages/util/home-paths + '@deepseek-ai/dsh-hooks-claude-code': + specifier: workspace:^ + version: link:../../packages/hooks/hooks-claude-code + '@deepseek-ai/dsh-hooks-codex': + specifier: workspace:^ + version: link:../../packages/hooks/hooks-codex '@deepseek-ai/dsh-jobs-local': specifier: workspace:^ version: link:../../packages/jobs/jobs-local @@ -216,6 +222,9 @@ importers: '@deepseek-ai/dsh-schedule': specifier: workspace:^ version: link:../../packages/schedule/schedule + '@deepseek-ai/dsh-sdk-app': + specifier: workspace:^ + version: link:../../packages/bundle/sdk-app '@deepseek-ai/dsh-session-projection': specifier: workspace:^ version: link:../../packages/session/session-projection @@ -270,6 +279,9 @@ importers: '@deepseek-ai/dsh-tool-pwsh': specifier: workspace:^ version: link:../../packages/shell/tool-pwsh + '@deepseek-ai/dsh-tool-pwsh-persistent': + specifier: workspace:^ + version: link:../../packages/shell/tool-pwsh-persistent '@deepseek-ai/dsh-tool-ralph': specifier: workspace:^ version: link:../../packages/workflow/tool-ralph @@ -297,6 +309,12 @@ importers: '@deepseek-ai/dsh-web-app': specifier: workspace:^ version: link:../../packages/bundle/web-app + '@deepseek-ai/dsh-webhook': + specifier: workspace:^ + version: link:../../packages/webhook/webhook + '@deepseek-ai/dsh-webhook-github': + specifier: workspace:^ + version: link:../../packages/webhook/webhook-github '@deepseek-ai/dsh-workflow-worker-thread': specifier: workspace:^ version: link:../../packages/workflow/workflow-worker-thread @@ -310,6 +328,9 @@ importers: specifier: ^0.1.4 version: 0.1.4 devDependencies: + '@agentclientprotocol/sdk': + specifier: 1.4.0 + version: 1.4.0(zod@4.4.3) '@deepseek-ai/dsh-agent': specifier: workspace:^ version: link:../../packages/core/agent @@ -354,16 +375,6 @@ importers: version: 10.0.0 apps/web: - dependencies: - '@deepseek-ai/dsh-client-web': - specifier: workspace:^ - version: link:../../packages/client/web - react: - specifier: ^18.2.0 - version: 18.3.1 - react-dom: - specifier: ^18.2.0 - version: 18.3.1(react@18.3.1) devDependencies: '@deepseek-ai/cordis-plugin-group': specifier: workspace:^ @@ -377,12 +388,18 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../../packages/client/ui-slots - '@deepseek-ai/dsh-client-web-react': + '@deepseek-ai/dsh-client-web': specifier: workspace:^ - version: link:../../packages/client/web-react + version: link:../../packages/client/web '@deepseek-ai/dsh-cmdline': specifier: workspace:^ version: link:../../packages/boot/cmdline + '@deepseek-ai/dsh-experimental-webworker-packer': + specifier: workspace:^ + version: link:../../packages/experimental/webworker-packer + '@deepseek-ai/dsh-experimental-webworker-runtime': + specifier: workspace:^ + version: link:../../packages/experimental/webworker-runtime '@deepseek-ai/dsh-pwsh-local': specifier: workspace:^ version: link:../../packages/shell/pwsh-local @@ -401,9 +418,18 @@ importers: fflate: specifier: ^0.8.2 version: 0.8.3 + http-server: + specifier: ^14.1.1 + version: 14.1.1 playwright: specifier: ^1.49.0 version: 1.61.1 + react: + specifier: ^18.2.0 + version: 18.3.1 + react-dom: + specifier: ^18.2.0 + version: 18.3.1(react@18.3.1) typescript: specifier: ^6.0.3 version: 6.0.3 @@ -412,10 +438,13 @@ importers: version: 6.4.3(@types/node@22.20.0)(jiti@2.7.0)(lightningcss@1.32.0)(tsx@4.22.4)(yaml@2.9.0) vitest: specifier: ^4.1.8 - version: 4.1.8(@opentelemetry/api@1.9.0)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@6.4.3(@types/node@22.20.0)(jiti@2.7.0)(lightningcss@1.32.0)(tsx@4.22.4)(yaml@2.9.0)) + version: 4.1.8(@opentelemetry/api@1.9.0)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1(@noble/hashes@2.3.0))(vite@6.4.3(@types/node@22.20.0)(jiti@2.7.0)(lightningcss@1.32.0)(tsx@4.22.4)(yaml@2.9.0)) examples: dependencies: + '@agentclientprotocol/sdk': + specifier: 1.4.0 + version: 1.4.0(zod@4.4.3) '@deepseek-ai/cordis-plugin-hmr': specifier: workspace:* version: link:../vendor/hmr @@ -428,9 +457,9 @@ importers: '@deepseek-ai/cordis-plugin-timer': specifier: workspace:* version: link:../vendor/timer - '@deepseek-ai/dsh-acp-demo': + '@deepseek-ai/dsh-acp': specifier: workspace:* - version: link:../packages/examples/acp-demo + version: link:../packages/acp/acp '@deepseek-ai/dsh-agent': specifier: workspace:* version: link:../packages/core/agent @@ -482,9 +511,18 @@ importers: '@deepseek-ai/dsh-credentials-local': specifier: workspace:* version: link:../packages/credentials/credentials-local + '@deepseek-ai/dsh-deepseek-llm-api-extensions': + specifier: workspace:* + version: link:../packages/llm/deepseek-llm-api-extensions '@deepseek-ai/dsh-e2b': specifier: workspace:* version: link:../packages/e2b/e2b + '@deepseek-ai/dsh-experimental-agent-team': + specifier: workspace:* + version: link:../packages/experimental/agent-team + '@deepseek-ai/dsh-experimental-tool-agent-team': + specifier: workspace:* + version: link:../packages/experimental/tool-agent-team '@deepseek-ai/dsh-fs-e2b': specifier: workspace:* version: link:../packages/e2b/fs-e2b @@ -542,9 +580,15 @@ importers: '@deepseek-ai/dsh-plan-mode': specifier: workspace:* version: link:../packages/plan/plan-mode + '@deepseek-ai/dsh-plugin-package-inventory-deepseek': + specifier: workspace:* + version: link:../packages/llm/plugin-package-inventory-deepseek '@deepseek-ai/dsh-pwsh-local': specifier: workspace:* version: link:../packages/shell/pwsh-local + '@deepseek-ai/dsh-pwsh-sandbox': + specifier: workspace:* + version: link:../packages/shell/pwsh-sandbox '@deepseek-ai/dsh-repeat-tool-reminder': specifier: workspace:* version: link:../packages/guard/repeat-tool-reminder @@ -569,6 +613,9 @@ importers: '@deepseek-ai/dsh-session-checkpoint-policy': specifier: workspace:* version: link:../packages/session/session-checkpoint-policy + '@deepseek-ai/dsh-session-log-deepseek': + specifier: workspace:* + version: link:../packages/session/session-log-deepseek '@deepseek-ai/dsh-session-persistence-jsonl': specifier: workspace:* version: link:../packages/session/session-persistence-jsonl @@ -689,6 +736,9 @@ importers: '@deepseek-ai/dsh-tool-pwsh': specifier: workspace:* version: link:../packages/shell/tool-pwsh + '@deepseek-ai/dsh-tool-pwsh-persistent': + specifier: workspace:* + version: link:../packages/shell/tool-pwsh-persistent '@deepseek-ai/dsh-tool-ralph': specifier: workspace:* version: link:../packages/workflow/tool-ralph @@ -737,9 +787,18 @@ importers: '@deepseek-ai/dsh-web-fetch-http': specifier: workspace:* version: link:../packages/web/web-fetch-http + '@deepseek-ai/dsh-webhook': + specifier: workspace:* + version: link:../packages/webhook/webhook + '@deepseek-ai/dsh-webhook-github': + specifier: workspace:* + version: link:../packages/webhook/webhook-github '@deepseek-ai/dsh-workflow-worker-thread': specifier: workspace:* version: link:../packages/workflow/workflow-worker-thread + '@deepseek-ai/schemastery': + specifier: link:../vendor/schemastery + version: link:../vendor/schemastery native/landlock-run: devDependencies: @@ -772,8 +831,8 @@ importers: packages/acp/acp: dependencies: '@agentclientprotocol/sdk': - specifier: 0.25.1 - version: 0.25.1(zod@4.4.3) + specifier: 1.4.0 + version: 1.4.0(zod@4.4.3) '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -790,15 +849,30 @@ importers: '@deepseek-ai/dsh-agent-loop-testkit': specifier: workspace:^ version: link:../../test-support/agent-loop-testkit + '@deepseek-ai/dsh-attachment': + specifier: workspace:^ + version: link:../../attachment/attachment '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm + '@deepseek-ai/dsh-mcp-client': + specifier: workspace:^ + version: link:../../mcp/mcp-client '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session + '@deepseek-ai/dsh-session-persistence': + specifier: workspace:^ + version: link:../../session/session-persistence + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:^ + version: link:../../session/session-persistence-jsonl + '@deepseek-ai/dsh-token-meter': + specifier: workspace:^ + version: link:../../llm/token-meter '@deepseek-ai/dsh-tools': specifier: workspace:^ version: link:../../core/tools @@ -858,6 +932,9 @@ importers: '@deepseek-ai/dsh-credentials': specifier: workspace:^ version: link:../../credentials/credentials + '@deepseek-ai/dsh-file-reference': + specifier: workspace:^ + version: link:../../context/file-reference '@deepseek-ai/dsh-goal': specifier: workspace:^ version: link:../../goal/goal @@ -879,6 +956,9 @@ importers: '@deepseek-ai/dsh-session-persistence': specifier: workspace:^ version: link:../../session/session-persistence + '@deepseek-ai/dsh-session-reference': + specifier: workspace:^ + version: link:../../context/session-reference '@deepseek-ai/dsh-settings': specifier: workspace:^ version: link:../../settings/settings @@ -978,6 +1058,28 @@ importers: specifier: ^15.0.0 version: 15.0.0 + packages/bundle/acp-app: + dependencies: + '@deepseek-ai/dsh-acp': + specifier: workspace:^ + version: link:../../acp/acp + '@deepseek-ai/dsh-cmdline': + specifier: workspace:^ + version: link:../../boot/cmdline + commander: + specifier: ^15.0.0 + version: 15.0.0 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + packages/bundle/base: dependencies: '@deepseek-ai/cordis-plugin-hmr': @@ -1028,6 +1130,9 @@ importers: '@deepseek-ai/dsh-credentials-local': specifier: workspace:^ version: link:../../credentials/credentials-local + '@deepseek-ai/dsh-deepseek-llm-api-extensions': + specifier: workspace:^ + version: link:../../llm/deepseek-llm-api-extensions '@deepseek-ai/dsh-fs-local': specifier: workspace:^ version: link:../../fs/fs-local @@ -1064,6 +1169,9 @@ importers: '@deepseek-ai/dsh-plan-mode': specifier: workspace:^ version: link:../../plan/plan-mode + '@deepseek-ai/dsh-plugin-package-inventory-deepseek': + specifier: workspace:^ + version: link:../../llm/plugin-package-inventory-deepseek '@deepseek-ai/dsh-pwsh-sandbox': specifier: workspace:^ version: link:../../shell/pwsh-sandbox @@ -1082,6 +1190,9 @@ importers: '@deepseek-ai/dsh-session-checkpoint-policy': specifier: workspace:^ version: link:../../session/session-checkpoint-policy + '@deepseek-ai/dsh-session-log-deepseek': + specifier: workspace:^ + version: link:../../session/session-log-deepseek '@deepseek-ai/dsh-session-persistence-jsonl': specifier: workspace:^ version: link:../../session/session-persistence-jsonl @@ -1256,6 +1367,28 @@ importers: specifier: workspace:^ version: link:../../core/session + packages/bundle/sdk-app: + dependencies: + '@deepseek-ai/dsh-cmdline': + specifier: workspace:^ + version: link:../../boot/cmdline + '@deepseek-ai/dsh-sdk-jsonrpc-server': + specifier: workspace:^ + version: link:../../sdk/server + commander: + specifier: ^15.0.0 + version: 15.0.0 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + packages/bundle/web-app: dependencies: '@deepseek-ai/dsh-agent-presets': @@ -1285,6 +1418,12 @@ importers: '@deepseek-ai/dsh-client-ui-agent-preset': specifier: workspace:^ version: link:../../client/ui-agent-preset + '@deepseek-ai/dsh-client-ui-attachment': + specifier: workspace:^ + version: link:../../client/ui-attachment + '@deepseek-ai/dsh-client-ui-brand-official': + specifier: workspace:^ + version: link:../../client/ui-brand-official '@deepseek-ai/dsh-client-ui-commands': specifier: workspace:^ version: link:../../client/ui-commands @@ -1327,6 +1466,12 @@ importers: '@deepseek-ai/dsh-client-ui-plan': specifier: workspace:^ version: link:../../client/ui-plan + '@deepseek-ai/dsh-client-ui-reference': + specifier: workspace:^ + version: link:../../client/ui-reference + '@deepseek-ai/dsh-client-ui-renderer': + specifier: workspace:^ + version: link:../../client/ui-renderer '@deepseek-ai/dsh-client-ui-settings': specifier: workspace:^ version: link:../../client/ui-settings @@ -1381,6 +1526,12 @@ importers: '@deepseek-ai/dsh-cordis-host-runner': specifier: workspace:^ version: link:../../extensions/cordis-host-runner + '@deepseek-ai/dsh-file-reference': + specifier: workspace:^ + version: link:../../context/file-reference + '@deepseek-ai/dsh-file-reference-local': + specifier: workspace:^ + version: link:../../context/file-reference-local '@deepseek-ai/dsh-host-apiproxy': specifier: workspace:^ version: link:../../host/apiproxy @@ -1402,6 +1553,9 @@ importers: '@deepseek-ai/dsh-host-webserver': specifier: workspace:^ version: link:../../host/webserver + '@deepseek-ai/dsh-launch-environment': + specifier: workspace:^ + version: link:../../util/launch-environment '@deepseek-ai/dsh-message-feedback': specifier: workspace:^ version: link:../../feedback/message-feedback @@ -1411,6 +1565,9 @@ importers: '@deepseek-ai/dsh-session-projection-cache': specifier: workspace:^ version: link:../../session/session-projection-cache + '@deepseek-ai/dsh-session-reference': + specifier: workspace:^ + version: link:../../context/session-reference '@deepseek-ai/dsh-session-stats': specifier: workspace:^ version: link:../../session/session-stats @@ -1423,6 +1580,9 @@ importers: '@deepseek-ai/dsh-storage-json': specifier: workspace:^ version: link:../../storage/storage-json + '@deepseek-ai/dsh-subprocess': + specifier: workspace:^ + version: link:../../subprocess/subprocess '@deepseek-ai/dsh-web-frontend': specifier: workspace:^ version: link:../../../apps/web @@ -1435,6 +1595,9 @@ importers: commander: specifier: ^15.0.0 version: 15.0.0 + open: + specifier: ^11.0.0 + version: 11.0.0 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -1454,24 +1617,6 @@ importers: packages/client/connection: dependencies: - '@deepseek-ai/dsh-attachment': - specifier: workspace:^ - version: link:../../attachment/attachment - '@deepseek-ai/dsh-commands': - specifier: workspace:^ - version: link:../../interaction/commands - '@deepseek-ai/dsh-host-apiproxy': - specifier: workspace:^ - version: link:../../host/apiproxy - '@deepseek-ai/dsh-llm': - specifier: workspace:^ - version: link:../../llm/llm - '@deepseek-ai/dsh-session': - specifier: workspace:^ - version: link:../../core/session - '@deepseek-ai/dsh-tools': - specifier: workspace:^ - version: link:../../core/tools '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -1482,12 +1627,33 @@ importers: '@deepseek-ai/cordis': specifier: workspace:^ version: link:../../../vendor/cordis + '@deepseek-ai/dsh-attachment': + specifier: workspace:^ + version: link:../../attachment/attachment + '@deepseek-ai/dsh-commands': + specifier: workspace:^ + version: link:../../interaction/commands + '@deepseek-ai/dsh-host-apiproxy': + specifier: workspace:^ + version: link:../../host/apiproxy '@deepseek-ai/dsh-host-webserver': specifier: workspace:^ version: link:../../host/webserver '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-tool-todo': + specifier: workspace:^ + version: link:../../todo/tool-todo + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../../core/tools '@types/ws': specifier: ^8.18.1 version: 8.18.1 @@ -1516,12 +1682,6 @@ importers: packages/client/locale: dependencies: - '@deepseek-ai/dsh-client-connection': - specifier: workspace:^ - version: link:../connection - '@deepseek-ai/dsh-settings': - specifier: workspace:^ - version: link:../../settings/settings '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -1532,9 +1692,15 @@ importers: '@deepseek-ai/dsh-api-remotes': specifier: workspace:^ version: link:../../api/remotes + '@deepseek-ai/dsh-client-connection': + specifier: workspace:^ + version: link:../connection '@deepseek-ai/dsh-client-runtime': specifier: workspace:^ version: link:../runtime + '@deepseek-ai/dsh-client-test-runtime': + specifier: workspace:^ + version: link:../../test-support/client-runtime '@deepseek-ai/dsh-client-ui-primitives': specifier: workspace:^ version: link:../ui-primitives @@ -1547,6 +1713,9 @@ importers: '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-settings': + specifier: workspace:^ + version: link:../../settings/settings '@types/react': specifier: ~18.3.1 version: 18.3.31 @@ -1571,9 +1740,22 @@ importers: packages/client/runtime: dependencies: + immer: + specifier: ^10.1.1 + version: 10.2.0 + zustand: + specifier: ~4.4.7 + version: 4.4.7(@types/react@18.3.31)(immer@10.2.0)(react@18.3.1) + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis '@deepseek-ai/dsh-agent': specifier: workspace:^ version: link:../../core/agent + '@deepseek-ai/dsh-api-remotes': + specifier: workspace:^ + version: link:../../api/remotes '@deepseek-ai/dsh-attachment': specifier: workspace:^ version: link:../../attachment/attachment @@ -1589,6 +1771,9 @@ importers: '@deepseek-ai/dsh-host-apiproxy': specifier: workspace:^ version: link:../../host/apiproxy + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm @@ -1604,31 +1789,15 @@ importers: '@deepseek-ai/dsh-session-title': specifier: workspace:^ version: link:../../session/session-title - '@deepseek-ai/dsh-tools': - specifier: workspace:^ - version: link:../../core/tools - immer: - specifier: ^10.1.1 - version: 10.2.0 - react: - specifier: ^18.2.0 - version: 18.3.1 - zustand: - specifier: ~4.4.7 - version: 4.4.7(@types/react@18.3.31)(immer@10.2.0)(react@18.3.1) - devDependencies: - '@deepseek-ai/cordis': - specifier: workspace:^ - version: link:../../../vendor/cordis - '@deepseek-ai/dsh-api-remotes': - specifier: workspace:^ - version: link:../../api/remotes - '@deepseek-ai/dsh-invariants': - specifier: workspace:^ - version: link:../../runtime-diagnostics/invariants '@deepseek-ai/dsh-timeout': specifier: workspace:^ version: link:../../util/timeout + '@deepseek-ai/dsh-tool-todo': + specifier: workspace:^ + version: link:../../todo/tool-todo + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../../core/tools '@deepseek-ai/dsh-typert-protocol': specifier: workspace:^ version: link:../../typert/protocol @@ -1638,19 +1807,9 @@ importers: '@types/react': specifier: ~18.3.1 version: 18.3.31 - - packages/client/schema-form: - dependencies: - '@deepseek-ai/schemastery': - specifier: link:../../../vendor/schemastery - version: link:../../../vendor/schemastery - devDependencies: - '@deepseek-ai/cordis': - specifier: workspace:^ - version: link:../../../vendor/cordis - '@deepseek-ai/dsh-invariants': - specifier: workspace:^ - version: link:../../runtime-diagnostics/invariants + react: + specifier: ^18.2.0 + version: 18.3.1 packages/client/ui-agent-preset: devDependencies: @@ -1684,9 +1843,6 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots - '@deepseek-ai/dsh-client-web-react': - specifier: workspace:^ - version: link:../web-react '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -1699,25 +1855,28 @@ importers: packages/client/ui-attachment: dependencies: - '@deepseek-ai/dsh-attachment': - specifier: workspace:^ - version: link:../../attachment/attachment - '@deepseek-ai/dsh-client-ui-primitives': - specifier: workspace:^ - version: link:../ui-primitives clsx: specifier: ^2.0.0 version: 2.1.1 - react: - specifier: ^18.2.0 - version: 18.3.1 - react-dom: - specifier: ^18.2.0 - version: 18.3.1(react@18.3.1) devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ version: link:../../../vendor/cordis + '@deepseek-ai/dsh-attachment': + specifier: workspace:^ + version: link:../../attachment/attachment + '@deepseek-ai/dsh-client-runtime': + specifier: workspace:^ + version: link:../runtime + '@deepseek-ai/dsh-client-ui-conversation': + specifier: workspace:^ + version: link:../ui-conversation + '@deepseek-ai/dsh-client-ui-primitives': + specifier: workspace:^ + version: link:../ui-primitives + '@deepseek-ai/dsh-client-ui-slots': + specifier: workspace:^ + version: link:../ui-slots '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -1727,6 +1886,45 @@ importers: '@types/react-dom': specifier: ~18.3.0 version: 18.3.7(@types/react@18.3.31) + react: + specifier: ^18.2.0 + version: 18.3.1 + react-dom: + specifier: ^18.2.0 + version: 18.3.1(react@18.3.1) + + packages/client/ui-brand-official: + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-client-runtime': + specifier: workspace:^ + version: link:../runtime + '@deepseek-ai/dsh-client-ui-conversation': + specifier: workspace:^ + version: link:../ui-conversation + '@deepseek-ai/dsh-client-ui-primitives': + specifier: workspace:^ + version: link:../ui-primitives + '@deepseek-ai/dsh-client-ui-sidebar': + specifier: workspace:^ + version: link:../ui-sidebar + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@testing-library/react': + specifier: ^16.1.0 + version: 16.3.2(@testing-library/dom@10.4.1)(@types/react-dom@18.3.7(@types/react@18.3.31))(@types/react@18.3.31)(react-dom@18.3.1(react@18.3.1))(react@18.3.1) + '@types/react': + specifier: ~18.3.1 + version: 18.3.31 + react: + specifier: ^18.2.0 + version: 18.3.1 + react-dom: + specifier: ^18.2.0 + version: 18.3.1(react@18.3.1) packages/client/ui-commands: dependencies: @@ -1779,9 +1977,6 @@ importers: packages/client/ui-conversation: dependencies: - '@deepseek-ai/dsh-settings': - specifier: workspace:^ - version: link:../../settings/settings '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -1816,9 +2011,6 @@ importers: '@deepseek-ai/dsh-client-test-runtime': specifier: workspace:^ version: link:../../test-support/client-runtime - '@deepseek-ai/dsh-client-ui-attachment': - specifier: workspace:^ - version: link:../ui-attachment '@deepseek-ai/dsh-client-ui-input-trigger': specifier: workspace:^ version: link:../ui-input-trigger @@ -1861,6 +2053,9 @@ importers: '@deepseek-ai/dsh-session-stats': specifier: workspace:^ version: link:../../session/session-stats + '@deepseek-ai/dsh-settings': + specifier: workspace:^ + version: link:../../settings/settings '@deepseek-ai/dsh-token-meter': specifier: workspace:^ version: link:../../llm/token-meter @@ -1870,6 +2065,9 @@ importers: '@deepseek-ai/dsh-tools': specifier: workspace:^ version: link:../../core/tools + '@deepseek-ai/dsh-util-crypto': + specifier: workspace:^ + version: link:../../util/crypto '@types/react': specifier: ~18.3.1 version: 18.3.31 @@ -1878,10 +2076,6 @@ importers: version: 18.3.1 packages/client/ui-deliverables: - dependencies: - react: - specifier: ^18.2.0 - version: 18.3.1 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -1901,6 +2095,9 @@ importers: '@deepseek-ai/dsh-client-ui-conversation': specifier: workspace:^ version: link:../ui-conversation + '@deepseek-ai/dsh-client-ui-primitives': + specifier: workspace:^ + version: link:../ui-primitives '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots @@ -1913,6 +2110,9 @@ importers: '@types/react': specifier: ~18.3.1 version: 18.3.31 + react: + specifier: ^18.2.0 + version: 18.3.1 packages/client/ui-directory-picker-browse: dependencies: @@ -1965,9 +2165,6 @@ importers: '@deepseek-ai/dsh-client-runtime': specifier: workspace:^ version: link:../runtime - '@deepseek-ai/dsh-client-ui-slots': - specifier: workspace:^ - version: link:../ui-slots '@deepseek-ai/dsh-client-ui-workspace': specifier: workspace:^ version: link:../ui-workspace @@ -2022,6 +2219,12 @@ importers: '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-typert-protocol': + specifier: workspace:^ + version: link:../../typert/protocol '@testing-library/react': specifier: ^16.1.0 version: 16.3.2(@testing-library/dom@10.4.1)(@types/react-dom@18.3.7(@types/react@18.3.31))(@types/react@18.3.31)(react-dom@18.3.1(react@18.3.1))(react@18.3.1) @@ -2059,6 +2262,9 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots + '@deepseek-ai/dsh-file-reference': + specifier: workspace:^ + version: link:../../context/file-reference '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -2070,10 +2276,6 @@ importers: version: 18.3.1 packages/client/ui-jobs: - dependencies: - react: - specifier: ^18.2.0 - version: 18.3.1 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -2102,6 +2304,9 @@ importers: '@types/react': specifier: ~18.3.1 version: 18.3.31 + react: + specifier: ^18.2.0 + version: 18.3.1 packages/client/ui-layout: devDependencies: @@ -2174,6 +2379,9 @@ importers: '@types/react': specifier: ~18.3.1 version: 18.3.31 + '@types/react-dom': + specifier: ~18.3.0 + version: 18.3.7(@types/react@18.3.31) react: specifier: ^18.2.0 version: 18.3.1 @@ -2182,6 +2390,10 @@ importers: version: 18.3.1(react@18.3.1) packages/client/ui-model-selection: + dependencies: + clsx: + specifier: ^2.1.1 + version: 2.1.1 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -2222,9 +2434,6 @@ importers: '@types/react': specifier: ~18.3.1 version: 18.3.31 - clsx: - specifier: ^2.1.1 - version: 2.1.1 react: specifier: ^18.2.0 version: 18.3.1 @@ -2246,9 +2455,6 @@ importers: '@deepseek-ai/dsh-client-runtime': specifier: workspace:^ version: link:../runtime - '@deepseek-ai/dsh-client-schema-form': - specifier: workspace:^ - version: link:../schema-form '@deepseek-ai/dsh-client-test-runtime': specifier: workspace:^ version: link:../../test-support/client-runtime @@ -2267,9 +2473,6 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots - '@deepseek-ai/dsh-client-web-react': - specifier: workspace:^ - version: link:../web-react '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -2309,9 +2512,6 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots - '@deepseek-ai/dsh-client-web-react': - specifier: workspace:^ - version: link:../web-react '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -2401,11 +2601,7 @@ importers: specifier: ~18.3.0 version: 18.3.7(@types/react@18.3.31) - packages/client/ui-settings: - dependencies: - '@deepseek-ai/dsh-client-connection': - specifier: workspace:^ - version: link:../connection + packages/client/ui-reference: devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -2413,12 +2609,86 @@ importers: '@deepseek-ai/dsh-api-remotes': specifier: workspace:^ version: link:../../api/remotes + '@deepseek-ai/dsh-client-locale': + specifier: workspace:^ + version: link:../locale '@deepseek-ai/dsh-client-runtime': specifier: workspace:^ version: link:../runtime - '@deepseek-ai/dsh-client-schema-form': + '@deepseek-ai/dsh-client-ui-input-trigger': specifier: workspace:^ - version: link:../schema-form + version: link:../ui-input-trigger + '@deepseek-ai/dsh-client-ui-slots': + specifier: workspace:^ + version: link:../ui-slots + '@deepseek-ai/dsh-file-reference': + specifier: workspace:^ + version: link:../../context/file-reference + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-session-reference': + specifier: workspace:^ + version: link:../../context/session-reference + '@deepseek-ai/dsh-typert-protocol': + specifier: workspace:^ + version: link:../../typert/protocol + + packages/client/ui-renderer: + dependencies: + use-sync-external-store: + specifier: 1.2.0 + version: 1.2.0(react@18.3.1) + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-client-runtime': + specifier: workspace:^ + version: link:../runtime + '@deepseek-ai/dsh-client-test-runtime': + specifier: workspace:^ + version: link:../../test-support/client-runtime + '@deepseek-ai/dsh-client-ui-slots': + specifier: workspace:^ + version: link:../ui-slots + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@types/react': + specifier: ~18.3.1 + version: 18.3.31 + '@types/react-dom': + specifier: ~18.3.0 + version: 18.3.7(@types/react@18.3.31) + '@types/use-sync-external-store': + specifier: ^1.5.0 + version: 1.5.0 + react: + specifier: ^18.2.0 + version: 18.3.1 + react-dom: + specifier: ^18.2.0 + version: 18.3.1(react@18.3.1) + + packages/client/ui-settings: + dependencies: + '@deepseek-ai/schemastery': + specifier: link:../../../vendor/schemastery + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-api-remotes': + specifier: workspace:^ + version: link:../../api/remotes + '@deepseek-ai/dsh-client-connection': + specifier: workspace:^ + version: link:../connection + '@deepseek-ai/dsh-client-runtime': + specifier: workspace:^ + version: link:../runtime '@deepseek-ai/dsh-client-test-runtime': specifier: workspace:^ version: link:../../test-support/client-runtime @@ -2440,9 +2710,6 @@ importers: packages/client/ui-settings-general: dependencies: - '@deepseek-ai/dsh-settings': - specifier: workspace:^ - version: link:../../settings/settings '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -2480,12 +2747,12 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots - '@deepseek-ai/dsh-client-web-react': - specifier: workspace:^ - version: link:../web-react '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-settings': + specifier: workspace:^ + version: link:../../settings/settings '@types/react': specifier: ~18.3.1 version: 18.3.31 @@ -2510,9 +2777,6 @@ importers: '@deepseek-ai/dsh-client-runtime': specifier: workspace:^ version: link:../runtime - '@deepseek-ai/dsh-client-schema-form': - specifier: workspace:^ - version: link:../schema-form '@deepseek-ai/dsh-client-test-runtime': specifier: workspace:^ version: link:../../test-support/client-runtime @@ -2525,9 +2789,6 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots - '@deepseek-ai/dsh-client-web-react': - specifier: workspace:^ - version: link:../web-react '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -2613,9 +2874,6 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots - '@deepseek-ai/dsh-client-web-react': - specifier: workspace:^ - version: link:../web-react '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -2724,10 +2982,6 @@ importers: version: 18.3.31 packages/client/ui-subagent: - dependencies: - react: - specifier: ^18.2.0 - version: 18.3.1 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -2765,15 +3019,18 @@ importers: '@types/react': specifier: ~18.3.1 version: 18.3.31 + '@types/react-dom': + specifier: ~18.3.0 + version: 18.3.7(@types/react@18.3.31) + react: + specifier: ^18.2.0 + version: 18.3.1 + react-dom: + specifier: ^18.2.0 + version: 18.3.1(react@18.3.1) packages/client/ui-theme: dependencies: - '@deepseek-ai/dsh-client-connection': - specifier: workspace:^ - version: link:../connection - '@deepseek-ai/dsh-settings': - specifier: workspace:^ - version: link:../../settings/settings '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -2787,6 +3044,9 @@ importers: '@deepseek-ai/dsh-api-remotes': specifier: workspace:^ version: link:../../api/remotes + '@deepseek-ai/dsh-client-connection': + specifier: workspace:^ + version: link:../connection '@deepseek-ai/dsh-client-locale': specifier: workspace:^ version: link:../locale @@ -2811,6 +3071,9 @@ importers: '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-settings': + specifier: workspace:^ + version: link:../../settings/settings '@types/react': specifier: ~18.3.1 version: 18.3.31 @@ -2851,9 +3114,6 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots - '@deepseek-ai/dsh-client-web-react': - specifier: workspace:^ - version: link:../web-react '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -2927,9 +3187,25 @@ importers: packages/client/ui-user-questions: dependencies: + clsx: + specifier: ^2.0.0 + version: 2.1.1 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-api-remotes': + specifier: workspace:^ + version: link:../../api/remotes '@deepseek-ai/dsh-client-connection': specifier: workspace:^ version: link:../connection + '@deepseek-ai/dsh-client-locale': + specifier: workspace:^ + version: link:../locale '@deepseek-ai/dsh-client-runtime': specifier: workspace:^ version: link:../runtime @@ -2942,25 +3218,6 @@ importers: '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../ui-slots - clsx: - specifier: ^2.0.0 - version: 2.1.1 - react: - specifier: ^18.2.0 - version: 18.3.1 - devDependencies: - '@deepseek-ai/cordis': - specifier: workspace:^ - version: link:../../../vendor/cordis - '@deepseek-ai/dsh-agent': - specifier: workspace:^ - version: link:../../core/agent - '@deepseek-ai/dsh-api-remotes': - specifier: workspace:^ - version: link:../../api/remotes - '@deepseek-ai/dsh-client-locale': - specifier: workspace:^ - version: link:../locale '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -2976,12 +3233,11 @@ importers: '@types/react': specifier: ~18.3.1 version: 18.3.31 - - packages/client/ui-workflow-run: - dependencies: react: specifier: ^18.2.0 version: 18.3.1 + + packages/client/ui-workflow-run: devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -3019,6 +3275,9 @@ importers: '@types/react': specifier: ~18.3.1 version: 18.3.31 + react: + specifier: ^18.2.0 + version: 18.3.1 packages/client/ui-workspace: dependencies: @@ -3029,6 +3288,9 @@ importers: '@deepseek-ai/cordis': specifier: workspace:^ version: link:../../../vendor/cordis + '@deepseek-ai/dsh-client-connection': + specifier: workspace:^ + version: link:../connection '@deepseek-ai/dsh-client-locale': specifier: workspace:^ version: link:../locale @@ -3061,34 +3323,6 @@ importers: version: 18.3.1 packages/client/web: - dependencies: - '@deepseek-ai/dsh-client-modules': - specifier: workspace:^ - version: link:../modules - '@deepseek-ai/dsh-client-schema-form': - specifier: workspace:^ - version: link:../schema-form - '@deepseek-ai/dsh-client-ui-attachment': - specifier: workspace:^ - version: link:../ui-attachment - '@deepseek-ai/dsh-client-ui-primitives': - specifier: workspace:^ - version: link:../ui-primitives - '@deepseek-ai/dsh-client-ui-slots': - specifier: workspace:^ - version: link:../ui-slots - '@deepseek-ai/dsh-client-ui-theme': - specifier: workspace:^ - version: link:../ui-theme - '@deepseek-ai/dsh-client-web-react': - specifier: workspace:^ - version: link:../web-react - react: - specifier: ^18.2.0 - version: 18.3.1 - react-dom: - specifier: ^18.2.0 - version: 18.3.1(react@18.3.1) devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -3096,12 +3330,18 @@ importers: '@deepseek-ai/cordis-plugin-loader': specifier: workspace:^ version: link:../../../vendor/loader - '@deepseek-ai/dsh-client-runtime': + '@deepseek-ai/dsh-client-modules': specifier: workspace:^ - version: link:../runtime - '@deepseek-ai/dsh-client-test-runtime': + version: link:../modules + '@deepseek-ai/dsh-client-ui-primitives': specifier: workspace:^ - version: link:../../test-support/client-runtime + version: link:../ui-primitives + '@deepseek-ai/dsh-client-ui-renderer': + specifier: workspace:^ + version: link:../ui-renderer + '@deepseek-ai/dsh-client-ui-slots': + specifier: workspace:^ + version: link:../ui-slots '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -3111,21 +3351,17 @@ importers: '@types/react-dom': specifier: ~18.3.0 version: 18.3.7(@types/react@18.3.31) + react: + specifier: ^18.2.0 + version: 18.3.1 + react-dom: + specifier: ^18.2.0 + version: 18.3.1(react@18.3.1) typescript: specifier: ^6.0.3 version: 6.0.3 - packages/client/web-react: - dependencies: - '@deepseek-ai/dsh-client-ui-slots': - specifier: workspace:^ - version: link:../ui-slots - react: - specifier: ^18.2.0 - version: 18.3.1 - use-sync-external-store: - specifier: 1.2.0 - version: 1.2.0(react@18.3.1) + packages/code-runtime/code-runtime: devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -3133,11 +3369,8 @@ importers: '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants - '@types/react': - specifier: ~18.3.1 - version: 18.3.31 - packages/code-runtime/code-runtime: + packages/code-runtime/code-runtime-python: devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -3351,11 +3584,58 @@ importers: specifier: workspace:^ version: link:../../core/tools + packages/context/file-reference: + dependencies: + zod: + specifier: ^4.4.3 + version: 4.4.3 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-typert-protocol': + specifier: workspace:^ + version: link:../../typert/protocol + + packages/context/file-reference-local: + dependencies: + '@deepseek-ai/schemastery': + specifier: link:../../../vendor/schemastery + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-file-reference': + specifier: workspace:^ + version: link:../file-reference + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../../core/system-prompt + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../../core/tools + packages/context/session-reference: dependencies: '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery + zod: + specifier: ^4.4.3 + version: 4.4.3 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -3381,6 +3661,9 @@ importers: '@deepseek-ai/dsh-session-query': specifier: workspace:^ version: link:../../session-query/session-query + '@deepseek-ai/dsh-typert-protocol': + specifier: workspace:^ + version: link:../../typert/protocol packages/context/time-context: dependencies: @@ -3653,6 +3936,21 @@ importers: specifier: workspace:^ version: link:../../interaction/user-approval + packages/credentials/authorization: + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-credentials': + specifier: workspace:^ + version: link:../credentials + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + packages/credentials/credentials: devDependencies: '@deepseek-ai/cordis': @@ -3755,58 +4053,6 @@ importers: specifier: workspace:^ version: link:../../util/timeout - packages/examples/acp-demo: - dependencies: - '@deepseek-ai/schemastery': - specifier: link:../../../vendor/schemastery - version: link:../../../vendor/schemastery - devDependencies: - '@deepseek-ai/cordis': - specifier: workspace:^ - version: link:../../../vendor/cordis - '@deepseek-ai/cordis-plugin-include': - specifier: workspace:^ - version: link:../../../vendor/include - '@deepseek-ai/cordis-plugin-loader': - specifier: workspace:^ - version: link:../../../vendor/loader - '@deepseek-ai/dsh-acp': - specifier: workspace:^ - version: link:../../acp/acp - '@deepseek-ai/dsh-agent': - specifier: workspace:^ - version: link:../../core/agent - '@deepseek-ai/dsh-agent-instructions': - specifier: workspace:^ - version: link:../../context/agent-instructions - '@deepseek-ai/dsh-agent-spine-demo': - specifier: workspace:^ - version: link:../agent-spine-demo - '@deepseek-ai/dsh-app-boot': - specifier: workspace:^ - version: link:../../boot/app-boot - '@deepseek-ai/dsh-invariants': - specifier: workspace:^ - version: link:../../runtime-diagnostics/invariants - '@deepseek-ai/dsh-session-checkpoint-policy': - specifier: workspace:^ - version: link:../../session/session-checkpoint-policy - '@deepseek-ai/dsh-session-persistence-jsonl': - specifier: workspace:^ - version: link:../../session/session-persistence-jsonl - '@deepseek-ai/dsh-session-query': - specifier: workspace:^ - version: link:../../session-query/session-query - '@deepseek-ai/dsh-session-query-sqlite': - specifier: workspace:^ - version: link:../../session-query/session-query-sqlite - '@deepseek-ai/dsh-system-prompt': - specifier: workspace:^ - version: link:../../core/system-prompt - '@deepseek-ai/dsh-tools': - specifier: workspace:^ - version: link:../../core/tools - packages/examples/agent-spine-demo: dependencies: '@deepseek-ai/schemastery': @@ -3919,11 +4165,127 @@ importers: specifier: workspace:^ version: link:../../../native/landlock-run/packages/entry - packages/examples/jsonrpc-demo: + packages/experimental/agent-team: dependencies: - '@deepseek-ai/dsh-app-boot': + '@deepseek-ai/schemastery': + specifier: link:../../../vendor/schemastery + version: link:../../../vendor/schemastery + zod: + specifier: ^4.4.3 + version: 4.4.3 + devDependencies: + '@deepseek-ai/cordis': specifier: workspace:^ - version: link:../../boot/app-boot + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-agent-loop': + specifier: workspace:^ + version: link:../../core/agent-loop + '@deepseek-ai/dsh-agent-loop-testkit': + specifier: workspace:^ + version: link:../../test-support/agent-loop-testkit + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-session-persistence': + specifier: workspace:^ + version: link:../../session/session-persistence + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:^ + version: link:../../session/session-persistence-jsonl + '@deepseek-ai/dsh-session-persistence-sqlite': + specifier: workspace:^ + version: link:../../session/session-persistence-sqlite + '@deepseek-ai/dsh-subagent': + specifier: workspace:^ + version: link:../../subagent/subagent + '@deepseek-ai/dsh-subagent-fork-in-process': + specifier: workspace:^ + version: link:../../subagent/subagent-fork-in-process + '@deepseek-ai/dsh-subagent-spawn-in-process': + specifier: workspace:^ + version: link:../../subagent/subagent-spawn-in-process + + packages/experimental/tool-agent-team: + dependencies: + '@deepseek-ai/schemastery': + specifier: link:../../../vendor/schemastery + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-agent-loop': + specifier: workspace:^ + version: link:../../core/agent-loop + '@deepseek-ai/dsh-agent-loop-testkit': + specifier: workspace:^ + version: link:../../test-support/agent-loop-testkit + '@deepseek-ai/dsh-experimental-agent-team': + specifier: workspace:^ + version: link:../agent-team + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-scope': + specifier: workspace:^ + version: link:../../core/scope + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:^ + version: link:../../session/session-persistence-jsonl + '@deepseek-ai/dsh-subagent': + specifier: workspace:^ + version: link:../../subagent/subagent + '@deepseek-ai/dsh-subagent-fork-in-process': + specifier: workspace:^ + version: link:../../subagent/subagent-fork-in-process + '@deepseek-ai/dsh-subagent-spawn-in-process': + specifier: workspace:^ + version: link:../../subagent/subagent-spawn-in-process + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../../core/system-prompt + '@deepseek-ai/dsh-tool-subagent-control': + specifier: workspace:^ + version: link:../../subagent/tool-subagent-control + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../../core/tools + + packages/experimental/webworker-packer: + dependencies: + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@deepseek-ai/dsh-experimental-webworker-runtime': + specifier: workspace:^ + version: link:../webworker-runtime + '@deepseek-ai/dsh-home-paths': + specifier: workspace:^ + version: link:../../util/home-paths + js-yaml: + specifier: ^4.2.0 + version: 4.3.1 + picomatch: + specifier: ^4.0.4 + version: 4.0.4 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -3931,6 +4293,58 @@ importers: '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + '@types/js-yaml': + specifier: ^4.0.9 + version: 4.0.9 + '@types/picomatch': + specifier: ^3.0.2 + version: 3.0.2 + + packages/experimental/webworker-runtime: + dependencies: + '@deepseek-ai/dsh-util-crypto': + specifier: workspace:^ + version: link:../../util/crypto + '@noble/hashes': + specifier: ^2.3.0 + version: 2.3.0 + '@yarnpkg/parsers': + specifier: ^3.1.0 + version: 3.1.0 + acorn: + specifier: ^8.17.0 + version: 8.17.0 + buffer: + specifier: ^6.0.3 + version: 6.0.3 + picomatch: + specifier: ^4.0.4 + version: 4.0.4 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader + '@deepseek-ai/dsh-client-modules': + specifier: workspace:^ + version: link:../../client/modules + '@deepseek-ai/dsh-host-apiproxy': + specifier: workspace:^ + version: link:../../host/apiproxy + '@deepseek-ai/dsh-host-webserver': + specifier: workspace:^ + version: link:../../host/webserver + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local + '@types/picomatch': + specifier: ^3.0.2 + version: 3.0.2 packages/extensions/cordis-client-runner: devDependencies: @@ -3952,9 +4366,6 @@ importers: '@deepseek-ai/dsh-client-runtime': specifier: workspace:^ version: link:../../client/runtime - '@deepseek-ai/dsh-client-ui-slots': - specifier: workspace:^ - version: link:../../client/ui-slots '@deepseek-ai/dsh-client-ui-theme': specifier: workspace:^ version: link:../../client/ui-theme @@ -4770,6 +5181,9 @@ importers: '@deepseek-ai/dsh-user-questions': specifier: workspace:^ version: link:../../interaction/user-questions + '@deepseek-ai/dsh-util-crypto': + specifier: workspace:^ + version: link:../../util/crypto '@deepseek-ai/dsh-workspace': specifier: workspace:^ version: link:../../workspace/workspace @@ -4959,6 +5373,9 @@ importers: packages/interaction/commands: dependencies: + '@deepseek-ai/dsh-util-crypto': + specifier: workspace:^ + version: link:../../util/crypto zod: specifier: ^4.4.3 version: 4.4.3 @@ -4969,12 +5386,18 @@ importers: '@deepseek-ai/dsh-agent': specifier: workspace:^ version: link:../../core/agent + '@deepseek-ai/dsh-attachment': + specifier: workspace:^ + version: link:../../attachment/attachment '@deepseek-ai/dsh-brand': specifier: workspace:^ version: link:../../util/brand '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm '@deepseek-ai/dsh-scope': specifier: workspace:^ version: link:../../core/scope @@ -5187,8 +5610,20 @@ importers: specifier: workspace:^ version: link:../../core/tools + packages/llm/deepseek-llm-api-extensions: + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + packages/llm/llm: dependencies: + '@deepseek-ai/dsh-util-crypto': + specifier: workspace:^ + version: link:../../util/crypto '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -5221,12 +5656,30 @@ importers: '@deepseek-ai/cordis': specifier: workspace:^ version: link:../../../vendor/cordis + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent '@deepseek-ai/dsh-anonymous-user-id': specifier: workspace:^ version: link:../../identity/anonymous-user-id + '@deepseek-ai/dsh-atomic-write': + specifier: workspace:^ + version: link:../../util/atomic-write + '@deepseek-ai/dsh-attachment': + specifier: workspace:^ + version: link:../../attachment/attachment + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand '@deepseek-ai/dsh-credentials': specifier: workspace:^ version: link:../../credentials/credentials + '@deepseek-ai/dsh-deepseek-llm-api-extensions': + specifier: workspace:^ + version: link:../deepseek-llm-api-extensions + '@deepseek-ai/dsh-home-paths': + specifier: workspace:^ + version: link:../../util/home-paths '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -5236,6 +5689,15 @@ importers: '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../llm + '@deepseek-ai/dsh-plugin-package-inventory-deepseek': + specifier: workspace:^ + version: link:../plugin-package-inventory-deepseek + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-session-log-deepseek': + specifier: workspace:^ + version: link:../../session/session-log-deepseek '@deepseek-ai/dsh-settings': specifier: workspace:^ version: link:../../settings/settings @@ -5258,6 +5720,9 @@ importers: '@deepseek-ai/dsh-attachment': specifier: workspace:^ version: link:../../attachment/attachment + '@deepseek-ai/dsh-authorization': + specifier: workspace:^ + version: link:../../credentials/authorization '@deepseek-ai/dsh-credentials': specifier: workspace:^ version: link:../../credentials/credentials @@ -5338,6 +5803,40 @@ importers: specifier: workspace:^ version: link:../../core/tools + packages/llm/plugin-package-inventory-deepseek: + dependencies: + '@deepseek-ai/schemastery': + specifier: link:../../../vendor/schemastery + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@deepseek-ai/cordis-plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-agent-presets': + specifier: workspace:^ + version: link:../../preset/agent-presets + '@deepseek-ai/dsh-deepseek-llm-api-extensions': + specifier: workspace:^ + version: link:../deepseek-llm-api-extensions + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-scope': + specifier: workspace:^ + version: link:../../core/scope + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + packages/llm/token-meter: dependencies: '@deepseek-ai/schemastery': @@ -5485,12 +5984,21 @@ importers: '@deepseek-ai/cordis': specifier: workspace:^ version: link:../../../vendor/cordis + '@deepseek-ai/dsh-attachment': + specifier: workspace:^ + version: link:../../attachment/attachment + '@deepseek-ai/dsh-attachment-local': + specifier: workspace:^ + version: link:../../attachment/attachment-local '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm + '@deepseek-ai/dsh-scope': + specifier: workspace:^ + version: link:../../core/scope '@deepseek-ai/dsh-subprocess': specifier: workspace:^ version: link:../../subprocess/subprocess @@ -5704,6 +6212,9 @@ importers: packages/sandbox/sandbox-windows-acl: dependencies: + '@deepseek-ai/dsh-win32-process': + specifier: workspace:^ + version: link:../../subprocess/win32-process koffi: specifier: ^3.1.0 version: 3.1.1 @@ -5764,6 +6275,10 @@ importers: version: link:../../core/tools packages/sdk/client: + dependencies: + '@deepseek-ai/dsh': + specifier: workspace:* + version: link:../../../apps/cli devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -5799,6 +6314,19 @@ importers: specifier: workspace:^ version: link:../../subagent/subagent + packages/sdk/python-runtime: + dependencies: + '@deepseek-ai/dsh-app-boot': + specifier: workspace:^ + version: link:../../boot/app-boot + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + packages/sdk/server: dependencies: '@deepseek-ai/schemastery': @@ -5910,6 +6438,9 @@ importers: '@deepseek-ai/dsh-session-title': specifier: workspace:^ version: link:../../session/session-title + '@deepseek-ai/dsh-tool-todo': + specifier: workspace:^ + version: link:../../todo/tool-todo packages/session-query/session-query-sqlite: dependencies: @@ -6027,6 +6558,25 @@ importers: specifier: workspace:^ version: link:../../core/tools + packages/session/session-log-deepseek: + dependencies: + '@deepseek-ai/schemastery': + specifier: link:../../../vendor/schemastery + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-deepseek-llm-api-extensions': + specifier: workspace:^ + version: link:../../llm/deepseek-llm-api-extensions + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + packages/session/session-persistence: devDependencies: '@deepseek-ai/cordis': @@ -6079,15 +6629,27 @@ importers: '@deepseek-ai/cordis': specifier: workspace:^ version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@deepseek-ai/cordis-plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session '@deepseek-ai/dsh-session-persistence': specifier: workspace:^ version: link:../session-persistence + typescript: + specifier: ^6.0.3 + version: 6.0.3 packages/session/session-projection: dependencies: @@ -6729,6 +7291,61 @@ importers: specifier: workspace:^ version: link:../../interaction/user-approval + packages/shell/tool-pwsh-persistent: + dependencies: + '@deepseek-ai/schemastery': + specifier: link:../../../vendor/schemastery + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@deepseek-ai/cordis-plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-pwsh-local': + specifier: workspace:^ + version: link:../pwsh-local + '@deepseek-ai/dsh-sandbox': + specifier: workspace:^ + version: link:../../sandbox/sandbox + '@deepseek-ai/dsh-sandbox-policy': + specifier: workspace:^ + version: link:../../sandbox/sandbox-policy + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../../core/system-prompt + '@deepseek-ai/dsh-terminal': + specifier: workspace:^ + version: link:../../terminal/terminal + '@deepseek-ai/dsh-terminal-bash': + specifier: workspace:^ + version: link:../../terminal/terminal-bash + '@deepseek-ai/dsh-timeout': + specifier: workspace:^ + version: link:../../util/timeout + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../../core/tools + packages/skill/skill: dependencies: '@deepseek-ai/schemastery': @@ -7023,8 +7640,8 @@ importers: packages/subagent/subagent-acp: dependencies: '@agentclientprotocol/sdk': - specifier: 0.25.1 - version: 0.25.1(zod@4.4.3) + specifier: 1.4.0 + version: 1.4.0(zod@4.4.3) '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -7074,6 +7691,12 @@ importers: '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery + '@modelcontextprotocol/sdk': + specifier: ^1.29.0 + version: 1.29.0(zod@4.4.3) + zod: + specifier: ^4.4.3 + version: 4.4.3 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -7108,9 +7731,15 @@ importers: packages/subagent/subagent-codex: dependencies: + '@deepseek-ai/dsh-sdk-protocol': + specifier: workspace:^ + version: link:../../sdk/protocol '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery + '@openai/codex': + specifier: 0.147.0 + version: 0.147.0 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -7130,9 +7759,6 @@ importers: '@deepseek-ai/dsh-loader-smoke': specifier: workspace:^ version: link:../../test-support/loader-smoke - '@deepseek-ai/dsh-sdk-protocol': - specifier: workspace:^ - version: link:../../sdk/protocol '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session @@ -7148,9 +7774,6 @@ importers: '@deepseek-ai/dsh-timeout': specifier: workspace:^ version: link:../../util/timeout - '@openai/codex': - specifier: 0.147.0 - version: 0.147.0 packages/subagent/subagent-dsh-sdk: dependencies: @@ -7492,9 +8115,12 @@ importers: packages/subprocess/subprocess-local: dependencies: + koffi: + specifier: ^3.1.0 + version: 3.1.1 node-pty: - specifier: ^1.1.0 - version: 1.1.0(patch_hash=7a0c04f1f49d798a9ffe2f7f414c01064a44ca2489772d0c3e1235ab336755e6) + specifier: 1.2.0-beta.15 + version: 1.2.0-beta.15(patch_hash=b40ae545608897914bd25fb009c97eeac478c34e8a910298ddcb01b746534bb0) devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -7512,6 +8138,19 @@ importers: specifier: workspace:^ version: link:../../util/timeout + packages/subprocess/win32-process: + dependencies: + koffi: + specifier: ^3.1.0 + version: 3.1.1 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + packages/terminal/terminal: devDependencies: '@deepseek-ai/cordis': @@ -7532,6 +8171,9 @@ importers: packages/terminal/terminal-bash: dependencies: + '@deepseek-ai/dsh-pwsh-local': + specifier: workspace:^ + version: link:../../shell/pwsh-local '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -7628,14 +8270,20 @@ importers: packages/test-support/acp-snapshot: dependencies: '@agentclientprotocol/sdk': - specifier: 0.25.1 - version: 0.25.1(zod@4.4.3) + specifier: 1.4.0 + version: 1.4.0(zod@4.4.3) + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:* + version: link:../../../vendor/include '@deepseek-ai/dsh-loader-smoke': specifier: workspace:* version: link:../loader-smoke + js-yaml: + specifier: ^4.2.0 + version: 4.2.0 vitest: specifier: ^4.1.8 - version: 4.1.8(@opentelemetry/api@1.9.0)(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) + version: 4.1.8(@opentelemetry/api@1.9.0)(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1(@noble/hashes@2.3.0))(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -7646,6 +8294,9 @@ importers: '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session + '@types/js-yaml': + specifier: ^4.0.9 + version: 4.0.9 packages/test-support/agent-loop-testkit: devDependencies: @@ -7684,7 +8335,7 @@ importers: version: 16.3.2(@testing-library/dom@10.4.1)(@types/react-dom@18.3.7(@types/react@18.3.31))(@types/react@18.3.31)(react-dom@18.3.1(react@18.3.1))(react@18.3.1) vitest: specifier: ^4.1.8 - version: 4.1.8(@opentelemetry/api@1.9.1)(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) + version: 4.1.8(@opentelemetry/api@1.9.1)(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1(@noble/hashes@2.3.0))(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -7692,12 +8343,12 @@ importers: '@deepseek-ai/dsh-client-runtime': specifier: workspace:^ version: link:../../client/runtime + '@deepseek-ai/dsh-client-ui-renderer': + specifier: workspace:^ + version: link:../../client/ui-renderer '@deepseek-ai/dsh-client-ui-slots': specifier: workspace:^ version: link:../../client/ui-slots - '@deepseek-ai/dsh-client-web-react': - specifier: workspace:^ - version: link:../../client/web-react '@deepseek-ai/dsh-host-apiproxy': specifier: workspace:^ version: link:../../host/apiproxy @@ -7734,6 +8385,9 @@ importers: '@deepseek-ai/dsh-compaction': specifier: workspace:^ version: link:../../compaction/compaction + '@deepseek-ai/dsh-deepseek-llm-api-extensions': + specifier: workspace:^ + version: link:../../llm/deepseek-llm-api-extensions '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants @@ -7911,6 +8565,15 @@ importers: specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + packages/util/crypto: + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + packages/util/home-paths: devDependencies: '@deepseek-ai/cordis': @@ -8118,6 +8781,82 @@ importers: specifier: workspace:^ version: link:../web + packages/webhook/webhook: + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@deepseek-ai/cordis-plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-agent-default-model': + specifier: workspace:^ + version: link:../../core/agent-default-model + '@deepseek-ai/dsh-agent-presets': + specifier: workspace:^ + version: link:../../preset/agent-presets + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-permission-presets': + specifier: workspace:^ + version: link:../../interaction/permission-presets + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-session-title': + specifier: workspace:^ + version: link:../../session/session-title + '@deepseek-ai/dsh-workspace': + specifier: workspace:^ + version: link:../../workspace/workspace + + packages/webhook/webhook-github: + dependencies: + '@deepseek-ai/schemastery': + specifier: link:../../../vendor/schemastery + version: link:../../../vendor/schemastery + '@octokit/webhooks': + specifier: ^14.2.0 + version: 14.2.0 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@deepseek-ai/cordis-plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader + '@deepseek-ai/dsh-credentials': + specifier: workspace:^ + version: link:../../credentials/credentials + '@deepseek-ai/dsh-host-webserver': + specifier: workspace:^ + version: link:../../host/webserver + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-webhook': + specifier: workspace:^ + version: link:../webhook + packages/workflow/tool-ralph: dependencies: '@deepseek-ai/schemastery': @@ -8340,15 +9079,24 @@ importers: '@deepseek-ai/dsh-agent-spine-demo': specifier: workspace:^ version: link:../../packages/examples/agent-spine-demo + '@deepseek-ai/dsh-agent-tool-presentation': + specifier: workspace:^ + version: link:../../packages/core/agent-tool-presentation '@deepseek-ai/dsh-anonymous-user-id': specifier: workspace:^ version: link:../../packages/identity/anonymous-user-id '@deepseek-ai/dsh-app-boot': specifier: workspace:^ version: link:../../packages/boot/app-boot + '@deepseek-ai/dsh-atomic-write': + specifier: workspace:^ + version: link:../../packages/util/atomic-write '@deepseek-ai/dsh-attachment': specifier: workspace:^ version: link:../../packages/attachment/attachment + '@deepseek-ai/dsh-authorization': + specifier: workspace:^ + version: link:../../packages/credentials/authorization '@deepseek-ai/dsh-bash-local': specifier: workspace:^ version: link:../../packages/shell/bash-local @@ -8361,6 +9109,9 @@ importers: '@deepseek-ai/dsh-code-runtime-worker-thread': specifier: workspace:^ version: link:../../packages/code-runtime/code-runtime-worker-thread + '@deepseek-ai/dsh-command-compact': + specifier: workspace:^ + version: link:../../packages/compaction/command-compact '@deepseek-ai/dsh-command-goal': specifier: workspace:^ version: link:../../packages/goal/command-goal @@ -8382,6 +9133,9 @@ importers: '@deepseek-ai/dsh-credentials': specifier: workspace:^ version: link:../../packages/credentials/credentials + '@deepseek-ai/dsh-deepseek-llm-api-extensions': + specifier: workspace:^ + version: link:../../packages/llm/deepseek-llm-api-extensions '@deepseek-ai/dsh-fs': specifier: workspace:^ version: link:../../packages/fs/fs @@ -8436,15 +9190,27 @@ importers: '@deepseek-ai/dsh-llm-retry': specifier: workspace:^ version: link:../../packages/llm/llm-retry + '@deepseek-ai/dsh-mcp-client': + specifier: workspace:^ + version: link:../../packages/mcp/mcp-client '@deepseek-ai/dsh-output-retention': specifier: workspace:^ version: link:../../packages/util/output-retention '@deepseek-ai/dsh-permission-presets': specifier: workspace:^ version: link:../../packages/interaction/permission-presets + '@deepseek-ai/dsh-persona': + specifier: workspace:^ + version: link:../../packages/preset/persona '@deepseek-ai/dsh-plan-mode': specifier: workspace:^ version: link:../../packages/plan/plan-mode + '@deepseek-ai/dsh-plugin-package-inventory-deepseek': + specifier: workspace:^ + version: link:../../packages/llm/plugin-package-inventory-deepseek + '@deepseek-ai/dsh-pwsh-local': + specifier: workspace:^ + version: link:../../packages/shell/pwsh-local '@deepseek-ai/dsh-repeat-tool-reminder': specifier: workspace:^ version: link:../../packages/guard/repeat-tool-reminder @@ -8460,21 +9226,24 @@ importers: '@deepseek-ai/dsh-scope': specifier: workspace:^ version: link:../../packages/core/scope - '@deepseek-ai/dsh-sdk-jsonrpc-demo': - specifier: workspace:^ - version: link:../../packages/examples/jsonrpc-demo '@deepseek-ai/dsh-sdk-jsonrpc-server': specifier: workspace:^ version: link:../../packages/sdk/server '@deepseek-ai/dsh-sdk-protocol': specifier: workspace:^ version: link:../../packages/sdk/protocol + '@deepseek-ai/dsh-sdk-python-runtime': + specifier: workspace:^ + version: link:../../packages/sdk/python-runtime '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../packages/core/session '@deepseek-ai/dsh-session-checkpoint-policy': specifier: workspace:^ version: link:../../packages/session/session-checkpoint-policy + '@deepseek-ai/dsh-session-log-deepseek': + specifier: workspace:^ + version: link:../../packages/session/session-log-deepseek '@deepseek-ai/dsh-session-persistence': specifier: workspace:^ version: link:../../packages/session/session-persistence @@ -8514,6 +9283,9 @@ importers: '@deepseek-ai/dsh-skill-filesystem': specifier: workspace:^ version: link:../../packages/skill/skill-filesystem + '@deepseek-ai/dsh-spill': + specifier: workspace:^ + version: link:../../packages/spill/spill '@deepseek-ai/dsh-subagent': specifier: workspace:^ version: link:../../packages/subagent/subagent @@ -8568,12 +9340,18 @@ importers: '@deepseek-ai/dsh-tool-fs': specifier: workspace:^ version: link:../../packages/fs/tool-fs + '@deepseek-ai/dsh-tool-fs-search': + specifier: workspace:^ + version: link:../../packages/fs/tool-fs-search '@deepseek-ai/dsh-tool-goal': specifier: workspace:^ version: link:../../packages/goal/tool-goal '@deepseek-ai/dsh-tool-jobs': specifier: workspace:^ version: link:../../packages/jobs/tool-jobs + '@deepseek-ai/dsh-tool-ralph': + specifier: workspace:^ + version: link:../../packages/workflow/tool-ralph '@deepseek-ai/dsh-tool-skill': specifier: workspace:^ version: link:../../packages/skill/tool-skill @@ -8784,8 +9562,8 @@ importers: packages: - '@agentclientprotocol/sdk@0.25.1': - resolution: {integrity: sha512-jx2rF3bdpGwZ75Q/meyEDLLbYmbtxk82Uh9hDCdxDvcEedBnNSF5hZAnL/kJR5VNz56JqwOmqnAqasC84MwwkQ==} + '@agentclientprotocol/sdk@1.4.0': + resolution: {integrity: sha512-/eufudw+aFY1LKLolT6yFE6UMmYRl7fMJ/DEONSIyR6wI3slHWITBsANRGqXEY8FRzqUxwh7QEaGiZHcJPVThg==} peerDependencies: zod: ^3.25.0 || ^4.0.0 @@ -10120,9 +10898,34 @@ packages: '@emnapi/core': ^1.7.1 '@emnapi/runtime': ^1.7.1 + '@noble/hashes@2.3.0': + resolution: {integrity: sha512-oN+QwyX7VSHotibwubG3kpzbwKrfnyR6OOO+3Nk/53ADL7FmgHHz4TgrbaYKvvOw09u6QTx0oiH1cNCIOuN0CQ==} + engines: {node: '>= 20.19.0'} + '@nodable/entities@2.2.0': resolution: {integrity: sha512-9uGyhaQavEUMC8AIddIjau4NsnsXhou+j5sBAGojCM1oxmQpVKTWR/9JxABD6UAv12vpIms55fPZKFQEhG6uBg==} + '@octokit/openapi-types@28.0.0': + resolution: {integrity: sha512-0rFyLuyHvIj6uuZWuDslxkowFYdPXoNIkeAv4b27dzm2Tf4vGWXnPsMcxs7d65kLdMERgP3wc1AEPlqMz8e1cQ==} + + '@octokit/openapi-webhooks-types@12.1.0': + resolution: {integrity: sha512-WiuzhOsiOvb7W3Pvmhf8d2C6qaLHXrWiLBP4nJ/4kydu+wpagV5Fkz9RfQwV2afYzv3PB+3xYgp4mAdNGjDprA==} + + '@octokit/request-error@7.1.1': + resolution: {integrity: sha512-+eaY7G2VVpSf2pc5Gn1+mph837V/d/TYTJAgWL9Tb0ogGYcpN3IlAVFgjL+Vv93F/sevrxkvsYCedtpLdcFLzA==} + engines: {node: '>= 20'} + + '@octokit/types@17.0.0': + resolution: {integrity: sha512-ByP1v7YL5SMveFPP7+sj0/ZuWCOOg/Chs4NafOMpq6WNIM/hdGY0S7C0TCGDBWu1aGmOxmUIhMx3cO+IdwYZ1Q==} + + '@octokit/webhooks-methods@6.0.0': + resolution: {integrity: sha512-MFlzzoDJVw/GcbfzVC1RLR36QqkTLUf79vLVO3D+xn7r0QgxnFoLZgtrzxiQErAjFUOdH6fas2KeQJ1yr/qaXQ==} + engines: {node: '>= 20'} + + '@octokit/webhooks@14.2.0': + resolution: {integrity: sha512-da6KbdNCV5sr1/txD896V+6W0iamFWrvVl8cHkBSPT+YlvmT3DwXa4jxZnQc+gnuTEqSWbBeoSZYTayXH9wXcw==} + engines: {node: '>= 20'} + '@openai/codex@0.147.0': resolution: {integrity: sha512-EQLEXecAG2ptxI7UpBMo2TR/ga5596/c/OsYF/0LoUDh5JANZ7IoGqlzBEWbuEVQ76JePIbtTW/ihCkp1a7Z3w==} engines: {node: '>=16'} @@ -11342,6 +12145,9 @@ packages: '@types/unist@3.0.3': resolution: {integrity: sha512-ko/gIFJRv177XgZsZcBwnqJN5x/Gien8qNOn0D5bQU/zAzVf9Zt3BlcUiLqhV9y4ARk0GbT3tnUiPNgnTXzc/Q==} + '@types/use-sync-external-store@1.5.0': + resolution: {integrity: sha512-5dyB8nLC/qogMrlCizZnYWQTA4lnb/v+It+sqNl5YnSRAPMlIqY/X0Xn+gZw8vOL+TgTTr28VEbn3uf8fUtAkw==} + '@types/web-bluetooth@0.0.21': resolution: {integrity: sha512-oIQLCGWtcFZy2JW77j9k8nHzAOpqMHLQejDA48XXMWH6tjCQHz5RCFz1bzsmROyL6PUm+LLnUiI4BCn221inxA==} @@ -11565,6 +12371,10 @@ packages: engines: {node: '>=18.12.0'} hasBin: true + '@yarnpkg/parsers@3.1.0': + resolution: {integrity: sha512-WoxUM/Be4hfsX06FxsvpGgfYqwgivMV7/Ol7aFuSfSmY6rRaiju4QxOEe9RUS0iYcSHWl5i9AhB1cMoE0p+XiA==} + engines: {node: '>=18.12.0'} + accepts@2.0.0: resolution: {integrity: sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==} engines: {node: '>= 0.6'} @@ -11648,6 +12458,9 @@ packages: ast-v8-to-istanbul@1.0.4: resolution: {integrity: sha512-0bC0/4bTSrnwdhU3IsZDwEdojvuPrSg59OYZfKsLRtJZ0u8VBx9DebfqqG8bRdCC0I7vjgxmPi41P0lpkhJHtA==} + async@3.2.6: + resolution: {integrity: sha512-htCUDlxyyCLMgaM3xXg0C0LW2xqfuQ6p05pCEIsXuyQ+a1koYKTuBMzRNwmybfLgvJDMd0r1LTn4+E0Ti6C2AA==} + balanced-match@1.0.2: resolution: {integrity: sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==} @@ -11663,6 +12476,10 @@ packages: engines: {node: '>=6.0.0'} hasBin: true + basic-auth@2.0.1: + resolution: {integrity: sha512-NF+epuEdnUYVlGuhaxbbq+dvJttwLnGY+YixlXlME5KpQ5W3CnXA5cVTneY3SPbPDRkcjMbifrwmFYcClgOZeg==} + engines: {node: '>= 0.8'} + bidi-js@1.0.3: resolution: {integrity: sha512-RKshQI1R3YQ+n9YJz2QQ147P66ELpa1FQEg20Dk8oW9t2KgLbpDLLp9aGZ7y8WHSshDknG0bknqGw5/tyCs5tw==} @@ -11697,10 +12514,17 @@ packages: buffer-equal-constant-time@1.0.1: resolution: {integrity: sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA==} + buffer@6.0.3: + resolution: {integrity: sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==} + builtin-modules@3.3.0: resolution: {integrity: sha512-zhaCDicdLuWN5UbN5IMnFqNMhNfo919sH85y2/ea+5Yg9TsTkeZxpL+JLbp6cgYFS4sRLp3YV4S6yDuqVWHYOw==} engines: {node: '>=6'} + bundle-name@4.1.0: + resolution: {integrity: sha512-tjwM5exMg6BGRI+kNmTntNsvdZS1X8BFYS6tnJ2hdH0kVxM6/eVZ2xy+FqStSWvYmtfFMDLIxurorHwDKfDz5Q==} + engines: {node: '>=18'} + bytes@3.1.2: resolution: {integrity: sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==} engines: {node: '>= 0.8'} @@ -11727,6 +12551,10 @@ packages: resolution: {integrity: sha512-NUPRluOfOiTKBKvWPtSD4PhFvWCqOi0BGStNWs57X9js7XGTprSmFoz5F0tWhR4WPjNeR9jXqdC7/UpSJTnlRg==} engines: {node: '>=18'} + chalk@4.1.2: + resolution: {integrity: sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==} + engines: {node: '>=10'} + chalk@5.6.2: resolution: {integrity: sha512-7NzBL0rN6fMUW+f7A6Io4h40qQlG+xGmtMxfbnH/K7TAtt8JQWVQK+6g0UXKMeVJoyV5EkkNsErQ8pVD3bLHbA==} engines: {node: ^12.17.0 || ^14.13 || >=16.0.0} @@ -11815,6 +12643,10 @@ packages: resolution: {integrity: sha512-tJtZBBHA6vjIAaF6EnIaq6laBBP9aq/Y3ouVJjEfoHbRBcHBAHYcMh/w8LDrk2PvIMMq8gmopa5D4V8RmbrxGw==} engines: {node: '>= 0.10'} + corser@2.0.1: + resolution: {integrity: sha512-utCYNzRSQIZNPIcGZdQc92UVJYAhtGAteCFg0yRaFm8f0P+CPtyGyHXJcGXnffjCybUCEx3FQ2G7U3/o9eIkVQ==} + engines: {node: '>= 0.4.0'} + cose-base@1.0.3: resolution: {integrity: sha512-s9whTXInMSgAp/NVXVNuVxVKzGH2qck3aQlVHxDCdAEPgtMKwc4Wq6/QKhgdEdgbLSi9rBTAcPoRa6JpiG4ksg==} @@ -12017,6 +12849,18 @@ packages: deep-is@0.1.4: resolution: {integrity: sha512-oIPzksmTg4/MriiaYGO+okXDT7ztn/w3Eptv/+gSIdMdKsJo0u4CfYNFJPy+4SKMuCqGw2wxnA+URMg3t8a/bQ==} + default-browser-id@5.0.1: + resolution: {integrity: sha512-x1VCxdX4t+8wVfd1so/9w+vQ4vx7lKd2Qp5tDRutErwmR85OgmfX7RlLRMWafRMY7hbEiXIbudNrjOAPa/hL8Q==} + engines: {node: '>=18'} + + default-browser@5.5.0: + resolution: {integrity: sha512-H9LMLr5zwIbSxrmvikGuI/5KGhZ8E2zH3stkMgM5LpOWDutGM2JZaj460Udnf1a+946zc7YBgrqEWwbk7zHvGw==} + engines: {node: '>=18'} + + define-lazy-prop@3.0.0: + resolution: {integrity: sha512-N+MeXYoqr3pOgn8xfyRPREN7gHakLYjhsHhWGT3fWAiL4IkAt0iDw14QiiEm2bE30c5XX5q0FtAA3CK5f9/BUg==} + engines: {node: '>=12'} + defu@6.1.7: resolution: {integrity: sha512-7z22QmUWiQ/2d0KkdYmANbRUVABpZ9SNYyH5vx6PZ+nE5bcC0l7uFvEfHlyld/HcGBFTL536ClDt3DEcSlEJAQ==} @@ -12222,6 +13066,9 @@ packages: resolution: {integrity: sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==} engines: {node: '>= 0.6'} + eventemitter3@4.0.7: + resolution: {integrity: sha512-8guHBZCwKnFhYdHr2ysuRWErTwhoN2X8XELRlrRwpmfeY2jjuUN4taQMsULKUVo1K4DvZl+0pgfyoysHxvmvEw==} + eventsource-parser@3.1.0: resolution: {integrity: sha512-kJezFj9YFAMLeORyi7aCLxLbD5/qWMQnoMVlVPyHIll7lgRJCc3JVln9Vgl9nwQi0YkMnhdGTMNn7CkRRAptMg==} engines: {node: '>=18.0.0'} @@ -12319,6 +13166,15 @@ packages: focus-trap@7.8.0: resolution: {integrity: sha512-/yNdlIkpWbM0ptxno3ONTuf+2g318kh2ez3KSeZN5dZ8YC6AAmgeWz+GasYYiBJPFaYcSAPeu4GfhUaChzIJXA==} + follow-redirects@1.16.0: + resolution: {integrity: sha512-y5rN/uOsadFT/JfYwhxRS5R7Qce+g3zG97+JrtFZlC9klX/W5hD7iiLzScI4nZqUS7DNUdhPgw4xI8W2LuXlUw==} + engines: {node: '>=4.0'} + peerDependencies: + debug: '*' + peerDependenciesMeta: + debug: + optional: true + foreground-child@3.3.1: resolution: {integrity: sha512-gIXjKqtFuWEgzFRJA9WCQeSJLZDjgJUOMCMzxtvFq/37KojM1BFGufqsCy0r4qSQmYLsZYMeyRqzIWOMup03sw==} engines: {node: '>=14'} @@ -12442,6 +13298,10 @@ packages: hast-util-whitespace@3.0.0: resolution: {integrity: sha512-88JUN06ipLwsnv+dVn+OIYOvAuvBMy/Qoi6O7mQHxdPXpjy+Cd6xRkWwux7DKO+4sYILtLBRIKgsdpS2gQc7qw==} + he@1.2.0: + resolution: {integrity: sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw==} + hasBin: true + hono@4.12.29: resolution: {integrity: sha512-1hNiRjawYrLq/4m3DQQjPGFg0VZkk4RjQJDff/excI6Dm9BiL75qxGrd7/c6YOxPdq6AscP3LiXhQ6fKFC1Waw==} engines: {node: '>=16.9.0'} @@ -12452,6 +13312,10 @@ packages: hookable@6.1.1: resolution: {integrity: sha512-U9LYDy1CwhMCnprUfeAZWZGByVbhd54hwepegYTK7Pi5NvqEj63ifz5z+xukznehT7i6NIZRu89Ay1AZmRsLEQ==} + html-encoding-sniffer@3.0.0: + resolution: {integrity: sha512-oWv4T4yJ52iKrufjnyZPkrN0CH3QnrUqdB6In1g5Fe1mia8GmF36gnfNySxoZtxD5+NmYw1EElVXiBk93UeskA==} + engines: {node: '>=12'} + html-encoding-sniffer@6.0.0: resolution: {integrity: sha512-CV9TW3Y3f8/wT0BRFc1/KAVQ3TUHiXmaAb6VW9vtiMFf7SLoMd1PdAc4W3KFOFETBJUb90KatHqlsZMWV+R9Gg==} engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} @@ -12470,6 +13334,15 @@ packages: resolution: {integrity: sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig==} engines: {node: '>= 14'} + http-proxy@1.18.1: + resolution: {integrity: sha512-7mz/721AbnJwIVbnaSv1Cz3Am0ZLT/UBwkC92VlxhXv/k/BBQfM2fXElQNC27BVGr0uwUpplYPQM9LnaBMR5NQ==} + engines: {node: '>=8.0.0'} + + http-server@14.1.1: + resolution: {integrity: sha512-+cbxadF40UXd9T01zUHgA+rlo2Bg1Srer4+B4NwIHdaGxAGGv59nYRnGGDJ9LBk7alpS0US+J+bLLdQOOkJq4A==} + engines: {node: '>=12'} + hasBin: true + https-proxy-agent@7.0.6: resolution: {integrity: sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==} engines: {node: '>= 14'} @@ -12486,6 +13359,9 @@ packages: resolution: {integrity: sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ==} engines: {node: '>=0.10.0'} + ieee754@1.2.1: + resolution: {integrity: sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==} + ignore@5.3.2: resolution: {integrity: sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g==} engines: {node: '>= 4'} @@ -12525,6 +13401,11 @@ packages: resolution: {integrity: sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==} engines: {node: '>= 0.10'} + is-docker@3.0.0: + resolution: {integrity: sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + hasBin: true + is-extglob@2.1.1: resolution: {integrity: sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ==} engines: {node: '>=0.10.0'} @@ -12537,6 +13418,15 @@ packages: resolution: {integrity: sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg==} engines: {node: '>=0.10.0'} + is-in-ssh@1.0.0: + resolution: {integrity: sha512-jYa6Q9rH90kR1vKB6NM7qqd1mge3Fx4Dhw5TVlK1MUBqhEOuCagrEHMevNuCcbECmXZ0ThXkRm+Ymr51HwEPAw==} + engines: {node: '>=20'} + + is-inside-container@1.0.0: + resolution: {integrity: sha512-KIYLCCJghfHZxqjYBE7rEy0OBuTd5xCHS7tHVgvCLkx7StIoaxwNW3hCALgEUjFfeRk+MG/Qxmp/vtETEF3tRA==} + engines: {node: '>=14.16'} + hasBin: true + is-plain-obj@4.1.0: resolution: {integrity: sha512-+Pgi+vMuUNkJyExiMBt5IlFoMyKnr5zhJ4Uspz58WOhBF5QoIZkFyNHIbBAtHwzVAgk5RtndVNsDRN61/mmDqg==} engines: {node: '>=12'} @@ -12559,6 +13449,10 @@ packages: resolution: {integrity: sha512-oG7cgbmg5kLYae2N5IVd3jm2s+vldjxJzK1pcu9LfpGuQ93MQSzo0okvRna+7y5ifrD+20FE8FvjusyGaz14fw==} engines: {node: '>=18'} + is-wsl@3.1.1: + resolution: {integrity: sha512-e6rvdUCiQCAuumZslxRJWR/Doq4VpPR82kqclvcS0efgt430SlGIk05vdCN58+VrzgtIcfNODjozVielycD4Sw==} + engines: {node: '>=16'} + isarray@1.0.0: resolution: {integrity: sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ==} @@ -12601,6 +13495,10 @@ packages: resolution: {integrity: sha512-ePWsvanv0DWuDRsW8dnt+R4jQ31SCRCQ7hhNcPXZPsoBZiemuZNYGf7adZdqX2D86j6rvKp3RpCxVTSb8WQlOw==} hasBin: true + js-yaml@4.3.1: + resolution: {integrity: sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==} + hasBin: true + jscpd-darwin-arm64@5.0.12: resolution: {integrity: sha512-CQCDYhQY9yOGGeauzkRGYW/QtXurPCjfDkEhUoM/M5UdmpzxfG3i9mnNsaheJ0RdFRf73mL7JaLVRP8U2l5/kA==} cpu: [arm64] @@ -13059,6 +13957,11 @@ packages: resolution: {integrity: sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A==} engines: {node: '>=18'} + mime@1.6.0: + resolution: {integrity: sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg==} + engines: {node: '>=4'} + hasBin: true + minimatch@10.2.5: resolution: {integrity: sha512-MULkVLfKGYDFYejP07QOurDLLQpcjk7Fw+7jXS2R2czRQzR56yHRveU5NDJEOviH+hETZKSkIk5c+T23GjFUMg==} engines: {node: 18 || 20 || >=22} @@ -13067,6 +13970,9 @@ packages: resolution: {integrity: sha512-OBwBN9AL4dqmETlpS2zasx+vTeWclWzkblfZk7KTA5j3jeOONz/tRCnZomUyvNg83wL5Zv9Ss6HMJXAgL8R2Yg==} engines: {node: '>=16 || 14 >=14.17'} + minimist@1.2.8: + resolution: {integrity: sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==} + minipass@7.1.3: resolution: {integrity: sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==} engines: {node: '>=16 || 14 >=14.17'} @@ -13164,8 +14070,8 @@ packages: resolution: {integrity: sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==} engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} - node-pty@1.1.0: - resolution: {integrity: sha512-20JqtutY6JPXTUnL0ij1uad7Qe1baT46lyolh2sSENDd4sTzKZ4nmAFkeAARDKwmlLjPx6XKRlwRUxwjOy+lUg==} + node-pty@1.2.0-beta.15: + resolution: {integrity: sha512-vORSzHXi4Ofl7HemVWpuudLqCPdaQb4LfpRCUpE5HPxhp4JYscl8zZwxh11p26v2wvW24WMwnMfLjhRLixrfxA==} node-releases@2.0.51: resolution: {integrity: sha512-wRNIrw4DmVLKQlbgOMdkMx27Wrpzes2hh5Jtbi2bjPd+4wJstWIqP5A+lscnqbm0xxmT5Bpg8Lec5ItEBwx6BQ==} @@ -13206,6 +14112,10 @@ packages: oniguruma-to-es@4.3.6: resolution: {integrity: sha512-csuQ9x3Yr0cEIs/Zgx/OEt9iBw9vqIunAPQkx19R/fiMq2oGVTgcMqO/V3Ybqefr1TBvosI6jU539ksaBULJyA==} + open@11.0.0: + resolution: {integrity: sha512-smsWv2LzFjP03xmvFoJ331ss6h+jixfA4UUV/Bsiyuu4YJPfN+FIQGOIiv4w9/+MoHkfkJ22UIaQWRVFRfH6Vw==} + engines: {node: '>=20'} + openai@6.26.0: resolution: {integrity: sha512-zd23dbWTjiJ6sSAX6s0HrCZi41JwTA1bQVs0wLQPZ2/5o2gxOJA5wh7yOAUgwYybfhDXyhwlpeQf7Mlgx8EOCA==} hasBin: true @@ -13224,6 +14134,10 @@ packages: openapi-typescript-helpers@0.0.15: resolution: {integrity: sha512-opyTPaunsklCBpTK8JGef6mfPhLSnyy5a0IN9vKtx3+4aExf+KxEqYwIy3hqkedXIB97u357uLMJsOnm3GVjsw==} + opener@1.5.2: + resolution: {integrity: sha512-ur5UIdyw5Y7yEj9wLzhqXiy6GZ3Mwx0yGI+5sMn2r0N0v3cKJvUmFH5yPP+WXh9e0xfyzyJX95D8l088DNFj7A==} + hasBin: true + optionator@0.9.4: resolution: {integrity: sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g==} engines: {node: '>= 0.8.0'} @@ -13353,10 +14267,18 @@ packages: points-on-path@0.2.1: resolution: {integrity: sha512-25ClnWWuw7JbWZcgqY/gJ4FQWadKxGWk+3kR/7kD0tCaDtPPMj7oHu2ToLaVhfpnHrZzYby2w6tUA0eOIuUg8g==} + portfinder@1.0.38: + resolution: {integrity: sha512-rEwq/ZHlJIKw++XtLAO8PPuOQA/zaPJOZJ37BVuN97nLpMJeuDVLVGRwbFoBgLudgdTMP2hdRJP++H+8QOA3vg==} + engines: {node: '>= 10.12'} + postcss@8.5.15: resolution: {integrity: sha512-FfR8sjd4em2T6fb3I2MwAJU7HWVMr9zba+enmQeeWFfCbm+UOC/0X4DS8XtpUTMwWMGbjKYP7xjfNekzyGmB3A==} engines: {node: ^10 || ^12 || >=14} + powershell-utils@0.1.0: + resolution: {integrity: sha512-dM0jVuXJPsDN6DvRpea484tCUaMiXWjuCn++HGTqUWzGDjv5tZkEZldAJ/UMlqRYGFrD/etByo4/xOuC/snX2A==} + engines: {node: '>=20'} + preact@10.29.7: resolution: {integrity: sha512-DCHYrK/B10yUD3ZjLfhZ3WIE/9Vf9VFUODcRE2dRomTYDpJk6z6L9wecSfhfE6M9ZTHUdyQkoC46arIDhEV84Q==} peerDependencies: @@ -13466,6 +14388,9 @@ packages: resolution: {integrity: sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==} engines: {node: '>=0.10.0'} + requires-port@1.0.0: + resolution: {integrity: sha512-KigOCHcocU3XODJxsu8i/j8T9tzT4adHiecwORRQ0ZZFcp7ahwXuRU1m+yuO90C5ZUyGeGfocHDI14M3L3yDAQ==} + resolve-pkg-maps@1.0.0: resolution: {integrity: sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==} @@ -13520,6 +14445,10 @@ packages: resolution: {integrity: sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ==} engines: {node: '>= 18'} + run-applescript@7.1.0: + resolution: {integrity: sha512-DPe5pVFaAsinSaV6QjQ6gdiedWDcRCbUuiQfQa2wmWV7+xC9bGulGI8+TdRmoFkAPaBXk8CrAbnlY2ISniJ47Q==} + engines: {node: '>=18'} + rw@1.3.3: resolution: {integrity: sha512-PdhdWy89SiZogBLaw42zdeqtRJ//zFd2PgQavcICDUgJT5oW10QCRKbJ6bg4r0/UY2M6BWd5tkxuGFRvCkgfHQ==} @@ -13550,6 +14479,9 @@ packages: search-insights@2.17.3: resolution: {integrity: sha512-RQPdCYTa8A68uM2jwxoY842xDhvx3E5LFL1LxvxCNMev4o5mLuokczhzjAgGwUZBAmOKZknArSxLKmXtIi2AxQ==} + secure-compare@3.0.1: + resolution: {integrity: sha512-AckIIV90rPDcBcglUwXPF3kg0P0qmPsPXAj6BBEENQE1p5yA1xfmDJzfi1Tappj37Pv2mVbKpL3Z1T+Nn7k1Qw==} + semver@6.3.1: resolution: {integrity: sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA==} hasBin: true @@ -13877,6 +14809,10 @@ packages: resolution: {integrity: sha512-+QBBXBCvifc56fsbuxZQ6Sic3wqqc3WWaqxs58gvJrcOuN83HGTCwz3oS5phzU9LthRNE9VrJCFCLUgHeeFnfA==} engines: {node: '>=18'} + union@0.5.0: + resolution: {integrity: sha512-N6uOhuW6zO95P3Mel2I2zMsbsanvvtgn6jVqJv4vbVcz/JN0OkL9suomjQGmWtxJQXOCqUJvquc1sMeNz/IwlA==} + engines: {node: '>= 0.8.0'} + unist-util-is@6.0.1: resolution: {integrity: sha512-LsiILbtBETkDz8I9p1dQ0uyRUWuaQzd/cuEeS1hoRSyW5E5XGmTzlwY1OrNzzakGowI9Dr/I8HVaw4hTtnxy8g==} @@ -13908,6 +14844,9 @@ packages: uri-js@4.4.1: resolution: {integrity: sha512-7rKUyy33Q1yc98pQ1DAmLtwX109F7TIfWlW1Ydo8Wl1ii1SeHieeh0HHfPeL2fMXK6z0s8ecKs9frCuLJvndBg==} + url-join@4.0.1: + resolution: {integrity: sha512-jk1+QP6ZJqyOiuEI9AEWQfju/nB2Pw466kbA0LEZljHwKeMgd9WrAEgEGxjPDD2+TNbbb37rTyhEfrCXfuKXnA==} + use-sync-external-store@1.2.0: resolution: {integrity: sha512-eEgnFxGQ1Ife9bzYs6VLi8/4X6CObHMw9Qr9tPY43iKwsPw8xE8+EFsf/2cFZ5S3esXgpWgtSCtLNS41F+sKPA==} peerDependencies: @@ -14149,6 +15088,11 @@ packages: resolution: {integrity: sha512-BMhLD/Sw+GbJC21C/UgyaZX41nPt8bUTg+jWyDeg7e7YN4xOM05YPSIXceACnXVtqyEw/LMClUQMtMZ+PGGpqQ==} engines: {node: '>=20'} + whatwg-encoding@2.0.0: + resolution: {integrity: sha512-p41ogyeMUrw3jWclHWTQg1k05DSVXPLcVxRTYsXUk+ZooOCZLcoYgPZ/HL/D/N+uQPOtcp1me1WhBEaX02mhWg==} + engines: {node: '>=12'} + deprecated: Use @exodus/bytes instead for a more spec-conformant and faster implementation + whatwg-mimetype@5.0.0: resolution: {integrity: sha512-sXcNcHOC51uPGF0P/D4NVtrkjSU2fNsm9iog4ZvZJsL3rjoDAzXZhkm2MWt1y+PUdggKAYVoMAIYcs78wJ51Cw==} engines: {node: '>=20'} @@ -14199,6 +15143,10 @@ packages: utf-8-validate: optional: true + wsl-utils@0.3.1: + resolution: {integrity: sha512-g/eziiSUNBSsdDJtCLB8bdYEUMj4jR7AGeUo96p/3dTafgjHhpF4RiCFPiRILwjQoDXx5MqkBr4fwWtR3Ky4Wg==} + engines: {node: '>=20'} + xml-name-validator@5.0.0: resolution: {integrity: sha512-EvGK8EJ3DhaHfbRlETOWAS5pO9MZITeauHKJyb8wyajUfQUenkIg2MvLDTZ4T/TgIcm3HU0TFBgWWboAZ30UHg==} engines: {node: '>=18'} @@ -14258,7 +15206,7 @@ packages: snapshots: - '@agentclientprotocol/sdk@0.25.1(zod@4.4.3)': + '@agentclientprotocol/sdk@1.4.0(zod@4.4.3)': dependencies: zod: 4.4.3 @@ -15198,7 +16146,9 @@ snapshots: '@eslint/core': 1.2.1 levn: 0.4.1 - '@exodus/bytes@1.15.1': {} + '@exodus/bytes@1.15.1(@noble/hashes@2.3.0)': + optionalDependencies: + '@noble/hashes': 2.3.0 '@google/genai@1.52.0(@modelcontextprotocol/sdk@1.29.0(zod@4.4.3))': dependencies: @@ -15519,8 +16469,30 @@ snapshots: '@tybys/wasm-util': 0.10.2 optional: true + '@noble/hashes@2.3.0': {} + '@nodable/entities@2.2.0': {} + '@octokit/openapi-types@28.0.0': {} + + '@octokit/openapi-webhooks-types@12.1.0': {} + + '@octokit/request-error@7.1.1': + dependencies: + '@octokit/types': 17.0.0 + + '@octokit/types@17.0.0': + dependencies: + '@octokit/openapi-types': 28.0.0 + + '@octokit/webhooks-methods@6.0.0': {} + + '@octokit/webhooks@14.2.0': + dependencies: + '@octokit/openapi-webhooks-types': 12.1.0 + '@octokit/request-error': 7.1.1 + '@octokit/webhooks-methods': 6.0.0 + '@openai/codex@0.147.0': optionalDependencies: '@openai/codex-darwin-arm64': '@openai/codex@0.147.0-darwin-arm64' @@ -16449,6 +17421,8 @@ snapshots: '@types/unist@3.0.3': {} + '@types/use-sync-external-store@1.5.0': {} + '@types/web-bluetooth@0.0.21': {} '@types/ws@8.18.1': @@ -16493,7 +17467,7 @@ snapshots: obug: 2.1.3 std-env: 4.1.0 tinyrainbow: 3.1.0 - vitest: 4.1.8(@opentelemetry/api@1.9.1)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) + vitest: 4.1.8(@opentelemetry/api@1.9.1)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1(@noble/hashes@2.3.0))(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) '@vitest/expect@4.1.8': dependencies: @@ -16704,6 +17678,11 @@ snapshots: '@yarnpkg/cli-dist@4.17.1': {} + '@yarnpkg/parsers@3.1.0': + dependencies: + js-yaml: 4.3.1 + tslib: 2.8.1 + accepts@2.0.0: dependencies: mime-types: 3.0.2 @@ -16790,6 +17769,8 @@ snapshots: estree-walker: 3.0.3 js-tokens: 10.0.0 + async@3.2.6: {} + balanced-match@1.0.2: {} balanced-match@4.0.4: {} @@ -16798,6 +17779,10 @@ snapshots: baseline-browser-mapping@2.10.43: {} + basic-auth@2.0.1: + dependencies: + safe-buffer: 5.1.2 + bidi-js@1.0.3: dependencies: require-from-string: 2.0.2 @@ -16842,8 +17827,17 @@ snapshots: buffer-equal-constant-time@1.0.1: {} + buffer@6.0.3: + dependencies: + base64-js: 1.5.1 + ieee754: 1.2.1 + builtin-modules@3.3.0: {} + bundle-name@4.1.0: + dependencies: + run-applescript: 7.1.0 + bytes@3.1.2: {} cac@7.0.0: {} @@ -16864,6 +17858,11 @@ snapshots: chai@6.2.2: {} + chalk@4.1.2: + dependencies: + ansi-styles: 4.3.0 + supports-color: 7.2.0 + chalk@5.6.2: {} character-entities-html4@2.1.0: {} @@ -16923,6 +17922,8 @@ snapshots: object-assign: 4.1.1 vary: 1.1.2 + corser@2.0.1: {} + cose-base@1.0.3: dependencies: layout-base: 1.0.2 @@ -17130,10 +18131,10 @@ snapshots: data-uri-to-buffer@4.0.1: {} - data-urls@7.0.0: + data-urls@7.0.0(@noble/hashes@2.3.0): dependencies: whatwg-mimetype: 5.0.0 - whatwg-url: 16.0.1 + whatwg-url: 16.0.1(@noble/hashes@2.3.0) transitivePeerDependencies: - '@noble/hashes' @@ -17151,6 +18152,15 @@ snapshots: deep-is@0.1.4: {} + default-browser-id@5.0.1: {} + + default-browser@5.5.0: + dependencies: + bundle-name: 4.1.0 + default-browser-id: 5.0.1 + + define-lazy-prop@3.0.0: {} + defu@6.1.7: {} delaunator@5.1.0: @@ -17433,6 +18443,8 @@ snapshots: etag@1.8.1: {} + eventemitter3@4.0.7: {} + eventsource-parser@3.1.0: {} eventsource@3.0.7: @@ -17571,6 +18583,8 @@ snapshots: dependencies: tabbable: 6.5.0 + follow-redirects@1.16.0: {} + foreground-child@3.3.1: dependencies: cross-spawn: 7.0.6 @@ -17716,15 +18730,21 @@ snapshots: dependencies: '@types/hast': 3.0.5 + he@1.2.0: {} + hono@4.12.29: {} hookable@5.5.3: {} hookable@6.1.1: {} - html-encoding-sniffer@6.0.0: + html-encoding-sniffer@3.0.0: dependencies: - '@exodus/bytes': 1.15.1 + whatwg-encoding: 2.0.0 + + html-encoding-sniffer@6.0.0(@noble/hashes@2.3.0): + dependencies: + '@exodus/bytes': 1.15.1(@noble/hashes@2.3.0) transitivePeerDependencies: - '@noble/hashes' @@ -17747,6 +18767,33 @@ snapshots: transitivePeerDependencies: - supports-color + http-proxy@1.18.1: + dependencies: + eventemitter3: 4.0.7 + follow-redirects: 1.16.0 + requires-port: 1.0.0 + transitivePeerDependencies: + - debug + + http-server@14.1.1: + dependencies: + basic-auth: 2.0.1 + chalk: 4.1.2 + corser: 2.0.1 + he: 1.2.0 + html-encoding-sniffer: 3.0.0 + http-proxy: 1.18.1 + mime: 1.6.0 + minimist: 1.2.8 + opener: 1.5.2 + portfinder: 1.0.38 + secure-compare: 3.0.1 + union: 0.5.0 + url-join: 4.0.1 + transitivePeerDependencies: + - debug + - supports-color + https-proxy-agent@7.0.6: dependencies: agent-base: 7.1.4 @@ -17764,6 +18811,8 @@ snapshots: dependencies: safer-buffer: 2.1.2 + ieee754@1.2.1: {} + ignore@5.3.2: {} immediate@3.0.6: {} @@ -17786,6 +18835,8 @@ snapshots: ipaddr.js@1.9.1: {} + is-docker@3.0.0: {} + is-extglob@2.1.1: {} is-fullwidth-code-point@3.0.0: {} @@ -17794,6 +18845,12 @@ snapshots: dependencies: is-extglob: 2.1.1 + is-in-ssh@1.0.0: {} + + is-inside-container@1.0.0: + dependencies: + is-docker: 3.0.0 + is-plain-obj@4.1.0: {} is-potential-custom-element-name@1.0.1: {} @@ -17806,6 +18863,10 @@ snapshots: is-what@5.5.0: {} + is-wsl@3.1.1: + dependencies: + is-inside-container: 1.0.0 + isarray@1.0.0: {} isexe@2.0.0: {} @@ -17845,6 +18906,10 @@ snapshots: dependencies: argparse: 2.0.1 + js-yaml@4.3.1: + dependencies: + argparse: 2.0.1 + jscpd-darwin-arm64@5.0.12: optional: true @@ -17872,17 +18937,17 @@ snapshots: jscpd-linux-x64-musl: 5.0.12 jscpd-windows-x64-msvc: 5.0.12 - jsdom@29.1.1: + jsdom@29.1.1(@noble/hashes@2.3.0): dependencies: '@asamuzakjp/css-color': 5.1.11 '@asamuzakjp/dom-selector': 7.1.1 '@bramus/specificity': 2.4.2 '@csstools/css-syntax-patches-for-csstree': 1.1.6(css-tree@3.2.1) - '@exodus/bytes': 1.15.1 + '@exodus/bytes': 1.15.1(@noble/hashes@2.3.0) css-tree: 3.2.1 - data-urls: 7.0.0 + data-urls: 7.0.0(@noble/hashes@2.3.0) decimal.js: 10.6.0 - html-encoding-sniffer: 6.0.0 + html-encoding-sniffer: 6.0.0(@noble/hashes@2.3.0) is-potential-custom-element-name: 1.0.1 lru-cache: 11.5.1 parse5: 8.0.1 @@ -17893,7 +18958,7 @@ snapshots: w3c-xmlserializer: 5.0.0 webidl-conversions: 8.0.1 whatwg-mimetype: 5.0.0 - whatwg-url: 16.0.1 + whatwg-url: 16.0.1(@noble/hashes@2.3.0) xml-name-validator: 5.0.0 transitivePeerDependencies: - '@noble/hashes' @@ -18501,6 +19566,8 @@ snapshots: dependencies: mime-db: 1.54.0 + mime@1.6.0: {} + minimatch@10.2.5: dependencies: brace-expansion: 5.0.6 @@ -18509,6 +19576,8 @@ snapshots: dependencies: brace-expansion: 2.1.2 + minimist@1.2.8: {} + minipass@7.1.3: {} minisearch@7.2.0: {} @@ -18588,7 +19657,7 @@ snapshots: fetch-blob: 3.2.0 formdata-polyfill: 4.0.10 - node-pty@1.1.0(patch_hash=7a0c04f1f49d798a9ffe2f7f414c01064a44ca2489772d0c3e1235ab336755e6): + node-pty@1.2.0-beta.15(patch_hash=b40ae545608897914bd25fb009c97eeac478c34e8a910298ddcb01b746534bb0): dependencies: node-addon-api: 7.1.1 @@ -18630,6 +19699,15 @@ snapshots: regex: 6.1.0 regex-recursion: 6.0.2 + open@11.0.0: + dependencies: + default-browser: 5.5.0 + define-lazy-prop: 3.0.0 + is-in-ssh: 1.0.0 + is-inside-container: 1.0.0 + powershell-utils: 0.1.0 + wsl-utils: 0.3.1 + openai@6.26.0(ws@8.21.0)(zod@4.4.3): optionalDependencies: ws: 8.21.0 @@ -18641,6 +19719,8 @@ snapshots: openapi-typescript-helpers@0.0.15: {} + opener@1.5.2: {} + optionator@0.9.4: dependencies: deep-is: 0.1.4 @@ -18807,12 +19887,21 @@ snapshots: path-data-parser: 0.1.0 points-on-curve: 0.2.0 + portfinder@1.0.38: + dependencies: + async: 3.2.6 + debug: 4.4.3 + transitivePeerDependencies: + - supports-color + postcss@8.5.15: dependencies: nanoid: 3.3.12 picocolors: 1.1.1 source-map-js: 1.2.1 + powershell-utils@0.1.0: {} + preact@10.29.7: {} prelude-ls@1.2.1: {} @@ -18926,6 +20015,8 @@ snapshots: require-from-string@2.0.2: {} + requires-port@1.0.0: {} + resolve-pkg-maps@1.0.0: {} retry@0.13.1: {} @@ -19040,6 +20131,8 @@ snapshots: transitivePeerDependencies: - supports-color + run-applescript@7.1.0: {} + rw@1.3.3: {} sade@1.8.1: @@ -19068,6 +20161,8 @@ snapshots: search-insights@2.17.3: {} + secure-compare@3.0.1: {} + semver@6.3.1: {} semver@7.8.4: {} @@ -19400,6 +20495,10 @@ snapshots: unicorn-magic@0.3.0: {} + union@0.5.0: + dependencies: + qs: 6.15.3 + unist-util-is@6.0.1: dependencies: '@types/unist': 3.0.3 @@ -19440,6 +20539,8 @@ snapshots: dependencies: punycode: 2.3.1 + url-join@4.0.1: {} + use-sync-external-store@1.2.0(react@18.3.1): dependencies: react: 18.3.1 @@ -19583,7 +20684,7 @@ snapshots: - typescript - universal-cookie - vitest@4.1.8(@opentelemetry/api@1.9.0)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@6.4.3(@types/node@22.20.0)(jiti@2.7.0)(lightningcss@1.32.0)(tsx@4.22.4)(yaml@2.9.0)): + vitest@4.1.8(@opentelemetry/api@1.9.0)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1(@noble/hashes@2.3.0))(vite@6.4.3(@types/node@22.20.0)(jiti@2.7.0)(lightningcss@1.32.0)(tsx@4.22.4)(yaml@2.9.0)): dependencies: '@vitest/expect': 4.1.8 '@vitest/mocker': 4.1.8(vite@6.4.3(@types/node@22.20.0)(jiti@2.7.0)(lightningcss@1.32.0)(tsx@4.22.4)(yaml@2.9.0)) @@ -19609,11 +20710,11 @@ snapshots: '@opentelemetry/api': 1.9.0 '@types/node': 22.20.0 '@vitest/coverage-v8': 4.1.8(vitest@4.1.8) - jsdom: 29.1.1 + jsdom: 29.1.1(@noble/hashes@2.3.0) transitivePeerDependencies: - msw - vitest@4.1.8(@opentelemetry/api@1.9.0)(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): + vitest@4.1.8(@opentelemetry/api@1.9.0)(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1(@noble/hashes@2.3.0))(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): dependencies: '@vitest/expect': 4.1.8 '@vitest/mocker': 4.1.8(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) @@ -19639,11 +20740,11 @@ snapshots: '@opentelemetry/api': 1.9.0 '@types/node': 25.9.3 '@vitest/coverage-v8': 4.1.8(vitest@4.1.8) - jsdom: 29.1.1 + jsdom: 29.1.1(@noble/hashes@2.3.0) transitivePeerDependencies: - msw - vitest@4.1.8(@opentelemetry/api@1.9.1)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): + vitest@4.1.8(@opentelemetry/api@1.9.1)(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1(@noble/hashes@2.3.0))(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): dependencies: '@vitest/expect': 4.1.8 '@vitest/mocker': 4.1.8(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) @@ -19669,11 +20770,11 @@ snapshots: '@opentelemetry/api': 1.9.1 '@types/node': 22.20.0 '@vitest/coverage-v8': 4.1.8(vitest@4.1.8) - jsdom: 29.1.1 + jsdom: 29.1.1(@noble/hashes@2.3.0) transitivePeerDependencies: - msw - vitest@4.1.8(@opentelemetry/api@1.9.1)(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): + vitest@4.1.8(@opentelemetry/api@1.9.1)(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1(@noble/hashes@2.3.0))(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): dependencies: '@vitest/expect': 4.1.8 '@vitest/mocker': 4.1.8(vite@8.0.16(@types/node@26.1.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) @@ -19699,7 +20800,7 @@ snapshots: '@opentelemetry/api': 1.9.1 '@types/node': 25.9.3 '@vitest/coverage-v8': 4.1.8(vitest@4.1.8) - jsdom: 29.1.1 + jsdom: 29.1.1(@noble/hashes@2.3.0) transitivePeerDependencies: - msw @@ -19736,11 +20837,15 @@ snapshots: webidl-conversions@8.0.1: {} + whatwg-encoding@2.0.0: + dependencies: + iconv-lite: 0.6.3 + whatwg-mimetype@5.0.0: {} - whatwg-url@16.0.1: + whatwg-url@16.0.1(@noble/hashes@2.3.0): dependencies: - '@exodus/bytes': 1.15.1 + '@exodus/bytes': 1.15.1(@noble/hashes@2.3.0) tr46: 6.0.0 webidl-conversions: 8.0.1 transitivePeerDependencies: @@ -19775,6 +20880,11 @@ snapshots: ws@8.21.0: {} + wsl-utils@0.3.1: + dependencies: + is-wsl: 3.1.1 + powershell-utils: 0.1.0 + xml-name-validator@5.0.0: {} xml-naming@0.1.0: {} diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index e8d8ee5bec..ec6cfd3af9 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -69,4 +69,4 @@ minimumReleaseAgeExclude: - node-addon-require-builtin@0.1.4 patchedDependencies: - node-pty@1.1.0: patches/node-pty@1.1.0.patch + node-pty@1.2.0-beta.15: patches/node-pty@1.2.0-beta.15.patch diff --git a/python/README.i18n.yaml b/python/README.i18n.yaml index 342b3b0017..8c5a23e37b 100644 --- a/python/README.i18n.yaml +++ b/python/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write python/README.md README.md: 75276a915eb4b63f84e0876de46e6d8d63540b59 -README.zh.md: cb6d05cdc2b6395cb460671d863169f5184351b9 +README.zh.md: f41822d66520869e5235a6fedd895b83a52639c9 diff --git a/python/README.zh.md b/python/README.zh.md index cb6d05cdc2..f41822d665 100644 --- a/python/README.zh.md +++ b/python/README.zh.md @@ -8,13 +8,13 @@ | 目录 | 分发名/模块 | 职责 | |---|---|---| -| [sdk](sdk/README.md) | `deepseek-harness-sdk` / `deepseek_harness` | 高层轮次 API 与低层 JSON-RPC 客户端 | -| [sdk-runtime](sdk-runtime/README.md) | `deepseek-harness-runtime-bin` / `deepseek_harness_runtime` | 内置运行时二进制与默认 agent(智能体)配置 | +| [sdk](sdk/README.zh.md) | `deepseek-harness-sdk` / `deepseek_harness` | 高层轮次 API 与低层 JSON-RPC 客户端 | +| [sdk-runtime](sdk-runtime/README.zh.md) | `deepseek-harness-runtime-bin` / `deepseek_harness_runtime` | 内置运行时二进制与默认 agent(智能体)配置 | ## 行为 -除非调用方选择显式通道,否则 SDK 会启动匹配的内置运行时。客户端选择通道并提供默认配置;运行时本身始终要求显式配置。[SDK 参考](sdk/README.md)和[运行时载体参考](sdk-runtime/README.md)定义完整的运行时选择与配置约定。 +除非调用方选择显式通道,否则 SDK 会启动匹配的内置运行时。客户端选择通道并提供默认配置;运行时本身始终要求显式配置。[SDK 参考](sdk/README.zh.md)和[运行时载体参考](sdk-runtime/README.zh.md)定义完整的运行时选择与配置约定。 ## 贡献者工作流 -[Python 贡献者工作流](development.md)介绍运行时产物构建、包验证、源码模式开发和分发。 +[Python 贡献者工作流](development.zh.md)介绍运行时产物构建、包验证、源码模式开发和分发。 diff --git a/python/development.i18n.yaml b/python/development.i18n.yaml index d5570fba11..0e515656a3 100644 --- a/python/development.i18n.yaml +++ b/python/development.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write python/development.md -development.md: 31dc254b58c05c2a19c7c4cd5dc1e53207517902 -development.zh.md: dbb85a0cffc5e06c7ca781b2b01f6993395204e3 +development.md: 6d326a1ac273e9594cbea4714e1bc9449f5e5ca7 +development.zh.md: d07ce7f95a195ea0fe24fa44529dc53b527c766a diff --git a/python/development.md b/python/development.md index 31dc254b58..6d326a1ac2 100644 --- a/python/development.md +++ b/python/development.md @@ -27,6 +27,15 @@ uv run --project python/sdk pytest `python/sdk/tests/test_bundled_runtime.py` exercises available bundled carriers and skips a carrier when its artifact has not been built. For repository-wide test policy, see [Testing](../docs/testing.md). +That suite drives fake runtime peers. `scripts/smoke-python-runtime.py` drives the real packaged runtime instead, and the required `python-runtime` CI job runs every scenario against a freshly built executable: + +```sh +uv run --project python/sdk python scripts/smoke-python-runtime.py \ + --scenario sdk-minimal --exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64 +``` + +Two scenarios compare committed expected output under `scripts/snapshots/python-sdk-single-exe/`. `minimal/model-visible.json` pins the checked-in minimal composition's assembled system prompts, advertised tool schemas, and model-visible messages, so a plugin that contributes an unintended system section or user message fails the job; it drops the dynamic runtime-context snapshot, which the same composition emits on macOS and not on Linux ([#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488)). `advanced/` pins the SDK result and the persisted session logs. Rerun the owning scenario with `--update-snapshots` and review that diff before committing it. + An interactive smoke test needs `DEEPSEEK_API_KEY` in the environment or repository-root `.env`: ```python @@ -41,7 +50,7 @@ with DeepSeekHarness() as harness: Repository contributors can select either development carrier: - Set `DSH_RUNTIME_MODE=node` to use the built Node carrier on system Node `>=22.19`. The build script refreshes this carrier, but distributions never include or auto-select it. -- Set `launch_args_override=("./node_modules/.bin/tsx", "packages/examples/jsonrpc-demo/src/bin.ts")` with the repository root as `cwd` to run unbuilt TypeScript source. Supply `cordis=...` when the default configuration is not suitable. +- Set `launch_args_override=("./node_modules/.bin/tsx", "packages/sdk/python-runtime/src/packaged-bin.ts")` with the repository root as `cwd` to run the private carrier's unbuilt TypeScript source. Supply `cordis=...` when the default configuration is not suitable. See `python/sdk/tests/manual_sdk_agent_smoke.py` for a complete source-mode invocation. @@ -52,17 +61,25 @@ The root `package.json` version is authoritative for both Python distributions. Build the pure SDK wheel once and one runtime wheel on each native platform: ```sh -version="$(node -p "require('./package.json').version")" +version="$(python - <<'PY' +import runpy + +release = runpy.run_path("scripts/build-python-release.py") +print(release["pep440_version"](release["repository_version"]())) +PY +)" python scripts/build-python-release.py --package sdk --output-dir dist-python python scripts/build-python-release.py --package runtime --platform macos-arm64 --runtime-exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64 --output-dir dist-python -pip install --find-links dist-python deepseek-harness-sdk=="$version" +pip install \ + "dist-python/deepseek_harness_sdk-$version-py3-none-any.whl" \ + "dist-python/deepseek_harness_runtime_bin-$version-py3-none-macosx_14_0_arm64.whl" ``` The runtime distribution is wheel-only. The release pipeline publishes three platform wheels with the pure SDK wheel: Linux x64, Linux arm64, and macOS 14 or newer on arm64. A `python-v` tag is accepted only when it matches the repository version; prerelease repository versions such as `0.0.1-rc.1` use their normalized PEP 440 spelling, such as `0.0.1rc1`, inside wheel filenames and metadata. ## Validate a release candidate -Label a pull request `python-release-dry-run`, or manually run the GitHub `Release (Python)` workflow with `publish=false`, to build all four wheels, install the Linux release set on Python 3.10 and 3.14, check exact filenames and metadata, enforce PyPI's default per-file size limit, and retain one aggregate artifact with SHA-256 hashes. Both paths have no registry credentials; a pull request run cannot enter either publication job. +Manually run the GitHub `Release (Python)` workflow with `publish=false` to build all four wheels, install the Linux release set on Python 3.10 and 3.14, check exact filenames and metadata, enforce PyPI's default per-file size limit, and retain one aggregate artifact with SHA-256 hashes. The run has no registry credentials; a dry run cannot enter either publication job. Public publication runs from the private automation repository; package metadata points to the separate read-only public source mirror, which does not run release Actions. The private repository defines the repository variable `PYPI_PUBLISHER_REPOSITORY` as its own `owner/name` and keeps `PUBLIC_PYPI_RELEASE_ENABLED=false` except during an intentional release. diff --git a/python/development.zh.md b/python/development.zh.md index dbb85a0cff..d07ce7f95a 100644 --- a/python/development.zh.md +++ b/python/development.zh.md @@ -2,7 +2,7 @@ [English](development.md) | 中文 -根据所需的贡献者成果选择工作流:构建运行时产物、验证 SDK、从源码运行或构建分发包。包行为分别见 [SDK 参考](sdk/README.md) 和[运行时载体参考](sdk-runtime/README.md)。 +根据所需的贡献者成果选择工作流:构建运行时产物、验证 SDK、从源码运行或构建分发包。包行为分别见 [SDK 参考](sdk/README.zh.md) 和[运行时载体参考](sdk-runtime/README.zh.md)。 ## 构建运行时产物 @@ -25,7 +25,16 @@ uv sync --project python/sdk --group test uv run --project python/sdk pytest ``` -`python/sdk/tests/test_bundled_runtime.py` 会运行可用的内置载体;某个载体的产物尚未构建时,会跳过该载体。仓库级测试政策见 [测试](../docs/testing.md)。 +`python/sdk/tests/test_bundled_runtime.py` 会运行可用的内置载体;某个载体的产物尚未构建时,会跳过该载体。仓库级测试政策见 [测试](../docs/testing.zh.md)。 + +该套件面向的是伪造的运行时对端。`scripts/smoke-python-runtime.py` 面向真实的打包运行时;必需的 `python-runtime` CI 任务会用新构建的可执行文件运行全部场景: + +```sh +uv run --project python/sdk python scripts/smoke-python-runtime.py \ + --scenario sdk-minimal --exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64 +``` + +其中两个场景会比对 `scripts/snapshots/python-sdk-single-exe/` 下已提交的期望输出。`minimal/model-visible.json` 固定了签入的极简组合所组装的系统提示词、对外公布的工具 schema 以及模型可见消息,因此插件一旦贡献出计划外的系统分段或 user 消息,该任务即失败;它会丢弃动态运行时上下文快照——同一组合在 macOS 上会发出它,在 Linux 上不会([#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488))。`advanced/` 固定 SDK 结果与持久化的会话日志。重新运行对应场景时加上 `--update-snapshots`,并在提交前审阅该差异。 交互式冒烟测试需要环境变量或仓库根目录 `.env` 中存在 `DEEPSEEK_API_KEY`: @@ -41,7 +50,7 @@ with DeepSeekHarness() as harness: 仓库贡献者可以选择以下任一开发载体: - 设置 `DSH_RUNTIME_MODE=node`,在系统 Node `>=22.19` 上使用已构建的 Node 载体。构建脚本会刷新该载体,但分发物绝不会包含或自动选择它。 -- 将仓库根目录设为 `cwd`,并设置 `launch_args_override=("./node_modules/.bin/tsx", "packages/examples/jsonrpc-demo/src/bin.ts")`,以运行未构建的 TypeScript 源码。默认配置不合适时,请提供 `cordis=...`。 +- 将仓库根目录设为 `cwd`,并设置 `launch_args_override=("./node_modules/.bin/tsx", "packages/sdk/python-runtime/src/packaged-bin.ts")`,以运行私有载体未构建的 TypeScript 源码。默认配置不合适时,请提供 `cordis=...`。 完整的源码模式调用见 `python/sdk/tests/manual_sdk_agent_smoke.py`。 @@ -52,17 +61,25 @@ with DeepSeekHarness() as harness: 纯 SDK wheel 包只需构建一次;每个原生平台分别构建一个运行时 wheel 包: ```sh -version="$(node -p "require('./package.json').version")" +version="$(python - <<'PY' +import runpy + +release = runpy.run_path("scripts/build-python-release.py") +print(release["pep440_version"](release["repository_version"]())) +PY +)" python scripts/build-python-release.py --package sdk --output-dir dist-python python scripts/build-python-release.py --package runtime --platform macos-arm64 --runtime-exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64 --output-dir dist-python -pip install --find-links dist-python deepseek-harness-sdk=="$version" +pip install \ + "dist-python/deepseek_harness_sdk-$version-py3-none-any.whl" \ + "dist-python/deepseek_harness_runtime_bin-$version-py3-none-macosx_14_0_arm64.whl" ``` 运行时分发包仅提供 wheel 包。发布流水线会连同纯 SDK wheel 包一起发布三个平台 wheel 包:Linux x64、Linux arm64 和 macOS 14 或更高版本的 arm64。只有与仓库版本匹配时,才接受 `python-v` 标签;`0.0.1-rc.1` 之类的仓库预发布版本在 wheel 包文件名和元数据中使用规范化的 PEP 440 写法,例如 `0.0.1rc1`。 ## 验证候选发行版 -为拉取请求添加 `python-release-dry-run` 标签,或手动运行 GitHub 的 `Release (Python)` 工作流并设置 `publish=false`,即可构建全部四个 wheel 包,在 Python 3.10 和 3.14 上安装 Linux 发行集合,检查精确文件名和元数据,执行 PyPI 默认单文件大小限制,并保留一份带 SHA-256 哈希的汇总产物。两条路径都没有注册表凭据,拉取请求运行无法进入任何发布作业。 +手动运行 GitHub 的 `Release (Python)` 工作流并设置 `publish=false`,即可构建全部四个 wheel 包,在 Python 3.10 和 3.14 上安装 Linux 发行集合,检查精确文件名和元数据,执行 PyPI 默认单文件大小限制,并保留一份带 SHA-256 哈希的汇总产物。该运行没有注册表凭据,dry-run 运行无法进入任何发布作业。 公开发布从私有自动化仓库运行;包元数据指向独立的只读公开源码镜像,该镜像不运行发布 Actions。私有仓库把仓库变量 `PYPI_PUBLISHER_REPOSITORY` 定义为自身的 `owner/name`,并且只在有意发布期间把 `PUBLIC_PYPI_RELEASE_ENABLED` 从 `false` 改为 `true`。 diff --git a/python/sdk-runtime/README.i18n.yaml b/python/sdk-runtime/README.i18n.yaml index 1d4b9fd739..c4ccb18bf1 100644 --- a/python/sdk-runtime/README.i18n.yaml +++ b/python/sdk-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write python/sdk-runtime/README.md -README.md: 592ce091f3b6c4bc151dfdee9d313b7c970c95de -README.zh.md: 0a29281a39c885bc3c40e017bdc9475e0065605f +README.md: 67d3842a9255250f66f22ff1f9422b26c3cf3282 +README.zh.md: 47c94b29d68eb915fae5303274fe2888274c1f82 diff --git a/python/sdk-runtime/README.md b/python/sdk-runtime/README.md index 592ce091f3..67d3842a92 100644 --- a/python/sdk-runtime/README.md +++ b/python/sdk-runtime/README.md @@ -8,19 +8,21 @@ Runtime carrier package for the Python SDK (dist `deepseek-harness-runtime-bin`, Two carriers coexist under `src/deepseek_harness_runtime/runtime/`, both injected by the repo's `scripts/build-exe-for-python-sdk.ts` build and both gitignored: -- **exe (production)** — a single-file Node executable `dsh-jsonrpc-agent-pkg--` (platform: `linux`/`macos`; arch: `x64`/`arm64`). macOS builds also ship the native `-spawn-helper` sibling that `node-pty` uses there. No Node installation is needed on the target machine. This is the only carrier that ships in wheel distributions; this package does not publish sdists. -- **node (dev-only)** — the full deploy closure under `runtime/node/` (`package.json` + `node_modules/`), executed as `node runtime/node/node_modules/@deepseek-ai/dsh-sdk-jsonrpc-demo/lib/packaged-bin.js` on a system Node >= 22.19. It is the current checkout's source build, meant for repo-local development and verification only; it is never selected automatically and is excluded from distributions. +- **exe (production)** — a single-file Node executable `dsh-jsonrpc-agent-pkg--` (platform: `linux`/`macos`; arch: `x64`/`arm64`) with a target-native ripgrep `-rg` sidecar. macOS builds also ship the native `-spawn-helper` sibling that `node-pty` uses there. No Node installation is needed on the target machine. This is the only carrier that ships in wheel distributions; this package does not publish sdists. +- **node (dev-only)** — the full deploy closure under `runtime/node/` (`package.json` + `node_modules/`), executed as `node runtime/node/node_modules/@deepseek-ai/dsh-sdk-python-runtime/lib/packaged-bin.js` on a system Node >= 22.19. It is the current checkout's source build, meant for repo-local development and verification only; it is never selected automatically and is excluded from distributions. -Both carriers hold the same content, defined once: the [package.json](https://github.com/deepseek-ai/deepseek-harness/blob/master/python/sdk-runtime/package.json) at this package's root is the deploy root of the single-exe pipeline — a pure dependency manifest (no code of its own) whose dependency closure IS both the plugin set compiled into the exe and the tree materialized into `runtime/node/`. Adding a plugin to the distribution means adding one dependency line there and rebuilding. +Both carriers hold the same content, defined once: the [package.json](https://github.com/deepseek-ai/deepseek-harness/blob/master/python/sdk-runtime/package.json) at this package's root is the private `dsh-sdk-python-runtime-closure` deploy root of the single-exe pipeline — a pure dependency manifest (no code of its own) whose dependency closure IS both the plugin set compiled into the exe and the tree materialized into `runtime/node/`. Adding a plugin to the distribution means adding one dependency line there and rebuilding. + +The bundled plugin set includes `@deepseek-ai/dsh-mcp-client`, so an external Cordis config can connect to stdio or Streamable HTTP MCP servers and expose their tools to the model. The wheel does not bundle MCP server programs or credentials: a stdio config supplies its executable and arguments, while a Streamable HTTP config supplies its URL and headers. The bridge supports MCP tools; MCP Resources and Prompts remain unsupported. A missing exe raises `FileNotFoundError` naming both acquisition routes: build via `scripts/build-exe-for-python-sdk.ts` in a deepseek-harness checkout, or install the matching platform runtime wheel produced by the `build-exe-for-python-sdk` CI workflow. A missing dev-only node carrier names its sole route, the build script. The workflow retains wheels rather than standalone executable archives. Acquisition strategy is deliberately separate from the lookup interface, so an on-demand download can replace it later without touching callers. -Each wheel contains exactly one runtime executable. The macOS wheel also contains its matching native spawn helper; a missing sidecar makes that installation incomplete and is a hard startup error, even for a selected Cordis composition that does not use PTY tools. Linux wheels contain no spawn helper because `node-pty` uses the staged `pty.node` addon directly. The fixed tags are `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, and `py3-none-macosx_14_0_arm64`; the macOS tag conservatively matches the bundled Node 24 executable's macOS 13.5 deployment target. This package's `platforms.json` owns the fixed tag and executable-name pairs used by both the repository release builder and the isolated build hook. The build hook rejects `py3-none-any`, absent or multiple runtime files, non-executable files, and unsupported platform tags. The repository root `package.json` supplies the shared version for this package and the SDK, and a `python-v` release tag must match it. +Each wheel contains exactly one runtime executable and its matching ripgrep `-rg` sidecar. The macOS wheel also contains its matching native spawn helper; any missing sidecar makes that installation incomplete and is a hard startup error, even for a selected Cordis composition that does not use filesystem-search or PTY tools. Linux wheels contain no spawn helper because `node-pty` uses the staged `pty.node` addon directly. The fixed tags are `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, and `py3-none-macosx_14_0_arm64`; the macOS tag conservatively matches the bundled Node 24 executable's macOS 13.5 deployment target. This package's `platforms.json` owns the fixed tag and executable-name pairs used by both the repository release builder and the isolated build hook. The build hook rejects `py3-none-any`, absent or multiple runtime executables, missing or extra sidecars, non-executable files, and unsupported platform tags. The repository root `package.json` supplies the shared version for this package and the SDK, and a `python-v` release tag must match it. ## Resolution API - `resolve_bundled_launch_args(mode=None) -> tuple[str, ...]` — the argv tuple that launches the bundled runtime: `(exe_path,)` in exe mode, `(node_path, bin_js_path)` in node mode. Mode selection: explicit argument > `DSH_RUNTIME_MODE` env var (`exe` | `node`) > automatic. Automatic resolution finds the production exe ONLY — the dev-only node carrier must be opted into explicitly so a production deployment can never silently ride on a source build. -- `bundled_runtime_path() -> Path` — the platform exe path (exe carrier only; on macOS it validates that the required sibling `-spawn-helper` is also installed). The node carrier has no single-path equivalent and launches via the argv tuple above. +- `bundled_runtime_path() -> Path` — the platform exe path (exe carrier only); it validates the required sibling `-rg` sidecar on every platform and the `-spawn-helper` sidecar on macOS. The node carrier has no single-path equivalent and launches via the argv tuple above. - `bundled_default_config_path() -> Path` — the checked-in default config (see below). - `bundled_package_dir() -> Path` — the installed package data root. diff --git a/python/sdk-runtime/README.zh.md b/python/sdk-runtime/README.zh.md index 0a29281a39..47c94b29d6 100644 --- a/python/sdk-runtime/README.zh.md +++ b/python/sdk-runtime/README.zh.md @@ -8,19 +8,21 @@ Python SDK 的运行时载体包(分发名 `deepseek-harness-runtime-bin`, 两种载体并存于 `src/deepseek_harness_runtime/runtime/` 之下,均由仓库的 `scripts/build-exe-for-python-sdk.ts` 构建注入,且均被 git 忽略: -- **exe(生产)**——单文件 Node 可执行程序 `dsh-jsonrpc-agent-pkg--`(platform:`linux`/`macos`;arch:`x64`/`arm64`)。macOS 构建还会随附 `node-pty` 在该平台使用的原生 `-spawn-helper` 伴随文件。目标机器无需安装 Node。这是唯一随 wheel 包分发的载体;本包不发布 sdist。 -- **node(仅限开发)**——`runtime/node/` 下的完整部署闭包(`package.json` + `node_modules/`),在系统 Node >= 22.19 上以 `node runtime/node/node_modules/@deepseek-ai/dsh-sdk-jsonrpc-demo/lib/packaged-bin.js` 执行。它是当前检出的源码构建,仅用于仓库本地的开发与验证;不会被自动选中,也不进入分发物。 +- **exe(生产)**——单文件 Node 可执行程序 `dsh-jsonrpc-agent-pkg--`(platform:`linux`/`macos`;arch:`x64`/`arm64`),以及匹配目标平台的 ripgrep `-rg` 伴随文件。macOS 构建还会随附 `node-pty` 在该平台使用的原生 `-spawn-helper` 伴随文件。目标机器无需安装 Node。这是唯一随 wheel 包分发的载体;本包不发布 sdist。 +- **node(仅限开发)**——`runtime/node/` 下的完整部署闭包(`package.json` + `node_modules/`),在系统 Node >= 22.19 上以 `node runtime/node/node_modules/@deepseek-ai/dsh-sdk-python-runtime/lib/packaged-bin.js` 执行。它是当前检出的源码构建,仅用于仓库本地的开发与验证;不会被自动选中,也不进入分发物。 -两种载体承载相同的内容,且只定义一次:本包根目录的 [package.json](https://github.com/deepseek-ai/deepseek-harness/blob/master/python/sdk-runtime/package.json) 是 single-exe 流水线的部署根目录——一份零代码的纯依赖 manifest,其依赖闭包既是编译进 exe 的插件集,也是物化到 `runtime/node/` 的文件树。往分发物里加插件,就是在那里加一行依赖再重新构建。 +两种载体承载相同的内容,且只定义一次:本包根目录的 [package.json](https://github.com/deepseek-ai/deepseek-harness/blob/master/python/sdk-runtime/package.json) 是 single-exe 流水线的私有 `dsh-sdk-python-runtime-closure` 部署根目录——一份零代码的纯依赖 manifest,其依赖闭包既是编译进 exe 的插件集,也是物化到 `runtime/node/` 的文件树。往分发物里加插件,就是在那里加一行依赖再重新构建。 + +内置插件集合包含 `@deepseek-ai/dsh-mcp-client`,因此外部 Cordis 配置可以连接 stdio 或 Streamable HTTP MCP server,并向模型提供这些 server 的工具。wheel 包不包含 MCP server 程序或凭据:stdio 配置需要提供可执行程序及其参数,Streamable HTTP 配置需要提供 URL 和请求头。该桥接仅支持 MCP 工具,尚不支持 MCP Resources 与 Prompts。 exe 缺失时抛出 `FileNotFoundError`,并写明两种获取途径:在 deepseek-harness 检出中经 `scripts/build-exe-for-python-sdk.ts` 构建,或安装 `build-exe-for-python-sdk` CI 工作流生成的对应平台运行时 wheel 包。仅限开发的 node 载体缺失时只提示构建脚本这一条途径。该工作流只保留 wheel 包,不保留独立 exe 归档。获取策略与查找接口刻意分离,之后可以换成按需下载而不改动任何调用方。 -每个 wheel 包只包含一个运行时可执行文件。macOS wheel 包还包含与其匹配的原生 spawn helper;缺少伴随文件意味着该安装不完整,并会在启动时硬失败,即使所选 Cordis 组合不使用 PTY 工具也是如此。Linux wheel 包不包含 spawn helper,因为 `node-pty` 直接使用暂存的 `pty.node` 原生插件。固定标签为 `py3-none-manylinux_2_28_x86_64`、`py3-none-manylinux_2_28_aarch64` 与 `py3-none-macosx_14_0_arm64`;macOS 标签保守匹配内置 Node 24 可执行文件的 macOS 13.5 部署目标。本包的 `platforms.json` 统一定义仓库发行构建器与隔离构建钩子使用的固定标签和可执行文件名。构建钩子会拒绝 `py3-none-any`、不存在运行时文件、存在多个运行时文件、文件不可执行以及不支持的平台标签。仓库根目录的 `package.json` 为本包和 SDK 提供共同版本,`python-v` 发布标签必须与其匹配。 +每个 wheel 包只包含一个运行时可执行文件及其匹配的 ripgrep `-rg` 伴随文件。macOS wheel 包还包含与其匹配的原生 spawn helper;缺少任一伴随文件都意味着该安装不完整,并会在启动时硬失败,即使所选 Cordis 组合不使用文件系统搜索或 PTY 工具也是如此。Linux wheel 包不包含 spawn helper,因为 `node-pty` 直接使用暂存的 `pty.node` 原生插件。固定标签为 `py3-none-manylinux_2_28_x86_64`、`py3-none-manylinux_2_28_aarch64` 与 `py3-none-macosx_14_0_arm64`;macOS 标签保守匹配内置 Node 24 可执行文件的 macOS 13.5 部署目标。本包的 `platforms.json` 统一定义仓库发行构建器与隔离构建钩子使用的固定标签和可执行文件名。构建钩子会拒绝 `py3-none-any`、不存在或存在多个运行时可执行文件、缺失或多余的伴随文件、文件不可执行以及不支持的平台标签。仓库根目录的 `package.json` 为本包和 SDK 提供共同版本,`python-v` 发布标签必须与其匹配。 ## 解析 API - `resolve_bundled_launch_args(mode=None) -> tuple[str, ...]`——启动内置运行时的 argv 元组:exe 模式下为 `(exe_path,)`,node 模式下为 `(node_path, bin_js_path)`。模式选择:显式参数 > `DSH_RUNTIME_MODE` 环境变量(`exe` | `node`)> 自动。自动解析只找生产 exe——仅限开发的 node 载体必须显式选用,从而生产部署绝不会悄悄跑在源码构建上。 -- `bundled_runtime_path() -> Path`——平台 exe 路径(仅 exe 载体,并会在 macOS 上校验必要的 `-spawn-helper` 伴随文件也已安装)。node 载体没有单一路径的等价物,经由上面的 argv 元组启动。 +- `bundled_runtime_path() -> Path`——平台 exe 路径(仅 exe 载体);它会在所有平台校验必要的 `-rg` 伴随文件,并在 macOS 上额外校验 `-spawn-helper` 伴随文件。node 载体没有单一路径的等价物,经由上面的 argv 元组启动。 - `bundled_default_config_path() -> Path`——检入的默认配置(见下文)。 - `bundled_package_dir() -> Path`——已安装包的数据根目录。 diff --git a/python/sdk-runtime/hatch_build.py b/python/sdk-runtime/hatch_build.py index 400d9d585b..ef5a621f9d 100644 --- a/python/sdk-runtime/hatch_build.py +++ b/python/sdk-runtime/hatch_build.py @@ -67,7 +67,7 @@ class RuntimeBuildHook(BuildHookInterface): expected_executable = matches[0][1] runtime_dir = Path(self.root) / "src" / "deepseek_harness_runtime" / "runtime" runtime_files = sorted(runtime_dir.glob("dsh-jsonrpc-agent-pkg-*") if runtime_dir.is_dir() else []) - expected_files = [expected_executable] + expected_files = [expected_executable, f"{expected_executable}-rg"] if "-macos-" in expected_executable: expected_files.append(f"{expected_executable}-spawn-helper") found_files = [path.name for path in runtime_files] diff --git a/python/sdk-runtime/package.json b/python/sdk-runtime/package.json index 89d4b81b22..dad5db82b7 100644 --- a/python/sdk-runtime/package.json +++ b/python/sdk-runtime/package.json @@ -1,5 +1,5 @@ { - "name": "dsh-jsonrpc-agent-pkg", + "name": "dsh-sdk-python-runtime-closure", "description": "Dependency-only deploy root defining the executable and Python runtime closure; pnpm deploy materializes this manifest and node_modules.", "version": "0.0.1", "private": true, @@ -14,20 +14,24 @@ "@deepseek-ai/dsh-agent": "workspace:^", "@deepseek-ai/dsh-agent-loop": "workspace:^", "@deepseek-ai/dsh-agent-spine-demo": "workspace:^", + "@deepseek-ai/dsh-agent-tool-presentation": "workspace:^", "@deepseek-ai/dsh-app-boot": "workspace:^", "@deepseek-ai/dsh-attachment": "workspace:^", + "@deepseek-ai/dsh-atomic-write": "workspace:^", "@deepseek-ai/dsh-shell": "workspace:^", "@deepseek-ai/dsh-shell-env": "workspace:^", "@deepseek-ai/dsh-bash-local": "workspace:^", "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-code-runtime": "workspace:^", "@deepseek-ai/dsh-code-runtime-worker-thread": "workspace:^", + "@deepseek-ai/dsh-command-compact": "workspace:^", "@deepseek-ai/dsh-command-goal": "workspace:^", "@deepseek-ai/dsh-commands": "workspace:^", "@deepseek-ai/dsh-compaction": "workspace:^", "@deepseek-ai/dsh-compaction-basic": "workspace:^", "@deepseek-ai/dsh-compaction-tool-result-pruner": "workspace:^", "@deepseek-ai/dsh-cordis-host-runner": "workspace:^", + "@deepseek-ai/dsh-authorization": "workspace:^", "@deepseek-ai/dsh-credentials": "workspace:^", "@deepseek-ai/dsh-launch-environment": "workspace:^", "@deepseek-ai/dsh-fs": "workspace:^", @@ -41,14 +45,20 @@ "@deepseek-ai/dsh-hooks-codex": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-sdk-jsonrpc-server": "workspace:^", - "@deepseek-ai/dsh-sdk-jsonrpc-demo": "workspace:^", + "@deepseek-ai/dsh-sdk-python-runtime": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-llm-deepseek": "workspace:^", + "@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^", + "@deepseek-ai/dsh-plugin-package-inventory-deepseek": "workspace:^", + "@deepseek-ai/dsh-session-log-deepseek": "workspace:^", "@deepseek-ai/dsh-llm-pi-ai": "workspace:^", "@deepseek-ai/dsh-llm-retry": "workspace:^", + "@deepseek-ai/dsh-mcp-client": "workspace:^", "@deepseek-ai/dsh-home-paths": "workspace:^", "@deepseek-ai/dsh-permission-presets": "workspace:^", "@deepseek-ai/dsh-plan-mode": "workspace:^", + "@deepseek-ai/dsh-persona": "workspace:^", + "@deepseek-ai/dsh-pwsh-local": "workspace:^", "@deepseek-ai/dsh-terminal": "workspace:^", "@deepseek-ai/dsh-terminal-bash": "workspace:^", "@deepseek-ai/dsh-repeat-tool-reminder": "workspace:^", @@ -71,6 +81,7 @@ "@deepseek-ai/dsh-settings": "workspace:^", "@deepseek-ai/dsh-skill": "workspace:^", "@deepseek-ai/dsh-skill-filesystem": "workspace:^", + "@deepseek-ai/dsh-spill": "workspace:^", "@deepseek-ai/dsh-subagent": "workspace:^", "@deepseek-ai/dsh-subagent-acp": "workspace:^", "@deepseek-ai/dsh-subagent-fork-in-process": "workspace:^", @@ -89,7 +100,9 @@ "@deepseek-ai/dsh-tool-bash-persistent": "workspace:^", "@deepseek-ai/dsh-tool-cordis": "workspace:^", "@deepseek-ai/dsh-tool-fs": "workspace:^", + "@deepseek-ai/dsh-tool-fs-search": "workspace:^", "@deepseek-ai/dsh-tool-goal": "workspace:^", + "@deepseek-ai/dsh-tool-ralph": "workspace:^", "@deepseek-ai/dsh-tool-skill": "workspace:^", "@deepseek-ai/dsh-tool-str-replace-editor": "workspace:^", "@deepseek-ai/dsh-tool-subagent": "workspace:^", diff --git a/python/sdk-runtime/src/deepseek_harness_runtime/__init__.py b/python/sdk-runtime/src/deepseek_harness_runtime/__init__.py index 16af912def..94aca48d9f 100644 --- a/python/sdk-runtime/src/deepseek_harness_runtime/__init__.py +++ b/python/sdk-runtime/src/deepseek_harness_runtime/__init__.py @@ -5,11 +5,11 @@ Two runtime carriers coexist under ``runtime/``, both injected by the repo's - **exe (production)**: single-file Node executables named ``dsh-jsonrpc-agent-pkg--`` (platform in {linux, macos}, arch in - {x64, arm64}); macOS also uses a sibling ``-spawn-helper``. The target machine - needs no Node installation. + {x64, arm64}) with a sibling ``-rg`` executable; macOS also uses a sibling + ``-spawn-helper``. The target machine needs no Node installation. - **node (dev-only)**: the full deploy closure under ``runtime/node/`` (``package.json`` + ``node_modules/``), executed as ``node - runtime/node/node_modules/@deepseek-ai/dsh-sdk-jsonrpc-demo/lib/packaged-bin.js`` on a + runtime/node/node_modules/@deepseek-ai/dsh-sdk-python-runtime/lib/packaged-bin.js`` on a system Node >= 22.19. It is the current checkout's source build, never selected automatically, and excluded from wheel/sdist distributions. @@ -71,10 +71,11 @@ def bundled_runtime_path() -> Path: """Absolute path of the bundled single-file runtime executable for the current platform. Raises FileNotFoundError when the platform is unsupported, the executable - has not been placed into this package, or the required macOS spawn helper is - missing; the message names the acquisition routes (acquisition strategy is - deliberately separate from this lookup interface, so an on-demand download - can replace it without touching callers). + has not been placed into this package, the required ripgrep sidecar is + missing, or the required macOS spawn helper is missing; the message names + the acquisition routes (acquisition strategy is deliberately separate from + this lookup interface, so an on-demand download can replace it without + touching callers). """ tag = _current_platform_tag() path = bundled_package_dir() / "runtime" / f"dsh-jsonrpc-agent-pkg-{tag}" @@ -83,6 +84,12 @@ def bundled_runtime_path() -> Path: f"deepseek-harness-runtime-bin is missing the runtime executable at {path}. " + _EXE_ACQUISITION_HINT ) + ripgrep = Path(f"{path}-rg") + if not ripgrep.is_file(): + raise FileNotFoundError( + f"deepseek-harness-runtime-bin is missing the ripgrep sidecar at {ripgrep}. " + + _EXE_ACQUISITION_HINT + ) if tag.startswith("macos-"): helper = Path(f"{path}-spawn-helper") if not helper.is_file(): @@ -134,7 +141,7 @@ def _node_launch_args() -> tuple[str, str]: node_root / "node_modules" / "@deepseek-ai" - / "dsh-sdk-jsonrpc-demo" + / "dsh-sdk-python-runtime" / "lib" / "packaged-bin.js" ) diff --git a/python/sdk-runtime/src/deepseek_harness_runtime/runtime/cordis.yml b/python/sdk-runtime/src/deepseek_harness_runtime/runtime/cordis.yml index 4b9caf846a..02f8cac145 100644 --- a/python/sdk-runtime/src/deepseek_harness_runtime/runtime/cordis.yml +++ b/python/sdk-runtime/src/deepseek_harness_runtime/runtime/cordis.yml @@ -17,6 +17,15 @@ # credential seam and, with no provider mounted here, from the launching # environment; DEEPSEEK_BASE_URL follows the same environment ladder. Neither # is inlined, so this file names no secret and no route. +- id: deepseek-llm-api-extensions + name: '@deepseek-ai/dsh-deepseek-llm-api-extensions' + +- id: session-log-deepseek + name: '@deepseek-ai/dsh-session-log-deepseek' + +- id: plugin-package-inventory-deepseek + name: '@deepseek-ai/dsh-plugin-package-inventory-deepseek' + - id: llm-deepseek name: '@deepseek-ai/dsh-llm-deepseek' diff --git a/python/sdk/README.i18n.yaml b/python/sdk/README.i18n.yaml index 23ae780335..4b93a8d04e 100644 --- a/python/sdk/README.i18n.yaml +++ b/python/sdk/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write python/sdk/README.md -README.md: 9de0791f1394640ccdf757ecc170a9e803d49969 -README.zh.md: c1c0c673216eb0170546d488bd18bc743639ad0b +README.md: 99515c52e6314dc29788a324338699d75c0a4451 +README.zh.md: 6ec268545107478ce9f347cdfb7f17d4a8afd151 diff --git a/python/sdk/README.md b/python/sdk/README.md index 9de0791f13..99515c52e6 100644 --- a/python/sdk/README.md +++ b/python/sdk/README.md @@ -33,14 +33,14 @@ with DeepSeekHarness( provider="deepseek-official", model="deepseek-v4-flash", max_tokens=49_152, - cordis="examples/jsonrpc-agent/cordis.yml", + cordis="examples/python-sdk-agent/cordis.yml", ) as harness: result = harness.run("Make the requested code change.") ``` `provider` selects a provider route registered by the chosen Cordis composition; `model` is the model id resolved by that adapter. `max_tokens` is an optional positive per-request output-token cap for the root agent and its in-process descendants; omission leaves the provider default in control. Compaction summaries keep the separate limit configured by their compaction plugin. The bundled default composition registers `deepseek-official`. A custom composition can mount `llm-pi-ai`, configure provider-specific credentials/endpoints there, and select any provider/model present in pi-ai's installed catalog. -The [Python SDK tutorial](https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/user/guide/python-sdk.md) provides an ordered installation and first-run path without the Web UI. The [`jsonrpc-agent` example](https://github.com/deepseek-ai/deepseek-harness/blob/master/examples/jsonrpc-agent/README.md) owns the complete standalone Cordis file used there. +The [Python SDK tutorial](https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/user/guide/python-sdk.md) provides an ordered installation and first-run path without the Web UI. The [`python-sdk-agent` example](https://github.com/deepseek-ai/deepseek-harness/blob/master/examples/python-sdk-agent/README.md) owns the complete standalone Cordis file used there. `Session.run()` owns an activity interval from its prompt's durable inbox receipt through the next whole-agent idle and returns `RunResult(session_id, final_response, finish_reason, events, notifications, session_root)`. `final_response` is the last committed root-session assistant text in the interval. `finish_reason` is the `kind` of the last root-session `turn/end` in the interval, such as `completed`, `max-tokens`, or `error`, and is `None` when no turn ended. A `turn/end` without a string `data.reason.kind` violates the runtime protocol and raises `SdkProtocolError`. Both result fields describe the owned interval rather than an output or ending causally assigned to the prompt. Steering, injected context, and other queued work may contribute before idle. diff --git a/python/sdk/README.zh.md b/python/sdk/README.zh.md index c1c0c67321..6ec2685451 100644 --- a/python/sdk/README.zh.md +++ b/python/sdk/README.zh.md @@ -30,14 +30,14 @@ with DeepSeekHarness( provider="deepseek-official", model="deepseek-v4-flash", max_tokens=49_152, - cordis="examples/jsonrpc-agent/cordis.yml", + cordis="examples/python-sdk-agent/cordis.yml", ) as harness: result = harness.run("Make the requested code change.") ``` `provider` 选择指定 Cordis 组合所注册的提供方路由;`model` 是该适配器解析出的模型 ID。`max_tokens` 是一个可选的正整数,用于限制根 agent 及其进程内后代在每次请求中输出的 token 数量;省略该参数时,由提供方的默认行为决定输出上限。压缩摘要继续使用压缩插件单独配置的上限。内置默认组合注册 `deepseek-official`。自定义组合可以挂载 `llm-pi-ai`,在其中配置各提供方专属的凭据和端点,并选择 pi-ai 已安装 catalog 中存在的任意提供方/模型组合。 -[Python SDK 教程](https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/user/guide/python-sdk.md)提供一套无需使用 Web UI、按步骤完成安装和首次运行的流程。该教程所用的完整独立 Cordis 配置文件位于 [`jsonrpc-agent` 示例](https://github.com/deepseek-ai/deepseek-harness/blob/master/examples/jsonrpc-agent/README.md)中。 +[Python SDK 教程](https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/user/guide/python-sdk.md)提供一套无需使用 Web UI、按步骤完成安装和首次运行的流程。该教程所用的完整独立 Cordis 配置文件位于 [`python-sdk-agent` 示例](https://github.com/deepseek-ai/deepseek-harness/blob/master/examples/python-sdk-agent/README.md)中。 `Session.run()` 的活动区间从其提示词被持久 inbox 接收时开始,到整个 agent 下一次进入空闲状态时结束,并返回 `RunResult(session_id, final_response, finish_reason, events, notifications, session_root)`。`final_response` 是该区间内根会话最后提交的助手文本。`finish_reason` 是该区间内根会话最后一个 `turn/end` 的 `kind`,例如 `completed`、`max-tokens` 或 `error`;没有轮次结束时为 `None`。缺少字符串 `data.reason.kind` 的 `turn/end` 违反运行时协议,并会抛出 `SdkProtocolError`。这两个结果字段描述的是 `Session.run()` 所界定的活动区间,并不表示某项输出或结束原因在因果上归属于该提示词。steering(中途引导)、注入的上下文和其他排队工作,也可能在 agent 进入空闲状态前参与这段活动。 diff --git a/python/sdk/tests/manual_sdk_agent_smoke.py b/python/sdk/tests/manual_sdk_agent_smoke.py index ae94f1bb07..c0305d9211 100644 --- a/python/sdk/tests/manual_sdk_agent_smoke.py +++ b/python/sdk/tests/manual_sdk_agent_smoke.py @@ -44,7 +44,7 @@ class MockCompletionHandler(BaseHTTPRequestHandler): def run_smoke(repo_root: Path, keep_sessions: bool) -> None: session_root = Path(tempfile.mkdtemp(prefix="dsh-sdk-smoke-sessions-")) - runtime_entry = repo_root / "packages/examples/jsonrpc-demo/src/bin.ts" + runtime_entry = repo_root / "packages/sdk/python-runtime/src/packaged-bin.ts" server = ThreadingHTTPServer(("127.0.0.1", 0), MockCompletionHandler) thread = threading.Thread(target=server.serve_forever, name="mock-openai-compatible-server", daemon=True) thread.start() diff --git a/python/sdk/tests/test_bundled_runtime.py b/python/sdk/tests/test_bundled_runtime.py index 791a164d1d..52d84cd161 100644 --- a/python/sdk/tests/test_bundled_runtime.py +++ b/python/sdk/tests/test_bundled_runtime.py @@ -16,7 +16,7 @@ from deepseek_harness_runtime import resolve_bundled_launch_args _MODES = ("exe", "node") _REPO_ROOT = Path(__file__).parents[3] -_MINIMAL_CONFIG = _REPO_ROOT / "examples" / "jsonrpc-agent" / "minimal.cordis.yml" +_MINIMAL_CONFIG = _REPO_ROOT / "examples" / "python-sdk-agent" / "minimal.cordis.yml" # The config must include the JSON-RPC serving plugin. _CORDIS_YML = """\ diff --git a/python/sdk/tests/test_release_version.py b/python/sdk/tests/test_release_version.py index 7ff9b15d59..b21a485949 100644 --- a/python/sdk/tests/test_release_version.py +++ b/python/sdk/tests/test_release_version.py @@ -92,6 +92,10 @@ def test_stage_runtime_copies_platform_payload( executable.write_bytes(b"runtime") executable.chmod(0o755) expected = {executable.name: b"runtime"} + ripgrep = Path(f"{executable}-rg") + ripgrep.write_bytes(b"ripgrep") + ripgrep.chmod(0o755) + expected[ripgrep.name] = b"ripgrep" if with_helper: spawn_helper = Path(f"{executable}-spawn-helper") spawn_helper.write_bytes(b"helper") diff --git a/python/sdk/tests/test_runtime_resolution.py b/python/sdk/tests/test_runtime_resolution.py index 778203f4d1..14e90f3283 100644 --- a/python/sdk/tests/test_runtime_resolution.py +++ b/python/sdk/tests/test_runtime_resolution.py @@ -51,7 +51,10 @@ def test_runtime_requires_spawn_helper_only_on_macos( runtime_dir.mkdir() linux = runtime_dir / "dsh-jsonrpc-agent-pkg-linux-x64" linux.touch() - (runtime_dir / "dsh-jsonrpc-agent-pkg-macos-arm64").touch() + Path(f"{linux}-rg").touch() + macos = runtime_dir / "dsh-jsonrpc-agent-pkg-macos-arm64" + macos.touch() + Path(f"{macos}-rg").touch() monkeypatch.setattr(runtime, "bundled_package_dir", lambda: tmp_path) monkeypatch.setattr(runtime, "_current_platform_tag", lambda: "macos-arm64") @@ -59,3 +62,16 @@ def test_runtime_requires_spawn_helper_only_on_macos( runtime.bundled_runtime_path() monkeypatch.setattr(runtime, "_current_platform_tag", lambda: "linux-x64") assert runtime.bundled_runtime_path() == linux + + +def test_runtime_requires_ripgrep_sidecar( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + runtime_dir = tmp_path / "runtime" + runtime_dir.mkdir() + (runtime_dir / "dsh-jsonrpc-agent-pkg-linux-x64").touch() + monkeypatch.setattr(runtime, "bundled_package_dir", lambda: tmp_path) + monkeypatch.setattr(runtime, "_current_platform_tag", lambda: "linux-x64") + + with pytest.raises(FileNotFoundError, match="ripgrep sidecar"): + runtime.bundled_runtime_path() diff --git a/python/sdk/tests/test_smoke_model.py b/python/sdk/tests/test_smoke_model.py index d4be72032d..c0e6b9f70e 100644 --- a/python/sdk/tests/test_smoke_model.py +++ b/python/sdk/tests/test_smoke_model.py @@ -30,3 +30,44 @@ def test_child_prompt_precedes_runtime_context(prompt_name: str, expected: str) for chunk in chunks for choice in chunk.get("choices", []) ) + + +def test_mcp_smoke_requests_the_discovered_tool() -> None: + chunks = SMOKE["completion_chunks"]({ + "messages": [{"role": "user", "content": SMOKE["MCP_PROMPT"]}], + "tools": [{"type": "function", "function": {"name": "mcp__fixture__add"}}], + }) + + calls = [ + call + for chunk in chunks + for choice in chunk.get("choices", []) + for call in choice.get("delta", {}).get("tool_calls", []) + ] + assert calls[0]["function"] == { + "name": "mcp__fixture__add", + "arguments": '{"a": 19, "b": 23}', + } + + +def test_mcp_smoke_accepts_the_external_server_result() -> None: + chunks = SMOKE["completion_chunks"]({ + "messages": [ + {"role": "user", "content": SMOKE["MCP_PROMPT"]}, + { + "role": "assistant", + "tool_calls": [{ + "id": "mcp-add", + "type": "function", + "function": {"name": "mcp__fixture__add", "arguments": '{}'}, + }], + }, + {"role": "tool", "tool_call_id": "mcp-add", "content": "42"}, + ], + }) + + assert any( + choice.get("delta", {}).get("content") == SMOKE["MCP_TEXT"] + for chunk in chunks + for choice in chunk.get("choices", []) + ) diff --git a/scripts/build-exe-for-python-sdk-native-pty.spec.ts b/scripts/build-exe-for-python-sdk-native-pty.spec.ts new file mode 100644 index 0000000000..5dd6588955 --- /dev/null +++ b/scripts/build-exe-for-python-sdk-native-pty.spec.ts @@ -0,0 +1,49 @@ +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { resolveLinuxNodePtyAddon } from './build-exe-for-python-sdk-native-pty.ts' + +const roots: string[] = [] + +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }) +}) + +describe('resolveLinuxNodePtyAddon', () => { + it('prefers the manylinux build produced by the release workflow', () => { + const root = temporaryPackage() + const built = createAddon(root, 'build', 'Release', 'pty.node') + createAddon(root, 'prebuilds', 'linux-x64', 'pty.node') + + expect(resolveLinuxNodePtyAddon(root, 'x64')).toBe(built) + }) + + it('uses the target prebuild after an ordinary beta install', () => { + const root = temporaryPackage() + const prebuilt = createAddon(root, 'prebuilds', 'linux-arm64', 'pty.node') + + expect(resolveLinuxNodePtyAddon(root, 'arm64')).toBe(prebuilt) + }) + + it('reports both expected locations when no addon is installed', () => { + const root = temporaryPackage() + + expect(() => resolveLinuxNodePtyAddon(root, 'x64')).toThrow( + `node-pty addon is absent from both ${join(root, 'build', 'Release', 'pty.node')} and ${join(root, 'prebuilds', 'linux-x64', 'pty.node')}`, + ) + }) +}) + +function temporaryPackage(): string { + const root = mkdtempSync(join(tmpdir(), 'dsh-node-pty-addon-')) + roots.push(root) + return root +} + +function createAddon(root: string, ...segments: string[]): string { + const path = join(root, ...segments) + mkdirSync(dirname(path), { recursive: true }) + writeFileSync(path, '') + return path +} diff --git a/scripts/build-exe-for-python-sdk-native-pty.ts b/scripts/build-exe-for-python-sdk-native-pty.ts new file mode 100644 index 0000000000..02fa864d73 --- /dev/null +++ b/scripts/build-exe-for-python-sdk-native-pty.ts @@ -0,0 +1,23 @@ +/** Resolve the native node-pty input used by the Python SDK runtime builder. */ + +import { existsSync } from 'node:fs' +import { join } from 'node:path' + +/** + * Prefer the workflow's manylinux build and fall back to node-pty's target prebuild. + * @param packageDirectory - installed node-pty package directory. + * @param arch - Linux target architecture. + * @returns the existing addon path. + */ +export function resolveLinuxNodePtyAddon( + packageDirectory: string, + arch: 'x64' | 'arm64', +): string { + const built = join(packageDirectory, 'build', 'Release', 'pty.node') + if (existsSync(built)) return built + const prebuilt = join(packageDirectory, 'prebuilds', `linux-${arch}`, 'pty.node') + if (existsSync(prebuilt)) return prebuilt + throw new Error( + `build-exe-for-python-sdk: node-pty addon is absent from both ${built} and ${prebuilt}.`, + ) +} diff --git a/scripts/build-exe-for-python-sdk.ts b/scripts/build-exe-for-python-sdk.ts index da1cea67c4..7516fcabb6 100644 --- a/scripts/build-exe-for-python-sdk.ts +++ b/scripts/build-exe-for-python-sdk.ts @@ -11,13 +11,15 @@ import { existsSync, statSync } from 'node:fs' import { chmod, copyFile, cp, lstat, mkdir, readFile, readdir, realpath, rm, writeFile } from 'node:fs/promises' import { basename, dirname, join, resolve, sep } from 'node:path' import { parseArgs } from 'node:util' +import { resolveLinuxNodePtyAddon } from './build-exe-for-python-sdk-native-pty.ts' const root = resolve(import.meta.dirname, '..') /** The closure manifest whose dependencies define the executable. */ -const DEPLOY_ROOT_PACKAGE = 'dsh-jsonrpc-agent-pkg' +const DEPLOY_ROOT_PACKAGE = 'dsh-sdk-python-runtime-closure' /** The closed-runtime app entry inside the deployed closure. */ -const ENTRY_BIN = 'node_modules/@deepseek-ai/dsh-sdk-jsonrpc-demo/lib/packaged-bin.js' +const ENTRY_BIN = 'node_modules/@deepseek-ai/dsh-sdk-python-runtime/lib/packaged-bin.js' +/** Stable Python-visible executable basename; rename with the later Python runtime migration. */ const OUTPUT_BASENAME = 'dsh-jsonrpc-agent-pkg' /** Default Node major; SEA mode requires at least Node 22. */ const DEFAULT_NODE_RANGE = 'node24' @@ -377,7 +379,7 @@ class SingleExeBuild { /** * Package one target; SEA mode accepts one target per invocation. * @param target - the pkg target triple to build. - * @returns the executable path and, on macOS, its helper path. + * @returns the executable and ripgrep sidecar paths, plus the macOS spawn helper path when required. */ async pack(target: Target): Promise { const product = join(this.outDir, `${OUTPUT_BASENAME}-${target.platform}-${target.arch}`) @@ -396,7 +398,8 @@ class SingleExeBuild { if (!this.cli.dryRun && !existsSync(product)) { throw new Error(`build-exe-for-python-sdk: product ${product} is missing after the pkg run; inspect ${this.outDir}.`) } - if (target.platform !== 'macos') return [product] + const ripgrep = await this.copyRipgrepSidecar(target, product) + if (target.platform !== 'macos') return [product, ripgrep] const spawnHelper = `${product}-spawn-helper` const source = join(this.staging, 'node_modules', 'node-pty', 'prebuilds', `darwin-${target.arch}`, 'spawn-helper') if (this.cli.dryRun) { @@ -405,12 +408,36 @@ class SingleExeBuild { await copyFile(source, spawnHelper) await chmod(spawnHelper, 0o755) } - return [product, spawnHelper] + return [product, ripgrep, spawnHelper] + } + + /** Copy the target ripgrep binary beside the executable so Node can spawn it outside pkg's virtual filesystem. */ + private async copyRipgrepSidecar(target: Target, product: string): Promise { + const platform = target.platform === 'macos' ? 'darwin' : target.platform + const source = join( + this.staging, + 'node_modules', + '@vscode', + `ripgrep-${platform}-${target.arch}`, + 'bin', + 'rg', + ) + const destination = `${product}-rg` + if (this.cli.dryRun) { + console.log(`build-exe-for-python-sdk: [dry-run] cp ${source} ${destination}`) + return destination + } + if (!existsSync(source)) { + throw new Error(`build-exe-for-python-sdk: target ripgrep binary is missing at ${source}.`) + } + await copyFile(source, destination) + await chmod(destination, 0o755) + return destination } /** - * Put the target node-pty addon in the staged closure. Linux npm installs - * build it from source, but legacy deploy omits that side-effect directory. + * Put the target node-pty addon in the staged closure. The release workflow + * provides a manylinux build; ordinary installs use node-pty's target prebuild. * @param target - the pkg target whose native addon is being staged. */ private async prepareNativePty(target: Target): Promise { @@ -418,8 +445,16 @@ class SingleExeBuild { if (this.cli.dryRun) console.log(`build-exe-for-python-sdk: [dry-run] rm -rf ${stagedBuild}`) else await rm(stagedBuild, { recursive: true, force: true }) if (target.platform !== 'linux') return - const source = join(root, 'packages', 'subprocess', 'subprocess-local', 'node_modules', 'node-pty', 'build', 'Release', 'pty.node') + const packageDirectory = join( + root, + 'packages', + 'subprocess', + 'subprocess-local', + 'node_modules', + 'node-pty', + ) const destination = join(stagedBuild, 'Release', 'pty.node') + const source = resolveLinuxNodePtyAddon(packageDirectory, target.arch) if (this.cli.dryRun) { console.log(`build-exe-for-python-sdk: [dry-run] cp ${source} ${destination}`) return diff --git a/scripts/build-python-release.py b/scripts/build-python-release.py index 18660b3411..0b4157f40d 100644 --- a/scripts/build-python-release.py +++ b/scripts/build-python-release.py @@ -48,7 +48,8 @@ PLATFORMS = load_platforms() def runtime_suffixes(executable_name: str) -> tuple[str, ...]: - return ("", "-spawn-helper") if "-macos-" in executable_name else ("",) + suffixes = ("", "-rg") + return (*suffixes, "-spawn-helper") if "-macos-" in executable_name else suffixes def main() -> None: diff --git a/scripts/build.ts b/scripts/build.ts new file mode 100644 index 0000000000..b8ffa8dbc0 --- /dev/null +++ b/scripts/build.ts @@ -0,0 +1,53 @@ +/** Run the complete repository build and bind its client artifacts to their public environment. */ + +import { spawnSync } from 'node:child_process' +import { rmSync } from 'node:fs' +import { resolve } from 'node:path' +import { parseArgs } from 'node:util' +import { + CLIENT_BUILD_RECORD_PATH, + clientBuildProcessEnvironment, + repositoryCommitHash, + resolveClientBuildEnvironment, + writeClientBuildRecord, +} from './client-build-environment.ts' +import { pnpmInvocation } from './pnpm-invocation.ts' + +/** Run one package script through the package manager that invoked this build. */ +function runScript(script: string, environment: NodeJS.ProcessEnv): void { + const invocation = pnpmInvocation(['run', script], environment) + const result = spawnSync(invocation.command, invocation.args, { + cwd: resolve(import.meta.dirname, '..'), + env: environment, + stdio: 'inherit', + }) + if (result.error !== undefined) throw result.error + if (result.status !== 0) { + throw new Error(`build: ${script} exited with ${String(result.status ?? result.signal)}`) + } +} + +/** Run the full build selected by `--profile` or `DSH_BUILD_CLIENT_PROFILE`. */ +function main(): void { + const { values } = parseArgs({ + options: { profile: { type: 'string' } }, + allowPositionals: false, + }) + const root = resolve(import.meta.dirname, '..') + const parentEnvironment = { + ...process.env, + DSH_CLIENT_COMMIT_HASH: repositoryCommitHash(root, process.env), + } + const clientEnvironment = resolveClientBuildEnvironment(parentEnvironment, values.profile) + const buildEnvironment = clientBuildProcessEnvironment(parentEnvironment, clientEnvironment) + + rmSync(resolve(root, CLIENT_BUILD_RECORD_PATH), { force: true }) + runScript('build:lib', buildEnvironment) + runScript('build:web', buildEnvironment) + const record = writeClientBuildRecord(root, clientEnvironment) + console.log( + `build: recorded ${String(record.artifacts.fileCount)} client artifact(s) with ${String(Object.keys(record.environment).length)} public value(s)`, + ) +} + +if (import.meta.main) main() diff --git a/scripts/check-workspace-constraints.spec.ts b/scripts/check-workspace-constraints.spec.ts new file mode 100644 index 0000000000..4bd39ac878 --- /dev/null +++ b/scripts/check-workspace-constraints.spec.ts @@ -0,0 +1,100 @@ +/** Experimental-package publication and dependency constraints. */ + +import { readFileSync } from 'node:fs' +import { join } from 'node:path' +import { describe, expect, it } from 'vitest' +import { + checkExperimentalDependencyIsolation, + checkExperimentalManifest, + checkWorkspaceManifest, + type WorkspaceManifest, +} from './check-workspace-constraints.ts' + +const experimental: WorkspaceManifest = { + dir: 'packages/experimental/prototype', + manifest: { name: '@deepseek-ai/dsh-experimental-prototype', private: true }, +} + +describe('experimental workspace constraints', () => { + it('requires the experimental package-name prefix', () => { + expect(checkExperimentalManifest({ + ...experimental, + manifest: { ...experimental.manifest, name: '@deepseek-ai/dsh-prototype' }, + })).toEqual([ + '@deepseek-ai/dsh-prototype: experimental package name must start with "@deepseek-ai/dsh-experimental-"', + ]) + }) + + it('requires private manifests without publication metadata', () => { + expect(checkExperimentalManifest(experimental)).toEqual([]) + expect(checkExperimentalManifest({ + ...experimental, + manifest: { ...experimental.manifest, private: false, publishConfig: { access: 'public' } }, + })).toEqual([ + '@deepseek-ai/dsh-experimental-prototype: experimental package must set "private": true', + '@deepseek-ai/dsh-experimental-prototype: experimental package must omit publishConfig', + ]) + }) + + it.each(['dependencies', 'optionalDependencies', 'peerDependencies'] as const)( + 'rejects release %s on an experimental package', + (section) => { + expect(checkExperimentalDependencyIsolation([experimental, { + dir: 'packages/core/consumer', + manifest: { + name: '@deepseek-ai/dsh-consumer', + [section]: { '@deepseek-ai/dsh-experimental-prototype': 'workspace:^' }, + }, + }])).toEqual([ + `@deepseek-ai/dsh-consumer: ${section}.@deepseek-ai/dsh-experimental-prototype must not reference an experimental package`, + ]) + }, + ) + + it('allows development and experimental consumers but rejects the Python release runtime', () => { + const manifests: WorkspaceManifest[] = [experimental, { + dir: 'packages/core/test-only', + manifest: { + name: '@deepseek-ai/dsh-test-only', + devDependencies: { '@deepseek-ai/dsh-experimental-prototype': 'workspace:^' }, + }, + }, { + dir: 'packages/experimental/consumer', + manifest: { + name: '@deepseek-ai/dsh-experimental-consumer', + dependencies: { '@deepseek-ai/dsh-experimental-prototype': 'workspace:^' }, + }, + }, { + dir: 'python/sdk-runtime', + manifest: { + name: '@deepseek-ai/dsh-python-runtime', + dependencies: { '@deepseek-ai/dsh-experimental-prototype': 'workspace:^' }, + }, + }] + + expect(checkExperimentalDependencyIsolation(manifests)).toEqual([ + '@deepseek-ai/dsh-python-runtime: dependencies.@deepseek-ai/dsh-experimental-prototype must not reference an experimental package', + ]) + }) +}) + +describe('private Python runtime carrier', () => { + const manifest = JSON.parse( + readFileSync(new URL('../packages/sdk/python-runtime/package.json', import.meta.url), 'utf8'), + ) as WorkspaceManifest['manifest'] + + it('participates in dsh package checks without becoming an npm release member', () => { + expect(checkWorkspaceManifest({ dir: 'packages/sdk/python-runtime', manifest })).toEqual([]) + }) + + it('rejects publication metadata on the private carrier', () => { + const path = join('packages', 'sdk', 'python-runtime', 'package.json') + expect(checkWorkspaceManifest({ + dir: 'packages/sdk/python-runtime', + manifest: { ...manifest, private: false, publishConfig: { access: 'public' } }, + })).toEqual([ + `${path}: @deepseek-ai/dsh-sdk-python-runtime: private carrier must set "private": true`, + `${path}: @deepseek-ai/dsh-sdk-python-runtime: private carrier must omit publishConfig`, + ]) + }) +}) diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index ee30778f3e..830386133d 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -7,6 +7,7 @@ import { existsSync, readdirSync, readFileSync } from 'node:fs' import { join, relative, resolve } from 'node:path' +import { pathToFileURL } from 'node:url' import { hasTypertRemoteNavigation, isForbiddenPublicationFile } from './publication-payload.ts' import { collectProjectReferenceFaceViolations } from './project-reference-faces.ts' @@ -47,19 +48,26 @@ const repositoryUrl = 'git+https://github.com/deepseek-harness/deepseek-harness. * their trusted publishing against the repository that runs the workflow. */ const publishedRepositoryUrl = 'git+https://github.com/deepseek-ai/deepseek-harness.git' +/** Private packages that participate in workspace checks but not releases. */ +const experimentalPackageDirectory = /^packages\/experimental\/[^/]+$/ +/** npm namespace reserved for private experimental packages. */ +const experimentalPackageNamePrefix = '@deepseek-ai/dsh-experimental-' /** Directories whose packages this repository publishes: one release member each. */ -const releaseMemberDirectory = /^(?:packages\/[^/]+\/[^/]+|apps\/[^/]+|vendor\/[^/]+)$/ +const releaseMemberDirectory = /^(?:packages\/(?!experimental\/)[^/]+\/[^/]+|apps\/[^/]+|vendor\/[^/]+)$/ +/** Named dsh packages that remain private because another distribution embeds them. */ +const privateCarrierDirectories = new Set(['packages/sdk/python-runtime']) const localArtifactDirs = new Set(['node_modules']) const appPackageFiles: Readonly> = { '@deepseek-ai/dsh': ['lib/*.js', 'config'], - // The Web build emits sourcemaps for browser debugging; publishing them is - // what the payload policy forbids, so the bundle ships without them. - '@deepseek-ai/dsh-web-frontend': ['dist', '!dist/**/*.map'], + // Sourcemaps stay out by payload policy; the worker-preview surface + // (dist/preview.html and dist/preview/) backs private experimental + // packages and is not published. + '@deepseek-ai/dsh-web-frontend': ['dist', '!dist/**/*.map', '!dist/preview.html', '!dist/preview'], } /** The subset of package.json fields this constraint check cares about. */ -interface PackageManifest { +export interface PackageManifest { name?: string version?: string private?: boolean @@ -84,10 +92,15 @@ interface PackageManifest { devDependencies?: Record dependencies?: Record optionalDependencies?: Record + dsh?: { + bundle?: { + patch?: string + } + } } /** One workspace manifest and its repo-relative path. */ -interface WorkspaceManifest { +export interface WorkspaceManifest { dir: string manifest: PackageManifest } @@ -131,19 +144,29 @@ function workspaceManifests(): WorkspaceManifest[] { } const packageFileExtras: Readonly> = { - // Profile bundles publish their dsh.bundle.patch layer beside the lib. - '@deepseek-ai/dsh-base': ['cordis.patch.yml'], - '@deepseek-ai/dsh-web-app': ['cordis.patch.yml'], - '@deepseek-ai/dsh-headless': ['cordis.patch.yml'], + // Statically linked client libraries keep their stylesheets next to the emitted + // JavaScript, which imports them by relative path: the compile shell runs + // them through its own CSS pipeline, so the sheets are published artifacts. + // The glob covers whichever sheets a package emits; sourcemaps stay + // unpublished, as everywhere else in the repository. + '@deepseek-ai/dsh-client-ui-primitives': ['lib/**/*.css'], + '@deepseek-ai/dsh-client-web': ['lib/**/*.css'], '@deepseek-ai/dsh-client-ui-theme': ['lib/styles'], - // The Python runtime uses a distinct closed-resolution bin; the public CLI - // keeps config-owned bare-package resolution through lib/bin.js. - '@deepseek-ai/dsh-sdk-jsonrpc-demo': ['lib/packaged-bin.js'], + // The CPython side ships as source .py files, published as-is rather than built. + '@deepseek-ai/dsh-code-runtime-python': ['py/**/*.py'], + // The private Python carrier ships only its closed-resolution entry. + '@deepseek-ai/dsh-sdk-python-runtime': ['lib/packaged-bin.js'], // The argv-prefix runner entry ships beside the lib as its own bundle; // sandbox-local resolves it through the package's ./runner export. tsdown // also shares its generated FFI code through a hashed runtime chunk. '@deepseek-ai/dsh-sandbox-windows-acl': ['lib/runner.js', 'lib/types-*.js'], + // SQLite loads every statement from immutable package resources at runtime. + '@deepseek-ai/dsh-session-persistence-sqlite': ['resources/sql/**/*.sql'], '@deepseek-ai/dsh-skill-badge': ['assets'], + // tsdown shares the repository/pack code between the lib entry and the bin + // through a hashed chunk. The committed bin.js is the link target pnpm can + // resolve at install time, before the build produces lib/bin.js. + '@deepseek-ai/dsh-experimental-webworker-packer': ['bin.js', 'lib/repository-*.js'], '@deepseek-ai/dsh-subprocess-local': ['scripts/ensure-spawn-helper.mjs'], } @@ -152,14 +175,23 @@ function sameStringList(actual: readonly string[] | undefined, expected: readonl } function expectedDshPackageFiles(manifest: PackageManifest): readonly string[] { - const extras = manifest.name ? packageFileExtras[manifest.name] ?? [] : [] + const declaredPatch = manifest.dsh?.bundle?.patch + const bundleFiles = declaredPatch === undefined ? [] : [declaredPatch.replace(/^\.\//, '')] + const extras = [ + ...bundleFiles, + ...(manifest.name ? packageFileExtras[manifest.name] ?? [] : []), + ] return [ 'lib/index.js', // Every package publishes its invariant ownership companion as a separate // bundle; the package-invariant gate validates the companion itself. 'lib/invariant.js', ...manifest.bin ? ['lib/bin.js'] : [], - ...manifest.exports?.['./worker'] ? ['lib/worker.cjs'] : [], + // Worker-thread packages ship a CJS worker entry; the browser worker + // bundle is an ES module a page loads with `new Worker(type: 'module')`. + // Keyed on the artifact path, like ./client below. + ...exportDefault(manifest, './worker') === './lib/worker.cjs' ? ['lib/worker.cjs'] : [], + ...exportDefault(manifest, './worker') === './lib/worker.js' ? ['lib/worker.js'] : [], // UI plugin packages ship their browser bundle beside the node lib // (single-artifact ruling: dist/ retired, ./client resolves lib/client.js). // Keyed on the artifact path, not the subpath name: apiproxy's ./client is @@ -167,7 +199,7 @@ function expectedDshPackageFiles(manifest: PackageManifest): readonly string[] { ...exportDefault(manifest, './client') === './lib/client.js' ? ['lib/client.js'] : [], // runtime's shell-held loader subpath ships as its own bundle beside the client half. ...exportDefault(manifest, './loader') === './lib/loader.js' ? ['lib/loader.js'] : [], - // web-react's store subpath ships its own bundle (single-entry builds; no shared chunk). + // A store subpath ships its own bundle (single-entry builds; no shared chunk). ...exportDefault(manifest, './store') === './lib/store/index.js' ? ['lib/store/index.js'] : [], // A surface bundle's startup row is its own bundle: the Loader imports it // as a row module, so it cannot ride inside the package entry. @@ -219,8 +251,26 @@ function usesEmittedTreeDefaults(manifest: PackageManifest): boolean { exportDefault(manifest, subpath)?.startsWith('./lib/types/') === true) } -function checkWorkspace({ dir, manifest }: WorkspaceManifest): string[] { +/** Experimental manifest requirements enforced independently from release metadata. */ +export function checkExperimentalManifest({ dir, manifest }: WorkspaceManifest): string[] { + if (!experimentalPackageDirectory.test(dir)) return [] + const label = manifest.name ?? dir const errors: string[] = [] + if (manifest.name?.startsWith(experimentalPackageNamePrefix) !== true) { + errors.push(`${label}: experimental package name must start with ${JSON.stringify(experimentalPackageNamePrefix)}`) + } + if (manifest.private !== true) errors.push(`${label}: experimental package must set "private": true`) + if (manifest.publishConfig !== undefined) errors.push(`${label}: experimental package must omit publishConfig`) + return errors +} + +/** + * Check one workspace manifest against publication and dsh-package policy. + * @param workspace - package directory and parsed manifest. + * @returns path-qualified policy violations. + */ +export function checkWorkspaceManifest({ dir, manifest }: WorkspaceManifest): string[] { + const errors = checkExperimentalManifest({ dir, manifest }) const label = manifest.name ?? dir const isLandlockPackageDir = dir.startsWith('native/landlock-run/packages/') const isPublicLandlockPackage = isLandlockPackageDir @@ -240,6 +290,13 @@ function checkWorkspace({ dir, manifest }: WorkspaceManifest): string[] { || manifest.repository.directory !== expectedDirectory) { errors.push(`${label}: published Landlock package repository must use ${repositoryUrl} with directory ${expectedDirectory} for trusted publishing`) } + } else if (privateCarrierDirectories.has(dir)) { + if (manifest.private !== true) { + errors.push(`${label}: private carrier must set "private": true`) + } + if (manifest.publishConfig !== undefined) { + errors.push(`${label}: private carrier must omit publishConfig`) + } } else if (releaseMemberDirectory.test(dir)) { // Release members state that they are publishable: npm refuses a private // package, and the repository field is how a consumer finds the source of @@ -251,19 +308,18 @@ function checkWorkspace({ dir, manifest }: WorkspaceManifest): string[] { // public. A mixed scope is why no publish path passes `--access` — one flag // cannot serve both, so each packed manifest decides // ([rationale](../.agents/notes/implemented/process/2026-08-13-public-vendor-and-native-sequences.md)). - const expectedAccess = dir.startsWith('vendor/') ? 'public' : 'restricted' if (manifest.private === true) { errors.push(`${label}: release member must not set "private": true`) } - if (manifest.publishConfig?.access !== expectedAccess) { - errors.push(`${label}: release member must set publishConfig.access to "${expectedAccess}"`) + if (manifest.publishConfig?.access !== 'public') { + errors.push(`${label}: release member must set publishConfig.access to "public"`) } if (manifest.repository?.type !== 'git' || manifest.repository.url !== publishedRepositoryUrl || manifest.repository.directory !== dir) { errors.push(`${label}: release member repository must use ${publishedRepositoryUrl} with directory ${dir}`) } - } else if (manifest.private !== true) { + } else if (!experimentalPackageDirectory.test(dir) && manifest.private !== true) { errors.push(`${label}: package.json must set "private": true`) } @@ -382,6 +438,31 @@ function checkRepositoryVersion(): string[] { /** Dependency sections whose ranges reach a published tarball or a local install. */ const dependencySections = ['dependencies', 'devDependencies', 'peerDependencies', 'optionalDependencies'] as const +/** Dependency sections present in an installed runtime. */ +const runtimeDependencySections = ['dependencies', 'optionalDependencies', 'peerDependencies'] as const + +/** + * Prevent an official runtime from requiring a package its release omits. + * @param manifests - release, private experimental, and deployment-root manifests. + * @returns One error for each forbidden runtime dependency. + */ +export function checkExperimentalDependencyIsolation(manifests: readonly WorkspaceManifest[]): string[] { + const experimentalNames = new Set(manifests + .filter(entry => experimentalPackageDirectory.test(entry.dir)) + .map(entry => entry.manifest.name) + .filter(name => name !== undefined)) + const errors: string[] = [] + for (const { dir, manifest } of manifests) { + if (!releaseMemberDirectory.test(dir) && dir !== 'python/sdk-runtime') continue + for (const section of runtimeDependencySections) { + for (const name of Object.keys(manifest[section] ?? {})) { + if (!experimentalNames.has(name)) continue + errors.push(`${manifest.name ?? dir}: ${section}.${name} must not reference an experimental package`) + } + } + } + return errors +} /** * Require the `workspace:` protocol for every reference to a workspace member. @@ -407,15 +488,25 @@ function checkWorkspaceProtocol(manifests: readonly WorkspaceManifest[]): string return errors } -const manifests = workspaceManifests() -const errors = [ - ...checkRepositoryVersion(), - ...manifests.flatMap(checkWorkspace), - ...checkWorkspaceProtocol(manifests), - ...checkHierarchyShape(), - ...collectProjectReferenceFaceViolations(root), -] -if (errors.length > 0) { - console.error(errors.join('\n')) - process.exitCode = 1 +/** Run the repository constraint gate. */ +export function main(): void { + const manifests = workspaceManifests() + const dependencyManifests = [ + ...manifests, + { dir: 'python/sdk-runtime', manifest: readJson(join(root, 'python/sdk-runtime/package.json')) }, + ] + const errors = [ + ...checkRepositoryVersion(), + ...manifests.flatMap(checkWorkspaceManifest), + ...checkWorkspaceProtocol(manifests), + ...checkExperimentalDependencyIsolation(dependencyManifests), + ...checkHierarchyShape(), + ...collectProjectReferenceFaceViolations(root), + ] + if (errors.length > 0) { + console.error(errors.join('\n')) + process.exitCode = 1 + } } + +if (process.argv[1] !== undefined && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) main() diff --git a/scripts/ci-workflow.spec.ts b/scripts/ci-workflow.spec.ts index b54c729a38..1c84f28c78 100644 --- a/scripts/ci-workflow.spec.ts +++ b/scripts/ci-workflow.spec.ts @@ -5,43 +5,60 @@ import { describe, expect, it } from 'vitest' const root = resolve(import.meta.dirname, '..') const runnerPrivatePnpmDestination = '${{ runner.temp }}/setup-pnpm' +const nativeWindowsPnpmDestination = '${{ runner.temp }}/setup-pnpm-js' describe('CI workflow', () => { it('isolates every pnpm action setup destination per runner', () => { - const workflow: unknown = yaml.load(readFileSync(resolve(root, '.github/workflows/ci.yml'), 'utf8')) - if (!isRecord(workflow) || !isRecord(workflow.jobs)) throw new TypeError('CI workflow must define jobs') - - const setups = Object.entries(workflow.jobs).flatMap(([jobName, job]) => { - if (!isRecord(job) || !Array.isArray(job.steps)) return [] - return job.steps.flatMap((step) => { - if (!isRecord(step) || typeof step.uses !== 'string' || !step.uses.startsWith('pnpm/action-setup@')) return [] - return [{ jobName, step }] - }) - }) + const files = ['.github/workflows/ci.yml', '.github/workflows/ci-master.yml'] + const setups: Array<{ jobName: string; step: unknown }> = [] + for (const file of files) { + const workflow: unknown = yaml.load(readFileSync(resolve(root, file), 'utf8')) + if (!isRecord(workflow) || !isRecord(workflow.jobs)) throw new TypeError(`${file} must define jobs`) + for (const [jobName, job] of Object.entries(workflow.jobs)) { + if (!isRecord(job) || !Array.isArray(job.steps)) continue + for (const step of job.steps) { + if (!isRecord(step) || typeof step.uses !== 'string' || !step.uses.startsWith('pnpm/action-setup@')) continue + setups.push({ jobName, step }) + } + } + } expect(setups.length).toBeGreaterThan(0) for (const { jobName, step } of setups) { expect(step, `${jobName} must not share pnpm/action-setup's default destination`).toMatchObject({ - with: { dest: runnerPrivatePnpmDestination }, + with: { + dest: jobName === 'windows-native' + ? nativeWindowsPnpmDestination + : runnerPrivatePnpmDestination, + }, }) + if (jobName === 'windows-native') expect(step).not.toMatchObject({ with: { standalone: true } }) } }) - it('keeps a required Wine Windows job, a non-blocking native Windows job with failover, and a master-only standby', () => { + it('keeps required Wine and native Windows jobs with failover, plus a master-only standby', () => { const workflow = loadWorkflow('.github/workflows/ci.yml') + const masterWorkflow = loadWorkflow('.github/workflows/ci-master.yml') if (!isRecord(workflow.jobs) || !isRecord(workflow.jobs.windows) || !isRecord(workflow.jobs['windows-native']) - || !isRecord(workflow.jobs['wine-apt-cache']) - || !isRecord(workflow.jobs['serial-windows']) - || !isRecord(workflow.jobs['all-checks-passed'])) { - throw new TypeError('CI workflow must define windows, windows-native, wine-apt-cache, serial-windows, and all-checks-passed jobs') + || !isRecord(workflow.jobs['node-24']) + || !isRecord(workflow.jobs['node-24-coverage']) + || !isRecord(workflow.jobs['node-24-consumers']) + || !isRecord(workflow.jobs['all-checks-passed']) + || !isRecord(masterWorkflow.jobs) + || !isRecord(masterWorkflow.jobs['wine-apt-cache']) + || !isRecord(masterWorkflow.jobs['serial-windows'])) { + throw new TypeError('CI workflow must define windows, windows-native, node-24, node-24-coverage, node-24-consumers, and all-checks-passed; ci-master must define wine-apt-cache and serial-windows') } const windows = workflow.jobs.windows const windowsNative = workflow.jobs['windows-native'] - const wineAptCache = workflow.jobs['wine-apt-cache'] - const serialWindows = workflow.jobs['serial-windows'] + const wineAptCache = masterWorkflow.jobs['wine-apt-cache'] + const serialWindows = masterWorkflow.jobs['serial-windows'] + const node24 = workflow.jobs['node-24'] + const node24Coverage = workflow.jobs['node-24-coverage'] + const node24Consumers = workflow.jobs['node-24-consumers'] const aggregate = workflow.jobs['all-checks-passed'] if (!Array.isArray(windows.steps) || !Array.isArray(aggregate.needs)) { throw new TypeError('Windows job must define steps and the aggregate must define needs') @@ -56,38 +73,72 @@ describe('CI workflow', () => { expect(windows.if).toBe("github.event_name == 'pull_request'") expect(commandSteps.some(step => step.run.includes('wine-windows-gates.sh'))).toBe(true) - // windows-native: non-blocking native job with failover, runs windows-complete. + // windows-native: blocking native job with failover, runs windows-complete. + // Its pool is resolved by the Windows-specific switch. expect(typeof windowsNative['runs-on']).toBe('string') - expect(windowsNative['runs-on']).toContain('DSH_CI_FAILOVER') + expect(windowsNative['runs-on']).toContain('DSH_CI_FAILOVER_WINDOWS') + expect(windowsNative['runs-on']).not.toContain('DSH_CI_FAILOVER_LINUX') expect(windowsNative['runs-on']).toContain('self-hosted') expect(windowsNative['runs-on']).toContain('dsh-win-ci') expect(windowsNative['runs-on']).toContain('dsh-windows-2025-16core') expect(windowsNative.name).toBe('windows node 24 / native complete') expect(windowsNative.if).toBe("github.event_name == 'pull_request'") - const nativeCommandSteps = (windowsNative.steps as unknown[]).filter((step): step is Record & { run: string } => ( + expect(windowsNative.env).toMatchObject({ + DSH_COVERAGE_MAX_WORKERS: '12', + DSH_COVERAGE_PARTITIONS: '16', + DSH_COVERAGE_TEST_TIMEOUT_MS: '30000', + DSH_GATE_CONCURRENCY: '8', + }) + const nativeSteps = windowsNative.steps as unknown[] + const nativeCommandSteps = nativeSteps.filter((step): step is Record & { run: string } => ( isRecord(step) && typeof step.run === 'string' )) expect(nativeCommandSteps.map(step => step.run)).toContain('pnpm run check:ci:windows-complete') - // wine-apt-cache: master-only, seeds the Wine apt cache. + // wine-apt-cache: master-only, seeds the Wine apt cache, lives in ci-master. expect(wineAptCache.if).toBe("github.event_name == 'push' && github.ref == 'refs/heads/master'") expect(wineAptCache['runs-on']).toBe('ubuntu-latest') - // serial-windows: master-only standby, self-hosted, non-blocking. + // serial-windows: master-only standby, self-hosted, non-blocking, lives in ci-master. expect(serialWindows.if).toBe("github.event_name == 'push' && github.ref == 'refs/heads/master'") expect(serialWindows['runs-on']).toEqual(['self-hosted', 'dsh-win-ci', 'windows']) expect(serialWindows.name).toBe('serial / windows (self-hosted standby)') - // Aggregate: Wine `windows` required, native `windows-native` excluded. + // Aggregate: both complementary Windows jobs are required. expect(aggregate.needs).toContain('windows') - expect(aggregate.needs).not.toContain('windows-native') + expect(aggregate.needs).toContain('windows-native') expect(aggregate.needs).not.toContain('serial-windows') + + // Linux failover is a separate switch: the three required Linux workers + // and the verdict job resolve their pool through DSH_CI_FAILOVER_LINUX, + // never the Windows switch. + for (const [jobName, job] of [['node-24', node24], ['node-24-coverage', node24Coverage], ['node-24-consumers', node24Consumers]] as const) { + expect(typeof job['runs-on']).toBe('string') + expect(job['runs-on'], `${jobName} runs-on must use the Linux failover switch`).toContain('DSH_CI_FAILOVER_LINUX') + expect(job['runs-on'], `${jobName} runs-on must not use the Windows failover switch`).not.toContain('DSH_CI_FAILOVER_WINDOWS') + expect(job['runs-on']).toContain('vm-backup') + } + expect(aggregate['runs-on']).toContain('DSH_CI_FAILOVER_LINUX') + expect(aggregate['runs-on']).not.toContain('DSH_CI_FAILOVER_WINDOWS') + expect(aggregate['runs-on']).toContain('vm-backup') }) - it('exempts push from cancellation, so one master merge does not cancel the running drill', () => { - const workflow = loadWorkflow('.github/workflows/ci.yml') + it('gives the Wine Host TypeScript compile the repository heap budget', () => { + const wineGates = readFileSync(resolve(root, 'scripts/wine-windows-gates.sh'), 'utf8') + + expect(wineGates).toContain( + 'wine_node "$scratch/logs/host-tsc.log" --max-old-space-size=4096 "$tsc_js" -b tsconfig.host.json --pretty false', + ) + }) + + it('exempts push from cancellation in ci-master, so one master merge does not cancel the running drill', () => { + const workflow = loadWorkflow('.github/workflows/ci-master.yml') + const prWorkflow = loadWorkflow('.github/workflows/ci.yml') if (!isRecord(workflow.jobs) || !isRecord(workflow.concurrency)) { - throw new TypeError('CI workflow must define jobs and a workflow-level concurrency block') + throw new TypeError('ci-master workflow must define jobs and a workflow-level concurrency block') + } + if (!isRecord(prWorkflow.jobs)) { + throw new TypeError('ci workflow must define jobs') } // Cancellation applies to the whole superseded RUN, so this has to be @@ -96,11 +147,26 @@ describe('CI workflow', () => { // a drill takes longer than the interval between master merges. The negated // form is load-bearing: `== 'pull_request'` would also stop cancelling // workflow_dispatch, and a re-dispatched runner benchmark holds up to 12 - // larger runners for 15 minutes in this same group on master. The - // expression is evaluated against the NEWLY TRIGGERED run, so a dispatch on - // master still cancels a mid-flight drill; the runbook records that bound. + // larger runners for 15 minutes in this same group on master. expect(workflow.concurrency['cancel-in-progress']).toBe("${{ github.event_name != 'push' }}") + // The PR-only ci.yml still cancels a superseded run on a new push, so a + // fresh head does not stack a second full 9-job run behind a stale one. + // Unlike ci-master it has no push carve-out: every PR event supersedes. + expect(prWorkflow.concurrency).toMatchObject({ + 'cancel-in-progress': true, + }) + + // The exact event sets are what keep master-only jobs out of the PR check + // panel: ci-master triggers only on push(master) + workflow_dispatch and + // never on pull_request; ci.yml is exactly pull_request-only. Assert the + // full sets so losing the wrong event, or gaining an extra one, fails. + if (!isRecord(workflow.on) || !isRecord(prWorkflow.on)) { + throw new TypeError('both CI workflows must define on') + } + expect(Object.keys(workflow.on).sort()).toEqual(['push', 'workflow_dispatch']) + expect(Object.keys(prWorkflow.on)).toEqual(['pull_request']) + // Neither drill may carry a job-level group: it would not exempt the job // from run-scoped cancellation. for (const name of ['serial-linux-selfhosted', 'serial-windows']) { @@ -114,14 +180,7 @@ describe('CI workflow', () => { // What bounds the cost of exempting push: a master push may only carry the // cache seeder and the two drills. Any job reachable on push would start // accumulating uncancelled runs, so the set is pinned here. - // - // Classification is an exact allowlist of the conditions in use, not a - // substring match: `github.event_name != 'pull_request'` mentions - // `pull_request` yet IS push-reachable, so matching on the event name alone - // would silently misclassify it as gated. const NOT_PUSH_REACHABLE = new Set([ - "github.event_name == 'pull_request'", - "always() && github.event_name == 'pull_request'", "github.event_name == 'workflow_dispatch' && inputs.suite == 'larger-runner-benchmark'", "github.event_name == 'workflow_dispatch' && inputs.suite == 'consolidated-runner-benchmark'", ]) @@ -187,6 +246,29 @@ describe('CI workflow', () => { }) }) +describe('DeepSeek e2e workflow', () => { + it('prepares bubblewrap from the pinned payload without a package transaction', () => { + const workflow = loadWorkflow('.github/workflows/e2e.yml') + const e2e = workflowJob(workflow, 'e2e') + if (!Array.isArray(e2e.steps)) throw new TypeError('DeepSeek e2e workflow must define steps') + + const steps = e2e.steps.filter(isRecord) + expect(steps.find(step => step.name === 'Prepare bubblewrap (unrestrict userns)')).toMatchObject({ + run: 'bash scripts/prepare-ci-bubblewrap.sh', + }) + expect(JSON.stringify(steps)).not.toContain('apt-get') + }) + + it('bounds profile subprocess fan-out to the tested e2e default', () => { + const workflow = loadWorkflow('.github/workflows/e2e.yml') + const e2e = workflowJob(workflow, 'e2e') + if (!Array.isArray(e2e.steps)) throw new TypeError('DeepSeek e2e workflow must define steps') + + const step = e2e.steps.filter(isRecord).find(candidate => candidate.name === 'E2E tests (real DeepSeek API)') + expect(step).toMatchObject({ env: { DSH_E2E_MAX_WORKERS: 4 } }) + }) +}) + describe('E2B e2e workflow', () => { it('is manual-only and fails loud before running the focused live suite', () => { const workflow = loadWorkflow('.github/workflows/e2b-e2e.yml') @@ -218,7 +300,6 @@ describe('Python release workflows', () => { it('keeps complete wheel validation separate from protected public publication', () => { const workflow = loadWorkflow('.github/workflows/python-release.yml') const dispatch = workflowEvent(workflow, 'workflow_dispatch') - const pullRequest = workflowEvent(workflow, 'pull_request') const build = workflowJob(workflow, 'build') const pythonCompat = workflowJob(workflow, 'python-compat') const validate = workflowJob(workflow, 'validate') @@ -234,9 +315,9 @@ describe('Python release workflows', () => { } expect(dispatch.inputs.publish).toMatchObject({ type: 'boolean', default: false }) - expect(pullRequest).toEqual({ types: ['labeled'] }) + if (!isRecord(workflow.on)) throw new TypeError('python-release workflow must define on') + expect(Object.keys(workflow.on)).toEqual(['workflow_dispatch']) expect(build).toMatchObject({ - if: "github.event_name == 'workflow_dispatch' || github.event.label.name == 'python-release-dry-run'", uses: './.github/workflows/build-exe-for-python-sdk.yml', with: { targets: 'node24-linux-x64,node24-linux-arm64,node24-macos-arm64', @@ -244,7 +325,10 @@ describe('Python release workflows', () => { }, }) expect(pythonCompat.strategy).toMatchObject({ matrix: { python: ['3.10', '3.14'] } }) - expect(JSON.stringify(pythonCompat.steps)).toContain('deepseek-harness-sdk==${{ steps.compatibility-version.outputs.version }}') + const pythonCompatSteps = JSON.stringify(pythonCompat.steps) + expect(pythonCompatSteps).toContain('dist/deepseek_harness_sdk-$VERSION-py3-none-any.whl') + expect(pythonCompatSteps).toContain('dist/deepseek_harness_runtime_bin-$VERSION-py3-none-manylinux_2_28_x86_64.whl') + expect(pythonCompatSteps).not.toContain('--find-links') const validateSteps = JSON.stringify(validate.steps) const authorize = validate.steps.filter(isRecord).find(step => step.name === 'Authorize publication request') if (!isRecord(authorize) || typeof authorize.run !== 'string') { @@ -317,10 +401,18 @@ describe('Python release workflows', () => { expect(plan.if).toContain('inputs.ci') expect(plan.if).toContain('inputs.release') expect(JSON.stringify(plan.steps)).toContain('pep440_version') - expect(JSON.stringify(workflow)).toContain('macosx_14_0_arm64') + const workflowJson = JSON.stringify(workflow) + expect(workflowJson).toContain('macosx_14_0_arm64') + expect(workflowJson).toContain('dist-python/$SDK_WHEEL') + expect(workflowJson).toContain('dist-python/$RUNTIME_WHEEL') + expect(workflowJson).toContain('/work/dist-python/$SDK_WHEEL') + expect(workflowJson).toContain('/work/dist-python/$RUNTIME_WHEEL') + expect(workflowJson).not.toContain('--find-links dist-python') + expect(workflowJson).not.toContain('--find-links /work/dist-python') expect(manylinuxAddon).toMatchObject({ if: "runner.os == 'Linux'" }) expect(JSON.stringify(manylinuxAddon)).toContain('manylinux_2_28_x86_64') expect(JSON.stringify(manylinuxAddon)).toContain('manylinux_2_28_aarch64') + expect(JSON.stringify(manylinuxAddon)).toContain('npm_config_build_from_source=true pnpm run install') expect(JSON.stringify(manylinuxAddon)).toContain('$HOME/setup-pnpm:$HOME/setup-pnpm:ro') expect(JSON.stringify(manylinuxAddon)).toContain('node-pty-glibc-versions.txt') expect(JSON.stringify(manylinuxAddon)).toContain('le 2.28') @@ -351,24 +443,103 @@ describe('Python release workflows', () => { }) describe('Issue lifecycle workflow', () => { - it('uses explicit review handoff events without rerunning when a draft becomes ready', () => { + it('runs the lifecycle job on every PR/review event but gates token and board steps', () => { const lifecycle = loadWorkflow('.github/workflows/issue-lifecycle.yml') + const policy = loadWorkflow('.github/workflows/issue-policy.yml') + const lifecycleJob = workflowJob(lifecycle, 'lifecycle') + if (!Array.isArray(lifecycleJob.steps)) throw new TypeError('Issue lifecycle job must define steps') + + // The job has no job-level `if`, so it is listed on every pull_request / + // pull_request_review event and reports success instead of a gray skip. The + // write-capable steps are gated at step level so approved/commented reviews + // never mint a Project/Issue App token nor touch the board. + expect(lifecycle.on).toHaveProperty('pull_request') + expect(lifecycle.on).toHaveProperty('pull_request_review') + expect(lifecycleJob.if).toBeUndefined() + // Keep the subscription-type gates: issue-lifecycle does not re-subscribe + // ready_for_review (issue-policy owns that) and only reacts to submitted + // review events. const lifecyclePullRequest = workflowEvent(lifecycle, 'pull_request') const lifecycleReview = workflowEvent(lifecycle, 'pull_request_review') - const lifecycleJob = workflowJob(lifecycle, 'lifecycle') - const policy = loadWorkflow('.github/workflows/issue-policy.yml') - const policyPullRequest = workflowEvent(policy, 'pull_request') - expect(lifecyclePullRequest.types).not.toContain('ready_for_review') expect(lifecyclePullRequest.types).toContain('review_requested') expect(lifecycleReview.types).toEqual(['submitted']) - expect(lifecycleJob.if).toBe( - "${{ github.event_name != 'pull_request_review' || (github.event.action == 'submitted' && github.event.review.state == 'changes_requested') }}", - ) + const gated = "${{ github.event_name != 'pull_request_review' || github.event.review.state == 'changes_requested' }}" + const steps = lifecycleJob.steps.filter(isRecord) + const tokenStep = steps.find(s => s.name === 'Create project token') + const handleStep = steps.find(s => s.name === 'Handle repository event') + expect(tokenStep).toMatchObject({ if: gated }) + expect(handleStep).toMatchObject({ if: gated }) + + // issue-policy owns PR validation; it is read-only and a real gate. + const policyPullRequest = workflowEvent(policy, 'pull_request') expect(policyPullRequest.types).toContain('ready_for_review') }) }) +describe('npm release workflows', () => { + it('keeps publication dispatch-only and pack in the PR workflow', () => { + // pack stays in the PR/master release workflows so a PR proves the set packs. + for (const file of ['release.yml', 'release-vendor.yml']) { + const workflow = loadWorkflow(`.github/workflows/${file}`) + if (!isRecord(workflow.jobs)) throw new TypeError(`${file} must define jobs`) + expect(Object.keys(workflow.jobs).sort()).toEqual(['pack']) + } + + // publication is workflow_dispatch-only (never a PR check) and keeps the + // npm-publish environment plus the shared dist-tag group. + for (const file of ['release-publish.yml', 'release-vendor-publish.yml']) { + const workflow = loadWorkflow(`.github/workflows/${file}`) + if (!isRecord(workflow.on) || !isRecord(workflow.jobs)) throw new TypeError(`${file} must define on and jobs`) + expect(Object.keys(workflow.on)).toEqual(['workflow_dispatch']) + const publish = workflow.jobs.publish + if (!isRecord(publish)) throw new TypeError(`${file} must define a publish job`) + expect(publish.environment).toBe('npm-publish') + expect(publish.concurrency).toMatchObject({ group: 'Release-publish' }) + } + }) +}) + +describe('Documentation site publication', () => { + it('keeps Pages deployment dispatch-only from a dsh-v* tag', () => { + const workflow = loadWorkflow('.github/workflows/docs-pages.yml') + const build = workflowJob(workflow, 'build') + const deploy = workflowJob(workflow, 'deploy') + if (!isRecord(workflow.on) || !isRecord(workflow.env) || !Array.isArray(build.steps)) { + throw new TypeError('Documentation deployment must define on, env, and build steps') + } + + // The site presents a released snapshot: a merge must never publish it, and + // publication must never appear as a PR check. + expect(Object.keys(workflow.on)).toEqual(['workflow_dispatch']) + + // RELEASE_PUBLISH makes release:verify reject every ref that is not a dsh-v* + // tag naming this tree's version, so the site and the npm sequence share one + // definition of a released version. + const steps = build.steps.filter(isRecord) + const verify = steps.find(step => step.name === 'Verify release version') + const checkout = steps.find( + step => typeof step.uses === 'string' && step.uses.startsWith('actions/checkout@'), + ) + expect(verify).toMatchObject({ + env: { RELEASE_PUBLISH: 'true' }, + run: 'pnpm run release:verify --family dsh', + }) + // Complete history: the release scripts read tags. + expect(checkout).toMatchObject({ with: { 'fetch-depth': 0 } }) + + // Projected source links stay on the public repository's master. That + // repository advances only to each release commit, so its master never + // carries unreleased work, while it retains only the most recent tags: + // following the dispatched tag would leave every source link on a deploy + // from an older tag unresolvable. + expect(workflow.env.DOCS_REPOSITORY_REF).toBe('master') + + // The environment owns the deployment tag policy and the required reviewers. + expect(deploy.environment).toMatchObject({ name: 'github-pages' }) + }) +}) + describe('Git hooks', () => { it('leaves frozen Agent Note sidecars to the archive verifier', () => { const lefthook = loadWorkflow('lefthook.yml') diff --git a/scripts/clean.spec.ts b/scripts/clean.spec.ts index c724453283..bd0c3be9af 100644 --- a/scripts/clean.spec.ts +++ b/scripts/clean.spec.ts @@ -38,6 +38,7 @@ describe('RepositoryCleaner', () => { write(join(root, 'products/shell/lib/types/index.js')) write(join(root, 'products/shell/lib/index.js')) write(join(root, '.typecheck/legacy.tsbuildinfo')) + write(join(root, '.dsh-build/client-build-environment.json')) write(join(root, 'root.tsbuildinfo')) write(join(root, 'packages/removed/ghost/node_modules/.bin/tool')) @@ -46,6 +47,7 @@ describe('RepositoryCleaner', () => { expect(existsSync(join(root, 'products/shell/lib'))).toBe(false) expect(existsSync(join(root, 'products/shell/src/index.ts'))).toBe(true) expect(existsSync(join(root, '.typecheck'))).toBe(false) + expect(existsSync(join(root, '.dsh-build'))).toBe(false) expect(existsSync(join(root, 'root.tsbuildinfo'))).toBe(false) expect(existsSync(join(root, 'packages/removed/ghost'))).toBe(false) }) diff --git a/scripts/clean.ts b/scripts/clean.ts index 68e4ff4e71..c0f7d71763 100644 --- a/scripts/clean.ts +++ b/scripts/clean.ts @@ -67,6 +67,8 @@ export class RepositoryCleaner { const unsafeOrphans: string[] = [] const canonicalRoot = await realpath(this.root) + await this.addIfPresent(targets, join(this.root, '.dsh-build'), canonicalRoot) + // These checks cover legacy root-level incremental state emitted by older configs. await this.addIfPresent(targets, join(this.root, '.typecheck'), canonicalRoot) for (const entry of await readdir(this.root, { withFileTypes: true })) { diff --git a/scripts/client-build-environment.client.spec.ts b/scripts/client-build-environment.client.spec.ts new file mode 100644 index 0000000000..e2eb0ebe72 --- /dev/null +++ b/scripts/client-build-environment.client.spec.ts @@ -0,0 +1,176 @@ +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join, resolve } from 'node:path' +import yaml from 'js-yaml' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + assertClientBuildEnvironment, + clientBuildEnvironmentDefines, + clientBuildProcessEnvironment, + readClientBuildRecord, + repositoryCommitHash, + resolveClientBuildEnvironment, + writeClientBuildRecord, +} from './client-build-environment.ts' +import { clientBundle } from '../packages/client/tsdown.client.ts' + +const root = resolve(import.meta.dirname, '..') +const PROBE_NAME = 'DSH_CLIENT_BUILD_TEST' +const COMMIT_HASH = '0123456789abcdef0123456789abcdef01234567' +const PROBE_KEY = `process.env.${PROBE_NAME}` +const originalProbe = process.env[PROBE_NAME] +const roots: string[] = [] +const dshBuildWorkflows = [ + 'build-exe-for-python-sdk.yml', + 'ci.yml', + 'e2b-e2e.yml', + 'e2e.yml', + 'release.yml', + 'release-publish.yml', + 'sandbox.yml', +] + +afterEach(() => { + if (originalProbe === undefined) Reflect.deleteProperty(process.env, PROBE_NAME) + else process.env[PROBE_NAME] = originalProbe + vi.resetModules() + for (const fixtureRoot of roots.splice(0)) rmSync(fixtureRoot, { recursive: true, force: true }) +}) + +function write(path: string, content: string): void { + mkdirSync(dirname(path), { recursive: true }) + writeFileSync(path, content) +} + +function buildFixture(environment: Record): string { + const fixtureRoot = mkdtempSync(join(tmpdir(), 'dsh-client-build-')) + roots.push(fixtureRoot) + write(join(fixtureRoot, 'apps/web/dist/index.html'), '
') + write(join(fixtureRoot, 'packages/client/example/lib/client.js'), 'module.exports = {}\n') + writeClientBuildRecord(fixtureRoot, environment) + return fixtureRoot +} + +describe('client build environment', () => { + it('requires an exact public environment for a named artifact profile', () => { + const expected = { + DSH_CLIENT_BUILD_PROFILE: 'official', + DSH_CLIENT_COMMIT_HASH: COMMIT_HASH.slice(0, 7), + DSH_CLIENT_TITLE: 'DeepSeek Harness', + } as const + + expect(() => { assertClientBuildEnvironment({ PATH: '/bin', ...expected }, expected) }).not.toThrow() + expect(() => { assertClientBuildEnvironment({}, expected) }).toThrow(/DSH_CLIENT_TITLE/) + expect(() => { assertClientBuildEnvironment({ DSH_CLIENT_TITLE: 'Other' }, expected) }).toThrow(/DSH_CLIENT_TITLE/) + expect(() => { + assertClientBuildEnvironment({ ...expected, DSH_CLIENT_UNDECLARED: 'value' }, expected) + }).toThrow(/DSH_CLIENT_UNDECLARED/) + }) + + it('inherits public values by default and isolates an explicit official profile', () => { + const parent = { + PATH: '/bin', + DSH_BUILD_CLIENT_PROFILE: 'official', + DSH_CLIENT_BUILD_PROFILE: 'local', + DSH_CLIENT_COMMIT_HASH: COMMIT_HASH.slice(0, 7), + DSH_CLIENT_TITLE: 'Local title', + DSH_CLIENT_EXTRA: 'local-extra', + } + + expect(resolveClientBuildEnvironment({ DSH_CLIENT_TITLE: 'Local title' })).toEqual({ + DSH_CLIENT_TITLE: 'Local title', + }) + expect(resolveClientBuildEnvironment(parent)).toEqual({ + DSH_CLIENT_BUILD_PROFILE: 'official', + DSH_CLIENT_COMMIT_HASH: COMMIT_HASH.slice(0, 7), + DSH_CLIENT_TITLE: 'DeepSeek Harness', + }) + expect(() => { + resolveClientBuildEnvironment({ DSH_BUILD_CLIENT_PROFILE: 'official' }) + }).toThrow(/DSH_CLIENT_COMMIT_HASH/) + expect(() => { resolveClientBuildEnvironment({}, 'unknown') }).toThrow(/unknown client build profile/) + expect(clientBuildProcessEnvironment(parent, { + DSH_CLIENT_BUILD_PROFILE: 'official', + DSH_CLIENT_COMMIT_HASH: COMMIT_HASH.slice(0, 7), + DSH_CLIENT_TITLE: 'DeepSeek Harness', + })).toEqual({ + PATH: '/bin', + DSH_CLIENT_BUILD_PROFILE: 'official', + DSH_CLIENT_COMMIT_HASH: COMMIT_HASH.slice(0, 7), + DSH_CLIENT_TITLE: 'DeepSeek Harness', + }) + expect(repositoryCommitHash('/unused', { DSH_CLIENT_COMMIT_HASH: COMMIT_HASH })).toBe(COMMIT_HASH.slice(0, 7)) + }) + + it('defines only public client values over a non-enumerable fallback', () => { + expect(clientBuildEnvironmentDefines({ + PATH: '/bin', + DSH_TEST_API_KEY: 'secret', + DSH_CLIENT_VARIANT: 'quoted "value"', + DSH_CLIENT_EMPTY: '', + DSH_CLIENT_UNSET: undefined, + })).toEqual({ + 'process.env': '{}', + 'process.env.DSH_CLIENT_EMPTY': '""', + 'process.env.DSH_CLIENT_VARIANT': '"quoted \\"value\\""', + }) + }) + + it('feeds the same build-process value to dynamic tsdown bundles and the Vite shell', async () => { + process.env[PROBE_NAME] = 'shared-value' + + const configs = clientBundle('@deepseek-ai/dsh-client-ui-sidebar', [ + 'lib/types/index.js', + 'lib/types/invariant.js', + ])({ env: { DSH_BUILD_FACE: 'client' } }) + if (!Array.isArray(configs)) throw new TypeError('client bundle config must be an array') + const dynamic = configs.find(config => config.name === '@deepseek-ai/dsh-client-ui-sidebar/client') + expect(dynamic?.define).toMatchObject({ + 'process.env': '{}', + [PROBE_KEY]: '"shared-value"', + }) + + const viteConfigPath = '../apps/web/vite.config.ts' + const viteModule: unknown = await import(viteConfigPath) + if (typeof viteModule !== 'object' || viteModule === null) { + throw new TypeError('web Vite config module must be an object') + } + const viteConfig: unknown = Reflect.get(viteModule, 'default') + if (typeof viteConfig === 'function') throw new TypeError('web Vite config must be an object') + if (typeof viteConfig !== 'object' || viteConfig === null) { + throw new TypeError('web Vite config must be an object') + } + expect(Reflect.get(viteConfig, 'define')).toMatchObject({ + 'process.env': '{}', + [PROBE_KEY]: '"shared-value"', + }) + }) + + it('binds the recorded environment to a complete set of client artifacts', () => { + const officialEnvironment = { + DSH_CLIENT_BUILD_PROFILE: 'official', + DSH_CLIENT_COMMIT_HASH: COMMIT_HASH.slice(0, 7), + DSH_CLIENT_TITLE: 'DeepSeek Harness', + } + const official = buildFixture(officialEnvironment) + const defaultBuild = buildFixture({}) + + expect(readClientBuildRecord(official, officialEnvironment).environment).toEqual(officialEnvironment) + expect(() => { readClientBuildRecord(defaultBuild, officialEnvironment) }).toThrow(/DSH_CLIENT_/) + expect(() => { readClientBuildRecord(join(defaultBuild, 'missing')) }).toThrow(/record.*missing/) + + write(join(official, 'apps/web/dist/index.html'), '
changed
') + expect(() => { readClientBuildRecord(official) }).toThrow(/artifacts differ/) + }) + + it('keeps public client values out of workflow-wide environments', () => { + for (const name of dshBuildWorkflows) { + const path = `.github/workflows/${name}` + const document: unknown = yaml.load(readFileSync(resolve(root, path), 'utf8')) + if (typeof document !== 'object' || document === null || Array.isArray(document)) { + throw new TypeError(`${path} must contain a workflow object`) + } + expect(JSON.stringify(document), path).not.toContain('DSH_CLIENT_') + } + }) +}) diff --git a/scripts/client-build-environment.ts b/scripts/client-build-environment.ts new file mode 100644 index 0000000000..2331db5f42 --- /dev/null +++ b/scripts/client-build-environment.ts @@ -0,0 +1,311 @@ +import { createHash } from 'node:crypto' +import { execFileSync } from 'node:child_process' +import { + existsSync, + globSync, + mkdirSync, + readFileSync, + statSync, + writeFileSync, +} from 'node:fs' +import { dirname, resolve } from 'node:path' + +/** Prefix reserved for build-time values that may be embedded in browser artifacts. */ +const CLIENT_BUILD_ENV_PREFIX = 'DSH_CLIENT_' + +/** Non-public selector used by build orchestration to request a named client profile. */ +export const CLIENT_BUILD_PROFILE_SELECTOR = 'DSH_BUILD_CLIENT_PROFILE' + +/** Public client environment required by official DSH artifacts. */ +const OFFICIAL_CLIENT_BUILD_ENVIRONMENT = { + DSH_CLIENT_BUILD_PROFILE: 'official', + DSH_CLIENT_TITLE: 'DeepSeek Harness', +} as const + +/** Public variable carrying the source commit embedded in client artifacts. */ +const CLIENT_COMMIT_HASH_VARIABLE = 'DSH_CLIENT_COMMIT_HASH' + +/** Repository-relative path of the complete client build record. */ +export const CLIENT_BUILD_RECORD_PATH = '.dsh-build/client-build-environment.json' + +const CLIENT_BUILD_RECORD_FORMAT = 1 +const CLIENT_ARTIFACT_PATTERNS = [ + 'apps/web/dist/**/*', + 'packages/*/*/lib/client.js', + 'packages/*/*/lib/client.js.map', +] as const + +/** Public values embedded in one set of client artifacts. */ +export type ClientBuildEnvironment = Readonly> + +/** + * Resolve the short source commit used by browser build metadata. + * @param root - repository root used when no explicit value is supplied. + * @param environment - environment that may already carry a commit value. + * @returns lowercase 7-character Git commit prefix. + */ +export function repositoryCommitHash(root: string, environment: NodeJS.ProcessEnv = process.env): string { + const explicit = environment[CLIENT_COMMIT_HASH_VARIABLE] + const value = explicit ?? execFileSync('git', ['rev-parse', 'HEAD'], { + cwd: root, + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'ignore'], + }).trim() + if (!/^[0-9a-f]{7,40}$/iu.test(value)) { + throw new Error(`${CLIENT_COMMIT_HASH_VARIABLE} must be a Git commit hash; got ${JSON.stringify(value)}`) + } + return value.slice(0, 7).toLowerCase() +} + +/** + * Resolve the exact public values required by an official build at one commit. + * @param root - repository root whose HEAD must match the built source. + * @param environment - optional explicit commit source for non-Git build environments. + * @returns complete official client environment. + */ +export function officialClientBuildEnvironment( + root: string, + environment: NodeJS.ProcessEnv = process.env, +): Readonly> { + return { + DSH_CLIENT_COMMIT_HASH: repositoryCommitHash(root, environment), + ...OFFICIAL_CLIENT_BUILD_ENVIRONMENT, + } +} + +/** Digest of every client artifact produced by the complete root build. */ +interface ClientArtifactDigest { + /** Number of files covered by the digest. */ + readonly fileCount: number + /** Lowercase SHA-256 digest of sorted paths and file contents. */ + readonly sha256: string +} + +/** Durable description of one complete root client build. */ +export interface ClientBuildRecord { + /** Record schema version. */ + readonly formatVersion: number + /** Exact public environment embedded by Vite and tsdown. */ + readonly environment: ClientBuildEnvironment + /** Digest that binds the environment to the current artifacts. */ + readonly artifacts: ClientArtifactDigest +} + +/** + * Collect the public client environment in deterministic key order. + * @param environment - environment inherited by the build process. + * @returns defined `DSH_CLIENT_*` values only. + */ +function clientBuildEnvironment(environment: NodeJS.ProcessEnv): ClientBuildEnvironment { + return Object.fromEntries(Object.entries(environment) + .filter(([name, value]) => name.startsWith(CLIENT_BUILD_ENV_PREFIX) && value !== undefined) + .sort(([left], [right]) => left.localeCompare(right))) as Record +} + +/** + * Resolve the exact public environment selected for a complete client build. + * @param environment - parent process environment. + * @param profile - explicit profile, or the non-public selector when omitted. + * @returns the inherited public values when no profile is selected, otherwise the named profile. + */ +export function resolveClientBuildEnvironment( + environment: NodeJS.ProcessEnv, + profile: string | undefined = environment[CLIENT_BUILD_PROFILE_SELECTOR], +): ClientBuildEnvironment { + if (profile === undefined) return clientBuildEnvironment(environment) + if (profile === 'official') { + const commitHash = environment[CLIENT_COMMIT_HASH_VARIABLE] + if (commitHash === undefined) { + throw new Error(`${CLIENT_COMMIT_HASH_VARIABLE} is required for the official client build profile`) + } + return { DSH_CLIENT_COMMIT_HASH: commitHash, ...OFFICIAL_CLIENT_BUILD_ENVIRONMENT } + } + throw new Error(`unknown client build profile ${JSON.stringify(profile)}; expected "official"`) +} + +/** + * Construct a subprocess environment containing exactly the selected public values. + * @param environment - parent process environment. + * @param clientEnvironment - complete public environment selected for the build. + * @returns the parent environment with selectors and inherited public values replaced. + */ +export function clientBuildProcessEnvironment( + environment: NodeJS.ProcessEnv, + clientEnvironment: ClientBuildEnvironment, +): NodeJS.ProcessEnv { + const child: NodeJS.ProcessEnv = {} + for (const [name, value] of Object.entries(environment)) { + if (name === CLIENT_BUILD_PROFILE_SELECTOR || name.startsWith(CLIENT_BUILD_ENV_PREFIX)) continue + child[name] = value + } + return { ...child, ...clientEnvironment } +} + +/** + * Require the public client environment to match an artifact profile exactly. + * + * An exact key set matters because every prefixed value is eligible for + * inlining: an unexpected variable can change published bytes just as surely + * as a missing or incorrect required value. + * + * @param environment - public environment from a build process or build record. + * @param expected - complete public client environment for the artifact profile. + */ +export function assertClientBuildEnvironment( + environment: Readonly>, + expected: Readonly>, +): void { + const actual = Object.fromEntries(Object.entries(environment) + .filter(([name, value]) => name.startsWith(CLIENT_BUILD_ENV_PREFIX) && value !== undefined) + .sort(([left], [right]) => left.localeCompare(right))) + const normalizedExpected = Object.fromEntries(Object.entries(expected) + .sort(([left], [right]) => left.localeCompare(right))) + if (JSON.stringify(actual) === JSON.stringify(normalizedExpected)) return + + const names = [...new Set([...Object.keys(actual), ...Object.keys(normalizedExpected)])].sort() + const differences = names.filter(name => actual[name] !== normalizedExpected[name]) + throw new Error(`client build environment differs from the required artifact profile: ${differences.join(', ')}`) +} + +/** + * Create bundler substitutions for public client build environment variables. + * + * The empty `process.env` fallback makes an unset static property read + * evaluate to `undefined` without providing a browser `process` global. + * Exact substitutions remain longer matches than that fallback. Dynamic + * property reads and enumeration deliberately observe the empty object. + * + * @param environment - environment inherited by the build process. + * @returns deterministic Vite/tsdown `define` expressions. + */ +export function clientBuildEnvironmentDefines( + environment: NodeJS.ProcessEnv, +): Record { + const defines: Record = { 'process.env': '{}' } + for (const [name, value] of Object.entries(clientBuildEnvironment(environment))) { + defines[`process.env.${name}`] = JSON.stringify(value) + } + return defines +} + +/** + * Write the build record after a complete root build succeeds. + * @param root - repository root containing the generated artifacts. + * @param environment - exact public environment supplied to both bundlers. + * @returns the record written to disk. + */ +export function writeClientBuildRecord( + root: string, + environment: ClientBuildEnvironment, +): ClientBuildRecord { + const record: ClientBuildRecord = { + formatVersion: CLIENT_BUILD_RECORD_FORMAT, + environment: clientBuildEnvironment(environment), + artifacts: clientArtifactDigest(root), + } + const path = resolve(root, CLIENT_BUILD_RECORD_PATH) + mkdirSync(dirname(path), { recursive: true }) + writeFileSync(path, `${JSON.stringify(record, null, 2)}\n`) + return record +} + +/** + * Read a complete build record and prove it still describes the current artifacts. + * @param root - repository root containing the record and generated artifacts. + * @param expected - optional exact public environment required by a consumer. + * @returns the parsed and artifact-verified record. + */ +export function readClientBuildRecord( + root: string, + expected?: Readonly>, +): ClientBuildRecord { + const path = resolve(root, CLIENT_BUILD_RECORD_PATH) + if (!existsSync(path)) { + throw new Error(`client build record ${CLIENT_BUILD_RECORD_PATH} is missing; run a complete pnpm run build first`) + } + + let parsed: unknown + try { + parsed = JSON.parse(readFileSync(path, 'utf8')) + } catch (error) { + const detail = error instanceof Error ? error.message : String(error) + throw new Error(`client build record ${CLIENT_BUILD_RECORD_PATH} is invalid JSON: ${detail}`) + } + const record = parseClientBuildRecord(parsed) + if (expected !== undefined) assertClientBuildEnvironment(record.environment, expected) + + const current = clientArtifactDigest(root) + if (current.fileCount !== record.artifacts.fileCount || current.sha256 !== record.artifacts.sha256) { + throw new Error( + `client artifacts differ from ${CLIENT_BUILD_RECORD_PATH}; run a complete pnpm run build before consuming them`, + ) + } + return record +} + +/** Return the deterministic digest of every artifact affected by the public client environment. */ +function clientArtifactDigest(root: string): ClientArtifactDigest { + const paths = globSync([...CLIENT_ARTIFACT_PATTERNS], { cwd: root }) + .map(path => path.replaceAll('\\', '/')) + .filter(path => statSync(resolve(root, path)).isFile()) + .sort() + if (paths.length === 0) throw new Error('complete client build produced no Vite or dynamic client artifacts') + + const digest = createHash('sha256') + for (const path of paths) { + const content = readFileSync(resolve(root, path)) + digest.update(`${Buffer.byteLength(path)}:`) + digest.update(path) + digest.update(`${content.byteLength}:`) + digest.update(content) + } + return { fileCount: paths.length, sha256: digest.digest('hex') } +} + +/** Parse and validate the persisted record before any consumer trusts it. */ +function parseClientBuildRecord(value: unknown): ClientBuildRecord { + if (!isObject(value) || !hasExactKeys(value, ['artifacts', 'environment', 'formatVersion'])) { + throw new Error(`client build record ${CLIENT_BUILD_RECORD_PATH} has an invalid top-level schema`) + } + if (value.formatVersion !== CLIENT_BUILD_RECORD_FORMAT) { + throw new Error( + `client build record ${CLIENT_BUILD_RECORD_PATH} uses format ${String(value.formatVersion)}; expected ${String(CLIENT_BUILD_RECORD_FORMAT)}`, + ) + } + if (!isObject(value.environment)) { + throw new Error(`client build record ${CLIENT_BUILD_RECORD_PATH} has an invalid environment`) + } + const environment: Record = {} + for (const [name, entry] of Object.entries(value.environment).sort(([left], [right]) => left.localeCompare(right))) { + if (!name.startsWith(CLIENT_BUILD_ENV_PREFIX) || typeof entry !== 'string') { + throw new Error(`client build record ${CLIENT_BUILD_RECORD_PATH} has an invalid environment entry ${name}`) + } + environment[name] = entry + } + if (!isObject(value.artifacts) || !hasExactKeys(value.artifacts, ['fileCount', 'sha256'])) { + throw new Error(`client build record ${CLIENT_BUILD_RECORD_PATH} has an invalid artifact digest`) + } + if (!Number.isSafeInteger(value.artifacts.fileCount) || Number(value.artifacts.fileCount) < 1) { + throw new Error(`client build record ${CLIENT_BUILD_RECORD_PATH} has an invalid artifact count`) + } + if (typeof value.artifacts.sha256 !== 'string' || !/^[0-9a-f]{64}$/.test(value.artifacts.sha256)) { + throw new Error(`client build record ${CLIENT_BUILD_RECORD_PATH} has an invalid SHA-256 digest`) + } + return { + formatVersion: CLIENT_BUILD_RECORD_FORMAT, + environment, + artifacts: { + fileCount: Number(value.artifacts.fileCount), + sha256: value.artifacts.sha256, + }, + } +} + +function isObject(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +function hasExactKeys(value: Record, expected: readonly string[]): boolean { + const actual = Object.keys(value).sort() + return actual.length === expected.length && actual.every((key, index) => key === expected[index]) +} diff --git a/scripts/client-bundle-css.spec.ts b/scripts/client-bundle-css.spec.ts index 41b3eb3679..b426e541ea 100644 --- a/scripts/client-bundle-css.spec.ts +++ b/scripts/client-bundle-css.spec.ts @@ -1,6 +1,6 @@ /** - * CSS Modules enter client bundles through virtual modules, so the loader must - * explicitly register the underlying stylesheet as a watch dependency. + * Stylesheets enter client bundles through virtual modules, so the loader must + * register their physical files as watch dependencies. */ import { mkdtemp, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' @@ -14,7 +14,7 @@ interface CssPlugin { load?: (this: { addWatchFile(id: string): void }, id: string) => Promise } -function cssPlugin(): CssPlugin { +function cssPlugin(name: 'dsh-css-modules-inline' | 'dsh-css-global-inline' | 'dsh-css-text-inline'): CssPlugin { const configs = clientBundle( '@deepseek-ai/dsh-client-test', ['lib/types/index.js', 'lib/types/invariant.js'], @@ -22,8 +22,8 @@ function cssPlugin(): CssPlugin { const client = configs.find(config => config.platform === 'browser') if (client === undefined) throw new Error('client config missing') const plugins = (client as { plugins: CssPlugin[] }).plugins - const plugin = plugins.find(candidate => candidate.name === 'dsh-css-modules-inline') - if (plugin === undefined) throw new Error('CSS Modules plugin missing from client config') + const plugin = plugins.find(candidate => candidate.name === name) + if (plugin === undefined) throw new Error(`${name} missing from client config`) return plugin } @@ -34,7 +34,7 @@ describe('client bundle CSS Modules', () => { const stylesheet = join(root, 'Fixture.module.css') const importer = join(root, 'index.ts') await writeFile(stylesheet, '.root { color: red; }\n') - const plugin = cssPlugin() + const plugin = cssPlugin('dsh-css-modules-inline') const virtualId = plugin.resolveId?.('./Fixture.module.css', importer) if (typeof virtualId !== 'string' || plugin.load === undefined) { throw new Error('CSS Modules plugin hooks are incomplete') @@ -50,3 +50,51 @@ describe('client bundle CSS Modules', () => { } }) }) + +describe('client bundle global CSS', () => { + it('compiles a side-effect stylesheet into a watched style injector', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-client-global-css-watch-')) + try { + const stylesheet = join(root, 'base.css') + const importer = join(root, 'index.ts') + await writeFile(stylesheet, 'body { color: red; }\n') + const plugin = cssPlugin('dsh-css-global-inline') + const virtualId = plugin.resolveId?.('./base.css', importer) + if (typeof virtualId !== 'string' || plugin.load === undefined) { + throw new Error('global CSS plugin hooks are incomplete') + } + const watched: string[] = [] + + const output = await plugin.load.call({ addWatchFile: id => watched.push(id) }, virtualId) + + expect(watched).toEqual([stylesheet]) + expect(output).toContain('data-plugin-css') + expect(output).toContain('body{color:red}') + } finally { + await rm(root, { recursive: true, force: true }) + } + }) + + it('compiles inline stylesheets as watched text without a module side effect', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-client-inline-css-watch-')) + try { + const stylesheet = join(root, 'base.css') + const importer = join(root, 'index.ts') + await writeFile(stylesheet, 'body { color: red; }\n') + const plugin = cssPlugin('dsh-css-text-inline') + const virtualId = plugin.resolveId?.('./base.css?inline', importer) + if (typeof virtualId !== 'string' || plugin.load === undefined) { + throw new Error('inline CSS plugin hooks are incomplete') + } + const watched: string[] = [] + + const output = await plugin.load.call({ addWatchFile: id => watched.push(id) }, virtualId) + + expect(watched).toEqual([stylesheet]) + expect(output).toContain('export default "body{color:red}"') + expect(output).not.toContain('data-plugin-css') + } finally { + await rm(root, { recursive: true, force: true }) + } + }) +}) diff --git a/scripts/client-bundle-purity.spec.ts b/scripts/client-bundle-purity.spec.ts index 4077c806f3..4278624bca 100644 --- a/scripts/client-bundle-purity.spec.ts +++ b/scripts/client-bundle-purity.spec.ts @@ -4,7 +4,7 @@ */ import { fileURLToPath } from 'node:url' import { describe, expect, it, vi } from 'vitest' -import { CLIENT_EXTERNALS, clientBundle } from '../packages/client/tsdown.client.ts' +import { clientBundle, requestedExternals } from '../packages/client/tsdown.client.ts' type ResolveId = (source: string) => null | { id: string; external: boolean } @@ -14,7 +14,10 @@ interface CssModulePlugin { load?: (this: { addWatchFile: (id: string) => void }, id: string) => Promise } -function clientConfigs(id = '@deepseek-ai/dsh-client-test') { +/** A representative dynamic bundle using the shared client baseline. */ +const REQUESTING_PACKAGE = '@deepseek-ai/dsh-client-ui-conversation' + +function clientConfigs(id = REQUESTING_PACKAGE) { return clientBundle(id, ['lib/types/index.js', 'lib/types/invariant.js'])( { env: { DSH_BUILD_FACE: 'client' } }, ).filter(config => config.platform === 'browser') @@ -36,10 +39,10 @@ function clientSourceMapPath(packagePath: string): string { return fileURLToPath(new URL(`../packages/${packagePath}/lib/client.js.map`, import.meta.url)) } -function purityResolveId(): ResolveId { +function purityResolveId(id = REQUESTING_PACKAGE): ResolveId { // libEntry is spelled at every call site (no default) so the // package-invariants text check can see the invariant entry per package. - const configs = clientConfigs() + const configs = clientConfigs(id) const plugins = (configs[0] as { plugins: { name: string; resolveId?: unknown }[] }).plugins const gate = plugins.find(p => p.name === 'dsh-client-bundle-purity') if (gate?.resolveId === undefined) throw new Error('purity plugin missing from client config') @@ -59,15 +62,16 @@ function cssModulePlugin(): CssModulePlugin { describe('client bundle purity gate', () => { const resolveId = purityResolveId() - it('leaves platform table entries and non-scoped specifiers alone', () => { + it('leaves default externals and non-scoped specifiers alone', () => { expect(resolveId('@deepseek-ai/dsh-client-ui-slots')).toBeNull() - expect(resolveId('@deepseek-ai/dsh-client-web-react')).toBeNull() expect(resolveId('@deepseek-ai/dsh-client-ui-primitives')).toBeNull() + expect(resolveId('@deepseek-ai/dsh-client-runtime/client')).toBeNull() expect(resolveId('react')).toBeNull() expect(resolveId('zod')).toBeNull() }) - it('rejects retired table entries (web-react/store left the 8-entry seed)', () => { + it('rejects the retired web-react platform package', () => { + expect(() => resolveId('@deepseek-ai/dsh-client-web-react')).toThrow(/purity/) expect(() => resolveId('@deepseek-ai/dsh-client-web-react/store')).toThrow(/purity/) }) @@ -89,17 +93,49 @@ describe('client bundle purity gate', () => { expect(() => resolveId('@deepseek-ai/dsh-client-web')).toThrow(/purity/) }) - it('throws on cross-plugin value imports — bare plugin names and /client subpaths alike (the rewrite arm is gone)', () => { + it('throws on cross-plugin value imports — bare plugin names and /client subpaths alike', () => { expect(() => resolveId('@deepseek-ai/dsh-client-connection')).toThrow(/purity/) expect(() => resolveId('@deepseek-ai/dsh-client-runtime')).toThrow(/purity/) expect(() => resolveId('@deepseek-ai/dsh-client-ui-layout/client')).toThrow(/purity/) }) - it('carries exactly one documented temporary exemption: runtime/client (store engine pending rehoming)', () => { + it('admits the parser-preloaded runtime for every dynamic bundle', () => { expect(resolveId('@deepseek-ai/dsh-client-runtime/client')).toBeNull() - const clientChannels = CLIENT_EXTERNALS.filter( - entry => entry.startsWith('@deepseek-ai/') && entry.endsWith('/client')) - expect(clientChannels).toEqual(['@deepseek-ai/dsh-client-runtime/client']) + const withoutRequest = purityResolveId('@deepseek-ai/dsh-client-ui-goal') + expect(withoutRequest('@deepseek-ai/dsh-client-runtime/client')).toBeNull() + }) + + it('externalizes the baseline independently of each package manifest', () => { + const requesting = clientConfigs()[0]?.deps as { neverBundle: (specifier: string) => boolean } + const plain = clientConfigs('@deepseek-ai/dsh-client-connection')[0]?.deps as { + neverBundle: (specifier: string) => boolean + } + + expect(requesting.neverBundle('react')).toBe(true) + expect(requesting.neverBundle('zod')).toBe(false) + expect(plain.neverBundle('react')).toBe(true) + expect(plain.neverBundle('@deepseek-ai/dsh-client-runtime/client')).toBe(true) + }) +}) + +describe('client bundle module requests', () => { + it('requests what the declaration lists', () => { + const requests = requestedExternals('@deepseek-ai/dsh-client-fixture', { + external: ['react', 'react/jsx-runtime', '@deepseek-ai/dsh-client-ui-slots'], + }) + + expect([...requests].sort()).toEqual([ + '@deepseek-ai/dsh-client-ui-slots', 'react', 'react/jsx-runtime', + ]) + }) + + it('requests nothing when the declaration is absent', () => { + expect(requestedExternals('@deepseek-ai/dsh-client-fixture', {}).size).toBe(0) + }) + + it('rejects a malformed declaration instead of reading past it', () => { + expect(() => requestedExternals('@deepseek-ai/dsh-client-fixture', { external: 'react' })) + .toThrow(/dsh\.client\.external must be a string array/) }) }) diff --git a/scripts/cordis-yaml.ts b/scripts/cordis-yaml.ts new file mode 100644 index 0000000000..e7db335c6a --- /dev/null +++ b/scripts/cordis-yaml.ts @@ -0,0 +1,54 @@ +/** + * Cordis YAML parsing and Loader-entry classification shared by repository checks. + * @module scripts/cordis-yaml + */ + +import * as yaml from 'js-yaml' + +/** A Loader `!!js` expression preserved as data instead of executed. */ +export interface JsExpr { + __jsExpr: string +} + +const jsExprType = new yaml.Type('tag:yaml.org,2002:js', { + kind: 'scalar', + resolve: data => typeof data === 'string', + construct: (data: unknown): JsExpr => { + if (typeof data !== 'string') throw new TypeError('!!js requires a scalar string') + return { __jsExpr: data } + }, +}) +const schema = yaml.JSON_SCHEMA.extend(jsExprType) + +/** + * Parse a Cordis config while preserving Loader `!!js` expressions as data. + * @param source - Cordis YAML source text. + * @returns the parsed YAML value. + */ +export function loadCordisYaml(source: string): unknown { + return yaml.load(source, { schema }) +} + +/** + * Test whether a value is a preserved Loader `!!js` expression. + * @param value - parsed YAML value. + * @returns whether the value contains one preserved expression. + */ +export function isJsExpr(value: unknown): value is JsExpr { + return typeof value === 'object' + && value !== null + && typeof (value as Record).__jsExpr === 'string' +} + +/** + * Test whether a Loader entry owns nested entries in its `config` array. + * @param value - parsed Loader entry. + * @returns whether the entry is an explicit or package-named Cordis group. + */ +export function isCordisGroupEntry(value: unknown): value is Record & { config: unknown[] } { + return typeof value === 'object' + && value !== null + && Array.isArray((value as Record).config) + && ((value as Record).group === true + || (value as Record).name === '@deepseek-ai/cordis-plugin-group') +} diff --git a/scripts/coverage-exempt.ts b/scripts/coverage-exempt.ts index b560567014..2e68b53b60 100644 --- a/scripts/coverage-exempt.ts +++ b/scripts/coverage-exempt.ts @@ -38,4 +38,11 @@ export const coverageExemptHeavySuites: readonly CoverageExemptSuite[] = [ { filter: 'scripts/install-lefthook.spec.ts', exclude: 'scripts/install-lefthook.spec.ts' }, { filter: 'scripts/oxlint-contract.spec.ts', exclude: 'scripts/oxlint-contract.spec.ts' }, { filter: 'scripts/change-scope.spec.ts', exclude: 'scripts/change-scope.spec.ts' }, + { filter: 'scripts/translation-pairing-merge.spec.ts', exclude: 'scripts/translation-pairing-merge.spec.ts' }, + // The real corpus transform runs package src only in a spawned Node process, + // outside the parent Vitest worker's v8 coverage session. + { + filter: 'packages/experimental/webworker-runtime/tests/compile/transform-corpus.spec.ts', + exclude: 'packages/experimental/webworker-runtime/tests/compile/transform-corpus.spec.ts', + }, ] diff --git a/scripts/coverage-partitions.spec.ts b/scripts/coverage-partitions.spec.ts new file mode 100644 index 0000000000..264a2686c4 --- /dev/null +++ b/scripts/coverage-partitions.spec.ts @@ -0,0 +1,257 @@ +import { access, mkdir, mkdtemp, symlink, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + COVERAGE_PARTITION_MODE_ENV, + COVERAGE_PARTITIONS_ENV, + COVERAGE_TEST_TIMEOUT_ENV, + CoveragePartitionCoordinator, + coverageTestTimeoutArgs, + forwardedCoverageArgs, + parseCoveragePartitionCount, + type CoverageCommand, + type CoverageCommandResult, +} from './coverage-partitions.ts' + +const passed: CoverageCommandResult = { exitCode: 0, signalCode: null } + +afterEach(() => vi.restoreAllMocks()) + +async function writeBlob(command: CoverageCommand): Promise { + if (command.blobPath === undefined) return + await mkdir(dirname(command.blobPath), { recursive: true }) + await writeFile(command.blobPath, '{}') +} + +async function temporaryRoot(): Promise { + return await mkdtemp(join(tmpdir(), 'dsh-coverage-partitions-')) +} + +function successfulCommandRecorder(commands: CoverageCommand[]) { + return vi.fn(async (command: CoverageCommand) => { + commands.push(command) + await writeBlob(command) + return passed + }) +} + +describe('coverage partition count', () => { + it.each([ + [undefined, undefined], + ['', undefined], + ['2', 2], + ['3', 3], + ])('parses %j as %j', (raw, expected) => { + expect(parseCoveragePartitionCount(raw)).toBe(expected) + }) + + it.each(['0', '1', '2.5', '02', 'many'])('rejects %j', (raw) => { + expect(() => parseCoveragePartitionCount(raw)) + .toThrow(`${COVERAGE_PARTITIONS_ENV} must be an integer greater than 1`) + }) +}) + +describe('coverage partition timeout', () => { + it('applies one configured timeout to tests and polling', () => { + expect(coverageTestTimeoutArgs('30000')).toEqual([ + '--testTimeout=30000', + '--expect.poll.timeout=30000', + ]) + }) + + it('keeps Vitest defaults when the timeout is absent', () => { + expect(coverageTestTimeoutArgs(undefined)).toEqual([]) + }) + + it('rejects invalid timeout input', () => { + expect(() => coverageTestTimeoutArgs('0')) + .toThrow(`${COVERAGE_TEST_TIMEOUT_ENV} must be a positive integer`) + }) +}) + +describe('coverage forwarded arguments', () => { + it('removes one package-script separator', () => { + expect(forwardedCoverageArgs(['--', 'scripts/example.spec.ts'])).toEqual(['scripts/example.spec.ts']) + }) + + it('preserves direct arguments and a subsequent Vitest separator', () => { + expect(forwardedCoverageArgs(['--testNamePattern=example'])).toEqual(['--testNamePattern=example']) + expect(forwardedCoverageArgs(['--', '--', 'example'])).toEqual(['--', 'example']) + }) +}) + +describe('coverage partition coordinator', () => { + it('runs every single-worker partition before one merged threshold check', async () => { + const root = await temporaryRoot() + const commands: CoverageCommand[] = [] + const runCommand = successfulCommandRecorder(commands) + const coordinator = new CoveragePartitionCoordinator({ + root, + partitions: 3, + pnpmEntrypoint: '/pnpm.cjs', + vitestArgs: ['--testTimeout=30000'], + runCommand, + }) + + await expect(coordinator.run()).resolves.toBe(0) + + expect(commands.map(command => command.label)).toEqual([ + 'partition 1/3', + 'partition 2/3', + 'partition 3/3', + 'merged coverage report', + ]) + for (const [index, command] of commands.slice(0, 3).entries()) { + expect(command.command).toBe(process.execPath) + expect(command.args[0]).toBe('/pnpm.cjs') + expect(command.args).toEqual(expect.arrayContaining([ + '--coverage', + '--coverage.reportOnFailure', + '--maxWorkers=1', + `--shard=${index + 1}/3`, + '--reporter=default', + '--reporter=blob', + '--testTimeout=30000', + ])) + expect(command.env).toEqual({ + [COVERAGE_PARTITIONS_ENV]: undefined, + [COVERAGE_PARTITION_MODE_ENV]: '1', + }) + } + const mergeCommand = commands[3] + if (mergeCommand === undefined) throw new Error('coverage merge command was not observed') + expect(mergeCommand.args).toContain('--coverage') + expect(mergeCommand.args.some(argument => argument.startsWith('--merge-reports='))).toBe(true) + expect(mergeCommand.env).toEqual({ + [COVERAGE_PARTITIONS_ENV]: undefined, + [COVERAGE_PARTITION_MODE_ENV]: undefined, + }) + }) + + it('runs a native pnpm entrypoint directly', async () => { + const root = await temporaryRoot() + const commands: CoverageCommand[] = [] + const runCommand = successfulCommandRecorder(commands) + const coordinator = new CoveragePartitionCoordinator({ + root, + partitions: 2, + pnpmEntrypoint: '/tools/pnpm', + runCommand, + }) + + await expect(coordinator.run()).resolves.toBe(0) + expect(commands).toHaveLength(3) + for (const command of commands) { + expect(command.command).toBe('/tools/pnpm') + expect(command.args[0]).toBe('exec') + } + }) + + it('merges normal test failures and returns their failed status', async () => { + const root = await temporaryRoot() + const reported = vi.spyOn(console, 'error').mockImplementation(() => undefined) + const runCommand = vi.fn(async (command: CoverageCommand) => { + await writeBlob(command) + return command.label === 'partition 2/2' + ? { exitCode: 1, signalCode: null, outputTail: 'specific Vitest failure' } + : passed + }) + const coordinator = new CoveragePartitionCoordinator({ + root, + partitions: 2, + pnpmEntrypoint: '/pnpm.cjs', + runCommand, + }) + + await expect(coordinator.run()).resolves.toBe(1) + expect(reported).toHaveBeenCalledWith('coverage-partitions: FAIL partition 2/2 (exit 1)') + expect(reported).toHaveBeenCalledWith( + 'coverage-partitions: output tail for partition 2/2:\nspecific Vitest failure', + ) + expect(runCommand).toHaveBeenCalledTimes(3) + }) + + it('rejects a missing partition blob before merge', async () => { + const root = await temporaryRoot() + const runCommand = vi.fn(async (command: CoverageCommand) => { + if (command.label !== 'partition 2/2') await writeBlob(command) + return passed + }) + const coordinator = new CoveragePartitionCoordinator({ + root, + partitions: 2, + pnpmEntrypoint: '/pnpm.cjs', + runCommand, + }) + + await expect(coordinator.run()).rejects.toThrow('coverage partitions produced') + expect(runCommand).toHaveBeenCalledTimes(2) + }) + + it('reports signal termination before missing-blob validation', async () => { + const root = await temporaryRoot() + const reported = vi.spyOn(console, 'error').mockImplementation(() => undefined) + const runCommand = vi.fn(async (command: CoverageCommand) => { + if (command.label === 'partition 1/2') await writeBlob(command) + return command.label === 'partition 2/2' + ? { exitCode: null, signalCode: 'SIGTERM' as const } + : passed + }) + const coordinator = new CoveragePartitionCoordinator({ + root, + partitions: 2, + pnpmEntrypoint: '/pnpm.cjs', + runCommand, + }) + + await expect(coordinator.run()).rejects.toThrow('coverage partitions produced') + expect(reported).toHaveBeenCalledWith('coverage-partitions: FAIL partition 2/2 (signal SIGTERM)') + }) + + it('waits for every partition after one spawn failure', async () => { + const root = await temporaryRoot() + const reported = vi.spyOn(console, 'error').mockImplementation(() => undefined) + let secondFinished = false + const runCommand = vi.fn(async (command: CoverageCommand) => { + await writeBlob(command) + if (command.label === 'partition 1/2') { + return { exitCode: null, signalCode: null, error: 'spawn unavailable' } + } + if (command.label === 'partition 2/2') secondFinished = true + return passed + }) + const coordinator = new CoveragePartitionCoordinator({ + root, + partitions: 2, + pnpmEntrypoint: '/pnpm.cjs', + runCommand, + }) + + await expect(coordinator.run()).resolves.toBe(1) + expect(reported).toHaveBeenCalledWith('coverage-partitions: FAIL partition 1/2 (spawn unavailable)') + expect(secondFinished).toBe(true) + expect(runCommand).toHaveBeenCalledTimes(3) + }) + + it('unlinks a link-shaped coverage path without touching its target', async () => { + const root = await temporaryRoot() + const target = await temporaryRoot() + const marker = join(target, 'marker.txt') + await writeFile(marker, 'owned elsewhere') + await symlink(target, join(root, 'coverage'), process.platform === 'win32' ? 'junction' : 'dir') + const runCommand = vi.fn(async (command: CoverageCommand) => { + await writeBlob(command) + return passed + }) + const coordinator = new CoveragePartitionCoordinator({ + root, + partitions: 2, + pnpmEntrypoint: '/pnpm.cjs', + runCommand, + }) + + await expect(coordinator.run()).resolves.toBe(0) + await expect(access(marker)).resolves.toBeUndefined() + }) +}) diff --git a/scripts/coverage-partitions.ts b/scripts/coverage-partitions.ts new file mode 100644 index 0000000000..fed0706f5e --- /dev/null +++ b/scripts/coverage-partitions.ts @@ -0,0 +1,272 @@ +/** Coordinate single-worker Vitest coverage partitions and one merged report. */ +import { spawn } from 'node:child_process' +import { lstat, mkdir, readdir, rm, unlink } from 'node:fs/promises' +import { join, relative, sep } from 'node:path' +import { pnpmInvocation } from './pnpm-invocation.ts' + +/** Environment variable selecting the number of instrumented coverage processes. */ +export const COVERAGE_PARTITIONS_ENV = 'DSH_COVERAGE_PARTITIONS' + +/** Internal marker that suppresses reports and thresholds inside a partition process. */ +export const COVERAGE_PARTITION_MODE_ENV = 'DSH_COVERAGE_PARTITION_MODE' + +/** Environment variable overriding instrumented test and polling timeouts. */ +export const COVERAGE_TEST_TIMEOUT_ENV = 'DSH_COVERAGE_TEST_TIMEOUT_MS' + +/** One child command owned by the coverage coordinator. */ +export interface CoverageCommand { + /** Diagnostic identity. */ + label: string + /** Executable launched without a platform shell. */ + command: string + /** Arguments passed to the executable. */ + args: string[] + /** Environment additions for the child. */ + env: Record + /** Working directory for the child. */ + cwd: string + /** Blob the partition must produce; absent for the merge command. */ + blobPath?: string +} + +/** Observable child-process completion. */ +export interface CoverageCommandResult { + /** Numeric process status, or `null` when a signal ended the child. */ + exitCode: number | null + /** Terminating signal, or `null` after an ordinary exit. */ + signalCode: NodeJS.Signals | null + /** Spawn failure recorded independently from process completion. */ + error?: string + /** Bounded combined stdout/stderr tail repeated when the command fails. */ + outputTail?: string +} + +/** Execute one coordinator command with inherited output. */ +export type CoverageCommandRunner = (command: CoverageCommand) => Promise + +/** Construction inputs for {@link CoveragePartitionCoordinator}. */ +export interface CoveragePartitionCoordinatorOptions { + /** Repository root that owns coverage output. */ + root: string + /** Number of concurrent single-worker Vitest processes. */ + partitions: number + /** pnpm JavaScript or executable entrypoint from `npm_execpath`. */ + pnpmEntrypoint: string + /** Additional arguments shared by every partition. */ + vitestArgs?: string[] + /** Child executor, injectable for scheduler tests. */ + runCommand?: CoverageCommandRunner +} + +/** Parse an optional coverage partition count. */ +export function parseCoveragePartitionCount(raw: string | undefined): number | undefined { + if (raw === undefined || raw === '') return undefined + const parsed = Number.parseInt(raw, 10) + if (!Number.isSafeInteger(parsed) || parsed < 2 || String(parsed) !== raw) { + throw new Error(`${COVERAGE_PARTITIONS_ENV} must be an integer greater than 1, got ${JSON.stringify(raw)}.`) + } + return parsed +} + +/** Resolve the paired Vitest timeout arguments used by coverage partitions. */ +export function coverageTestTimeoutArgs(raw: string | undefined): string[] { + if (raw === undefined || raw === '') return [] + const parsed = Number.parseInt(raw, 10) + if (!Number.isSafeInteger(parsed) || parsed < 1 || String(parsed) !== raw) { + throw new Error(`${COVERAGE_TEST_TIMEOUT_ENV} must be a positive integer, got ${JSON.stringify(raw)}.`) + } + return [`--testTimeout=${raw}`, `--expect.poll.timeout=${raw}`] +} + +/** Remove pnpm's package-script separator before forwarding Vitest arguments. */ +export function forwardedCoverageArgs(args: readonly string[]): string[] { + return [...args.slice(args[0] === '--' ? 1 : 0)] +} + +/** Run instrumented partitions, validate their blobs, and merge once. */ +export class CoveragePartitionCoordinator { + private readonly root: string + private readonly partitions: number + private readonly pnpmEntrypoint: string + private readonly vitestArgs: string[] + private readonly runCommand: CoverageCommandRunner + private readonly temporaryRoot: string + private readonly blobsRoot: string + + /** Create a coordinator from validated process-independent inputs. */ + public constructor(options: CoveragePartitionCoordinatorOptions) { + if (!Number.isSafeInteger(options.partitions) || options.partitions < 2) { + throw new Error(`coverage partitions must be an integer greater than 1, got ${String(options.partitions)}.`) + } + this.root = options.root + this.partitions = options.partitions + this.pnpmEntrypoint = options.pnpmEntrypoint + this.vitestArgs = options.vitestArgs ?? [] + this.runCommand = options.runCommand ?? runCoverageCommand + this.temporaryRoot = join(this.root, 'coverage', '.partitioned') + this.blobsRoot = join(this.temporaryRoot, 'blobs') + } + + /** + * Run every partition before one merged threshold check. + * @returns zero only when every partition and the merge command succeed. + */ + public async run(): Promise { + await removeOwnedTree(join(this.root, 'coverage')) + await mkdir(this.blobsRoot, { recursive: true }) + + try { + const commands = Array.from( + { length: this.partitions }, + (_, index) => this.partitionCommand(index + 1), + ) + const results = await Promise.all(commands.map(async (command) => { + console.log(`coverage-partitions: start ${command.label}`) + const result = await this.runCommand(command) + if (commandFailed(result)) { + console.error(`coverage-partitions: FAIL ${command.label} (${commandFailureReason(result)})`) + if (result.outputTail !== undefined && result.outputTail !== '') { + console.error(`coverage-partitions: output tail for ${command.label}:\n${result.outputTail}`) + } + } + return result + })) + await this.assertCompleteBlobSet(commands) + + const mergeCommand = this.mergeCommand() + console.log(`coverage-partitions: start ${mergeCommand.label}`) + const mergeResult = await this.runCommand(mergeCommand) + return results.some(commandFailed) || commandFailed(mergeResult) ? 1 : 0 + } finally { + await removeOwnedTree(this.temporaryRoot) + } + } + + private partitionCommand(index: number): CoverageCommand { + const blobPath = join(this.blobsRoot, `partition-${index}.json`) + const reportsDirectory = join(this.temporaryRoot, `coverage-${index}`) + const invocation = pnpmInvocation([ + 'exec', + 'vitest', + 'run', + '--coverage', + '--coverage.reportOnFailure', + '--maxWorkers=1', + `--shard=${index}/${this.partitions}`, + '--reporter=default', + '--reporter=blob', + `--outputFile.blob=${this.relativePath(blobPath)}`, + `--coverage.reportsDirectory=${this.relativePath(reportsDirectory)}`, + ...this.vitestArgs, + ], { npm_execpath: this.pnpmEntrypoint }) + return { + label: `partition ${index}/${this.partitions}`, + ...invocation, + env: { + [COVERAGE_PARTITIONS_ENV]: undefined, + [COVERAGE_PARTITION_MODE_ENV]: '1', + }, + cwd: this.root, + blobPath, + } + } + + private mergeCommand(): CoverageCommand { + const invocation = pnpmInvocation([ + 'exec', + 'vitest', + `--merge-reports=${this.relativePath(this.blobsRoot)}`, + '--coverage', + ], { npm_execpath: this.pnpmEntrypoint }) + return { + label: 'merged coverage report', + ...invocation, + env: { + [COVERAGE_PARTITIONS_ENV]: undefined, + [COVERAGE_PARTITION_MODE_ENV]: undefined, + }, + cwd: this.root, + } + } + + private relativePath(path: string): string { + return relative(this.root, path).split(sep).join('/') + } + + private async assertCompleteBlobSet(commands: CoverageCommand[]): Promise { + const expected = commands.map((command) => { + if (command.blobPath === undefined) throw new Error(`${command.label} has no blob path.`) + return this.relativePath(command.blobPath) + }).sort() + const actual = (await readdir(this.blobsRoot)) + .map(name => this.relativePath(join(this.blobsRoot, name))) + .sort() + if (actual.length !== expected.length || actual.some((name, index) => name !== expected[index])) { + throw new Error(`coverage partitions produced ${JSON.stringify(actual)}; expected ${JSON.stringify(expected)}.`) + } + } +} + +/** Spawn one pnpm-backed command without a platform shell. */ +function runCoverageCommand(command: CoverageCommand): Promise { + return new Promise((resolveCommand) => { + let outputTail = '' + const env = { ...process.env } + for (const [name, value] of Object.entries(command.env)) { + if (value === undefined) Reflect.deleteProperty(env, name) + else env[name] = value + } + const child = spawn(command.command, command.args, { + cwd: command.cwd, + env, + stdio: ['ignore', 'pipe', 'pipe'], + }) + child.stdout.setEncoding('utf8') + child.stderr.setEncoding('utf8') + child.stdout.on('data', (chunk: string) => { + process.stdout.write(chunk) + outputTail = appendOutputTail(outputTail, chunk) + }) + child.stderr.on('data', (chunk: string) => { + process.stderr.write(chunk) + outputTail = appendOutputTail(outputTail, chunk) + }) + child.once('error', (error: Error) => { + resolveCommand({ exitCode: null, signalCode: null, error: error.message, outputTail }) + }) + child.once('close', (exitCode, signalCode) => { + resolveCommand({ exitCode, signalCode, outputTail }) + }) + }) +} + +function appendOutputTail(previous: string, chunk: string): string { + const combined = previous + chunk + return combined.length <= 65_536 ? combined : combined.slice(-65_536) +} + +function commandFailed(result: CoverageCommandResult): boolean { + return result.exitCode !== 0 || result.signalCode !== null || result.error !== undefined +} + +function commandFailureReason(result: CoverageCommandResult): string { + const facts = [ + result.error, + result.exitCode === null ? undefined : `exit ${result.exitCode}`, + result.signalCode === null ? undefined : `signal ${result.signalCode}`, + ].filter((fact): fact is string => fact !== undefined) + return facts.join(', ') || 'no exit code or signal' +} + +async function removeOwnedTree(path: string): Promise { + const metadata = await lstat(path).catch((error: unknown) => { + if (error instanceof Error && 'code' in error && error.code === 'ENOENT') return undefined + throw error + }) + if (metadata === undefined) return + if (metadata.isSymbolicLink() || !metadata.isDirectory()) { + await unlink(path) + return + } + await rm(path, { recursive: true, force: true }) +} diff --git a/scripts/demo-code-mode.mjs b/scripts/demo-code-mode.mjs index 7ddbbae2cc..a395ee9951 100644 --- a/scripts/demo-code-mode.mjs +++ b/scripts/demo-code-mode.mjs @@ -8,9 +8,13 @@ if (process.argv.length > 2) { const child = spawn(process.execPath, [ '--import', - 'tsx', - 'packages/examples/acp-demo/src/bin.ts', - '--config', + 'tsx/esm', + 'apps/cli/src/bin.ts', + '--profile', + 'acp', + '--patch', + 'examples/acp-agent/cordis.yml', + '--patch', 'examples/acp-agent/code-mode.cordis.yml', ], { stdio: 'inherit' }) child.on('exit', (code, signal) => { process.exit(signal !== null ? 1 : code ?? 1) }) diff --git a/scripts/demo-cordis.mjs b/scripts/demo-cordis.mjs index 43a23ab250..a2564c5fb0 100644 --- a/scripts/demo-cordis.mjs +++ b/scripts/demo-cordis.mjs @@ -6,8 +6,12 @@ import { spawn } from 'node:child_process' const SURFACES = new Map([ // The browser surface with the cordis toolset layered on: `dsh web --config` // applies this overlay over the shipped web composition; it owns port 3081. - ['web', ['--import', 'tsx', 'apps/cli/src/bin.ts', 'web', '--patch', 'examples/web-cordis/cordis.yml']], - ['acp', ['--import', 'tsx', 'packages/examples/acp-demo/src/bin.ts', '--config', 'examples/acp-agent/cordis-tools.cordis.yml']], + ['web', ['--import', 'tsx/esm', 'apps/cli/src/bin.ts', 'web', '--patch', 'examples/web-cordis/cordis.yml']], + ['acp', [ + '--import', 'tsx/esm', 'apps/cli/src/bin.ts', '--profile', 'acp', + '--patch', 'examples/acp-agent/cordis.yml', + '--patch', 'examples/acp-agent/cordis-tools.cordis.yml', + ]], ]) const surface = process.argv[2] ?? 'web' diff --git a/scripts/dev-web.spec.ts b/scripts/dev-web.spec.ts index 293c6e94f0..f185ee6754 100644 --- a/scripts/dev-web.spec.ts +++ b/scripts/dev-web.spec.ts @@ -3,7 +3,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { expect, it } from 'vitest' import type { TsdownBundle } from 'tsdown' -import { discoverPluginDirs, watchClientPlugins } from './dev-web.ts' +import { discoverLibraryDirs, discoverPluginDirs, watchClientPlugins } from './dev-web.ts' it('discovers dsh.client packages with sibling roles', async () => { const root = await mkdtemp(join(tmpdir(), 'dsh-dev-web-discovery-')) @@ -24,6 +24,31 @@ it('discovers dsh.client packages with sibling roles', async () => { } }) +it('discovers client-preset packages the shell links, excluding loader-delivered and test infrastructure', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-dev-web-library-')) + try { + const write = async (dir: string, manifest: unknown, config: string): Promise => { + await mkdir(join(root, dir), { recursive: true }) + await writeFile(join(root, dir, 'package.json'), JSON.stringify(manifest)) + await writeFile(join(root, dir, 'tsdown.config.ts'), config) + } + const clientPreset = "import { clientLibrary } from '../tsdown.client.ts'\nexport default clientLibrary('x', [])\n" + + // Linked by the compile shell: client preset, no loader-delivered half. + await write('packages/client/linked', {}, clientPreset) + // Loader-delivered: discoverPluginDirs owns it, so it must not appear twice. + await write('packages/client/delivered', { dsh: { client: { platform: 'web' } } }, clientPreset) + // Test infrastructure builds through the preset but never enters the shell graph. + await write('packages/test-support/harness', {}, clientPreset) + // Host package with its own config: not a client-face build at all. + await write('packages/host/server', {}, "import { defineConfig } from 'tsdown'\nexport default defineConfig({})\n") + + expect(discoverLibraryDirs(root)).toEqual(['packages/client/linked']) + } finally { + await rm(root, { recursive: true, force: true }) + } +}) + it('rebuilds a client-plugin bundle after its source changes', async () => { const root = await mkdtemp(join(tmpdir(), 'dsh-dev-web-watch-')) let bundles: TsdownBundle[] = [] diff --git a/scripts/dev-web.ts b/scripts/dev-web.ts index 06eb23411a..05f929f1b8 100644 --- a/scripts/dev-web.ts +++ b/scripts/dev-web.ts @@ -1,17 +1,28 @@ /** - * Watch-build for client-plugin HMR: runs every `dsh.client` plugin package - * through the tsdown JS API in watch mode. Reload signaling is not this - * script's business — the host webserver stat-polls the bundles it serves and - * broadcasts `rebuilt` frames itself (`dsh web`), so any process that - * rewrites `lib/client.js` files triggers reloads; this script is merely the - * convenient way to keep them all rebuilt on source change. + * Watch-build for the web dev loop: rebuilds every artifact the browser reads + * from a source edit. Reload signaling is not this script's business — the host + * webserver stat-polls the bundles it serves and broadcasts `rebuilt` frames + * itself (`dsh web`), so any process that rewrites `lib/client.js` files + * triggers reloads; this script is merely the convenient way to keep them all + * rebuilt on source change. * - * Usage: `pnpm exec tsx scripts/dev-web.ts [--poll[=ms]]`. Requires the - * packages' node halves built once (`tsc -b tsconfig.build.json`): the lib - * config's entries are tsc output. `--poll` switches the source-file watcher - * to polling (default 500ms): network mounts (weka) deliver no inotify + * Three stages, because the compile shell links built lib products rather than + * sources: `tsc -b tsconfig.client.json` emits `lib/types` (the tsdown lib + * entries are that emit, not `src`), tsdown bundles `lib/index.js` and + * `lib/client.js`, and `vite build` rewrites `apps/web/dist`, which `dsh web` + * serves. A missing stage does not fail — it silently shows the previous + * artifact, so an edit appears to do nothing. + * + * MUST NOT run concurrently with `pnpm run build`: both write the same + * `lib/` and `apps/web/dist/` trees. + * + * Usage: `pnpm exec tsx scripts/dev-web.ts [--poll[=ms]]`. Requires one prior + * `pnpm run build`: every stage is incremental over the previous stage's output + * and none of them bootstraps a missing tree. `--poll` switches the source + * watchers to polling (default 500ms): network mounts (weka) deliver no inotify * events, so native watching sees the initial build only and never a source - * change. + * change. Polling has to reach tsc too — a native-watching tsc never re-emits + * `lib/types`, which strands the other two stages on stale input. * * Each package keeps its own tsdown.config.ts untouched: this script layers * `watch` through API-level inline config (tsdown workspace mode fills inline @@ -20,11 +31,24 @@ import { globSync, readFileSync } from 'node:fs' import { dirname, join, resolve, sep } from 'node:path' import { fileURLToPath, pathToFileURL } from 'node:url' +import { execa } from 'execa' import { build } from 'tsdown' import type { TsdownBundle } from 'tsdown' const repoRoot = fileURLToPath(new URL('..', import.meta.url)) +/** Client-face type emit feeding every tsdown lib entry in the watch set. */ +const CLIENT_TYPE_PROGRAM = 'tsconfig.client.json' + +/** Compile-shell workspace whose dist `dsh web` serves. */ +const SHELL_PACKAGE = '@deepseek-ai/dsh-web-frontend' + +/** + * Test infrastructure builds through the client preset but never enters the + * shell's module graph, so it is not a dev-loop artifact. + */ +const TEST_INFRASTRUCTURE_PREFIX = 'packages/test-support/' + /** * Discover the watch workspace by declaration: every packages// * whose package.json carries `dsh.client` with platform "web" is a client @@ -44,6 +68,32 @@ export function discoverPluginDirs(root = repoRoot): string[] { return dirs } +/** + * Discover the statically linked library packages: the other half of the same + * partition {@link discoverPluginDirs} takes. A package that builds through the + * client preset without declaring `dsh.client` has no loader-delivered browser + * half, so the compile shell links its `lib/index.js` instead — and an edit to + * its source reaches the browser only once that bundle is rewritten. Deriving + * the set from the build preset rather than a hand list keeps it correct when + * dependency sections move around; deriving it from `dependencies` would not, + * because client packages declare their build inputs as devDependencies. + * @param root - repository root containing the grouped package directories. + * @returns workspace-relative library package directories. + */ +export function discoverLibraryDirs(root = repoRoot): string[] { + const dirs: string[] = [] + for (const configPath of globSync('packages/*/*/tsdown.config.ts', { cwd: root }).sort()) { + const dir = dirname(configPath).split(sep).join('/') + if (dir.startsWith(TEST_INFRASTRUCTURE_PREFIX)) continue + if (!readFileSync(join(root, configPath), 'utf8').includes('tsdown.client.ts')) continue + const manifest = JSON.parse(readFileSync(join(root, dir, 'package.json'), 'utf8')) as { + dsh?: { client?: unknown } + } + if (manifest.dsh?.client === undefined) dirs.push(dir) + } + return dirs +} + /** * Start the tsdown watch build used by `pnpm run dev:web`. * @param root - repository or fixture root passed to tsdown. @@ -85,14 +135,56 @@ export async function watchClientPlugins( return bundles } +/** + * Live watcher processes to terminate when this script is interrupted. Stages + * register themselves as they start, so the set is complete from the first + * spawn: an interrupt during a later stage's startup still tears down the + * earlier ones instead of orphaning them. + */ +const stages: StageHandle[] = [] + +/** + * Spawn one watcher stage, inheriting stdio, registering it for teardown, and + * failing loud if it ever exits: a dead stage leaves the artifact chain silently + * stale, which reads as "my edit did nothing" — the one failure this script + * exists to prevent. + * @param stage - command label used in the exit diagnostic. + * @param command - executable, resolved from the workspace bin when local. + * @param args - command arguments. + * @param local - whether to resolve `command` from the workspace's installed bins. + */ +function spawnStage(stage: string, command: string, args: readonly string[], local: boolean): void { + const child = execa(command, [...args], { + cwd: repoRoot, + stdio: 'inherit', + preferLocal: local, + reject: false, + }) + stages.push({ kill: () => { child.kill() } }) + void child.then((result) => { + console.error(`dev-web: ${stage} exited (code ${String(result.exitCode)}); the artifact chain is now stale`) + process.exit(1) + }) +} + +/** The only capability this script needs from a live watcher process. */ +interface StageHandle { + readonly kill: () => void +} + const invokedPath = process.argv[1] const isMain = invokedPath !== undefined && import.meta.url === pathToFileURL(resolve(invokedPath)).href if (isMain) { const pluginDirs = discoverPluginDirs() + const libraryDirs = discoverLibraryDirs() if (pluginDirs.length === 0) { console.error('dev-web: no dsh.client (platform "web") packages found under packages/') process.exit(1) } + if (libraryDirs.length === 0) { + console.error('dev-web: no client-preset library packages found under packages/ — the compile shell links their lib products, so an empty set means the discovery predicate is stale') + process.exit(1) + } const args = process.argv.slice(2) const pollArg = args.find(a => a === '--poll' || a.startsWith('--poll=')) @@ -106,9 +198,41 @@ if (isMain) { process.exit(1) } - await watchClientPlugins(repoRoot, pluginDirs, pollInterval) + // Registered before any stage starts: `stages` is read at signal time, so an + // interrupt during tsdown's initial builds still kills whatever is running. + const stop = (): void => { for (const stage of stages) stage.kill() } + process.once('SIGINT', stop) + process.once('SIGTERM', stop) + + // tsc has no polling interval flag, so `--poll` selects its fixed-interval + // watchers rather than an interval. Dropping that translation leaves tsc + // natively watching on a network mount where inotify never fires: it stops + // re-emitting lib/types, and the two later stages then rebuild forever from + // stale input without printing anything. + spawnStage(`tsc -b ${CLIENT_TYPE_PROGRAM} --watch`, 'tsc', [ + '-b', CLIENT_TYPE_PROGRAM, '--watch', '--preserveWatchOutput', + ...pollInterval !== undefined + ? ['--watchFile', 'fixedPollingInterval', '--watchDirectory', 'fixedPollingInterval'] + : [], + ], true) + + // tsdown's initial builds are awaited before the dist watcher starts so vite's + // first build reads current lib bundles rather than whatever the last full + // build left. Its own watch then covers later lib rewrites — those files are + // in its module graph. + await watchClientPlugins(repoRoot, [...pluginDirs, ...libraryDirs], pollInterval) + // Through the shell's own `watch` script rather than vite's API: vite is not a + // repository-root dependency, and more importantly the vite root is its + // working directory — `resolve.dedupe` resolves react from that root, so + // running vite from anywhere but apps/web silently switches which react copy + // the bundle gets. + spawnStage('vite build --watch', 'pnpm', ['--filter', SHELL_PACKAGE, 'run', 'watch'], false) + console.log( `dev-web: watching ${String(pluginDirs.length)} dsh.client plugin packages` - + `${pollInterval !== undefined ? ` (polling ${String(pollInterval)}ms)` : ''}:\n ${pluginDirs.join('\n ')}`, + + ` and ${String(libraryDirs.length)} statically linked library packages` + + (pollInterval !== undefined ? ` (polling ${String(pollInterval)}ms)` : '') + + `, plus tsc -b ${CLIENT_TYPE_PROGRAM} and the ${SHELL_PACKAGE} dist build:\n ` + + [...pluginDirs, ...libraryDirs].join('\n '), ) } diff --git a/scripts/doc-budgets.manifest.json b/scripts/doc-budgets.manifest.json index 6358eae682..050cab0ffd 100644 --- a/scripts/doc-budgets.manifest.json +++ b/scripts/doc-budgets.manifest.json @@ -1,5 +1,5 @@ { - "AGENTS.md": 1900, + "AGENTS.md": 1950, "docs/AGENTS.md": 1320, "docs/architecture.md": 2400, "docs/cordis-primer.md": 600, diff --git a/scripts/doc-typecheck-paths.ts b/scripts/doc-typecheck-paths.ts index b054126aec..ec17f1b29c 100644 --- a/scripts/doc-typecheck-paths.ts +++ b/scripts/doc-typecheck-paths.ts @@ -12,7 +12,7 @@ export function builtDeclarationPath(candidate: string): string { if (sourceFile?.[1] && sourceFile[2]) { return `${sourceFile[1]}/lib/types/${sourceFile[2]}.d.ts` } - // Directory subpath entries (web-react's /store, runtime's /client): the + // Directory subpath entries (for example, runtime's /client): the // source dir maps to the same dir under lib/types (index resolution applies). const sourceDir = /^(.*)\/src\/(.+)$/.exec(candidate) if (sourceDir?.[1] && sourceDir[2]) { diff --git a/scripts/gen-client-catalog.ts b/scripts/gen-client-catalog.ts index 854bc75c6d..d8980ed95b 100644 --- a/scripts/gen-client-catalog.ts +++ b/scripts/gen-client-catalog.ts @@ -42,7 +42,7 @@ const MAX_DECL_CHARS = 1200 /** * Line budget for ONE slot's expanded report. The whole point of narrowing to a * single slot is to spend less context, so a report a model cannot finish - * reading is a defect rather than a detail. Today's widest slot renders 60 + * reading is a defect rather than a detail. The widest measured slot renders 60 * lines, so this leaves room to document a slot properly while catching the two * ways a report runs away: an owner share that hands down a subsystem instead of * a share, and prose that grew into a manual. @@ -552,7 +552,6 @@ export function main(): void { console.log(`gen-client-catalog: wrote ${OUT}.`) } -// Run only when invoked as a script, not when imported by a test. if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { main() } diff --git a/scripts/gen-config-catalog.ts b/scripts/gen-config-catalog.ts index 83fe8e7809..9c341f7684 100644 --- a/scripts/gen-config-catalog.ts +++ b/scripts/gen-config-catalog.ts @@ -874,7 +874,6 @@ function main(): void { console.log(`gen-config-catalog: wrote ${OUT}.`) } -// Run only when invoked as a script, not when imported by a test. if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { main() } diff --git a/scripts/gen-cordis-catalog-record.spec.ts b/scripts/gen-cordis-catalog-record.spec.ts index 73720e8be8..3b51562768 100644 --- a/scripts/gen-cordis-catalog-record.spec.ts +++ b/scripts/gen-cordis-catalog-record.spec.ts @@ -10,7 +10,13 @@ import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'nod import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, describe, expect, it } from 'vitest' -import { maybeRecordPair, REGION_BEGIN, REGION_END, spliceRegion } from './gen-cordis-catalog.ts' +import { + localizePageRegion, + maybeRecordPair, + REGION_BEGIN, + REGION_END, + spliceRegion, +} from './gen-cordis-catalog.ts' import { blobHash, renderPairMeta } from './translation-pairing.ts' const PAGE = 'docs/subsystems/fix.md' @@ -142,3 +148,28 @@ describe('spliceRegion', () => { .toThrow('found 2 BEGIN/2 END') }) }) + +describe('localizePageRegion', () => { + it('changes only paired Markdown paths for the Chinese generated region', () => { + const root = mkdtempSync(join(tmpdir(), 'cordis-region-locale-')) + roots.push(root) + mkdirSync(join(root, 'docs/subsystems'), { recursive: true }) + mkdirSync(join(root, 'packages'), { recursive: true }) + mkdirSync(join(root, 'scripts'), { recursive: true }) + writeFileSync(join(root, 'docs/subsystems/target.md'), '# Target\n') + writeFileSync(join(root, 'docs/subsystems/target.zh.md'), '# 目标\n') + writeFileSync(join(root, 'docs/subsystems/excluded.md'), '# Excluded\n') + writeFileSync(join(root, 'docs/subsystems/excluded.zh.md'), '# 排除\n') + writeFileSync(join(root, 'packages/outside.md'), '# Outside\n') + writeFileSync(join(root, 'packages/outside.zh.md'), '# 范围外\n') + writeFileSync(join(root, 'scripts/translation-pairing.manifest.json'), JSON.stringify({ + excluded: ['docs/subsystems/excluded.md'], + })) + const region = `${REGION_BEGIN}\n[Target](target.md#api) [Excluded](excluded.md) [Outside](../../packages/outside.md)\n${REGION_END}` + + expect(localizePageRegion(region, 'docs/subsystems/page.md', root)).toBe(region) + expect(localizePageRegion(region, 'docs/subsystems/page.zh.md', root)).toBe( + `${REGION_BEGIN}\n[Target](target.zh.md#api) [Excluded](excluded.md) [Outside](../../packages/outside.md)\n${REGION_END}`, + ) + }) +}) diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index ed622a12ee..6b19d4220f 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -3,8 +3,9 @@ * Typert catalog projection. Every harness `ctx.` service and event scope * maps to exactly one `docs/subsystems/` page through the curated tables below; * the generator injects each page's Cordis API reference between its GENERATED markers — - * byte-identically into both language sides of the pair — and re-records a - * pair's `.i18n.yaml` only when nothing outside the region changed. The + * into both language sides of the pair, localizing paired document paths for + * the Chinese side while retaining every other byte — and re-records a pair's + * `.i18n.yaml` only when nothing outside the region changed. The * projection enforces event modes, JSDoc parameter/return completeness, and * signature type-link coverage; the inherited (vendor) tier renders to * `docs/cordis-api/inherited.md`. `--check` verifies every generated artifact. @@ -32,9 +33,12 @@ import { contextKeyMap, contextMergeFiles, eventNameList } from './cordis-walk.t import { blobHash, parsePairMeta, + parseTranslationPairingManifest, partitionGeneratedRegions, renderPairMeta, + translationPairSourcePredicate, } from './translation-pairing.ts' +import { rewriteTranslationLinkLocales } from './translation-links.ts' const root = resolve(import.meta.dirname, '..') const SUBSYSTEMS_DIR = 'docs/subsystems' @@ -64,10 +68,13 @@ export const SERVICE_PAGE: Record = { commands: 'commands.md', compaction: 'compaction.md', cordisInspect: 'extensions.md', + authorization: 'credentials.md', credentials: 'credentials.md', directoryPicker: 'workspace.md', + deepseekLlmApiExtensions: 'llm-streaming.md', dynamicCordisRunner: 'extensions.md', e2b: 'subprocess.md', + fileReferences: 'session-reference.md', fs: 'filesystem.md', goals: 'goal.md', webServer: 'web-server.md', @@ -97,6 +104,7 @@ export const SERVICE_PAGE: Record = { systemPrompt: 'system-prompt.md', jobs: 'jobs.md', sessionTelemetry: 'session-telemetry.md', + agentTeams: 'agent-team.md', tokenMeter: 'token-meter.md', toolResultPruner: 'compaction.md', tools: 'tools.md', @@ -105,6 +113,7 @@ export const SERVICE_PAGE: Record = { userQuestions: 'user-questions.md', web: 'web.md', workflowEngine: 'workflow.md', + webhookRuntime: 'webhook.md', workspaceRegistry: 'workspace.md', } @@ -128,6 +137,7 @@ export const SERVICE_PAGE: Record = { */ export const SERVICE_WALK_EXEMPTIONS: Record = { agent: 'not a service: the DX accessor field on Agent.ctx (root accessor defaulting to undefined) — docs/subsystems/core.md owns the Agent handle', + appReady: 'not a service: launcher-provided successful-startup signal — packages/boot/cmdline/README.md owns the launcher contract', appExit: 'not a service: launcher-provided bounded process-exit callback — packages/boot/cmdline/README.md owns the launcher contract', cmdlineArgs: 'not a service: launcher-provided immutable app argument accessor — packages/boot/cmdline/README.md owns the launcher contract', configuredAgentIdentities: 'not a service: launcher-provided boot-context value (ConfiguredAgentIdentities | undefined) — packages/core/agent-loop/README.md owns this launcher contract', @@ -135,7 +145,8 @@ export const SERVICE_WALK_EXEMPTIONS: Record = { dshHomePath: 'not a service: boot-provided root accessor function (typeof dshHomePath | undefined) for Loader !!js config expressions — packages/boot/app-boot/README.md owns the boot contract', launchEnvironment: 'not a service: launcher-provided root accessor value (LaunchEnvironmentSnapshot | undefined) — packages/util/launch-environment/README.md owns this launcher contract', connection: 'interface-typed (HostConnectionHandle); implementing class HostConnectionService is declared in rpc-host.ts — packages/client/connection/README.md owns the API', - appShell: 'client-side interface-typed browser service — packages/client/web/README.md owns the API', + uiRenderer: 'client-side interface-typed browser service — packages/client/ui-renderer/README.md owns the API', + settingsSchema: 'client-side schema introspection service — packages/client/ui-settings/README.md owns the API', settingsScope: 'client-side settings-namespace transport service — packages/client/ui-settings/README.md owns the API', chatFileMentions: 'client-side slot-contract accessor (ChatFileMentions) — packages/client/ui-conversation/README.md owns the API', commandUi: 'client-side interface-typed browser service — packages/client/ui-commands/README.md owns the API', @@ -169,6 +180,7 @@ export const EVENT_SCOPE_PAGE: Record = { 'approval': 'approval.md', 'commands': 'commands.md', 'cordis': 'extensions.md', + 'authorization': 'credentials.md', 'credentials': 'credentials.md', 'domain': 'storage.md', 'fs': 'filesystem.md', @@ -181,6 +193,7 @@ export const EVENT_SCOPE_PAGE: Record = { 'system-prompt': 'system-prompt.md', 'session-telemetry': 'session-telemetry.md', 'tools': 'tools.md', + 'webserver': 'web-server.md', 'workflow': 'workflow.md', } @@ -230,6 +243,9 @@ export const LINK_MAP: Readonly> = { SettleReason: 'core.md', AdapterRegistrationHandle: 'llm-streaming.md', DirectoryRegistrationHandle: 'llm-streaming.md', + DeepSeekLlmApiExtensionMap: 'llm-streaming.md', + DeepSeekLlmApiExtensionProvider: 'llm-streaming.md', + DeepSeekLlmApiExtensionRequest: 'llm-streaming.md', LlmCallConfig: 'llm-streaming.md', LlmModelContext: 'llm-streaming.md', LlmModelReasoningInfo: 'llm-streaming.md', @@ -268,7 +284,9 @@ export const LINK_MAP: Readonly> = { RequestErrorAction: 'core.md', RequestFailureContext: 'core.md', PreparedReferencedMessage: 'session-reference.md', + FileReferenceCandidate: 'session-reference.md', SessionReferenceCandidate: 'session-reference.md', + SessionReferenceMentionCandidate: 'session-reference.md', SessionReferenceInput: 'session-reference.md', SessionEvent: 'session.md', SessionId: 'core.md', @@ -279,7 +297,10 @@ export const LINK_MAP: Readonly> = { ApprovalPolicy: 'approval.md', ApprovalRequest: 'approval.md', ApprovalService: 'approval.md', + EncodedImageAttachment: 'attachment.md', ImageAttachmentRef: 'attachment.md', + ImageRequestPolicy: 'attachment.md', + RequestImageAttachment: 'attachment.md', SaveImageAttachment: 'attachment.md', StoredImageAttachment: 'attachment.md', ShellExecRequest: 'shell.md', @@ -328,6 +349,7 @@ export const LINK_MAP: Readonly> = { LspQueryResult: 'lsp.md', LlmAdapter: 'llm-streaming.md', PreparedLlmCall: 'llm-streaming.md', + PreparedDeepSeekLlmApiExtensions: 'llm-streaming.md', LlmRuntime: 'llm-streaming.md', StreamChunk: 'llm-streaming.md', SkillProviderControl: 'skills.md', @@ -422,6 +444,18 @@ export const LINK_MAP: Readonly> = { JobSnapshot: 'jobs.md', JobStart: 'jobs.md', JobsChangedListener: 'jobs.md', + CreateTeamTaskRequest: 'agent-team.md', + SendTeamMessageRequest: 'agent-team.md', + SendTeamMessageResult: 'agent-team.md', + SpawnTeammateRequest: 'agent-team.md', + SpawnTeammateResult: 'agent-team.md', + TeamId: 'agent-team.md', + TeamMemberView: 'agent-team.md', + TeamMembership: 'agent-team.md', + TeamTaskId: 'agent-team.md', + TeamTaskView: 'agent-team.md', + TeamWaitResult: 'agent-team.md', + UpdateTeamTaskRequest: 'agent-team.md', TokenMeasurement: 'token-meter.md', CodeDispatchLog: 'tools.md', PostToolDecision: 'tools.md', @@ -445,8 +479,23 @@ export const LINK_MAP: Readonly> = { SettingsPathOp: 'settings.md', SettingsDescribeOptions: 'settings.md', SettingsUpdateSource: 'settings.md', + AuthorizationEntry: 'credentials.md', + AuthorizationFlow: 'credentials.md', + AuthorizationInteraction: 'credentials.md', + AuthorizationMethod: 'credentials.md', + AuthorizationNotice: 'credentials.md', + AuthorizationOutcome: 'credentials.md', + AuthorizationPrompt: 'credentials.md', + AuthorizationRequest: 'credentials.md', + AuthorizationSession: 'credentials.md', + AuthorizationSettlement: 'credentials.md', + AuthorizationStatus: 'credentials.md', CredentialRef: 'credentials.md', + CredentialKey: 'credentials.md', CredentialInfo: 'credentials.md', + CredentialRecord: 'credentials.md', + CredentialRecordEntry: 'credentials.md', + CredentialRecordInfo: 'credentials.md', ResolvedCredential: 'credentials.md', AskUserQuestionAnswer: 'user-questions.md', AskUserQuestionRequest: 'user-questions.md', @@ -458,10 +507,13 @@ export const LINK_MAP: Readonly> = { WebSearchRequest: 'web.md', WebSearchResult: 'web.md', WorkflowRun: 'workflow.md', + VerifiedWebhookDelivery: 'webhook.md', + WebhookRule: 'webhook.md', PresetOption: 'permission-presets.md', PresetSpec: 'permission-presets.md', InvariantInstaller: 'invariants.md', WebRoute: 'web-server.md', + IndexInjection: 'web-server.md', StorageBackend: 'storage.md', StorageForms: 'storage.md', Domain: 'storage.md', @@ -476,6 +528,7 @@ export const LINK_MAP: Readonly> = { WorkflowStartRequest: 'workflow.md', ProjectionDefinition: 'session-projection.md', SessionProjectionMap: 'session-projection.md', + SessionProjectionStateMap: 'session-projection.md', ProjectionChangeListener: 'session-projection.md', ProjectionSnapshot: 'session-projection.md', ProjectionCheckpoint: 'session-projection.md', @@ -494,7 +547,11 @@ export const FOUNDATION_TYPE_NAMES: ReadonlySet = new Set([ 'AsyncIterable', 'Context', 'Error', + 'EntryTree', + 'Exclude', 'Map', + 'NonNullable', + 'Omit', 'Partial', 'Pick', 'Promise', @@ -698,6 +755,19 @@ export interface WalkPartitionMaps { readonly eventWalkExemptions: Readonly> } +/** Project paired Markdown destinations in one generated region to the page's locale. */ +export function localizePageRegion(region: string, pageRel: string, scanRoot: string = root): string { + if (!pageRel.endsWith('.zh.md')) return region + const manifest = parseTranslationPairingManifest( + readFileSync(resolve(scanRoot, 'scripts/translation-pairing.manifest.json'), 'utf8'), + ) + return rewriteTranslationLinkLocales(region, { + repoRoot: scanRoot, + sourcePath: pageRel, + isTranslationPairSource: translationPairSourcePredicate(manifest), + }).content +} + /** * Judge the rendered API and the independent AST scan against the curated * partition maps, fail-closed in both directions for services AND events: a @@ -821,6 +891,7 @@ export function computeOutputs(): [string, string][] { ) for (const side of [page, page.replace(/\.md$/, '.zh.md')]) { const rel = `${SUBSYSTEMS_DIR}/${side}` + const localizedRegion = localizePageRegion(region, rel) let current: string try { current = readFileSync(resolve(root, rel), 'utf8') @@ -831,7 +902,7 @@ export function computeOutputs(): [string, string][] { continue } try { - outputs.push([rel, spliceRegion(current, region)]) + outputs.push([rel, spliceRegion(current, localizedRegion)]) } catch (error) { problems.push(`${rel}: ${error instanceof Error ? error.message : String(error)}`) } @@ -947,7 +1018,6 @@ export function main(): void { console.log(`gen-cordis-catalog: ${outputs.length} artifact(s) computed, ${changedPages} written, ${recorded} pair record(s) refreshed.`) } -// Run only when invoked as a script, not when imported by a test. if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { main() } diff --git a/scripts/gen-doc-graphs.ts b/scripts/gen-doc-graphs.ts index 0ee6ae5a13..1aeb42b436 100644 --- a/scripts/gen-doc-graphs.ts +++ b/scripts/gen-doc-graphs.ts @@ -67,6 +67,7 @@ const GROUP_ORDER = [ 'core', 'typert', 'goal', + 'experimental', 'process', 'bash', 'pty', @@ -79,6 +80,7 @@ const GROUP_ORDER = [ 'tasks', 'workflow', 'web', + 'webhook', 'spill', 'todo', 'plan', @@ -114,6 +116,15 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['agent-loop', 'compaction-basic'], note: 'Adapters register provider implementations; the loop and compaction call the provider-neutral stream service.', }, + { + key: 'deepseekLlmApiExtensions', + pkg: 'deepseek-llm-api-extensions', + title: 'Official DeepSeek request extensions', + mode: 'seam', + implementations: ['session-log-deepseek', 'plugin-package-inventory-deepseek'], + consumers: ['llm-deepseek'], + note: 'Plugins prepare independent top-level fields; the official adapter merges them and commits their delivery state after HTTP acceptance.', + }, { key: 'tokenMeter', pkg: 'token-meter', @@ -188,6 +199,15 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['llm-deepseek', 'llm-pi-ai', 'apiproxy'], note: 'Configuration carries references to secrets; providers own the values. Consumers resolve per operation, so a rotated credential reaches the very next request; the web gateway exposes value-free views and write-only storage.', }, + { + key: 'authorization', + pkg: 'authorization', + title: 'Authorization flow registry', + mode: 'seam', + implementations: [], + consumers: ['llm-pi-ai'], + note: 'Flows are registered by the plugin that knows how to obtain one credential and keyed by the record they write; the seam owns the conversation and the one-attempt-per-key lifecycle, never the protocol.', + }, { key: 'sessionTelemetry', pkg: 'session-telemetry', @@ -238,6 +258,14 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['session-reference', 'tool-session-query'], note: 'The interface supplies exact reads, filters, and traces; its concrete backend adds full-text reconciliation, ranking, snippets, and cursor generations, while the model consumer owns workspace authority and cursor-free rendering.', }, + { + key: 'fileReferences', + pkg: 'file-reference', + title: 'File reference discovery', + mode: 'seam', + implementations: ['file-reference-local'], + note: 'The interface returns path-only completion candidates within the addressed Agent cwd through its unary Remote contract; providers own namespace access and ranking without reading file contents.', + }, { key: 'sessionReferenceResolver', pkg: 'session-reference', @@ -469,6 +497,14 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['tool-subagent', 'tool-subagent-control', 'tool-ralph'], note: 'Providers implement transports; the service also owns optional Activation-based continuation orchestration, tool-subagent selects one-shot or continuable delegation, tool-subagent-control delivers follow-ups, and tool-ralph requires one fresh structured-output route.', }, + { + key: 'agentTeams', + pkg: 'agent-team', + title: 'Agent Teams coordination domain', + mode: 'core', + consumers: ['tool-agent-team'], + note: 'Owns the implicit-root roster, durable peer mailbox, shared task DAG, and continuable-child lifecycle; tool-agent-team contributes the scoped model policy and controls.', + }, { key: 'jobs', pkg: 'jobs', @@ -530,6 +566,14 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['tool-workflow', 'tool-ralph'], note: 'One engine per context, as in bash, with no named-provider registry; the general workflow and fixed Ralph consumers start runs whose agent() calls fan out through ctx.subagents.', }, + { + key: 'webhookRuntime', + pkg: 'webhook', + title: 'Webhook rule runtime', + mode: 'core', + consumers: ['webhook-github'], + note: 'Provider adapters dispatch authenticated deliveries; trusted plugins register independent process-local rules, and the runtime turns non-null results into ordinary Workspace-backed Sessions without delivery or completion state.', + }, { key: 'lsp', pkg: 'lsp', @@ -717,34 +761,20 @@ const APP_EXAMPLES = [ { id: 'acp', rel: 'examples/acp-agent/composition.md', - title: 'ACP Automation App Composition', + title: 'ACP Automation Profile Patch', label: 'examples/acp-agent', config: 'examples/acp-agent/cordis.yml', - summary: 'The ACP demo exposes fresh baseline-prompt agent sessions to programmatic clients over JSON-RPC stdio, with no stdout logger, human UI, or pre-created agent.', + summary: 'The ACP example patches the shipped base + acp-app profile for demos and snapshots; dsh owns launch, and the ACP bridge exposes fresh automation sessions without a stdout logger or pre-created agent.', }, ] type AppExample = typeof APP_EXAMPLES[number] -function renderAppExpansion(lines: string[], appNode: string, pluginName: string): void { - const agentCore = nodeId('bundle', 'agent_core') - const jsonl = nodeId('bundle', 'jsonl') - lines.push(` ${appNode} --> ${agentCore}["@deepseek-ai/dsh-agent-spine-demo"]`) - lines.push(` ${appNode} --> ${jsonl}["@deepseek-ai/dsh-session-persistence-jsonl"]`) - if (pluginName === '@deepseek-ai/dsh-acp-demo') { - lines.push(` ${appNode} --> ${nodeId('entrypoint', 'acp')}["@deepseek-ai/dsh-acp
automation-only JSON-RPC stdio
fresh sessions created by client"]`) - } - lines.push( - ` ${agentCore} --> ${nodeId('spine', 'llm')}["ctx.llm"]`, - ` ${agentCore} --> ${nodeId('spine', 'sessions')}["ctx.sessions"]`, - ` ${agentCore} --> ${nodeId('spine', 'tools')}["ctx.tools + tool-bash"]`, - ` ${agentCore} --> ${nodeId('spine', 'loop')}["ctx.agents + ctx.agentLoop"]`, - ) -} - function renderAppComposition(example: AppExample): string { const plugins = parseExampleCordis(example.config) - const maintenance = 'hybrid: the leaf plugin list is parsed from its `cordis.yml`; app package expansion is curated from package source' + const maintenance = example.id === 'acp' + ? 'hybrid: the patch row list is parsed from its `cordis.yml`; the scope summary is curated' + : 'hybrid: the leaf plugin list is parsed from its `cordis.yml`; app package expansion is curated from package source' const lines = generatedHeader(example.title) lines.push( example.summary, @@ -757,9 +787,6 @@ function renderAppComposition(example: AppExample): string { const pluginNode = nodeId(`plugin_${example.id}`, plugin.id) lines.push(` ${pluginNode}["${escLabel(plugin.id)}
${escLabel(plugin.name)}"]`) lines.push(` cfg --> ${pluginNode}`) - if (plugin.name === '@deepseek-ai/dsh-acp-demo') { - renderAppExpansion(lines, pluginNode, plugin.name) - } } lines.push( '```', diff --git a/scripts/gen-persistence-catalog.ts b/scripts/gen-persistence-catalog.ts index 38100fd531..a6ea8d9392 100644 --- a/scripts/gen-persistence-catalog.ts +++ b/scripts/gen-persistence-catalog.ts @@ -36,22 +36,27 @@ const EVENT_ENVELOPE_TYPE_NAMES = [ type EventEnvelopeTypeName = typeof EVENT_ENVELOPE_TYPE_NAMES[number] -/** Primary subsystems page for linked payload types. */ +/** Documentation target, relative to `docs/`, for linked payload types. */ const LINK_MAP: Record = { - CallId: 'core.md', - ContentBlock: 'core.md', - MessageSource: 'core.md', - ScheduleChange: 'schedule.md', - StreamChunk: 'llm-streaming.md', - TokenUsage: 'llm-streaming.md', - TodoItem: 'session.md', - TurnTrigger: 'session.md', - TurnEndReason: 'session.md', - SessionTitleEventData: 'session-title.md', - SessionTitleLlmRequestEventData: 'session-title.md', - SessionTitleModelProvenance: 'session-title.md', - SessionTitleProviderId: 'session-title.md', - SessionTitleSource: 'session-title.md', + CallId: 'subsystems/core.md', + ContentBlock: 'subsystems/core.md', + MessageSource: 'subsystems/core.md', + ScheduleChange: 'subsystems/schedule.md', + StreamChunk: 'subsystems/llm-streaming.md', + TokenUsage: 'subsystems/llm-streaming.md', + TodoItem: 'subsystems/todo.md', + TurnTrigger: 'subsystems/session.md', + TurnEndReason: 'subsystems/session.md', + SessionTitleEventData: 'subsystems/session-title.md', + SessionTitleLlmRequestEventData: 'subsystems/session-title.md', + SessionTitleModelProvenance: 'subsystems/session-title.md', + SessionTitleProviderId: 'subsystems/session-title.md', + SessionTitleSource: 'subsystems/session-title.md', + TeamId: 'subsystems/agent-team.md', + TeamMemberSnapshot: 'subsystems/agent-team.md', + TeamMessageId: 'subsystems/agent-team.md', + TeamMessageSnapshot: 'subsystems/agent-team.md', + TeamTaskSnapshot: 'subsystems/agent-team.md', } /** One log event, extracted from a `SessionEventMap` declaration. */ @@ -336,7 +341,7 @@ function typeLinks(payload: string): string { if (new RegExp(`\\b${name}\\b`).test(payload)) seen.add(name) } if (seen.size === 0) return '' - const links = [...seen].sort().map(n => `[${n}](subsystems/${LINK_MAP[n]})`) + const links = [...seen].sort().map(n => `[${n}](${LINK_MAP[n]})`) return `Types: ${links.join(' · ')}` } @@ -461,7 +466,6 @@ function main(): void { } } -// Run only when invoked as a script, not when imported by a test. if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { main() } diff --git a/scripts/gen-third-party-notices.ts b/scripts/gen-third-party-notices.ts index f2411f5eff..42b7d29c2c 100644 --- a/scripts/gen-third-party-notices.ts +++ b/scripts/gen-third-party-notices.ts @@ -673,7 +673,6 @@ export function render(): string { ) ? collectClaudeDistribution() : undefined - const nonPermissiveDev = devDeps.filter(dep => !isPermissive(dep.license)) // A copyleft license reaching a shipped surface is a distribution decision, // not a rendering detail; the notices cannot quietly absorb it. @@ -693,7 +692,7 @@ export function render(): string { DeepSeek Harness is licensed under [MIT](LICENSE). It depends on the third-party software listed below. Each project remains under its own license; nothing in this file changes those terms. -This file lists **direct** dependencies declared by the workspace and the explicitly disclosed official Claude platform payload closure. It is generated from the workspace manifests by \`scripts/gen-third-party-notices.ts\`: a pre-commit hook regenerates it whenever a staged file changes one of its inputs, and \`scripts/gen-third-party-notices.spec.ts\` asserts in the test lane that the committed bytes match. Deleting a manifest runs no hook, so that case is caught by the assertion instead. Run \`pnpm run verify-third-party-notices\` for the standalone check. +This file lists **direct** dependencies declared by the workspace and the explicitly disclosed official Claude Code platform payload closure. It is generated from the workspace manifests by \`scripts/gen-third-party-notices.ts\`: a pre-commit hook regenerates it whenever a staged file changes one of its inputs, and \`scripts/gen-third-party-notices.spec.ts\` asserts in the test lane that the committed bytes match. Deleting a manifest runs no hook, so that case is caught by the assertion instead. Run \`pnpm run verify-third-party-notices\` for the standalone check. The complete npm transitive closure, including the Landlock launcher workspace, is recorded with exact pinned versions in [\`pnpm-lock.yaml\`](pnpm-lock.yaml) — inspect it with \`pnpm licenses list\`. The Python closure is recorded separately in [\`python/sdk/uv.lock\`](python/sdk/uv.lock). @@ -769,7 +768,6 @@ function main(): void { console.log(`gen-third-party-notices: wrote ${OUT}.`) } -// Run only when invoked as a script, not when imported by a test. if (process.argv[1] !== undefined && import.meta.filename === resolve(process.argv[1])) { main() } diff --git a/scripts/gen-tool-catalog.ts b/scripts/gen-tool-catalog.ts index 005eb4aef9..19ab10c455 100644 --- a/scripts/gen-tool-catalog.ts +++ b/scripts/gen-tool-catalog.ts @@ -43,6 +43,7 @@ import * as ToolAskUser from '@deepseek-ai/dsh-tool-ask-user' import * as ToolBash from '@deepseek-ai/dsh-tool-bash' import * as ToolPwsh from '@deepseek-ai/dsh-tool-pwsh' import * as ToolBashPersistent from '@deepseek-ai/dsh-tool-bash-persistent' +import * as ToolPwshPersistent from '@deepseek-ai/dsh-tool-pwsh-persistent' import CordisHostRunner from '@deepseek-ai/dsh-cordis-host-runner' import * as ToolCordis from '@deepseek-ai/dsh-tool-cordis' import * as ToolFs from '@deepseek-ai/dsh-tool-fs' @@ -57,6 +58,8 @@ import * as ToolLsp from '@deepseek-ai/dsh-tool-lsp' import * as ToolSkill from '@deepseek-ai/dsh-tool-skill' import * as ToolSessionQuery from '@deepseek-ai/dsh-tool-session-query' import * as ToolTasks from '@deepseek-ai/dsh-tool-jobs' +import type TeamService from '@deepseek-ai/dsh-experimental-agent-team' +import * as ToolTeam from '@deepseek-ai/dsh-experimental-tool-agent-team' import * as ToolTodo from '@deepseek-ai/dsh-tool-todo' import * as ToolSubagent from '@deepseek-ai/dsh-tool-subagent' import * as ToolWeb from '@deepseek-ai/dsh-tool-web' @@ -72,6 +75,7 @@ class CatalogAttachmentStore extends AttachmentStore { maxImagesPerMessage: 1, maxMessageImageBytes: 1, maxImagePixels: 1, + maxImageDimension: 1, mediaTypes: Object.freeze(['image/png'] as const), }) @@ -280,6 +284,19 @@ const TOOL_PACKAGES: ToolPackage[] = [ note: 'One owner-isolated persistent bash tool; deployment composition supplies the PTY backend and may override the model-facing environment description.', }, + { + pkg: '@deepseek-ai/dsh-tool-pwsh-persistent', + dir: 'tool-pwsh-persistent', + source: 'packages/shell/tool-pwsh-persistent/src/index.ts', + requires: ['ctx.tools', 'ctx.terminals', 'an owning Agent at execution time'], + writes: ['tool/call', 'PTY shell state', 'tool/result'], + async mount(ctx) { + await ctx.plugin(TerminalSessionService) + await ctx.plugin(ToolPwshPersistent) + }, + note: + 'One owner-isolated persistent pwsh tool, the Windows counterpart of the persistent bash tool; deployment composition supplies a pwsh-dialect PTY backend and may override the model-facing environment description.', + }, { pkg: '@deepseek-ai/dsh-tool-str-replace-editor', dir: 'tool-str-replace-editor', @@ -297,18 +314,18 @@ const TOOL_PACKAGES: ToolPackage[] = [ pkg: '@deepseek-ai/dsh-tool-fs', dir: 'tool-fs', source: 'packages/fs/tool-fs/src/index.ts', - requires: ['ctx.tools', 'ctx.fs', 'ctx.systemPrompt', 'ctx.attachments (read_image registration)', 'ctx.llm + an image-capable route (read_image execution)'], + requires: ['ctx.tools', 'ctx.fs', 'ctx.systemPrompt', 'ctx.attachments (image-tool registration)', 'ctx.llm + an image-capable route (image-tool execution)'], writes: ['tool/call', 'fs/write-intent or fs/edit-intent for mutations', 'fs/observed after read presence/absence or successful file operation', 'durable attachment (read_image)', 'tool/result'], async mount(ctx) { // The tool needs `fs`; the bare provider is sufficient because policy // changes behavior, not schema shape. The catalog seam marker opts into - // the attachments-conditional read_image schema without attachment I/O. + // the attachments-conditional image schema without attachment I/O. await ctx.plugin(LocalFileSystem) await ctx.plugin(CatalogAttachmentStore) await ctx.plugin(ToolFs) }, note: - 'The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. `read_image` is not registered without `ctx.attachments`; its schema is route-independent, and execution refuses unless the exact routed model declares image input.', + 'The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The image tool is not registered without `ctx.attachments`; its schema is route-independent, and execution refuses unless the exact routed model declares image input.', }, { pkg: '@deepseek-ai/dsh-tool-fs-search', @@ -506,6 +523,44 @@ const TOOL_PACKAGES: ToolPackage[] = [ note: 'The kind-agnostic background-job controller: background bash commands, PTY sends, and subagents are read, listed, and killed through the same three tools. Loading the plugin attaches the controller that arms producers\' `ctx.jobs.start()`.', }, + { + pkg: '@deepseek-ai/dsh-experimental-tool-agent-team', + dir: 'tool-agent-team', + source: 'packages/experimental/tool-agent-team/src/index.ts', + requires: ['ctx.tools', 'ctx.systemPrompt', 'ctx.agentTeams', 'an exact live Team member Agent'], + writes: ['tool/call', 'team/member', 'team/message/queued', 'team/message/delivered', 'team/task', 'tool/result'], + async mount(ctx) { + await ctx.plugin(AgentRegistry) + await ctx.plugin(SessionStore) + const session = ctx.sessions.create(SessionId('tool-catalog-team-lead')) + let agent!: Agent + const membership = { + get root() { return agent }, + id: session.id, + role: 'lead' as const, + name: 'lead', + } + ctx.provide('agentTeams', { + tryMembership: (candidate: Agent) => candidate === agent ? membership : undefined, + membership: () => membership, + } as unknown as TeamService) + await ctx.plugin(Object.assign((inner: Context) => { + agent = { + id: session.id, + session, + options: {}, + status: 'idle', + } as unknown as Agent + Object.assign(agent, { ctx: createScope(inner, agent).ctx }) + inner.agents.register(agent) + }, { inject: ['tools', 'systemPrompt', 'agents', 'agentTeams'] })) + await ctx.plugin(ToolTeam) + catalogChildScopes.set(ctx, agent) + }, + scope: ctx => catalogChildScopes.get(ctx) as Agent, + note: + 'All ten tools are scoped to implicit Team Leads and durable teammates. The shipped dsh-base bundle keeps the package disabled; the documented Agent Teams profile patch enables it while disabling the legacy continuable-child control names.', + }, { pkg: '@deepseek-ai/dsh-tool-todo', dir: 'tool-todo', @@ -596,9 +651,9 @@ export function assertManifestComplete(packages: ToolPackage[] = TOOL_PACKAGES, * A tool package that boots without registering anything is a broken boot, not * an empty catalog section. The usual cause is an `inject` the entry's `mount` * does not satisfy: cordis leaves the plugin PENDING, every step here still - * succeeds, and the generator writes a catalog missing that package's tools — - * with the freshness gate green on it, because the omission is now what the - * generator produces. {@link assertManifestComplete} cannot see this: the + * succeeds, and the generator writes a catalog missing that package's tools. + * The freshness gate stays green because regeneration reproduces the omission. + * {@link assertManifestComplete} cannot see this: the * package IS listed, it just contributed nothing. * @param entry - the manifest entry that was booted. * @param harvested - how many schemas its boot registered. @@ -753,7 +808,6 @@ async function main(): Promise { console.log(`gen-tool-catalog: wrote ${OUT}.`) } -// Run only when invoked as a script, not when imported by a test. if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { await main() } diff --git a/scripts/gen-translation-brief.ts b/scripts/gen-translation-brief.ts index 0a0d03cc61..fac9e9bcb4 100644 --- a/scripts/gen-translation-brief.ts +++ b/scripts/gen-translation-brief.ts @@ -16,11 +16,13 @@ import { existsSync, globSync, mkdtempSync, readFileSync, rmSync, writeFileSync import { tmpdir } from 'node:os' import { basename, join, resolve, sep } from 'node:path' import { + isTranslationPairingManifestExcluded, isTranslationScopeFile, pairAnchorOfArgument, parseTranslationMarkdown, parseTranslationPairingManifest, TRANSLATION_SCOPE_GLOB_EXCLUDES, + translationPairSourcePredicate, translationStructureDiff, translationStructureSignature, } from './translation-pairing.ts' @@ -41,10 +43,11 @@ import { const root = resolve(import.meta.dirname, '..') const manifest = parseTranslationPairingManifest(readFileSync(join(root, 'scripts/translation-pairing.manifest.json'), 'utf8')) +const isTranslationPairSource = translationPairSourcePredicate(manifest) const terminology = readFileSync(join(root, 'docs/i18n/terminology.md'), 'utf8') function isExcluded(file: string): boolean { - return manifest.excluded.some(entry => (entry.endsWith('/') ? file.startsWith(entry) : file === entry)) + return isTranslationPairingManifestExcluded(file, manifest) } /** Recorded hashes of one consistency record: basename → blob hash. */ @@ -215,12 +218,23 @@ function planScope( /** Validate a computed mechanical counterpart and write it. */ function applyMechanical(counterpartPath: string, sourceCurrent: string, result: string): void { const counterpartBase = basename(counterpartPath) + const sourcePath = counterpartPath.endsWith('.zh.md') + ? counterpartPath.replace(/\.zh\.md$/, '.md') + : counterpartPath.replace(/\.md$/, '.zh.md') const sourceBase = counterpartBase.endsWith('.zh.md') ? counterpartBase.replace(/\.zh\.md$/, '.md') : counterpartBase.replace(/\.md$/, '.zh.md') const errors = translationStructureDiff( - translationStructureSignature(parseTranslationMarkdown(sourceCurrent), counterpartBase), - translationStructureSignature(parseTranslationMarkdown(result), sourceBase), + translationStructureSignature( + parseTranslationMarkdown(sourceCurrent), + counterpartBase, + { repoRoot: root, sourcePath, isTranslationPairSource, markdown: sourceCurrent }, + ), + translationStructureSignature( + parseTranslationMarkdown(result), + sourceBase, + { repoRoot: root, sourcePath: counterpartPath, isTranslationPairSource, markdown: result }, + ), ) if (errors.length > 0) { throw new Error(`gen-translation-brief: computed mechanical update for ${counterpartPath} violates the pair structure: ${errors.join('; ')}`) diff --git a/scripts/install-lefthook.mjs b/scripts/install-lefthook.mjs index 198f428b0a..3f8a4904ed 100644 --- a/scripts/install-lefthook.mjs +++ b/scripts/install-lefthook.mjs @@ -23,7 +23,7 @@ const OWNERSHIP_MARKER_VERSION = 1 const OWNERSHIP_MARKER_OWNER = 'deepseek-harness worktree-local lefthook hooks' const INSTALL_LOCK = 'dsh-lefthook-install.lock' const INSTALL_LOCK_TIMEOUT_MS = 30_000 -const INSTALL_LOCK_INITIALIZATION_TIMEOUT_MS = 1_000 +const INSTALL_LOCK_INITIALIZATION_TIMEOUT_MS = 5_000 const INSTALL_LOCK_POLL_MS = 50 const ALLOW_HOOKS_PATH_OVERRIDE = 'DSH_LEFTHOOK_ALLOW_HOOKS_PATH_OVERRIDE' const REPOSITORY_EXTENSION_PATTERN = '^extensions\\.' diff --git a/scripts/install-lefthook.spec.ts b/scripts/install-lefthook.spec.ts index c5a92a675e..f0c76ead6f 100644 --- a/scripts/install-lefthook.spec.ts +++ b/scripts/install-lefthook.spec.ts @@ -23,9 +23,9 @@ const pairingMergeDriver = 'scripts/merge-translation-pairing-driver.sh %O %A %B const scriptsDirectory = fileURLToPath(new URL('.', import.meta.url)) const tsxPackageDirectory = dirname(fileURLToPath(import.meta.resolve('tsx/package.json'))) const fixtures: string[] = [] -// Multi-worktree cases spawn several Git and Node subprocesses; coverage concurrency can -// legitimately exceed Vitest's default deadline without changing the installer behavior. -const MULTI_PROCESS_TEST_TIMEOUT_MS = 20_000 +// Multi-worktree cases spawn several Git and Node subprocesses; native Windows +// coverage concurrency can delay them without changing installer behavior. +const MULTI_PROCESS_TEST_TIMEOUT_MS = 30_000 interface Fixture { container: string @@ -184,7 +184,7 @@ function installLockPath(fixture: Fixture): string { } async function waitForPath(path: string): Promise { - const deadline = Date.now() + 5_000 + const deadline = Date.now() + 10_000 while (!existsSync(path)) { if (Date.now() >= deadline) throw new Error(`timed out waiting for ${path}`) await new Promise(resolveWait => setTimeout(resolveWait, 10)) @@ -211,7 +211,7 @@ function runInstaller( }) } -describe('worktree-local Lefthook installer', { timeout: 15_000 }, () => { +describe('worktree-local Lefthook installer', { timeout: 30_000 }, () => { for (const [label, extraEnv] of [ ['CI', { CI: 'true' }], ['GitHub Actions', { GITHUB_ACTIONS: 'true' }], @@ -529,7 +529,12 @@ describe('worktree-local Lefthook installer', { timeout: 15_000 }, () => { const lockPath = installLockPath(fixture) const runningPath = join(hooksPath(fixture, fixture.main), '.fake-lefthook-running') const install = runInstaller(fixture, fixture.main, { DSH_TEST_LEFTHOOK_DELAY_MS: '250' }) - await waitForPath(runningPath) + try { + await waitForPath(runningPath) + } catch (error) { + await install + throw error + } const replacementRecord = 'replacement owner\n' writeFileSync(lockPath, replacementRecord) diff --git a/scripts/lint-rule-fingerprint.spec.ts b/scripts/lint-rule-fingerprint.spec.ts index 0db617ba59..4aecd9e985 100644 --- a/scripts/lint-rule-fingerprint.spec.ts +++ b/scripts/lint-rule-fingerprint.spec.ts @@ -18,19 +18,19 @@ interface Profile { // snapshot; they do not re-evaluate that deleted baseline or track its preset. const profiles = { source: { - count: 88, + count: 89, indexes: [0, 1, 4, 5], - sha256: 'da1dfd77cb6eb66be93d8d3820f9b9b68b7aa391c24680f8851c0910298f9e3b', + sha256: '3e8b8455827254d6da59afb803d30f062b9fe1264b4df75ce4c8cd6e742b93dd', }, example: { - count: 87, + count: 88, indexes: [0, 1, 2, 4, 5], - sha256: '6a2606053bc1ec1de3b02611de88ea51d201dac13a1f193e4934d33c08b95f08', + sha256: 'e2b944c028ef1971302d47b2eb0e95c05e740623c7cbaebb4b1312dbf73bf156', }, test: { - count: 83, + count: 84, indexes: [0, 3, 4, 5], - sha256: '7995e14926a36c40bd65c474637735222a95fb030395681685f03060e50a7b78', + sha256: '7800bf57f0d689a5623e4ad60315b10173158fb9cf988d3f005cb115f2245691', }, } as const satisfies Record @@ -85,7 +85,7 @@ describe('Oxlint repository rule fingerprint', () => { const overrides: readonly unknown[] = parsed.overrides it('pins every override field', () => { - expect(overrides).toHaveLength(8) + expect(overrides).toHaveLength(9) }) it.each(Object.entries(profiles))('pins the %s rule profile', (_name, profile) => { diff --git a/scripts/locale-dictionary-parity.spec.ts b/scripts/locale-dictionary-parity.spec.ts new file mode 100644 index 0000000000..b51630f105 --- /dev/null +++ b/scripts/locale-dictionary-parity.spec.ts @@ -0,0 +1,308 @@ +/** + * Gate for the invariant `FALLBACK_LOCALE` rests on: every shipped dictionary + * declares the same keys in `zh` and `en`. + * + * The locale runtime resolves a key through the active locale, then through + * the single fallback locale (`en`), then surfaces the key itself. With + * symmetric dictionaries that middle step always resolves, so one constant can + * serve as both the opening locale and the dictionary fallback. A key added to + * only one side breaks that: a reader of the other language sees a bare key + * such as `list.aria` instead of text. This gate fails on the asymmetry rather + * than waiting for the bare key to reach a UI. + * + * Discovery is deliberately broad, because a gate that silently narrows is + * worse than no gate. It sweeps every workspace package (not just + * `packages/client`), reads dictionaries wherever they are declared — + * `locales.ts`, a `locales/` directory, or inline in the plugin body — and + * pairs `zh`/`en` across sibling files as well as within one module. A `zh` + * dictionary whose `en` counterpart cannot be found anywhere is an error, not + * a skip. + */ + +import type { Dirent } from 'node:fs' +import { readdirSync, readFileSync } from 'node:fs' +import { dirname, resolve } from 'node:path' +import { fileURLToPath } from 'node:url' +import ts from 'typescript' +import { describe, expect, it } from 'vitest' + +const root = fileURLToPath(new URL('..', import.meta.url)) + +/** Repo-relative path with `/` separators, so messages and suffix tests match on every OS. */ +function relative(file: string): string { + return file.slice(root.length).replaceAll('\\', '/') +} + +/** Every `.ts` source file under each workspace package's `src`, excluding declarations. */ +function sourceFiles(): string[] { + const files: string[] = [] + const packagesRoot = resolve(root, 'packages') + for (const group of directories(packagesRoot)) { + for (const pkg of directories(resolve(packagesRoot, group))) { + walk(resolve(packagesRoot, group, pkg, 'src'), files) + } + } + return files.sort() +} + +/** Immediate subdirectory names, or none when the path is not a directory. */ +function directories(dir: string): string[] { + return readEntries(dir).filter(entry => entry.isDirectory()).map(entry => entry.name) +} + +/** + * Directory entries, treating only a genuinely absent directory as empty. + * Any other failure (`EACCES`, I/O) rethrows: silently reading it as "absent" + * would narrow the sweep and let the gate pass while checking less. + * @param dir - absolute directory path. + * @returns entries, or none when the directory does not exist. + */ +function readEntries(dir: string): Dirent[] { + try { + return readdirSync(dir, { withFileTypes: true }) + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return [] + throw error + } +} + +function walk(dir: string, out: string[]): void { + for (const entry of readEntries(dir)) { + const full = resolve(dir, entry.name) + if (entry.isDirectory()) walk(full, out) + else if (entry.name.endsWith('.ts') && !entry.name.endsWith('.d.ts')) out.push(full) + } +} + +/** One discovered dictionary: which file and export name declared it. */ +interface Dictionary { + /** Repo-relative declaring file. */ + file: string + /** Export name, or the registration site for an inline literal. */ + name: string + /** Declared keys, sorted. */ + keys: string[] +} + +/** + * Keys of every top-level `export const = { ... }` object literal whose + * name identifies a locale dictionary, plus inline `register(ns, locale, {...})` + * literals. Read from the AST so the gate never executes package code. + * @param file - absolute path of a candidate module. + * @returns discovered dictionaries, keyed by locale-bearing name. + */ +function dictionariesIn(file: string): Dictionary[] { + const text = readFileSync(file, 'utf8') + // Cheap pre-filter: parsing every package source is wasteful. The pattern + // must admit every shape `localeOf` accepts, or a file would be skipped + // before parsing — the silent narrowing this gate exists to prevent. A bare + // `\b(zh|en)\b` misses `zhSettings`/`accessZh`, because `\b` does not hold + // between `h` and an uppercase letter. + if (!/\b(zh|en)\b|\b(zh|en)[A-Z]|(Zh|En)\b/.test(text)) return [] + const source = ts.createSourceFile(file, text, ts.ScriptTarget.ESNext, true) + const found: Dictionary[] = [] + const rel = relative(file) + + // Module-scope variable declarations, keyed by name. A 3-arg + // `register(NS, 'zh'|'en', dict)` whose third argument is an identifier — + // e.g. a local dictionary variable rather than an inline literal — resolves + // through here so the gate still verifies its symmetry. + const moduleConsts = new Map() + for (const statement of source.statements) { + if (!ts.isVariableStatement(statement)) continue + for (const decl of statement.declarationList.declarations) { + if (ts.isIdentifier(decl.name) && decl.initializer !== undefined) { + moduleConsts.set(decl.name.text, decl.initializer) + } + } + } + + for (const statement of source.statements) { + if (!ts.isVariableStatement(statement)) continue + if (statement.modifiers?.some(m => m.kind === ts.SyntaxKind.ExportKeyword) !== true) continue + for (const decl of statement.declarationList.declarations) { + if (!ts.isIdentifier(decl.name)) continue + const literal = unwrap(decl.initializer) + if (literal === undefined || !ts.isObjectLiteralExpression(literal)) continue + if (localeOf(decl.name.text) === undefined) continue + found.push({ file: rel, name: decl.name.text, keys: keysOf(literal) }) + } + } + + // A 3-arg `register(ns, 'zh'|'en', dict)` call whose dictionary argument we + // cannot turn into an object literal. We refuse instead of skipping: a + // registration we cannot measure is exactly the silent narrowing this gate + // exists to catch. + const refuse = (ns: string, tag: string, why: string): never => { + throw new Error(`cannot verify register('${ns}', '${tag}', ...) in ${rel}: ${why}`) + } + + // Inline registrations, two shapes. A `[['zh', {...}], ['en', {...}]]` pair + // handed to a registration loop keys off the enclosing array; separate + // `register(NS, 'zh', {...})` / `register(NS, 'en', {...})` calls key off the + // namespace argument, so the two calls pair with each other. + const visit = (node: ts.Node): void => { + if (ts.isCallExpression(node)) { + const callee = node.expression + const name = ts.isPropertyAccessExpression(callee) + ? callee.name.text + : ts.isIdentifier(callee) && callee.text === 'register' ? 'register' : undefined + if (name === 'register' && node.arguments.length >= 3) { + const [ns, tag, dict] = node.arguments + if (ns === undefined || tag === undefined || !ts.isStringLiteral(tag)) return + if (tag.text !== 'zh' && tag.text !== 'en') return + const raw = unwrap(dict) + const literal = raw !== undefined && ts.isIdentifier(raw) + ? (() => { + const resolved = moduleConsts.get(raw.text) + return resolved === undefined ? undefined : unwrap(resolved) + })() + : raw + const why = raw !== undefined && ts.isIdentifier(raw) + ? `third argument ${raw.text} does not resolve to an inline or module-scope object literal` + : 'third argument is neither an object literal nor a resolvable dictionary variable' + if (literal === undefined || !ts.isObjectLiteralExpression(literal)) { + // The dictionary argument must resolve to an object literal; the + // gate refuses rather than skips, so the symmetry it verifies never + // silently narrows. + refuse(ns.getText(source), tag.text, why) + } + const dictionary: ts.ObjectLiteralExpression = literal as ts.ObjectLiteralExpression + // The namespace expression's source text identifies the pair, so the + // zh and en calls for one namespace meet and calls for different + // namespaces stay apart. + found.push({ file: rel, name: `${tag.text}@register:${ns.getText(source)}`, keys: keysOf(dictionary) }) + } + } + if (ts.isArrayLiteralExpression(node) && node.elements.length === 2) { + const site = source.getLineAndCharacterOfPosition(node.getStart(source)).line + 1 + for (const element of node.elements) { + if (!ts.isArrayLiteralExpression(element) || element.elements.length !== 2) continue + const [tag, dict] = element.elements + const literal = unwrap(dict) + if (tag === undefined || !ts.isStringLiteral(tag)) continue + if (literal === undefined || !ts.isObjectLiteralExpression(literal)) continue + if (tag.text !== 'zh' && tag.text !== 'en') continue + found.push({ file: rel, name: `${tag.text}@inline:${site}`, keys: keysOf(literal) }) + } + } + ts.forEachChild(node, visit) + } + visit(source) + return found +} + +/** Declared property names of an object literal, sorted. */ +function keysOf(literal: ts.ObjectLiteralExpression): string[] { + const keys: string[] = [] + for (const prop of literal.properties) { + if (!ts.isPropertyAssignment(prop)) continue + if (ts.isIdentifier(prop.name) || ts.isStringLiteral(prop.name)) keys.push(prop.name.text) + } + return keys.sort() +} + +/** Look through `satisfies`/`as`/parenthesized wrappers to the literal. */ +function unwrap(node: ts.Expression | undefined): ts.Expression | undefined { + let current = node + while ( + current !== undefined + && (ts.isSatisfiesExpression(current) || ts.isAsExpression(current) || ts.isParenthesizedExpression(current)) + ) { + current = current.expression + } + return current +} + +/** + * The locale a dictionary name declares, and the namespace-ish remainder that + * identifies which pair it belongs to. `zh`/`en`, `zhSettings`/`enSettings`, + * and `settingsZh`/`settingsEn` are the shapes this repo uses. A name-prefix + * shape requires an uppercase ASCII letter at the third position (`[A-Z]`), + * matching the admission of the cheap pre-filter, so `zh2Foo`/`zh_probe` + * cannot be treated as dictionaries in one place and skipped in another. + * @param name - export name or synthetic inline name. + * @returns locale plus pair key, or undefined when the name names no locale. + */ +function localeOf(name: string): { locale: 'zh' | 'en'; pair: string } | undefined { + for (const locale of ['zh', 'en'] as const) { + const other = locale === 'zh' ? 'Zh' : 'En' + if (name === locale) return { locale, pair: '' } + // Synthetic names for inline shapes carry their own pair key after the + // first ':' (the enclosing array's line, or the namespace expression). + if (name.startsWith(`${locale}@`)) return { locale, pair: name.slice(name.indexOf(':')) } + if (name.startsWith(locale) && name.length > 2 && /[A-Z]/.test(name[2] ?? '')) { + return { locale, pair: name.slice(2) } + } + if (name.endsWith(other)) return { locale, pair: name.slice(0, -2) } + } + return undefined +} + +describe('shipped locale dictionaries', () => { + it('declares the same keys in zh and en, so the single fallback locale always resolves', () => { + const files = sourceFiles() + // Guard the discovery itself: an empty or narrowed sweep would pass every + // assertion below while checking nothing. + expect(files.length).toBeGreaterThan(500) + + // Pair within a file first; a dictionary whose counterpart is not in the + // same module then pairs with a sibling in the same directory. Both shapes + // ship here: `locales/settings.ts` exports zh+en together, while + // `locales/zh.ts` + `locales/en.ts` split the common pair across files. + const perFile = new Map() + for (const file of files) { + const dicts = dictionariesIn(file) + if (dicts.length > 0) perFile.set(relative(file), dicts) + } + + const groups = new Map>() + const place = (key: string, locale: 'zh' | 'en', dict: Dictionary): void => { + const slot = groups.get(key) ?? new Map<'zh' | 'en', Dictionary>() + if (slot.has(locale)) { + throw new Error(`two ${locale} dictionaries claim pair ${key}: ${slot.get(locale)?.file} and ${dict.file}`) + } + slot.set(locale, dict) + groups.set(key, slot) + } + + for (const [rel, dicts] of perFile) { + for (const dict of dicts) { + const parsed = localeOf(dict.name) + if (parsed === undefined) continue + const sameFileCounterpart = dicts.some((other) => { + const otherParsed = localeOf(other.name) + return otherParsed !== undefined + && otherParsed.pair === parsed.pair + && otherParsed.locale !== parsed.locale + }) + // Same-file pairs key by file so two pairs in one directory stay + // distinct; split pairs key by directory so siblings meet. + const key = sameFileCounterpart ? `${rel}::${parsed.pair}` : `${dirname(rel)}::${parsed.pair}` + place(key, parsed.locale, dict) + } + } + + const problems: string[] = [] + let comparedPairs = 0 + for (const [key, slot] of [...groups].sort()) { + const zh = slot.get('zh') + const en = slot.get('en') + if (zh === undefined || en === undefined) { + const present = zh ?? en + problems.push(`${present?.file} declares ${present?.name} with no counterpart for pair ${key}`) + continue + } + comparedPairs++ + const zhOnly = zh.keys.filter(k => !en.keys.includes(k)) + const enOnly = en.keys.filter(k => !zh.keys.includes(k)) + if (zhOnly.length > 0) problems.push(`${zh.file} ${zh.name} has keys absent from ${en.name}: ${zhOnly.join(', ')}`) + if (enOnly.length > 0) problems.push(`${en.file} ${en.name} has keys absent from ${zh.name}: ${enOnly.join(', ')}`) + } + + // The shipped dictionary count only grows; a collapse means discovery or + // pairing broke, which would hide real asymmetry. + expect(comparedPairs).toBeGreaterThan(25) + expect(problems).toEqual([]) + }) +}) diff --git a/scripts/markdown.ts b/scripts/markdown.ts index 1d40e1d8bb..f221d1c342 100644 --- a/scripts/markdown.ts +++ b/scripts/markdown.ts @@ -56,6 +56,101 @@ export function visitMarkdown(node: Nodes, visitor: (node: Nodes) => boolean | v } } +/** Markdown nodes whose authored destination occupies a replaceable source range. */ +export type MarkdownDestinationNode = Extract + +/** One authored Markdown destination and its absolute source offsets. */ +export interface MarkdownDestination { + start: number + end: number + url: string +} + +/** Whether a Markdown URL is external, repository-root absolute, or purely in-page. */ +export function isExternalOrAbsoluteMarkdownUrl(url: string): boolean { + return url.startsWith('#') + || url.startsWith('//') + || url.startsWith('/') + || /^[a-zA-Z][a-zA-Z0-9+.-]*:/.test(url) +} + +/** Split one Markdown URL without normalizing its query or fragment suffix. */ +export function splitMarkdownUrlTarget(url: string): { path: string; suffix: string } { + const boundary = url.search(/[?#]/) + if (boundary === -1) return { path: url, suffix: '' } + return { path: url.slice(0, boundary), suffix: url.slice(boundary) } +} + +function skipWhitespace(source: string, start: number): number { + let index = start + while (/\s/.test(source[index] ?? '')) index += 1 + return index +} + +function labelEnd(source: string): number { + const first = source.indexOf('[') + if (first === -1) return -1 + let depth = 0 + for (let index = first; index < source.length; index += 1) { + const char = source[index] + if (char === '\\') index += 1 + else if (char === '[') depth += 1 + else if (char === ']') { + depth -= 1 + if (depth === 0) return index + } + } + return -1 +} + +function destinationRange(rawNode: string, type: MarkdownDestinationNode['type']): { start: number; end: number } { + const endOfLabel = labelEnd(rawNode) + if (endOfLabel === -1) throw new Error(`markdown: cannot locate label end in ${JSON.stringify(rawNode)}`) + let start: number + if (type === 'definition') { + const colon = rawNode.indexOf(':', endOfLabel + 1) + if (colon === -1) throw new Error(`markdown: cannot locate definition separator in ${JSON.stringify(rawNode)}`) + start = skipWhitespace(rawNode, colon + 1) + } else { + if (rawNode[endOfLabel + 1] !== '(') { + throw new Error(`markdown: cannot locate inline destination in ${JSON.stringify(rawNode)}`) + } + start = skipWhitespace(rawNode, endOfLabel + 2) + } + if (rawNode[start] === '<') { + for (let index = start + 1; index < rawNode.length; index += 1) { + if (rawNode[index] === '\\') index += 1 + else if (rawNode[index] === '>') return { start: start + 1, end: index } + } + throw new Error(`markdown: cannot locate angle-bracket destination end in ${JSON.stringify(rawNode)}`) + } + let depth = 0 + for (let index = start; index < rawNode.length; index += 1) { + const char = rawNode[index] + if (char === '\\') index += 1 + else if (char === '(') depth += 1 + else if (char === ')') { + if (depth === 0) return { start, end: index } + depth -= 1 + } else if (/\s/.test(char ?? '') && depth === 0) { + return { start, end: index } + } + } + return { start, end: rawNode.length } +} + +/** Locate one parsed destination in the original Markdown without reserializing it. */ +export function markdownDestination(source: string, node: MarkdownDestinationNode): MarkdownDestination { + const start = node.position?.start.offset + const end = node.position?.end.offset + if (start === undefined || end === undefined) { + throw new Error(`markdown: destination ${JSON.stringify(node.url)} has no source offsets`) + } + const range = destinationRange(source.slice(start, end), node.type) + const absolute = { start: start + range.start, end: start + range.end } + return { ...absolute, url: source.slice(absolute.start, absolute.end) } +} + /** * Extract every parsed code block with its info string, in document order. * @param source - Markdown source to scan. diff --git a/scripts/merge-translation-pairing.ts b/scripts/merge-translation-pairing.ts index dda2621a36..b14305bddb 100644 --- a/scripts/merge-translation-pairing.ts +++ b/scripts/merge-translation-pairing.ts @@ -4,6 +4,7 @@ import { execFileSync } from 'node:child_process' import { readFileSync, writeFileSync } from 'node:fs' import { mergeTranslationPairingRecords, + repositoryTranslationPairSource, resolveTranslationPairingConflicts, } from './translation-pairing-merge.ts' @@ -16,7 +17,7 @@ try { const root = execFileSync('git', ['rev-parse', '--show-toplevel'], { encoding: 'utf8' }).trim() if (args[0] === '--resolve') { if (args.length !== 1) throw new Error('--resolve takes no paths; it inspects the unmerged index') - const resolved = resolveTranslationPairingConflicts(root) + const resolved = resolveTranslationPairingConflicts(root, repositoryTranslationPairSource(root)) if (resolved.length === 0) { console.log('merge-translation-pairing: no unresolved pairing records') } else { @@ -36,6 +37,7 @@ try { readFileSync(ancestorPath, 'utf8'), readFileSync(currentPath, 'utf8'), readFileSync(otherPath, 'utf8'), + repositoryTranslationPairSource(root), ) writeFileSync(currentPath, result.record) } diff --git a/scripts/migrate-packed-session-fixtures.ts b/scripts/migrate-packed-session-fixtures.ts index f934d91cd2..7c2866f51d 100644 --- a/scripts/migrate-packed-session-fixtures.ts +++ b/scripts/migrate-packed-session-fixtures.ts @@ -1,6 +1,6 @@ #!/usr/bin/env node /** - * Temporary branch-convergence command for canonical packed session fixtures. + * Temporary branch-convergence command for canonical projected session fixtures. * * @see ../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md */ @@ -18,4 +18,4 @@ for (const fixture of changed) { writeFileSync(resolve(root, fixture.path), fixture.canonical) console.log(fixture.path) } -console.log(`packed session fixtures: ${changed.length} rewritten, ${fixtures.length} inspected`) +console.log(`session snapshot fixtures: ${changed.length} rewritten, ${fixtures.length} inspected`) diff --git a/scripts/oxlint-contract.spec.ts b/scripts/oxlint-contract.spec.ts index def26f0a78..c1f39f8cc8 100644 --- a/scripts/oxlint-contract.spec.ts +++ b/scripts/oxlint-contract.spec.ts @@ -1,8 +1,8 @@ import { spawnSync } from 'node:child_process' import { randomUUID } from 'node:crypto' import { existsSync } from 'node:fs' -import { mkdir, readFile, rm, writeFile } from 'node:fs/promises' -import { join, relative } from 'node:path' +import { mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises' +import { basename, dirname, join, relative } from 'node:path' import { fileURLToPath } from 'node:url' import { flattenDiagnosticMessageText, parseConfigFileTextToJson } from 'typescript' import { describe, expect, it } from 'vitest' @@ -39,6 +39,26 @@ function normalizedOutput(result: ReturnType): string { return `${result.stdout}${result.stderr}`.replaceAll('\\', '/') } +/** @returns A transient filename excluded from concurrent repository-wide glob discovery. */ +function hiddenProbeName(prefix: string, suffix: string, extension = '.ts'): string { + return `.${prefix}-${suffix}${extension}` +} + +/** + * Publish a complete probe so concurrent repository scans never read a partial write. + * @param path - Final probe path that the owning project must discover. + * @param source - Complete TypeScript source to publish. + */ +async function publishProbe(path: string, source: string): Promise { + const staging = join(dirname(path), `.${basename(path)}.staging`) + try { + await writeFile(staging, source) + await rename(staging, path) + } finally { + await rm(staging, { force: true }) + } +} + async function writeContractConfig(suffix: string): Promise { const path = join(repositoryRoot, `.oxlintrc.contract-${suffix}.json`) await writeFile(path, JSON.stringify({ extends: ['./.oxlintrc.json'], ignorePatterns: [] })) @@ -59,7 +79,8 @@ describe('Oxlint executable contract', () => { ['example', 'examples/headless-agent/tests', 'tsconfig.host.json'], ['website', 'website', 'tsconfig.host.json'], ] as const - const source = `export function probePromise(): Promise { + const source = `/** Produce a settled promise for type-aware linting. */ +export function probePromise(): Promise { return Promise.resolve() } @@ -70,7 +91,7 @@ probePromise() const paths: Array = [] for (const [label, parent, tsconfig, extension = '.ts'] of probes) { const path = join(repositoryRoot, parent, `oxlint-contract-${suffix}${extension}`) - await writeFile(path, source) + await publishProbe(path, source) paths.push([label, relative(repositoryRoot, path), tsconfig]) } const clientScript = 'scripts/client-bundle-purity.spec.ts' @@ -88,7 +109,7 @@ probePromise() expect(result.error).toBeUndefined() expect(result.status, output).toBe(1) for (const [label, path, tsconfig] of paths) { - expect(output, label).toContain(`${path.replaceAll('\\', '/')}:5:1: Promises must be awaited`) + expect(output, label).toContain(`${path.replaceAll('\\', '/')}:6:1: Promises must be awaited`) expect(output, `${label} project`).toContain( `Got tsconfig for file ${join(repositoryRoot, path).replaceAll('\\', '/')}: ${join(repositoryRoot, tsconfig).replaceAll('\\', '/')}`, ) @@ -110,7 +131,7 @@ probePromise() it('runs JavaScript compatibility and nursery rules', async () => { const suffix = randomUUID() const configPath = await writeContractConfig(suffix) - const path = join(repositoryRoot, 'scripts', `oxlint-contract-${suffix}.ts`) + const path = join(repositoryRoot, 'scripts', hiddenProbeName('oxlint-contract', suffix)) const source = `export function firstProbe(): number { const first = 1 const second = 2 @@ -230,7 +251,7 @@ export const longProbe = 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + it('reports an unused suppression', async () => { const suffix = randomUUID() const configPath = await writeContractConfig(suffix) - const path = join(repositoryRoot, 'scripts', `oxlint-contract-${suffix}.ts`) + const path = join(repositoryRoot, 'scripts', hiddenProbeName('oxlint-contract', suffix)) try { await writeFile(path, '// oxlint-disable-next-line no-console\nexport const value = 1\n') @@ -280,7 +301,7 @@ export const longProbe = 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + expect(stagedConfig.ignorePatterns).not.toContain('packages/typert/generator/tests/fixtures/type-model/**') const suffix = randomUUID() - const path = join(repositoryRoot, 'scripts', `staged-lint-probe-${suffix}.ts`) + const path = join(repositoryRoot, 'scripts', hiddenProbeName('staged-lint-probe', suffix)) try { await writeFile(path, 'export const value={answer:1};\n') const lint = runOxlint([ @@ -303,7 +324,7 @@ export const longProbe = 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + it('preserves successful fix output channels', async () => { const suffix = randomUUID() - const path = join(repositoryRoot, 'scripts', `staged-lint-probe-${suffix}.ts`) + const path = join(repositoryRoot, 'scripts', hiddenProbeName('staged-lint-probe', suffix)) try { await writeFile(path, '// oxlint-disable-next-line no-console\nexport const value = 1\n') @@ -327,7 +348,7 @@ export const longProbe = 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + it('prints only the final diagnostics when a fix retry still fails', async () => { const suffix = randomUUID() - const path = join(repositoryRoot, 'scripts', `staged-lint-probe-${suffix}.ts`) + const path = join(repositoryRoot, 'scripts', hiddenProbeName('staged-lint-probe', suffix)) try { await writeFile(path, `export const longProbe = ${'1 + '.repeat(80)}1\n`) diff --git a/scripts/pnpm-invocation.spec.ts b/scripts/pnpm-invocation.spec.ts new file mode 100644 index 0000000000..575e21491b --- /dev/null +++ b/scripts/pnpm-invocation.spec.ts @@ -0,0 +1,33 @@ +import { describe, expect, it } from 'vitest' +import { pnpmInvocation } from './pnpm-invocation.ts' + +describe('pnpm invocation', () => { + it.each([ + '/tools/pnpm.js', + '/tools/pnpm.cjs', + '/tools/pnpm.mjs', + '/tools/PNPM.CJS', + '/tools/with spaces/工具/$pnpm;.mjs', + ])('runs the JavaScript entrypoint %j through Node', (entrypoint) => { + expect(pnpmInvocation(['run', 'build'], { npm_execpath: entrypoint })).toEqual({ + command: process.execPath, + args: [entrypoint, 'run', 'build'], + }) + }) + + it.each([ + '/tools/pnpm', + '/tools/with spaces/$pnpm;', + String.raw`C:\Program Files\工具\$pnpm;\pnpm.exe`, + ])('runs the executable entrypoint %j directly', (entrypoint) => { + expect(pnpmInvocation(['run', 'build'], { npm_execpath: entrypoint })).toEqual({ + command: entrypoint, + args: ['run', 'build'], + }) + }) + + it.each([undefined, ''])('rejects an unavailable lifecycle entrypoint', (entrypoint) => { + expect(() => pnpmInvocation([], { npm_execpath: entrypoint })) + .toThrow('npm_execpath is unavailable; invoke the script through pnpm run') + }) +}) diff --git a/scripts/pnpm-invocation.ts b/scripts/pnpm-invocation.ts new file mode 100644 index 0000000000..8ca5a2df91 --- /dev/null +++ b/scripts/pnpm-invocation.ts @@ -0,0 +1,21 @@ +/** Resolve shell-free child-process invocations for the pnpm process that launched a package script. */ + +/** + * Resolve pnpm's executable and arguments from its lifecycle environment. + * @param args - Arguments to pass to pnpm. + * @param environment - Lifecycle environment containing `npm_execpath`. + * @returns A command and argument array suitable for `spawn` or `spawnSync` without a shell. + */ +export function pnpmInvocation( + args: readonly string[], + environment: NodeJS.ProcessEnv = process.env, +): { command: string; args: string[] } { + const entrypoint = environment.npm_execpath + if (entrypoint === undefined || entrypoint === '') { + throw new Error('pnpm invocation: npm_execpath is unavailable; invoke the script through pnpm run.') + } + if (/\.[cm]?js$/iu.test(entrypoint)) { + return { command: process.execPath, args: [entrypoint, ...args] } + } + return { command: entrypoint, args: [...args] } +} diff --git a/scripts/prepare-ci-bubblewrap.sh b/scripts/prepare-ci-bubblewrap.sh index e5f0902750..1efaee200a 100755 --- a/scripts/prepare-ci-bubblewrap.sh +++ b/scripts/prepare-ci-bubblewrap.sh @@ -28,5 +28,5 @@ printf '%s\n' "$root/usr/bin" >> "$GITHUB_PATH" sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 \ || echo 'apparmor userns knob absent — the functional probe decides' "$root/usr/bin/bwrap" --version -"$root/usr/bin/bwrap" --ro-bind / / --dev /dev --proc /proc --die-with-parent -- true +"$root/usr/bin/bwrap" --ro-bind / / --dev /dev --unshare-pid --proc /proc --die-with-parent -- true echo 'bubblewrap functional probe passed' diff --git a/scripts/project-doc-site.spec.ts b/scripts/project-doc-site.spec.ts index b5f5218de7..05ff7356f1 100644 --- a/scripts/project-doc-site.spec.ts +++ b/scripts/project-doc-site.spec.ts @@ -3,11 +3,16 @@ import { execFileSync } from 'node:child_process' import { existsSync, globSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' -import { basename, join, resolve } from 'node:path' -import { afterEach, describe, expect, it } from 'vitest' +import { basename, dirname, join, resolve } from 'node:path' +import { fromMarkdown } from 'mdast-util-from-markdown' +import { gfmFromMarkdown } from 'mdast-util-gfm' +import { gfm } from 'micromark-extension-gfm' +import type { Nodes } from 'mdast' +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest' import { docsPages, landingLink, routeLink, sectionSpec, type DocsPage } from '../website/docs.ts' import { - addProjectionFrontmatter, projectedPageContent, publishableImage, rewriteMarkdown, + addProjectionFrontmatter, emitRawMarkdownPages, llmsTxt, projectedPageContent, publishableImage, + rawMarkdownFiles, rawMarkdownPageContent, rawMarkdownRoute, resolveRepositoryRef, rewriteMarkdown, } from './project-doc-site.ts' const roots: string[] = [] @@ -91,6 +96,16 @@ describe('publishableImage', () => { }) }) +describe('resolveRepositoryRef', () => { + it('defaults to public master instead of a private workflow SHA', () => { + expect(resolveRepositoryRef({ GITHUB_SHA: 'private-sha' })).toBe('master') + }) + + it('accepts an explicit public repository ref', () => { + expect(resolveRepositoryRef({ DOCS_REPOSITORY_REF: 'public-sha' })).toBe('public-sha') + }) +}) + describe('rewriteMarkdown', () => { it('maps published pages and pins unpublished source links', () => { const { root, pages } = fixture() @@ -213,10 +228,15 @@ describe('rewriteMarkdown', () => { ) }) - it('routes a pair switcher across locales while ordinary links stay in locale', () => { + it('routes switchers across locales and explicit locale siblings within their locale', () => { const { root, pages } = fixture() writeFileSync(join(root, 'docs/a.zh.md'), '# A\n') - const paired = pages.filter(page => page.source !== 'docs/a.md') + writeFileSync(join(root, 'docs/b.zh.md'), '# B\n') + const paired = pages.filter(page => page.source !== 'docs/a.md').map(page => ( + page.locale === 'root' && page.source === 'docs/b.md' + ? { ...page, source: 'docs/b.zh.md', sourceAliases: ['docs/b.md'] } + : page + )) paired.push( { locale: 'root', contentLocale: 'zh-CN', source: 'docs/a.zh.md', sourceAliases: ['docs/a.md'], @@ -227,7 +247,7 @@ describe('rewriteMarkdown', () => { route: 'en/guide/a.md', label: 'A', sidebar: 'en-guide', section: 'Test', order: 1, }, ) - expect(rewriteMarkdown('[English](a.md) [B](b.md)\n', { + expect(rewriteMarkdown('[English](a.md) [B](b.zh.md)\n', { locale: 'root', sourcePath: 'docs/a.zh.md', route: 'guide/a.md', @@ -235,6 +255,14 @@ describe('rewriteMarkdown', () => { repoRoot: root, repositoryRef: 'abc123', })).toBe('[English](../en/guide/a.md) [B](../reference-root/b.md)\n') + expect(rewriteMarkdown('[中文](a.zh.md) [B](b.md)\n', { + locale: 'en', + sourcePath: 'docs/a.md', + route: 'en/guide/a.md', + pages: paired, + repoRoot: root, + repositoryRef: 'abc123', + })).toBe('[中文](../../guide/a.md) [B](../reference/b.md)\n') }) it('fails loud when a relative target is missing', () => { @@ -287,6 +315,37 @@ describe('docsPages locale routes', () => { } }) + it('projects the audited tutorial entry links from explicit locale index pages', () => { + const entries = [ + ['docs/user/develop/basic/config.md', '../framework/index.md'], + ['docs/user/develop/basic/publish.md', '../framework/index.md'], + ['docs/user/develop/basic/tool.md', './index.md'], + ['docs/user/develop/basic/tool.md', '../practice/index.md'], + ['docs/user/develop/framework/events.md', '../practice/index.md'], + ['docs/user/develop/framework/service.md', '../practice/index.md'], + ['docs/user/develop/practice/index.md', '../basic/index.md'], + ['docs/user/guide/index.md', '../develop/basic/index.md'], + ] as const + + for (const [englishSource, englishTarget] of entries) { + for (const locale of ['en', 'root'] as const) { + const source = locale === 'root' ? englishSource.replace(/\.md$/, '.zh.md') : englishSource + const target = locale === 'root' ? englishTarget.replace(/\.md$/, '.zh.md') : englishTarget + const page = docsPages.find(candidate => candidate.locale === locale && candidate.source === source) + expect(page, `${locale}:${source}`).toBeDefined() + expect(readFileSync(resolve(repositoryRoot, source), 'utf8')).toContain(`](${target})`) + expect(rewriteMarkdown(`[Entry](${target})\n`, { + locale, + sourcePath: source, + route: page!.route, + pages: docsPages, + repoRoot: repositoryRoot, + repositoryRef: 'abc123', + })).toBe(`[Entry](${englishTarget})\n`) + } + } + }) + it('indexes every subsystem page in both sides of the folder README', () => { const pages = globSync(join(repositoryRoot, 'docs/subsystems/*.md')) .map(page => basename(page)) @@ -295,11 +354,22 @@ describe('docsPages locale routes', () => { expect(pages.length).toBeGreaterThan(0) for (const readme of ['README.md', 'README.zh.md']) { const rows = readFileSync(join(repositoryRoot, 'docs/subsystems', readme), 'utf8') - const missing = pages.filter(page => !rows.includes(`| [${page}](${page}) |`)) + const missing = pages.filter((page) => { + const target = readme.endsWith('.zh.md') ? page.replace(/\.md$/, '.zh.md') : page + return !rows.includes(`| [${page}](${target}) |`) + }) expect(missing, `${readme} must carry one table row per subsystem page`).toEqual([]) } }) + it('places the shared todo fragment alias on the translated todo section', () => { + const catalog = readFileSync(resolve(repositoryRoot, 'docs/tool-catalog.zh.md'), 'utf8') + expect(catalog.match(/<\/a>/g)).toHaveLength(1) + expect(catalog).toContain( + '\n\n## `@deepseek-ai/dsh-tool-todo`', + ) + }) + it('projects every published subsystem page in Chinese', () => { const rootPages = docsPages.filter(page => ( page.locale === 'root' && page.route.startsWith('reference/subsystems/') @@ -492,3 +562,207 @@ describe('projectedPageContent', () => { .toThrow('locale home source "docs/index.zh.md" must start with YAML frontmatter') }) }) + +describe('rawMarkdownPageContent', () => { + it('keeps the home body the rendered site omits and drops the VitePress frontmatter', () => { + expect(rawMarkdownPageContent( + '---\nlayout: false\nhead:\n - - meta\n - http-equiv: refresh\n content: 0; url=./guide/quickstart\n---\n\n# Harness\n\nEnglish | [中文](./index.md)\n\nBody.\n', + 'docs/user/index.zh.md', + )).toBe('# Harness\n\nBody.\n') + }) + + it('drops the language switcher and repository badge like the rendered site', () => { + const badge = '[![](https://img.shields.io/badge/powered_by-dsh-4D6BFE?style=flat-square)](https://github.com/deepseek-ai/deepseek-harness)' + expect(rawMarkdownPageContent(`# Guide\n\nEnglish | [中文](./x)\n\nBody.\n\n${badge}\n`, 'docs/guide.md')) + .toBe('# Guide\n\nBody.\n') + }) + + it('rejects unclosed frontmatter and names the page', () => { + // The twin pass is the first place an ordinary page's frontmatter is + // parsed, so an anonymous error would leave 168 routes to search. + expect(() => rawMarkdownPageContent('---\nlayout: false\n', 'docs/broken.md')) + .toThrow('project-doc-site: "docs/broken.md" has unclosed YAML frontmatter') + }) +}) + +describe('emitRawMarkdownPages', () => { + function mirrorDir(): string { + const out = mkdtempSync(join(tmpdir(), 'dsh-doc-mirror-')) + roots.push(out) + return out + } + + it('writes every route with rewritten links, placed images, and no projection frontmatter', () => { + const { root, pages } = fixture() + writeFileSync(join(root, 'docs/a.md'), '[B](b.md) ![logo](../packages/logo.svg)\n') + const out = mirrorDir() + + // The real path, because image placement proves containment via realpath. + emitRawMarkdownPages(out, { pages, repoRoot: realpathSync(root), repositoryRef: 'abc123' }) + + expect(readFileSync(join(out, 'a.md'), 'utf8')).toBe('[B](./reference-root/b.md) ![logo](./logo.svg)\n') + expect(readFileSync(join(out, 'en/a.md'), 'utf8')).toBe('[B](./reference/b.md) ![logo](./logo.svg)\n') + expect(readFileSync(join(out, 'reference-root/b.md'), 'utf8')).toBe('# B\n') + expect(existsSync(join(out, 'logo.svg'))).toBe(true) + expect(existsSync(join(out, 'en/logo.svg'))).toBe(true) + }) + + it('emits the full body of a locale home page', () => { + const { root, pages } = fixture() + writeFileSync(join(root, 'docs/home.md'), '---\nlayout: false\n---\n\n# Home\n\n[A](a.md)\n') + pages.push({ + locale: 'root', contentLocale: 'zh-CN', source: 'docs/home.md', route: 'index.md', + label: 'Home', sidebar: null, section: 'Home', order: 0, + }) + const out = mirrorDir() + + emitRawMarkdownPages(out, { pages, repoRoot: root, repositoryRef: 'abc123' }) + + expect(readFileSync(join(out, 'index.md'), 'utf8')).toBe('# Home\n\n[A](./a.md)\n') + }) + + it('emits a parent-level alias for an index route with links recomputed', () => { + // A copied alias would carry the index page's relative links one directory + // too high, so the alias is its own projection over the alias route. + const { root, pages } = fixture() + writeFileSync(join(root, 'docs/c.md'), '# C\n\n[A](a.md)\n') + pages.push({ + locale: 'root', contentLocale: 'en-US', source: 'docs/c.md', route: 'guide/index.md', + label: 'C', sidebar: 'zh-guide', section: 'Test', order: 3, + }) + const out = mirrorDir() + + emitRawMarkdownPages(out, { pages, repoRoot: root, repositoryRef: 'abc123' }) + + expect(readFileSync(join(out, 'guide/index.md'), 'utf8')).toBe('# C\n\n[A](../a.md)\n') + expect(readFileSync(join(out, 'guide.md'), 'utf8')).toBe('# C\n\n[A](./a.md)\n') + }) + + it('refuses to overwrite a file the build already carries', () => { + // The twin pass writes into a populated build directory, and VitePress has + // already copied `website/public/` there; a page image sharing one of + // those names must fail loud instead of silently replacing the site file. + const { root, pages } = fixture() + writeFileSync(join(root, 'docs/a.md'), '![logo](../packages/logo.svg)\n') + const out = mirrorDir() + writeFileSync(join(out, 'logo.svg'), 'public copy\n') + + expect(() => { + emitRawMarkdownPages(out, { pages, repoRoot: realpathSync(root), repositoryRef: 'abc123' }) + }).toThrow('would overwrite') + expect(readFileSync(join(out, 'logo.svg'), 'utf8')).toBe('public copy\n') + }) +}) + +describe('rawMarkdownFiles', () => { + it('lists every route plus a parent alias per index route', () => { + const files = rawMarkdownFiles() + for (const page of docsPages) expect(files).toContain(page.route) + expect(files).toContain('reference.md') + expect(files).toContain('en/reference.md') + expect(files).toContain('en.md') + // The root home has no parent to alias into; `/` is documented as `/index.md`. + expect(files).not.toContain('.md') + expect(new Set(files).size).toBe(files.length) + }) +}) + +describe('raw Markdown projection of the published manifest', () => { + let mirror: string + + // Coverage instrumentation on a loaded CI runner stretches the full-manifest + // emission and the 181-file link walk past vitest's 5s default. + beforeAll(() => { + mirror = mkdtempSync(join(tmpdir(), 'dsh-doc-mirror-real-')) + emitRawMarkdownPages(mirror, { pages: docsPages, repoRoot: repositoryRoot, repositoryRef: 'master' }) + }, 60_000) + + afterAll(() => { + rmSync(mirror, { recursive: true, force: true }) + }) + + it('emits every published route and every index alias', () => { + for (const file of rawMarkdownFiles()) { + expect(existsSync(join(mirror, file)), file).toBe(true) + } + }) + + it('emits home pages with their bodies instead of the frontmatter stub', () => { + for (const route of ['index.md', 'en/index.md']) { + const home = readFileSync(join(mirror, route), 'utf8') + expect(home.startsWith('---'), route).toBe(false) + expect(home, route).toContain('# DeepSeek Harness') + } + }) + + it('resolves every relative link inside the emitted tree', { timeout: 60_000 }, () => { + // Raw pages are read outside the site, so a relative target that only the + // rendered site serves would strand every agent following it. + const broken: string[] = [] + for (const file of globSync('**/*.md', { cwd: mirror }).sort()) { + for (const target of relativeTargets(readFileSync(join(mirror, file), 'utf8'))) { + if (!existsSync(resolve(mirror, dirname(file), target))) broken.push(`${file}: ${target}`) + } + } + expect(broken).toEqual([]) + }) +}) + +function relativeTargets(markdown: string): string[] { + const tree = fromMarkdown(markdown, { extensions: [gfm()], mdastExtensions: [gfmFromMarkdown()] }) + const targets: string[] = [] + const visit = (node: Nodes): void => { + if ((node.type === 'link' || node.type === 'image' || node.type === 'definition') && 'url' in node) { + const external = node.url.startsWith('#') + || node.url.startsWith('/') + || /^[a-zA-Z][a-zA-Z0-9+.-]*:/.test(node.url) + const path = node.url.split(/[?#]/)[0] ?? '' + if (!external && path !== '') targets.push(decodeURIComponent(path)) + } + if ('children' in node) { + for (const child of node.children) visit(child) + } + } + visit(tree) + return targets +} + +describe('llmsTxt', () => { + const site = { base: '/x/', title: 'DeepSeek Harness', description: '插件化 SDK' } + + it('lists every sidebar page as a base-prefixed raw-Markdown link', () => { + const text = llmsTxt(site) + for (const page of docsPages) { + if (page.sidebar === null) expect(text, page.route).not.toContain(`](/x/${page.route})`) + else expect(text, page.route).toContain(`- [${page.label}](/x/${page.route}): ${page.section}`) + } + }) + + it('groups the two locale trees under their own headings', () => { + const text = llmsTxt(site) + expect(text.indexOf('## 简体中文')).toBeGreaterThan(-1) + expect(text.indexOf('## English')).toBeGreaterThan(text.indexOf('## 简体中文')) + }) + + it('carries the site identity and the raw-Markdown convention', () => { + const text = llmsTxt(site) + expect(text.startsWith('# DeepSeek Harness\n')).toBe(true) + expect(text).toContain('> 插件化 SDK') + expect(text).toMatch(/`\.md`/) + }) +}) + +describe('rawMarkdownRoute', () => { + it('projects one published route on demand', () => { + const { root, pages } = fixture() + writeFileSync(join(root, 'docs/a.md'), '# A\n\n[B](b.md)\n') + + expect(rawMarkdownRoute('en/a.md', { pages, repoRoot: root, repositoryRef: 'abc123' })) + .toBe('# A\n\n[B](./reference/b.md)\n') + }) + + it('returns undefined for a path the manifest does not publish', () => { + const { root, pages } = fixture() + expect(rawMarkdownRoute('en/missing.md', { pages, repoRoot: root, repositoryRef: 'abc123' })).toBeUndefined() + }) +}) diff --git a/scripts/project-doc-site.ts b/scripts/project-doc-site.ts index 1d0ea9072a..1efac68ec5 100644 --- a/scripts/project-doc-site.ts +++ b/scripts/project-doc-site.ts @@ -3,6 +3,9 @@ * * The generated tree is disposable: sources stay in their owning `docs/` * tier, while this adapter rewrites cross-source links for the public site. + * The same projection also emits a raw-Markdown twin of every route into the + * build output, so a page's URL, minus any trailing slash, plus `.md` serves + * it as plain Markdown. */ import { @@ -13,23 +16,33 @@ import { fromMarkdown } from 'mdast-util-from-markdown' import { gfmFromMarkdown } from 'mdast-util-gfm' import { gfm } from 'micromark-extension-gfm' import type { Nodes } from 'mdast' -import { docsPages, type DocsLocale, type DocsPage } from '../website/docs.ts' +import { docsPages, localeCollections, orderedPages, type DocsLocale, type DocsPage } from '../website/docs.ts' +import { + isExternalOrAbsoluteMarkdownUrl, + markdownDestination, + splitMarkdownUrlTarget, +} from './markdown.ts' const REPOSITORY_URL = 'https://github.com/deepseek-ai/deepseek-harness' const root = resolve(import.meta.dirname, '..') const generatedRoot = resolve(root, 'website/.generated') +/** + * Resolve the public repository ref used by projected source links. + * + * @param environment Build environment containing an optional explicit public ref. + * @returns The configured public ref, or `master`. + */ +export function resolveRepositoryRef(environment: NodeJS.ProcessEnv): string { + return environment.DOCS_REPOSITORY_REF ?? 'master' +} + interface Replacement { start: number end: number value: string } -interface DestinationRange { - start: number - end: number -} - type RewritableNode = Extract /** Inputs for rewriting one canonical Markdown page. */ @@ -55,94 +68,12 @@ function repoPath(absPath: string, repoRoot: string): string { return relative(repoRoot, absPath).split(sep).join('/') } -function isExternalOrSiteAbsolute(url: string): boolean { - return url.startsWith('#') - || url.startsWith('//') - || url.startsWith('/') - || /^[a-zA-Z][a-zA-Z0-9+.-]*:/.test(url) -} - -function skipWhitespace(source: string, start: number): number { - let index = start - while (/\s/.test(source[index] ?? '')) index += 1 - return index -} - -function labelEnd(source: string): number { - const first = source.indexOf('[') - if (first === -1) return -1 - let depth = 0 - for (let index = first; index < source.length; index += 1) { - const char = source[index] - if (char === '\\') { - index += 1 - } else if (char === '[') { - depth += 1 - } else if (char === ']') { - depth -= 1 - if (depth === 0) return index - } - } - return -1 -} - -function destinationRange(rawNode: string, type: 'link' | 'image' | 'definition'): DestinationRange { - const endOfLabel = labelEnd(rawNode) - if (endOfLabel === -1) { - throw new Error(`project-doc-site: cannot locate label end in ${JSON.stringify(rawNode)}.`) - } - - let start: number - if (type === 'definition') { - const colon = rawNode.indexOf(':', endOfLabel + 1) - if (colon === -1) { - throw new Error(`project-doc-site: cannot locate definition separator in ${JSON.stringify(rawNode)}.`) - } - start = skipWhitespace(rawNode, colon + 1) - } else { - if (rawNode[endOfLabel + 1] !== '(') { - throw new Error(`project-doc-site: cannot locate inline destination in ${JSON.stringify(rawNode)}.`) - } - start = skipWhitespace(rawNode, endOfLabel + 2) - } - - if (rawNode[start] === '<') { - for (let index = start + 1; index < rawNode.length; index += 1) { - if (rawNode[index] === '\\') index += 1 - else if (rawNode[index] === '>') return { start: start + 1, end: index } - } - throw new Error(`project-doc-site: cannot locate angle-bracket destination end in ${JSON.stringify(rawNode)}.`) - } - - let depth = 0 - for (let index = start; index < rawNode.length; index += 1) { - const char = rawNode[index] - if (char === '\\') { - index += 1 - } else if (char === '(') { - depth += 1 - } else if (char === ')') { - if (depth === 0) return { start, end: index } - depth -= 1 - } else if (/\s/.test(char ?? '') && depth === 0) { - return { start, end: index } - } - } - return { start, end: rawNode.length } -} - // `#fragment` suffixes pass through verbatim. Generated cordis-surface // headings carry explicit `` anchors with the GitHub slug, so those // fragments resolve on the published site too; hand-written headings rely on // VitePress's own slugger, which differs from GitHub's for punctuation-heavy // text — hand-authored cross-page fragments should prefer plain-text headings // or explicit anchors. -function splitTarget(url: string): { path: string; suffix: string } { - const boundary = url.search(/[?#]/) - if (boundary === -1) return { path: url, suffix: '' } - return { path: url.slice(0, boundary), suffix: url.slice(boundary) } -} - function decodePath(path: string): string { try { return decodeURIComponent(path) @@ -229,8 +160,8 @@ export function rewriteMarkdown(source: string, options: RewriteMarkdownOptions) const replacements: Replacement[] = [] const rewrite = (node: RewritableNode): void => { - if (isExternalOrSiteAbsolute(node.url)) return - const { path, suffix } = splitTarget(node.url) + if (isExternalOrAbsoluteMarkdownUrl(node.url)) return + const { path, suffix } = splitMarkdownUrlTarget(node.url) if (path === '') return const { absPath, line } = resolveRepositoryTarget(sourceAbs, path, options.repoRoot) const targetPath = repoPath(absPath, options.repoRoot) @@ -247,16 +178,10 @@ export function rewriteMarkdown(source: string, options: RewriteMarkdownOptions) ? `${options.placeImage(absPath)}${suffix}` : githubTarget(absPath, line, suffix, options.repositoryRef, options.repoRoot, node.type === 'image') - const start = node.position?.start.offset - const end = node.position?.end.offset - if (start === undefined || end === undefined) { - throw new Error(`project-doc-site: link ${JSON.stringify(node.url)} has no source offsets.`) - } - const rawNode = source.slice(start, end) - const rawDestination = destinationRange(rawNode, node.type) + const destination = markdownDestination(source, node) replacements.push({ - start: start + rawDestination.start, - end: start + rawDestination.end, + start: destination.start, + end: destination.end, value: nextUrl, }) } @@ -395,34 +320,66 @@ export function docsSourceFiles(): string[] { return [...new Set([...docsPages.map(page => resolve(root, page.source)), ...referencedImages()])] } -/** Rebuild the disposable VitePress source tree from the publication manifest. */ -export function projectDocs(): void { +/** Manifest and repository inputs for one projection pass. */ +export interface ProjectionContext { + /** Pages to project. */ + pages: DocsPage[] + /** Repository root every source and placed image must live under. */ + repoRoot: string + /** Public ref used by projected GitHub links. */ + repositoryRef: string +} + +function defaultProjectionContext(): ProjectionContext { + return { pages: docsPages, repoRoot: root, repositoryRef: resolveRepositoryRef(process.env) } +} + +/** + * Project every page and its images into one target tree. + * + * `entries` are what gets emitted; link resolution always reads the canonical + * `context.pages`, so an alias entry sharing a source with its index route + * emits at its own path while links keep targeting canonical routes. + */ +function projectPagesInto( + targetRoot: string, + context: ProjectionContext, + pageContent: (markdown: string, page: DocsPage) => string, + entries: DocsPage[] = context.pages, +): void { const routes = new Set() /** Projected path to the repository file that claimed it, pages and images alike. */ const claimed = new Map() - const repositoryRef = process.env.GITHUB_SHA ?? 'master' - rmSync(generatedRoot, { recursive: true, force: true }) /** Reserve one projected path, refusing a second source for it. */ const claim = (target: string, sourceAbs: string): void => { const holder = claimed.get(target) if (holder !== undefined && holder !== sourceAbs) { throw new Error( - `project-doc-site: ${repoPath(sourceAbs, root)} and ${repoPath(holder, root)}` - + ` both project to ${relative(generatedRoot, target).split(sep).join('/')}.`, + `project-doc-site: ${repoPath(sourceAbs, context.repoRoot)} and ${repoPath(holder, context.repoRoot)}` + + ` both project to ${relative(targetRoot, target).split(sep).join('/')}.`, + ) + } + // A file the projection did not claim is another producer's output — in + // the twin pass, the build VitePress just wrote, including `public/` + // copies. Overwriting one would silently corrupt the site. + if (holder === undefined && existsSync(target)) { + throw new Error( + `project-doc-site: ${repoPath(sourceAbs, context.repoRoot)} would overwrite existing build file` + + ` ${relative(targetRoot, target).split(sep).join('/')}.`, ) } claimed.set(target, sourceAbs) } - for (const page of docsPages) { + for (const page of entries) { if (routes.has(page.route)) throw new Error(`project-doc-site: duplicate route ${JSON.stringify(page.route)}.`) routes.add(page.route) - const sourceAbs = resolve(root, page.source) + const sourceAbs = resolve(context.repoRoot, page.source) if (!existsSync(sourceAbs) || !lstatSync(sourceAbs).isFile()) { throw new Error(`project-doc-site: source ${JSON.stringify(page.source)} does not exist or is not a file.`) } - const output = resolve(generatedRoot, page.route) + const output = resolve(targetRoot, page.route) // Claimed before the images are placed: a page and an image landing on one // path would otherwise overwrite each other in whichever order they ran. claim(output, sourceAbs) @@ -432,14 +389,14 @@ export function projectDocs(): void { sourcePath: page.source, locale: page.locale, route: page.route, - pages: docsPages, - repoRoot: root, - repositoryRef, + pages: context.pages, + repoRoot: context.repoRoot, + repositoryRef: context.repositoryRef, placeImage: (absPath) => { - const real = publishableImage(absPath, root) + const real = publishableImage(absPath, context.repoRoot) if (real === undefined) { throw new Error( - `project-doc-site: ${page.source} references image ${repoPath(absPath, root)},` + `project-doc-site: ${page.source} references image ${repoPath(absPath, context.repoRoot)},` + ' which is not a regular file inside the repository.', ) } @@ -455,6 +412,164 @@ export function projectDocs(): void { return `./${encodeURI(name)}` }, }) - writeFileSync(output, addProjectionFrontmatter(projectedPageContent(projected, page), page)) + writeFileSync(output, pageContent(projected, page)) } } + +/** Rebuild the disposable VitePress source tree from the publication manifest. */ +export function projectDocs(): void { + rmSync(generatedRoot, { recursive: true, force: true }) + projectPagesInto(generatedRoot, defaultProjectionContext(), (markdown, page) => + addProjectionFrontmatter(projectedPageContent(markdown, page), page)) +} + +/** + * Strip the leading YAML frontmatter of a projected page. + * + * @param markdown Rewritten canonical Markdown content. + * @param source Repository-relative page source, named by the failure. + * @returns The content after the frontmatter block, or the input when none opens it. + */ +function withoutFrontmatter(markdown: string, source: string): string { + if (!markdown.startsWith('---\n')) return markdown + const closingDelimiter = '\n---\n' + const closing = markdown.indexOf(closingDelimiter, 4) + if (closing === -1) { + throw new Error(`project-doc-site: ${JSON.stringify(source)} has unclosed YAML frontmatter.`) + } + return markdown.slice(closing + closingDelimiter.length).replace(/^\n+/, '') +} + +/** + * The raw-Markdown twin of one published page. + * + * Frontmatter is VitePress rendering configuration and is dropped. A locale + * home page therefore keeps its body here, while the rendered site truncates + * it to the frontmatter redirect. + * + * @param markdown Rewritten canonical Markdown content. + * @param source Repository-relative page source, named by frontmatter failures. + * @returns Plain Markdown without frontmatter or repository chrome. + */ +export function rawMarkdownPageContent(markdown: string, source: string): string { + return withoutRepositoryChrome(withoutFrontmatter(markdown, source)) +} + +/** + * Parent-level alias route of an index route, or `undefined` for other routes. + * + * The rendered site shows an index route as a directory URL, so "append + * `.md`" naturally lands on `.md` once the trailing slash is dropped. + * The root `index.md` has no parent to alias into. + */ +function indexAliasRoute(route: string): string | undefined { + const match = /^(.+)\/index\.md$/.exec(route) + return match?.[1] === undefined ? undefined : `${match[1]}.md` +} + +/** + * Site-relative Markdown files the raw-Markdown projection emits: every + * route, plus one parent-level alias per index route. + * + * @param pages Pages to project, defaulting to the publication manifest. + * @returns The emitted paths, routes first. + */ +export function rawMarkdownFiles(pages: DocsPage[] = docsPages): string[] { + const aliases = pages.map(page => indexAliasRoute(page.route)).filter(alias => alias !== undefined) + return [...pages.map(page => page.route), ...aliases] +} + +/** + * Emit the raw-Markdown twin of every published route into a built site, so + * static hosting serves the page's URL, minus any trailing slash, plus `.md` + * as plain Markdown. Each index route also emits a parent-level alias twin, + * projected over the alias route so its relative links stay correct. + * Referenced images are copied beside the pages, keeping the same relative + * URLs valid in both trees. Existing build files stay in place, and a name + * collision with one fails the emission. + * + * @param outDir Build output directory to emit into. + * @param context Manifest and repository inputs, defaulting to this repository. + */ +export function emitRawMarkdownPages(outDir: string, context: ProjectionContext = defaultProjectionContext()): void { + const aliases = context.pages.flatMap((page) => { + const alias = indexAliasRoute(page.route) + return alias === undefined ? [] : [{ ...page, route: alias }] + }) + projectPagesInto( + outDir, + context, + (markdown, page) => rawMarkdownPageContent(markdown, page.source), + [...context.pages, ...aliases], + ) +} + +/** + * Raw Markdown served for one site route. + * + * Dev-server counterpart of {@link emitRawMarkdownPages}: images are not + * copied because the generated tree already serves them beside the page. + * + * @param route Manifest route, including its `.md` suffix. + * @param context Manifest and repository inputs, defaulting to this repository. + * @returns The projected page, or `undefined` when the manifest does not publish the route. + */ +export function rawMarkdownRoute(route: string, context: ProjectionContext = defaultProjectionContext()): string | undefined { + const page = context.pages.find(candidate => candidate.route === route) + if (page === undefined) return undefined + const markdown = readFileSync(resolve(context.repoRoot, page.source), 'utf8') + return rawMarkdownPageContent(rewriteMarkdown(markdown, { + sourcePath: page.source, + locale: page.locale, + route: page.route, + pages: context.pages, + repoRoot: context.repoRoot, + repositoryRef: context.repositoryRef, + placeImage: absPath => `./${encodeURI(basename(absPath))}`, + }), page.source) +} + +/** Site identity written into llms.txt. */ +export interface LlmsTxtSite { + /** Site base path, carrying the leading and trailing slashes VitePress requires. */ + base: string + /** Site title. */ + title: string + /** Site description. */ + description: string +} + +/** Locale groups llms.txt lists, in the order the site's navigation presents them. */ +const llmsTxtLocales: readonly { heading: string; locale: DocsLocale }[] = [ + { heading: '简体中文', locale: 'root' }, + { heading: 'English', locale: 'en' }, +] + +/** + * The llms.txt index of every published page's raw-Markdown twin. + * + * Links are site-absolute so an agent resolves them against the host it + * fetched llms.txt from; locale home pages stay out because this file is the + * agent-facing entry point itself. + * + * @param site Site identity and base path. + * @returns llms.txt content listing both locale trees. + */ +export function llmsTxt(site: LlmsTxtSite): string { + const lines = [ + `# ${site.title}`, + '', + `> ${site.description}`, + '', + '页面 URL 去掉末尾斜杠再加 `.md` 即为该页原始 Markdown(根路径用 `/index.md`);下方列表是各页精确地址。Drop any trailing slash and append `.md` to a page URL for its raw Markdown (the site root is `/index.md`); the list below carries the exact addresses.', + ] + for (const { heading, locale } of llmsTxtLocales) { + lines.push('', `## ${heading}`, '') + for (const collection of localeCollections[locale]) { + for (const page of orderedPages(locale, collection)) { + lines.push(`- [${page.label}](${site.base}${page.route}): ${page.section}`) + } + } + } + return `${lines.join('\n')}\n` +} diff --git a/scripts/publint-all.spec.ts b/scripts/publint-all.spec.ts index 9e6d4ef26e..ca692d2c7b 100644 --- a/scripts/publint-all.spec.ts +++ b/scripts/publint-all.spec.ts @@ -13,7 +13,11 @@ afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }) }) -function fixture(exportPath = './lib/index.js'): string { +function fixture(options: { + exportPath?: string + indexSource?: string + files?: Record +} = {}): string { const root = mkdtempSync(join(tmpdir(), 'dsh-publint-all-')) roots.push(root) const packageDir = join(root, 'packages/core/probe') @@ -26,10 +30,14 @@ function fixture(exportPath = './lib/index.js'): string { engines: { node: '>=22.19' }, sideEffects: false, files: ['lib'], - exports: { '.': { default: exportPath } }, + exports: { '.': { default: options.exportPath ?? './lib/index.js' } }, }, null, 2)}\n`) writeFileSync(join(packageDir, 'README.md'), '# Probe\n') - writeFileSync(join(packageDir, 'lib/index.js'), 'export const probe = true\n') + writeFileSync(join(packageDir, 'lib/index.js'), options.indexSource ?? 'export const probe = true\n') + for (const [path, source] of Object.entries(options.files ?? {})) { + mkdirSync(join(packageDir, path, '..'), { recursive: true }) + writeFileSync(join(packageDir, path), source) + } writeFileSync(join(packageDir, 'unpublished.js'), 'export const hidden = true\n') return root } @@ -54,14 +62,34 @@ describe('publint package runner', () => { }) it('rejects an export that exists in the workspace but is not published', () => { - const result = run(fixture('./unpublished.js')) + const result = run(fixture({ exportPath: './unpublished.js' })) expect(result.status).toBe(1) expect(result.stdout).toContain('unpublished.js') }) it('rejects a public export whose built file is missing', () => { - const result = run(fixture('./lib/missing.js')) + const result = run(fixture({ exportPath: './lib/missing.js' })) expect(result.status).toBe(1) expect(result.stdout).toContain('missing.js') }) + + it('accepts published relative JavaScript and CSS targets', () => { + const result = run(fixture({ + indexSource: "export { helper } from './helper.js'\nimport './theme.css'\n", + files: { + 'lib/helper.js': 'export const helper = true\n', + 'lib/theme.css': ':root {}\n', + }, + })) + expect(result.status, result.stderr).toBe(0) + }) + + it('rejects unpublished relative JavaScript and CSS targets', () => { + const result = run(fixture({ + indexSource: "export { helper } from './missing.js'\nimport './missing.css'\n", + })) + expect(result.status).toBe(1) + expect(result.stderr).toContain('imports "./missing.js"') + expect(result.stderr).toContain('imports "./missing.css"') + }) }) diff --git a/scripts/publint-all.ts b/scripts/publint-all.ts index b6ddba1451..34d04a0c0d 100644 --- a/scripts/publint-all.ts +++ b/scripts/publint-all.ts @@ -7,10 +7,11 @@ import { statSync, } from 'node:fs' import { availableParallelism } from 'node:os' -import { dirname, relative, resolve, sep } from 'node:path' +import { dirname, posix, relative, resolve, sep } from 'node:path' import { parseArgs } from 'node:util' import { publint, type Message, type PackFile } from 'publint' import { formatMessage } from 'publint/utils' +import ts from 'typescript' const CONCURRENCY_ENV = 'DSH_PUBLINT_CONCURRENCY' const repositoryRoot = resolve(import.meta.dirname, '..') @@ -32,8 +33,21 @@ interface PackageManifest { } type PublintResult = - | { path: string; status: 'passed'; messages: Message[]; manifest: Record } - | { path: string; status: 'failed'; messages: Message[]; manifest: Record; failure?: string } + | { + path: string + status: 'passed' + messages: Message[] + closureViolations: string[] + manifest: Record + } + | { + path: string + status: 'failed' + messages: Message[] + closureViolations: string[] + manifest: Record + failure?: string + } function workspacePackages(): PackageTarget[] { return globSync('packages/*/*/package.json', { cwd: packagesRoot }) @@ -103,21 +117,101 @@ function addPath(path: string, paths: Set): void { } } +interface RelativeImport { + specifier: string + line: number +} + +/** Return relative imports whose targets are absent from the publication view. */ +function publicationClosureViolations(target: PackageTarget, files: readonly PackFile[]): string[] { + const published = new Set(files.map(file => file.name)) + const violations: string[] = [] + for (const file of files) { + if (!/\.(?:js|mjs|cjs)$/.test(file.name)) continue + const bytes = file.data instanceof ArrayBuffer ? new Uint8Array(file.data) : file.data + const source = typeof bytes === 'string' ? bytes : Buffer.from(bytes).toString('utf8') + for (const imported of relativeImports(file.name, source)) { + const resolved = posix.normalize(posix.join(posix.dirname(file.name), imported.specifier)) + if (resolutionCandidates(resolved).some(candidate => published.has(candidate))) continue + violations.push( + `${target.path}/${file.name.slice('package/'.length)}:${String(imported.line)}` + + ` imports ${JSON.stringify(imported.specifier)}, but ${target.manifest.name ?? target.path}` + + ` does not publish ${JSON.stringify(resolved.slice('package/'.length))}`, + ) + } + } + return violations +} + +/** Paths a relative JavaScript module request can resolve to in a published package. */ +function resolutionCandidates(target: string): string[] { + const base = target.replace(/\/+$/, '') + return [ + target, + ...['.js', '.mjs', '.cjs', '/index.js', '/index.mjs', '/index.cjs'].map(suffix => base + suffix), + ] +} + +/** Extract relative static imports, re-exports, dynamic imports, and requires. */ +function relativeImports(file: string, sourceText: string): RelativeImport[] { + const source = ts.createSourceFile(file, sourceText, ts.ScriptTarget.Latest, false, ts.ScriptKind.JS) + const imports: RelativeImport[] = [] + const record = (node: ts.Node, literal: ts.Expression | undefined): void => { + if (literal === undefined || !ts.isStringLiteralLike(literal) || !literal.text.startsWith('.')) return + imports.push({ + specifier: literal.text, + line: source.getLineAndCharacterOfPosition(node.getStart(source)).line + 1, + }) + } + const visit = (node: ts.Node): void => { + if (ts.isImportDeclaration(node) || ts.isExportDeclaration(node)) { + record(node, node.moduleSpecifier) + } else if (ts.isCallExpression(node) + && (node.expression.kind === ts.SyntaxKind.ImportKeyword + || ts.isIdentifier(node.expression) && node.expression.text === 'require')) { + record(node, node.arguments[0]) + } + ts.forEachChild(node, visit) + } + visit(source) + return imports +} + +/** + * publint reads the CommonJS interop preamble inside the prebuilt browser + * bundles and reports CJS-written-as-ESM. Node never resolves those files: + * `lib/client.js` is evaluated by the page module system as a classic script, + * and `lib/worker.js` by `new Worker(url, { type: 'module' })` — both outside + * the Node resolution publint models. Exactly that verdict on exactly those + * files is suppressed; every other publint error stays fatal. + */ +function isBrowserBundleFormatFalsePositive(message: Message): boolean { + if (message.code !== 'FILE_INVALID_FORMAT') return false + const filePath = (message.args as { actualFilePath?: string }).actualFilePath ?? '' + const exportKey = Array.isArray(message.path) ? message.path.join('/') : '' + return /(^|\/)lib\/(client|worker)\.js$/.test(filePath) + || /(^|\/)\.\/(client|worker)$/.test(exportKey) +} + async function runPublint(target: PackageTarget): Promise { try { + const files = publicationFiles(target) + const closureViolations = publicationClosureViolations(target, files) const result = await publint({ pkgDir: 'package', - pack: { files: publicationFiles(target) }, + pack: { files }, }) const manifest = result.pkg as Record - return result.messages.some(message => message.type === 'error') - ? { path: target.path, status: 'failed', messages: result.messages, manifest } - : { path: target.path, status: 'passed', messages: result.messages, manifest } + const messages = result.messages.filter(message => !isBrowserBundleFormatFalsePositive(message)) + return messages.some(message => message.type === 'error') || closureViolations.length > 0 + ? { path: target.path, status: 'failed', messages, closureViolations, manifest } + : { path: target.path, status: 'passed', messages, closureViolations, manifest } } catch (error: unknown) { return { path: target.path, status: 'failed', messages: [], + closureViolations: [], manifest: target.manifest as Record, failure: error instanceof Error ? error.message : String(error), } @@ -150,7 +244,10 @@ function printResult(result: PublintResult): void { for (const message of result.messages) { console.log(formatMessage(message, result.manifest, { color: false }) ?? message.code) } - if (result.status === 'passed' && result.messages.length === 0) console.log('All good!') + for (const violation of result.closureViolations) console.error(violation) + if (result.status === 'passed' && result.messages.length === 0 && result.closureViolations.length === 0) { + console.log('All good!') + } } const packages = workspacePackages() diff --git a/scripts/publish-npm-baseline.ts b/scripts/publish-npm-baseline.ts index d69923910f..4e98065719 100644 --- a/scripts/publish-npm-baseline.ts +++ b/scripts/publish-npm-baseline.ts @@ -24,7 +24,7 @@ const DEFAULT_REGISTRY = 'https://registry.npm.harnessment.com' const DEFAULT_OUTPUT_DIRECTORY = '.artifacts/npm-baseline' const PACKAGE_PATTERNS = [ 'vendor/*/package.json', - 'packages/*/*/package.json', + 'packages/!(experimental)/*/package.json', 'apps/*/package.json', ] as const const DEPENDENCY_SECTIONS = [ @@ -42,7 +42,7 @@ node, bin_path, cwd, timeout_seconds = sys.argv[1:] pid, fd = pty.fork() if pid == 0: os.chdir(cwd) - os.execvpe(node, [node, bin_path, "web", "--host", "127.0.0.1", "--port", "0"], os.environ.copy()) + os.execvpe(node, [node, bin_path, "web", "--no-open", "--host", "127.0.0.1", "--port", "0"], os.environ.copy()) output = bytearray() ready_seen = False diff --git a/scripts/release/bump.ts b/scripts/release/bump.ts index 27679d8fa2..69f6582838 100644 --- a/scripts/release/bump.ts +++ b/scripts/release/bump.ts @@ -3,7 +3,8 @@ * readable from the repository rather than derived inside CI * ([rationale](../../.agents/notes/implemented/process/2026-08-10-npm-release-sequences.md)). * - * The dsh family shares one version across its members and the workspace root: + * The dsh family shares one version across its publishable members, private + * package manifests, and the workspace root: * `major`, `minor`, `patch`, or an explicit `x.y.z` (including a prerelease such * as `0.0.1-rc.1`). The vendored family has one version line per package, but * every release advances and publishes the complete family so the next release @@ -14,7 +15,7 @@ * the tag after the commit merges. CI never writes to the repository. */ -import { readFileSync, writeFileSync } from 'node:fs' +import { globSync, readFileSync, writeFileSync } from 'node:fs' import { join, matchesGlob } from 'node:path' import { parseArgs } from 'node:util' import { releaseFamily, type ReleaseFamily, type ReleaseMember } from './families.ts' @@ -39,16 +40,22 @@ const ROOT_MANIFEST = 'package.json' /** One manifest the bump rewrites, and the tag its new version will carry. */ interface PlannedVersion { + readonly manifestPath: string + readonly label: string + readonly from: string + readonly to: string + /** The tag this version publishes from, or undefined for a non-published manifest. */ + readonly tag: string | undefined +} + +/** One private dsh package whose version follows the publishable family. */ +interface PrivateDshVersion { /** Repository-relative manifest path. */ readonly manifestPath: string - /** Label for the log line. */ + /** Package directory used in bump output. */ readonly label: string - /** The version the manifest currently carries. */ - readonly from: string - /** The version to write. */ - readonly to: string - /** The tag this version publishes from, or undefined for the workspace root. */ - readonly tag: string | undefined + /** Current manifest version. */ + readonly version: string } /** @@ -236,14 +243,43 @@ function rootVersion(root: string): string { } /** - * Plan the dsh family's rewrite: one version for every member and the root. + * Discover private package manifests that share the dsh version without joining + * its publish set. + * @param root - repository root. + * @returns Private package manifests sorted by path. + */ +function privateDshVersions(root: string): PrivateDshVersion[] { + return globSync('packages/*/*/package.json', { cwd: root }) + .map(path => path.replaceAll('\\', '/')) + .sort() + .flatMap((manifestPath) => { + const parsed: unknown = JSON.parse(readFileSync(join(root, manifestPath), 'utf8')) + if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) { + throw new Error(`${manifestPath} is not a JSON object`) + } + const manifest = parsed as Record + if (manifest.private !== true) return [] + if (typeof manifest.version !== 'string') { + throw new Error(`${manifestPath} must declare a string version`) + } + return [{ + manifestPath, + label: manifestPath.slice(0, -'/package.json'.length), + version: manifest.version, + }] + }) +} + +/** + * Plan the dsh family's rewrite: one version for every publishable member, + * private package, and the root. * @param family - the dsh family. * @param root - repository root. * @param members - the family's members. * @param request - `major`, `minor`, `patch`, or an explicit version. * @returns The manifests to rewrite and the shared target version. */ -function planShared( +export function planShared( family: ReleaseFamily, root: string, members: readonly ReleaseMember[], @@ -259,13 +295,24 @@ function planShared( ] for (const member of members) { planned.push({ - manifestPath: join(member.directory, 'package.json'), + manifestPath: `${member.directory}/package.json`, label: member.directory, from: member.version, to: version, tag: family.tagFor({ ...member, version }), }) } + const publishableManifests = new Set(members.map(member => `${member.directory}/package.json`)) + for (const entry of privateDshVersions(root)) { + if (publishableManifests.has(entry.manifestPath)) continue + planned.push({ + manifestPath: entry.manifestPath, + label: entry.label, + from: entry.version, + to: version, + tag: undefined, + }) + } return { planned, version } } @@ -287,7 +334,7 @@ function planPerPackage( const tagged = lastTaggedVersion(family, member) const to = nextVendorVersion(member.version, tagged, prerelease) planned.push({ - manifestPath: join(member.directory, 'package.json'), + manifestPath: `${member.directory}/package.json`, label: member.directory, from: member.version, to, diff --git a/scripts/release/families.spec.ts b/scripts/release/families.spec.ts index 369fc62b16..d22db243d8 100644 --- a/scripts/release/families.spec.ts +++ b/scripts/release/families.spec.ts @@ -1,8 +1,12 @@ /** Release family discovery, publish order, tag naming, and the bump judgements. */ -import { describe, expect, it } from 'vitest' +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join, resolve } from 'node:path' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { officialClientBuildEnvironment, writeClientBuildRecord } from '../client-build-environment.ts' import { releaseFamily, type ReleaseMember } from './families.ts' -import { compareVersions, nextVendorVersion, reachesPayload } from './bump.ts' +import { compareVersions, nextVendorVersion, planShared, reachesPayload } from './bump.ts' /** * A release member standing in for a manifest on disk. @@ -15,7 +19,53 @@ function member(directory: string, name: string, manifest: Record): string { + const root = mkdtempSync(join(tmpdir(), 'dsh-release-build-')) + roots.push(root) + write(join(root, 'apps/web/dist/index.html'), '
') + write(join(root, 'packages/client/example/lib/client.js'), 'module.exports = {}\n') + writeClientBuildRecord(root, environment) + return root +} + +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }) + vi.unstubAllEnvs() +}) + describe('release families', () => { + it('excludes private experimental packages from the dsh release', () => { + const members = releaseFamily('dsh').members(resolve(import.meta.dirname, '../..')) + + expect(members.some(member => member.directory.startsWith('packages/experimental/'))).toBe(false) + expect(members.map(member => member.name)).not.toContain('@deepseek-ai/dsh-experimental-agent-team') + }) + + it('bumps private dsh packages without adding release tags', () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-release-version-')) + roots.push(root) + write(join(root, 'package.json'), '{"version":"0.0.1"}\n') + write(join(root, 'packages/experimental/prototype/package.json'), '{"version":"0.0.1","private":true}\n') + write(join(root, 'packages/core/unselected/package.json'), '{"version":"0.0.1"}\n') + + const dsh = releaseFamily('dsh') + const published = member('packages/core/published', '@deepseek-ai/dsh-published') + const { planned } = planShared(dsh, root, [published], '0.0.2') + + expect(planned.map(entry => ({ path: entry.manifestPath, tag: entry.tag }))).toEqual([ + { path: 'package.json', tag: undefined }, + { path: 'packages/core/published/package.json', tag: 'dsh-v0.0.2' }, + { path: 'packages/experimental/prototype/package.json', tag: undefined }, + ]) + }) + it('names one tag for the whole dsh family and one per vendored package', () => { const dsh = releaseFamily('dsh') const vendor = releaseFamily('vendor') @@ -49,6 +99,23 @@ describe('release families', () => { expect(() => { vendor.verifyVersions([{ ...members[0]!, version: 'latest' }]) }).toThrow(/unpublishable version/) }) + it('requires a current official client build only for dsh artifacts', () => { + const dsh = releaseFamily('dsh') + const vendor = releaseFamily('vendor') + const officialEnvironment = officialClientBuildEnvironment(resolve(import.meta.dirname, '../..')) + vi.stubEnv('DSH_CLIENT_COMMIT_HASH', officialEnvironment.DSH_CLIENT_COMMIT_HASH) + const official = buildFixture(officialEnvironment) + const defaultBuild = buildFixture({}) + + expect(() => { dsh.verifyBuildArtifacts(official) }).not.toThrow() + expect(() => { dsh.verifyBuildArtifacts(defaultBuild) }).toThrow(/DSH_CLIENT_TITLE/) + expect(() => { dsh.verifyBuildArtifacts(join(defaultBuild, 'missing')) }).toThrow(/record.*missing/) + expect(() => { vendor.verifyBuildArtifacts(join(defaultBuild, 'missing')) }).not.toThrow() + + write(join(official, 'packages/client/example/lib/client.js'), 'module.exports = { changed: true }\n') + expect(() => { dsh.verifyBuildArtifacts(official) }).toThrow(/artifacts differ/) + }) + it('publishes a dependency before its consumer, and orders ties by name', () => { const dsh = releaseFamily('dsh') const members = [ @@ -57,7 +124,7 @@ describe('release families', () => { member('packages/a/zebra', '@deepseek-ai/dsh-zebra'), ] - expect(dsh.publishOrder(members).map(entry => entry.name)).toEqual([ + expect(dsh.publishOrder(members).order.map(entry => entry.name)).toEqual([ '@deepseek-ai/dsh-library', '@deepseek-ai/dsh-consumer', '@deepseek-ai/dsh-zebra', @@ -74,6 +141,92 @@ describe('release families', () => { expect(() => { dsh.publishOrder(members) }).toThrow(/dependency cycle/) }) + it('publishes a peer before its consumer', () => { + const dsh = releaseFamily('dsh') + const members = [ + member('packages/a/consumer', '@deepseek-ai/dsh-consumer', { peerDependencies: { '@deepseek-ai/dsh-zebra': 'workspace:^' } }), + member('packages/a/zebra', '@deepseek-ai/dsh-zebra'), + ] + + // Name order alone would place the consumer first; the peer edge moves it. + expect(dsh.publishOrder(members).order.map(entry => entry.name)).toEqual([ + '@deepseek-ai/dsh-zebra', + '@deepseek-ai/dsh-consumer', + ]) + }) + + it('orders around a peer cycle rather than refusing to publish, and reports the edge it dropped', () => { + const dsh = releaseFamily('dsh') + const members = [ + member('packages/a/left', '@deepseek-ai/dsh-left', { peerDependencies: { '@deepseek-ai/dsh-right': 'workspace:^' } }), + member('packages/a/right', '@deepseek-ai/dsh-right', { peerDependencies: { '@deepseek-ai/dsh-left': 'workspace:^' } }), + ] + + // Sibling packages declare each other as peers, and npm treats an unmet peer + // as a warning, so this pair has to publish rather than fail the release. + const plan = dsh.publishOrder(members) + expect(plan.order.map(entry => entry.name)).toEqual([ + '@deepseek-ai/dsh-right', + '@deepseek-ai/dsh-left', + ]) + // One of the two edges has to give, and which one it is belongs in the log. + expect(plan.droppedPeerEdges).toEqual([ + { consumer: '@deepseek-ai/dsh-right', peer: '@deepseek-ai/dsh-left' }, + ]) + }) + + it('honours an install edge even when a peer cycle surrounds it', () => { + const dsh = releaseFamily('dsh') + const members = [ + member('packages/a/base', '@deepseek-ai/dsh-base', { peerDependencies: { '@deepseek-ai/dsh-consumer': 'workspace:^' } }), + member('packages/a/consumer', '@deepseek-ai/dsh-consumer', { + dependencies: { '@deepseek-ai/dsh-base': 'workspace:^' }, + peerDependencies: { '@deepseek-ai/dsh-base': 'workspace:^' }, + }), + ] + + // The install edge is absolute: base publishes first, and the peer edge that + // would reverse it is the one dropped. + const plan = dsh.publishOrder(members) + expect(plan.order.map(entry => entry.name)).toEqual([ + '@deepseek-ai/dsh-base', + '@deepseek-ai/dsh-consumer', + ]) + expect(plan.droppedPeerEdges).toEqual([ + { consumer: '@deepseek-ai/dsh-base', peer: '@deepseek-ai/dsh-consumer' }, + ]) + }) + + it('refuses an order that would publish a consumer before a dependency it installs', () => { + const dsh = releaseFamily('dsh') + const members = [ + member('packages/a/alpha', '@deepseek-ai/dsh-alpha', { peerDependencies: { '@deepseek-ai/dsh-bravo': 'workspace:^' } }), + member('packages/a/bravo', '@deepseek-ai/dsh-bravo', { peerDependencies: { '@deepseek-ai/dsh-charlie': 'workspace:^' } }), + member('packages/a/charlie', '@deepseek-ai/dsh-charlie', { dependencies: { '@deepseek-ai/dsh-alpha': 'workspace:^' } }), + ] + + // A cycle of two peer edges closed by one install edge: dropping a peer edge + // would order this, and the traversal drops the install edge instead. That + // order would publish charlie before the alpha it installs, so it is refused + // here rather than published. + expect(() => { dsh.publishOrder(members) }).toThrow(/no publish order honours @deepseek-ai\/dsh-charlie -> @deepseek-ai\/dsh-alpha/) + }) + + it('ignores devDependencies when ordering', () => { + const dsh = releaseFamily('dsh') + const members = [ + member('packages/a/alpha', '@deepseek-ai/dsh-alpha', { devDependencies: { '@deepseek-ai/dsh-zebra': 'workspace:^' } }), + member('packages/a/zebra', '@deepseek-ai/dsh-zebra'), + ] + + // A dev dependency is absent from the published package, so it must not move + // the consumer behind it. + expect(dsh.publishOrder(members).order.map(entry => entry.name)).toEqual([ + '@deepseek-ai/dsh-alpha', + '@deepseek-ai/dsh-zebra', + ]) + }) + it('applies the harness payload policy to dsh and keeps upstream payloads for vendored packages', () => { const dsh = releaseFamily('dsh') const vendor = releaseFamily('vendor') diff --git a/scripts/release/families.ts b/scripts/release/families.ts index d39552636d..7a73323244 100644 --- a/scripts/release/families.ts +++ b/scripts/release/families.ts @@ -11,23 +11,56 @@ import { globSync, readFileSync } from 'node:fs' import { resolve } from 'node:path' +import { + officialClientBuildEnvironment, + readClientBuildRecord, +} from '../client-build-environment.ts' import { validateTarballPayload } from '../publication-payload.ts' -/** Dependency sections that constrain publish order: a consumer must publish after its dependency. */ -const ORDER_SECTIONS = ['dependencies', 'optionalDependencies'] as const +/** + * Dependency sections a consumer must publish after, because npm resolves them + * when the package is installed: publishing a consumer first would leave a + * window where its own tree cannot be assembled. + */ +const INSTALL_SECTIONS = ['dependencies', 'optionalDependencies'] as const + +/** + * Peer declarations also order the publication, but they cannot constrain it. + * npm never installs a peer on the package's behalf — an unmet peer is a + * warning, not a resolution failure — and sibling packages legitimately declare + * each other as peers, which makes these edges the ones that close cycles. They + * order what they can and are dropped where they would deadlock. + */ +const PEER_SECTIONS = ['peerDependencies'] as const /** The workspace root manifest, which is never a release member. */ const WORKSPACE_ROOT_PACKAGE = '@deepseek-ai/dsh-root' +/** One peer declaration the publish order leaves unordered. */ +interface DroppedPeerEdge { + readonly consumer: string + /** The declared peer, which publishes after `consumer` or alongside it in a cycle. */ + readonly peer: string +} + +/** + * A family's publish order together with the ordering it could not honour. + * + * The dropped edges are part of the result rather than a detail of forming it: + * a release drops real ordering constraints, and the operator reading the pack + * log is the only one who can judge whether a newly dropped edge is expected. + */ +export interface PublishPlan { + readonly order: readonly ReleaseMember[] + /** Peer declarations left unordered, in the order the traversal reached them. */ + readonly droppedPeerEdges: readonly DroppedPeerEdge[] +} + /** One publishable package of a release family. */ export interface ReleaseMember { - /** Repository-relative package directory, for example `packages/core/session`. */ readonly directory: string - /** Package name from its manifest. */ readonly name: string - /** Package version from its manifest. */ readonly version: string - /** The parsed manifest, for payload policy and publication checks. */ readonly manifest: Readonly> } @@ -59,23 +92,28 @@ function requireString(manifest: Record, field: string, context /** The executable a family's installed artifacts are driven through. */ export interface InstalledEntry { - /** Package that carries the executable. */ readonly packageName: string - /** Path to the executable inside that package. */ readonly binPath: string } /** A release sequence: its members, its version baseline, and its tag naming. */ export abstract class ReleaseFamily { - /** Workflow-facing identifier, also the `--family` argument. */ + /** Workflow-facing `--family` identifier. */ abstract readonly id: string - /** Glob patterns, relative to the repository root, that select this family's manifests. */ + /** Repository-relative glob patterns selecting this family's manifests. */ abstract readonly patterns: readonly string[] /** Git tag prefix this family publishes from. */ abstract readonly tagPrefix: string + /** + * Assert that built artifacts match this release family's required profile. + * Families without environment-selected artifacts accept every build tree. + * @param _root - repository root containing generated artifacts. + */ + verifyBuildArtifacts(_root: string): void {} + /** * Discover this family's members. * @param root - repository root. @@ -107,45 +145,122 @@ export abstract class ReleaseFamily { } /** - * Order members so every package publishes after the family members it depends on. + * Order members so every package publishes after the family members it + * depends on, which is what makes a partial publication self-consistent: an + * interrupted run leaves a prefix whose packages never point at something + * absent from the registry. + * + * Install edges are honoured absolutely — a cycle among them is a defect this + * reports rather than works around. Peer edges order what they can and are + * dropped where honouring one would deadlock: sibling packages declare each + * other as peers, and npm treats an unmet peer as a warning rather than a + * resolution failure ([rationale](../../.agents/notes/implemented/process/2026-08-10-npm-release-sequences.md)). + * Every dropped edge is reported, because dropping one is a decision about a + * real release rather than an implementation detail. * @param members - this family's members. - * @returns The same members in publish order; ties break by name for determinism. + * @returns The order, ties broken by name for determinism, and the peer edges it left unordered. */ - publishOrder(members: readonly ReleaseMember[]): ReleaseMember[] { + publishOrder(members: readonly ReleaseMember[]): PublishPlan { const byName = new Map(members.map(member => [member.name, member])) - const ordered: ReleaseMember[] = [] - const placed = new Set() - const visiting = new Set() + const byNameSorted = [...members].sort((left, right) => left.name.localeCompare(right.name)) + const edges = (member: ReleaseMember, sections: readonly string[]): ReleaseMember[] => + this.orderEdges(member, byName, sections) - const visit = (member: ReleaseMember, path: readonly string[]): void => { - if (placed.has(member.name)) return - if (visiting.has(member.name)) { + // Install edges alone must be acyclic, and that is checked on its own graph: + // a peer edge leading into an install edge would otherwise read as a cycle + // where the install edges are perfectly orderable. + const installVisiting = new Set() + const installDone = new Set() + const checkInstall = (member: ReleaseMember, path: readonly string[]): void => { + if (installDone.has(member.name)) return + if (installVisiting.has(member.name)) { throw new Error(`dependency cycle in release family ${this.id}: ${[...path, member.name].join(' -> ')}`) } - visiting.add(member.name) - for (const dependency of this.orderEdges(member, byName)) { - visit(dependency, [...path, member.name]) + installVisiting.add(member.name) + for (const dependency of edges(member, INSTALL_SECTIONS)) checkInstall(dependency, [...path, member.name]) + installVisiting.delete(member.name) + installDone.add(member.name) + } + for (const member of byNameSorted) checkInstall(member, []) + + // Emit the order over both kinds of edge. A node already on the stack closes + // a cycle, and that cycle carries at least one peer edge because the install + // edges were just proved acyclic — but the back edge that reaches the stacked + // node is not necessarily the peer one, so the post-condition below decides + // whether the emitted order survived. + const ordered: ReleaseMember[] = [] + const droppedPeerEdges: DroppedPeerEdge[] = [] + const placed = new Set() + const onStack = new Set() + // Members reachable from one member through install edges. A peer edge is + // dropped when the peer installs the member declaring it: honouring it would + // emit a package before something it installs, and the install edge wins. + const installClosure = (member: ReleaseMember): Set => { + const reached = new Set() + const walk = (current: ReleaseMember): void => { + for (const dependency of edges(current, INSTALL_SECTIONS)) { + if (reached.has(dependency.name)) continue + reached.add(dependency.name) + walk(dependency) + } } - visiting.delete(member.name) + walk(member) + return reached + } + const visit = (member: ReleaseMember): void => { + if (placed.has(member.name) || onStack.has(member.name)) return + onStack.add(member.name) + for (const dependency of edges(member, INSTALL_SECTIONS)) visit(dependency) + for (const peer of edges(member, PEER_SECTIONS)) { + if (installClosure(peer).has(member.name)) { + droppedPeerEdges.push({ consumer: member.name, peer: peer.name }) + continue + } + // A peer already on the stack is an ancestor, so it publishes after this + // member rather than before it: the edge is dropped, not honoured. + if (onStack.has(peer.name)) droppedPeerEdges.push({ consumer: member.name, peer: peer.name }) + visit(peer) + } + onStack.delete(member.name) placed.add(member.name) ordered.push(member) } + for (const member of byNameSorted) visit(member) - for (const member of [...members].sort((left, right) => left.name.localeCompare(right.name))) { - visit(member, []) + // A cycle mixing both kinds of edge can put an install edge's target on the + // stack, where the traversal skips it like a peer edge and emits a consumer + // before something it installs. Nothing downstream can detect that, and it + // would only surface as an unresolvable install for whoever consumes the + // published packages, so the emitted order is checked against the edges it + // exists to honour. + const position = new Map(ordered.map((entry, index) => [entry.name, index])) + for (const [index, member] of ordered.entries()) { + for (const dependency of edges(member, INSTALL_SECTIONS)) { + const dependencyIndex = position.get(dependency.name) + if (dependencyIndex !== undefined && dependencyIndex < index) continue + throw new Error( + `release family ${this.id}: no publish order honours ${member.name} -> ${dependency.name};` + + ' a cycle mixing peer and dependency declarations reaches this dependency through a peer edge', + ) + } } - return ordered + return { order: ordered, droppedPeerEdges } } /** - * The family members one member depends on at runtime. + * The family members one member declares in the given sections. * @param member - the dependent member. * @param byName - every family member by package name. - * @returns Dependencies inside this family, sorted by name. + * @param sections - manifest sections to read. + * @returns Members of this family named there, sorted by name. */ - private orderEdges(member: ReleaseMember, byName: ReadonlyMap): ReleaseMember[] { + private orderEdges( + member: ReleaseMember, + byName: ReadonlyMap, + sections: readonly string[], + ): ReleaseMember[] { const edges: ReleaseMember[] = [] - for (const section of ORDER_SECTIONS) { + for (const section of sections) { const dependencies = member.manifest[section] if (dependencies === null || typeof dependencies !== 'object' || Array.isArray(dependencies)) continue for (const name of Object.keys(dependencies)) { @@ -193,12 +308,17 @@ export abstract class ReleaseFamily { abstract readonly installedEntry: InstalledEntry | undefined } -/** `packages/*` and `apps/*`: one shared version across the whole family. */ +/** Release packages and apps: one shared version across the whole family. */ class DshFamily extends ReleaseFamily { readonly id = 'dsh' - readonly patterns = ['packages/*/*/package.json', 'apps/*/package.json'] as const + readonly patterns = ['packages/!(experimental)/*/package.json', 'apps/*/package.json'] as const readonly tagPrefix = 'dsh-v' + /** Require current artifacts from a complete official client build. */ + override verifyBuildArtifacts(root: string): void { + readClientBuildRecord(root, officialClientBuildEnvironment(root)) + } + /** * Require one version across the family, the way a single tag can name it. * @param members - this family's members. diff --git a/scripts/release/pack.ts b/scripts/release/pack.ts index 47a33a26ac..3b68a1e49c 100644 --- a/scripts/release/pack.ts +++ b/scripts/release/pack.ts @@ -45,7 +45,8 @@ function main(): void { const family = releaseFamily(values.family) const root = process.cwd() const destination = resolve(root, values.out ?? DEFAULT_OUTPUT) - const members = family.publishOrder(family.members(root)) + const members = family.publishOrder(family.members(root)).order + family.verifyBuildArtifacts(root) family.verifyVersions(members) rmSync(destination, { recursive: true, force: true }) diff --git a/scripts/release/process.ts b/scripts/release/process.ts index 746f24ac36..f73d8d53b8 100644 --- a/scripts/release/process.ts +++ b/scripts/release/process.ts @@ -9,9 +9,7 @@ import { fileURLToPath } from 'node:url' /** Where and with what environment a release step runs a command. */ export interface RunOptions { - /** Working directory; defaults to the current one. */ readonly cwd?: string - /** Child environment; defaults to this process's. */ readonly env?: NodeJS.ProcessEnv } @@ -19,9 +17,7 @@ export interface RunOptions { export interface CommandResult { /** Exit status, or null when a signal ended the process. */ readonly status: number | null - /** Captured standard output. */ readonly stdout: string - /** Captured standard error. */ readonly stderr: string } @@ -38,6 +34,27 @@ export function attempt(command: string, args: readonly string[], options: RunOp return { status: result.status, stdout: result.stdout, stderr: result.stderr } } +/** + * Run a command, then echo and return its captured output. Output is buffered + * until exit and stdout precedes stderr. + * @param command - executable name. + * @param args - command arguments. + * @param options - working directory and environment. + * @returns The exit status and captured streams. + */ +export function attemptEchoed(command: string, args: readonly string[], options: RunOptions = {}): CommandResult { + const result = spawnSync(command, [...args], { + cwd: options.cwd, + env: options.env, + encoding: 'utf8', + stdio: ['inherit', 'pipe', 'pipe'], + }) + if (result.error !== undefined) throw result.error + if (result.stdout !== '') process.stdout.write(result.stdout) + if (result.stderr !== '') process.stderr.write(result.stderr) + return { status: result.status, stdout: result.stdout, stderr: result.stderr } +} + /** * Run a command, capture its standard output, and fail on a non-zero exit. * @param command - executable name. @@ -67,11 +84,7 @@ export function run(command: string, args: readonly string[], options: RunOption } /** - * Whether this module is the process entry point. - * - * The release scripts are both commands and modules: a test imports their pure - * logic, and importing a module runs its body, so an unguarded `main()` would - * run the wrong command with the wrong arguments. + * Return whether Node started the given module as the process entry point. * @param moduleUrl - the caller's `import.meta.url`. * @returns True when Node started this module. */ diff --git a/scripts/release/publish.ts b/scripts/release/publish.ts index 11ad01173c..2590e96e52 100644 --- a/scripts/release/publish.ts +++ b/scripts/release/publish.ts @@ -18,7 +18,7 @@ import { join, resolve } from 'node:path' import { setTimeout as sleep } from 'node:timers/promises' import { parseArgs } from 'node:util' import { releaseFamily } from './families.ts' -import { attempt, isEntry } from './process.ts' +import { attempt, attemptEchoed, isEntry } from './process.ts' import { packedIdentity, readPublishOrder } from './tarball.ts' /** @@ -102,7 +102,7 @@ async function publishTarball(tarball: string, name: string, version: string): P // command-line flag could not serve both and would override the manifest // that does. Each packed manifest decides, and // check-workspace-constraints holds every manifest to its sequence's level. - const result = attempt('npm', ['publish', tarball, ...tagArgs]) + const result = attemptEchoed('npm', ['publish', tarball, ...tagArgs]) const output = `${result.stdout}${result.stderr}` if (result.status === 0) return @@ -136,9 +136,15 @@ async function main(): Promise { const family = releaseFamily(values.family) const directory = resolve(process.cwd(), values.from) + // Every entry in the order settles as either published or already present, so + // one counter answers "how far along is this run" for whoever is watching a + // release that takes minutes per family. + const order = readPublishOrder(directory) + const total = String(order.length) let published = 0 let skipped = 0 - for (const filename of readPublishOrder(directory)) { + for (const [index, filename] of order.entries()) { + const progress = `[${String(index + 1)}/${total}]` const tarball = join(directory, filename) const { name, version } = packedIdentity(tarball) const state = registryState(name, version) @@ -151,7 +157,7 @@ async function main(): Promise { + '\nBump the version, or investigate why the build is not reproducible.', ) } - console.log(`release publish: ${name}@${version} already published, skipping`) + console.log(`release publish: ${progress} ${name}@${version} already published, skipping`) skipped += 1 continue } @@ -159,11 +165,14 @@ async function main(): Promise { // only skips does not wait at all. if (published > 0) await sleep(PUBLISH_SPACING_MS) await publishTarball(tarball, name, version) - console.log(`release publish: ${name}@${version} published`) + console.log(`release publish: ${progress} ${name}@${version} published`) published += 1 } - console.log(`release publish: family ${family.id}, ${String(published)} published, ${String(skipped)} already present`) + console.log( + `release publish: family ${family.id}, ${total} member(s),` + + ` ${String(published)} published, ${String(skipped)} already present`, + ) } if (isEntry(import.meta.url)) await main() diff --git a/scripts/release/verify.ts b/scripts/release/verify.ts index 1bd74c84d6..5829087f97 100644 --- a/scripts/release/verify.ts +++ b/scripts/release/verify.ts @@ -9,7 +9,33 @@ import { parseArgs } from 'node:util' import { isEntry } from './process.ts' -import { releaseFamily, type ReleaseFamily, type ReleaseMember } from './families.ts' +import { releaseFamily, type PublishPlan, type ReleaseFamily, type ReleaseMember } from './families.ts' + +/** + * Print the publish order the release will follow, and the peer declarations it + * leaves unordered. + * + * The order is the release's own plan: an interrupted publication leaves exactly + * a prefix of it, so reading it is how anyone judges what a partial run left on + * the registry, and printing it on every pull request is what makes a change to + * the order reviewable rather than only observable during a publication. + * @param family - the release family. + * @param plan - the resolved order and its dropped edges. + */ +function reportPublishOrder(family: ReleaseFamily, plan: PublishPlan): void { + console.log(`release verify: publish order for family ${family.id}, ${String(plan.order.length)} member(s):`) + const width = String(plan.order.length).length + for (const [index, member] of plan.order.entries()) { + console.log(` ${String(index + 1).padStart(width, ' ')} ${member.name}@${member.version}`) + } + if (plan.droppedPeerEdges.length === 0) return + console.log( + `release verify: ${String(plan.droppedPeerEdges.length)} peer declaration(s) publish unordered,` + + ' because the peer cannot precede the package declaring it without contradicting a dependency edge' + + ' or its own cycle. npm treats an unmet peer as a warning, so this orders nothing and blocks nothing:', + ) + for (const edge of plan.droppedPeerEdges) console.log(` ${edge.consumer} -> ${edge.peer}`) +} /** * Assert every member may be published: npm refuses a `private` package. @@ -55,6 +81,16 @@ function main(): void { const family = releaseFamily(values.family) const members = family.members(process.cwd()) family.verifyVersions(members) + // Resolve the publish order here, before the build: an install-edge cycle + // makes the order unrepresentable, and that has to surface at the first gate + // rather than when pack is already writing tarballs. + const plan = family.publishOrder(members) + if (plan.order.length !== members.length) { + throw new Error( + `release family ${family.id}: publish order covers ${String(plan.order.length)} of ${String(members.length)} members`, + ) + } + reportPublishOrder(family, plan) const publishing = process.env.RELEASE_PUBLISH === 'true' if (publishing) { @@ -64,7 +100,11 @@ function main(): void { const versions = [...new Set(members.map(member => member.version))] const summary = versions.length === 1 ? versions[0] : `${String(versions.length)} versions` - console.log(`release verify: family ${family.id}, ${String(members.length)} member(s), ${summary}${publishing ? ', publish gates passed' : ''}`) + console.log( + `release verify: family ${family.id}, ${String(members.length)} member(s), ${summary},` + + ` publish order resolved, ${String(plan.droppedPeerEdges.length)} peer declaration(s) unordered` + + (publishing ? ', publish gates passed' : ''), + ) } if (isEntry(import.meta.url)) main() diff --git a/scripts/rescope-vendor.ts b/scripts/rescope-vendor.ts index 3f5cb525c2..134dfad203 100644 --- a/scripts/rescope-vendor.ts +++ b/scripts/rescope-vendor.ts @@ -77,8 +77,6 @@ interface GenericSkip { } const GENERIC_SKIPS: readonly GenericSkip[] = [ - // `vendorPackages` lists vendor/ directory names, joined with 'vendor' below it. - { file: 'packages/examples/acp-demo/tests/built-bin.e2e.ts', upstream: ['cordis', 'cosmokit', 'schemastery'] }, // `Symbol.for('schemastery')` and the `vendor:` metadata field are upstream identifiers. { file: 'vendor/schemastery/src/index.ts', upstream: ['schemastery'] }, // Asserts the vendored-manifest table, which gains an upstream-name column. @@ -104,6 +102,35 @@ const GENERIC_SKIPS: readonly GenericSkip[] = [ // GROUP_ORDER holds `packages//` directory names, not package names. { file: 'scripts/gen-module-graph.ts', upstream: ['cordis'] }, { file: 'scripts/gen-doc-graphs.ts', upstream: ['cordis'] }, + // `cordis/*` is the extensions event domain, not a package subpath. The + // generated catalogs and every producer/consumer must preserve that wire id. + { file: 'docs/event-producer-consumer.md', upstream: ['cordis'] }, + { file: 'docs/event-producer-consumer.zh.md', upstream: ['cordis'] }, + { file: 'docs/subsystems/extensions.md', upstream: ['cordis'] }, + { file: 'docs/subsystems/extensions.zh.md', upstream: ['cordis'] }, + { file: 'packages/api/remotes/src/remote-events.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-client-runner/src/client/index.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-client-runner/src/client/runtime.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-client-runner/tests/orchestrator.client.spec.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-client-runner/tests/plugin.client.spec.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-host-runner/src/index.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-host-runner/src/inspect-registry.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-host-runner/src/types.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-host-runner/tests/helpers.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-host-runner/tests/runner.spec.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/cordis-host-runner/tests/versioning.spec.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/tool-cordis/src/api-catalog.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/tool-cordis/src/providers.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/ui-cordis/src/client/index.ts', upstream: ['cordis'] }, + { file: 'packages/extensions/ui-cordis/src/client/inventory.ts', upstream: ['cordis'] }, + { file: 'scripts/gen-cordis-catalog.ts', upstream: ['cordis'] }, + // The UI locale namespace and input-trigger source id are product keys. + { file: 'packages/client/ui-settings-plugin-inventory/src/client/PluginInventorySettingsTab.tsx', upstream: ['cordis'] }, + { file: 'packages/extensions/ui-cordis/src/client/CordisActionRow.tsx', upstream: ['cordis'] }, + { file: 'packages/extensions/ui-cordis/src/client/CordisDefineRow.tsx', upstream: ['cordis'] }, + { file: 'packages/extensions/ui-cordis/src/client/CordisPanel.tsx', upstream: ['cordis'] }, + { file: 'packages/extensions/ui-cordis/src/client/CordisRunRow.tsx', upstream: ['cordis'] }, + { file: 'packages/extensions/ui-cordis/src/client/locales.ts', upstream: ['cordis'] }, ] /** A string that must appear exactly `count` times once the rescope has run. */ @@ -132,7 +159,6 @@ const POSTCONDITIONS: readonly PostCondition[] = [ // The preset id the shipped composition documents to its own model. { file: 'apps/cli/config/agent-presets/cordis/agent.cordis.yml', text: 'The `cordis` agent preset', count: 1 }, { file: 'apps/cli/config/agent-presets/cordis/agent.cordis.yml', text: 'corrupting the `cordis` preset', count: 1 }, - { file: 'packages/examples/acp-demo/tests/built-bin.e2e.ts', text: '\'cordis\', \'loader\', \'include\', \'timer\', \'hmr\', \'logger-console\',', count: 1 }, ] /** @@ -275,8 +301,8 @@ const EXACT_EDITS: readonly ExactEdit[] = [ replace: `/** * Vendored framework libraries: rescoped into @deepseek-ai, so the gate below * would read them as plugin packages. They carry no cross-plugin runtime - * identity to share — the framework itself is a platform module (external), - * while these are ordinary libraries a browser bundle inlines. + * identity to share — the framework itself is a requested module-table row + * (external), while these are ordinary libraries a browser bundle inlines. */ const VENDORED_LIBRARY = /^@deepseek-ai\\/(cosmokit|schemastery)(\\/|$)/ @@ -405,8 +431,8 @@ const VENDORED_LIBRARY = /^@deepseek-ai\\/(cosmokit|schemastery)(\\/|$)/ expect: 1, }, { - // The framework peer is no longer a registry name, so the rehearsal must install this - // repository's vendored copies; cosmokit comes along as cordis's own dependency. + // The framework peer is a rescoped package, so the rehearsal installs this + // repository's vendored copies; cosmokit arrives as cordis's dependency. id: 'packed-install-vendored-peer', file: 'packages/sandbox/sandbox-local/tests/packed-install.e2e.ts', find: ` 'packages/runtime-diagnostics/invariants', @@ -523,7 +549,7 @@ function rewriteLine(line: string, file: string, all: readonly Pattern[]): strin * Markdown splits in two. Every fence is code a reader copies or a * configuration they mount, so every fence follows the rename regardless of its * info string. Prose follows it only under `docs/`, where a sentence quoting - * `` `cordis` `` teaches a name this repository no longer resolves; elsewhere + * `` `cordis` `` teaches an unresolved package name; elsewhere * prose is a record of what was true when it was written, and the same spelling * can mean something else entirely — the Python SDK's `cordis` option, or the * unvendored `@cordisjs/plugin-http`. diff --git a/scripts/run-coverage-partitions.ts b/scripts/run-coverage-partitions.ts new file mode 100644 index 0000000000..8626665b96 --- /dev/null +++ b/scripts/run-coverage-partitions.ts @@ -0,0 +1,30 @@ +/** CLI entry for partitioned Vitest coverage. */ +import { resolve } from 'node:path' +import { + COVERAGE_PARTITIONS_ENV, + COVERAGE_TEST_TIMEOUT_ENV, + CoveragePartitionCoordinator, + coverageTestTimeoutArgs, + forwardedCoverageArgs, + parseCoveragePartitionCount, +} from './coverage-partitions.ts' + +const partitions = parseCoveragePartitionCount(process.env[COVERAGE_PARTITIONS_ENV]) +if (partitions === undefined) { + throw new Error(`${COVERAGE_PARTITIONS_ENV} is required by partitioned coverage.`) +} +const pnpmEntrypoint = process.env.npm_execpath +if (pnpmEntrypoint === undefined || pnpmEntrypoint === '') { + throw new Error('partitioned coverage must be invoked through a pnpm package script.') +} + +const coordinator = new CoveragePartitionCoordinator({ + root: resolve(import.meta.dirname, '..'), + partitions, + pnpmEntrypoint, + vitestArgs: [ + ...coverageTestTimeoutArgs(process.env[COVERAGE_TEST_TIMEOUT_ENV]), + ...forwardedCoverageArgs(process.argv.slice(2)), + ], +}) +process.exitCode = await coordinator.run() diff --git a/scripts/run-gates.spec.ts b/scripts/run-gates.spec.ts index f2218ad210..02dc89c868 100644 --- a/scripts/run-gates.spec.ts +++ b/scripts/run-gates.spec.ts @@ -31,9 +31,9 @@ function resultFor(subject: Gate, status: GateResult['status'] = 'passed'): Gate } } -function withPnpmEntrypoint(action: () => T): T { +function withPnpmEntrypoint(action: () => T, entrypoint = '/private/pnpm.cjs'): T { const previous = process.env.npm_execpath - process.env.npm_execpath = '/private/pnpm.cjs' + process.env.npm_execpath = entrypoint try { return action() } finally { @@ -69,6 +69,7 @@ describe('gate graph validation', () => { 'ci-windows-observational', 'node-compat', 'check-all', + 'hygiene', 'doc-sync', ] as const)('constructs and executes preflight for a valid non-empty %s graph', async (mode) => { const subject = withPnpmEntrypoint(() => gatesForMode(mode)) @@ -83,6 +84,40 @@ describe('gate graph validation', () => { expect(ids).toContain('public-repository-links') }) + it('keeps the hygiene aggregate aligned with the package script checks', () => { + const ids = withPnpmEntrypoint(() => gatesForMode('hygiene').map(subject => subject.id)) + + expect(ids).toEqual([ + 'rescope-vendor', 'knip', 'publint', 'constraints', 'application-entrypoints', + 'dsh-package-licenses', 'package-invariants', 'built-package-invariants', 'node-next-types', + 'optional-dependency-imports', 'client-packages', 'cordis-config', + 'runtime-closure', 'vendored-links', + ]) + expect(defaultConcurrency('hygiene', ids.length, 8)).toEqual({ + workers: 4, + source: '8 available CPU(s), hygiene cap 4', + }) + }) + + it('schedules the longest documentation leaves before short checks', () => { + const ids = withPnpmEntrypoint(() => gatesForMode('doc-sync').map(subject => subject.id)) + + expect(ids.slice(0, 10)).toEqual([ + 'doc-typecheck', 'docs-site-build', 'doc-graphs', 'markdown-links', 'type-equivalence', + 'cordis-catalog', 'mermaid', 'scoped-events', 'translation-pairing', 'markdown-wrap', + ]) + }) + + it('launches a native pnpm entrypoint directly', () => { + const entrypoint = String.raw`C:\Program Files\pnpm\pnpm.exe` + const subject = withPnpmEntrypoint(() => gatesForMode('ci-windows-blocking')[0], entrypoint) + + expect(subject).toMatchObject({ + command: entrypoint, + args: ['run', 'build'], + }) + }) + it.each(['ci-primary', 'ci-static', 'check-all'] as const)( 'keeps the DSH package license policy in %s', (mode) => { @@ -92,20 +127,97 @@ describe('gate graph validation', () => { }, ) - it('keeps native Windows coverage blocking while portability inventory remains observational', () => { - const gates = withPnpmEntrypoint(() => gatesForMode('ci-windows-complete')) - const byId = new Map(gates.map(subject => [subject.id, subject])) + it.each(['ci-primary', 'ci-static', 'check-all'] as const)( + 'keeps the client dependency policy in %s', + (mode) => { + const ids = withPnpmEntrypoint(() => gatesForMode(mode).map(subject => subject.id)) + + expect(ids).toContain('client-packages') + }, + ) + + it.each(['ci-primary', 'ci-static', 'check-all', 'hygiene'] as const)( + 'keeps application entrypoint enforcement in %s', + (mode) => { + const ids = withPnpmEntrypoint(() => gatesForMode(mode).map(subject => subject.id)) + + expect(ids).toContain('application-entrypoints') + }, + ) + + it('keeps native Windows coverage blocking while retaining the observational inventory', () => { + const complete = withPnpmEntrypoint(() => gatesForMode('ci-windows-complete')) + const observational = withPnpmEntrypoint(() => gatesForMode('ci-windows-observational')) + .filter(gate => gate.id !== 'build' && gate.id !== 'docs-site-build') + const byId = new Map(complete.map(subject => [subject.id, subject])) expect(byId.get('coverage')?.allowFailure).not.toBe(true) expect(byId.get('coverage-exempt-heavy')?.allowFailure).not.toBe(true) - expect(byId.get('duplication')?.allowFailure).toBe(true) + expect(byId.get('coverage-exempt-heavy')?.needs).toContain('build') + expect(byId.get('coverage-exempt-heavy')?.after).toContain('coverage') + expect(observational).not.toHaveLength(0) + for (const gate of observational) { + const completeGate = byId.get(gate.id) + expect(completeGate?.allowFailure).toBe(true) + expect(completeGate?.after).toContain('coverage') + expect(completeGate?.after).not.toContain('coverage-exempt-heavy') + expect(completeGate?.needs).toEqual(gate.needs) + } + }) + + it('applies one configured test and polling timeout to both coverage gates', () => { + const gates = withEnv('DSH_COVERAGE_TEST_TIMEOUT_MS', '15000', () => + withPnpmEntrypoint(() => gatesForMode('ci-windows-complete'))) + + for (const id of ['coverage', 'coverage-exempt-heavy']) { + expect(gates.find(subject => subject.id === id)?.args).toEqual(expect.arrayContaining([ + '--testTimeout=15000', + '--expect.poll.timeout=15000', + ])) + } + }) + + it('keeps Vitest timeout defaults when the coverage override is absent', () => { + const gates = withEnv('DSH_COVERAGE_TEST_TIMEOUT_MS', undefined, () => + withPnpmEntrypoint(() => gatesForMode('ci-windows-complete'))) + + for (const id of ['coverage', 'coverage-exempt-heavy']) { + expect(gates.find(subject => subject.id === id)?.args).not.toEqual(expect.arrayContaining([ + expect.stringMatching(/^--(?:testTimeout|expect\.poll\.timeout)=/), + ])) + } + }) + + it('rejects an invalid coverage timeout before starting a gate', () => { + expect(() => withEnv('DSH_COVERAGE_TEST_TIMEOUT_MS', '0', () => + withPnpmEntrypoint(() => gatesForMode('ci-windows-complete')))) + .toThrow('DSH_COVERAGE_TEST_TIMEOUT_MS must be a positive integer') + }) + + it('selects partitioned coverage only when explicitly configured', () => { + const coverage = withEnv('DSH_COVERAGE_PARTITIONS', '3', () => + withPnpmEntrypoint(() => gatesForMode('ci-windows-complete').find(subject => subject.id === 'coverage'))) + + expect(coverage).toMatchObject({ + displayCommand: 'DSH_COVERAGE_PARTITIONS=3 pnpm run test:coverage:partitioned', + args: ['/private/pnpm.cjs', 'run', 'test:coverage:partitioned'], + streamOutput: true, + }) + }) + + it('rejects an invalid coverage partition count before starting a gate', () => { + expect(() => withEnv('DSH_COVERAGE_PARTITIONS', '1', () => + withPnpmEntrypoint(() => gatesForMode('ci-windows-complete')))) + .toThrow('DSH_COVERAGE_PARTITIONS must be an integer greater than 1') }) it.each([ ['empty', [], /gate graph has no gates/], ['duplicate ids', [gate('same'), gate('same')], /duplicate gate id "same"/], ['unknown dependencies', [gate('subject', { needs: ['missing'] })], /depends on unknown gate "missing"/], + ['unknown ordering predecessors', [gate('subject', { after: ['missing'] })], /waits for unknown gate "missing"/], ['cycles', [gate('first', { needs: ['second'] }), gate('second', { needs: ['first'] })], /dependency cycle: first -> second -> first/], + ['mixed cycles', [gate('first', { after: ['second'] }), gate('second', { needs: ['first'] })], /dependency cycle: first -> second -> first/], ] as const)('rejects %s before starting a child', async (_label, invalid, message) => { const execute = vi.fn(async (subject: Gate) => resultFor(subject)) @@ -131,6 +243,29 @@ describe('gate graph validation', () => { expect(execute).toHaveBeenCalledWith(root) expect(results[0]).toMatchObject({ gate: dependent, status: 'skipped', error: 'dependency failed or skipped: root' }) }) + + it('runs an ordered follower after its predecessor fails', async () => { + const follower = gate('follower', { after: ['root'] }) + const root = gate('root') + const execute = vi.fn(async (subject: Gate) => resultFor(subject, subject === root ? 'failed' : 'passed')) + + const results = await runGates([follower, root], 2, execute) + + expect(execute.mock.calls.map(([subject]) => subject.id)).toEqual(['root', 'follower']) + expect(results.map(result => result.status)).toEqual(['passed', 'failed']) + }) + + it('runs an ordered follower after its predecessor is skipped', async () => { + const follower = gate('follower', { after: ['dependent'] }) + const dependent = gate('dependent', { needs: ['root'] }) + const root = gate('root') + const execute = vi.fn(async (subject: Gate) => resultFor(subject, subject === root ? 'failed' : 'passed')) + + const results = await runGates([follower, dependent, root], 2, execute) + + expect(execute.mock.calls.map(([subject]) => subject.id)).toEqual(['root', 'follower']) + expect(results.map(result => result.status)).toEqual(['passed', 'skipped', 'failed']) + }) }) describe('Oxlint gate', () => { @@ -252,7 +387,13 @@ describe('Node 24 lane ownership', () => { 'built-bin-smoke', ]) expect(subject.find(item => item.id === 'publint')?.needs).toEqual(['build']) - expect(subject.find(item => item.id === 'built-package-invariants')?.needs).toEqual(['publint']) + expect(subject.find(item => item.id === 'build')?.env).toEqual({ + DSH_BUILD_CLIENT_PROFILE: 'official', + }) + expect(subject.find(item => item.id === 'node-compat')?.env).toEqual({ + DSH_BUILD_CLIENT_PROFILE: 'official', + }) + expect(subject.find(item => item.id === 'built-package-invariants')?.needs).toEqual(['build']) expect(subject.find(item => item.id === 'lint-and-duplication')?.needs).toEqual(['built-package-invariants']) for (const id of [ 'snapshot', @@ -294,6 +435,22 @@ describe('Linux primary graph', () => { }) describe('gate process outcomes', () => { + it('streams selected gate output without retaining it', async () => { + const write = vi.spyOn(process.stdout, 'write').mockReturnValue(true) + try { + const result = await runGate(gate('streamed', { + args: ['-e', "process.stdout.write('live output')"], + streamOutput: true, + })) + + expect(result.status).toBe('passed') + expect(result.output).toEqual([]) + expect(write).toHaveBeenCalledWith('live output') + } finally { + write.mockRestore() + } + }) + it.skipIf(process.platform === 'win32')('reports signal termination independently from exit status', async () => { const result = await runGate(gate('terminated', { args: ['-e', "process.kill(process.pid, 'SIGTERM')"], diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 27664fab5e..872041c32b 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -9,7 +9,15 @@ import { spawn } from 'node:child_process' import { availableParallelism } from 'node:os' import { resolve } from 'node:path' import { performance } from 'node:perf_hooks' +import { CLIENT_BUILD_PROFILE_SELECTOR } from './client-build-environment.ts' import { COVERAGE_EXEMPT_ENV, coverageExemptHeavySuites } from './coverage-exempt.ts' +import { + COVERAGE_PARTITIONS_ENV, + COVERAGE_TEST_TIMEOUT_ENV, + coverageTestTimeoutArgs, + parseCoveragePartitionCount, +} from './coverage-partitions.ts' +import { pnpmInvocation } from './pnpm-invocation.ts' /** A named aggregate exposed by the gate runner. */ export type Mode = @@ -26,6 +34,7 @@ export type Mode = | 'ci-windows-observational' | 'node-compat' | 'check-all' + | 'hygiene' | 'doc-sync' type GateResultStatus = 'passed' | 'failed' | 'skipped' @@ -39,8 +48,13 @@ export interface Gate { command: string args: string[] needs?: string[] + /** Gate ids that must settle, regardless of outcome, before this gate starts. */ + after?: string[] env?: Record + /** Keep a failure visible without failing the aggregate. */ allowFailure?: boolean + /** Write child output as it arrives instead of buffering it until completion. */ + streamOutput?: boolean } /** The observed outcome of one gate process. */ @@ -111,11 +125,12 @@ function parseMode(raw: string | undefined): Mode { case 'ci-windows-observational': case 'node-compat': case 'check-all': + case 'hygiene': case 'doc-sync': return raw default: throw new Error( - `run-gates: expected mode ci-primary | ci-linux-primary | ci-static | ci-lint-contracts-ready | ci-coverage | ci-snapshot | ci-artifacts | ci-consumers | ci-windows-blocking | ci-windows-complete | ci-windows-observational | node-compat | check-all | doc-sync, got ${JSON.stringify(raw)}.`, + `run-gates: expected mode ci-primary | ci-linux-primary | ci-static | ci-lint-contracts-ready | ci-coverage | ci-snapshot | ci-artifacts | ci-consumers | ci-windows-blocking | ci-windows-complete | ci-windows-observational | node-compat | check-all | hygiene | doc-sync, got ${JSON.stringify(raw)}.`, ) } } @@ -135,7 +150,7 @@ export function defaultConcurrency( if (selectedMode === 'ci-consumers') return { workers: total, source: 'ci-consumers gate count' } // Local modes cap workers: several doc gates each build a full ts.Program, // so an uncapped default on a large host trades wall clock for memory blowups. - const localCap = selectedMode === 'check-all' || selectedMode === 'doc-sync' + const localCap = selectedMode === 'check-all' || selectedMode === 'hygiene' || selectedMode === 'doc-sync' const modeLimit = localCap ? Math.min(4, available) : available return { workers: Math.min(total, modeLimit), @@ -165,6 +180,14 @@ function pnpmScript(id: string, script: string, options: Partial = {}): Ga } } +/** Build official client artifacts inside a CI aggregate without changing sibling gate environments. */ +function ciBuildGate(id = 'build', options: Partial = {}): Gate { + return pnpmScript(id, 'build', { + ...options, + env: { ...options.env, [CLIENT_BUILD_PROFILE_SELECTOR]: 'official' }, + }) +} + function pnpmExec(id: string, args: string[], options: Partial = {}): Gate { return { id, @@ -175,15 +198,6 @@ function pnpmExec(id: string, args: string[], options: Partial = {}): Gate } } -function pnpmInvocation(args: string[]): Pick { - const entrypoint = process.env.npm_execpath - if (entrypoint === undefined || entrypoint === '') { - throw new Error('run-gates: npm_execpath is unavailable; invoke the runner through a pnpm package script.') - } - // Windows cannot spawn the pnpm.cmd shim directly; the JavaScript entrypoint keeps every host shell-free. - return { command: process.execPath, args: [entrypoint, ...args] } -} - /** * Construct the complete gate list for a named aggregate. * @param selected - aggregate mode to construct. @@ -205,7 +219,7 @@ export function gatesForMode(selected: Mode): Gate[] { case 'ci-coverage': return coverageGates() case 'ci-snapshot': - return [pnpmScript('build', 'build'), snapshotGate()] + return [ciBuildGate(), snapshotGate()] case 'ci-artifacts': return ciArtifactGates() case 'ci-consumers': @@ -237,6 +251,13 @@ export function gatesForMode(selected: Mode): Gate[] { }), pnpmScript('module-graph', 'verify-module-graph', { label: 'module graph' }), ] + case 'hygiene': + return [ + ...hygieneLeafGates(), + pnpmScript('cordis-config', 'verify-cordis-config', { label: 'Cordis config' }), + pnpmScript('runtime-closure', 'verify-runtime-closure', { label: 'runtime closure' }), + pnpmScript('vendored-links', 'verify-vendored-links', { label: 'vendored links' }), + ] case 'doc-sync': return docSyncLeafGates() } @@ -245,10 +266,15 @@ export function gatesForMode(selected: Mode): Gate[] { function ciSharedStaticGates(): Gate[] { return [ pnpmScript('runtime-closure', 'verify-runtime-closure', { label: 'runtime closure' }), + pnpmScript('application-entrypoints', 'verify-application-entrypoints', { label: 'application entrypoints' }), pnpmScript('constraints', 'constraints'), pnpmScript('dsh-package-licenses', 'verify-dsh-package-licenses', { label: 'DSH package licenses' }), pnpmScript('package-invariants', 'verify-package-invariants', { label: 'package invariants' }), pnpmScript('cordis-config', 'verify-cordis-config', { label: 'Cordis config' }), + pnpmScript('optional-dependency-imports', 'verify-optional-dependency-imports', { + label: 'optional dependency imports', + }), + pnpmScript('client-packages', 'verify-client-packages', { label: 'client packages' }), pnpmScript('issue-management', 'test:issue-management', { label: 'Issue management policy' }), ] } @@ -272,7 +298,7 @@ function ciPrimaryGates(): Gate[] { // The prepared typecheck and build both drive Client tsc, while build also // repeats the Host contract pass. Wait for all three consumers so build // neither races tsbuildinfo nor replaces declarations while they are read. - pnpmScript('build', 'build', { needs: ['typecheck', 'lint', 'doc-typecheck'] }), + ciBuildGate('build', { needs: ['typecheck', 'lint', 'doc-typecheck'] }), pnpmScript('publint', 'publint', { needs: ['build'] }), pnpmScript('node-next-types', 'verify-node-next-types', { label: 'node-next types', @@ -354,7 +380,7 @@ function runningNodeMajor(): number { function ciStaticGates(options: { ownsBuild: boolean }): Gate[] { return [ ...ciSharedStaticGates(), - ...options.ownsBuild ? [pnpmScript('build', 'build')] : [], + ...options.ownsBuild ? [ciBuildGate()] : [], ...docSyncLeafGates({ includeDocTypecheck: options.ownsBuild, ...options.ownsBuild @@ -373,7 +399,7 @@ function ciStaticGates(options: { ownsBuild: boolean }): Gate[] { function ciArtifactGates(): Gate[] { return [ - pnpmScript('build', 'build'), + ciBuildGate(), pnpmScript('publint', 'publint', { needs: ['build'] }), pnpmScript('node-next-types', 'verify-node-next-types', { label: 'node-next types', @@ -388,10 +414,13 @@ function ciConsumerGates(): Gate[] { const builtTree = ['build'] const validatedBuild = ['built-package-invariants'] return [ - pnpmScript('build', 'build'), - pnpmScript('node-compat', 'check:node-compat', { label: 'Node compatibility' }), + ciBuildGate(), + pnpmScript('node-compat', 'check:node-compat', { + label: 'Node compatibility', + env: { [CLIENT_BUILD_PROFILE_SELECTOR]: 'official' }, + }), pnpmScript('publint', 'publint', { needs: builtTree }), - builtPackageInvariantsGate(['publint']), + builtPackageInvariantsGate(builtTree), pnpmScript('lint-and-duplication', 'check:ci:lint:contracts-ready', { label: 'lint and duplication', needs: validatedBuild, @@ -411,6 +440,20 @@ function ciConsumerGates(): Gate[] { } function webSnapshotGate(needs: string[]): Gate { + const workerRaw = process.env.DSH_WEB_SNAPSHOT_WORKERS + if (workerRaw !== undefined && workerRaw !== '') { + const workers = Number.parseInt(workerRaw, 10) + if (!Number.isSafeInteger(workers) || workers < 2 || String(workers) !== workerRaw) { + throw new Error(`run-gates: DSH_WEB_SNAPSHOT_WORKERS must be an integer greater than 1, got ${JSON.stringify(workerRaw)}.`) + } + return pnpmScript('web-snapshot', 'test:web:ci', { + label: 'web browser snapshot', + displayCommand: `DSH_SNAPSHOT=replay DSH_WEB_SNAPSHOT_WORKERS=${workers} pnpm run test:web:ci`, + env: { DSH_SNAPSHOT: 'replay' }, + needs, + streamOutput: true, + }) + } return pnpmScript('web-snapshot', 'test:web:built', { label: 'web browser snapshot', displayCommand: 'DSH_SNAPSHOT=replay pnpm run test:web:built', @@ -421,21 +464,32 @@ function webSnapshotGate(needs: string[]): Gate { function ciWindowsBlockingGates(): Gate[] { return [ - pnpmScript('windows-build', 'build', { label: 'build' }), + ciBuildGate('windows-build', { label: 'build' }), pnpmScript('windows-site', 'docs:build', { label: 'production site' }), ] } function ciWindowsCompleteGates(): Gate[] { + const coverage = coverageGates().map(gate => gate.id === 'coverage-exempt-heavy' + ? { + ...gate, + needs: [...new Set(['build', ...(gate.needs ?? [])])], + after: [...new Set(['coverage', ...(gate.after ?? [])])], + } + : gate) const observational = ciWindowsObservationalGates() // The required production site replaces the observational MPA build; both // VitePress modes write the same output directory and cannot overlap. .filter(gate => gate.id !== 'build' && gate.id !== 'docs-site-build') - .map(gate => ({ ...gate, allowFailure: true })) + .map(gate => ({ + ...gate, + allowFailure: true, + after: [...new Set(['coverage', ...(gate.after ?? [])])], + })) return [ - pnpmScript('build', 'build'), + ciBuildGate(), pnpmScript('windows-site', 'docs:build', { label: 'production site' }), - ...coverageGates(), + ...coverage, ...observational, ] } @@ -475,13 +529,17 @@ function lintGate(options: { needs?: string[] } = {}): Gate { // under v8 instrumentation while contributing nothing the thresholds need // (membership rules in scripts/coverage-exempt.ts). // -// DSH_COVERAGE_MAX_WORKERS is the lane's worker budget, so the two parallel -// gates split it instead of each claiming it whole (the failover pool's -// 8 x 6-instance bound assumes one lane never exceeds its value). The exempt +// DSH_COVERAGE_MAX_WORKERS is the ordinary lane's worker budget, so the two +// parallel gates split it instead of each claiming it whole. When +// DSH_COVERAGE_PARTITIONS is set, its single-worker processes replace the +// instrumented share while this budget still sizes the exempt gate. The exempt // gate's wall clock is dominated by its longest single file, so it takes the -// small share. A budget of 1 gives each gate 1 worker; lanes that need a -// strict total of one (the serial reference jobs) also set -// DSH_GATE_CONCURRENCY=1, which keeps the gates from overlapping at all. +// small share. A budget of 1 gives each gate 1 worker; lanes that need a strict +// total of one (the serial reference jobs) also set DSH_GATE_CONCURRENCY=1, +// which keeps the gates from overlapping at all. +// DSH_COVERAGE_TEST_TIMEOUT_MS raises Vitest's per-test and expect.poll +// defaults together for instrumented lanes whose scheduling overhead exceeds +// those defaults. Explicit fixture timeouts remain authoritative. function coverageWorkerArgs(): { instrumented: string[]; exempt: string[] } { const [flag] = positiveIntArg('DSH_COVERAGE_MAX_WORKERS', '--maxWorkers') if (flag === undefined) return { instrumented: [], exempt: [] } @@ -496,21 +554,33 @@ function coverageWorkerArgs(): { instrumented: string[]; exempt: string[] } { function coverageGates(): Gate[] { const workers = coverageWorkerArgs() - return [ - pnpmExec('coverage', [ + const timeouts = coverageTestTimeoutArgs(process.env[COVERAGE_TEST_TIMEOUT_ENV]) + const partitions = parseCoveragePartitionCount(process.env[COVERAGE_PARTITIONS_ENV]) + const instrumented = partitions === undefined + ? pnpmExec('coverage', [ 'vitest', 'run', '--coverage', ...workers.instrumented, + ...timeouts, ], { label: 'test:coverage', env: { [COVERAGE_EXEMPT_ENV]: '1' }, - }), + }) + : pnpmScript('coverage', 'test:coverage:partitioned', { + label: 'test:coverage', + displayCommand: `${COVERAGE_PARTITIONS_ENV}=${partitions} pnpm run test:coverage:partitioned`, + env: { [COVERAGE_EXEMPT_ENV]: '1' }, + streamOutput: true, + }) + return [ + instrumented, pnpmExec('coverage-exempt-heavy', [ 'vitest', 'run', ...coverageExemptHeavySuites.map(suite => suite.filter), ...workers.exempt, + ...timeouts, ], { label: 'test:coverage-exempt-heavy', }), @@ -558,6 +628,7 @@ function hygieneLeafGates(options: { artifactNeeds?: string[] } = {}): Gate[] { pnpmScript('knip', 'knip'), pnpmScript('publint', 'publint', artifactOptions), pnpmScript('constraints', 'constraints'), + pnpmScript('application-entrypoints', 'verify-application-entrypoints', { label: 'application entrypoints' }), pnpmScript('dsh-package-licenses', 'verify-dsh-package-licenses', { label: 'DSH package licenses' }), pnpmScript('package-invariants', 'verify-package-invariants', { label: 'package invariants' }), builtPackageInvariantsGate(options.artifactNeeds), @@ -565,6 +636,10 @@ function hygieneLeafGates(options: { artifactNeeds?: string[] } = {}): Gate[] { label: 'node-next types', ...artifactOptions, }), + pnpmScript('optional-dependency-imports', 'verify-optional-dependency-imports', { + label: 'optional dependency imports', + }), + pnpmScript('client-packages', 'verify-client-packages', { label: 'client packages' }), ] } @@ -579,38 +654,38 @@ function docSyncLeafGates(options: { if (options.docTypecheckNeeds !== undefined) docTypecheckOptions.needs = options.docTypecheckNeeds if (options.docTypecheckEnv !== undefined) docTypecheckOptions.env = options.docTypecheckEnv return [ + // Stable FIFO starts the longest leaves first; only docs-site-build writes website/.generated. ...options.includeDocTypecheck === false ? [] : [pnpmScript('doc-typecheck', options.docTypecheckScript ?? 'doc-typecheck', docTypecheckOptions)], + pnpmScript('docs-site-build', options.docsBuildScript ?? 'docs:build', { label: 'documentation build' }), + pnpmScript('doc-graphs', 'verify-doc-graphs', { label: 'doc graphs' }), + pnpmScript('markdown-links', 'verify-md-links', { label: 'markdown links' }), + pnpmScript('type-equivalence', 'verify-type-equiv', { label: 'type equivalence' }), pnpmScript('cordis-catalog', 'verify-cordis-catalog', { label: 'cordis catalog' }), + pnpmScript('mermaid', 'verify-mermaid'), + pnpmScript('scoped-events', 'verify-scoped-events', { label: 'scoped events' }), + pnpmScript('translation-pairing', 'verify-translation-pairing', { label: 'translation pairing' }), + pnpmScript('markdown-wrap', 'verify-md-wrap', { label: 'markdown wrap' }), pnpmScript('client-catalog', 'verify-client-catalog', { label: 'client catalog' }), pnpmScript('export-jsdoc', 'verify-export-jsdoc', { label: 'export jsdoc' }), pnpmScript('tool-catalog', 'verify-tool-catalog', { label: 'tool catalog' }), pnpmScript('config-catalog', 'verify-config-catalog', { label: 'config catalog' }), pnpmScript('persistence-catalog', 'verify-persistence-catalog', { label: 'persistence catalog' }), - pnpmScript('doc-graphs', 'verify-doc-graphs', { label: 'doc graphs' }), - pnpmScript('scoped-events', 'verify-scoped-events', { label: 'scoped events' }), - pnpmScript('markdown-wrap', 'verify-md-wrap', { label: 'markdown wrap' }), - pnpmScript('markdown-links', 'verify-md-links', { label: 'markdown links' }), pnpmScript('public-repository-links', 'verify-public-repository-links', { label: 'public repository links' }), pnpmScript('doc-refs', 'verify-doc-refs', { label: 'doc refs' }), pnpmScript('package-paths', 'verify-package-paths', { label: 'package paths' }), pnpmScript('config-source-ownership', 'verify-config-source-ownership', { label: 'config source ownership' }), pnpmScript('package-readme-model-experience', 'verify-package-readme-model-experience', { label: 'package README model experience' }), - pnpmScript('mermaid', 'verify-mermaid'), pnpmScript('agent-note-classification', 'verify-agent-note-classification', { label: 'agent note classification' }), pnpmScript('agent-note-format', 'verify-agent-note-format', { label: 'agent note format' }), pnpmScript('archived-agent-notes', 'verify-archived-agent-notes', { label: 'archived agent notes' }), - pnpmScript('type-equivalence', 'verify-type-equiv', { label: 'type equivalence' }), pnpmScript('skill-invocation-metadata', 'verify-skill-invocation-metadata', { label: 'skill invocation metadata' }), pnpmScript('translation-prompt', 'verify-translation-prompt', { label: 'translation prompt' }), - pnpmScript('translation-pairing', 'verify-translation-pairing', { label: 'translation pairing' }), pnpmScript('doc-budgets', 'verify-doc-budgets', { label: 'doc budgets' }), pnpmExec('docs-site-projection', ['vitest', 'run', 'scripts/project-doc-site.spec.ts', 'scripts/verify-doc-site-fragments.spec.ts'], { label: 'documentation site checks', }), - // Keep the VitePress build itself in one gate because projection rewrites website/.generated. - pnpmScript('docs-site-build', options.docsBuildScript ?? 'docs:build', { label: 'documentation build' }), pnpmScript('package-readme-limitations', 'verify-package-readme-limitations', { label: 'package README limitations' }), ] } @@ -623,7 +698,6 @@ function builtBinSmokeGate(needs: string[] = ['build']): Gate { 'vitest.e2e.config.ts', 'examples/headless-agent/tests/keyless-smoke.e2e.ts', 'apps/cli/tests/built-bin.e2e.ts', - 'packages/examples/acp-demo/tests/built-bin.e2e.ts', 'packages/host/directory-picker-native/tests/built-worker.e2e.ts', 'packages/sdk/server/tests/built-scope-carrier.e2e.ts', 'packages/subagent/subagent-codex/tests/loader-composition.e2e.ts', @@ -660,6 +734,11 @@ function validateGateGraph(gates: readonly Gate[]): void { throw new Error(`run-gates: gate ${JSON.stringify(gate.id)} depends on unknown gate ${JSON.stringify(dependency)}.`) } } + for (const predecessor of gate.after ?? []) { + if (!ids.has(predecessor)) { + throw new Error(`run-gates: gate ${JSON.stringify(gate.id)} waits for unknown gate ${JSON.stringify(predecessor)}.`) + } + } } const cycle = findDependencyCycle(gates) @@ -681,8 +760,8 @@ function findDependencyCycle(gates: readonly Gate[]): string[] | undefined { active.set(id, path.length) path.push(id) - for (const dependency of gate.needs ?? []) { - const cycle = visit(dependency) + for (const predecessor of [...(gate.needs ?? []), ...(gate.after ?? [])]) { + const cycle = visit(predecessor) if (cycle !== undefined) return cycle } path.pop() @@ -723,7 +802,7 @@ export async function runGates( for (;;) { let madeProgress = false while (running.length < maxActive) { - const ready = gates.find(gate => states.get(gate.id) === 'pending' && dependenciesPassed(gate, states)) + const ready = gates.find(gate => states.get(gate.id) === 'pending' && predecessorsReady(gate, states)) if (ready === undefined) break states.set(ready.id, 'running') running.push({ gate: ready, promise: execute(ready) }) @@ -732,32 +811,24 @@ export async function runGates( } if (running.length === 0) { - let pending = gates.filter(gate => states.get(gate.id) === 'pending') - while (pending.length > 0) { - const gate = pending.find(item => (item.needs ?? []).some((id) => { - const state = states.get(id) - return state === 'failed' || state === 'skipped' - })) - if (gate === undefined) throw new Error('run-gates: validated graph stalled without a failed dependency.') - const failedDeps = (gate.needs ?? []).filter((id) => { - const state = states.get(id) - return state === 'failed' || state === 'skipped' - }) - const result: GateResult = { - gate, - status: 'skipped', - durationMs: 0, - output: [], - exitCode: null, - signalCode: null, - error: `dependency failed or skipped: ${failedDeps.join(', ')}`, - } - states.set(gate.id, 'skipped') - results.set(gate.id, result) - observe(result) - pending = pending.filter(item => item !== gate) + const pending = gates.filter(gate => states.get(gate.id) === 'pending') + if (pending.length === 0) break + const gate = pending.find(item => (item.needs ?? []).some(id => gateFailed(states.get(id)))) + if (gate === undefined) throw new Error('run-gates: validated graph stalled without a failed dependency.') + const failedDeps = (gate.needs ?? []).filter(id => gateFailed(states.get(id))) + const result: GateResult = { + gate, + status: 'skipped', + durationMs: 0, + output: [], + exitCode: null, + signalCode: null, + error: `dependency failed or skipped: ${failedDeps.join(', ')}`, } - break + states.set(gate.id, 'skipped') + results.set(gate.id, result) + observe(result) + continue } if (!madeProgress) { @@ -776,8 +847,17 @@ export async function runGates( }) } -function dependenciesPassed(gate: Gate, states: Map): boolean { +function predecessorsReady(gate: Gate, states: Map): boolean { return (gate.needs ?? []).every(id => states.get(id) === 'passed') + && (gate.after ?? []).every(id => gateSettled(states.get(id))) +} + +function gateSettled(state: GateState | undefined): boolean { + return state === 'passed' || state === 'failed' || state === 'skipped' +} + +function gateFailed(state: GateState | undefined): boolean { + return state === 'failed' || state === 'skipped' } /** @@ -802,10 +882,12 @@ export async function runGate(gate: Gate): Promise { child.stdout.setEncoding('utf8') child.stderr.setEncoding('utf8') child.stdout.on('data', (chunk: string) => { - output.push({ stream: 'stdout', text: chunk }) + if (gate.streamOutput === true) process.stdout.write(chunk) + else output.push({ stream: 'stdout', text: chunk }) }) child.stderr.on('data', (chunk: string) => { - output.push({ stream: 'stderr', text: chunk }) + if (gate.streamOutput === true) process.stderr.write(chunk) + else output.push({ stream: 'stderr', text: chunk }) }) child.on('error', (error) => { spawnError = `failed to start command: ${error.message}` @@ -859,7 +941,7 @@ function printResult(result: GateResult): void { console.error(`command: ${result.gate.displayCommand}`) console.error(`outcome: ${formatGateResultReason(result)}`) } - printOutput(result.output) + if (result.gate.streamOutput !== true) printOutput(result.output) } function printSummary(results: GateResult[], durationMs: number): void { diff --git a/scripts/run-oxlint.spec.ts b/scripts/run-oxlint.spec.ts index 25245add32..85628382e8 100644 --- a/scripts/run-oxlint.spec.ts +++ b/scripts/run-oxlint.spec.ts @@ -16,6 +16,18 @@ describe('Oxlint invocation', () => { }) }) + it('uses location-preserving diagnostics in CI', () => { + expect(resolveOxlintInvocation(['.'], { CI: 'true', DSH_OXLINT_THREADS: '4' })).toEqual({ + args: ['.', '--format=unix', '--threads=4'], + env: { CI: 'true', DSH_OXLINT_THREADS: '4', GOMAXPROCS: '4' }, + }) + }) + + it('preserves an explicitly selected CI formatter', () => { + expect(resolveOxlintInvocation(['.', '--format', 'github'], { CI: 'true' }).args) + .toEqual(['.', '--format', 'github']) + }) + it.each(['0', '-1', '1.5', 'auto'])('rejects invalid worker bound %s', (value) => { expect(() => resolveOxlintInvocation(['.'], { DSH_OXLINT_THREADS: value })) .toThrow('DSH_OXLINT_THREADS must be a positive integer') diff --git a/scripts/run-oxlint.ts b/scripts/run-oxlint.ts index 13ed84799f..bcbddb5011 100644 --- a/scripts/run-oxlint.ts +++ b/scripts/run-oxlint.ts @@ -10,6 +10,14 @@ function isFixInvocation(args: readonly string[]): boolean { return args.some(arg => FIX_FLAGS.has(arg)) } +function hasOutputFormat(args: readonly string[]): boolean { + return args.some(arg => + arg === '-f' + || arg.startsWith('-f=') + || arg === '--format' + || arg.startsWith('--format=')) +} + /** Complete Oxlint child-process arguments and environment. */ export interface OxlintInvocation { readonly args: readonly string[] @@ -23,8 +31,10 @@ export interface OxlintInvocation { * @returns the complete CLI arguments and child environment. */ export function resolveOxlintInvocation(args: readonly string[], env: NodeJS.ProcessEnv): OxlintInvocation { + const resolvedArgs = [...args] + if (env.CI === 'true' && !hasOutputFormat(args)) resolvedArgs.push('--format=unix') const raw = env.DSH_OXLINT_THREADS - if (raw === undefined || raw === '') return { args: [...args], env: { ...env } } + if (raw === undefined || raw === '') return { args: resolvedArgs, env: { ...env } } const parsed = Number.parseInt(raw, 10) if (!Number.isSafeInteger(parsed) || parsed < 1 || String(parsed) !== raw) { throw new Error(`run-oxlint: DSH_OXLINT_THREADS must be a positive integer, got ${JSON.stringify(raw)}.`) @@ -33,7 +43,7 @@ export function resolveOxlintInvocation(args: readonly string[], env: NodeJS.Pro throw new Error('run-oxlint: use DSH_OXLINT_THREADS instead of passing --threads directly.') } return { - args: [...args, `--threads=${raw}`], + args: [...resolvedArgs, `--threads=${raw}`], env: { ...env, GOMAXPROCS: raw }, } } diff --git a/scripts/run-web-snapshots.ts b/scripts/run-web-snapshots.ts new file mode 100644 index 0000000000..17ce81d987 --- /dev/null +++ b/scripts/run-web-snapshots.ts @@ -0,0 +1,44 @@ +/** Run serial browser owners before one bounded snapshot pool. */ +import { spawn } from 'node:child_process' +import { pnpmInvocation } from './pnpm-invocation.ts' + +const serialFiles = [ + 'apps/web/tests/hmr-live.e2e.ts', + 'apps/web/tests/cordis-tool-round.e2e.ts', +] +const workerRaw = process.env.DSH_WEB_SNAPSHOT_WORKERS +const workers = Number.parseInt(workerRaw ?? '', 10) +if (!Number.isSafeInteger(workers) || workers < 2 || String(workers) !== workerRaw) { + throw new Error(`DSH_WEB_SNAPSHOT_WORKERS must be an integer greater than 1, got ${JSON.stringify(workerRaw)}.`) +} +const invocation = pnpmInvocation(['exec', 'vitest', 'run', '--config', 'vitest.web.config.ts']) +let serialStatus = 0 +for (const file of serialFiles) { + serialStatus = await run(invocation.command, [...invocation.args, file]) + if (serialStatus !== 0) break +} +if (serialStatus === 0) { + process.exitCode = await run(invocation.command, [ + ...invocation.args, + ...serialFiles.map(file => `--exclude=${file}`), + '--fileParallelism', + `--maxWorkers=${String(workers)}`, + ]) +} else { + process.exitCode = serialStatus +} + +function run(command: string, args: string[]): Promise { + return new Promise((resolveRun, reject) => { + const child = spawn(command, args, { stdio: 'inherit' }) + child.once('error', reject) + child.once('exit', (exitCode, signalCode) => { + if (signalCode !== null) { + console.error(`web snapshots terminated by ${signalCode}`) + resolveRun(1) + return + } + resolveRun(exitCode ?? 1) + }) + }) +} diff --git a/scripts/session-fixture-layout.snapshot.ts b/scripts/session-fixture-layout.snapshot.ts index eddf94c249..ae8d11c907 100644 --- a/scripts/session-fixture-layout.snapshot.ts +++ b/scripts/session-fixture-layout.snapshot.ts @@ -1,4 +1,4 @@ -/** Repository-wide canonical-layout check for committed session fixtures. */ +/** Repository-wide canonical-layout check for committed session snapshots. */ import { resolve } from 'node:path' import { expect, it } from 'vitest' @@ -6,7 +6,7 @@ import { inspectSessionFixtureLayouts } from './session-fixture-layout.ts' const root = resolve(import.meta.dirname, '..') -it('keeps every session-format JSONL fixture in canonical packed layout', () => { +it('keeps every session-format JSONL fixture projected into canonical packed layout', () => { const nonCanonical = inspectSessionFixtureLayouts(root) .filter(fixture => fixture.source !== fixture.canonical) .map(fixture => fixture.path) diff --git a/scripts/session-fixture-layout.spec.ts b/scripts/session-fixture-layout.spec.ts index 5ba5fdfeaa..c0b87953b7 100644 --- a/scripts/session-fixture-layout.spec.ts +++ b/scripts/session-fixture-layout.spec.ts @@ -1,5 +1,6 @@ import { describe, expect, it } from 'vitest' -import { decodeStorageRecord, type SessionEvent } from '@deepseek-ai/dsh-session' +import { type SessionEvent } from '@deepseek-ai/dsh-session' +import { parseSessionLog } from '@deepseek-ai/dsh-llm-replay' import { canonicalSessionFixture } from './session-fixture-layout.ts' const HEADER = ' {"type":"session","version":0,"id":"fixture","createdAt":1,"delegationDepth":0} ' @@ -22,8 +23,7 @@ function unpackedFixture(): string { } function decodedBody(content: string): SessionEvent[] { - return content.trimEnd().split('\n').slice(1) - .flatMap(line => decodeStorageRecord(JSON.parse(line) as unknown)) + return parseSessionLog(content) } describe('canonicalSessionFixture', () => { @@ -31,8 +31,12 @@ describe('canonicalSessionFixture', () => { const canonical = canonicalSessionFixture(unpackedFixture(), 'fixture.jsonl') expect(canonical).toBeDefined() expect(canonical?.split('\n')[0]).toBe(HEADER) - expect(JSON.parse(canonical?.split('\n')[1] ?? '{}')).toMatchObject({ type: 'text-chunks' }) - expect(decodedBody(canonical ?? '')).toStrictEqual(chunkRun()) + const packed = JSON.parse(canonical?.split('\n')[1] ?? '{}') as Record + expect(packed).toMatchObject({ type: 'text-chunks' }) + expect(packed).not.toHaveProperty('seq0') + expect(packed).not.toHaveProperty('time0') + expect(decodedBody(canonical ?? '').map(({ seq: _seq, time: _time, ...event }) => event)) + .toStrictEqual(chunkRun().map(({ seq: _seq, time: _time, ...event }) => event)) }) it('ignores JSONL whose first record is not a session header', () => { @@ -45,13 +49,22 @@ describe('canonicalSessionFixture', () => { expect(canonicalSessionFixture(packed ?? '')).toBe(packed) }) + it('is idempotent for an already projected fixture', () => { + const projected = [ + HEADER, + '{"type":"turn/start","data":{"turn":1,"seq":99,"time":100}}', + '', + ].join('\n') + expect(canonicalSessionFixture(projected)).toBe(projected) + }) + it('fails loud on malformed records after a session header', () => { expect(() => canonicalSessionFixture(`${HEADER}\n{not-json}\n`, 'broken.jsonl')) - .toThrow(/broken\.jsonl:2: invalid JSON/) + .toThrow(/broken\.jsonl: session snapshot line 2 contains invalid JSON/) }) it('labels malformed packed rows with the fixture path and line', () => { expect(() => canonicalSessionFixture(`${HEADER}\n{"type":"text-chunks"}\n`, 'broken.jsonl')) - .toThrow(/broken\.jsonl:2: invalid session storage record: malformed text-chunks storage row/) + .toThrow(/broken\.jsonl: session snapshot line 2: malformed text-chunks storage row/) }) }) diff --git a/scripts/session-fixture-layout.ts b/scripts/session-fixture-layout.ts index d1d0359374..996b40fbca 100644 --- a/scripts/session-fixture-layout.ts +++ b/scripts/session-fixture-layout.ts @@ -1,97 +1,85 @@ -/** Canonical packed-row layout helpers for repository session fixtures. */ +/** Canonical packed-row and envelope projection helpers for repository session fixtures. */ import { deepStrictEqual } from 'node:assert' import { execFileSync } from 'node:child_process' import { existsSync, readFileSync } from 'node:fs' import { resolve } from 'node:path' -import { decodeStorageRecord, packChunkRuns, type SessionEvent } from '@deepseek-ai/dsh-session' +import { packChunkRuns, type SessionEvent } from '@deepseek-ai/dsh-session' +import { parseSessionLog } from '@deepseek-ai/dsh-llm-replay' -/** One repository session fixture and its canonical packed representation. */ +/** One repository session fixture and its canonical projected representation. */ export interface SessionFixtureLayout { /** Repository-relative path with `/` separators. */ path: string /** Current fixture bytes decoded as UTF-8. */ source: string - /** Canonical packed fixture bytes. */ + /** Canonical projected fixture bytes. */ canonical: string } -interface RecordLine { - line: number - text: string -} - -function recordLines(content: string): RecordLine[] { - return content.split(/\r?\n/).flatMap((text, index) => ( - text.trim().length === 0 ? [] : [{ line: index + 1, text }] - )) -} - -function parseRecord(line: RecordLine, label: string): unknown { - try { - return JSON.parse(line.text) as unknown - } catch (error) { - const detail = error instanceof Error ? error.message : String(error) - throw new Error(`${label}:${line.line}: invalid JSON: ${detail}`, { cause: error }) - } -} - function isSessionHeader(value: unknown): boolean { return value !== null && typeof value === 'object' && (value as { type?: unknown }).type === 'session' } -function decodeBody(lines: readonly RecordLine[], label: string): SessionEvent[] { - return lines.flatMap((line) => { - const record = parseRecord(line, label) - try { - return decodeStorageRecord(record) - } catch (error) { - const detail = error instanceof Error ? error.message : String(error) - throw new Error(`${label}:${line.line}: invalid session storage record: ${detail}`, { cause: error }) - } - }) -} - function renderFixture(headerLine: string, events: readonly SessionEvent[]): string { return [ headerLine, - ...packChunkRuns(events).map(record => JSON.stringify(record)), + ...packChunkRuns(events).map((stored) => { + const record = stored as unknown as Record + delete record.seq + delete record.time + delete record.seq0 + delete record.time0 + return JSON.stringify(record) + }), '', ].join('\n') } +function withoutEnvelope(events: readonly SessionEvent[]): Array> { + return events.map((event) => { + const { seq: _seq, time: _time, ...projected } = event + return projected + }) +} + /** * Canonicalize one JSONL document when its first record is a session header. - * The header line remains byte-identical; body records decode to logical events - * and re-encode with {@link packChunkRuns}. Non-session JSONL returns undefined. + * The header line remains byte-identical; body records decode to logical events, + * re-encode with {@link packChunkRuns}, and omit storage sequence/time envelopes. + * Non-session JSONL returns undefined. * * @param content - JSONL source text. * @param label - path-like diagnostic label. * @returns Canonical text for a session fixture, otherwise undefined. */ export function canonicalSessionFixture(content: string, label = ''): string | undefined { - const lines = recordLines(content) - const header = lines[0] - if (header === undefined) return undefined + const headerLine = content.split(/\r?\n/).find(line => line.trim().length > 0) + if (headerLine === undefined) return undefined let headerValue: unknown try { - headerValue = JSON.parse(header.text) as unknown + headerValue = JSON.parse(headerLine) as unknown } catch { return undefined } if (!isSessionHeader(headerValue)) return undefined - const events = decodeBody(lines.slice(1), label) - const canonical = renderFixture(header.text, events) - const canonicalLines = recordLines(canonical) - const decoded = decodeBody(canonicalLines.slice(1), label) + let events try { - deepStrictEqual(decoded, events) + events = parseSessionLog(content) } catch (error) { - throw new Error(`${label}: packed rewrite changed the decoded event stream`, { cause: error }) + const detail = error instanceof Error ? error.message : String(error) + throw new Error(`${label}: ${detail}`, { cause: error }) } - if (renderFixture(header.text, decoded) !== canonical) { + const canonical = renderFixture(headerLine, events) + const decoded = parseSessionLog(canonical) + try { + deepStrictEqual(withoutEnvelope(decoded), withoutEnvelope(events)) + } catch (error) { + throw new Error(`${label}: packed snapshot rewrite changed the event payload stream`, { cause: error }) + } + if (renderFixture(headerLine, decoded) !== canonical) { throw new Error(`${label}: packed rewrite is not idempotent`) } return canonical diff --git a/scripts/smoke-python-runtime.py b/scripts/smoke-python-runtime.py index 700cfb7c79..6c4010e427 100644 --- a/scripts/smoke-python-runtime.py +++ b/scripts/smoke-python-runtime.py @@ -9,6 +9,7 @@ import json import os import queue import subprocess +import sys import tempfile import threading import time @@ -28,9 +29,13 @@ WORKFLOW_WORKER_TEXT = "workflow worker smoke ok" MINIMAL_PROMPT = "Exercise the packaged minimal agent's persistent Bash and string-replacement editor." MINIMAL_TEXT = "minimal agent smoke ok" MINIMAL_EDITOR_PATH_PREFIX = "Editor path: " -MINIMAL_SYSTEM_PROMPT = "You are a helpful software engineer assistant." +FS_SEARCH_PROMPT = "Exercise the packaged filesystem search tools." +FS_SEARCH_TEXT = "filesystem search smoke ok" +FS_SEARCH_MARKER = "PACKAGED_FS_SEARCH_OK" +MCP_PROMPT = "Exercise the packaged MCP client with one external stdio server." +MCP_TEXT = "MCP client smoke ok" MINIMAL_CORDIS = ( - Path(__file__).resolve().parent.parent / "examples" / "jsonrpc-agent" / "minimal.cordis.yml" + Path(__file__).resolve().parent.parent / "examples" / "python-sdk-agent" / "minimal.cordis.yml" ) MINIMAL_BASH_COMMAND = ( "counter=$(( ${counter:-0} + 1 )); export counter; " @@ -65,13 +70,27 @@ SNAPSHOT_WORKFLOW_SCRIPT = ( f"const reply = await agent('{SNAPSHOT_WORKFLOW_CHILD_PROMPT}', {{ label: 'workflow-child' }})\n" "return { reply }" ) -SNAPSHOT_DIRECTORY = ( +ADVANCED_SNAPSHOT_DIRECTORY = ( Path(__file__).resolve().parent / "snapshots" / "python-sdk-single-exe" / "advanced" ) -SNAPSHOT_FILENAMES = ("result.json", "session.jsonl", "session.1.jsonl", "session.2.jsonl") +ADVANCED_SNAPSHOT_FILENAMES = ("result.json", "session.jsonl", "session.1.jsonl", "session.2.jsonl") +MINIMAL_SNAPSHOT_DIRECTORY = ( + Path(__file__).resolve().parent / "snapshots" / "python-sdk-single-exe" / "minimal" +) +MINIMAL_SNAPSHOT_FILENAMES = ("model-visible.json",) +# The agent loop's dynamic runtime-context snapshot is the one model-visible message this +# expected output cannot carry: the same composition emits it on macOS and not on Linux +# (deepseek-harness#2488), and the file must replay on both. Everything else is compared. +RUNTIME_CONTEXT_PREFIX = "Current runtime context" CUSTOM_CORDIS = """\ - id: sdk-jsonrpc-server name: '@deepseek-ai/dsh-sdk-jsonrpc-server' +- id: deepseek-llm-api-extensions + name: '@deepseek-ai/dsh-deepseek-llm-api-extensions' +- id: session-log-deepseek + name: '@deepseek-ai/dsh-session-log-deepseek' + config: + enabled: true - id: agent-core name: '@deepseek-ai/dsh-agent-spine-demo' config: @@ -109,6 +128,144 @@ CUSTOM_CORDIS = """\ - id: cordis-tool name: '@deepseek-ai/dsh-tool-cordis' """ +FS_SEARCH_CORDIS = """\ +- id: sdk-jsonrpc-server + name: '@deepseek-ai/dsh-sdk-jsonrpc-server' +- id: agent-core + name: '@deepseek-ai/dsh-agent-spine-demo' + config: + workspaceContext: false + skills: + enabled: false + toolBash: false + toolJobs: false +- id: sessions + name: '@deepseek-ai/dsh-session-persistence-jsonl' + config: + root: !!js process.env.DSH_SESSION_ROOT + compression: 'none' +- id: subprocess + name: '@deepseek-ai/dsh-subprocess-local' +- id: fs-search + name: '@deepseek-ai/dsh-tool-fs-search' + config: + sampleOverCapGlobResults: false +""" +MCP_SERVER_SCRIPT = """\ +import json +import os +import sys +import time + + +log_path = os.environ.get("MCP_SMOKE_LOG") + + +def send(message): + sys.stdout.write(json.dumps(message, separators=(",", ":")) + "\\n") + sys.stdout.flush() + + +for line in sys.stdin: + request = json.loads(line) + if log_path is not None: + with open(log_path, "a", encoding="utf-8") as log: + log.write(str(request.get("method")) + "\\n") + request_id = request.get("id") + if request_id is None: + continue + method = request.get("method") + if method == "initialize": + send({ + "jsonrpc": "2.0", + "id": request_id, + "result": { + "protocolVersion": request["params"]["protocolVersion"], + "capabilities": {"tools": {"listChanged": False}}, + "serverInfo": {"name": "python-wheel-fixture", "version": "1.0.0"}, + }, + }) + elif method == "tools/list": + # Keep discovery pending long enough that an SDK runtime answering + # initialize before discovery completes makes its first model request + # without this tool and fails deterministically. + time.sleep(0.25) + send({ + "jsonrpc": "2.0", + "id": request_id, + "result": { + "tools": [{ + "name": "add", + "description": "Add two numbers.", + "inputSchema": { + "type": "object", + "properties": {"a": {"type": "number"}, "b": {"type": "number"}}, + "required": ["a", "b"], + "additionalProperties": False, + }, + }], + }, + }) + elif method == "tools/call": + params = request["params"] + if params.get("name") != "add" or params.get("arguments") != {"a": 19, "b": 23}: + send({ + "jsonrpc": "2.0", + "id": request_id, + "error": {"code": -32602, "message": "unexpected tool call"}, + }) + continue + send({ + "jsonrpc": "2.0", + "id": request_id, + "result": {"content": [{"type": "text", "text": "42"}]}, + }) + else: + send({ + "jsonrpc": "2.0", + "id": request_id, + "error": {"code": -32601, "message": f"unsupported method: {method}"}, + }) +""" + + +def mcp_cordis(server_script: Path) -> str: + """Build an external config that mounts the packaged MCP client.""" + return json.dumps([ + { + "id": "sdk-jsonrpc-server", + "name": "@deepseek-ai/dsh-sdk-jsonrpc-server", + }, + { + "id": "agent-core", + "name": "@deepseek-ai/dsh-agent-spine-demo", + "config": { + "workspaceContext": False, + "skills": {"enabled": False}, + "toolBash": False, + }, + }, + { + "id": "sessions", + "name": "@deepseek-ai/dsh-session-persistence-jsonl", + "config": {"root": "./sessions", "compression": "none"}, + }, + { + "id": "mcp-fixture", + "name": "@deepseek-ai/dsh-mcp-client", + "config": { + "serverName": "fixture", + "transport": "stdio", + "command": sys.executable, + "args": [str(server_script)], + "env": {"MCP_SMOKE_LOG": str(server_script.with_suffix(".log"))}, + "failOnStartupError": True, + "reconnect": {"enabled": False}, + }, + }, + ], indent=2) + + class MockModelHandler(BaseHTTPRequestHandler): """Return deterministic text, worker, and orchestration completions.""" @@ -143,6 +300,12 @@ def completion_chunks(body: dict[str, object]) -> list[dict[str, object]]: if latest.get("role") == "tool": call_id, tool_name = latest_tool_call(messages) tool_text = message_text(latest.get("content")) + mcp = mcp_tool_followup(call_id, tool_name, tool_text) + if mcp is not None: + return mcp + fs_search = fs_search_tool_followup(call_id, tool_name, tool_text) + if fs_search is not None: + return fs_search minimal = minimal_tool_followup(body, call_id, tool_name, tool_text) if minimal is not None: return minimal @@ -170,17 +333,9 @@ def completion_chunks(body: dict[str, object]) -> list[dict[str, object]]: ), None, ) + # The minimal composition's assembled system prompt, advertised tool schemas, and + # model-visible messages are pinned by its snapshot, not asserted here. if minimal_prompt is not None: - names = advertised_tool_names(body) - if names != {"bash", "str_replace_editor"}: - raise AssertionError(f"minimal agent smoke advertised unexpected tools: {names}") - system_prompts = [ - message_text(message.get("content")) - for message in messages - if isinstance(message, dict) and message.get("role") == "system" - ] - if system_prompts != [MINIMAL_SYSTEM_PROMPT]: - raise AssertionError(f"minimal agent smoke assembled unexpected system prompts: {system_prompts}") return tool_call_chunks( "minimal-bash-1", "bash", @@ -192,6 +347,8 @@ def completion_chunks(body: dict[str, object]) -> list[dict[str, object]]: SNAPSHOT_PROMPT, CODE_PROMPT, WORKFLOW_PROMPT, + FS_SEARCH_PROMPT, + MCP_PROMPT, } prompt = next( (candidate for candidate in user_prompts if candidate in scenario_prompts), @@ -233,9 +390,60 @@ def completion_chunks(body: dict[str, object]) -> list[dict[str, object]]: }, }, ) + if prompt == FS_SEARCH_PROMPT: + assert_advertised_tool(body, "grep") + assert_advertised_tool(body, "glob") + return tool_call_chunks( + "fs-search-grep", + "grep", + {"pattern": FS_SEARCH_MARKER, "path": "."}, + ) + if prompt == MCP_PROMPT: + assert_advertised_tool(body, "mcp__fixture__add") + return tool_call_chunks( + "mcp-add", + "mcp__fixture__add", + {"a": 19, "b": 23}, + ) return text_chunks(EXPECTED_TEXT) +def mcp_tool_followup( + call_id: str, + tool_name: str, + tool_text: str, +) -> list[dict[str, object]] | None: + """Verify one tool call through the packaged MCP client.""" + if call_id != "mcp-add": + return None + if tool_name != "mcp__fixture__add" or "42" not in tool_text: + raise AssertionError(f"packaged MCP call returned an unexpected result: {tool_name}: {tool_text}") + return text_chunks(MCP_TEXT) + + +def fs_search_tool_followup( + call_id: str, + tool_name: str, + tool_text: str, +) -> list[dict[str, object]] | None: + """Exercise both ripgrep-backed tools through the packaged executable.""" + if not call_id.startswith("fs-search-"): + return None + if call_id == "fs-search-grep" and tool_name == "grep": + if "needle.txt" not in tool_text or FS_SEARCH_MARKER not in tool_text: + raise AssertionError(f"packaged grep returned no marker: {tool_text}") + return tool_call_chunks( + "fs-search-glob", + "glob", + {"pattern": "**/*.txt"}, + ) + if call_id == "fs-search-glob" and tool_name == "glob": + if "needle.txt" not in tool_text: + raise AssertionError(f"packaged glob returned no fixture path: {tool_text}") + return text_chunks(FS_SEARCH_TEXT) + raise AssertionError(f"unexpected filesystem-search follow-up: {call_id} {tool_name}: {tool_text}") + + def minimal_tool_followup( body: dict[str, object], call_id: str, @@ -477,16 +685,16 @@ def main() -> None: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument( "--scenario", - choices=("all", "sdk-default", "sdk-custom", "sdk-minimal", "sdk-snapshot", "direct"), + choices=("all", "sdk-default", "sdk-custom", "sdk-minimal", "sdk-fs-search", "sdk-mcp", "sdk-snapshot", "direct"), default="all", ) parser.add_argument("--exe", type=Path) parser.add_argument("--update-snapshots", action="store_true") args = parser.parse_args() - if args.scenario in {"all", "sdk-custom", "sdk-minimal", "sdk-snapshot", "direct"} and args.exe is None: + if args.scenario in {"all", "sdk-custom", "sdk-minimal", "sdk-fs-search", "sdk-snapshot", "direct"} and args.exe is None: parser.error("--exe is required for custom, minimal, snapshot, and direct scenarios") - if args.update_snapshots and args.scenario not in {"all", "sdk-snapshot"}: - parser.error("--update-snapshots requires --scenario sdk-snapshot or all") + if args.update_snapshots and args.scenario not in {"all", "sdk-minimal", "sdk-snapshot"}: + parser.error("--update-snapshots requires --scenario sdk-minimal, sdk-snapshot, or all") if args.exe is not None and not args.exe.is_file(): parser.error(f"runtime executable does not exist: {args.exe}") @@ -498,7 +706,12 @@ def main() -> None: smoke_sdk_custom(model.url, args.exe.resolve()) if args.scenario in {"all", "sdk-minimal"}: assert args.exe is not None - smoke_sdk_minimal(model.url, args.exe.resolve()) + smoke_sdk_minimal(model.url, args.exe.resolve(), args.update_snapshots) + if args.scenario in {"all", "sdk-fs-search"}: + assert args.exe is not None + smoke_sdk_fs_search(model.url, args.exe.resolve()) + if args.scenario in {"all", "sdk-mcp"}: + smoke_sdk_mcp(model.url, None if args.exe is None else args.exe.resolve()) if args.scenario in {"all", "sdk-snapshot"}: assert args.exe is not None smoke_sdk_snapshot(model.url, args.exe.resolve(), args.update_snapshots) @@ -558,10 +771,12 @@ def smoke_sdk_custom(base_url: str, executable: Path) -> None: assert_session_log(sessions, root, EXPECTED_TEXT, CODE_WORKER_TEXT, WORKFLOW_WORKER_TEXT) -def smoke_sdk_minimal(base_url: str, executable: Path) -> None: +def smoke_sdk_minimal(base_url: str, executable: Path, update_snapshots: bool) -> None: """Exercise the checked-in minimal composition through the packaged executable.""" from deepseek_harness import DeepSeekHarness + # One mock model serves every scenario of a run, so the snapshot takes this turn's slice. + first_request = len(MockModelHandler.requests) with tempfile.TemporaryDirectory(prefix="dsh-sdk-minimal-") as temporary: root = Path(temporary).resolve() editor_path = root / "created.txt" @@ -587,6 +802,73 @@ def smoke_sdk_minimal(base_url: str, executable: Path) -> None: raise AssertionError(f"packaged editor wrote unexpected content: {editor_path.read_text()!r}") assert_session_log(sessions, root, MINIMAL_TEXT, "COUNT=1", "COUNT=2 CWD=/tmp") + files = build_minimal_snapshot_files(MockModelHandler.requests[first_request:], root) + compare_snapshot_files( + files, update_snapshots, MINIMAL_SNAPSHOT_DIRECTORY, MINIMAL_SNAPSHOT_FILENAMES, + ) + + +def smoke_sdk_fs_search(base_url: str, executable: Path) -> None: + """Exercise real grep and glob spawns through the packaged executable.""" + from deepseek_harness import DeepSeekHarness + + with tempfile.TemporaryDirectory(prefix="dsh-sdk-fs-search-") as temporary: + root = Path(temporary).resolve() + (root / "needle.txt").write_text(f"{FS_SEARCH_MARKER}\n") + sessions = root / "sessions" + cordis = root / "cordis.yml" + cordis.write_text(FS_SEARCH_CORDIS) + with DeepSeekHarness( + provider="deepseek-official", + model="smoke-model", + cwd=str(root), + session_root=str(sessions), + cordis=str(cordis), + runtime_bin=str(executable), + api_key="sk-keyless-smoke", + base_url=base_url, + request_timeout_seconds=60, + ) as harness: + result = harness.run(FS_SEARCH_PROMPT, session_id="fs-search-smoke") + + assert result.final_response == FS_SEARCH_TEXT, result.final_response + assert_session_log(sessions, root, FS_SEARCH_TEXT, FS_SEARCH_MARKER, "needle.txt") + + +def smoke_sdk_mcp(base_url: str, executable: Path | None) -> None: + """Discover and call an external stdio MCP tool through the packaged client.""" + from deepseek_harness import DeepSeekHarness + + with tempfile.TemporaryDirectory(prefix="dsh-sdk-mcp-") as temporary: + root = Path(temporary).resolve() + sessions = root / "sessions" + server_script = root / "mcp_server.py" + server_script.write_text(MCP_SERVER_SCRIPT) + cordis = root / "cordis.yml" + cordis.write_text(mcp_cordis(server_script)) + discovery_log = server_script.with_suffix(".log") + with DeepSeekHarness( + provider="deepseek-official", + model="smoke-model", + cwd=str(root), + session_root=str(sessions), + cordis=str(cordis), + runtime_bin=None if executable is None else str(executable), + api_key="sk-keyless-smoke", + base_url=base_url, + request_timeout_seconds=60, + ) as harness: + result = harness.run(MCP_PROMPT, session_id="mcp-smoke") + + assert result.final_response == MCP_TEXT, result.final_response + assert discovery_log.read_text().splitlines() == [ + "initialize", + "notifications/initialized", + "tools/list", + "tools/call", + ] + assert_session_log(sessions, root, MCP_TEXT, "mcp__fixture__add", "42") + def smoke_sdk_snapshot(base_url: str, executable: Path, update_snapshots: bool) -> None: """Drive and compare the advanced SDK/executable behavioral snapshot.""" @@ -628,7 +910,9 @@ def smoke_sdk_snapshot(base_url: str, executable: Path, update_snapshots: bool) raise AssertionError("second advanced child log has no workflow-subagent result") files = build_snapshot_files(result, logs, child_ids, root) - compare_snapshot_files(files, update_snapshots) + compare_snapshot_files( + files, update_snapshots, ADVANCED_SNAPSHOT_DIRECTORY, ADVANCED_SNAPSHOT_FILENAMES, + ) def smoke_direct(base_url: str, executable: Path) -> None: @@ -802,6 +1086,79 @@ def snapshot_child_ids(result: "RunResult") -> list[str]: return child_ids +def build_minimal_snapshot_files( + requests: list[dict[str, object]], + cwd: Path, +) -> dict[str, str]: + """Render the minimal composition's model-visible surface as expected output. + + Every assembled system prompt, advertised tool schema, and system or user message is + kept verbatim: they carry what the deployment actually shows the model, so a plugin + that contributes an unintended system section or user message cannot pass unnoticed. + Assistant and tool payloads keep only their call identity, and the dynamic + runtime-context snapshot is dropped, because their text differs across the platforms + this expected output must replay on. + """ + snapshot = [] + for body in requests: + messages = body.get("messages") + if not isinstance(messages, list): + raise AssertionError(f"minimal model request has no messages: {body}") + snapshot.append({ + "tools": minimal_snapshot_text(body.get("tools"), cwd), + "messages": [ + minimal_snapshot_message(message, cwd) + for message in messages + if not is_runtime_context_message(message) + ], + }) + return {"model-visible.json": json.dumps(snapshot, indent=2, ensure_ascii=False) + "\n"} + + +def is_runtime_context_message(message: object) -> bool: + """Identify the agent loop's dynamic runtime-context snapshot, current or cleared.""" + return ( + isinstance(message, dict) + and message.get("role") == "user" + and message_text(message.get("content")).startswith(RUNTIME_CONTEXT_PREFIX) + ) + + +def minimal_snapshot_message(message: object, cwd: Path) -> dict[str, object]: + """Reduce one model-visible message to its stable, behavior-carrying parts.""" + if not isinstance(message, dict): + raise AssertionError(f"minimal model request has an invalid message: {message}") + role = message.get("role") + if role in ("system", "user"): + return {"role": role, "text": minimal_snapshot_text(message_text(message.get("content")), cwd)} + if role == "assistant": + calls = message.get("tool_calls") + if not isinstance(calls, list): + raise AssertionError(f"minimal assistant message has no tool calls: {message}") + return { + "role": role, + "toolCalls": [ + {"id": call.get("id"), "name": (call.get("function") or {}).get("name")} + for call in calls + if isinstance(call, dict) + ], + } + if role == "tool": + return {"role": role, "toolCallId": message.get("tool_call_id"), "text": "{{tool-result}}"} + raise AssertionError(f"minimal model request has an unexpected message role: {message}") + + +def minimal_snapshot_text(value: object, cwd: Path) -> object: + """Replace the scenario's temporary working directory everywhere it appears.""" + if isinstance(value, str): + return value.replace(str(cwd), "{{cwd}}") + if isinstance(value, list): + return [minimal_snapshot_text(item, cwd) for item in value] + if isinstance(value, dict): + return {key: minimal_snapshot_text(item, cwd) for key, item in value.items()} + return value + + def build_snapshot_files( result: "RunResult", logs: dict[str, list[dict[str, object]]], @@ -831,15 +1188,17 @@ def build_snapshot_files( files = { "result.json": json.dumps(normalized_result, indent=2, ensure_ascii=False) + "\n", "session.jsonl": render_jsonl( - [normalize_snapshot_value(record, replacements) for record in logs[SNAPSHOT_SESSION_ID]] + project_session_snapshot([ + normalize_snapshot_value(record, replacements) for record in logs[SNAPSHOT_SESSION_ID] + ]) ), } for index, child_id in enumerate(child_ids, start=1): files[f"session.{index}.jsonl"] = render_jsonl( - [normalize_snapshot_value(record, replacements) for record in logs[child_id]] + project_session_snapshot([ + normalize_snapshot_value(record, replacements) for record in logs[child_id] + ]) ) - if tuple(files) != SNAPSHOT_FILENAMES: - raise AssertionError(f"advanced snapshot file set drifted: {tuple(files)}") return files @@ -930,27 +1289,44 @@ def render_jsonl(records: list[object]) -> str: ) -def compare_snapshot_files(files: dict[str, str], update: bool) -> None: - """Write or exactly compare the advanced executable snapshot files.""" +def project_session_snapshot(records: list[dict[str, object]]) -> list[dict[str, object]]: + """Omit storage sequence/time envelopes from snapshot body records.""" + projected = [dict(record) for record in records] + for record in projected[1:]: + for key in ("seq", "time", "seq0", "time0"): + record.pop(key, None) + return projected + + +def compare_snapshot_files( + files: dict[str, str], + update: bool, + directory: Path, + filenames: tuple[str, ...], +) -> None: + """Write or exactly compare one scenario's expected snapshot files.""" + scenario = directory.name + if tuple(files) != filenames: + raise AssertionError(f"{scenario} snapshot builder produced {tuple(files)}, expected {filenames}") if update: - SNAPSHOT_DIRECTORY.mkdir(parents=True, exist_ok=True) + directory.mkdir(parents=True, exist_ok=True) for name, content in files.items(): - (SNAPSHOT_DIRECTORY / name).write_text(content, encoding="utf-8") - print(f"smoke-python-runtime: updated snapshots in {SNAPSHOT_DIRECTORY}") + (directory / name).write_text(content, encoding="utf-8") + print(f"smoke-python-runtime: updated snapshots in {directory}") existing = { path.name - for path in SNAPSHOT_DIRECTORY.iterdir() + for path in directory.iterdir() if path.is_file() - } if SNAPSHOT_DIRECTORY.is_dir() else set() - expected = set(SNAPSHOT_FILENAMES) + } if directory.is_dir() else set() + expected = set(filenames) if existing != expected: raise AssertionError( - "advanced snapshot files differ: " + f"{scenario} snapshot files differ: " f"missing={sorted(expected - existing)}, unexpected={sorted(existing - expected)}" ) for name, actual in files.items(): - expected_text = (SNAPSHOT_DIRECTORY / name).read_text(encoding="utf-8") + expected_text = (directory / name).read_text(encoding="utf-8") if actual == expected_text: continue diff = "".join(difflib.unified_diff( @@ -960,7 +1336,7 @@ def compare_snapshot_files(files: dict[str, str], update: bool) -> None: tofile=f"actual/{name}", )) raise AssertionError( - f"advanced executable snapshot mismatch in {name}; " + f"{scenario} executable snapshot mismatch in {name}; " "rerun with --update-snapshots after reviewing the behavior\n" f"{diff}" ) diff --git a/scripts/snapshots/python-sdk-single-exe/advanced/result.json b/scripts/snapshots/python-sdk-single-exe/advanced/result.json index 2548efb6cb..0889fd6b9f 100644 --- a/scripts/snapshots/python-sdk-single-exe/advanced/result.json +++ b/scripts/snapshots/python-sdk-single-exe/advanced/result.json @@ -134,9 +134,18 @@ } }, { - "type": "assistant/chunk", + "type": "session-log-deepseek/delivery-accepted", "seq": 8, "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 7 + } + }, + { + "type": "assistant/chunk", + "seq": 9, + "time": 0, "data": { "turn": 1, "step": 1, @@ -149,7 +158,7 @@ }, { "type": "assistant/chunk", - "seq": 9, + "seq": 10, "time": 0, "data": { "turn": 1, @@ -165,7 +174,7 @@ }, { "type": "assistant/chunk", - "seq": 10, + "seq": 11, "time": 0, "data": { "turn": 1, @@ -184,7 +193,7 @@ }, { "type": "assistant/chunk", - "seq": 11, + "seq": 12, "time": 0, "data": { "turn": 1, @@ -200,7 +209,7 @@ }, { "type": "assistant/chunk", - "seq": 12, + "seq": 13, "time": 0, "data": { "turn": 1, @@ -215,7 +224,7 @@ }, { "type": "assistant/message", - "seq": 13, + "seq": 14, "time": 0, "data": { "turn": 1, @@ -243,17 +252,17 @@ } }, "sourceEventSeqs": [ - 8, 9, 10, 11, - 12 + 12, + 13 ], "surfaceOp": "append" }, { "type": "tool/call", - "seq": 14, + "seq": 15, "time": 0, "data": { "turn": 1, @@ -265,7 +274,7 @@ }, { "type": "tool/result", - "seq": 15, + "seq": 16, "time": 0, "data": { "turn": 1, @@ -297,13 +306,13 @@ } }, "sourceEventSeqs": [ - 14 + 15 ], "surfaceOp": "append" }, { "type": "step/end", - "seq": 16, + "seq": 17, "time": 0, "data": { "turn": 1, @@ -312,16 +321,25 @@ }, { "type": "step/start", - "seq": 17, + "seq": 18, "time": 0, "data": { "turn": 1, "step": 2 } }, + { + "type": "session-log-deepseek/delivery-accepted", + "seq": 19, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 18 + } + }, { "type": "assistant/chunk", - "seq": 18, + "seq": 20, "time": 0, "data": { "turn": 1, @@ -335,7 +353,7 @@ }, { "type": "assistant/chunk", - "seq": 19, + "seq": 21, "time": 0, "data": { "turn": 1, @@ -351,7 +369,7 @@ }, { "type": "assistant/chunk", - "seq": 20, + "seq": 22, "time": 0, "data": { "turn": 1, @@ -370,7 +388,7 @@ }, { "type": "assistant/chunk", - "seq": 21, + "seq": 23, "time": 0, "data": { "turn": 1, @@ -386,7 +404,7 @@ }, { "type": "assistant/chunk", - "seq": 22, + "seq": 24, "time": 0, "data": { "turn": 1, @@ -401,7 +419,7 @@ }, { "type": "assistant/message", - "seq": 23, + "seq": 25, "time": 0, "data": { "turn": 1, @@ -429,17 +447,17 @@ } }, "sourceEventSeqs": [ - 18, - 19, 20, 21, - 22 + 22, + 23, + 24 ], "surfaceOp": "append" }, { "type": "tool/call", - "seq": 24, + "seq": 26, "time": 0, "data": { "turn": 1, @@ -451,7 +469,7 @@ }, { "type": "tool/result", - "seq": 25, + "seq": 27, "time": 0, "data": { "turn": 1, @@ -484,13 +502,13 @@ } }, "sourceEventSeqs": [ - 24 + 26 ], "surfaceOp": "append" }, { "type": "step/end", - "seq": 26, + "seq": 28, "time": 0, "data": { "turn": 1, @@ -499,7 +517,7 @@ }, { "type": "step/start", - "seq": 27, + "seq": 29, "time": 0, "data": { "turn": 1, @@ -508,7 +526,7 @@ }, { "type": "request/header", - "seq": 28, + "seq": 30, "time": 0, "data": { "header": { @@ -543,9 +561,18 @@ "reason": "change" } }, + { + "type": "session-log-deepseek/delivery-accepted", + "seq": 31, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 30 + } + }, { "type": "assistant/chunk", - "seq": 29, + "seq": 32, "time": 0, "data": { "turn": 1, @@ -559,7 +586,7 @@ }, { "type": "assistant/chunk", - "seq": 30, + "seq": 33, "time": 0, "data": { "turn": 1, @@ -575,7 +602,7 @@ }, { "type": "assistant/chunk", - "seq": 31, + "seq": 34, "time": 0, "data": { "turn": 1, @@ -594,7 +621,7 @@ }, { "type": "assistant/chunk", - "seq": 32, + "seq": 35, "time": 0, "data": { "turn": 1, @@ -610,7 +637,7 @@ }, { "type": "assistant/chunk", - "seq": 33, + "seq": 36, "time": 0, "data": { "turn": 1, @@ -625,7 +652,7 @@ }, { "type": "assistant/message", - "seq": 34, + "seq": 37, "time": 0, "data": { "turn": 1, @@ -653,17 +680,17 @@ } }, "sourceEventSeqs": [ - 29, - 30, - 31, 32, - 33 + 33, + 34, + 35, + 36 ], "surfaceOp": "append" }, { "type": "tool/call", - "seq": 35, + "seq": 38, "time": 0, "data": { "turn": 1, @@ -675,7 +702,7 @@ }, { "type": "tool/code-dispatch-start", - "seq": 36, + "seq": 39, "time": 0, "data": { "rootCallId": "advanced-code", @@ -689,7 +716,7 @@ }, { "type": "tool/code-dispatch", - "seq": 37, + "seq": 40, "time": 0, "data": { "rootCallId": "advanced-code", @@ -710,7 +737,7 @@ }, { "type": "tool/result", - "seq": 38, + "seq": 41, "time": 0, "data": { "turn": 1, @@ -738,13 +765,13 @@ } }, "sourceEventSeqs": [ - 35 + 38 ], "surfaceOp": "append" }, { "type": "step/end", - "seq": 39, + "seq": 42, "time": 0, "data": { "turn": 1, @@ -753,16 +780,25 @@ }, { "type": "step/start", - "seq": 40, + "seq": 43, "time": 0, "data": { "turn": 1, "step": 4 } }, + { + "type": "session-log-deepseek/delivery-accepted", + "seq": 44, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 43 + } + }, { "type": "assistant/chunk", - "seq": 41, + "seq": 45, "time": 0, "data": { "turn": 1, @@ -776,7 +812,7 @@ }, { "type": "assistant/chunk", - "seq": 42, + "seq": 46, "time": 0, "data": { "turn": 1, @@ -792,7 +828,7 @@ }, { "type": "assistant/chunk", - "seq": 43, + "seq": 47, "time": 0, "data": { "turn": 1, @@ -811,7 +847,7 @@ }, { "type": "assistant/chunk", - "seq": 44, + "seq": 48, "time": 0, "data": { "turn": 1, @@ -827,7 +863,7 @@ }, { "type": "assistant/chunk", - "seq": 45, + "seq": 49, "time": 0, "data": { "turn": 1, @@ -842,7 +878,7 @@ }, { "type": "assistant/message", - "seq": 46, + "seq": 50, "time": 0, "data": { "turn": 1, @@ -870,17 +906,17 @@ } }, "sourceEventSeqs": [ - 41, - 42, - 43, - 44, - 45 + 45, + 46, + 47, + 48, + 49 ], "surfaceOp": "append" }, { "type": "tool/call", - "seq": 47, + "seq": 51, "time": 0, "data": { "turn": 1, @@ -892,7 +928,7 @@ }, { "type": "tool/result", - "seq": 48, + "seq": 52, "time": 0, "data": { "turn": 1, @@ -920,13 +956,13 @@ } }, "sourceEventSeqs": [ - 47 + 51 ], "surfaceOp": "append" }, { "type": "step/end", - "seq": 49, + "seq": 53, "time": 0, "data": { "turn": 1, @@ -935,16 +971,25 @@ }, { "type": "step/start", - "seq": 50, + "seq": 54, "time": 0, "data": { "turn": 1, "step": 5 } }, + { + "type": "session-log-deepseek/delivery-accepted", + "seq": 55, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 54 + } + }, { "type": "assistant/chunk", - "seq": 51, + "seq": 56, "time": 0, "data": { "turn": 1, @@ -958,7 +1003,7 @@ }, { "type": "assistant/chunk", - "seq": 52, + "seq": 57, "time": 0, "data": { "turn": 1, @@ -974,7 +1019,7 @@ }, { "type": "assistant/chunk", - "seq": 53, + "seq": 58, "time": 0, "data": { "turn": 1, @@ -993,7 +1038,7 @@ }, { "type": "assistant/chunk", - "seq": 54, + "seq": 59, "time": 0, "data": { "turn": 1, @@ -1009,7 +1054,7 @@ }, { "type": "assistant/chunk", - "seq": 55, + "seq": 60, "time": 0, "data": { "turn": 1, @@ -1024,7 +1069,7 @@ }, { "type": "assistant/message", - "seq": 56, + "seq": 61, "time": 0, "data": { "turn": 1, @@ -1052,17 +1097,17 @@ } }, "sourceEventSeqs": [ - 51, - 52, - 53, - 54, - 55 + 56, + 57, + 58, + 59, + 60 ], "surfaceOp": "append" }, { "type": "tool/call", - "seq": 57, + "seq": 62, "time": 0, "data": { "turn": 1, @@ -1074,7 +1119,7 @@ }, { "type": "tool-workflow/run-start", - "seq": 58, + "seq": 63, "time": 0, "data": { "runId": "{{workflow-run}}", @@ -1083,7 +1128,7 @@ }, { "type": "tool-workflow/agent-start", - "seq": 59, + "seq": 64, "time": 0, "data": { "runId": "{{workflow-run}}", @@ -1095,7 +1140,7 @@ }, { "type": "tool-workflow/agent-end", - "seq": 60, + "seq": 65, "time": 0, "data": { "runId": "{{workflow-run}}", @@ -1105,7 +1150,7 @@ }, { "type": "tool-workflow/run-end", - "seq": 61, + "seq": 66, "time": 0, "data": { "runId": "{{workflow-run}}", @@ -1114,7 +1159,7 @@ }, { "type": "tool/result", - "seq": 62, + "seq": 67, "time": 0, "data": { "turn": 1, @@ -1142,13 +1187,13 @@ } }, "sourceEventSeqs": [ - 57 + 62 ], "surfaceOp": "append" }, { "type": "step/end", - "seq": 63, + "seq": 68, "time": 0, "data": { "turn": 1, @@ -1157,16 +1202,25 @@ }, { "type": "step/start", - "seq": 64, + "seq": 69, "time": 0, "data": { "turn": 1, "step": 6 } }, + { + "type": "session-log-deepseek/delivery-accepted", + "seq": 70, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 69 + } + }, { "type": "assistant/chunk", - "seq": 65, + "seq": 71, "time": 0, "data": { "turn": 1, @@ -1180,7 +1234,7 @@ }, { "type": "assistant/chunk", - "seq": 66, + "seq": 72, "time": 0, "data": { "turn": 1, @@ -1196,7 +1250,7 @@ }, { "type": "assistant/chunk", - "seq": 67, + "seq": 73, "time": 0, "data": { "turn": 1, @@ -1215,7 +1269,7 @@ }, { "type": "assistant/chunk", - "seq": 68, + "seq": 74, "time": 0, "data": { "turn": 1, @@ -1231,7 +1285,7 @@ }, { "type": "assistant/chunk", - "seq": 69, + "seq": 75, "time": 0, "data": { "turn": 1, @@ -1246,7 +1300,7 @@ }, { "type": "assistant/message", - "seq": 70, + "seq": 76, "time": 0, "data": { "turn": 1, @@ -1274,17 +1328,17 @@ } }, "sourceEventSeqs": [ - 65, - 66, - 67, - 68, - 69 + 71, + 72, + 73, + 74, + 75 ], "surfaceOp": "append" }, { "type": "tool/call", - "seq": 71, + "seq": 77, "time": 0, "data": { "turn": 1, @@ -1296,7 +1350,7 @@ }, { "type": "tool/result", - "seq": 72, + "seq": 78, "time": 0, "data": { "turn": 1, @@ -1324,13 +1378,13 @@ } }, "sourceEventSeqs": [ - 71 + 77 ], "surfaceOp": "append" }, { "type": "step/end", - "seq": 73, + "seq": 79, "time": 0, "data": { "turn": 1, @@ -1339,7 +1393,7 @@ }, { "type": "step/start", - "seq": 74, + "seq": 80, "time": 0, "data": { "turn": 1, @@ -1348,7 +1402,7 @@ }, { "type": "request/header", - "seq": 75, + "seq": 81, "time": 0, "data": { "header": { @@ -1382,9 +1436,18 @@ "reason": "change" } }, + { + "type": "session-log-deepseek/delivery-accepted", + "seq": 82, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 81 + } + }, { "type": "assistant/chunk", - "seq": 76, + "seq": 83, "time": 0, "data": { "turn": 1, @@ -1398,7 +1461,7 @@ }, { "type": "assistant/chunk", - "seq": 77, + "seq": 84, "time": 0, "data": { "turn": 1, @@ -1412,7 +1475,7 @@ }, { "type": "assistant/chunk", - "seq": 78, + "seq": 85, "time": 0, "data": { "turn": 1, @@ -1429,7 +1492,7 @@ }, { "type": "assistant/chunk", - "seq": 79, + "seq": 86, "time": 0, "data": { "turn": 1, @@ -1445,7 +1508,7 @@ }, { "type": "assistant/chunk", - "seq": 80, + "seq": 87, "time": 0, "data": { "turn": 1, @@ -1460,7 +1523,7 @@ }, { "type": "assistant/message", - "seq": 81, + "seq": 88, "time": 0, "data": { "turn": 1, @@ -1486,17 +1549,17 @@ } }, "sourceEventSeqs": [ - 76, - 77, - 78, - 79, - 80 + 83, + 84, + 85, + 86, + 87 ], "surfaceOp": "append" }, { "type": "step/end", - "seq": 82, + "seq": 89, "time": 0, "data": { "turn": 1, @@ -1505,7 +1568,7 @@ }, { "type": "turn/end", - "seq": 83, + "seq": 90, "time": 0, "data": { "turn": 1, @@ -1707,9 +1770,24 @@ "payload": { "sessionId": "{{parent}}", "event": { - "type": "assistant/chunk", + "type": "session-log-deepseek/delivery-accepted", "seq": 8, "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 7 + } + } + } + }, + { + "method": "session.event", + "payload": { + "sessionId": "{{parent}}", + "event": { + "type": "assistant/chunk", + "seq": 9, + "time": 0, "data": { "turn": 1, "step": 1, @@ -1728,7 +1806,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 9, + "seq": 10, "time": 0, "data": { "turn": 1, @@ -1750,7 +1828,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 10, + "seq": 11, "time": 0, "data": { "turn": 1, @@ -1775,7 +1853,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 11, + "seq": 12, "time": 0, "data": { "turn": 1, @@ -1797,7 +1875,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 12, + "seq": 13, "time": 0, "data": { "turn": 1, @@ -1818,7 +1896,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/message", - "seq": 13, + "seq": 14, "time": 0, "data": { "turn": 1, @@ -1846,11 +1924,11 @@ } }, "sourceEventSeqs": [ - 8, 9, 10, 11, - 12 + 12, + 13 ], "surfaceOp": "append" } @@ -1862,7 +1940,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/call", - "seq": 14, + "seq": 15, "time": 0, "data": { "turn": 1, @@ -1880,7 +1958,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/result", - "seq": 15, + "seq": 16, "time": 0, "data": { "turn": 1, @@ -1912,7 +1990,7 @@ } }, "sourceEventSeqs": [ - 14 + 15 ], "surfaceOp": "append" } @@ -1924,7 +2002,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/end", - "seq": 16, + "seq": 17, "time": 0, "data": { "turn": 1, @@ -1939,7 +2017,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/start", - "seq": 17, + "seq": 18, "time": 0, "data": { "turn": 1, @@ -1948,13 +2026,28 @@ } } }, + { + "method": "session.event", + "payload": { + "sessionId": "{{parent}}", + "event": { + "type": "session-log-deepseek/delivery-accepted", + "seq": 19, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 18 + } + } + } + }, { "method": "session.event", "payload": { "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 18, + "seq": 20, "time": 0, "data": { "turn": 1, @@ -1974,7 +2067,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 19, + "seq": 21, "time": 0, "data": { "turn": 1, @@ -1996,7 +2089,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 20, + "seq": 22, "time": 0, "data": { "turn": 1, @@ -2021,7 +2114,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 21, + "seq": 23, "time": 0, "data": { "turn": 1, @@ -2043,7 +2136,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 22, + "seq": 24, "time": 0, "data": { "turn": 1, @@ -2064,7 +2157,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/message", - "seq": 23, + "seq": 25, "time": 0, "data": { "turn": 1, @@ -2092,11 +2185,11 @@ } }, "sourceEventSeqs": [ - 18, - 19, 20, 21, - 22 + 22, + 23, + 24 ], "surfaceOp": "append" } @@ -2108,7 +2201,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/call", - "seq": 24, + "seq": 26, "time": 0, "data": { "turn": 1, @@ -2126,7 +2219,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/result", - "seq": 25, + "seq": 27, "time": 0, "data": { "turn": 1, @@ -2159,7 +2252,7 @@ } }, "sourceEventSeqs": [ - 24 + 26 ], "surfaceOp": "append" } @@ -2171,7 +2264,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/end", - "seq": 26, + "seq": 28, "time": 0, "data": { "turn": 1, @@ -2186,7 +2279,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/start", - "seq": 27, + "seq": 29, "time": 0, "data": { "turn": 1, @@ -2201,7 +2294,7 @@ "sessionId": "{{parent}}", "event": { "type": "request/header", - "seq": 28, + "seq": 30, "time": 0, "data": { "header": { @@ -2238,13 +2331,28 @@ } } }, + { + "method": "session.event", + "payload": { + "sessionId": "{{parent}}", + "event": { + "type": "session-log-deepseek/delivery-accepted", + "seq": 31, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 30 + } + } + } + }, { "method": "session.event", "payload": { "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 29, + "seq": 32, "time": 0, "data": { "turn": 1, @@ -2264,7 +2372,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 30, + "seq": 33, "time": 0, "data": { "turn": 1, @@ -2286,7 +2394,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 31, + "seq": 34, "time": 0, "data": { "turn": 1, @@ -2311,7 +2419,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 32, + "seq": 35, "time": 0, "data": { "turn": 1, @@ -2333,7 +2441,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 33, + "seq": 36, "time": 0, "data": { "turn": 1, @@ -2354,7 +2462,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/message", - "seq": 34, + "seq": 37, "time": 0, "data": { "turn": 1, @@ -2382,11 +2490,11 @@ } }, "sourceEventSeqs": [ - 29, - 30, - 31, 32, - 33 + 33, + 34, + 35, + 36 ], "surfaceOp": "append" } @@ -2398,7 +2506,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/call", - "seq": 35, + "seq": 38, "time": 0, "data": { "turn": 1, @@ -2416,7 +2524,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/code-dispatch-start", - "seq": 36, + "seq": 39, "time": 0, "data": { "rootCallId": "advanced-code", @@ -2436,7 +2544,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/code-dispatch", - "seq": 37, + "seq": 40, "time": 0, "data": { "rootCallId": "advanced-code", @@ -2463,7 +2571,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/result", - "seq": 38, + "seq": 41, "time": 0, "data": { "turn": 1, @@ -2491,7 +2599,7 @@ } }, "sourceEventSeqs": [ - 35 + 38 ], "surfaceOp": "append" } @@ -2503,7 +2611,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/end", - "seq": 39, + "seq": 42, "time": 0, "data": { "turn": 1, @@ -2518,7 +2626,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/start", - "seq": 40, + "seq": 43, "time": 0, "data": { "turn": 1, @@ -2527,13 +2635,28 @@ } } }, + { + "method": "session.event", + "payload": { + "sessionId": "{{parent}}", + "event": { + "type": "session-log-deepseek/delivery-accepted", + "seq": 44, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 43 + } + } + } + }, { "method": "session.event", "payload": { "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 41, + "seq": 45, "time": 0, "data": { "turn": 1, @@ -2553,7 +2676,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 42, + "seq": 46, "time": 0, "data": { "turn": 1, @@ -2575,7 +2698,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 43, + "seq": 47, "time": 0, "data": { "turn": 1, @@ -2600,7 +2723,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 44, + "seq": 48, "time": 0, "data": { "turn": 1, @@ -2622,7 +2745,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 45, + "seq": 49, "time": 0, "data": { "turn": 1, @@ -2643,7 +2766,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/message", - "seq": 46, + "seq": 50, "time": 0, "data": { "turn": 1, @@ -2671,11 +2794,11 @@ } }, "sourceEventSeqs": [ - 41, - 42, - 43, - 44, - 45 + 45, + 46, + 47, + 48, + 49 ], "surfaceOp": "append" } @@ -2687,7 +2810,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/call", - "seq": 47, + "seq": 51, "time": 0, "data": { "turn": 1, @@ -2948,9 +3071,24 @@ "payload": { "sessionId": "{{child-1}}", "event": { - "type": "assistant/chunk", + "type": "session-log-deepseek/delivery-accepted", "seq": 10, "time": 0, + "data": { + "sessionId": "{{child-1}}", + "throughSeq": 9 + } + } + } + }, + { + "method": "session.event", + "payload": { + "sessionId": "{{child-1}}", + "event": { + "type": "assistant/chunk", + "seq": 11, + "time": 0, "data": { "turn": 1, "step": 1, @@ -2969,7 +3107,7 @@ "sessionId": "{{child-1}}", "event": { "type": "assistant/chunk", - "seq": 11, + "seq": 12, "time": 0, "data": { "turn": 1, @@ -2989,7 +3127,7 @@ "sessionId": "{{child-1}}", "event": { "type": "assistant/chunk", - "seq": 12, + "seq": 13, "time": 0, "data": { "turn": 1, @@ -3012,7 +3150,7 @@ "sessionId": "{{child-1}}", "event": { "type": "assistant/chunk", - "seq": 13, + "seq": 14, "time": 0, "data": { "turn": 1, @@ -3034,7 +3172,7 @@ "sessionId": "{{child-1}}", "event": { "type": "assistant/chunk", - "seq": 14, + "seq": 15, "time": 0, "data": { "turn": 1, @@ -3055,7 +3193,7 @@ "sessionId": "{{child-1}}", "event": { "type": "assistant/message", - "seq": 15, + "seq": 16, "time": 0, "data": { "turn": 1, @@ -3081,11 +3219,11 @@ } }, "sourceEventSeqs": [ - 10, 11, 12, 13, - 14 + 14, + 15 ], "surfaceOp": "append" } @@ -3097,7 +3235,7 @@ "sessionId": "{{child-1}}", "event": { "type": "step/end", - "seq": 16, + "seq": 17, "time": 0, "data": { "turn": 1, @@ -3112,7 +3250,7 @@ "sessionId": "{{child-1}}", "event": { "type": "turn/end", - "seq": 17, + "seq": 18, "time": 0, "data": { "turn": 1, @@ -3153,7 +3291,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/result", - "seq": 48, + "seq": 52, "time": 0, "data": { "turn": 1, @@ -3181,7 +3319,7 @@ } }, "sourceEventSeqs": [ - 47 + 51 ], "surfaceOp": "append" } @@ -3193,7 +3331,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/end", - "seq": 49, + "seq": 53, "time": 0, "data": { "turn": 1, @@ -3208,7 +3346,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/start", - "seq": 50, + "seq": 54, "time": 0, "data": { "turn": 1, @@ -3217,13 +3355,28 @@ } } }, + { + "method": "session.event", + "payload": { + "sessionId": "{{parent}}", + "event": { + "type": "session-log-deepseek/delivery-accepted", + "seq": 55, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 54 + } + } + } + }, { "method": "session.event", "payload": { "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 51, + "seq": 56, "time": 0, "data": { "turn": 1, @@ -3243,7 +3396,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 52, + "seq": 57, "time": 0, "data": { "turn": 1, @@ -3265,7 +3418,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 53, + "seq": 58, "time": 0, "data": { "turn": 1, @@ -3290,7 +3443,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 54, + "seq": 59, "time": 0, "data": { "turn": 1, @@ -3312,7 +3465,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 55, + "seq": 60, "time": 0, "data": { "turn": 1, @@ -3333,7 +3486,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/message", - "seq": 56, + "seq": 61, "time": 0, "data": { "turn": 1, @@ -3361,11 +3514,11 @@ } }, "sourceEventSeqs": [ - 51, - 52, - 53, - 54, - 55 + 56, + 57, + 58, + 59, + 60 ], "surfaceOp": "append" } @@ -3377,7 +3530,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/call", - "seq": 57, + "seq": 62, "time": 0, "data": { "turn": 1, @@ -3395,7 +3548,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool-workflow/run-start", - "seq": 58, + "seq": 63, "time": 0, "data": { "runId": "{{workflow-run}}", @@ -3653,7 +3806,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool-workflow/agent-start", - "seq": 59, + "seq": 64, "time": 0, "data": { "runId": "{{workflow-run}}", @@ -3670,9 +3823,24 @@ "payload": { "sessionId": "{{child-2}}", "event": { - "type": "assistant/chunk", + "type": "session-log-deepseek/delivery-accepted", "seq": 10, "time": 0, + "data": { + "sessionId": "{{child-2}}", + "throughSeq": 9 + } + } + } + }, + { + "method": "session.event", + "payload": { + "sessionId": "{{child-2}}", + "event": { + "type": "assistant/chunk", + "seq": 11, + "time": 0, "data": { "turn": 1, "step": 1, @@ -3691,7 +3859,7 @@ "sessionId": "{{child-2}}", "event": { "type": "assistant/chunk", - "seq": 11, + "seq": 12, "time": 0, "data": { "turn": 1, @@ -3711,7 +3879,7 @@ "sessionId": "{{child-2}}", "event": { "type": "assistant/chunk", - "seq": 12, + "seq": 13, "time": 0, "data": { "turn": 1, @@ -3734,7 +3902,7 @@ "sessionId": "{{child-2}}", "event": { "type": "assistant/chunk", - "seq": 13, + "seq": 14, "time": 0, "data": { "turn": 1, @@ -3756,7 +3924,7 @@ "sessionId": "{{child-2}}", "event": { "type": "assistant/chunk", - "seq": 14, + "seq": 15, "time": 0, "data": { "turn": 1, @@ -3777,7 +3945,7 @@ "sessionId": "{{child-2}}", "event": { "type": "assistant/message", - "seq": 15, + "seq": 16, "time": 0, "data": { "turn": 1, @@ -3803,11 +3971,11 @@ } }, "sourceEventSeqs": [ - 10, 11, 12, 13, - 14 + 14, + 15 ], "surfaceOp": "append" } @@ -3819,7 +3987,7 @@ "sessionId": "{{child-2}}", "event": { "type": "step/end", - "seq": 16, + "seq": 17, "time": 0, "data": { "turn": 1, @@ -3834,7 +4002,7 @@ "sessionId": "{{child-2}}", "event": { "type": "turn/end", - "seq": 17, + "seq": 18, "time": 0, "data": { "turn": 1, @@ -3875,7 +4043,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool-workflow/agent-end", - "seq": 60, + "seq": 65, "time": 0, "data": { "runId": "{{workflow-run}}", @@ -3891,7 +4059,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool-workflow/run-end", - "seq": 61, + "seq": 66, "time": 0, "data": { "runId": "{{workflow-run}}", @@ -3906,7 +4074,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/result", - "seq": 62, + "seq": 67, "time": 0, "data": { "turn": 1, @@ -3934,7 +4102,7 @@ } }, "sourceEventSeqs": [ - 57 + 62 ], "surfaceOp": "append" } @@ -3946,7 +4114,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/end", - "seq": 63, + "seq": 68, "time": 0, "data": { "turn": 1, @@ -3961,7 +4129,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/start", - "seq": 64, + "seq": 69, "time": 0, "data": { "turn": 1, @@ -3970,13 +4138,28 @@ } } }, + { + "method": "session.event", + "payload": { + "sessionId": "{{parent}}", + "event": { + "type": "session-log-deepseek/delivery-accepted", + "seq": 70, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 69 + } + } + } + }, { "method": "session.event", "payload": { "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 65, + "seq": 71, "time": 0, "data": { "turn": 1, @@ -3996,7 +4179,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 66, + "seq": 72, "time": 0, "data": { "turn": 1, @@ -4018,7 +4201,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 67, + "seq": 73, "time": 0, "data": { "turn": 1, @@ -4043,7 +4226,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 68, + "seq": 74, "time": 0, "data": { "turn": 1, @@ -4065,7 +4248,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 69, + "seq": 75, "time": 0, "data": { "turn": 1, @@ -4086,7 +4269,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/message", - "seq": 70, + "seq": 76, "time": 0, "data": { "turn": 1, @@ -4114,11 +4297,11 @@ } }, "sourceEventSeqs": [ - 65, - 66, - 67, - 68, - 69 + 71, + 72, + 73, + 74, + 75 ], "surfaceOp": "append" } @@ -4130,7 +4313,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/call", - "seq": 71, + "seq": 77, "time": 0, "data": { "turn": 1, @@ -4148,7 +4331,7 @@ "sessionId": "{{parent}}", "event": { "type": "tool/result", - "seq": 72, + "seq": 78, "time": 0, "data": { "turn": 1, @@ -4176,7 +4359,7 @@ } }, "sourceEventSeqs": [ - 71 + 77 ], "surfaceOp": "append" } @@ -4188,7 +4371,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/end", - "seq": 73, + "seq": 79, "time": 0, "data": { "turn": 1, @@ -4203,7 +4386,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/start", - "seq": 74, + "seq": 80, "time": 0, "data": { "turn": 1, @@ -4218,7 +4401,7 @@ "sessionId": "{{parent}}", "event": { "type": "request/header", - "seq": 75, + "seq": 81, "time": 0, "data": { "header": { @@ -4254,13 +4437,28 @@ } } }, + { + "method": "session.event", + "payload": { + "sessionId": "{{parent}}", + "event": { + "type": "session-log-deepseek/delivery-accepted", + "seq": 82, + "time": 0, + "data": { + "sessionId": "{{parent}}", + "throughSeq": 81 + } + } + } + }, { "method": "session.event", "payload": { "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 76, + "seq": 83, "time": 0, "data": { "turn": 1, @@ -4280,7 +4478,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 77, + "seq": 84, "time": 0, "data": { "turn": 1, @@ -4300,7 +4498,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 78, + "seq": 85, "time": 0, "data": { "turn": 1, @@ -4323,7 +4521,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 79, + "seq": 86, "time": 0, "data": { "turn": 1, @@ -4345,7 +4543,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/chunk", - "seq": 80, + "seq": 87, "time": 0, "data": { "turn": 1, @@ -4366,7 +4564,7 @@ "sessionId": "{{parent}}", "event": { "type": "assistant/message", - "seq": 81, + "seq": 88, "time": 0, "data": { "turn": 1, @@ -4392,11 +4590,11 @@ } }, "sourceEventSeqs": [ - 76, - 77, - 78, - 79, - 80 + 83, + 84, + 85, + 86, + 87 ], "surfaceOp": "append" } @@ -4408,7 +4606,7 @@ "sessionId": "{{parent}}", "event": { "type": "step/end", - "seq": 82, + "seq": 89, "time": 0, "data": { "turn": 1, @@ -4423,7 +4621,7 @@ "sessionId": "{{parent}}", "event": { "type": "turn/end", - "seq": 83, + "seq": 90, "time": 0, "data": { "turn": 1, diff --git a/scripts/snapshots/python-sdk-single-exe/advanced/session.1.jsonl b/scripts/snapshots/python-sdk-single-exe/advanced/session.1.jsonl index e18ff2c585..6e2f62bfe1 100644 --- a/scripts/snapshots/python-sdk-single-exe/advanced/session.1.jsonl +++ b/scripts/snapshots/python-sdk-single-exe/advanced/session.1.jsonl @@ -1,19 +1,20 @@ {"type":"session","version":0,"id":"{{child-1}}","createdAt":0,"cwd":"{{cwd}}","parentSession":"{{parent}}","origin":"subagent","delegationDepth":1} -{"type":"agent/inbox/spliced","seq":0,"time":0,"data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Reply with exactly DIRECT_CHILD_OK and nothing else."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"}]}} -{"type":"turn/start","seq":1,"time":0,"data":{"turn":1}} -{"type":"agent/inbox/spliced","seq":2,"time":0,"data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} -{"type":"subagent/descriptor","seq":3,"time":0,"data":{"version":2,"mode":"one-shot","provider":"spawn","label":"Check direct child"}} -{"type":"step/start","seq":4,"time":0,"data":{"turn":1,"step":1}} -{"type":"user/message","seq":5,"time":0,"data":{"content":[{"type":"text","text":"Reply with exactly DIRECT_CHILD_OK and nothing else."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} -{"type":"user/message","seq":6,"time":0,"data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nYou are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"subagent:delegation","text":"You are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}]},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} -{"type":"session/title","seq":7,"time":0,"data":{"title":"Reply with exactly DIRECT_CHILD_OK and","messageSeqs":[5],"source":{"kind":"fallback"}}} -{"type":"request/header","seq":8,"time":0,"data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","snapshot_double","subagent","workflow"]},"reason":"initial"}} -{"type":"request/context","seq":9,"time":0,"data":{"provider":"deepseek-official","model":"smoke-model","contextWindow":1000000}} -{"type":"assistant/chunk","seq":10,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} -{"type":"assistant/chunk","seq":11,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"DIRECT_CHILD_OK"}}} -{"type":"assistant/chunk","seq":12,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DIRECT_CHILD_OK"}}}} -{"type":"assistant/chunk","seq":13,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} -{"type":"assistant/chunk","seq":14,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} -{"type":"assistant/message","seq":15,"time":0,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"DIRECT_CHILD_OK"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[10,11,12,13,14],"surfaceOp":"append"} -{"type":"step/end","seq":16,"time":0,"data":{"turn":1,"step":1}} -{"type":"turn/end","seq":17,"time":0,"data":{"turn":1,"reason":{"kind":"completed"}}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Reply with exactly DIRECT_CHILD_OK and nothing else."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"}]}} +{"type":"turn/start","data":{"turn":1}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"subagent/descriptor","data":{"version":2,"mode":"one-shot","provider":"spawn","label":"Check direct child"}} +{"type":"step/start","data":{"turn":1,"step":1}} +{"type":"user/message","data":{"content":[{"type":"text","text":"Reply with exactly DIRECT_CHILD_OK and nothing else."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} +{"type":"user/message","data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nYou are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"subagent:delegation","text":"You are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}]},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} +{"type":"session/title","data":{"title":"Reply with exactly DIRECT_CHILD_OK and","messageSeqs":[5],"source":{"kind":"fallback"}}} +{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","snapshot_double","subagent","workflow"]},"reason":"initial"}} +{"type":"request/context","data":{"provider":"deepseek-official","model":"smoke-model","contextWindow":1000000}} +{"type":"session-log-deepseek/delivery-accepted","data":{"sessionId":"{{child-1}}","throughSeq":9}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"DIRECT_CHILD_OK"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DIRECT_CHILD_OK"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"DIRECT_CHILD_OK"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[11,12,13,14,15],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":1}} +{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/scripts/snapshots/python-sdk-single-exe/advanced/session.2.jsonl b/scripts/snapshots/python-sdk-single-exe/advanced/session.2.jsonl index a407d19417..e11cec5548 100644 --- a/scripts/snapshots/python-sdk-single-exe/advanced/session.2.jsonl +++ b/scripts/snapshots/python-sdk-single-exe/advanced/session.2.jsonl @@ -1,19 +1,20 @@ {"type":"session","version":0,"id":"{{child-2}}","createdAt":0,"cwd":"{{cwd}}","parentSession":"{{parent}}","origin":"subagent","delegationDepth":1} -{"type":"agent/inbox/spliced","seq":0,"time":0,"data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Reply with exactly WORKFLOW_CHILD_OK and nothing else."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"}]}} -{"type":"turn/start","seq":1,"time":0,"data":{"turn":1}} -{"type":"agent/inbox/spliced","seq":2,"time":0,"data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} -{"type":"subagent/descriptor","seq":3,"time":0,"data":{"version":2,"mode":"one-shot","provider":"spawn"}} -{"type":"step/start","seq":4,"time":0,"data":{"turn":1,"step":1}} -{"type":"user/message","seq":5,"time":0,"data":{"content":[{"type":"text","text":"Reply with exactly WORKFLOW_CHILD_OK and nothing else."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} -{"type":"user/message","seq":6,"time":0,"data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nYou are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"subagent:delegation","text":"You are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}]},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} -{"type":"session/title","seq":7,"time":0,"data":{"title":"Reply with exactly WORKFLOW_CHILD_OK and","messageSeqs":[5],"source":{"kind":"fallback"}}} -{"type":"request/header","seq":8,"time":0,"data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","snapshot_double","subagent","workflow"]},"reason":"initial"}} -{"type":"request/context","seq":9,"time":0,"data":{"provider":"deepseek-official","model":"smoke-model","contextWindow":1000000}} -{"type":"assistant/chunk","seq":10,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} -{"type":"assistant/chunk","seq":11,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"WORKFLOW_CHILD_OK"}}} -{"type":"assistant/chunk","seq":12,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"WORKFLOW_CHILD_OK"}}}} -{"type":"assistant/chunk","seq":13,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} -{"type":"assistant/chunk","seq":14,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} -{"type":"assistant/message","seq":15,"time":0,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"WORKFLOW_CHILD_OK"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[10,11,12,13,14],"surfaceOp":"append"} -{"type":"step/end","seq":16,"time":0,"data":{"turn":1,"step":1}} -{"type":"turn/end","seq":17,"time":0,"data":{"turn":1,"reason":{"kind":"completed"}}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Reply with exactly WORKFLOW_CHILD_OK and nothing else."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"}]}} +{"type":"turn/start","data":{"turn":1}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"subagent/descriptor","data":{"version":2,"mode":"one-shot","provider":"spawn"}} +{"type":"step/start","data":{"turn":1,"step":1}} +{"type":"user/message","data":{"content":[{"type":"text","text":"Reply with exactly WORKFLOW_CHILD_OK and nothing else."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} +{"type":"user/message","data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nYou are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"subagent:delegation","text":"You are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}]},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} +{"type":"session/title","data":{"title":"Reply with exactly WORKFLOW_CHILD_OK and","messageSeqs":[5],"source":{"kind":"fallback"}}} +{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","snapshot_double","subagent","workflow"]},"reason":"initial"}} +{"type":"request/context","data":{"provider":"deepseek-official","model":"smoke-model","contextWindow":1000000}} +{"type":"session-log-deepseek/delivery-accepted","data":{"sessionId":"{{child-2}}","throughSeq":9}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"WORKFLOW_CHILD_OK"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"WORKFLOW_CHILD_OK"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"WORKFLOW_CHILD_OK"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[11,12,13,14,15],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":1}} +{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/scripts/snapshots/python-sdk-single-exe/advanced/session.jsonl b/scripts/snapshots/python-sdk-single-exe/advanced/session.jsonl index 9b37ce12c6..2e65037b37 100644 --- a/scripts/snapshots/python-sdk-single-exe/advanced/session.jsonl +++ b/scripts/snapshots/python-sdk-single-exe/advanced/session.jsonl @@ -1,85 +1,92 @@ {"type":"session","version":0,"id":"{{parent}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} -{"type":"agent/inbox/spliced","seq":0,"time":0,"data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Run the advanced packaged-runtime snapshot scenario."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"}]}} -{"type":"turn/start","seq":1,"time":0,"data":{"turn":1}} -{"type":"agent/inbox/spliced","seq":2,"time":0,"data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} -{"type":"step/start","seq":3,"time":0,"data":{"turn":1,"step":1}} -{"type":"user/message","seq":4,"time":0,"data":{"content":[{"type":"text","text":"Run the advanced packaged-runtime snapshot scenario."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} -{"type":"session/title","seq":5,"time":0,"data":{"title":"Run the advanced packaged-runtime snapsh","messageSeqs":[4],"source":{"kind":"fallback"}}} -{"type":"request/header","seq":6,"time":0,"data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","subagent","workflow"]},"reason":"initial"}} -{"type":"request/context","seq":7,"time":0,"data":{"provider":"deepseek-official","model":"smoke-model","contextWindow":1000000}} -{"type":"assistant/chunk","seq":8,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} -{"type":"assistant/chunk","seq":9,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-define","name":"cordis_define","argumentsDelta":"{\"plugin\": {\"kind\": \"new\", \"idPrefix\": \"snap\"}, \"name\": \"Snapshot Double\", \"purpose\": \"Expose a deterministic doubling tool for executable snapshot verification.\", \"code\": {\"host\": \"return (ctx) => {\\n harness.registerTool(ctx, harness.defineTool({\\n name: 'snapshot_double',\\n description: 'Double a number for executable snapshot verification.',\\n parameters: { value: { type: 'number', required: true } },\\n output: {\\n schema: { type: 'number' },\\n render(_args, value) {\\n return [{ type: 'text', text: String(value) }]\\n }\\n },\\n async execute(args) {\\n return args.value * 2\\n }\\n }))\\n}\\n\"}}"}}} -{"type":"assistant/chunk","seq":10,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-define","name":"cordis_define","arguments":"{\"plugin\": {\"kind\": \"new\", \"idPrefix\": \"snap\"}, \"name\": \"Snapshot Double\", \"purpose\": \"Expose a deterministic doubling tool for executable snapshot verification.\", \"code\": {\"host\": \"return (ctx) => {\\n harness.registerTool(ctx, harness.defineTool({\\n name: 'snapshot_double',\\n description: 'Double a number for executable snapshot verification.',\\n parameters: { value: { type: 'number', required: true } },\\n output: {\\n schema: { type: 'number' },\\n render(_args, value) {\\n return [{ type: 'text', text: String(value) }]\\n }\\n },\\n async execute(args) {\\n return args.value * 2\\n }\\n }))\\n}\\n\"}}"}}}} -{"type":"assistant/chunk","seq":11,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} -{"type":"assistant/chunk","seq":12,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} -{"type":"assistant/message","seq":13,"time":0,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-define","name":"cordis_define","arguments":"{\"plugin\": {\"kind\": \"new\", \"idPrefix\": \"snap\"}, \"name\": \"Snapshot Double\", \"purpose\": \"Expose a deterministic doubling tool for executable snapshot verification.\", \"code\": {\"host\": \"return (ctx) => {\\n harness.registerTool(ctx, harness.defineTool({\\n name: 'snapshot_double',\\n description: 'Double a number for executable snapshot verification.',\\n parameters: { value: { type: 'number', required: true } },\\n output: {\\n schema: { type: 'number' },\\n render(_args, value) {\\n return [{ type: 'text', text: String(value) }]\\n }\\n },\\n async execute(args) {\\n return args.value * 2\\n }\\n }))\\n}\\n\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[8,9,10,11,12],"surfaceOp":"append"} -{"type":"tool/call","seq":14,"time":0,"data":{"turn":1,"step":1,"callId":"advanced-define","name":"cordis_define","arguments":"{\"plugin\": {\"kind\": \"new\", \"idPrefix\": \"snap\"}, \"name\": \"Snapshot Double\", \"purpose\": \"Expose a deterministic doubling tool for executable snapshot verification.\", \"code\": {\"host\": \"return (ctx) => {\\n harness.registerTool(ctx, harness.defineTool({\\n name: 'snapshot_double',\\n description: 'Double a number for executable snapshot verification.',\\n parameters: { value: { type: 'number', required: true } },\\n output: {\\n schema: { type: 'number' },\\n render(_args, value) {\\n return [{ type: 'text', text: String(value) }]\\n }\\n },\\n async execute(args) {\\n return args.value * 2\\n }\\n }))\\n}\\n\"}}"}} -{"type":"tool/result","seq":15,"time":0,"data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"advanced-define"},"content":[{"type":"tool-result","toolCallId":"advanced-define","content":[{"type":"text","text":"Defined snap-1/pkg-1 (Snapshot Double); it is not running yet. Use cordis_run to activate this Package."}],"isError":false}],"role":"user","id":"{{messageId}}"},"meta":{"pluginId":"snap-1","packageId":"pkg-1"}},"sourceEventSeqs":[14],"surfaceOp":"append"} -{"type":"step/end","seq":16,"time":0,"data":{"turn":1,"step":1}} -{"type":"step/start","seq":17,"time":0,"data":{"turn":1,"step":2}} -{"type":"assistant/chunk","seq":18,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} -{"type":"assistant/chunk","seq":19,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-run","name":"cordis_run","argumentsDelta":"{\"pluginId\": \"snap-1\", \"packageId\": \"pkg-1\", \"mode\": \"run\"}"}}} -{"type":"assistant/chunk","seq":20,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-run","name":"cordis_run","arguments":"{\"pluginId\": \"snap-1\", \"packageId\": \"pkg-1\", \"mode\": \"run\"}"}}}} -{"type":"assistant/chunk","seq":21,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} -{"type":"assistant/chunk","seq":22,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} -{"type":"assistant/message","seq":23,"time":0,"data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-run","name":"cordis_run","arguments":"{\"pluginId\": \"snap-1\", \"packageId\": \"pkg-1\", \"mode\": \"run\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[18,19,20,21,22],"surfaceOp":"append"} -{"type":"tool/call","seq":24,"time":0,"data":{"turn":1,"step":2,"callId":"advanced-run","name":"cordis_run","arguments":"{\"pluginId\": \"snap-1\", \"packageId\": \"pkg-1\", \"mode\": \"run\"}"}} -{"type":"tool/result","seq":25,"time":0,"data":{"turn":1,"step":2,"message":{"source":{"kind":"tool","callId":"advanced-run"},"content":[{"type":"tool-result","toolCallId":"advanced-run","content":[{"type":"text","text":"snap-1/pkg-1 is running (run-1)."}],"isError":false}],"role":"user","id":"{{messageId}}"},"meta":{"pluginId":"snap-1","packageId":"pkg-1","pluginRunId":"run-1"}},"sourceEventSeqs":[24],"surfaceOp":"append"} -{"type":"step/end","seq":26,"time":0,"data":{"turn":1,"step":2}} -{"type":"step/start","seq":27,"time":0,"data":{"turn":1,"step":3}} -{"type":"request/header","seq":28,"time":0,"data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","snapshot_double","subagent","workflow"]},"reason":"change"}} -{"type":"assistant/chunk","seq":29,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} -{"type":"assistant/chunk","seq":30,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-code","name":"run_code","argumentsDelta":"{\"code\": \"return await tools.snapshot_double({ value: 21 })\", \"description\": \"Run the temporary Plugin tool\"}"}}} -{"type":"assistant/chunk","seq":31,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-code","name":"run_code","arguments":"{\"code\": \"return await tools.snapshot_double({ value: 21 })\", \"description\": \"Run the temporary Plugin tool\"}"}}}} -{"type":"assistant/chunk","seq":32,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} -{"type":"assistant/chunk","seq":33,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} -{"type":"assistant/message","seq":34,"time":0,"data":{"turn":1,"step":3,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-code","name":"run_code","arguments":"{\"code\": \"return await tools.snapshot_double({ value: 21 })\", \"description\": \"Run the temporary Plugin tool\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[29,30,31,32,33],"surfaceOp":"append"} -{"type":"tool/call","seq":35,"time":0,"data":{"turn":1,"step":3,"callId":"advanced-code","name":"run_code","arguments":"{\"code\": \"return await tools.snapshot_double({ value: 21 })\", \"description\": \"Run the temporary Plugin tool\"}"}} -{"type":"tool/code-dispatch-start","seq":36,"time":0,"data":{"rootCallId":"advanced-code","parentCallId":"advanced-code","subCallId":"advanced-code:code:1","name":"snapshot_double","arguments":{"value":21}}} -{"type":"tool/code-dispatch","seq":37,"time":0,"data":{"rootCallId":"advanced-code","parentCallId":"advanced-code","subCallId":"advanced-code:code:1","name":"snapshot_double","arguments":{"value":21},"isError":false,"content":[{"type":"text","text":"42"}]}} -{"type":"tool/result","seq":38,"time":0,"data":{"turn":1,"step":3,"message":{"source":{"kind":"tool","callId":"advanced-code"},"content":[{"type":"tool-result","toolCallId":"advanced-code","content":[{"type":"text","text":"42"}],"isError":false}],"role":"user","id":"{{messageId}}"}},"sourceEventSeqs":[35],"surfaceOp":"append"} -{"type":"step/end","seq":39,"time":0,"data":{"turn":1,"step":3}} -{"type":"step/start","seq":40,"time":0,"data":{"turn":1,"step":4}} -{"type":"assistant/chunk","seq":41,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} -{"type":"assistant/chunk","seq":42,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-direct-child","name":"subagent","argumentsDelta":"{\"description\": \"Check direct child\", \"prompt\": \"Reply with exactly DIRECT_CHILD_OK and nothing else.\"}"}}} -{"type":"assistant/chunk","seq":43,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-direct-child","name":"subagent","arguments":"{\"description\": \"Check direct child\", \"prompt\": \"Reply with exactly DIRECT_CHILD_OK and nothing else.\"}"}}}} -{"type":"assistant/chunk","seq":44,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} -{"type":"assistant/chunk","seq":45,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} -{"type":"assistant/message","seq":46,"time":0,"data":{"turn":1,"step":4,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-direct-child","name":"subagent","arguments":"{\"description\": \"Check direct child\", \"prompt\": \"Reply with exactly DIRECT_CHILD_OK and nothing else.\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[41,42,43,44,45],"surfaceOp":"append"} -{"type":"tool/call","seq":47,"time":0,"data":{"turn":1,"step":4,"callId":"advanced-direct-child","name":"subagent","arguments":"{\"description\": \"Check direct child\", \"prompt\": \"Reply with exactly DIRECT_CHILD_OK and nothing else.\"}"}} -{"type":"tool/result","seq":48,"time":0,"data":{"turn":1,"step":4,"message":{"source":{"kind":"tool","callId":"advanced-direct-child"},"content":[{"type":"tool-result","toolCallId":"advanced-direct-child","content":[{"type":"text","text":"DIRECT_CHILD_OK"}],"isError":false}],"role":"user","id":"{{messageId}}"}},"sourceEventSeqs":[47],"surfaceOp":"append"} -{"type":"step/end","seq":49,"time":0,"data":{"turn":1,"step":4}} -{"type":"step/start","seq":50,"time":0,"data":{"turn":1,"step":5}} -{"type":"assistant/chunk","seq":51,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} -{"type":"assistant/chunk","seq":52,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-workflow","name":"workflow","argumentsDelta":"{\"script\": \"phase('Delegate')\\nconst reply = await agent('Reply with exactly WORKFLOW_CHILD_OK and nothing else.', { label: 'workflow-child' })\\nreturn { reply }\", \"meta\": {\"name\": \"advanced-exe-snapshot\", \"description\": \"exercise one packaged workflow child\"}}"}}} -{"type":"assistant/chunk","seq":53,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-workflow","name":"workflow","arguments":"{\"script\": \"phase('Delegate')\\nconst reply = await agent('Reply with exactly WORKFLOW_CHILD_OK and nothing else.', { label: 'workflow-child' })\\nreturn { reply }\", \"meta\": {\"name\": \"advanced-exe-snapshot\", \"description\": \"exercise one packaged workflow child\"}}"}}}} -{"type":"assistant/chunk","seq":54,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} -{"type":"assistant/chunk","seq":55,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} -{"type":"assistant/message","seq":56,"time":0,"data":{"turn":1,"step":5,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-workflow","name":"workflow","arguments":"{\"script\": \"phase('Delegate')\\nconst reply = await agent('Reply with exactly WORKFLOW_CHILD_OK and nothing else.', { label: 'workflow-child' })\\nreturn { reply }\", \"meta\": {\"name\": \"advanced-exe-snapshot\", \"description\": \"exercise one packaged workflow child\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[51,52,53,54,55],"surfaceOp":"append"} -{"type":"tool/call","seq":57,"time":0,"data":{"turn":1,"step":5,"callId":"advanced-workflow","name":"workflow","arguments":"{\"script\": \"phase('Delegate')\\nconst reply = await agent('Reply with exactly WORKFLOW_CHILD_OK and nothing else.', { label: 'workflow-child' })\\nreturn { reply }\", \"meta\": {\"name\": \"advanced-exe-snapshot\", \"description\": \"exercise one packaged workflow child\"}}"}} -{"type":"tool-workflow/run-start","seq":58,"time":0,"data":{"runId":"{{workflow-run}}","name":"advanced-exe-snapshot"}} -{"type":"tool-workflow/agent-start","seq":59,"time":0,"data":{"runId":"{{workflow-run}}","seq":1,"label":"workflow-child","phase":"Delegate","childId":"{{child-2}}"}} -{"type":"tool-workflow/agent-end","seq":60,"time":0,"data":{"runId":"{{workflow-run}}","seq":1,"outcome":"completed"}} -{"type":"tool-workflow/run-end","seq":61,"time":0,"data":{"runId":"{{workflow-run}}","stopReason":"completed"}} -{"type":"tool/result","seq":62,"time":0,"data":{"turn":1,"step":5,"message":{"source":{"kind":"tool","callId":"advanced-workflow"},"content":[{"type":"tool-result","toolCallId":"advanced-workflow","content":[{"type":"text","text":"workflow \"advanced-exe-snapshot\" completed (1 agent).\nReturn value:\n{\n \"reply\": \"WORKFLOW_CHILD_OK\"\n}"}],"isError":false}],"role":"user","id":"{{messageId}}"}},"sourceEventSeqs":[57],"surfaceOp":"append"} -{"type":"step/end","seq":63,"time":0,"data":{"turn":1,"step":5}} -{"type":"step/start","seq":64,"time":0,"data":{"turn":1,"step":6}} -{"type":"assistant/chunk","seq":65,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} -{"type":"assistant/chunk","seq":66,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-undefine","name":"cordis_undefine","argumentsDelta":"{\"pluginId\": \"snap-1\"}"}}} -{"type":"assistant/chunk","seq":67,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-undefine","name":"cordis_undefine","arguments":"{\"pluginId\": \"snap-1\"}"}}}} -{"type":"assistant/chunk","seq":68,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} -{"type":"assistant/chunk","seq":69,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} -{"type":"assistant/message","seq":70,"time":0,"data":{"turn":1,"step":6,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-undefine","name":"cordis_undefine","arguments":"{\"pluginId\": \"snap-1\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[65,66,67,68,69],"surfaceOp":"append"} -{"type":"tool/call","seq":71,"time":0,"data":{"turn":1,"step":6,"callId":"advanced-undefine","name":"cordis_undefine","arguments":"{\"pluginId\": \"snap-1\"}"}} -{"type":"tool/result","seq":72,"time":0,"data":{"turn":1,"step":6,"message":{"source":{"kind":"tool","callId":"advanced-undefine"},"content":[{"type":"tool-result","toolCallId":"advanced-undefine","content":[{"type":"text","text":"Removed dynamic Plugin snap-1 and all of its Packages."}],"isError":false}],"role":"user","id":"{{messageId}}"}},"sourceEventSeqs":[71],"surfaceOp":"append"} -{"type":"step/end","seq":73,"time":0,"data":{"turn":1,"step":6}} -{"type":"step/start","seq":74,"time":0,"data":{"turn":1,"step":7}} -{"type":"request/header","seq":75,"time":0,"data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","subagent","workflow"]},"reason":"change"}} -{"type":"assistant/chunk","seq":76,"time":0,"data":{"turn":1,"step":7,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} -{"type":"assistant/chunk","seq":77,"time":0,"data":{"turn":1,"step":7,"chunk":{"type":"text-delta","index":0,"text":"ADVANCED_EXECUTABLE_OK"}}} -{"type":"assistant/chunk","seq":78,"time":0,"data":{"turn":1,"step":7,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"ADVANCED_EXECUTABLE_OK"}}}} -{"type":"assistant/chunk","seq":79,"time":0,"data":{"turn":1,"step":7,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} -{"type":"assistant/chunk","seq":80,"time":0,"data":{"turn":1,"step":7,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} -{"type":"assistant/message","seq":81,"time":0,"data":{"turn":1,"step":7,"message":{"role":"assistant","content":[{"type":"text","text":"ADVANCED_EXECUTABLE_OK"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[76,77,78,79,80],"surfaceOp":"append"} -{"type":"step/end","seq":82,"time":0,"data":{"turn":1,"step":7}} -{"type":"turn/end","seq":83,"time":0,"data":{"turn":1,"reason":{"kind":"completed"}}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Run the advanced packaged-runtime snapshot scenario."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"}]}} +{"type":"turn/start","data":{"turn":1}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","data":{"turn":1,"step":1}} +{"type":"user/message","data":{"content":[{"type":"text","text":"Run the advanced packaged-runtime snapshot scenario."}],"source":{"kind":"user"},"role":"user","id":"{{messageId}}"},"surfaceOp":"append"} +{"type":"session/title","data":{"title":"Run the advanced packaged-runtime snapsh","messageSeqs":[4],"source":{"kind":"fallback"}}} +{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","subagent","workflow"]},"reason":"initial"}} +{"type":"request/context","data":{"provider":"deepseek-official","model":"smoke-model","contextWindow":1000000}} +{"type":"session-log-deepseek/delivery-accepted","data":{"sessionId":"{{parent}}","throughSeq":7}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-define","name":"cordis_define","argumentsDelta":"{\"plugin\": {\"kind\": \"new\", \"idPrefix\": \"snap\"}, \"name\": \"Snapshot Double\", \"purpose\": \"Expose a deterministic doubling tool for executable snapshot verification.\", \"code\": {\"host\": \"return (ctx) => {\\n harness.registerTool(ctx, harness.defineTool({\\n name: 'snapshot_double',\\n description: 'Double a number for executable snapshot verification.',\\n parameters: { value: { type: 'number', required: true } },\\n output: {\\n schema: { type: 'number' },\\n render(_args, value) {\\n return [{ type: 'text', text: String(value) }]\\n }\\n },\\n async execute(args) {\\n return args.value * 2\\n }\\n }))\\n}\\n\"}}"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-define","name":"cordis_define","arguments":"{\"plugin\": {\"kind\": \"new\", \"idPrefix\": \"snap\"}, \"name\": \"Snapshot Double\", \"purpose\": \"Expose a deterministic doubling tool for executable snapshot verification.\", \"code\": {\"host\": \"return (ctx) => {\\n harness.registerTool(ctx, harness.defineTool({\\n name: 'snapshot_double',\\n description: 'Double a number for executable snapshot verification.',\\n parameters: { value: { type: 'number', required: true } },\\n output: {\\n schema: { type: 'number' },\\n render(_args, value) {\\n return [{ type: 'text', text: String(value) }]\\n }\\n },\\n async execute(args) {\\n return args.value * 2\\n }\\n }))\\n}\\n\"}}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-define","name":"cordis_define","arguments":"{\"plugin\": {\"kind\": \"new\", \"idPrefix\": \"snap\"}, \"name\": \"Snapshot Double\", \"purpose\": \"Expose a deterministic doubling tool for executable snapshot verification.\", \"code\": {\"host\": \"return (ctx) => {\\n harness.registerTool(ctx, harness.defineTool({\\n name: 'snapshot_double',\\n description: 'Double a number for executable snapshot verification.',\\n parameters: { value: { type: 'number', required: true } },\\n output: {\\n schema: { type: 'number' },\\n render(_args, value) {\\n return [{ type: 'text', text: String(value) }]\\n }\\n },\\n async execute(args) {\\n return args.value * 2\\n }\\n }))\\n}\\n\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[9,10,11,12,13],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":1,"callId":"advanced-define","name":"cordis_define","arguments":"{\"plugin\": {\"kind\": \"new\", \"idPrefix\": \"snap\"}, \"name\": \"Snapshot Double\", \"purpose\": \"Expose a deterministic doubling tool for executable snapshot verification.\", \"code\": {\"host\": \"return (ctx) => {\\n harness.registerTool(ctx, harness.defineTool({\\n name: 'snapshot_double',\\n description: 'Double a number for executable snapshot verification.',\\n parameters: { value: { type: 'number', required: true } },\\n output: {\\n schema: { type: 'number' },\\n render(_args, value) {\\n return [{ type: 'text', text: String(value) }]\\n }\\n },\\n async execute(args) {\\n return args.value * 2\\n }\\n }))\\n}\\n\"}}"}} +{"type":"tool/result","data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"advanced-define"},"content":[{"type":"tool-result","toolCallId":"advanced-define","content":[{"type":"text","text":"Defined snap-1/pkg-1 (Snapshot Double); it is not running yet. Use cordis_run to activate this Package."}],"isError":false}],"role":"user","id":"{{messageId}}"},"meta":{"pluginId":"snap-1","packageId":"pkg-1"}},"sourceEventSeqs":[15],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":1}} +{"type":"step/start","data":{"turn":1,"step":2}} +{"type":"session-log-deepseek/delivery-accepted","data":{"sessionId":"{{parent}}","throughSeq":18}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-run","name":"cordis_run","argumentsDelta":"{\"pluginId\": \"snap-1\", \"packageId\": \"pkg-1\", \"mode\": \"run\"}"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-run","name":"cordis_run","arguments":"{\"pluginId\": \"snap-1\", \"packageId\": \"pkg-1\", \"mode\": \"run\"}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-run","name":"cordis_run","arguments":"{\"pluginId\": \"snap-1\", \"packageId\": \"pkg-1\", \"mode\": \"run\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[20,21,22,23,24],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":2,"callId":"advanced-run","name":"cordis_run","arguments":"{\"pluginId\": \"snap-1\", \"packageId\": \"pkg-1\", \"mode\": \"run\"}"}} +{"type":"tool/result","data":{"turn":1,"step":2,"message":{"source":{"kind":"tool","callId":"advanced-run"},"content":[{"type":"tool-result","toolCallId":"advanced-run","content":[{"type":"text","text":"snap-1/pkg-1 is running (run-1)."}],"isError":false}],"role":"user","id":"{{messageId}}"},"meta":{"pluginId":"snap-1","packageId":"pkg-1","pluginRunId":"run-1"}},"sourceEventSeqs":[26],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":2}} +{"type":"step/start","data":{"turn":1,"step":3}} +{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","snapshot_double","subagent","workflow"]},"reason":"change"}} +{"type":"session-log-deepseek/delivery-accepted","data":{"sessionId":"{{parent}}","throughSeq":30}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-code","name":"run_code","argumentsDelta":"{\"code\": \"return await tools.snapshot_double({ value: 21 })\", \"description\": \"Run the temporary Plugin tool\"}"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-code","name":"run_code","arguments":"{\"code\": \"return await tools.snapshot_double({ value: 21 })\", \"description\": \"Run the temporary Plugin tool\"}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":3,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-code","name":"run_code","arguments":"{\"code\": \"return await tools.snapshot_double({ value: 21 })\", \"description\": \"Run the temporary Plugin tool\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[32,33,34,35,36],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":3,"callId":"advanced-code","name":"run_code","arguments":"{\"code\": \"return await tools.snapshot_double({ value: 21 })\", \"description\": \"Run the temporary Plugin tool\"}"}} +{"type":"tool/code-dispatch-start","data":{"rootCallId":"advanced-code","parentCallId":"advanced-code","subCallId":"advanced-code:code:1","name":"snapshot_double","arguments":{"value":21}}} +{"type":"tool/code-dispatch","data":{"rootCallId":"advanced-code","parentCallId":"advanced-code","subCallId":"advanced-code:code:1","name":"snapshot_double","arguments":{"value":21},"isError":false,"content":[{"type":"text","text":"42"}]}} +{"type":"tool/result","data":{"turn":1,"step":3,"message":{"source":{"kind":"tool","callId":"advanced-code"},"content":[{"type":"tool-result","toolCallId":"advanced-code","content":[{"type":"text","text":"42"}],"isError":false}],"role":"user","id":"{{messageId}}"}},"sourceEventSeqs":[38],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":3}} +{"type":"step/start","data":{"turn":1,"step":4}} +{"type":"session-log-deepseek/delivery-accepted","data":{"sessionId":"{{parent}}","throughSeq":43}} +{"type":"assistant/chunk","data":{"turn":1,"step":4,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":4,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-direct-child","name":"subagent","argumentsDelta":"{\"description\": \"Check direct child\", \"prompt\": \"Reply with exactly DIRECT_CHILD_OK and nothing else.\"}"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":4,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-direct-child","name":"subagent","arguments":"{\"description\": \"Check direct child\", \"prompt\": \"Reply with exactly DIRECT_CHILD_OK and nothing else.\"}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":4,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":4,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":4,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-direct-child","name":"subagent","arguments":"{\"description\": \"Check direct child\", \"prompt\": \"Reply with exactly DIRECT_CHILD_OK and nothing else.\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[45,46,47,48,49],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":4,"callId":"advanced-direct-child","name":"subagent","arguments":"{\"description\": \"Check direct child\", \"prompt\": \"Reply with exactly DIRECT_CHILD_OK and nothing else.\"}"}} +{"type":"tool/result","data":{"turn":1,"step":4,"message":{"source":{"kind":"tool","callId":"advanced-direct-child"},"content":[{"type":"tool-result","toolCallId":"advanced-direct-child","content":[{"type":"text","text":"DIRECT_CHILD_OK"}],"isError":false}],"role":"user","id":"{{messageId}}"}},"sourceEventSeqs":[51],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":4}} +{"type":"step/start","data":{"turn":1,"step":5}} +{"type":"session-log-deepseek/delivery-accepted","data":{"sessionId":"{{parent}}","throughSeq":54}} +{"type":"assistant/chunk","data":{"turn":1,"step":5,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":5,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-workflow","name":"workflow","argumentsDelta":"{\"script\": \"phase('Delegate')\\nconst reply = await agent('Reply with exactly WORKFLOW_CHILD_OK and nothing else.', { label: 'workflow-child' })\\nreturn { reply }\", \"meta\": {\"name\": \"advanced-exe-snapshot\", \"description\": \"exercise one packaged workflow child\"}}"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":5,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-workflow","name":"workflow","arguments":"{\"script\": \"phase('Delegate')\\nconst reply = await agent('Reply with exactly WORKFLOW_CHILD_OK and nothing else.', { label: 'workflow-child' })\\nreturn { reply }\", \"meta\": {\"name\": \"advanced-exe-snapshot\", \"description\": \"exercise one packaged workflow child\"}}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":5,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":5,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":5,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-workflow","name":"workflow","arguments":"{\"script\": \"phase('Delegate')\\nconst reply = await agent('Reply with exactly WORKFLOW_CHILD_OK and nothing else.', { label: 'workflow-child' })\\nreturn { reply }\", \"meta\": {\"name\": \"advanced-exe-snapshot\", \"description\": \"exercise one packaged workflow child\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[56,57,58,59,60],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":5,"callId":"advanced-workflow","name":"workflow","arguments":"{\"script\": \"phase('Delegate')\\nconst reply = await agent('Reply with exactly WORKFLOW_CHILD_OK and nothing else.', { label: 'workflow-child' })\\nreturn { reply }\", \"meta\": {\"name\": \"advanced-exe-snapshot\", \"description\": \"exercise one packaged workflow child\"}}"}} +{"type":"tool-workflow/run-start","data":{"runId":"{{workflow-run}}","name":"advanced-exe-snapshot"}} +{"type":"tool-workflow/agent-start","data":{"runId":"{{workflow-run}}","seq":1,"label":"workflow-child","phase":"Delegate","childId":"{{child-2}}"}} +{"type":"tool-workflow/agent-end","data":{"runId":"{{workflow-run}}","seq":1,"outcome":"completed"}} +{"type":"tool-workflow/run-end","data":{"runId":"{{workflow-run}}","stopReason":"completed"}} +{"type":"tool/result","data":{"turn":1,"step":5,"message":{"source":{"kind":"tool","callId":"advanced-workflow"},"content":[{"type":"tool-result","toolCallId":"advanced-workflow","content":[{"type":"text","text":"workflow \"advanced-exe-snapshot\" completed (1 agent).\nReturn value:\n{\n \"reply\": \"WORKFLOW_CHILD_OK\"\n}"}],"isError":false}],"role":"user","id":"{{messageId}}"}},"sourceEventSeqs":[62],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":5}} +{"type":"step/start","data":{"turn":1,"step":6}} +{"type":"session-log-deepseek/delivery-accepted","data":{"sessionId":"{{parent}}","throughSeq":69}} +{"type":"assistant/chunk","data":{"turn":1,"step":6,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":6,"chunk":{"type":"tool-call-delta","index":0,"id":"advanced-undefine","name":"cordis_undefine","argumentsDelta":"{\"pluginId\": \"snap-1\"}"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":6,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"advanced-undefine","name":"cordis_undefine","arguments":"{\"pluginId\": \"snap-1\"}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":6,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":6,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":6,"message":{"role":"assistant","content":[{"type":"tool-call","id":"advanced-undefine","name":"cordis_undefine","arguments":"{\"pluginId\": \"snap-1\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[71,72,73,74,75],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":6,"callId":"advanced-undefine","name":"cordis_undefine","arguments":"{\"pluginId\": \"snap-1\"}"}} +{"type":"tool/result","data":{"turn":1,"step":6,"message":{"source":{"kind":"tool","callId":"advanced-undefine"},"content":[{"type":"tool-result","toolCallId":"advanced-undefine","content":[{"type":"text","text":"Removed dynamic Plugin snap-1 and all of its Packages."}],"isError":false}],"role":"user","id":"{{messageId}}"}},"sourceEventSeqs":[77],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":6}} +{"type":"step/start","data":{"turn":1,"step":7}} +{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"smoke-model","maxTokens":256000,"reasoningEffort":"high"},"adapterDefaults":{"reasoningEffort":true,"maxTokens":true},"system":"{{system}}","tools":["cordis_define","cordis_inspect_list","cordis_inspect_query","cordis_inspect_self","cordis_run","cordis_stop","cordis_undefine","job_kill","job_list","job_output","run_code","subagent","workflow"]},"reason":"change"}} +{"type":"session-log-deepseek/delivery-accepted","data":{"sessionId":"{{parent}}","throughSeq":81}} +{"type":"assistant/chunk","data":{"turn":1,"step":7,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":7,"chunk":{"type":"text-delta","index":0,"text":"ADVANCED_EXECUTABLE_OK"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":7,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"ADVANCED_EXECUTABLE_OK"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":7,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":7,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","data":{"turn":1,"step":7,"message":{"role":"assistant","content":[{"type":"text","text":"ADVANCED_EXECUTABLE_OK"}],"source":{"kind":"model","provider":"deepseek-official","model":"smoke-model"},"id":"{{messageId}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[83,84,85,86,87],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":7}} +{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/scripts/snapshots/python-sdk-single-exe/minimal/model-visible.json b/scripts/snapshots/python-sdk-single-exe/minimal/model-visible.json new file mode 100644 index 0000000000..a3223c8d76 --- /dev/null +++ b/scripts/snapshots/python-sdk-single-exe/minimal/model-visible.json @@ -0,0 +1,430 @@ +[ + { + "tools": [ + { + "type": "function", + "function": { + "name": "bash", + "description": "Run commands in a bash shell\n* When invoking this tool, the contents of the \"command\" parameter does NOT need to be XML-escaped.\n* You don't have access to the internet via this tool.\n* You do have access to a mirror of common linux and python packages via apt and pip.\n* State is persistent across command calls and discussions with the user.\n* To inspect a particular line range of a file, e.g. lines 10-25, try 'sed -n 10,25p /path/to/the/file'.\n* Please avoid commands that may produce a very large amount of output.\n* Please run long lived commands in the background, e.g. 'sleep 10 &' or start a server in the background.", + "parameters": { + "type": "object", + "properties": { + "command": { + "type": "string", + "description": "The bash command to run. Relative path is preferred in the command." + } + }, + "required": [ + "command" + ] + } + } + }, + { + "type": "function", + "function": { + "name": "str_replace_editor", + "description": "Custom editing tool for viewing, creating and editing files\n* State is persistent across command calls and discussions with the user\n* If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep\n* The `create` command cannot be used if the specified `path` already exists as a file\n* If a `command` generates a long output, it will be truncated and marked with ``\n\nNotes for using the `str_replace` command:\n* The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces!\n* If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique\n* The `new_str` parameter should contain the edited lines that should replace the `old_str`", + "parameters": { + "type": "object", + "properties": { + "command": { + "type": "string", + "description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.", + "enum": [ + "view", + "create", + "str_replace", + "insert" + ] + }, + "path": { + "type": "string", + "description": "Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`." + }, + "file_text": { + "type": "string", + "description": "Required parameter of `create` command, with the content of the file to be created." + }, + "insert_line": { + "type": "integer", + "description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`." + }, + "new_str": { + "type": "string", + "description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert." + }, + "old_str": { + "type": "string", + "description": "Required parameter of `str_replace` command containing the string in `path` to replace." + }, + "view_range": { + "type": "array", + "description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.", + "items": { + "type": "integer" + } + } + }, + "required": [ + "command", + "path" + ] + } + } + } + ], + "messages": [ + { + "role": "system", + "text": "You are a helpful software engineer assistant." + }, + { + "role": "user", + "text": "Exercise the packaged minimal agent's persistent Bash and string-replacement editor.\nEditor path: {{cwd}}/created.txt" + } + ] + }, + { + "tools": [ + { + "type": "function", + "function": { + "name": "bash", + "description": "Run commands in a bash shell\n* When invoking this tool, the contents of the \"command\" parameter does NOT need to be XML-escaped.\n* You don't have access to the internet via this tool.\n* You do have access to a mirror of common linux and python packages via apt and pip.\n* State is persistent across command calls and discussions with the user.\n* To inspect a particular line range of a file, e.g. lines 10-25, try 'sed -n 10,25p /path/to/the/file'.\n* Please avoid commands that may produce a very large amount of output.\n* Please run long lived commands in the background, e.g. 'sleep 10 &' or start a server in the background.", + "parameters": { + "type": "object", + "properties": { + "command": { + "type": "string", + "description": "The bash command to run. Relative path is preferred in the command." + } + }, + "required": [ + "command" + ] + } + } + }, + { + "type": "function", + "function": { + "name": "str_replace_editor", + "description": "Custom editing tool for viewing, creating and editing files\n* State is persistent across command calls and discussions with the user\n* If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep\n* The `create` command cannot be used if the specified `path` already exists as a file\n* If a `command` generates a long output, it will be truncated and marked with ``\n\nNotes for using the `str_replace` command:\n* The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces!\n* If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique\n* The `new_str` parameter should contain the edited lines that should replace the `old_str`", + "parameters": { + "type": "object", + "properties": { + "command": { + "type": "string", + "description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.", + "enum": [ + "view", + "create", + "str_replace", + "insert" + ] + }, + "path": { + "type": "string", + "description": "Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`." + }, + "file_text": { + "type": "string", + "description": "Required parameter of `create` command, with the content of the file to be created." + }, + "insert_line": { + "type": "integer", + "description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`." + }, + "new_str": { + "type": "string", + "description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert." + }, + "old_str": { + "type": "string", + "description": "Required parameter of `str_replace` command containing the string in `path` to replace." + }, + "view_range": { + "type": "array", + "description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.", + "items": { + "type": "integer" + } + } + }, + "required": [ + "command", + "path" + ] + } + } + } + ], + "messages": [ + { + "role": "system", + "text": "You are a helpful software engineer assistant." + }, + { + "role": "user", + "text": "Exercise the packaged minimal agent's persistent Bash and string-replacement editor.\nEditor path: {{cwd}}/created.txt" + }, + { + "role": "assistant", + "toolCalls": [ + { + "id": "minimal-bash-1", + "name": "bash" + } + ] + }, + { + "role": "tool", + "toolCallId": "minimal-bash-1", + "text": "{{tool-result}}" + } + ] + }, + { + "tools": [ + { + "type": "function", + "function": { + "name": "bash", + "description": "Run commands in a bash shell\n* When invoking this tool, the contents of the \"command\" parameter does NOT need to be XML-escaped.\n* You don't have access to the internet via this tool.\n* You do have access to a mirror of common linux and python packages via apt and pip.\n* State is persistent across command calls and discussions with the user.\n* To inspect a particular line range of a file, e.g. lines 10-25, try 'sed -n 10,25p /path/to/the/file'.\n* Please avoid commands that may produce a very large amount of output.\n* Please run long lived commands in the background, e.g. 'sleep 10 &' or start a server in the background.", + "parameters": { + "type": "object", + "properties": { + "command": { + "type": "string", + "description": "The bash command to run. Relative path is preferred in the command." + } + }, + "required": [ + "command" + ] + } + } + }, + { + "type": "function", + "function": { + "name": "str_replace_editor", + "description": "Custom editing tool for viewing, creating and editing files\n* State is persistent across command calls and discussions with the user\n* If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep\n* The `create` command cannot be used if the specified `path` already exists as a file\n* If a `command` generates a long output, it will be truncated and marked with ``\n\nNotes for using the `str_replace` command:\n* The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces!\n* If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique\n* The `new_str` parameter should contain the edited lines that should replace the `old_str`", + "parameters": { + "type": "object", + "properties": { + "command": { + "type": "string", + "description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.", + "enum": [ + "view", + "create", + "str_replace", + "insert" + ] + }, + "path": { + "type": "string", + "description": "Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`." + }, + "file_text": { + "type": "string", + "description": "Required parameter of `create` command, with the content of the file to be created." + }, + "insert_line": { + "type": "integer", + "description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`." + }, + "new_str": { + "type": "string", + "description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert." + }, + "old_str": { + "type": "string", + "description": "Required parameter of `str_replace` command containing the string in `path` to replace." + }, + "view_range": { + "type": "array", + "description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.", + "items": { + "type": "integer" + } + } + }, + "required": [ + "command", + "path" + ] + } + } + } + ], + "messages": [ + { + "role": "system", + "text": "You are a helpful software engineer assistant." + }, + { + "role": "user", + "text": "Exercise the packaged minimal agent's persistent Bash and string-replacement editor.\nEditor path: {{cwd}}/created.txt" + }, + { + "role": "assistant", + "toolCalls": [ + { + "id": "minimal-bash-1", + "name": "bash" + } + ] + }, + { + "role": "tool", + "toolCallId": "minimal-bash-1", + "text": "{{tool-result}}" + }, + { + "role": "assistant", + "toolCalls": [ + { + "id": "minimal-bash-2", + "name": "bash" + } + ] + }, + { + "role": "tool", + "toolCallId": "minimal-bash-2", + "text": "{{tool-result}}" + } + ] + }, + { + "tools": [ + { + "type": "function", + "function": { + "name": "bash", + "description": "Run commands in a bash shell\n* When invoking this tool, the contents of the \"command\" parameter does NOT need to be XML-escaped.\n* You don't have access to the internet via this tool.\n* You do have access to a mirror of common linux and python packages via apt and pip.\n* State is persistent across command calls and discussions with the user.\n* To inspect a particular line range of a file, e.g. lines 10-25, try 'sed -n 10,25p /path/to/the/file'.\n* Please avoid commands that may produce a very large amount of output.\n* Please run long lived commands in the background, e.g. 'sleep 10 &' or start a server in the background.", + "parameters": { + "type": "object", + "properties": { + "command": { + "type": "string", + "description": "The bash command to run. Relative path is preferred in the command." + } + }, + "required": [ + "command" + ] + } + } + }, + { + "type": "function", + "function": { + "name": "str_replace_editor", + "description": "Custom editing tool for viewing, creating and editing files\n* State is persistent across command calls and discussions with the user\n* If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep\n* The `create` command cannot be used if the specified `path` already exists as a file\n* If a `command` generates a long output, it will be truncated and marked with ``\n\nNotes for using the `str_replace` command:\n* The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces!\n* If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique\n* The `new_str` parameter should contain the edited lines that should replace the `old_str`", + "parameters": { + "type": "object", + "properties": { + "command": { + "type": "string", + "description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.", + "enum": [ + "view", + "create", + "str_replace", + "insert" + ] + }, + "path": { + "type": "string", + "description": "Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`." + }, + "file_text": { + "type": "string", + "description": "Required parameter of `create` command, with the content of the file to be created." + }, + "insert_line": { + "type": "integer", + "description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`." + }, + "new_str": { + "type": "string", + "description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert." + }, + "old_str": { + "type": "string", + "description": "Required parameter of `str_replace` command containing the string in `path` to replace." + }, + "view_range": { + "type": "array", + "description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.", + "items": { + "type": "integer" + } + } + }, + "required": [ + "command", + "path" + ] + } + } + } + ], + "messages": [ + { + "role": "system", + "text": "You are a helpful software engineer assistant." + }, + { + "role": "user", + "text": "Exercise the packaged minimal agent's persistent Bash and string-replacement editor.\nEditor path: {{cwd}}/created.txt" + }, + { + "role": "assistant", + "toolCalls": [ + { + "id": "minimal-bash-1", + "name": "bash" + } + ] + }, + { + "role": "tool", + "toolCallId": "minimal-bash-1", + "text": "{{tool-result}}" + }, + { + "role": "assistant", + "toolCalls": [ + { + "id": "minimal-bash-2", + "name": "bash" + } + ] + }, + { + "role": "tool", + "toolCallId": "minimal-bash-2", + "text": "{{tool-result}}" + }, + { + "role": "assistant", + "toolCalls": [ + { + "id": "minimal-editor", + "name": "str_replace_editor" + } + ] + }, + { + "role": "tool", + "toolCallId": "minimal-editor", + "text": "{{tool-result}}" + } + ] + } +] diff --git a/scripts/snapshots/translation-prompt-v4/request-response.expected.json b/scripts/snapshots/translation-prompt-v4/request-response.expected.json index 87d39e4609..506a8110b5 100644 --- a/scripts/snapshots/translation-prompt-v4/request-response.expected.json +++ b/scripts/snapshots/translation-prompt-v4/request-response.expected.json @@ -4,7 +4,7 @@ "messages": [ { "role": "system", - "content": "# Translation Prompt\n\nYou are a senior technical translator specializing in LLM and agent development documentation. Your task is to translate the complete source document from English to Chinese, producing natural, professional technical prose.\n\nRead each complete semantic unit, understand it, and restate it as a native technical author would write it in the target language. Do not mechanically preserve source-language syntax. Then verify the translation against the source clause by clause: preserve every proposition and add none. Fluency never justifies losing or altering meaning, and completeness never justifies unnatural word-for-word prose.\n\n## Priority\n\nApply these authorities in order:\n\n1. Preserve the source meaning and the required document structure, protected content, and formatting.\n2. Follow the injected terminology table exactly.\n3. Use the injected whole-document gold pairs to calibrate target-language voice and phrasing.\n4. Apply the general writing guidance and illustrative examples in this prompt.\n\nA lower-priority rule may refine but never override a higher-priority requirement. Gold pairs calibrate voice; they are not a translation memory. No style preference, gold-pair phrasing, or embedded example may override source meaning, required structure, protected content, or the terminology table.\n\n## Quality Requirements\n\n### Structure and Format Preservation\n- Output a complete translated document that maintains the same document frame as the source: heading hierarchy and order, list kinds and item counts, ordered-list starts, table rows and columns, link targets, and code blocks.\n- Paragraph boundaries may change within the same structural unit when the target language needs different semantic grouping. Do not merge or move content across headings, list items, table cells, or other independent structural units.\n- Keep each prose paragraph on one physical line. Use paragraph breaks, not hard-wrapped lines inside a paragraph.\n- Fenced code blocks must be byte-identical to the source, including info strings, whitespace, and ALL comments inside them. Do NOT translate or reformat any content inside code blocks. This is a hard rule with no exceptions.\n- Inline code spans must be kept verbatim. This includes commands, flags, paths, identifiers, API and event names, config keys, protocol values, version numbers, and other machine-readable tokens. Never translate or reformat them.\n- Every relative link must point to the same target as in the source. Translate link text; do not change link targets.\n- Language switcher line: when an English source contains `English | [中文](source-filename.zh.md)`, write `[English](source-filename.md) | 中文`. When a Chinese source contains `[English](source-filename.md) | 中文`, write `English | [中文](source-filename.zh.md)`. Do NOT copy the source switcher unchanged. If the source has no switcher, do not invent a filename or switcher; the pipeline inserts the canonical target switcher after parsing ``.\n- Preserve emphasis marker types and the semantic spans they cover. Do not add, remove, move, or change bold and italic markers.\n\n### Faithfulness\n- Preserve every proposition in the source and add none. Every sentence, list item, note, FIXME, warning, example, caveat, prerequisite, and guarantee must have an equivalent in the translation. Count list items on both sides.\n- Preserve actors, objects, conditions, exceptions, negation, modality, causal relationships, and distinctions between concepts.\n- Preserve the exact strength and orientation of contracts. Completion and lifecycle conditions, failure behavior, directions and data flow, normal and exceptional result channels, ownership changes, and quantitative bounds must not be weakened, strengthened, reversed, or merged.\n- Translate ideas rather than source-language idioms, but never use fluency as a reason to omit or alter meaning.\n\n### Tone and Style\n- The translation must read as if originally written in the target language by a native technical author. If an expression sounds like a word-for-word rendering from the source language, rephrase it.\n- Write in a professional, formal tone appropriate for developer documentation. Never use colloquial or casual expressions.\n- Name an actor when the target language would otherwise obscure an actor that the source states or unambiguously implies. Never invent responsibility merely to avoid a passive construction.\n- Prefer established target-language engineering terms over literal renderings. Replace metaphors with direct descriptions that preserve the source meaning.\n- Use polite imperative forms where the text instructs the reader to do something. In Chinese, address the reader as `你`, not `您`.\n- Keep the author's register: concise stays concise, detailed stays detailed.\n\n### Sentence Structure\n- Break long sentences where the target language needs a pause. Avoid run-on sentences.\n- Use active voice when it improves clarity without changing or inventing the actor. Retain passive voice when the actor is unknown, irrelevant, or intentionally omitted.\n- Restructure source-language syntax into clear target-language syntax. Preserve the logical scope of conditions, concessions, negation, coordination, and modifiers.\n- Split or combine clauses when needed for readability, provided every source relationship remains explicit.\n- Translate meaning, not words. Do not invent words or expressions that a native technical author would not use.\n\n### Word Choice\n- Prefer precise, formal vocabulary over casual or colloquial alternatives.\n- When multiple synonyms exist, choose the one most commonly used in professional technical documentation of the target language.\n- Translate ordinary prose when an established target-language expression is clear. Preserve proper nouns, canonical product names, code identifiers, APIs, paths, package names, and terms that the terminology table requires to remain in the source language.\n- Use context to resolve polysemous words. A familiar word does not have one fixed rendering in every technical domain.\n- Avoid slang, internal jargon, or overly literal translations that would not be recognized by the general developer audience.\n- Do not use the same word to translate distinct source-language concepts when their distinction matters.\n- Avoid repeating the same ordinary verb in close proximity when a natural equivalent preserves the exact meaning. Never vary a terminology-table form, defined concept, or contract verb merely for stylistic variety.\n\n#### When translating into Chinese\n- When a number modifies a noun, include a natural Chinese classifier or measure word when Chinese grammar requires one. For example: \"three-role capability seam\" → \"包含三种角色的能力 seam\", not \"三角色 seam\". Do not add classifiers to code, identifiers, versions, units, or fixed names.\n\n### Punctuation\n\n#### When translating into Chinese\n- Use full-width Chinese punctuation in Chinese prose: `,。:;?!()「」`. Keep half-width punctuation inside code spans, numbers, and complete verbatim English text.\n- Prefer colons, periods, commas, or parentheses over em dashes when they make the sentence clearer or more natural. Keep an em dash when it is the clearest natural punctuation.\n- Use enumeration commas (、) between parallel Chinese items, not regular commas.\n- Keep list-item endings consistent with their grammar. Complete sentences may end with periods or other grammatically required punctuation; do not end list items with commas.\n- Put one half-width space between Chinese text and Latin words or numerals. Do not add a space next to full-width punctuation, and do not leave a meaningless half-width space between two Chinese characters.\n- Markdown emphasis markers do not create a word boundary. Determine spacing from the rendered adjacent characters: Chinese next to Chinese takes no space, while Chinese next to a Latin word or numeral takes one half-width space.\n- Use half-width digits and Latin letters, never full-width forms.\n- For RFC 2119 keywords (MUST, MUST NOT, SHOULD, MAY), translate to the corresponding Chinese term (必须、禁止、应当、可以), preserve the SOURCE emphasis span exactly, and do not weaken its normative strength: plain source stays plain (必须), italic source stays italic (*必须*), and bold source stays bold (**必须**).\n\n#### When translating into English\n- Use half-width English punctuation and standard English spacing. Preserve full-width punctuation only in verbatim Chinese text.\n- Convert enumeration commas (、) to English commas and Chinese prose quotation marks to English double quotes.\n- Convert Chinese topic-comment sentences and omitted-subject constructions into clear English subjects when the actor is stated or unambiguously implied. Do not invent an actor.\n- Use concise professional developer prose and established English technical terms. Do not transliterate Chinese engineering idioms literally.\n- Use the terminology table's English column exactly and do not carry Chinese first-occurrence glosses into English prose.\n\n## Terminology\n\nA terminology table is provided below. Follow it strictly:\n- Render every listed term exactly as specified.\n- When the target language is Chinese, use the \"中文\" column. On the document's first prose occurrence, write the \"首次出现\" value when one is specified; on later occurrences, write only the part before the parenthetical gloss.\n- When the target language is English, use the \"English\" column without a Chinese gloss; do not copy the \"中文\" or \"首次出现\" value into English prose.\n- If a term has already been glossed as part of a compound term, do not gloss it again when it appears alone later.\n- NEVER use translations listed in the \"不要译作\" column.\n- Code spans and other protected tokens remain verbatim even when their text resembles a listed term.\n- For an unlisted technical term, use an established target-language technical term when its meaning is unambiguous in context. For a Chinese target, use an established Chinese rendering from a major Chinese-language OSS or vendor source; if you cannot reliably determine such a rendering, preserve the source term and record `[Terminology: pending]` in `` with a tentative rendering for human review. For an English target, use the established English technical term; if the source term has no unambiguous established equivalent, preserve it with the shortest English gloss needed to make it intelligible and record `[Terminology: pending]` in ``. A tentative rendering may appear in `` but must not be silently adopted in `` or ``, and you must not invent or claim a specific external precedent. This rule applies to terminology only; for general prose, freely restructure and paraphrase for natural expression.\n\n# Terminology\n\n本表约定本仓库的中英术语统一译法。\n\n**通用规则:**\n- \"中文\"列为中文译文的正文默认用词。若该列为英文,则中文译文的正文中保留英文不翻译。\n- 首次出现按\"首次出现\"列书写(带括号注释);后续出现只写括号前的部分(可能为中文,也可能为英文),不出现括号内的注释。\n- \"不要译作\"列为严格禁止的译法。\n- 如果某术语已经作为另一个术语的组成部分被括注过(如 `agent loop(智能体循环)` 中已包含 `agent` 的括注),则该术语后续单独出现时无需再次括注。\n\n## 缩写类(中英文文本中均使用缩写)\n\n| English | 中文 | 首次出现 | 不要译作 | 备注 |\n|---|---|---|---|---|\n| ACP | ACP | ACP(Agent Client Protocol) | | |\n| AI | AI | AI(人工智能) | | |\n| API | API | | | |\n| CI | CI | | | |\n| CLI | CLI | CLI(命令行界面) | | |\n| e2e | e2e | | | |\n| HMR | HMR | HMR(热模块替换) | | |\n| JSON Schema | JSON Schema | | | |\n| JSONL | JSONL | | | |\n| LLM | LLM | LLM(大语言模型) | | |\n| MCP | MCP | | | |\n| PR | PR | PR(Pull Request) | | |\n| RAG | RAG | RAG(检索增强生成) | | |\n| SDK | SDK | | | 只指受支持的 Python 与 TypeScript SDK 所使用的 JSON-RPC 客户端/服务器协议;DeepSeek Harness 项目本身不是 SDK |\n| SSE | SSE | SSE(Server-Sent Events) | | |\n\n## 英文类(中英文文本中均使用英文)\n\n| English | 中文 | 首次出现 | 不要译作 | 备注 |\n|---|---|---|---|---|\n| agent | agent | agent(智能体) | | |\n| Agent Note | Agent Note | | 智能体注记、智能体笔记 | 仓库定义的文档类型,涵盖提案、已实现决策和被否决提案;中文对侧 H1 保持固定前缀 `# Agent Note: `,标题中不加术语括注 |\n| agent harness | agent harness | agent harness(智能体框架) | | agent 组合词(agent harness/workflow/loop/skill 等)整体保留英文;未括注过 agent 时首现按对应组合词或 agent 行处理 |\n| agent loop | agent loop | agent loop(智能体循环) | | |\n| blob hash | blob hash | | | `git hash-object` 的结果 |\n| coding agent | coding agent | coding agent(编程智能体) | | agent 组合词,正文保留英文 |\n| Cordis | Cordis | | | |\n| dispose | dispose | dispose(资源释放) | | |\n| doc-sync | doc-sync | doc-sync(文档同步门禁) | | |\n| fiber | fiber | | | |\n| fixture | fixture | fixture(测试前置数据) | | |\n| fork | fork | | | |\n| Function Calling | Function Calling | Function Calling(函数调用) | | |\n| harness | harness | | | |\n| harness engineering | harness engineering | | | |\n| KV Cache | KV Cache | | | 专有技术名称,保持大小写与空格 |\n| lint | lint | | | |\n| mock | mock | | | 保留英文;指测试替身 |\n| loader | loader | | | |\n| manifest | manifest | manifest(元数据清单) | | |\n| monorepo | monorepo | | | |\n| Round | Round | | 回合、目标回合、Ralph 回合 | 外层策略使用 Round 时,领域层级为 Session > Round > Turn(轮次) > Step(步骤);Round 是可选的外层策略迭代,并非每个会话轮次都具有的通用层级。Goal Round 与 Ralph Round 均保留英文。一个 Round 承载一个轮次,步骤隶属于该轮次;明确的零步骤轮次仍保持原义。 |\n| schema | schema | | | |\n| schema DSL | schema DSL | | | |\n| seam | seam | | 接缝 | 一个可替换能力的整体,包含 Service Definition / Service Provider / Consumer 三种角色;角色需要独立演化时才拆包,也可由同一包承担多个角色。以 `packages/shell` 为范例;Service Definition 是 Cordis `Service`(抽象类或具体 registry 服务),不是 TypeScript interface。任何单一角色、普通边界或扩展点都不能称为 seam。本仓库正文保留英文;与 `extension point` 是不同概念 |\n| Service Provider | Service Provider | | Service provider | 能力 seam 的命名角色;单数固定写作 Service Provider,复数写作 Service Providers。泛指提供服务的 provider 不适用本词条 |\n| skill | skill | skill(技能) | | |\n| slot | slot | | 坑位、孔位 | 客户端架构中的具名可注册位置,保留英文 |\n| spill | spill | | | 工具输出超限落盘机制;组合词写 `spill 文件`、`spill 路径` |\n| spawn | spawn | | | |\n| steering | steering | steering(中途引导) | | |\n| job id | job id | | 任务 id | 保留英文 |\n| subagent | subagent | | | |\n| transcript | transcript | transcript(文本记录) | | 指会话渲染给用户或编辑器的完整文本,区别于事件日志 |\n| Typert | Typert | | TypeRT、typeRT、Type RT | DeepSeek Harness 类型图、生成器、loader 与运行时 registry 的产品拼写 |\n| waterfall | waterfall | waterfall(瀑布式事件) | | |\n| wheel | wheel 包 | | | Python 打包格式 |\n| worktree | worktree | | | git 工作区概念 |\n| Zstandard | Zstandard | | | RFC 8878 compression format; `zstd` remains a code value. |\n\n## 双语类(中英文文本各自使用中英文)\n\n| English | 中文 | 首次出现 | 不要译作 | 备注 |\n|---|---|---|---|---|\n| adapter | 适配器 | | | |\n| adapter contract | 适配器约定 | 适配器约定(adapter contract) | | |\n| append-only | 仅追加 | | | |\n| artifact | 产物 | | 制品 | |\n| backend | 后端 | | | |\n| binder | 绑定器 | | | 命名角色:把已声明接口绑定到调用方 context 或生命周期 |\n| config | 配置 | | | 命名角色:一个已解析配置值或边界严格的配置记录 |\n| controller | 控制器 | | | 命名角色:接受意图并改变一项既有领域或展示状态 |\n| directory | 目录 | | | 命名角色:暴露供发现或选择的条目及元数据 |\n| engine | 引擎 | | | 命名角色:实现领域算法或有状态执行模型 |\n| gateway | 网关 | | | 命名角色:适配进程、网络、RPC 或 API 边界 |\n| handle | 句柄 | | | 命名角色:引用并控制或观察一个实时资源 |\n| policy | 策略 | | | 命名角色:决定允许、选择、限制或观察什么 |\n| presenter | 展示转换器 | | | 命名角色:把领域值纯转换为渲染意图 |\n| resolver | 解析器 | | | 命名角色:根据输入计算或定位一个答案 |\n| store | 存储 | | | 命名角色:拥有一组数据并主要提供数据操作 |\n| background job | 后台任务 | | | |\n| block | 块 | | | |\n| build target | 构建目标 | | | |\n| cancel | 取消 | | | |\n| canary test | canary 测试 | | 金丝雀测试 | 本仓库保留 `canary` |\n| capability | 能力 | | | 必须与 `feature` → `功能` 区分 |\n| capability seam | 能力 seam | | 功能 seam、能力接缝 | 本仓库 Service Definition、Service Provider 与 Consumer 三种角色组成完整可替换能力的命名架构概念;普通 `seam` 仍按其词条处理 |\n| feature | 功能 | | 能力 | SDK 产品与工程模型中的可管理产品单元 |\n| feature option | 功能选项 | | variant | 一项 SDK 功能内有限、可选择的实现或配置 |\n| checkpoint | 检查点 | | | |\n| chunk | 分片 | | | |\n| compaction | 压缩 | 压缩(compaction) | | |\n| companion tool | 配套工具 | | | |\n| composition bundle | 组合包 | | | 只约束应用或插件的组合语境,不约束所有 `bundle` |\n| Cordis plugin config | Cordis 插件配置 | | | Cordis 插件公开的 `Config` 对象或配置结构 |\n| config key | 配置键 | | | Cordis 插件配置中的单个字段 |\n| consumer | 消费方 | | 消费者 | |\n| content block | 内容块 | | | |\n| Cookbook | 实操手册 | | | 文档标题用语 |\n| context | 上下文 | | | |\n| counterpart | 对侧文件 | | 对应物、配对物 | 双语配对语境;泛指\"另一侧\"时可写「另一侧」 |\n| configurable-provider directory | 可配置提供方目录 | | | llm seam 中 `registerConfigurableProviders()` 维护的目录;沿用 Service Catalog →「服务目录」先例 |\n| context compaction | 上下文压缩 | 上下文压缩(context compaction) | | |\n| contract | 约定 | | | 如:`pairing contract` →`配对约定` |\n| Cordis config entry | Cordis 配置项 | | | 指 `cordis.yml` 插件列表中的一项;插件实现本身写`Cordis 插件` |\n| Cordis plugin | Cordis 插件 | | | Cordis 加载的插件实现,不指 `cordis.yml` 中的一项配置 |\n| crash recovery | 崩溃恢复 | | | |\n| deploy root | 部署根目录 | | | |\n| dormant | 休眠 | | 睡眠、蛰伏 | 指已声明可配置但当前未注册路由的提供方 |\n| durability | 持久性 | | | |\n| feature requirement | 功能依赖 | | | 功能或功能选项通过 `requires` 声明的关系 |\n| event | 事件 | | | |\n| event log | 事件日志 | | | |\n| event stream | 事件流 | | | |\n| event-sourced | 事件溯源 | | | 沿用 DDD 社区通行译法 |\n| Executive summary | 摘要 | | | 事故复盘标题用语 |\n| executor | 执行器 | | | |\n| expected output | 预期输出 | | 金标 | 指 snapshot 比较产物;翻译语料的人工校准样例不在此列 |\n| extension | 扩展 | | | |\n| extension point | 扩展点 | | | 注意与 `seam` 区分 |\n| fail-fast | 快速失败 | | | |\n| fenced code block | 围栏代码块 | | | 沿用 MDN 中文翻译 |\n| fingerprint | 指纹 | | | 通用内容指纹;双语配对机制使用 sidecar record 记录两侧 blob hash |\n| finish reason | 结束原因 | | | |\n| fold | 折叠区 | | | 配置界面语境:默认收起的字段分区(collapsed →「收起」)|\n| foreground run | 前台运行 | | | |\n| freshness | 新鲜度 | | | 沿用 MDN 中文翻译;在本项目中指译文相对源文的同步状态 |\n| hook | 钩子 | | | |\n| implementation | 实现 | | | |\n| inference | 推理 | 推理(inference) | | 需要和 `reasoning` 区分时保留英文括注 |\n| info string | 信息字符串 | | | 沿用 CommonMark 中文翻译;指代码围栏 ``` 之后的语言标注 |\n| injection | 注入 | | | |\n| integration | 集成 | | | |\n| interface | 接口 | | | |\n| language switcher | 语言切换行 | | | i18n 配对机制用语:双语配对文件顶部的互链行 |\n| merge | 合并 | | | |\n| message | 消息 | | | |\n| mod | 模组 | | | |\n| model provider | 模型提供方 | | | |\n| model selection | 模型选择 | | 模型目标 | 面向 Agent 的提供方、模型和可选推理强度选择。 |\n| module | 模块 | | | |\n| non-escalation | 非升权 | | 非升级、不可升级 | 仅用于安全与权限语境,指主体不得获得超出既有授权的权限;普通升级不适用此行 |\n| npm dependency | NPM 依赖 | | | `package.json` 中的包关系;`dependencies`、`devDependencies` 等字段保持原样 |\n| opt-out ratio | opt-out 比例 | | 退出检查比例 | |\n| orphan | 遗留 | | 孤儿、孤立 | 指英文源已不存在的 `.zh.md`(如「遗留译文」);进程语境按 OS 惯用语译「孤儿进程」 |\n| orphan branch | 孤立分支 | | 孤儿分支 | 沿用 git 官方中文翻译 |\n| package | 包 | | | 指 npm 包(`@deepseek-ai/dsh-*`);`package.json` 等代码标识保持原样 |\n| pairing | 配对 | | | |\n| parent-subset grants | 父级子集授权 | | 父集合授权 | 指授权范围仅限于父级所持授权的子集 |\n| peer dependency | 对等依赖 | 对等依赖(peer dependency) | | |\n| permission | 权限 | | | |\n| persistence | 持久化 | | | |\n| pipeline | 流水线 | | | |\n| plugin | 插件 | | | |\n| postmortem | 事故复盘 | 事故复盘(postmortem) | 事后分析、事故记录 | 事故记录与分析文档;目录或路径中的 `postmortem` 保持代码形式 |\n| prompt | 提示词 | | | |\n| provider | 提供方 | | | |\n| provider-neutral | 提供方无关 | | 提供方中立 | |\n| quality gate | 质量门禁 | | | |\n| quiescence | 完全停稳 | | 静默、静止状态 | 指生命周期工作全部结算后的状态 |\n| reasoning | 推理 | 推理(reasoning) | | 需要和 `inference` 区分时保留英文括注 |\n| reasoning_content | 思考内容 | | | |\n| registry | 注册表 | | | |\n| replay | 回放 | | | |\n| resume | 恢复 | | | |\n| runtime | 运行时 | | | |\n| same-world subprocess | 与宿主共享文件系统和内核的子进程 | | 同世界子进程 | |\n| sandbox | 沙箱 | | | |\n| service | 服务 | | | |\n| serving interface | 对外服务接口 | | | |\n| session | 会话 | | | |\n| session event | 会话事件 | | | |\n| setup card | 设置卡片 | | | 首次运行时代替行卡直接展开的配置卡 |\n| sidecar file | 伴随文件 | | | 指与文档同目录的普通伴随文件 |\n| sidecar record | 伴随记录 | | 旁挂记录 | 指与文档同目录的伴随记录文件 |\n| smoke test | 冒烟测试 | | | |\n| snapshot | 快照 | | | |\n| source of truth | 真源 | | 事实来源、唯一来源 | |\n| spine | 主干 | | | |\n| stale | 陈旧 | | 过期 | 与 `fresh`(`新鲜`)成对;门禁输出中保留英文 `stale` 不翻译;`expired` 才译为`过期` |\n| step | 步骤 | | | |\n| stream | 流 | | | |\n| structural signature | 结构签名 | | | i18n 配对机制用语:门禁比对两侧文件时提取的有序结构序列(标题层级、代码块、列表等) |\n| Summary | 概述 | | | 事故复盘标题用语 |\n| system prompt | 系统提示词 | | | |\n| taxonomy | 分类体系 | | | |\n| token usage | token 用量 | | | |\n| tool | 工具 | | | |\n| tool call | 工具调用 | | | |\n| tool result | 工具结果 | | | |\n| tool schema | 工具 schema | | | |\n| toolkit | 工具包 | | | |\n| turn | 轮次 | | | |\n| VFS | VFS | 虚拟文件系统(VFS) | | |\n| typecheck | 类型检查 | | | |\n| vocabulary | 词汇 | | | |\n| wire format | 协议格式 | 协议格式(wire format) | | |\n| workflow | 工作流 | | | |\n| wrapper | 包装层 | | | 软件层或 SDK 包装层 |\n| wrapper script | 包装脚本 | | | 可执行脚本包装层 |\n\n\n## Output Format\n\nReturn exactly three raw XML sections in the order shown below. Do not wrap the response in a Markdown code fence and do not add analysis or text before, between, or after the sections. The fence below only displays the required format; do not reproduce the fence.\n\nThe outer section tags are framing. If Markdown inside any section body contains a line consisting only of ``, ``, ``, ``, ``, or ``, prefix that line with `\\`. If the original line already has one or more backslashes immediately before the tag, add one more. The parser removes exactly one framing escape; tags mentioned inline need no escaping.\n\n```xml\n\n(First pass: the complete translation, written as natural target-language technical prose)\n\n\n\n(Second pass: actual corrections only, one correction per line with a category tag, e.g.)\n- [Tone] \"旁挂记录\" → \"伴随记录\"(生造词)\n- [Sentence] 第 3 段补充逗号断句\n- [Punctuation] 两处破折号替换为冒号\n- [Terminology: pending] source term → tentative rendering\n- 无修正\n\n\n\n(Complete final translation after corrections)\n\n```\n\n## Self-Review Instructions\n\nAfter writing ``, verify it in two directions. First re-read it in the target language only without comparing it with the source; this makes awkward phrasing easier to notice. Then compare it against the source clause by clause for completeness and exact meaning. Resolve doubts before writing ``; do not include reasoning transcripts, checks that passed, tentative suggestions, retractions, or no-op corrections.\n\n**Structure**\n- Are the heading hierarchy and order, list kind and item count, ordered-list start, table dimensions, and code block content identical to the source?\n- Are ALL comments and info strings inside code blocks left untranslated and byte-identical to the source?\n- Are inline code spans and machine-readable tokens verbatim?\n- Is an existing language switcher correctly flipped, and is no switcher or filename invented when the source lacks one?\n- Are link targets and emphasis spans preserved?\n- Does spacing across emphasis boundaries follow the same Chinese/Latin/numeral rule as ordinary prose?\n- Are wrapper-tag lines inside section bodies escaped with one additional backslash?\n\n**Faithfulness**\n- Clause by clause, is anything added, dropped, weakened, strengthened, reversed, merged, or re-bounded? Are list item counts identical on both sides?\n- Do actors, objects, conditions, exceptions, negation, modality, causal relationships, guarantees, contract directions, result channels, ownership changes, and quantities survive exactly?\n\n**Tone & Style**\n- Does every sentence read as if originally written by a native technical author?\n- Is there any colloquial, casual, overly informal, promotional, or metaphorical phrasing?\n- Are actors explicit where the target language needs them, without inventing responsibility?\n\n**Sentence Structure**\n- Are there run-on sentences that need breaking?\n- Are there stiff passive constructions that can safely become active, or active constructions that invent an actor?\n- Are conditions, concessions, negation, coordination, and modifiers scoped clearly?\n\n**Word Choice**\n- Are there overly literal translations that sound unnatural?\n- Are ordinary prose words left untranslated despite an established target-language expression?\n- Does each polysemous word fit its local context?\n- Is the same target-language word used for distinct source concepts, or is a defined term varied merely to avoid repetition?\n- Is any slang or internal jargon present?\n\n**Terminology**\n- For a Chinese target, are first-occurrence glosses correctly applied to the true first prose occurrence, neither missing nor repeated? For an English target, are Chinese glosses absent?\n- Are any \"不要译作\" forbidden translations present?\n- Do protected tokens remain untouched even when they resemble terminology entries?\n- For an unlisted term, does a Chinese target use an established Chinese rendering or preserve the source term as pending when no reliable rendering is known, and does an English target use the established English technical term or preserve only an ambiguous source term with the shortest necessary gloss and a pending notice?\n\n**Punctuation** (when target is Chinese)\n- Are punctuation, mixed-script spacing, quotation marks, Latin letters, and digits in their required forms?\n- Are there em dashes that make the sentence less clear and should be replaced, while natural em dashes remain intact?\n- Are list-item endings grammatically consistent, with none ending in commas?\n- Do RFC 2119 keywords preserve the source emphasis span and normative strength exactly?\n\nRecord actual corrections in ``, then output the corrected complete document in ``. If no correction or pending terminology notice is needed, write exactly `- 无修正` in `` and copy `` unchanged into ``. If `` contains only pending terminology notices, copy `` unchanged into ``.\n\n## Examples\n\nBelow are representative examples of common problems and their corrections. Follow the \"Good\" versions within the rule each example illustrates; examples do not override source context or higher-priority requirements.\n\n### Colloquial verb → Professional verb\n- Source: `The repo pins pnpm@11.7.0 in package.json`\n- Bad: `仓库在 package.json 中钉住 pnpm@11.7.0`\n- Good: `该仓库在 package.json 中固定使用 pnpm@11.7.0`\n\n### Run-on sentence → Natural phrasing with pause\n- Source: `Read docs/architecture.md before changing anything under packages/.`\n- Bad: `改动 packages/ 下的任何东西之前先读 docs/architecture.md。`\n- Good: `在修改 packages/ 目录下的任何内容之前,请先阅读 docs/architecture.md。`\n\n### Stiff passive voice → Active and natural\n- Source: `a green gate means the pair was confirmed consistent at these exact contents, not that the confirmation was sound.`\n- Bad: `门禁绿意味着这对文档曾在当前内容上被确认一致,不意味着这次确认本身是对的。`\n- Good: `门禁通过意味着这组文档在当前内容上的一致性得到了确认,不代表确认本身正确可靠。`\n\n### Invented word → Natural expression\n- Source: `A sidecar record of both blob hashes makes consistency checkable`\n- Bad: `旁挂记录两侧 blob hash,使一致性可检查`\n- Good: `伴随记录保存两侧 blob hash,使一致性可检查`\n\n### Em-dash → Colon/period\n- Source: `FIXME — an issue that should block a new release. A release should not ship with an open FIXME unless reviewers explicitly agree the change can be merged anyway.`\n- Bad: `FIXME——应当阻塞新版本发布的问题。除非评审者明确同意可以照常合入,发布不应带着未解决的 FIXME 出门。`\n- Good: `FIXME:应当阻塞新版本发布的问题。除非评审者明确同意该更改可以合并,否则发布版本不应包含未解决的 FIXME。`\n\n### Overly literal → Meaningful rendering\n- Source: `awkward phrasing is easier to notice when you read the translation without comparing it with the source`\n- Bad: `不把译文和原文比较时,尴尬的措辞更容易被注意`\n- Good: `不对照原文阅读译文时,更容易察觉别扭的表达`\n\n### Terminology — do not translate what should be kept in English\n- Source: `typed service seams, and explicit extension points`\n- Bad: `类型化的服务 seam(扩展点)与显式扩展点`\n- Good: `类型化的服务 seam 与显式扩展点`\n\n### Slang/jargon → Professional phrasing\n- Source: `The committed agent workflow lives in .agents/skills/dsh-translate-docs`\n- Bad: `进仓的 agent 工作流见 .agents/skills/dsh-translate-docs`\n- Good: `仓库内置的 agent 工作流见 .agents/skills/dsh-translate-docs`\n\n### \"For humans\" — translate the intent, not the word\n- Source: `For humans, start with the development guide`\n- Bad: `对于人工读者,请先从开发指南开始`(\"人工读者\"生硬)\n- Good: `面向开发者:请先阅读开发指南`(\"开发者\"自然,且中文里冒号在此处更自然)\n\n### Code block comments — NEVER translate\n- Source code block contains: `# full-screen TUI coding agent (needs DEEPSEEK_API_KEY)`\n- Bad: `# 全屏 TUI coding agent(需要 DEEPSEEK_API_KEY)`\n- Good: `# full-screen TUI coding agent (needs DEEPSEEK_API_KEY)` (keep exactly as-is, byte-for-byte)\n\n### Language switcher — flip direction\n- Source file (English) has: `English | [中文](README.zh.md)`\n- Bad (copying source unchanged): `English | [中文](README.zh.md)`\n- Good (flipped for Chinese file): `[English](README.md) | 中文`\n\n---\n\nNow translate the following document:" + "content": "# Translation Prompt\n\nYou are a senior technical translator specializing in LLM and agent development documentation. Your task is to translate the complete source document from English to Chinese, producing natural, professional technical prose.\n\nRead each complete semantic unit, understand it, and restate it as a native technical author would write it in the target language. Do not mechanically preserve source-language syntax. Then verify the translation against the source clause by clause: preserve every proposition and add none. Fluency never justifies losing or altering meaning, and completeness never justifies unnatural word-for-word prose.\n\n## Priority\n\nApply these authorities in order:\n\n1. Preserve the source meaning and the required document structure, protected content, and formatting.\n2. Follow the injected terminology table exactly.\n3. Use the injected whole-document gold pairs to calibrate target-language voice and phrasing.\n4. Apply the general writing guidance and illustrative examples in this prompt.\n\nA lower-priority rule may refine but never override a higher-priority requirement. Gold pairs calibrate voice; they are not a translation memory. No style preference, gold-pair phrasing, or embedded example may override source meaning, required structure, protected content, or the terminology table.\n\n## Quality Requirements\n\n### Structure and Format Preservation\n- Output a complete translated document that maintains the same document frame as the source: heading hierarchy and order, list kinds and item counts, ordered-list starts, table rows and columns, link order and semantic targets, and code blocks.\n- Paragraph boundaries may change within the same structural unit when the target language needs different semantic grouping. Do not merge or move content across headings, list items, table cells, or other independent structural units.\n- Keep each prose paragraph on one physical line. Use paragraph breaks, not hard-wrapped lines inside a paragraph.\n- Fenced code blocks must be byte-identical to the source, including info strings, whitespace, and ALL comments inside them. Do NOT translate or reformat any content inside code blocks. This is a hard rule with no exceptions.\n- Inline code spans must be kept verbatim. This includes commands, flags, paths, identifiers, API and event names, config keys, protocol values, version numbers, and other machine-readable tokens. Never translate or reformat them.\n- Every repository-relative document link must keep the source link's semantic target and exact query/fragment suffix. When the target belongs to the active bilingual corpus, English output uses its `.md` path and Chinese output uses its `.zh.md` path; a missing counterpart in that corpus is an error, while targets outside it keep the original path. External URLs, images, and pure in-page fragments stay unchanged. Translate link text.\n- Language switcher line: when an English source contains `English | [中文](source-filename.zh.md)`, write `[English](source-filename.md) | 中文`. When a Chinese source contains `[English](source-filename.md) | 中文`, write `English | [中文](source-filename.zh.md)`. Do NOT copy the source switcher unchanged. If the source has no switcher, do not invent a filename or switcher; the pipeline inserts the canonical target switcher after parsing ``.\n- Preserve emphasis marker types and the semantic spans they cover. Do not add, remove, move, or change bold and italic markers.\n\n### Faithfulness\n- Preserve every proposition in the source and add none. Every sentence, list item, note, FIXME, warning, example, caveat, prerequisite, and guarantee must have an equivalent in the translation. Count list items on both sides.\n- Preserve actors, objects, conditions, exceptions, negation, modality, causal relationships, and distinctions between concepts.\n- Preserve the exact strength and orientation of contracts. Completion and lifecycle conditions, failure behavior, directions and data flow, normal and exceptional result channels, ownership changes, and quantitative bounds must not be weakened, strengthened, reversed, or merged.\n- Translate ideas rather than source-language idioms, but never use fluency as a reason to omit or alter meaning.\n\n### Tone and Style\n- The translation must read as if originally written in the target language by a native technical author. If an expression sounds like a word-for-word rendering from the source language, rephrase it.\n- Write in a professional, formal tone appropriate for developer documentation. Never use colloquial or casual expressions.\n- Name an actor when the target language would otherwise obscure an actor that the source states or unambiguously implies. Never invent responsibility merely to avoid a passive construction.\n- Prefer established target-language engineering terms over literal renderings. Replace metaphors with direct descriptions that preserve the source meaning.\n- Use polite imperative forms where the text instructs the reader to do something. In Chinese, address the reader as `你`, not `您`.\n- Keep the author's register: concise stays concise, detailed stays detailed.\n\n### Sentence Structure\n- Break long sentences where the target language needs a pause. Avoid run-on sentences.\n- Use active voice when it improves clarity without changing or inventing the actor. Retain passive voice when the actor is unknown, irrelevant, or intentionally omitted.\n- Restructure source-language syntax into clear target-language syntax. Preserve the logical scope of conditions, concessions, negation, coordination, and modifiers.\n- Split or combine clauses when needed for readability, provided every source relationship remains explicit.\n- Translate meaning, not words. Do not invent words or expressions that a native technical author would not use.\n\n### Word Choice\n- Prefer precise, formal vocabulary over casual or colloquial alternatives.\n- When multiple synonyms exist, choose the one most commonly used in professional technical documentation of the target language.\n- Translate ordinary prose when an established target-language expression is clear. Preserve proper nouns, canonical product names, code identifiers, APIs, paths, package names, and terms that the terminology table requires to remain in the source language.\n- Use context to resolve polysemous words. A familiar word does not have one fixed rendering in every technical domain.\n- Avoid slang, internal jargon, or overly literal translations that would not be recognized by the general developer audience.\n- Do not use the same word to translate distinct source-language concepts when their distinction matters.\n- Avoid repeating the same ordinary verb in close proximity when a natural equivalent preserves the exact meaning. Never vary a terminology-table form, defined concept, or contract verb merely for stylistic variety.\n\n#### When translating into Chinese\n- When a number modifies a noun, include a natural Chinese classifier or measure word when Chinese grammar requires one. For example: \"three-role capability seam\" → \"包含三种角色的能力 seam\", not \"三角色 seam\". Do not add classifiers to code, identifiers, versions, units, or fixed names.\n\n### Punctuation\n\n#### When translating into Chinese\n- Use full-width Chinese punctuation in Chinese prose: `,。:;?!()「」`. Keep half-width punctuation inside code spans, numbers, and complete verbatim English text.\n- Prefer colons, periods, commas, or parentheses over em dashes when they make the sentence clearer or more natural. Keep an em dash when it is the clearest natural punctuation.\n- Use enumeration commas (、) between parallel Chinese items, not regular commas.\n- Keep list-item endings consistent with their grammar. Complete sentences may end with periods or other grammatically required punctuation; do not end list items with commas.\n- Put one half-width space between Chinese text and Latin words or numerals. Do not add a space next to full-width punctuation, and do not leave a meaningless half-width space between two Chinese characters.\n- Markdown emphasis markers do not create a word boundary. Determine spacing from the rendered adjacent characters: Chinese next to Chinese takes no space, while Chinese next to a Latin word or numeral takes one half-width space.\n- Use half-width digits and Latin letters, never full-width forms.\n- For RFC 2119 keywords (MUST, MUST NOT, SHOULD, MAY), translate to the corresponding Chinese term (必须、禁止、应当、可以), preserve the SOURCE emphasis span exactly, and do not weaken its normative strength: plain source stays plain (必须), italic source stays italic (*必须*), and bold source stays bold (**必须**).\n\n#### When translating into English\n- Use half-width English punctuation and standard English spacing. Preserve full-width punctuation only in verbatim Chinese text.\n- Convert enumeration commas (、) to English commas and Chinese prose quotation marks to English double quotes.\n- Convert Chinese topic-comment sentences and omitted-subject constructions into clear English subjects when the actor is stated or unambiguously implied. Do not invent an actor.\n- Use concise professional developer prose and established English technical terms. Do not transliterate Chinese engineering idioms literally.\n- Use the terminology table's English column exactly and do not carry Chinese first-occurrence glosses into English prose.\n\n## Terminology\n\nA terminology table is provided below. Follow it strictly:\n- Render every listed term exactly as specified.\n- When the target language is Chinese, use the \"中文\" column. On the document's first prose occurrence, write the \"首次出现\" value when one is specified; on later occurrences, write only the part before the parenthetical gloss.\n- When the target language is English, use the \"English\" column without a Chinese gloss; do not copy the \"中文\" or \"首次出现\" value into English prose.\n- If a term has already been glossed as part of a compound term, do not gloss it again when it appears alone later.\n- NEVER use translations listed in the \"不要译作\" column.\n- Code spans and other protected tokens remain verbatim even when their text resembles a listed term.\n- For an unlisted technical term, use an established target-language technical term when its meaning is unambiguous in context. For a Chinese target, use an established Chinese rendering from a major Chinese-language OSS or vendor source; if you cannot reliably determine such a rendering, preserve the source term and record `[Terminology: pending]` in `` with a tentative rendering for human review. For an English target, use the established English technical term; if the source term has no unambiguous established equivalent, preserve it with the shortest English gloss needed to make it intelligible and record `[Terminology: pending]` in ``. A tentative rendering may appear in `` but must not be silently adopted in `` or ``, and you must not invent or claim a specific external precedent. This rule applies to terminology only; for general prose, freely restructure and paraphrase for natural expression.\n\n# Terminology\n\n本表约定本仓库的中英术语统一译法。\n\n**通用规则:**\n- \"中文\"列为中文译文的正文默认用词。若该列为英文,则中文译文的正文中保留英文不翻译。\n- 首次出现按\"首次出现\"列书写(带括号注释);后续出现只写括号前的部分(可能为中文,也可能为英文),不出现括号内的注释。\n- \"不要译作\"列为严格禁止的译法。\n- 如果某术语已经作为另一个术语的组成部分被括注过(如 `agent loop(智能体循环)` 中已包含 `agent` 的括注),则该术语后续单独出现时无需再次括注。\n\n## 缩写类(中英文文本中均使用缩写)\n\n| English | 中文 | 首次出现 | 不要译作 | 备注 |\n|---|---|---|---|---|\n| ACP | ACP | ACP(Agent Client Protocol) | | |\n| AI | AI | AI(人工智能) | | |\n| API | API | | | |\n| CI | CI | | | |\n| CLI | CLI | CLI(命令行界面) | | |\n| e2e | e2e | | | |\n| HMR | HMR | HMR(热模块替换) | | |\n| JSON Schema | JSON Schema | | | |\n| JSONL | JSONL | | | |\n| LLM | LLM | LLM(大语言模型) | | |\n| MCP | MCP | | | |\n| PR | PR | PR(Pull Request) | | |\n| RAG | RAG | RAG(检索增强生成) | | |\n| SDK | SDK | | | 只指受支持的 Python 与 TypeScript SDK 所使用的 JSON-RPC 客户端/服务器协议;DeepSeek Harness 项目本身不是 SDK |\n| SSE | SSE | SSE(Server-Sent Events) | | |\n\n## 英文类(中英文文本中均使用英文)\n\n| English | 中文 | 首次出现 | 不要译作 | 备注 |\n|---|---|---|---|---|\n| agent | agent | agent(智能体) | | |\n| Agent Note | Agent Note | | 智能体注记、智能体笔记 | 仓库定义的文档类型,涵盖提案、已实现决策和被否决提案;中文对侧 H1 保持固定前缀 `# Agent Note: `,标题中不加术语括注 |\n| agent harness | agent harness | agent harness(智能体框架) | | agent 组合词(agent harness/workflow/loop/skill 等)整体保留英文;未括注过 agent 时首现按对应组合词或 agent 行处理 |\n| agent loop | agent loop | agent loop(智能体循环) | | |\n| blob hash | blob hash | | | `git hash-object` 的结果 |\n| coding agent | coding agent | coding agent(编程智能体) | | agent 组合词,正文保留英文 |\n| Cordis | Cordis | | | |\n| dispose | dispose | dispose(资源释放) | | |\n| doc-sync | doc-sync | doc-sync(文档同步门禁) | | |\n| fiber | fiber | | | |\n| fixture | fixture | fixture(测试前置数据) | | |\n| fork | fork | | | |\n| Function Calling | Function Calling | Function Calling(函数调用) | | |\n| harness | harness | | | |\n| harness engineering | harness engineering | | | |\n| KV Cache | KV Cache | | | 专有技术名称,保持大小写与空格 |\n| lint | lint | | | |\n| mock | mock | | | 保留英文;指测试替身 |\n| loader | loader | | | |\n| manifest | manifest | manifest(元数据清单) | | |\n| monorepo | monorepo | | | |\n| Round | Round | | 回合、目标回合、Ralph 回合 | 外层策略使用 Round 时,领域层级为 Session > Round > Turn(轮次) > Step(步骤);Round 是可选的外层策略迭代,并非每个会话轮次都具有的通用层级。Goal Round 与 Ralph Round 均保留英文。一个 Round 承载一个轮次,步骤隶属于该轮次;明确的零步骤轮次仍保持原义。 |\n| schema | schema | | | |\n| schema DSL | schema DSL | | | |\n| seam | seam | | 接缝 | 一个可替换能力的整体,包含 Service Definition / Service Provider / Consumer 三种角色;角色需要独立演化时才拆包,也可由同一包承担多个角色。以 `packages/shell` 为范例;Service Definition 是 Cordis `Service`(抽象类或具体 registry 服务),不是 TypeScript interface。任何单一角色、普通边界或扩展点都不能称为 seam。本仓库正文保留英文;与 `extension point` 是不同概念 |\n| Service Provider | Service Provider | | Service provider | 能力 seam 的命名角色;单数固定写作 Service Provider,复数写作 Service Providers。泛指提供服务的 provider 不适用本词条 |\n| skill | skill | skill(技能) | | |\n| slot | slot | | 坑位、孔位 | 客户端架构中的具名可注册位置,保留英文 |\n| spill | spill | | | 工具输出超限落盘机制;组合词写 `spill 文件`、`spill 路径` |\n| spawn | spawn | | | |\n| steering | steering | steering(中途引导) | | |\n| job id | job id | | 任务 id | 保留英文 |\n| subagent | subagent | | | |\n| transcript | transcript | transcript(文本记录) | | 指会话渲染给用户或编辑器的完整文本,区别于事件日志 |\n| Typert | Typert | | TypeRT、typeRT、Type RT | DeepSeek Harness 类型图、生成器、loader 与运行时 registry 的产品拼写 |\n| waterfall | waterfall | waterfall(瀑布式事件) | | |\n| wheel | wheel 包 | | | Python 打包格式 |\n| worktree | worktree | | | git 工作区概念 |\n| Zstandard | Zstandard | | | RFC 8878 compression format; `zstd` remains a code value. |\n\n## 双语类(中英文文本各自使用中英文)\n\n| English | 中文 | 首次出现 | 不要译作 | 备注 |\n|---|---|---|---|---|\n| adapter | 适配器 | | | |\n| adapter contract | 适配器约定 | 适配器约定(adapter contract) | | |\n| append-only | 仅追加 | | | |\n| artifact | 产物 | | 制品 | |\n| backend | 后端 | | | |\n| binder | 绑定器 | | | 命名角色:把已声明接口绑定到调用方 context 或生命周期 |\n| config | 配置 | | | 命名角色:一个已解析配置值或边界严格的配置记录 |\n| controller | 控制器 | | | 命名角色:接受意图并改变一项既有领域或展示状态 |\n| directory | 目录 | | | 命名角色:暴露供发现或选择的条目及元数据 |\n| engine | 引擎 | | | 命名角色:实现领域算法或有状态执行模型 |\n| gateway | 网关 | | | 命名角色:适配进程、网络、RPC 或 API 边界 |\n| handle | 句柄 | | | 命名角色:引用并控制或观察一个实时资源 |\n| policy | 策略 | | | 命名角色:决定允许、选择、限制或观察什么 |\n| presenter | 展示转换器 | | | 命名角色:把领域值纯转换为渲染意图 |\n| resolver | 解析器 | | | 命名角色:根据输入计算或定位一个答案 |\n| store | 存储 | | | 命名角色:拥有一组数据并主要提供数据操作 |\n| background job | 后台任务 | | | |\n| block | 块 | | | |\n| build target | 构建目标 | | | |\n| cancel | 取消 | | | |\n| canary test | canary 测试 | | 金丝雀测试 | 本仓库保留 `canary` |\n| capability | 能力 | | | 必须与 `feature` → `功能` 区分 |\n| capability seam | 能力 seam | | 功能 seam、能力接缝 | 本仓库 Service Definition、Service Provider 与 Consumer 三种角色组成完整可替换能力的命名架构概念;普通 `seam` 仍按其词条处理 |\n| feature | 功能 | | 能力 | SDK 产品与工程模型中的可管理产品单元 |\n| feature option | 功能选项 | | variant | 一项 SDK 功能内有限、可选择的实现或配置 |\n| checkpoint | 检查点 | | | |\n| chunk | 分片 | | | |\n| compaction | 压缩 | 压缩(compaction) | | |\n| companion tool | 配套工具 | | | |\n| composition bundle | 组合包 | | | 只约束应用或插件的组合语境,不约束所有 `bundle` |\n| Cordis plugin config | Cordis 插件配置 | | | Cordis 插件公开的 `Config` 对象或配置结构 |\n| config key | 配置键 | | | Cordis 插件配置中的单个字段 |\n| consumer | 消费方 | | 消费者 | |\n| content block | 内容块 | | | |\n| Cookbook | 实操手册 | | | 文档标题用语 |\n| context | 上下文 | | | |\n| counterpart | 对侧文件 | | 对应物、配对物 | 双语配对语境;泛指\"另一侧\"时可写「另一侧」 |\n| configurable-provider directory | 可配置提供方目录 | | | llm seam 中 `registerConfigurableProviders()` 维护的目录;沿用 Service Catalog →「服务目录」先例 |\n| context compaction | 上下文压缩 | 上下文压缩(context compaction) | | |\n| contract | 约定 | | | 如:`pairing contract` →`配对约定` |\n| Cordis config entry | Cordis 配置项 | | | 指 `cordis.yml` 插件列表中的一项;插件实现本身写`Cordis 插件` |\n| Cordis plugin | Cordis 插件 | | | Cordis 加载的插件实现,不指 `cordis.yml` 中的一项配置 |\n| crash recovery | 崩溃恢复 | | | |\n| deploy root | 部署根目录 | | | |\n| dormant | 休眠 | | 睡眠、蛰伏 | 指已声明可配置但当前未注册路由的提供方 |\n| durability | 持久性 | | | |\n| feature requirement | 功能依赖 | | | 功能或功能选项通过 `requires` 声明的关系 |\n| event | 事件 | | | |\n| event log | 事件日志 | | | |\n| event stream | 事件流 | | | |\n| event-sourced | 事件溯源 | | | 沿用 DDD 社区通行译法 |\n| Executive summary | 摘要 | | | 事故复盘标题用语 |\n| executor | 执行器 | | | |\n| expected output | 预期输出 | | 金标 | 指 snapshot 比较产物;翻译语料的人工校准样例不在此列 |\n| extension | 扩展 | | | |\n| extension point | 扩展点 | | | 注意与 `seam` 区分 |\n| fail-fast | 快速失败 | | | |\n| fenced code block | 围栏代码块 | | | 沿用 MDN 中文翻译 |\n| fingerprint | 指纹 | | | 通用内容指纹;双语配对机制使用 sidecar record 记录两侧 blob hash |\n| finish reason | 结束原因 | | | |\n| fold | 折叠区 | | | 配置界面语境:默认收起的字段分区(collapsed →「收起」)|\n| foreground run | 前台运行 | | | |\n| freshness | 新鲜度 | | | 沿用 MDN 中文翻译;在本项目中指译文相对源文的同步状态 |\n| hook | 钩子 | | | |\n| implementation | 实现 | | | |\n| inference | 推理 | 推理(inference) | | 需要和 `reasoning` 区分时保留英文括注 |\n| info string | 信息字符串 | | | 沿用 CommonMark 中文翻译;指代码围栏 ``` 之后的语言标注 |\n| injection | 注入 | | | |\n| integration | 集成 | | | |\n| interface | 接口 | | | |\n| language switcher | 语言切换行 | | | i18n 配对机制用语:双语配对文件顶部的互链行 |\n| merge | 合并 | | | |\n| message | 消息 | | | |\n| mod | 模组 | | | |\n| model provider | 模型提供方 | | | |\n| model selection | 模型选择 | | 模型目标 | 面向 Agent 的提供方、模型和可选推理强度选择。 |\n| module | 模块 | | | |\n| non-escalation | 非升权 | | 非升级、不可升级 | 仅用于安全与权限语境,指主体不得获得超出既有授权的权限;普通升级不适用此行 |\n| npm dependency | NPM 依赖 | | | `package.json` 中的包关系;`dependencies`、`devDependencies` 等字段保持原样 |\n| opt-out ratio | opt-out 比例 | | 退出检查比例 | |\n| orphan | 遗留 | | 孤儿、孤立 | 指英文源已不存在的 `.zh.md`(如「遗留译文」);进程语境按 OS 惯用语译「孤儿进程」 |\n| orphan branch | 孤立分支 | | 孤儿分支 | 沿用 git 官方中文翻译 |\n| package | 包 | | | 指 npm 包(`@deepseek-ai/dsh-*`);`package.json` 等代码标识保持原样 |\n| pairing | 配对 | | | |\n| parent-subset grants | 父级子集授权 | | 父集合授权 | 指授权范围仅限于父级所持授权的子集 |\n| peer dependency | 对等依赖 | 对等依赖(peer dependency) | | |\n| permission | 权限 | | | |\n| persistence | 持久化 | | | |\n| pipeline | 流水线 | | | |\n| plugin | 插件 | | | |\n| postmortem | 事故复盘 | 事故复盘(postmortem) | 事后分析、事故记录 | 事故记录与分析文档;目录或路径中的 `postmortem` 保持代码形式 |\n| prompt | 提示词 | | | |\n| provider | 提供方 | | | |\n| provider-neutral | 提供方无关 | | 提供方中立 | |\n| quality gate | 质量门禁 | | | |\n| quiescence | 完全停稳 | | 静默、静止状态 | 指生命周期工作全部结算后的状态 |\n| reasoning | 推理 | 推理(reasoning) | | 需要和 `inference` 区分时保留英文括注 |\n| reasoning_content | 思考内容 | | | |\n| registry | 注册表 | | | |\n| replay | 回放 | | | |\n| resume | 恢复 | | | |\n| runtime | 运行时 | | | |\n| same-world subprocess | 与宿主共享文件系统和内核的子进程 | | 同世界子进程 | |\n| sandbox | 沙箱 | | | |\n| service | 服务 | | | |\n| serving interface | 对外服务接口 | | | |\n| session | 会话 | | | |\n| session event | 会话事件 | | | |\n| setup card | 设置卡片 | | | 首次运行时代替行卡直接展开的配置卡 |\n| sidecar file | 伴随文件 | | | 指与文档同目录的普通伴随文件 |\n| sidecar record | 伴随记录 | | 旁挂记录 | 指与文档同目录的伴随记录文件 |\n| smoke test | 冒烟测试 | | | |\n| snapshot | 快照 | | | |\n| source of truth | 真源 | | 事实来源、唯一来源 | |\n| spine | 主干 | | | |\n| stale | 陈旧 | | 过期 | 与 `fresh`(`新鲜`)成对;门禁输出中保留英文 `stale` 不翻译;`expired` 才译为`过期` |\n| step | 步骤 | | | |\n| stream | 流 | | | |\n| structural signature | 结构签名 | | | i18n 配对机制用语:门禁比对两侧文件时提取的有序结构序列(标题层级、代码块、列表等) |\n| Summary | 概述 | | | 事故复盘标题用语 |\n| system prompt | 系统提示词 | | | |\n| taxonomy | 分类体系 | | | |\n| token usage | token 用量 | | | |\n| tool | 工具 | | | |\n| tool call | 工具调用 | | | |\n| tool result | 工具结果 | | | |\n| tool schema | 工具 schema | | | |\n| toolkit | 工具包 | | | |\n| turn | 轮次 | | | |\n| VFS | VFS | 虚拟文件系统(VFS) | | |\n| typecheck | 类型检查 | | | |\n| vocabulary | 词汇 | | | |\n| wire format | 协议格式 | 协议格式(wire format) | | |\n| workflow | 工作流 | | | |\n| wrapper | 包装层 | | | 软件层或 SDK 包装层 |\n| wrapper script | 包装脚本 | | | 可执行脚本包装层 |\n\n\n## Output Format\n\nReturn exactly three raw XML sections in the order shown below. Do not wrap the response in a Markdown code fence and do not add analysis or text before, between, or after the sections. The fence below only displays the required format; do not reproduce the fence.\n\nThe outer section tags are framing. If Markdown inside any section body contains a line consisting only of ``, ``, ``, ``, ``, or ``, prefix that line with `\\`. If the original line already has one or more backslashes immediately before the tag, add one more. The parser removes exactly one framing escape; tags mentioned inline need no escaping.\n\n```xml\n\n(First pass: the complete translation, written as natural target-language technical prose)\n\n\n\n(Second pass: actual corrections only, one correction per line with a category tag, e.g.)\n- [Tone] \"旁挂记录\" → \"伴随记录\"(生造词)\n- [Sentence] 第 3 段补充逗号断句\n- [Punctuation] 两处破折号替换为冒号\n- [Terminology: pending] source term → tentative rendering\n- 无修正\n\n\n\n(Complete final translation after corrections)\n\n```\n\n## Self-Review Instructions\n\nAfter writing ``, verify it in two directions. First re-read it in the target language only without comparing it with the source; this makes awkward phrasing easier to notice. Then compare it against the source clause by clause for completeness and exact meaning. Resolve doubts before writing ``; do not include reasoning transcripts, checks that passed, tentative suggestions, retractions, or no-op corrections.\n\n**Structure**\n- Are the heading hierarchy and order, list kind and item count, ordered-list start, table dimensions, and code block content identical to the source?\n- Are ALL comments and info strings inside code blocks left untranslated and byte-identical to the source?\n- Are inline code spans and machine-readable tokens verbatim?\n- Is an existing language switcher correctly flipped, and is no switcher or filename invented when the source lacks one?\n- Do links preserve their semantic targets and exact query/fragment suffixes while using target-locale paths, and are emphasis spans preserved?\n- Does spacing across emphasis boundaries follow the same Chinese/Latin/numeral rule as ordinary prose?\n- Are wrapper-tag lines inside section bodies escaped with one additional backslash?\n\n**Faithfulness**\n- Clause by clause, is anything added, dropped, weakened, strengthened, reversed, merged, or re-bounded? Are list item counts identical on both sides?\n- Do actors, objects, conditions, exceptions, negation, modality, causal relationships, guarantees, contract directions, result channels, ownership changes, and quantities survive exactly?\n\n**Tone & Style**\n- Does every sentence read as if originally written by a native technical author?\n- Is there any colloquial, casual, overly informal, promotional, or metaphorical phrasing?\n- Are actors explicit where the target language needs them, without inventing responsibility?\n\n**Sentence Structure**\n- Are there run-on sentences that need breaking?\n- Are there stiff passive constructions that can safely become active, or active constructions that invent an actor?\n- Are conditions, concessions, negation, coordination, and modifiers scoped clearly?\n\n**Word Choice**\n- Are there overly literal translations that sound unnatural?\n- Are ordinary prose words left untranslated despite an established target-language expression?\n- Does each polysemous word fit its local context?\n- Is the same target-language word used for distinct source concepts, or is a defined term varied merely to avoid repetition?\n- Is any slang or internal jargon present?\n\n**Terminology**\n- For a Chinese target, are first-occurrence glosses correctly applied to the true first prose occurrence, neither missing nor repeated? For an English target, are Chinese glosses absent?\n- Are any \"不要译作\" forbidden translations present?\n- Do protected tokens remain untouched even when they resemble terminology entries?\n- For an unlisted term, does a Chinese target use an established Chinese rendering or preserve the source term as pending when no reliable rendering is known, and does an English target use the established English technical term or preserve only an ambiguous source term with the shortest necessary gloss and a pending notice?\n\n**Punctuation** (when target is Chinese)\n- Are punctuation, mixed-script spacing, quotation marks, Latin letters, and digits in their required forms?\n- Are there em dashes that make the sentence less clear and should be replaced, while natural em dashes remain intact?\n- Are list-item endings grammatically consistent, with none ending in commas?\n- Do RFC 2119 keywords preserve the source emphasis span and normative strength exactly?\n\nRecord actual corrections in ``, then output the corrected complete document in ``. If no correction or pending terminology notice is needed, write exactly `- 无修正` in `` and copy `` unchanged into ``. If `` contains only pending terminology notices, copy `` unchanged into ``.\n\n## Examples\n\nBelow are representative examples of common problems and their corrections. Follow the \"Good\" versions within the rule each example illustrates; examples do not override source context or higher-priority requirements.\n\n### Colloquial verb → Professional verb\n- Source: `The repo pins pnpm@11.7.0 in package.json`\n- Bad: `仓库在 package.json 中钉住 pnpm@11.7.0`\n- Good: `该仓库在 package.json 中固定使用 pnpm@11.7.0`\n\n### Run-on sentence → Natural phrasing with pause\n- Source: `Read docs/architecture.md before changing anything under packages/.`\n- Bad: `改动 packages/ 下的任何东西之前先读 docs/architecture.md。`\n- Good: `在修改 packages/ 目录下的任何内容之前,请先阅读 docs/architecture.md。`\n\n### Stiff passive voice → Active and natural\n- Source: `a green gate means the pair was confirmed consistent at these exact contents, not that the confirmation was sound.`\n- Bad: `门禁绿意味着这对文档曾在当前内容上被确认一致,不意味着这次确认本身是对的。`\n- Good: `门禁通过意味着这组文档在当前内容上的一致性得到了确认,不代表确认本身正确可靠。`\n\n### Invented word → Natural expression\n- Source: `A sidecar record of both blob hashes makes consistency checkable`\n- Bad: `旁挂记录两侧 blob hash,使一致性可检查`\n- Good: `伴随记录保存两侧 blob hash,使一致性可检查`\n\n### Em-dash → Colon/period\n- Source: `FIXME — an issue that should block a new release. A release should not ship with an open FIXME unless reviewers explicitly agree the change can be merged anyway.`\n- Bad: `FIXME——应当阻塞新版本发布的问题。除非评审者明确同意可以照常合入,发布不应带着未解决的 FIXME 出门。`\n- Good: `FIXME:应当阻塞新版本发布的问题。除非评审者明确同意该更改可以合并,否则发布版本不应包含未解决的 FIXME。`\n\n### Overly literal → Meaningful rendering\n- Source: `awkward phrasing is easier to notice when you read the translation without comparing it with the source`\n- Bad: `不把译文和原文比较时,尴尬的措辞更容易被注意`\n- Good: `不对照原文阅读译文时,更容易察觉别扭的表达`\n\n### Terminology — do not translate what should be kept in English\n- Source: `typed service seams, and explicit extension points`\n- Bad: `类型化的服务 seam(扩展点)与显式扩展点`\n- Good: `类型化的服务 seam 与显式扩展点`\n\n### Slang/jargon → Professional phrasing\n- Source: `The committed agent workflow lives in .agents/skills/dsh-translate-docs`\n- Bad: `进仓的 agent 工作流见 .agents/skills/dsh-translate-docs`\n- Good: `仓库内置的 agent 工作流见 .agents/skills/dsh-translate-docs`\n\n### \"For humans\" — translate the intent, not the word\n- Source: `For humans, start with the development guide`\n- Bad: `对于人工读者,请先从开发指南开始`(\"人工读者\"生硬)\n- Good: `面向开发者:请先阅读开发指南`(\"开发者\"自然,且中文里冒号在此处更自然)\n\n### Code block comments — NEVER translate\n- Source code block contains: `# full-screen TUI coding agent (needs DEEPSEEK_API_KEY)`\n- Bad: `# 全屏 TUI coding agent(需要 DEEPSEEK_API_KEY)`\n- Good: `# full-screen TUI coding agent (needs DEEPSEEK_API_KEY)` (keep exactly as-is, byte-for-byte)\n\n### Language switcher — flip direction\n- Source file (English) has: `English | [中文](README.zh.md)`\n- Bad (copying source unchanged): `English | [中文](README.zh.md)`\n- Good (flipped for Chinese file): `[English](README.md) | 中文`\n\n---\n\nNow translate the following document:" }, { "role": "user", diff --git a/scripts/test-invariants.ts b/scripts/test-invariants.ts index a9f4cf335b..a3b96a90a3 100644 --- a/scripts/test-invariants.ts +++ b/scripts/test-invariants.ts @@ -117,6 +117,7 @@ class TestAttachmentStore extends AttachmentStore { maxImagesPerMessage: 1, maxMessageImageBytes: 1, maxImagePixels: 1, + maxImageDimension: 1, mediaTypes: ['image/png'], } diff --git a/scripts/translation-brief.spec.ts b/scripts/translation-brief.spec.ts index 0e0bc8b24c..85a49a0179 100644 --- a/scripts/translation-brief.spec.ts +++ b/scripts/translation-brief.spec.ts @@ -215,6 +215,8 @@ describe('brief rendering', () => { expect(brief).toContain('## First-occurrence notes') expect(brief).toContain('agent(智能体)') expect(brief).toContain('首次出现 annotations attach to the document-wide first occurrence only') + expect(brief).toContain('targets in the active bilingual corpus use `.zh.md` for Chinese') + expect(brief).toContain('a missing in-scope counterpart is an error') expect(brief).toContain('verify-translation-pairing --write docs/foo.md') }) @@ -266,6 +268,8 @@ describe('brief rendering', () => { scope: { kind: 'units', bundles: [bundle], firstOccurrenceNotes: [] }, }) expect(brief).toContain('exactly what the new Chinese states') + expect(brief).toContain('targets in the active bilingual corpus use `.md` for English') + expect(brief).toContain('targets outside the corpus keep the authored path') expect(brief).toContain('verify-translation-pairing --write docs/foo.md') }) diff --git a/scripts/translation-brief.ts b/scripts/translation-brief.ts index 9380974fa0..1c76bccc31 100644 --- a/scripts/translation-brief.ts +++ b/scripts/translation-brief.ts @@ -386,7 +386,7 @@ const ZH_TARGET_DIGEST = [ '- Nothing added, nothing dropped: the Chinese must state exactly what the new English states.', '- Write natural institutional technical Chinese, not word-by-word gloss; terse stays terse.', '- Code fences byte-identical to the English side, comments included; inline code spans verbatim.', - '- Relative links keep the `.md` target; only the switcher line links `.zh.md`.', + '- Repository-relative document links keep the same semantic target and exact query/fragment; targets in the active bilingual corpus use `.zh.md` for Chinese, a missing in-scope counterpart is an error, and targets outside the corpus keep the authored path. The switcher remains the cross-locale exception.', '- Structure mirrors the counterpart: heading depths and order, list kinds and item counts, table rows and columns.', '- 首次出现 annotations attach to the document-wide first occurrence only; later occurrences use the bare form, and an empty 首次出现 cell means never gloss.', '- Typography: one half-width space between Chinese and Latin or digits; full-width punctuation in Chinese prose; 顿号 for enumerations; second person is 你.', @@ -398,7 +398,7 @@ const EN_TARGET_DIGEST = [ '- Nothing added, nothing dropped: the English must state exactly what the new Chinese states.', '- Write concise professional developer prose, not word-by-word gloss; terse stays terse.', '- Code fences byte-identical to the Chinese side, comments included; inline code spans verbatim.', - '- Relative links keep the `.md` target; only the switcher line links `.zh.md`.', + '- Repository-relative document links keep the same semantic target and exact query/fragment; targets in the active bilingual corpus use `.md` for English, a missing in-scope counterpart is an error, and targets outside the corpus keep the authored path. The switcher remains the cross-locale exception.', '- Structure mirrors the counterpart: heading depths and order, list kinds and item counts, table rows and columns.', '- One physical line per paragraph; exactly one trailing newline.', ] diff --git a/scripts/translation-links.spec.ts b/scripts/translation-links.spec.ts new file mode 100644 index 0000000000..35efde120e --- /dev/null +++ b/scripts/translation-links.spec.ts @@ -0,0 +1,253 @@ +/** Regression coverage for locale-aware bilingual Markdown links. */ + +import { mkdirSync, mkdtempSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { + normalizeTranslationMarkdownLinks, + rewriteTranslationLinkLocales, + translationLinkLocaleViolations, + type TranslationLinkContext, +} from './translation-links.ts' +import { removeFixtureSafely } from './test-fixture-cleanup.ts' + +const roots: string[] = [] + +afterEach(() => { + for (const root of roots.splice(0)) removeFixtureSafely(root) +}) + +function fixture(): string { + const root = mkdtempSync(join(tmpdir(), 'dsh-translation-links-')) + roots.push(root) + mkdirSync(join(root, 'docs/section'), { recursive: true }) + mkdirSync(join(root, 'packages'), { recursive: true }) + writeFileSync(join(root, 'docs/guide.md'), '# Guide\n') + writeFileSync(join(root, 'docs/guide.zh.md'), '# 指南\n') + writeFileSync(join(root, 'docs/reference.md'), '# Overview\n') + writeFileSync(join(root, 'docs/reference.zh.md'), '# 概览\n') + writeFileSync(join(root, 'docs/unpaired.md'), '# Only\n') + writeFileSync(join(root, 'docs/section/index.md'), '# Section\n') + writeFileSync(join(root, 'docs/section/index.zh.md'), '# 章节\n') + writeFileSync(join(root, 'packages/outside.md'), '# Outside\n') + writeFileSync(join(root, 'packages/outside.zh.md'), '# 范围外\n') + return root +} + +function linkContext( + root: string, + sourcePath: string, + repositoryFileExists?: (repoPath: string) => boolean, +): TranslationLinkContext { + return { + repoRoot: root, + sourcePath, + isTranslationPairSource: path => path.startsWith('docs/'), + ...(repositoryFileExists === undefined ? {} : { repositoryFileExists }), + } +} + +function expectUnchangedLinkInput(root: string, input: string): void { + const context = linkContext(root, 'docs/guide.md') + expect(translationLinkLocaleViolations(input, context)).toEqual([]) + expect(rewriteTranslationLinkLocales(input, context)).toEqual({ content: input, rewritten: 0 }) + expect(normalizeTranslationMarkdownLinks(input, context)).toBe(input) +} + +describe('translation link locale validation', () => { + it('rejects a Chinese link to the English sibling with an exact diagnostic', () => { + const root = fixture() + expect(translationLinkLocaleViolations( + '# 指南\n\n正文。\n\n[概览](reference.md?view=full#overview)\n', + linkContext(root, 'docs/guide.zh.md'), + )).toEqual([{ + sourcePath: 'docs/guide.zh.md', + line: 5, + url: 'reference.md?view=full#overview', + expectedUrl: 'reference.zh.md?view=full#overview', + }]) + }) + + it('rewrites an encoded exact filename without changing its query or fragment suffix', () => { + const root = fixture() + const input = '[概览](reference%2Emd?view=full&mode=all#overview)\n' + expect(translationLinkLocaleViolations( + input, + linkContext(root, 'docs/guide.zh.md'), + )[0]).toMatchObject({ + url: 'reference%2Emd?view=full&mode=all#overview', + expectedUrl: 'reference.zh.md?view=full&mode=all#overview', + }) + expect(rewriteTranslationLinkLocales(input, linkContext(root, 'docs/guide.zh.md'))).toEqual({ + content: '[概览](reference.zh.md?view=full&mode=all#overview)\n', + rewritten: 1, + }) + }) + + it('encodes each exact path segment with only RFC 3986 unreserved characters', () => { + const root = fixture() + const input = '[保留](a%29%23%3Fb%2Emd?view=full#section)\n' + const repositoryFiles = new Set(['docs/a)#?b.md', 'docs/a)#?b.zh.md']) + expect(rewriteTranslationLinkLocales( + input, + linkContext(root, 'docs/guide.zh.md', path => repositoryFiles.has(path)), + )).toEqual({ + content: '[保留](a%29%23%3Fb.zh.md?view=full#section)\n', + rewritten: 1, + }) + }) + + it('accepts the target-locale sibling and an out-of-scope target with its own sibling', () => { + const root = fixture() + expect(translationLinkLocaleViolations( + '[paired](reference.zh.md) [outside](../packages/outside.md)\n', + linkContext(root, 'docs/guide.zh.md'), + )).toEqual([]) + }) + + it('does not fall back when an active target is missing its locale sibling', () => { + const root = fixture() + expect(translationLinkLocaleViolations( + '[missing](unpaired.md)\n', + linkContext(root, 'docs/guide.zh.md'), + )[0]).toMatchObject({ expectedUrl: 'unpaired.zh.md' }) + }) + + it('requires English sources to use the English sibling', () => { + const root = fixture() + expect(translationLinkLocaleViolations( + '[Reference](reference.zh.md)\n', + linkContext(root, 'docs/guide.md'), + )[0]).toMatchObject({ + url: 'reference.zh.md', + expectedUrl: 'reference.md', + }) + }) + + it('does not infer an index page from a directory target', () => { + const root = fixture() + const input = '[Section](section/)\n' + expect(translationLinkLocaleViolations(input, linkContext(root, 'docs/guide.zh.md'))).toEqual([]) + expect(rewriteTranslationLinkLocales(input, linkContext(root, 'docs/guide.zh.md'))) + .toEqual({ content: input, rewritten: 0 }) + }) + + it('exempts the language switcher target explicitly', () => { + const root = fixture() + expect(translationLinkLocaleViolations( + '# 指南\n\n[English](guide.md) | 中文\n', + linkContext(root, 'docs/guide.zh.md'), + ['guide.md'], + )).toEqual([]) + }) + + it('does not exempt an ordinary body link to the counterpart', () => { + const root = fixture() + const markdown = '# 指南\n\n[English](guide.md) | 中文\n\n[正文](guide.md)\n' + expect(translationLinkLocaleViolations( + markdown, + linkContext(root, 'docs/guide.zh.md'), + ['guide.md'], + )).toEqual([{ + sourcePath: 'docs/guide.zh.md', + line: 5, + url: 'guide.md', + expectedUrl: 'guide.zh.md', + }]) + expect(rewriteTranslationLinkLocales( + markdown, + linkContext(root, 'docs/guide.zh.md'), + ['guide.md'], + ).content).toBe('# 指南\n\n[English](guide.md) | 中文\n\n[正文](guide.zh.md)\n') + }) + + it('uses the selected content plane for target existence without deriving scope from siblings', () => { + const root = fixture() + const staged = new Set(['docs/reference.md', 'docs/reference.zh.md']) + expect(translationLinkLocaleViolations( + '[概览](reference.md)\n', + linkContext(root, 'docs/guide.zh.md', path => staged.has(path)), + )).toHaveLength(1) + staged.delete('docs/reference.zh.md') + expect(translationLinkLocaleViolations( + '[概览](reference.md)\n', + linkContext(root, 'docs/guide.zh.md', path => staged.has(path)), + )).toHaveLength(1) + staged.delete('docs/reference.md') + expect(translationLinkLocaleViolations( + '[概览](reference.md)\n', + linkContext(root, 'docs/guide.zh.md', path => staged.has(path)), + )).toEqual([]) + }) +}) + +describe('translation link rewriting and normalization', () => { + it('rewrites only the destination while preserving the suffix and title', () => { + const root = fixture() + const input = '[概览](reference.md?view=full&mode=all#overview "reference.md title")\n' + expect(rewriteTranslationLinkLocales( + input, + linkContext(root, 'docs/guide.zh.md'), + )).toEqual({ + content: '[概览](reference.zh.md?view=full&mode=all#overview "reference.md title")\n', + rewritten: 1, + }) + }) + + it('rewrites link definitions without changing their labels', () => { + const root = fixture() + expect(rewriteTranslationLinkLocales( + '[概览][ref]\n\n[ref]: "title"\n', + linkContext(root, 'docs/guide.zh.md'), + ).content).toBe('[概览][ref]\n\n[ref]: "title"\n') + }) + + it('uses only the first duplicate reference definition', () => { + const root = fixture() + expect(translationLinkLocaleViolations( + '[概览][ref]\n\n[ref]: reference.zh.md\n[ref]: reference.md\n', + linkContext(root, 'docs/guide.zh.md'), + )).toEqual([]) + }) + + it('does not treat an image-only definition as a document link', () => { + const root = fixture() + const input = '![preview][asset]\n\n[asset]: reference.zh.md#overview\n' + expectUnchangedLinkInput(root, input) + }) + + it.each([ + '\n', + 'https://example.com/reference.md\n', + ])('leaves GFM autolink source unchanged: %s', (input) => { + const root = fixture() + expectUnchangedLinkInput(root, input) + }) + + it('normalizes only paired locale paths and retains other bytes', () => { + const root = fixture() + const english = '[Reference](reference.md#overview) [Outside](../packages/outside.md)\n' + const chinese = '[Reference](reference.zh.md#overview) [Outside](../packages/outside.md)\n' + expect(normalizeTranslationMarkdownLinks( + english, + linkContext(root, 'docs/guide.md'), + )).toBe(normalizeTranslationMarkdownLinks( + chinese, + linkContext(root, 'docs/guide.zh.md'), + )) + }) + + it('retains authored query bytes during normalization', () => { + const root = fixture() + const escaped = '[Reference](reference.md?x=1&y=2#overview)\n' + const literal = '[Reference](reference.zh.md?x=1&y=2#overview)\n' + expect(normalizeTranslationMarkdownLinks( + escaped, + linkContext(root, 'docs/guide.md'), + )).not.toBe(normalizeTranslationMarkdownLinks( + literal, + linkContext(root, 'docs/guide.zh.md'), + )) + }) +}) diff --git a/scripts/translation-links.ts b/scripts/translation-links.ts new file mode 100644 index 0000000000..af7c991fd4 --- /dev/null +++ b/scripts/translation-links.ts @@ -0,0 +1,328 @@ +/** Locale-aware resolution and byte-preserving rewrites for bilingual Markdown links. */ + +import { existsSync, statSync } from 'node:fs' +import { posix, resolve } from 'node:path' +import type { Nodes } from 'mdast' +import { + isExternalOrAbsoluteMarkdownUrl, + markdownDestination, + parseMarkdown, + splitMarkdownUrlTarget, + visitMarkdown, + type MarkdownDestination, +} from './markdown.ts' + +/** Repository and source document used to resolve one relative link. */ +export interface TranslationLinkContext { + /** Absolute repository root. */ + repoRoot: string + /** Repository-relative Markdown source path. */ + sourcePath: string + /** Whether an English Markdown path belongs to the active bilingual corpus. */ + isTranslationPairSource: (sourcePath: string) => boolean + /** Selected content plane; defaults to regular files in the working tree. */ + repositoryFileExists?: (repoPath: string) => boolean +} + +/** One relative document link whose target uses the wrong locale sibling. */ +export interface TranslationLinkLocaleViolation { + sourcePath: string + line: number + url: string + expectedUrl: string +} + +/** Result of rewriting wrong-locale relative document links. */ +export interface TranslationLinkRewriteResult { + content: string + rewritten: number +} + +interface TranslationPairTarget { + source: string + zh: string +} + +interface ResolvedTranslationLink { + pair: TranslationPairTarget + targetPath: string + suffix: string + expectedPath: string + expectedUrl: string + locale: 'en' | 'zh' +} + +interface Replacement { + start: number + end: number + value: string +} + +type LinkNode = Extract + +/** Offset of the one top-level switcher link immediately following the H1. */ +export function languageSwitcherLinkOffset( + tree: Nodes, + markdown: string, + acceptedTargets: string | readonly string[], +): number | undefined { + if (tree.type !== 'root') return undefined + const accepted = new Set(typeof acceptedTargets === 'string' ? [acceptedTargets] : acceptedTargets) + const headingIndex = tree.children.findIndex(node => node.type === 'heading' && node.depth === 1) + if (headingIndex < 0) return undefined + for (const node of tree.children.slice(headingIndex + 1)) { + if (node.type === 'heading') return undefined + if (node.type !== 'paragraph' || node.position === undefined) continue + const start = node.position.start.offset + const end = node.position.end.offset + if (start === undefined || end === undefined) continue + const authored = markdown.slice(start, end) + if (!/^(?:English \| \[中文\]\([^\n]+\)|\[English\]\([^\n]+\) \| 中文)$/.test(authored)) continue + const links = node.children.filter((child): child is Extract => child.type === 'link') + if (links.length === 1 && accepted.has(links[0]?.url ?? '')) { + return links[0]?.position?.start.offset + } + } + return undefined +} + +/** Whether the tree carries its canonical top-level language switcher. */ +export function hasLanguageSwitcher( + tree: Nodes, + markdown: string, + acceptedTargets: string | readonly string[], +): boolean { + return languageSwitcherLinkOffset(tree, markdown, acceptedTargets) !== undefined +} + +function decodePath(path: string): string { + try { + return decodeURIComponent(path) + } catch { + return path + } +} + +function worktreeFileExists(repoRoot: string, repoPath: string): boolean { + try { + const path = resolve(repoRoot, repoPath) + return existsSync(path) && statSync(path).isFile() + } catch { + return false + } +} + +function repositoryFileExists(context: TranslationLinkContext, repoPath: string): boolean { + return context.repositoryFileExists?.(repoPath) ?? worktreeFileExists(context.repoRoot, repoPath) +} + +function repositoryRelativePath(path: string): string | undefined { + const normalized = posix.normalize(path) + if (normalized === '' || normalized === '.' || normalized === '..' || normalized.startsWith('../') || posix.isAbsolute(normalized)) { + return undefined + } + return normalized +} + +function resolveRepositoryTarget( + rawPath: string, + context: TranslationLinkContext, +): string | undefined { + const decoded = decodePath(rawPath) + const exact = repositoryRelativePath(posix.join(posix.dirname(context.sourcePath), decoded)) + if (exact === undefined) return undefined + return repositoryFileExists(context, exact) ? exact : undefined +} + +function translationPairTarget(targetPath: string, context: TranslationLinkContext): TranslationPairTarget | undefined { + const source = targetPath.endsWith('.zh.md') + ? targetPath.replace(/\.zh\.md$/, '.md') + : targetPath.endsWith('.md') ? targetPath : undefined + if (source === undefined || !context.isTranslationPairSource(source)) return undefined + const zh = source.replace(/\.md$/, '.zh.md') + return { source, zh } +} + +function encodePathSegment(segment: string): string { + return encodeURIComponent(segment).replace(/[!'()*]/g, character => ( + `%${character.charCodeAt(0).toString(16).toUpperCase()}` + )) +} + +function relativeExpectedPath( + context: TranslationLinkContext, + expectedPath: string, + rawPath: string, +): string { + const relative = posix.relative(posix.dirname(context.sourcePath), expectedPath) + const encoded = relative.split('/').map(encodePathSegment).join('/') + return rawPath.startsWith('./') && !encoded.startsWith('.') ? `./${encoded}` : encoded +} + +function expectedLocalePath( + rawPath: string, + locale: 'en' | 'zh', + context: TranslationLinkContext, + expectedPath: string, +): string { + if (locale === 'zh' && rawPath.endsWith('.md') && !rawPath.endsWith('.zh.md')) { + return rawPath.replace(/\.md$/, '.zh.md') + } + if (locale === 'en' && rawPath.endsWith('.zh.md')) return rawPath.replace(/\.zh\.md$/, '.md') + return relativeExpectedPath(context, expectedPath, rawPath) +} + +function resolveTranslationLink( + url: string, + context: TranslationLinkContext, + authoredUrl: string, +): ResolvedTranslationLink | undefined { + if (isExternalOrAbsoluteMarkdownUrl(url)) return undefined + const { path } = splitMarkdownUrlTarget(url) + const authored = splitMarkdownUrlTarget(authoredUrl) + if (path === '') return undefined + const targetPath = resolveRepositoryTarget(path, context) + if (targetPath === undefined) return undefined + const pair = translationPairTarget(targetPath, context) + if (pair === undefined) return undefined + const locale = context.sourcePath.endsWith('.zh.md') ? 'zh' : 'en' + const expectedPath = locale === 'zh' ? pair.zh : pair.source + return { + pair, + targetPath, + suffix: authored.suffix, + expectedPath, + expectedUrl: `${expectedLocalePath(authored.path, locale, context, expectedPath)}${authored.suffix}`, + locale, + } +} + +function hasExpectedLocale(resolved: ResolvedTranslationLink): boolean { + return resolved.targetPath === resolved.expectedPath +} + +function replacementFor(destination: MarkdownDestination, value: string): Replacement { + return { start: destination.start, end: destination.end, value } +} + +function authoredExternalTarget(markdown: string, node: LinkNode): string { + const start = node.position?.start.offset + const end = node.position?.end.offset + if (start === undefined || end === undefined) { + throw new Error(`translation-links: external link ${JSON.stringify(node.url)} has no source offsets`) + } + const raw = markdown.slice(start, end) + if (node.type === 'definition' || raw.startsWith('[')) return markdownDestination(markdown, node).url + if (raw.startsWith('<') && raw.endsWith('>')) return raw.slice(1, -1) + return raw +} + +function applyReplacements(markdown: string, replacements: Replacement[]): string { + let output = markdown + for (const replacement of replacements.sort((left, right) => right.start - left.start)) { + output = output.slice(0, replacement.start) + replacement.value + output.slice(replacement.end) + } + return output +} + +function visitDocumentLinkNodes( + markdown: string, + skipTargets: readonly string[], + visitor: (node: LinkNode) => void, +): void { + const tree = parseMarkdown(markdown) + const switcherOffset = languageSwitcherLinkOffset(tree, markdown, skipTargets) + const referencedIdentifiers = new Set() + const visitedDefinitions = new Set() + visitMarkdown(tree, (node) => { + if (node.type === 'linkReference') referencedIdentifiers.add(node.identifier) + }) + visitMarkdown(tree, (node) => { + if (node.type === 'link' && node.position?.start.offset === switcherOffset) return + if (node.type === 'link') { + visitor(node) + } else if (node.type === 'definition' + && referencedIdentifiers.has(node.identifier) + && !visitedDefinitions.has(node.identifier)) { + visitedDefinitions.add(node.identifier) + visitor(node) + } + }) +} + +function visitResolvedDocumentLinks( + markdown: string, + context: TranslationLinkContext, + skipTargets: readonly string[], + visitor: (node: LinkNode, destination: MarkdownDestination, resolved: ResolvedTranslationLink) => void, +): void { + visitDocumentLinkNodes(markdown, skipTargets, (node) => { + if (isExternalOrAbsoluteMarkdownUrl(node.url)) return + const destination = markdownDestination(markdown, node) + const resolved = resolveTranslationLink(node.url, context, destination.url) + if (resolved !== undefined) visitor(node, destination, resolved) + }) +} + +/** Return one violation per wrong-locale link or link definition. */ +export function translationLinkLocaleViolations( + markdown: string, + context: TranslationLinkContext, + skipTargets: readonly string[] = [], +): TranslationLinkLocaleViolation[] { + const violations: TranslationLinkLocaleViolation[] = [] + visitResolvedDocumentLinks(markdown, context, skipTargets, (node, destination, resolved) => { + if (hasExpectedLocale(resolved)) return + violations.push({ + sourcePath: context.sourcePath, + line: node.position?.start.line ?? 0, + url: destination.url, + expectedUrl: resolved.expectedUrl, + }) + }) + return violations +} + +/** Rewrite wrong-locale document links without reserializing surrounding Markdown. */ +export function rewriteTranslationLinkLocales( + markdown: string, + context: TranslationLinkContext, + skipTargets: readonly string[] = [], +): TranslationLinkRewriteResult { + const replacements: Replacement[] = [] + visitResolvedDocumentLinks(markdown, context, skipTargets, (_node, destination, resolved) => { + if (hasExpectedLocale(resolved)) return + replacements.push(replacementFor(destination, resolved.expectedUrl)) + }) + return { content: applyReplacements(markdown, replacements), rewritten: replacements.length } +} + +/** Normalize only paired-document locale paths while retaining every other byte and URL suffix. */ +export function normalizeTranslationMarkdownLinks( + markdown: string, + context: TranslationLinkContext, + skipTargets: readonly string[] = [], +): string { + const replacements: Replacement[] = [] + visitResolvedDocumentLinks(markdown, context, skipTargets, (_node, destination, resolved) => { + replacements.push(replacementFor( + destination, + `dsh-translation-target:${resolved.pair.source}${resolved.suffix}`, + )) + }) + return applyReplacements(markdown, replacements) +} + +/** Semantic target of one authored inline link or referenced definition. */ +export function semanticTranslationLinkNodeTarget( + node: LinkNode, + markdown: string, + context: TranslationLinkContext, +): string { + if (isExternalOrAbsoluteMarkdownUrl(node.url)) return authoredExternalTarget(markdown, node) + const destination = markdownDestination(markdown, node) + const resolved = resolveTranslationLink(node.url, context, destination.url) + return resolved === undefined + ? destination.url + : `dsh-translation-target:${resolved.pair.source}${resolved.suffix}` +} diff --git a/scripts/translation-pairing-git.ts b/scripts/translation-pairing-git.ts index 5f1ddd4fc0..041a0bdb00 100644 --- a/scripts/translation-pairing-git.ts +++ b/scripts/translation-pairing-git.ts @@ -42,12 +42,48 @@ export function runGit(root: string, args: string[], operation: string, input?: /** One regular stage-zero Git index entry and its exact blob bytes. */ export interface GitIndexBlob { - /** Object ID recorded in the index. */ objectId: string - /** Blob bytes stored under that object ID. */ content: Buffer } +/** Every stage-zero path currently present in the Git index. */ +export function gitIndexPaths(root: string): Set { + const paths = new Set() + const entries = runGit(root, ['ls-files', '--stage', '-z'], 'listing Git index paths') + .toString('utf8') + .split('\0') + .filter(Boolean) + for (const entry of entries) { + const match = /^\d+ [0-9a-f]+ ([0-3])\t([\s\S]+)$/.exec(entry) + if (!match?.[1] || match[2] === undefined) throw new Error('git ls-files --stage returned a malformed entry') + if (match[1] === '0') paths.add(match[2]) + } + return paths +} + +/** + * Paths visible to a custom merge driver from the current index plus every + * merge head Git advertises through `GITHEAD_` environment entries. + * + * Git invokes custom drivers before it writes clean additions from the other + * heads into stage zero. The explicit post-conflict resolver has no GITHEAD + * entries and therefore uses the already-merged index alone. + */ +export function gitMergeInputPaths(root: string, environment: NodeJS.ProcessEnv = process.env): Set { + const paths = gitIndexPaths(root) + const heads = Object.keys(environment) + .flatMap(key => /^GITHEAD_([0-9a-f]{40})$/.exec(key)?.[1] ?? []) + .sort() + for (const head of heads) { + const files = runGit(root, ['ls-tree', '-r', '--name-only', '-z', head], `listing merge-head ${head} paths`) + .toString('utf8') + .split('\0') + .filter(Boolean) + for (const file of files) paths.add(file) + } + return paths +} + /** * Read one path from the Git index without consulting working-tree bytes. * diff --git a/scripts/translation-pairing-merge.spec.ts b/scripts/translation-pairing-merge.spec.ts index 32924416a6..7293989c2e 100644 --- a/scripts/translation-pairing-merge.spec.ts +++ b/scripts/translation-pairing-merge.spec.ts @@ -6,6 +6,7 @@ import { mkdtempSync, mkdirSync, readFileSync, + rmSync, symlinkSync, writeFileSync, } from 'node:fs' @@ -15,8 +16,8 @@ import { fileURLToPath } from 'node:url' import { afterEach, describe, expect, it } from 'vitest' import { gitBlobHash, storeGitBlob } from './translation-pairing-git.ts' import { - mergeTranslationPairingRecords, - resolveTranslationPairingConflicts, + mergeTranslationPairingRecords as mergeTranslationPairingRecordsWithScope, + resolveTranslationPairingConflicts as resolveTranslationPairingConflictsWithScope, } from './translation-pairing-merge.ts' import { renderTranslationPairingRecord, @@ -35,6 +36,27 @@ interface Fixture { root: string } +function mergeTranslationPairingRecords( + root: string, + metaPath: string, + ancestorRecord: string, + currentRecord: string, + otherRecord: string, +) { + return mergeTranslationPairingRecordsWithScope( + root, + metaPath, + ancestorRecord, + currentRecord, + otherRecord, + () => true, + ) +} + +function resolveTranslationPairingConflicts(root: string): string[] { + return resolveTranslationPairingConflictsWithScope(root, () => true) +} + afterEach(() => { for (const fixture of fixtures.splice(0)) removeFixtureSafely(fixture) }) @@ -252,6 +274,19 @@ describe('translation pairing merge composition', { timeout: 15_000 }, () => { )).toThrow('pairing record escapes the repository') }) + it('rejects a pairing record excluded from the active corpus', () => { + const fixture = createFixture(false) + + expect(() => mergeTranslationPairingRecordsWithScope( + fixture.root, + 'docs/guide.i18n.yaml', + '', + '', + '', + () => false, + )).toThrow('docs/guide.i18n.yaml is excluded from the active bilingual documentation corpus') + }) + it('merges the owner blobs named by three valid records', () => { const fixture = createFixture(false) git(fixture, ['config', 'merge.default', 'text']) @@ -271,6 +306,48 @@ describe('translation pairing merge composition', { timeout: 15_000 }, () => { expect(result.zhHash).toBe(gitBlobHash(Buffer.from(mergedZh))) }) + it('accepts locale-specific paths to the same paired document', () => { + const fixture = createFixture(false) + write(fixture.root, 'docs/reference.md', '# Overview\n') + write(fixture.root, 'docs/reference.zh.md', '# 概览\n') + git(fixture, ['add', 'docs/reference.md', 'docs/reference.zh.md']) + const source = baseSource.replace('Alpha base.', '[Reference](reference.md#overview)') + const zh = baseZh.replace('甲基础。', '[参考](reference.zh.md#overview)') + const ancestor = record(fixture.root, 'docs/guide.md', source, zh) + const current = record(fixture.root, 'docs/guide.md', source, zh) + const other = record(fixture.root, 'docs/guide.md', source, zh) + rmSync(join(fixture.root, 'docs/reference.md')) + rmSync(join(fixture.root, 'docs/reference.zh.md')) + + expect(mergeTranslationPairingRecords( + fixture.root, + 'docs/guide.i18n.yaml', + ancestor, + current, + other, + ).zhContent.toString('utf8')).toBe(zh) + }) + + it('rejects a clean merge whose Chinese link uses the English sibling', () => { + const fixture = createFixture(false) + write(fixture.root, 'docs/reference.md', '# Overview\n') + write(fixture.root, 'docs/reference.zh.md', '# 概览\n') + git(fixture, ['add', 'docs/reference.md', 'docs/reference.zh.md']) + const source = baseSource.replace('Alpha base.', '[Reference](reference.md)') + const zh = baseZh.replace('甲基础。', '[参考](reference.md)') + const ancestor = record(fixture.root, 'docs/guide.md', source, zh) + const current = record(fixture.root, 'docs/guide.md', source, zh) + const other = record(fixture.root, 'docs/guide.md', source, zh) + + expect(() => mergeTranslationPairingRecords( + fixture.root, + 'docs/guide.i18n.yaml', + ancestor, + current, + other, + )).toThrow('docs/guide.zh.md:5 clean merge uses "reference.md"; expected "reference.zh.md"') + }) + it('merges a generated source without an English language switcher', () => { const fixture = createFixture(false) const ancestor = record(fixture.root, 'docs/module-graph.md', generatedBaseSource, generatedBaseZh) @@ -414,6 +491,49 @@ describe('translation pairing merge composition', { timeout: 15_000 }, () => { expectMergedPair(fixture) }) + it('sees a paired link target added by the other branch', () => { + const fixture = createFixture() + commitPair(fixture, baseSource, baseZh, 'base') + git(fixture, ['switch', '-c', 'current']) + commitPair(fixture, currentSource, currentZh, 'current guide') + git(fixture, ['switch', 'master']) + record( + fixture.root, + 'docs/guide.md', + baseSource.replace('Beta base.', '[Reference](reference.md#overview)'), + baseZh.replace('乙基础。', '[参考](reference.zh.md#overview)'), + ) + record( + fixture.root, + 'docs/reference.md', + '# Reference\n\nEnglish | [中文](reference.zh.md)\n\nOverview.\n', + '# 参考\n\n[English](reference.md) | 中文\n\n概览。\n', + ) + git(fixture, ['add', '.']) + git(fixture, ['commit', '-m', 'other guide and target']) + git(fixture, ['switch', 'current']) + installFixtureRuntime(fixture.root) + git(fixture, [ + 'config', + 'merge.dsh-translation-pairing.driver', + 'scripts/merge-translation-pairing-driver.sh %O %A %B %P', + ]) + + const merge = spawnSync('git', ['-C', fixture.root, 'merge', '--no-edit', 'master'], { + encoding: 'utf8', + env: fixture.env, + }) + + expect(merge.status, merge.stderr).toBe(0) + expect(git(fixture, ['diff', '--name-only', '--diff-filter=U'])).toBe('') + expect(readFileSync(join(fixture.root, 'docs/guide.md'), 'utf8')).toContain( + '[Reference](reference.md#overview)', + ) + expect(readFileSync(join(fixture.root, 'docs/guide.zh.md'), 'utf8')).toContain( + '[参考](reference.zh.md#overview)', + ) + }) + it('leaves an ordinary recoverable conflict when the configured runtime is unavailable', () => { const fixture = createFixture() const records = createDivergedPair(fixture) diff --git a/scripts/translation-pairing-merge.ts b/scripts/translation-pairing-merge.ts index 856c909091..7f0f25114b 100644 --- a/scripts/translation-pairing-merge.ts +++ b/scripts/translation-pairing-merge.ts @@ -7,6 +7,7 @@ import { basename, isAbsolute, join, relative, resolve, sep } from 'node:path' import { GIT_COMMAND_MAX_BUFFER, gitBlobHash, + gitMergeInputPaths, readGitIndexBlob, runGit, storeGitBlob, @@ -14,12 +15,17 @@ import { import { isTranslationScopeFile, languageSwitcherTargets, - linksTo, parseTranslationMarkdown, + parseTranslationPairingManifest, requiresSourceLanguageSwitcher, + translationPairSourcePredicate, translationStructureDiff, translationStructureSignature, } from './translation-pairing.ts' +import { + hasLanguageSwitcher, + translationLinkLocaleViolations, +} from './translation-links.ts' import { parseTranslationPairingRecord, renderTranslationPairingRecord, @@ -162,20 +168,62 @@ function loadRecordOwners( } } -function assertMergedPairStructure(paths: TranslationPairPaths, source: Buffer, zh: Buffer): void { - const sourceTree = parseTranslationMarkdown(source.toString('utf8')) - const zhTree = parseTranslationMarkdown(zh.toString('utf8')) +function assertMergedPairStructure( + root: string, + paths: TranslationPairPaths, + source: Buffer, + zh: Buffer, + isTranslationPairSource: (sourcePath: string) => boolean, +): void { + const sourceText = source.toString('utf8') + const zhText = zh.toString('utf8') + const sourceTree = parseTranslationMarkdown(sourceText) + const zhTree = parseTranslationMarkdown(zhText) + const indexFiles = gitMergeInputPaths(root) + const repositoryFileExists = (path: string): boolean => indexFiles.has(path) const sourceSwitcherTargets = languageSwitcherTargets(paths.source) const zhSwitcherTargets = languageSwitcherTargets(paths.zh) - if (requiresSourceLanguageSwitcher(paths.source) && !linksTo(sourceTree, zhSwitcherTargets)) { + if (requiresSourceLanguageSwitcher(paths.source) + && !hasLanguageSwitcher(sourceTree, sourceText, zhSwitcherTargets)) { throw new Error(`${paths.source} clean merge lost its language-switcher link to ${basename(paths.zh)}`) } - if (!linksTo(zhTree, sourceSwitcherTargets)) { + if (!hasLanguageSwitcher(zhTree, zhText, sourceSwitcherTargets)) { throw new Error(`${paths.zh} clean merge lost its language-switcher link to ${basename(paths.source)}`) } + const localeViolations = [ + ...translationLinkLocaleViolations(sourceText, { + repoRoot: root, + sourcePath: paths.source, + isTranslationPairSource, + repositoryFileExists, + }, zhSwitcherTargets), + ...translationLinkLocaleViolations(zhText, { + repoRoot: root, + sourcePath: paths.zh, + isTranslationPairSource, + repositoryFileExists, + }, sourceSwitcherTargets), + ] + if (localeViolations.length > 0) { + const violation = localeViolations[0] + if (violation === undefined) throw new Error('translation locale violation disappeared') + throw new Error(`${violation.sourcePath}:${violation.line} clean merge uses ${JSON.stringify(violation.url)}; expected ${JSON.stringify(violation.expectedUrl)}`) + } const divergences = translationStructureDiff( - translationStructureSignature(sourceTree, zhSwitcherTargets), - translationStructureSignature(zhTree, sourceSwitcherTargets), + translationStructureSignature(sourceTree, zhSwitcherTargets, { + repoRoot: root, + sourcePath: paths.source, + isTranslationPairSource, + repositoryFileExists, + markdown: sourceText, + }), + translationStructureSignature(zhTree, sourceSwitcherTargets, { + repoRoot: root, + sourcePath: paths.zh, + isTranslationPairSource, + repositoryFileExists, + markdown: zhText, + }), ) if (divergences.length > 0) { throw new Error(`${paths.source} and ${paths.zh} clean merges diverge structurally: ${divergences.join('; ')}`) @@ -212,19 +260,23 @@ export function mergeTranslationPairingRecords( ancestorRecord: string, currentRecord: string, otherRecord: string, + isTranslationPairSource: (sourcePath: string) => boolean, ): TranslationPairingMergeResult { const normalizedMeta = normalizeMetaPath(root, metaPath) if (!isTranslationScopeFile(normalizedMeta)) { throw new Error(`${normalizedMeta} is outside the active bilingual documentation corpus`) } const paths = translationPairPathsFromMeta(normalizedMeta) + if (!isTranslationPairSource(paths.source)) { + throw new Error(`${normalizedMeta} is excluded from the active bilingual documentation corpus`) + } assertDefaultTextMerge(root, paths) const ancestor = loadRecordOwners(root, 'ancestor', ancestorRecord, paths) const current = loadRecordOwners(root, 'current', currentRecord, paths) const other = loadRecordOwners(root, 'other', otherRecord, paths) const sourceContent = mergeBlobTriplet(root, paths.source, ancestor.source, current.source, other.source) const zhContent = mergeBlobTriplet(root, paths.zh, ancestor.zh, current.zh, other.zh) - assertMergedPairStructure(paths, sourceContent, zhContent) + assertMergedPairStructure(root, paths, sourceContent, zhContent, isTranslationPairSource) const sourceHash = storeGitBlob(root, sourceContent) const zhHash = storeGitBlob(root, zhContent) return { @@ -236,6 +288,16 @@ export function mergeTranslationPairingRecords( } } +/** Read the repository manifest and return its active bilingual-source predicate. */ +export function repositoryTranslationPairSource(root: string): (sourcePath: string) => boolean { + const path = 'scripts/translation-pairing.manifest.json' + const content = readGitIndexBlob(root, path)?.content ?? readFileSync(join(root, path)) + const manifest = parseTranslationPairingManifest( + content.toString('utf8'), + ) + return translationPairSourcePredicate(manifest) +} + function unmergedSidecars(root: string): Map { const output = runGit(root, ['ls-files', '--unmerged', '-z'], 'listing unresolved merge entries').toString('utf8') const records = new Map() @@ -289,7 +351,10 @@ function assertUneditedSidecar( * @param root - Repository root with an in-progress merge-like operation. * @returns Repository-relative sidecar paths resolved and staged. */ -export function resolveTranslationPairingConflicts(root: string): string[] { +export function resolveTranslationPairingConflicts( + root: string, + isTranslationPairSource: (sourcePath: string) => boolean, +): string[] { const resolutions: { path: string; record: string }[] = [] const failures: { path: string; reason: string }[] = [] for (const [metaPath, stages] of [...unmergedSidecars(root)].sort(([left], [right]) => left.localeCompare(right))) { @@ -307,6 +372,7 @@ export function resolveTranslationPairingConflicts(root: string): string[] { ancestorRecord, currentRecord, otherRecord, + isTranslationPairSource, ) const paths = translationPairPathsFromMeta(metaPath) if (readGitIndexBlob(root, paths.source)?.objectId !== result.sourceHash) { diff --git a/scripts/translation-pairing.spec.ts b/scripts/translation-pairing.spec.ts index efddff2bbe..c4dda468fe 100644 --- a/scripts/translation-pairing.spec.ts +++ b/scripts/translation-pairing.spec.ts @@ -1,11 +1,16 @@ /** Regression tests for bilingual snapshots, corpus scope, and structure. */ import { execFileSync, spawnSync } from 'node:child_process' -import { mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { describe, expect, it } from 'vitest' -import { gitBlobHash, readGitIndexBlob, storeGitBlob } from './translation-pairing-git.ts' +import { + gitBlobHash, + gitIndexPaths, + readGitIndexBlob, + storeGitBlob, +} from './translation-pairing-git.ts' import { parseTranslationPairingRecord, renderTranslationPairingRecord, @@ -13,21 +18,44 @@ import { } from './translation-pairing-record.ts' import { blobHash, + isTranslationPairingManifestExcluded, isTranslationScopeFile, languageSwitcherTargets, - linksTo, pairAnchorOfArgument, parseTranslationMarkdown, parseTranslationPairingCliArgs, parseTranslationPairingManifest, partitionGeneratedRegions, requiresSourceLanguageSwitcher, + translationPairSourcePredicate, translationStructureDiff, translationStructureSignature, } from './translation-pairing.ts' +const fixturePairSource = (): boolean => true + function signature(markdown: string) { - return translationStructureSignature(parseTranslationMarkdown(markdown), 'counterpart.zh.md') + return translationStructureSignature( + parseTranslationMarkdown(markdown), + 'counterpart.zh.md', + { + repoRoot: process.cwd(), sourcePath: 'counterpart.md', + isTranslationPairSource: fixturePairSource, markdown, + }, + ) +} + +function fixtureSignature( + root: string, + sourcePath: string, + markdown: string, + switcherTarget: string, +) { + return translationStructureSignature( + parseTranslationMarkdown(markdown), + switcherTarget, + { repoRoot: root, sourcePath, isTranslationPairSource: fixturePairSource, markdown }, + ) } function gitSupportsObjectFormat(format: 'sha256'): boolean { @@ -106,6 +134,26 @@ describe('translation pairing snapshots', () => { } }) + it('lists exact index files without treating a directory prefix as one entry', () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-translation-pairing-index-')) + try { + execFileSync('git', ['init', '--quiet', root], { + env: { ...process.env, GIT_DEFAULT_HASH: 'sha1' }, + }) + mkdirSync(join(root, 'docs'), { recursive: true }) + writeFileSync(join(root, 'docs/reference.md'), '# Reference\n') + writeFileSync(join(root, 'docs/reference.zh.md'), '# 参考\n') + execFileSync('git', ['-C', root, 'add', 'docs']) + + expect(gitIndexPaths(root)).toEqual(new Set([ + 'docs/reference.md', + 'docs/reference.zh.md', + ])) + } finally { + rmSync(root, { recursive: true, force: true }) + } + }) + it.skipIf(!supportsSha256ObjectFormat)('rejects an object format that pairing records cannot represent', () => { const root = mkdtempSync(join(tmpdir(), 'dsh-translation-pairing-')) try { @@ -119,11 +167,16 @@ describe('translation pairing snapshots', () => { describe('translation pairing manifest', () => { it('accepts an exclusions-only manifest', () => { - expect(parseTranslationPairingManifest(JSON.stringify({ + const manifest = parseTranslationPairingManifest(JSON.stringify({ excluded: ['docs/generated/'], - }))).toEqual({ + })) + expect(manifest).toEqual({ excluded: ['docs/generated/'], }) + expect(isTranslationPairingManifestExcluded('docs/generated/page.md', manifest)).toBe(true) + expect(translationPairSourcePredicate(manifest)('docs/generated/page.md')).toBe(false) + expect(translationPairSourcePredicate(manifest)('docs/guide.md')).toBe(true) + expect(translationPairSourcePredicate(manifest)('packages/example/guide.md')).toBe(false) }) it.each([ @@ -156,16 +209,44 @@ describe('translation pairing switchers', () => { it('accepts only the canonical public URL for an absolute switcher', () => { const targets = languageSwitcherTargets('python/sdk/README.zh.md') - const canonical = parseTranslationMarkdown( - '[中文](https://github.com/deepseek-ai/deepseek-harness/blob/master/python/sdk/README.zh.md)', - ) - const wrongPath = parseTranslationMarkdown( - '[中文](https://github.com/deepseek-ai/deepseek-harness/blob/master/other/README.zh.md)', - ) + const canonicalMarkdown = '# README\n\nEnglish | [中文](https://github.com/deepseek-ai/deepseek-harness/blob/master/python/sdk/README.zh.md)\n' + const canonical = parseTranslationMarkdown(canonicalMarkdown) + const wrongMarkdown = '# README\n\nEnglish | [中文](https://github.com/deepseek-ai/deepseek-harness/blob/master/other/README.zh.md)\n' + const wrongPath = parseTranslationMarkdown(wrongMarkdown) - expect(linksTo(canonical, targets)).toBe(true) - expect(translationStructureSignature(canonical, targets).links).toEqual([]) - expect(linksTo(wrongPath, targets)).toBe(false) + expect(translationStructureSignature(canonical, targets, { + repoRoot: process.cwd(), + sourcePath: 'python/sdk/README.md', + isTranslationPairSource: fixturePairSource, + markdown: canonicalMarkdown, + }).links).toEqual([]) + expect(translationStructureSignature(wrongPath, targets, { + repoRoot: process.cwd(), + sourcePath: 'python/sdk/README.md', + isTranslationPairSource: fixturePairSource, + markdown: wrongMarkdown, + }).links).toEqual([ + 'https://github.com/deepseek-ai/deepseek-harness/blob/master/other/README.zh.md', + ]) + }) + + it('excludes only the header switcher from the structural links', () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-translation-switcher-')) + try { + writeFileSync(join(root, 'guide.md'), '# Guide\n') + writeFileSync(join(root, 'guide.zh.md'), '# 指南\n') + const markdown = '# 指南\n\n[English](guide.md) | 中文\n\n[正文](guide.md)\n' + expect(translationStructureSignature( + parseTranslationMarkdown(markdown), + languageSwitcherTargets('guide.md'), + { + repoRoot: root, sourcePath: 'guide.zh.md', + isTranslationPairSource: fixturePairSource, markdown, + }, + ).links).toEqual(['dsh-translation-target:guide.md']) + } finally { + rmSync(root, { recursive: true, force: true }) + } }) }) @@ -201,6 +282,12 @@ describe('translation scope discovery', () => { 'CONTRIBUTING.md', 'CONTRIBUTING.zh.md', 'CONTRIBUTING.i18n.yaml', + 'BRAND_GUIDELINES.md', + 'BRAND_GUIDELINES.zh.md', + 'BRAND_GUIDELINES.i18n.yaml', + 'SAFETY.md', + 'SAFETY.zh.md', + 'SAFETY.i18n.yaml', 'apps/cli/README.md', 'future/subtree/readme.md', 'packages/example/README.zh.md', @@ -215,6 +302,7 @@ describe('translation scope discovery', () => { it.each([ 'packages/example/guide.md', 'packages/example/CONTRIBUTING.md', + 'packages/example/BRAND_GUIDELINES.md', 'examples/tutorial.md', 'website/reference.md', 'packages/example/README.txt', @@ -230,6 +318,81 @@ describe('translation scope discovery', () => { }) describe('translation structural signature', () => { + it('retains external GFM autolinks without parsing inline-link syntax', () => { + const markdown = '\n' + expect(signature(markdown).links).toEqual(['https://example.com/reference.md']) + }) + + it('retains exact authored bytes for ordinary external link targets', () => { + const escaped = signature('[External](https://example.com/?x=1&y=2)\n') + const literal = signature('[External](https://example.com/?x=1&y=2)\n') + expect(escaped.links).toEqual(['https://example.com/?x=1&y=2']) + expect(translationStructureDiff(escaped, literal)).toEqual([ + 'link target #1 diverges between the pair: "https://example.com/?x=1&y=2" vs "https://example.com/?x=1&y=2"', + ]) + }) + + it('treats target-locale siblings as one semantic link target', () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-translation-structure-')) + try { + writeFileSync(join(root, 'reference.md'), '# Reference\n') + writeFileSync(join(root, 'reference.zh.md'), '# 参考\n') + const sourceMarkdown = '[Reference](reference.md?view=full#section)\n' + const counterpartMarkdown = '[参考](reference.zh.md?view=full#section)\n' + const source = fixtureSignature(root, 'guide.md', sourceMarkdown, 'guide.zh.md') + const counterpart = fixtureSignature(root, 'guide.zh.md', counterpartMarkdown, 'guide.md') + expect(translationStructureDiff(source, counterpart)).toEqual([]) + } finally { + rmSync(root, { recursive: true, force: true }) + } + }) + + it('includes reference-style document links but excludes image-only definitions', () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-translation-structure-')) + try { + writeFileSync(join(root, 'reference.md'), '# Reference\n') + writeFileSync(join(root, 'reference.zh.md'), '# 参考\n') + const markdown = [ + '[Reference][doc]', + '', + '![Preview][asset]', + '', + '[doc]: reference.md', + '[asset]: reference.zh.md', + '', + ].join('\n') + expect(translationStructureSignature( + parseTranslationMarkdown(markdown), + 'guide.zh.md', + { + repoRoot: root, sourcePath: 'guide.md', + isTranslationPairSource: fixturePairSource, markdown, + }, + ).links).toEqual(['dsh-translation-target:reference.md']) + } finally { + rmSync(root, { recursive: true, force: true }) + } + }) + + it('compares the first duplicate reference definition that CommonMark resolves', () => { + const root = mkdtempSync(join(tmpdir(), 'dsh-translation-structure-')) + try { + for (const name of ['reference', 'different', 'other']) { + writeFileSync(join(root, `${name}.md`), `# ${name}\n`) + writeFileSync(join(root, `${name}.zh.md`), `# ${name} zh\n`) + } + const sourceMarkdown = '[Reference][ref]\n\n[ref]: reference.md\n[ref]: other.md\n' + const counterpartMarkdown = '[参考][ref]\n\n[ref]: different.zh.md\n[ref]: other.zh.md\n' + const source = fixtureSignature(root, 'guide.md', sourceMarkdown, 'guide.zh.md') + const counterpart = fixtureSignature(root, 'guide.zh.md', counterpartMarkdown, 'guide.md') + expect(translationStructureDiff(source, counterpart)).toEqual([ + 'link target #1 diverges between the pair: "dsh-translation-target:reference.md" vs "dsh-translation-target:different.md"', + ]) + } finally { + rmSync(root, { recursive: true, force: true }) + } + }) + it('accepts matching list kinds, starts, and item counts', () => { const source = signature('3. One\n4. Two\n\n- A\n- B\n') const counterpart = signature('3. 一\n4. 二\n\n- 甲\n- 乙\n') diff --git a/scripts/translation-pairing.ts b/scripts/translation-pairing.ts index 930764034b..95b70c48b1 100644 --- a/scripts/translation-pairing.ts +++ b/scripts/translation-pairing.ts @@ -12,6 +12,11 @@ import { fromMarkdown } from 'mdast-util-from-markdown' import { gfmFromMarkdown } from 'mdast-util-gfm' import { gfm } from 'micromark-extension-gfm' import type { Nodes } from 'mdast' +import { + languageSwitcherLinkOffset, + semanticTranslationLinkNodeTarget, + type TranslationLinkContext, +} from './translation-links.ts' /** Complete opening marker line: `` (slug captured). */ const GENERATED_REGION_BEGIN_LINE = /^$/ @@ -125,7 +130,7 @@ export interface TranslationPairingManifest { } const README_ARTIFACT = /(?:^|\/)readme(?:\.md|\.zh\.md|\.i18n\.yaml)$/i -const ROOT_CONTRIBUTING_ARTIFACT = /^contributing(?:\.md|\.zh\.md|\.i18n\.yaml)$/i +const ROOT_PAIRED_DOCUMENT_ARTIFACT = /^(?:brand_guidelines|contributing|safety)(?:\.md|\.zh\.md|\.i18n\.yaml)$/i const NON_SOURCE_DIRECTORIES = new Set([ 'node_modules', 'lib', @@ -180,7 +185,7 @@ function isTranslationSourceExcluded(file: string): boolean { export function isTranslationScopeFile(file: string): boolean { return !file.startsWith('.agents/notes/archived/') && !isTranslationSourceExcluded(file) && (README_ARTIFACT.test(file) - || ROOT_CONTRIBUTING_ARTIFACT.test(file) + || ROOT_PAIRED_DOCUMENT_ARTIFACT.test(file) || file.startsWith('.agents/notes/') || file.startsWith('docs/') || file.startsWith('python/')) @@ -213,6 +218,22 @@ export function parseTranslationPairingManifest(content: string): TranslationPai return { excluded: excludedField(record) } } +/** Whether a manifest entry excludes one exact file or a directory subtree. */ +export function isTranslationPairingManifestExcluded( + file: string, + manifest: TranslationPairingManifest, +): boolean { + return manifest.excluded.some(entry => (entry.endsWith('/') ? file.startsWith(entry) : file === entry)) +} + +/** Build the active bilingual-source predicate shared by every link consumer. */ +export function translationPairSourcePredicate( + manifest: TranslationPairingManifest, +): (sourcePath: string) => boolean { + return sourcePath => isTranslationScopeFile(sourcePath) + && !isTranslationPairingManifestExcluded(sourcePath, manifest) +} + /** * Normalize one CLI pair argument to its English anchor path: any of the * pair's three files (`foo.md`, `foo.zh.md`, `foo.i18n.yaml`) or the bare @@ -309,18 +330,6 @@ export function languageSwitcherTargets(counterpart: string): string[] { return [basename(counterpart), `${PUBLIC_REPOSITORY_BLOB_ROOT}${counterpart}`] } -/** Whether the tree contains a link to any accepted target. */ -export function linksTo(tree: Nodes, targets: string | readonly string[]): boolean { - const accepted = new Set(typeof targets === 'string' ? [targets] : targets) - let found = false - const visit = (node: Nodes): void => { - if (node.type === 'link' && accepted.has(node.url)) found = true - if ('children' in node) for (const child of node.children) visit(child) - } - visit(tree) - return found -} - /** Generated English sources cannot carry a switcher without making their generator stale. */ export function requiresSourceLanguageSwitcher(source: string): boolean { return ![ @@ -347,11 +356,21 @@ export function requiresSourceLanguageSwitcher(source: string): boolean { export function translationStructureSignature( tree: Nodes, switcherTargets: string | readonly string[], + linkContext: TranslationLinkContext & { markdown: string }, ): TranslationStructureSignature { - const acceptedSwitchers = new Set( - typeof switcherTargets === 'string' ? [switcherTargets] : switcherTargets, - ) + const switcherOffset = languageSwitcherLinkOffset(tree, linkContext.markdown, switcherTargets) const sig: TranslationStructureSignature = { headings: [], code: [], tables: [], lists: [], links: [] } + const definitions = new Map>() + const collectDefinitions = (node: Nodes): void => { + if (node.type === 'definition' && !definitions.has(node.identifier)) { + definitions.set(node.identifier, node) + } + if ('children' in node) for (const child of node.children) collectDefinitions(child) + } + collectDefinitions(tree) + const linkTarget = (node: Extract): string => ( + semanticTranslationLinkNodeTarget(node, linkContext.markdown, linkContext) + ) const visit = (node: Nodes): void => { switch (node.type) { case 'heading': @@ -369,8 +388,17 @@ export function translationStructureSignature( : `bullet:items=${node.children.length}`) break case 'link': - if (!acceptedSwitchers.has(node.url)) sig.links.push(node.url) + if (node.position?.start.offset !== switcherOffset) { + sig.links.push(linkTarget(node)) + } break + case 'linkReference': { + const definition = definitions.get(node.identifier) + if (definition !== undefined) { + sig.links.push(linkTarget(definition)) + } + break + } default: // Every other node kind is prose or a container, not part of the signature. break diff --git a/scripts/translation-prompt.spec.ts b/scripts/translation-prompt.spec.ts index 65160896c7..96ddc98991 100644 --- a/scripts/translation-prompt.spec.ts +++ b/scripts/translation-prompt.spec.ts @@ -58,6 +58,10 @@ describe('translation prompt rendering', () => { expect(rendered).toContain('Markdown emphasis markers do not create a word boundary') expect(rendered).toContain('Never invent responsibility merely to avoid a passive construction') expect(rendered).toContain('Never vary a terminology-table form, defined concept, or contract verb merely for stylistic variety') + expect(rendered).toContain('Chinese output uses its `.zh.md` path') + expect(rendered).toContain('belongs to the active bilingual corpus') + expect(rendered).toContain('a missing counterpart in that corpus is an error') + expect(rendered).toContain('exact query/fragment suffix') expect(rendered).toContain('Return exactly three raw XML sections') }) diff --git a/scripts/ts-project.ts b/scripts/ts-project.ts index 9a0400a39d..53b100ceb0 100644 --- a/scripts/ts-project.ts +++ b/scripts/ts-project.ts @@ -11,6 +11,12 @@ interface ProjectGraph { options: ts.CompilerOptions } +/** + * A compiler face: the two aggregates a repository-wide program may seed from. + * The root solution is never one of them. + */ +export type CompilerFace = 'host' | 'client' + /** TypeScript config host shared by repository scripts. */ export const repositoryConfigHost: ts.ParseConfigFileHost = { useCaseSensitiveFileNames: ts.sys.useCaseSensitiveFileNames, @@ -24,12 +30,12 @@ export const repositoryConfigHost: ts.ParseConfigFileHost = { } /** - * Parse the host aggregate tsconfig and flatten all referenced projects into one + * Parse one face aggregate tsconfig and flatten all referenced projects into one * semantic graph. Never seed the root solution: flattening host+client into one * program collides the cordis Context merges. */ -function loadProjectGraph(projectRoot: string): ProjectGraph { - const rootConfigPath = resolve(projectRoot, 'tsconfig.host.json') +function loadProjectGraph(projectRoot: string, face: CompilerFace): ProjectGraph { + const rootConfigPath = resolve(projectRoot, `tsconfig.${face}.json`) const rootConfig = parseConfig(rootConfigPath) const rootNames = new Set() const visited = new Set() @@ -81,8 +87,12 @@ export class TypeScriptProject { /** The checker shared by every semantic query in this project. */ readonly checker: ts.TypeChecker - constructor(private readonly projectRoot: string) { - const graph = loadProjectGraph(projectRoot) + /** + * @param projectRoot - repository root the program is seeded and reported from. + * @param face - which compiler face aggregate to flatten. + */ + constructor(readonly projectRoot: string, face: CompilerFace = 'host') { + const graph = loadProjectGraph(projectRoot, face) this.program = ts.createProgram(graph.rootNames, semanticCompilerOptions(graph.options)) this.checker = this.program.getTypeChecker() } diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index 5e6184b88d..30a98799b6 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -301,6 +301,26 @@ "symbol": "ScheduleView", "source": "packages/schedule/schedule/src/types.ts" }, + { + "doc": "docs/subsystems/agent-team.md", + "symbol": "TeamMemberSnapshot", + "source": "packages/experimental/agent-team/src/types.ts" + }, + { + "doc": "docs/subsystems/agent-team.md", + "symbol": "TeamMessageSnapshot", + "source": "packages/experimental/agent-team/src/types.ts" + }, + { + "doc": "docs/subsystems/agent-team.md", + "symbol": "TeamMessageSource", + "source": "packages/experimental/agent-team/src/types.ts" + }, + { + "doc": "docs/subsystems/agent-team.md", + "symbol": "TeamTaskSnapshot", + "source": "packages/experimental/agent-team/src/types.ts" + }, { "doc": "docs/subsystems/commands.md", "symbol": "CommandInputDescriptor", @@ -351,6 +371,11 @@ "symbol": "ToolProviderResult", "source": "packages/core/system-prompt/src/index.ts" }, + { + "doc": "docs/subsystems/llm-streaming.md", + "symbol": "ReplayEnvelope", + "source": "packages/llm/llm/src/types.ts" + }, { "doc": "docs/subsystems/llm-streaming.md", "symbol": "StreamChunk", @@ -424,9 +449,9 @@ "source": "packages/core/session/src/types.ts" }, { - "doc": "docs/subsystems/session.md", + "doc": "docs/subsystems/todo.md", "symbol": "TodoItem", - "source": "packages/core/session/src/types.ts" + "source": "packages/todo/tool-todo/src/types.ts" }, { "doc": "docs/subsystems/session.md", @@ -660,11 +685,21 @@ "symbol": "SessionReferenceInput", "source": "packages/context/session-reference/src/types.ts" }, + { + "doc": "docs/subsystems/session-reference.md", + "symbol": "FileReferenceCandidate", + "source": "packages/context/file-reference/src/types.ts" + }, { "doc": "docs/subsystems/session-reference.md", "symbol": "SessionReferenceCandidate", "source": "packages/context/session-reference/src/types.ts" }, + { + "doc": "docs/subsystems/session-reference.md", + "symbol": "SessionReferenceMentionCandidate", + "source": "packages/context/session-reference/src/types.ts" + }, { "doc": "docs/subsystems/session-reference.md", "symbol": "PreparedReferencedMessage", @@ -880,6 +915,11 @@ "symbol": "ImageAttachmentLimits", "source": "packages/attachment/attachment/src/types.ts" }, + { + "doc": "docs/subsystems/attachment.md", + "symbol": "EncodedImageAttachment", + "source": "packages/attachment/attachment/src/types.ts" + }, { "doc": "docs/subsystems/attachment.md", "symbol": "SaveImageAttachment", @@ -890,6 +930,16 @@ "symbol": "StoredImageAttachment", "source": "packages/attachment/attachment/src/types.ts" }, + { + "doc": "docs/subsystems/attachment.md", + "symbol": "ImageRequestPolicy", + "source": "packages/attachment/attachment/src/types.ts" + }, + { + "doc": "docs/subsystems/attachment.md", + "symbol": "RequestImageAttachment", + "source": "packages/attachment/attachment/src/types.ts" + }, { "doc": "docs/subsystems/shell.md", "symbol": "ShellExecRequest", diff --git a/scripts/types/client-build-environment/index.d.ts b/scripts/types/client-build-environment/index.d.ts new file mode 100644 index 0000000000..db7b9f24fe --- /dev/null +++ b/scripts/types/client-build-environment/index.d.ts @@ -0,0 +1,7 @@ +/** Build-time values that bundlers replace before client code reaches a browser. */ +declare const process: { + readonly env: { + readonly NODE_ENV?: string + readonly [name: `DSH_CLIENT_${string}`]: string | undefined + } +} diff --git a/scripts/verify-application-entrypoints.spec.ts b/scripts/verify-application-entrypoints.spec.ts new file mode 100644 index 0000000000..02ba9a2196 --- /dev/null +++ b/scripts/verify-application-entrypoints.spec.ts @@ -0,0 +1,104 @@ +/** Application-entrypoint classification and dsh-launch enforcement. */ + +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join, resolve } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { applicationEntrypointViolations } from './verify-application-entrypoints.ts' + +const cleanups: string[] = [] + +afterEach(() => { + for (const path of cleanups.splice(0)) rmSync(path, { recursive: true, force: true }) +}) + +function fixture(): string { + const root = mkdtempSync(join(tmpdir(), 'dsh-application-entrypoints-')) + cleanups.push(root) + return root +} + +function write(root: string, path: string, content: string): void { + const target = resolve(root, path) + mkdirSync(dirname(target), { recursive: true }) + writeFileSync(target, content) +} + +describe('application entrypoints', () => { + it('accepts the repository launcher inventory', () => { + expect(applicationEntrypointViolations(resolve(import.meta.dirname, '..'))).toEqual([]) + }) + + it('rejects a package-level application bin', () => { + const root = fixture() + write(root, 'packages/example/app/package.json', JSON.stringify({ bin: { app: 'lib/bin.js' } })) + + expect(applicationEntrypointViolations(root)).toEqual([ + 'packages/example/app/package.json: package bin bypasses the dsh launcher; applications use apps/cli profiles', + ]) + }) + + it('rejects an unclassified executable source', () => { + const root = fixture() + write(root, 'packages/example/app/src/bin.ts', '#!/usr/bin/env node\n') + + expect(applicationEntrypointViolations(root)).toEqual([ + 'packages/example/app/src/bin.ts: executable source has no application/build/test classification', + ]) + }) + + it('rejects an executable at an application package root', () => { + const root = fixture() + write(root, 'apps/example/rogue.mjs', '#!/usr/bin/env node\n') + + expect(applicationEntrypointViolations(root)).toEqual([ + 'apps/example/rogue.mjs: executable source has no application/build/test classification', + ]) + }) + + it('rejects an executable at the repository root', () => { + const root = fixture() + write(root, 'rogue.mjs', '#!/usr/bin/env node\n') + + expect(applicationEntrypointViolations(root)).toEqual([ + 'rogue.mjs: executable source has no application/build/test classification', + ]) + }) + + it('rejects an unclassified executable in an example workspace', () => { + const root = fixture() + write(root, 'examples/rogue/src/bin.ts', '#!/usr/bin/env node\n') + + expect(applicationEntrypointViolations(root)).toEqual([ + 'examples/rogue/src/bin.ts: executable source has no application/build/test classification', + ]) + }) + + it('accepts the temporary private Python carrier source without an npm bin', () => { + const root = fixture() + write(root, 'packages/sdk/python-runtime/package.json', JSON.stringify({ private: true })) + write(root, 'packages/sdk/python-runtime/src/packaged-bin.ts', '#!/usr/bin/env node\n') + + expect(applicationEntrypointViolations(root)).toEqual([]) + }) + + it('rejects a classified demo wrapper that launches a package entry', () => { + const root = fixture() + write(root, 'package.json', JSON.stringify({ scripts: { 'demo:code-mode': 'node scripts/demo-code-mode.mjs' } })) + write(root, 'scripts/demo-code-mode.mjs', "spawn('node', ['packages/example/app/src/bin.ts'])\n") + + expect(applicationEntrypointViolations(root)).toEqual([ + 'scripts/demo-code-mode.mjs: application demo wrapper must launch apps/cli/src/bin.ts', + 'scripts/demo-code-mode.mjs: application demo wrapper must not launch a package entry directly', + ]) + }) + + it('rejects a new root demo until its launch role is classified', () => { + const root = fixture() + write(root, 'package.json', JSON.stringify({ scripts: { 'demo:new-app': 'dsh --profile new-app' } })) + + expect(applicationEntrypointViolations(root)).toEqual([ + 'package.json scripts.demo:new-app: demo launcher has no explicit dsh or in-process classification', + ]) + }) +}) diff --git a/scripts/verify-application-entrypoints.ts b/scripts/verify-application-entrypoints.ts new file mode 100644 index 0000000000..7dec320e79 --- /dev/null +++ b/scripts/verify-application-entrypoints.ts @@ -0,0 +1,199 @@ +/** + * Enforce dsh profiles as the only supported Node application launcher. + * Vendor CLIs, build tools, test tools, and the temporary private Python + * runtime carrier are explicit classifications rather than implicit holes. + */ + +import { existsSync, globSync, readFileSync } from 'node:fs' +import { resolve, sep } from 'node:path' +import { pathToFileURL } from 'node:url' + +type ManifestBin = string | Record + +interface PackageManifest { + readonly bin?: unknown +} + +interface RootManifest { + readonly scripts?: Record +} + +interface DemoPolicy { + readonly kind: 'dsh-direct' | 'dsh-wrapper' + readonly wrapper?: string +} + +/** Public product launcher plus the private build-only WebWorker packer. */ +const MANIFEST_BIN_ALLOWLIST = new Map([ + ['apps/cli/package.json', { dsh: 'lib/bin.js' }], + ['packages/experimental/webworker-packer/package.json', { 'dsh-pack-vfs-image': './bin.js' }], +]) + +/** Every executable in a Node application workspace has one explicit role. */ +const EXECUTABLE_SOURCE_ALLOWLIST = new Map([ + ['apps/cli/src/bin.ts', 'supported dsh application launcher'], + ['examples/acp-agent/tests/fixtures/shell/tool-pwsh/driver.ts', 'test-only subprocess driver'], + ['examples/acp-agent/tests/fixtures/subagent/subagent-acp/driver.ts', 'test-only subprocess driver'], + ['examples/acp-agent/tests/fixtures/subagent/subagent-claude-code/driver.ts', 'test-only subprocess driver'], + ['examples/acp-agent/tests/fixtures/subagent/subagent-codex/driver.ts', 'test-only subprocess driver'], + ['examples/headless-agent/tests/fixtures/headless-driver.ts', 'test-only subprocess driver'], + ['examples/headless-agent/tests/fixtures/session-telemetry-otel-driver.ts', 'test-only subprocess driver'], + ['examples/headless-agent/tests/fixtures/time-context-driver.ts', 'test-only subprocess driver'], + ['examples/python-sdk-agent/tests/fixtures/subagent/subagent-dsh-sdk/driver.ts', 'test-only subprocess driver'], + ['packages/experimental/webworker-packer/bin.js', 'private build-only wrapper'], + ['packages/experimental/webworker-packer/src/bin.ts', 'private build-only implementation'], + ['packages/sdk/client/tests/fake-runtime.ts', 'test-only SDK runtime peer'], + ['packages/sdk/python-runtime/src/packaged-bin.ts', 'temporary private Python runtime carrier'], + ['packages/test-support/llm-mock-server/src/bin.ts', 'test-only model server'], +]) + +/** Root demos are application wrappers and therefore must visibly select dsh. */ +const ROOT_DEMO_POLICIES = new Map([ + ['demo:acp', { kind: 'dsh-direct' }], + ['demo:code-mode', { kind: 'dsh-wrapper', wrapper: 'scripts/demo-code-mode.mjs' }], + ['demo:cordis', { kind: 'dsh-wrapper', wrapper: 'scripts/demo-cordis.mjs' }], +]) + +const SOURCE_PATTERNS = [ + '*.ts', + '*.js', + '*.mjs', + '*.cjs', + 'apps/**/*.ts', + 'apps/**/*.js', + 'apps/**/*.mjs', + 'apps/**/*.cjs', + 'examples/**/*.ts', + 'examples/**/*.js', + 'examples/**/*.mjs', + 'examples/**/*.cjs', + 'packages/**/*.ts', + 'packages/**/*.js', + 'packages/**/*.mjs', + 'packages/**/*.cjs', +] + +const SOURCE_EXCLUDES = [ + '**/node_modules/**', + '**/lib/**', + '**/dist/**', + '**/coverage/**', +] + +/** Convert a host path from glob output to the repository's slash form. */ +function repositoryPath(path: string): string { + return path.split(sep).join('/') +} + +/** Stable comparison for string and object npm `bin` declarations. */ +function normalizedBin(value: unknown): string | undefined { + if (typeof value === 'string') return JSON.stringify(value) + if (!isRecord(value)) return undefined + const entries = Object.entries(value) + if (!entries.every(([, target]) => typeof target === 'string')) return undefined + return JSON.stringify(Object.fromEntries(entries.sort(([left], [right]) => left.localeCompare(right)))) +} + +function manifestBinViolations(root: string): string[] { + const failures: string[] = [] + const manifests = globSync(['apps/*/package.json', 'packages/*/*/package.json'], { cwd: root }).sort() + for (const rawPath of manifests) { + const path = repositoryPath(rawPath) + const manifest = JSON.parse(readFileSync(resolve(root, path), 'utf8')) as PackageManifest + if (manifest.bin === undefined) continue + const expected = MANIFEST_BIN_ALLOWLIST.get(path) + if (expected === undefined) { + failures.push(`${path}: package bin bypasses the dsh launcher; applications use apps/cli profiles`) + continue + } + if (normalizedBin(manifest.bin) !== normalizedBin(expected)) { + failures.push(`${path}: classified bin must remain ${JSON.stringify(expected)}, got ${JSON.stringify(manifest.bin)}`) + } + } + return failures +} + +function executableSourceViolations(root: string): string[] { + const failures: string[] = [] + for (const rawPath of globSync(SOURCE_PATTERNS, { cwd: root, exclude: SOURCE_EXCLUDES }).sort()) { + const path = repositoryPath(rawPath) + const source = readFileSync(resolve(root, path), 'utf8') + if (!source.startsWith('#!')) continue + if (!EXECUTABLE_SOURCE_ALLOWLIST.has(path)) { + failures.push(`${path}: executable source has no application/build/test classification`) + } + } + return failures +} + +function referencesDshCli(source: string): boolean { + return source.includes('apps/cli/src/bin.ts') +} + +function referencesPackageEntry(source: string): boolean { + return /packages\/[^/\s'"`]+\/[^/\s'"`]+\/(?:src|lib)\/[^\s'"`]+/.test(source) +} + +function rootDemoViolations(root: string): string[] { + const manifestPath = resolve(root, 'package.json') + if (!existsSync(manifestPath)) return [] + const manifest = JSON.parse(readFileSync(manifestPath, 'utf8')) as RootManifest + const failures: string[] = [] + for (const [name, commandValue] of Object.entries(manifest.scripts ?? {}).sort(([left], [right]) => left.localeCompare(right))) { + if (!name.startsWith('demo:')) continue + const command = typeof commandValue === 'string' ? commandValue : '' + const policy = ROOT_DEMO_POLICIES.get(name) + if (policy === undefined) { + failures.push(`package.json scripts.${name}: demo launcher has no explicit dsh or in-process classification`) + continue + } + if (policy.kind === 'dsh-direct') { + if (!referencesDshCli(command)) failures.push(`package.json scripts.${name}: application demo must launch apps/cli/src/bin.ts`) + if (referencesPackageEntry(command)) failures.push(`package.json scripts.${name}: application demo must not launch a package entry directly`) + continue + } + const wrapper = policy.wrapper + if (wrapper === undefined || !command.includes(wrapper)) { + failures.push(`package.json scripts.${name}: classified wrapper must be ${String(wrapper)}`) + continue + } + const wrapperPath = resolve(root, wrapper) + if (!existsSync(wrapperPath)) { + failures.push(`${wrapper}: classified demo wrapper is missing`) + continue + } + const source = readFileSync(wrapperPath, 'utf8') + if (!referencesDshCli(source)) failures.push(`${wrapper}: application demo wrapper must launch apps/cli/src/bin.ts`) + if (referencesPackageEntry(source)) failures.push(`${wrapper}: application demo wrapper must not launch a package entry directly`) + } + return failures +} + +/** + * Find unsupported application entrypoints below a repository root. + * @param root - repository or test-fixture root. + * @returns deterministic path-qualified violations. + */ +export function applicationEntrypointViolations(root: string): string[] { + return [ + ...manifestBinViolations(root), + ...executableSourceViolations(root), + ...rootDemoViolations(root), + ] +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +if (process.argv[1] !== undefined && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) { + const root = resolve(import.meta.dirname, '..') + const failures = applicationEntrypointViolations(root) + if (failures.length > 0) { + console.error('verify-application-entrypoints: unsupported launcher(s):') + for (const failure of failures) console.error(` ${failure}`) + process.exitCode = 1 + } else { + console.log('verify-application-entrypoints: dsh is the only supported Node application launcher; the private Python carrier remains the temporary exception.') + } +} diff --git a/scripts/verify-client-domain-graph.spec.ts b/scripts/verify-client-domain-graph.spec.ts new file mode 100644 index 0000000000..4710f6eab7 --- /dev/null +++ b/scripts/verify-client-domain-graph.spec.ts @@ -0,0 +1,14 @@ +import { describe, expect, it } from 'vitest' +import { resolveClientImport } from './verify-client-domain-graph.ts' + +describe('client domain import resolution', () => { + it('preserves imports that leave src/client from a top-level file', () => { + expect(resolveClientImport('styles.ts', '../styles/base.css?inline')) + .toBe('../styles/base.css?inline') + }) + + it('normalizes imports between domains inside src/client', () => { + expect(resolveClientImport('input/hub.ts', '../queue/store.ts')) + .toBe('queue/store.ts') + }) +}) diff --git a/scripts/verify-client-domain-graph.ts b/scripts/verify-client-domain-graph.ts index 3fe8b750eb..e2017f19de 100644 --- a/scripts/verify-client-domain-graph.ts +++ b/scripts/verify-client-domain-graph.ts @@ -15,7 +15,7 @@ */ import { globSync, readdirSync, readFileSync, statSync } from 'node:fs' -import { join, resolve, sep } from 'node:path' +import { join, posix, resolve, sep } from 'node:path' const root = resolve(import.meta.dirname, '..') const CLIENT_DIR = join(root, 'packages/client') @@ -41,6 +41,16 @@ function domainOf(rel: string): string { return ix === -1 ? '' : rel.slice(0, ix) } +/** + * Resolve one relative import to a client-directory-relative path. + * @param file - Importing file relative to `src/client`. + * @param specifier - Relative module specifier from that file. + * @returns Normalized path, preserving leading `..` segments outside `src/client`. + */ +export function resolveClientImport(file: string, specifier: string): string { + return posix.normalize(posix.join(posix.dirname(file), specifier)) +} + function checkPackage(pkgName: string, clientDir: string): Violation[] { const violations: Violation[] = [] const files = listSources(clientDir) @@ -52,17 +62,8 @@ function checkPackage(pkgName: string, clientDir: string): Violation[] { for (const match of source.matchAll(/from\s+['"](\.[^'"]+)['"]/g)) { const spec = match[1] if (spec === undefined) continue - // Resolve the relative specifier against the importing file's directory - // to a client-dir-relative path. - const fromDir = rel.includes('/') ? rel.slice(0, rel.lastIndexOf('/')) : '' - const parts = (fromDir ? fromDir.split('/') : []) - for (const seg of spec.split('/')) { - if (seg === '.') continue - if (seg === '..') parts.pop() - else parts.push(seg) - } - const target = parts.join('/') - if (target.startsWith('..')) continue // out of client dir (package root) — package-level rules govern + const target = resolveClientImport(rel, spec) + if (target === '..' || target.startsWith('../')) continue // package-level rules govern const toDomain = domainOf(target) if (toDomain === '' || CONTRACT_DIRS.has(toDomain)) continue // top-level shared file or contract layer if (fromDomain === toDomain) continue // inside one domain @@ -78,21 +79,26 @@ function checkPackage(pkgName: string, clientDir: string): Violation[] { return violations } -const violations: Violation[] = [] -for (const pkg of readdirSync(CLIENT_DIR)) { - const clientDir = join(CLIENT_DIR, pkg, 'src/client') - try { - if (!statSync(clientDir).isDirectory()) continue - } catch { - // No client half in this package — nothing to layer-check. - continue +function main(): void { + const violations: Violation[] = [] + for (const pkg of readdirSync(CLIENT_DIR)) { + const clientDir = join(CLIENT_DIR, pkg, 'src/client') + try { + if (!statSync(clientDir).isDirectory()) continue + } catch { + // No client half in this package — nothing to layer-check. + continue + } + violations.push(...checkPackage(pkg, clientDir)) } - violations.push(...checkPackage(pkg, clientDir)) + + if (violations.length > 0) { + console.error(`verify-client-domain-graph: ${violations.length} violation(s):`) + for (const v of violations) console.error(` ${v.file} -> ${v.imported}\n ${v.reason}`) + process.exitCode = 1 + return + } + console.log('verify-client-domain-graph: client domain layering clean.') } -if (violations.length > 0) { - console.error(`verify-client-domain-graph: ${violations.length} violation(s):`) - for (const v of violations) console.error(` ${v.file} -> ${v.imported}\n ${v.reason}`) - process.exit(1) -} -console.log('verify-client-domain-graph: client domain layering clean.') +if (import.meta.filename === resolve(process.argv[1] ?? '')) main() diff --git a/scripts/verify-client-packages.spec.ts b/scripts/verify-client-packages.spec.ts new file mode 100644 index 0000000000..ca378f4b59 --- /dev/null +++ b/scripts/verify-client-packages.spec.ts @@ -0,0 +1,402 @@ +/** Tests for client package modes, dependency sections, and module requests. */ + +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { + collectClientPackageViolations, + collectRuntimeSourcePackageUses, + collectSourcePackageUses, + fixClientPackageManifests, + readClientDeclarations, + type ClientDeclaration, + type ClientPackage, + type ClientPackageFacts, +} from './verify-client-packages.ts' + +const CORDIS = '@deepseek-ai/cordis' +const roots: string[] = [] + +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }) +}) + +function declaration( + short: string, + fields: Partial> = {}, +): ClientDeclaration { + return { + name: short.startsWith('@') ? short : '@deepseek-ai/dsh-client-' + short, + manifest: 'packages/client/' + short.replace(/^.*\//, '') + '/package.json', + dynamic: true, + external: [], + inject: [], + ...fields, + } +} + +function pkg( + short: string, + fields: Partial> = {}, +): ClientPackage { + return { + ...declaration(short), + staticLinked: false, + sourceUses: {}, + runtimeSourceUses: {}, + dependencies: {}, + peerDependencies: { [CORDIS]: 'workspace:^' }, + devDependencies: { [CORDIS]: 'workspace:^' }, + ...fields, + } +} + +function facts( + packages: readonly ClientPackage[], + options: Partial> = {}, +): ClientPackageFacts { + return { + packages, + declarations: options.declarations ?? packages, + staticLinkedPackages: options.staticLinkedPackages ?? new Set( + packages.filter(item => item.staticLinked).map(item => item.name), + ), + platformModules: options.platformModules ?? [], + preloadedExternals: options.preloadedExternals ?? [], + parserPreloadIds: options.parserPreloadIds + ?? (options.preloadedExternals ?? []).map(value => value.replace(/\/client$/, '')), + malformed: options.malformed ?? [], + } +} + +describe('source package uses', () => { + it('counts type imports, module augmentations, dynamic imports, and JSX', () => { + const uses = collectSourcePackageUses('feature.tsx', [ + "import type { A } from '@deepseek-ai/dsh-a/subpath'", + "declare module '@deepseek-ai/dsh-client-ui-slots' {}", + "const load = () => import('@deepseek-ai/dsh-b')", + 'export const view =
', + "export type { Local } from './local.ts'", + ].join('\n')) + + expect([...uses].sort()).toEqual([ + '@deepseek-ai/dsh-a', + '@deepseek-ai/dsh-b', + '@deepseek-ai/dsh-client-ui-slots', + 'react', + ]) + expect([...collectRuntimeSourcePackageUses('feature.tsx', [ + "import type { A } from '@deepseek-ai/dsh-a/subpath'", + "declare module '@deepseek-ai/dsh-client-ui-slots' {}", + "const load = () => import('@deepseek-ai/dsh-b')", + 'export const view =
', + ].join('\n'))].sort()).toEqual([ + '@deepseek-ai/dsh-b', + 'react', + ]) + }) +}) + +describe('package modes', () => { + it('accepts one dynamic package and one statically linked package', () => { + const dynamic = pkg('runtime') + const shell = pkg('ui-slots', { dynamic: false, staticLinked: true }) + expect(collectClientPackageViolations(facts([dynamic, shell]))).toEqual([]) + }) + + it('rejects a package with both modes or neither mode', () => { + const both = pkg('both', { staticLinked: true }) + const neither = pkg('neither', { dynamic: false }) + const found = collectClientPackageViolations(facts([both, neither])) + expect(found).toHaveLength(2) + expect(found.join('\n')).toContain('must be dynamic or statically linked, not both') + expect(found.join('\n')).toContain('has no supported client package mode') + }) + + it('requires seeded workspace packages to use staticLinked and preloads to name dynamic rows', () => { + const slots = declaration('ui-slots', { dynamic: false }) + const runtime = declaration('runtime', { dynamic: false }) + const found = collectClientPackageViolations(facts([], { + declarations: [slots, runtime], + platformModules: [slots.name], + preloadedExternals: [runtime.name + '/client'], + })) + expect(found).toHaveLength(2) + expect(found.join('\n')).toContain('does not use the staticLinked preset') + expect(found.join('\n')).toContain('has no dynamic dsh.client row') + }) + + it('requires every preloaded external to have a parser preload row', () => { + const runtime = declaration('runtime') + expect(collectClientPackageViolations(facts([], { + declarations: [runtime], + preloadedExternals: [runtime.name + '/client'], + parserPreloadIds: [], + }))).toEqual([ + 'packages/client/web/src/platform.ts: parser-preloaded external ' + + '"@deepseek-ai/dsh-client-runtime/client" has no matching PARSER_PRELOAD_IDS row in ' + + 'packages/client/modules/src/index.ts', + ]) + }) +}) + +describe('dependency sections', () => { + it('accepts dynamic peer plus dev relationships, static dev inputs, and private dependencies', () => { + const slots = pkg('ui-slots', { dynamic: false, staticLinked: true }) + const runtime = pkg('runtime', { + inject: ['@deepseek-ai/dsh-client-feature'], + sourceUses: { + '@deepseek-ai/dsh-agent': ['packages/client/runtime/src/index.ts'], + '@deepseek-ai/dsh-client-ui-slots': ['packages/client/runtime/src/client/slots.ts'], + react: ['packages/client/runtime/src/client/view.tsx'], + }, + dependencies: { immer: '^10.1.1' }, + peerDependencies: { + [CORDIS]: 'workspace:^', + '@deepseek-ai/dsh-agent': 'workspace:^', + '@deepseek-ai/dsh-client-feature': 'workspace:^', + }, + devDependencies: { + [CORDIS]: 'workspace:^', + '@deepseek-ai/dsh-agent': 'workspace:^', + '@deepseek-ai/dsh-client-feature': 'workspace:^', + '@deepseek-ai/dsh-client-ui-slots': 'workspace:^', + react: '^18.2.0', + }, + }) + expect(collectClientPackageViolations(facts([slots, runtime], { + platformModules: ['react', slots.name], + }))).toEqual([]) + }) + + it('rejects internal dependencies, static peers, and mismatched peer development ranges', () => { + const slots = pkg('ui-slots', { dynamic: false, staticLinked: true }) + const subject = pkg('feature', { + sourceUses: { + '@deepseek-ai/dsh-agent': ['packages/client/feature/src/index.ts'], + [slots.name]: ['packages/client/feature/src/view.tsx'], + }, + dependencies: { '@deepseek-ai/dsh-agent': 'workspace:^' }, + peerDependencies: { [CORDIS]: 'workspace:^', [slots.name]: 'workspace:^' }, + devDependencies: { [CORDIS]: 'workspace:^', [slots.name]: 'workspace:*' }, + }) + const found = collectClientPackageViolations(facts([slots, subject])) + expect(found).toHaveLength(2) + expect(found.join('\n')).toContain('peer-installed DSH relationship') + expect(found.join('\n')).toContain('static client input') + }) + + it('requires every peer to have the same development range', () => { + const subject = pkg('feature', { + peerDependencies: { [CORDIS]: 'workspace:^', '@deepseek-ai/cordis-plugin-loader': 'workspace:^' }, + }) + expect(collectClientPackageViolations(facts([subject]))).toEqual([ + 'packages/client/feature/package.json: peerDependencies.@deepseek-ai/cordis-plugin-loader' + + ' is workspace:^, so devDependencies.@deepseek-ai/cordis-plugin-loader must use the same range;' + + ' found no declaration', + ]) + }) + + it('requires statically linked third-party runtime imports in dependencies', () => { + const primitives = pkg('ui-primitives', { + dynamic: false, + staticLinked: true, + runtimeSourceUses: { shiki: ['packages/client/ui-primitives/src/highlight.ts'] }, + devDependencies: { [CORDIS]: 'workspace:^', shiki: '^4.3.1' }, + }) + const found = collectClientPackageViolations(facts([primitives])) + expect(found).toHaveLength(1) + expect(found[0]).toContain('runtime import retained by a statically linked artifact') + expect(found[0]).toContain('declare it only in dependencies') + + const valid = { ...primitives, dependencies: { shiki: '^4.3.1' }, devDependencies: { [CORDIS]: 'workspace:^' } } + expect(collectClientPackageViolations(facts([valid]))).toEqual([]) + }) + + it('keeps the web shell runtime inputs development-only', () => { + const web = pkg('web', { + dynamic: false, + staticLinked: true, + runtimeSourceUses: { + '@deepseek-ai/cordis-plugin-loader': ['packages/client/web/src/boot.ts'], + react: ['packages/client/web/src/seed.ts'], + }, + devDependencies: { + [CORDIS]: 'workspace:^', + '@deepseek-ai/cordis-plugin-loader': 'workspace:^', + react: '^18.2.0', + }, + }) + expect(collectClientPackageViolations(facts([web]))).toEqual([]) + }) + + it('allows npm dependency cycles', () => { + const a = pkg('a', { + peerDependencies: { [CORDIS]: 'workspace:^', '@deepseek-ai/dsh-client-b': 'workspace:^' }, + devDependencies: { [CORDIS]: 'workspace:^', '@deepseek-ai/dsh-client-b': 'workspace:^' }, + }) + const b = pkg('b', { + peerDependencies: { [CORDIS]: 'workspace:^', '@deepseek-ai/dsh-client-a': 'workspace:^' }, + devDependencies: { [CORDIS]: 'workspace:^', '@deepseek-ai/dsh-client-a': 'workspace:^' }, + }) + expect(collectClientPackageViolations(facts([a, b]))).toEqual([]) + }) +}) + +describe('module requests', () => { + it('accepts a dynamic row supplier and its client subpath', () => { + const ui = declaration('ui', { external: ['@deepseek-ai/dsh-client-slots/client'] }) + const slots = declaration('slots') + expect(collectClientPackageViolations(facts([], { declarations: [ui, slots] }))).toEqual([]) + }) + + it('rejects an explicit baseline request', () => { + const ui = declaration('ui', { external: ['react'] }) + expect(collectClientPackageViolations(facts([], { + declarations: [ui], + platformModules: ['react'], + }))).toEqual([ + ui.manifest + ': dsh.client.external repeats baseline module "react"; remove the explicit declaration', + ]) + }) + + it('rejects duplicates, empty values, self-requests, and missing suppliers', () => { + const ui = declaration('ui', { + external: ['', '@deepseek-ai/dsh-client-ui', '@deepseek-ai/dsh-missing', '@deepseek-ai/dsh-missing'], + inject: ['', '@deepseek-ai/dsh-a', '@deepseek-ai/dsh-a'], + }) + const found = collectClientPackageViolations(facts([], { declarations: [ui] })) + expect(found).toHaveLength(6) + expect(found.join('\n')).toContain('dsh.client.external contains an empty value') + expect(found.join('\n')).toContain('dsh.client.inject contains an empty value') + expect(found.join('\n')).toContain('names its own row') + expect(found.join('\n')).toContain('has no supplier') + }) + + it('rejects synchronous module-request cycles but ignores inject cycles', () => { + const a = declaration('a', { + external: ['@deepseek-ai/dsh-client-b'], + inject: ['@deepseek-ai/dsh-client-b'], + }) + const b = declaration('b', { + external: ['@deepseek-ai/dsh-client-a'], + inject: ['@deepseek-ai/dsh-client-a'], + }) + const found = collectClientPackageViolations(facts([], { declarations: [a, b] })) + expect(found).toHaveLength(1) + expect(found[0]).toContain('synchronous dsh.client.external cycle') + }) +}) + +describe('manifest declarations', () => { + it('reports malformed arrays without hiding other packages', () => { + const root = mkdtempSync(join(tmpdir(), 'client-packages-')) + roots.push(root) + const files: Record = { + 'packages/g/a/package.json': { + name: '@f/a', dsh: { client: { external: 'react', inject: ['@f/b', 1] } }, + }, + 'packages/g/b/package.json': { name: '@f/b', dsh: { client: {} } }, + } + for (const [path, value] of Object.entries(files)) { + mkdirSync(dirname(join(root, path)), { recursive: true }) + writeFileSync(join(root, path), JSON.stringify(value)) + } + + const result = readClientDeclarations(root) + expect(result.declarations).toHaveLength(2) + expect(result.malformed).toEqual([ + 'packages/g/a/package.json: @f/a dsh.client.external must be a string array', + 'packages/g/a/package.json: @f/a dsh.client.inject must be a string array', + ]) + }) + + it('fixes unambiguous dependency sections and declaration entries', () => { + const root = mkdtempSync(join(tmpdir(), 'client-packages-fix-')) + roots.push(root) + const subject = pkg('feature', { + external: ['', 'react', '@deepseek-ai/dsh-client-feature', '@deepseek-ai/dsh-missing'], + inject: ['', '@deepseek-ai/dsh-agent', '@deepseek-ai/dsh-agent'], + sourceUses: { + '@deepseek-ai/dsh-agent': ['packages/client/feature/src/index.ts'], + '@deepseek-ai/dsh-client-ui-slots': ['packages/client/feature/src/view.tsx'], + }, + dependencies: { + [CORDIS]: 'workspace:^', + '@deepseek-ai/dsh-agent': 'workspace:*', + }, + peerDependencies: { + '@deepseek-ai/dsh-client-ui-slots': 'workspace:^', + '@deepseek-ai/cordis-plugin-loader': 'workspace:^', + }, + devDependencies: {}, + }) + const slots = declaration('ui-slots', { dynamic: false }) + const manifest = { + name: subject.name, + dsh: { client: { external: subject.external, inject: subject.inject, platform: 'web' } }, + dependencies: subject.dependencies, + peerDependencies: subject.peerDependencies, + devDependencies: subject.devDependencies, + } + mkdirSync(dirname(join(root, subject.manifest)), { recursive: true }) + writeFileSync(join(root, subject.manifest), JSON.stringify(manifest)) + writeFileSync(join(root, 'package.json'), JSON.stringify({ private: true })) + + expect(fixClientPackageManifests(root, facts([subject], { + declarations: [subject, slots], + staticLinkedPackages: new Set([slots.name]), + platformModules: ['react', slots.name], + }))).toEqual([subject.manifest]) + + const fixed = JSON.parse(readFileSync(join(root, subject.manifest), 'utf8')) as { + dsh: { client: { external: string[]; inject: string[] } } + dependencies?: Record + peerDependencies: Record + devDependencies: Record + } + expect(fixed.dsh.client).toMatchObject({ + external: ['@deepseek-ai/dsh-missing'], + inject: ['@deepseek-ai/dsh-agent'], + }) + expect(fixed.dependencies).toBeUndefined() + expect(fixed.peerDependencies).toEqual({ + '@deepseek-ai/cordis-plugin-loader': 'workspace:^', + [CORDIS]: 'workspace:^', + '@deepseek-ai/dsh-agent': 'workspace:*', + }) + expect(fixed.devDependencies).toEqual({ + '@deepseek-ai/dsh-client-ui-slots': 'workspace:^', + [CORDIS]: 'workspace:^', + '@deepseek-ai/dsh-agent': 'workspace:*', + '@deepseek-ai/cordis-plugin-loader': 'workspace:^', + }) + }) + + it('fixes a statically linked runtime import into dependencies', () => { + const root = mkdtempSync(join(tmpdir(), 'client-packages-static-fix-')) + roots.push(root) + const subject = pkg('ui-primitives', { + dynamic: false, + staticLinked: true, + runtimeSourceUses: { shiki: ['packages/client/ui-primitives/src/highlight.ts'] }, + devDependencies: { [CORDIS]: 'workspace:^', shiki: '^4.3.1' }, + }) + mkdirSync(dirname(join(root, subject.manifest)), { recursive: true }) + writeFileSync(join(root, subject.manifest), JSON.stringify({ + name: subject.name, + peerDependencies: subject.peerDependencies, + devDependencies: subject.devDependencies, + })) + writeFileSync(join(root, 'package.json'), JSON.stringify({ private: true })) + + expect(fixClientPackageManifests(root, facts([subject]))).toEqual([subject.manifest]) + const fixed = JSON.parse(readFileSync(join(root, subject.manifest), 'utf8')) as { + dependencies: Record + devDependencies: Record + } + expect(fixed.dependencies).toEqual({ shiki: '^4.3.1' }) + expect(fixed.devDependencies).toEqual({ [CORDIS]: 'workspace:^' }) + }) +}) diff --git a/scripts/verify-client-packages.ts b/scripts/verify-client-packages.ts new file mode 100644 index 0000000000..225f944bdf --- /dev/null +++ b/scripts/verify-client-packages.ts @@ -0,0 +1,895 @@ +/** + * Verify client package modes, npm dependency sections, and the synchronous + * browser module-request graph. + */ + +import { globSync, readFileSync, writeFileSync } from 'node:fs' +import { dirname, resolve, sep } from 'node:path' +import { pathToFileURL } from 'node:url' +import ts from 'typescript' +import { TypeScriptProject } from './ts-project.ts' + +const GATE = 'verify-client-packages' +const CLIENT_MANIFEST_GLOB = 'packages/client/*/package.json' +const MANIFEST_GLOBS = ['packages/*/*/package.json', 'apps/*/package.json', 'vendor/*/package.json'] +const CONFIG_GLOB = 'packages/*/*/tsdown.config.ts' +const PLATFORM_SOURCE = 'packages/client/web/src/platform.ts' +const PARSER_PRELOAD_SOURCE = 'packages/client/modules/src/index.ts' +const STATIC_PRESET_SOURCE = 'packages/client/tsdown.client.ts' +const CORDIS = '@deepseek-ai/cordis' +const DSH_PREFIX = '@deepseek-ai/dsh-' +const CLIENT_WEB = '@deepseek-ai/dsh-client-web' + +/** One workspace package's browser-module declaration. */ +export interface ClientDeclaration { + readonly name: string + readonly manifest: string + readonly dynamic: boolean + readonly external: readonly string[] + /** Informational package dependencies declared by the row. */ + readonly inject: readonly string[] +} + +/** One package directly under packages/client. */ +export interface ClientPackage extends ClientDeclaration { + readonly staticLinked: boolean + readonly sourceUses: Readonly> + readonly runtimeSourceUses: Readonly> + readonly dependencies: Readonly> + readonly peerDependencies: Readonly> + readonly devDependencies: Readonly> +} + +/** Complete source-plane input to the client package verifier. */ +export interface ClientPackageFacts { + readonly packages: readonly ClientPackage[] + readonly declarations: readonly ClientDeclaration[] + readonly staticLinkedPackages: ReadonlySet + readonly platformModules: readonly string[] + readonly preloadedExternals: readonly string[] + readonly parserPreloadIds: readonly string[] + readonly malformed: readonly string[] +} + +/** Result of reading every workspace browser-module declaration. */ +export interface ClientDeclarations { + readonly declarations: ClientDeclaration[] + readonly malformed: string[] +} + +/** + * Collect bare packages referenced by one production source file. + * @param path - File path used to select TypeScript's parser mode. + * @param source - Source text to inspect. + * @returns Bare package names referenced by imports, declarations, or JSX. + */ +export function collectSourcePackageUses(path: string, source: string): Set { + const sourceFile = ts.createSourceFile(path, source, ts.ScriptTarget.Latest, true) + return collectSourceFilePackageUses(sourceFile, false) +} + +/** + * Collect bare packages whose values one production source file reaches at runtime. + * @param path - File path used to select TypeScript's parser mode. + * @param source - Source text to inspect. + * @returns Bare package names retained by runtime imports, exports, requires, or JSX. + */ +export function collectRuntimeSourcePackageUses(path: string, source: string): Set { + const sourceFile = ts.createSourceFile(path, source, ts.ScriptTarget.Latest, true) + return collectSourceFilePackageUses(sourceFile, true) +} + +function importCarriesRuntimeValue(node: ts.ImportDeclaration): boolean { + const clause = node.importClause + if (clause === undefined) return true + if (clause.phaseModifier === ts.SyntaxKind.TypeKeyword) return false + const bindings = clause.namedBindings + return clause.name !== undefined + || bindings === undefined + || ts.isNamespaceImport(bindings) + || bindings.elements.length === 0 + || bindings.elements.some(element => !element.isTypeOnly) +} + +function exportCarriesRuntimeValue(node: ts.ExportDeclaration): boolean { + if (node.isTypeOnly) return false + const clause = node.exportClause + if (clause === undefined || ts.isNamespaceExport(clause)) return true + return clause.elements.length === 0 || clause.elements.some(element => !element.isTypeOnly) +} + +function collectSourceFilePackageUses(sourceFile: ts.SourceFile, runtimeOnly: boolean): Set { + const uses = new Set() + + const add = (specifier: ts.Expression | undefined): void => { + if (specifier === undefined || !ts.isStringLiteral(specifier) || !isBareSpecifier(specifier.text)) return + uses.add(packageNameOf(specifier.text)) + } + const visit = (node: ts.Node): void => { + if (ts.isImportDeclaration(node)) { + if (!runtimeOnly || importCarriesRuntimeValue(node)) add(node.moduleSpecifier) + } else if (ts.isExportDeclaration(node)) { + if (!runtimeOnly || exportCarriesRuntimeValue(node)) add(node.moduleSpecifier) + } else if (ts.isImportEqualsDeclaration(node) && ts.isExternalModuleReference(node.moduleReference)) { + if (!runtimeOnly || !node.isTypeOnly) add(node.moduleReference.expression) + } else if (!runtimeOnly && ts.isImportTypeNode(node) && ts.isLiteralTypeNode(node.argument)) { + add(node.argument.literal) + } else if (ts.isCallExpression(node) + && (node.expression.kind === ts.SyntaxKind.ImportKeyword + || ts.isIdentifier(node.expression) && node.expression.text === 'require')) { + add(node.arguments[0]) + } else if (!runtimeOnly && ts.isModuleDeclaration(node) && ts.isStringLiteral(node.name)) { + add(node.name) + } else if (ts.isJsxElement(node) || ts.isJsxSelfClosingElement(node) || ts.isJsxFragment(node)) { + uses.add('react') + } + ts.forEachChild(node, visit) + } + + visit(sourceFile) + return uses +} + +/** + * Read browser-module declarations from workspace manifests. + * @param root - Absolute repository root. + * @returns Declarations and malformed dsh.client fields. + */ +export function readClientDeclarations(root: string): ClientDeclarations { + const malformed: string[] = [] + const declarations = globSync(MANIFEST_GLOBS, { cwd: root }) + .map(normalizePath) + .sort() + .flatMap(path => readDeclaration(root, path, malformed) ?? []) + return { declarations, malformed } +} + +/** + * Return every client package policy violation. + * @param facts - Package modes, manifests, source uses, and platform module lists. + * @returns Stable self-contained diagnostics. + */ +export function collectClientPackageViolations(facts: ClientPackageFacts): string[] { + return [ + ...facts.malformed, + ...collectModeViolations(facts), + ...collectDependencyViolations(facts), + ...collectModuleViolations(facts), + ].sort((left, right) => left.localeCompare(right)) +} + +interface ManifestDocument { + readonly path: string + readonly manifest: Manifest + changed: boolean +} + +type DependencySection = 'dependencies' | 'peerDependencies' | 'devDependencies' + +/** + * Repair manifest declarations whose intended result follows uniquely from the policy. + * @param root - Absolute repository root. + * @param facts - Facts used by the verification pass. + * @returns Repository-relative manifests written by the fixer. + */ +export function fixClientPackageManifests(root: string, facts: ClientPackageFacts): string[] { + const documents = new Map() + const document = (path: string): ManifestDocument => { + const cached = documents.get(path) + if (cached !== undefined) return cached + const loaded: ManifestDocument = { + path, + manifest: JSON.parse(readFileSync(resolve(root, path), 'utf8')) as Manifest, + changed: false, + } + documents.set(path, loaded) + return loaded + } + + const baseline = new Set([...facts.platformModules, ...facts.preloadedExternals]) + for (const declaration of facts.declarations.filter(entry => entry.dynamic)) { + const target = document(declaration.manifest) + const dsh = isRecord(target.manifest.dsh) ? target.manifest.dsh : undefined + const client = isRecord(dsh?.client) ? dsh.client : undefined + if (client === undefined) continue + target.changed = normalizeClientArray(client, 'inject', () => false) || target.changed + target.changed = normalizeClientArray( + client, + 'external', + value => baseline.has(value) || rowPackageOf(value, new Set([declaration.name])) === declaration.name, + ) || target.changed + } + + const staticInputs = new Set([ + ...facts.staticLinkedPackages, + ...facts.platformModules.map(packageNameOf), + ]) + staticInputs.delete(CORDIS) + const inferredRanges = dependencyRangeCandidates(root) + for (const pkg of facts.packages) { + const target = document(pkg.manifest) + const expected = expectedSections(pkg, staticInputs) + for (const [name, rule] of expected) { + const range = preferredRange(target.manifest, name, rule.kind, inferredRanges) + if (range === undefined) continue + target.changed = rule.kind === 'dependency' + ? ensureDependencyOnly(target.manifest, name, range) || target.changed + : rule.kind === 'dev' + ? ensureDevOnly(target.manifest, name, range) || target.changed + : ensurePeerDev(target.manifest, name, range) || target.changed + } + + if (pkg.dynamic) { + const productionNames = new Set([ + ...Object.keys(section(target.manifest, 'dependencies')), + ...Object.keys(section(target.manifest, 'peerDependencies')), + ]) + for (const name of productionNames) { + if (expected.has(name)) continue + const range = preferredRange( + target.manifest, + name, + staticInputs.has(name) ? 'dev' : 'peer-dev', + inferredRanges, + ) + if (range === undefined) continue + if (staticInputs.has(name)) { + target.changed = ensureDevOnly(target.manifest, name, range) || target.changed + } else if (section(target.manifest, 'dependencies')[name] !== undefined && isInternalDsh(name)) { + target.changed = ensurePeerDev(target.manifest, name, range) || target.changed + } + } + } + + for (const [name, range] of Object.entries(section(target.manifest, 'peerDependencies'))) { + target.changed = setDependency(target.manifest, 'devDependencies', name, range) || target.changed + } + target.changed = deleteEmptySections(target.manifest) || target.changed + } + + const changed = [...documents.values()].filter(target => target.changed).sort((left, right) => + left.path.localeCompare(right.path)) + for (const target of changed) { + writeFileSync(resolve(root, target.path), JSON.stringify(target.manifest, null, 2) + '\n') + } + return changed.map(target => target.path) +} + +function normalizeClientArray( + client: Record, + field: 'external' | 'inject', + remove: (value: string) => boolean, +): boolean { + const value = client[field] + if (!Array.isArray(value) || value.some(entry => typeof entry !== 'string')) return false + const seen = new Set() + const normalized = value.filter((entry: string) => { + if (entry === '' || seen.has(entry) || remove(entry)) return false + seen.add(entry) + return true + }) + if (normalized.length === value.length && normalized.every((entry, index) => entry === value[index])) return false + if (normalized.length === 0) { + if (field === 'external') delete client.external + else delete client.inject + } else { + client[field] = normalized + } + return true +} + +function ensureDevOnly(manifest: Manifest, name: string, range: string): boolean { + let changed = deleteDependency(manifest, 'dependencies', name) + changed = deleteDependency(manifest, 'peerDependencies', name) || changed + return setDependency(manifest, 'devDependencies', name, range) || changed +} + +function ensureDependencyOnly(manifest: Manifest, name: string, range: string): boolean { + let changed = deleteDependency(manifest, 'peerDependencies', name) + changed = deleteDependency(manifest, 'devDependencies', name) || changed + return setDependency(manifest, 'dependencies', name, range) || changed +} + +function ensurePeerDev(manifest: Manifest, name: string, range: string): boolean { + let changed = deleteDependency(manifest, 'dependencies', name) + changed = setDependency(manifest, 'peerDependencies', name, range) || changed + return setDependency(manifest, 'devDependencies', name, range) || changed +} + +function setDependency(manifest: Manifest, field: DependencySection, name: string, range: string): boolean { + const dependencies = mutableSection(manifest, field) + if (dependencies[name] === range) return false + dependencies[name] = range + return true +} + +function deleteDependency(manifest: Manifest, field: DependencySection, name: string): boolean { + const dependencies = section(manifest, field) + if (dependencies[name] === undefined) return false + manifest[field] = Object.fromEntries(Object.entries(dependencies).filter(([key]) => key !== name)) + return true +} + +function deleteEmptySections(manifest: Manifest): boolean { + let changed = false + for (const field of ['dependencies', 'peerDependencies', 'devDependencies'] as const) { + if (manifest[field] === undefined || Object.keys(section(manifest, field)).length > 0) continue + if (field === 'dependencies') delete manifest.dependencies + else if (field === 'peerDependencies') delete manifest.peerDependencies + else delete manifest.devDependencies + changed = true + } + return changed +} + +function preferredRange( + manifest: Manifest, + name: string, + kind: ExpectedRule['kind'], + inferred: ReadonlyMap>, +): string | undefined { + const order: readonly DependencySection[] = kind === 'dependency' + ? ['dependencies', 'devDependencies', 'peerDependencies'] + : kind === 'dev' + ? ['devDependencies', 'peerDependencies', 'dependencies'] + : ['peerDependencies', 'devDependencies', 'dependencies'] + for (const field of order) { + const range = section(manifest, field)[name] + if (range !== undefined) return range + } + if (isInternalDsh(name)) return 'workspace:^' + const candidates = inferred.get(name) + return candidates?.size === 1 ? [...candidates][0] : undefined +} + +function dependencyRangeCandidates(root: string): Map> { + const candidates = new Map>() + const paths = globSync([ + 'package.json', + ...MANIFEST_GLOBS, + 'website/package.json', + ], { cwd: root }).map(normalizePath) + for (const path of new Set(paths)) { + const manifest = JSON.parse(readFileSync(resolve(root, path), 'utf8')) as Manifest + for (const field of ['dependencies', 'peerDependencies', 'devDependencies'] as const) { + for (const [name, range] of Object.entries(section(manifest, field))) { + const ranges = candidates.get(name) ?? new Set() + ranges.add(range) + candidates.set(name, ranges) + } + } + } + return candidates +} + +function section(manifest: Manifest, field: DependencySection): Record { + return manifest[field] ?? {} +} + +function mutableSection(manifest: Manifest, field: DependencySection): Record { + const value = manifest[field] + if (value !== undefined) return value + const created: Record = {} + manifest[field] = created + return created +} + +function collectModeViolations(facts: ClientPackageFacts): string[] { + const violations: string[] = [] + for (const pkg of facts.packages) { + if (pkg.dynamic && pkg.staticLinked) { + violations.push( + pkg.manifest + ': ' + pkg.name + ' declares dsh.client and uses the staticLinked preset;' + + ' a client package must be dynamic or statically linked, not both', + ) + } else if (!pkg.dynamic && !pkg.staticLinked) { + violations.push( + pkg.manifest + ': ' + pkg.name + ' has no supported client package mode;' + + ' declare dsh.client or use the staticLinked preset', + ) + } + } + + const workspaceNames = new Set(facts.declarations.map(entry => entry.name)) + for (const specifier of facts.platformModules) { + const owner = packageNameOf(specifier) + if (!workspaceNames.has(owner) || owner === CORDIS || facts.staticLinkedPackages.has(owner)) continue + violations.push( + PLATFORM_SOURCE + ': seeded workspace module ' + JSON.stringify(specifier) + + ' belongs to ' + owner + ', whose build does not use the staticLinked preset', + ) + } + + const rows = rowNames(facts.declarations) + for (const specifier of facts.preloadedExternals) { + if (rowPackageOf(specifier, rows) === undefined) { + violations.push( + PLATFORM_SOURCE + ': parser-preloaded external ' + JSON.stringify(specifier) + + ' has no dynamic dsh.client row', + ) + } + if (!facts.parserPreloadIds.includes(stripClientSuffix(specifier))) { + violations.push( + PLATFORM_SOURCE + ': parser-preloaded external ' + JSON.stringify(specifier) + + ' has no matching PARSER_PRELOAD_IDS row in ' + PARSER_PRELOAD_SOURCE, + ) + } + } + return violations +} + +interface ExpectedRule { + readonly kind: 'dependency' | 'dev' | 'peer-dev' + readonly origins: Set +} + +function collectDependencyViolations(facts: ClientPackageFacts): string[] { + const violations: string[] = [] + const staticInputs = new Set([ + ...facts.staticLinkedPackages, + ...facts.platformModules.map(packageNameOf), + ]) + staticInputs.delete(CORDIS) + + for (const pkg of [...facts.packages].sort((left, right) => left.manifest.localeCompare(right.manifest))) { + const expected = expectedSections(pkg, staticInputs) + for (const [name, rule] of [...expected].sort(([left], [right]) => left.localeCompare(right))) { + const actual = declaredSections(pkg, name) + if (rule.kind === 'dependency') { + if (actual.length === 1 && actual[0] === 'dependencies') continue + violations.push( + pkg.manifest + ': ' + name + ' (' + describeOrigins(rule.origins) + ') is a runtime import' + + ' retained by a statically linked artifact; declare it only in dependencies, found ' + + describeSections(actual), + ) + continue + } + if (rule.kind === 'dev') { + if (actual.length === 1 && actual[0] === 'devDependencies') continue + violations.push( + pkg.manifest + ': ' + name + ' (' + describeOrigins(rule.origins) + ') is a static client input;' + + ' declare it only in devDependencies, found ' + describeSections(actual), + ) + continue + } + + const peerRange = pkg.peerDependencies[name] + const devRange = pkg.devDependencies[name] + if (actual.length === 2 + && actual.includes('peerDependencies') + && actual.includes('devDependencies') + && peerRange === devRange) continue + violations.push( + pkg.manifest + ': ' + name + ' (' + describeOrigins(rule.origins) + ')' + + ' is a peer-installed DSH relationship; declare it in peerDependencies and devDependencies' + + ' with matching ranges, not dependencies; found ' + describeSections(actual) + + describeRangeMismatch(peerRange, devRange), + ) + } + + for (const [name, peerRange] of Object.entries(pkg.peerDependencies).sort(([left], [right]) => left.localeCompare(right))) { + if (expected.has(name)) continue + const devRange = pkg.devDependencies[name] + if (devRange === peerRange) continue + violations.push( + pkg.manifest + ': peerDependencies.' + name + ' is ' + peerRange + ', so devDependencies.' + name + + ' must use the same range; found ' + (devRange ?? 'no declaration'), + ) + } + + if (!pkg.dynamic) continue + for (const section of ['dependencies', 'peerDependencies'] as const) { + for (const name of Object.keys(pkg[section]).sort()) { + if (expected.has(name)) continue + if (staticInputs.has(name)) { + violations.push( + pkg.manifest + ': dynamic package declares static input ' + name + ' in ' + section + ';' + + ' move it to devDependencies or delete the stale declaration', + ) + } else if (section === 'dependencies' && isInternalDsh(name)) { + violations.push( + pkg.manifest + ': dynamic package declares ' + name + ' in dependencies;' + + ' dynamic DSH relationships are peer plus dev, and static client inputs are dev-only', + ) + } + } + } + } + return violations +} + +function expectedSections(pkg: ClientPackage, staticInputs: ReadonlySet): Map { + const expected = new Map([ + [CORDIS, { kind: 'peer-dev', origins: new Set(['client package baseline']) }], + ]) + if (!pkg.dynamic) { + if (pkg.name === CLIENT_WEB) return expected + for (const [name, locations] of Object.entries(pkg.runtimeSourceUses)) { + if (name === pkg.name || name === CORDIS || isInternalDsh(name)) continue + expected.set(name, { kind: 'dependency', origins: new Set(locations) }) + } + return expected + } + + const add = (name: string, origin: string): void => { + if (name === pkg.name) return + const kind = staticInputs.has(name) ? 'dev' : isInternalDsh(name) ? 'peer-dev' : undefined + if (kind === undefined) return + const current = expected.get(name) + if (current !== undefined) current.origins.add(origin) + else expected.set(name, { kind, origins: new Set([origin]) }) + } + for (const [name, locations] of Object.entries(pkg.sourceUses)) { + for (const location of locations) add(name, location) + } + for (const name of pkg.inject) add(name, 'dsh.client.inject') + return expected +} + +interface ModuleEdge { + readonly from: string + readonly to: string + readonly specifier: string +} + +function collectModuleViolations(facts: ClientPackageFacts): string[] { + const violations: string[] = [] + const baseline = new Set([...facts.platformModules, ...facts.preloadedExternals]) + const rows = rowNames(facts.declarations) + const byName = new Map(facts.declarations.map(entry => [entry.name, entry])) + const edges: ModuleEdge[] = [] + + for (const pkg of facts.declarations.filter(entry => entry.dynamic)) { + for (const field of ['external', 'inject'] as const) { + const seen = new Set() + for (const value of pkg[field]) { + if (value === '') violations.push(pkg.manifest + ': dsh.client.' + field + ' contains an empty value') + else if (seen.has(value)) { + violations.push(pkg.manifest + ': dsh.client.' + field + ' lists ' + JSON.stringify(value) + ' twice') + } + seen.add(value) + } + } + + for (const specifier of new Set(pkg.external)) { + if (specifier === '') continue + if (baseline.has(specifier)) { + violations.push( + pkg.manifest + ': dsh.client.external repeats baseline module ' + JSON.stringify(specifier) + + '; remove the explicit declaration', + ) + continue + } + const supplier = rowPackageOf(specifier, rows) + if (supplier === pkg.name) { + violations.push(pkg.manifest + ': dsh.client.external names its own row ' + JSON.stringify(specifier)) + } else if (supplier !== undefined) { + edges.push({ from: pkg.name, to: supplier, specifier }) + } else { + const owner = stripClientSuffix(specifier) + violations.push( + pkg.manifest + ': dsh.client.external ' + JSON.stringify(specifier) + ' has no supplier;' + + (byName.has(owner) + ? ' workspace package ' + owner + + ' declares no dynamic dsh.client row and the shell does not seed this specifier' + : ' no dynamic row or PLATFORM_MODULES entry answers it'), + ) + } + } + } + + violations.push(...collectModuleCycles(edges, byName)) + return violations +} + +function collectModuleCycles( + edges: readonly ModuleEdge[], + byName: ReadonlyMap, +): string[] { + const outgoing = new Map() + for (const edge of [...edges].sort((left, right) => left.specifier.localeCompare(right.specifier))) { + outgoing.set(edge.from, [...outgoing.get(edge.from) ?? [], edge]) + } + const finished = new Set() + const onPath = new Set() + const path: ModuleEdge[] = [] + const reported = new Map() + + const walk = (name: string): void => { + onPath.add(name) + for (const edge of outgoing.get(name) ?? []) { + if (onPath.has(edge.to)) { + const start = path.findIndex(entry => entry.from === edge.to) + const cycle = start === -1 ? [edge] : [...path.slice(start), edge] + const key = cycleKey(cycle) + if (!reported.has(key)) reported.set(key, formatCycle(cycle, byName)) + } else if (!finished.has(edge.to)) { + path.push(edge) + walk(edge.to) + path.pop() + } + } + onPath.delete(name) + finished.add(name) + } + + for (const name of [...outgoing.keys()].sort()) { + if (!finished.has(name)) walk(name) + } + return [...reported.values()] +} + +function cycleKey(cycle: readonly ModuleEdge[]): string { + const labels = cycle.map(edge => edge.from + ' ' + edge.specifier) + const first = [...labels].sort()[0] + const offset = first === undefined ? 0 : labels.indexOf(first) + return [...labels.slice(offset), ...labels.slice(0, offset)].join(' -> ') +} + +function formatCycle( + cycle: readonly ModuleEdge[], + byName: ReadonlyMap, +): string { + const entry = cycle[0] + const chain = cycle.map(edge => edge.from + ' --(' + edge.specifier + ')-->').join(' ') + const manifest = entry === undefined ? 'packages/client' : byName.get(entry.from)?.manifest ?? entry.from + return manifest + ': synchronous dsh.client.external cycle: ' + chain + ' ' + (entry?.from ?? '') +} + +interface Manifest { + name?: unknown + dsh?: unknown + dependencies?: Record + peerDependencies?: Record + devDependencies?: Record +} + +function readDeclaration( + root: string, + manifestPath: string, + malformed: string[], +): ClientDeclaration | undefined { + const manifest = JSON.parse(readFileSync(resolve(root, manifestPath), 'utf8')) as Manifest + if (typeof manifest.name !== 'string') return undefined + const dsh = isRecord(manifest.dsh) ? manifest.dsh : undefined + const rawClient = dsh?.client + if (rawClient === undefined) { + return { name: manifest.name, manifest: manifestPath, dynamic: false, external: [], inject: [] } + } + if (!isRecord(rawClient)) { + malformed.push(manifestPath + ': ' + manifest.name + ' dsh.client must be an object') + return { name: manifest.name, manifest: manifestPath, dynamic: false, external: [], inject: [] } + } + return { + name: manifest.name, + manifest: manifestPath, + dynamic: true, + external: stringArray(rawClient.external, manifest.name, manifestPath, 'external', malformed), + inject: stringArray(rawClient.inject, manifest.name, manifestPath, 'inject', malformed), + } +} + +function stringArray( + value: unknown, + packageName: string, + manifestPath: string, + field: string, + malformed: string[], +): readonly string[] { + if (value === undefined) return [] + if (!Array.isArray(value) || value.some(entry => typeof entry !== 'string')) { + malformed.push(manifestPath + ': ' + packageName + ' dsh.client.' + field + ' must be a string array') + return [] + } + return value as string[] +} + +async function readStaticLinkedRoster(root: string): Promise> { + const presetUrl = pathToFileURL(resolve(import.meta.dirname, '..', STATIC_PRESET_SOURCE)).href + const preset = await import(presetUrl) as { isStaticLinkedConfig?: unknown } + if (typeof preset.isStaticLinkedConfig !== 'function') { + throw new Error(GATE + ': ' + STATIC_PRESET_SOURCE + ' exports no isStaticLinkedConfig') + } + const predicate = preset.isStaticLinkedConfig as (configs: readonly unknown[]) => boolean + const roster = new Set() + for (const configPath of globSync(CONFIG_GLOB, { cwd: root }).map(normalizePath).sort()) { + const loaded = await import(pathToFileURL(resolve(root, configPath)).href) as { default?: unknown } + if (typeof loaded.default !== 'function') continue + const configs = (loaded.default as (input: { env: Record }) => unknown)({ + env: { DSH_BUILD_FACE: 'client' }, + }) + if (!Array.isArray(configs) || !predicate(configs)) continue + const manifest = JSON.parse( + readFileSync(resolve(root, configPath.replace(/tsdown\.config\.ts$/, 'package.json')), 'utf8'), + ) as Manifest + if (typeof manifest.name === 'string') roster.add(manifest.name) + } + return roster +} + +function unwrapExpression(expression: ts.Expression): ts.Expression { + let current = expression + while (ts.isAsExpression(current) || ts.isSatisfiesExpression(current) || ts.isParenthesizedExpression(current)) { + current = current.expression + } + return current +} + +function readStringLiteralArray(root: string, sourcePath: string, name: string): string[] { + const path = resolve(root, sourcePath) + const source = ts.createSourceFile(path, readFileSync(path, 'utf8'), ts.ScriptTarget.Latest, false, ts.ScriptKind.TS) + const constants = new Map() + for (const statement of source.statements) { + if (!ts.isVariableStatement(statement)) continue + for (const declaration of statement.declarationList.declarations) { + if (!ts.isIdentifier(declaration.name) || declaration.initializer === undefined) continue + const initializer = unwrapExpression(declaration.initializer) + if (ts.isStringLiteral(initializer)) constants.set(declaration.name.text, initializer.text) + } + } + for (const statement of source.statements) { + if (!ts.isVariableStatement(statement)) continue + for (const declaration of statement.declarationList.declarations) { + if (!ts.isIdentifier(declaration.name) || declaration.name.text !== name) continue + const expression = declaration.initializer === undefined ? undefined : unwrapExpression(declaration.initializer) + if (expression === undefined || !ts.isArrayLiteralExpression(expression)) { + throw new Error(GATE + ': ' + name + ' in ' + sourcePath + ' must be an array literal') + } + return expression.elements.map((element) => { + const value = unwrapExpression(element) + if (ts.isStringLiteral(value)) return value.text + if (ts.isIdentifier(value) && constants.has(value.text)) return constants.get(value.text) as string + throw new Error(GATE + ': ' + name + ' in ' + sourcePath + ' must contain only string constants') + }) + } + } + throw new Error(GATE + ': ' + sourcePath + ' declares no ' + name) +} + +async function readFacts(root: string): Promise { + const { declarations, malformed } = readClientDeclarations(root) + const byManifest = new Map(declarations.map(entry => [entry.manifest, entry])) + const staticLinkedPackages = await readStaticLinkedRoster(root) + const project = new TypeScriptProject(root, 'client') + const packages: ClientPackage[] = [] + + for (const manifestPath of globSync(CLIENT_MANIFEST_GLOB, { cwd: root }).map(normalizePath).sort()) { + const declaration = byManifest.get(manifestPath) + if (declaration === undefined) throw new Error(GATE + ': no declaration facts for ' + manifestPath) + const manifest = JSON.parse(readFileSync(resolve(root, manifestPath), 'utf8')) as Manifest + if (typeof manifest.name !== 'string') throw new Error(GATE + ': ' + manifestPath + ' has no package name') + const sourceUses = new Map>() + const runtimeSourceUses = new Map>() + const packageDirectory = dirname(manifestPath) + const sourcePrefix = packageDirectory + '/src/' + for (const sourceFile of project.sourceFiles()) { + if (sourceFile.isDeclarationFile) continue + const file = project.relativePath(sourceFile) + if (!file.startsWith(sourcePrefix)) continue + for (const name of collectSourceFilePackageUses(sourceFile, false)) { + const locations = sourceUses.get(name) ?? new Set() + locations.add(file) + sourceUses.set(name, locations) + } + for (const name of collectSourceFilePackageUses(sourceFile, true)) { + const locations = runtimeSourceUses.get(name) ?? new Set() + locations.add(file) + runtimeSourceUses.set(name, locations) + } + } + packages.push({ + ...declaration, + staticLinked: staticLinkedPackages.has(declaration.name), + sourceUses: Object.fromEntries( + [...sourceUses].sort(([left], [right]) => left.localeCompare(right)) + .map(([name, locations]) => [name, [...locations].sort()]), + ), + runtimeSourceUses: Object.fromEntries( + [...runtimeSourceUses].sort(([left], [right]) => left.localeCompare(right)) + .map(([name, locations]) => [name, [...locations].sort()]), + ), + dependencies: manifest.dependencies ?? {}, + peerDependencies: manifest.peerDependencies ?? {}, + devDependencies: manifest.devDependencies ?? {}, + }) + } + + return { + packages, + declarations, + staticLinkedPackages, + platformModules: readStringLiteralArray(root, PLATFORM_SOURCE, 'PLATFORM_MODULES'), + preloadedExternals: readStringLiteralArray(root, PLATFORM_SOURCE, 'PRELOADED_CLIENT_EXTERNALS'), + parserPreloadIds: readStringLiteralArray(root, PARSER_PRELOAD_SOURCE, 'PARSER_PRELOAD_IDS'), + malformed, + } +} + +function packageNameOf(specifier: string): string { + const segments = specifier.split('/') + return segments.slice(0, specifier.startsWith('@') ? 2 : 1).join('/') +} + +function stripClientSuffix(specifier: string): string { + return specifier.endsWith('/client') ? specifier.slice(0, -'/client'.length) : specifier +} + +function rowNames(declarations: readonly ClientDeclaration[]): Set { + return new Set(declarations.filter(entry => entry.dynamic).map(entry => entry.name)) +} + +function rowPackageOf(specifier: string, rows: ReadonlySet): string | undefined { + if (rows.has(specifier)) return specifier + const stripped = stripClientSuffix(specifier) + return rows.has(stripped) ? stripped : undefined +} + +function declaredSections(pkg: ClientPackage, name: string): string[] { + return (['dependencies', 'peerDependencies', 'devDependencies'] as const) + .filter(section => pkg[section][name] !== undefined) +} + +function describeSections(sections: readonly string[]): string { + return sections.length === 0 ? 'no dependency declaration' : sections.join(' + ') +} + +function describeRangeMismatch(peer: string | undefined, dev: string | undefined): string { + if (peer === undefined || dev === undefined || peer === dev) return '' + return ' (peer ' + peer + ', dev ' + dev + ')' +} + +function describeOrigins(origins: ReadonlySet): string { + const sorted = [...origins].sort() + const [first, second, ...rest] = sorted + if (first === undefined) return 'production use' + if (second === undefined) return first + return rest.length === 0 ? first + ', ' + second : first + ', ' + second + ', and ' + String(rest.length) + ' more' +} + +function isInternalDsh(name: string): boolean { + return name === CORDIS || name.startsWith(DSH_PREFIX) +} + +function isBareSpecifier(specifier: string): boolean { + return !specifier.startsWith('.') && !specifier.startsWith('/') && !specifier.startsWith('#') +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +function normalizePath(path: string): string { + return path.split(sep).join('/') +} + +async function main(): Promise { + const root = resolve(import.meta.dirname, '..') + let facts = await readFacts(root) + if (process.argv.includes('--fix')) { + const changed = fixClientPackageManifests(root, facts) + console.log( + changed.length === 0 + ? GATE + ': no mechanically fixable manifest changes.' + : GATE + ': fixed ' + String(changed.length) + ' manifest(s): ' + changed.join(', '), + ) + facts = await readFacts(root) + } + const violations = collectClientPackageViolations(facts) + if (violations.length > 0) { + console.error(GATE + ': ' + String(violations.length) + ' violation(s):') + for (const violation of violations) console.error(' ' + violation) + process.exit(1) + } + + const dynamic = facts.packages.filter(pkg => pkg.dynamic).length + const requests = facts.declarations.reduce((total, pkg) => total + pkg.external.length, 0) + console.log( + GATE + ': ' + String(facts.packages.length) + ' client packages (' + String(dynamic) + ' dynamic, ' + + String(facts.packages.length - dynamic) + ' statically linked) satisfy dependency and module-request rules; ' + + String(requests) + ' explicit external request(s).', + ) +} + +if (process.argv[1] !== undefined && import.meta.filename === resolve(process.argv[1])) { + await main() +} diff --git a/scripts/verify-config-source-ownership.spec.ts b/scripts/verify-config-source-ownership.spec.ts index 41026c5fe4..3a099156c2 100644 --- a/scripts/verify-config-source-ownership.spec.ts +++ b/scripts/verify-config-source-ownership.spec.ts @@ -14,7 +14,7 @@ describe('configuration source ownership gate', () => { it('rejects inline endpoints in shipped bundle patches', () => { const root = mkdtempSync(join(tmpdir(), 'dsh-config-source-ownership-')) roots.push(root) - const directory = join(root, 'packages/bundle/base') + const directory = join(root, 'packages/subagent/subagent-claude-code') mkdirSync(directory, { recursive: true }) writeFileSync( join(directory, 'cordis.patch.yml'), @@ -22,7 +22,7 @@ describe('configuration source ownership gate', () => { ) expect(collectConfigSourceOwnershipViolations(root)).toEqual([ - 'packages/bundle/base/cordis.patch.yml:2: inlines a credential or endpoint from the environment.' + 'packages/subagent/subagent-claude-code/cordis.patch.yml:2: inlines a credential or endpoint from the environment.' + ' The adapter resolves apiKeyEnv through ctx.credentials and the endpoint through the' + ' environment snapshot; inlining here bypasses both ladders.', ]) diff --git a/scripts/verify-config-source-ownership.ts b/scripts/verify-config-source-ownership.ts index 0684124215..c42e7cb793 100644 --- a/scripts/verify-config-source-ownership.ts +++ b/scripts/verify-config-source-ownership.ts @@ -14,7 +14,8 @@ const SHIPPED_CONFIG_GLOBS = [ 'apps/*/config/*.yml', 'examples/*/*.cordis.yml', 'examples/*/cordis.yml', - 'packages/bundle/*/cordis.patch.yml', + // Bundle identity comes from the package manifest, not the domain directory. + 'packages/*/*/cordis.patch.yml', // The Python runtime ships its own default composition inside the wheel. 'python/*/src/**/cordis.yml', ] diff --git a/scripts/verify-cordis-config.spec.ts b/scripts/verify-cordis-config.spec.ts index 6c1304e16a..f63031e46c 100644 --- a/scripts/verify-cordis-config.spec.ts +++ b/scripts/verify-cordis-config.spec.ts @@ -4,8 +4,15 @@ * metadata field must stay static, and a disabled expression must parse. */ +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' import { describe, expect, it } from 'vitest' -import { metadataExpressionErrors } from './verify-cordis-config.ts' +import { + bundleManifestPaths, + bundlePluginDependencyErrors, + metadataExpressionErrors, +} from './verify-cordis-config.ts' describe('verify-cordis-config metadata expressions', () => { it('accepts a disabled !!js expression', () => { @@ -37,3 +44,45 @@ describe('verify-cordis-config metadata expressions', () => { expect(problems.some(problem => problem.includes('[0].disabled: disabled expression does not parse'))).toBe(true) }) }) + +describe('workspace Bundle discovery and product dependency closures', () => { + it('discovers a Bundle outside packages/bundle from its manifest declaration', () => { + const fixture = mkdtempSync(join(tmpdir(), 'dsh-bundle-discovery-')) + try { + const bundleDir = join(fixture, 'packages/subagent/example') + const plainDir = join(fixture, 'packages/bundle/plain') + mkdirSync(bundleDir, { recursive: true }) + mkdirSync(plainDir, { recursive: true }) + writeFileSync(join(bundleDir, 'package.json'), JSON.stringify({ + name: '@deepseek-ai/dsh-subagent-example', + dsh: { bundle: { patch: './cordis.patch.yml' } }, + })) + writeFileSync(join(plainDir, 'package.json'), JSON.stringify({ + name: '@deepseek-ai/dsh-plain', + })) + + expect(bundleManifestPaths(fixture)).toEqual([ + 'packages/subagent/example/package.json', + ]) + } finally { + rmSync(fixture, { recursive: true, force: true }) + } + }) + + it('allows a Bundle to mount itself but rejects an undeclared plugin package', () => { + const manifestPath = 'packages/subagent/example/package.json' + const file = 'packages/subagent/example/cordis.patch.yml' + const manifest = { + name: '@deepseek-ai/dsh-subagent-example', + dependencies: {}, + } + const self = { file, name: '@deepseek-ai/dsh-subagent-example' } + expect(bundlePluginDependencyErrors(manifestPath, manifest, [self])).toEqual([]) + expect(bundlePluginDependencyErrors(manifestPath, manifest, [ + self, + { file, name: '@deepseek-ai/dsh-missing-plugin' }, + ])).toEqual([ + `${file}: @deepseek-ai/dsh-missing-plugin must be declared in ${manifestPath} dependencies`, + ]) + }) +}) diff --git a/scripts/verify-cordis-config.ts b/scripts/verify-cordis-config.ts index f2e0aef46c..24b06baede 100644 --- a/scripts/verify-cordis-config.ts +++ b/scripts/verify-cordis-config.ts @@ -12,20 +12,19 @@ import { globSync, readFileSync } from 'node:fs' import { dirname, relative, resolve } from 'node:path' -import * as yaml from 'js-yaml' +import { Script } from 'node:vm' import ts from 'typescript' import { cordisConfigFiles } from './cordis-config-files.ts' +import { isCordisGroupEntry, isJsExpr, loadCordisYaml } from './cordis-yaml.ts' -interface JsExpr { - __jsExpr: string -} - -interface PackageManifest { +export interface PackageManifest { name?: string dependencies?: Record + optionalDependencies?: Record + dsh?: { bundle?: { patch?: string } } } -interface PluginReference { +export interface PluginReference { file: string name: string } @@ -34,7 +33,9 @@ const root = resolve(import.meta.dirname, '..') // These example files are overlays consumed by the built dsh app, so their bare // specifiers resolve from apps/cli rather than the examples workspace. const appOverlayFiles = new Set([ + 'examples/web-github-review/tests/fixtures/real-cli/cordis.yml', 'examples/web-cordis/cordis.yml', + 'examples/web-github-review/cordis.yml', 'examples/web-schedule/cordis.yml', ...globSync('examples/mcp-memory/*.cordis.yml', { cwd: root }), ]) @@ -56,16 +57,6 @@ const CHOOSER_BACKEND_PACKAGES = [ '@deepseek-ai/dsh-client-ui-directory-picker-browse', '@deepseek-ai/dsh-client-ui-directory-picker-native', ] -const jsExprType = new yaml.Type('tag:yaml.org,2002:js', { - kind: 'scalar', - resolve: data => typeof data === 'string', - construct: (data: unknown): JsExpr => { - if (typeof data !== 'string') throw new TypeError('!!js requires a scalar string') - return { __jsExpr: data } - }, -}) -const schema = yaml.JSON_SCHEMA.extend(jsExprType) - const errors: string[] = [] const pluginReferences: PluginReference[] = [] @@ -73,7 +64,7 @@ if (import.meta.main) { const files = cordisConfigFiles(root) for (const file of files) { - const document: unknown = yaml.load(readFileSync(resolve(root, file), 'utf8'), { schema }) + const document = loadCordisYaml(readFileSync(resolve(root, file), 'utf8')) if (!isUnknownArray(document)) { errors.push(`${file}: root must be a Loader entry array`) continue @@ -172,7 +163,7 @@ function validatePresetPlaneSeparation(): string[] { /** Every entry of one config file, or an empty list when it is not an entry array. */ function loadEntries(file: string): unknown[] { - const document: unknown = yaml.load(readFileSync(resolve(root, file), 'utf8'), { schema }) + const document = loadCordisYaml(readFileSync(resolve(root, file), 'utf8')) return isUnknownArray(document) ? document : [] } @@ -204,7 +195,7 @@ function validateEntry(value: unknown, file: string, path: string): void { } recordPlugin(value, file) validateMetadata(value, file, path) - if ((value.group === true || value.name === '@deepseek-ai/cordis-plugin-group') && isUnknownArray(value.config)) { + if (isCordisGroupEntry(value)) { for (let index = 0; index < value.config.length; index++) { validateEntry(value.config[index], file, `${path}.config[${index}]`) } @@ -260,11 +251,14 @@ function validateExampleResolution(): string[] { function validateAppResolution(): string[] { const violations: string[] = [] + const bundleManifests = bundleManifestPaths() // App overlays (and any config left under apps/cli/config) resolve from the // dsh app's own dependency surface — the profile module fallback mirrors it. const appDependencies = { ...readManifest('apps/cli/package.json').dependencies, - // The fallback also links every bundle's own dependencies (healProfilesModuleFallback). + // The fallback also links every in-box bundle's own dependencies + // (healProfilesModuleFallback). Optional Profile bundles stay outside the + // app installation until that Profile installs them. ...Object.fromEntries(globSync('packages/bundle/*/package.json', { cwd: root }) .flatMap(file => Object.entries(readManifest(file).dependencies ?? {}))), } @@ -274,20 +268,50 @@ function validateAppResolution(): string[] { violations.push(...missingPluginDependencies(appReferences, appDependencies, 'apps/cli/package.json or a bundle manifest')) // Each bundle's patch rows must resolve from that bundle's own dependencies: // per-layer resolution anchors on the bundle package directory. - for (const manifestPath of globSync('packages/bundle/*/package.json', { cwd: root })) { + for (const manifestPath of bundleManifests) { const bundleDir = manifestPath.replace(/\/package\.json$/, '') const manifest = readManifest(manifestPath) - const references = pluginReferences.filter(reference => reference.file.startsWith(`${bundleDir}/`)) - violations.push(...missingPluginDependencies( - // A bundle may mount its own package (the web-app runtime row). - references.filter(reference => packageNameFromSpecifier(reference.name) !== manifest.name), - manifest.dependencies ?? {}, - manifestPath, - )) + const patch = manifest.dsh?.bundle?.patch + if (typeof patch !== 'string') continue + const patchFile = relative(root, resolve(root, bundleDir, patch)).replaceAll('\\', '/') + const references = pluginReferences.filter(reference => reference.file === patchFile) + violations.push(...bundlePluginDependencyErrors(manifestPath, manifest, references)) } return violations } +/** + * Discover workspace Bundle packages from their manifest declaration. + * @param repoRoot Repository root to scan. + * @returns Sorted slash-normalized repository-relative package manifest paths. + */ +export function bundleManifestPaths(repoRoot: string = root): string[] { + return globSync('packages/*/*/package.json', { cwd: repoRoot }) + .filter(path => typeof readManifest(path, repoRoot).dsh?.bundle?.patch === 'string') + .map(path => path.replaceAll('\\', '/')) + .sort() +} + +/** + * Validate plugin packages referenced by one Bundle patch. + * @param manifestPath Repository-relative Bundle manifest path. + * @param manifest Parsed Bundle manifest. + * @param references Plugin rows read from the Bundle package directory. + * @returns Missing production dependency diagnostics. + */ +export function bundlePluginDependencyErrors( + manifestPath: string, + manifest: PackageManifest, + references: readonly PluginReference[], +): string[] { + return missingPluginDependencies( + // A Bundle may mount its own package (for example, its provider or runtime row). + references.filter(reference => packageNameFromSpecifier(reference.name) !== manifest.name), + manifest.dependencies ?? {}, + manifestPath, + ) +} + /** * Every configured specifier of a local workspace package must resolve through * the tsconfig `paths` facade to a `.ts`/`.tsx` source file. The `dsh` source @@ -363,8 +387,8 @@ function missingPluginDependencies( : `${[...locations].join(', ')}: ${packageName} must be declared in ${manifestPath} dependencies`) } -function readManifest(path: string): PackageManifest { - return JSON.parse(readFileSync(resolve(root, path), 'utf8')) as PackageManifest +function readManifest(path: string, repoRoot: string = root): PackageManifest { + return JSON.parse(readFileSync(resolve(repoRoot, path), 'utf8')) as PackageManifest } function localPackageDirectories(): Map { @@ -461,9 +485,8 @@ export function metadataExpressionErrors(entry: Record, path: s */ function disabledExpressionProblem(expression: string): string | undefined { try { - // Compilation only — the constructor never executes the body. - // oxlint-disable-next-line typescript/no-implied-eval - new Function(`return (${expression})`) + // Compilation only — constructing a Script does not execute its source. + new Script(`(${expression})`) return undefined } catch (error) { const detail = error instanceof Error ? error.message : String(error) @@ -484,10 +507,6 @@ function collectExpressionPaths(value: unknown, path: string, output: string[]): for (const [key, child] of Object.entries(value)) collectExpressionPaths(child, `${path}.${key}`, output) } -function isJsExpr(value: unknown): value is JsExpr { - return isRecord(value) && typeof value.__jsExpr === 'string' -} - function isRecord(value: unknown): value is Record { return value !== null && typeof value === 'object' } diff --git a/scripts/verify-doc-site-fragments.spec.ts b/scripts/verify-doc-site-fragments.spec.ts index d06d46f53b..a85914cb54 100644 --- a/scripts/verify-doc-site-fragments.spec.ts +++ b/scripts/verify-doc-site-fragments.spec.ts @@ -4,7 +4,7 @@ import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, describe, expect, it } from 'vitest' -import { inspectSiteFragments } from './verify-doc-site-fragments.ts' +import { inspectSiteFragments, missingSiteFiles } from './verify-doc-site-fragments.ts' const roots: string[] = [] @@ -85,3 +85,13 @@ describe('inspectSiteFragments', () => { ]) }) }) + +describe('missingSiteFiles', () => { + it('reports the expected files a build did not emit', () => { + const root = fixture() + writeFileSync(join(root, 'guide/start.md'), '# Ready\n') + + expect(missingSiteFiles(root, ['guide/start.md', 'guide/absent.md', 'llms.txt'])) + .toEqual(['guide/absent.md', 'llms.txt']) + }) +}) diff --git a/scripts/verify-doc-site-fragments.ts b/scripts/verify-doc-site-fragments.ts index c086dedccb..5e1ba882cc 100644 --- a/scripts/verify-doc-site-fragments.ts +++ b/scripts/verify-doc-site-fragments.ts @@ -1,15 +1,17 @@ /** - * Verify fragment links against the HTML emitted by VitePress. Markdown and - * VitePress use different heading-slug algorithms, so source-link validation - * alone cannot prove that a published fragment exists. + * Verify fragment links against the HTML emitted by VitePress, and that the + * build carries the raw-Markdown twin of every route plus llms.txt. Markdown + * and VitePress use different heading-slug algorithms, so source-link + * validation alone cannot prove that a published fragment exists. * * This runs as part of `docs:build` and can also run directly after a build * with `tsx scripts/verify-doc-site-fragments.ts`. */ -import { globSync, readFileSync } from 'node:fs' +import { existsSync, globSync, readFileSync } from 'node:fs' import { resolve, sep } from 'node:path' import { JSDOM } from 'jsdom' +import { rawMarkdownFiles } from './project-doc-site.ts' const root = resolve(import.meta.dirname, '..') @@ -138,18 +140,40 @@ export function inspectSiteFragments(distRoot: string): SiteFragmentReport { return { checked, broken } } +/** + * Expected files a build did not emit. + * + * @param distRoot - Directory containing the built site. + * @param expected - Site-relative files the build must carry. + * @returns The absent files, in the given order. + */ +export function missingSiteFiles(distRoot: string, expected: readonly string[]): string[] { + return expected.filter(file => !existsSync(resolve(distRoot, file))) +} + function main(): number { const distRoot = resolve(root, 'website/.dist') const report = inspectSiteFragments(distRoot) - if (report.broken.length === 0) { - console.log(`verify-doc-site-fragments: ${report.checked} internal fragment reference(s) resolve.`) + const expected = rawMarkdownFiles() + const missing = missingSiteFiles(distRoot, [...expected, 'llms.txt']) + if (report.broken.length === 0 && missing.length === 0) { + console.log( + `verify-doc-site-fragments: ${report.checked} internal fragment reference(s) resolve;` + + ` ${expected.length} raw-Markdown file(s) and llms.txt emitted.`, + ) return 0 } - console.error(`verify-doc-site-fragments: ${report.broken.length} broken fragment reference(s):`) - for (const item of report.broken) { - const target = item.target === undefined ? 'target route was not built' : `${item.target} has no id ${JSON.stringify(item.fragment)}` - console.error(` ${item.source}: ${JSON.stringify(item.href)} (${target})`) + if (report.broken.length > 0) { + console.error(`verify-doc-site-fragments: ${report.broken.length} broken fragment reference(s):`) + for (const item of report.broken) { + const target = item.target === undefined ? 'target route was not built' : `${item.target} has no id ${JSON.stringify(item.fragment)}` + console.error(` ${item.source}: ${JSON.stringify(item.href)} (${target})`) + } + } + if (missing.length > 0) { + console.error(`verify-doc-site-fragments: ${missing.length} expected raw-Markdown file(s) missing from the build:`) + for (const file of missing) console.error(` ${file}`) } return 1 } diff --git a/scripts/verify-export-jsdoc.ts b/scripts/verify-export-jsdoc.ts index f1efd361bb..ae92d07ac1 100644 --- a/scripts/verify-export-jsdoc.ts +++ b/scripts/verify-export-jsdoc.ts @@ -22,37 +22,28 @@ const PROTOCOL_EXPORTS = new Set(['Config', 'inject', 'name', 'reusable', 'apply /** Per-file walk state threaded through the scope recursion. */ interface Walk { - /** Repo-relative path of the file being walked. */ rel: string - /** The parsed source file. */ sf: ts.SourceFile - /** Raw file text (rawJsDoc reads comment ranges out of it). */ text: string - /** The program's checker, consulted only for heritage-member lookups. */ checker: ts.TypeChecker - /** The aggregate violation list, appended in place. */ violations: string[] } -/** True when a statement carries the `export` modifier. */ function isExported(stmt: ts.Statement): boolean { return ts.canHaveModifiers(stmt) && (ts.getModifiers(stmt)?.some(m => m.kind === ts.SyntaxKind.ExportKeyword) ?? false) } -/** True for a class member a consumer cannot reach: `private`/`protected`/`#name`. */ function isNonPublic(member: ts.ClassElement): boolean { const mods = ts.canHaveModifiers(member) ? ts.getModifiers(member) : undefined return (mods?.some(m => m.kind === ts.SyntaxKind.PrivateKeyword || m.kind === ts.SyntaxKind.ProtectedKeyword) ?? false) || ('name' in member && ts.isPrivateIdentifier(member.name)) } -/** True when a class member carries the `static` modifier. */ function isStatic(member: ts.ClassElement): boolean { const mods = ts.canHaveModifiers(member) ? ts.getModifiers(member) : undefined return mods?.some(m => m.kind === ts.SyntaxKind.StaticKeyword) ?? false } -/** The `this`-receiver exemption every function-like check shares. */ function thisReceiver(p: ts.ParameterDeclaration): boolean { return ts.isIdentifier(p.name) && p.name.text === 'this' } @@ -611,7 +602,6 @@ function main(): void { process.exit(1) } -// Run only when invoked as a script, not when imported by a test. if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { main() } diff --git a/scripts/verify-md-links.ts b/scripts/verify-md-links.ts index 63ea5e531a..e8f0e94850 100644 --- a/scripts/verify-md-links.ts +++ b/scripts/verify-md-links.ts @@ -195,7 +195,6 @@ export function findViolations( return out } -// Run only when invoked as a script, not when imported by the spec. if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { // Archived notes remain valid link targets, but their historical outbound links are frozen. const files = uniqueRepoFiles(root, PATTERNS, isArchivedAgentNotePath) diff --git a/scripts/verify-optional-dependency-imports.spec.ts b/scripts/verify-optional-dependency-imports.spec.ts new file mode 100644 index 0000000000..3bb857050f --- /dev/null +++ b/scripts/verify-optional-dependency-imports.spec.ts @@ -0,0 +1,130 @@ +/** + * Tests for the optional-dependency load gate: which import and re-export forms + * survive emit, and therefore load a package the installed tree may not carry. + * + * The expectations here match what `tsc` emits with `verbatimModuleSyntax` off: + * `import type`, `import {}`, an inline `type` specifier, and a named binding + * that resolves to a type all disappear; a bare import, a value binding, and a + * star re-export remain. + */ + +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { afterAll, describe, expect, it } from 'vitest' +import { TypeScriptProject } from './ts-project.ts' +import { collectOptionalImportViolations } from './verify-optional-dependency-imports.ts' + +const FIXTURE: Record = { + 'tsconfig.host.json': JSON.stringify({ + compilerOptions: { + target: 'es2022', + module: 'esnext', + moduleResolution: 'bundler', + noEmit: true, + skipLibCheck: true, + types: [], + paths: { + '@f/opt': ['./packages/f/opt/src/index.ts'], + '@f/hard': ['./packages/f/hard/src/index.ts'], + }, + }, + include: ['packages/**/*.ts'], + }), + + 'packages/f/opt/package.json': JSON.stringify({ name: '@f/opt', version: '0.0.1' }), + 'packages/f/opt/src/index.ts': [ + 'export interface Shape { a: number }', + 'export const runtimeValue = 1', + '', + ].join('\n'), + + 'packages/f/hard/package.json': JSON.stringify({ name: '@f/hard', version: '0.0.1' }), + 'packages/f/hard/src/index.ts': 'export const hardValue = 2\n', + + // The consumer allows @f/opt to be absent and requires @f/hard. + 'packages/f/consumer/package.json': JSON.stringify({ + name: '@f/consumer', + version: '0.0.1', + dependencies: { '@f/hard': '*' }, + peerDependencies: { '@f/opt': '*' }, + peerDependenciesMeta: { '@f/opt': { optional: true } }, + }), + + // Elided by the compiler, so each of these is allowed. + 'packages/f/consumer/src/allowed-type-only.ts': [ + "import type {} from '@f/opt'", + 'export const a = 1', + '', + ].join('\n'), + 'packages/f/consumer/src/allowed-empty.ts': [ + "import {} from '@f/opt'", + 'export const b = 1', + '', + ].join('\n'), + 'packages/f/consumer/src/allowed-inline-type.ts': [ + "import { type Shape } from '@f/opt'", + 'export const c: Shape = { a: 1 }', + '', + ].join('\n'), + 'packages/f/consumer/src/allowed-type-binding.ts': [ + "import { Shape } from '@f/opt'", + 'export const d: Shape = { a: 1 }', + '', + ].join('\n'), + 'packages/f/consumer/src/allowed-type-reexport.ts': [ + "export type { Shape } from '@f/opt'", + '', + ].join('\n'), + // A hard dependency may be loaded at module scope; only optional ones may not. + 'packages/f/consumer/src/allowed-hard-dependency.ts': [ + "import { hardValue } from '@f/hard'", + 'export const e = hardValue', + '', + ].join('\n'), + + // Kept by the compiler, so each of these loads a package that may be absent. + 'packages/f/consumer/src/rejected-bare.ts': [ + "import '@f/opt'", + 'export const f = 1', + '', + ].join('\n'), + 'packages/f/consumer/src/rejected-value.ts': [ + "import { runtimeValue } from '@f/opt'", + 'export const g = runtimeValue', + '', + ].join('\n'), + 'packages/f/consumer/src/rejected-star-reexport.ts': [ + "export * from '@f/opt'", + '', + ].join('\n'), +} + +const root = mkdtempSync(join(tmpdir(), 'optional-imports-')) +for (const [rel, content] of Object.entries(FIXTURE)) { + mkdirSync(dirname(join(root, rel)), { recursive: true }) + writeFileSync(join(root, rel), content) +} +const violations = collectOptionalImportViolations(new TypeScriptProject(root)) + +afterAll(() => { + rmSync(root, { recursive: true, force: true }) +}) + +describe('optional dependency loads', () => { + it('reports every form the compiler keeps, and nothing else', () => { + expect(violations.map(violation => violation.split(' loads ')[0])).toEqual([ + 'packages/f/consumer/src/rejected-bare.ts:1', + 'packages/f/consumer/src/rejected-star-reexport.ts:1', + 'packages/f/consumer/src/rejected-value.ts:1', + ]) + }) + + it('names the package, the declaration that made it optional, and the way out', () => { + expect(violations[0]).toBe( + 'packages/f/consumer/src/rejected-bare.ts:1 loads @f/opt at module scope,' + + ' declared optional in peerDependenciesMeta; import it as a type,' + + ' or restructure so module scope does not need it', + ) + }) +}) diff --git a/scripts/verify-optional-dependency-imports.ts b/scripts/verify-optional-dependency-imports.ts new file mode 100644 index 0000000000..11314f36e0 --- /dev/null +++ b/scripts/verify-optional-dependency-imports.ts @@ -0,0 +1,213 @@ +/** + * Reject a static value import of an optional dependency. + * + * A dependency declared in `optionalDependencies`, or as a peer carrying + * `peerDependenciesMeta..optional`, may be absent from an installed tree — + * that absence is what "optional" promises a consumer. A static import is + * evaluated when the importing module loads, so one absent package turns + * "this capability is unavailable" into a load failure for everything that + * reaches the importing module. + * + * The way out, in order: import it as a type, which emits nothing and is all + * that declaration merging needs; or restructure so nothing at module scope + * needs the package. A dynamic `import()` only moves the failure to first use, + * so it belongs to a caller that genuinely requires the package and handles its + * absence — it is a last resort, not the default answer, and reaching for it is + * a sign the dependency is not optional. + * + * Value-vs-type is decided against a bound Program rather than the import + * syntax, because `verbatimModuleSyntax` is off: a named import used only in + * type positions is elided and does not load anything. The decision is + * deliberately conservative in one direction — a value binding the compiler + * would elide because nothing references it in a value position is still + * reported, and the fix it asks for (`import type`, or dropping the binding) is + * what the published package wants regardless. Both compiler faces are scanned, + * and only files that ship — a published package's `src` — are subject. + */ + +import { existsSync, readFileSync } from 'node:fs' +import { resolve } from 'node:path' +import ts from 'typescript' +import { TypeScriptProject, type CompilerFace } from './ts-project.ts' + +const root = resolve(import.meta.dirname, '..') + +/** Directories whose `src` ships as a published package. */ +const PUBLISHED_SOURCE = /^(?:packages\/[^/]+\/[^/]+|apps\/[^/]+)\/src\// + +/** How a manifest marked a dependency optional, for the violation message. */ +type OptionalKind = 'optionalDependencies' | 'peerDependenciesMeta' + +/** + * The package name a module specifier resolves to. + * @param specifier - an import specifier, possibly a subpath. + * @returns The bare package name, keeping a leading scope. + */ +function packageOf(specifier: string): string { + const parts = specifier.split('/') + return specifier.startsWith('@') ? parts.slice(0, 2).join('/') : parts[0] ?? specifier +} + +/** + * Read a manifest field as a record. + * @param manifest - parsed manifest. + * @param field - field name. + * @returns The field value, or an empty record. + */ +function record(manifest: Record, field: string): Record { + const value = manifest[field] + if (value === null || typeof value !== 'object' || Array.isArray(value)) return {} + return value as Record +} + +/** + * The dependencies one manifest allows to be absent. + * @param manifest - parsed manifest. + * @returns Each optional package name and how it was marked. + */ +function optionalDependencies(manifest: Record): Map { + const optional = new Map() + for (const name of Object.keys(record(manifest, 'optionalDependencies'))) { + optional.set(name, 'optionalDependencies') + } + const peers = record(manifest, 'peerDependencies') + for (const [name, meta] of Object.entries(record(manifest, 'peerDependenciesMeta'))) { + if (meta === null || typeof meta !== 'object') continue + if ((meta as Record).optional !== true) continue + // A meta entry for an undeclared peer is check-workspace-constraints' business. + if (!(name in peers)) continue + optional.set(name, 'peerDependenciesMeta') + } + return optional +} + +/** One package directory's optional dependencies, resolved once per directory. */ +const optionalByDirectory = new Map>() + +/** + * The optional dependencies of the package owning a source file. + * @param projectRoot - root the relative path is resolved against. + * @param relativePath - repository-relative path of a source file. + * @returns That package's optional dependencies, empty when it declares none. + */ +function optionalFor(projectRoot: string, relativePath: string): Map { + const directory = resolve(projectRoot, relativePath.slice(0, relativePath.indexOf('/src/'))) + const cached = optionalByDirectory.get(directory) + if (cached !== undefined) return cached + const manifestPath = resolve(directory, 'package.json') + const parsed: unknown = existsSync(manifestPath) ? JSON.parse(readFileSync(manifestPath, 'utf8')) : {} + const manifest = parsed !== null && typeof parsed === 'object' && !Array.isArray(parsed) + ? parsed as Record + : {} + const optional = optionalDependencies(manifest) + optionalByDirectory.set(directory, optional) + return optional +} + +/** + * Whether one binding of an import or re-export names a value. + * @param name - the local binding name node. + * @param checker - the program's checker. + * @returns True when the binding carries value meaning, and on an unresolved + * symbol, so an unresolvable binding fails closed. + */ +function bindsValue(name: ts.Identifier | ts.StringLiteral, checker: ts.TypeChecker): boolean { + const symbol = checker.getSymbolAtLocation(name) + if (symbol === undefined) return true + const target = (symbol.flags & ts.SymbolFlags.Alias) === 0 ? symbol : checker.getAliasedSymbol(symbol) + return (target.flags & ts.SymbolFlags.Value) !== 0 +} + +/** + * Whether an import declaration loads its module at run time. + * @param declaration - the import declaration. + * @param checker - the program's checker. + * @returns True when the emitted module keeps the import. + */ +function importLoadsModule(declaration: ts.ImportDeclaration, checker: ts.TypeChecker): boolean { + const clause = declaration.importClause + // A bare `import 'x'` is kept for its side effects. + if (clause === undefined) return true + // Only the type phase erases the import. `import defer` still resolves and + // links the module, deferring evaluation alone, so an absent package fails + // exactly as it would without the modifier. + if (clause.phaseModifier === ts.SyntaxKind.TypeKeyword) return false + if (clause.name !== undefined) return true + const bindings = clause.namedBindings + if (bindings === undefined || ts.isNamespaceImport(bindings)) return true + return bindings.elements.some(element => !element.isTypeOnly && bindsValue(element.name, checker)) +} + +/** + * Whether a re-export loads its module at run time. + * @param declaration - the export declaration, which carries a module specifier. + * @param checker - the program's checker. + * @returns True when the emitted module keeps the re-export. + */ +function exportLoadsModule(declaration: ts.ExportDeclaration, checker: ts.TypeChecker): boolean { + if (declaration.isTypeOnly) return false + const clause = declaration.exportClause + // `export * from 'x'` re-exports whatever values the module has. + if (clause === undefined || ts.isNamespaceExport(clause)) return true + return clause.elements.some(element => !element.isTypeOnly && bindsValue(element.name, checker)) +} + +/** + * Collect every static value import of an optional dependency in one face. + * @param project - a bound repository project. + * @returns One message per violation, sorted by location. + */ +export function collectOptionalImportViolations(project: TypeScriptProject): string[] { + const checker = project.checker + const violations: string[] = [] + for (const sourceFile of project.sourceFiles()) { + if (sourceFile.isDeclarationFile) continue + const relativePath = project.relativePath(sourceFile) + if (!PUBLISHED_SOURCE.test(relativePath)) continue + const optional = optionalFor(project.projectRoot, relativePath) + if (optional.size === 0) continue + + for (const statement of sourceFile.statements) { + const isImport = ts.isImportDeclaration(statement) + if (!isImport && !ts.isExportDeclaration(statement)) continue + const specifierNode = statement.moduleSpecifier + if (specifierNode === undefined || !ts.isStringLiteral(specifierNode)) continue + const kind = optional.get(packageOf(specifierNode.text)) + if (kind === undefined) continue + const loads = isImport + ? importLoadsModule(statement, checker) + : exportLoadsModule(statement, checker) + if (!loads) continue + const { line } = sourceFile.getLineAndCharacterOfPosition(statement.getStart(sourceFile)) + violations.push( + `${relativePath}:${String(line + 1)} loads ${specifierNode.text} at module scope,` + + ` declared optional in ${kind}; import it as a type, or restructure so module scope does not need it`, + ) + } + } + return violations.sort((left, right) => left.localeCompare(right)) +} + +/** CLI entry: list every violation and exit 1, or confirm the invariant holds. */ +function main(): void { + const faces: readonly CompilerFace[] = ['host', 'client'] + const violations = new Set() + for (const face of faces) { + for (const violation of collectOptionalImportViolations(new TypeScriptProject(root, face))) { + violations.add(violation) + } + } + if (violations.size === 0) { + console.log('verify-optional-dependency-imports: no optional dependency is loaded at module scope.') + return + } + console.error(`verify-optional-dependency-imports: ${String(violations.size)} optional dependency load(s) at module scope:`) + for (const violation of [...violations].sort((left, right) => left.localeCompare(right))) { + console.error(` ${violation}`) + } + process.exit(1) +} + +if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { + main() +} diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index d38b4743c5..2cc5dcd2a8 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -51,8 +51,11 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/code-runtime/code-runtime': { kind: 'indirect', reason: 'The service interface delegates model rendering to Code Mode in dsh-tools.' }, 'packages/core/agent-tool-presentation': { kind: 'indirect', reason: 'The row only selects between the two projections dsh-tools owns; it registers no prompt, schema, or result of its own.' }, 'packages/code-runtime/code-runtime-worker-thread': { kind: 'indirect', reason: 'The worker backend delegates model rendering to Code Mode in dsh-tools.' }, + 'packages/code-runtime/code-runtime-python': { kind: 'indirect', reason: 'The CPython subprocess backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/client/ui-agent-preset': { kind: 'indirect', reason: 'Browser-side settings row; the preset it selects owns every model-facing effect.' }, + 'packages/util/crypto': { kind: 'indirect', reason: 'Pure identifier minting; the ids consumers mint with it never enter prompts as semantic content.' }, 'packages/core/agent-default-model': { kind: 'indirect', reason: 'The service supplies a ModelSelection; request assembly and adapters own the model-visible request.' }, + 'packages/llm/deepseek-llm-api-extensions': { kind: 'indirect', reason: 'The registry contributes model-hidden provider fields; dsh-llm-deepseek owns their wire placement.' }, 'packages/preset/agent-presets': { kind: 'indirect', reason: 'The mount installs a preset\'s own plugins, which own every model-facing registration it makes visible.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, @@ -60,22 +63,25 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/client/hmr': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, 'packages/client/modules': { kind: 'none', reason: 'Browser-side module-loading kernel machinery; registers nothing model-facing.' }, 'packages/test-support/client-runtime': { kind: 'none', reason: 'Browser-side test infrastructure (jsdom bench); registers nothing model-facing.' }, + 'packages/experimental/webworker-runtime': { kind: 'none', reason: 'Browser-side host runtime and Node-compatibility layer; the plugins it boots own every model-facing registration.' }, + 'packages/experimental/webworker-packer': { kind: 'none', reason: 'Build-time image writer; its output reaches a model only through the tree the worker then boots.' }, 'packages/client/ui-slots': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, 'packages/client/ui-attachment': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, 'packages/client/ui-primitives': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, - 'packages/client/web-react': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, - 'packages/client/schema-form': { kind: 'none', reason: 'Browser-side form-rendering library; registers nothing model-facing.' }, + 'packages/client/ui-renderer': { kind: 'none', reason: 'Browser-side render assembly; registers nothing model-facing.' }, 'packages/client/connection': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, 'packages/api/remotes': { kind: 'none', reason: 'The Remote BFF selects business methods and identity policy; selected services own any model-visible effect.' }, 'packages/client/runtime': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, 'packages/client/ui-layout': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, 'packages/client/ui-sidebar': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, + 'packages/client/ui-brand-official': { kind: 'none', reason: 'Browser-side presentation occupants; registers nothing model-facing.' }, 'packages/client/ui-conversation': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, 'packages/client/ui-message-feedback': { kind: 'none', reason: 'Browser-side controls over the message-feedback sidecar; ratings and notes never enter the Session log, model context, or telemetry.' }, 'packages/client/ui-tool': { kind: 'none', reason: 'Browser-side Tool presentation layer; renders logged calls without changing model context.' }, 'packages/client/ui-jobs': { kind: 'none', reason: 'Browser-side read-only projection of ctx.jobs records; dsh-tool-jobs owns the model-facing behavior.' }, 'packages/client/ui-workflow-run': { kind: 'none', reason: 'Browser-side UI plugin layer; renders durable workflow records without changing model context.' }, 'packages/client/ui-input-trigger': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, + 'packages/client/ui-reference': { kind: 'indirect', reason: 'Browser-side reference selection delegates file guidance and session snapshot preparation to Host-owned providers.' }, 'packages/client/ui-commands': { kind: 'indirect', reason: 'The dispatch paths trigger the host command.execute RPC; each command handler\'s host package owns any model-visible effect.' }, 'packages/client/ui-model-selection': { kind: 'indirect', reason: 'Selection routes session.selectModel; the Host snapshots the selection at the next prompt-assembly boundary and owns the model-visible effect.' }, 'packages/client/ui-goal': { kind: 'indirect', reason: 'The strip verbs route goal.* mutations; the host GoalService owns the model-visible goal/change context message.' }, @@ -96,6 +102,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/client/locale': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, 'packages/client/web': { kind: 'none', reason: 'Browser-side UI plugin layer; registers nothing model-facing.' }, 'packages/examples/agent-spine-demo': { kind: 'indirect', reason: 'The bundle only mounts model-facing child plugins.' }, + 'packages/context/file-reference': { kind: 'indirect', reason: 'The discovery seam and grammar delegate model guidance to the composed provider.' }, 'packages/fs/fs': { kind: 'indirect', reason: 'The service interface delegates model rendering to dsh-tool-fs.' }, 'packages/e2b/fs-e2b': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, 'packages/fs/fs-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, @@ -106,6 +113,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/host/directory-picker-browse': { kind: 'none', reason: 'The GUI-host picking backend registers nothing model-facing.' }, 'packages/host/directory-picker-native': { kind: 'none', reason: 'The GUI-host picking backend registers nothing model-facing.' }, 'packages/host/webserver': { kind: 'none', reason: 'The HTTP carrier bridges browser and API handler and registers nothing model-facing.' }, + 'packages/webhook/webhook-github': { kind: 'indirect', reason: 'The adapter delegates model-visible text to matching rules and dsh-webhook.' }, 'packages/host/frontend-static': { kind: 'none', reason: 'The SPA dist server answers browser asset requests and registers nothing model-facing.' }, 'packages/host/plugin-inventory': { kind: 'none', reason: 'Host-side read-only Loader projection; registers nothing model-facing.' }, 'packages/bundle/base': { kind: 'indirect', reason: 'The bundle is a patch-list carrier; each inserted row\'s package owns its model-facing behavior.' }, @@ -130,6 +138,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/settings/settings-file': { kind: 'indirect', reason: 'The file provider stores and publishes namespace sections; consumers of ctx.settings own any model-facing behavior.' }, 'packages/credentials/credentials': { kind: 'indirect', reason: 'The seam resolves credential references; the consuming adapter owns every model-facing use a value authorizes.' }, 'packages/credentials/credentials-local': { kind: 'indirect', reason: 'The file/environment provider stores credential values; consumers of ctx.credentials own any model-facing behavior.' }, + 'packages/credentials/authorization': { kind: 'none', reason: 'A configuration-time conversation with a human; no flow, notice, or prompt reaches a model request.' }, 'packages/util/atomic-write': { kind: 'none', reason: 'Pure filesystem write primitive; registers nothing model-facing.' }, 'packages/session/session-telemetry': { kind: 'none', reason: 'The seam observes the session stream and hands redacted copies outward; it registers nothing model-facing.' }, 'packages/session/session-telemetry-otel': { kind: 'none', reason: 'The backend forwards seam records into the OTel SDK pipeline and registers nothing model-facing.' }, @@ -150,10 +159,9 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/typert/generator': { kind: 'none', reason: 'The build-time generator runs outside any agent runtime and touches no model request.' }, 'packages/jobs/jobs': { kind: 'indirect', reason: 'Producer and controller plugins own all model rendering over the job registry.' }, 'packages/jobs/jobs-local': { kind: 'indirect', reason: 'The registry backend delegates model rendering to producer plugins and dsh-tool-jobs.' }, - 'packages/examples/acp-demo': { kind: 'indirect', reason: 'The app bundle delegates request composition to dsh-agent-spine-demo and dsh-acp.' }, 'packages/boot/app-boot': { kind: 'indirect', reason: 'Only the loaded plugin tree contributes model context.' }, 'packages/boot/cmdline': { kind: 'none', reason: 'Resolves the process command line before any session exists; configured rows own every model-visible consequence.' }, - 'packages/examples/jsonrpc-demo': { kind: 'indirect', reason: 'Only the externally configured plugin tree contributes model context.' }, + 'packages/sdk/python-runtime': { kind: 'indirect', reason: 'Only the externally configured plugin tree contributes model context.' }, 'packages/interaction/permission-presets': { kind: 'indirect', reason: 'The service writes mechanism events rendered by dsh-user-approval and dsh-tool-bash.' }, 'packages/interaction/user-questions': { kind: 'indirect', reason: 'Model-facing consumers render provider answers and seam errors.' }, 'packages/util/timeout': { kind: 'indirect', reason: 'Only timeout consumers render timeout outcomes.' }, diff --git a/scripts/verify-runtime-closure.spec.ts b/scripts/verify-runtime-closure.spec.ts new file mode 100644 index 0000000000..09a1b044ab --- /dev/null +++ b/scripts/verify-runtime-closure.spec.ts @@ -0,0 +1,165 @@ +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { verifyRuntimeClosure } from './verify-runtime-closure.ts' + +const roots: string[] = [] + +function fixture(files: Record>): string { + const root = mkdtempSync(join(tmpdir(), 'dsh-runtime-closure-')) + roots.push(root) + for (const [relative, value] of Object.entries(files)) { + const path = join(root, relative) + mkdirSync(dirname(path), { recursive: true }) + writeFileSync(path, typeof value === 'string' ? value : `${JSON.stringify(value, null, 2)}\n`) + } + return root +} + +const platforms = { + 'linux-x64': { tag: 'manylinux_2_28_x86_64', executable: 'runtime-linux-x64' }, + 'linux-arm64': { tag: 'manylinux_2_28_aarch64', executable: 'runtime-linux-arm64' }, + 'macos-arm64': { tag: 'macosx_14_0_arm64', executable: 'runtime-macos-arm64' }, +} + +function workspace(root: string, name: string, manifest: Record): void { + const packageName = name.replace('@scope/', '') + const path = join(root, 'packages/core', packageName, 'package.json') + mkdirSync(dirname(path), { recursive: true }) + writeFileSync(path, `${JSON.stringify({ name, ...manifest }, null, 2)}\n`) +} + +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }) +}) + +describe('verifyRuntimeClosure', () => { + it('requires only plugins active for a Linux or macOS target', async () => { + const root = fixture({ + 'python/sdk-runtime/package.json': { name: 'runtime', dependencies: { '@scope/shared': 'workspace:^' } }, + 'python/sdk-runtime/platforms.json': platforms, + 'apps/cli/config/agent-presets/standard/agent.cordis.yml': ` +- id: tools + name: cordis:group + group: true + config: + - id: shared + name: '@scope/shared' + - id: linux + name: '@scope/linux' + disabled: !!js process.platform !== 'linux' + - id: macos + name: '@scope/macos' + disabled: !!js process.platform !== 'darwin' +`, + }) + + const result = await verifyRuntimeClosure(root) + + expect(result.presetCount).toBe(1) + expect(result.failures).toEqual([ + 'standard preset -> @scope/linux (linux-arm64, linux-x64)', + 'standard preset -> @scope/macos (macos-arm64)', + ]) + }) + + it('treats an unsupported disabled expression as active on every target', async () => { + const root = fixture({ + 'python/sdk-runtime/package.json': { name: 'runtime', dependencies: {} }, + 'python/sdk-runtime/platforms.json': platforms, + 'apps/cli/config/agent-presets/standard/agent.cordis.yml': ` +- id: conditional + name: '@scope/conditional' + disabled: !!js process.env.DSH_DISABLE_CONDITIONAL === '1' +`, + }) + + const result = await verifyRuntimeClosure(root) + + expect(result.failures).toEqual([ + 'standard preset -> @scope/conditional (linux-arm64, linux-x64, macos-arm64)', + ]) + }) + + it('does not interpret an ordinary plugin array config as nested Loader entries', async () => { + const root = fixture({ + 'python/sdk-runtime/package.json': { name: 'runtime', dependencies: { '@scope/plugin': 'workspace:^' } }, + 'python/sdk-runtime/platforms.json': platforms, + 'apps/cli/config/agent-presets/standard/agent.cordis.yml': ` +- id: plugin + name: '@scope/plugin' + config: + - name: '@scope/config-value' +`, + }) + + const result = await verifyRuntimeClosure(root) + + expect(result.failures).toEqual([]) + }) + + it('requires preset plugins to be linked from the workspace', async () => { + const root = fixture({ + 'python/sdk-runtime/package.json': { name: 'runtime', dependencies: { '@scope/plugin': '1.2.3' } }, + 'python/sdk-runtime/platforms.json': platforms, + 'apps/cli/config/agent-presets/standard/agent.cordis.yml': ` +- id: plugin + name: '@scope/plugin' +`, + }) + + const result = await verifyRuntimeClosure(root) + + expect(result.failures).toEqual([ + 'standard preset -> @scope/plugin [runtime dependency is "1.2.3"; expected workspace:] (linux-arm64, linux-x64, macos-arm64)', + ]) + }) + + it('fails when no shipped preset is discovered', async () => { + const root = fixture({ + 'python/sdk-runtime/package.json': { name: 'runtime', dependencies: {} }, + 'python/sdk-runtime/platforms.json': platforms, + }) + + const result = await verifyRuntimeClosure(root) + + expect(result.presetCount).toBe(0) + expect(result.failures).toEqual([ + 'no agent presets matched apps/cli/config/agent-presets/*/agent.cordis.yml', + ]) + }) + + it('fails when the runtime platform manifest has no targets', async () => { + const root = fixture({ + 'python/sdk-runtime/package.json': { name: 'runtime', dependencies: {} }, + 'python/sdk-runtime/platforms.json': {}, + 'apps/cli/config/agent-presets/standard/agent.cordis.yml': '[]\n', + }) + + const result = await verifyRuntimeClosure(root) + + expect(result.failures).toEqual([ + 'python/sdk-runtime/platforms.json defines no runtime targets', + ]) + }) + + it('retains the required workspace-peer closure check', async () => { + const root = fixture({ + 'python/sdk-runtime/package.json': { name: 'runtime', dependencies: { '@scope/root': 'workspace:^' } }, + 'python/sdk-runtime/platforms.json': platforms, + 'apps/cli/config/agent-presets/minimal/agent.cordis.yml': '[]\n', + }) + workspace(root, '@scope/root', { + peerDependencies: { '@scope/required': 'workspace:^', '@scope/optional': 'workspace:^' }, + peerDependenciesMeta: { '@scope/optional': { optional: true } }, + }) + workspace(root, '@scope/required', {}) + workspace(root, '@scope/optional', {}) + + const result = await verifyRuntimeClosure(root) + + expect(result.workspacePackageCount).toBe(1) + expect(result.failures).toEqual(['runtime -> @scope/root -> @scope/required']) + }) +}) diff --git a/scripts/verify-runtime-closure.ts b/scripts/verify-runtime-closure.ts index c87d562f59..927bae8db5 100644 --- a/scripts/verify-runtime-closure.ts +++ b/scripts/verify-runtime-closure.ts @@ -1,12 +1,14 @@ /** - * Verify that the executable deploy manifest supplies every required workspace - * peer in its dependency graph. With auto peer installation disabled, a missing - * root peer can otherwise fail only when Cordis loads the packaged plugin. + * Verify that the executable deploy manifest supplies every plugin referenced + * by a shipped agent preset and every required workspace peer in its dependency + * graph. With auto peer installation disabled, either omission can otherwise + * fail only when Cordis loads the packaged plugin. */ import { globSync } from 'node:fs' import { readFile } from 'node:fs/promises' -import { resolve } from 'node:path' +import { basename, dirname, resolve } from 'node:path' import { parseArgs } from 'node:util' +import { isCordisGroupEntry, loadCordisYaml } from './cordis-yaml.ts' interface PackageManifest { name?: string @@ -21,58 +23,181 @@ interface WorkspacePackage { manifest: PackageManifest } -const root = resolve(import.meta.dirname, '..') -const { values } = parseArgs({ - args: process.argv.slice(2), - options: { manifest: { type: 'string' } }, -}) -const runtimeManifestPath = resolve(root, values.manifest ?? 'python/sdk-runtime/package.json') -const runtimeManifest = await loadManifest(runtimeManifestPath) -const runtimeName = runtimeManifest.name ?? 'python/sdk-runtime' -const workspace = await loadWorkspacePackages() -const runtimeDependencies = runtimeManifest.dependencies ?? {} -const parents = new Map() -const queue: string[] = [] - -for (const dependency of Object.keys(runtimeDependencies).sort()) { - if (!workspace.has(dependency)) continue - parents.set(dependency, undefined) - queue.push(dependency) +interface RuntimePlatform { + tag: string + executable: string } -const failures: string[] = [] -for (let index = 0; index < queue.length; index += 1) { - const packageName = queue[index] - if (packageName === undefined) continue - const current = workspace.get(packageName) - if (current === undefined) continue - const peers = current.manifest.peerDependencies ?? {} - const peerMeta = current.manifest.peerDependenciesMeta ?? {} - for (const peer of Object.keys(peers).sort()) { - if (!workspace.has(peer) || peerMeta[peer]?.optional === true) continue - if (runtimeDependencies[peer]?.startsWith('workspace:') === true) continue - failures.push(`${formatChain(runtimeName, packageName, parents)} -> ${peer}`) - } - const dependencies = { - ...current.manifest.dependencies, - ...current.manifest.optionalDependencies, - } - for (const dependency of Object.keys(dependencies).sort()) { - if (!workspace.has(dependency) || parents.has(dependency)) continue - parents.set(dependency, packageName) +type RuntimePlatformManifest = Record + +const AGENT_PRESET_GLOB = 'apps/cli/config/agent-presets/*/agent.cordis.yml' + +export interface RuntimeClosureResult { + failures: string[] + presetCount: number + workspacePackageCount: number +} + +/** + * Check that the runtime manifest contains every shipped-preset plugin and workspace peer. + * @param root repository root containing the runtime manifest and shipped presets. + * @param manifestPath runtime manifest path relative to {@link root}. + * @returns the discovered preset count, reachable workspace package count, and violations. + */ +export async function verifyRuntimeClosure( + root: string, + manifestPath = 'python/sdk-runtime/package.json', +): Promise { + const runtimeManifest = await loadManifest(resolve(root, manifestPath)) + const runtimeName = runtimeManifest.name ?? manifestPath + const workspace = await loadWorkspacePackages(root) + const runtimeDependencies = runtimeManifest.dependencies ?? {} + const platforms = await loadJson(resolve(root, 'python/sdk-runtime/platforms.json')) + const presetPaths = globSync(AGENT_PRESET_GLOB, { cwd: root }).sort() + const targets = Object.keys(platforms).sort() + const parents = new Map() + const queue: string[] = [] + + for (const dependency of Object.keys(runtimeDependencies).sort()) { + if (!workspace.has(dependency)) continue + parents.set(dependency, undefined) queue.push(dependency) } + + const failures: string[] = [] + if (presetPaths.length === 0) failures.push(`no agent presets matched ${AGENT_PRESET_GLOB}`) + if (targets.length === 0) failures.push('python/sdk-runtime/platforms.json defines no runtime targets') + failures.push(...await missingPresetPlugins(root, runtimeDependencies, presetPaths, targets)) + for (let index = 0; index < queue.length; index += 1) { + const packageName = queue[index] + if (packageName === undefined) continue + const current = workspace.get(packageName) + if (current === undefined) continue + const peers = current.manifest.peerDependencies ?? {} + const peerMeta = current.manifest.peerDependenciesMeta ?? {} + for (const peer of Object.keys(peers).sort()) { + if (!workspace.has(peer) || peerMeta[peer]?.optional === true) continue + if (runtimeDependencies[peer]?.startsWith('workspace:') === true) continue + failures.push(`${formatChain(runtimeName, packageName, parents)} -> ${peer}`) + } + const dependencies = { + ...current.manifest.dependencies, + ...current.manifest.optionalDependencies, + } + for (const dependency of Object.keys(dependencies).sort()) { + if (!workspace.has(dependency) || parents.has(dependency)) continue + parents.set(dependency, packageName) + queue.push(dependency) + } + } + + return { + failures, + presetCount: presetPaths.length, + workspacePackageCount: queue.length, + } } -if (failures.length > 0) { - console.error('verify-runtime-closure: required workspace peers are missing from python/sdk-runtime dependencies:') - for (const failure of failures) console.error(` ${failure}`) - process.exit(1) +if (import.meta.main) { + const root = resolve(import.meta.dirname, '..') + const { values } = parseArgs({ + args: process.argv.slice(2), + options: { manifest: { type: 'string' } }, + }) + const result = await verifyRuntimeClosure(root, values.manifest) + if (result.failures.length > 0) { + console.error('verify-runtime-closure: preset plugins or required workspace peers are missing from python/sdk-runtime dependencies:') + for (const failure of result.failures) console.error(` ${failure}`) + process.exitCode = 1 + } else { + console.log( + `verify-runtime-closure: ${result.presetCount} agent presets and ${result.workspacePackageCount} workspace packages form a closed runtime dependency graph.`, + ) + } } -console.log(`verify-runtime-closure: ${queue.length} workspace packages form a closed runtime dependency graph.`) +async function missingPresetPlugins( + root: string, + runtimeDependencies: Readonly>, + presetPaths: readonly string[], + targets: readonly string[], +): Promise { + const missing = new Map>() + const failures: string[] = [] + for (const presetPath of presetPaths) { + const document = loadCordisYaml(await readFile(resolve(root, presetPath), 'utf8')) + if (!Array.isArray(document)) { + failures.push(`${presetPath}: preset root must be a Loader entry array`) + continue + } + for (const target of targets) { + const processPlatform = processPlatformForTarget(target) + for (const plugin of activeBarePluginPackages(document, processPlatform)) { + const version = runtimeDependencies[plugin] + if (version?.startsWith('workspace:') === true) continue + const preset = basename(dirname(presetPath)) + const declaration = version === undefined + ? '' + : ` [runtime dependency is ${JSON.stringify(version)}; expected workspace:]` + const key = `${preset} preset -> ${plugin}${declaration}` + const targets = missing.get(key) ?? new Set() + targets.add(target) + missing.set(key, targets) + } + } + } + failures.push(...[...missing.entries()].map(([chain, targets]) => + `${chain} (${[...targets].sort().join(', ')})`)) + return failures +} -async function loadWorkspacePackages(): Promise> { +function activeBarePluginPackages(entries: unknown[], processPlatform: string): Set { + const packages = new Set() + const visit = (value: unknown, parentDisabled: boolean): void => { + if (!isRecord(value)) return + const disabled = parentDisabled || disabledOnPlatform(value.disabled, processPlatform) + if (disabled) return + if (typeof value.name === 'string') { + const packageName = barePackageName(value.name) + if (packageName !== undefined) packages.add(packageName) + } + if (isCordisGroupEntry(value)) { + for (const child of value.config) visit(child, disabled) + } + } + for (const entry of entries) visit(entry, false) + return packages +} + +function disabledOnPlatform(value: unknown, processPlatform: string): boolean { + if (typeof value === 'boolean') return value + if (!isRecord(value) || typeof value.__jsExpr !== 'string') return false + const match = /^process\.platform\s*(===|!==)\s*(['"])(win32|linux|darwin)\2$/.exec(value.__jsExpr.trim()) + if (match === null) return false + const [, operator, , expected] = match + return operator === '===' ? processPlatform === expected : processPlatform !== expected +} + +function processPlatformForTarget(target: string): string { + if (target.startsWith('linux-')) return 'linux' + if (target.startsWith('macos-')) return 'darwin' + throw new Error(`verify-runtime-closure: unsupported runtime target ${JSON.stringify(target)}`) +} + +function barePackageName(specifier: string): string | undefined { + if (specifier.startsWith('.') || specifier.startsWith('/') || specifier.includes(':')) return undefined + const parts = specifier.split('/') + if (specifier.startsWith('@')) { + return parts.length >= 2 ? `${parts[0]}/${parts[1]}` : undefined + } + return parts[0] || undefined +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +async function loadWorkspacePackages(root: string): Promise> { const paths = globSync(['packages/*/*/package.json', 'vendor/*/package.json'], { cwd: root }) .sort() .map(relative => resolve(root, relative)) @@ -85,7 +210,11 @@ async function loadWorkspacePackages(): Promise> { } async function loadManifest(path: string): Promise { - return JSON.parse(await readFile(path, 'utf8')) as PackageManifest + return loadJson(path) +} + +async function loadJson(path: string): Promise { + return JSON.parse(await readFile(path, 'utf8')) as T } function formatChain( diff --git a/scripts/verify-translation-pairing.ts b/scripts/verify-translation-pairing.ts index 9bfc1a1d73..5b2b4052a4 100644 --- a/scripts/verify-translation-pairing.ts +++ b/scripts/verify-translation-pairing.ts @@ -10,9 +10,14 @@ * See `docs/i18n/README.md` for the owning contract. */ -import { existsSync, globSync, readFileSync, writeFileSync } from 'node:fs' +import { existsSync, globSync, readFileSync, statSync, writeFileSync } from 'node:fs' import { basename, join, resolve, sep } from 'node:path' -import { gitBlobHash, readGitIndexBlob, storeGitBlob } from './translation-pairing-git.ts' +import { + gitBlobHash, + gitIndexPaths, + readGitIndexBlob, + storeGitBlob, +} from './translation-pairing-git.ts' import { parseTranslationPairingRecord, renderTranslationPairingRecord, @@ -20,17 +25,23 @@ import { } from './translation-pairing-record.ts' import { languageSwitcherTargets, - linksTo, parseTranslationMarkdown, parseTranslationPairingCliArgs, parseTranslationPairingManifest, partitionGeneratedRegions, requiresSourceLanguageSwitcher, + isTranslationPairingManifestExcluded, isTranslationScopeFile, TRANSLATION_SCOPE_GLOB_EXCLUDES, + translationPairSourcePredicate, translationStructureDiff, translationStructureSignature, } from './translation-pairing.ts' +import { + hasLanguageSwitcher, + normalizeTranslationMarkdownLinks, + translationLinkLocaleViolations, +} from './translation-links.ts' const root = resolve(import.meta.dirname, '..') let request: ReturnType @@ -43,6 +54,7 @@ try { const listMode = request.mode === 'list' const writeMode = request.mode === 'write' const indexMode = request.input === 'index' +const indexFiles = indexMode ? gitIndexPaths(root) : undefined const contentCache = new Map() @@ -50,15 +62,17 @@ const contentCache = new Map() function readRepositoryFile(file: string): Buffer | undefined { if (contentCache.has(file)) return contentCache.get(file) const content = indexMode - ? readGitIndexBlob(root, file)?.content - : existsSync(join(root, file)) ? readFileSync(join(root, file)) : undefined + ? indexFiles?.has(file) ? readGitIndexBlob(root, file)?.content : undefined + : existsSync(join(root, file)) && statSync(join(root, file)).isFile() + ? readFileSync(join(root, file)) + : undefined contentCache.set(file, content) return content } /** Whether one path exists in the selected content plane. */ function repositoryFileExists(file: string): boolean { - return readRepositoryFile(file) !== undefined + return indexMode ? indexFiles?.has(file) === true : readRepositoryFile(file) !== undefined } /** Discover source Markdown and pairing sidecars before applying the corpus predicate. */ @@ -74,6 +88,7 @@ if (manifestContent === undefined) { throw new Error('scripts/translation-pairing.manifest.json is missing from the selected content plane') } const manifest = parseTranslationPairingManifest(manifestContent.toString('utf8')) +const isTranslationPairSource = translationPairSourcePredicate(manifest) /** * An excluded entry ending in `/` excludes the whole directory. The trailing @@ -82,7 +97,7 @@ const manifest = parseTranslationPairingManifest(manifestContent.toString('utf8' * manifest must keep their trailing slash. */ function isExcluded(file: string): boolean { - return manifest.excluded.some(entry => (entry.endsWith('/') ? file.startsWith(entry) : file === entry)) + return isTranslationPairingManifestExcluded(file, manifest) } // Enumerate the scope once: the whole corpus, or exactly the named pairs' @@ -230,40 +245,83 @@ for (const source of [...pairAnchors].sort()) { continue } - // Generated regions are language-invariant: the exact same generator output - // (markers included) must appear in both sides, in the same order. The - // structural signature below compares the region content again as part of - // the whole document; this dedicated check exists to name the divergence - // precisely and to reject a region grammar violation on either side. + const sourceText = sourceContent.toString('utf8') + const zhText = zhContent.toString('utf8') + const sourceSwitcherTargets = languageSwitcherTargets(source) + const zhSwitcherTargets = languageSwitcherTargets(zh) + for (const violation of [ + ...translationLinkLocaleViolations(sourceText, { + repoRoot: root, + sourcePath: source, + isTranslationPairSource, + repositoryFileExists, + }, zhSwitcherTargets), + ...translationLinkLocaleViolations(zhText, { + repoRoot: root, + sourcePath: zh, + isTranslationPairSource, + repositoryFileExists, + }, sourceSwitcherTargets), + ]) { + errors.push(`${violation.sourcePath}:${violation.line}: link target ${JSON.stringify(violation.url)} uses the wrong locale; expected ${JSON.stringify(violation.expectedUrl)}`) + state.set(source, 'out-of-sync') + } + + // Generated regions must remain byte-identical after paired document paths + // are normalized to one semantic target. The structural signature below + // compares their contents again as part of the whole document; this named + // check rejects any prose, ordering, code, marker, or non-locale URL drift. let sourceRegions: { regions: string[]; stripped: string } let zhRegions: { regions: string[]; stripped: string } try { - sourceRegions = partitionGeneratedRegions(sourceContent.toString('utf8')) - zhRegions = partitionGeneratedRegions(zhContent.toString('utf8')) + sourceRegions = partitionGeneratedRegions(sourceText) + zhRegions = partitionGeneratedRegions(zhText) } catch (error) { errors.push(`${source} ↔ ${zh}: ${error instanceof Error ? error.message : String(error)}`) state.set(source, 'out-of-sync') continue } - if (sourceRegions.regions.length !== zhRegions.regions.length - || sourceRegions.regions.some((region, index) => region !== zhRegions.regions[index])) { - errors.push(`${source} ↔ ${zh}: generated regions differ between the pair — regenerate (the generator writes both sides byte-identically)`) + const normalizedSourceRegions = sourceRegions.regions.map(region => normalizeTranslationMarkdownLinks(region, { + repoRoot: root, + sourcePath: source, + isTranslationPairSource, + repositoryFileExists, + })) + const normalizedZhRegions = zhRegions.regions.map(region => normalizeTranslationMarkdownLinks(region, { + repoRoot: root, + sourcePath: zh, + isTranslationPairSource, + repositoryFileExists, + })) + if (normalizedSourceRegions.length !== normalizedZhRegions.length + || normalizedSourceRegions.some((region, index) => region !== normalizedZhRegions[index])) { + errors.push(`${source} ↔ ${zh}: generated regions differ beyond paired-document locale paths — regenerate both sides`) state.set(source, 'out-of-sync') } - const sourceTree = parseTranslationMarkdown(sourceContent.toString('utf8')) - const zhTree = parseTranslationMarkdown(zhContent.toString('utf8')) - const sourceSwitcherTargets = languageSwitcherTargets(source) - const zhSwitcherTargets = languageSwitcherTargets(zh) - if (!linksTo(zhTree, sourceSwitcherTargets)) { + const sourceTree = parseTranslationMarkdown(sourceText) + const zhTree = parseTranslationMarkdown(zhText) + if (!hasLanguageSwitcher(zhTree, zhText, sourceSwitcherTargets)) { errors.push(`${zh}: missing language switcher — no link to ${basename(source)}`) } - if (requiresSourceLanguageSwitcher(source) && !linksTo(sourceTree, zhSwitcherTargets)) { + if (requiresSourceLanguageSwitcher(source) && !hasLanguageSwitcher(sourceTree, sourceText, zhSwitcherTargets)) { errors.push(`${source}: missing language switcher — no link back to ${basename(zh)}`) } for (const divergence of translationStructureDiff( - translationStructureSignature(sourceTree, zhSwitcherTargets), - translationStructureSignature(zhTree, sourceSwitcherTargets), + translationStructureSignature(sourceTree, zhSwitcherTargets, { + repoRoot: root, + sourcePath: source, + isTranslationPairSource, + repositoryFileExists, + markdown: sourceText, + }), + translationStructureSignature(zhTree, sourceSwitcherTargets, { + repoRoot: root, + sourcePath: zh, + isTranslationPairSource, + repositoryFileExists, + markdown: zhText, + }), )) { errors.push(`${source} ↔ ${zh}: ${divergence}`) } diff --git a/scripts/wine-windows-gates.sh b/scripts/wine-windows-gates.sh index 5e46b75f91..f9f04faea0 100755 --- a/scripts/wine-windows-gates.sh +++ b/scripts/wine-windows-gates.sh @@ -243,7 +243,7 @@ grep -q '^smoke: win32 x64' "$scratch/logs/smoke.log" || { echo 'wine-windows-ga # Host face before compiling and bundling the Client face. # Both statuses are captured so one failure cannot hide the other's result. build_gate() { - wine_node "$scratch/logs/host-tsc.log" "$tsc_js" -b tsconfig.host.json --pretty false || return $? + wine_node "$scratch/logs/host-tsc.log" --max-old-space-size=4096 "$tsc_js" -b tsconfig.host.json --pretty false || return $? wine_node "$scratch/logs/host-tsdown.log" "$tsdown_js" --env.DSH_BUILD_FACE host || return $? wine_node "$scratch/logs/client-tsc.log" "$tsc_js" -b tsconfig.client.json --pretty false || return $? wine_node "$scratch/logs/client-tsdown.log" "$tsdown_js" --env.DSH_BUILD_FACE client diff --git a/tsconfig.base.client.json b/tsconfig.base.client.json index a6efecbe2d..9222c69f26 100644 --- a/tsconfig.base.client.json +++ b/tsconfig.base.client.json @@ -6,6 +6,7 @@ "compilerOptions": { "jsx": "react-jsx", "lib": ["ES2024", "DOM", "DOM.Iterable"], - "types": [] + "typeRoots": ["./scripts/types", "./node_modules/@types"], + "types": ["client-build-environment"] } } diff --git a/tsconfig.base.json b/tsconfig.base.json index 16d69db6db..b92fdb61e9 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -53,6 +53,7 @@ "@deepseek-ai/dsh-session/invariant": ["./packages/core/session/src/invariant.ts"], "@deepseek-ai/dsh-typert-registry/types": ["./packages/typert/registry/src/types.ts"], "@deepseek-ai/dsh-typert-generator": ["./packages/typert/generator/src/index.ts"], + "@deepseek-ai/dsh-attachment/types": ["./packages/attachment/attachment/src/types.ts"], "@deepseek-ai/dsh-session/types": ["./packages/core/session/src/types.ts"], "@deepseek-ai/dsh-session/surface": ["./packages/core/session/src/surface.ts"], "@deepseek-ai/dsh-session-projection/types": ["./packages/session/session-projection/src/types.ts"], @@ -74,6 +75,7 @@ "@deepseek-ai/dsh-goal/client": ["./packages/goal/goal/src/client.ts"], "@deepseek-ai/dsh-llm/types": ["./packages/llm/llm/src/types.ts"], "@deepseek-ai/dsh-llm/brand": ["./packages/llm/llm/src/brand.ts"], + "@deepseek-ai/dsh-deepseek-llm-api-extensions/types": ["./packages/llm/deepseek-llm-api-extensions/src/types.ts"], "@deepseek-ai/dsh-llm-retry/types": ["./packages/llm/llm-retry/src/types.ts"], "@deepseek-ai/dsh-workflow/types": ["./packages/workflow/workflow/src/types.ts"], "@deepseek-ai/dsh-tool-workflow/types": ["./packages/workflow/tool-workflow/src/types.ts"], @@ -82,11 +84,15 @@ "@deepseek-ai/dsh-commands/types": ["./packages/interaction/commands/src/types.ts"], "@deepseek-ai/dsh-jobs/brand": ["./packages/jobs/jobs/src/brand.ts"], "@deepseek-ai/dsh-credentials/types": ["./packages/credentials/credentials/src/types.ts"], + "@deepseek-ai/dsh-authorization/types": ["./packages/credentials/authorization/src/types.ts"], "@deepseek-ai/dsh-settings/types": ["./packages/settings/settings/src/types.ts"], "@deepseek-ai/dsh-api-remotes/types": ["./packages/api/remotes/src/types.ts"], "@deepseek-ai/dsh-api-remotes/invariant": ["./packages/api/remotes/src/invariant.ts"], "@deepseek-ai/dsh-compaction/checkpoint": ["./packages/compaction/compaction/src/checkpoint.ts"], "@deepseek-ai/dsh-compaction/types": ["./packages/compaction/compaction/src/types.ts"], + "@deepseek-ai/dsh-file-reference/grammar": ["./packages/context/file-reference/src/grammar.ts"], + "@deepseek-ai/dsh-file-reference/types": ["./packages/context/file-reference/src/types.ts"], + "@deepseek-ai/dsh-session-reference/types": ["./packages/context/session-reference/src/types.ts"], "@deepseek-ai/dsh-tools/presentation": ["./packages/core/tools/src/presentation.ts"], "@deepseek-ai/dsh-tools/types": ["./packages/core/tools/src/types.ts"], "@deepseek-ai/dsh-tool-subagent-control/list-agents": ["./packages/subagent/tool-subagent-control/src/list-agents.ts"], @@ -97,6 +103,11 @@ "@deepseek-ai/dsh-agent/invariant": ["./packages/core/agent/src/invariant.ts"], "@deepseek-ai/dsh-scope/invariant": ["./packages/core/scope/src/invariant.ts"], "@deepseek-ai/dsh-agent-loop/invariant": ["./packages/core/agent-loop/src/invariant.ts"], + "@deepseek-ai/dsh-experimental-agent-team/invariant": ["./packages/experimental/agent-team/src/invariant.ts"], + "@deepseek-ai/dsh-experimental-tool-agent-team/invariant": ["./packages/experimental/tool-agent-team/src/invariant.ts"], + "@deepseek-ai/dsh-experimental-webworker-runtime/invariant": ["./packages/experimental/webworker-runtime/src/invariant.ts"], + "@deepseek-ai/dsh-experimental-webworker-packer/invariant": ["./packages/experimental/webworker-packer/src/invariant.ts"], + "@deepseek-ai/dsh-util-crypto/invariant": ["./packages/util/crypto/src/invariant.ts"], "@deepseek-ai/dsh-*/invariant": [ "./packages/core/*/src/invariant.ts", "./packages/prompt/*/src/invariant.ts", @@ -117,7 +128,9 @@ "./packages/preset/*/src/invariant.ts", "./packages/subagent/*/src/invariant.ts", "./packages/jobs/*/src/invariant.ts", + "./packages/experimental/*/src/invariant.ts", "./packages/workflow/*/src/invariant.ts", + "./packages/webhook/*/src/invariant.ts", "./packages/web/*/src/invariant.ts", "./packages/attachment/*/src/invariant.ts", "./packages/spill/*/src/invariant.ts", @@ -174,9 +187,9 @@ "@deepseek-ai/dsh-client-ui-slots": ["./packages/client/ui-slots/src"], "@deepseek-ai/dsh-client-ui-attachment": ["./packages/client/ui-attachment/src"], "@deepseek-ai/dsh-client-ui-primitives": ["./packages/client/ui-primitives/src"], - "@deepseek-ai/dsh-client-schema-form": ["./packages/client/schema-form/src"], - "@deepseek-ai/dsh-client-schema-form/invariant": ["./packages/client/schema-form/src/invariant.ts"], - "@deepseek-ai/dsh-client-web-react": ["./packages/client/web-react/src"], + "@deepseek-ai/dsh-client-ui-renderer": ["./packages/client/ui-renderer/src"], + "@deepseek-ai/dsh-client-ui-renderer/client": ["./packages/client/ui-renderer/src/client"], + "@deepseek-ai/dsh-client-ui-renderer/invariant": ["./packages/client/ui-renderer/src/invariant.ts"], "@deepseek-ai/dsh-client-connection": ["./packages/client/connection/src"], "@deepseek-ai/dsh-api-remotes": ["./packages/api/remotes/src"], "@deepseek-ai/dsh-api-remotes/client": ["./packages/api/remotes/src/client/index.ts"], @@ -194,11 +207,13 @@ "@deepseek-ai/dsh-client-test-runtime/invariant": ["./packages/test-support/client-runtime/src/invariant.ts"], "@deepseek-ai/dsh-client-ui-layout": ["./packages/client/ui-layout/src"], "@deepseek-ai/dsh-client-ui-sidebar": ["./packages/client/ui-sidebar/src"], + "@deepseek-ai/dsh-client-ui-brand-official": ["./packages/client/ui-brand-official/src"], "@deepseek-ai/dsh-client-ui-conversation": ["./packages/client/ui-conversation/src"], "@deepseek-ai/dsh-client-ui-tool": ["./packages/client/ui-tool/src"], "@deepseek-ai/dsh-client-ui-deliverables": ["./packages/client/ui-deliverables/src"], "@deepseek-ai/dsh-client-ui-workflow-run": ["./packages/client/ui-workflow-run/src"], "@deepseek-ai/dsh-client-ui-input-trigger": ["./packages/client/ui-input-trigger/src"], + "@deepseek-ai/dsh-client-ui-reference": ["./packages/client/ui-reference/src"], "@deepseek-ai/dsh-client-ui-commands": ["./packages/client/ui-commands/src"], "@deepseek-ai/dsh-client-ui-model-selection": ["./packages/client/ui-model-selection/src"], "@deepseek-ai/dsh-client-ui-goal": ["./packages/client/ui-goal/src"], @@ -225,6 +240,11 @@ "@deepseek-ai/dsh-sdk-client": ["./packages/sdk/client/src"], "@deepseek-ai/dsh-sdk-protocol": ["./packages/sdk/protocol/src"], "@deepseek-ai/dsh-sdk-jsonrpc-server": ["./packages/sdk/server/src"], + "@deepseek-ai/dsh-experimental-agent-team": ["./packages/experimental/agent-team/src"], + "@deepseek-ai/dsh-experimental-tool-agent-team": ["./packages/experimental/tool-agent-team/src"], + "@deepseek-ai/dsh-experimental-webworker-runtime": ["./packages/experimental/webworker-runtime/src"], + "@deepseek-ai/dsh-experimental-webworker-packer": ["./packages/experimental/webworker-packer/src"], + "@deepseek-ai/dsh-util-crypto": ["./packages/util/crypto/src"], "@deepseek-ai/dsh-*": [ "./packages/core/*/src", "./packages/prompt/*/src", @@ -247,7 +267,9 @@ "./packages/preset/*/src", "./packages/subagent/*/src", "./packages/jobs/*/src", + "./packages/experimental/*/src", "./packages/workflow/*/src", + "./packages/webhook/*/src", "./packages/web/*/src", "./packages/attachment/*/src", "./packages/spill/*/src", diff --git a/tsconfig.client.json b/tsconfig.client.json index e599f76f1b..b5874b6fde 100644 --- a/tsconfig.client.json +++ b/tsconfig.client.json @@ -26,6 +26,8 @@ // Client package outside packages/client with a local CSS module face. "packages/extensions/ui-cordis/src/css-modules.d.ts", "packages/client/tsdown.client.ts", + "scripts/client-build-environment.ts", + "scripts/*.client.spec.ts", "scripts/client-bundle-css.spec.ts", "scripts/client-bundle-purity.spec.ts" ], @@ -48,10 +50,8 @@ // rootDir rules make both paths depend on this runtime project reference. { "path": "./packages/compaction/compaction" }, { "path": "./packages/client/ui-slots" }, - { "path": "./packages/client/schema-form" }, { "path": "./packages/client/ui-attachment" }, { "path": "./packages/client/ui-primitives" }, - { "path": "./packages/client/web-react" }, { "path": "./packages/client/modules" }, { "path": "./packages/client/hmr" }, { "path": "./packages/client/connection/tsconfig.client.json" }, @@ -64,6 +64,7 @@ { "path": "./packages/test-support/client-runtime" }, { "path": "./packages/client/ui-layout" }, { "path": "./packages/client/ui-sidebar" }, + { "path": "./packages/client/ui-brand-official" }, { "path": "./packages/client/ui-conversation" }, { "path": "./packages/client/ui-tool" }, { "path": "./packages/client/ui-deliverables" }, @@ -72,6 +73,7 @@ { "path": "./packages/client/ui-input-trigger" }, { "path": "./packages/client/ui-commands" }, { "path": "./packages/client/ui-skill" }, + { "path": "./packages/client/ui-reference" }, { "path": "./packages/client/ui-subagent" }, { "path": "./packages/client/ui-jobs" }, { "path": "./packages/client/ui-directory-picker-browse" }, @@ -92,6 +94,7 @@ { "path": "./packages/client/ui-settings-models" }, { "path": "./packages/client/ui-settings-plugin-inventory" }, { "path": "./packages/client/locale" }, + { "path": "./packages/client/ui-renderer" }, { "path": "./packages/client/web" }, { "path": "./apps/web" } ] diff --git a/tsconfig.host.json b/tsconfig.host.json index 459036247a..c3ead24d3c 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -10,14 +10,17 @@ "include": [ "apps/web/tests/scaffold.ts", "apps/web/tests/default-model.e2e.ts", + "apps/web/tests/github-ready-review.e2e.ts", "apps/web/tests/declared-reasoning.e2e.ts", "apps/web/tests/support.ts", "apps/web/tests/scaffold-hermetic.e2e.ts", + "apps/web/tests/startup-rpc-budget.e2e.ts", "apps/web/tests/minimal-preset.snapshot.ts", "apps/web/tests/message-feedback-protocol.snapshot.ts", "apps/web/tests/live-interactions.e2e.ts", "apps/web/tests/question-composer.e2e.ts", "apps/web/tests/approval-composer.e2e.ts", + "apps/web/tests/plan-control-row.e2e.ts", "apps/web/tests/plan-review.e2e.ts", "apps/web/tests/steering.e2e.ts", "apps/web/tests/navigation-panes.e2e.ts", @@ -34,10 +37,12 @@ "apps/web/tests/workspace-management.e2e.ts", "apps/web/tests/replay-round-trip.e2e.ts", "apps/web/tests/hmr-live.e2e.ts", + "apps/web/tests/preview-boot.e2e.ts", "apps/web/tests/seeded-history.e2e.ts", "apps/web/tests/cold-blank-session.e2e.ts", "apps/web/tests/stats-paged-history.e2e.ts", "apps/web/tests/sidebar-scrollbar.e2e.ts", + "apps/web/tests/rail-search-expand.e2e.ts", "apps/web/tests/conversation-column-overflow.e2e.ts", "apps/web/tests/code-mode-round.e2e.ts", "apps/web/tests/composer-draft-scroll.e2e.ts", @@ -45,7 +50,10 @@ "apps/web/tests/web-search-round.e2e.ts", "apps/web/tests/message-actions.e2e.ts", "apps/web/tests/message-feedback.e2e.ts", + "apps/web/tests/message-feedback-layout.e2e.ts", "apps/web/tests/markdown-images.e2e.ts", + "apps/web/tests/reference-composer.e2e.ts", + "apps/web/tests/markdown-wide-table.e2e.ts", "apps/web/tests/math-rendering.e2e.ts", "apps/web/tests/markdown-cjk-strong.e2e.ts", "apps/web/tests/markdown-inline-code-links.e2e.ts", @@ -102,6 +110,7 @@ "packages/*/*/tests/**/*.client.spec.ts", "packages/*/*/tests/**/*.client.spec.tsx", "packages/client/tsdown.client.ts", + "scripts/*.client.spec.ts", "scripts/client-bundle-css.spec.ts", "packages/typert/generator/tests/fixtures/**", "scripts/client-bundle-purity.spec.ts" @@ -121,11 +130,13 @@ { "path": "./packages/util/native-command" }, { "path": "./packages/util/home-paths" }, { "path": "./packages/util/timeout" }, + { "path": "./packages/util/crypto" }, { "path": "./packages/util/output-retention" }, { "path": "./packages/util/atomic-write" }, { "path": "./packages/attachment/attachment" }, { "path": "./packages/attachment/attachment-local" }, { "path": "./packages/llm/llm" }, + { "path": "./packages/llm/deepseek-llm-api-extensions" }, { "path": "./packages/llm/token-meter" }, { "path": "./packages/core/session" }, { "path": "./packages/core/scope" }, @@ -136,6 +147,7 @@ { "path": "./packages/typert/loader" }, { "path": "./packages/session/session-persistence" }, { "path": "./packages/session/session-checkpoint-policy" }, + { "path": "./packages/session/session-log-deepseek" }, { "path": "./packages/session/session-persistence-jsonl" }, { "path": "./packages/session/session-persistence-sqlite" }, { "path": "./packages/session/session-projection" }, @@ -147,6 +159,7 @@ { "path": "./packages/settings/settings-file" }, { "path": "./packages/credentials/credentials" }, { "path": "./packages/credentials/credentials-local" }, + { "path": "./packages/credentials/authorization" }, { "path": "./packages/session-query/tool-session-query" }, { "path": "./packages/storage/storage" }, { "path": "./packages/storage/storage-json" }, @@ -172,6 +185,8 @@ { "path": "./packages/feedback/command-feedback" }, { "path": "./packages/schedule/schedule" }, { "path": "./packages/context/time-context" }, + { "path": "./packages/context/file-reference" }, + { "path": "./packages/context/file-reference-local" }, { "path": "./packages/context/tmux-context" }, { "path": "./packages/context/session-reference" }, { "path": "./packages/interaction/user-questions" }, @@ -190,14 +205,17 @@ { "path": "./packages/examples/agent-spine-demo" }, { "path": "./packages/subprocess/subprocess" }, { "path": "./packages/subprocess/subprocess-local" }, + { "path": "./packages/subprocess/win32-process" }, { "path": "./packages/e2b/e2b" }, { "path": "./packages/e2b/subprocess-e2b" }, { "path": "./packages/shell/shell" }, { "path": "./packages/terminal/terminal" }, { "path": "./packages/terminal/terminal-bash" }, { "path": "./packages/shell/tool-bash-persistent" }, + { "path": "./packages/shell/tool-pwsh-persistent" }, { "path": "./packages/terminal/tool-terminal" }, { "path": "./packages/code-runtime/code-runtime" }, + { "path": "./packages/code-runtime/code-runtime-python" }, { "path": "./packages/code-runtime/code-runtime-worker-thread" }, { "path": "./packages/llm/llm-deepseek" }, { "path": "./packages/llm/llm-pi-ai" }, @@ -238,19 +256,21 @@ { "path": "./packages/runtime-diagnostics/invariants" }, { "path": "./packages/test-support/agent-loop-testkit" }, { "path": "./packages/acp/acp" }, - { "path": "./packages/examples/acp-demo" }, + { "path": "./packages/bundle/acp-app" }, { "path": "./packages/bundle/base" }, { "path": "./packages/bundle/headless" }, + { "path": "./packages/bundle/sdk-app" }, { "path": "./packages/bundle/web-app" }, { "path": "./packages/boot/app-boot" }, { "path": "./packages/boot/cmdline" }, { "path": "./packages/sdk/server" }, - { "path": "./packages/examples/jsonrpc-demo" }, + { "path": "./packages/sdk/python-runtime" }, { "path": "./packages/test-support/llm-replay" }, { "path": "./packages/typert/generator" }, { "path": "./packages/test-support/acp-snapshot" }, { "path": "./packages/test-support/loader-smoke" }, { "path": "./packages/test-support/llm-mock-server" }, + { "path": "./packages/experimental/webworker-packer" }, { "path": "./packages/subagent/subagent" }, { "path": "./packages/subagent/tool-subagent" }, { "path": "./packages/subagent/tool-subagent-control" }, @@ -265,10 +285,14 @@ { "path": "./packages/jobs/jobs" }, { "path": "./packages/jobs/jobs-local" }, { "path": "./packages/jobs/tool-jobs" }, + { "path": "./packages/experimental/agent-team" }, + { "path": "./packages/experimental/tool-agent-team" }, { "path": "./packages/workflow/workflow" }, { "path": "./packages/workflow/workflow-worker-thread" }, { "path": "./packages/workflow/tool-workflow" }, { "path": "./packages/workflow/tool-ralph" }, + { "path": "./packages/webhook/webhook" }, + { "path": "./packages/webhook/webhook-github" }, { "path": "./packages/todo/tool-todo" }, { "path": "./packages/plan/plan-mode" }, { "path": "./packages/preset/agent-presets" }, @@ -287,6 +311,7 @@ { "path": "./packages/host/directory-picker-native" }, { "path": "./packages/host/frontend-static" }, { "path": "./packages/host/plugin-inventory" }, + { "path": "./packages/llm/plugin-package-inventory-deepseek" }, { "path": "./packages/host/webserver" }, { "path": "./packages/sdk/client" }, { "path": "./packages/sdk/protocol" }, diff --git a/vitest.config.ts b/vitest.config.ts index 1844b5cbcc..7ac8101425 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -5,6 +5,7 @@ import { resolvePwshPath } from './packages/shell/pwsh-local/src/resolve.ts' import { defineConfig } from 'vitest/config' import { standardDecoratorPlugin, vitestExecArgv } from './vitest.shared.ts' import { COVERAGE_EXEMPT_ENV, coverageExemptHeavySuites } from './scripts/coverage-exempt.ts' +import { COVERAGE_PARTITION_MODE_ENV } from './scripts/coverage-partitions.ts' // Prints exact `path:line:col` records for every uncovered statement, branch // path, and function when a file misses the per-file 100% gate — the built-in @@ -42,6 +43,17 @@ const windowsUnsupportedTests = process.platform === 'win32' 'packages/subprocess/subprocess-local/tests/process-inspector.spec.ts', 'packages/subprocess/subprocess-local/tests/spawn.spec.ts', 'packages/subprocess/subprocess-local/tests/terminal.spec.ts', + // Oracle-diff suites: they compare the worker's POSIX path/url faces + // against the host Node's own answers, which are win32 semantics on + // Windows. The worker always speaks POSIX; the Linux lanes hold the diff. + 'packages/experimental/webworker-runtime/tests/node/path-diff.spec.ts', + 'packages/experimental/webworker-runtime/tests/node/shim-diff.spec.ts', + // The subprocess ladder over the worker's child_process face: its kill + // rung reaches the in-worker process table through `process.kill`, + // which the ladder's win32 branch replaces with taskkill-by-real-pid — + // undeliverable to a table pid. The worker host always reports 'linux', + // so the Linux lanes hold the ladder. + 'packages/experimental/webworker-runtime/tests/node/child-process.spec.ts', ] : [] @@ -56,6 +68,10 @@ const windowsUnsupportedCoveragePackages = process.platform === 'win32' const windowsOnlyCoverageExclusions = process.platform !== 'win32' ? [ 'packages/sandbox/sandbox-windows-acl/src/**/*.ts', + // The koffi-backed Win32 table (Toolhelp32/GetProcessTimes/taskkill) + // executes only on win32; its decision logic is unit-pinned on every + // host through the injected-internals suites. + 'packages/subprocess/subprocess-local/src/windows-inspector.ts', ] : [] @@ -100,6 +116,12 @@ const coverageExemptExcludes = coverageExemptRaw === '1' ? coverageExemptHeavySuites.map(suite => suite.exclude) : [] +const coveragePartitionRaw = process.env[COVERAGE_PARTITION_MODE_ENV] +if (coveragePartitionRaw !== undefined && coveragePartitionRaw !== '' && coveragePartitionRaw !== '1') { + throw new Error(`vitest config: ${COVERAGE_PARTITION_MODE_ENV} must be '1' or unset, got ${JSON.stringify(coveragePartitionRaw)}.`) +} +const coveragePartitionMode = coveragePartitionRaw === '1' + // These suites exercise process-global state, process APIs, or timing-sensitive process I/O // that worker threads cannot isolate reliably under aggregate gate contention. // Keep the narrow exception in forks while the rest of the inventory avoids per-file processes. @@ -186,7 +208,7 @@ export default defineConfig({ 'packages/client/ui-primitives/src/RiskConfirmation.tsx', 'packages/client/ui-workspace/src/client/WorkspaceBrowser.tsx', 'packages/client/ui-workspace/src/client/WorkspacePicker.tsx', - 'packages/client/web-react/src/*', + 'packages/client/ui-renderer/src/client/*', // This isolated settings-scope lifecycle has complete unit coverage; // keep it out of the broader client-runtime GUI debt exemption. 'packages/client/runtime/src/**/!(settings-scope).ts', @@ -200,6 +222,14 @@ export default defineConfig({ 'packages/client/ui-layout/src/*', 'packages/client/web/src/*', 'packages/host/webserver/src/*', + // The browser-worker runtime and its image packer: the executing + // composition is a real dedicated Worker driven by the web browser lane + // (apps/web/tests/preview-boot.e2e.ts), which unit-process V8 coverage + // cannot observe. Unit specs cover the algorithmic cores; the assembled + // evidence is that boot. TODO(webworker): revisit when a browser-grade + // coverage lane exists. + 'packages/experimental/webworker-runtime/src/**', + 'packages/experimental/webworker-packer/src/*', 'packages/client/modules/src/client/system.ts', 'packages/client/hmr/src/client/index.ts', // Web config-tree boot round: the new host-side web-transport halves @@ -243,6 +273,8 @@ export default defineConfig({ 'packages/client/ui-workspace/src/client/index.ts', 'packages/test-support/client-runtime/src/translate.ts', 'packages/client/ui-primitives/src/JsonTree.tsx', + 'packages/client/ui-settings-models/src/client/DeepSeekOnboardingDialog.tsx', + 'packages/client/ui-settings-models/src/client/welcome-store.ts', 'packages/extensions/*/src/**/*.ts', 'packages/extensions/*/src/**/*.tsx', // Typert generator: correctness is pinned by its fixture suites and @@ -268,16 +300,20 @@ export default defineConfig({ // Per-file so a well-covered big file can't subsidize a bare one. // Every v8 ignore comment must carry a reason — see the quality-gates Agent Note // (.agents/notes/implemented/process/2026-06-11-quality-gates.md). - thresholds: { - perFile: true, - statements: 100, - branches: 100, - functions: 100, - lines: 100, - }, - reporter: process.env.CI - ? ['text', uncoveredLocationsReporter] - : ['text', 'html', uncoveredLocationsReporter], + thresholds: coveragePartitionMode + ? undefined + : { + perFile: true, + statements: 100, + branches: 100, + functions: 100, + lines: 100, + }, + reporter: coveragePartitionMode + ? [] + : process.env.CI + ? ['text', uncoveredLocationsReporter] + : ['text', 'html', uncoveredLocationsReporter], }, }, }) diff --git a/vitest.web.config.ts b/vitest.web.config.ts index 1179144f61..7c20ab6462 100644 --- a/vitest.web.config.ts +++ b/vitest.web.config.ts @@ -27,7 +27,8 @@ export default defineConfig({ 'apps/web/tests/**/*.e2e.ts', 'apps/web/tests/**/*.snapshot.ts', ], - // Browser boot + real-model turns are slow; files share one browser, run serial. + // Local and record runs stay serial. CI runs workspace-mutating HMR and + // dynamic Cordis lifecycle coverage before parallelizing the remaining files. testTimeout: 180_000, hookTimeout: 120_000, fileParallelism: false, diff --git a/website/.vitepress/config.ts b/website/.vitepress/config.ts index 86e22c7706..07fc6bad92 100644 --- a/website/.vitepress/config.ts +++ b/website/.vitepress/config.ts @@ -1,12 +1,12 @@ /** VitePress configuration for the locally projected documentation site. */ -import { readFileSync } from 'node:fs' +import { readFileSync, writeFileSync } from 'node:fs' import { resolve } from 'node:path' -import type { DefaultTheme, PageData } from 'vitepress' +import type { DefaultTheme, PageData, SiteConfig } from 'vitepress' import type { ViteDevServer } from 'vite' import { withMermaid } from 'vitepress-plugin-mermaid' -import { landingLink, orderedPages, routeLink, sectionSpec, type DocsLocale, type DocsPage, type DocsSidebar } from '../docs.ts' -import { docsSourceFiles, projectDocs } from '../../scripts/project-doc-site.ts' +import { landingLink, localeCollections, orderedPages, routeLink, sectionSpec, type DocsLocale, type DocsPage, type DocsSidebar } from '../docs.ts' +import { docsSourceFiles, emitRawMarkdownPages, llmsTxt, projectDocs, rawMarkdownRoute } from '../../scripts/project-doc-site.ts' projectDocs() @@ -58,14 +58,14 @@ interface GuideModules { */ const guideModules = { root: { - guide: 'zh-guide', - develop: { label: '开发', collection: 'zh-develop' }, - reference: { label: '参考', collection: 'zh-reference' }, + guide: localeCollections.root[0], + develop: { label: '开发', collection: localeCollections.root[1] }, + reference: { label: '参考', collection: localeCollections.root[2] }, }, en: { - guide: 'en-guide', - develop: { label: 'Development', collection: 'en-develop' }, - reference: { label: 'Reference', collection: 'en-reference' }, + guide: localeCollections.en[0], + develop: { label: 'Development', collection: localeCollections.en[1] }, + reference: { label: 'Reference', collection: localeCollections.en[2] }, }, } satisfies Record @@ -111,6 +111,46 @@ function watchCanonicalDocs(server: ViteDevServer): void { }) } +/** + * Serve the raw-Markdown twin of each route and llms.txt during development, + * matching what `buildEnd` emits into the static build. Pages project from + * their canonical sources per request, so an edit shows without a rebuild. + */ +function serveRawMarkdown(server: ViteDevServer): void { + server.middlewares.use((req, res, next) => { + if (req.url === undefined || (req.method !== 'GET' && req.method !== 'HEAD')) { + next() + return + } + // The dev client imports page modules at these same `.md` URLs, and a + // module script must reach Vite's transform. Browsers declare the purpose: + // `script` for module imports, `document` for address-bar navigation. + // Header-less clients (curl, agents) read the raw twin. In-page fetch() + // (`empty`) also passes to Vite — a deliberate dev-only divergence that + // keeps Vite's own requests unbroken, while production static hosting + // answers such a fetch with the raw file. + const fetchDest = req.headers['sec-fetch-dest'] + if (fetchDest !== undefined && fetchDest !== 'document') { + next() + return + } + const pathname = req.url.split(/[?#]/, 1)[0] ?? '' + const sitePath = pathname.startsWith(base) ? pathname.slice(base.length) : pathname.replace(/^\//, '') + if (sitePath === 'llms.txt') { + res.setHeader('Content-Type', 'text/plain; charset=utf-8') + res.end(llmsTxt({ base, ...siteIdentity })) + return + } + const content = sitePath.endsWith('.md') ? rawMarkdownRoute(sitePath) : undefined + if (content === undefined) { + next() + return + } + res.setHeader('Content-Type', 'text/markdown; charset=utf-8') + res.end(content) + }) +} + function escapeVueInterpolation(html: string): string { return html.replaceAll('{{', '{{').replaceAll('}}', '}}') } @@ -163,6 +203,12 @@ const sharedTheme: Pick` would freeze the mark at the colors the file declares. @@ -247,9 +293,14 @@ function siteTitle(previewTag: string): string { } export default withMermaid({ - title: 'DeepSeek Harness', - description: '用于构建 Agent Harness 的插件化 SDK', + title: siteIdentity.title, + description: siteIdentity.description, base, + /** Emit the raw-Markdown twin of every route plus llms.txt beside the rendered site. */ + buildEnd(siteConfig: SiteConfig) { + emitRawMarkdownPages(siteConfig.outDir) + writeFileSync(resolve(siteConfig.outDir, 'llms.txt'), llmsTxt({ base, ...siteIdentity })) + }, head: [ // VitePress leaves head hrefs untouched, so the base belongs here explicitly. ['link', { rel: 'icon', type: 'image/svg+xml', href: `${base}favicon.svg` }], @@ -322,7 +373,10 @@ export default withMermaid({ plugins: [ { name: 'deepseek-harness-doc-projector', - configureServer: watchCanonicalDocs, + configureServer(server) { + watchCanonicalDocs(server) + serveRawMarkdown(server) + }, }, ], }, @@ -330,11 +384,29 @@ export default withMermaid({ config(md) { const renderText = md.renderer.rules.text const renderCode = md.renderer.rules.code_inline - if (renderText === undefined || renderCode === undefined) { - throw new Error('VitePress Markdown renderer is missing its text or inline-code rule.') - } + const renderFence = md.renderer.rules.fence + if (renderText === undefined) throw new Error('VitePress Markdown renderer is missing the text rendering rule.') + if (renderCode === undefined) throw new Error('VitePress Markdown renderer is missing the inline-code rendering rule.') + if (renderFence === undefined) throw new Error('VitePress Markdown renderer is missing the fence rendering rule.') md.renderer.rules.text = (...args) => escapeVueInterpolation(renderText(...args)) md.renderer.rules.code_inline = (...args) => escapeVueInterpolation(renderCode(...args)) + const renderedFences = new Map() + md.renderer.rules.fence = (...args) => { + const [tokens, index] = args + const token = tokens[index] + if (token === undefined) throw new Error('VitePress code-fence renderer received no token.') + // Mermaid output embeds the token position, and VitePress snippets resolve source files during rendering. + if (['mermaid', 'mmd'].includes(token.info.trim().split(/\s+/, 1)[0] ?? '')) return renderFence(...args) + if (Reflect.get(token, 'src') !== undefined) return renderFence(...args) + // Keep the cache build-local; a dev renderer can survive many HMR updates. + if (process.env.NODE_ENV !== 'production') return renderFence(...args) + const key = JSON.stringify([token.content, token.info, token.markup, token.attrs]) + const cached = renderedFences.get(key) + if (cached !== undefined) return cached + const html = renderFence(...args) + renderedFences.set(key, html) + return html + } }, }, mermaid: {}, diff --git a/website/AGENTS.md b/website/AGENTS.md index 333041fd28..89af73f6b3 100644 --- a/website/AGENTS.md +++ b/website/AGENTS.md @@ -10,4 +10,6 @@ Keep canonical prose and generated catalogs in their owning `docs/` tier, then e The projector writes disposable Markdown to the ignored `website/.generated/` directory. Never edit or commit `.generated/`, `.cache/`, or `.dist/`. +The build also emits each route's raw-Markdown twin (with a parent-level alias per index route) and a root `llms.txt` index into `.dist/`, so a page's URL, minus any trailing slash, plus `.md` serves it as plain Markdown. Both derive from the publication manifest at build time; neither is ever a file in this tree. + Run `pnpm docs:check` after changing this subtree; the gate rejects additional non-ignored Markdown under `website/`. diff --git a/website/docs.ts b/website/docs.ts index 97fdfddd2a..5e6226ccf0 100644 --- a/website/docs.ts +++ b/website/docs.ts @@ -328,6 +328,8 @@ const subsystemsReference = subsystemGroups.flatMap(([rootSection, enSection, fi )) const reference = [ + // `docs/deepseek-llm-api-wire-extensions.md` is a repository-only provider protocol reference. + // Projected links intentionally resolve to its GitHub source instead of a public site route. ...pairedPages(([ ['docs/architecture.md', 'reference/index.md', '架构', 'Architecture', 0], ] as const).map(([source, route, rootLabel, enLabel, order]): PairedPage => ({ @@ -392,6 +394,7 @@ const reference = [ ['adding-a-package.md', '新增 Package', 'Adding a package'], ['adding-a-tool.md', '新增 Tool', 'Adding a tool'], ['adding-an-llm-adapter.md', '新增 LLM Adapter', 'Adding an LLM adapter'], + ['adding-a-settings-card.md', '新增设置卡片', 'Adding a settings card'], ['extension-cookbook.md', '扩展模式', 'Extension patterns'], ] as const).map(([file, rootLabel, enLabel], order): PairedPage => ({ source: `docs/cookbook/${file}`, @@ -407,10 +410,20 @@ const reference = [ label: { root: '新增 Conversation Node', en: 'Adding a Conversation Node' }, sidebar: { root: 'zh-reference', en: 'en-reference' }, section: { root: '开发手册', en: 'Cookbook' }, - order: 4, + order: 5, }]), ] +/** + * Sidebar collections of each locale, in the order the site's navigation + * presents them. The navigation bar and the llms.txt index both read this + * sequence, so a new collection lands in both surfaces together. + */ +export const localeCollections = { + root: ['zh-guide', 'zh-develop', 'zh-reference'], + en: ['en-guide', 'en-develop', 'en-reference'], +} as const satisfies Record + /** A sidebar group, matched to pages by `label`. */ export interface DocsSection { /** Group heading, equal to the `section` field of every page it holds. */