Merge remote-tracking branch 'origin/feat/cua-explicit-inspect' into fix/cua-localized-app
# Conflicts: # packages/experimental/cua-macos/README.i18n.yaml
This commit is contained in:
commit
6e68211a28
246 files changed
+3451
-1935
No files matched your search
+4
-4
@@ -9,11 +9,11 @@
|
||||
en: a22ddcfb339e503d
|
||||
zh: a1f4fadb46fce1e7
|
||||
/agent-note-native-cua-objects-independent-of-evaluator-ownership/decision:
|
||||
en: bbde2baca9167362
|
||||
zh: 69e6eb14818aed25
|
||||
en: d08345b0b22b0884
|
||||
zh: 18a843df211bcabc
|
||||
/agent-note-native-cua-objects-independent-of-evaluator-ownership/alternatives-considered:
|
||||
en: 3d0bc77e3c432cb9
|
||||
zh: 50a20eda1811f16c
|
||||
en: f2bb7165b7c168a0
|
||||
zh: e6234e3464d971ee
|
||||
/agent-note-native-cua-objects-independent-of-evaluator-ownership/consequences:
|
||||
en: 528c644d5810e60b
|
||||
zh: 9649dfc3b1172a6d
|
||||
@@ -22,7 +22,7 @@ The operation schema owns shared data declarations. Host-only limits and launch
|
||||
|
||||
Clipboard paste is distinct from Unicode key input: Return and Tab key events can trigger actions instead of inserting clipboard content. Pasteboard access runs on the AppKit main actor and retains promised data until a reader requests it. AX command completion alone does not establish that the application has fetched the data. Restoration checks ownership before replacing contents; the caller still verifies the application result. After successful command dispatch, `pasteReadTimeoutMs` bounds waiting for a data request, with a default of 1000 ms. Expiry returns `PASTE_NOT_CONSUMED`, closes the data provider against late delivery, and restores the clipboard only while still owned. The failure reports no read within that interval; it does not prove the application will never process its queued Paste command. A delayed command may read restored clipboard contents, so callers inspect the target and never retry automatically. Foreground-dependent consumption was observed in the installed ChatGPT/Codex application (`com.openai.codex`); other applications need their own validation. Guidance presents this as an example and applies recovery instructions to any `PASTE_NOT_CONSUMED` result.
|
||||
|
||||
CUA registers its own Harness tool and returns operation values directly. Screenshot capture requires a resolved image-capable model and the shared attachment service. The tool admits each screenshot once and renders its durable image reference; the window object and image sink retain the same `CuaScreenshot` containing encoded `bytes`, `mimeType`, `pixelWidth`, `pixelHeight` and `windowId` from the validated screenshot result. The bridge preserves this metadata without another upload or image emission. Callers use the reported format and dimensions when saving or declaring images.
|
||||
CUA registers its own Harness tool and returns operation values directly. Screenshot capture requires a resolved image-capable model and the shared attachment service. The attachment pipeline stores the screenshot at the dimensions used for coordinate mapping; the native tool result describes its encoded format and pixels. The program facade returns observation results, including the stored Image for a screenshot. The [explicit PTC output decision](2026-11-09-explicit-ptc-output.md) owns image identity and caller-selected display.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -32,7 +32,7 @@ A CUA-specific persistent evaluator duplicates execution lifecycle policy and pr
|
||||
|
||||
Reusing the MCP result adapter would impose MCP envelopes and wire validation on native CUA operations. Direct Harness tool output and shared attachment admission provide the needed logging and images without a second protocol.
|
||||
|
||||
**Fixing the screenshot format** would require otherwise unnecessary re-encoding and discard the attachment pipeline’s pass-through optimization. A format convention alone still forces callers to guess, and cannot tell them when encoding policy changes. Returning the already computed format and dimensions preserves pass-through and makes each result describe its actual bytes.
|
||||
**Fixing the screenshot format** would require otherwise unnecessary re-encoding and discard the attachment pipeline’s pass-through optimization. The native result describes its encoding; program Images expose the stored file and its pixel dimensions, so callers can use the file without a facade-specific byte representation.
|
||||
|
||||
**Waiting until the full native-operation timeout** hides successful command dispatch behind a generic timeout. A separate configurable read budget reports the missing acknowledgement without treating slow applications as unsupported. Requiring every application to be foreground would claim a prerequisite that the available observations do not establish.
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ Computer-use 程序需要一致的应用和窗口操作。将这些操作与 CUA
|
||||
|
||||
剪贴板粘贴与 Unicode 按键输入分别实现,因为 Return 和 Tab 按键可能触发动作,而不是插入剪贴板内容。剪贴板访问使用 AppKit 主线程,保留延迟提供的数据直到读取方请求。AX 命令完成本身不能证明应用已取得数据。恢复前检查剪贴板归属;调用方仍需验证应用中的结果。 命令成功分派后,`pasteReadTimeoutMs` 限制等待数据请求的时间,默认 1000 毫秒。到期返回 `PASTE_NOT_CONSUMED`,关闭数据提供方以阻止迟到的数据交付,并仅在仍持有剪贴板时恢复原内容。该失败只说明这段时间内没有读取,不证明应用以后不会处理已排队的 Paste 命令。迟到的命令可能读取恢复后的剪贴板内容,因此调用方必须检查目标,不能自动重试。已安装的 ChatGPT/Codex 应用(`com.openai.codex`)已观察到依赖前台才能读取的行为,其他应用需要分别验证。指引将其写为例子,并对任何 `PASTE_NOT_CONSUMED` 结果提供恢复步骤。
|
||||
|
||||
CUA 注册自己的 Harness 工具,直接返回操作结果。截图要求解析到支持图片输入的模型,并提供共享附件服务。工具只接入一次截图,再渲染持久图片引用;窗口对象和图片输出回调保留同一个 `CuaScreenshot`,其中的编码 `bytes`、`mimeType`、`pixelWidth`、`pixelHeight` 和 `windowId` 来自已校验的截图结果。桥接保留这些元数据,不再次上传或输出图片。调用方落盘或声明图片时使用返回的格式和尺寸。
|
||||
CUA 注册自己的 Harness 工具,直接返回操作结果。截图要求解析到支持图片输入的模型,并提供共享附件服务。附件流水线按坐标换算使用的尺寸保存截图,原生工具结果描述编码格式和像素。程序对象接口返回观察结果,截图操作返回已存储的 `Image`。[PTC 显式输出决策](2026-11-09-explicit-ptc-output.zh.md)负责图片身份及调用方选择展示的规则。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
@@ -32,7 +32,7 @@ CUA 专用持久执行器会重复实现执行生命周期规则,使 PTC 无
|
||||
|
||||
复用 MCP 结果适配器会给原生 CUA 操作引入 MCP 信封和协议校验。直接使用 Harness 工具输出与共享附件接入,即可取得所需的日志和图片能力,无需再经过一种协议。
|
||||
|
||||
**固定截图格式**会要求原本不需要的重新编码,并失去附件处理的原样直通优化。仅约定格式仍要求调用方猜测,也无法告知编码策略的变化。返回已经算出的格式和尺寸,既保留直通优化,也让每个结果说明其实际字节。
|
||||
**固定截图格式**会要求原本不需要的重新编码,并失去附件处理的原样直通优化。原生结果描述编码格式;程序 Image 提供已存储文件及其像素尺寸,使调用方直接使用文件,无需对象接口专属的字节表示。
|
||||
|
||||
**等到原生操作的总超时**会用一条通用超时掩盖命令已经成功分派的事实。独立、可配置的读取等待上限能报告缺失的读取确认,而不将响应慢的应用认定为不支持粘贴。要求所有应用都必须位于前台,则超出了现有观察能支持的结论。
|
||||
|
||||
|
||||
@@ -9,8 +9,8 @@
|
||||
en: 87ce83e36a6ab68e
|
||||
zh: 8aa98dc7dae03c41
|
||||
/agent-note-cua-programs-outside-the-ptc-preset/decision:
|
||||
en: 98e4a571dc7d4c10
|
||||
zh: 2465f2abff5eb64c
|
||||
en: 13e079e19b5f2d6d
|
||||
zh: 2ce9b6aa981162d7
|
||||
/agent-note-cua-programs-outside-the-ptc-preset/alternatives-considered:
|
||||
en: b5d8e00cbe4b53ae
|
||||
zh: 4ef81611150a8d75
|
||||
|
||||
@@ -12,7 +12,7 @@ Computer Use needs reusable application and window objects in Standard and other
|
||||
|
||||
CUA keeps `agentPreset: ptc` as the default selector for its raw `cua` tool. Every other composed preset receives `cua_run_code`; Agents without a composed preset receive neither tool. `agentPreset: false` retains global raw-tool registration. A base owner uses Agent scopes for registration and disposal, so enabling and disabling the provider affects existing and later Agents without editing presets. Raw registration depends only on base CUA services and preset selection. The private program producer publishes and withdraws its definition by identity; private dependency loss or startup failure leaves raw tools and prompts available.
|
||||
|
||||
`cua_run_code` composes the shared [program-tool factory](../../../../packages/core/tool-executor-ptc/README.md) with one isolated Node `persistent-in-memory` runtime owned by the CUA provider. Agent `stateOwner` identities select separate retained processes. Its only binding is the private `tools.cua` definition; the model calls ordinary session tools directly. Nested CUA operations retain tool policy, guards, dispatch events, and image admission. The provider releases held keys when either program tool settles. Provider unload drains its own CUA programs before native teardown; releasing input held by other PTC programs does not wait for their unrelated work. State is in memory, separate from the profile's PTC state, and released with its Agent or provider.
|
||||
`cua_run_code` composes the shared [program-tool factory](../../../../packages/core/tool-executor-ptc/README.md) with one isolated Node `persistent-in-memory` runtime owned by the CUA provider. Agent `stateOwner` identities select separate retained processes. Its only binding is the private `tools.cua` definition; the model calls ordinary session tools directly. Nested CUA operations retain tool policy, guards, dispatch events, and image admission. Programs select text and screenshots through `inspect(...values)`, following the [explicit output decision](2026-11-09-explicit-ptc-output.md). The provider releases held keys when either program tool settles. Provider unload drains its own CUA programs before native teardown; releasing input held by other PTC programs does not wait for their unrelated work. State is in memory, separate from the profile's PTC state, and released with its Agent or provider.
|
||||
|
||||
CUA registers a source program through `registerEnvironmentInitializer` before its isolated runtime admits a run. Each new environment evaluates it once after installing current bindings, including temporary escalated environments. The child awaits Host acceptance before starting model code. Initialization shares the first run's time and output budgets; failure discards the environment and prevents model execution without rolling back external effects. Separate evaluation preserves model source locations. An initializer-owning scope depends only on the runtime, so program-tool dependency reloads retain the registration. PTC keeps its explicit bridge initialization.
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ Computer Use 需要在标准及其他 preset 中复用应用和窗口对象,
|
||||
|
||||
CUA 保留 `agentPreset: ptc`,用默认选择器指定获得原始 `cua` 工具的 preset。其他已组合的 preset 获得 `cua_run_code`;未组合 preset 的 agent(智能体)不获得这两个工具。`agentPreset: false` 保留全局原始工具注册。基础注册的所有者通过 Agent 作用域管理注册和销毁,因此启停提供方会影响已有和后续 Agent,无需修改 preset。原始工具的注册只依赖基础 CUA 服务和 preset 选择。私有程序工具的提供方按定义对象的身份发布和撤回其定义;私有依赖缺失或启动失败时,原始工具和提示词仍然可用。
|
||||
|
||||
`cua_run_code` 将共享[程序工具工厂](../../../../packages/core/tool-executor-ptc/README.zh.md)与 CUA 提供方持有的单个隔离 Node `persistent-in-memory` 运行时组合。Agent 的 `stateOwner` 身份选择各自独立的保留进程。它只绑定私有 `tools.cua` 定义;模型直接调用其他普通会话工具。嵌套 CUA 操作保留工具策略、guard、派发事件和图像接纳。任一程序工具结束时,提供方都会释放该程序仍按住的键。提供方卸载时,先等待自己持有的 CUA 程序结束再清理原生资源;释放其他 PTC 程序持有的输入时,不等待这些程序中的其他工作。状态保存在内存中,独立于 profile 的 PTC 状态,随所属 Agent 或提供方释放。
|
||||
`cua_run_code` 将共享[程序工具工厂](../../../../packages/core/tool-executor-ptc/README.zh.md)与 CUA 提供方持有的单个隔离 Node `persistent-in-memory` 运行时组合。Agent 的 `stateOwner` 身份选择各自独立的保留进程。它只绑定私有 `tools.cua` 定义;模型直接调用其他普通会话工具。嵌套 CUA 操作保留工具策略、guard、派发事件和图像接纳。程序通过 `inspect(...values)` 选择文字和截图,遵循[显式输出决策](2026-11-09-explicit-ptc-output.zh.md)。任一程序工具结束时,提供方都会释放该程序仍按住的键。提供方卸载时,先等待自己持有的 CUA 程序结束再清理原生资源;释放其他 PTC 程序持有的输入时,不等待这些程序中的其他工作。状态保存在内存中,独立于 profile 的 PTC 状态,随所属 Agent 或提供方释放。
|
||||
|
||||
CUA 在隔离运行时首次接纳运行前,通过 `registerEnvironmentInitializer` 注册一段源码。每个新环境安装本次绑定后执行该程序一次,包括临时提权环境。子进程等待 Host 确认后才开始执行模型代码。初始化与首次运行共享时间和输出预算;失败会丢弃环境并阻止模型代码执行,但不回滚外部效果。独立求值保留模型源码位置。初始化程序所属的作用域只依赖运行时,因此程序工具的依赖重载时仍保留这项注册。PTC 保留显式 bridge 初始化。
|
||||
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
# Bilingual-pair consistency record for 2026-11-09-explicit-ptc-output.md (docs/i18n/README.md): per heading
|
||||
# section, a hash of its English and Chinese blocks outside code blocks and generated regions.
|
||||
# After editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-11-09-explicit-ptc-output.md
|
||||
/agent-note-explicit-text-and-image-output-in-ptc:
|
||||
en: b62f1fc259e6f4a7
|
||||
zh: 340942c222a38e2a
|
||||
/agent-note-explicit-text-and-image-output-in-ptc/problem:
|
||||
en: 51667474a3f9053d
|
||||
zh: 70c6af69e943f4f5
|
||||
/agent-note-explicit-text-and-image-output-in-ptc/decision:
|
||||
en: ec82c0ce9f6a371d
|
||||
zh: 74596c0ab6a3cdd7
|
||||
/agent-note-explicit-text-and-image-output-in-ptc/alternatives-considered:
|
||||
en: cfbca328947567f3
|
||||
zh: 3b2f619365d2250e
|
||||
/agent-note-explicit-text-and-image-output-in-ptc/consequences:
|
||||
en: 636609d91f12a960
|
||||
zh: b26397c72f972827
|
||||
@@ -0,0 +1,43 @@
|
||||
# Agent Note: Explicit text and image output in PTC
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-11-09-explicit-ptc-output.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
PTC lets a program acquire and filter intermediate tool results before choosing what the model sees. A final expression or return value can expose incidental data, while automatically attaching every acquired image defeats that selection. Separate screenshot bytes and image-output callbacks also make programs convert images before comparing or displaying them.
|
||||
|
||||
## Decision
|
||||
|
||||
TypeScript `run_code` programs select text, objects and images through top-level `inspect(...values)`. Arguments are rendered independently: strings appear directly, and other values use Node's object formatting without depth, array-length or string-length preview limits. Inspection displays undefined properties, Error objects and circular references; requiring lossless JSON for display would interrupt subsequent work after tools have already executed. Images can occur inside arrays or objects. The runtime Image's custom inspection hook collects images reached by Node's formatter; each formatting call owns its collector. One output sequence preserves the relationship between explanatory text and selected images.
|
||||
|
||||
The tool executor requests `discardCompletion: true`, so completion values are discarded before JSON validation and serialization. This is a consumer choice: direct runtime and workflow consumers retain completion values when their protocols require them. The [PTC foundation](../feature/2026-06-15-ptc.md) owns dispatch and settlement. The [typed-return decision](../feature/2026-07-20-ptc-typed-tool-returns.md) continues to own ordinary JSON binding values, failures and output accounting; this note owns TypeScript output selection and image projection.
|
||||
|
||||
The Node environment supplies `inspect` alongside the complete native console. Native console methods remain available to programs and imported libraries, and their stdout and stderr writes share the captured output sequence. Model instructions direct programs to select output through `inspect`.
|
||||
|
||||
`read_image` and CUA screenshot bindings expose stored `Image` values. Readonly `attachmentId`, `path`, `width` and `height` identify the stored image and its readable execution-world file. The dimensions describe that file's pixels, including the image used for CUA coordinate mapping. The [CUA object API](2026-09-17-experimental-cua-object-api.md) returns screenshot Images and text observations directly to the program. The program passes selected values to `inspect` for display.
|
||||
|
||||
The host carries the attachment reference with the program-readable path. The runtime transports image metadata and keeps the attachment reference behind a process-owned image ID. Inspection resolves that ID directly to the registered reference. Image identity is established by the runtime; matching object properties alone do not grant access to an image output. Retained program Images follow their owning process's lifetime.
|
||||
|
||||
Nested tool dispatches retain their complete rendered content in the Session log. Successful outer `run_code` results interleave inspected images with text in emission order, where model history and UI rendering use the same durable attachment references. Successful outer-result replacement remains authoritative over the selected output.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Display the completion value alongside printed output.** The last expression or return then controls context admission as well as program control flow. Discarding it before validation lets a program finish with an arbitrary internal value while explicitly selecting a smaller visible result.
|
||||
|
||||
**Automatically forward every acquired image.** Acquisition then implies model-context cost, even when the program needs several screenshots to select one. Explicit inspection gives text and images the same selection rule.
|
||||
|
||||
**Keep a CUA byte-returning facade and image sink.** This maintains a separate screenshot representation and a second image-emission path. Returning the runtime's Image lets file reads and screenshots share inspection and preserves the stored attachment reference through display.
|
||||
|
||||
**Discover images with a separate object traversal.** A separate traversal must duplicate Node's handling of collections, symbols and custom formatters and can emit images absent from the text representation. The Image hook ties collection to Node's formatting.
|
||||
|
||||
**Bounded object previews.** Item and string preview limits can omit file contents that a program explicitly selected. Removing those limits keeps selected contents complete within the shared output budget.
|
||||
|
||||
**Restrict console availability.** Missing bindings and incomplete replacements can break program code and imported libraries. The native console preserves Node methods while prompt guidance directs output selection through `inspect`.
|
||||
|
||||
## Consequences
|
||||
|
||||
Programs explicitly choose the output needed for the next model request, including image comparisons and selected nested results. The [upgrade guide](../../../../docs/upgrade-guide/v0.2.2-rc.1/ptc-inspect-output/guide.md) owns caller migration. Acquiring an image still performs storage and processing work; selection controls its presentation to the model.
|
||||
|
||||
The runtime owns process-local image identity and output ordering; the tool executor owns attachment projection and durable result content. Text and image-reference metadata share the outer output budget. Runtime teardown releases retained image identities, while stored attachments follow the attachment store's lifetime.
|
||||
@@ -0,0 +1,43 @@
|
||||
# Agent Note: PTC 显式文本与图片输出
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-11-09-explicit-ptc-output.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
PTC 允许程序先获取和筛选中间工具结果,再选择模型看到的内容。末尾表达式或返回值可能暴露偶然产生的数据,而自动附加每张已获取的图片会使这种筛选失效。独立的截图字节和图片输出回调也要求程序先转换图片,再进行比较或展示。
|
||||
|
||||
## 决策
|
||||
|
||||
TypeScript `run_code` 程序通过顶层 `inspect(...values)` 选择文本、对象和图片。各参数独立渲染:字符串直接展示,其他值使用 Node 的对象格式,不限制预览深度、数组项数或字符串长度。inspect 展示 undefined 属性、Error 对象和循环引用;要求展示值必须是无损 JSON,会在工具已经执行后中断后续工作。图片也可以嵌入数组或对象。运行时 Image 的自定义 inspect 钩子收集 Node 格式化函数访问到的图片;每次格式化调用拥有自己的收集器。同一输出序列保留说明文字与所选图片的对应关系。
|
||||
|
||||
工具执行器请求 `discardCompletion: true`,在 JSON 校验和序列化前丢弃完成值。这是消费方的选择:直接运行时调用方和工作流消费方在其协议需要时保留完成值。[PTC 基础](../feature/2026-06-15-ptc.zh.md)负责分发与结算。[类型化返回值决策](../feature/2026-07-20-ptc-typed-tool-returns.zh.md)继续负责普通 JSON 绑定值、失败和输出计量;本记录负责 TypeScript 输出选择与图片投影。
|
||||
|
||||
Node 环境同时提供 `inspect` 和完整的原生 console。程序与导入的库均可使用原生 console 方法,其 stdout 和 stderr 写入共用捕获输出序列。模型指引要求程序通过 `inspect` 选择输出。
|
||||
|
||||
`read_image` 和 CUA 截图绑定提供已存储的 `Image` 值。只读的 `attachmentId`、`path`、`width` 和 `height` 标识已存储图片及其在执行环境中可读取的文件。尺寸描述该文件的像素,也对应 CUA 坐标换算所用的图片。[CUA 对象 API](2026-09-17-experimental-cua-object-api.zh.md)将截图的 `Image` 和观察文本直接返回给程序。程序将选中的值传给 `inspect` 展示。
|
||||
|
||||
宿主将附件引用与程序可读取的路径一起传递。运行时传输图片元数据,并用进程拥有的图片 ID 保存对应的附件引用。展示时,该 ID 直接解析为已登记的引用。图片身份由运行时建立,仅有相同对象属性不能取得图片输出权限。保留在程序中的 Image 跟随其所属进程的生命周期。
|
||||
|
||||
嵌套工具分发将完整渲染内容保留在 Session 日志中。成功的外层 `run_code` 结果按输出顺序穿插显式展示的图片和文本,模型历史和 UI 渲染使用相同的持久附件引用。成功外层结果的替换仍对所选输出具有最终决定权。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**同时展示完成值与打印输出。** 末尾表达式或返回值会同时控制上下文纳入和程序控制流。在校验前丢弃完成值,使程序可以用任意内部值结束,并显式选择更小的可见结果。
|
||||
|
||||
**自动转发每张已获取的图片。** 即使程序需要先获取多张截图才能选出一张,获取动作本身也会产生模型上下文成本。显式展示让文本和图片遵循同一选择规则。
|
||||
|
||||
**保留返回字节的 CUA 对象接口和图片输出回调。** 这会维护独立的截图表示和第二条图片输出路径。返回运行时 Image,使文件读取和截图共用展示方式,并在展示过程中保留已存储附件引用。
|
||||
|
||||
**单独遍历对象以发现图片。** 独立遍历必须重复实现 Node 对集合、Symbol 和自定义格式化的处理,还可能输出文本表示中未出现的图片。Image 钩子让收集跟随 Node 的格式化。
|
||||
|
||||
**限制对象预览。** 条目和字符串的预览上限可能省略程序已明确选中的文件内容。取消这些限制,在共享输出预算内完整保留选中的内容。
|
||||
|
||||
**限制 console 的可用性。** 缺失的绑定和不完整的替代实现可能使程序及导入的库出错。原生 console 保留 Node 方法,提示词指引程序通过 `inspect` 选择输出。
|
||||
|
||||
## 后果
|
||||
|
||||
程序显式选择下一次模型请求需要的输出,包括图片比较和嵌套结果中的所选内容。[升级指南](../../../../docs/upgrade-guide/v0.2.2-rc.1/ptc-inspect-output/guide.zh.md)负责调用方迁移。获取图片仍会执行存储和处理工作,选择控制的是向模型展示的内容。
|
||||
|
||||
运行时负责进程内图片身份和输出排序;工具执行器负责附件投影和持久结果内容。文本和图片引用元数据共用外层输出预算。运行时拆卸释放保留的图片身份,已存储附件则遵循附件存储的生命周期。
|
||||
@@ -9,8 +9,8 @@
|
||||
en: 4b312be894531bee
|
||||
zh: d655648ea5843ef5
|
||||
/agent-note-ptc-mode-the-model-writes-typescript-against-the-tool-registry/decision:
|
||||
en: a1732e2b1b46cb39
|
||||
zh: ddfc2bdcffa099a5
|
||||
en: 109e418dd92ead24
|
||||
zh: e2564f0f98cc4641
|
||||
/agent-note-ptc-mode-the-model-writes-typescript-against-the-tool-registry/decision/per-program-execution-controls:
|
||||
en: da694641e1d009de
|
||||
zh: 9d7492d39039af66
|
||||
@@ -18,23 +18,23 @@
|
||||
en: 4cd76a34f4712e1a
|
||||
zh: 303ef72a18644d7c
|
||||
/agent-note-ptc-mode-the-model-writes-typescript-against-the-tool-registry/decision/the-run-code-tool-and-the-dispatch-bridge:
|
||||
en: c3d079330748cd24
|
||||
zh: 4e2cc41d00227780
|
||||
en: 69e4683ac3150bdf
|
||||
zh: d8eaa12eaba6d67e
|
||||
/agent-note-ptc-mode-the-model-writes-typescript-against-the-tool-registry/decision/observability-tool-ptc-dispatch:
|
||||
en: 5423da23d4e8d1d9
|
||||
zh: f1198d4f224f3fe6
|
||||
en: 951fb03a27841eea
|
||||
zh: 38928d1ca859eced
|
||||
/agent-note-ptc-mode-the-model-writes-typescript-against-the-tool-registry/decision/the-ptc-runtime-seam:
|
||||
en: 4c423f4e9cef6055
|
||||
zh: 5c23a3637eb420c0
|
||||
/agent-note-ptc-mode-the-model-writes-typescript-against-the-tool-registry/decision/the-node-process-runtime:
|
||||
en: 0b846ed33977e383
|
||||
zh: e45158758d657f9a
|
||||
en: b2a8d7d654d1832b
|
||||
zh: 5852a2a3660e5a2e
|
||||
/agent-note-ptc-mode-the-model-writes-typescript-against-the-tool-registry/decision/trust-posture:
|
||||
en: d62b41a6a5c58e5f
|
||||
zh: ef72c5a9b32b370e
|
||||
/agent-note-ptc-mode-the-model-writes-typescript-against-the-tool-registry/decision/what-the-model-sees:
|
||||
en: 81c12f19f7a967c1
|
||||
zh: 29734ca964acdec2
|
||||
en: 109d4c326bfcf572
|
||||
zh: 9f6be5afc281054d
|
||||
/agent-note-ptc-mode-the-model-writes-typescript-against-the-tool-registry/consequences:
|
||||
en: 761eb6c1aeb7af18
|
||||
zh: 5434456a3d0a1b65
|
||||
|
||||
@@ -22,7 +22,7 @@ Three decisions, each elaborated in its own section below:
|
||||
2. **Code execution is a capability seam** — `packages/ptc-runtime/` contains the Service Definition package `@deepseek-ai/dsh-ptc-runtime`, which owns `ctx.ptcRuntime` ([capability seams](../architecture/2026-06-13-capability-seams.md); Consumer = `dsh-tools`, with core-consumes-a-seam precedent in `agent-loop` → `dsh-llm`). The runtime knows nothing about tools: it is handed a program and named async bindings, runs the program, and reports `{ value, logs, error? }`. Language and substrate are backend properties, so a future Python or container backend is another Service Provider package, not a redesign.
|
||||
3. **The shipped implementation is `@deepseek-ai/dsh-ptc-runtime-node`**: each program runs in a fresh managed Node process under the calling Session's resolved sandbox policy. Direct Node APIs remain available within that policy; named host bindings cross a separately validated control channel.
|
||||
|
||||
This note owns PTC presentation, composition and settlement. The [typed tool-return decision](2026-07-20-ptc-typed-tool-returns.md) owns generated outputs, canonical binding values, `ToolCallError` and lossless outer output. The [sandboxed Node decision](../architecture/2026-09-11-sandboxed-node-ptc-runtime.md) supersedes the worker execution, trust posture and budget realization while preserving this note's consumer rules.
|
||||
This note owns PTC presentation, composition and settlement. The [typed tool-return decision](2026-07-20-ptc-typed-tool-returns.md) owns generated outputs, ordinary canonical binding values, `ToolCallError` and lossless runtime output. The [explicit output decision](../architecture/2026-11-09-explicit-ptc-output.md) owns TypeScript completion suppression and text/image selection. The [sandboxed Node decision](../architecture/2026-09-11-sandboxed-node-ptc-runtime.md) supersedes the worker execution, trust posture and budget realization while preserving this note's consumer rules.
|
||||
|
||||
### Per-program execution controls
|
||||
|
||||
@@ -48,9 +48,9 @@ The [Node execution decision](../architecture/2026-09-11-sandboxed-node-ptc-runt
|
||||
|
||||
Under `'ptc'` the registry owns `run_code` as a reserved presentation transport with two required parameters, `{ code: string; description: string }` (the description labels the call in UIs, the bash precedent). It is represented by a normal `ToolDefinition` for dispatch but stays outside the filterable capability layers, so restrictions cannot accidentally remove PTC mode's only entry point. Calls traverse the complete tool pipeline — `tools/pre-execute` → monotonic guards → `tools/execute` around dispatch → `tools/post-execute` → optional definition-owned `finalizeContent` → immutable `tools/result` notification — exactly like native calls; a permission plugin can inspect the program text before it runs, and final-result observers see the normalized outer outcome. Its `execute(args, exec)`:
|
||||
|
||||
1. **Build bindings.** One run-scoped signal follows outer cancellation and is aborted whenever the run settles. Each visible tool binding snapshots lossless-JSON arguments, enters the native-contract dispatch pool (the [live-parallel reference](../../../../packages/core/tools/README.md) owns the scheduling design), executes with a deterministic call id and the outer token as `parent`, defers returned contexts through the outer execution, and logs the `tool/ptc-dispatch-start`/`tool/ptc-dispatch` pair, the settle side carrying the full rendered result content. Success returns the tool's final canonical JSON value; failure becomes the program-visible `ToolCallError`. Every sub-call retains its own immutable execution identity and traverses the full tool pipeline.
|
||||
1. **Build bindings.** One run-scoped signal follows outer cancellation and is aborted whenever the run settles. Each visible tool binding snapshots lossless-JSON arguments, enters the native-contract dispatch pool (the [live-parallel reference](../../../../packages/core/tools/README.md) owns the scheduling design), executes with a deterministic call id and the outer token as `parent`, defers returned contexts through the outer execution, and logs the `tool/ptc-dispatch-start`/`tool/ptc-dispatch` pair, the settle side carrying the full rendered result content. Success returns the consumer-facing value, including program Images for the selected TypeScript bindings; failure becomes the program-visible `ToolCallError`. Every sub-call retains its own immutable execution identity and traverses the full tool pipeline.
|
||||
2. **Runs resolved inputs:** `ctx.ptcRuntime.run(ctx.ptcRuntime.resolve(request))`. The request carries the program, bindings, run-scoped signal, calling Session cwd and applicable resolved file policy. The run-scoped signal connects outer settlement to execution cancellation.
|
||||
3. **Settle after quiescence.** When the runtime settles, the bridge aborts outstanding work and drains the dispatch queue before returning. Success returns captured logs and the completion value as canonical output; the registry renders that value into durable `tool/result.content`, which the result card reads directly. A runtime failure becomes `CodeRunFailedError`; backend rejection uses the registry's normal error boundary. Both produce structured error results, and no sub-call can append after `run_code` settles.
|
||||
3. **Settle after quiescence.** When the runtime settles, the bridge aborts outstanding work and drains the dispatch queue before returning. TypeScript success returns captured logs as canonical output and projects inspected images into durable `tool/result.content`; other language consumers also render their completion value. The result card reads that content directly. A runtime failure becomes `CodeRunFailedError`; backend rejection uses the registry's normal error boundary. Both produce structured error results, and no sub-call can append after `run_code` settles.
|
||||
|
||||
**Sub-call contexts are deferred through the parent.** Injecting inside `run_code` would break parent call/result adjacency, so `ToolRunContext.deferContext()` collects every sub-result `additionalContexts` entry in dispatch order. The registry carries that array even when the program later throws, and the loop appends each entry only after the outer result and every sibling result in the step. An outer post-execute block discards tool-deferred entries and exposes only contexts explicitly attached by the blocking decision.
|
||||
|
||||
@@ -58,13 +58,13 @@ Under `'ptc'` the registry owns `run_code` as a reserved presentation transport
|
||||
|
||||
**Tool-call allowance.** Counting attempts before validation and queuing bounds loops that repeatedly fail or enqueue work. The allowance scales with the resolved timeout and permits bursts; it does not enforce a rolling window. Exhaustion uses the run-scoped abort path so a program cannot catch a binding failure and keep running. Active calls are canceled, queued calls are abandoned, and settlement waits for cleanup. Completed effects remain; a new run receives a fresh allowance. The [executor README](../../../../packages/core/tool-executor-ptc/README.md#use-this-package) owns configuration and failure reporting.
|
||||
|
||||
**Presentation.** `run_code`'s render intent is decided here per the [render-intent Agent Note](../architecture/2026-07-02-tool-render-intent-union.md): `presentCall` creates a `generic` card with `kind: 'execute'`, the program text as its title, and the same program text as `rawInput`; `run_code` intentionally declares no `presentResult`, so the TUI and host/client runtime (Web) complete that card through their generic raw-content fallback using the final durable `tool/result.content`, including captured logs plus the returned value, failure, or post-policy spill preview. This is not a `terminal` card: that card's semantics are "a shell command in a working directory", which a program is not. See the [result-card completeness note](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md).
|
||||
**Presentation.** `run_code` uses a `generic` card with `kind: 'execute'`, the model-authored description as its title and the submitted program as `rawInput`. UI adapters read durable `tool/result.content` for captured text, selected images, failure or the post-policy spill preview. The [explicit output decision](../architecture/2026-11-09-explicit-ptc-output.md) owns selection of that content.
|
||||
|
||||
### Observability: `tool/ptc-dispatch`
|
||||
|
||||
Each sub-dispatch appends a log-only `tool/ptc-dispatch-start` event at pool entry and a `tool/ptc-dispatch` settle event containing parent and child call ids, tool identity, normalized arguments, and the complete rendered `content`/`isError` outcome. It remains outside model history but available to persistence and UIs. Appends occur inside the open `run_code` turn. Direct executions without an agent still run but cannot log the event.
|
||||
|
||||
New sub-calls use `<parent>:ptc:<n>` ids, numbered by tool-call attempt. Invalid arguments consume a number without a dispatch event, so logged ids may have gaps. All call ids are opaque to consumers: migration preserves every historical id byte-for-byte, including `:code:` substrings, so dispatch pairs, spill references, and other correlations remain intact. The bridge attributes forwarded image context to `{ kind: 'plugin', plugin: 'tools-ptc' }`.
|
||||
New sub-calls use `<parent>:ptc:<n>` ids, numbered by tool-call attempt. Invalid arguments consume a number without a dispatch event, so logged ids may have gaps. All call ids are opaque to consumers: migration preserves every historical id byte-for-byte, including `:code:` substrings, so dispatch pairs, spill references, and other correlations remain intact. The [explicit output decision](../architecture/2026-11-09-explicit-ptc-output.md) owns TypeScript image presentation; other language consumers attribute forwarded image contexts to `{ kind: 'ptc-mode' }`.
|
||||
|
||||
The [V2-to-V3 PTC specification](../../../../packages/session/session-format-v2-to-v3/README.md#ptc-vocabulary) owns exact historical tag and attribution conversion; [native V3 admission](../../../../packages/session/session-format-v2-to-v3/README.md#native-v3-admission) owns predecessor-tag refusal. These are not runtime aliases: an opaque extension must not acquire PTC lifecycle meaning merely through a version change.
|
||||
|
||||
@@ -78,7 +78,7 @@ Program outcomes resolve as result fields; caller misuse may reject. Lossless bi
|
||||
|
||||
### The Node process runtime
|
||||
|
||||
The Node provider strips erasable TypeScript, resolves its executable/bootstrap in the configured filesystem/subprocess world, and launches through the shared sandbox and managed process owners. Each child has an empty model environment and fresh program globals. Typed rejection constructors and the capturing console remain program-visible.
|
||||
The Node provider strips erasable TypeScript, resolves its executable/bootstrap in the configured filesystem/subprocess world, and launches through the shared sandbox and managed process owners. Each child has an empty model environment and fresh program globals. Typed rejection constructors and `inspect` are program-visible; optional console support shares the inspection function.
|
||||
|
||||
Binding traffic uses a dedicated control channel separate from stdout/stderr. The host validates frames, call identities and allowed bindings before dispatch. Output, pending calls and channel buffers have configured limits; completion, deadline, cancellation and protocol failure terminate and await the managed range. The provider README and sandboxed Node decision own exact bounds and platform limits.
|
||||
|
||||
@@ -88,7 +88,7 @@ The selected OS backend constrains direct Node effects under the same policy use
|
||||
|
||||
### What the model sees
|
||||
|
||||
The SDK instructs the model to write an async body in the loaded runtime's language (an erasable-TypeScript body by default; a Python `async` body under a Python runtime — see the [language-dispatch note](../../archived/feature/2026-07-31-ptc-language-dispatch.md)), call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Both flavors state the same contract in their own primitive: independent read-only calls MAY overlap under `Promise.all` (TypeScript) or `asyncio.gather` (Python), mutating calls run alone in submission order, and dependent work sequences with `await`. The declaration prefix can be as large as native schemas, but remains stable for provider caching.
|
||||
The SDK declares the loaded runtime's source format and visible tool bindings, instructs the model to call tools through `await tools.name(args)` and catch rejected tool calls when needed, and presents language-specific output instructions. TypeScript selects output through `inspect`; the Python flavor retains return and print output. Independent read-only calls may overlap under `Promise.all` or `asyncio.gather`, mutating calls run alone in submission order, and dependent work sequences with `await`. The declaration prefix can be as large as native schemas, but remains stable for provider caching.
|
||||
|
||||
The transport's own `description` and both SDK instruction flavors open by naming `code` and `description` as the call's two required arguments. Prose that describes the call as passing a program leaves the second argument discoverable only through the parameter schema, and a model that emits `{code}` alone loses the whole written program to an `INVALID_ARGS` rejection.
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一
|
||||
2. **代码执行是一个能力 seam**——`packages/ptc-runtime/` 包含 Service Definition 包 `@deepseek-ai/dsh-ptc-runtime`,拥有 `ctx.ptcRuntime`([能力 seam](../architecture/2026-06-13-capability-seams.zh.md);消费方 = `dsh-tools`,core 消费 seam 的先例见 `agent-loop` → `dsh-llm`)。运行时对工具一无所知:它接收一段程序和命名的异步绑定,执行程序,报告 `{ value, logs, error? }`。语言和基底是后端属性,因此未来的 Python 或容器后端只是另一个 Service Provider 包,而非重新设计。
|
||||
3. **交付的实现是 `@deepseek-ai/dsh-ptc-runtime-node`**:每个程序在全新的受管 Node 进程中,按调用 Session 的已解析沙箱策略运行。直接 Node API 在该策略内仍可使用;具名 Host 绑定通过单独验证的控制通道调用。
|
||||
|
||||
本说明负责 PTC 呈现、组合与结算。[类型化工具返回决策](2026-07-20-ptc-typed-tool-returns.zh.md)负责生成输出、规范绑定值、`ToolCallError` 与无损外层输出。[沙箱 Node 决策](../architecture/2026-09-11-sandboxed-node-ptc-runtime.zh.md)取代 worker 执行、信任姿态与预算实现,同时保留本说明的消费方规则。
|
||||
本说明负责 PTC 呈现、组合与结算。[类型化工具返回决策](2026-07-20-ptc-typed-tool-returns.zh.md)负责生成输出、普通规范绑定值、`ToolCallError` 与无损运行时输出。[显式输出决策](../architecture/2026-11-09-explicit-ptc-output.zh.md)负责 TypeScript 完成值抑制及文本/图片选择。[沙箱 Node 决策](../architecture/2026-09-11-sandboxed-node-ptc-runtime.zh.md)取代 worker 执行、信任姿态与预算实现,同时保留本说明的消费方规则。
|
||||
|
||||
### 每段程序的执行控制
|
||||
|
||||
@@ -48,9 +48,9 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一
|
||||
|
||||
在 `'ptc'` 下,注册表拥有 `run_code` 作为保留的呈现传输通道,带两个必需参数 `{ code: string; description: string }`(description 为 UI 标注该调用,沿用 bash 的先例)。它由一个正常的 `ToolDefinition` 表示以供分发,但位于可过滤的能力层之外,因此限制规则不会意外移除 PTC mode 的唯一入口。调用遍历完整的工具流水线——`tools/pre-execute` → 单调性守卫 → `tools/execute` 包裹分发 → `tools/post-execute` → 由定义拥有的可选 `finalizeContent` → 不可变的 `tools/result` 通知——与原生调用完全一致;权限插件可以在程序运行前检查程序文本,最终结果观察者看到的是规范化的外层结果。其 `execute(args, exec)`:
|
||||
|
||||
1. **构建绑定。** 一个 run 级别的 signal 跟随外层取消,并在 run 结算时被 abort。每个可见工具绑定都会对无损 JSON 参数创建快照,进入原生约定的分发池(调度设计由[实时并行 参考](../../../../packages/core/tools/README.zh.md) 负责),以确定性的 call id 和外层 token 作为 `parent` 执行,通过外层 execution 延后返回的上下文,并记录 `tool/ptc-dispatch-start`/`tool/ptc-dispatch` 事件对,其中结算侧携带完整渲染后的结果内容。成功时返回工具最终的规范 JSON 值;失败则变为程序可见的 `ToolCallError`。每个子调用保留自己不可变的执行标识,并遍历完整的工具流水线。
|
||||
1. **构建绑定。** 一个 run 级别的 signal 跟随外层取消,并在 run 结算时被 abort。每个可见工具绑定都会对无损 JSON 参数创建快照,进入原生约定的分发池(调度设计由[实时并行 参考](../../../../packages/core/tools/README.zh.md) 负责),以确定性的 call id 和外层 token 作为 `parent` 执行,通过外层 execution 延后返回的上下文,并记录 `tool/ptc-dispatch-start`/`tool/ptc-dispatch` 事件对,其中结算侧携带完整渲染后的结果内容。成功时返回面向消费方的值,包括指定 TypeScript 绑定的程序 Image;失败则变为程序可见的 `ToolCallError`。每个子调用保留自己不可变的执行标识,并遍历完整的工具流水线。
|
||||
2. **运行已解析输入:** `ctx.ptcRuntime.run(ctx.ptcRuntime.resolve(request))`。请求携带程序、绑定、运行级信号、调用 Session 的 cwd 与适用的已解析文件策略。运行级信号将外层结算与执行取消关联。
|
||||
3. **完全停稳后结算。** 运行时结算后,桥 abort 未完成的工作并排空分发队列后再返回。成功时返回捕获的日志和完成值,将其作为规范输出;注册表再把该值渲染为持久化的 `tool/result.content`,供结果卡片直接读取。运行时失败变为 `CodeRunFailedError`;后端拒绝使用注册表的正常错误边界。两者都产生结构化的错误结果,且 `run_code` 结算后不允许子调用追加。
|
||||
3. **完全停稳后结算。** 运行时结算后,桥接层 abort 未完成的工作并排空分发队列后再返回。TypeScript 成功结果将捕获日志作为规范输出,并将显式展示的图片投影到持久化的 `tool/result.content`;其他语言消费方还会渲染完成值。结果卡片直接读取该内容。运行时失败变为 `CodeRunFailedError`;后端拒绝使用注册表的正常错误边界。两者都产生结构化的错误结果,且 `run_code` 结算后不允许子调用追加。
|
||||
|
||||
**子调用上下文通过父调用延后。** 在 `run_code` 内部注入会破坏父调用/结果的相邻性,因此 `ToolRunContext.deferContext()` 按分发顺序收集每个子结果的 `additionalContexts` 条目。即使程序后来抛出异常,注册表仍携带该数组;循环只在外层结果与步骤中所有兄弟结果之后追加每个条目。外层 post-execute 阻止会丢弃工具延后的条目,只暴露阻止 decision 显式附加的上下文。
|
||||
|
||||
@@ -58,13 +58,13 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一
|
||||
|
||||
**工具调用额度。** 在验证和排队之前对尝试计数,可以限制重复失败或持续入队的循环。额度随最终超时时间缩放,允许集中调用,不实施滑动窗口限制。额度耗尽时使用运行级取消路径,使程序无法捕获绑定错误后继续运行。活动调用被取消,排队调用被放弃,结算会等待清理完成。已完成的操作仍然生效;新运行获得完整额度。[执行器 README](../../../../packages/core/tool-executor-ptc/README.zh.md#use-this-package) 负责配置与失败报告。
|
||||
|
||||
**呈现。** `run_code` 的 render intent 按[呈现意图 Agent Note](../architecture/2026-07-02-tool-render-intent-union.zh.md)在此决定:`presentCall` 创建一个 `generic` 卡片,`kind: 'execute'`,以程序文本作为标题,并将同一程序文本作为 `rawInput`;`run_code` 有意不声明 `presentResult`,因此 TUI 和宿主/客户端运行时(Web)会通过通用原始内容回退机制,使用最终持久化的 `tool/result.content` 补全该卡片,其中包括捕获的日志,以及返回值、失败信息或 post-policy spill 预览。这不是 `terminal` 卡片:该卡片的语义是「工作目录中的 shell 命令」,程序不是。参见[结果卡片完整性说明](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md)。
|
||||
**呈现。** `run_code` 使用 `kind: 'execute'` 的 `generic` 卡片,标题为模型填写的描述,`rawInput` 为提交的程序。UI 适配器读取持久化的 `tool/result.content`,展示捕获文本、所选图片、失败或 post-policy spill 预览。[显式输出决策](../architecture/2026-11-09-explicit-ptc-output.zh.md)负责这些内容的选择。
|
||||
|
||||
### 可观测性:`tool/ptc-dispatch`
|
||||
|
||||
每次子分发在进入分发池时追加一个仅日志的 `tool/ptc-dispatch-start` 事件,并以一个 `tool/ptc-dispatch` 结算事件收尾,后者包含父子 call id、工具标识、规范化参数以及完整渲染后的 `content`/`isError` 结果。它不进入模型历史,但可供持久化和 UI 使用。追加发生在开放的 `run_code` 轮次内。没有 agent 的直接执行仍然运行,但无法记录该事件。
|
||||
|
||||
新子调用使用 `<parent>:ptc:<n>` 标识,按工具调用尝试编号。无效参数会占用编号而不产生分发事件,因此日志中的编号可能有空缺。消费者将所有 call id 视为不透明值:迁移逐字节保留每个历史标识,包括 `:code:` 子串,因此分发事件对、spill 引用及其他关联保持完整。桥接层将转发图片上下文的来源标记为 `{ kind: 'plugin', plugin: 'tools-ptc' }`。
|
||||
新子调用使用 `<parent>:ptc:<n>` id,按工具调用尝试编号。无效参数在没有分发事件时仍消耗编号,因此日志中的 id 可能有空缺。所有调用 id 对消费方均不透明:迁移逐字节保留每个历史 id,包括 `:code:` 子串,使分发事件对、spill 引用和其他关联保持完整。[显式输出决策](../architecture/2026-11-09-explicit-ptc-output.zh.md)负责 TypeScript 图片呈现;其他语言消费方将转发的图片上下文归属为 `{ kind: 'ptc-mode' }`。
|
||||
|
||||
[V2 到 V3 PTC 规范](../../../../packages/session/session-format-v2-to-v3/README.zh.md#ptc-vocabulary)负责精确的历史标签与归属转换;[原生 V3 准入](../../../../packages/session/session-format-v2-to-v3/README.zh.md#native-v3-admission)负责前代标签拒绝。这些不是运行时别名:不透明扩展不能仅因版本变化就获得 PTC 生命周期含义。
|
||||
|
||||
@@ -78,7 +78,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一
|
||||
|
||||
### Node 进程运行时
|
||||
|
||||
Node 提供方擦除可擦除 TypeScript,在配置的文件系统/子进程世界中解析可执行文件与 bootstrap,并通过共享沙箱与受管进程所有者启动。每个子进程都有空模型环境和全新程序全局对象。类型化拒绝构造器与捕获 console 仍对程序可见。
|
||||
Node 提供方擦除可擦除 TypeScript,在配置的文件系统/子进程世界中解析可执行文件与 bootstrap,并通过共享沙箱与受管进程所有者启动。每个子进程都有空模型环境和全新程序全局对象。类型化拒绝构造器与 `inspect` 对程序可见;可选 console 支持复用展示函数。
|
||||
|
||||
绑定通信使用与 stdout/stderr 分开的专用控制通道。Host 在分派前验证帧、调用身份与允许的绑定。输出、待处理调用与通道缓冲有配置上限;完成、截止、取消和协议失败会终止并等待受管范围。提供方 README 与沙箱 Node 决策负责确切上限和平台限制。
|
||||
|
||||
@@ -88,7 +88,7 @@ Node 提供方擦除可擦除 TypeScript,在配置的文件系统/子进程
|
||||
|
||||
### 模型看到的内容
|
||||
|
||||
SDK 指示模型编写一个所加载运行时语言的异步函数体(默认可擦除 TypeScript;Python 运行时下为 Python `async` 函数体——见[语言分发 note](../../archived/feature/2026-07-31-ptc-language-dispatch.md)),通过 `await tools.name(args)` 调用工具,在需要时捕获被拒绝的工具调用,并仅 return 或 log 应重新进入上下文的输出。两种 flavor 用各自的原语陈述同一约定:相互独立的只读调用可以(MAY)在 `Promise.all`(TypeScript)或 `asyncio.gather`(Python)下重叠,有副作用的调用按提交顺序单独运行,有依赖的工作用 `await` 排序。声明前缀可能与原生 schema 一样大,但对提供方缓存保持稳定。
|
||||
SDK 声明所加载运行时的源码格式和可见工具绑定,指示模型通过 `await tools.name(args)` 调用工具、在需要时捕获被拒绝的工具调用,并提供各语言的输出说明。TypeScript 通过 `inspect` 选择输出;Python flavor 保留 return 和 print 输出。相互独立的只读调用可通过 `Promise.all` 或 `asyncio.gather` 重叠,有副作用的调用按提交顺序单独运行,有依赖的工作用 `await` 排序。声明前缀可能与原生 schema 一样大,但对提供方缓存保持稳定。
|
||||
|
||||
传输自身的 `description` 与两种 flavor 的 SDK 说明都以点名 `code` 和 `description` 这两个必填参数开头。把该调用描述成「传入一个程序」的散文会让第二个参数只能从参数 schema 中发现,而只发出 `{code}` 的模型会因 `INVALID_ARGS` 被拒,连同已写好的整个程序一起丢失。
|
||||
|
||||
|
||||
@@ -9,32 +9,32 @@
|
||||
en: e2403bc358c4e14b
|
||||
zh: dd37923097a3b24c
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/decision:
|
||||
en: 2218fac13ef57c77
|
||||
zh: d4a18daa9debc77c
|
||||
en: 6dd62ee16c69536c
|
||||
zh: aec9fa1c74be28e5
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/decision/generated-sdk:
|
||||
en: 30583e9ac6715952
|
||||
zh: 3a85c9482abf574c
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/decision/binding-values-and-failures:
|
||||
en: ecb95cafa2dc1297
|
||||
zh: ccdf300ca936ca12
|
||||
en: 8b10f08e67ca9323
|
||||
zh: e57adb520049f407
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/decision/outer-result-and-output-ledger:
|
||||
en: d907d0a8205157e1
|
||||
zh: 1ed99168266a7ca4
|
||||
en: d57456d76fed9c6d
|
||||
zh: 453775faad48588c
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/decision/typed-handles-and-lifetime:
|
||||
en: b2b9318c2812e3a4
|
||||
zh: 73cf34bfd487e2f1
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/decision/persistence-metadata-and-spill:
|
||||
en: c388619f862a92ba
|
||||
zh: d557eae40c7b6b7a
|
||||
en: 2c3ad4ec376ed2d5
|
||||
zh: 49e8206a4c17b1b3
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/testing:
|
||||
en: 728513c779f031f8
|
||||
zh: f6d869f18add7242
|
||||
en: 81a4355914c918cf
|
||||
zh: 36aa414cd382ff9c
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/alternatives-considered:
|
||||
en: dedd87b9b35d8fab
|
||||
zh: 78db4c7e8c4e5f61
|
||||
en: 069a1e170c7d1dfb
|
||||
zh: 14ce5a56e602595c
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/consequences:
|
||||
en: f5726b5e7a2fc73f
|
||||
zh: 0b9aa884db5d920f
|
||||
/agent-note-typed-tool-returns-in-ptc-mode/known-limitations-and-deferred-work:
|
||||
en: a42af629ad6872cc
|
||||
zh: 41e5c0bb3de0ae4d
|
||||
en: 19828c2e5796df2d
|
||||
zh: 2ef63df81fca7a72
|
||||
@@ -14,7 +14,7 @@ The [canonical tool-output contract](../../../../packages/core/tools/README.md)
|
||||
|
||||
## Decision
|
||||
|
||||
PTC mode is a typed projection of the visible tool registry. Each successful binding resolves to the final canonical `JsonValue` after post-execute policy, while a failed binding rejects with a real `ToolCallError`. Intermediate values remain inside the run and cross the worker boundary whole. The outer `run_code` logs, completion value, or failure diagnostic enter the configurable output ledger and model-facing spill pipeline; a successfully settled sub-call whose final Native content contains an image additionally defers that complete ordered content through the parent result as logged, source-attributed context.
|
||||
PTC mode is a typed projection of the visible tool registry. Ordinary successful bindings resolve to the final canonical `JsonValue` after post-execute policy, while failed bindings reject with a real `ToolCallError`. Intermediate values remain inside the run and cross the process boundary whole. The [explicit output decision](../architecture/2026-11-09-explicit-ptc-output.md) owns TypeScript Image bindings, completion suppression and selection of visible text and images. Other language consumers retain completion rendering and deferred image-bearing content.
|
||||
|
||||
This note owns the return and failure contract layered on the original [PTC mode foundation](2026-06-15-ptc.md). The unified schema vocabulary is owned by the [JSON-value schema DSL reference](../../../../packages/core/tools/README.md), and Native rendering and policy projection remain owned by the canonical-output note.
|
||||
|
||||
@@ -49,15 +49,15 @@ declare const tools: {
|
||||
|
||||
### Binding values and failures
|
||||
|
||||
Before dispatch the bridge snapshots binding arguments as lossless JSON and snapshots the detached value again for an independent durable summary event. Host-side detachment, immutable execution, and output-schema projection all use iterative traversals rather than nested structured clone or recursive freezing. `undefined`, non-finite numbers, `-0`, sparse arrays, cycles, functions, and exotic objects reject that call before the tool runs. Successful dispatch returns `ToolExecutionResult.value`; Native `content`, metadata, and internal error information do not cross to the program. Image-bearing final content is not a second binding value: the bridge ferries it after the outer result so the next model request can see the durable image, while post-execute block/content replacement remains authoritative and text-only results are not duplicated.
|
||||
Before dispatch the bridge snapshots binding arguments as lossless JSON and snapshots the detached value again for an independent durable summary event. Host-side detachment, immutable execution, and output-schema projection all use iterative traversals rather than nested structured clone or recursive freezing. `undefined`, non-finite numbers, `-0`, sparse arrays, cycles, functions, and exotic objects reject that call before the tool runs. Ordinary successful dispatch returns `ToolExecutionResult.value`; the selected TypeScript image bindings use the final image attachment to construct a program Image. Post-execute blocking and result projection remain authoritative. Native content envelopes, metadata and internal error information stay outside ordinary binding values.
|
||||
|
||||
PTC mode declares its rejection capability on the runtime request as `{ name: "ToolCallError", memberNameProperty: "toolName" }`. The runtime Service Definition treats those names as data: the worker materializes and injects the actual constructor used for `tools` binding failures, so `error instanceof ToolCallError` works without making a generic runtime know about tools. The worker constructs failures and defines their public fields through module-captured error and property-definition intrinsics plus null-prototype descriptors, so model mutations cannot replace the promised rejection with a worker failure. The error has the standard `Error` message plus the exact `toolName`; it deliberately omits `ToolFailure.info`, error codes, and Native content. This is an exception contract for control flow, not a failure union for programmatic classification.
|
||||
|
||||
Binding arguments and resolutions are revalidated as lossless JSON on both sides of the hostile worker protocol and have no byte cap. Before crossing through structured clone, each detached value is encoded as a flat pre-order token stream whose transport nesting is bounded; the receiver rebuilds it iteratively. Valid application nesting therefore has neither a JavaScript call-stack depth cap nor a platform-specific nested structured-clone limit. At module initialization the worker captures its own realm's `Array.prototype` and `Object.prototype` identities, the native function-source intrinsic used only to recognize foreign-realm plain-container prototypes, and every structural and metering intrinsic used by the JSON boundary. Property writes use null-prototype descriptors, while private array and set operations invoke captured methods without consulting mutable global or prototype slots. Model code can therefore replace helpers such as `Object.keys`, `Array.isArray`, collection methods, string methods, or `Buffer.byteLength`, rewrite intrinsic-prototype constructor slots, or add descriptor-shaped fields to `Object.prototype` without changing validation, wire transport, or byte accounting. The foreign-realm native function-source check still rejects user-authored constructors that imitate `Object` or `Array`. The dependency-light runtime Service Definition names its structural equivalent `PtcJsonValue` so it need not depend on the session-owned canonical type; the generated SDK and tool API use `JsonValue`. Intermediate values are not prompt-truncated, context-spilled, or persisted. This preserves full acquired search, workflow, task, filesystem, and MCP values for programmatic filtering while leaving provider and executor acquisition limits truthful.
|
||||
Ordinary binding arguments and JSON resolutions are revalidated as lossless JSON on both sides of the hostile worker protocol and have no byte cap. Before crossing through structured clone, each detached value is encoded as a flat pre-order token stream whose transport nesting is bounded; the receiver rebuilds it iteratively. Valid application nesting therefore has neither a JavaScript call-stack depth cap nor a platform-specific nested structured-clone limit. At module initialization the worker captures its own realm's `Array.prototype` and `Object.prototype` identities, the native function-source intrinsic used only to recognize foreign-realm plain-container prototypes, and every structural and metering intrinsic used by the JSON boundary. Property writes use null-prototype descriptors, while private array and set operations invoke captured methods without consulting mutable global or prototype slots. Model code can therefore replace helpers such as `Object.keys`, `Array.isArray`, collection methods, string methods, or `Buffer.byteLength`, rewrite intrinsic-prototype constructor slots, or add descriptor-shaped fields to `Object.prototype` without changing validation, wire transport, or byte accounting. The foreign-realm native function-source check still rejects user-authored constructors that imitate `Object` or `Array`. The dependency-light runtime Service Definition names its structural equivalent `PtcJsonValue` so it need not depend on the session-owned canonical type; the generated SDK and tool API use `JsonValue`. Intermediate values are not prompt-truncated, context-spilled, or persisted. This preserves full acquired search, workflow, task, filesystem, and MCP values for programmatic filtering while leaving provider and executor acquisition limits truthful.
|
||||
|
||||
### Outer result and output ledger
|
||||
|
||||
The runtime accepts an exact lossless JSON completion of any root. Returning `undefined` omits the completion; returning `null` is an explicit result. `run_code` exposes the canonical outer value `{ logs: string[], result?: JsonValue }`. Its Native renderer emits logs first, renders a string result raw, and renders every other JSON root with an iterative pretty printer. Total indentation is capped at ten characters and deeper subtrees remain compact, preserving the established shallow text while keeping traversal stack-safe and formatted size linear in the canonical JSON size.
|
||||
Runtime consumers that retain completions receive exact lossless JSON: `undefined` omits the completion and `null` is an explicit result. TypeScript `run_code` requests completion discard and returns captured logs plus optional sandbox facts as its canonical value; selected images enter its rendered content separately. Other language consumers also expose `result?: JsonValue`. Their renderer emits logs first, renders a string result raw, and renders every other JSON root with an iterative pretty printer. Total indentation is capped at ten characters and deeper subtrees remain compact, keeping traversal stack-safe and formatted size linear in the canonical JSON size.
|
||||
|
||||
`NodePtcRuntime` replaces the former independent log and value caps with configurable `maxOutputBytes`, defaulting to `67_108_864` bytes. The worker charges captured logs by their exact JSON-string serialization and preflights the detached completion or program exception against the remaining combined budget before posting a terminal message. A giant thrown string or stack therefore crosses the worker port only as the fixed `output-limit` diagnostic. The host repeats the hostile-peer ledger for forged traffic and native pipe writes the worker cannot observe. Fixed `PtcRunResult` field names, braces, the bounded error-kind tag, and later presentation whitespace are deliberately outside this variable-payload ledger. Neither stage materializes an over-limit serialized completion. A result at or below the cap is exact. A completion that cannot survive lossless JSON snapshotting fails as `invalid-output`; a value, diagnostic, or combined outcome over the cap fails as `output-limit` rather than becoming inspected or truncated text.
|
||||
|
||||
@@ -73,13 +73,13 @@ Temporary Cordis Plugins follow the same rule: `cordis_mount` returns `{ id, plu
|
||||
|
||||
### Persistence, metadata, and spill
|
||||
|
||||
Nested dispatch logs the sub-call's full rendered `content`/`isError` on `tool/ptc-dispatch` but does not persist canonical values. `tool/result` continues to persist only rendered content, error, and optional metadata. A successful final content sequence containing an image is also wrapped in a source-attributed user message and deferred through the outer result; the normal session event makes that model-visible input reconstructable. The [PTC mode note](2026-06-15-ptc.md) owns durable event names and historical identity preservation; replay cannot recreate intermediate canonical program values.
|
||||
Nested dispatch logs the sub-call's full rendered `content`/`isError` on `tool/ptc-dispatch` but does not persist canonical values. TypeScript programs select images for the outer `tool/result.content` through explicit inspection. Other language consumers defer successful image-bearing content as source-attributed user messages after the outer result. The normal Session events make these model-visible inputs reconstructable. The [PTC mode note](2026-06-15-ptc.md) owns durable event names and historical identity preservation; replay cannot recreate intermediate canonical program values.
|
||||
|
||||
The opaque `exec.parent` token marks nested calls. Presentation metadata and generic or tool-owned spill projections skip those calls; their canonical values never enter context. The Client can derive [nested terminal cards](../../../../packages/client/ui-tool/README.md) from raw dispatch events without metadata. The outer `run_code` call produces the model-facing result and may spill its final post-policy presentation; `run_code` intentionally declares neither a result presenter nor presentation metadata, so UI adapters complete the card through their generic raw-content fallback using durable `tool/result.content`.
|
||||
|
||||
## Testing
|
||||
|
||||
Compile-time and snapshot tests pin exact `ToolArgsMap`, `ToolOutputMap`, `ToolName`, schema-to-TypeScript coverage, exotic names, and assembled PTC mode image forwarding. Registry and real-worker tests cover scalar, array, object, and null values; raw string rendering; absent `undefined`; consumer-declared real rejection classes, including `ToolCallError`; invalid arguments and completions, including intrinsic-looking forged prototypes; model-mutated JSON-boundary globals, prototype methods, constructor slots, and inherited descriptor fields; typed binding failures after those mutations; large uncapped intermediate bindings; nested spill suppression; generic image-bearing context deferral plus post-execute replacement/block precedence; exact and over-limit 64 MiB accounting; combined logs/value/diagnostic accounting; giant thrown stacks; bounded failure spill; hostile forged traffic; and built-package execution.
|
||||
Compile-time and snapshot tests pin exact `ToolArgsMap`, `ToolOutputMap`, `ToolName`, schema-to-TypeScript coverage, exotic names, and assembled PTC mode image projection. Registry and real-worker tests cover scalar, array, object, and null values; raw string rendering; absent `undefined`; consumer-declared real rejection classes, including `ToolCallError`; invalid arguments and completions, including intrinsic-looking forged prototypes; model-mutated JSON-boundary globals, prototype methods, constructor slots, and inherited descriptor fields; typed binding failures after those mutations; large uncapped intermediate bindings; nested spill suppression; consumer-specific image projection plus post-execute replacement/block precedence; exact and over-limit 64 MiB accounting; combined logs/value/diagnostic accounting; giant thrown stacks; bounded failure spill; hostile forged traffic; and built-package execution.
|
||||
|
||||
Keyless real-worker integration tests pin the two handle workflows that prose results could not safely support. A background bash call returns its job id, the outer run settles, and a later run polls that id to completion; separate cases prove pre-abort creates no task, post-publication call abort preserves the task, foreground execution stays signal-coupled, and `job_kill` owns cancellation. A Cordis program reads an active or pending mount's id and `waitingFor` fields directly, unmounts by that id, and confirms removal without parsing rendered text.
|
||||
|
||||
@@ -93,9 +93,9 @@ Keyless real-worker integration tests pin the two handle workflows that prose re
|
||||
|
||||
**Silently inspect or truncate an oversized completion.** Rejected because changing a JSON value into a string is lossy and type-incorrect. The explicit `output-limit` failure lets the model choose a smaller result, while the retained logs and diagnostic can still use normal outer spill.
|
||||
|
||||
**Require each rich leaf tool to inspect `exec.parent` and defer itself.** Rejected because it couples leaf tools to PTC mode internals, duplicates policy handling, and misses future rich tools. The dispatch bridge owns generic forwarding from the already settled final result.
|
||||
**Require each rich leaf tool to inspect `exec.parent` and defer itself.** Rejected because it couples leaf tools to PTC mode internals and duplicates policy handling. The dispatch bridge owns consumer-specific image projection from the already settled final result.
|
||||
|
||||
**Expose Native rich content as part of every binding's canonical value.** Rejected because a canonical value is lossless JSON and tool-specific; attachment blocks are a model projection with durable lifecycle semantics. Keeping the value and projection separate preserves typed programs without dropping images from later model context.
|
||||
**Expose Native rich content as part of every binding's canonical value.** Rejected because a canonical value is lossless JSON and tool-specific; attachment blocks are a model projection with durable lifecycle semantics. Consumer-owned program Image bindings preserve typed tool values while leaving image selection to the program.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -111,6 +111,6 @@ The worker performs bounded-depth flat-wire transport and lossless validation bu
|
||||
- Intermediate values have no byte cap and can exhaust process or worker memory through retention, flat-wire copies, or structured-clone cost.
|
||||
- The 64 MiB hard cap applies only to the outer variable payloads, excluding fixed result-envelope syntax and presentation whitespace; spill cannot recover bytes rejected beyond that cap.
|
||||
- Provider or executor acquisition limits may already have discarded source data before a canonical value reaches PTC mode.
|
||||
- Unsupported MCP output schemas fall back to `JsonValue`; admitted MCP images use the generic deferred projection, while audio and embedded-resource payloads remain diagnostic-only.
|
||||
- Unsupported MCP output schemas fall back to `JsonValue`; image projection follows the consumer-specific rules above, while audio and embedded-resource payloads remain diagnostic-only.
|
||||
- Nested calls have no separate model-facing result; Client child cards do not add model context.
|
||||
- Code failures expose `ToolCallError` message and tool name only, without a programmatic error-code union.
|
||||
@@ -14,7 +14,7 @@ PTC mode 过去会把每个嵌套工具的结果从 `ContentBlock[]` 重新投
|
||||
|
||||
## 决策
|
||||
|
||||
PTC mode 是可见工具注册表的类型化投影。每个成功的绑定调用都会解析为 post-execute 策略处理后的最终规范 `JsonValue`,失败的绑定调用则会以真正的 `ToolCallError` 拒绝 Promise。中间值只存在于本次运行中,并完整跨越 worker 边界。外层 `run_code` 的日志、完成值或失败诊断会进入可配置的输出账本以及面向模型的输出落盘流水线;如果成功结算的子调用最终 Native 内容包含图片,其完整有序内容还会经父结果延后为写入日志且带来源归属的上下文。
|
||||
PTC mode 是可见工具注册表的类型化投影。普通成功绑定调用解析为 post-execute 策略处理后的最终规范 `JsonValue`,失败的绑定调用则以真正的 `ToolCallError` 拒绝 Promise。中间值保留在运行中,并完整跨越进程边界。[显式输出决策](../architecture/2026-11-09-explicit-ptc-output.zh.md)负责 TypeScript Image 绑定、完成值抑制及可见文本和图片的选择。其他语言消费方保留完成值渲染和含图片内容的延后转发。
|
||||
|
||||
本文档定义叠加在原始 [PTC mode 基础](2026-06-15-ptc.zh.md)之上的返回值与失败约定。统一 schema 词汇由 [JSON 值 schema DSL reference](../../../../packages/core/tools/README.zh.md)负责定义;Native 渲染与策略投影仍由规范输出 Agent Note 负责定义。
|
||||
|
||||
@@ -49,15 +49,15 @@ declare const tools: {
|
||||
|
||||
### 绑定值与失败
|
||||
|
||||
分发前,桥接层会把绑定参数快照为无损 JSON,再对分离后的值生成一次快照,供独立的持久摘要事件使用。宿主侧的值分离、执行数据的不可变处理与输出 schema 投影均采用迭代遍历,而不使用嵌套结构化克隆或递归冻结。`undefined`、非有限数、`-0`、稀疏数组、循环引用、函数和非普通对象都会使该调用在工具运行前被拒绝。成功分发会返回 `ToolExecutionResult.value`;Native `content`、元数据和内部错误信息不会传入程序。含图片的最终内容不是第二份绑定值:桥接层会在外层结果之后转运它,使下一次模型请求可以看到持久图片;post-execute 阻止/内容替换仍具有权威性,纯文本结果不会重复。
|
||||
分发前,桥接层会把绑定参数快照为无损 JSON,再对分离后的值生成一次快照,供独立的持久摘要事件使用。宿主侧的值分离、执行数据的不可变处理与输出 schema 投影均采用迭代遍历,而不使用嵌套结构化克隆或递归冻结。`undefined`、非有限数、`-0`、稀疏数组、循环引用、函数和非普通对象都会使该调用在工具运行前被拒绝。普通成功分发返回 `ToolExecutionResult.value`;指定的 TypeScript 图片绑定使用最终图片附件构造程序 Image。Post-execute 阻止和结果投影仍具有最终决定权。普通绑定值不包含 Native 内容信封、元数据和内部错误信息。
|
||||
|
||||
PTC mode 通过运行时请求中的 `{ name: "ToolCallError", memberNameProperty: "toolName" }` 声明其以异常拒绝 Promise 的能力。运行时 Service Definition 只把这些名称视为数据:worker 会动态生成并注入真正用于 `tools` 绑定失败的构造函数,因此无需让通用运行时了解工具,`error instanceof ToolCallError` 也能成立。worker 使用模块初始化时捕获的 Error 构造函数与属性定义内建方法,配合原型为 null 的属性描述符,构造失败对象并定义其公开字段,因此模型代码的修改不会把约定承诺的 reject 变成 worker 失败。该错误包含标准的 `Error` 消息和确切的 `toolName`,并有意省略 `ToolFailure.info`、错误代码与 Native 内容。这是一项用于控制流的异常约定,而不是供程序分类的失败联合。
|
||||
|
||||
绑定参数与绑定返回值会在不可信 worker 协议的两端重新校验为无损 JSON,且不设字节上限。每个分离后的值在通过结构化克隆跨越边界前,都会编码为扁平的前序 token 流,其传输结构的嵌套深度有界;接收方再以迭代方式重建该值。因此,有效应用数据的嵌套深度既不受 JavaScript 调用栈深度上限限制,也不受特定平台对嵌套结构化克隆施加的上限限制。模块初始化时,worker 会捕获自身 JavaScript 运行域中 `Array.prototype` 和 `Object.prototype` 的引用、仅用于识别其他运行域普通容器原型、可获取原生函数源码的内建函数,以及 JSON 边界用于结构处理和计量的全部内建方法。属性写入使用原型为 null 的属性描述符;内部的数组与集合操作直接调用捕获的方法,不会访问可变的全局或原型槽位。因此,即使模型代码替换 `Object.keys`、`Array.isArray`、集合方法、字符串方法或 `Buffer.byteLength` 等辅助方法,重写内建原型的构造函数槽位,或向 `Object.prototype` 添加形如属性描述符的字段,也不会改变校验、协议传输或字节计量。面向其他运行域的原生函数源码检查仍会拒绝由用户编写、冒充 `Object` 或 `Array` 的构造函数。为保持依赖轻量,运行时 Service Definition 将结构等价类型命名为 `PtcJsonValue`,从而无需依赖会话侧拥有的规范类型;生成的 SDK 和工具 API 则使用 `JsonValue`。这些值不会经过提示词截断、上下文 spill 或持久化。因此,程序可以完整筛选已经采集的搜索、工作流、任务、文件系统与 MCP 值,同时提供方和执行器的采集上限仍会实际生效。
|
||||
普通绑定参数与 JSON 返回值会在不可信 worker 协议的两端重新校验为无损 JSON,且不设字节上限。每个分离后的值在通过结构化克隆跨越边界前,都会编码为扁平的前序 token 流,其传输结构的嵌套深度有界;接收方再以迭代方式重建该值。因此,有效应用数据的嵌套深度既不受 JavaScript 调用栈深度上限限制,也不受特定平台对嵌套结构化克隆施加的上限限制。模块初始化时,worker 会捕获自身 JavaScript 运行域中 `Array.prototype` 和 `Object.prototype` 的引用、仅用于识别其他运行域普通容器原型、可获取原生函数源码的内建函数,以及 JSON 边界用于结构处理和计量的全部内建方法。属性写入使用原型为 null 的属性描述符;内部的数组与集合操作直接调用捕获的方法,不会访问可变的全局或原型槽位。因此,即使模型代码替换 `Object.keys`、`Array.isArray`、集合方法、字符串方法或 `Buffer.byteLength` 等辅助方法,重写内建原型的构造函数槽位,或向 `Object.prototype` 添加形如属性描述符的字段,也不会改变校验、协议传输或字节计量。面向其他运行域的原生函数源码检查仍会拒绝由用户编写、冒充 `Object` 或 `Array` 的构造函数。为保持依赖轻量,运行时 Service Definition 将结构等价类型命名为 `PtcJsonValue`,从而无需依赖会话侧拥有的规范类型;生成的 SDK 和工具 API 则使用 `JsonValue`。这些值不会经过提示词截断、上下文 spill 或持久化。因此,程序可以完整筛选已经采集的搜索、工作流、任务、文件系统与 MCP 值,同时提供方和执行器的采集上限仍会实际生效。
|
||||
|
||||
### 外层结果与输出账本
|
||||
|
||||
运行时接受以任意 JSON 类型为根的精确无损完成值。返回 `undefined` 表示省略完成值;返回 `null` 则是显式结果。`run_code` 暴露规范外层值 `{ logs: string[], result?: JsonValue }`。其 Native 渲染器先输出日志;字符串结果保持原文,其他所有 JSON 根值则使用迭代式美化渲染器。总缩进长度上限为 10 个字符,更深的子树保持紧凑格式,既保留既有的浅层文本,又确保遍历不受调用栈深度限制,且格式化输出大小与规范 JSON 大小呈线性关系。
|
||||
保留完成值的运行时消费方接收精确无损 JSON:`undefined` 省略完成值,`null` 是显式结果。TypeScript `run_code` 请求丢弃完成值,将捕获日志和可选沙箱信息作为规范值返回;所选图片单独进入渲染内容。其他语言消费方还提供 `result?: JsonValue`。其渲染器先输出日志,字符串结果保持原文,其他 JSON 根值使用迭代式美化渲染器。总缩进长度上限为十个字符,更深子树保持紧凑格式,使遍历不受调用栈深度限制,且格式化大小与规范 JSON 大小呈线性关系。
|
||||
|
||||
`NodePtcRuntime` 以可配置的 `maxOutputBytes` 取代彼此独立的日志与值上限,默认值为 `67_108_864` 字节。worker 会将已捕获日志序列化为 JSON 字符串后的精确字节数计入账本,并在发送终态消息前,根据组合账本的剩余额度预检分离后的完成值或程序异常。因此,即使抛出的字符串或堆栈极大,通过 worker 端口的也只会是固定的 `output-limit` 诊断。宿主侧会针对伪造流量以及 worker 无法观察的原生管道写入,重复执行这套面向不可信对端的账本校验。固定的 `PtcRunResult` 字段名、花括号、有界的错误类型标签及后续展示空白有意不计入这份可变负载账本。这两个阶段都不会实际生成超出上限的完成值序列化结果。结果不超过上限时会保持精确。完成值无法通过无损 JSON 快照时,以 `invalid-output` 失败;值、诊断或包含日志的组合结果超过上限时,以 `output-limit` 失败,而不会变成检查格式化后或截断的文本。
|
||||
|
||||
@@ -73,13 +73,13 @@ PTC mode 通过运行时请求中的 `{ name: "ToolCallError", memberNamePropert
|
||||
|
||||
### 持久化、元数据与 spill
|
||||
|
||||
嵌套分发在 `tool/ptc-dispatch` 上记录子调用完整渲染后的 `content`/`isError`,但不会持久化规范值。`tool/result` 继续只持久化渲染后的内容、错误和可选元数据。包含图片的成功最终内容序列还会包装成带来源归属的用户消息,并经外层结果延后;普通会话事件使该模型可见输入可以重建。[PTC mode note](2026-06-15-ptc.zh.md) 负责持久事件名称与历史标识保留规则;回放无法重建程序的规范中间值。
|
||||
嵌套分发在 `tool/ptc-dispatch` 上记录子调用完整渲染后的 `content`/`isError`,但不会持久化规范值。TypeScript 程序通过显式展示为外层 `tool/result.content` 选择图片。其他语言消费方将成功的含图片内容作为带来源归属的用户消息,延后到外层结果之后。普通 Session 事件使这些模型可见输入可以重建。[PTC mode note](2026-06-15-ptc.zh.md)负责持久事件名称与历史标识保留规则;回放无法重建程序的规范中间值。
|
||||
|
||||
不透明的 `exec.parent` token 用于标识嵌套调用。展示元数据以及通用或工具自有的 spill 投影都会跳过这些调用;它们的规范值永远不会进入上下文。Client 可以从原始分发事件派生[嵌套 terminal 卡片](../../../../packages/client/ui-tool/README.zh.md),无需元数据。外层 `run_code` 调用产生面向模型的结果,并且可能对 post-policy 处理后的最终展示执行 spill;`run_code` 有意既不声明结果展示器,也不声明展示元数据,因此 UI 适配器会通过通用的原始内容回退机制,使用持久化的 `tool/result.content` 补全该卡片。
|
||||
|
||||
## 测试
|
||||
|
||||
编译期测试与快照测试锁定了精确的 `ToolArgsMap`、`ToolOutputMap`、`ToolName`、schema 到 TypeScript 的覆盖范围、特殊名称,以及组装后的 PTC mode 图片转发。注册表与真实 worker 测试覆盖标量、数组、对象和 null 值;字符串原文渲染;缺席的 `undefined`;消费方声明、实际用于拒绝 Promise 的异常类,包括 `ToolCallError`;无效参数与完成值,包括伪装为内建原型的伪造原型;模型代码修改过的 JSON 边界全局对象、原型方法、构造函数槽位,以及继承而来的属性描述符字段;上述修改后的类型化绑定失败;不设上限的大型中间绑定值;嵌套输出落盘抑制;通用含图片上下文延后以及 post-execute 替换/阻止优先级;64 MiB 上限内外的精确计量;日志、值与诊断的组合计量;抛出的超大堆栈;有界失败的输出落盘;不可信对端伪造的流量;以及构建后包的执行。
|
||||
编译期测试与快照测试锁定了精确的 `ToolArgsMap`、`ToolOutputMap`、`ToolName`、schema 到 TypeScript 的覆盖范围、特殊名称,以及组装后的 PTC mode 图片投影。注册表与真实 worker 测试覆盖标量、数组、对象和 null 值;字符串原文渲染;缺席的 `undefined`;消费方声明、实际用于拒绝 Promise 的异常类,包括 `ToolCallError`;无效参数与完成值,包括伪装为内建原型的伪造原型;模型代码修改过的 JSON 边界全局对象、原型方法、构造函数槽位,以及继承而来的属性描述符字段;上述修改后的类型化绑定失败;不设上限的大型中间绑定值;嵌套输出落盘抑制;按消费方规则投影图片以及 post-execute 替换/阻止优先级;64 MiB 上限内外的精确计量;日志、值与诊断的组合计量;抛出的超大堆栈;有界失败的输出落盘;不可信对端伪造的流量;以及构建后包的执行。
|
||||
|
||||
无密钥的真实 worker 集成测试锁定了自然语言结果无法安全支持的两种句柄工作流。后台 bash 调用返回 job id,外层运行结束,之后的运行再根据该 id 轮询直至任务完成;其他用例分别证明,预先中止不会创建任务、发布后的调用取消会保留任务、前台执行仍与信号耦合,并且由 `job_kill` 负责取消。Cordis 程序会直接读取 active 或 pending 挂载的 id 和 `waitingFor` 字段,按该 id 卸载,并在不解析渲染文本的情况下确认挂载已移除。
|
||||
|
||||
@@ -93,9 +93,9 @@ PTC mode 通过运行时请求中的 `{ name: "ToolCallError", memberNamePropert
|
||||
|
||||
**静默检查格式化或截断过大的完成值:**不予采纳。把 JSON 值改成字符串既有损又违反类型。显式的 `output-limit` 失败让模型可以选择返回更小的结果,而保留的日志和诊断仍可使用普通的外层 spill 机制。
|
||||
|
||||
**要求每个丰富叶子工具检查 `exec.parent` 并自行延后。** 不予采用,因为这会把叶子工具与 PTC mode 内部机制耦合、重复策略处理,并遗漏未来丰富工具。分发桥接层负责从已经结算的最终结果通用转发。
|
||||
**要求每个丰富叶子工具检查 `exec.parent` 并自行延后。** 不予采用,因为这会把叶子工具与 PTC mode 内部机制耦合,并重复策略处理。分发桥接层负责按消费方规则,从已经结算的最终结果投影图片。
|
||||
|
||||
**把 Native 丰富内容暴露为每个绑定规范值的一部分。** 不予采用,因为规范值是无损 JSON 且由工具定义;附件块是具有持久生命周期语义的模型投影。保持值与投影分离,既能保留类型化程序,也不会从后续模型上下文中丢弃图片。
|
||||
**把 Native 丰富内容暴露为每个绑定规范值的一部分。** 不予采用,因为规范值是无损 JSON 且由工具定义;附件块是具有持久生命周期语义的模型投影。消费方拥有的程序 Image 绑定保留类型化工具值,并将图片选择交给程序。
|
||||
|
||||
## 后果
|
||||
|
||||
@@ -111,6 +111,6 @@ worker 会以嵌套深度有界的扁平协议格式传输数据并执行无损
|
||||
- 中间值没有字节上限,可能因值的保留、扁平协议格式副本或结构化克隆开销而耗尽进程或 worker 内存。
|
||||
- 64 MiB 硬上限只适用于外层可变负载,不计固定的结果封装语法与展示空白;spill 无法恢复超出该上限后被拒绝的字节。
|
||||
- 提供方或执行器的采集上限可能在规范值到达 PTC mode 前就已丢弃部分源数据。
|
||||
- 不支持的 MCP 输出 schema 会回退为 `JsonValue`;已准入的 MCP 图片使用通用延后投影,而音频和嵌入资源载荷仍只提供诊断。
|
||||
- 不支持的 MCP 输出 schema 回退为 `JsonValue`;图片投影遵循上述消费方规则,音频和嵌入资源负载仍仅用于诊断。
|
||||
- 嵌套调用没有独立的面向模型的结果;Client 子调用卡片不会增加模型上下文。
|
||||
- PTC mode 失败只暴露 `ToolCallError` 的消息与工具名,不提供程序可用的错误代码联合。
|
||||
@@ -189,7 +189,7 @@ describe.skipIf(!existsSync(join(root, 'apps/cli/lib/bin.js')))('Official option
|
||||
expect(names(minimal.agent)).toContain('session_search')
|
||||
await turn(coded.agent, 'Read this session lineage through run_code.', [
|
||||
toolCallResponse('ptc-search', 'run_code', {
|
||||
code: 'console.log(await tools.session_trace({}));',
|
||||
code: 'inspect(await tools.session_trace({}));',
|
||||
description: 'Read the current session lineage',
|
||||
}), textResponse('PTC_SEARCH_DONE'),
|
||||
])
|
||||
@@ -340,7 +340,7 @@ describe.skipIf(!existsSync(join(root, 'apps/cli/lib/bin.js')))('Official option
|
||||
if (coded === undefined) throw new Error('the PTC Agent was not created')
|
||||
await turn(coded, 'Read this session lineage and load the badge through one program.', [
|
||||
toolCallResponse('combined-program', 'run_code', {
|
||||
code: 'console.log(await tools.session_trace({})); console.log(await tools.skill({ name: "dsh-badge" }));',
|
||||
code: 'inspect(await tools.session_trace({})); inspect(await tools.skill({ name: "dsh-badge" }));',
|
||||
description: 'Use optional session search and badge together',
|
||||
}), textResponse('COMBINED_DONE'),
|
||||
])
|
||||
|
||||
@@ -38,7 +38,7 @@ import * as CordisInspectProviders from '@deepseek-ai/dsh-tool-cordis/host'
|
||||
*/
|
||||
|
||||
const PERSONA = 'You are a coding agent. You work by writing TypeScript programs for run_code: '
|
||||
+ 'batch related tool work into one program and print or return ONLY the findings that matter.'
|
||||
+ 'batch related tool work into one program and inspect ONLY the findings that matter.'
|
||||
const WORKSPACE_PROBE = 'dragonfruit-8675309'
|
||||
|
||||
let ctx: Context | undefined
|
||||
@@ -106,14 +106,17 @@ function runCode(
|
||||
})
|
||||
}
|
||||
|
||||
/** Read the optional completion from a successful canonical `run_code` value. */
|
||||
/** Read a JSON value explicitly inspected by the program. */
|
||||
function completion(result: ToolExecutionResult): unknown {
|
||||
if (result.isError) {
|
||||
throw new Error(result.content.filter(block => block.type === 'text').map(block => block.text).join('\n'))
|
||||
}
|
||||
const value = result.value
|
||||
if (typeof value !== 'object' || value === null || Array.isArray(value)) throw new Error('invalid run_code result')
|
||||
return value.result
|
||||
if (!Array.isArray(value.logs)) throw new Error('run_code result has no captured output')
|
||||
const text = value.logs.at(-1)
|
||||
if (typeof text !== 'string') throw new Error('run_code did not inspect a JSON value')
|
||||
return JSON.parse(text)
|
||||
}
|
||||
|
||||
async function mountRuntime(harness: Context): Promise<void> {
|
||||
@@ -184,7 +187,7 @@ describe('PTC mode typed values: keyless real-process contracts', () => {
|
||||
exposesInfo: 'info' in error,
|
||||
};
|
||||
}
|
||||
return { length: large.length, failure };
|
||||
inspect(JSON.stringify({ length: large.length, failure }));
|
||||
`))
|
||||
|
||||
expect(value).toEqual({
|
||||
@@ -211,12 +214,12 @@ describe('PTC mode typed values: keyless real-process contracts', () => {
|
||||
description: 'Run completion marker in background',
|
||||
run_in_background: true,
|
||||
});
|
||||
return started.jobId;
|
||||
inspect(JSON.stringify(started.jobId));
|
||||
`))
|
||||
expect(jobId).toMatch(/^bash-[0-9a-f]{16}$/)
|
||||
|
||||
const polled = completion(await runCode(ctx, `
|
||||
return await tools.job_output({ job_id: ${JSON.stringify(jobId)}, wait: true, timeout_ms: 5000 });
|
||||
inspect(JSON.stringify(await tools.job_output({ job_id: ${JSON.stringify(jobId)}, wait: true, timeout_ms: 5000 })));
|
||||
`))
|
||||
if (typeof polled !== 'object' || polled === null || Array.isArray(polled)) throw new Error('invalid job_output completion')
|
||||
const taskOutput = polled as Record<string, unknown>
|
||||
@@ -231,7 +234,7 @@ describe('PTC mode typed values: keyless real-process contracts', () => {
|
||||
const pre = new AbortController()
|
||||
pre.abort('pre-aborted')
|
||||
const preResult = await runCode(ctx, `
|
||||
return await tools.bash({ command: 'sleep 10', description: 'Must never start', run_in_background: true });
|
||||
inspect(JSON.stringify(await tools.bash({ command: 'sleep 10', description: 'Must never start', run_in_background: true })));
|
||||
`, pre.signal)
|
||||
expect(preResult.isError).toBe(true)
|
||||
expect(ctx.jobs.list()).toEqual([])
|
||||
@@ -246,7 +249,7 @@ describe('PTC mode typed values: keyless real-process contracts', () => {
|
||||
const afterPublication = new AbortController()
|
||||
const running = runCode(ctx, `
|
||||
const started = await tools.bash({ command: 'sleep 10', description: 'Wait for explicit task kill', run_in_background: true });
|
||||
console.log(started.jobId);
|
||||
inspect(started.jobId);
|
||||
await new Promise(() => {});
|
||||
`, afterPublication.signal)
|
||||
await Promise.race([
|
||||
@@ -265,11 +268,11 @@ describe('PTC mode typed values: keyless real-process contracts', () => {
|
||||
expect(ctx.jobs.list()[0]).toMatchObject({ id: job!.id, status: 'running' })
|
||||
|
||||
const killed = completion(await runCode(ctx, `
|
||||
return await tools.job_kill({ job_id: ${JSON.stringify(job!.id)}, reason: 'test owns cancellation' });
|
||||
inspect(JSON.stringify(await tools.job_kill({ job_id: ${JSON.stringify(job!.id)}, reason: 'test owns cancellation' })));
|
||||
`))
|
||||
expect(killed).toMatchObject({ outcome: 'cancellation-requested', job: { id: job!.id } })
|
||||
const settled = completion(await runCode(ctx, `
|
||||
return await tools.job_output({ job_id: ${JSON.stringify(job!.id)}, wait: true, timeout_ms: 5000 });
|
||||
inspect(JSON.stringify(await tools.job_output({ job_id: ${JSON.stringify(job!.id)}, wait: true, timeout_ms: 5000 })));
|
||||
`))
|
||||
expect(settled).toMatchObject({ job: { id: job!.id, status: 'killed' } })
|
||||
})
|
||||
@@ -280,7 +283,7 @@ describe('PTC mode typed values: keyless real-process contracts', () => {
|
||||
const controller = new AbortController()
|
||||
const startedAt = Date.now()
|
||||
const pending = runCode(ctx, `
|
||||
return await tools.bash({ command: 'sleep 10', description: 'Run cancellable foreground command' });
|
||||
inspect(JSON.stringify(await tools.bash({ command: 'sleep 10', description: 'Run cancellable foreground command' })));
|
||||
`, controller.signal)
|
||||
setTimeout(() => { controller.abort('stop-foreground') }, 200)
|
||||
const result = await pending
|
||||
@@ -306,7 +309,7 @@ describe('PTC mode typed values: keyless real-process contracts', () => {
|
||||
const inspected = await tools.cordis_inspect_query({
|
||||
platform: provider.platform, provider: provider.id, method: provider.methods[0].name,
|
||||
});
|
||||
return { provider: provider.id, names: inspected.data.tools.map(tool => tool.name) };
|
||||
inspect(JSON.stringify({ provider: provider.id, names: inspected.data.tools.map(tool => tool.name) }));
|
||||
`, testToolSignal, agent))
|
||||
|
||||
expect(value).toEqual({ provider: 'Tool', names: ['cordis_inspect_list', 'cordis_inspect_query'] })
|
||||
@@ -335,7 +338,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('PTC mode: real model writes a pr
|
||||
type: 'text',
|
||||
text: 'Using one run_code program: run `echo alpha-7` with the bash tool, run `echo beta-9` with the bash tool, '
|
||||
+ 'then write both outputs joined by a plus sign into combined.txt (bash heredoc or redirect), '
|
||||
+ 'and return only the joined string.',
|
||||
+ 'and inspect the joined string. Reply with that string only.',
|
||||
}], source: { kind: 'user' } }))
|
||||
await waitForIdle(ctx, agent)
|
||||
const events: readonly SessionEvent[] = agent.session.snapshotEvents()
|
||||
|
||||
@@ -46,7 +46,7 @@ it.for(['standard', 'ptc'] as const)('keeps %s plugin changes pending until the
|
||||
} catch (error) { fixtureError = error }
|
||||
const call = mode === 'ptc'
|
||||
? { type: 'tool_use', id: `manage-${requests.length}`, name: 'run_code', input: {
|
||||
code: `return await tools.plugin_manager(${JSON.stringify(args)});`, description: 'Disable web tools after restart',
|
||||
code: `inspect(await tools.plugin_manager(${JSON.stringify(args)}));`, description: 'Disable web tools after restart',
|
||||
} }
|
||||
: { type: 'tool_use', id: `manage-${requests.length}`, name: 'plugin_manager', input: args }
|
||||
response.writeHead(200, { 'content-type': 'text/event-stream' })
|
||||
|
||||
@@ -80,14 +80,20 @@ it.skipIf(mode === 'record')('shows standard CUA programs and their recorded sou
|
||||
expect(await source.textContent()).toContain('var app = await cua.getApp("test.fixture")')
|
||||
expect(await page.locator('[data-subcalls] [data-tool="cua"]').count()).toBe(8)
|
||||
expect(events.filter(event => event.type === 'tool/ptc-dispatch' && event.data.content.some(block => block.type === 'image'))).toHaveLength(1)
|
||||
expect(events.filter(event => event.type === 'user/message' && event.data.source.kind === 'ptc-mode'
|
||||
&& event.data.content.some(block => block.type === 'image'))).toHaveLength(1)
|
||||
expect(events.filter(event => event.type === 'tool/result'
|
||||
&& event.data.message.content.some(block => block.type === 'image'))).toHaveLength(1)
|
||||
expect(await calls.first().textContent()).toContain('(1x1px, 90 bytes)')
|
||||
expect(await calls.first().textContent()).not.toContain('"type": "image"')
|
||||
await compareOrRefreshGolden(join(snapshotDirectory, 'ui.expected.md'),
|
||||
await captureStableAria(page, '[class*="centerCol"]', scaffold.workspaceCwd), mode)
|
||||
if (mode === 'refresh') {
|
||||
const evidence = fileURLToPath(new URL('../../../.playwright-mcp/cua-run-code-ui', import.meta.url))
|
||||
await mkdir(evidence, { recursive: true })
|
||||
await page.screenshot({ path: join(evidence, 'standard-code-cards.png'), animations: 'disabled' })
|
||||
const viewport = page.viewportSize()
|
||||
await page.setViewportSize({ width: 800, height: 900 })
|
||||
await page.screenshot({ path: join(evidence, 'standard-code-cards-narrow.png'), animations: 'disabled' })
|
||||
if (viewport !== null) await page.setViewportSize(viewport)
|
||||
}
|
||||
await page.getByRole('tab', { name: 'Trajectory', exact: true }).click()
|
||||
const row = page.getByRole('row', { name: new RegExp(`cua_run_code ${description}`) })
|
||||
|
||||
@@ -796,7 +796,7 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
const callId = ToolCallId('long-ptc-result')
|
||||
const call = agent.session.append('tool/call', {
|
||||
turn: 3, step: 1, callId, name: 'run_code',
|
||||
arguments: JSON.stringify({ code: 'console.log("PTC output")', description: 'Long program output' }),
|
||||
arguments: JSON.stringify({ code: 'inspect("PTC output")', description: 'Long program output' }),
|
||||
})
|
||||
await page.getByRole('tab', { name: 'Trajectory' }).click()
|
||||
await page.locator('tr[data-kind="tool"]').filter({ hasText: 'Long program output' }).click()
|
||||
|
||||
@@ -141,7 +141,7 @@ describe.skipIf(MODE === 'record')('web snapshot: Official native subagent bundl
|
||||
const firstTurnEvents = agent.session.snapshotEvents()
|
||||
const toolResults = firstTurnEvents.filter(event => event.type === 'tool/result').map(event => event.data)
|
||||
expect(toolResults, `PTC execution must start the native child before waiting for its result: ${JSON.stringify(toolResults)}`)
|
||||
.toMatchObject([{ message: { isError: false, content: [{ type: 'text', text: 'agent-1' }] } }])
|
||||
.toMatchObject([{ message: { isError: false, content: [{ type: 'text', text: 'agent-1\n' }] } }])
|
||||
const dispatches = firstTurnEvents.filter(event => event.type === 'tool/ptc-dispatch').map(event => event.data)
|
||||
expect(dispatches).toMatchObject([{ name: 'subagent_claude_code', isError: false }])
|
||||
const catalog = firstTurnEvents.filter(event => event.type === 'subagent/catalog').map(event => event.data)
|
||||
|
||||
@@ -57,7 +57,8 @@ fs.appendFileSync(${JSON.stringify(openLog)}, JSON.stringify({ path, action, con
|
||||
openInAppEnvironment: createLaunchEnvironmentSnapshot([{ source: 'process', values: { SSH_CONNECTION: '10.0.0.2 55000 10.0.0.9 22' } }]),
|
||||
extraOverlayPath: fileURLToPath(new URL('./present.overlay.yml', import.meta.url)),
|
||||
agentPresets: { default: 'ptc' }, compareReplaySession: true,
|
||||
...(MODE === 'record' ? {} : { replayFixture: FIXTURE }),
|
||||
// The recorded UI includes decode throughput from an incremental stream.
|
||||
...(MODE === 'record' ? {} : { replayFixture: FIXTURE, paceMs: 15 }),
|
||||
})
|
||||
// File associations belong to the desktop rather than the recorded Session.
|
||||
const controller = scaffold.ctx.get('sessionController')
|
||||
|
||||
@@ -9,7 +9,7 @@ import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type {} from '@deepseek-ai/dsh-user-approval'
|
||||
import {
|
||||
assertFinalWorkspaceSnapshot, assertFixtureInventory, captureStableAria, compareOrRefreshGolden, fixtureUserPrompts,
|
||||
launchWebScaffold, recordFixture, watchConsole, webSnapshotMode, type WebScaffold,
|
||||
launchWebScaffold, recordFixture, selectedSessionFixture, watchConsole, webSnapshotMode, type WebScaffold,
|
||||
} from './scaffold.ts'
|
||||
import { connectFreshWorkspace, newEnglishPage, saveFailureShot } from './support.ts'
|
||||
|
||||
@@ -19,7 +19,7 @@ const UI_EXPECTED = join(SNAPSHOT_DIR, 'approval.expected.md')
|
||||
const MODE = webSnapshotMode()
|
||||
const PROMPT = 'Use run_code with timeoutMs 120000 and direct Node filesystem access to create approved.txt in the working directory containing exactly "approved\\n". '
|
||||
+ 'Use await import("node:fs/promises") and writeFile; do not call nested tools. First attempt the write under the current read-only sandbox without escalation. '
|
||||
+ 'In that first program, catch only filesystem errors with code EPERM, EACCES or EROFS and return exactly "EXPECTED_SANDBOX_DENIAL"; rethrow any other error. '
|
||||
+ 'In that first program, catch only filesystem errors with code EPERM, EACCES or EROFS and use `inspect(...)` to display exactly "EXPECTED_SANDBOX_DENIAL"; rethrow any other error. '
|
||||
+ 'If the sandbox denies it, explicitly retry the program with sandbox_permissions "workspace-write" and justification "Create the file requested by the user". '
|
||||
+ 'I will answer the approval prompt. After the file is written, reply DONE and stop.'
|
||||
|
||||
@@ -52,7 +52,7 @@ describe('web e2e: PTC program sandbox escalation', () => {
|
||||
|
||||
it('keeps the file absent until approval and records the granted program', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-ptc-escalation'))
|
||||
if (MODE !== 'record') expect(fixtureUserPrompts(await readFile(FIXTURE, 'utf8'))).toEqual([PROMPT])
|
||||
if (MODE !== 'record') expect(fixtureUserPrompts(await readFile(await selectedSessionFixture(FIXTURE), 'utf8'))).toEqual([PROMPT])
|
||||
const input = page.locator('[data-composer-input]').first()
|
||||
await input.waitFor({ timeout: 10_000 })
|
||||
await page.locator('[aria-label^="Access mode"]').click()
|
||||
|
||||
@@ -9,7 +9,7 @@ import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
acknowledgeReloadConnectionLoss, captureExpandedTurnProcessAria, captureStableAria, compareOrRefreshGolden, fixtureUserPrompts,
|
||||
launchWebScaffold, recordFixture, watchConsole, webSnapshotMode, type WebScaffold,
|
||||
launchWebScaffold, recordFixture, selectedSessionFixture, watchConsole, webSnapshotMode, type WebScaffold,
|
||||
} from './scaffold.ts'
|
||||
import { connectFreshWorkspace, expandOwningTurnProcess, newEnglishPage, openSettings, saveFailureShot } from './support.ts'
|
||||
|
||||
@@ -23,7 +23,7 @@ const MODE = webSnapshotMode()
|
||||
|
||||
// Elicits the successful and failed sub-rows this scenario asserts.
|
||||
const PROMPT = 'Using ONE run_code program: run bash `echo CODE_ROUND_OK`, then read the file missing.txt '
|
||||
+ 'catching its error in the program. Return an object with both outcomes. Then reply DONE and stop.'
|
||||
+ 'catching its error in the program. Use `inspect(...)` to display an object with both outcomes. Then reply DONE and stop.'
|
||||
|
||||
describe('web e2e: PTC mode round renders nested sub-calls', () => {
|
||||
let scaffold: WebScaffold
|
||||
@@ -58,7 +58,7 @@ describe('web e2e: PTC mode round renders nested sub-calls', () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-ptc-drive'))
|
||||
const requestStates: string[] = []
|
||||
if (MODE !== 'record') {
|
||||
expect(fixtureUserPrompts(await readFile(FIXTURE, 'utf8'))).toEqual([PROMPT])
|
||||
expect(fixtureUserPrompts(await readFile(await selectedSessionFixture(FIXTURE), 'utf8'))).toEqual([PROMPT])
|
||||
}
|
||||
const input = page.locator('[data-composer-input]').first()
|
||||
await input.waitFor({ timeout: 10_000 })
|
||||
|
||||
@@ -586,6 +586,7 @@ flowchart TD
|
||||
pkg_skill_badge --> pkg_skill
|
||||
pkg_host_product_telemetry_otel --> pkg_otel
|
||||
pkg_persona --> pkg_system_prompt
|
||||
pkg_ptc_runtime --> pkg_attachment
|
||||
pkg_ptc_runtime --> pkg_sandbox
|
||||
pkg_sandbox_local --> pkg_llm
|
||||
pkg_sandbox_local --> pkg_sandbox
|
||||
@@ -1752,7 +1753,7 @@ flowchart TD
|
||||
| [`skill-badge`](../packages/experimental/skill-badge) | `experimental` | [`skill`](../packages/skill/skill) |
|
||||
| [`host-product-telemetry-otel`](../packages/host/product-telemetry-otel) | `host` | [`otel`](../packages/telemetry/otel) |
|
||||
| [`persona`](../packages/preset/persona) | `preset` | [`system-prompt`](../packages/core/system-prompt) |
|
||||
| [`ptc-runtime`](../packages/ptc-runtime/ptc-runtime) | `ptc-runtime` | [`sandbox`](../packages/sandbox/sandbox) |
|
||||
| [`ptc-runtime`](../packages/ptc-runtime/ptc-runtime) | `ptc-runtime` | [`attachment`](../packages/attachment/attachment), [`sandbox`](../packages/sandbox/sandbox) |
|
||||
| [`sandbox-local`](../packages/sandbox/sandbox-local) | `sandbox` | [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox), [`session`](../packages/core/session) |
|
||||
| [`sandbox-windows-acl`](../packages/sandbox/sandbox-windows-acl) | `sandbox` | [`skill`](../packages/skill/skill), [`subprocess`](../packages/subprocess/subprocess) |
|
||||
| [`session-persistence-jsonl`](../packages/session/session-persistence-jsonl) | `session` | [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence) |
|
||||
|
||||
@@ -588,6 +588,7 @@ flowchart TD
|
||||
pkg_skill_badge --> pkg_skill
|
||||
pkg_host_product_telemetry_otel --> pkg_otel
|
||||
pkg_persona --> pkg_system_prompt
|
||||
pkg_ptc_runtime --> pkg_attachment
|
||||
pkg_ptc_runtime --> pkg_sandbox
|
||||
pkg_sandbox_local --> pkg_llm
|
||||
pkg_sandbox_local --> pkg_sandbox
|
||||
@@ -1754,7 +1755,7 @@ flowchart TD
|
||||
| [`skill-badge`](../packages/experimental/skill-badge) | `experimental` | [`skill`](../packages/skill/skill) |
|
||||
| [`host-product-telemetry-otel`](../packages/host/product-telemetry-otel) | `host` | [`otel`](../packages/telemetry/otel) |
|
||||
| [`persona`](../packages/preset/persona) | `preset` | [`system-prompt`](../packages/core/system-prompt) |
|
||||
| [`ptc-runtime`](../packages/ptc-runtime/ptc-runtime) | `ptc-runtime` | [`sandbox`](../packages/sandbox/sandbox) |
|
||||
| [`ptc-runtime`](../packages/ptc-runtime/ptc-runtime) | `ptc-runtime` | [`attachment`](../packages/attachment/attachment), [`sandbox`](../packages/sandbox/sandbox) |
|
||||
| [`sandbox-local`](../packages/sandbox/sandbox-local) | `sandbox` | [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox), [`session`](../packages/core/session) |
|
||||
| [`sandbox-windows-acl`](../packages/sandbox/sandbox-windows-acl) | `sandbox` | [`skill`](../packages/skill/skill), [`subprocess`](../packages/subprocess/subprocess) |
|
||||
| [`session-persistence-jsonl`](../packages/session/session-persistence-jsonl) | `session` | [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence) |
|
||||
|
||||
@@ -12,8 +12,8 @@
|
||||
en: 182147ddf388c600
|
||||
zh: c0a7d77602c78bcd
|
||||
/ptc-runtime/captured-output-and-the-failure-taxonomy:
|
||||
en: 8eb1607888ab6efc
|
||||
zh: 580c852d17b9413f
|
||||
en: ef553620e77bd638
|
||||
zh: 40ff2f5ba3de5ed5
|
||||
/ptc-runtime/the-service:
|
||||
en: a82f828fa006c7e9
|
||||
zh: 2279d36867335b57
|
||||
@@ -26,6 +26,8 @@ interface PtcRunRequest {
|
||||
* becomes {@link PtcRunResult.value}.
|
||||
*/
|
||||
program: string
|
||||
/** Discard the completion without JSON validation when true; omission preserves the completion value. */
|
||||
discardCompletion?: boolean
|
||||
/**
|
||||
* Object-identity owner for optional in-memory state. Omission requests a fresh
|
||||
* environment. Consumers must await runtime.release(owner) when the owner ends.
|
||||
@@ -93,11 +95,12 @@ interface PtcRunResult {
|
||||
*/
|
||||
value?: PtcJsonValue
|
||||
/**
|
||||
* Captured text. Each source channel preserves emission order; interleaving
|
||||
* Captured text fragments retain their emitted line breaks. Concatenate adjacent text without separators.
|
||||
* Each source channel preserves emission order; interleaving
|
||||
* across independent channels is backend-dependent. Bounded only as part of
|
||||
* the outer result.
|
||||
*/
|
||||
logs: string[]
|
||||
output: PtcOutput[]
|
||||
/** Present iff the run failed; see {@link PtcRunFailure} for the taxonomy. */
|
||||
error?: PtcRunFailure
|
||||
}
|
||||
@@ -150,7 +153,7 @@ interface PtcBindingNamespace {
|
||||
* words, so the same namespace list works against every backend regardless
|
||||
* of `language` — a JS-only spelling like `$tools` is rejected by design,
|
||||
* not just by the Python backend. Names that satisfy the identifier rule but
|
||||
* name a backend-owned slot (`RESERVED_BINDING_GLOBALS`, e.g. `console`,
|
||||
* name a backend-owned slot (`RESERVED_BINDING_GLOBALS`, e.g. `inspect`, `console`, `Image`,
|
||||
* `__dsh_main__`) are also refused everywhere; see its declaration for the
|
||||
* exact set and why each entry is reserved.
|
||||
*/
|
||||
@@ -171,7 +174,7 @@ type PtcJsonValue = null | boolean | number | string | PtcJsonValue[] | { [key:
|
||||
/**
|
||||
* One host-side function exposed to the program as an async callable. The
|
||||
* runtime bridges calls to it (possibly across a serialization boundary), so
|
||||
* `args` and the resolution value MUST be lossless JSON. A runtime rejects a
|
||||
* `args` must be lossless JSON; resolutions also support host-issued PtcImage values. A runtime rejects a
|
||||
* lossy or non-cloneable value with a descriptive error rather than corrupting
|
||||
* the run. No seam-level byte cap applies to a binding resolution. A rejection
|
||||
* of this function surfaces inside the program as a rejection of the
|
||||
@@ -182,7 +185,29 @@ type PtcBindingFunction = (args: unknown) => Promise<PtcJsonValue>
|
||||
|
||||
## Captured output and the failure taxonomy
|
||||
|
||||
Logs are plain strings. Each source channel preserves emission order, while interleaving across independent channels is backend-dependent because channel metadata is not part of the seam. The runtime captures the program's console and stream output, and consumers render only the text. Implementations cap the serialized outer log-array plus completion-value or failure-message payload; fixed result-envelope syntax and consumer presentation whitespace are not part of that variable-payload ledger. Overflow is an explicit failure rather than in-band value substitution.
|
||||
`output` contains text and image entries in emission order: `{ type: 'text', text }` or `{ type: 'image', attachment }`. Text entries retain emitted newline characters; concatenate adjacent text without separators. Each source channel preserves emission order; interleaving across independent channels is backend-dependent. Host bindings return `PtcImage` descriptors containing the attachment reference and a readable execution-world `path`; the program receives an `Image` with that path and the stored pixel dimensions. The Node provider caps serialized text, image descriptors, and completion or diagnostic payloads together; overflow is an explicit failure.
|
||||
|
||||
```ts public-api
|
||||
/** A stored image readable from the program's execution environment. */
|
||||
declare class PtcImage {
|
||||
readonly [key: string]: PtcJsonValue;
|
||||
/** Absolute attachment file path readable in the execution world. */
|
||||
readonly path: string;
|
||||
/** Stored image selected when the program inspects this value. */
|
||||
readonly attachment: Readonly<ImageAttachmentRef>;
|
||||
/**
|
||||
* Describe one immutable image file in the execution world.
|
||||
* @param path - Absolute path to the stored image.
|
||||
* @param attachment - Reference to the immutable stored image.
|
||||
*/
|
||||
constructor(path: string, attachment: ImageAttachmentRef);
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
/** Raw text fragments and inspected images in emission order; concatenate adjacent text without separators. */
|
||||
type PtcOutput = { type: 'text'; text: string } | { type: 'image'; attachment: ImageAttachmentRef }
|
||||
```
|
||||
|
||||
Failure kinds are **orthogonal outcomes reported independently** (per [defensive-patterns](../defensive-patterns/general.md)): a budget expiry is not an exception, an abort is not a timeout, and a substrate death (e.g. OOM) is neither:
|
||||
|
||||
|
||||
@@ -26,6 +26,8 @@ interface PtcRunRequest {
|
||||
* becomes {@link PtcRunResult.value}.
|
||||
*/
|
||||
program: string
|
||||
/** Discard the completion without JSON validation when true; omission preserves the completion value. */
|
||||
discardCompletion?: boolean
|
||||
/**
|
||||
* Object-identity owner for optional in-memory state. Omission requests a fresh
|
||||
* environment. Consumers must await runtime.release(owner) when the owner ends.
|
||||
@@ -93,11 +95,12 @@ interface PtcRunResult {
|
||||
*/
|
||||
value?: PtcJsonValue
|
||||
/**
|
||||
* Captured text. Each source channel preserves emission order; interleaving
|
||||
* Captured text fragments retain their emitted line breaks. Concatenate adjacent text without separators.
|
||||
* Each source channel preserves emission order; interleaving
|
||||
* across independent channels is backend-dependent. Bounded only as part of
|
||||
* the outer result.
|
||||
*/
|
||||
logs: string[]
|
||||
output: PtcOutput[]
|
||||
/** Present iff the run failed; see {@link PtcRunFailure} for the taxonomy. */
|
||||
error?: PtcRunFailure
|
||||
}
|
||||
@@ -150,7 +153,7 @@ interface PtcBindingNamespace {
|
||||
* words, so the same namespace list works against every backend regardless
|
||||
* of `language` — a JS-only spelling like `$tools` is rejected by design,
|
||||
* not just by the Python backend. Names that satisfy the identifier rule but
|
||||
* name a backend-owned slot (`RESERVED_BINDING_GLOBALS`, e.g. `console`,
|
||||
* name a backend-owned slot (`RESERVED_BINDING_GLOBALS`, e.g. `inspect`, `console`, `Image`,
|
||||
* `__dsh_main__`) are also refused everywhere; see its declaration for the
|
||||
* exact set and why each entry is reserved.
|
||||
*/
|
||||
@@ -171,7 +174,7 @@ type PtcJsonValue = null | boolean | number | string | PtcJsonValue[] | { [key:
|
||||
/**
|
||||
* One host-side function exposed to the program as an async callable. The
|
||||
* runtime bridges calls to it (possibly across a serialization boundary), so
|
||||
* `args` and the resolution value MUST be lossless JSON. A runtime rejects a
|
||||
* `args` must be lossless JSON; resolutions also support host-issued PtcImage values. A runtime rejects a
|
||||
* lossy or non-cloneable value with a descriptive error rather than corrupting
|
||||
* the run. No seam-level byte cap applies to a binding resolution. A rejection
|
||||
* of this function surfaces inside the program as a rejection of the
|
||||
@@ -182,7 +185,29 @@ type PtcBindingFunction = (args: unknown) => Promise<PtcJsonValue>
|
||||
|
||||
## 捕获的输出与失败分类体系
|
||||
|
||||
日志是纯字符串。每个来源通道保留自身的发出顺序;由于通道元数据不属于 seam,相互独立的通道如何交错由后端决定。运行时捕获程序的 console 与流输出,Consumer 只渲染文本。实现会对序列化后的外层日志数组,以及完成值或失败消息的组合载荷设置上限;固定的结果封装语法与 Consumer 展示空白不计入这份可变载荷计量。超限会显式失败,而不会在值中插入替代内容。
|
||||
`output` 按发出顺序保存文本和图片条目:文本使用 `{ type: 'text', text }`,图片使用 `{ type: 'image', attachment }`。文本条目保留实际输出的换行,相邻文本直接拼接,不额外插入分隔符。每个来源通道都保留发出顺序;独立通道之间的交错顺序由后端决定。宿主绑定返回 `PtcImage` 描述符,包含附件引用和可在执行环境读取的 `path`;程序收到带有该路径和存储像素尺寸的 `Image`。Node 提供方对序列化文本、图片描述符、完成值或诊断一起计量;超限会显式失败。
|
||||
|
||||
```ts public-api
|
||||
/** A stored image readable from the program's execution environment. */
|
||||
declare class PtcImage {
|
||||
readonly [key: string]: PtcJsonValue;
|
||||
/** Absolute attachment file path readable in the execution world. */
|
||||
readonly path: string;
|
||||
/** Stored image selected when the program inspects this value. */
|
||||
readonly attachment: Readonly<ImageAttachmentRef>;
|
||||
/**
|
||||
* Describe one immutable image file in the execution world.
|
||||
* @param path - Absolute path to the stored image.
|
||||
* @param attachment - Reference to the immutable stored image.
|
||||
*/
|
||||
constructor(path: string, attachment: ImageAttachmentRef);
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
/** Raw text fragments and inspected images in emission order; concatenate adjacent text without separators. */
|
||||
type PtcOutput = { type: 'text'; text: string } | { type: 'image'; attachment: ImageAttachmentRef }
|
||||
```
|
||||
|
||||
失败类型是**正交的结果,独立报告**(见 [defensive-patterns](../defensive-patterns/general.zh.md)):预算耗尽不是异常,中止不是超时,基底崩溃(如 OOM)也不是二者中的任何一个:
|
||||
|
||||
|
||||
@@ -72,8 +72,8 @@
|
||||
en: 8c8643ef46fd075f
|
||||
zh: 8c8643ef46fd075f
|
||||
/tool-schema-catalog/deepseek-ai-dsh-tool-executor-ptc/run-code:
|
||||
en: 7917e16cbc2c366f
|
||||
zh: c39eb0420544f04a
|
||||
en: 2c7a3166dfd9751c
|
||||
zh: 1b52ae2d0f33b9a4
|
||||
/tool-schema-catalog/deepseek-ai-dsh-plan-mode:
|
||||
en: b500b530845e26a9
|
||||
zh: b500b530845e26a9
|
||||
|
||||
@@ -623,7 +623,7 @@ ask_user_question keeps the original blocking behavior by default; set `mode: ti
|
||||
|
||||
### `run_code`
|
||||
|
||||
Execute a TypeScript program against the available tools. Takes two required arguments: `code`, the BODY of an async function (erasable syntax only; top-level `await` and `return` work), and `description`, a short summary of what the program does. Call tools as `await tools.name(args)` per the declarations in the system prompt. Only what you print or return is program output — curate it. Image-bearing subtool results are attached after the run. The program is the body of an async TypeScript function; only erasable syntax is accepted, with no enum or namespaces. Type annotations are advisory. Top-level await and return are supported. Emit output with console.log(...) or return a lossless JSON value. Only printed values and the returned value become program output. Each call runs in a fresh Node process. Node APIs are available through await import(...). Relative paths use the supplied working directory; process.env starts empty. Direct filesystem access follows this execution's sandbox policy. Each program starts in the Session's current directory. Running programs keep their initial directory. A sandbox escalation approves this complete program for one execution only. Nested tools retain their own policies and approvals. Request wider access only after evidence of a denial. Earlier effects may already have completed: inspect them before explicitly retrying. Programs are never replayed automatically.
|
||||
Execute a TypeScript program against the available tools. Takes two required arguments: `code`, the BODY of an async function (erasable syntax only; top-level `await` and `return` work), and `description`, a short summary of what the program does. Call tools as `await tools.name(args)` per the declarations in the system prompt. Show selected text and images with inspect(...values). The program is the body of an async TypeScript function; only erasable syntax is accepted, with no enum or namespaces. Type annotations are advisory. Top-level await and return are supported. Use inspect(...values) to display output. Do not use console. Each call runs in a fresh Node process. Node APIs are available through await import(...). Relative paths use the supplied working directory; process.env starts empty. Direct filesystem access follows this execution's sandbox policy. Each program starts in the Session's current directory. Running programs keep their initial directory. A sandbox escalation approves this complete program for one execution only. Nested tools retain their own policies and approvals. Request wider access only after evidence of a denial. Earlier effects may already have completed: inspect them before explicitly retrying. Programs are never replayed automatically.
|
||||
|
||||
```json
|
||||
{
|
||||
|
||||
@@ -627,7 +627,7 @@ ask_user_question 默认保持原有阻塞行为;设置 `mode: timed` 后才
|
||||
|
||||
### `run_code`
|
||||
|
||||
针对可用工具执行 TypeScript 程序。接受两个必填参数:`code`,异步函数的函数体(仅限可擦除语法,支持顶层 `await` 和 `return`);以及 `description`,简要说明程序做什么。请按系统提示词中的声明,以 `await tools.name(args)` 调用工具。只有打印或返回的内容构成程序输出,请筛选这些内容。含图片的子工具结果会在运行结束后附加。程序是异步 TypeScript 函数的函数体,只接受可擦除语法,不支持 enum 或 namespace。类型标注只作提示。支持顶层 await 和 return。使用 console.log(...) 或返回无损 JSON 值来输出。只有打印值和返回值成为程序输出。每次调用在新的 Node 进程中运行,可通过 await import(...) 使用 Node API。相对路径以传入的工作目录为准;process.env 初始为空。直接文件系统访问遵循本次执行的沙箱策略。每个程序从 Session 的当前目录启动,运行中保留初始目录。沙箱提权只为本次完整程序执行授权。嵌套工具保留自身策略与审批。只有在获得拒绝证据后才请求更宽访问。先前效果可能已经发生,请在显式重试前检查。程序不会自动重放。
|
||||
针对可用工具执行 TypeScript 程序。接受两个必填参数:`code`,异步函数的函数体(仅限可擦除语法,支持顶层 `await` 和 `return`);以及 `description`,简要说明程序做什么。请按系统提示词中的声明,以 `await tools.name(args)` 调用工具。使用 inspect(...values) 展示选定的文本和图片。程序是异步 TypeScript 函数的函数体,只接受可擦除语法,不支持 enum 或 namespace。类型标注只作提示。支持顶层 await 和 return。使用 inspect(...values) 展示输出。不要使用 console。每次调用在新的 Node 进程中运行,可通过 await import(...) 使用 Node API。相对路径以传入的工作目录为准;process.env 初始为空。直接文件系统访问遵循本次执行的沙箱策略。每个程序从 Session 的当前目录启动,运行中保留初始目录。沙箱提权只为本次完整程序执行授权。嵌套工具保留自身策略与审批。只有在获得拒绝证据后才请求更宽访问。先前效果可能已经发生,请在显式重试前检查。程序不会自动重放。
|
||||
|
||||
```json
|
||||
{
|
||||
|
||||
@@ -12,5 +12,5 @@
|
||||
en: d3b7e02c362b0049
|
||||
zh: 49f1a190495e813d
|
||||
/native-cua-application-and-window-api/migration:
|
||||
en: 90d4d985da7e6618
|
||||
zh: 9d9b4d3444cf0e3b
|
||||
en: fe1cd8e7812f6fb2
|
||||
zh: 449cca5b18275ebd
|
||||
@@ -16,7 +16,7 @@ Providers supply startup capabilities and implement `getGuidance(kind)` instead
|
||||
## Migration
|
||||
|
||||
1. After `cua.getApp(selector)`, select the target from `app.getWindows()` with `app.getWindow(windowId)`. Use `window.getScreenshot()`, `window.click()`, `window.scroll()`, and `window.drag()`. Replace `hover` with `window.mouseMove()`. On macOS, `window.click(id)` sends mouse events at the element center. Switch to `invoke(id)` for the default accessibility action, or `performAction(id, "AXShowMenu")` to request an advertised menu action.
|
||||
2. Replace `getAXState()` with `getAccessibilityTree()`. Replace `getAXStateAndScreenshot()` with separate tree and screenshot reads. Application and window selection does not read AX. Before using element IDs, call `getAccessibilityTree()` explicitly; use `{ emit: false }` to retain the snapshot without displaying it. A screenshot does not require a successful AX read. Before coordinate input, including relative coordinates, obtain a successful AX observation covering the window or a screenshot; absolute coordinates require a screenshot. See [macOS AX read limits](../cua-ax-read-limits/guide.md) for changes to traversal defaults and configuration.
|
||||
2. Replace `getAXState()` with `getAccessibilityTree()`. Replace `getAXStateAndScreenshot()` with separate tree and screenshot reads. Application and window selection does not read AX. Before using element IDs, call `getAccessibilityTree()` explicitly. To display the tree in TypeScript PTC, pass its return value to `inspect`. A screenshot does not require a successful AX read. Before coordinate input, including relative coordinates, obtain a successful AX observation covering the window or a screenshot; absolute coordinates require a screenshot. See [macOS AX read limits](../cua-ax-read-limits/guide.md) for changes to traversal defaults and configuration.
|
||||
3. Pass the opaque string ID from the latest tree wherever you passed a numeric element index. In raw requests use `elementId` with `snapshotId`. Replace `performSecondaryAction` with `listActions(id)` and `performAction(id, exactName, argument?)`, or use `invoke(id)` for the default action. `setValue` accepts the provider’s supported JSON values. On macOS, `performAction(id, 'AXRaise')` performs the advertised raise action. Call `app.activate()` first when the application must become active.
|
||||
4. Replace a packed chord such as `pressKey('shift+Tab')` with `pressKey('Tab', ['shift'])`. Check `(await cua.getCapabilities()).methods` before using `sendKeyEvent`. On macOS, call it inside one `run_code` or `cua_run_code` to hold a key across tool calls. To request one repeat, send another down for a non-modifier key held by that execution. Returning, throwing or pausing releases any remaining keys. If your script handles unknown macOS key names as `NATIVE_ERROR`, handle `INVALID_ARGUMENT` instead. Correct the key name before retrying.
|
||||
5. Read `dispatched` from action results, then observe the application to verify the outcome. `invoke` returns `dispatched: false` when the element has no default action. Remove reads of the old `verified` field and of `pid` on selected objects or window records. Use `listApps()` or running `findApps()` matches when you need a PID.
|
||||
|
||||
@@ -16,7 +16,7 @@ description: "迁移 CUA 脚本和自定义原生提供方,使用显式窗口
|
||||
## 迁移
|
||||
|
||||
1. 调用 `cua.getApp(selector)` 后,从 `app.getWindows()` 返回的列表选定目标,再调用 `app.getWindow(windowId)`。在窗口上调用 `getScreenshot()`、`click()`、`scroll()` 和 `drag()`,将原有的 `hover` 调用改为 `window.mouseMove()`。在 macOS 上,`window.click(id)` 在元素中心发送鼠标事件。需要执行默认辅助功能动作时,改用 `invoke(id)`。需要打开控件菜单且控件声明了 `AXShowMenu` 时,调用 `performAction(id, "AXShowMenu")`。
|
||||
2. 将 `getAXState()` 改为 `getAccessibilityTree()`,将 `getAXStateAndScreenshot()` 拆成读取树和截图两次调用。选择应用或窗口不读取 AX。使用元素 ID 前须显式调用 `getAccessibilityTree()`;使用 `{ emit: false }` 可保留快照但不显示树。截图不要求 AX 读取成功。所有坐标输入(包括相对坐标)之前都须成功读取覆盖该窗口的 AX 或取得截图;绝对坐标要求截图。[读取上限的迁移说明](../cua-ax-read-limits/guide.zh.md)列出了 macOS 默认值和配置含义的变化。
|
||||
2. 将 `getAXState()` 改为 `getAccessibilityTree()`,将 `getAXStateAndScreenshot()` 拆成读取树和截图两次调用。选择应用或窗口不读取 AX。使用元素 ID 前须显式调用 `getAccessibilityTree()`。在 TypeScript PTC 中,将返回值传给 `inspect` 即可展示这棵树。截图不要求 AX 读取成功。所有坐标输入(包括相对坐标)之前都须成功读取覆盖该窗口的 AX 或取得截图;绝对坐标要求截图。[读取上限的迁移说明](../cua-ax-read-limits/guide.zh.md)列出了 macOS 默认值和配置含义的变化。
|
||||
3. 将数字元素索引改为从最新树中取得的字符串 ID。直接发送请求时使用 `elementId` 和 `snapshotId`。原先调用 `performSecondaryAction` 的地方,先用 `listActions(id)` 取得完整动作名,再调用 `performAction(id, exactName, argument?)`。要执行默认动作,可调用 `invoke(id)`。`setValue` 接受提供方支持的 JSON 值。 在 macOS 上,`performAction(id, 'AXRaise')` 只执行元素声明的抬起窗口动作。需要把应用切到前台时先调用 `app.activate()`。
|
||||
4. 将 `pressKey('shift+Tab')` 之类的组合键写法改为 `pressKey('Tab', ['shift'])`。调用 `sendKeyEvent` 前检查 `(await cua.getCapabilities()).methods`。在 macOS 上,同一次 `run_code` 或 `cua_run_code` 中可调用此方法跨工具调用保持按键。如果需要一次重复按下,可在这次执行中对已按住的普通键再发一次 `down`。执行返回、异常退出或用户暂停时会释放仍按着的键。如果脚本用 `NATIVE_ERROR` 识别 macOS 上的未知键名,请改为处理 `INVALID_ARGUMENT`,并在重试前修正键名。
|
||||
5. 从动作结果读取 `dispatched`,再观察应用确认效果。元素没有默认动作时,`invoke` 返回 `dispatched: false`。移除读取旧字段 `verified` 的代码,以及对已选对象和窗口记录中 `pid` 的读取。需要 PID 时,使用 `listApps()` 或 `findApps()` 返回的运行中应用记录。
|
||||
|
||||
@@ -9,8 +9,8 @@
|
||||
en: 5c10a48d83f56e19
|
||||
zh: 1c8d8832764d558f
|
||||
/node-ptc-bootstrap-protocol-version-3/change:
|
||||
en: 09a683eb8a4ec57c
|
||||
zh: 7b702b0011ac83b8
|
||||
en: 6d4af1d37654f401
|
||||
zh: a06ee0ee8a9326ee
|
||||
/node-ptc-bootstrap-protocol-version-3/migration:
|
||||
en: 4c25b296ed471589
|
||||
zh: 46071ce14e092f48
|
||||
@@ -13,6 +13,8 @@ The Node PTC Host requires bootstrap protocol version 3 instead of version 2. It
|
||||
|
||||
Version 3 carries optional `ProgramBootData.initialization` source for each new environment. The bootstrap evaluates it separately after installing current bindings, then awaits the Host's `initialized()` RPC acceptance and rechecks cancellation before running the model program. Initialization failure discards the environment and prevents model execution. The Node provider supports initializer registration only in `persistent-in-memory` mode.
|
||||
|
||||
Version 3 also carries `ProgramBootData.discardCompletion` to discard completion values before JSON validation and serialization. The bootstrap provides `inspect` and sends selected images through `HostMethods.image(id)` in output order. Custom bootstraps must support these fields and methods alongside environment initialization.
|
||||
|
||||
## Migration
|
||||
|
||||
1. If your Node PTC configuration sets `launch.bootstrapPath`, deploy the matching version of `@deepseek-ai/dsh-ptc-runtime-node` in its execution environment. Use the package's built `lib/process.js` or REPL-default `lib/persistent-process.js` with its dependencies. Keep the Host and bootstrap versions aligned; changing only the advertised version is insufficient.
|
||||
|
||||
@@ -13,6 +13,8 @@ Node PTC Host 要求 bootstrap 协议版本从 2 升至 3。它会在派发程
|
||||
|
||||
版本 3 为每个新环境携带可选的 `ProgramBootData.initialization` 源码。bootstrap 安装本次绑定后独立执行初始化,随后等待 Host 对 `initialized()` RPC 的确认,再次检查取消状态后才执行模型程序。初始化失败会丢弃环境并阻止模型代码执行。Node 提供方仅在 `persistent-in-memory` 模式下支持注册初始化程序。
|
||||
|
||||
版本 3 还携带 `ProgramBootData.discardCompletion`,用于在 JSON 校验和序列化前丢弃完成值。bootstrap 提供 `inspect`,并通过 `HostMethods.image(id)` 按输出顺序发送选中的图片。自定义 bootstrap 必须同时支持这些字段、方法和环境初始化。
|
||||
|
||||
## 迁移
|
||||
|
||||
1. 如果 Node PTC 配置设置了 `launch.bootstrapPath`,请在其执行环境中部署匹配版本的 `@deepseek-ai/dsh-ptc-runtime-node`。使用包内构建产物 `lib/process.js`,或默认采用 REPL 的 `lib/persistent-process.js`,并保留其依赖。Host 与 bootstrap 版本必须匹配;只修改声明的版本号并不够。
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
# Bilingual-pair consistency record for guide.md (docs/i18n/README.md): per heading
|
||||
# section, a hash of its English and Chinese blocks outside code blocks and generated regions.
|
||||
# After editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/upgrade-guide/v0.2.2-rc.1/ptc-inspect-output/guide.md
|
||||
/:
|
||||
en: aa6195ec0952307e
|
||||
zh: 1d80177284309ac6
|
||||
/explicit-output-in-typescript-ptc-programs:
|
||||
en: 064ec0081b5c2946
|
||||
zh: 4b67a513ea949220
|
||||
/explicit-output-in-typescript-ptc-programs/change:
|
||||
en: a4b3d449a4eedd9f
|
||||
zh: ae9b0ba7f9f4acbd
|
||||
/explicit-output-in-typescript-ptc-programs/migration:
|
||||
en: 0953d3ac2f2f01e3
|
||||
zh: b3614bf85d3d2fcf
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
kind: upgrade-guide
|
||||
description: "TypeScript PTC programs display selected values through inspect and return ordered text and image output."
|
||||
---
|
||||
|
||||
# Explicit output in TypeScript PTC programs
|
||||
|
||||
English | [中文](guide.zh.md)
|
||||
|
||||
## Change
|
||||
|
||||
TypeScript `run_code` programs display selected values through `inspect(...values)`. Function return values and final expressions previously appeared in the result; they are now omitted. Each inspected argument is rendered independently: strings appear directly, and other values use Node's object formatting without depth, array-length or string-length preview limits. Undefined properties, Error objects and circular references remain visible.
|
||||
|
||||
CUA observations previously produced output automatically. `getCapabilities()`, `listApps()`, `findApps(query)`, `getState()`, `app.getWindows()` and app or window `getAccessibilityTree()` now return their results to the program. Use `inspect` to display the values you need.
|
||||
|
||||
Only CUA screenshots and `read_image` return ordinary `Image` values in TypeScript programs. Their `path` points to an attachment file readable in the execution environment; `width` and `height` describe its stored pixels. Pass selected images to `inspect` alongside explanatory text, including images in arrays, Map entries, Set values and enumerable object properties. Nested image selection follows Node's custom formatting. Read CUA window IDs from the window object’s `windowId`.
|
||||
|
||||
Other image-producing tools retain their declared program values and automatically attach their pictures to the model context. Image adaptation requires a readable attachment path in the program filesystem. `fs-ssh` does not provide this mapping, so `read_image` and CUA screenshot bindings fail in that composition. Use a local filesystem composition for program Images.
|
||||
|
||||
Formatting selected values can require more memory than the retained output; see the [Node formatting memory limit](../../../../packages/ptc-runtime/ptc-runtime-node/README.md#known-limitations-and-deferred-work).
|
||||
|
||||
## Migration
|
||||
|
||||
1. Update programs that rely on completion values to explicitly inspect the values they need:
|
||||
|
||||
```js
|
||||
const result = await tools.bash({ command: 'pwd', description: 'Show current directory' })
|
||||
inspect(result)
|
||||
```
|
||||
|
||||
2. Remove output callbacks from `connectCua` and `createCuaApi`, the `emit` observation option and the unused `disableDiffing` AX option. Pass observations you want to display to `inspect`, for example `inspect(await app.getWindows())`. A custom dispatcher must return a program Image for screenshot operations.
|
||||
|
||||
```js
|
||||
const { connectCua } = await import("<host-provided bridge file URL>")
|
||||
const cua = connectCua(() => tools.cua)
|
||||
const app = await cua.getApp("TextEdit")
|
||||
const windows = await app.getWindows()
|
||||
const win = await app.getWindow(windows[0].windowId)
|
||||
const before = await win.getScreenshot()
|
||||
await app.pressKey("Tab")
|
||||
const after = await win.getScreenshot()
|
||||
inspect("before", { windowId: win.windowId }, before, "after", after)
|
||||
```
|
||||
|
||||
3. Direct runtime consumers can set `discardCompletion: true` to select the same explicit-output behavior.
|
||||
4. Third-party `PtcRuntime` providers replace returned `logs` arrays with `output: PtcOutput[]`; direct runtime consumers read `result.output`. Text entries are `{ type: 'text', text }`, image entries are `{ type: 'image', attachment }`, and array order is display order. Preserve emitted line breaks and concatenate adjacent text entries without separators.
|
||||
5. For custom `launch.bootstrapPath` deployments, follow the [Node PTC bootstrap upgrade guide](../ptc-bootstrap-protocol/guide.md).
|
||||
6. Run an updated program and verify that inspected values appear once and its final expression adds no output.
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
kind: upgrade-guide
|
||||
description: "TypeScript PTC 程序通过 inspect 展示选定的值,并返回有序的文本与图片输出。"
|
||||
---
|
||||
|
||||
# TypeScript PTC 程序的显式输出
|
||||
|
||||
[English](guide.md) | 中文
|
||||
|
||||
## 变更
|
||||
|
||||
TypeScript `run_code` 程序通过 `inspect(...values)` 展示选定的值。此前函数返回值和末尾表达式的值会进入结果,现在会被省略。各个 inspect 参数独立渲染:字符串直接展示,其他值使用 Node 的对象格式,不限制预览深度、数组项数或字符串长度。undefined 属性、Error 对象和循环引用均会展示。
|
||||
|
||||
此前,CUA 观察方法默认会输出结果。现在,`getCapabilities()`、`listApps()`、`findApps(query)`、`getState()`、`app.getWindows()` 以及应用或窗口的 `getAccessibilityTree()` 都将结果返回给程序。需要展示哪些值,就把它们传给 `inspect`。
|
||||
|
||||
只有 CUA 截图与 `read_image` 在 TypeScript 程序中返回普通 `Image`,其 `path` 指向执行环境可读取的附件文件,`width` 与 `height` 为该文件的像素尺寸。使用 `inspect` 将选中的图片与说明一起展示;图片可嵌入数组、Map 条目、Set 的值以及可枚举的对象属性。嵌套图片的选择遵循 Node 的自定义格式化。CUA 窗口 ID 从窗口对象的 `windowId` 读取。
|
||||
|
||||
其他图片工具保留其声明的程序返回值,并将图片自动附加到模型上下文。图片适配要求附件在程序文件系统中有可读路径。`fs-ssh` 未提供这项映射,因此该组合中的 `read_image` 与 CUA 截图绑定会失败。需要程序 Image 时,使用本地文件系统组合。
|
||||
|
||||
格式化选定的值所需内存可能大于保留的输出;参见 [Node 格式化的内存限制](../../../../packages/ptc-runtime/ptc-runtime-node/README.zh.md#known-limitations-and-deferred-work)。
|
||||
|
||||
## 迁移
|
||||
|
||||
1. 将依赖完成值的程序,改为显式 inspect 所需的值:
|
||||
|
||||
```js
|
||||
const result = await tools.bash({ command: 'pwd', description: 'Show current directory' })
|
||||
inspect(result)
|
||||
```
|
||||
|
||||
2. 删除 `connectCua` 和 `createCuaApi` 的输出回调、观察选项 `emit` 和已无作用的 AX 选项 `disableDiffing`。将要展示的观察结果传给 `inspect`,例如 `inspect(await app.getWindows())`。自定义分派器须为截图操作直接返回程序可用的 `Image`。
|
||||
|
||||
```js
|
||||
const { connectCua } = await import("<host-provided bridge file URL>")
|
||||
const cua = connectCua(() => tools.cua)
|
||||
const app = await cua.getApp("TextEdit")
|
||||
const windows = await app.getWindows()
|
||||
const win = await app.getWindow(windows[0].windowId)
|
||||
const before = await win.getScreenshot()
|
||||
await app.pressKey("Tab")
|
||||
const after = await win.getScreenshot()
|
||||
inspect("before", { windowId: win.windowId }, before, "after", after)
|
||||
```
|
||||
|
||||
3. 直接调用运行时的消费方可设置 `discardCompletion: true`,选择相同的显式输出行为。
|
||||
4. 第三方 `PtcRuntime` 提供方将返回的 `logs` 数组替换为 `output: PtcOutput[]`;直接消费运行时结果的调用方改为读取 `result.output`。文本条目为 `{ type: 'text', text }`,图片条目为 `{ type: 'image', attachment }`,数组顺序就是展示顺序。保留实际输出的换行,相邻文本条目直接拼接,不额外插入分隔符。
|
||||
5. 使用自定义 `launch.bootstrapPath` 的部署,请按 [Node PTC bootstrap 升级指南](../ptc-bootstrap-protocol/guide.zh.md)更新。
|
||||
6. 运行更新后的程序,确认 inspect 的值只出现一次,且末尾表达式不额外输出。
|
||||
@@ -18,8 +18,8 @@
|
||||
en: 4d8cb38024095705
|
||||
zh: 1baf6fa42f3fb80f
|
||||
/deepseek-ai-dsh-client-ui-tool/use-this-package/registering-a-business-tool-view:
|
||||
en: e458aff61e634190
|
||||
zh: 650e417f450b3edf
|
||||
en: 3dd6bd71dfc5527d
|
||||
zh: 9fe3c2135014bb7b
|
||||
/deepseek-ai-dsh-client-ui-tool/use-this-package/built-in-views:
|
||||
en: 33f41ad70d0728f7
|
||||
zh: b7c65108c5921134
|
||||
|
||||
@@ -47,7 +47,7 @@ ctx.slots.inject('tool.call.toolview', () =>
|
||||
}, BusinessToolRow))
|
||||
```
|
||||
|
||||
The owner payload is `ToolCallOwnerProps`: `callId`, `toolName`, the `phase` discriminant and its stage-specific `block`, optional `cwd` and `home`, the Gateway-authorized `loadImage` loader (for a view whose result carries durable images), and plain `openFile`/`inspect` callbacks. A PTC dispatch block retains its event's `parentCallId`; a root Session call has no such field, so descendants route through the same keyed dispatch — a registered view such as `read_image` renders its card there, and unregistered descendants keep the generic flattened form. Path summaries relativize to the Session cwd first, then replace a leftover POSIX Host home with `~`; `filePath` and Host open prefer the operation-time absolute result path while displayed labels keep the authored argument. Successful older root results retain their argument fallback; pending relative paths wait for a result. The registration receives the normal Session slot runtime share but no React node or Runtime service.
|
||||
The owner payload is `ToolCallOwnerProps`: `callId`, `toolName`, the `phase` discriminant and its stage-specific `block`, optional `cwd` and `home`, the Gateway-authorized `loadImage` loader (for a view whose result carries durable images), and plain `openFile`/`inspect` callbacks. A PTC dispatch block retains its event's `parentCallId`; a root Session call has no such field, so descendants route through the same keyed dispatch — a registered view such as `read_image` renders its card there, and unregistered descendants keep the generic flattened form. Path summaries relativize to the Session cwd first, then replace a leftover POSIX Host home with `~`; `filePath` and Host open prefer the operation-time absolute result path while displayed labels keep the authored argument. Successful older root results retain their argument fallback; pending relative paths wait for a result. The registration receives the normal Session slot runtime share but no React node or Runtime service. Generic result output preserves block order and displays image attachment names or IDs with dimensions and byte counts.
|
||||
|
||||
### Built-in views
|
||||
|
||||
|
||||
@@ -47,7 +47,7 @@ ctx.slots.inject('tool.call.toolview', () =>
|
||||
}, BusinessToolRow))
|
||||
```
|
||||
|
||||
owner 载荷为 `ToolCallOwnerProps`:`callId`、`toolName`、`phase` 判别字段及对应阶段的 `block`、可选 `cwd` 与 `home`、经 Gateway 授权的 `loadImage` loader(供结果携带持久图像的视图使用),以及普通的 `openFile`/`inspect` 回调。PTC dispatch 块保留事件的 `parentCallId`;根会话调用没有该字段,因此后代调用都走同一条按 key 分发路径:已注册视图的调用(如 `read_image`)也会在嵌套处渲染对应卡片,未注册的后代调用则保持通用压平形式。路径摘要先相对会话 cwd 缩短,再把剩余的 POSIX Host home 写成 `~`;`filePath` 与 Host 打开优先使用操作发生时记录的绝对结果路径,显示标签仍保留原始参数。成功的旧根调用结果继续回退到原参数;运行中的相对路径等待结果。注册项会收到常规的会话 slot 运行时共享数据,但不会收到 React 节点或运行时服务。
|
||||
owner 载荷为 `ToolCallOwnerProps`:`callId`、`toolName`、`phase` 判别字段及对应阶段的 `block`、可选 `cwd` 与 `home`、经 Gateway 授权的 `loadImage` loader(供结果携带持久图像的视图使用),以及普通的 `openFile`/`inspect` 回调。PTC dispatch 块保留事件的 `parentCallId`;根会话调用没有该字段,因此后代调用都走同一条按 key 分发路径:已注册视图的调用(如 `read_image`)也会在嵌套处渲染对应卡片,未注册的后代调用则保持通用压平形式。路径摘要先相对会话 cwd 缩短,再把剩余的 POSIX Host home 写成 `~`;`filePath` 与 Host 打开优先使用操作发生时记录的绝对结果路径,显示标签仍保留原始参数。成功的旧根调用结果继续回退到原参数;运行中的相对路径等待结果。注册项会收到常规的会话 slot 运行时共享数据,但不会收到 React 节点或运行时服务。 通用结果输出保留块的顺序,并以附件名称或 ID、尺寸和字节数展示图片引用。
|
||||
|
||||
### 内置视图
|
||||
|
||||
|
||||
@@ -203,21 +203,26 @@ function deriveAutoReviewDenial(block: ToolCallBlock): AutoReviewDenial | null {
|
||||
|
||||
/**
|
||||
* Flatten a settled result's content blocks to display text: text blocks
|
||||
* verbatim, other block shapes as pretty JSON. Empty content on a failed call
|
||||
* falls back to the structured error's `name: code` line.
|
||||
* verbatim, images through the supplied formatter, and other blocks
|
||||
* as pretty JSON. Empty failure content falls back to the structured error's
|
||||
* `name: code` line.
|
||||
* @param node - the settled result node.
|
||||
* @param formatImage - Optional formatter for unvalidated image attachments; omission or undefined preserves image JSON.
|
||||
* @returns the flattened result text (may be empty).
|
||||
*/
|
||||
export function resultText(node: ToolResultNode): string {
|
||||
export function resultText(node: ToolResultNode, formatImage?: (image: unknown) => string | undefined): string {
|
||||
const parts: string[] = []
|
||||
for (const block of node.content) {
|
||||
if (block.type === 'text') parts.push(block.text)
|
||||
else parts.push(JSON.stringify(block, null, 2))
|
||||
else {
|
||||
const image = block.type === 'image' ? formatImage?.(block.attachment) : undefined
|
||||
parts.push(image ?? JSON.stringify(block, null, 2))
|
||||
}
|
||||
}
|
||||
if (parts.length === 0 && node.error !== undefined) {
|
||||
parts.push(`${node.error.name}: ${node.error.code}`)
|
||||
}
|
||||
return parts.join('\n')
|
||||
return parts.reduce((text, part, index) => text + (index === 0 || text.endsWith('\n') ? '' : '\n') + part, '')
|
||||
}
|
||||
|
||||
function parseArgs(argsRaw: string): unknown {
|
||||
@@ -368,9 +373,13 @@ export function formatToolBody(variant: ToolRowVariant, argsRaw: string): string
|
||||
* @param block - preparing call, dispatched call, or result from the snapshot.
|
||||
* @param cwd - session workspace root; workspace-rooted path summaries display relative to it.
|
||||
* @param home - host account home; a leftover POSIX home path displays as `~`.
|
||||
* @param formatImage - Optional image formatter; omission preserves image JSON.
|
||||
* @returns the row model.
|
||||
*/
|
||||
export function toolRowModel(toolName: string, block: ToolCallBlock, cwd?: string, home?: string): ToolRowModel {
|
||||
export function toolRowModel(
|
||||
toolName: string, block: ToolCallBlock, cwd?: string, home?: string,
|
||||
formatImage?: (image: unknown) => string | undefined,
|
||||
): ToolRowModel {
|
||||
const variant = classifyTool(toolName)
|
||||
const titleKey = toolTitleKey(toolName)
|
||||
const done = 'kind' in block
|
||||
@@ -404,7 +413,7 @@ export function toolRowModel(toolName: string, block: ToolCallBlock, cwd?: strin
|
||||
// The empty string is "no text" for both derived result fields: a settled
|
||||
// call with blank content has nothing to expand, and a blank first line
|
||||
// would erase the collapsed error row's summary slot.
|
||||
const output = done ? (resultText(block) || null) : null
|
||||
const output = done ? (resultText(block, formatImage) || null) : null
|
||||
const errorSummary = state === 'error' && output !== null ? firstLine(output) : null
|
||||
const bodyRaw = argsRaw === '' ? null : argsRaw
|
||||
return {
|
||||
|
||||
@@ -36,7 +36,14 @@ export type GenericToolCardProps = ToolCallOwnerProps & {
|
||||
export function GenericToolCard({
|
||||
toolName, block, cwd, home, openFile, getFileTriggerProps, inspect, useDisclosure, t,
|
||||
}: GenericToolCardProps) {
|
||||
const model = toolRowModel(toolName, block, cwd, home)
|
||||
const model = toolRowModel(toolName, block, cwd, home, (image) => {
|
||||
if (typeof image !== 'object' || image === null || Array.isArray(image)) return undefined
|
||||
const { attachmentId, name, width, height, bytes } = image as Record<string, unknown>
|
||||
if (typeof attachmentId !== 'string' || attachmentId === ''
|
||||
|| name !== undefined && typeof name !== 'string'
|
||||
|| ![width, height, bytes].every(value => typeof value === 'number' && Number.isInteger(value) && value > 0)) return undefined
|
||||
return `${name ?? attachmentId} (${t('row.imageSize', { width, height, bytes })})`
|
||||
})
|
||||
const autoReview = model.autoReviewDenial === null
|
||||
? null
|
||||
: localizeAutoReviewDenial(model.autoReviewDenial, t)
|
||||
|
||||
@@ -62,7 +62,7 @@ async function executeShell(text: string, nested: boolean, name = 'bash', maxInl
|
||||
if (tool === undefined) throw new Error('missing fixture binding')
|
||||
const blocks = await tool(shellArgs)
|
||||
expect(blocks).toEqual([{ type: 'text', text }])
|
||||
return { logs: [], value: true }
|
||||
return { output: [{ type: 'text', text: 'true' }] }
|
||||
}
|
||||
}
|
||||
await ctx.plugin(BindingRuntime)
|
||||
@@ -86,7 +86,7 @@ async function executeShell(text: string, nested: boolean, name = 'bash', maxInl
|
||||
signal: new AbortController().signal,
|
||||
name: nested ? 'run_code' : name,
|
||||
arguments: nested ? {
|
||||
code: `const blocks = await tools.${name}(${JSON.stringify(shellArgs)}); return blocks[0].text === ${JSON.stringify(text)};`,
|
||||
code: `const blocks = await tools.${name}(${JSON.stringify(shellArgs)}); inspect(blocks[0].text === ${JSON.stringify(text)});`,
|
||||
description: 'Dispatch shell fixture through the tool pipeline',
|
||||
} : shellArgs,
|
||||
})
|
||||
@@ -95,7 +95,7 @@ async function executeShell(text: string, nested: boolean, name = 'bash', maxInl
|
||||
|
||||
let block: ToolResultNode
|
||||
if (nested) {
|
||||
expect(result.value).toMatchObject({ result: true })
|
||||
expect(result.value).toMatchObject({ logs: ['true'] })
|
||||
const dispatches = session.snapshotEvents().filter(event => event.type === 'tool/ptc-dispatch')
|
||||
expect(dispatches).toHaveLength(1)
|
||||
const event = dispatches[0]!
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { SessionId } from '@deepseek-ai/dsh-session/types'
|
||||
import { AttachmentId } from '@deepseek-ai/dsh-attachment'
|
||||
// @vitest-environment jsdom
|
||||
import { contextMenuHook } from '@deepseek-ai/dsh-client-test-runtime'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
@@ -14,7 +15,7 @@ import {
|
||||
} from '../src/client/tool/models/tool-call-model.ts'
|
||||
import { ToolRow } from '../src/client/tool/components/ToolRow.tsx'
|
||||
import { GenericToolCard, type GenericToolCardProps } from '../src/client/tool/toolviews/GenericToolCard.tsx'
|
||||
import { zh } from '@deepseek-ai/dsh-client-ui-conversation/src/client/locales.ts'
|
||||
import { en, zh } from '@deepseek-ai/dsh-client-ui-conversation/src/client/locales.ts'
|
||||
import { PartialArguments } from '@deepseek-ai/dsh-util-values'
|
||||
|
||||
afterEach(() => {
|
||||
@@ -238,21 +239,50 @@ describe('tool-call-model', () => {
|
||||
.toBe('{\n "code": ""\n}')
|
||||
})
|
||||
|
||||
it('resultText flattens text blocks verbatim, other shapes as JSON, empty error content to name: code', () => {
|
||||
it('resultText retains text and image JSON and uses the structured failure when content is empty', () => {
|
||||
expect(resultText(result({ content: [{ type: 'text', text: 'a\nb' }] }))).toBe('a\nb')
|
||||
expect(resultText(result({ content: [{ type: 'text', text: 'a' }, { type: 'image', data: 'x' } as never] })))
|
||||
.toBe(`a\n${JSON.stringify({ type: 'image', data: 'x' }, null, 2)}`)
|
||||
const image = { type: 'image' as const, attachment: { attachmentId: AttachmentId('sha256:image'), mediaType: 'image/png' as const, width: 8, height: 4, bytes: 64 } }
|
||||
expect(resultText(result({ content: [image] }))).toBe(JSON.stringify(image, null, 2))
|
||||
const reasoning = { type: 'reasoning' as const, text: 'detail' }
|
||||
expect(resultText(result({ content: [reasoning] }))).toBe(JSON.stringify(reasoning, null, 2))
|
||||
expect(resultText(result({ content: [], isError: true, error: { name: 'ToolError', code: 'denied' } })))
|
||||
.toBe('ToolError: denied')
|
||||
expect(resultText(result({ content: [] }))).toBe('')
|
||||
})
|
||||
|
||||
it('preserves image JSON when the formatter declines', () => {
|
||||
for (const content of ['[{"type":"image"}]', '[{"type":"image","attachment":null}]',
|
||||
'[{"type":"image","attachment":{"width":"8"}}]']) {
|
||||
const blocks = JSON.parse(content) as ToolResultNode['content']
|
||||
expect(resultText(result({ content: blocks }), () => undefined)).toBe(JSON.stringify(blocks[0], null, 2))
|
||||
}
|
||||
})
|
||||
|
||||
it('separates image references without duplicating an emitted newline', () => {
|
||||
const attachment = { attachmentId: AttachmentId('sha256:image'), mediaType: 'image/png' as const, width: 8, height: 4, bytes: 64 }
|
||||
for (const text of ['a', 'a\n']) {
|
||||
expect(resultText(result({ content: [{ type: 'text', text }, { type: 'image', attachment }] }), () => 'sha256:image'))
|
||||
.toBe('a\nsha256:image')
|
||||
}
|
||||
expect(resultText(result({ content: [{ type: 'text', text: 'a\n\n' }, { type: 'image', attachment }] }), () => 'sha256:image'))
|
||||
.toBe('a\n\nsha256:image')
|
||||
})
|
||||
|
||||
it('derives output from the settled result and null while running or blank', () => {
|
||||
expect(toolRowModel('bash', result({ content: [{ type: 'text', text: 'out' }] })).output).toBe('out')
|
||||
expect(toolRowModel('bash', running()).output).toBeNull()
|
||||
expect(toolRowModel('bash', result({ content: [] })).output).toBeNull()
|
||||
})
|
||||
|
||||
it.each([false, true])('shares formatted image output with its error summary, failure=%s', (isError) => {
|
||||
const image = { type: 'image' as const, attachment: { attachmentId: AttachmentId('sha256:image'), mediaType: 'image/png' as const, width: 8, height: 4, bytes: 64 } }
|
||||
const formatImage = vi.fn(() => 'selected image')
|
||||
const model = toolRowModel('run_code', result({ content: [image], isError }), undefined, undefined, formatImage)
|
||||
expect(model.output).toBe('selected image')
|
||||
expect(model.errorSummary).toBe(isError ? 'selected image' : null)
|
||||
expect(formatImage).toHaveBeenCalledOnce()
|
||||
})
|
||||
|
||||
it('derives errorSummary as the first output line on error rows only', () => {
|
||||
const failed = result({ content: [{ type: 'text', text: 'boom\ndetail' }], isError: true })
|
||||
expect(toolRowModel('bash', failed).errorSummary).toBe('boom')
|
||||
@@ -591,6 +621,33 @@ describe('GenericToolCard', () => {
|
||||
useDisclosure, callId: 'c1', toolName, ...('kind' in block ? { phase: 'result' as const, block: block } : { phase: block.phase, block: block }), openFile: vi.fn(), t,
|
||||
})
|
||||
|
||||
it.each([
|
||||
[t, 'sha256:image (8x4px,64 字节)'],
|
||||
[makeTranslate(en), 'sha256:image (8x4px, 64 bytes)'],
|
||||
] as const)('localizes compact image references in the generic card', (translate, expected) => {
|
||||
const attachment = { attachmentId: AttachmentId('sha256:image'), mediaType: 'image/png' as const, width: 8, height: 4, bytes: 64 }
|
||||
const block = result({ content: [{ type: 'text', text: 'before\n' }, { type: 'image', attachment }] })
|
||||
const view = render(<GenericToolCard {...props('run_code', block)} t={translate} />)
|
||||
fireEvent.click(view.getByRole('button'))
|
||||
expect(view.container.textContent).toContain(expected)
|
||||
expect(view.container.querySelector('[class*="output"]')?.textContent ?? view.container.textContent).not.toContain('before\n\nsha256:')
|
||||
})
|
||||
|
||||
it.each([
|
||||
'{"type":"image"}',
|
||||
'{"type":"image","attachment":null}',
|
||||
'{"type":"image","attachment":[]}',
|
||||
'{"type":"image","attachment":{"attachmentId":"image","width":"8","height":4,"bytes":64}}',
|
||||
'{"type":"image","attachment":{"attachmentId":"image","width":8,"height":4,"bytes":0}}',
|
||||
'{"type":"image","attachment":{"attachmentId":"image","name":12,"width":8,"height":4,"bytes":64}}',
|
||||
'{"type":"image","attachment":{"width":8,"height":4,"bytes":64}}',
|
||||
])('renders malformed image references as JSON: %s', (json) => {
|
||||
const content = JSON.parse(`[${json}]`) as ToolResultNode['content']
|
||||
const view = render(<GenericToolCard {...props('run_code', result({ content }))} />)
|
||||
fireEvent.click(view.getByRole('button'))
|
||||
expect(view.container.textContent).toContain(JSON.stringify(content[0], null, 2))
|
||||
})
|
||||
|
||||
it('renders the classified variant row from the frozen slice', () => {
|
||||
const view = render(<GenericToolCard {...props('bash', result())}
|
||||
getFileTriggerProps={() => ({})} />)
|
||||
|
||||
@@ -942,7 +942,7 @@ describe('PTC mode native-tool denial through the agent loop', () => {
|
||||
readonly language = 'typescript'
|
||||
readonly isolation = 'fake' as const
|
||||
async run(_request: PtcRunRequest): Promise<PtcRunResult> {
|
||||
return { logs: [] }
|
||||
return { output: [] }
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ class StubRuntime extends PtcRuntime {
|
||||
readonly isolation = 'stub'
|
||||
|
||||
run(_request: PtcRunRequest): Promise<PtcRunResult> {
|
||||
return Promise.resolve({ logs: [] })
|
||||
return Promise.resolve({ output: [] })
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -21,8 +21,8 @@
|
||||
en: 78708b3779d999a9
|
||||
zh: ad6173f9f01630de
|
||||
/deepseek-ai-dsh-tool-executor-ptc/understand-the-implementation/ptc-mode:
|
||||
en: 5623dd6cdd26bda2
|
||||
zh: 0b93be49e84ef52c
|
||||
en: 7d0df521ba40881d
|
||||
zh: 65d4738c69cbe0d4
|
||||
/deepseek-ai-dsh-tool-executor-ptc/further-exploration:
|
||||
en: ee69078c38c655a9
|
||||
zh: fb0873407f23c8c5
|
||||
@@ -33,8 +33,8 @@
|
||||
en: eaefd7303f22977d
|
||||
zh: 9c425f9501128224
|
||||
/deepseek-ai-dsh-tool-executor-ptc/model-experience/ptc-mode-schema-and-system-prompt/what-the-model-sees:
|
||||
en: 3c4ee15c38a82543
|
||||
zh: c032ede27425f917
|
||||
en: e2f91d6cb1140eb5
|
||||
zh: 39f30237b11349c6
|
||||
/deepseek-ai-dsh-tool-executor-ptc/model-experience/ptc-mode-schema-and-system-prompt/what-the-model-sees/typescript-ptc-mode-sdk-instructions-with-bash:
|
||||
en: 3a8df15e4ae32f9e
|
||||
zh: fc4a800ba363e835
|
||||
@@ -60,8 +60,8 @@
|
||||
en: f4703836365dc3d4
|
||||
zh: a09ff48a6550fd69
|
||||
/deepseek-ai-dsh-tool-executor-ptc/known-limitations-and-deferred-work:
|
||||
en: 86d79fbb28aced73
|
||||
zh: aac0ffa1c3ad5e9a
|
||||
en: 14cbc970c6df692c
|
||||
zh: ae6ce4ddb596847c
|
||||
/deepseek-ai-dsh-tool-executor-ptc/known-limitations-and-deferred-work/dev-note:
|
||||
en: e67f633dc6fe7773
|
||||
zh: 3bb0c20c93f16f98
|
||||
@@ -64,7 +64,7 @@ The PTC executor exposes the reserved `run_code` transport and a deterministic S
|
||||
|
||||
New sub-calls use `<parent>:ptc:<n>` ids. Consumers treat these ids as opaque and correlate events by exact equality; restored historical ids retain their original bytes. The [PTC mode decision](../../../.agents/notes/implemented/feature/2026-06-15-ptc.md) owns durable naming and restoration rules.
|
||||
|
||||
Successful image-bearing subcall results become deferred user-message context with `source.kind` set to `ptc-mode`. Other additional contexts retain their producing tool's attribution.
|
||||
TypeScript bindings for `read_image` and CUA screenshot operations return an `Image` with a readable `path`, `width`, and `height`. Passing it to `inspect` places its durable attachment among the captured text in the `run_code` result. Images nested in arrays and objects are displayed with their containing value. On program failure, inspected output accompanies the original diagnostic; outer result replacements remain authoritative. If an output policy removes a screenshot or read_image image, the binding rejects with ToolCallError and includes the retained text notice. Additional contexts retain their producing tool’s attribution.
|
||||
|
||||
When the caller cancels and the runtime reports `abort`, `run_code` waits for cleanup and returns the native tool cancellation result: `AbortError` / `ABORTED`, with `Error: tool call aborted` as its content. Retained runtimes append their cancellation diagnostic, including whether variables were preserved or cleared. The Agent records the specific cancellation cause in `turn/end`, including a hook's reason. Other runtime failures retain `CODE_RUN_FAILED` and their diagnostics, even if the caller also cancels. Cancellation can discard retained runtime state and does not undo completed tool effects.
|
||||
|
||||
@@ -95,7 +95,7 @@ Custom host integrations retain their `PtcExecutor` instance and call `reset(age
|
||||
|
||||
#### What the model sees
|
||||
|
||||
PTC mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-executor-ptc), the SDK instructions below, and the generated exact SDK block for the loaded runtime's language. Both runtime languages list `code` before `description`; this is presentation order, not an enforced JSON member order. REPL and Script runtimes share the generic tool description and SDK instructions; their `executionInstructions` supply syntax and state rules. The TypeScript bash example returns a value for function-body runtimes, and prints output for REPL and Script runtimes. The instructions identify generated declarations as program-only bindings. When the current `bash` parameter schema accepts the example arguments, they also show a complete `run_code` call around `tools.bash(...)`. The `tools:sdk` section uses first-party order 5000 and disables prompt-variable interpolation, preserving literal `{{…}}` text in tool descriptions and schemas for both runtime languages. Under `ptc`, the prompt also carries the `tools:ptc-only` rule earlier in the first-party order, so the model reads which tools it may call before it reads what each one is for. Exhausting the tool-call allowance returns a failed `run_code` result with the limit, rejected call number, termination notice, and possible infinite-loop warning. Runtime cleanup diagnostics and captured output remain visible.
|
||||
PTC mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-executor-ptc), the SDK instructions below, and the generated exact SDK block for the loaded runtime's language. Both runtime languages list `code` before `description`; this is presentation order, not an enforced JSON member order. REPL and Script runtimes share the generic tool description and SDK instructions; their `executionInstructions` supply syntax and state rules. The TypeScript SDK declares `inspect(...values)` and uses it in the bash example for every execution mode. The instructions identify generated declarations as program-only bindings. When the current `bash` parameter schema accepts the example arguments, they also show a complete `run_code` call around `tools.bash(...)`. The `tools:sdk` section uses first-party order 5000 and disables prompt-variable interpolation, preserving literal `{{…}}` text in tool descriptions and schemas for both runtime languages. Under `ptc`, the prompt also carries the `tools:ptc-only` rule earlier in the first-party order, so the model reads which tools it may call before it reads what each one is for. Exhausting the tool-call allowance returns a failed `run_code` result with the limit, rejected call number, termination notice, and possible infinite-loop warning. Runtime cleanup diagnostics and captured output remain visible. TypeScript programs show selected values with `inspect(...values)`. Each argument is rendered independently. The executor omits function returns and final expression values. The Node provider retains the complete native console and captures its stdout and stderr writes; model instructions direct programs to use `inspect`.
|
||||
|
||||
##### TypeScript PTC mode SDK instructions with bash
|
||||
|
||||
@@ -104,14 +104,14 @@ PTC mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md
|
||||
|
||||
`run_code` takes two required arguments: `code`, the body of an async TypeScript function (erasable syntax only — no `enum` or namespaces; type annotations are advisory, the code runs type-stripped), and `description`, a short summary of what the program does. The declarations below are SDK bindings for this program. A declaration does not make its name a directly callable tool; only names supplied as separate tool schemas may be called directly. When no separate `bash` schema is supplied, invoke a declared `bash` binding inside `run_code`:
|
||||
|
||||
`run_code({ code: "return await tools.bash({ description: 'Show current directory', command: 'pwd' })", description: "Show current directory" })`
|
||||
`run_code({ code: "inspect(await tools.bash({ description: 'Show current directory', command: 'pwd' }))", description: "Show current directory" })`
|
||||
|
||||
Inside the program:
|
||||
|
||||
- Call tools as `await tools.name(args)` — quoted access for exotic names: `tools["my-tool"](args)`. Every call resolves to the tool's typed canonical JSON value. Tool arguments must be lossless JSON.
|
||||
- Call tools as `await tools.name(args)` — quoted access for exotic names: `tools["my-tool"](args)`. Every call resolves to its declared value, which may be an Image. Tool arguments must be lossless JSON.
|
||||
- A FAILED tool call rejects with `ToolCallError`, whose `toolName` identifies the failed tool and whose `message` is human-readable — `try/catch` it to handle and continue.
|
||||
- Independent read-only calls MAY overlap under `Promise.all` (safe calls run concurrently; mutating calls run alone, in submission order). Sequence dependent work with `await`.
|
||||
- Emit results with `return` and/or `console.log(...)`. Only what you print or return is program output. A successful tool result containing an image is attached after the run so you can inspect it on the next step; every other intermediate result stays out of the conversation, so extract just what you need.
|
||||
- Show selected values and images with `inspect(...values)`; every other intermediate result stays out of the conversation, so extract just what you need.
|
||||
|
||||
Program-only SDK bindings:
|
||||
```
|
||||
@@ -150,6 +150,8 @@ The schema remains stable while the caller's description and runtime settings re
|
||||
|
||||
These constraints apply to the host integration and accepted calls.
|
||||
|
||||
- **Program Images cover read_image and CUA screenshots** — other image-producing tools keep their declared values and automatically attach their pictures as additional model context.
|
||||
- **Program Images require attachment path mapping** — the filesystem must map stored host attachments to readable program paths. `fs-ssh` has no such mapping, so image adaptation rejects in that composition.
|
||||
- **No built-in reset control** — the shipped Loader plugin does not expose its executor instance through a service, GUI control or SDK endpoint. Hosts needing reset must mount and retain `PtcExecutor` in their own composition; that integration owns retries after a failed reset.
|
||||
- **PTC mode's SDK language follows the one loaded runtime, and a presentation is per agent rather than per tool** — PTC execution rejects prompt assembly unless `ctx.ptcRuntime.language` has a registered SDK renderer; within one agent no tool can be native-only while another is ptc-only.
|
||||
- **PTC mode intermediate values are execution-local and unbounded by bytes** — they cannot be reconstructed from session replay and may exhaust process or worker memory; only the outer `run_code` output has the worker's configurable hard cap.
|
||||
|
||||
@@ -64,7 +64,7 @@ PTC 执行器公开保留的 `run_code` 传输,以及按所加载运行时语
|
||||
|
||||
新子调用使用 `<parent>:ptc:<n>` 标识。消费方将这些标识视为不透明值,并通过精确相等关联事件;恢复的历史标识保留原始字节。[PTC mode 决策](../../../.agents/notes/implemented/feature/2026-06-15-ptc.zh.md) 负责持久化命名与恢复规则。
|
||||
|
||||
成功且包含图片的子调用结果会成为延后的 user-message 上下文,其 `source.kind` 为 `ptc-mode`。其他 additional context 保留其生产工具的归属。
|
||||
TypeScript 中的 `read_image` 与 CUA 截图操作返回 `Image`,其中包含可读取的 `path`、`width` 和 `height`。将其传给 `inspect` 后,持久附件会按顺序与文本一起进入 `run_code` 结果。数组和对象内嵌的图片会随所在值一起展示。程序失败时,已 inspect 的输出随原始诊断一起呈现;外层替换的结果保持生效。输出策略移除截图或 read_image 图片时,绑定以 ToolCallError 拒绝,并包含保留的文本提示。额外上下文保留生成工具的归属。
|
||||
|
||||
调用方取消且运行时报告 `abort` 时,`run_code` 等待清理完成,并返回原生工具的取消结果:`AbortError` / `ABORTED`,内容为 `Error: tool call aborted`。保留状态的运行时会附加取消诊断,包括变量被保留还是清空。Agent 在 `turn/end` 中记录具体取消原因,包括 hook 的理由。其他运行时失败保留 `CODE_RUN_FAILED` 及其诊断,即使调用方同时取消。取消可能丢弃保留的运行时状态,也不会撤销已完成的工具操作。
|
||||
|
||||
@@ -95,7 +95,7 @@ PTC 执行器公开保留的 `run_code` 传输,以及按所加载运行时语
|
||||
|
||||
#### 模型看到什么
|
||||
|
||||
PTC mode 会公开生成的 [`run_code` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-executor-ptc)、下方 SDK 说明,以及按所加载运行时语言生成的精确 SDK 块。两种运行时语言都将 `code` 列在 `description` 前;这是呈现顺序,不强制 JSON 成员顺序。REPL 与 Script 运行时共用通用工具描述和 SDK 说明;语法与状态规则由各自的 `executionInstructions` 提供。TypeScript bash 示例对函数体运行时返回值,对 REPL 和 Script 运行时打印输出。说明将生成声明明确标为只能在程序内使用的绑定。当当前 `bash` 参数 schema 接受示例参数时,说明还会给出以 `run_code` 包住 `tools.bash(...)` 的完整调用。`tools:sdk` 段使用 first-party 顺序 5000,并关闭提示词变量插值,使两种运行时语言都原样保留工具描述和 schema 中的 `{{…}}` 文本。在 `ptc` 下,提示词还会带上处于更早 first-party 顺序的 `tools:ptc-only` 规则,让模型先读到「可以调用哪些工具」再读「每个工具做什么」。 工具调用额度耗尽时,`run_code` 返回失败结果,说明额度、被拒绝的调用序号、程序已终止,以及可能存在死循环。运行时清理诊断和已捕获的输出仍然可见。
|
||||
PTC mode 会公开生成的 [`run_code` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-executor-ptc)、下方 SDK 说明,以及按所加载运行时语言生成的精确 SDK 块。两种运行时语言都将 `code` 列在 `description` 前;这是呈现顺序,不强制 JSON 成员顺序。REPL 与 Script 运行时共用通用工具描述和 SDK 说明;语法与状态规则由各自的 `executionInstructions` 提供。TypeScript SDK 声明 `inspect(...values)`,并在所有执行模式的 bash 示例中使用它。说明将生成声明明确标为只能在程序内使用的绑定。当当前 `bash` 参数 schema 接受示例参数时,说明还会给出以 `run_code` 包住 `tools.bash(...)` 的完整调用。`tools:sdk` 段使用 first-party 顺序 5000,并关闭提示词变量插值,使两种运行时语言都原样保留工具描述和 schema 中的 `{{…}}` 文本。在 `ptc` 下,提示词还会带上处于更早 first-party 顺序的 `tools:ptc-only` 规则,让模型先读到「可以调用哪些工具」再读「每个工具做什么」。 工具调用额度耗尽时,`run_code` 返回失败结果,说明额度、被拒绝的调用序号、程序已终止,以及可能存在死循环。运行时清理诊断和已捕获的输出仍然可见。 TypeScript 程序通过 `inspect(...values)` 展示选定的值。每个参数独立渲染。执行器省略函数返回值和末尾表达式的值。Node 提供方保留完整的原生 console,并捕获其 stdout 和 stderr 写入;模型指引要求程序使用 `inspect`。
|
||||
|
||||
##### 带 bash 的 TypeScript PTC mode SDK 说明
|
||||
|
||||
@@ -104,14 +104,14 @@ PTC mode 会公开生成的 [`run_code` schema](../../../docs/tool-catalog.zh.md
|
||||
|
||||
`run_code` takes two required arguments: `code`, the body of an async TypeScript function (erasable syntax only — no `enum` or namespaces; type annotations are advisory, the code runs type-stripped), and `description`, a short summary of what the program does. The declarations below are SDK bindings for this program. A declaration does not make its name a directly callable tool; only names supplied as separate tool schemas may be called directly. When no separate `bash` schema is supplied, invoke a declared `bash` binding inside `run_code`:
|
||||
|
||||
`run_code({ code: "return await tools.bash({ description: 'Show current directory', command: 'pwd' })", description: "Show current directory" })`
|
||||
`run_code({ code: "inspect(await tools.bash({ description: 'Show current directory', command: 'pwd' }))", description: "Show current directory" })`
|
||||
|
||||
Inside the program:
|
||||
|
||||
- Call tools as `await tools.name(args)` — quoted access for exotic names: `tools["my-tool"](args)`. Every call resolves to the tool's typed canonical JSON value. Tool arguments must be lossless JSON.
|
||||
- Call tools as `await tools.name(args)` — quoted access for exotic names: `tools["my-tool"](args)`. Every call resolves to its declared value, which may be an Image. Tool arguments must be lossless JSON.
|
||||
- A FAILED tool call rejects with `ToolCallError`, whose `toolName` identifies the failed tool and whose `message` is human-readable — `try/catch` it to handle and continue.
|
||||
- Independent read-only calls MAY overlap under `Promise.all` (safe calls run concurrently; mutating calls run alone, in submission order). Sequence dependent work with `await`.
|
||||
- Emit results with `return` and/or `console.log(...)`. Only what you print or return is program output. A successful tool result containing an image is attached after the run so you can inspect it on the next step; every other intermediate result stays out of the conversation, so extract just what you need.
|
||||
- Show selected values and images with `inspect(...values)`; every other intermediate result stays out of the conversation, so extract just what you need.
|
||||
|
||||
Program-only SDK bindings:
|
||||
```
|
||||
@@ -150,6 +150,8 @@ Error: code run failed (abort): run_code exceeded its limit of 2048 tool calls (
|
||||
|
||||
以下约束适用于宿主集成及已开始的调用。
|
||||
|
||||
- **程序 Image 仅覆盖 read_image 与 CUA 截图**:其他图片工具保留其声明的返回值,并将图片自动附加到模型的额外上下文。
|
||||
- **程序 Image 要求附件路径映射**:文件系统必须将宿主存储的附件映射为程序可读的路径。`fs-ssh` 没有这项映射,因此该组合中的图片适配会拒绝。
|
||||
- **没有内置重置控件**:自带的 Loader 插件不通过服务、GUI 控件或 SDK 端点暴露执行器实例。需要重置的宿主必须在自己的组合中挂载并保留 `PtcExecutor`,该集成负责失败后的重试。
|
||||
- **PTC mode 的 SDK 语言由当前加载的运行时决定,且呈现方式按 agent 而非按工具**:PTC 会拒绝组装提示词,除非 `ctx.ptcRuntime.language` 有已注册的 SDK 渲染器;同一个 agent 内不能让一个工具仅使用 Native,而另一个仅使用 PTC。
|
||||
- **PTC mode 中间值只存在于执行局部,且没有字节上限**:它们无法从会话回放重建,并可能耗尽进程或 worker 内存;只有外层 `run_code` 输出受 worker 可配置的硬上限约束。
|
||||
|
||||
@@ -26,8 +26,9 @@ export class ToolCallLimitExceededError extends Error {
|
||||
* exception, a budget expiry, a runtime-local abort, or substrate death. Caller
|
||||
* cancellation uses the shared `AbortError` / `ABORTED` tool outcome. Extends
|
||||
* {@link HarnessError} (`code: 'CODE_RUN_FAILED'`); the registry's execution
|
||||
* pipeline converts it into a structured `isError` result whose text carries
|
||||
* the failure kind plus the captured logs, so the model can self-correct.
|
||||
* pipeline converts it into a structured `isError` result with the failure
|
||||
* kind. Captured text and images accompany the original diagnostic;
|
||||
* outer result replacements remain authoritative.
|
||||
*/
|
||||
export class CodeRunFailedError extends HarnessError {
|
||||
constructor(message: string) {
|
||||
|
||||
@@ -6,12 +6,15 @@
|
||||
* @module
|
||||
*/
|
||||
|
||||
import { isDeepStrictEqual } from 'node:util'
|
||||
import type { PtcDispatchLog } from './runtime-types.ts'
|
||||
import { symbols, type Context } from '@deepseek-ai/cordis'
|
||||
import { brandString } from '@deepseek-ai/dsh-brand'
|
||||
import { createUserMessage, HarnessError } from '@deepseek-ai/dsh-llm'
|
||||
import { createUserMessage, HarnessError, resolveImageAttachmentAccess } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
import type { ContentBlock, ToolCallId, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import { PtcImage } from '@deepseek-ai/dsh-ptc-runtime'
|
||||
import type { PtcOutput } from '@deepseek-ai/dsh-ptc-runtime'
|
||||
import type { PtcBindingFunction, PtcRunResult, PtcRunSandbox, PtcRuntime } from '@deepseek-ai/dsh-ptc-runtime'
|
||||
import { approveEscalation, ESCALATION_TARGETS, validateEscalationArgs } from '@deepseek-ai/dsh-sandbox'
|
||||
import type { SandboxExecutionPolicy } from '@deepseek-ai/dsh-sandbox'
|
||||
@@ -27,7 +30,7 @@ import type { PtcCalls } from './ptc-calls.ts'
|
||||
import { CodeRunFailedError, ToolCallLimitExceededError } from './errors.ts'
|
||||
import type { Config } from './config.ts'
|
||||
import type { RunCodeToolOptions } from './run-code.ts'
|
||||
import type { ToolDefinition, ToolExecutionResult, ToolRunContext } from '@deepseek-ai/dsh-tools'
|
||||
import type { ToolDefinition, ToolExecution, ToolExecutionFailure, ToolExecutionResult, ToolRunContext } from '@deepseek-ai/dsh-tools'
|
||||
import type {} from './types.ts'
|
||||
|
||||
/** The model-facing name of the PTC mode tool. */
|
||||
@@ -42,27 +45,26 @@ interface RunCodeFlavor {
|
||||
}
|
||||
|
||||
/** Keep tool-call rules here; the runtime owns evaluation syntax and state retention. */
|
||||
function runCodeFlavor(language: string, execution = {
|
||||
function runCodeFlavor(language: string, outputInstruction: string, execution = {
|
||||
programDescription: 'the program to execute',
|
||||
outputInstruction: 'Curate the program output.',
|
||||
codeDescription: `The ${language} program to execute, following this tool's execution instructions.`,
|
||||
}): RunCodeFlavor {
|
||||
return {
|
||||
description: `Execute a ${language} program against the available tools. Takes two required arguments: `
|
||||
+ `\`code\`, ${execution.programDescription}, and \`description\`, a short summary of what the program does. `
|
||||
+ 'Call tools as `await tools.name(args)` per the declarations in the system prompt. '
|
||||
+ `${execution.outputInstruction} Image-bearing subtool results are attached after the run.`,
|
||||
+ outputInstruction,
|
||||
codeDescription: execution.codeDescription,
|
||||
}
|
||||
}
|
||||
|
||||
const TYPESCRIPT_FLAVOR = runCodeFlavor('TypeScript')
|
||||
const TYPESCRIPT_FUNCTION_FLAVOR = runCodeFlavor('TypeScript', {
|
||||
const TYPESCRIPT_OUTPUT_INSTRUCTION = 'Show selected text and images with inspect(...values).'
|
||||
const TYPESCRIPT_FLAVOR = runCodeFlavor('TypeScript', TYPESCRIPT_OUTPUT_INSTRUCTION)
|
||||
const TYPESCRIPT_FUNCTION_FLAVOR = runCodeFlavor('TypeScript', TYPESCRIPT_OUTPUT_INSTRUCTION, {
|
||||
programDescription: 'the BODY of an async function (erasable syntax only; top-level `await` and `return` work)',
|
||||
outputInstruction: 'Only what you print or return is program output — curate it.',
|
||||
codeDescription: 'The program: the body of an async TypeScript function.',
|
||||
})
|
||||
const PYTHON_FLAVOR = runCodeFlavor('Python')
|
||||
const PYTHON_FLAVOR = runCodeFlavor('Python', 'Curate the program output. Image-bearing subtool results are attached after the run.')
|
||||
|
||||
/** Languages with both a transport schema and an SDK renderer. */
|
||||
export type PtcSdkLanguage = 'typescript' | 'python'
|
||||
@@ -255,6 +257,33 @@ function renderValue(value: JsonValue): string {
|
||||
/** Canonical value returned by the outer PTC mode transport. */
|
||||
type RunCodeOutput = { logs: string[]; result?: JsonValue; sandbox?: PtcRunSandbox }
|
||||
|
||||
/** Render captured text, selected images, and the applied file policy in emission order. */
|
||||
function renderRunOutput(
|
||||
value: RunCodeOutput,
|
||||
guidance: string,
|
||||
output: PtcOutput[] = value.logs.map(text => ({ type: 'text', text })),
|
||||
toolName: string,
|
||||
): ContentBlock[] {
|
||||
const content: ContentBlock[] = []
|
||||
const appendText = (text: string): void => { if (text.length > 0) content.push({ type: 'text', text }) }
|
||||
const remaining: string[] = []
|
||||
for (const block of output) {
|
||||
if (block.type === 'text') remaining.push(block.text)
|
||||
else {
|
||||
appendText(remaining.join(''))
|
||||
remaining.length = 0
|
||||
content.push(block)
|
||||
}
|
||||
}
|
||||
appendText(remaining.join(''))
|
||||
const details: string[] = []
|
||||
if (value.result !== undefined) details.push(renderValue(value.result))
|
||||
if (value.sandbox?.enforcement === 'partial') details.push('File sandbox enforcement is partial on this host.')
|
||||
if (value.sandbox?.denied) details.push(`The ${value.sandbox.mode} file sandbox denied an operation.${guidance}`)
|
||||
appendText(details.filter(text => text.length > 0).join('\n'))
|
||||
return content.length > 0 ? content : [{ type: 'text', text: `(${toolName} completed with no output)` }]
|
||||
}
|
||||
|
||||
/** Unowned programs do not require Session directory state. */
|
||||
async function resolveWorkingDirectory(ctx: Context, exec: ToolRunContext): Promise<string | undefined> {
|
||||
if (exec.agent === undefined) return undefined
|
||||
@@ -306,6 +335,20 @@ export function createRunCodeTransport(
|
||||
}
|
||||
}
|
||||
|
||||
type ImageRef = Extract<ContentBlock, { type: 'image' }>['attachment']
|
||||
const inspections = new WeakMap<ToolExecution, {
|
||||
value?: RunCodeOutput
|
||||
failure?: Pick<ToolExecutionFailure, 'error' | 'content'>
|
||||
output: PtcOutput[]
|
||||
}>()
|
||||
const programImage = (ref: ImageRef): PtcImage => {
|
||||
const attachments = ctx.get('attachments')
|
||||
const fs = ctx.get('fs')
|
||||
const access = attachments === undefined || fs === undefined ? undefined
|
||||
: resolveImageAttachmentAccess(attachments, path => fs.processPathFromHostPath(path), ref)
|
||||
if (access === undefined) throw new Error('Image storage must expose a readable path in the program filesystem')
|
||||
return new PtcImage(access.readonlyPath, ref)
|
||||
}
|
||||
const stateOwners = new WeakMap<object, WeakSet<PtcRuntime>>()
|
||||
const definition = defineTool({
|
||||
name: toolName,
|
||||
@@ -343,13 +386,8 @@ export function createRunCodeTransport(
|
||||
},
|
||||
},
|
||||
},
|
||||
render: (_args, value) => {
|
||||
const rendered = value.result === undefined ? '' : renderValue(value.result)
|
||||
const parts = [value.logs.join('\n'), rendered].filter(part => part.length > 0)
|
||||
if (value.sandbox?.enforcement === 'partial') parts.push('File sandbox enforcement is partial on this host.')
|
||||
if (value.sandbox?.denied) parts.push(`The ${value.sandbox.mode} file sandbox denied an operation.${escalationGuidance(options === undefined ? ctx.get('ptcRuntime')?.sandboxMode : options.runtime.sandboxMode)}`)
|
||||
return [{ type: 'text', text: parts.length > 0 ? parts.join('\n') : `(${toolName} completed with no output)` }]
|
||||
},
|
||||
render: (_args, value) => renderRunOutput(value,
|
||||
escalationGuidance(options === undefined ? ctx.get('ptcRuntime')?.sandboxMode : options.runtime.sandboxMode), undefined, toolName),
|
||||
},
|
||||
async execute(args, exec): Promise<RunCodeOutput> {
|
||||
if (args.description.trim().length === 0) {
|
||||
@@ -515,6 +553,8 @@ export function createRunCodeTransport(
|
||||
|
||||
const binding = (schema: ToolSchema): PtcBindingFunction => async (rawArgs: unknown): Promise<JsonValue> => {
|
||||
const { name } = schema
|
||||
// FIXME: Let image-producing tools declare their program result adaptation.
|
||||
const usesProgramImage = runtime.language === 'typescript' && (name === 'read_image' || name === 'cua')
|
||||
if (runOver()) {
|
||||
throw new Error(`${toolName} run is over (${String(runController.signal.reason)}); ${name} not dispatched`)
|
||||
}
|
||||
@@ -538,13 +578,14 @@ export function createRunCodeTransport(
|
||||
parent: exec.token,
|
||||
signal: runController.signal,
|
||||
}
|
||||
type DispatchOutcome = { isError: true; message: string } | { isError: false; value: JsonValue }
|
||||
type DispatchOutcome = { isError: true; message: string }
|
||||
| { isError: false; value: JsonValue; content: ContentBlock[]; imageRemoved: boolean }
|
||||
const outcome = await new Promise<DispatchOutcome>((resolve, reject) => {
|
||||
// Set by the dispatch stage (or start() for a pre-settled result): what commit() finalizes in submission order.
|
||||
let parked:
|
||||
| { kind: 'post-result' | 'final-result'; exec: ToolRunContext; result: ToolExecutionResult }
|
||||
| undefined
|
||||
const settle = (result: ToolExecutionResult): void => {
|
||||
const settle = (result: ToolExecutionResult, imageRemoved: boolean): void => {
|
||||
// The program gets its value NOW: the log-content listener (for
|
||||
// example, a spill backend) must never delay the binding or occupy
|
||||
// a dispatch slot. The event append is tracked side work; the run's
|
||||
@@ -553,7 +594,7 @@ export function createRunCodeTransport(
|
||||
// chain cannot reject).
|
||||
resolve(result.isError
|
||||
? { isError: true, message: result.error.message }
|
||||
: { isError: false, value: result.value })
|
||||
: { isError: false, value: result.value, content: result.content, imageRemoved })
|
||||
const agent = exec.agent
|
||||
if (agent === undefined) return
|
||||
const task: Promise<void> = (async () => {
|
||||
@@ -619,7 +660,7 @@ export function createRunCodeTransport(
|
||||
/* v8 ignore next -- commit() runs only after `settled` flipped, which set parked. */
|
||||
if (parked === undefined) return
|
||||
const result = await executor.complete(parked.exec, parked)
|
||||
if (!result.isError && result.content.some(block => block.type === 'image')) {
|
||||
if (!usesProgramImage && !result.isError && result.content.some(block => block.type === 'image')) {
|
||||
exec.deferContext(createUserMessage({
|
||||
content: result.content,
|
||||
source: { kind: 'ptc-mode' },
|
||||
@@ -634,7 +675,8 @@ export function createRunCodeTransport(
|
||||
// (ToolExecutionFailure types it never), so a policy-converted
|
||||
// failure cannot stop the turn through a recovering program.
|
||||
if (result.concludesTurn) exec.concludeTurn()
|
||||
settle(result)
|
||||
settle(result, parked.result.content.some(block => block.type === 'image')
|
||||
&& !result.content.some(block => block.type === 'image'))
|
||||
// Backpressure on pending event-append tasks: each task retains
|
||||
// a full result while a slow backend stores it, so the pool cap
|
||||
// bounds their count. Beyond the cap, the
|
||||
@@ -656,6 +698,16 @@ export function createRunCodeTransport(
|
||||
// only the binding name. Native content and internal error metadata
|
||||
// stay outside the program-facing failure contract.
|
||||
if (outcome.isError) throw new Error(outcome.message)
|
||||
if (usesProgramImage) {
|
||||
const images = outcome.content.filter(block => block.type === 'image')
|
||||
const first = images[0]
|
||||
if (first !== undefined && images.length === 1) return programImage(first.attachment)
|
||||
if (outcome.imageRemoved) {
|
||||
throw new Error('Image output was removed by a tool output policy.\n'
|
||||
+ outcome.content.filter(block => block.type === 'text').map(block => block.text).join('\n'))
|
||||
}
|
||||
if (name === 'read_image' || images.length > 1) throw new Error('Image-producing tools must return exactly one image')
|
||||
}
|
||||
return outcome.value
|
||||
}
|
||||
|
||||
@@ -694,6 +746,7 @@ export function createRunCodeTransport(
|
||||
}
|
||||
const spec = runtime.resolve({
|
||||
program: args.code,
|
||||
...runtime.language === 'typescript' ? { discardCompletion: true } : {},
|
||||
...exec.agent === undefined || policy?.mode !== standingPolicy?.mode ? {} : { stateOwner: exec.agent },
|
||||
bindings: [{
|
||||
global: 'tools',
|
||||
@@ -716,27 +769,41 @@ export function createRunCodeTransport(
|
||||
await drainDispatches()
|
||||
}
|
||||
|
||||
const logs = result.output.flatMap(block => block.type === 'text' ? [block.text] : [])
|
||||
const hasImages = result.output.some(block => block.type === 'image')
|
||||
if (result.error) {
|
||||
const limitError = runController.signal.reason instanceof ToolCallLimitExceededError
|
||||
? runController.signal.reason : undefined
|
||||
let failure: HarnessError
|
||||
if (result.error.kind === 'abort' && exec.signal.aborted && limitError === undefined) {
|
||||
const message = runtime.retainsState ? `tool call aborted: ${result.error.message}` : 'tool call aborted'
|
||||
const error = new HarnessError(message, TOOL_ABORTED)
|
||||
error.name = 'AbortError'
|
||||
throw error
|
||||
failure = new HarnessError(message, TOOL_ABORTED)
|
||||
failure.name = 'AbortError'
|
||||
} else {
|
||||
const logsText = logs.length > 0 && !hasImages
|
||||
? `\nCaptured output:\n${logs.join('')}` : ''
|
||||
const sandboxText = result.sandbox === undefined ? ''
|
||||
: `${logsText.endsWith('\n') ? '' : '\n'}File sandbox: ${result.sandbox.mode}${result.sandbox.enforcement === undefined ? '' : `; enforcement: ${result.sandbox.enforcement}`}${result.sandbox.denied ? '; operation denied' : ''}.`
|
||||
const failureText = limitError === undefined || limitError.message === result.error.message
|
||||
? result.error.message : `${limitError.message}\n${result.error.message}`
|
||||
failure = new CodeRunFailedError(`code run failed (${result.error.kind}): ${failureText}${logsText}${sandboxText}${result.sandbox?.denied ? escalationGuidance(runtime.sandboxMode) : ''}`)
|
||||
}
|
||||
const logsText = result.logs.length > 0 ? `\nCaptured output:\n${result.logs.join('\n')}` : ''
|
||||
const sandboxText = result.sandbox === undefined ? ''
|
||||
: `\nFile sandbox: ${result.sandbox.mode}${result.sandbox.enforcement === undefined ? '' : `; enforcement: ${result.sandbox.enforcement}`}${result.sandbox.denied ? '; operation denied' : ''}.`
|
||||
const failureText = limitError === undefined || limitError.message === result.error.message
|
||||
? result.error.message : `${limitError.message}\n${result.error.message}`
|
||||
throw new CodeRunFailedError(`code run failed (${result.error.kind}): ${failureText}${logsText}${sandboxText}${result.sandbox?.denied ? escalationGuidance(runtime.sandboxMode) : ''}`)
|
||||
if (hasImages) inspections.set(exec, {
|
||||
failure: {
|
||||
error: { message: failure.message, info: { name: failure.name, code: failure.code } },
|
||||
content: [{ type: 'text', text: `Error: ${failure.message}` }],
|
||||
},
|
||||
output: result.output,
|
||||
})
|
||||
throw failure
|
||||
}
|
||||
return {
|
||||
logs: result.logs,
|
||||
const value: RunCodeOutput = {
|
||||
logs,
|
||||
...result.sandbox === undefined ? {} : { sandbox: result.sandbox },
|
||||
...result.value !== undefined ? { result: result.value } : {},
|
||||
...runtime.language !== 'typescript' && result.value !== undefined ? { result: result.value } : {},
|
||||
}
|
||||
if (hasImages) inspections.set(exec, { value, output: result.output })
|
||||
return value
|
||||
} finally {
|
||||
callSignal.removeEventListener('abort', onOuterAbort)
|
||||
}
|
||||
@@ -745,6 +812,18 @@ export function createRunCodeTransport(
|
||||
: calls.run(exec.agent.session.id, exec.signal,
|
||||
() => runtime.sandboxMode === undefined ? undefined : resolveSandboxPolicy(ctx, exec).mode, run)
|
||||
},
|
||||
projectContent(exec, result) {
|
||||
const selected = inspections.get(exec)
|
||||
inspections.delete(exec)
|
||||
if (selected === undefined) return undefined
|
||||
if (result.isError) {
|
||||
if (!isDeepStrictEqual(selected.failure, { error: result.error, content: result.content })) return undefined
|
||||
return [...result.content, ...renderRunOutput({ logs: [] }, '', selected.output, toolName)]
|
||||
}
|
||||
if (selected.value === undefined || !isDeepStrictEqual(selected.value, result.value)) return undefined
|
||||
return renderRunOutput(selected.value,
|
||||
escalationGuidance(options === undefined ? ctx.get('ptcRuntime')?.sandboxMode : options.runtime.sandboxMode), selected.output, toolName)
|
||||
},
|
||||
// The model-authored description is the call's always-visible UI label
|
||||
// (the bash `description` precedent); the program itself rides rawInput.
|
||||
presentCall: args => ({
|
||||
|
||||
@@ -258,16 +258,13 @@ function renderSdkInstructions(programFormat: PtcProgramFormat): string {
|
||||
}
|
||||
|
||||
/** Render tool execution and output instructions for the selected execution syntax. */
|
||||
function renderSdkProgramInstructions(programFormat: PtcProgramFormat): string {
|
||||
const outputInstruction = programFormat === 'function-body'
|
||||
? 'Emit results with `return` and/or `console.log(...)`. Only what you print or return is program output.'
|
||||
: 'Only values emitted as program output enter the conversation.'
|
||||
function renderSdkProgramInstructions(): string {
|
||||
return `Inside the program:
|
||||
|
||||
- Call tools as \`await tools.name(args)\` — quoted access for exotic names: \`tools["my-tool"](args)\`. Every call resolves to the tool's typed canonical JSON value. Tool arguments must be lossless JSON.
|
||||
- Call tools as \`await tools.name(args)\` — quoted access for exotic names: \`tools["my-tool"](args)\`. Every call resolves to its declared value, which may be an Image. Tool arguments must be lossless JSON.
|
||||
- A FAILED tool call rejects with \`ToolCallError\`, whose \`toolName\` identifies the failed tool and whose \`message\` is human-readable — \`try/catch\` it to handle and continue.
|
||||
- Independent read-only calls MAY overlap under \`Promise.all\` (safe calls run concurrently; mutating calls run alone, in submission order). Sequence dependent work with \`await\`.
|
||||
- ${outputInstruction} A successful tool result containing an image is attached after the run so you can inspect it on the next step; every other intermediate result stays out of the conversation, so extract just what you need.
|
||||
- Show selected values and images with \`inspect(...values)\`; every other intermediate result stays out of the conversation, so extract just what you need.
|
||||
|
||||
Program-only SDK bindings:`
|
||||
}
|
||||
@@ -280,7 +277,7 @@ function acceptsExampleString(schema: JsonSchemaNode | undefined, value: string)
|
||||
}
|
||||
|
||||
/** Render the bash example only when its literal arguments satisfy the current parameter schema. */
|
||||
function renderBashExample(schemas: ToolSdkSchema[], programFormat: PtcProgramFormat): string {
|
||||
function renderBashExample(schemas: ToolSdkSchema[]): string {
|
||||
const bash = schemas.find(schema => schema.name === 'bash')
|
||||
if (bash === undefined) return ''
|
||||
const parameters = bash.parameters as JsonSchemaNode
|
||||
@@ -291,9 +288,7 @@ function renderBashExample(schemas: ToolSdkSchema[], programFormat: PtcProgramFo
|
||||
const needsDescription = required.includes('description')
|
||||
if (needsDescription && !acceptsExampleString(parameters.properties?.description, 'Show current directory')) return ''
|
||||
const description = needsDescription ? "description: 'Show current directory', " : ''
|
||||
const program = programFormat === 'function-body'
|
||||
? `return await tools.bash({ ${description}command: 'pwd' })`
|
||||
: `console.log(await tools.bash({ ${description}command: 'pwd' }))`
|
||||
const program = `inspect(await tools.bash({ ${description}command: 'pwd' }))`
|
||||
return ` When no separate \`bash\` schema is supplied, invoke a declared \`bash\` binding inside \`run_code\`:\n\n\`run_code({ code: "${program}", description: "Show current directory" })\``
|
||||
}
|
||||
|
||||
@@ -317,11 +312,16 @@ export function renderToolsSdk(schemas: readonly ToolSdkSchema[], programFormat:
|
||||
for (const schema of sorted) {
|
||||
argsMembers.push(...docLines(schema.description, 1))
|
||||
argsMembers.push(`${pad(1)}${renderKey(schema.name)}: ${jsonSchemaToTs(schema.parameters, 1)};`)
|
||||
outputMembers.push(`${pad(1)}${renderKey(schema.name)}: ${jsonSchemaToTs(schema.output, 1)};`)
|
||||
// FIXME: Derive program return types from tool-declared adaptations, together with the binding adapter in ptc.ts.
|
||||
const output = schema.name === 'read_image' ? 'Image'
|
||||
: `${schema.name === 'cua' ? 'Image | ' : ''}${jsonSchemaToTs(schema.output, 1)}`
|
||||
outputMembers.push(`${pad(1)}${renderKey(schema.name)}: ${output};`)
|
||||
}
|
||||
const argsMap = `interface ToolArgsMap {${argsMembers.length > 0 ? `\n${argsMembers.join('\n')}\n` : ''}}`
|
||||
const outputMap = `interface ToolOutputMap {${outputMembers.length > 0 ? `\n${outputMembers.join('\n')}\n` : ''}}`
|
||||
const declaration = [
|
||||
'declare class Image { private constructor(); readonly attachmentId: string; readonly path: string; readonly width: number; readonly height: number; }',
|
||||
'declare function inspect(...values: unknown[]): void;',
|
||||
argsMap,
|
||||
outputMap,
|
||||
'type ToolName = keyof ToolOutputMap',
|
||||
@@ -330,6 +330,6 @@ export function renderToolsSdk(schemas: readonly ToolSdkSchema[], programFormat:
|
||||
].join('\n\n')
|
||||
const jsonValue = 'type JsonValue = null | boolean | number | string | JsonValue[] | { [key: string]: JsonValue }'
|
||||
const instructions = renderSdkInstructions(programFormat)
|
||||
const programInstructions = renderSdkProgramInstructions(programFormat)
|
||||
return `${instructions}${renderBashExample(sorted, programFormat)}\n\n${programInstructions}\n\n\`\`\`ts\n${jsonValue}\n\n${declaration}\n\`\`\``
|
||||
const programInstructions = renderSdkProgramInstructions()
|
||||
return `${instructions}${renderBashExample(sorted)}\n\n${programInstructions}\n\n\`\`\`ts\n${jsonValue}\n\n${declaration}\n\`\`\``
|
||||
}
|
||||
@@ -25,7 +25,7 @@ try {
|
||||
const runtime = {
|
||||
language: 'typescript', programFormat: 'repl', retainsState: false,
|
||||
resolve(request) { return { ...request, cwd: process.cwd(), timeoutMs: 60000 } },
|
||||
async run() { return { logs: [], error: { kind: 'exception', message: 'probe failure' } } },
|
||||
async run() { return { output: [], error: { kind: 'exception', message: 'probe failure' } } },
|
||||
}
|
||||
const capability = defineTool({
|
||||
name: 'private_probe', description: 'Probe private binding.', parameters: {},
|
||||
@@ -43,10 +43,12 @@ try {
|
||||
error => error instanceof ptc.CodeRunFailedError && error.code === 'CODE_RUN_FAILED')
|
||||
runtime.run = async (spec) => {
|
||||
assert.deepEqual(Object.keys(spec.bindings[0].functions), ['private_probe'])
|
||||
return { logs: [], value: await spec.bindings[0].functions.private_probe({}) }
|
||||
assert.equal(spec.discardCompletion, true)
|
||||
assert.equal(await spec.bindings[0].functions.private_probe({}), 'private result')
|
||||
return { output: [{ type: 'text', text: 'private result' }] }
|
||||
}
|
||||
const result = await definition.execute({ code: 'probe', description: 'Check the private binding' }, execution)
|
||||
assert.deepEqual(result, { logs: [], result: 'private result' })
|
||||
assert.deepEqual(result, { logs: ['private result'] })
|
||||
assert.equal(ctx.tools.get('private_probe'), undefined)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
|
||||
@@ -49,7 +49,7 @@ class FakeRuntime extends PtcRuntime {
|
||||
|
||||
readonly language: string
|
||||
readonly isolation = 'fake'
|
||||
behavior: (request: PtcRunRequest) => Promise<PtcRunResult> = () => Promise.resolve({ logs: [] })
|
||||
behavior: (request: PtcRunRequest) => Promise<PtcRunResult> = () => Promise.resolve({ output: [] })
|
||||
lastRequest?: PtcRunRequest
|
||||
|
||||
constructor(ctx: Context, config: { language?: string } = {}) {
|
||||
@@ -63,6 +63,11 @@ class FakeRuntime extends PtcRuntime {
|
||||
}
|
||||
}
|
||||
|
||||
/** Output from a program that explicitly prints a selected JSON value. */
|
||||
function printed(value: JsonValue): PtcRunResult {
|
||||
return { output: [{ type: 'text', text: typeof value === 'string' ? value : JSON.stringify(value, null, 2) }] }
|
||||
}
|
||||
|
||||
interface SetupOptions {
|
||||
mode?: 'native' | 'ptc'
|
||||
maxSubCallsPerMinute?: number
|
||||
@@ -188,7 +193,7 @@ it('logs detached frozen nested metadata without changing the binding value or l
|
||||
expect(value).toBe('original contents')
|
||||
metadata.cwd = '/later'
|
||||
metadata.path = '/later/other.txt'
|
||||
return { logs: [], value }
|
||||
return printed(value)
|
||||
}
|
||||
const result = await runCode(ctx, 'return await tools.location({})', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -234,7 +239,7 @@ describe('mode-aware wire contribution', () => {
|
||||
ctx.tools.register(definition, { when: () => true })
|
||||
await expect(functions.conditional!({})).rejects.toThrow('unknown tool "conditional"')
|
||||
}
|
||||
return { logs: [] }
|
||||
return { output: [] }
|
||||
}
|
||||
const result = await executeCall(ctx, {
|
||||
agent, toolSet, signal: testToolSignal, callId: ToolCallId('conditional-program'),
|
||||
@@ -423,10 +428,7 @@ describe('mode-aware wire contribution', () => {
|
||||
expect(sdk).toContain('echo: {')
|
||||
expect(sdk).not.toContain('hidden:')
|
||||
|
||||
runtime.behavior = request => Promise.resolve({
|
||||
logs: [],
|
||||
value: Object.keys(request.bindings[0]!.functions).sort().join(','),
|
||||
})
|
||||
runtime.behavior = request => Promise.resolve(printed(Object.keys(request.bindings[0]!.functions).sort().join(',')))
|
||||
const result = await runCode(ctx, 'return Object.keys(tools)', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.content).toEqual([{ type: 'text', text: 'echo' }])
|
||||
@@ -450,10 +452,7 @@ describe('mode-aware wire contribution', () => {
|
||||
expect(sdk).not.toContain('denied:')
|
||||
expect(sdk).toContain('kept: {')
|
||||
|
||||
runtime.behavior = request => Promise.resolve({
|
||||
logs: [],
|
||||
value: Object.keys(request.bindings[0]!.functions).sort().join(','),
|
||||
})
|
||||
runtime.behavior = request => Promise.resolve(printed(Object.keys(request.bindings[0]!.functions).sort().join(',')))
|
||||
const result = await runCode(ctx, 'return Object.keys(tools)', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.content).toEqual([{ type: 'text', text: 'kept' }])
|
||||
@@ -519,15 +518,12 @@ describe('mode-aware wire contribution', () => {
|
||||
memberNameProperty: 'toolName',
|
||||
})
|
||||
const functions = request.bindings[0]!.functions
|
||||
return Promise.resolve({
|
||||
logs: [],
|
||||
value: JSON.stringify({
|
||||
names: Object.keys(functions).sort(),
|
||||
// Own-property AND prototype-chain reads both come back empty —
|
||||
// there is no handle a program could re-enter run_code through.
|
||||
runCode: String(functions[RUN_CODE_NAME]),
|
||||
}),
|
||||
})
|
||||
return Promise.resolve(printed(JSON.stringify({
|
||||
names: Object.keys(functions).sort(),
|
||||
// Own-property AND prototype-chain reads both come back empty —
|
||||
// there is no handle a program could re-enter run_code through.
|
||||
runCode: String(functions[RUN_CODE_NAME]),
|
||||
})))
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -688,7 +684,7 @@ describe('the per-program tool-call limit', () => {
|
||||
await expect(echo({ value: 'rejected' })).rejects.toThrow(message)
|
||||
expect(request.signal?.aborted).toBe(true)
|
||||
await expect(echo({ value: 'caught the error' })).rejects.toThrow('run_code run is over')
|
||||
return { logs: [], error: { kind: 'abort', message } }
|
||||
return { output: [], error: { kind: 'abort', message } }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.error).toMatchObject({ info: { code: 'CODE_RUN_FAILED' } })
|
||||
@@ -697,7 +693,7 @@ describe('the per-program tool-call limit', () => {
|
||||
expect(events.filter(event => event.type === 'tool/ptc-dispatch-start')).toHaveLength(2048)
|
||||
expect(events.filter(event => event.type === 'tool/ptc-dispatch')).toHaveLength(2048)
|
||||
|
||||
runtime.behavior = async request => ({ logs: [], value: await request.bindings[0]!.functions.echo!({ value: 'next run' }) })
|
||||
runtime.behavior = async request => (printed(await request.bindings[0]!.functions.echo!({ value: 'next run' })))
|
||||
expect((await runCode(ctx, 'next program', { agent })).isError).toBe(false)
|
||||
expect(calls).toHaveLength(2049)
|
||||
})
|
||||
@@ -721,7 +717,7 @@ describe('the per-program tool-call limit', () => {
|
||||
for (let n = 0; n < limit; n++) await echo({ value: String(n) })
|
||||
await expect(echo({ value: 'rejected' })).rejects.toThrow(`limit of ${limit} tool calls (4 per minute of its timeout); call ${limit + 1} was not dispatched`)
|
||||
expect(request.signal?.aborted).toBe(true)
|
||||
return { logs: [], error: { kind: 'abort', message: String(request.signal?.reason) } }
|
||||
return { output: [], error: { kind: 'abort', message: String(request.signal?.reason) } }
|
||||
}
|
||||
expect((await runCode(ctx, 'program', { timeoutMs: requestedMs })).isError).toBe(true)
|
||||
expect(calls).toHaveLength(limit)
|
||||
@@ -737,7 +733,7 @@ describe('the per-program tool-call limit', () => {
|
||||
const calls = registerEcho(ctx)
|
||||
runtime.behavior = async (request) => {
|
||||
for (let n = 0; n < 3; n++) await request.bindings[0]!.functions.echo!({ value: String(n) })
|
||||
return { logs: [] }
|
||||
return { output: [] }
|
||||
}
|
||||
expect((await runCode(ctx, 'program')).isError).toBe(false)
|
||||
expect(calls).toHaveLength(3)
|
||||
@@ -769,7 +765,7 @@ describe('the per-program tool-call limit', () => {
|
||||
void functions.queued!({ value: 'queued' }).catch(() => { outcomes.push('queued abandoned') })
|
||||
await expect(functions.queued!({ value: 'excess' })).rejects.toThrow('limit of 2 tool calls (1 per minute of its timeout); call 3 was not dispatched')
|
||||
expect(request.signal?.aborted).toBe(true)
|
||||
return { logs: [], error: { kind: 'abort', message: 'execution canceled' } }
|
||||
return { output: [], error: { kind: 'abort', message: 'execution canceled' } }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.error).toMatchObject({ info: { code: 'CODE_RUN_FAILED' } })
|
||||
@@ -798,7 +794,7 @@ describe('the per-program tool-call limit', () => {
|
||||
await expect(echo({ value: 'excess' })).rejects.toThrow('limit of 2 tool calls')
|
||||
expect(request.signal?.aborted).toBe(true)
|
||||
if (cancelCaller) controller.abort('later caller cancellation')
|
||||
return { logs: ['before termination'], error: failure }
|
||||
return { output: [{ type: 'text', text: 'before termination' }], error: failure }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { signal: controller.signal })
|
||||
expect(result.error).toMatchObject({ info: { code: 'CODE_RUN_FAILED' } })
|
||||
@@ -827,7 +823,7 @@ describe('the per-program tool-call limit', () => {
|
||||
await expect(functions.fail!({})).rejects.toThrow('tool failed')
|
||||
await expect(functions.echo!({ value: 'rejected' })).rejects.toThrow('limit of 3 tool calls (3 per minute of its timeout); call 4 was not dispatched')
|
||||
expect(request.signal?.aborted).toBe(true)
|
||||
return { logs: [], error: { kind: 'abort', message: String(request.signal?.reason) } }
|
||||
return { output: [], error: { kind: 'abort', message: String(request.signal?.reason) } }
|
||||
}
|
||||
expect((await runCode(ctx, 'program')).isError).toBe(true)
|
||||
expect(calls).toEqual([])
|
||||
@@ -885,13 +881,13 @@ describe('the sub-dispatch scheduler (native concurrency contract)', () => {
|
||||
// All three must be START-able without any completion (overlap proof).
|
||||
await expect.poll(() => gated.pending()).toBe(3)
|
||||
gated.releaseAll()
|
||||
return { logs: [], value: (await all).map(String).join(',') }
|
||||
return printed((await all).map(String).join(','))
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(gated.peakLive()).toBe(3)
|
||||
if (result.isError) throw new Error('expected success')
|
||||
expect(result.value).toMatchObject({ result: 'safe_read:a,safe_read:b,safe_read:c' })
|
||||
expect(result.value).toMatchObject({ logs: ['safe_read:a,safe_read:b,safe_read:c'] })
|
||||
// One start per dispatch, paired with its settle by subCallId, starts in submission order.
|
||||
const starts = events.filter(event => event.type === 'tool/ptc-dispatch-start').map(event => event.data as { subCallId: string })
|
||||
const settles = events.filter(event => event.type === 'tool/ptc-dispatch').map(event => event.data as { subCallId: string })
|
||||
@@ -919,7 +915,7 @@ describe('the sub-dispatch scheduler (native concurrency contract)', () => {
|
||||
await expect.poll(() => safe.pending()).toBe(1)
|
||||
safe.releaseAll()
|
||||
await Promise.all([...reads, write, tail])
|
||||
return { logs: [], value: 'ordered' }
|
||||
return printed('ordered')
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -941,9 +937,9 @@ describe('the sub-dispatch scheduler (native concurrency contract)', () => {
|
||||
await release.promise
|
||||
return next()
|
||||
})
|
||||
runtime.behavior = async request => ({ logs: [], value: await Promise.all(
|
||||
runtime.behavior = async request => (printed(await Promise.all(
|
||||
[0, 1, 2, 3].map(value => request.bindings[0]!.functions.echo!({ value: String(value) })),
|
||||
) })
|
||||
)))
|
||||
const running = runCode(ctx, 'program', { agent })
|
||||
try {
|
||||
await blocked.promise
|
||||
@@ -967,7 +963,7 @@ describe('the sub-dispatch scheduler (native concurrency contract)', () => {
|
||||
await expect.poll(() => gated.pending()).toBe(2)
|
||||
gated.releaseAll()
|
||||
await all
|
||||
return { logs: [], value: 'capped' }
|
||||
return printed('capped')
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -997,12 +993,12 @@ describe('the sub-dispatch scheduler (native concurrency contract)', () => {
|
||||
dispose()
|
||||
const message = await request.bindings[0]!.functions.ephemeral!({})
|
||||
.then(() => 'resolved', (error: unknown) => error instanceof Error ? error.message : String(error))
|
||||
return { logs: [], value: message }
|
||||
return printed(message)
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
if (result.isError) throw new Error('expected success')
|
||||
expect(result.value).toMatchObject({ result: 'unknown tool "ephemeral"' })
|
||||
expect(result.value).toMatchObject({ logs: ['unknown tool "ephemeral"'] })
|
||||
expect(calls).toEqual([])
|
||||
})
|
||||
|
||||
@@ -1032,7 +1028,7 @@ describe('the sub-dispatch scheduler (native concurrency contract)', () => {
|
||||
await expect.poll(() => gated.pending()).toBe(2)
|
||||
gated.releaseAll()
|
||||
await all
|
||||
return { logs: [], value: 'ordered-prepare' }
|
||||
return printed('ordered-prepare')
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -1072,7 +1068,7 @@ describe('the sub-dispatch scheduler (native concurrency contract)', () => {
|
||||
await expect.poll(() => reader.pending()).toBe(1)
|
||||
reader.releaseAll()
|
||||
await r
|
||||
return { logs: [], value: 'barrier-through-commit' }
|
||||
return printed('barrier-through-commit')
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -1098,7 +1094,7 @@ describe('the sub-dispatch scheduler (native concurrency contract)', () => {
|
||||
gated.release()
|
||||
await expect.poll(() => releasePost !== undefined).toBe(true)
|
||||
queueMicrotask(() => { releasePost!() })
|
||||
return { logs: [], value: 'returned-early' }
|
||||
return printed('returned-early')
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -1138,7 +1134,7 @@ describe('the sub-dispatch scheduler (native concurrency contract)', () => {
|
||||
gated.release() // releases a (FIFO gate) — invert: release twice reversed is not possible;
|
||||
gated.releaseAll()
|
||||
await all
|
||||
return { logs: [], value: 'ordered-commit' }
|
||||
return printed('ordered-commit')
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -1210,7 +1206,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
await releaseFirst.promise
|
||||
}
|
||||
const value = await request.bindings[0]!.functions.probe!({ value: request.program })
|
||||
return { logs: [], value: JSON.stringify(value) }
|
||||
return printed(JSON.stringify(value))
|
||||
}
|
||||
const first = runCode(ctx, 'first')
|
||||
try {
|
||||
@@ -1237,12 +1233,12 @@ describe('the run_code dispatch bridge', () => {
|
||||
const first = await tools.echo!({ value: 'one' })
|
||||
const second = await tools.echo!({ value: 'two' })
|
||||
if (typeof first !== 'string' || typeof second !== 'string') throw new Error('echo returned a non-string')
|
||||
return { logs: [`saw ${first}`], value: second }
|
||||
return { output: [{ type: 'text', text: `saw ${first}\n` }, { type: 'text', text: second }] }
|
||||
}
|
||||
const result = await runCode(ctx, 'const …: string = …', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
if (result.isError) throw new Error('expected run_code success')
|
||||
expect(result.value).toEqual({ logs: ['saw echo:one'], result: 'echo:two' })
|
||||
expect(result.value).toEqual({ logs: ['saw echo:one\n', 'echo:two'] })
|
||||
expect(result.content).toEqual([{ type: 'text', text: 'saw echo:one\necho:two' }])
|
||||
expect(calls).toEqual([{ value: 'one' }, { value: 'two' }])
|
||||
const dispatches = events.filter(event => event.type === 'tool/ptc-dispatch')
|
||||
@@ -1264,7 +1260,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
registerEcho(ctx)
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'nested' })
|
||||
return { logs: [], value: 'done' }
|
||||
return printed('done')
|
||||
}
|
||||
|
||||
// Freeze the nested observer's parent correlation. If that were the live
|
||||
@@ -1303,7 +1299,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.finalize!({})
|
||||
return { logs: [], value: 'program complete' }
|
||||
return printed('program complete')
|
||||
}
|
||||
|
||||
const concluded = await runCode(ctx, 'await tools.finalize({})')
|
||||
@@ -1318,7 +1314,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
})
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.finalize!({}).catch(() => undefined)
|
||||
return { logs: [], value: 'recovered' }
|
||||
return printed('recovered')
|
||||
}
|
||||
const recovered = await runCode(ctx, 'await tools.finalize({}).catch(() => {})')
|
||||
veto()
|
||||
@@ -1352,7 +1348,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
const tools = request.bindings[0]!.functions
|
||||
const values = await Promise.all([tools.probe!({ id: 'a' }), tools.probe!({ id: 'b' }), tools.probe!({ id: 'c' })])
|
||||
if (!values.every(value => typeof value === 'string')) throw new Error('probe returned a non-string')
|
||||
return { logs: [], value: values.join(',') }
|
||||
return printed(values.join(','))
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -1375,9 +1371,9 @@ describe('the run_code dispatch bridge', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
try {
|
||||
await request.bindings[0]!.functions.fail!({})
|
||||
return { logs: [], value: 'unreachable' }
|
||||
return printed('unreachable')
|
||||
} catch (error: unknown) {
|
||||
return { logs: [], value: `caught: ${error instanceof Error ? error.message : String(error)}` }
|
||||
return printed(`caught: ${error instanceof Error ? error.message : String(error)}`)
|
||||
}
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
@@ -1409,7 +1405,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
firstLog.resolve(undefined)
|
||||
expect(await third).toBe('echo:three')
|
||||
secondLog.resolve(undefined)
|
||||
return { logs: [], value: 'complete' }
|
||||
return printed('complete')
|
||||
}
|
||||
try {
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
@@ -1428,7 +1424,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
const { agent, events } = fakeAgent()
|
||||
runtime.behavior = async (request) => {
|
||||
const value = await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
return { logs: [], value: value as string }
|
||||
return printed(value)
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -1452,12 +1448,12 @@ describe('the run_code dispatch bridge', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
const message = await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
.then(() => 'resolved', (error: unknown) => error instanceof Error ? error.message : String(error))
|
||||
return { logs: [], value: message }
|
||||
return printed(message)
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
if (result.isError) throw new Error('expected success')
|
||||
expect(result.value).toMatchObject({ result: 'gate exploded' })
|
||||
expect(result.value).toMatchObject({ logs: ['gate exploded'] })
|
||||
// The pipeline failure is final: the body never ran and post-execute was
|
||||
// skipped, yet the settle event still carries the error outcome.
|
||||
expect(calls).toEqual([])
|
||||
@@ -1488,9 +1484,9 @@ describe('the run_code dispatch bridge', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
try {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
return { logs: [], value: 'unreachable' }
|
||||
return printed('unreachable')
|
||||
} catch (error: unknown) {
|
||||
return { logs: [], value: `denied: ${error instanceof Error ? error.message : String(error)}` }
|
||||
return printed(`denied: ${error instanceof Error ? error.message : String(error)}`)
|
||||
}
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
@@ -1539,7 +1535,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
const { agent, events } = fakeAgent()
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
return { logs: [], value: 'unreachable' }
|
||||
return printed('unreachable')
|
||||
}
|
||||
|
||||
const result = await runCode(ctx, 'await tools.echo({ value: "x" })', { agent })
|
||||
@@ -1567,9 +1563,9 @@ describe('the run_code dispatch bridge', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
try {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
return { logs: [], value: 'unreachable' }
|
||||
return printed('unreachable')
|
||||
} catch (error: unknown) {
|
||||
return { logs: [], value: error instanceof Error ? error.message : String(error) }
|
||||
return printed(error instanceof Error ? error.message : String(error))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1593,9 +1589,9 @@ describe('the run_code dispatch bridge', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
try {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x', big: 1n })
|
||||
return { logs: [], value: 'unreachable' }
|
||||
return printed('unreachable')
|
||||
} catch (error: unknown) {
|
||||
return { logs: [], value: error instanceof Error ? error.message : String(error) }
|
||||
return printed(error instanceof Error ? error.message : String(error))
|
||||
}
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
@@ -1611,7 +1607,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
const args = Object.assign(Object.create(null) as Record<string, unknown>, { value: 'x', nested: ['same'] })
|
||||
await request.bindings[0]!.functions.echo!(args)
|
||||
return { logs: [] }
|
||||
return { output: [] }
|
||||
}
|
||||
await runCode(ctx, 'program', { agent })
|
||||
expect(calls).toEqual([{ value: 'x', nested: ['same'] }])
|
||||
@@ -1637,7 +1633,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
await request.bindings[0]!.functions.echo!({ value: 'y' })
|
||||
return { logs: [], value: 'done' }
|
||||
return printed('done')
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -1655,8 +1651,8 @@ describe('the run_code dispatch bridge', () => {
|
||||
])
|
||||
})
|
||||
|
||||
it('defers image-bearing final sub-call content onto the outer run_code result', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
it.each(['typescript', 'python'])('records image-bearing sub-call results for a %s runtime', async (language) => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc', runtime: { language } })
|
||||
ctx.tools.register(defineContentToolFixture({
|
||||
name: 'image_result',
|
||||
description: 'Return one durable image.',
|
||||
@@ -1674,19 +1670,90 @@ describe('the run_code dispatch bridge', () => {
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.image_result!({})
|
||||
return { logs: [], value: 'done' }
|
||||
return printed('done')
|
||||
}
|
||||
|
||||
const result = await runCode(ctx, 'program')
|
||||
|
||||
expect(result.additionalContexts).toMatchObject([{
|
||||
role: 'user',
|
||||
source: { kind: 'ptc-mode' },
|
||||
content: [
|
||||
{ type: 'text', text: 'image result' },
|
||||
{ type: 'image', attachment: { mediaType: 'image/png', bytes: 1, width: 1, height: 1 } },
|
||||
{ type: 'image', attachment: { attachmentId: 'sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' } },
|
||||
],
|
||||
}])
|
||||
expect(result.content).toEqual([{ type: 'text', text: 'done' }])
|
||||
})
|
||||
|
||||
it.each([
|
||||
{ name: 'read_image', count: 0, message: 'Image-producing tools must return exactly one image' },
|
||||
{ name: 'read_image', count: 2, message: 'Image-producing tools must return exactly one image' },
|
||||
{ name: 'cua', count: 2, message: 'Image-producing tools must return exactly one image' },
|
||||
{ name: 'read_image', count: 1, message: 'Image storage must expose a readable path in the program filesystem' },
|
||||
])('rejects $name with $count images when its program image is unavailable', async ({ name, count, message }) => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
onTestFinished(() => ctx.fiber.dispose())
|
||||
ctx.tools.register(defineContentToolFixture({
|
||||
name, description: 'Return image content.', parameters: {},
|
||||
execute: () => Promise.resolve(Array.from({ length: count }, () => ({
|
||||
type: 'image' as const,
|
||||
attachment: {
|
||||
attachmentId: 'sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' as never,
|
||||
mediaType: 'image/png' as const, bytes: 1, width: 1, height: 1,
|
||||
},
|
||||
}))),
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions[name]!({})
|
||||
return { output: [] }
|
||||
}
|
||||
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.content).toEqual([{ type: 'text', text: expect.stringContaining(message) as string }])
|
||||
})
|
||||
|
||||
it('returns CUA text observations as program values', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
onTestFinished(() => ctx.fiber.dispose())
|
||||
ctx.tools.register(defineContentToolFixture({
|
||||
name: 'cua', description: 'Return a text observation.', parameters: {},
|
||||
execute: () => Promise.resolve([{ type: 'text', text: 'Window selected' }]),
|
||||
}))
|
||||
runtime.behavior = async request => printed(await request.bindings[0]!.functions.cua!({}))
|
||||
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.content).toEqual([{ type: 'text', text: JSON.stringify([{ type: 'text', text: 'Window selected' }], null, 2) }])
|
||||
})
|
||||
|
||||
it.each(['read_image', 'cua'])('reports the policy notice when %s image output is removed', async (name) => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
ctx.tools.register(defineContentToolFixture({
|
||||
name, description: 'Return image content.', parameters: {},
|
||||
execute: () => Promise.resolve([{
|
||||
type: 'image', attachment: {
|
||||
attachmentId: 'sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' as never,
|
||||
mediaType: 'image/png', bytes: 1, width: 1, height: 1,
|
||||
},
|
||||
}]),
|
||||
}))
|
||||
ctx.on('tools/post-execute', async (exec, _result, next): Promise<PostToolDecision> => {
|
||||
const decision = await next()
|
||||
return exec.name === name && decision.kind === 'accept'
|
||||
? { kind: 'accept', content: [{ type: 'text', text: 'image omitted; read the saved result' }] }
|
||||
: decision
|
||||
})
|
||||
runtime.behavior = async (request) => {
|
||||
await expect(request.bindings[0]!.functions[name]!({})).rejects.toThrow(
|
||||
'Image output was removed by a tool output policy.\nimage omitted; read the saved result',
|
||||
)
|
||||
return printed('recovered')
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.content).toEqual([{ type: 'text', text: 'recovered' }])
|
||||
expect(result.additionalContexts).toBeUndefined()
|
||||
})
|
||||
|
||||
it('does not defer images removed by a nested post-execute decision', async () => {
|
||||
@@ -1712,7 +1779,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
})
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.image_result!({}).catch(() => undefined)
|
||||
return { logs: [], value: 'done' }
|
||||
return printed('done')
|
||||
}
|
||||
|
||||
const result = await runCode(ctx, 'program')
|
||||
@@ -1737,7 +1804,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
})
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
return { logs: [], error: { kind: 'exception', message: 'program failed later' } }
|
||||
return { output: [], error: { kind: 'exception', message: 'program failed later' } }
|
||||
}
|
||||
|
||||
const result = await runCode(ctx, 'program')
|
||||
@@ -1751,13 +1818,24 @@ describe('the run_code dispatch bridge', () => {
|
||||
}])
|
||||
})
|
||||
|
||||
it.each([false, true])('preserves raw text fragments, failure=%s', async (failed) => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
runtime.behavior = () => Promise.resolve({
|
||||
output: ['sel', 'ected\n', 'log\n', 'warn\n', 'after\n'].map(text => ({ type: 'text' as const, text })),
|
||||
...failed ? { error: { kind: 'exception' as const, message: 'stopped' } } : {},
|
||||
})
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(failed)
|
||||
expect(result.content).toEqual([{ type: 'text', text: `${failed ? 'Error: code run failed (exception): stopped\nCaptured output:\n' : ''}selected\nlog\nwarn\nafter\n` }])
|
||||
})
|
||||
|
||||
it.each([
|
||||
['before code started', []],
|
||||
['during execution', ['got this far']],
|
||||
] as const)('reports a timeout %s with its captured output', async (phase, logs) => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
runtime.behavior = () => Promise.resolve({
|
||||
logs: [...logs],
|
||||
output: logs.map(text => ({ type: 'text' as const, text })),
|
||||
error: { kind: 'timeout', message: `execution timed out ${phase} (300ms); retained variables were cleared` },
|
||||
})
|
||||
const result = await runCode(ctx, 'program')
|
||||
@@ -1801,7 +1879,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
await Promise.all(calls)
|
||||
// A real runtime would be terminated by the abort; the fake honors the
|
||||
// contract by reporting the abort as the run failure.
|
||||
return { logs: [], error: { kind: 'abort', message: 'user-cancel' } }
|
||||
return { output: [], error: { kind: 'abort', message: 'user-cancel' } }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { signal: controller.signal })
|
||||
expect(result.isError).toBe(true)
|
||||
@@ -1850,7 +1928,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
const calls = registerEcho(ctx)
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
return { logs: [], value: 'ok' }
|
||||
return printed('ok')
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -1880,7 +1958,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
const { agent, events } = fakeAgent()
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'blocked' })
|
||||
return { logs: [] }
|
||||
return { output: [] }
|
||||
}
|
||||
|
||||
const result = await runCode(ctx, 'return await tools.echo({ value: "blocked" })', { agent })
|
||||
@@ -1918,10 +1996,10 @@ describe('the run_code dispatch bridge', () => {
|
||||
})
|
||||
|
||||
it.each([
|
||||
['logs only', { logs: ['printed'] }, 'printed'],
|
||||
['result only', { logs: [], value: 'returned' }, 'returned'],
|
||||
['logs plus result', { logs: ['printed'], value: 'returned' }, 'printed\nreturned'],
|
||||
['no output', { logs: [] }, '(run_code completed with no output)'],
|
||||
['logs only', { output: [{ type: 'text', text: 'printed' }] }, 'printed'],
|
||||
['completion only', { output: [], value: 'returned' }, '(run_code completed with no output)'],
|
||||
['logs plus completion', { output: [{ type: 'text', text: 'printed' }], value: 'returned' }, 'printed'],
|
||||
['no output', { output: [] }, '(run_code completed with no output)'],
|
||||
] as [string, PtcRunResult, string][])('keeps %s in durable content without a result presenter', async (_name, output, text) => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
runtime.behavior = () => Promise.resolve(output)
|
||||
@@ -1939,7 +2017,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
it('keeps a post-policy spill preview in durable content without a result presenter', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
const preview = 'HEAD\n\n(Omitted 100 bytes. Full formatted result stored at: /tmp/run-code.txt.)\n\nTAIL'
|
||||
runtime.behavior = () => Promise.resolve({ logs: ['printed'], value: 'returned' })
|
||||
runtime.behavior = () => Promise.resolve({ output: [{ type: 'text', text: 'printed' }], value: 'returned' })
|
||||
ctx.on('tools/post-execute', (exec, _result, next): Promise<PostToolDecision> => {
|
||||
if (exec.name !== RUN_CODE_NAME) return next()
|
||||
return Promise.resolve({ kind: 'accept', content: [{ type: 'text', text: preview }] })
|
||||
@@ -1955,7 +2033,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
it('keeps canonical failure content durable without a result presenter', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
runtime.behavior = () => Promise.resolve({
|
||||
logs: ['captured before failure'],
|
||||
output: [{ type: 'text', text: 'captured before failure' }],
|
||||
error: { kind: 'output-limit', message: 'outer output exceeded 8 bytes' },
|
||||
})
|
||||
|
||||
@@ -1991,7 +2069,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
const value = await request.bindings[0]!.functions.mixed!({})
|
||||
return { logs: [], value }
|
||||
return printed(value)
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -2010,19 +2088,16 @@ describe('the run_code dispatch bridge', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
const echo = request.bindings[0]!.functions.echo!
|
||||
const catchMessage = (promise: Promise<unknown>) => promise.then(() => 'resolved', (error: unknown) => error instanceof Error ? error.message : String(error))
|
||||
return {
|
||||
logs: [],
|
||||
value: [
|
||||
// Root undefined must reject up front: the event log rejects it as
|
||||
// data, and nothing may execute unlogged.
|
||||
await catchMessage(echo(undefined)),
|
||||
await catchMessage(echo(Object.defineProperty({}, 'bad', { enumerable: true, get() { throw 'raw-throw' } }))),
|
||||
await catchMessage(echo(Object.defineProperty({}, 'bad', { enumerable: true, get() { throw new Error('error-throw') } }))),
|
||||
await catchMessage(echo(new Date(0))),
|
||||
// A bare function is a value JSON cannot represent at all.
|
||||
await catchMessage(echo(() => 1)),
|
||||
].join(' | '),
|
||||
}
|
||||
return printed([
|
||||
// Root undefined must reject up front: the event log rejects it as
|
||||
// data, and nothing may execute unlogged.
|
||||
await catchMessage(echo(undefined)),
|
||||
await catchMessage(echo(Object.defineProperty({}, 'bad', { enumerable: true, get() { throw 'raw-throw' } }))),
|
||||
await catchMessage(echo(Object.defineProperty({}, 'bad', { enumerable: true, get() { throw new Error('error-throw') } }))),
|
||||
await catchMessage(echo(new Date(0))),
|
||||
// A bare function is a value JSON cannot represent at all.
|
||||
await catchMessage(echo(() => 1)),
|
||||
].join(' | '))
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
const text = (result.content[0] as { text: string }).text
|
||||
@@ -2065,13 +2140,13 @@ describe('the run_code dispatch bridge', () => {
|
||||
let nested: JsonValue = 'leaf'
|
||||
for (let index = 0; index < depth; index++) nested = [nested]
|
||||
const value = await request.bindings[0]!.functions.deep_args!({ nested })
|
||||
return { logs: [], value }
|
||||
return printed(value)
|
||||
}
|
||||
|
||||
const result = await runCode(ctx, 'return tools.deep_args(...)', { agent })
|
||||
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.isError ? undefined : result.value).toEqual({ logs: [], result: depth })
|
||||
expect(result.isError ? undefined : result.value).toEqual({ logs: [String(depth)] })
|
||||
expect({ observedDepth, observedLeaf }).toEqual({ observedDepth: depth, observedLeaf: 'leaf' })
|
||||
const dispatch = session.snapshotEvents().find(event => event.type === 'tool/ptc-dispatch')
|
||||
if (dispatch === undefined) throw new Error('expected a durable tool/ptc-dispatch event')
|
||||
@@ -2101,7 +2176,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.mutator!({ list: ['original'] })
|
||||
return { logs: [] }
|
||||
return { output: [] }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -2126,38 +2201,38 @@ describe('the run_code dispatch bridge', () => {
|
||||
const functions = request.bindings[0]!.functions
|
||||
expect(Object.getPrototypeOf(functions)).toBeNull()
|
||||
const value = await functions['__proto__']!({})
|
||||
return { logs: [], value }
|
||||
return printed(value)
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.content[0]).toEqual({ type: 'text', text: 'proto-tool-ok' })
|
||||
})
|
||||
|
||||
it('renders every non-string JSON root as pretty JSON while preserving strings raw', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
runtime.behavior = () => Promise.resolve({ logs: [], value: { n: 42, ok: true } })
|
||||
it('renders Python completion values as pretty JSON while preserving strings raw', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc', runtime: { language: 'python' } })
|
||||
runtime.behavior = () => Promise.resolve({ output: [], value: { n: 42, ok: true } })
|
||||
expect((await runCode(ctx, 'object')).content[0]).toEqual({ type: 'text', text: '{\n "n": 42,\n "ok": true\n}' })
|
||||
runtime.behavior = () => Promise.resolve({ logs: [], value: {} })
|
||||
runtime.behavior = () => Promise.resolve({ output: [], value: {} })
|
||||
expect((await runCode(ctx, 'empty object')).content[0]).toEqual({ type: 'text', text: '{}' })
|
||||
const nested = { outer: [{ inner: true }] }
|
||||
runtime.behavior = () => Promise.resolve({ logs: [], value: nested })
|
||||
runtime.behavior = () => Promise.resolve({ output: [], value: nested })
|
||||
expect((await runCode(ctx, 'nested')).content[0]).toEqual({ type: 'text', text: JSON.stringify(nested, null, 2) })
|
||||
runtime.behavior = () => Promise.resolve({ logs: [], value: ['x', 2] })
|
||||
runtime.behavior = () => Promise.resolve({ output: [], value: ['x', 2] })
|
||||
expect((await runCode(ctx, 'array')).content[0]).toEqual({ type: 'text', text: '[\n "x",\n 2\n]' })
|
||||
runtime.behavior = () => Promise.resolve({ logs: [], value: [] })
|
||||
runtime.behavior = () => Promise.resolve({ output: [], value: [] })
|
||||
expect((await runCode(ctx, 'empty array')).content[0]).toEqual({ type: 'text', text: '[]' })
|
||||
runtime.behavior = () => Promise.resolve({ logs: [], value: null })
|
||||
runtime.behavior = () => Promise.resolve({ output: [], value: null })
|
||||
expect((await runCode(ctx, 'null')).content[0]).toEqual({ type: 'text', text: 'null' })
|
||||
runtime.behavior = () => Promise.resolve({ logs: [], value: 'raw' })
|
||||
runtime.behavior = () => Promise.resolve({ output: [], value: 'raw' })
|
||||
expect((await runCode(ctx, 'string')).content[0]).toEqual({ type: 'text', text: 'raw' })
|
||||
runtime.behavior = () => Promise.resolve({ logs: [] })
|
||||
runtime.behavior = () => Promise.resolve({ output: [] })
|
||||
const absent = await runCode(ctx, 'undefined')
|
||||
expect(absent.content[0]).toEqual({ type: 'text', text: '(run_code completed with no output)' })
|
||||
expect(absent.isError ? undefined : absent.value).toEqual({ logs: [] })
|
||||
})
|
||||
|
||||
it('renders deeply nested JSON without recursive traversal or quadratic indentation', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc' })
|
||||
it('renders deeply nested Python completions without recursive traversal or quadratic indentation', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'ptc', runtime: { language: 'python' } })
|
||||
let value: JsonValue = {
|
||||
emptyArray: [],
|
||||
emptyObject: {},
|
||||
@@ -2165,7 +2240,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
record: { first: true, second: null },
|
||||
}
|
||||
for (let depth = 0; depth < 5_000; depth++) value = [value]
|
||||
runtime.behavior = () => Promise.resolve({ logs: [], value })
|
||||
runtime.behavior = () => Promise.resolve({ output: [], value })
|
||||
|
||||
const result = await runCode(ctx, 'deep result')
|
||||
|
||||
@@ -2182,8 +2257,8 @@ describe('the run_code dispatch bridge', () => {
|
||||
const calls = registerEcho(ctx)
|
||||
runtime.behavior = (request) => {
|
||||
// The fake honors the seam contract for an already-aborted signal.
|
||||
if (request.signal?.aborted) return Promise.resolve({ logs: [], error: { kind: 'abort' as const, message: String(request.signal.reason) } })
|
||||
return Promise.resolve({ logs: [], value: 'unreachable' })
|
||||
if (request.signal?.aborted) return Promise.resolve({ output: [], error: { kind: 'abort' as const, message: String(request.signal.reason) } })
|
||||
return Promise.resolve(printed('unreachable'))
|
||||
}
|
||||
const controller = new AbortController()
|
||||
controller.abort('too-late')
|
||||
@@ -2209,7 +2284,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
controller.abort('cancelled-mid-run')
|
||||
const message = await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
.then(() => 'resolved', (error: unknown) => error instanceof Error ? error.message : String(error))
|
||||
return { logs: [], value: message }
|
||||
return printed(message)
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { signal: controller.signal })
|
||||
expect(result.isError).toBe(true)
|
||||
@@ -2228,7 +2303,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
const message = `execution canceled; retained variables were ${state}`
|
||||
runtime.behavior = async () => {
|
||||
controller.abort({ kind: 'user' })
|
||||
return { logs: [], error: { kind: 'abort', message } }
|
||||
return { output: [], error: { kind: 'abort', message } }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { signal: controller.signal })
|
||||
expect(result.error).toEqual({
|
||||
@@ -2254,7 +2329,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
expect(request.signal?.reason).toBe(controller.signal.reason)
|
||||
const reason: unknown = request.signal?.reason
|
||||
const message = reason instanceof Error ? reason.message : String(reason)
|
||||
return { logs: [], error: { kind: 'abort', message } }
|
||||
return { output: [], error: { kind: 'abort', message } }
|
||||
}
|
||||
|
||||
const result = await runCode(ctx, 'program', { signal: controller.signal })
|
||||
@@ -2280,7 +2355,7 @@ describe('the run_code dispatch bridge', () => {
|
||||
const controller = new AbortController()
|
||||
runtime.behavior = async () => {
|
||||
if (cancelled) controller.abort({ kind: 'user' })
|
||||
return { logs: ['got this far'], error: failure }
|
||||
return { output: [{ type: 'text', text: 'got this far' }], error: failure }
|
||||
}
|
||||
|
||||
const result = await runCode(ctx, 'program', { signal: controller.signal })
|
||||
@@ -2599,7 +2674,7 @@ describe('host environment reset', () => {
|
||||
void request.bindings[0]!.functions.blocked!({}).catch(() => {})
|
||||
return program.promise
|
||||
}
|
||||
const release = vi.spyOn(runtime, 'release').mockImplementation(async () => { program.resolve({ logs: [], error: { kind: 'abort', message: 'environment disposed' } }) })
|
||||
const release = vi.spyOn(runtime, 'release').mockImplementation(async () => { program.resolve({ output: [], error: { kind: 'abort', message: 'environment disposed' } }) })
|
||||
const running = runCode(ctx, 'await tools.blocked({}); while (true) {}', { agent })
|
||||
await toolStarted.promise
|
||||
let settled = false
|
||||
@@ -2611,7 +2686,7 @@ describe('host environment reset', () => {
|
||||
cleanup.resolve(undefined)
|
||||
await resetting
|
||||
expect((await running).isError).toBe(true)
|
||||
} finally { cleanup.resolve(undefined); program.resolve({ logs: [] }); await Promise.all([running, resetting]) }
|
||||
} finally { cleanup.resolve(undefined); program.resolve({ output: [] }); await Promise.all([running, resetting]) }
|
||||
})
|
||||
|
||||
it('reports failed cleanup, refuses further code, and allows explicit reset retry', async () => {
|
||||
@@ -2663,20 +2738,32 @@ describe('PTC standing file policy and sandbox outcomes', () => {
|
||||
it('preserves partial enforcement and observed denial in successful output', async () => {
|
||||
const { ctx, runtime } = await setup()
|
||||
try {
|
||||
runtime.behavior = async () => ({ logs: [], sandbox: { mode: 'read-only', enforcement: 'partial', denied: true } })
|
||||
runtime.behavior = async () => ({ output: [], sandbox: { mode: 'read-only', enforcement: 'partial', denied: true } })
|
||||
const result = await runCode(ctx, 'return 1')
|
||||
expect(result.value).toEqual({ logs: [], sandbox: { mode: 'read-only', enforcement: 'partial', denied: true } })
|
||||
expect(result.content).toEqual([{ type: 'text', text: 'File sandbox enforcement is partial on this host.\nThe read-only file sandbox denied an operation.' }])
|
||||
} finally { await ctx.fiber.dispose() }
|
||||
})
|
||||
|
||||
it.each(['line', 'line\n', 'line\n\n'])('keeps written line breaks before sandbox diagnostics: %j', async (log) => {
|
||||
const { ctx, runtime } = await setup()
|
||||
try {
|
||||
runtime.behavior = async () => ({ output: [{ type: 'text', text: log }],
|
||||
error: { kind: 'exception', message: 'failed' }, sandbox: { mode: 'read-only', denied: false } })
|
||||
const result = await runCode(ctx, 'inspect("line"); throw new Error("failed")')
|
||||
const text = result.content.filter(block => block.type === 'text').map(block => block.text).join('')
|
||||
const written = log === 'line' ? 'line\n' : log
|
||||
expect(text).toBe(`Error: code run failed (exception): failed\nCaptured output:\n${written}File sandbox: read-only.`)
|
||||
} finally { await ctx.fiber.dispose() }
|
||||
})
|
||||
|
||||
it.each([
|
||||
{ mode: 'danger-full-access' as const, denied: false },
|
||||
{ mode: 'read-only' as const, denied: true, enforcement: 'full' as const },
|
||||
])('includes the available sandbox facts with a failed program ($mode)', async (sandbox) => {
|
||||
const { ctx, runtime } = await setup()
|
||||
try {
|
||||
runtime.behavior = async () => ({ logs: [], error: { kind: 'exception', message: 'failed' }, sandbox })
|
||||
runtime.behavior = async () => ({ output: [], error: { kind: 'exception', message: 'failed' }, sandbox })
|
||||
const result = await runCode(ctx, 'throw new Error("failed")')
|
||||
expect(result.isError).toBe(true)
|
||||
const text = result.content.filter(block => block.type === 'text').map(block => block.text).join('\n')
|
||||
@@ -2855,7 +2942,7 @@ describe('per-program execution controls', () => {
|
||||
let effects = 0
|
||||
runtime.behavior = () => {
|
||||
effects++
|
||||
return Promise.resolve({ logs: ['first effect completed'],
|
||||
return Promise.resolve({ output: [{ type: 'text', text: 'first effect completed' }],
|
||||
error: { kind: 'exception', message: 'access denied' },
|
||||
sandbox: { mode: 'read-only', denied: true, enforcement: 'full' } })
|
||||
}
|
||||
@@ -2887,7 +2974,7 @@ describe('per-program execution controls', () => {
|
||||
expect(ctx.sandboxPolicy.resolve({ session: agent.session }).mode).toBe('read-only')
|
||||
const value = await request.bindings[0]!.functions.echo!({ value: 'nested' })
|
||||
.then(() => 'unexpected nested grant', () => 'nested approval rejected')
|
||||
return { logs: [], value }
|
||||
return printed(value)
|
||||
}
|
||||
const resolver = vi.spyOn(runtime, 'resolve')
|
||||
const result = execute({ sandbox_permissions: 'workspace-write', justification: 'Write generated source', timeoutMs: 1234 })
|
||||
@@ -2963,7 +3050,7 @@ it.each([
|
||||
runtime.behavior = async (request) => {
|
||||
entered.resolve(undefined)
|
||||
await release.promise
|
||||
return { logs: [], value: await request.bindings[0]!.functions.echo!({}) }
|
||||
return printed(await request.bindings[0]!.functions.echo!({}))
|
||||
}
|
||||
const agent = { ctx, id: SessionId('unloading-ptc'), session: Session.create(SessionId('unloading-ptc')) } as Agent
|
||||
const input = {
|
||||
@@ -2978,7 +3065,7 @@ it.each([
|
||||
await (unload === 'runtime' ? runtimeProvider : provider).dispose()
|
||||
await expect(ctx.tools.execute(input)).rejects.toThrow('selected tool executor is unavailable')
|
||||
release.resolve(undefined)
|
||||
expect((await pending).results[0]?.result).toMatchObject({ isError: false, value: { logs: [], result: 'settled' } })
|
||||
expect((await pending).results[0]?.result).toMatchObject({ isError: false, value: { logs: ['settled'] } })
|
||||
if (owned) expect(agent.session.snapshotEvents().filter(event => event.type === 'tool/ptc-dispatch'))
|
||||
.toMatchObject([{ data: { name: 'echo', isError: false, content: [{ type: 'text', text: 'settled' }] } }])
|
||||
} finally {
|
||||
|
||||
@@ -21,7 +21,7 @@ class ControlledRuntime extends PtcRuntime {
|
||||
override get programFormat() { return 'repl' as const }
|
||||
override get executionInstructions() { return 'Top-level await is supported; the last expression is returned.' }
|
||||
override get timeout() { return { defaultMs: 120_000, maxMs: 600_000 } }
|
||||
behavior: (spec: PtcRunSpec) => Promise<PtcRunResult> = async () => ({ logs: [] })
|
||||
behavior: (spec: PtcRunSpec) => Promise<PtcRunResult> = async () => ({ output: [] })
|
||||
|
||||
resolve(request: PtcRunRequest): PtcRunSpec {
|
||||
return { ...request, cwd: request.cwd ?? process.cwd(), timeoutMs: request.timeoutMs ?? this.timeout.defaultMs }
|
||||
@@ -96,9 +96,11 @@ describe('registered program tool factory', () => {
|
||||
expect(request.program).toBe('await cua.inspect()')
|
||||
expect(request.stateOwner).toBe(agent)
|
||||
expect(Object.keys(request.bindings[0]!.functions)).toEqual(['cua'])
|
||||
return { logs: [], value: await request.bindings[0]!.functions.cua!({}) }
|
||||
expect(request.discardCompletion).toBe(true)
|
||||
expect(await request.bindings[0]!.functions.cua!({})).toBe('private result')
|
||||
return { output: [{ type: 'text', text: 'private result' }] }
|
||||
}
|
||||
expect((await execute()).results[0]?.result).toMatchObject({ isError: false, value: { logs: [], result: 'private result' } })
|
||||
expect((await execute()).results[0]?.result).toMatchObject({ isError: false, value: { logs: ['private result'] } })
|
||||
expect(body).toHaveBeenCalledOnce()
|
||||
expect(unrelated).not.toHaveBeenCalled()
|
||||
expect(Object.isFrozen(definition.parameters)).toBe(false)
|
||||
@@ -117,7 +119,7 @@ describe('registered program tool factory', () => {
|
||||
? { reason: 'Computer operation blocked', info: { name: 'ComputerGuardError', code: 'COMPUTER_BLOCKED' } } : undefined)
|
||||
runtime.behavior = async (request) => {
|
||||
await expect(request.bindings[0]!.functions.cua!({})).rejects.toThrow()
|
||||
return { logs: [] }
|
||||
return { output: [] }
|
||||
}
|
||||
expect((await execute()).results[0]?.result.isError).toBe(false)
|
||||
approval = 'allowed-once'
|
||||
@@ -141,7 +143,7 @@ describe('registered program tool factory', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
expect(request.stateOwner).toBeUndefined()
|
||||
expect(request.sandboxPolicy?.mode).toBe('workspace-write')
|
||||
return { logs: [], sandbox: { mode: 'workspace-write', denied: true } }
|
||||
return { output: [], sandbox: { mode: 'workspace-write', denied: true } }
|
||||
}
|
||||
const result = (await execute({ sandbox_permissions: 'workspace-write', justification: 'Write computer output' })).results[0]!.result
|
||||
expect(result.isError).toBe(false)
|
||||
@@ -149,14 +151,14 @@ describe('registered program tool factory', () => {
|
||||
expect(JSON.stringify(result.content)).toContain('Nested tools retain their own policies and approvals')
|
||||
})
|
||||
|
||||
it('records nested results while forwarding screenshots and deferred context after the program', async () => {
|
||||
it('records nested results while forwarding images and deferred context from private tools', async () => {
|
||||
type Attachment = Extract<ContentBlock, { type: 'image' }>['attachment']
|
||||
const image: ContentBlock = { type: 'image', attachment: {
|
||||
attachmentId: brandString<Attachment['attachmentId']>('sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'),
|
||||
mediaType: 'image/png', width: 1, height: 1, bytes: 1,
|
||||
} }
|
||||
const definition = defineTool({
|
||||
name: 'cua', description: 'Capture the test computer.', parameters: {},
|
||||
name: 'capture', description: 'Capture the test computer.', parameters: {},
|
||||
output: {
|
||||
schema: { type: 'string' }, render: () => [image], presentationMeta: () => ({ screen: 1 }),
|
||||
},
|
||||
@@ -169,8 +171,8 @@ describe('registered program tool factory', () => {
|
||||
const { ctx, runtime, session, execute } = await setup(definition)
|
||||
ctx.on('tools/ptc-dispatch-log', async () => [{ type: 'text', text: 'durable screenshot preview' }])
|
||||
runtime.behavior = async (request) => {
|
||||
expect(await request.bindings[0]!.functions.cua!({})).toBe('raw snapshot')
|
||||
return { logs: ['inspection complete'] }
|
||||
expect(await request.bindings[0]!.functions.capture!({})).toBe('raw snapshot')
|
||||
return { output: [{ type: 'text', text: 'inspection complete' }] }
|
||||
}
|
||||
const batch = await execute()
|
||||
expect(batch.results[0]?.result.content).toEqual([{ type: 'text', text: 'inspection complete' }])
|
||||
@@ -179,7 +181,7 @@ describe('registered program tool factory', () => {
|
||||
const events = session.snapshotEvents().filter(event => event.type === 'tool/ptc-dispatch-start' || event.type === 'tool/ptc-dispatch')
|
||||
expect(events.map(event => event.type)).toEqual(['tool/ptc-dispatch-start', 'tool/ptc-dispatch'])
|
||||
expect(events[1]?.data).toMatchObject({
|
||||
rootCallId: 'program', parentCallId: 'program', subCallId: 'program:ptc:1', name: 'cua', arguments: {},
|
||||
rootCallId: 'program', parentCallId: 'program', subCallId: 'program:ptc:1', name: 'capture', arguments: {},
|
||||
isError: false, content: [{ type: 'text', text: 'durable screenshot preview' }], meta: { screen: 1 },
|
||||
})
|
||||
})
|
||||
@@ -205,7 +207,7 @@ describe('registered program tool factory', () => {
|
||||
await new Promise<void>((resolve) => {
|
||||
request.signal?.addEventListener('abort', () => { resolve() }, { once: true })
|
||||
})
|
||||
return { logs: [], error: { kind: 'abort', message: 'runtime cancelled' } }
|
||||
return { output: [], error: { kind: 'abort', message: 'runtime cancelled' } }
|
||||
}
|
||||
const running = execute()
|
||||
let disposing: Promise<void> | undefined
|
||||
@@ -238,7 +240,7 @@ describe('registered program tool factory', () => {
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.cua!({})
|
||||
await expect(request.bindings[0]!.functions.cua!({})).rejects.toThrow('cua_run_code exceeded its limit of 1 tool calls')
|
||||
return { logs: [], error: { kind: 'abort', message: 'allowance exhausted' } }
|
||||
return { output: [], error: { kind: 'abort', message: 'allowance exhausted' } }
|
||||
}
|
||||
const result = (await execute({ timeoutMs: 1000 })).results[0]!.result
|
||||
expect(result).toMatchObject({ isError: true, error: { info: { code: 'CODE_RUN_FAILED' } } })
|
||||
|
||||
@@ -137,6 +137,7 @@ describe('renderToolsSdk', () => {
|
||||
expect(text).toContain('declare class ToolCallError extends Error')
|
||||
expect(text).toContain('readonly toolName: ToolName;')
|
||||
expect(text).toContain('declare const tools: {')
|
||||
expect(text).toContain('readonly attachmentId: string;')
|
||||
expect(text).toContain('type JsonValue = null | boolean | number | string')
|
||||
expect(text.indexOf('bash: {')).toBeGreaterThan(0)
|
||||
expect(text).toContain('"my-mcp.tool":')
|
||||
@@ -152,6 +153,16 @@ describe('renderToolsSdk', () => {
|
||||
expect(text).toContain('lossless JSON')
|
||||
})
|
||||
|
||||
it('declares Image results for read_image and image-or-value results for cua', () => {
|
||||
const tools: ToolSdkSchema[] = ['read_image', 'cua'].map(name => ({
|
||||
name, description: 'Image-producing tool.', parameters: {}, output: { type: 'string' },
|
||||
}))
|
||||
const text = renderToolsSdk(tools)
|
||||
const outputs = text.slice(text.indexOf('interface ToolOutputMap'), text.indexOf('type ToolName'))
|
||||
expect(outputs).toContain('read_image: Image;')
|
||||
expect(outputs).toContain('cua: Image | string;')
|
||||
})
|
||||
|
||||
it('lists the program before its description', () => {
|
||||
const text = renderToolsSdk([bash])
|
||||
expect(text).toContain('`code`')
|
||||
@@ -163,14 +174,14 @@ describe('renderToolsSdk', () => {
|
||||
const text = renderToolsSdk([bash])
|
||||
expect(text).toContain('A declaration does not make its name a directly callable tool')
|
||||
expect(text).toContain('only names supplied as separate tool schemas may be called directly')
|
||||
expect(text).toContain('`run_code({ code: "return await tools.bash({ description: \'Show current directory\', command: \'pwd\' })", description: "Show current directory" })`')
|
||||
expect(text).toContain('`run_code({ code: "inspect(await tools.bash({ description: \'Show current directory\', command: \'pwd\' }))", description: "Show current directory" })`')
|
||||
expect(text).toContain('Program-only SDK bindings:')
|
||||
expect(text).not.toContain('The available tools:')
|
||||
})
|
||||
|
||||
it('keeps REPL output syntax distinct from function bodies', () => {
|
||||
const text = renderToolsSdk([bash], 'repl')
|
||||
expect(text).toContain('run_code({ code: "console.log(await tools.bash(')
|
||||
expect(text).toContain('run_code({ code: "inspect(await tools.bash(')
|
||||
expect(text).not.toContain('return await tools.bash(')
|
||||
expect(text).not.toContain('the body of an async TypeScript function')
|
||||
expect(text).not.toContain('Emit results with `return`')
|
||||
@@ -222,9 +233,10 @@ describe('renderToolsSdk', () => {
|
||||
expect(renderToolsSdk([constrained])).toContain('tools.bash({ command: \'pwd\' })')
|
||||
})
|
||||
|
||||
it('uses a returned result in function-body mode', () => {
|
||||
it('uses explicit inspection in function-body mode', () => {
|
||||
const text = renderToolsSdk([bash])
|
||||
expect(text).toContain('return await tools.bash(')
|
||||
expect(text).toContain('inspect(await tools.bash(')
|
||||
expect(text).toContain('- Show selected values and images with `inspect(...values)`; every other intermediate result stays out of the conversation, so extract just what you need.')
|
||||
expect(text).toContain('body of an async TypeScript function')
|
||||
expect(text).not.toContain('TypeScript REPL statements')
|
||||
})
|
||||
|
||||
@@ -55,7 +55,7 @@ class SkillRuntime extends PtcRuntime {
|
||||
return { ...request, cwd: request.cwd ?? process.cwd(), timeoutMs: request.timeoutMs ?? 120_000 }
|
||||
}
|
||||
async run(request: PtcRunRequest): Promise<PtcRunResult> {
|
||||
return { logs: [], value: await request.bindings[0]!.functions.skill!({ name: this.skill }) }
|
||||
return { output: [], value: await request.bindings[0]!.functions.skill!({ name: this.skill }) }
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -76,7 +76,7 @@ class CatalogAdapter extends LlmAdapter {
|
||||
class FakeRuntime extends PtcRuntime {
|
||||
readonly language = 'typescript'
|
||||
readonly isolation = 'fake'
|
||||
behavior: (request: PtcRunRequest) => Promise<PtcRunResult> = () => Promise.resolve({ logs: [] })
|
||||
behavior: (request: PtcRunRequest) => Promise<PtcRunResult> = () => Promise.resolve({ output: [] })
|
||||
resolve(request: PtcRunRequest): PtcRunSpec {
|
||||
return { ...request, cwd: request.cwd ?? process.cwd(), timeoutMs: request.timeoutMs ?? 120_000 }
|
||||
}
|
||||
@@ -648,12 +648,12 @@ describe('arguments and paths', () => {
|
||||
})
|
||||
|
||||
describe('PTC dispatch', () => {
|
||||
it('forwards nested render images and applies execution policy to the nested call', async () => {
|
||||
it('records nested render images and applies execution policy to the nested call', async () => {
|
||||
const { ctx } = await setup({ ptc: true })
|
||||
const runtime = ctx.ptcRuntime as FakeRuntime
|
||||
runtime.behavior = async (request) => {
|
||||
const value = await request.bindings[0]!.functions.office_render!({ input: 'deck.pptx' })
|
||||
return { logs: [], value }
|
||||
await request.bindings[0]!.functions.office_render!({ input: 'deck.pptx' })
|
||||
return { output: [] }
|
||||
}
|
||||
const agent = { ...agentOn('vision'), ctx }
|
||||
await loadOffice(ctx, agent)
|
||||
@@ -663,9 +663,15 @@ describe('PTC dispatch', () => {
|
||||
signal, callId: ToolCallId(`run-${++calls}`), name: RUN_CODE_NAME, arguments: args, agent: agent as never,
|
||||
toolSet,
|
||||
})
|
||||
const result = await run({ code: 'return await tools.office_render({ input: "deck.pptx" })', description: 'Render' })
|
||||
const result = await run({ code: 'await tools.office_render({ input: "deck.pptx" })', description: 'Render' })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.additionalContexts?.[0]?.content.some(block => block.type === 'image')).toBe(true)
|
||||
const dispatch = agent.session.snapshotEvents().find(event => event.type === 'tool/ptc-dispatch')
|
||||
expect(dispatch?.data.content.some(block => block.type === 'image')).toBe(true)
|
||||
expect(result.content.some(block => block.type === 'image')).toBe(false)
|
||||
expect(result.additionalContexts).toHaveLength(1)
|
||||
expect(result.additionalContexts?.[0]).toMatchObject({
|
||||
source: { kind: 'ptc-mode' }, content: dispatch?.data.content,
|
||||
})
|
||||
|
||||
const denied: string[] = []
|
||||
ctx.on('tools/pre-execute', (exec, next) => {
|
||||
@@ -673,7 +679,7 @@ describe('PTC dispatch', () => {
|
||||
denied.push(exec.name)
|
||||
return Promise.resolve({ kind: 'deny' as const, reason: 'office denied by policy' })
|
||||
}, { prepend: true })
|
||||
const refused = await run({ code: 'return await tools.office_render({ input: "deck.pptx" })', description: 'Render' })
|
||||
const refused = await run({ code: 'await tools.office_render({ input: "deck.pptx" })', description: 'Render' })
|
||||
expect(denied).toEqual(['office_render'])
|
||||
expect(text(refused)).toContain('office denied by policy')
|
||||
})
|
||||
|
||||
@@ -15,11 +15,11 @@
|
||||
en: 1777e7bf6785f986
|
||||
zh: e3daac1074ebb74d
|
||||
/deepseek-ai-dsh-experimental-cua-macos/use-this-package:
|
||||
en: bfd9f81f48c780e1
|
||||
zh: c2de74848db16003
|
||||
en: 1707074b060e3ddf
|
||||
zh: 89bf1457c0a6e808
|
||||
/deepseek-ai-dsh-experimental-cua-macos/use-this-package/connect-an-external-execution-environment:
|
||||
en: 8bd8e80f83dd2599
|
||||
zh: 7d4988effe28c40a
|
||||
en: 7058be5d1b778643
|
||||
zh: 821de6b0ad7aba27
|
||||
/deepseek-ai-dsh-experimental-cua-macos/understand-the-implementation:
|
||||
en: 4e82b85a5aa72eee
|
||||
zh: 4e004b7a205d8688
|
||||
@@ -39,8 +39,8 @@
|
||||
en: 9142df90fd94ef91
|
||||
zh: cad17384c7f7452b
|
||||
/deepseek-ai-dsh-experimental-cua-macos/model-experience/native-application-operations/token-effect:
|
||||
en: 86b0b7e406627a47
|
||||
zh: 53ad627fb3521f28
|
||||
en: 75db5557a3bfb338
|
||||
zh: 52b34e266737c582
|
||||
/deepseek-ai-dsh-experimental-cua-macos/model-experience/native-application-operations/kv-cache-effect:
|
||||
en: a2c9acb5a4320dda
|
||||
zh: e8d1586af60215c4
|
||||
|
||||
@@ -94,7 +94,7 @@ AX observations show menu-bar entries by default. Use `app.getAccessibilityTree(
|
||||
|
||||
Values and placeholders are separate. Named, stateful or actionable groups and scroll areas retain their own rows. Repeated names, descriptions and help text on one node are shown once. Values, placeholders and URLs keep their distinct meanings even when their text matches; body text shared by distinct nodes remains intact. Writability is explicit on relevant controls, including `AXIncrementor`; `unknown` means the system did not report it. Missing optional state fields do not imply false. During tree observation, an AXValue or AXSelectedTextRange writability probe returning -25200 leaves that field unknown and preserves the remaining nodes. Actual writes remain strict; permission, communication and invalid-element errors still propagate. UTF-16 ranges outside the observed value are marked unavailable. Compact tree labels can omit AppKit target and selector details. Pass the exact name returned by `listActions(elementId)` to `performAction`.
|
||||
|
||||
Observations emit by default and also return their value. In persistent PTC or `cua_run_code`, use `void await app.getAccessibilityTree()` for one emitted copy, or `emit:false` and one explicit print. Capturing a screenshot after an emitted tree can fail independently; retain the successful AX output.
|
||||
Call `inspect(await app.getAccessibilityTree())` to display the tree, or filter the returned text and pass the matches to `inspect`. Text inspected before a later screenshot failure remains visible.
|
||||
|
||||
Screenshot diagnostics use `[cua-screenshot]` records keyed by the tool request ID. Native phase records return over the originating Agent connection and cover window/AX geometry checks, system capture, PNG/Base64 encoding and response writing. Host debug logs add dispatch, reply and cleanup timings. Each request retains at most 8192 bytes of complete diagnostic records, with explicit truncation; screenshot failures append these records to the error for Session export. Records contain stages, timings and error codes, not window titles, AX text or image bytes.
|
||||
|
||||
@@ -106,12 +106,12 @@ To run through persistent PTC, add `--patch packages/experimental/cua-macos/exam
|
||||
|
||||
```javascript
|
||||
import { connectCua } from '@deepseek-ai/dsh-cua/bridge'
|
||||
const cua = connectCua(() => tools.cua, text => console.log(text))
|
||||
const cua = connectCua(() => tools.cua)
|
||||
const app = await cua.getApp("com.apple.TextEdit")
|
||||
const windows = await app.getWindows()
|
||||
if (windows.length === 0) throw new Error("Open a TextEdit window first")
|
||||
const window = await app.getWindow(windows[0].windowId)
|
||||
void await window.getScreenshot()
|
||||
inspect(await window.getScreenshot())
|
||||
```
|
||||
|
||||
An independent floating panel has its own window ID. Select it from `app.getWindows()` to capture or click the panel.
|
||||
@@ -132,7 +132,7 @@ Use `win.click(id)` to send mouse events at the center of an element, or `win.cl
|
||||
|
||||
Absolute coordinates for `mouseMove`, `click`, and `drag` use `[x, y]` tuples; `scroll` uses named `x` and `y`. All use pixels in the latest returned screenshot. Pass `relative: true` for integer thousandths of window width and height, from 0 to 999; omission or `false` uses pixels. Relative input can use a successful AX read or screenshot, while absolute input requires a screenshot. The Host maps the delivered image to native points, including attachment and model-request resizing. A value of 999 remains inside the window edge. Coordinate options do not apply to element IDs. A width or height change since the latest observation returns `WINDOW_CHANGED` before input, including raw coordinate calls without a snapshot ID; read fresh AX or take a new screenshot before proceeding. Window movement alone preserves window-local coordinates. Capture and pointer input reject known disagreement between the selected window’s accessibility frame and displayed bounds with `WINDOW_CHANGED`, before posting pointer events. This includes Stage Manager transformations when AX retains the full window frame. Read fresh state; any activation or window raise remains explicit. Advertised AX actions remain available. Missing AX geometry retains the AXless path and does not establish that a transformed window can be operated.
|
||||
|
||||
Retained objects resolve the current tool binding on every call. The external runtime owns persistence, imports, cancellation and completion-value serialization. Screenshots return encoded bytes with their MIME type, pixel dimensions and window ID; the standard tool pipeline admits the image attachment exactly once. `emit: false` suppresses facade output only. A host that uses another dispatcher can build objects through `createCuaApi` from `./api`.
|
||||
Retained objects resolve the current tool binding on every call. The external runtime owns persistence, imports, cancellation and completion-value serialization. Screenshots return stored `Image` values for explicit display through `inspect`. A caller with another program dispatcher can build objects through `createCuaApi(dispatch)` from `./api`.
|
||||
|
||||
An AppKit sheet can appear in its document window’s tree with a separate window ID. For pointer input on the sheet, use `app.getWindow(sheetId)` and an element ID from that window’s tree. Keyboard input follows a system Open/Save panel only when the application’s focused sheet identifies one Apple panel-service window and that service still focuses the same panel. Unrelated or ambiguous remote recipients fail before dispatch.
|
||||
|
||||
@@ -183,19 +183,19 @@ The configured PTC preset receives one `cua` operation tool with method-specific
|
||||
```markdown
|
||||
macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.
|
||||
|
||||
Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation.
|
||||
Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation.
|
||||
|
||||
Use void await for observations that emit, and never stringify screenshot image bytes.
|
||||
Call inspect(...values) to display selected observation results.
|
||||
|
||||
Signatures show resolved return values after await. ActionResult = { dispatched: boolean, foregroundChanged?: boolean | null }; foregroundChanged is omitted when not sampled and null when the sample is unavailable. JsonValue = string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue }; numbers must be finite. App is the application object with readonly app.appHandle: string (opaque) and the methods listed below for app level. Window is the window object with readonly window.windowId: number and the methods listed below for window level. Neither object carries pid.
|
||||
|
||||
Setup and discovery:
|
||||
- cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, systemPointer: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions.
|
||||
- cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications.
|
||||
- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid.
|
||||
- cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation.
|
||||
- cua.getCapabilities() -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, systemPointer: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions.
|
||||
- cua.listApps() -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications.
|
||||
- cua.findApps(query: string) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid.
|
||||
- cua.getState() -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation.
|
||||
- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and return without reading AX or activating the app.
|
||||
- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership.
|
||||
- app.getWindows() -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership.
|
||||
- app.getWindow(windowId: number) -> Window — select a window by an ID from getWindows without reading AX. Providers reject closed or foreign windows when operating on them.
|
||||
- app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required.
|
||||
- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation.
|
||||
@@ -205,7 +205,7 @@ Setup and discovery:
|
||||
Available at app or window level depending on the platform.
|
||||
|
||||
Read getAccessibilityTree explicitly before using element ids. Selection and screenshots do not read or replace the current AX snapshot. Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- getAccessibilityTree(options?: { includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree(). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Use inspect to display the matches you need. Use element ids from the file only while that observation is current.
|
||||
- listActions(elementId: string) -> string[] — list the actions an element advertises.
|
||||
- invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none.
|
||||
- performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument.
|
||||
@@ -214,7 +214,7 @@ Read getAccessibilityTree explicitly before using element ids. Selection and scr
|
||||
|
||||
## Simulated mouse
|
||||
Use an explicit window for screenshots and pointer input. Before coordinate input, including relative coordinates, obtain a successful getAccessibilityTree observation covering that window or take its screenshot. Absolute pixel coordinates require a returned screenshot.
|
||||
- getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG.
|
||||
- getScreenshot() -> Image — capture the selected window as a stored image with readonly attachmentId, path, width and height; pass it to inspect to display it.
|
||||
- mouseMove(target: string | [number, number], options?: { relative?: boolean, useSystemPointer?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request a mouse move to the target without pressing a button; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without that route, HOVER_UNAVAILABLE is returned before sending input. Physical movement requires an unobscured foreground target. Explain any necessary activation and observe the target again.
|
||||
- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying.
|
||||
- scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left.
|
||||
@@ -229,14 +229,14 @@ Available at app or window level depending on the platform.
|
||||
- pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string.
|
||||
- sendKeyEvent(key: string, direction: 'down' | 'up') -> ActionResult — send 'down' to hold a key across tool calls in one program, for example W down, D down, D up, W up. Holding a key on a physical keyboard can produce repeated key-down events after a delay. To send one such repeat for a non-modifier key this run already holds, send 'down' again. Each call posts one event. A control can filter repeats. Use the app's response to decide whether and when to repeat or release the key. When this program returns, throws, or the user pauses it, macOS releases only its remaining keys at their original recipients.
|
||||
|
||||
Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree maps to getState; getWindow has no raw method; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name.
|
||||
Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. getAccessibilityTree maps to getState; getWindow has no raw method; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name.
|
||||
|
||||
Only if the task cannot be completed without moving the user’s pointer, call await win.mouseMove(target, { useSystemPointer: true }). Prefer an advertised accessibility action, a click on a visible control, or browser tools when possible. The question interrupts the user. When approval is disabled an explicit move can act immediately. Pass true on every physical move, even after approval. Omitting the option or passing false uses the default hover route without prompting. Approval or refusal lasts until this session closes or DSH restarts. A refusal sends no input. If the user dismisses the request, ask again only if they request it. Wait for an outstanding request to finish. Subagents cannot request physical movement.
|
||||
```
|
||||
|
||||
#### Token effect
|
||||
|
||||
Enabling the plugin adds a fixed schema and guidance. AX observations, operation results and admitted images add model input. Retained objects are not serialized into prompts by this package. A higher default or per-read override can include more AX text in model input when the window has more nodes to visit. Each visible auxiliary window, such as a status item, menu or tooltip, adds one window record to `getWindows()` and raw `getState` output; it does not automatically add a screenshot.
|
||||
Enabling the plugin adds a fixed schema and guidance. Native `cua` calls send result text and admitted screenshot images to the model. In TypeScript PTC and `cua_run_code`, programs select text and images with `inspect`. Retaining an app or window object in the program adds no prompt content. A higher default or per-read override can return more AX text when the window has more nodes to visit. Each visible auxiliary window, such as a status item, menu or tooltip, adds one window record to `getWindows()` and raw `getState` output; it does not automatically add a screenshot.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
|
||||
@@ -94,7 +94,7 @@ AX 观察默认只展示菜单栏入口。使用 `app.getAccessibilityTree({ inc
|
||||
|
||||
实际值与占位文字分别展示。有名称、状态或可用动作的分组和滚动区域保留独立行。同一节点中重复的名称、描述和帮助只显示一次。实际值、占位文字和 URL 即使文字相同也保留各自含义,不删除不同节点共有的正文。相关控件明确展示可写性,包括 `AXIncrementor`;`unknown` 表示系统未报告。缺少可选状态字段不代表 false。读树时,AXValue 或 AXSelectedTextRange 的可写性探测返回 -25200,会将该字段保留为未知并继续读取其他节点。实际写入仍严格校验;权限、通信和失效元素错误仍向上传播。超出当前值长度的 UTF-16 选区标为不可用。树中的动作简称可能省略 AppKit 的 target 和 selector 信息。调用 `performAction` 时使用 `listActions(elementId)` 返回的完整名称。
|
||||
|
||||
观察方法默认自动输出,同时返回结果。在 persistent PTC 或 `cua_run_code` 中使用 `void await app.getAccessibilityTree()` 输出一次,或指定 `emit:false` 后自行打印一次。AX 输出后的截图可能单独失败,已成功取得的 AX 输出仍保留。
|
||||
调用 `inspect(await app.getAccessibilityTree())` 展示整棵树,也可以先筛选返回的文本,再将匹配项传给 `inspect`。若后续截图失败,先前已显式输出的 AX 文本仍然可见。
|
||||
|
||||
截图诊断使用带工具请求 ID 的 `[cua-screenshot]` 记录。原生阶段记录经本次 Agent 连接返回,覆盖窗口及 AX 几何检查、系统截图、PNG/Base64 编码和响应写出。Host 调试日志补充发送、响应和清理耗时。每次请求最多保留 8192 字节完整诊断记录,超限会明确标记;截图失败时附在错误中,保留到 Session trace。记录只含阶段、耗时和错误码,不含窗口标题、AX 正文或图片内容。
|
||||
|
||||
@@ -106,12 +106,12 @@ AX 观察默认只展示菜单栏入口。使用 `app.getAccessibilityTree({ inc
|
||||
|
||||
```javascript
|
||||
import { connectCua } from '@deepseek-ai/dsh-cua/bridge'
|
||||
const cua = connectCua(() => tools.cua, text => console.log(text))
|
||||
const cua = connectCua(() => tools.cua)
|
||||
const app = await cua.getApp("com.apple.TextEdit")
|
||||
const windows = await app.getWindows()
|
||||
if (windows.length === 0) throw new Error("Open a TextEdit window first")
|
||||
const window = await app.getWindow(windows[0].windowId)
|
||||
void await window.getScreenshot()
|
||||
inspect(await window.getScreenshot())
|
||||
```
|
||||
|
||||
独立浮层有自己的窗口 ID。需要截图或点击浮层时,从 `app.getWindows()` 中选中它。
|
||||
@@ -132,7 +132,7 @@ void await window.getScreenshot()
|
||||
|
||||
`mouseMove`、`click` 和 `drag` 的绝对坐标使用 `[x, y]` 元组;`scroll` 使用命名的 `x` 和 `y`。它们都使用最近返回截图中的像素。传入 `relative: true` 时使用 0~999 的千分比整数;省略或传入 `false` 时使用像素。相对坐标可基于成功的 AX 读取或截图,绝对坐标必须先截图。Host 根据实际送入模型的图片尺寸换算原生 point,包含附件归一化和模型请求阶段的缩放。999 映射到窗口边缘以内。坐标选项不适用于元素 ID。窗口宽高与最近一次观察不同时,在输入前返回 `WINDOW_CHANGED`,即使原始坐标调用不传快照 ID 也会检查;重新读取 AX 或截图后再操作。单纯移动窗口不使窗口内坐标失效。截图和鼠标输入会核对所选窗口的辅助功能框与显示边界;已知两者不一致时,在发送任何鼠标事件前返回 `WINDOW_CHANGED`。当 AX 保留完整窗口框时,这也会拦住 Stage Manager 变换后的窗口。需要重新读取状态;激活应用或抬起窗口仍须显式调用。元素声明的 AX 动作保持可用。缺少 AX 几何时保留无 AX 路径,不代表变换后的窗口一定可以操作。
|
||||
|
||||
保留的对象会在每次调用时获取当前工具绑定。外部运行时负责持久化、导入、取消和最终表达式值的序列化。截图返回编码字节及其 MIME 类型、像素尺寸和窗口 ID,标准工具链负责将图片作为附件传递一次。`emit: false` 只控制对象层的输出。使用其他分派器的宿主可以通过 `./api` 的 `createCuaApi` 创建对象。
|
||||
保留的对象会在每次调用时获取当前工具绑定。外部运行时负责持久化、导入、取消和最终表达式值的序列化。截图返回保存的 `Image` 值,通过 `inspect` 显式展示。使用其他程序分派器的调用方可以通过 `./api` 的 `createCuaApi(dispatch)` 创建对象。
|
||||
|
||||
AppKit 的 sheet 可以出现在文档窗口的树里,并有独立的窗口 ID。对 sheet 发送鼠标输入时,先用 `app.getWindow(sheetId)` 选择它,再使用该窗口树中的元素 ID。只有应用当前获得焦点的 sheet 指向唯一的 Apple 面板服务窗口,且该服务仍聚焦于同一面板时,键盘输入才转交系统打开或保存面板。无法确认或存在歧义的远端接收方会在输入前报错。
|
||||
|
||||
@@ -183,19 +183,19 @@ AppKit 的 sheet 可以出现在文档窗口的树里,并有独立的窗口 ID
|
||||
```markdown
|
||||
macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.
|
||||
|
||||
Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation.
|
||||
Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation.
|
||||
|
||||
Use void await for observations that emit, and never stringify screenshot image bytes.
|
||||
Call inspect(...values) to display selected observation results.
|
||||
|
||||
Signatures show resolved return values after await. ActionResult = { dispatched: boolean, foregroundChanged?: boolean | null }; foregroundChanged is omitted when not sampled and null when the sample is unavailable. JsonValue = string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue }; numbers must be finite. App is the application object with readonly app.appHandle: string (opaque) and the methods listed below for app level. Window is the window object with readonly window.windowId: number and the methods listed below for window level. Neither object carries pid.
|
||||
|
||||
Setup and discovery:
|
||||
- cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, systemPointer: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions.
|
||||
- cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications.
|
||||
- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid.
|
||||
- cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation.
|
||||
- cua.getCapabilities() -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, systemPointer: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions.
|
||||
- cua.listApps() -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications.
|
||||
- cua.findApps(query: string) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid.
|
||||
- cua.getState() -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation.
|
||||
- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and return without reading AX or activating the app.
|
||||
- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership.
|
||||
- app.getWindows() -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership.
|
||||
- app.getWindow(windowId: number) -> Window — select a window by an ID from getWindows without reading AX. Providers reject closed or foreign windows when operating on them.
|
||||
- app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required.
|
||||
- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation.
|
||||
@@ -205,7 +205,7 @@ Setup and discovery:
|
||||
Available at app or window level depending on the platform.
|
||||
|
||||
Read getAccessibilityTree explicitly before using element ids. Selection and screenshots do not read or replace the current AX snapshot. Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- getAccessibilityTree(options?: { includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree(). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Use inspect to display the matches you need. Use element ids from the file only while that observation is current.
|
||||
- listActions(elementId: string) -> string[] — list the actions an element advertises.
|
||||
- invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none.
|
||||
- performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument.
|
||||
@@ -214,7 +214,7 @@ Read getAccessibilityTree explicitly before using element ids. Selection and scr
|
||||
|
||||
## Simulated mouse
|
||||
Use an explicit window for screenshots and pointer input. Before coordinate input, including relative coordinates, obtain a successful getAccessibilityTree observation covering that window or take its screenshot. Absolute pixel coordinates require a returned screenshot.
|
||||
- getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG.
|
||||
- getScreenshot() -> Image — capture the selected window as a stored image with readonly attachmentId, path, width and height; pass it to inspect to display it.
|
||||
- mouseMove(target: string | [number, number], options?: { relative?: boolean, useSystemPointer?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request a mouse move to the target without pressing a button; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without that route, HOVER_UNAVAILABLE is returned before sending input. Physical movement requires an unobscured foreground target. Explain any necessary activation and observe the target again.
|
||||
- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying.
|
||||
- scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left.
|
||||
@@ -229,14 +229,14 @@ Available at app or window level depending on the platform.
|
||||
- pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string.
|
||||
- sendKeyEvent(key: string, direction: 'down' | 'up') -> ActionResult — send 'down' to hold a key across tool calls in one program, for example W down, D down, D up, W up. Holding a key on a physical keyboard can produce repeated key-down events after a delay. To send one such repeat for a non-modifier key this run already holds, send 'down' again. Each call posts one event. A control can filter repeats. Use the app's response to decide whether and when to repeat or release the key. When this program returns, throws, or the user pauses it, macOS releases only its remaining keys at their original recipients.
|
||||
|
||||
Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree maps to getState; getWindow has no raw method; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name.
|
||||
Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. getAccessibilityTree maps to getState; getWindow has no raw method; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name.
|
||||
|
||||
Only if the task cannot be completed without moving the user’s pointer, call await win.mouseMove(target, { useSystemPointer: true }). Prefer an advertised accessibility action, a click on a visible control, or browser tools when possible. The question interrupts the user. When approval is disabled an explicit move can act immediately. Pass true on every physical move, even after approval. Omitting the option or passing false uses the default hover route without prompting. Approval or refusal lasts until this session closes or DSH restarts. A refusal sends no input. If the user dismisses the request, ask again only if they request it. Wait for an outstanding request to finish. Subagents cannot request physical movement.
|
||||
```
|
||||
|
||||
#### Token 影响
|
||||
|
||||
启用插件会增加固定参数定义和指导文本。AX 观察、操作结果和已接入的图片会增加模型输入。本包不将保留的对象序列化进提示。窗口中有更多节点可读时,调高默认值或单次读取上限可能使进入模型的 AX 文本变长。每个可见辅助窗口(例如状态栏图标、菜单或工具提示)都会给 `getWindows()` 和原始 `getState` 的结果增加一条窗口记录。截图需要单独请求。
|
||||
启用插件会增加固定参数定义和指导文本。直接调用原生 `cua` 工具时,结果文本和通过检查的截图会进入模型输入。在 TypeScript PTC 和 `cua_run_code` 中,程序用 `inspect` 选择要展示的文本和图片。在程序中保留应用或窗口对象不会增加提示词内容。窗口中有更多节点可读时,调高默认值或单次读取上限可能使返回的 AX 文本变长。每个可见辅助窗口(例如状态栏图标、菜单或工具提示)都会给 `getWindows()` 和原始 `getState` 的结果增加一条窗口记录。截图需要单独请求。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ ${COMMON_GUIDANCE.replace(/^- ([\w.]+)\(.*$/gm, (entry, method: string) => {
|
||||
}
|
||||
|
||||
/** Native PTC operation instructions with registered file tools for large observations. */
|
||||
export const GUIDANCE = operationGuidance('In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need.')
|
||||
export const GUIDANCE = operationGuidance('In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree(). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Use inspect to display the matches you need.')
|
||||
|
||||
/** CUA-only program instructions; other session tools remain outside the program. */
|
||||
export const PROGRAM_GUIDANCE = operationGuidance('Capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory and print only the matches you need. If needed, save it to a new workspace file with node:fs/promises and use the session’s file tools to search it.')
|
||||
export const PROGRAM_GUIDANCE = operationGuidance('Capture a large tree with const tree = await win.getAccessibilityTree(). Filter its lines in memory and use inspect to display the matches you need. If needed, save it to a new workspace file with node:fs/promises and use the session’s file tools to search it.')
|
||||
@@ -72,25 +72,26 @@ it('enumerates every available method once in its catalog entry', () => {
|
||||
})
|
||||
|
||||
it('documents option fields and return fields', () => {
|
||||
for (const token of ['emit', 'includeMenus', 'details', 'maxElements', 'maxDepth', 'relative', 'mouseButton', 'clickCount', 'holdMs', 'modifiers', 'format', 'prefix', 'suffix', 'selectionType']) expect(COMMON_GUIDANCE).toContain(token)
|
||||
for (const token of ['dispatched', 'foregroundChanged', 'hoverRoute', 'systemPointerMoved', 'bytes', 'mimeType', 'pixelWidth', 'pixelHeight', 'windowId']) expect(COMMON_GUIDANCE).toContain(token)
|
||||
for (const token of ['includeMenus', 'details', 'maxElements', 'maxDepth', 'relative', 'mouseButton', 'clickCount', 'holdMs', 'modifiers', 'format', 'prefix', 'suffix', 'selectionType']) expect(COMMON_GUIDANCE).toContain(token)
|
||||
for (const token of ['dispatched', 'foregroundChanged', 'hoverRoute', 'systemPointerMoved', 'path', 'width', 'height', 'windowId']) expect(COMMON_GUIDANCE).toContain(token)
|
||||
})
|
||||
|
||||
it('inlines discovery, restoration, screenshot, and mouse-move return records', () => {
|
||||
it('inlines discovery, restoration, and mouse-move return records', () => {
|
||||
const lines = COMMON_GUIDANCE.split('\n')
|
||||
for (const name of ['cua.getCapabilities', 'cua.listApps', 'cua.findApps', 'cua.getState', 'app.getWindows', 'app.show', 'window.show', 'getScreenshot', 'mouseMove']) {
|
||||
for (const name of ['cua.getCapabilities', 'cua.listApps', 'cua.findApps', 'cua.getState', 'app.getWindows', 'app.show', 'window.show', 'mouseMove']) {
|
||||
expect(lines.find(line => line.startsWith(`- ${name}(`))).toMatch(/ -> \(?\{ /)
|
||||
}
|
||||
expect(COMMON_GUIDANCE).toContain("{ platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, systemPointer: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false }")
|
||||
expect(COMMON_GUIDANCE).toContain('{ id: string, displayName: string, isRunning: true, pid: number }[]')
|
||||
expect(COMMON_GUIDANCE).toContain('({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[]')
|
||||
expect(COMMON_GUIDANCE).toContain('getScreenshot() -> Image')
|
||||
expect(COMMON_GUIDANCE).toContain('bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null')
|
||||
})
|
||||
|
||||
it('defines every non-builtin type name used in method signatures', () => {
|
||||
const signatures = COMMON_GUIDANCE.split('\n').filter(line => line.startsWith('- ')).map(line => line.split(' — ')[0]).join('\n')
|
||||
const names = signatures.replace(/'[^']*'/g, '').match(/\b[A-Z][A-Za-z0-9_]*\b/g) ?? []
|
||||
expect([...new Set(names)].sort()).toEqual(['ActionResult', 'App', 'JsonValue', 'Uint8Array', 'Window'])
|
||||
expect([...new Set(names)].sort()).toEqual(['ActionResult', 'App', 'Image', 'JsonValue', 'Window'])
|
||||
expect(COMMON_GUIDANCE).toContain('ActionResult = { dispatched: boolean, foregroundChanged?: boolean | null }')
|
||||
expect(COMMON_GUIDANCE).toContain('JsonValue = string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue }')
|
||||
expect(COMMON_GUIDANCE).toContain('App is the application object with readonly app.appHandle: string (opaque) and the methods listed below for app level')
|
||||
@@ -98,8 +99,7 @@ it('defines every non-builtin type name used in method signatures', () => {
|
||||
expect(COMMON_GUIDANCE).toContain('Signatures show resolved return values after await')
|
||||
})
|
||||
|
||||
it('preserves image formats and pointer modifier enums without restricting keyboard names', () => {
|
||||
expect(COMMON_GUIDANCE).toContain("mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'")
|
||||
it('documents pointer modifiers and platform-defined keyboard names', () => {
|
||||
const lines = COMMON_GUIDANCE.split('\n')
|
||||
for (const name of ['click', 'drag']) {
|
||||
expect(lines.find(line => line.startsWith(`- ${name}(`))).toContain("modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[]")
|
||||
@@ -120,11 +120,10 @@ it('states platform-dependent levels without qualifying grouped method names', (
|
||||
expect(COMMON_GUIDANCE).toContain('from: [number, number]')
|
||||
})
|
||||
|
||||
it('keeps consent, lock, emission, observation, and no-replay rules', () => {
|
||||
it('keeps consent, lock, observation, and no-replay rules', () => {
|
||||
expect(COMMON_GUIDANCE).toContain('consent')
|
||||
expect(COMMON_GUIDANCE).toContain('SESSION_LOCKED')
|
||||
expect(COMMON_GUIDANCE).toContain('Never replay uncertain input')
|
||||
expect(COMMON_GUIDANCE).toContain('emit: false suppresses sink output only')
|
||||
expect(COMMON_GUIDANCE).toContain('Read fresh state after actions')
|
||||
expect(COMMON_GUIDANCE).toMatch(/element ids are opaque strings/i)
|
||||
expect(COMMON_GUIDANCE).toContain('snapshotId')
|
||||
@@ -135,19 +134,12 @@ it('prioritizes reported API availability over instruction text in the prologue'
|
||||
expect(prologue).toContain('Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions.')
|
||||
})
|
||||
|
||||
it('separates facade options from raw RPC parameters', () => {
|
||||
expect(COMMON_GUIDANCE).toContain('facade-only')
|
||||
expect(COMMON_GUIDANCE).toContain('emit is local output control')
|
||||
expect(COMMON_GUIDANCE).toContain('raw operation schema')
|
||||
})
|
||||
|
||||
it('documents emitted observations and key arguments without assuming a transport', () => {
|
||||
expect(COMMON_GUIDANCE).not.toContain('not a preinstalled global')
|
||||
expect(COMMON_GUIDANCE).toContain('void await')
|
||||
it('documents explicit inspection and separate key arguments', () => {
|
||||
expect(COMMON_GUIDANCE).toContain('Call inspect(...values) to display selected observation results.')
|
||||
expect(COMMON_GUIDANCE).toContain('do not pack them into a chord string')
|
||||
expect(COMMON_GUIDANCE).toContain('perform a left-button double-click')
|
||||
})
|
||||
|
||||
it('omits stale names and unavailable platform operations', () => {
|
||||
for (const stale of ['getAXState', 'disableDiffing', 'performSecondaryAction', 'hover(', 'keyChord', 'window.typeText', 'window.pressKey', 'elementIndex', 'verified: false', 'browser tabs', 'not implemented']) expect(GUIDANCE).not.toContain(stale)
|
||||
for (const stale of ['getAXState', 'emit', 'disableDiffing', 'performSecondaryAction', 'hover(', 'keyChord', 'window.typeText', 'window.pressKey', 'elementIndex', 'verified: false', 'browser tabs', 'not implemented']) expect(GUIDANCE).not.toContain(stale)
|
||||
})
|
||||
@@ -109,7 +109,7 @@ it.skipIf(process.platform === 'win32')('delivers a valid PNG larger than 4 MiB
|
||||
if (this.requests.length === 1) {
|
||||
const code = 'const app = await tools.cua({ method: "getApp", params: { selector: "9" } }); '
|
||||
+ 'const result = await tools.cua({ method: "screenshot", params: { appHandle: app.appHandle, windowId: 7 } }); '
|
||||
+ 'return { mimeType: result.mimeType, encodedLength: result.data.length };'
|
||||
+ 'inspect({ path: result.path, width: result.width, height: result.height }, result);'
|
||||
const call = { type: 'tool-call' as const, id: ToolCallId('large-image-ptc'), name: 'run_code',
|
||||
arguments: JSON.stringify({ code, description: 'Inspect the generated large screenshot' }),
|
||||
}
|
||||
@@ -157,26 +157,26 @@ it.skipIf(process.platform === 'win32')('delivers a valid PNG larger than 4 MiB
|
||||
const result = messages.find(message => message.role === 'tool' && message.toolCallId === 'large-image-ptc')
|
||||
if (result?.role !== 'tool') throw new Error('The PTC result was not recorded')
|
||||
expect(result.isError, JSON.stringify(result.content)).toBe(false)
|
||||
expect(result.content.filter(block => block.type === 'text').map(block => block.text).join('\n'))
|
||||
.toContain(String(model.requestImages[0]?.toString('base64').length))
|
||||
const nextImages = model.requests[1]?.messages.flatMap(message => message.content).filter(block => block.type === 'image')
|
||||
expect(nextImages).toHaveLength(1)
|
||||
const image = nextImages?.[0]
|
||||
if (image === undefined) throw new Error('The next model request is missing the PTC screenshot')
|
||||
expect(image.attachment).toMatchObject({ width: WIDTH, height: HEIGHT })
|
||||
expect(image.attachment.width).toBeGreaterThan(0)
|
||||
expect(image.attachment.width).toBeLessThan(WIDTH)
|
||||
expect(result.content.filter(block => block.type === 'text').map(block => block.text).join('\n'))
|
||||
.toContain(`width: ${image.attachment.width}`)
|
||||
const stored = await context.attachments.readImage(image.attachment)
|
||||
const path = context.attachments.imageHostPath(image.attachment)
|
||||
if (path === undefined) throw new Error('The real local attachment has no disk path')
|
||||
expect(await readFile(path)).toEqual(Buffer.from(stored.data))
|
||||
expect(stored.data.byteLength).toBe(image.attachment.bytes)
|
||||
const decodedStored = await sharp(stored.data).raw().toBuffer({ resolveWithObject: true })
|
||||
expect(decodedStored.info).toMatchObject({ width: WIDTH, height: HEIGHT, channels: 3 })
|
||||
expect(decodedStored.info).toMatchObject({ width: image.attachment.width, height: image.attachment.height, channels: 3 })
|
||||
const dispatches = handle.agent.session.snapshotEvents().filter(event => event.type === 'tool/ptc-dispatch')
|
||||
expect(dispatches).toHaveLength(2)
|
||||
expect(dispatches.every(event => event.data.name === 'cua' && !event.data.isError)).toBe(true)
|
||||
expect(dispatches[1]?.data.content.find(block => block.type === 'image')?.attachment).toEqual(image.attachment)
|
||||
expect(messages.filter(message => message.source.kind === 'ptc-mode').flatMap(message => message.content))
|
||||
.toContainEqual({ type: 'image', attachment: image.attachment })
|
||||
expect(result.content).toContainEqual({ type: 'image', attachment: image.attachment })
|
||||
expect(JSON.stringify(model.requests[1]?.messages)).not.toContain(source.toString('base64'))
|
||||
|
||||
expect(model.requestImages).toHaveLength(1)
|
||||
|
||||
@@ -169,13 +169,19 @@ it.skipIf(process.platform === 'win32')('shares one application between retained
|
||||
const result = await executeToolFixture(context, { agent: owner, signal: new AbortController().signal,
|
||||
callId: ToolCallId(`retained-object-${++objectCall}`), name: 'cua', arguments: operation })
|
||||
if (result.isError) throw new Error(JSON.stringify(result.content))
|
||||
const image = result.content.find(block => block.type === 'image')
|
||||
if (image !== undefined) {
|
||||
const path = context.attachments.imageHostPath(image.attachment)
|
||||
if (path === undefined) throw new Error('Fixture image path is unavailable')
|
||||
return { attachmentId: image.attachment.attachmentId, path, width: image.attachment.width, height: image.attachment.height }
|
||||
}
|
||||
return z.json().parse(result.value)
|
||||
})
|
||||
const retainedApp = await objectApi(agent).getApp('test.fixture')
|
||||
const retainedWindow = await retainedApp.getWindow(7)
|
||||
await retainedApp.getAccessibilityTree({ maxDepth: 512 })
|
||||
expect(await retainedWindow.getAccessibilityTree({ maxElements: 100000, emit: false })).toContain('Count: 1')
|
||||
await retainedWindow.getAccessibilityTree({ emit: false })
|
||||
expect(await retainedWindow.getAccessibilityTree({ maxElements: 100000 })).toContain('Count: 1')
|
||||
await retainedWindow.getAccessibilityTree()
|
||||
const reads = (await readEvents()).filter(event => event.method === 'getState').slice(-3)
|
||||
expect(reads.map(event => event.params)).toEqual([
|
||||
{ appHandle: retainedApp.appHandle, maxElements: 5000, maxDepth: 512 },
|
||||
@@ -186,16 +192,16 @@ it.skipIf(process.platform === 'win32')('shares one application between retained
|
||||
const peerApp = await objectApi(peer.agent).getApp('test.fixture')
|
||||
const peerWindow = await peerApp.getWindow(7)
|
||||
expect(peerApp.appHandle).not.toBe(retainedApp.appHandle)
|
||||
const screenshot = await peerWindow.getScreenshot({ emit: false })
|
||||
expect(screenshot).toMatchObject({ mimeType: 'image/png', pixelWidth: 1, pixelHeight: 1, windowId: 7 })
|
||||
expect(Buffer.from(screenshot.bytes).toString('base64')).toBe(png)
|
||||
await (await peerApp.getWindow(8)).getAccessibilityTree({ emit: false })
|
||||
const screenshot = await peerWindow.getScreenshot()
|
||||
expect(screenshot).toMatchObject({ width: 1, height: 1 })
|
||||
expect((await readFile(screenshot.path)).toString('base64')).toBe(png)
|
||||
await (await peerApp.getWindow(8)).getAccessibilityTree()
|
||||
const recovery = 'Snapshot belongs to window 8; read getAccessibilityTree on window 7'
|
||||
await expect(peerWindow.click(element('0'))).rejects.toThrow(recovery)
|
||||
await expect(peerWindow.mouseMove(element('0'))).rejects.toThrow(recovery)
|
||||
expect(await peerWindow.getAccessibilityTree({ emit: false })).toContain('Count: 1')
|
||||
expect(await peerWindow.getAccessibilityTree()).toContain('Count: 1')
|
||||
await peerWindow.click(element('0'))
|
||||
expect(await retainedWindow.getAccessibilityTree({ emit: false })).toContain('Count: 2')
|
||||
expect(await retainedWindow.getAccessibilityTree()).toContain('Count: 2')
|
||||
const sharedEvents = await readEvents()
|
||||
expect(sharedEvents.filter(event => event.event === 'start')).toHaveLength(1)
|
||||
expect(sharedEvents.filter(event => event.event === 'connect')).toHaveLength(2)
|
||||
@@ -204,7 +210,7 @@ it.skipIf(process.platform === 'win32')('shares one application between retained
|
||||
expect((await readEvents()).filter(event => event.event === 'exit')).toHaveLength(0)
|
||||
expect((await readEvents()).filter(event => event.event === 'disconnect')).toHaveLength(1)
|
||||
await peerWindow.click(element('0'))
|
||||
expect(await peerWindow.getAccessibilityTree({ emit: false })).toContain('Count: 3')
|
||||
expect(await peerWindow.getAccessibilityTree()).toContain('Count: 3')
|
||||
await peer.dispose()
|
||||
expect((await readEvents()).filter(event => event.event === 'exit')).toHaveLength(1)
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@ import ComputerUseRegistry from '@deepseek-ai/dsh-computer-use'
|
||||
import FileSystem from '@deepseek-ai/dsh-fs-local'
|
||||
import { LlmAdapter, ToolCallId, type LlmResolvedModelInfo, type StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import NodeRuntime from '@deepseek-ai/dsh-ptc-runtime-node'
|
||||
import { PtcImage } from '@deepseek-ai/dsh-ptc-runtime'
|
||||
import Sandbox from '@deepseek-ai/dsh-sandbox-local'
|
||||
import SandboxPolicy from '@deepseek-ai/dsh-sandbox-policy'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
@@ -66,17 +67,34 @@ it.skipIf(process.platform === 'win32' || !existsSync(bridge) || !existsSync(boo
|
||||
calls.push({ phase, method: operation.method })
|
||||
const result = await executeToolFixture(ctx, { agent: handle.agent, signal: new AbortController().signal,
|
||||
callId: ToolCallId(`external-cua-${++count}`), name: 'cua', arguments: operation })
|
||||
if (result.isError) throw new Error(result.error.message)
|
||||
images += result.content.filter(block => block.type === 'image').length
|
||||
return result.value ?? null
|
||||
const image = result.content.find(block => block.type === 'image')
|
||||
if (image !== undefined) {
|
||||
const path = ctx.attachments.imageHostPath(image.attachment)
|
||||
if (path === undefined) throw new Error('Fixture image path is unavailable')
|
||||
return new PtcImage(path, image.attachment)
|
||||
}
|
||||
return result.value
|
||||
} } }],
|
||||
}))
|
||||
expect((await run('await import("@deepseek-ai/dsh-cua/bridge")', 'unresolved')).error?.message).toContain('Cannot find package')
|
||||
const section = (await ctx.systemPrompt.assemble()).sections.find(value => value.name === 'computer-use:dsh-cua')
|
||||
const initialization = section?.text.match(/Persistent PTC initialization:\n\n```ts\n([\s\S]*?)\n```/u)?.[1]
|
||||
expect(initialization).toContain('/lib/bridge.js')
|
||||
const first = await run(`${initialization}\nvar app = await cua.getApp("test.fixture"); var win = await app.getWindow(7); await win.getAccessibilityTree({emit:false}); await win.click("0"); var shot = await win.getScreenshot(); ({ mimeType: shot.mimeType, pixelWidth: shot.pixelWidth, pixelHeight: shot.pixelHeight, windowId: shot.windowId, byteLength: shot.bytes.length });`, 'first')
|
||||
const first = await run(`${initialization}
|
||||
var app = await cua.getApp("test.fixture");
|
||||
var win = await app.getWindow(7);
|
||||
await win.getAccessibilityTree();
|
||||
await win.click("0");
|
||||
var shot = await win.getScreenshot();
|
||||
var { readFile } = await import("node:fs/promises");
|
||||
({ isImage: shot instanceof Image, width: shot.width, height: shot.height,
|
||||
windowId: win.windowId, byteLength: (await readFile(shot.path)).byteLength });
|
||||
`, 'first')
|
||||
expect(first.error).toBeUndefined()
|
||||
expect(first.value).toEqual({ mimeType: 'image/png', pixelWidth: 1, pixelHeight: 1, windowId: 7, byteLength: 90 })
|
||||
expect(first.output).toEqual([])
|
||||
expect(first.value).toEqual({ isImage: true, width: 1, height: 1, windowId: 7, byteLength: 90 })
|
||||
expect(images).toBe(1)
|
||||
expect(JSON.stringify(first)).not.toContain('iVBORw0KGgo')
|
||||
const invalid = await run('await cua.findApps("");', 'invalid')
|
||||
@@ -84,10 +102,15 @@ it.skipIf(process.platform === 'win32' || !existsSync(bridge) || !existsSync(boo
|
||||
expect(invalid.error?.message).not.toContain('ZodError')
|
||||
expect(invalid.error?.message).not.toContain('"origin"')
|
||||
expect(calls.filter(call => call.phase === 'invalid')).toEqual([])
|
||||
const second = await run('await win.getAccessibilityTree({emit:false}); await win.click("0"); await win.getAccessibilityTree({emit:false});', 'second')
|
||||
const second = await run(`
|
||||
await win.getAccessibilityTree();
|
||||
await win.click("0");
|
||||
const tree = await win.getAccessibilityTree();
|
||||
inspect(tree.includes("Count: 2"), (await cua.getCapabilities()).platform);
|
||||
`, 'second')
|
||||
expect(second.error).toBeUndefined()
|
||||
expect(second.value).toContain('Count: 2')
|
||||
expect(calls.filter(call => call.phase === 'second').map(call => call.method)).toEqual(['getState', 'click', 'getState'])
|
||||
expect(second.output).toEqual([{ type: 'text', text: 'true macos\n' }])
|
||||
expect(calls.filter(call => call.phase === 'second').map(call => call.method)).toEqual(['getState', 'click', 'getState', 'getCapabilities'])
|
||||
} finally {
|
||||
try { await ctx.fiber.dispose() } finally {
|
||||
try { await closeFixture?.() } finally { await rm(cwd, { recursive: true, force: true }) }
|
||||
|
||||
@@ -15,8 +15,8 @@
|
||||
en: 5db4d4fea25493b9
|
||||
zh: 65ab64fc24eb0d09
|
||||
/deepseek-ai-dsh-cua/use-this-package:
|
||||
en: 8ca39a733b479dd3
|
||||
zh: 582c86bd18ef56fd
|
||||
en: ddfab1dd44960f47
|
||||
zh: 9d1c7b81e012f168
|
||||
/deepseek-ai-dsh-cua/use-this-package/permission-controls:
|
||||
en: 8941f2139fd1ca3a
|
||||
zh: 4c8fb953d821eaf1
|
||||
@@ -36,8 +36,8 @@
|
||||
en: b613371a761f0ab8
|
||||
zh: 826a85089ee3518c
|
||||
/deepseek-ai-dsh-cua/model-experience/authorized-application-operations/token-effect:
|
||||
en: 602a4ad8d6709512
|
||||
zh: de7ef79cc1abb442
|
||||
en: b9b9827c0baf946d
|
||||
zh: 137bf1a711bba968
|
||||
/deepseek-ai-dsh-cua/model-experience/authorized-application-operations/kv-cache-effect:
|
||||
en: 2d5df52dafa1d940
|
||||
zh: 06e815759b9150c3
|
||||
|
||||
@@ -31,7 +31,7 @@ Application consent supplies English, Chinese, and Japanese display text from `s
|
||||
|
||||
Mount the [macOS provider](../cua-macos/README.md) to enable native operations on macOS. External PTC hosts import `connectCua` from `@deepseek-ai/dsh-cua/bridge` and supply the current `tools.cua` binding. Source compositions and native preparation belong to the provider.
|
||||
|
||||
`cua_run_code` runs TypeScript in a separate persistent Node environment for each Agent. The global `cua` object is available before the program starts; `tools` contains only `cua`. Call ordinary tools directly outside the program. Use top-level `await` and a final expression for the result; top-level `return` is invalid. The tool accepts the same timeout and sandbox controls as `run_code`. A restarted runtime automatically recreates `cua`, but other variables and selected application objects must be created again.
|
||||
`cua_run_code` runs TypeScript in a separate persistent Node environment for each Agent. The global `cua` object is available before the program starts; `tools` contains only `cua`. Call ordinary tools directly outside the program. Use top-level `await` and `inspect(...values)` to display selected text and screenshots; top-level `return` is invalid. The tool accepts the same timeout and sandbox controls as `run_code`. A restarted runtime automatically recreates `cua`, but other variables and selected application objects must be created again.
|
||||
|
||||
Configure runtime resources and launch settings through the provider's `standaloneRunCode.runtime` and program call limits through `standaloneRunCode.executor`. Runtime execution is fixed to `persistent-in-memory`; setting `executionMode` or `scriptPersistence` there is rejected. These settings do not change the profile's shared PTC runtime.
|
||||
|
||||
@@ -43,9 +43,9 @@ Use `cua.getApp(selector)` to select an application; it obtains consent and opti
|
||||
|
||||
Keyboard methods are available in the scopes a provider declares; the provider owns how input is delivered within each scope. A lost connection invalidates its handles and coordinates. Cleanup failures are reported and the connection is not reused; select the application again and inspect fresh state, because earlier input may have partially executed. Screenshot image errors retain the failing stage and attachment error code. Invalid request arguments reject before dispatch with a readable method, parameter and requirement.
|
||||
|
||||
Observation methods emit and return their result. Use `void await app.getAccessibilityTree()` for one visible copy, or use `emit:false` and print the returned text once. `window.getScreenshot()` returns `CuaScreenshot` with `bytes`, `mimeType`, `pixelWidth`, `pixelHeight`, and `windowId`. The image sink receives the same object. Use its MIME type when selecting a file extension or declaring an image; the format can be PNG, JPEG, WebP, or GIF. Each screenshot enters the attachment pipeline once; `emit:false` only suppresses the object sink.
|
||||
Connect PTC with `connectCua(() => tools.cua)` or supply a custom dispatcher to `createCuaApi(dispatch)`. Observation methods return their results to the program. Call `inspect(await app.getWindows())` to display a result, or pass a filtered value. `window.getScreenshot()` returns an `Image` whose `path` is readable by the program and whose `width` and `height` describe the stored pixels. Use `inspect("before", before, "after", after)` to display selected screenshots. Read the window ID from `window.windowId`.
|
||||
|
||||
`tools.cua` returns operation values directly. Screenshot values retain the encoded image Base64 and add a durable `attachment` reference for the tool renderer; model content contains coordinate metadata and one image, without the encoded bytes. Before capture, the tool requires an attachment store and a resolved model route declaring image input. Admission and storage failures reject the call. The object bridge preserves the screenshot bytes and validated metadata without uploading or emitting the same image again.
|
||||
`tools.cua` screenshot operations return `Image` in TypeScript programs. Host-side canonical screenshot results carry encoded image data and a durable `attachment` reference; the native renderer projects coordinate metadata and the image into model content. Capture requires an attachment store and a resolved model route declaring image input. Admission and storage failures reject the call.
|
||||
|
||||
Application consent precedes launch, observation, capture, and input. Session access is stored by conversation and application identity and survives DSH restart and Computer Use disablement. Other conversations require their own consent. Saved grants require a platform-provided reusable application identity. Refusal and revocation close affected Agent connections, so their old objects require fresh selection. No input is replayed after cancellation, timeout, or reconnect.
|
||||
|
||||
@@ -142,7 +142,7 @@ With the default preset selection, PTC receives `cua` through its existing `run_
|
||||
|
||||
#### Token effect
|
||||
|
||||
The selected tool schema and CUA guidance add model input. Program output, operation results, and admitted images add history; only the CUA binding is described inside the program. Retained JavaScript objects are not serialized into prompts. Duplicate caller printing can repeat observation text.
|
||||
The selected tool schema and CUA guidance add model input. Native `cua` calls send their result text and admitted screenshot images to the model. In TypeScript PTC and `cua_run_code`, programs select text and images with `inspect`. Retaining an app or window object in the program adds no prompt content.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
|
||||
@@ -31,7 +31,7 @@ kind: "package-reference"
|
||||
|
||||
挂载 [macOS 提供方](../cua-macos/README.zh.md)以在 macOS 上启用原生操作。外部 PTC 宿主从 `@deepseek-ai/dsh-cua/bridge` 导入 `connectCua`,并提供当前 `tools.cua` 绑定。源码组合与原生组件准备由提供方负责。
|
||||
|
||||
`cua_run_code` 为每个 agent(智能体)在独立的持久 Node 环境中运行 TypeScript。程序开始前即可使用全局 `cua` 对象;`tools` 中只有 `cua`。其他普通工具在程序外直接调用。使用顶层 `await`,以最后一个表达式作为结果;顶层 `return` 无效。工具接受与 `run_code` 相同的超时和沙箱控制参数。运行时重启会自动重建 `cua`,但其他变量和已选择的应用对象需要重新创建。
|
||||
`cua_run_code` 为每个 agent(智能体)在独立的持久 Node 环境中运行 TypeScript。程序开始前即可使用全局 `cua` 对象;`tools` 中只有 `cua`。其他普通工具在程序外直接调用。使用顶层 `await`,通过 `inspect(...values)` 展示选中的文字和截图;顶层 `return` 无效。工具接受与 `run_code` 相同的超时和沙箱控制参数。运行时重启会自动重建 `cua`,但其他变量和已选择的应用对象需要重新创建。
|
||||
|
||||
通过提供方的 `standaloneRunCode.runtime` 配置运行时资源和启动设置,通过 `standaloneRunCode.executor` 配置程序调用上限。运行时固定使用 `persistent-in-memory`,在其中设置 `executionMode` 或 `scriptPersistence` 会被拒绝。这些设置不改变 profile 的共享 PTC 运行时。
|
||||
|
||||
@@ -43,9 +43,9 @@ kind: "package-reference"
|
||||
|
||||
键盘方法可在 provider 声明的作用域中使用;每个作用域内的输入投递由 provider 负责。连接丢失会使其句柄和坐标失效。清理失败会被报告,该连接不再复用;应重新选择应用并读取新状态,因为先前的输入可能已部分执行。截图图像错误会保留失败阶段和附件错误码。请求参数校验失败时,在分派前返回包含方法、参数和要求的可读消息。
|
||||
|
||||
观察方法会输出并返回结果。使用 `void await app.getAccessibilityTree()` 只展示一份内容,或使用 `emit:false` 后打印一次返回文本。`window.getScreenshot()` 返回包含 `bytes`、`mimeType`、`pixelWidth`、`pixelHeight` 和 `windowId` 的 `CuaScreenshot`,图片输出回调接收同一个对象。选择文件后缀或声明图片时使用其中的 MIME 类型;格式可能是 PNG、JPEG、WebP 或 GIF。每张截图只进入一次附件处理链;`emit:false` 只抑制对象层的输出。
|
||||
通过 `connectCua(() => tools.cua)` 连接 PTC,或将自定义分派器传给 `createCuaApi(dispatch)`。观察方法返回结果供程序使用。需要展示时,调用 `inspect(await app.getWindows())` 或传入筛选后的值。`window.getScreenshot()` 返回 `Image`,其 `path` 指向程序可读取的附件文件,`width` 和 `height` 是图片的像素尺寸。使用 `inspect("操作前", before, "操作后", after)` 展示选中的截图,从 `window.windowId` 读取窗口 ID。
|
||||
|
||||
`tools.cua` 直接返回操作结果。截图结果保留图片 Base64,并增加供工具渲染使用的持久 `attachment` 引用;模型内容只包含坐标元数据和一张图片,不包含编码字节。截图前要求附件服务可用,并确认当前模型支持图片输入。图片接入或保存失败会拒绝调用。对象桥接保留截图字节和已校验的元数据,不重复上传或输出同一张图片。
|
||||
`tools.cua` 的截图操作在 TypeScript 程序中返回 `Image`。宿主侧的规范化截图结果包含编码图像数据与持久的 `attachment` 引用;原生渲染器将坐标元数据和图片投影为模型内容。截图前要求附件服务可用,并确认当前模型支持图片输入。图片接入或保存失败会拒绝调用。
|
||||
|
||||
应用授权发生在启动、观察、截图和输入之前。会话许可按对话和应用身份持久化,重启 DSH 或关闭 Computer Use 后仍可复用,其他对话需要单独授权。长期许可要求平台提供可复用的应用身份。拒绝或撤销会关闭受影响的 Agent 连接,因此旧对象必须重新选择目标。取消、超时或重新连接后均不重放输入。
|
||||
|
||||
@@ -142,7 +142,7 @@ Computer Use 详情页的权限卡片、组件列表和来源信息使用 l3 边
|
||||
|
||||
#### Token 影响
|
||||
|
||||
选用的工具 schema 和 CUA 指引增加模型输入。程序输出、操作结果和接纳的图像增加历史;程序内只描述 CUA 绑定。保留的 JavaScript 对象不会序列化进提示词。调用方重复打印会重复观察文本。
|
||||
相应工具的参数定义和 CUA 使用说明会进入模型输入。直接调用原生 `cua` 工具时,结果文本和通过检查的截图会进入模型输入。在 TypeScript PTC 和 `cua_run_code` 中,程序用 `inspect` 选择要展示的文本和图片。在程序中保留应用或窗口对象不会增加提示词内容。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
/** Object facade over logged native operations; its caller owns evaluator persistence. */
|
||||
import { Buffer } from 'node:buffer'
|
||||
import type { z } from 'zod'
|
||||
import type { JsonValue } from '@deepseek-ai/dsh-util-values'
|
||||
import { ActionNamesResult, ActionResult, AppResult, AppsResult, CapabilitiesResult, FoundAppsResult, HoverResult, MouseMoveTargets, Request, ScreenshotResult, ShowResult, StateResult, WindowsResult } from './protocol.ts'
|
||||
import type { CuaApi, CuaApp, CuaAppHandle, CuaCapabilities, CuaDispatch, CuaObservationOptions, CuaObservationSink, CuaOperation, CuaScreenshot, CuaSurface, CuaWindow } from './types.ts'
|
||||
import { ActionNamesResult, ActionResult, AppResult, AppsResult, CapabilitiesResult, FoundAppsResult, HoverResult, MouseMoveTargets, Request, ShowResult, StateResult, WindowsResult } from './protocol.ts'
|
||||
import type { CuaApi, CuaApp, CuaAppHandle, CuaCapabilities, CuaDispatch, CuaOperation, CuaProgramImage, CuaSurface, CuaWindow } from './types.ts'
|
||||
export type * from './types.ts'
|
||||
|
||||
function requestIssueMessage(issues: readonly z.core.$ZodIssue[], parent: readonly PropertyKey[] = []): string {
|
||||
@@ -37,55 +36,38 @@ function validatedCall(dispatch: CuaDispatch) {
|
||||
}
|
||||
}
|
||||
|
||||
async function emit(sink: CuaObservationSink | undefined, value: object, options?: CuaObservationOptions): Promise<void> {
|
||||
if (options?.emit !== false) await sink?.text(JSON.stringify(value))
|
||||
}
|
||||
|
||||
/**
|
||||
* Create app/window objects using the caller's authorized dispatcher.
|
||||
* @param dispatch - Current tool authority; rejects propagate without retries.
|
||||
* @param sink - Optional text/image output; emit false suppresses this sink only.
|
||||
* Create app/window objects using the caller's authorized program dispatcher.
|
||||
* @param dispatch - Current tool authority; screenshots resolve to program Images and rejections propagate without retries.
|
||||
* @returns Objects independent of variable storage and evaluator lifecycle.
|
||||
*/
|
||||
export function createCuaApi(dispatch: CuaDispatch, sink?: CuaObservationSink): CuaApi {
|
||||
export function createCuaApi(dispatch: CuaDispatch): CuaApi {
|
||||
const call = validatedCall(dispatch)
|
||||
let capabilities: CuaCapabilities | undefined
|
||||
async function currentCapabilities(): Promise<CuaCapabilities> {
|
||||
if (capabilities === undefined) capabilities = CapabilitiesResult.parse(await call('getCapabilities', {}))
|
||||
return capabilities
|
||||
}
|
||||
const objects = createCuaObjects(call, sink)
|
||||
const objects = createCuaObjects(call)
|
||||
|
||||
const api: CuaApi = {
|
||||
async getCapabilities(options) {
|
||||
const result = CapabilitiesResult.parse(await call('getCapabilities', {}))
|
||||
capabilities = result
|
||||
await emit(sink, result, options)
|
||||
return result
|
||||
async getCapabilities() {
|
||||
capabilities = CapabilitiesResult.parse(await call('getCapabilities', {}))
|
||||
return capabilities
|
||||
},
|
||||
async listApps(options) {
|
||||
const result = AppsResult.parse(await call('listApps', {})).map(app => ({
|
||||
async listApps() {
|
||||
return AppsResult.parse(await call('listApps', {})).map(app => ({
|
||||
id: app.bundleId || String(app.pid), displayName: app.name, isRunning: true as const, pid: app.pid,
|
||||
}))
|
||||
await emit(sink, result, options)
|
||||
return result
|
||||
},
|
||||
async findApps(query, options) {
|
||||
const result = FoundAppsResult.parse(await call('findApps', { query }))
|
||||
await emit(sink, result, options)
|
||||
return result
|
||||
async findApps(query) {
|
||||
return FoundAppsResult.parse(await call('findApps', { query }))
|
||||
},
|
||||
async getState(options) {
|
||||
const result = { apps: await api.listApps({ emit: false }), browsers: [] as [] }
|
||||
await emit(sink, result, options)
|
||||
return result
|
||||
async getState() {
|
||||
return { apps: await api.listApps(), browsers: [] }
|
||||
},
|
||||
async getApp(selector) {
|
||||
const request = Request.safeParse({ method: 'getApp', params: { selector } })
|
||||
if (!request.success) throw new Error(`getApp ${requestIssueMessage(request.error.issues)}`)
|
||||
const capabilities = await currentCapabilities()
|
||||
const current = capabilities ?? await api.getCapabilities()
|
||||
const { appHandle } = AppResult.parse(await call('getApp', { selector }))
|
||||
return objects.createCuaApp(appHandle, capabilities)
|
||||
return objects.createCuaApp(appHandle, current)
|
||||
},
|
||||
}
|
||||
return api
|
||||
@@ -93,7 +75,7 @@ export function createCuaApi(dispatch: CuaDispatch, sink?: CuaObservationSink):
|
||||
|
||||
type CuaScopeParams = (params: Record<string, JsonValue>) => Record<string, JsonValue>
|
||||
|
||||
function createCuaObjects(call: ReturnType<typeof validatedCall>, sink: CuaObservationSink | undefined): {
|
||||
function createCuaObjects(call: ReturnType<typeof validatedCall>): {
|
||||
createCuaApp(appHandle: CuaAppHandle, capabilities: CuaCapabilities): CuaApp
|
||||
} {
|
||||
const observed = new Map<CuaAppHandle, { snapshotId: string; windowId: number | undefined }>()
|
||||
@@ -133,7 +115,6 @@ function createCuaObjects(call: ReturnType<typeof validatedCall>, sink: CuaObser
|
||||
throw new Error('Native observation belongs to another application or window')
|
||||
}
|
||||
observed.set(target.appHandle, { snapshotId: state.snapshotId, windowId: target.windowId })
|
||||
if (options?.emit !== false) await sink?.text(state.tree)
|
||||
return state.tree
|
||||
},
|
||||
async listActions(elementId) {
|
||||
@@ -192,18 +173,14 @@ function createCuaObjects(call: ReturnType<typeof validatedCall>, sink: CuaObser
|
||||
})
|
||||
return {
|
||||
...base.surface, windowId,
|
||||
async getScreenshot(options) {
|
||||
const image = ScreenshotResult.parse(await base.call('screenshot', {}))
|
||||
if (image.windowId !== windowId) throw new Error('Native screenshot belongs to another window')
|
||||
const screenshot: CuaScreenshot = {
|
||||
bytes: Uint8Array.from(Buffer.from(image.data, 'base64')),
|
||||
mimeType: image.mimeType,
|
||||
pixelWidth: image.pixelWidth,
|
||||
pixelHeight: image.pixelHeight,
|
||||
windowId: image.windowId,
|
||||
async getScreenshot() {
|
||||
const value = await base.call('screenshot', {})
|
||||
if (value === null || typeof value !== 'object' || Array.isArray(value)
|
||||
|| typeof value.attachmentId !== 'string'
|
||||
|| typeof value.path !== 'string' || typeof value.width !== 'number' || typeof value.height !== 'number') {
|
||||
throw new Error('The CUA program binding must return an Image for screenshots')
|
||||
}
|
||||
if (options?.emit !== false) await sink?.image(screenshot)
|
||||
return screenshot
|
||||
return value as CuaProgramImage
|
||||
},
|
||||
async mouseMove(targetPoint, options) {
|
||||
const relative = options?.relative === undefined ? {} : { relative: options.relative }
|
||||
@@ -259,10 +236,8 @@ function createCuaObjects(call: ReturnType<typeof validatedCall>, sink: CuaObser
|
||||
return {
|
||||
...base.surface, appHandle,
|
||||
async activate() { return base.action('activate', {}) },
|
||||
async getWindows(options) {
|
||||
const windows = WindowsResult.parse(await base.call('listWindows', {}))
|
||||
await emit(sink, windows, options)
|
||||
return windows
|
||||
async getWindows() {
|
||||
return WindowsResult.parse(await base.call('listWindows', {}))
|
||||
},
|
||||
getWindow(windowId): Promise<CuaWindow> {
|
||||
const result = WindowsResult.element.shape.windowId.safeParse(windowId)
|
||||
|
||||
@@ -3,19 +3,14 @@ import type { JsonValue } from '@deepseek-ai/dsh-util-values'
|
||||
import { createCuaApi } from './api.ts'
|
||||
import type { CuaApi, CuaOperation } from './types.ts'
|
||||
|
||||
/** Standard tools.cua binding; its image content is admitted by the host tool pipeline. */
|
||||
/** Standard tools.cua binding; screenshots resolve to stored program images. */
|
||||
export type CuaToolBinding = (operation: CuaOperation) => Promise<JsonValue>
|
||||
|
||||
/**
|
||||
* Connect the object API without owning an evaluator or its lifetime.
|
||||
* @param getTool - Resolves the current execution's tools.cua binding on every operation; do not capture a completed execution's binding.
|
||||
* @param onText - Optional text output captured by the caller's execution environment.
|
||||
* @returns Application and window objects using the current tool binding. Rejections propagate without retries.
|
||||
*/
|
||||
export function connectCua(getTool: () => CuaToolBinding, onText?: (text: string) => void | Promise<void>): CuaApi {
|
||||
return createCuaApi(operation => getTool()(operation), {
|
||||
text: value => onText?.(value),
|
||||
// The standard tools.cua result already carries the screenshot attachment.
|
||||
image: () => {},
|
||||
})
|
||||
export function connectCua(getTool: () => CuaToolBinding): CuaApi {
|
||||
return createCuaApi(operation => getTool()(operation))
|
||||
}
|
||||
@@ -2,19 +2,19 @@
|
||||
const hoverSignature = 'mouseMove(target: string | [number, number], options?: { relative?: boolean })'
|
||||
|
||||
/** Async method catalog with consent and observation rules, grouped by observation, mouse, and keyboard. */
|
||||
export const COMMON_GUIDANCE = `Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation.
|
||||
export const COMMON_GUIDANCE = `Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation.
|
||||
|
||||
Use void await for observations that emit, and never stringify screenshot image bytes.
|
||||
Call inspect(...values) to display selected observation results.
|
||||
|
||||
Signatures show resolved return values after await. ActionResult = { dispatched: boolean, foregroundChanged?: boolean | null }; foregroundChanged is omitted when not sampled and null when the sample is unavailable. JsonValue = string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue }; numbers must be finite. App is the application object with readonly app.appHandle: string (opaque) and the methods listed below for app level. Window is the window object with readonly window.windowId: number and the methods listed below for window level. Neither object carries pid.
|
||||
|
||||
Setup and discovery:
|
||||
- cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, systemPointer: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions.
|
||||
- cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications.
|
||||
- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid.
|
||||
- cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation.
|
||||
- cua.getCapabilities() -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, systemPointer: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions.
|
||||
- cua.listApps() -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications.
|
||||
- cua.findApps(query: string) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid.
|
||||
- cua.getState() -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation.
|
||||
- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and return without reading AX or activating the app.
|
||||
- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership.
|
||||
- app.getWindows() -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership.
|
||||
- app.getWindow(windowId: number) -> Window — select a window by an ID from getWindows without reading AX. Providers reject closed or foreign windows when operating on them.
|
||||
- app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required.
|
||||
- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation.
|
||||
@@ -24,7 +24,7 @@ Setup and discovery:
|
||||
Available at app or window level depending on the platform.
|
||||
|
||||
Read getAccessibilityTree explicitly before using element ids. Selection and screenshots do not read or replace the current AX snapshot. Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names.
|
||||
- getAccessibilityTree(options?: { includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names.
|
||||
- listActions(elementId: string) -> string[] — list the actions an element advertises.
|
||||
- invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none.
|
||||
- performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument.
|
||||
@@ -33,7 +33,7 @@ Read getAccessibilityTree explicitly before using element ids. Selection and scr
|
||||
|
||||
## Simulated mouse
|
||||
Use an explicit window for screenshots and pointer input. Before coordinate input, including relative coordinates, obtain a successful getAccessibilityTree observation covering that window or take its screenshot. Absolute pixel coordinates require a returned screenshot.
|
||||
- getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG.
|
||||
- getScreenshot() -> Image — capture the selected window as a stored image with readonly attachmentId, path, width, and height. Pass selected screenshots to inspect(...values) to display them; read the window ID from window.windowId.
|
||||
- ${hoverSignature} -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request a mouse move to the target without pressing a button; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration.
|
||||
- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press.
|
||||
- scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left.
|
||||
@@ -47,7 +47,7 @@ Available at app or window level depending on the platform.
|
||||
- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry.
|
||||
- pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string.
|
||||
|
||||
Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree maps to getState; getWindow has no raw method; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name.`
|
||||
Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. getAccessibilityTree maps to getState; getWindow has no raw method; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name.`
|
||||
|
||||
/**
|
||||
* Describe the system-pointer option for a provider that declares the capability.
|
||||
|
||||
@@ -262,13 +262,17 @@ export function installCua(ctx: Context, config: CuaConfig, backend: CuaBackend)
|
||||
}
|
||||
const refs = await stage('image admission', () => admitEncodedImages(attachments, [{ data: screenshot.data, mediaType: screenshot.mimeType }]))
|
||||
signal.throwIfAborted()
|
||||
const attachment = refs[0] as ImageAttachmentRef
|
||||
let attachment = refs[0] as ImageAttachmentRef
|
||||
const original = attachment.originalDimensions ?? attachment
|
||||
if (original.width !== screenshot.pixelWidth || original.height !== screenshot.pixelHeight) {
|
||||
throw new NativeError('NATIVE_PROTOCOL', 'Native screenshot dimensions disagree with its decoded image')
|
||||
}
|
||||
const image = await stage('model image preparation', () => attachments.readImageRequest(attachment, target(attachment), signal))
|
||||
signal.throwIfAborted()
|
||||
if (image.width !== attachment.width || image.height !== attachment.height) {
|
||||
attachment = await stage('image storage', () => attachments.saveImage({ data: image.data, mediaType: image.mediaType }))
|
||||
signal.throwIfAborted()
|
||||
}
|
||||
execution.coordinates.screenshot(parsedParams.appHandle as CuaAppHandle, screenshot.windowId, screenshot.bounds, image)
|
||||
return { ...screenshot, mimeType: image.mediaType, data: Buffer.from(image.data).toString('base64'),
|
||||
pixelWidth: image.width, pixelHeight: image.height, coordinateSpace: 'screenshot-pixels', attachment }
|
||||
|
||||
@@ -14,7 +14,7 @@ import type { CuaConfig } from './index.ts'
|
||||
const programDescription = 'Run a TypeScript program to observe and operate desktop applications. '
|
||||
+ 'The global cua object is ready to use; retain application and window objects across calls. '
|
||||
+ 'Only tools.cua is available inside the program. Call other session tools directly. '
|
||||
+ 'Print the values you need or leave a final expression; screenshots are attached after the program finishes.'
|
||||
+ 'Use inspect(...values) to display selected text and images.'
|
||||
|
||||
/**
|
||||
* Register raw CUA for its configured PTC preset and a persistent program tool for other presets.
|
||||
@@ -42,7 +42,7 @@ Persistent PTC initialization:
|
||||
|
||||
\`\`\`ts
|
||||
var { connectCua } = await import(${ptcBridgeUrl});
|
||||
var cua = connectCua(() => tools.cua, text => console.log(text));
|
||||
var cua = connectCua(() => tools.cua);
|
||||
\`\`\``
|
||||
} }
|
||||
const register = (scope: Context, definition: ToolDefinition, instructions: PromptSection): (() => void) => {
|
||||
@@ -92,7 +92,7 @@ var cua = connectCua(() => tools.cua, text => console.log(text));
|
||||
apply(owner: Context) {
|
||||
owner.effect(function* () {
|
||||
yield owner.ptcRuntime.registerEnvironmentInitializer(
|
||||
`globalThis.cua = (await import(${programBridgeUrl})).connectCua(() => tools.cua, text => console.log(text));`,
|
||||
`globalThis.cua = (await import(${programBridgeUrl})).connectCua(() => tools.cua);`,
|
||||
)
|
||||
const child = owner.plugin({
|
||||
name: 'cua-program-tools', inject: ['ptcRuntime', 'tools', 'workingDirectory'],
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
import type { JsonValue } from '@deepseek-ai/dsh-util-values'
|
||||
import type { z } from 'zod'
|
||||
import type { ActionResult, CapabilitiesResult, FoundAppsResult, HoverResult, SystemPointerRequest, ScreenshotResult, ShowResult, WindowsResult } from './protocol.ts'
|
||||
import type { ActionResult, CapabilitiesResult, FoundAppsResult, HoverResult, SystemPointerRequest, ShowResult, WindowsResult } from './protocol.ts'
|
||||
|
||||
/** Helper-issued application selection; public app objects hold only this opaque handle. */
|
||||
export type CuaAppHandle = Branded<'CuaAppHandle'>
|
||||
@@ -15,29 +15,10 @@ export type CuaOperation = z.output<typeof SystemPointerRequest>
|
||||
type CuaParameters<M extends CuaOperation['method']> = Extract<CuaOperation, { method: M }>['params']
|
||||
/** Dispatch exactly once; a failed action must not be retried implicitly. */
|
||||
export type CuaDispatch = (operation: CuaOperation) => Promise<JsonValue>
|
||||
/** Encoded model-request image with the format, pixel dimensions and window reported by the Host. */
|
||||
export type CuaScreenshot = Pick<z.output<typeof ScreenshotResult>, 'mimeType' | 'pixelWidth' | 'pixelHeight' | 'windowId'> & {
|
||||
bytes: Uint8Array
|
||||
}
|
||||
/** Optional output owned by the caller; persistence and image admission are external. */
|
||||
export interface CuaObservationSink {
|
||||
/**
|
||||
* Emit text.
|
||||
* @param value - observed application state.
|
||||
*/
|
||||
text(value: string): void | Promise<void>
|
||||
/**
|
||||
* Emit an image.
|
||||
* @param screenshot - encoded image and its validated format, dimensions and window.
|
||||
*/
|
||||
image(screenshot: CuaScreenshot): void | Promise<void>
|
||||
}
|
||||
/** Observations emit by default. Returned values are unaffected by this option. */
|
||||
export interface CuaObservationOptions { emit?: boolean }
|
||||
/** Stored screenshot returned to a program for explicit inspection. */
|
||||
export type CuaProgramImage = { readonly attachmentId: string; readonly path: string; readonly width: number; readonly height: number }
|
||||
/** Full observations of the requested content; each read replaces the application's current snapshot. */
|
||||
export interface CuaAxOptions extends CuaObservationOptions {
|
||||
/** Reserved compatibility option; observations are never diffs. */
|
||||
disableDiffing?: boolean
|
||||
export interface CuaAxOptions {
|
||||
/** Expand menu contents. The default shows menu-bar entries only. */
|
||||
includeMenus?: boolean
|
||||
/** Include identifiers, global native bounds and complete action names. */
|
||||
@@ -107,7 +88,7 @@ export interface CuaSurface {
|
||||
/**
|
||||
* Read the full accessibility tree, with menu contents collapsed by default.
|
||||
* The returned tree becomes the application's current snapshot; use element ids from its latest successful read.
|
||||
* @param options - Output, traversal limits, menu expansion and field detail options.
|
||||
* @param options - Traversal limits, menu expansion and field detail options.
|
||||
* @returns Accessibility tree text with element ids, up to the limits for this read.
|
||||
*/
|
||||
getAccessibilityTree(options?: CuaAxOptions): Promise<string>
|
||||
@@ -190,10 +171,9 @@ export interface CuaWindow extends CuaSurface {
|
||||
readonly windowId: number
|
||||
/**
|
||||
* Capture the selected window.
|
||||
* @param options - emission options.
|
||||
* @returns encoded model-request image with its format, pixel dimensions and window; use mimeType when saving or declaring the image.
|
||||
* @returns Stored Image with a readable path and pixel dimensions, ready for explicit inspection.
|
||||
*/
|
||||
getScreenshot(options?: CuaObservationOptions): Promise<CuaScreenshot>
|
||||
getScreenshot(): Promise<CuaProgramImage>
|
||||
/**
|
||||
* Request a mouse move to an element or a point without pressing a button.
|
||||
* The provider reports the route and whether the system pointer moved.
|
||||
@@ -239,10 +219,9 @@ export interface CuaApp extends CuaSurface {
|
||||
activate(): Promise<CuaActionResult>
|
||||
/**
|
||||
* Enumerate current windows.
|
||||
* @param options - output options.
|
||||
* @returns current window records.
|
||||
*/
|
||||
getWindows(options?: CuaObservationOptions): Promise<CuaWindowInfo[]>
|
||||
getWindows(): Promise<CuaWindowInfo[]>
|
||||
/**
|
||||
* Select a specific window.
|
||||
* @param windowId - ID obtained from getWindows.
|
||||
@@ -255,29 +234,25 @@ export interface CuaApp extends CuaSurface {
|
||||
export interface CuaApi {
|
||||
/**
|
||||
* Inspect permissions without requesting them.
|
||||
* @param options - output options.
|
||||
* @returns reported native capabilities.
|
||||
*/
|
||||
getCapabilities(options?: CuaObservationOptions): Promise<CuaCapabilities>
|
||||
getCapabilities(): Promise<CuaCapabilities>
|
||||
/**
|
||||
* Enumerate running applications.
|
||||
* @param options - output options.
|
||||
* @returns identifiers accepted by getApp.
|
||||
*/
|
||||
listApps(options?: CuaObservationOptions): Promise<CuaAppInfo[]>
|
||||
listApps(): Promise<CuaAppInfo[]>
|
||||
/**
|
||||
* Search the provider's installed application locations without launching an app.
|
||||
* @param query - Nonempty name, provider-specific application identifier or absolute application path.
|
||||
* @param options - Output options.
|
||||
* @returns Matching installed applications; overly broad queries reject instead of silently truncating.
|
||||
*/
|
||||
findApps(query: string, options?: CuaObservationOptions): Promise<CuaFoundAppInfo[]>
|
||||
findApps(query: string): Promise<CuaFoundAppInfo[]>
|
||||
/**
|
||||
* Discover apps.
|
||||
* @param options - output options.
|
||||
* @returns native apps and an empty browser list.
|
||||
*/
|
||||
getState(options?: CuaObservationOptions): Promise<{ apps: CuaAppInfo[]; browsers: [] }>
|
||||
getState(): Promise<{ apps: CuaAppInfo[]; browsers: [] }>
|
||||
/**
|
||||
* Select an app, obtain consent, and launch it in the background if necessary, without reading AX.
|
||||
* @param selector - Exact name, bundle ID, absolute application path or running PID string; ambiguous names reject.
|
||||
|
||||
@@ -400,5 +400,5 @@ it.each([
|
||||
expect(prompt?.includes('Persistent PTC initialization:')).toBe(runtime.retainsState)
|
||||
expect(prompt?.includes('use the host-provided bridge initialization once and retain the returned objects')).toBe(runtime.retainsState)
|
||||
expect(prompt?.includes('cua comes from that bridge and is not a preinstalled global')).toBe(runtime.retainsState)
|
||||
expect(prompt?.includes('var cua = connectCua(() => tools.cua, text => console.log(text));')).toBe(runtime.retainsState)
|
||||
expect(prompt?.includes('var cua = connectCua(() => tools.cua);')).toBe(runtime.retainsState)
|
||||
})
|
||||
@@ -1,15 +1,14 @@
|
||||
/** App and window behavior over a fixture operation dispatcher, without desktop access. */
|
||||
import { Buffer } from 'node:buffer'
|
||||
import { describe, expect, expectTypeOf, it, vi } from 'vitest'
|
||||
import { brandString } from '@deepseek-ai/dsh-brand'
|
||||
import type { JsonValue } from '@deepseek-ai/dsh-util-values'
|
||||
import { createCuaApi } from '../src/api.ts'
|
||||
import type { CuaApp, CuaCapabilities, CuaCoordinateOptions, CuaElementId, CuaOperation, CuaScreenshot, CuaWindow } from '../src/types.ts'
|
||||
import type { CuaApp, CuaCapabilities, CuaCoordinateOptions, CuaElementId, CuaOperation, CuaWindow } from '../src/types.ts'
|
||||
|
||||
/** Element ids are opaque strings; these fixtures use the macOS decimal form. */
|
||||
const element = (id: string): CuaElementId => brandString<CuaElementId>(id)
|
||||
|
||||
const PNG = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAADElEQVQImWNgZGIGAAAOAAeCcsnOAAAAAElFTkSuQmCC'
|
||||
const IMAGE = Object.freeze({ attachmentId: 'sha256:screenshot', path: '/images/screenshot.png', width: 1, height: 1 })
|
||||
const bounds = { x: -640, y: 100, width: 640, height: 480 }
|
||||
const selected = { appHandle: 'app-17', pid: 17, name: 'Editor', bundleId: 'test.editor', running: true }
|
||||
const delivered = { dispatched: true }
|
||||
@@ -19,11 +18,10 @@ function windowInfo(windowId: number) {
|
||||
return { windowId, appName: 'Editor', title: `Document ${windowId}`, bounds, isOnScreen: true, onCurrentSpace: null }
|
||||
}
|
||||
|
||||
function fixture(withSink = true) {
|
||||
function fixture() {
|
||||
let windows = [windowInfo(23)]
|
||||
let currentCapabilities: CuaCapabilities = capabilities
|
||||
let generation = 0
|
||||
const sink = { text: vi.fn<(value: string) => void>(), image: vi.fn<(screenshot: CuaScreenshot) => void>() }
|
||||
const dispatch = vi.fn(async ({ method, params }: CuaOperation): Promise<JsonValue> => {
|
||||
switch (method) {
|
||||
case 'getCapabilities': return currentCapabilities
|
||||
@@ -37,13 +35,13 @@ function fixture(withSink = true) {
|
||||
snapshotId: `snapshot-${++generation}`, tree: `Tree ${generation}`, truncated: false,
|
||||
windows: params.windowId === undefined ? windows : windows.filter(window => window.windowId === params.windowId),
|
||||
}
|
||||
case 'screenshot': return { mimeType: 'image/png', data: PNG, pixelWidth: 1, pixelHeight: 1, windowId: params.windowId, bounds, coordinateSpace: 'screenshot-pixels' }
|
||||
case 'screenshot': return IMAGE
|
||||
case 'listActions': return ['AXPress', 'AXShowMenu', 'AXTextArea.AXValue.Set']
|
||||
default: return delivered
|
||||
}
|
||||
})
|
||||
return {
|
||||
dispatch, sink, api: createCuaApi(dispatch, withSink ? sink : undefined),
|
||||
dispatch, api: createCuaApi(dispatch),
|
||||
setWindows: (ids: number[]) => { windows = ids.map(windowInfo) },
|
||||
setWindowRecords: (records: ReturnType<typeof windowInfo>[]) => { windows = records },
|
||||
setCapabilities: (value: CuaCapabilities) => { currentCapabilities = value },
|
||||
@@ -54,10 +52,6 @@ function state(overrides: Record<string, JsonValue> = {}): JsonValue {
|
||||
return { appHandle: selected.appHandle, windowId: null, snapshotId: 'changed', tree: 'Changed tree', windows: [windowInfo(23)], truncated: false, ...overrides }
|
||||
}
|
||||
|
||||
function screenshot(overrides: Record<string, JsonValue> = {}): JsonValue {
|
||||
return { mimeType: 'image/png', data: PNG, pixelWidth: 1, pixelHeight: 1, windowId: 23, bounds, coordinateSpace: 'screenshot-pixels', ...overrides }
|
||||
}
|
||||
|
||||
describe('application discovery', () => {
|
||||
it('reports permissions and discovers both bundle and PID selectors without activating an app', async () => {
|
||||
const f = fixture()
|
||||
@@ -67,27 +61,22 @@ describe('application discovery', () => {
|
||||
{ id: '18', displayName: 'Utility', isRunning: true, pid: 18 },
|
||||
], browsers: [] })
|
||||
expect(f.dispatch.mock.calls.map(([operation]) => operation.method)).toEqual(['getCapabilities', 'listApps'])
|
||||
const emitted = f.sink.text.mock.calls.map(([text]): unknown => JSON.parse(text))
|
||||
expect(emitted).toEqual([capabilities, await f.api.getState({ emit: false })])
|
||||
expect(f.sink.text).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('returns observations without requiring an output sink', async () => {
|
||||
const f = fixture(false)
|
||||
expect(await f.api.getCapabilities({ emit: false })).toEqual(capabilities)
|
||||
it('returns app handles and window records for the selected application', async () => {
|
||||
const f = fixture()
|
||||
expect(await f.api.getCapabilities()).toEqual(capabilities)
|
||||
expect(await f.api.listApps()).toHaveLength(2)
|
||||
const app = await f.api.getApp('Editor')
|
||||
expect(app).not.toHaveProperty('pid')
|
||||
expectTypeOf<CuaApp>().not.toHaveProperty('pid')
|
||||
expect(app.appHandle).toBe('app-17')
|
||||
const window = await app.getWindow(23)
|
||||
expect(await window.getScreenshot()).toEqual({ bytes: Uint8Array.from(Buffer.from(PNG, 'base64')), mimeType: 'image/png', pixelWidth: 1, pixelHeight: 1, windowId: 23 })
|
||||
expect(await window.getScreenshot()).toEqual(IMAGE)
|
||||
expect(f.dispatch.mock.calls.map(([operation]) => operation.method)).toEqual([
|
||||
'getCapabilities', 'listApps', 'getApp', 'screenshot',
|
||||
])
|
||||
expect(await app.getWindows()).toEqual([windowInfo(23)])
|
||||
expect(f.sink.text).not.toHaveBeenCalled()
|
||||
expect(f.sink.image).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('searches installed applications without turning ordinary discovery into an installed-app inventory', async () => {
|
||||
@@ -99,7 +88,6 @@ describe('application discovery', () => {
|
||||
f.dispatch.mockResolvedValueOnce(matches)
|
||||
expect(await f.api.findApps('Editor')).toEqual(matches)
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'findApps', params: { query: 'Editor' } })
|
||||
expect(f.sink.text).toHaveBeenLastCalledWith(JSON.stringify(matches))
|
||||
await expect(f.api.findApps(' ')).rejects.toThrow()
|
||||
})
|
||||
|
||||
@@ -117,7 +105,6 @@ describe('application discovery', () => {
|
||||
await expect(f.api.listApps()).rejects.toThrow()
|
||||
f.dispatch.mockResolvedValueOnce({ ...selected, appHandle: '' })
|
||||
await expect(f.api.getApp('Editor')).rejects.toThrow()
|
||||
expect(f.sink.text).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -160,13 +147,13 @@ describe('mouseMove across observations', () => {
|
||||
const f = fixture()
|
||||
f.setWindows([23, 24])
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
expect(await window.mouseMove(element('0'))).toEqual({ ...delivered, hoverRoute: 'accessibility', systemPointerMoved: false })
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getScreenshot({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
await window.getScreenshot()
|
||||
expect(await window.mouseMove([30, 40])).toEqual({ ...delivered, hoverRoute: 'accessibility', systemPointerMoved: false })
|
||||
expect(f.dispatch.mock.calls.map(([operation]) => operation)).toEqual([
|
||||
{ method: 'mouseMove', params: { appHandle: 'app-17', snapshotId: 'snapshot-2', elementId: '0', windowId: 23 } },
|
||||
@@ -175,7 +162,7 @@ describe('mouseMove across observations', () => {
|
||||
{ method: 'mouseMove', params: { appHandle: 'app-17', snapshotId: 'snapshot-3', windowId: 23, x: 30, y: 40 } },
|
||||
])
|
||||
const other = await app.getWindow(24)
|
||||
await other.getAccessibilityTree({ emit: false })
|
||||
await other.getAccessibilityTree()
|
||||
await other.mouseMove([5, 6])
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'mouseMove', params: {
|
||||
appHandle: 'app-17', snapshotId: 'snapshot-4', windowId: 24, x: 5, y: 6,
|
||||
@@ -190,9 +177,9 @@ describe('pointer buttons and modifiers', () => {
|
||||
it.each([0, 800])('preserves %d ms for window clicks at elements and coordinates', async (holdMs) => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
for (const target of [element('0'), [10, 20] as const]) {
|
||||
await window.click(target, { holdMs, mouseButton: 'right' })
|
||||
expect(f.dispatch.mock.calls.at(-1)?.[0]).toMatchObject({ method: 'click', params: { holdMs, mouseButton: 'right' } })
|
||||
@@ -205,9 +192,9 @@ describe('pointer buttons and modifiers', () => {
|
||||
it('retains middle-button and modifier intent through click and drag dispatch', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
await window.click(element('0'), { mouseButton: 'middle', modifiers: ['shift'] })
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'click', params: {
|
||||
appHandle: 'app-17', snapshotId: 'snapshot-2', elementId: '0', windowId: 23, mouseButton: 'middle', modifiers: ['shift'],
|
||||
@@ -232,7 +219,7 @@ describe('text selections', () => {
|
||||
it('binds literal text and optional disambiguation to the latest observed control', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
await app.selectText(element('1'), 'target')
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'selectText', params: {
|
||||
appHandle: 'app-17', snapshotId: 'snapshot-1', elementId: '1', text: 'target',
|
||||
@@ -252,7 +239,7 @@ describe('element action queries', () => {
|
||||
it('returns raw provider action names including attribute-setting operations without reading new state', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
expect(await app.listActions(element('42.12345'))).toEqual(['AXPress', 'AXShowMenu', 'AXTextArea.AXValue.Set'])
|
||||
expect(f.dispatch.mock.calls.map(([operation]) => operation)).toEqual([
|
||||
@@ -260,7 +247,6 @@ describe('element action queries', () => {
|
||||
])
|
||||
await expect(app.listActions(element(''))).rejects.toThrow()
|
||||
expect(f.dispatch).toHaveBeenCalledTimes(1)
|
||||
expect(f.sink.text).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -268,7 +254,7 @@ describe('element actions', () => {
|
||||
it('dispatches advertised actions with omitted, null or direct values without wrapping the value', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
expect(await app.performAction(element('42.12345'), 'AXPress')).toEqual(delivered)
|
||||
expect(await app.performAction(element('42.12345'), 'AXPress', null)).toEqual(delivered)
|
||||
@@ -280,13 +266,12 @@ describe('element actions', () => {
|
||||
{ method: 'performAction', params: { appHandle: 'app-17', snapshotId: 'snapshot-1', elementId: '42.12345', action: 'AXTextArea.AXValue.Set', argument: 'hello' } },
|
||||
{ method: 'performAction', params: { appHandle: 'app-17', snapshotId: 'snapshot-1', elementId: '42.12345', action: 'AXWindow.AXPosition.Set', argument: { x: 10, y: 20 } } },
|
||||
])
|
||||
expect(f.sink.text).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('rejects an empty action before dispatch', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
await expect(app.performAction(element('0'), '')).rejects.toThrow('performAction action')
|
||||
expect(f.dispatch).not.toHaveBeenCalled()
|
||||
@@ -297,19 +282,18 @@ describe('element default invocation', () => {
|
||||
it('uses the latest snapshot and opaque element id without reading fresh state', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
expect(await app.invoke(element('42.12345'))).toEqual(delivered)
|
||||
expect(f.dispatch.mock.calls.map(([operation]) => operation)).toEqual([
|
||||
{ method: 'invoke', params: { appHandle: 'app-17', snapshotId: 'snapshot-1', elementId: '42.12345' } },
|
||||
])
|
||||
expect(f.sink.text).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns confirmed absence of a default action as un-dispatched evidence', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
f.dispatch.mockResolvedValueOnce({ dispatched: false })
|
||||
await expect(app.invoke(element('9'))).resolves.toEqual({ dispatched: false })
|
||||
})
|
||||
@@ -317,7 +301,7 @@ describe('element default invocation', () => {
|
||||
it('rejects provider failures and malformed dispatch evidence', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const failure = new Error('PERMISSION_DENIED')
|
||||
f.dispatch.mockRejectedValueOnce(failure)
|
||||
await expect(app.invoke(element('0'))).rejects.toBe(failure)
|
||||
@@ -332,12 +316,12 @@ describe('application and window observations', () => {
|
||||
const f = fixture()
|
||||
f.setWindows([23, 24])
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
expect(await app.getWindows({ emit: false })).toEqual([windowInfo(23), windowInfo(24)])
|
||||
await app.getAccessibilityTree()
|
||||
expect(await app.getWindows()).toEqual([windowInfo(23), windowInfo(24)])
|
||||
const first = await app.getWindow(23)
|
||||
await first.getAccessibilityTree({ emit: false })
|
||||
await first.getAccessibilityTree()
|
||||
const second = await app.getWindow(24)
|
||||
await second.getAccessibilityTree({ emit: false })
|
||||
await second.getAccessibilityTree()
|
||||
await app.setValue(element('1'), 'First document')
|
||||
await first.click(element('2'))
|
||||
await second.click(element('3'))
|
||||
@@ -348,24 +332,22 @@ describe('application and window observations', () => {
|
||||
])
|
||||
expect(first.windowId).toBe(23)
|
||||
expect(second.windowId).toBe(24)
|
||||
expect(f.sink.text.mock.calls.map(([text]) => text)).toEqual([])
|
||||
await expect(second.getAccessibilityTree({ emit: false, disableDiffing: true })).resolves.toBe('Tree 4')
|
||||
await expect(second.getAccessibilityTree()).resolves.toBe('Tree 4')
|
||||
await second.click(element('3'))
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'click', params: { appHandle: 'app-17', snapshotId: 'snapshot-4', elementId: '3', windowId: 24 } })
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
await first.invoke(element('4'))
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'invoke', params: { appHandle: 'app-17', snapshotId: 'snapshot-5', elementId: '4', windowId: 23 } })
|
||||
expect(f.sink.text).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('keeps separate snapshots for different application handles', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
f.dispatch.mockResolvedValueOnce({ appHandle: 'app-18', name: 'Utility', running: true })
|
||||
f.dispatch.mockResolvedValueOnce(state({ appHandle: 'app-18', snapshotId: 'other', windows: [] }))
|
||||
const other = await f.api.getApp('Utility')
|
||||
await other.getAccessibilityTree({ emit: false })
|
||||
await other.getAccessibilityTree()
|
||||
await app.setValue(element('0'), 'Editor')
|
||||
await other.setValue(element('0'), 'Utility')
|
||||
expect(f.dispatch.mock.calls.slice(-2).map(([operation]) => operation)).toEqual([
|
||||
@@ -377,15 +359,15 @@ describe('application and window observations', () => {
|
||||
it('shares one current snapshot between repeated handles for the same application', async () => {
|
||||
const f = fixture()
|
||||
const first = await f.api.getApp('Editor')
|
||||
await first.getAccessibilityTree({ emit: false })
|
||||
await first.getAccessibilityTree()
|
||||
const second = await f.api.getApp('Editor')
|
||||
const window = await second.getWindow(23)
|
||||
await window.getScreenshot({ emit: false })
|
||||
await window.getScreenshot()
|
||||
await first.setValue(element('1'), 'Preserved')
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'setValue', params: {
|
||||
appHandle: 'app-17', snapshotId: 'snapshot-1', elementId: '1', value: 'Preserved',
|
||||
} })
|
||||
await second.getAccessibilityTree({ emit: false })
|
||||
await second.getAccessibilityTree()
|
||||
expect(second.appHandle).toBe(first.appHandle)
|
||||
await first.setValue(element('1'), 'Shared')
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'setValue', params: {
|
||||
@@ -396,11 +378,11 @@ describe('application and window observations', () => {
|
||||
it('exposes screenshots and pointer input only on explicit windows', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
expect(app).toHaveProperty('getAccessibilityTree')
|
||||
for (const method of ['getScreenshot', 'click', 'mouseMove', 'scroll', 'drag']) expect(app).not.toHaveProperty(method)
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
for (const method of ['getScreenshot', 'click', 'mouseMove', 'scroll', 'drag']) expect(window).toHaveProperty(method)
|
||||
expect(window).not.toHaveProperty('hover')
|
||||
expect(window).toHaveProperty('mouseMove')
|
||||
@@ -410,22 +392,22 @@ describe('application and window observations', () => {
|
||||
const f = fixture()
|
||||
f.setWindows([23, 24])
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const first = await app.getWindow(23)
|
||||
await first.getAccessibilityTree({ emit: false })
|
||||
await first.getAccessibilityTree()
|
||||
await first.click([10, 20])
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'click', params: {
|
||||
appHandle: 'app-17', windowId: 23, snapshotId: 'snapshot-2', x: 10, y: 20, mouseButton: 'left', clickCount: 1,
|
||||
} })
|
||||
const second = await app.getWindow(24)
|
||||
await second.getAccessibilityTree({ emit: false })
|
||||
await second.getAccessibilityTree()
|
||||
await first.click([10, 20])
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'click', params: {
|
||||
appHandle: 'app-17', windowId: 23, x: 10, y: 20, mouseButton: 'left', clickCount: 1,
|
||||
} })
|
||||
await second.click([10, 20])
|
||||
expect(f.dispatch.mock.calls.at(-1)?.[0]).toMatchObject({ params: { snapshotId: 'snapshot-3', windowId: 24 } })
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
await first.click([10, 20])
|
||||
expect(f.dispatch.mock.calls.at(-1)?.[0]).toMatchObject({ params: { snapshotId: 'snapshot-4', windowId: 23 } })
|
||||
})
|
||||
@@ -433,12 +415,11 @@ describe('application and window observations', () => {
|
||||
it('keeps a validated AXless window usable for screenshots and coordinate input', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
f.dispatch.mockResolvedValueOnce(state({ windowId: 23, snapshotId: 'axless', tree: '', elements: [] }))
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
expect(f.sink.text).not.toHaveBeenCalled()
|
||||
await expect(window.getScreenshot()).resolves.toEqual({ bytes: Uint8Array.from(Buffer.from(PNG, 'base64')), mimeType: 'image/png', pixelWidth: 1, pixelHeight: 1, windowId: 23 })
|
||||
await window.getAccessibilityTree()
|
||||
await expect(window.getScreenshot()).resolves.toEqual(IMAGE)
|
||||
await window.click([10, 20])
|
||||
await window.scroll({ x: 10, y: 20, deltaLinesX: 0, deltaLinesY: -1 })
|
||||
await window.drag([10, 20], [30, 40])
|
||||
@@ -469,10 +450,10 @@ describe('application and window observations', () => {
|
||||
]
|
||||
f.setWindowRecords(windows)
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
expect(await app.getWindows({ emit: false })).toEqual(windows)
|
||||
await app.getAccessibilityTree()
|
||||
expect(await app.getWindows()).toEqual(windows)
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
await window.getScreenshot()
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'screenshot', params: { appHandle: 'app-17', windowId: 23 } })
|
||||
await window.click([10, 20])
|
||||
@@ -480,7 +461,7 @@ describe('application and window observations', () => {
|
||||
appHandle: 'app-17', windowId: 23, snapshotId: 'snapshot-2', x: 10, y: 20, mouseButton: 'left', clickCount: 1,
|
||||
} })
|
||||
const hidden = await app.getWindow(25)
|
||||
await hidden.getAccessibilityTree({ emit: false })
|
||||
await hidden.getAccessibilityTree()
|
||||
await hidden.click([10, 20])
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'click', params: {
|
||||
appHandle: 'app-17', windowId: 25, snapshotId: 'snapshot-3', x: 10, y: 20, mouseButton: 'left', clickCount: 1,
|
||||
@@ -505,14 +486,13 @@ describe('application and window observations', () => {
|
||||
const app = await f.api.getApp('Editor')
|
||||
f.dispatch.mockResolvedValueOnce(state(identity))
|
||||
await expect(app.getAccessibilityTree()).rejects.toThrow('belongs to another application or window')
|
||||
expect(f.sink.text).not.toHaveBeenCalled()
|
||||
await expect(app.setValue(element('0'), 'stale')).rejects.toThrow('Read getAccessibilityTree')
|
||||
})
|
||||
|
||||
it('returns provider window records without process filtering', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const observed = windowInfo(24)
|
||||
f.dispatch.mockResolvedValueOnce(state({ windows: [observed] }))
|
||||
await expect(app.getAccessibilityTree()).resolves.toBe('Changed tree')
|
||||
@@ -538,7 +518,7 @@ describe('application and window observations', () => {
|
||||
{ method: 'typeText', params: { appHandle: 'app-17', text: 'keyboard without accessibility' } },
|
||||
])
|
||||
const window = await app.getWindow(23)
|
||||
await expect(window.getAccessibilityTree({ emit: false })).resolves.toBe('Tree 1')
|
||||
await expect(window.getAccessibilityTree()).resolves.toBe('Tree 1')
|
||||
await expect(app.getAccessibilityTree()).rejects.toThrow('Application-level accessibility is not supported on current platform; observe a window instead.')
|
||||
await expect(window.invoke(element('0'))).resolves.toEqual(delivered)
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'invoke', params: {
|
||||
@@ -550,9 +530,9 @@ describe('application and window observations', () => {
|
||||
const f = fixture()
|
||||
f.setCapabilities({ ...capabilities, keyboard: ['app', 'window'] })
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
await window.typeText('hello')
|
||||
await window.paste('<b>bold</b>', { format: 'html' })
|
||||
@@ -574,9 +554,9 @@ describe('application and window observations', () => {
|
||||
const f = fixture()
|
||||
f.setCapabilities({ ...capabilities, keyboard: ['window'] })
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
await expect(app.typeText('app text')).rejects.toThrow('App-scoped keyboard input is not supported on current platform.')
|
||||
await expect(app.paste('app text')).rejects.toThrow('App-scoped keyboard input is not supported on current platform.')
|
||||
@@ -590,9 +570,9 @@ describe('application and window observations', () => {
|
||||
it('accepts coordinate input without a current snapshot', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
f.dispatch.mockRejectedValueOnce(new Error('Native app is busy'))
|
||||
await expect(window.getAccessibilityTree()).rejects.toThrow('Native app is busy')
|
||||
await window.click([10, 20])
|
||||
@@ -604,23 +584,20 @@ describe('application and window observations', () => {
|
||||
it('clears a failed refresh and requires a new read before element operations', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
const failure = new Error('Native app is busy')
|
||||
f.dispatch.mockRejectedValueOnce(failure)
|
||||
await expect(app.getAccessibilityTree()).rejects.toBe(failure)
|
||||
await expect(window.setValue(element('0'), 'stale')).rejects.toThrow('Read getAccessibilityTree')
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
f.dispatch.mockRejectedValueOnce(failure)
|
||||
await expect(window.getAccessibilityTree()).rejects.toBe(failure)
|
||||
await expect(app.invoke(element('0'))).rejects.toThrow('Read getAccessibilityTree')
|
||||
f.sink.text.mockClear()
|
||||
const calls = f.dispatch.mock.calls.length
|
||||
await window.getScreenshot()
|
||||
expect(f.dispatch.mock.calls.slice(calls).map(([operation]) => operation.method)).toEqual(['screenshot'])
|
||||
expect(f.sink.text).not.toHaveBeenCalled()
|
||||
expect(f.sink.image).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -639,7 +616,6 @@ describe('screenshots', () => {
|
||||
const window = await app.getWindow(23)
|
||||
f.dispatch.mockRejectedValueOnce(new Error(code))
|
||||
await expect(window.getScreenshot()).rejects.toThrow(code)
|
||||
expect(f.sink.image).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it.each(['application', 'window'])('captures the first screenshot when %s AX reads fail', async (scope) => {
|
||||
@@ -651,11 +627,11 @@ describe('screenshots', () => {
|
||||
return dispatch(request)
|
||||
})
|
||||
const app = await f.api.getApp('Editor')
|
||||
expect(await app.getWindows({ emit: false })).toEqual([windowInfo(23)])
|
||||
expect(await app.getWindows()).toEqual([windowInfo(23)])
|
||||
const window = await app.getWindow(23)
|
||||
const target = scope === 'application' ? app : window
|
||||
await expect(target.getAccessibilityTree()).rejects.toBe(failure)
|
||||
await expect(window.getScreenshot()).resolves.toMatchObject({ windowId: 23, pixelWidth: 1 })
|
||||
await expect(window.getScreenshot()).resolves.toBe(IMAGE)
|
||||
await expect(window.invoke(element('0'))).rejects.toThrow('Read getAccessibilityTree')
|
||||
await window.click([0, 0])
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'click', params: {
|
||||
@@ -663,49 +639,34 @@ describe('screenshots', () => {
|
||||
} })
|
||||
})
|
||||
|
||||
it('returns screenshot metadata independently of input point bounds and supports selective output', async () => {
|
||||
it('returns the accessibility text and the stored screenshot Image', async () => {
|
||||
const f = fixture()
|
||||
f.setWindows([23, 24])
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
const window = await app.getWindow(24)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
f.sink.text.mockClear()
|
||||
const tree = await window.getAccessibilityTree()
|
||||
const image = await window.getScreenshot()
|
||||
expect(tree).toBe('Tree 3')
|
||||
expect(image).toEqual({ bytes: Uint8Array.from(Buffer.from(PNG, 'base64')), mimeType: 'image/png', pixelWidth: 1, pixelHeight: 1, windowId: 24 })
|
||||
expect(tree).toBe('Tree 1')
|
||||
expect(image).toBe(IMAGE)
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'screenshot', params: { appHandle: 'app-17', windowId: 24 } })
|
||||
expect(f.sink.text).toHaveBeenCalledWith('Tree 3')
|
||||
expect(f.sink.image).toHaveBeenCalledWith(image)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getScreenshot({ emit: false })
|
||||
expect(f.sink.text).toHaveBeenCalledTimes(1)
|
||||
expect(f.sink.image).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('rejects a screenshot from another window before emitting its image', async () => {
|
||||
it.each([null, 'text only', {}, { path: '/image.png', width: 1 }, { path: '/image.png', width: 1, height: 1 }])('rejects an invalid screenshot result %j', async (value) => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
f.dispatch.mockResolvedValueOnce(screenshot({ windowId: 24 }))
|
||||
await expect(window.getScreenshot()).rejects.toThrow('belongs to another window')
|
||||
expect(f.sink.image).not.toHaveBeenCalled()
|
||||
f.dispatch.mockResolvedValueOnce(value)
|
||||
await expect(window.getScreenshot()).rejects.toThrow('must return an Image')
|
||||
})
|
||||
|
||||
it.each(['image/jpeg', 'image/webp', 'image/gif'])('preserves Host-admitted %s bytes and metadata in the return value and image sink', async (mimeType) => {
|
||||
it.each(['jpeg', 'webp', 'gif'])('retains the Host Image for a %s screenshot', async (format) => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
const data = Buffer.from('admitted request image').toString('base64')
|
||||
f.dispatch.mockResolvedValueOnce(screenshot({ mimeType, data, pixelWidth: 320, pixelHeight: 240 }))
|
||||
const image = await window.getScreenshot()
|
||||
expect(image).toEqual({ bytes: Uint8Array.from(Buffer.from(data, 'base64')), mimeType, pixelWidth: 320, pixelHeight: 240, windowId: 23 })
|
||||
expect(f.sink.image).toHaveBeenCalledExactlyOnceWith(image)
|
||||
const image = Object.freeze({ ...IMAGE, path: '/images/screenshot.' + format, width: 320, height: 240 })
|
||||
f.dispatch.mockResolvedValueOnce(image)
|
||||
expect(await window.getScreenshot()).toBe(image)
|
||||
})
|
||||
|
||||
})
|
||||
@@ -714,9 +675,9 @@ describe('input actions', () => {
|
||||
it('passes explicit relative mode to every coordinate gesture and rejects it on element ids', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
await window.click([500, 250], { relative: true })
|
||||
await window.mouseMove([500, 250], { relative: true })
|
||||
await window.drag([0, 0], [999, 999], { relative: true })
|
||||
@@ -734,9 +695,9 @@ describe('input actions', () => {
|
||||
it('keeps the selected window when scroll options include another target identity', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
const options = {
|
||||
x: 10, y: 20, deltaLinesX: 0, deltaLinesY: 1,
|
||||
appHandle: 'another-app', snapshotId: 'another-snapshot', windowId: 24,
|
||||
@@ -752,9 +713,9 @@ describe('input actions', () => {
|
||||
it('dispatches window-point gestures and reports delivery without claiming the action succeeded', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
expect(await window.click([10, 20], { mouseButton: 'right' })).toEqual(delivered)
|
||||
expect(await window.click([11, 21], { clickCount: 2 })).toEqual(delivered)
|
||||
expect(await window.scroll({ x: 10, y: 20, deltaLinesX: -2, deltaLinesY: 3 })).toEqual(delivered)
|
||||
@@ -792,9 +753,9 @@ describe('input actions', () => {
|
||||
])('reports a readable validation failure: %s', async (message, run) => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
await expect(run({ app, window })).rejects.toMatchObject({ name: 'Error', message })
|
||||
expect(f.dispatch).not.toHaveBeenCalled()
|
||||
@@ -803,9 +764,9 @@ describe('input actions', () => {
|
||||
it.each(invalidActions)('rejects %s without invoking native input', async (_label, run) => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
const calls = f.dispatch.mock.calls.length
|
||||
await expect(run({ app, window })).rejects.toThrow()
|
||||
expect(f.dispatch).toHaveBeenCalledTimes(calls)
|
||||
@@ -851,7 +812,7 @@ describe('input actions', () => {
|
||||
it('passes literal text and keys to the selected app identity', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
await app.typeText('中文\nline two')
|
||||
await app.paste('粘贴\nplain text')
|
||||
await app.paste('<b>bold</b>', { format: 'html' })
|
||||
@@ -869,9 +830,9 @@ describe('input actions', () => {
|
||||
it('rejects keyboard input on a window surface without dispatching an app-level fallback', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
for (const run of [
|
||||
() => window.typeText('hello'),
|
||||
@@ -893,7 +854,7 @@ describe('input actions', () => {
|
||||
const f = fixture()
|
||||
f.setCapabilities({ ...capabilities, keyboard: ['app', 'window'] })
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const surface = scope === 'window' ? await app.getWindow(23) : app
|
||||
f.dispatch.mockClear()
|
||||
f.dispatch.mockResolvedValueOnce(delivered)
|
||||
@@ -907,7 +868,7 @@ describe('input actions', () => {
|
||||
it('returns an un-dispatched key-event result from one RPC without retrying', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
f.dispatch.mockClear()
|
||||
f.dispatch.mockResolvedValueOnce({ dispatched: false })
|
||||
await expect(app.sendKeyEvent('Command', 'down')).resolves.toEqual({ dispatched: false })
|
||||
@@ -937,9 +898,9 @@ describe('input actions', () => {
|
||||
it('retains the snapshot identity for element right and double clicks', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
await window.click(element('12'), { mouseButton: 'right' })
|
||||
await window.click(element('13'), { clickCount: 2 })
|
||||
expect(f.dispatch.mock.calls.slice(-2).map(([operation]) => operation)).toEqual([
|
||||
@@ -951,9 +912,9 @@ describe('input actions', () => {
|
||||
it('dispatches an opaque provider element id unchanged', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
await window.click(element('42.12345'))
|
||||
await window.mouseMove(element('42.12345'))
|
||||
expect(f.dispatch.mock.calls.slice(-2).map(([operation]) => operation)).toEqual([
|
||||
@@ -973,9 +934,9 @@ describe('input actions', () => {
|
||||
it('propagates native failures once and rejects malformed delivery claims', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
const failure = new Error('Application was restarted; select it again')
|
||||
f.dispatch.mockRejectedValueOnce(failure)
|
||||
await expect(app.typeText('hello')).rejects.toBe(failure)
|
||||
@@ -990,7 +951,7 @@ describe('input actions', () => {
|
||||
await expect(f.api.getApp('')).rejects.toThrow()
|
||||
expect(f.dispatch).not.toHaveBeenCalled()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ emit: false })
|
||||
await app.getAccessibilityTree()
|
||||
const calls = f.dispatch.mock.calls.length
|
||||
await expect(app.getWindow(0)).rejects.toThrow()
|
||||
expect(f.dispatch).toHaveBeenCalledTimes(calls)
|
||||
@@ -1000,7 +961,7 @@ describe('input actions', () => {
|
||||
it('expands menu contents and details only for the requested observation', async () => {
|
||||
const f = fixture()
|
||||
const app = await f.api.getApp('Editor')
|
||||
await app.getAccessibilityTree({ includeMenus: true, details: true, emit: false })
|
||||
await app.getAccessibilityTree({ includeMenus: true, details: true })
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'getState', params: { appHandle: selected.appHandle, includeMenus: true, details: true } })
|
||||
await app.getAccessibilityTree({ includeMenus: false, details: false })
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'getState', params: { appHandle: selected.appHandle, includeMenus: false, details: false } })
|
||||
@@ -1009,14 +970,14 @@ it('expands menu contents and details only for the requested observation', async
|
||||
it('forwards provider options while retaining the selected target and coordinate mode', async () => {
|
||||
const f = fixture()
|
||||
const win = await (await f.api.getApp('Editor')).getWindow(23)
|
||||
await win.getAccessibilityTree({ emit: false })
|
||||
await win.getAccessibilityTree()
|
||||
const defaultCoordinates: CuaCoordinateOptions = {}
|
||||
const elementOptions = { ...defaultCoordinates, trackingMode: 'precise', appHandle: 'another-app', windowId: 99, unused: undefined }
|
||||
await win.mouseMove(element('0'), elementOptions)
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'mouseMove', params: {
|
||||
appHandle: 'app-17', windowId: 23, snapshotId: 'snapshot-1', elementId: '0', trackingMode: 'precise',
|
||||
} })
|
||||
await win.getScreenshot({ emit: false })
|
||||
await win.getScreenshot()
|
||||
const pixelOptions = { relative: false, useSystemPointer: true, trackingMode: 'precise', x: 222 }
|
||||
await win.mouseMove([30, 40], pixelOptions)
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'mouseMove', params: {
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
/** Retained app objects use fresh execution bindings while tool results own image admission. */
|
||||
import { Buffer } from 'node:buffer'
|
||||
import { expect, it, vi } from 'vitest'
|
||||
import { brandString } from '@deepseek-ai/dsh-brand'
|
||||
import type { JsonValue } from '@deepseek-ai/dsh-util-values'
|
||||
@@ -10,7 +9,6 @@ import type { CuaCapabilities, CuaElementId, CuaOperation } from '../src/types.t
|
||||
/** Element ids are opaque strings; these fixtures use the macOS decimal form. */
|
||||
const element = (id: string): CuaElementId => brandString<CuaElementId>(id)
|
||||
|
||||
const PNG = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAADElEQVQImWNgZGIGAAAOAAeCcsnOAAAAAElFTkSuQmCC'
|
||||
const capabilities: CuaCapabilities = {
|
||||
platform: 'macos', systemPointer: true, permissions: { accessibility: true, screenRecording: true }, appLevelAccessibility: true,
|
||||
keyboard: ['app'], sessionLocked: false, methods: ['getCapabilities', 'getApp', 'getState', 'screenshot'],
|
||||
@@ -31,21 +29,18 @@ it('uses the current tool binding when a retained app is called by another execu
|
||||
return snapshot
|
||||
})
|
||||
let current: CuaToolBinding = first
|
||||
const text = vi.fn()
|
||||
const cua = connectCua(() => current, text)
|
||||
const cua = connectCua(() => current)
|
||||
const app = await cua.getApp('Editor')
|
||||
const window = await app.getWindow(23)
|
||||
await window.getAccessibilityTree({ emit: false })
|
||||
await window.getAccessibilityTree()
|
||||
const second = vi.fn(async (): Promise<JsonValue> => ({ dispatched: true }))
|
||||
current = second
|
||||
|
||||
await expect(window.click(element('0'))).resolves.toEqual({ dispatched: true })
|
||||
expect(first).toHaveBeenCalledTimes(3)
|
||||
expect(second).toHaveBeenCalledWith({ method: 'click', params: { appHandle: 'app-17', snapshotId: 'snapshot-1', elementId: '0', windowId: 23 } })
|
||||
expect(text).not.toHaveBeenCalled()
|
||||
current = first
|
||||
await app.getAccessibilityTree()
|
||||
expect(text).toHaveBeenCalledWith('0 button')
|
||||
await expect(app.getAccessibilityTree()).resolves.toBe('0 button')
|
||||
})
|
||||
|
||||
it('propagates a denied action without retrying it', async () => {
|
||||
@@ -56,7 +51,7 @@ it('propagates a denied action without retrying it', async () => {
|
||||
expect(tool).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('supports callers that retain observations without a text callback', async () => {
|
||||
it('returns normalized application records', async () => {
|
||||
const cua = connectCua(() => async () => [{ pid: 17, name: 'Editor' }])
|
||||
await expect(cua.listApps()).resolves.toEqual([{ id: '17', displayName: 'Editor', isRunning: true, pid: 17 }])
|
||||
})
|
||||
@@ -66,21 +61,17 @@ it.each([null, 'text only', { content: [{ type: 'text', text: '{}' }] }])('rejec
|
||||
await expect(cua.getState()).rejects.toThrow()
|
||||
})
|
||||
|
||||
it('returns screenshot bytes and metadata with one image-bearing tool call and no image text output', async () => {
|
||||
it('returns the program Image unchanged from one screenshot tool call', async () => {
|
||||
const image = Object.freeze({ attachmentId: 'sha256:screenshot', path: '/images/screenshot.png', width: 1, height: 1 })
|
||||
const tool = vi.fn(async (operation: CuaOperation): Promise<JsonValue> => {
|
||||
if (operation.method === 'getCapabilities') return capabilities
|
||||
if (operation.method === 'getApp') return selected
|
||||
if (operation.method === 'getState') return { ...snapshot, windowId: operation.params.windowId ?? null }
|
||||
return {
|
||||
mimeType: 'image/png', data: PNG, pixelWidth: 1, pixelHeight: 1, windowId: 23, bounds, coordinateSpace: 'screenshot-pixels',
|
||||
}
|
||||
return image
|
||||
})
|
||||
const text = vi.fn()
|
||||
const app = await connectCua(() => tool, text).getApp('Editor')
|
||||
const app = await connectCua(() => tool).getApp('Editor')
|
||||
const window = await app.getWindow(23)
|
||||
text.mockClear()
|
||||
|
||||
expect(await window.getScreenshot()).toEqual({ bytes: Uint8Array.from(Buffer.from(PNG, 'base64')), mimeType: 'image/png', pixelWidth: 1, pixelHeight: 1, windowId: 23 })
|
||||
expect(await window.getScreenshot()).toBe(image)
|
||||
expect(tool.mock.calls.map(([operation]) => operation.method)).toEqual(['getCapabilities', 'getApp', 'screenshot'])
|
||||
expect(text).not.toHaveBeenCalled()
|
||||
})
|
||||
@@ -32,8 +32,8 @@ import { CuaConfig, installCua } from '../src/index.ts'
|
||||
import { Request } from '../src/protocol.ts'
|
||||
|
||||
const programs = [
|
||||
'globalThis.visits = 1; var initialCua = cua; var app = await cua.getApp("test.fixture"); var win = await app.getWindow(7); await win.getAccessibilityTree({emit:false}); await win.click("0"); await win.getScreenshot(); ({ visits, bindings: Object.keys(tools), app: app.appHandle, window: win.windowId, pid: process.pid })',
|
||||
'globalThis.visits++; await win.getAccessibilityTree({emit:false}); await win.click("0"); ({ visits, sameCua: cua === initialCua, app: app.appHandle, window: win.windowId, tree: await win.getAccessibilityTree({emit:false}), pid: process.pid })',
|
||||
'globalThis.visits = 1; var initialCua = cua; var app = await cua.getApp("test.fixture"); var win = await app.getWindow(7); await win.getAccessibilityTree(); await win.click("0"); var screenshot = await win.getScreenshot(); inspect(JSON.stringify({ visits, bindings: Object.keys(tools), app: app.appHandle, window: win.windowId, pid: process.pid })); inspect(screenshot)',
|
||||
'globalThis.visits++; await win.getAccessibilityTree(); await win.click("0"); await win.getScreenshot(); inspect(JSON.stringify({ visits, sameCua: cua === initialCua, app: app.appHandle, window: win.windowId, tree: await win.getAccessibilityTree(), pid: process.pid }))',
|
||||
'await win.click("0")',
|
||||
]
|
||||
|
||||
@@ -164,12 +164,13 @@ it('retains CUA objects, guards and logs hidden subcalls, forwards screenshots,
|
||||
expect(dispatches.map(event => event.data.subCallId)).toEqual(guarded)
|
||||
expect(dispatches.every(event => event.data.name === 'cua')).toBe(true)
|
||||
expect(dispatches.filter(event => event.data.rootCallId === 'cua-program-2')
|
||||
.map(event => Request.parse(event.data.arguments).method)).toEqual(['getState', 'click', 'getState'])
|
||||
.map(event => Request.parse(event.data.arguments).method)).toEqual(['getState', 'click', 'screenshot', 'getState'])
|
||||
expect(dispatches.at(-1)?.data).toMatchObject({ rootCallId: 'cua-program-3', isError: true,
|
||||
error: { name: 'FixtureGuardError', code: 'FIXTURE_CLICK_DENIED' } })
|
||||
expect(native.calls.map(call => call.id)).toEqual(guarded.slice(0, -1))
|
||||
const images = model.requests[1]!.messages.flatMap(message => message.content.filter(block => block.type === 'image'))
|
||||
expect(images).toHaveLength(1)
|
||||
expect(model.requests[2]!.messages.flatMap(message => message.content.filter(block => block.type === 'image'))).toHaveLength(1)
|
||||
const image = images[0]!
|
||||
expect(image.attachment).toMatchObject({ mediaType: 'image/png', width: 1, height: 1 })
|
||||
expect(Buffer.from((await ctx.attachments.readImage(image.attachment)).data).toString('base64')).toBe(png)
|
||||
@@ -185,9 +186,9 @@ it('retains CUA objects, guards and logs hidden subcalls, forwards screenshots,
|
||||
|
||||
it('recreates the automatic facade when its environment is rebuilt', async () => {
|
||||
const { agent, run } = await setup([
|
||||
'var previousCua = cua; "created"',
|
||||
'var previousCua = cua; inspect("created")',
|
||||
'process.exit(0)',
|
||||
'({ ready: typeof cua.getApp === "function", previous: typeof previousCua })',
|
||||
'inspect(JSON.stringify({ ready: typeof cua.getApp === "function", previous: typeof previousCua }))',
|
||||
])
|
||||
await run()
|
||||
const results = agent.session.deriveMessages().filter(message => message.role === 'tool')
|
||||
@@ -198,7 +199,7 @@ it('recreates the automatic facade when its environment is rebuilt', async () =>
|
||||
|
||||
it('settles an in-flight native call before releasing the provider connection and program process', async () => {
|
||||
const { ctx, native, agent, provider, run } = await setup([
|
||||
'var app = await cua.getApp("test.fixture"); var win = await app.getWindow(7); await win.getAccessibilityTree({emit:false}); process.pid',
|
||||
'var app = await cua.getApp("test.fixture"); var win = await app.getWindow(7); await win.getAccessibilityTree(); inspect(process.pid)',
|
||||
'await win.click("0")',
|
||||
])
|
||||
const started = native.holdNextClick()
|
||||
|
||||
@@ -18,11 +18,11 @@
|
||||
en: c5ae9318211c3fb9
|
||||
zh: f297045a12d54098
|
||||
/deepseek-ai-dsh-experimental-ptc-runtime-python/use-this-package/what-you-get:
|
||||
en: ca7b9cc958069c7f
|
||||
zh: 8a4ac72d02ad76eb
|
||||
en: 5e339e8d7c293dd2
|
||||
zh: ff6a86ce83918aa4
|
||||
/deepseek-ai-dsh-experimental-ptc-runtime-python/use-this-package/the-wire:
|
||||
en: e690628dc6197c3b
|
||||
zh: 2b9cdad8b41dcde5
|
||||
en: b15b31d62db5253d
|
||||
zh: 3eec4849570c7916
|
||||
/deepseek-ai-dsh-experimental-ptc-runtime-python/use-this-package/what-can-go-wrong:
|
||||
en: 9a0c3294b510cd22
|
||||
zh: 5635886060044ae3
|
||||
@@ -33,8 +33,8 @@
|
||||
en: 706fb5750d6e766f
|
||||
zh: b8f054bffb4c1ff2
|
||||
/deepseek-ai-dsh-experimental-ptc-runtime-python/understand-the-implementation/wire-contract:
|
||||
en: 56cb353442d0aa01
|
||||
zh: bcb13e5dc761af4a
|
||||
en: 261a827d31bb983e
|
||||
zh: 0590a57d64eaa499
|
||||
/deepseek-ai-dsh-experimental-ptc-runtime-python/understand-the-implementation/lossless-json-crossing:
|
||||
en: 6252e86fa10e9dd0
|
||||
zh: 9b95b033ea0b5463
|
||||
@@ -54,8 +54,8 @@
|
||||
en: 40294c1c31b74824
|
||||
zh: d805c6bd7270505e
|
||||
/deepseek-ai-dsh-experimental-ptc-runtime-python/known-limitations-and-deferred-work:
|
||||
en: 5d62ed07fefbefc4
|
||||
zh: 2a03d9bc9265a200
|
||||
en: 09cd151c631bd9c2
|
||||
zh: 5c9a34fc6b4e9804
|
||||
/deepseek-ai-dsh-experimental-ptc-runtime-python/known-limitations-and-deferred-work/dev-note:
|
||||
en: e67f633dc6fe7773
|
||||
zh: 3bb0c20c93f16f98
|
||||
@@ -43,9 +43,13 @@ The program is the body of an async Python function. Top-level await and return
|
||||
|
||||
The package's default export is the `PythonPtcRuntime` plugin. Its public surface also re-exports the host-side protocol vocabulary: `validateChildFrame` (rebuilds every inbound frame), the lossless-JSON codec and meters (`encodeJsonPlain`, `checkDoneValue`, `hasUnsafeIntegerToken`, `hasNonLosslessNumber`), `logTruncationMarker` (the shared truncation-marker text), plus `resolvePythonBin` (interpreter lookup against the current `PATH`), `readProcessStart` (process-start statistics for tests), `detachResidual` (a test seam for the settled run's resource cleanup), and `hostFrameParseCeiling` (the heap-derived frame parse cap a given heap limit admits). Every cap is a validated `Config` field with a default: `cpuSeconds` (60), `maxWallMs` (600000), `addressSpaceMb` (512, not applied on Darwin), `maxLogBytes` (65536), `maxValueBytes` (32768), `graceMs` (3000), and `pythonBin` (`python3`, resolved, executable-checked, version-probed under a five-second force-kill deadline, and frozen at load). Each child receives only `TMPDIR`; ambient credentials, `PATH`, `HOME`, and other host state stay unavailable.
|
||||
|
||||
`resolve` rejects `discardCompletion: true` before starting a Python process. Omitted or false values preserve the program completion.
|
||||
|
||||
`result.output` text entries retain written newlines and unterminated tails; concatenate their `text` fields with `join('')`. `print('a', 'b')` contributes `'a b\n'`, while `sys.stdout.write('a'); sys.stdout.write('b')` contributes `'ab'`. Both the child and host include newline escaping in `maxLogBytes` accounting.
|
||||
|
||||
### The wire
|
||||
|
||||
Frames travel on the child's fd 3 as JSON-lines — one object per line — so stdout/stderr stay clear for the program's own output. Child → host: `boot-ack`, `call`, `log`, `done`. Host → child: `boot` (first frame, carrying every cap and the namespace declarations), `run` (after `boot-ack`, carrying only the program body), and one `reply` per `call`. A forged frame can carry both `value` and `error` on `done`, so a consumer must check `error` first and ignore `value` when it is set. A `log` frame's `open` flag marks an unterminated line committed by an explicit flush: the host appends the next log frame to the same entry, so `print('a', end='', flush=True); print('b')` reads back as one `'ab'` entry rather than a fake newline (the split-billing arithmetic lives in the fd-3 protocol Agent Note's wire-contract section). The one exception to merging is truncation: when a later over-budget frame trips the ledger, the already-billed prefix is committed as its own entry and the truncation marker follows it (the marker stays last, with no re-charge).
|
||||
Frames travel on the child's fd 3 as JSON-lines — one object per line — so stdout/stderr stay clear for the program's own output. Child → host: `boot-ack`, `call`, `log`, `done`. Host → child: `boot` (first frame, carrying every cap and the namespace declarations), `run` (after `boot-ack`, carrying only the program body), and one `reply` per `call`. A forged frame can carry both `value` and `error` on `done`, so a consumer must check `error` first and ignore `value` when it is set. A `log` frame's `open` flag marks an unterminated line committed by an explicit flush: the host appends the next log frame to the same entry, so `print('a', end='', flush=True); print('b')` reads back as one `'ab\n'` entry with its written newline (the split-billing arithmetic lives in the fd-3 protocol Agent Note's wire-contract section). The one exception to merging is truncation: when a later over-budget frame trips the ledger, the already-billed prefix is committed as its own entry and the truncation marker follows it (the marker stays last, with no re-charge).
|
||||
|
||||
### What can go wrong
|
||||
|
||||
@@ -71,7 +75,7 @@ The shipped bootstrap and protocol sources reach the interpreter as UTF-8 JSON o
|
||||
|
||||
### Wire contract
|
||||
|
||||
The frames are `boot` / `run` (host → child) and `boot-ack` / `call` / `log` / `done` plus one `reply` per call (child → host). The `log` frame's `truncated` flag marks the frame that IS the child ledger's truncation marker, so the host stops capturing at the same point the child did instead of inferring it from its own budget. The `log` frame's `open` flag marks an unterminated line committed by an explicit flush: the host merges the next log frame into the same entry, so `print('a', end='', flush=True); print('b')` reads back as one `'ab'` entry rather than a fake newline (the split-billing arithmetic lives in the fd-3 protocol Agent Note's wire-contract section). The one exception to merging is truncation: the already-billed prefix is committed as its own entry and the truncation marker follows it (marker last, no re-charge). `done.error.kind` is one of `exception`, `invalid-output`, `output-limit`, or `timeout` from the CPU check after model code returns; wall deadlines, aborts, and substrate death are observed host-side.
|
||||
The frames are `boot` / `run` (host → child) and `boot-ack` / `call` / `log` / `done` plus one `reply` per call (child → host). The `log` frame's `truncated` flag marks the frame that IS the child ledger's truncation marker, so the host stops capturing at the same point the child did instead of inferring it from its own budget. The `log` frame's `open` flag marks an unterminated line committed by an explicit flush: the host merges the next log frame into the same entry, so `print('a', end='', flush=True); print('b')` reads back as one `'ab\n'` entry with its written newline (the split-billing arithmetic lives in the fd-3 protocol Agent Note's wire-contract section). The one exception to merging is truncation: the already-billed prefix is committed as its own entry and the truncation marker follows it (marker last, no re-charge). `done.error.kind` is one of `exception`, `invalid-output`, `output-limit`, or `timeout` from the CPU check after model code returns; wall deadlines, aborts, and substrate death are observed host-side.
|
||||
|
||||
### Lossless JSON crossing
|
||||
|
||||
@@ -127,16 +131,16 @@ These limits define what the package does and does not cover; they are current p
|
||||
|
||||
- **The cross-language guard covers the executed surfaces and the frame field shapes, not the field types** — the mirror e2e compares required/optional field sets, not that `cpuSeconds` is an `int` on both sides; a type-level drift is caught by review plus the backend's real-subprocess suite.
|
||||
- **A descendant that escapes the child's process group with `setsid()` is not reaped by the group teardown** — `kill(-pid)` cannot reach it; the run still settles on the value the done frame decided, and the close-deadline backstop forces settlement if the orphan holds the pipes open, but the orphan itself outlives the fiber until it exits on its own.
|
||||
- **A `log` frame that arrives after settlement is dropped** — once the run has settled, host-side capture is closed; a late fd-3 `log` frame (from a thread that outlived the done frame) is discarded rather than appended to `logs`.
|
||||
- **A `log` frame that arrives after settlement is dropped** — once the run has settled, host-side capture is closed; a late fd-3 `log` frame (from a thread that outlived the done frame) is discarded rather than appended to `result.output`.
|
||||
- **A binding REPLY value has no seam-level byte or depth cap** — `maxValueBytes` meters only the done frame's completion value; a wide binding reply is rebuilt host-side (`snapshotJsonValue` traversal) and encoded whole, bounded on both sides only by process memory (like a binding argument, which has no child-side budget either).
|
||||
- **No shipped profile mounts this provider** — the keyless `ptc-python-turn` snapshot replaces the headless PTC runtime through the real Loader; released profiles use the sandboxed Node process backend.
|
||||
- **Workflow execution requires Node** — compositions using this Python provider disable `workflow-ptc`, `tool-workflow`, and `tool-ralph`; the workflow provider rejects an incompatible runtime when loaded.
|
||||
- **Cross-channel log interleaving is backend-dependent** — Python stdout, stderr, and fd-3 log frames travel independently; each channel preserves its own order, while their total order in `result.logs` may differ.
|
||||
- **Cross-channel log interleaving is backend-dependent** — Python stdout, stderr, and fd-3 log frames travel independently; each channel preserves its own order, while their total order in `result.output` may differ.
|
||||
- **CPython 3.10 or newer is required** — the configured executable is resolved and version-probed at load; unsupported interpreters fail before `ctx.ptcRuntime` is registered.
|
||||
- **Execution is local and Unix-only** — the interpreter, working directory, and sandbox runner must belong to the host running this plugin. The provider uses POSIX resource limits and process-group signals; it does not support Windows or remote execution providers.
|
||||
- **Sandbox enforcement inherits backend limits** — a successful program can report partial enforcement, and file confinement does not establish multi-tenant isolation or complete descendant cleanup.
|
||||
- **Diagnostic prefixes omit the package's experimental qualifier** — the marker `[dsh-ptc-runtime-python] log capture truncated at <N> bytes` identifies this provider independently of its npm package name. The protocol mirror verifies identical marker bytes in TypeScript and Python.
|
||||
- **`run()` is one-shot** — `logs` become available only after `PtcRunResult` resolves; there is no streaming-log or progress interface for output produced by a running program.
|
||||
- **`run()` is one-shot** — `output` becomes available only after `PtcRunResult` resolves; there is no streaming-log or progress interface for output produced by a running program.
|
||||
- **No state persists across runs** — every request executes in a fresh subprocess; a persistent REPL-style kernel stays deferred until a backend brings its own logging scheme.
|
||||
- **An fd-3 frame whose raw length exceeds the effective frame parse cap settles the run as a worker-exit** — the cap is 64 MiB, or lower when the host's configured heap cannot safely parse a near-cap frame (`hostFrameParseCeiling`); `maxLogBytes`/`maxValueBytes` are load-bounded to the same cap so an honest child's frames always fit; a model-constructed binding ARGUMENT above the cap (a value with no seam-level budget) trips it too — an accepted residual of the OOM guard.
|
||||
- **A child that stops reading its replies settles the run as a worker-exit once the reply backlog passes 1024 frames** — the host writes replies one at a time, waiting for `drain` when the pipe is full; a child that keeps sending calls without consuming replies would otherwise grow the retained backlog (and the binding results it pins) until the wall clock, so the backlog cap fails the run early. Binding results carry no seam-level byte cap, so this is a count bound, not a byte bound.
|
||||
|
||||
@@ -43,9 +43,13 @@ The program is the body of an async Python function. Top-level await and return
|
||||
|
||||
包的默认导出是 `PythonPtcRuntime` 插件。其公开面还重新导出宿主侧协议词汇:`validateChildFrame`(重建每条入站帧)、无损 JSON codec 与计量器(`encodeJsonPlain`、`checkDoneValue`、`hasUnsafeIntegerToken`、`hasNonLosslessNumber`)、`logTruncationMarker`(共享截断标记文本),以及 `resolvePythonBin`(对照当前 `PATH` 的解释器查找)、`readProcessStart`(供测试用的进程启动统计)、`detachResidual`(已结算运行的资源清理测试 seam)与 `hostFrameParseCeiling`(给定堆上限可容纳的堆推导帧解析上限)。每个上限都是带默认值并经校验的 `Config` 字段:`cpuSeconds`(60)、`maxWallMs`(600000)、`addressSpaceMb`(512,Darwin 上不生效)、`maxLogBytes`(65536)、`maxValueBytes`(32768)、`graceMs`(3000)与 `pythonBin`(`python3`,在加载期解析、检查可执行性,在五秒强制终止期限内探测版本并固定)。每个子进程只接收 `TMPDIR`;环境中的凭证、`PATH`、`HOME` 与其他宿主状态均不可见。
|
||||
|
||||
`resolve` 在启动 Python 进程前拒绝 `discardCompletion: true`。省略该选项或设为 false 时保留程序完成值。
|
||||
|
||||
`result.output` 的文本条目保留实际写入的换行和未换行尾段;用 `join('')` 连接各条目的 `text`。`print('a', 'b')` 输出 `'a b\n'`,而 `sys.stdout.write('a'); sys.stdout.write('b')` 输出 `'ab'`。子进程与宿主都将换行的 JSON 转义开销计入 `maxLogBytes`。
|
||||
|
||||
### wire
|
||||
|
||||
帧在子进程 fd 3 上以 JSON-lines 传输——每行一个对象——因此 stdout/stderr 留给程序自己的输出。子进程 → 宿主:`boot-ack`、`call`、`log`、`done`。宿主 → 子进程:`boot`(首帧,携带全部上限与命名空间声明)、`run`(`boot-ack` 之后,只携带程序体)与每个 `call` 一个 `reply`。伪造帧可在 `done` 上同时携带 `value` 与 `error`,因此消费方必须先检查 `error`,在它存在时忽略 `value`。`log` 帧的 `open` 标志标记由显式 flush 提交的未结束行:宿主把下一个 log 帧追加到同一条目,因此 `print('a', end='', flush=True); print('b')` 读回为一条 `'ab'` 条目而不是假换行(拆分计费算术见 fd-3 协议 Agent Note 的 wire-contract 一节)。合并的唯一例外是截断:当后续超预算帧触发账本时,已计费的前缀作为独立条目先提交,截断 marker 跟在后面(marker 保持末位,无重复计费)。
|
||||
帧在子进程 fd 3 上以 JSON-lines 传输——每行一个对象——因此 stdout/stderr 留给程序自己的输出。子进程 → 宿主:`boot-ack`、`call`、`log`、`done`。宿主 → 子进程:`boot`(首帧,携带全部上限与命名空间声明)、`run`(`boot-ack` 之后,只携带程序体)与每个 `call` 一个 `reply`。伪造帧可在 `done` 上同时携带 `value` 与 `error`,因此消费方必须先检查 `error`,在它存在时忽略 `value`。`log` 帧的 `open` 标志标记由显式 flush 提交的未结束行:宿主把下一个 log 帧追加到同一条目,因此 `print('a', end='', flush=True); print('b')` 读回为一条带有原始换行的 `'ab\n'` 条目(拆分计费算术见 fd-3 协议 Agent Note 的 wire-contract 一节)。合并的唯一例外是截断:当后续超预算帧触发账本时,已计费的前缀作为独立条目先提交,截断 marker 跟在后面(marker 保持末位,无重复计费)。
|
||||
|
||||
### 可能出错的地方
|
||||
|
||||
@@ -71,7 +75,7 @@ The program is the body of an async Python function. Top-level await and return
|
||||
|
||||
### wire 契约
|
||||
|
||||
帧为 `boot`/`run`(宿主 → 子进程)与 `boot-ack`/`call`/`log`/`done` (子进程 → 宿主),以及每个 `call` 对应一个 `reply`(宿主 → 子进程)。`log` 帧的 `truncated` 标志标记的就是子进程账本自己的截断标记帧,因此宿主在与子进程相同的点停止捕获,而不是从自己的预算推断。`log` 帧的 `open` 标志标记由显式 flush 提交的未结束行:宿主把下一个 log 帧合并进同一条目,因此 `print('a', end='', flush=True); print('b')` 读回为一条 `'ab'` 条目而不是假换行(拆分计费算术在 fd-3 协议 Agent Note 的 wire-contract 段)。合并的唯一例外是截断:当后续超预算帧触发账本时,已计费的前缀作为独立条目先提交,截断 marker 跟在后面(marker 保持末位,无重复计费)。`done.error.kind` 为 `exception`、`invalid-output`、`output-limit` 或模型代码返回后 CPU 检查产生的 `timeout` 之一;墙钟截止、中止与基底死亡在宿主侧观察。
|
||||
帧为 `boot`/`run`(宿主 → 子进程)与 `boot-ack`/`call`/`log`/`done` (子进程 → 宿主),以及每个 `call` 对应一个 `reply`(宿主 → 子进程)。`log` 帧的 `truncated` 标志标记的就是子进程账本自己的截断标记帧,因此宿主在与子进程相同的点停止捕获,而不是从自己的预算推断。`log` 帧的 `open` 标志标记由显式 flush 提交的未结束行:宿主把下一个 log 帧合并进同一条目,因此 `print('a', end='', flush=True); print('b')` 读回为一条带有原始换行的 `'ab\n'` 条目(拆分计费算术在 fd-3 协议 Agent Note 的 wire-contract 段)。合并的唯一例外是截断:当后续超预算帧触发账本时,已计费的前缀作为独立条目先提交,截断 marker 跟在后面(marker 保持末位,无重复计费)。`done.error.kind` 为 `exception`、`invalid-output`、`output-limit` 或模型代码返回后 CPU 检查产生的 `timeout` 之一;墙钟截止、中止与基底死亡在宿主侧观察。
|
||||
|
||||
### 无损 JSON 跨越
|
||||
|
||||
@@ -127,16 +131,16 @@ The program is the body of an async Python function. Top-level await and return
|
||||
|
||||
- **跨语言 guard 覆盖执行的表面与帧字段形状,而非字段类型**——mirror e2e 比较必填/可选字段集,而非 `cpuSeconds` 在两侧是否都是 `int`;类型级漂移由评审加后端的真实子进程套件捕获。
|
||||
- **以 `setsid()` 逃出子进程组后代不被组拆卸回收**——`kill(-pid)` 够不到它;运行仍按 done 帧决定的值结算,若该孤儿持有管道,close 截止兜底会强制结算,但孤儿本身在自行退出前一直存活到 fiber 之外。
|
||||
- **结算后到达的 `log` 帧被丢弃**——运行一旦结算,宿主侧捕获即关闭;迟到的 fd-3 `log` 帧(来自比 done 帧存活更久的线程)会被丢弃,而不是追加到 `logs`。
|
||||
- **结算后到达的 `log` 帧被丢弃**——运行一旦结算,宿主侧捕获即关闭;迟到的 fd-3 `log` 帧(来自比 done 帧存活更久的线程)会被丢弃,而不是追加到 `result.output`。
|
||||
- **binding 回复值没有 seam 级字节或深度上限**——`maxValueBytes` 只计量 done 帧的完成值;宽 binding 回复在宿主侧重建(`snapshotJsonValue` 遍历)并整帧编码,两侧都只受进程内存约束(与没有子进程侧预算的 binding 实参一样)。
|
||||
- **已发布 profile 均不挂载本提供方**——keyless `ptc-python-turn` 快照通过真实 Loader 替换 headless PTC 运行时;已发布 profile 使用沙箱 Node 进程后端。
|
||||
- **工作流执行需要 Node**——使用本 Python 提供方的组合禁用 `workflow-ptc`、`tool-workflow` 和 `tool-ralph`;工作流提供方在加载时拒绝不兼容的运行时。
|
||||
- **跨通道日志交错由后端决定**——Python stdout、stderr 与 fd-3 日志帧彼此独立传输;每个通道保留自身顺序,但它们在 `result.logs` 中的总顺序可能不同。
|
||||
- **跨通道日志交错由后端决定**——Python stdout、stderr 与 fd-3 日志帧彼此独立传输;每个通道保留自身顺序,但它们在 `result.output` 中的总顺序可能不同。
|
||||
- **需要 CPython 3.10 或更高版本**——配置的可执行文件会在加载期完成解析与版本探测;不受支持的解释器会在 `ctx.ptcRuntime` 注册前失败。
|
||||
- **执行限于本地 Unix 宿主**——解释器、工作目录与沙箱 runner 必须位于运行本插件的宿主上。提供方使用 POSIX 资源限制和进程组信号;不支持 Windows 或远程执行提供方。
|
||||
- **沙箱强制能力继承后端限制**——成功的程序也可能报告部分强制能力,文件约束不构成多租户隔离或完整的后代清理保证。
|
||||
- **诊断前缀省略包名中的 experimental 限定词**——标记 `[dsh-ptc-runtime-python] log capture truncated at <N> bytes` 独立于 npm 包名来标识本提供方。协议镜像检查 TypeScript 与 Python 的标记字节完全相同。
|
||||
- **`run()` 是一次性的**——`logs` 只有在 `PtcRunResult` resolve 后才能获得;没有为运行中程序产生的输出提供流式日志或进度接口。
|
||||
- **`run()` 是一次性的**——`output` 只有在 `PtcRunResult` resolve 后才能获得;没有为运行中程序产生的输出提供流式日志或进度接口。
|
||||
- **运行之间不保留状态**——每次请求都在全新子进程中执行;持久 REPL 风格内核在某个后端带来自己的日志方案之前保持延期。
|
||||
- **原始长度超过有效帧解析上限的 fd-3 帧会让本次运行以 worker-exit 结算**——上限为 64 MiB,或当宿主的配置堆无法安全解析接近上限的帧时更低(`hostFrameParseCeiling`);`maxLogBytes`/`maxValueBytes` 在加载期被限制到同一上限,因此诚实子进程的帧总能放得下;模型构造的超过该上限的 binding 实参(一个在 seam 层没有预算的值)会触发同一上限——这是该 OOM 防护的已接受残余。
|
||||
- **停止读取回复的子进程会在回复积压超过 1024 帧时以 worker-exit 结算运行**——宿主每次写一条回复,管道满时等待 `drain`;只持续发送调用而不消费回复的子进程会让保留的积压(及其钉住的 binding 结果)一直增长到墙钟,因此积压上限让运行提前失败。binding 结果在 seam 层没有字节上限,所以这是计数上限而非字节上限。
|
||||
|
||||
@@ -215,14 +215,9 @@ class _LogStream(io.TextIOBase):
|
||||
"""A newline-coalescing text stream backed by a :class:`LogBuffer`.
|
||||
|
||||
Installed as ``sys.stdout`` / ``sys.stderr`` before executing the model
|
||||
program. ``print(...)`` calls ``write`` once per argument, separator, and
|
||||
newline, so a raw one-push-per-write stream would emit
|
||||
``["a", " ", "b", "\\n"]`` for ``print("a", "b")`` — and PTC mode renders
|
||||
``logs`` with ``join('\\n')``, turning that into spurious blank lines. This
|
||||
stream instead buffers writes and pushes one LogBuffer entry per completed
|
||||
LINE (the text up to each ``\\n``, newline stripped), so the rendered join
|
||||
reproduces ``a b``. Any unterminated tail is flushed by :meth:`flush_line`
|
||||
after the program settles.
|
||||
program. Writes are buffered into complete lines including their original
|
||||
newline. Explicit flushes and program settlement emit unterminated tails
|
||||
without adding a newline. Each emitted fragment is charged as written.
|
||||
"""
|
||||
|
||||
def __init__(self, logs: LogBuffer) -> None:
|
||||
@@ -290,22 +285,10 @@ class _LogStream(io.TextIOBase):
|
||||
pos = 0
|
||||
if self._pending or self._pending_blocks:
|
||||
newline = text.index("\n")
|
||||
# The +3 cheap-bound overhead (quotes + separator) belongs to a
|
||||
# NEW entry. While an `open` entry is accumulating, the closing
|
||||
# line is that entry's TAIL: its cheap bound is the content
|
||||
# length alone, matching `_push_locked`'s open-aware bound.
|
||||
# This bound's observable behavior is invariant under the
|
||||
# overhead either way: when the +3 form trips and the open-aware
|
||||
# form does not (pending + newline in [remaining - 2, remaining]),
|
||||
# _push_bounded_prefix re-slices the SAME line text (the extra
|
||||
# slice `text[:newline]` carries no newline) and _push_locked
|
||||
# admits it under the same open-aware billing, byte for byte.
|
||||
# The open-aware form only keeps _push_bounded_prefix's "certain
|
||||
# to reject" precondition true, which is exactly what the scan
|
||||
# pre-check below does NOT preserve (its slice carries the
|
||||
# newline, so push genuinely rejects and truncates).
|
||||
# A complete line includes its newline. New entries also pay
|
||||
# the quotes and separator; open entries already paid them.
|
||||
overhead = 3 if not self._logs._open_started else 0
|
||||
if self._pending_chars + newline + overhead > self._logs.remaining:
|
||||
if self._pending_chars + newline + 1 + overhead > self._logs.remaining:
|
||||
# The reconstructed first line cannot fit the ledger, so
|
||||
# LogBuffer would reject it whole: copy only the prefix that
|
||||
# fails its cheap bound and drop the chunks. The slice is
|
||||
@@ -314,9 +297,9 @@ class _LogStream(io.TextIOBase):
|
||||
# join was (measured: MemoryError inside `sys.stdout.write`
|
||||
# under a 400 MiB addressSpaceMb), and `remaining + 4`
|
||||
# characters are all the helper can use.
|
||||
self._push_bounded_prefix(text[: min(newline, self._logs.remaining + 4)])
|
||||
self._push_bounded_prefix(text[: min(newline + 1, self._logs.remaining + 4)])
|
||||
else:
|
||||
self._pending.append(text[:newline])
|
||||
self._pending.append(text[:newline + 1])
|
||||
line = "".join(self._pending_parts())
|
||||
self._pending = []
|
||||
self._pending_blocks = []
|
||||
@@ -341,10 +324,10 @@ class _LogStream(io.TextIOBase):
|
||||
# prefix is longer than `remaining`), so the marker is emitted
|
||||
# and the oversized line is never materialized.
|
||||
overhead = 3 if not self._logs._open_started else 0
|
||||
if newline - pos + overhead > self._logs.remaining:
|
||||
if newline - pos + 1 + overhead > self._logs.remaining:
|
||||
self._logs.push(text[pos:pos + self._logs.remaining + 4])
|
||||
break
|
||||
self._logs.push(text[pos:newline])
|
||||
self._logs.push(text[pos:newline + 1])
|
||||
pos = newline + 1
|
||||
if pos < length:
|
||||
if self._logs.remaining > 0:
|
||||
@@ -368,14 +351,8 @@ class _LogStream(io.TextIOBase):
|
||||
# cheap length lower bound rejects immediately, emits the
|
||||
# marker, and never encodes the tail — and a push after the
|
||||
# marker is already out returns without emitting a second.
|
||||
# Reaching 0 EXACTLY (65 one-character lines against the
|
||||
# default 3-byte-per-entry serialized charge) leaves
|
||||
# `_truncated` unset, so without this the tail vanished with
|
||||
# no marker at all. Sliced to a budget-sized prefix, not the
|
||||
# whole tail: the tail can be hundreds of megabytes and the
|
||||
# copy would be the RLIMIT_AS death this bound exists to
|
||||
# avoid, while push only needs enough characters to fail its
|
||||
# own cheap length check.
|
||||
# Keep the rejected prefix bounded even when the unscanned
|
||||
# tail is much larger than the output budget.
|
||||
self._logs.push(text[pos:pos + self._logs.remaining + 4])
|
||||
else:
|
||||
self._pending.append(text)
|
||||
@@ -476,9 +453,8 @@ class _LogStream(io.TextIOBase):
|
||||
self._pending = []
|
||||
self._pending_blocks = []
|
||||
self._pending_chars = 0
|
||||
# The line has NO trailing newline: mark the frame `open` so the
|
||||
# host appends the next log frame to the same entry instead of
|
||||
# inserting a fake newline between two entries.
|
||||
# Keep unterminated fragments in the same billed entry across
|
||||
# explicit flushes and the next newline-terminated write.
|
||||
self._logs.push(line, open=True)
|
||||
|
||||
|
||||
|
||||
@@ -223,7 +223,7 @@ const CLOSE_REAP_MARGIN_MS = 2_000
|
||||
* of astral characters is ~4x the budget in each string that holds it. The
|
||||
* heaviest path holds THREE such copies at once: a single
|
||||
* `sys.stdout.write(line + "\n")` keeps the caller's `text` argument (alive for
|
||||
* the whole `write` call, ~4x), the line slice `text[pos:newline]` handed to
|
||||
* the whole `write` call, ~4x), the line slice `text[pos:newline + 1]` handed to
|
||||
* `LogBuffer.push` (~4x), and the `text.encode("utf-8")` copy `_push_locked`
|
||||
* takes to charge and ship it (~4x). The settlement `flush_line` path holds only
|
||||
* two (its `"".join(...)` and that encode copy — it drops the pending chunks
|
||||
@@ -974,11 +974,12 @@ export class PythonPtcRuntime extends PtcRuntime {
|
||||
* Resolve authority, directory, and the configured wall deadline.
|
||||
* @param request - Program inputs and optional resolved sandbox authority.
|
||||
* @returns Complete inputs for the provider's run method.
|
||||
* @throws When a timeout override is supplied, cwd is relative, or the provider is disposed.
|
||||
* @throws When a requested timeout or completion-discard option is unsupported, cwd is relative, or the provider is disposed.
|
||||
*/
|
||||
resolve(request: PtcRunRequest): PtcRunSpec {
|
||||
if (this.disposed) throw new Error('dsh-ptc-runtime-python: resolve() after disposal')
|
||||
if (request.timeoutMs !== undefined) throw new Error('dsh-ptc-runtime-python: per-call timeout is unsupported')
|
||||
if (request.discardCompletion === true) throw new Error('dsh-ptc-runtime-python: discardCompletion is unsupported')
|
||||
const sandboxPolicy = request.sandboxPolicy ?? this.ctx.sandboxPolicy.resolve()
|
||||
const cwd = request.cwd ?? sandboxPolicy.workspaceRoot
|
||||
if (!isAbsolute(cwd)) throw new Error('dsh-ptc-runtime-python: cwd must be absolute')
|
||||
@@ -1021,7 +1022,7 @@ export class PythonPtcRuntime extends PtcRuntime {
|
||||
try {
|
||||
input = bootstrapInput()
|
||||
} catch (error: unknown) {
|
||||
return { logs: [], sandbox, error: { kind: 'worker-exit', message: `failed to read the python bootstrap: ${messageOf(error)}` } }
|
||||
return { output: [], sandbox, error: { kind: 'worker-exit', message: `failed to read the python bootstrap: ${messageOf(error)}` } }
|
||||
}
|
||||
let confined: ConfinedArgv | undefined
|
||||
// Unbuffered native output stays observable before process-group teardown.
|
||||
@@ -1044,7 +1045,7 @@ export class PythonPtcRuntime extends PtcRuntime {
|
||||
return await pending
|
||||
} catch (error: unknown) {
|
||||
return {
|
||||
logs: [], sandbox,
|
||||
output: [], sandbox,
|
||||
error: controller.signal.aborted ? controller.signal.reason as PtcRunFailure
|
||||
: { kind: error instanceof SandboxUnavailableError ? 'sandbox-unavailable' : 'worker-exit', message: messageOf(error) },
|
||||
}
|
||||
@@ -1175,7 +1176,7 @@ export class PythonPtcRuntime extends PtcRuntime {
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
complete()
|
||||
return Promise.resolve({ logs: [], sandbox, error: {
|
||||
return Promise.resolve({ output: [], sandbox, error: {
|
||||
kind: confined !== undefined && isRunnerSpawnFailure(error, argv[0], request.cwd) ? 'sandbox-unavailable' : 'worker-exit',
|
||||
message: `python spawn error: ${messageOf(error)}`,
|
||||
} })
|
||||
@@ -1289,13 +1290,8 @@ export class PythonPtcRuntime extends PtcRuntime {
|
||||
// Stray-byte capture: anything the child writes to its stdout/stderr
|
||||
// (native prints, C-extension writes) still counts against the ledger.
|
||||
//
|
||||
// Output is admitted per LINE, not per transport chunk. `logs` entries
|
||||
// are joined with `\n` downstream (PTC mode), so each entry must be one
|
||||
// line: pushing a raw `data` chunk would turn every arbitrary pipe-read
|
||||
// boundary into a model-visible newline, so a single 200 KiB native write
|
||||
// split across pipe reads would read back with spurious line breaks. The
|
||||
// child's own `log` frames are already line-granular; stray capture
|
||||
// matches them by splitting on `\n`.
|
||||
// Complete lines retain their newline; final residuals retain their
|
||||
// unterminated text. Transport chunk boundaries do not add separators.
|
||||
//
|
||||
// Buffered as raw `Buffer` chunks with a running SERIALIZED-cost counter,
|
||||
// exactly like the fd-3 reader below and for the same reasons: a string
|
||||
@@ -1339,7 +1335,7 @@ export class PythonPtcRuntime extends PtcRuntime {
|
||||
stray.blocks = []
|
||||
let newline: number
|
||||
while ((newline = buffered.indexOf(0x0a)) >= 0) {
|
||||
admit(buffered.subarray(0, newline).toString('utf8'))
|
||||
admit(buffered.subarray(0, newline + 1).toString('utf8'))
|
||||
buffered = buffered.subarray(newline + 1)
|
||||
}
|
||||
// Carry the residual as a fresh right-sized copy, not the subarray view
|
||||
@@ -1445,8 +1441,8 @@ export class PythonPtcRuntime extends PtcRuntime {
|
||||
stray.blocks = []
|
||||
// Do not admit an EMPTY entry: when the whole residual is a single
|
||||
// unfinished multibyte sequence, `full` was drained into `keep` and no
|
||||
// complete byte stream remains to admit. `admit('')` would push a
|
||||
// model-visible bogus empty line (logs are joined with '\n' downstream).
|
||||
// complete byte stream remains to admit. `admit('')` would push an
|
||||
// empty output entry whose wrapper would consume the ledger.
|
||||
if (full.length > 0) admit(full.toString('utf8'))
|
||||
} else {
|
||||
stray.chunks = []
|
||||
@@ -2146,16 +2142,16 @@ export class PythonPtcRuntime extends PtcRuntime {
|
||||
let resolved = false
|
||||
// The decided terminal result for a live child, recorded by finish() and
|
||||
// read by the `close` handler that settles it once the pipes have drained.
|
||||
let decided: Omit<PtcRunResult, 'logs'>
|
||||
let decided: Omit<PtcRunResult, 'output'>
|
||||
|
||||
// The single settlement point: resolve run() with the decided result and
|
||||
// mark the fiber quiescent. Idempotent — the first call wins, so a later
|
||||
// `close` after done/timeout/abort is absorbed as a no-op.
|
||||
const settle = (result: Omit<PtcRunResult, 'logs'>): void => {
|
||||
const settle = (result: Omit<PtcRunResult, 'output'>): void => {
|
||||
if (resolved) return
|
||||
resolved = true
|
||||
if (closeDeadline !== undefined) clearTimeout(closeDeadline)
|
||||
resolve({ ...result, logs, sandbox: { ...sandbox } })
|
||||
resolve({ ...result, output: logs.map(text => ({ type: 'text', text })), sandbox: { ...sandbox } })
|
||||
// Mark the fiber quiescent for THIS run: drop it from `live` and resolve
|
||||
// `finished` (what teardown awaits). Deferred until the process group is
|
||||
// actually empty — dropping from `live` before then would let a
|
||||
@@ -2239,7 +2235,7 @@ export class PythonPtcRuntime extends PtcRuntime {
|
||||
pollGroup()
|
||||
}
|
||||
|
||||
const finish = (result: Omit<PtcRunResult, 'logs'>): void => {
|
||||
const finish = (result: Omit<PtcRunResult, 'output'>): void => {
|
||||
if (settled) return
|
||||
settled = true
|
||||
decided = result
|
||||
|
||||
@@ -88,6 +88,8 @@ describe('PythonPtcRuntime — seam descriptors and misuse', () => {
|
||||
expect(() => runtime.resolve({ ...request, cwd: 'relative' })).toThrow('cwd must be absolute')
|
||||
expect(() => runtime.resolve({ ...request, timeoutMs: 1 })).toThrow('per-call timeout is unsupported')
|
||||
expect(() => runtime.resolve({ ...request, timeoutMs: null })).toThrow('per-call timeout is unsupported')
|
||||
expect(() => runtime.resolve({ ...request, discardCompletion: true })).toThrow('discardCompletion is unsupported')
|
||||
expect(runtime.resolve({ ...request, discardCompletion: false }).discardCompletion).toBe(false)
|
||||
expect(() => runtime.registerEnvironmentInitializer('helper = {}')).toThrow('environment initialization is unsupported')
|
||||
const sandboxPolicy = { mode: 'danger-full-access' as const, workspaceRoot: cwd }
|
||||
expect(runtime.resolve({ ...request, sandboxPolicy }).sandboxPolicy).toBe(sandboxPolicy)
|
||||
@@ -561,7 +563,7 @@ describe('PythonPtcRuntime — seam descriptors and misuse', () => {
|
||||
const result = await runtime.run(runtime.resolve({ program: 'return 1', bindings: [], signal }))
|
||||
expect(result.error?.kind).toBe('abort')
|
||||
expect(result.error?.message).toContain('already-cancelled')
|
||||
expect(result.logs).toEqual([])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([])
|
||||
})
|
||||
|
||||
it('short-circuits on an already-aborted signal whose reason cannot be converted', async () => {
|
||||
@@ -575,7 +577,7 @@ describe('PythonPtcRuntime — seam descriptors and misuse', () => {
|
||||
const result = await runtime.run(runtime.resolve({ program: 'return 1', bindings: [], signal }))
|
||||
expect(result.error?.kind).toBe('abort')
|
||||
expect(result.error?.message).toBe('<unrenderable rejection value>')
|
||||
expect(result.logs).toEqual([])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([])
|
||||
})
|
||||
|
||||
it('delivers packaged Python modules without exposing a writable bootstrap file', async () => {
|
||||
@@ -670,7 +672,7 @@ describe('PythonPtcRuntime — seam descriptors and misuse', () => {
|
||||
const result = await runtime.run(runtime.resolve({ program: 'return 42', bindings: [] }))
|
||||
expect(result.error?.kind).toBe('worker-exit')
|
||||
expect(result.error?.message).toContain('failed to read the python bootstrap')
|
||||
expect(result.logs).toEqual([])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([])
|
||||
expect(result.value).toBeUndefined()
|
||||
} finally {
|
||||
failNextReadOf.value = undefined
|
||||
@@ -971,7 +973,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
expect(result.value).toEqual({ answer: 42 })
|
||||
// `print` in Python emits: text, ' ', text, '\n'. Concat the captured
|
||||
// fragments and assert the model-visible message survives.
|
||||
expect(result.logs.join('')).toContain('hello 42')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).toContain('hello 42')
|
||||
// 15s: this is usually the suite's first real subprocess — a cold python3
|
||||
// start (interpreter + asyncio import) on a loaded CI runner can exceed
|
||||
// the 5s default alone; later tests reuse the warm page cache.
|
||||
@@ -993,7 +995,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toEqual({ tmpdir: tmpdir(), path: null, home: null, token: null })
|
||||
expect(result.logs).toEqual([])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([])
|
||||
})
|
||||
|
||||
it('bridges binding calls both ways and rejects the program-side call on a host rejection', async () => {
|
||||
@@ -1258,17 +1260,42 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
expect(result.value).toBeUndefined()
|
||||
})
|
||||
|
||||
it('coalesces print arguments into one log line, not per-write fragments', async () => {
|
||||
// print("a","b") calls write() per arg/sep/newline; the stream must emit
|
||||
// one logical line "a b" so PTC mode's join(newline) does not insert
|
||||
// spurious blank lines. Two prints → exactly two entries, no empties.
|
||||
it.each([
|
||||
['print arguments', 'print("a", "b")', 'a b\n'],
|
||||
['raw writes', 'import sys; sys.stdout.write("a"); sys.stdout.write("b")', 'ab'],
|
||||
['native newlines', 'import os; os.write(1, b"one\\ntwo\\ntail")', 'one\ntwo\ntail'],
|
||||
['blank lines and trailing text', 'import sys; print(); print("a"); print(); sys.stdout.write("tail")', '\na\n\ntail'],
|
||||
])('preserves raw output from %s', async (_name, program, expected) => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run(runtime.resolve({ program, bindings: [] }))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).toBe(expected)
|
||||
})
|
||||
|
||||
it.each(['python', 'native'])('charges written newlines at the exact output cap for %s writes', async (channel) => {
|
||||
const { runtime } = await setup({ maxLogBytes: 64 })
|
||||
const program = (length: number): string => channel === 'python'
|
||||
? `print("a" * ${length})`
|
||||
: `import os; os.write(1, b"a" * ${length} + b"\\n")`
|
||||
const exact = await runtime.run(runtime.resolve({ program: program(58), bindings: [] }))
|
||||
const text = exact.output.filter(block => block.type === 'text').map(block => block.text)
|
||||
expect(exact.error).toBeUndefined()
|
||||
expect(text).toEqual(['a'.repeat(58) + '\n'])
|
||||
expect(Buffer.byteLength(JSON.stringify(text))).toBe(64)
|
||||
const over = await runtime.run(runtime.resolve({ program: program(59), bindings: [] }))
|
||||
expect(over.error).toBeUndefined()
|
||||
expect(over.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([logTruncationMarker(64)])
|
||||
})
|
||||
|
||||
it('coalesces print arguments and retains each written newline', async () => {
|
||||
// Per-argument writes coalesce without dropping the terminating newline.
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run(runtime.resolve({
|
||||
program: ['print("a", "b")', 'print("c")', 'return None'].join('\n'),
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['a b', 'c'])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['a b\n', 'c\n'])
|
||||
})
|
||||
|
||||
it('flushes a print with no trailing newline', async () => {
|
||||
@@ -1278,14 +1305,12 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['partial'])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['partial'])
|
||||
})
|
||||
|
||||
it('aggregates a large newline-free native write into one log entry, not one per pipe chunk', async () => {
|
||||
// A single `os.write` larger than one pipe read arrives as several Node
|
||||
// `data` chunks. `logs` entries are joined with `\n` downstream, so pushing
|
||||
// one entry per transport chunk would insert model-visible newlines at
|
||||
// arbitrary pipe boundaries inside one native write. Stray capture holds a
|
||||
// `data` chunks. Stray capture holds a
|
||||
// per-stream residual and admits only on a real `\n`, so a 200 KiB blast
|
||||
// with no newline reads back as exactly one entry with no interior breaks.
|
||||
const { runtime } = await setup({ maxLogBytes: 300_000 })
|
||||
@@ -1295,7 +1320,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['A'.repeat(size)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['A'.repeat(size)])
|
||||
})
|
||||
|
||||
it('splits native output on its own newlines, one entry per line', async () => {
|
||||
@@ -1307,7 +1332,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['one', 'two', 'three'])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['one\n', 'two\n', 'three'])
|
||||
})
|
||||
|
||||
it('preserves each native stream order while allowing backend-dependent interleaving', async () => {
|
||||
@@ -1324,8 +1349,8 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.indexOf('stdout-one')).toBeLessThan(result.logs.indexOf('stdout-two'))
|
||||
expect(result.logs.indexOf('stderr-one')).toBeLessThan(result.logs.indexOf('stderr-two'))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).indexOf('stdout-one\n')).toBeLessThan(result.output.filter(block => block.type === 'text').map(block => block.text).indexOf('stdout-two\n'))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).indexOf('stderr-one\n')).toBeLessThan(result.output.filter(block => block.type === 'text').map(block => block.text).indexOf('stderr-two\n'))
|
||||
})
|
||||
|
||||
it('bounds a newline-free native flood by the ledger instead of buffering it whole', async () => {
|
||||
@@ -1341,9 +1366,9 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.at(-1)).toBe(logTruncationMarker(4096))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).at(-1)).toBe(logTruncationMarker(4096))
|
||||
// The retained output is bounded by the budget, not the 2 MB flood.
|
||||
expect(result.logs.join('').length).toBeLessThan(4096)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('').length).toBeLessThan(4096)
|
||||
})
|
||||
|
||||
it('bounds a newline-free single-character Python write drip by the fragment cap, not OOM', async () => {
|
||||
@@ -1368,7 +1393,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.at(-1)).toBe(logTruncationMarker(4096))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).at(-1)).toBe(logTruncationMarker(4096))
|
||||
})
|
||||
|
||||
it('bounds a control-char-dense native residual by serialized cost, not raw length', async () => {
|
||||
@@ -1386,7 +1411,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.at(-1)).toBe(logTruncationMarker(4096))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).at(-1)).toBe(logTruncationMarker(4096))
|
||||
})
|
||||
|
||||
it('rejects an output budget that could breach addressSpaceMb during encode at load', async () => {
|
||||
@@ -1459,7 +1484,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.at(-1)).toBe(logTruncationMarker(4096))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).at(-1)).toBe(logTruncationMarker(4096))
|
||||
})
|
||||
|
||||
it('drops a second stray line in the same chunk once the first truncated the ledger', async () => {
|
||||
@@ -1478,8 +1503,8 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.at(-1)).toBe(logTruncationMarker(64))
|
||||
expect(result.logs.join('\n')).not.toContain('SECOND')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).at(-1)).toBe(logTruncationMarker(64))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).not.toContain('SECOND')
|
||||
})
|
||||
|
||||
it('charges a broken multibyte sequence its U+FFFD bytes, split across pipe chunks', async () => {
|
||||
@@ -1501,8 +1526,8 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.join('')).toContain('A')
|
||||
expect(result.logs.join('')).toContain('�')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).toContain('A')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).toContain('�')
|
||||
})
|
||||
|
||||
it('charges the exact serialized cost of short-escape and quote/backslash characters', async () => {
|
||||
@@ -1518,7 +1543,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['\ta"b\\c\x01é'])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['\ta"b\\c\x01é\n'])
|
||||
})
|
||||
|
||||
it('fails a completion dict with a non-string key as invalid-output (no key coercion)', async () => {
|
||||
@@ -2049,11 +2074,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
}, 90_000)
|
||||
|
||||
it('appends a flushed unterminated line to the next entry without a fake newline', async () => {
|
||||
// An explicit flush of an unterminated line (print(..., end='', flush=True))
|
||||
// used to push a full log frame, so the following print() landed in a
|
||||
// SECOND entry and logs.join('\n') rendered 'a\nb' for what the program
|
||||
// printed as one line. The flush frame now carries `open: true` and the
|
||||
// host appends the next frame to the same entry.
|
||||
// Explicitly flushed text and the closing newline share one billed entry.
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run(runtime.resolve({
|
||||
program: [
|
||||
@@ -2064,7 +2085,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['ab'])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['ab\n'])
|
||||
}, 15_000)
|
||||
|
||||
it('keeps a SEALED open hold when the run ends with it still open', async () => {
|
||||
@@ -2083,7 +2104,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['x' + 'a'.repeat(3000)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['x' + 'a'.repeat(3000)])
|
||||
}, 15_000)
|
||||
|
||||
it('keeps a flushed unterminated line when the run ends with it still open', async () => {
|
||||
@@ -2099,7 +2120,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['committed'])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['committed'])
|
||||
}, 15_000)
|
||||
|
||||
it('skips the hold for a zero-content open continuation', async () => {
|
||||
@@ -2119,7 +2140,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['xy'])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['xy'])
|
||||
}, 15_000)
|
||||
|
||||
it('seals the open hold past MAX_PENDING_CHUNKS without changing the merged entry', async () => {
|
||||
@@ -2141,7 +2162,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['x' + 'a'.repeat(3000) + 'y'])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['x' + 'a'.repeat(3000) + 'y'])
|
||||
}, 15_000)
|
||||
|
||||
it('bounds a forged open-frame flood against the log budget', async () => {
|
||||
@@ -2161,7 +2182,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['a'.repeat(60), logTruncationMarker(64)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['a'.repeat(60), logTruncationMarker(64)])
|
||||
}, 15_000)
|
||||
|
||||
it('commits a sealed open hold before the truncation marker', async () => {
|
||||
@@ -2185,8 +2206,8 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs[0]).toBe('x' + 'a'.repeat(3000))
|
||||
expect(result.logs[result.logs.length - 1]).toBe(logTruncationMarker(65536))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)[0]).toBe('x' + 'a'.repeat(3000))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)[result.output.filter(block => block.type === 'text').map(block => block.text).length - 1]).toBe(logTruncationMarker(65536))
|
||||
}, 15_000)
|
||||
|
||||
it('commits a flushed open prefix before the truncation marker', async () => {
|
||||
@@ -2203,7 +2224,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['committed', logTruncationMarker(64)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['committed', logTruncationMarker(64)])
|
||||
}, 15_000)
|
||||
|
||||
it('drops a forged fd-3 frame with illegal UTF-8 instead of accepting a mangled value', async () => {
|
||||
@@ -2240,14 +2261,14 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['a'.repeat(60), logTruncationMarker(64)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['a'.repeat(60), logTruncationMarker(64)])
|
||||
}, 15_000)
|
||||
|
||||
it('bills a merged open entry once, not per fragment', async () => {
|
||||
// A merged entry's wire cost is billed ONCE, split across its fragments
|
||||
// (first fragment pays quotes+separator, continuations pay only content).
|
||||
// Under maxLogBytes: 64, 16 single-character flushes merge to one 16-char
|
||||
// entry (2 quotes + 16 content + 1 separator = 19), which fits; per-
|
||||
// entry (2 quotes + 16 content + 2 escaped newline + 1 separator = 21), which fits; per-
|
||||
// fragment billing (each charged quotes+separator, ~4 bytes) would truncate
|
||||
// at 16 x 4 = 64.
|
||||
const { runtime } = await setup({ maxLogBytes: 64 })
|
||||
@@ -2261,15 +2282,11 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['x'.repeat(16)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['x'.repeat(16) + '\n'])
|
||||
}, 15_000)
|
||||
|
||||
it('admits a compliant merged entry whose closing frame fits the remaining budget', async () => {
|
||||
// The review's arithmetic check: print('a'*30, flush); print('b'*25) under
|
||||
// maxLogBytes: 64 has a merged wire cost of 2 quotes + 55 content + 1
|
||||
// separator = 58 <= 63, so it MUST be admitted as one entry. The earlier
|
||||
// cap math (logBudget - openCost) made the closing frame's walk see a
|
||||
// negative cap and truncate a compliant entry.
|
||||
// The merged text and newline cost 60 bytes against the 63-byte ledger.
|
||||
const { runtime } = await setup({ maxLogBytes: 64 })
|
||||
const result = await runtime.run(runtime.resolve({
|
||||
program: [
|
||||
@@ -2280,7 +2297,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['a'.repeat(30) + 'b'.repeat(25)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['a'.repeat(30) + 'b'.repeat(25) + '\n'])
|
||||
}, 15_000)
|
||||
|
||||
it('rejects an open frame that would overflow the ledger by one byte', async () => {
|
||||
@@ -2300,50 +2317,41 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual([logTruncationMarker(64)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([logTruncationMarker(64)])
|
||||
}, 15_000)
|
||||
|
||||
it('bills the closing frame as the merged tail under an exact-fit budget', async () => {
|
||||
// The child's split billing: a 30-char open + a 30-char closing frame cost
|
||||
// 2 + 60 + 1 = 63 = ledger 63 exactly; the closing frame must be billed as
|
||||
// the merged tail (content only), not as a fresh entry (which would
|
||||
// double-charge the quotes+separator and truncate an exact-fit entry).
|
||||
// The merged entry costs two quotes, 58 characters, the escaped
|
||||
// newline, and one separator: 63 bytes against the 63-byte ledger.
|
||||
const { runtime } = await setup({ maxLogBytes: 64 })
|
||||
const result = await runtime.run(runtime.resolve({
|
||||
program: [
|
||||
"print('a' * 30, end='', flush=True)",
|
||||
"print('b' * 30)",
|
||||
"print('b' * 28)",
|
||||
'return "done"',
|
||||
].join('\n'),
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['a'.repeat(30) + 'b'.repeat(30)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['a'.repeat(30) + 'b'.repeat(28) + '\n'])
|
||||
}, 15_000)
|
||||
|
||||
it('does not over-reject an exact-fit closing line while an open entry accumulates', async () => {
|
||||
// The write-path pre-check's cheap bound used +3 (quotes + separator) even
|
||||
// while an open entry was accumulating, so an exact-fit merged TAIL was
|
||||
// truncated. The recipe below goes through the SCAN pre-check (the
|
||||
// newline-terminated write arrives with an empty pending buffer, so the
|
||||
// buffered-chunks branch is skipped): 'a'*29 flush bills 32 (ledger 31
|
||||
// left), then one write of 'b'*30 + newline merges 30 more chars whose
|
||||
// cheap bound is 30, not 33 — the +3 form saw 30 + 3 = 33 > 31, sliced to
|
||||
// a budget prefix, and pushed past the ledger, emitting the marker for a
|
||||
// line that fits (merged cost 2 + 59 + 1 = 62 <= 63).
|
||||
// A closing tail pays its content and escaped newline; the first
|
||||
// fragment already paid the quotes and separator.
|
||||
const { runtime } = await setup({ maxLogBytes: 64 })
|
||||
const result = await runtime.run(runtime.resolve({
|
||||
program: [
|
||||
'import sys',
|
||||
"sys.stdout.write('a' * 29)",
|
||||
'sys.stdout.flush()',
|
||||
"sys.stdout.write('b' * 30 + chr(10))",
|
||||
"sys.stdout.write('b' * 29 + chr(10))",
|
||||
'return "done"',
|
||||
].join('\n'),
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['a'.repeat(29) + 'b'.repeat(30)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['a'.repeat(29) + 'b'.repeat(29) + '\n'])
|
||||
}, 15_000)
|
||||
|
||||
|
||||
@@ -2363,7 +2371,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['a'.repeat(59), logTruncationMarker(64)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['a'.repeat(59), logTruncationMarker(64)])
|
||||
}, 15_000)
|
||||
|
||||
it('truncates when the closing frame of a merged entry overflows the budget', async () => {
|
||||
@@ -2381,7 +2389,7 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toEqual(['x'.repeat(40), logTruncationMarker(64)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['x'.repeat(40), logTruncationMarker(64)])
|
||||
}, 15_000)
|
||||
|
||||
it('keeps a float completion exact when the program mutates the decimal context', async () => {
|
||||
@@ -2486,8 +2494,8 @@ describe('PythonPtcRuntime — programs and bindings', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('ok')
|
||||
expect(result.logs).toContain('first')
|
||||
expect(result.logs).toContain('second')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toContain('first')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toContain('second')
|
||||
})
|
||||
|
||||
it('raises the declared errorClass with the member name on rejection', async () => {
|
||||
@@ -3472,7 +3480,7 @@ describe('PythonPtcRuntime — budgets, termination, disposal', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('escaped')
|
||||
expect(result.logs).toContain('leader-diagnostic-no-newline')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toContain('leader-diagnostic-no-newline')
|
||||
}, 8000)
|
||||
|
||||
it('closes the child stdin so a program read sees EOF instead of blocking', async () => {
|
||||
@@ -3868,7 +3876,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
const markers = result.logs.filter(line => line.includes('log capture truncated at 128 bytes'))
|
||||
const markers = result.output.filter(block => block.type === 'text').map(block => block.text).filter(line => line.includes('log capture truncated at 128 bytes'))
|
||||
expect(markers.length).toBeGreaterThanOrEqual(1)
|
||||
})
|
||||
|
||||
@@ -4445,7 +4453,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs.some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
}, 20_000)
|
||||
|
||||
it('stops host capture at the child ledger truncation, keeping exactly one marker', async () => {
|
||||
@@ -4472,11 +4480,11 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
const markers = result.logs.filter(line => line.includes('log capture truncated'))
|
||||
const markers = result.output.filter(block => block.type === 'text').map(block => block.text).filter(line => line.includes('log capture truncated'))
|
||||
expect(markers).toHaveLength(1)
|
||||
// The marker is the LAST entry: nothing was retained after truncation.
|
||||
expect(result.logs.at(-1)).toBe(markers[0])
|
||||
expect(result.logs.join('\n')).not.toContain('AFTER')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).at(-1)).toBe(markers[0])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).not.toContain('AFTER')
|
||||
}, 20_000)
|
||||
|
||||
it('keeps one marker when a program forges repeated truncation frames', async () => {
|
||||
@@ -4499,13 +4507,13 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
// The non-boolean flag did not truncate, so its text was captured normally.
|
||||
expect(result.logs).toContain('first')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toContain('first')
|
||||
// The first genuine flag stopped capture and emitted the HOST's own marker;
|
||||
// the frame's own text is discarded, so neither payload appears.
|
||||
expect(result.logs).not.toContain('MARK-A')
|
||||
expect(result.logs).not.toContain('MARK-B')
|
||||
expect(result.logs.at(-1)).toBe(logTruncationMarker(4096))
|
||||
expect(result.logs.filter(line => line.includes('log capture truncated'))).toHaveLength(1)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).not.toContain('MARK-A')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).not.toContain('MARK-B')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).at(-1)).toBe(logTruncationMarker(4096))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).filter(line => line.includes('log capture truncated'))).toHaveLength(1)
|
||||
}, 20_000)
|
||||
|
||||
it('discards the text of a forged truncation frame instead of retaining it', async () => {
|
||||
@@ -4529,8 +4537,8 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
expect(result.value).toBe('done')
|
||||
// Only the host marker is kept, so the total stays orders of magnitude below
|
||||
// what the forgery carried — and below the cap it was trying to escape.
|
||||
expect(result.logs).toEqual([logTruncationMarker(64)])
|
||||
expect(result.logs.join('').length).toBeLessThan(forgedBytes / 1000)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([logTruncationMarker(64)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('').length).toBeLessThan(forgedBytes / 1000)
|
||||
}, 30_000)
|
||||
|
||||
it('coalesces unframed fd-3 fragments without recopying the sealed prefix', async () => {
|
||||
@@ -4586,7 +4594,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs).toContain('after-trickle')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toContain('after-trickle\n')
|
||||
// Sealing appends a finished block rather than re-merging everything held, so
|
||||
// each byte is copied once. Re-concatenating the whole buffer at every
|
||||
// threshold made the cumulative copy volume quadratic — 10 MiB trickled a
|
||||
@@ -4653,9 +4661,9 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs.some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
// The retained text is bounded by the budget, not the 1 MB the program wrote.
|
||||
expect(result.logs.join('\n').length).toBeLessThan(4096)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('').length).toBeLessThan(4096)
|
||||
}, 20_000)
|
||||
|
||||
it('discards empty writes instead of buffering one list slot each', async () => {
|
||||
@@ -4678,7 +4686,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs).toEqual([])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([])
|
||||
}, 30_000)
|
||||
|
||||
it('stops scanning a single-write newline flood once the log ledger truncates', async () => {
|
||||
@@ -4694,7 +4702,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs.filter(line => line.includes('log capture truncated'))).toHaveLength(1)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).filter(line => line.includes('log capture truncated'))).toHaveLength(1)
|
||||
expect(Date.now() - start).toBeLessThan(8_000)
|
||||
}, 15_000)
|
||||
|
||||
@@ -4721,7 +4729,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs).toEqual([logTruncationMarker(256)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([logTruncationMarker(256)])
|
||||
}, 40_000)
|
||||
|
||||
it('bounds a newline-free write against the already-buffered chunks before joining them', async () => {
|
||||
@@ -4752,7 +4760,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs).toEqual([logTruncationMarker(256)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([logTruncationMarker(256)])
|
||||
}, 40_000)
|
||||
|
||||
it('bounds a newline-terminated write against the already-buffered chunks before joining them', async () => {
|
||||
@@ -4779,7 +4787,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs).toEqual([logTruncationMarker(256)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([logTruncationMarker(256)])
|
||||
}, 40_000)
|
||||
|
||||
it('emits pending text on an explicit flush, before the run can be killed', async () => {
|
||||
@@ -4799,59 +4807,39 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error?.kind).toBe('timeout')
|
||||
expect(result.logs).toContain('before hang')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toContain('before hang')
|
||||
}, 15_000)
|
||||
|
||||
it('marks a dropped tail when the ledger lands on exactly zero remaining', async () => {
|
||||
// One 100-character line costs 103 serialized bytes (quotes + separator),
|
||||
// consuming a 104-byte budget minus the 1-byte array-envelope reservation
|
||||
// (104 - 1 = 103) EXACTLY. Landing on zero never trips
|
||||
// LogBuffer's "cost > remaining" branch, so `_truncated` stays unset and the
|
||||
// stream's own `remaining > 0` guard silently discarded the unscanned tail —
|
||||
// the run reported a complete log while dropping text. The tail must be
|
||||
// pushed so the marker is emitted. (This surfaced only after empty writes
|
||||
// stopped being buffered: `print` issues a trailing `write("")` whose
|
||||
// buffered-empty path used to force the marker out incidentally.) A single
|
||||
// wide line is used rather than many narrow ones so the CHILD ledger is the
|
||||
// one that lands on zero: the host's identical ledger truncates first when
|
||||
// many small entries precede the long marker text. `["y"*100]` serializes to
|
||||
// exactly 104 bytes (103 payload + 1 envelope), so 104 is the smallest
|
||||
// budget that admits the entry.
|
||||
const { runtime } = await setup({ maxLogBytes: 104, maxWallMs: 10_000 })
|
||||
// A 100-character line plus its escaped newline consumes a 106-byte
|
||||
// serialized array exactly; an unscanned tail must still report truncation.
|
||||
const { runtime } = await setup({ maxLogBytes: 106, maxWallMs: 10_000 })
|
||||
const result = await runtime.run(runtime.resolve({
|
||||
program: ['print("y" * 100 + "\\n" + "z" * 10, end="")', 'return "done"'].join('\n'),
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs).toContain('y'.repeat(100))
|
||||
expect(result.logs.filter(line => line.includes('log capture truncated'))).toHaveLength(1)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toContain('y'.repeat(100) + '\n')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).filter(line => line.includes('log capture truncated'))).toHaveLength(1)
|
||||
// The dropped tail is not retained, but its loss is now reported.
|
||||
expect(result.logs.some(line => line.includes('z'))).toBe(false)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).some(line => line.includes('z'))).toBe(false)
|
||||
}, 15_000)
|
||||
|
||||
it('keeps an admitted log within the serialized array envelope at the exact limit', async () => {
|
||||
// Each entry is charged its JSON-string cost plus one separator byte, and
|
||||
// the serialized outer logs array adds one more byte of envelope (two
|
||||
// brackets and n-1 commas). The ledgers reserve that byte, so a result that
|
||||
// exactly exhausts the ledger still serializes within the configured cap.
|
||||
// At the 64-byte floor (the smallest admissible maxLogBytes): ledger 63,
|
||||
// a 60-character line serializes as `"aaa...a"` (62 bytes) + 1 separator
|
||||
// = 63, exactly exhausting the ledger and serializing as `["aaa...a"]`
|
||||
// = 64 = the cap; a 61-character line costs 64 > 63 and truncates. The
|
||||
// marker rides envelope, so the serialized logs run to cap + marker.
|
||||
// At the 64-byte floor, 58 characters and their escaped newline
|
||||
// serialize to exactly 64 bytes including the array envelope.
|
||||
const { runtime } = await setup({ maxLogBytes: 64, maxWallMs: 10_000 })
|
||||
const result = await runtime.run(runtime.resolve({
|
||||
program: ['print("a" * 60 + "\\n" + "b" * 61, end="")', 'return "done"'].join('\n'),
|
||||
program: ['print("a" * 58 + "\\n" + "b" * 61, end="")', 'return "done"'].join('\n'),
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
// The 60-character line was admitted; the 61-character line was not (a
|
||||
// single 'b' would also match the marker's "bytes", so check for the line).
|
||||
expect(result.logs).toContain('a'.repeat(60))
|
||||
expect(result.logs.some(line => line.includes('b'.repeat(61)))).toBe(false)
|
||||
expect(result.logs.some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
// Only the first complete line fits before the marker.
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toContain('a'.repeat(58) + '\n')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).some(line => line.includes('b'.repeat(61)))).toBe(false)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
}, 15_000)
|
||||
|
||||
it('rejects a log budget too small to serialize the truncation marker', async () => {
|
||||
@@ -4876,10 +4864,10 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
// Serialized (escaped) size of retained entries stays in the budget's
|
||||
// neighborhood: well under the ~30 kB an uncharged flood would retain.
|
||||
const serialized = Buffer.byteLength(JSON.stringify(result.logs), 'utf8')
|
||||
const serialized = Buffer.byteLength(JSON.stringify(result.output.filter(block => block.type === 'text').map(block => block.text)), 'utf8')
|
||||
expect(serialized).toBeLessThan(1024)
|
||||
})
|
||||
|
||||
@@ -4902,9 +4890,9 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs.filter(line => line.includes('log capture truncated'))).toHaveLength(1)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).filter(line => line.includes('log capture truncated'))).toHaveLength(1)
|
||||
// Nothing of the line itself was retained: the ledger refused the whole entry.
|
||||
expect(result.logs.every(line => !line.includes(String.fromCharCode(0)))).toBe(true)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).every(line => !line.includes(String.fromCharCode(0)))).toBe(true)
|
||||
}, 30_000)
|
||||
|
||||
it('bounds a huge unterminated tail after an early newline without copying it whole', async () => {
|
||||
@@ -4932,7 +4920,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs.some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
}, 30_000)
|
||||
|
||||
it('flushes logs before framing the value so their peaks do not add against RLIMIT_AS', async () => {
|
||||
@@ -5367,8 +5355,8 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.length).toBeLessThanOrEqual(65)
|
||||
expect(result.logs.some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).length).toBeLessThanOrEqual(65)
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).some(line => line.includes('log capture truncated'))).toBe(true)
|
||||
})
|
||||
|
||||
it('reassembles multibyte UTF-8 split across stray-output pipe chunks', async () => {
|
||||
@@ -5398,7 +5386,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
const text = result.logs.join('')
|
||||
const text = result.output.filter(block => block.type === 'text').map(block => block.text).join('')
|
||||
expect(text).toContain('\u0900\u4f60\u597d\u{1f600}\u{10ffff}')
|
||||
expect(text).not.toContain('\ufffd')
|
||||
})
|
||||
@@ -5420,7 +5408,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.join('')).toContain('�')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).toContain('�')
|
||||
})
|
||||
|
||||
it('rejects reserved words of EITHER backend language as binding globals', async () => {
|
||||
@@ -5451,8 +5439,8 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.join('')).toContain('stray stdout')
|
||||
expect(result.logs.join('')).toContain('stray stderr')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).toContain('stray stdout')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).toContain('stray stderr')
|
||||
})
|
||||
|
||||
it('flushes bytes written through sys.__stdout__/sys.__stderr__ before the done frame', async () => {
|
||||
@@ -5482,8 +5470,8 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs.join('')).toContain('orig stdout')
|
||||
expect(result.logs.join('')).toContain('orig stderr')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).toContain('orig stdout')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).join('')).toContain('orig stderr')
|
||||
}, 15_000)
|
||||
|
||||
it('escalates to SIGKILL when the program traps SIGTERM and ignores the grace period', async () => {
|
||||
@@ -5576,7 +5564,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('settled')
|
||||
// The frame was dropped as one truncation marker, not retained.
|
||||
expect(result.logs).toEqual([logTruncationMarker(128)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([logTruncationMarker(128)])
|
||||
// The escaped copy (~144 MiB) was never materialized.
|
||||
expect(process.memoryUsage().heapUsed - before).toBeLessThan(256 * 1024 * 1024)
|
||||
}, 90_000)
|
||||
@@ -5597,7 +5585,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('settled')
|
||||
expect(result.logs).toEqual([logTruncationMarker(64)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual([logTruncationMarker(64)])
|
||||
}, 8000)
|
||||
|
||||
it('caps a forged done error.message from its code-unit prefix, never encoding the whole message', async () => {
|
||||
@@ -5834,7 +5822,7 @@ describe('PythonPtcRuntime — hostile peer', () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs).toContain('after-cap-frames')
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toContain('after-cap-frames')
|
||||
}, 120_000)
|
||||
|
||||
it('rejects an oversized first frame that lands on the sealing threshold with a newline', async () => {
|
||||
|
||||
@@ -45,7 +45,7 @@ it('seals stray fragments without recopying the sealed prefix', async () => {
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs).toEqual(['x'.repeat(60_000)])
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text)).toEqual(['x'.repeat(60_000) + '\n'])
|
||||
// Sealing copies each byte at most twice; merging every accumulated prefix
|
||||
// instead copies over a megabyte for these 60,001 controlled fragments.
|
||||
expect(maxFragments).toBeLessThanOrEqual(1024)
|
||||
@@ -75,7 +75,7 @@ it.each([
|
||||
bindings: [],
|
||||
}))
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.at(-1)).toBe(logTruncationMarker(3072))
|
||||
expect(result.output.filter(block => block.type === 'text').map(block => block.text).at(-1)).toBe(logTruncationMarker(3072))
|
||||
// Each raw byte decodes to U+FFFD (three UTF-8 bytes), so a 3072-byte
|
||||
// budget flushes near 1024 raw bytes. Charging raw or structural widths
|
||||
// instead retains over 2048 bytes before flushing these payloads.
|
||||
|
||||
@@ -8301,17 +8301,21 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
name: 'PtcJsonValue',
|
||||
declaration: 'export type PtcJsonValue = null | boolean | number | string | PtcJsonValue[] | {\n [key: string]: PtcJsonValue;\n};',
|
||||
},
|
||||
{
|
||||
name: 'PtcOutput',
|
||||
declaration: 'export type PtcOutput = {\n type: \'text\';\n text: string;\n} | {\n type: \'image\';\n attachment: ImageAttachmentRef;\n};',
|
||||
},
|
||||
{
|
||||
name: 'PtcRunFailure',
|
||||
declaration: 'export interface PtcRunFailure {\n kind: \'exception\' | \'timeout\' | \'abort\' | \'worker-exit\' | \'invalid-output\' | \'output-limit\' | \'protocol\' | \'sandbox-unavailable\';\n message: string;\n}',
|
||||
},
|
||||
{
|
||||
name: 'PtcRunRequest',
|
||||
declaration: 'export interface PtcRunRequest {\n program: string;\n stateOwner?: object;\n bindings: PtcBindingNamespace[];\n cwd?: string;\n timeoutMs?: number | null;\n sandboxPolicy?: SandboxExecutionPolicy;\n signal?: AbortSignal;\n}',
|
||||
declaration: 'export interface PtcRunRequest {\n program: string;\n discardCompletion?: boolean;\n stateOwner?: object;\n bindings: PtcBindingNamespace[];\n cwd?: string;\n timeoutMs?: number | null;\n sandboxPolicy?: SandboxExecutionPolicy;\n signal?: AbortSignal;\n}',
|
||||
},
|
||||
{
|
||||
name: 'PtcRunResult',
|
||||
declaration: 'export interface PtcRunResult {\n sandbox?: PtcRunSandbox;\n value?: PtcJsonValue;\n logs: string[];\n error?: PtcRunFailure;\n}',
|
||||
declaration: 'export interface PtcRunResult {\n sandbox?: PtcRunSandbox;\n value?: PtcJsonValue;\n output: PtcOutput[];\n error?: PtcRunFailure;\n}',
|
||||
},
|
||||
{
|
||||
name: 'PtcRunSandbox',
|
||||
|
||||
@@ -163,7 +163,7 @@ class PatchPtcRuntime extends PtcRuntime {
|
||||
return { ...request, cwd: request.cwd ?? process.cwd(), timeoutMs: request.timeoutMs ?? 120_000 }
|
||||
}
|
||||
async run(request: PtcRunSpec) {
|
||||
return { logs: [], value: await request.bindings[0]!.functions.apply_patch!({ input: this.patch }) }
|
||||
return { output: [{ type: 'text' as const, text: JSON.stringify(await request.bindings[0]!.functions.apply_patch!({ input: this.patch })) }] }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -179,10 +179,10 @@ it('dispatches native and PTC patch calls with identical values', async () => {
|
||||
await t.file('a', 'old\n')
|
||||
const ptc = await executeModelToolFixture(p.ctx, {
|
||||
signal: p.exec.signal, callId: p.exec.callId, agent: { ctx: p.ctx, session: { header: { cwd: t.workspace }, append() {} } } as never,
|
||||
name: 'run_code', arguments: { code: 'return await tools.apply_patch({patch})', description: 'Apply the fixture patch' },
|
||||
name: 'run_code', arguments: { code: 'inspect(JSON.stringify(await tools.apply_patch({patch})))', description: 'Apply the fixture patch' },
|
||||
})
|
||||
if (ptc.isError) throw new Error(JSON.stringify(ptc))
|
||||
expect(ptc.value).toEqual({ logs: [], result: native.value })
|
||||
expect(ptc.value).toEqual({ logs: [JSON.stringify(native.value)] })
|
||||
expect(await t.bytes('a')).toBe('new\n')
|
||||
})
|
||||
|
||||
|
||||
@@ -640,7 +640,7 @@ describe('current working directory', () => {
|
||||
if (read === undefined) throw new Error('missing read binding')
|
||||
const value = await read({ file_path: 'note.txt' })
|
||||
expect(JSON.stringify(value)).toContain('current')
|
||||
return { logs: [], value }
|
||||
return { output: [], value }
|
||||
}
|
||||
}
|
||||
await ctx.plugin(ReadBindingRuntime)
|
||||
|
||||
@@ -12,16 +12,17 @@ import * as NativeToolExecutor from '@deepseek-ai/dsh-tool-executor-native'
|
||||
import * as PtcToolExecutor from '@deepseek-ai/dsh-tool-executor-ptc'
|
||||
import { provideWorkingDirectoryFixture } from '@deepseek-ai/dsh-agent-loop-testkit'
|
||||
import { afterEach, beforeEach, describe, expect, it, onTestFinished } from 'vitest'
|
||||
import { mkdir, mkdtemp, realpath, rm, writeFile } from 'node:fs/promises'
|
||||
import { mkdir, mkdtemp, readFile, realpath, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import { PtcRuntime } from '@deepseek-ai/dsh-ptc-runtime'
|
||||
import { PtcImage, PtcRuntime } from '@deepseek-ai/dsh-ptc-runtime'
|
||||
import type { PtcRunRequest, PtcRunResult } from '@deepseek-ai/dsh-ptc-runtime'
|
||||
import { ToolCallId, LlmAdapter, LlmRuntime } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelInfo, LlmResolvedModelInfo, Message, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRuntime, { RUN_CODE_NAME } from '@deepseek-ai/dsh-tools'
|
||||
import type { PostToolDecision } from '@deepseek-ai/dsh-tools'
|
||||
import LocalFileSystem from '@deepseek-ai/dsh-fs-local'
|
||||
import * as FsPolicy from '@deepseek-ai/dsh-fs-observation-policy'
|
||||
import LocalAttachmentStore from '@deepseek-ai/dsh-attachment-local'
|
||||
@@ -80,7 +81,7 @@ class FakeRuntime extends PtcRuntime {
|
||||
|
||||
readonly language = 'typescript'
|
||||
readonly isolation = 'fake'
|
||||
behavior: (request: PtcRunRequest) => Promise<PtcRunResult> = () => Promise.resolve({ logs: [] })
|
||||
behavior: (request: PtcRunRequest) => Promise<PtcRunResult> = () => Promise.resolve({ output: [] })
|
||||
|
||||
run(request: PtcRunRequest): Promise<PtcRunResult> {
|
||||
return this.behavior(request)
|
||||
@@ -297,31 +298,100 @@ describe('read_image happy path', () => {
|
||||
expect(result.isError).toBe(false)
|
||||
})
|
||||
|
||||
it('forwards a nested PTC mode image through the outer run_code context', async () => {
|
||||
it('displays the selected program image among captured text', async () => {
|
||||
await writeFile(join(dir, 'red.png'), PNG_1X1)
|
||||
const ctx = await setup({ toolMode: 'ptc' })
|
||||
const runtime = ctx.ptcRuntime as FakeRuntime
|
||||
runtime.behavior = async (request) => {
|
||||
const value = await request.bindings[0]!.functions.read_image!({ file_path: 'red.png' })
|
||||
return { logs: [], value }
|
||||
expect(value).toBeInstanceOf(PtcImage)
|
||||
if (!(value instanceof PtcImage)) throw new Error('read_image must return an Image')
|
||||
expect(await readFile(value.path)).toEqual(PNG_1X1)
|
||||
expect([value.attachment.width, value.attachment.height]).toEqual([1, 1])
|
||||
return { output: [{ type: 'text', text: 'before' }, { type: 'image', attachment: value.attachment }, { type: 'text', text: 'after' }] }
|
||||
}
|
||||
|
||||
const result = await call(ctx, RUN_CODE_NAME, {
|
||||
code: 'return await tools.read_image({ file_path: "red.png" })',
|
||||
code: 'inspect("before", await tools.read_image({ file_path: "red.png" }), "after")',
|
||||
description: 'Read the image through PTC mode',
|
||||
}, { ...agentOn('vision-model'), ctx })
|
||||
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.content.every(block => block.type === 'text')).toBe(true)
|
||||
expect(result.additionalContexts).toHaveLength(1)
|
||||
const forwarded = result.additionalContexts?.[0]?.content
|
||||
expect(forwarded).toHaveLength(2)
|
||||
expect(forwarded?.[0]?.type).toBe('text')
|
||||
expect(forwarded?.[0]?.type === 'text' ? forwarded[0].text : '').toContain('<type>image</type>')
|
||||
expect(forwarded?.[1]).toMatchObject({
|
||||
type: 'image',
|
||||
attachment: { mediaType: 'image/png', width: 1, height: 1 },
|
||||
expect(result.additionalContexts).toBeUndefined()
|
||||
expect(result.content).toMatchObject([
|
||||
{ type: 'text', text: 'before' },
|
||||
{ type: 'image', attachment: { mediaType: 'image/png', width: 1, height: 1 } },
|
||||
{ type: 'text', text: 'after' },
|
||||
])
|
||||
})
|
||||
|
||||
it.each([
|
||||
['around', 'block', 'success'], ['around', 'block', 'failure'],
|
||||
['around', 'redact', 'failure'], ['around', 'replace', 'success'], ['around', 'recover', 'failure'],
|
||||
['post', 'block', 'success'], ['post', 'block', 'failure'], ['post', 'replace', 'success'],
|
||||
] as const)('honors an outer %s %s decision after program %s with inspected images', async (stage, decision, outcome) => {
|
||||
await writeFile(join(dir, 'red.png'), PNG_1X1)
|
||||
const ctx = await setup({ toolMode: 'ptc' })
|
||||
const runtime = ctx.ptcRuntime as FakeRuntime
|
||||
runtime.behavior = async (request) => {
|
||||
const image = await request.bindings[0]!.functions.read_image!({ file_path: 'red.png' })
|
||||
if (!(image instanceof PtcImage)) throw new Error('read_image must return an Image')
|
||||
return {
|
||||
output: [{ type: 'image', attachment: image.attachment }],
|
||||
...outcome === 'failure' ? { error: { kind: 'exception' as const, message: 'program stopped' } } : {},
|
||||
}
|
||||
}
|
||||
if (stage === 'around') {
|
||||
ctx.on('tools/execute', async (exec, next) => {
|
||||
const result = await next()
|
||||
if (exec.name !== RUN_CODE_NAME) return result
|
||||
if (decision === 'redact') return { ...result, content: [{ type: 'text', text: 'image blocked' }] }
|
||||
return decision === 'block'
|
||||
? { isError: true, error: { message: 'image blocked' }, content: [{ type: 'text', text: 'image blocked' }] }
|
||||
: { isError: false, value: { logs: ['image replaced'] }, content: [{ type: 'text', text: 'image replaced' }] }
|
||||
})
|
||||
} else {
|
||||
ctx.on('tools/post-execute', (exec, _result, next): Promise<PostToolDecision> => {
|
||||
if (exec.name !== RUN_CODE_NAME) return next()
|
||||
return Promise.resolve(decision === 'block'
|
||||
? { kind: 'block', feedback: [{ type: 'text', text: 'image blocked' }] }
|
||||
: { kind: 'accept', value: { logs: ['image replaced'] } })
|
||||
})
|
||||
}
|
||||
|
||||
const result = await call(ctx, RUN_CODE_NAME, {
|
||||
code: 'inspect(await tools.read_image({ file_path: "red.png" }))', description: 'Inspect an image',
|
||||
}, { ...agentOn('vision-model'), ctx })
|
||||
|
||||
const blocked = decision === 'block' || decision === 'redact'
|
||||
expect(result.isError).toBe(blocked)
|
||||
expect(result.content).toEqual([{ type: 'text', text: blocked ? 'image blocked' : 'image replaced' }])
|
||||
})
|
||||
|
||||
it.each(['exception', 'timeout'] as const)('preserves inspected text and image order after an %s', async (kind) => {
|
||||
await writeFile(join(dir, 'red.png'), PNG_1X1)
|
||||
const ctx = await setup({ toolMode: 'ptc' })
|
||||
const runtime = ctx.ptcRuntime as FakeRuntime
|
||||
runtime.behavior = async (request) => {
|
||||
const image = await request.bindings[0]!.functions.read_image!({ file_path: 'red.png' })
|
||||
if (!(image instanceof PtcImage)) throw new Error('read_image must return an Image')
|
||||
return {
|
||||
output: [{ type: 'text', text: 'before' }, { type: 'image', attachment: image.attachment }, { type: 'text', text: 'after' }],
|
||||
error: { kind, message: 'program stopped' },
|
||||
}
|
||||
}
|
||||
const result = await call(ctx, RUN_CODE_NAME, { code: 'program', description: 'Inspect before stopping' }, {
|
||||
...agentOn('vision-model'), ctx,
|
||||
})
|
||||
expect(result.isError).toBe(true)
|
||||
const diagnostic = result.content[0]
|
||||
if (diagnostic?.type !== 'text') throw new Error('Expected a separate failure diagnostic')
|
||||
expect(diagnostic.text).toContain(`code run failed (${kind}): program stopped`)
|
||||
expect(result.content.slice(1)).toMatchObject([
|
||||
{ type: 'text', text: 'before' },
|
||||
{ type: 'image', attachment: { mediaType: 'image/png', width: 1, height: 1 } },
|
||||
{ type: 'text', text: 'after' },
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
@@ -1225,7 +1225,7 @@ class GuidancePtcRuntime extends PtcRuntime {
|
||||
|
||||
readonly language = 'typescript'
|
||||
readonly isolation = 'fake'
|
||||
run() { return Promise.resolve({ logs: [] }) }
|
||||
run() { return Promise.resolve({ output: [] }) }
|
||||
}
|
||||
|
||||
describe('scope-aware PTC guidance', () => {
|
||||
|
||||
@@ -18,7 +18,7 @@ class BindingProbe extends PtcRuntime {
|
||||
readonly language = 'typescript'
|
||||
readonly isolation = 'in-process-test'
|
||||
behavior: (spec: PtcRunSpec) => Promise<PtcRunResult> = async spec => ({
|
||||
logs: [], value: Object.keys(spec.bindings[0]!.functions).sort(),
|
||||
output: [{ type: 'text', text: JSON.stringify(Object.keys(spec.bindings[0]!.functions).sort()) }],
|
||||
})
|
||||
|
||||
resolve(request: PtcRunRequest): PtcRunSpec {
|
||||
@@ -81,7 +81,7 @@ describe('MCP resource tools', () => {
|
||||
}
|
||||
if (mode !== 'native') {
|
||||
expect(await call(ctx, 'run_code', { code: 'inspect bindings', description: 'Inspect bindings' }))
|
||||
.toMatchObject({ isError: false, value: { result: [] } })
|
||||
.toMatchObject({ isError: false, value: { logs: ['[]'] } })
|
||||
}
|
||||
})
|
||||
|
||||
@@ -213,15 +213,15 @@ describe('MCP resource tools', () => {
|
||||
expect(await binding({ server: 'docs' })).toEqual({ resources: [] })
|
||||
remove()
|
||||
await expect(binding({ server: 'docs' })).rejects.toThrow('unknown tool')
|
||||
return { logs: [] }
|
||||
return { output: [] }
|
||||
}
|
||||
const result = await call(ctx, 'run_code', { code: 'invoke and remove provider', description: 'Use resource binding' })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(request).toHaveBeenCalledOnce()
|
||||
expect(renderPrompt(await ctx.systemPrompt.assemble())).not.toContain('mcp')
|
||||
runtime.behavior = async spec => ({ logs: [], value: Object.keys(spec.bindings[0]!.functions) })
|
||||
runtime.behavior = async spec => ({ output: [{ type: 'text', text: JSON.stringify(Object.keys(spec.bindings[0]!.functions)) }] })
|
||||
expect(await call(ctx, 'run_code', { code: 'inspect bindings', description: 'Inspect bindings' }))
|
||||
.toMatchObject({ isError: false, value: { result: [] } })
|
||||
.toMatchObject({ isError: false, value: { logs: ['[]'] } })
|
||||
})
|
||||
|
||||
it('retains tools for a configured provider while its requests fail', async () => {
|
||||
|
||||
@@ -503,7 +503,7 @@ describe('the soft layer', () => {
|
||||
|
||||
readonly language = 'typescript'
|
||||
readonly isolation = 'fake'
|
||||
run(_request: PtcRunRequest): Promise<PtcRunResult> { return Promise.resolve({ logs: [] }) }
|
||||
run(_request: PtcRunRequest): Promise<PtcRunResult> { return Promise.resolve({ output: [] }) }
|
||||
}
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
@@ -528,7 +528,7 @@ describe('the soft layer', () => {
|
||||
|
||||
readonly language = 'typescript'
|
||||
readonly isolation = 'fake'
|
||||
run(_request: PtcRunRequest): Promise<PtcRunResult> { return Promise.resolve({ logs: [] }) }
|
||||
run(_request: PtcRunRequest): Promise<PtcRunResult> { return Promise.resolve({ output: [] }) }
|
||||
}
|
||||
const withPlanMode = new Context()
|
||||
await withPlanMode.plugin(SystemPrompt)
|
||||
@@ -944,7 +944,7 @@ describe('exit_plan_mode', () => {
|
||||
async run(request: PtcRunRequest): Promise<PtcRunResult> {
|
||||
const exit = request.bindings[0]?.functions[EXIT_PLAN_MODE]
|
||||
if (exit === undefined) throw new Error('missing exit_plan_mode binding')
|
||||
return { logs: [], value: await exit({ plan }) }
|
||||
return { output: [], value: await exit({ plan }) }
|
||||
}
|
||||
}
|
||||
const ctx = new Context()
|
||||
|
||||
@@ -85,9 +85,10 @@ class ProgramFixture extends PtcRuntime {
|
||||
}
|
||||
|
||||
async run(request: PtcRunRequest): Promise<PtcRunResult> {
|
||||
expect(request.program).toBe('return await tools.echo({ value: "ptc" })')
|
||||
expect(request.program).toBe('inspect(await tools.echo({ value: "ptc" }))')
|
||||
const value = await request.bindings[0]!.functions.echo!({ value: 'ptc' })
|
||||
return { logs: [], value }
|
||||
if (typeof value !== 'string') throw new Error('expected a string echo result')
|
||||
return { output: [{ type: 'text', text: value }] }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -140,7 +141,7 @@ it.each([true, false])('loads cordis.yml executor presets with runtime availabil
|
||||
})).toMatchObject({ isError: false, value: 'echo:native' })
|
||||
expect(await executeModelToolFixture(ctx, {
|
||||
agent: ptc, name: 'run_code', callId: ToolCallId('program'), signal,
|
||||
arguments: { code: 'return await tools.echo({ value: "ptc" })', description: 'Echo through the program' },
|
||||
arguments: { code: 'inspect(await tools.echo({ value: "ptc" }))', description: 'Echo through the program' },
|
||||
})).toMatchObject({ isError: false, content: [{ type: 'text', text: 'echo:ptc' }] })
|
||||
expect(ptc.session.snapshotEvents().find(event => event.type === 'tool/ptc-dispatch')).toMatchObject({
|
||||
data: { name: 'echo', isError: false, content: [{ type: 'text', text: 'echo:ptc' }] },
|
||||
|
||||
@@ -15,23 +15,23 @@
|
||||
en: d152484eb41ac6b4
|
||||
zh: 725b47765ee8f30b
|
||||
/deepseek-ai-dsh-ptc-runtime-node/use-this-package:
|
||||
en: 9c742ab3175c4f80
|
||||
zh: 5fa7d1ce9c3920d2
|
||||
en: 70e83ccdf8159a10
|
||||
zh: 8282f1da5e844426
|
||||
/deepseek-ai-dsh-ptc-runtime-node/use-this-package/configuration:
|
||||
en: 8d42f921a1fa8e5c
|
||||
zh: 27b79ec4ec52b3fd
|
||||
/deepseek-ai-dsh-ptc-runtime-node/use-this-package/execution-and-results:
|
||||
en: 2a9fa3292bf07355
|
||||
zh: db0b5684820915eb
|
||||
en: b1a7149f2981cf2e
|
||||
zh: a218e2ab9a23465c
|
||||
/deepseek-ai-dsh-ptc-runtime-node/use-this-package/in-memory-modes:
|
||||
en: 2de0a876701fc2c2
|
||||
zh: 330ca97a69528872
|
||||
en: 7b221a8976e48d43
|
||||
zh: df61037a5b767275
|
||||
/deepseek-ai-dsh-ptc-runtime-node/use-this-package/deadlines-and-cancellation:
|
||||
en: d383ec7aeab8afec
|
||||
zh: 7c9c49e205fbbcd8
|
||||
/deepseek-ai-dsh-ptc-runtime-node/use-this-package/failures:
|
||||
en: 34a221d0f26b8e47
|
||||
zh: 2fd20ac9c9953c7f
|
||||
en: 74411c07b1484746
|
||||
zh: 65b826012219bbec
|
||||
/deepseek-ai-dsh-ptc-runtime-node/understand-the-implementation:
|
||||
en: 8a9dddeafde30b6d
|
||||
zh: d2c95d923f892cc8
|
||||
@@ -57,8 +57,8 @@
|
||||
en: 40294c1c31b74824
|
||||
zh: d91e7b78c0619c7e
|
||||
/deepseek-ai-dsh-ptc-runtime-node/known-limitations-and-deferred-work:
|
||||
en: 7f688098214c7c72
|
||||
zh: de06cbba4da172c5
|
||||
en: e98734a36f5afb88
|
||||
zh: a36cb3c71b7e4f24
|
||||
/deepseek-ai-dsh-ptc-runtime-node/known-limitations-and-deferred-work/dev-note:
|
||||
en: 6373aba4e08f7205
|
||||
zh: 2a8c33d1f2839df4
|
||||
@@ -27,6 +27,8 @@ Execute model-written TypeScript with host-provided async functions. The default
|
||||
|
||||
Mount this provider in a composition that supplies `fs`, `subprocess`, `sandbox` and `sandboxPolicy`. `dsh-tool-executor-ptc` supplies the calling Session's directory and standing policy; direct runtime consumers resolve those options before execution.
|
||||
|
||||
Programs display text, objects, and `Image` values through top-level `inspect(...values)`. One call can mix multiple texts and images, including images nested in arrays and objects. `Image.attachmentId` matches the attachment ID in the model's image description. `Image.path` names a stored attachment file readable in the execution environment; `width` and `height` describe that file’s pixels. Image descriptors share the `maxOutputBytes` budget with text.
|
||||
|
||||
### Configuration
|
||||
|
||||
Configure the provider row after its required services are available:
|
||||
@@ -65,11 +67,17 @@ Profile editors can import `resolvePersistentExecution` from `@deepseek-ai/dsh-p
|
||||
|
||||
### Execution and results
|
||||
|
||||
In `per-call` mode, programs are async function bodies: top-level `await` and `return` work, and only erasable TypeScript is accepted. Node globals remain available, and a function body may declare a local `process`, including `const process = await import("node:process")`. A successful call returns its lossless-JSON value as `result.value` and captured text as `result.logs`. `result.sandbox` reports the selected mode, observed denial and the backend's full or partial enforcement independently of the program outcome.
|
||||
In `per-call` mode, programs are async function bodies: top-level `await` and `return` work, and only erasable TypeScript is accepted. Node globals remain available, and a function body may declare a local `process`, including `const process = await import("node:process")`. A successful call returns its lossless-JSON value as `result.value` and captured text and images as ordered `result.output` entries. `result.sandbox` reports the selected mode, observed denial and the backend's full or partial enforcement independently of the program outcome.
|
||||
|
||||
`inspect(...values)` renders strings directly and other values with Node's `util.inspect`, without depth, array-length or string-length preview limits. Undefined properties, Error objects and circular references remain visible. Registered Images are emitted directly or when Node's formatting reaches them in nested values, including Map entries, Set values and enumerable Symbol properties. Custom inspection methods determine which nested values are visited. Text and selected images follow argument order within the shared output budget. If object formatting throws, preceding inspected arguments remain visible. Each emitted text group ends in a newline; raw stream writes retain their original text.
|
||||
|
||||
All execution modes provide the complete native Node console, whose stdout and stderr writes enter the captured output. Model instructions say: "Use inspect(...values) to display output. Do not use console."
|
||||
|
||||
`run_code` sets `discardCompletion: true`: function return values and final expression values are omitted without JSON serialization, while inspected output remains visible. Direct runtime consumers receive the usual lossless-JSON completion unless they set this option.
|
||||
|
||||
Direct filesystem, network and subprocess operations remain Node operations, subject to the selected OS sandbox. Nested host bindings cross the control channel; PTC tool calls retain the registry's visibility, ordering, logging and approval rules. Running a program does not change the Session's standing policy or automatically replay it after a denial.
|
||||
|
||||
Set `executionMode: vm-script` to evaluate erasable TypeScript through `node:vm` Script. Top-level declarations and `await` work, for example `var fs = await import("node:fs/promises"); await fs.readdir(".")`. Top-level `return` and static `import`/`export` are not supported. The runtime awaits the final expression and returns its lossless JSON value; `console.log(...)` also emits output. Relative dynamic imports resolve from the supplied working directory. Script syntax is non-strict unless the program opts into strict mode; the context is not a security boundary, and Node access remains subject to the process sandbox.
|
||||
Set `executionMode: vm-script` to evaluate erasable TypeScript through `node:vm` Script. Top-level declarations and `await` work, for example `var fs = await import("node:fs/promises"); inspect(await fs.readdir("."))`. Top-level `return` and static `import`/`export` are not supported. The runtime awaits the final expression; its lossless JSON value is returned unless `discardCompletion` is enabled. Relative dynamic imports resolve from the supplied working directory. Script syntax is non-strict unless the program opts into strict mode; the context is not a security boundary, and Node access remains subject to the process sandbox.
|
||||
|
||||
`scriptPersistence: false` is the default: every call uses a fresh process and context, ignoring `stateOwner`. Set it to `true` to retain a context per owner under the lifecycle rules below. Top-level declarations, functions and object identities survive subsequent calls, including values initialized through top-level `await`. Function locals remain local. This option requires `executionMode: vm-script`; both settings accept the same syntax.
|
||||
|
||||
@@ -93,7 +101,7 @@ The `persistent-in-memory` mode and Script mode with `scriptPersistence: true` r
|
||||
|
||||
Save the patch and pass its path to `dsh headless --patch <path>` in a composition with PTC tool presentation. The `per-call` mode and Script mode with `scriptPersistence: false` retain no program memory between calls. Changing the configuration reloads the provider and releases its execution resources. Packaged single-executable hosts use their embedded carrier by default; the host sends the selected program format and retention flag with each evaluation, so the embedded `process.js` bootstrap serves both per-call and retained execution. Remote script deployments use a `node-script` launch with `launch.bootstrapPath` pointing to this package's built `persistent-process.js`.
|
||||
|
||||
In REPL mode, each live Agent keeps its own process and top-level variables, functions, and imported modules across `run_code` calls and turns. Use `await import(...)`, `console.log(...)`, and a final JSON expression; top-level `return` is not supported. Assignments to undeclared names create retained globals under Node REPL’s non-strict evaluation. `_` and `_error` are ordinary retained globals, without interactive REPL result setters. Node REPL declaration rules apply, including errors on redeclaring existing `let`, `const`, or `class` names; assign to a mutable binding or choose a new name. A leading `{ ... }` is a statement block and can contain top-level await and temporary declarations; parenthesize an object expression to return it. Retained functions and saved tool methods call the current evaluation's bindings. All tool calls must be awaited; callbacks from a completed evaluation cannot invoke tools in a later evaluation.
|
||||
In REPL mode, each live Agent keeps its own process and top-level variables, functions, and imported modules across `run_code` calls and turns. Use `await import(...)` and `inspect(...)`; top-level `return` is not supported. Assignments to undeclared names create retained globals under Node REPL’s non-strict evaluation. `_` and `_error` are ordinary retained globals, without interactive REPL result setters. Node REPL declaration rules apply, including errors on redeclaring existing `let`, `const`, or `class` names; assign to a mutable binding or choose a new name. A leading `{ ... }` is a statement block and can contain top-level await and temporary declarations; parenthesize an object expression to return it. Retained functions and saved tool methods call the current evaluation's bindings. All tool calls must be awaited; callbacks from a completed evaluation cannot invoke tools in a later evaluation.
|
||||
|
||||
In `persistent-in-memory` mode, call `registerEnvironmentInitializer(program)` before the first run is admitted to install REPL source that executes once per new environment. The runtime installs current bindings first and evaluates initialization separately from the user program, sharing the first call's deadline, cancellation and output limits. It discards the completion value and includes printed output in the result. New owners, owner-less execution, release and process loss create environments that run initialization again; ordinary calls reuse initialized state. Assigning to an initialized global does not trigger initialization. Initialization lines do not shift user-program locations. Other modes reject registration.
|
||||
|
||||
@@ -113,7 +121,7 @@ Omitting `timeoutMs` uses the configured elapsed default; numeric requests are v
|
||||
|
||||
### Failures
|
||||
|
||||
Program parse errors and thrown exceptions are `exception`; deadline expiry is `timeout`; cancellation is `abort`; unparsable or oversized control traffic and invalid program messages are `protocol`; unavailable confinement is `sandbox-unavailable`; early process exit, lost control transport or failed managed cleanup is `worker-exit`. Control I/O failures retain the transport diagnostic alongside the observed process exit status. The substrate-independent failure name remains `worker-exit` for process providers. Lossy completions are `invalid-output`, and an oversized outer result is `output-limit`, retaining the fitting log prefix. Invalid or unsupported options and calls after disposal reject as caller misuse.
|
||||
Program parse errors and thrown exceptions are `exception`; deadline expiry is `timeout`; cancellation is `abort`; unparsable or oversized control traffic and invalid program messages are `protocol`; unavailable confinement is `sandbox-unavailable`; early process exit, lost control transport or failed managed cleanup is `worker-exit`. Control I/O failures retain the transport diagnostic alongside the observed process exit status. The substrate-independent failure name remains `worker-exit` for process providers. When completion capture is enabled, lossy values are `invalid-output`, and an oversized outer result is `output-limit`, retaining a fitting ordered text and image prefix alongside the diagnostic. Invalid or unsupported options and calls after disposal reject as caller misuse.
|
||||
|
||||
-----
|
||||
|
||||
@@ -228,9 +236,10 @@ These limits qualify the execution guarantees and retained output.
|
||||
- **Script values can cross realms** — binding results, binding rejection classes and imported modules can use constructors from outside the Script context. `instanceof Array` or `instanceof Error` can be false for those values; use `Array.isArray` and the declared binding error class.
|
||||
- **In-memory environments retain one process per live owner** — idle owners keep their process and heap until explicit release, owner disposal or failure. Fan-out can retain many processes; idle expiration remains deferred.
|
||||
- **Output caps reject rather than retain every byte** — spill can preserve only the bounded result delivered by this provider.
|
||||
- **Formatting allocates before output accounting** — complete object and TypedArray formatting can require much more memory than the retained output. The output budget does not cap these temporary allocations; exhausting the configured heap can end the run with `worker-exit`.
|
||||
- **Bindings are bounded at transport admission** — control limits do not bound the memory a host binding allocates while producing its result.
|
||||
- **Preinstalled bootstraps require a compatible protocol revision** — readiness rejects a missing or mismatched revision before dispatching code. Upgrade `launch.bootstrapPath` with the host; the runtime does not install remote bootstrap updates.
|
||||
- **The console shim has five methods** — `log`, `info`, `warn`, `error` and `debug`.
|
||||
- **Image references live with the process** — every acquired image retains host metadata until the process terminates or its owner is released, including images that the program never displays. Long capture loops in retained environments accumulate these references.
|
||||
|
||||
<a id="dev-note"></a>
|
||||
### Dev Note
|
||||
|
||||
Loaded 100 of 246 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user