merge: preserve CUA selection fixes with AX traversal limits
This commit is contained in:
Dudu-0223 committed
commit
d4100b4d3f
46 files changed
+303
-85
No files matched your search
@@ -0,0 +1,19 @@
|
||||
# Bilingual-pair consistency record for 2026-11-09-cua-ax-traversal-defaults.md (docs/i18n/README.md): per heading
|
||||
# section, a hash of its English and Chinese blocks outside code blocks and generated regions.
|
||||
# After editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-11-09-cua-ax-traversal-defaults.md
|
||||
/agent-note-ax-traversal-defaults-and-per-read-limits:
|
||||
en: 71a088b589135b94
|
||||
zh: 40a0accb1f6d100b
|
||||
/agent-note-ax-traversal-defaults-and-per-read-limits/problem:
|
||||
en: 18405553f792f64c
|
||||
zh: 1b6891403539c2f8
|
||||
/agent-note-ax-traversal-defaults-and-per-read-limits/decision:
|
||||
en: 89cb2b1f33b0551c
|
||||
zh: b57839093a735c2a
|
||||
/agent-note-ax-traversal-defaults-and-per-read-limits/alternatives-considered:
|
||||
en: c15d84e0bbb18b39
|
||||
zh: 4799291d7a66fa04
|
||||
/agent-note-ax-traversal-defaults-and-per-read-limits/consequences:
|
||||
en: 687f4ba2af13e847
|
||||
zh: ae5777022619b71d
|
||||
@@ -0,0 +1,21 @@
|
||||
# Agent Note: AX traversal defaults and per-read limits
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-11-09-cua-ax-traversal-defaults.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
A shallow read can omit a control in a deeply nested window. Increasing the profile settings to reach it also increases the work and possible output of subsequent reads.
|
||||
|
||||
## Decision
|
||||
|
||||
For an accessibility tree read, the Host takes each of `maxElements` and `maxDepth` from the request when present and from the provider configuration otherwise. A caller can raise or lower either value for one read. The native provider enforces its traversal ceilings. On macOS, Paste menu discovery always uses the configured values because the paste operation has no traversal options.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Cap each read at the configured values.** With a profile tuned for small reads, recovering a truncated branch would require changing the profile even when the provider can handle a larger request. Per-read overrides let the caller inspect that window and return to the usual limits on the next read.
|
||||
|
||||
## Consequences
|
||||
|
||||
Lowering a profile default reduces the budget for ordinary reads. An explicit larger request can take longer and produce more text. The native operation deadline and response size limit still apply. The [macOS configuration table](../../../../packages/experimental/cua-macos/README.md#use-this-package) lists the defaults.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Agent Note: AX 树遍历的默认值和单次读取上限
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-11-09-cua-ax-traversal-defaults.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
读取深度较小时，控件可能位于窗口中被截断的分支里。为了找到这个控件而调高配置值，会让以后的读取也可能访问更多节点、返回更多文本。
|
||||
|
||||
## 决策
|
||||
|
||||
读取辅助功能树时，Host 分别处理 `maxElements` 和 `maxDepth`。调用方显式传入的值只对本次读取生效，可以高于或低于配置值。省略的参数使用平台配置。原生实现在遍历时执行自身的硬上限。macOS 的粘贴操作没有遍历参数，查找粘贴菜单时始终使用配置值。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**把配置值作为每次读取的硬上限。** 如果日常读取使用较小的配置值，即使平台支持读取更大的树，调用方也必须先修改配置才能看到被截断的分支。允许单次覆盖后，调用方可以单独扩大这次读取的范围，下次读取继续使用原有配置。
|
||||
|
||||
## 影响
|
||||
|
||||
调低配置值可以缩小常规读取的范围。显式请求更大的树可能耗时更长，也可能返回更多文本。原生调用仍受超时和响应大小限制。默认值见 [macOS 配置表](../../../../packages/experimental/cua-macos/README.zh.md#use-this-package)。
|
||||
@@ -1163,7 +1163,7 @@ export interface Config {
|
||||
|
||||
- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents`
|
||||
- `refs`: [`CuaConfig`](../packages/experimental/cua/src/index.ts)
|
||||
- `source`: [`packages/experimental/cua-macos/src/index.ts:33`](../packages/experimental/cua-macos/src/index.ts)
|
||||
- `source`: [`packages/experimental/cua-macos/src/index.ts:34`](../packages/experimental/cua-macos/src/index.ts)
|
||||
|
||||
```ts config-catalog
|
||||
/** Native preparation, source-build signing, operation bounds, and profile-local application grants. */
|
||||
|
||||
@@ -1165,7 +1165,7 @@ export interface Config {
|
||||
|
||||
- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents`
|
||||
- `refs`: [`CuaConfig`](../packages/experimental/cua/src/index.ts)
|
||||
- `source`: [`packages/experimental/cua-macos/src/index.ts:33`](../packages/experimental/cua-macos/src/index.ts)
|
||||
- `source`: [`packages/experimental/cua-macos/src/index.ts:34`](../packages/experimental/cua-macos/src/index.ts)
|
||||
|
||||
```ts config-catalog
|
||||
/** Native preparation, source-build signing, operation bounds, and profile-local application grants. */
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
# Bilingual-pair consistency record for guide.md (docs/i18n/README.md): per heading
|
||||
# section, a hash of its English and Chinese blocks outside code blocks and generated regions.
|
||||
# After editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/upgrade-guide/v0.2.2-rc.1/cua-ax-read-limits/guide.md
|
||||
/:
|
||||
en: a8ca814cf60ea12e
|
||||
zh: f068d1ee13e80594
|
||||
/macos-ax-tree-read-limits:
|
||||
en: a7eed0c191ff2b32
|
||||
zh: 15291798ba91329b
|
||||
/macos-ax-tree-read-limits/change:
|
||||
en: e4281d86095b950f
|
||||
zh: 684ddf2957443ace
|
||||
/macos-ax-tree-read-limits/migration:
|
||||
en: 97a3885f5d1cb907
|
||||
zh: 19313c56eff68835
|
||||
@@ -0,0 +1,20 @@
|
||||
---
|
||||
kind: upgrade-guide
|
||||
description: "macOS reads more AX nodes by default and accepts per-read overrides of configured traversal budgets."
|
||||
---
|
||||
|
||||
# macOS AX tree read limits
|
||||
|
||||
English | [中文](guide.zh.md)
|
||||
|
||||
## Change
|
||||
|
||||
The default macOS node count rises from 2000 to 5000 and the default depth from 25 to 50. Reading a large window can take longer and emit more text to the model.
|
||||
|
||||
Configured `maxElements` and `maxDepth` now supply defaults independently for omitted arguments. Callers can request larger or smaller values for one read. Paste menu discovery continues to use the configured limits.
|
||||
|
||||
## Migration
|
||||
|
||||
1. To keep the previous budgets for ordinary reads, find the `@deepseek-ai/dsh-experimental-cua-macos` entry in your `cordis.yml` or profile patch. Set `maxElements: 2000` and `maxDepth: 25` in its existing `config`.
|
||||
2. Restart the profile and confirm both defaults in the Computer Use model instructions.
|
||||
3. If a script depends on specific traversal budgets, pass them explicitly to `getAccessibilityTree()`. An override applies only to that call.
|
||||
@@ -0,0 +1,20 @@
|
||||
---
|
||||
kind: upgrade-guide
|
||||
description: "macOS 默认读取更多 AX 节点，调用方可为单次读取覆盖配置值。"
|
||||
---
|
||||
|
||||
# macOS AX 树读取范围
|
||||
|
||||
[English](guide.md) | 中文
|
||||
|
||||
## 变更
|
||||
|
||||
macOS 的默认节点数从 2000 增至 5000，默认深度从 25 增至 50。窗口中有更多节点可访问时，读取可能耗时更长，输出到模型的文本也可能增加。
|
||||
|
||||
`maxElements` 和 `maxDepth` 分别决定省略对应参数时的读取范围。调用方可以为一次读取指定高于或低于配置的值。查找粘贴菜单始终使用配置值。
|
||||
|
||||
## 迁移
|
||||
|
||||
1. 如需沿用原先的常规读取范围，在 `cordis.yml` 或配置补丁中找到 `@deepseek-ai/dsh-experimental-cua-macos`，在现有的 `config` 下设置 `maxElements: 2000` 和 `maxDepth: 25`。
|
||||
2. 重启配置，确认 Computer Use 模型说明中的两个默认值与配置一致。
|
||||
3. 如果脚本依赖特定的遍历范围，在 `getAccessibilityTree()` 的调用处显式传入这两个参数。一次调用的覆盖值不会影响后续读取。
|
||||
@@ -12,5 +12,5 @@
|
||||
en: e5d1027faa55adba
|
||||
zh: 4b88ad6a70977fb3
|
||||
/native-cua-application-and-window-api/migration:
|
||||
en: 150890339b3967c8
|
||||
zh: 1b0b464f48dde313
|
||||
en: 071b67613c512b84
|
||||
zh: 7ee32d8dbcf7a0f2
|
||||
@@ -16,7 +16,7 @@ Providers supply startup capabilities alongside their platform guidance. The nat
|
||||
## Migration
|
||||
|
||||
1. After `cua.getApp(selector)`, select the target from `app.getWindows()` with `app.getWindow(windowId)`. Use `window.getScreenshot()`, `window.click()`, `window.scroll()`, and `window.drag()`. Replace `hover` with `window.mouseMove()`. On macOS, `window.click(id)` sends mouse events at the element center. Switch to `invoke(id)` for the default accessibility action, or `performAction(id, "AXShowMenu")` to request an advertised menu action.
|
||||
2. Replace `getAXState()` with `getAccessibilityTree()`. Replace `getAXStateAndScreenshot()` with separate tree and screenshot reads. Application and window selection does not read AX. Before using element IDs, call `getAccessibilityTree()` explicitly; use `{ emit: false }` to retain the snapshot without displaying it. A screenshot does not require a successful AX read. Before coordinate input, including relative coordinates, obtain a successful AX observation covering the window or a screenshot; absolute coordinates require a screenshot.
|
||||
2. Replace `getAXState()` with `getAccessibilityTree()`. Replace `getAXStateAndScreenshot()` with separate tree and screenshot reads. Application and window selection does not read AX. Before using element IDs, call `getAccessibilityTree()` explicitly; use `{ emit: false }` to retain the snapshot without displaying it. A screenshot does not require a successful AX read. Before coordinate input, including relative coordinates, obtain a successful AX observation covering the window or a screenshot; absolute coordinates require a screenshot. See [macOS AX read limits](../cua-ax-read-limits/guide.md) for changes to traversal defaults and configuration.
|
||||
3. Pass the opaque string ID from the latest tree wherever you passed a numeric element index. In raw requests use `elementId` with `snapshotId`. Replace `performSecondaryAction` with `listActions(id)` and `performAction(id, exactName, argument?)`, or use `invoke(id)` for the default action. `setValue` accepts the provider’s supported JSON values. On macOS, `performAction(id, 'AXRaise')` performs the advertised raise action. Call `app.activate()` first when the application must become active.
|
||||
4. Replace a packed chord such as `pressKey('shift+Tab')` with `pressKey('Tab', ['shift'])`. Check `(await cua.getCapabilities()).methods` before using `sendKeyEvent`. On macOS, call it inside one `run_code` to hold a key across tool calls. To request one repeat, send another down for a non-modifier key held by that execution. Returning, throwing or pausing releases any remaining keys. If your script handles unknown macOS key names as `NATIVE_ERROR`, handle `INVALID_ARGUMENT` instead. Correct the key name before retrying.
|
||||
5. Read `dispatched` from action results, then observe the application to verify the outcome. `invoke` returns `dispatched: false` when the element has no default action. Remove reads of the old `verified` field and of `pid` on selected objects or window records. Use `listApps()` or running `findApps()` matches when you need a PID.
|
||||
|
||||
@@ -16,7 +16,7 @@ CUA 脚本在指定窗口上截图和操作鼠标。App 和 Window 对象共享
|
||||
## 迁移
|
||||
|
||||
1. 调用 `cua.getApp(selector)` 后，从 `app.getWindows()` 返回的列表选定目标，再调用 `app.getWindow(windowId)`。在窗口上调用 `getScreenshot()`、`click()`、`scroll()` 和 `drag()`，将原有的 `hover` 调用改为 `window.mouseMove()`。在 macOS 上，`window.click(id)` 在元素中心发送鼠标事件。需要执行默认辅助功能动作时，改用 `invoke(id)`。需要打开控件菜单且控件声明了 `AXShowMenu` 时，调用 `performAction(id, "AXShowMenu")`。
|
||||
2. 将 `getAXState()` 改为 `getAccessibilityTree()`，将 `getAXStateAndScreenshot()` 拆成读取树和截图两次调用。选择应用或窗口不读取 AX。使用元素 ID 前须显式调用 `getAccessibilityTree()`；使用 `{ emit: false }` 可保留快照但不显示树。截图不要求 AX 读取成功。所有坐标输入（包括相对坐标）之前都须成功读取覆盖该窗口的 AX 或取得截图；绝对坐标要求截图。
|
||||
2. 将 `getAXState()` 改为 `getAccessibilityTree()`，将 `getAXStateAndScreenshot()` 拆成读取树和截图两次调用。选择应用或窗口不读取 AX。使用元素 ID 前须显式调用 `getAccessibilityTree()`；使用 `{ emit: false }` 可保留快照但不显示树。截图不要求 AX 读取成功。所有坐标输入（包括相对坐标）之前都须成功读取覆盖该窗口的 AX 或取得截图；绝对坐标要求截图。[读取上限的迁移说明](../cua-ax-read-limits/guide.zh.md)列出了 macOS 默认值和配置含义的变化。
|
||||
3. 将数字元素索引改为从最新树中取得的字符串 ID。直接发送请求时使用 `elementId` 和 `snapshotId`。原先调用 `performSecondaryAction` 的地方，先用 `listActions(id)` 取得完整动作名，再调用 `performAction(id, exactName, argument?)`。要执行默认动作，可调用 `invoke(id)`。`setValue` 接受提供方支持的 JSON 值。 在 macOS 上，`performAction(id, 'AXRaise')` 只执行元素声明的抬起窗口动作。需要把应用切到前台时先调用 `app.activate()`。
|
||||
4. 将 `pressKey('shift+Tab')` 之类的组合键写法改为 `pressKey('Tab', ['shift'])`。调用 `sendKeyEvent` 前检查 `(await cua.getCapabilities()).methods`。在 macOS 上，同一次 `run_code` 中可调用此方法跨工具调用保持按键。如果需要一次重复按下，可在这次执行中对已按住的普通键再发一次 `down`。执行返回、异常退出或用户暂停时会释放仍按着的键。如果脚本用 `NATIVE_ERROR` 识别 macOS 上的未知键名，请改为处理 `INVALID_ARGUMENT`，并在重试前修正键名。
|
||||
5. 从动作结果读取 `dispatched`，再观察应用确认效果。元素没有默认动作时，`invoke` 返回 `dispatched: false`。移除读取旧字段 `verified` 的代码，以及对已选对象和窗口记录中 `pid` 的读取。需要 PID 时，使用 `listApps()` 或 `findApps()` 返回的运行中应用记录。
|
||||
|
||||
@@ -15,8 +15,8 @@
|
||||
en: 1777e7bf6785f986
|
||||
zh: e3daac1074ebb74d
|
||||
/deepseek-ai-dsh-experimental-cua-macos/use-this-package:
|
||||
en: 98f739bfb1f2554b
|
||||
zh: 4d4809f851df6b35
|
||||
en: 233423a8675de85d
|
||||
zh: faf5cecb9999cfa7
|
||||
/deepseek-ai-dsh-experimental-cua-macos/use-this-package/connect-an-external-execution-environment:
|
||||
en: a08d37e868a0a860
|
||||
zh: 5dc8a2cdd165cc7e
|
||||
@@ -39,14 +39,14 @@
|
||||
en: 2dbc22ffd1f8a717
|
||||
zh: d4458872e75305d9
|
||||
/deepseek-ai-dsh-experimental-cua-macos/model-experience/native-application-operations/token-effect:
|
||||
en: ab52b46d874fa8b9
|
||||
zh: e87d3620ef77cb06
|
||||
en: f49bbb1f447feb1d
|
||||
zh: 664edf0f704caab6
|
||||
/deepseek-ai-dsh-experimental-cua-macos/model-experience/native-application-operations/kv-cache-effect:
|
||||
en: a2c9acb5a4320dda
|
||||
zh: e8d1586af60215c4
|
||||
/deepseek-ai-dsh-experimental-cua-macos/known-limitations-and-deferred-work:
|
||||
en: 4fefaf88ea9eefc6
|
||||
zh: d3d18cfbccf8c20f
|
||||
en: b362f9357fe79293
|
||||
zh: a74f0e9cffb647bc
|
||||
/deepseek-ai-dsh-experimental-cua-macos/known-limitations-and-deferred-work/dev-note:
|
||||
en: b836f01bd0bceb4a
|
||||
zh: 8122fa08b2aa95fa
|
||||
@@ -72,7 +72,7 @@ Source builds default to `sourceSigning: ad-hoc`, without creating certificates
|
||||
| `maxClipboardBytes` | `33554432` | Maximum original clipboard bytes preserved during a paste; oversized contents reject before replacement. |
|
||||
| `pasteReadTimeoutMs` | `1000` | Clipboard read deadline after the Paste command returns; expiry reports `PASTE_NOT_CONSUMED`. |
|
||||
| `maxAppMatches` | `20` | Maximum installed-app search matches; broader queries reject and require a narrower name. |
|
||||
| `maxElements` / `maxDepth` | `2000` / `25` | Accessibility traversal limits. |
|
||||
| `maxElements` / `maxDepth` | `5000` / `50` | Defaults for tree reads. Each read can raise or lower either value. Paste menu discovery always uses the configured limits. [Rationale](../../../.agents/notes/implemented/architecture/2026-11-09-cua-ax-traversal-defaults.md) |
|
||||
| `maxClickHoldMs` | `5000` | Maximum explicit click hold in milliseconds; accepts nonnegative integers. Zero only permits immediate release. |
|
||||
| `dragDurationMs` / `dragSteps` | `400` / `20` | Duration and movement segments for a drag. |
|
||||
| `textEventIntervalMs` | `0.25` | Delay after each Unicode down/up event, from 0 to 999 ms. Keystroke timing is independent. |
|
||||
@@ -89,7 +89,7 @@ Application consent is enforced before AX reads, window-title reads, screenshots
|
||||
|
||||
Raw `getState` exposes validated state fields and the complete text for the requested disclosure options. The native helper also returns structured element diagnostics with opaque string `elementId` values for native checks; the tool projection omits that duplicate array from model text and structured tool values.
|
||||
|
||||
AX observations show menu-bar entries by default. Use `app.getAccessibilityTree({ includeMenus: true })` to expand menu contents and `app.getAccessibilityTree({ details: true })` for identifiers, global-point bounds and complete action names. These options also apply to raw `getState`. Every read replaces the application's current snapshot; use element IDs from that read. Details do not bypass the configured node and depth limits. Truncation identifies the reached limit.
|
||||
AX observations show menu-bar entries by default. Use `app.getAccessibilityTree({ includeMenus: true })` to expand menu contents and `app.getAccessibilityTree({ details: true })` for identifiers, global-point bounds and complete action names. These options also apply to raw `getState`. Every read replaces the application's current snapshot; use element IDs from that read. Pass `maxElements` or `maxDepth` to override either configured limit for one read. Truncation identifies the reached limit.
|
||||
|
||||
Values and placeholders are separate. Named, stateful or actionable groups and scroll areas retain their own rows. Repeated names, descriptions and help text on one node are shown once. Values, placeholders and URLs keep their distinct meanings even when their text matches; body text shared by distinct nodes remains intact. Writability is explicit on relevant controls, including `AXIncrementor`; `unknown` means the system did not report it. Missing optional state fields do not imply false. During tree observation, an AXValue or AXSelectedTextRange writability probe returning -25200 leaves that field unknown and preserves the remaining nodes. Actual writes remain strict; permission, communication and invalid-element errors still propagate. UTF-16 ranges outside the observed value are marked unavailable. Compact tree labels can omit AppKit target and selector details. Pass the exact name returned by `listActions(elementId)` to `performAction`.
|
||||
|
||||
@@ -202,7 +202,7 @@ Setup and discovery:
|
||||
Available at app or window level depending on the platform.
|
||||
|
||||
Read getAccessibilityTree explicitly before using element ids. Selection and screenshots do not read or replace the current AX snapshot. Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the tree within the provider limits; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. If the tree reports maxElements or maxDepth truncation, select a narrower window or ask the user to raise the named macOS provider setting and read again. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- listActions(elementId: string) -> string[] — list the actions an element advertises.
|
||||
- invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none.
|
||||
- performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument.
|
||||
@@ -233,7 +233,7 @@ Only if the task cannot be completed without moving the user’s pointer, call a
|
||||
|
||||
#### Token effect
|
||||
|
||||
Enabling the plugin adds a fixed schema and guidance. AX observations, operation results and admitted images add model input. Retained objects are not serialized into prompts by this package.
|
||||
Enabling the plugin adds a fixed schema and guidance. AX observations, operation results and admitted images add model input. Retained objects are not serialized into prompts by this package. A higher default or per-read override can include more AX text in model input when the window has more nodes to visit.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
@@ -241,6 +241,8 @@ Tool results append to Session history. Changing plugin configuration or tool pr
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
The native operation deadline (`timeoutMs`), per-AX-call timeout (`axTimeoutMs`), and response size limit (`maxResponseBytes`) still apply to larger tree reads.
|
||||
|
||||
The queue orders individual native requests. Holding an app object does not reserve its application across a Session; [continuous interaction](../../../.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.md) grants no cross-call ownership. Capture RPCs that never return occupy the single capture slot; creating new app objects does not reset that system connection.
|
||||
|
||||
<a id="known-limitations-and-deferred-work"></a>
|
||||
|
||||
@@ -72,7 +72,7 @@ pnpm dsh headless --patch packages/experimental/cua-macos/examples/macos.cordis.
|
||||
| `maxClipboardBytes` | `33554432` | 粘贴时保存原剪贴板内容的字节上限；超限会在替换前拒绝。 |
|
||||
| `pasteReadTimeoutMs` | `1000` | Paste 命令返回后等待剪贴板读取的上限；到期返回 `PASTE_NOT_CONSUMED`。 |
|
||||
| `maxAppMatches` | `20` | 已安装应用搜索的结果上限；范围过大的查询会被拒绝，需要缩小名称范围。 |
|
||||
| `maxElements` / `maxDepth` | `2000` / `25` | 辅助功能树遍历限制。 |
|
||||
| `maxElements` / `maxDepth` | `5000` / `50` | 读取树时的默认值。每次读取可单独调高或调低任一项。查找粘贴菜单始终使用配置值。[决策理由](../../../.agents/notes/implemented/architecture/2026-11-09-cua-ax-traversal-defaults.zh.md) |
|
||||
| `maxClickHoldMs` | `5000` | 显式点击按住时长的上限，单位为毫秒，接受非负整数。设为 0 时只允许立即松开。 |
|
||||
| `dragDurationMs` / `dragSteps` | `400` / `20` | 拖动时长和移动分段数。 |
|
||||
| `textEventIntervalMs` | `0.25` | 每个 Unicode 按下或抬起事件后的间隔，范围为 0 至 1000 毫秒。击键节奏独立。 |
|
||||
@@ -89,7 +89,7 @@ CUA 工具在读取 AX、窗口标题、截图、激活和输入前检查应用
|
||||
|
||||
原始 `getState` 操作返回经过校验的状态字段，以及按本次披露选项生成的完整文本。原生 helper 还返回供原生检查使用的结构化元素诊断数组，其中 `elementId` 是不透明字符串；工具投影会从模型文本和结构化工具值中移除这份重复数组。
|
||||
|
||||
AX 观察默认只展示菜单栏入口。使用 `app.getAccessibilityTree({ includeMenus: true })` 展开菜单内容，使用 `app.getAccessibilityTree({ details: true })` 读取标识符、全局 point 坐标和完整动作名。原始 `getState` 也接受这两个选项。每次读取都会替换应用当前的快照；应使用该次读取返回的元素 ID。详情不会绕过配置的节点数和深度上限；截断提示会指明触发的上限。
|
||||
AX 观察默认只展示菜单栏入口。使用 `app.getAccessibilityTree({ includeMenus: true })` 展开菜单内容，使用 `app.getAccessibilityTree({ details: true })` 读取标识符、全局 point 坐标和完整动作名。原始 `getState` 也接受这两个选项。每次读取都会替换应用当前的快照；应使用该次读取返回的元素 ID。传入 `maxElements` 或 `maxDepth` 可调整本次读取的上限。树被截断时会提示触发的上限。
|
||||
|
||||
实际值与占位文字分别展示。有名称、状态或可用动作的分组和滚动区域保留独立行。同一节点中重复的名称、描述和帮助只显示一次。实际值、占位文字和 URL 即使文字相同也保留各自含义，不删除不同节点共有的正文。相关控件明确展示可写性，包括 `AXIncrementor`；`unknown` 表示系统未报告。缺少可选状态字段不代表 false。读树时，AXValue 或 AXSelectedTextRange 的可写性探测返回 -25200，会将该字段保留为未知并继续读取其他节点。实际写入仍严格校验；权限、通信和失效元素错误仍向上传播。超出当前值长度的 UTF-16 选区标为不可用。树中的动作简称可能省略 AppKit 的 target 和 selector 信息。调用 `performAction` 时使用 `listActions(elementId)` 返回的完整名称。
|
||||
|
||||
@@ -202,7 +202,7 @@ Setup and discovery:
|
||||
Available at app or window level depending on the platform.
|
||||
|
||||
Read getAccessibilityTree explicitly before using element ids. Selection and screenshots do not read or replace the current AX snapshot. Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the tree within the provider limits; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. If the tree reports maxElements or maxDepth truncation, select a narrower window or ask the user to raise the named macOS provider setting and read again. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- listActions(elementId: string) -> string[] — list the actions an element advertises.
|
||||
- invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none.
|
||||
- performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument.
|
||||
@@ -233,7 +233,7 @@ Only if the task cannot be completed without moving the user’s pointer, call a
|
||||
|
||||
#### Token 影响
|
||||
|
||||
启用插件会增加固定参数定义和指导文本。AX 观察、操作结果和已接入的图片会增加模型输入。本包不将保留的对象序列化进提示。
|
||||
启用插件会增加固定参数定义和指导文本。AX 观察、操作结果和已接入的图片会增加模型输入。本包不将保留的对象序列化进提示。窗口中有更多节点可读时，调高默认值或单次读取上限可能使进入模型的 AX 文本变长。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
@@ -241,6 +241,8 @@ Only if the task cannot be completed without moving the user’s pointer, call a
|
||||
|
||||
## 已知限制与后续工作
|
||||
|
||||
读取较大的树仍受原生操作超时（`timeoutMs`）、单次 AX 调用超时（`axTimeoutMs`）和响应大小限制（`maxResponseBytes`）约束。
|
||||
|
||||
队列只排序单次原生请求。持有 app 对象不会为 Session 保留应用，[连续交互](../../../.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.zh.md)不分配跨调用所有权。一直不返回的系统截图 RPC 会占用唯一捕获槽位，重新创建 app 对象不会重置该系统连接。
|
||||
|
||||
<a id="known-limitations-and-deferred-work"></a>
|
||||
|
||||
@@ -576,12 +576,16 @@ final class AXAccess {
|
||||
var lines = ["AX scope=\(requested == nil ? "application" : "window") offsets=UTF-16\(details ? " bounds=global-points" : "")"]
|
||||
var elements: [JSONValue] = []
|
||||
var indexed: [Int: AXUIElement] = [:]
|
||||
func walk(_ element: AXUIElement, level: Int, depth: Int) throws {
|
||||
var pending = roots.reversed().map { (element: $0, level: 0, depth: 0) }
|
||||
func enqueueChildren(_ children: [AXUIElement], level: Int, depth: Int) {
|
||||
for child in children.reversed() { pending.append((child, level, depth)) }
|
||||
}
|
||||
while let (element, level, depth) = pending.popLast() {
|
||||
try cancellation.check()
|
||||
let hash = CFHash(element)
|
||||
if visited[hash, default: []].contains(where: { CFEqual($0, element) }) { return }
|
||||
guard count < maxElements else { elementLimitReached = true; return }
|
||||
guard depth <= maxDepth else { depthLimitReached = true; return }
|
||||
if visited[hash, default: []].contains(where: { CFEqual($0, element) }) { continue }
|
||||
guard count < maxElements else { elementLimitReached = true; break }
|
||||
guard depth <= maxDepth else { depthLimitReached = true; continue }
|
||||
visited[hash, default: []].append(element)
|
||||
count += 1
|
||||
let observation = try nodeObservation(attributes: metadata(element), fallback: { try self.attribute(element, $0) })
|
||||
@@ -616,8 +620,8 @@ final class AXAccess {
|
||||
identifier: identifier, url: url, focused: focused, expanded: expanded)
|
||||
node.validateSelection()
|
||||
if node.transparent || (!addressable && !node.hasMeaning && role != "AXWindow" && role != "AXSheet") {
|
||||
for child in try childValues(observed("AXChildren"), attribute: "AXChildren") { try walk(child, level: level, depth: depth + 1) }
|
||||
return
|
||||
enqueueChildren(try childValues(observed("AXChildren"), attribute: "AXChildren"), level: level, depth: depth + 1)
|
||||
continue
|
||||
}
|
||||
let index: Int? = addressable ? indexed.count : nil
|
||||
if let index { indexed[index] = element }
|
||||
@@ -640,10 +644,9 @@ final class AXAccess {
|
||||
"selectedRange": node.selectedRange.map { .array([.number(Double($0.location)), .number(Double($0.length))]) } ?? .null,
|
||||
]))
|
||||
}
|
||||
if role == "AXMenuBarItem" && !includeMenus { menusCollapsed = true; return }
|
||||
for child in try childValues(observed("AXChildren"), attribute: "AXChildren") { try walk(child, level: level + 1, depth: depth + 1) }
|
||||
if role == "AXMenuBarItem" && !includeMenus { menusCollapsed = true; continue }
|
||||
enqueueChildren(try childValues(observed("AXChildren"), attribute: "AXChildren"), level: level + 1, depth: depth + 1)
|
||||
}
|
||||
for root in roots { try walk(root, level: 0, depth: 0) }
|
||||
if menusCollapsed { lines.append("Menu contents omitted; request includeMenus=true to expand.") }
|
||||
let truncated = elementLimitReached || depthLimitReached
|
||||
if truncated { lines.append(AXTextFormatter.truncation(maxElements: elementLimitReached ? maxElements : nil, maxDepth: depthLimitReached ? maxDepth : nil)) }
|
||||
|
||||
@@ -307,7 +307,7 @@ actor AXWorker {
|
||||
let projection: AXProjection
|
||||
do {
|
||||
projection = try ax().project(identity: identity, windowID: window,
|
||||
maxElements: params.integer("maxElements", in: 1...10000), maxDepth: params.integer("maxDepth", in: 1...100),
|
||||
maxElements: params.integer("maxElements", in: NativeProtocol.elementRange), maxDepth: params.integer("maxDepth", in: NativeProtocol.depthRange),
|
||||
includeMenus: params.optionalBoolean("includeMenus") ?? false, details: params.optionalBoolean("details") ?? false)
|
||||
} catch let failure as NativeFailure {
|
||||
throw Self.observationFailure(failure)
|
||||
@@ -343,7 +343,7 @@ actor AXWorker {
|
||||
return .paste(PastePlan(handle: handle, text: try params.string("text"), format: try params.optionalString("format") ?? "text",
|
||||
maxBytes: try params.integer("maxClipboardBytes", in: 1024...Int.max),
|
||||
readTimeout: .milliseconds(try params.integer("pasteReadTimeoutMs", in: 1...2147483647)),
|
||||
maxElements: try params.integer("maxElements", in: 1...10000), maxDepth: try params.integer("maxDepth", in: 1...100)))
|
||||
maxElements: try params.integer("maxElements", in: NativeProtocol.elementRange), maxDepth: try params.integer("maxDepth", in: NativeProtocol.depthRange)))
|
||||
case "sendKeyEvent":
|
||||
try inputs().0.keyDown(params.string("key"), app: handle, run: params.string("runId")) {
|
||||
try ax().keyboardRecipient(identity, catalog: catalog)
|
||||
|
||||
+2
@@ -1,5 +1,7 @@
|
||||
// Generated by native/generate-protocol.ts from @deepseek-ai/dsh-cua/native-schema. Do not edit.
|
||||
enum NativeProtocol {
|
||||
static let elementRange = 1...100000
|
||||
static let depthRange = 1...512
|
||||
static let fields: [String: Set<String>] = [
|
||||
"activate": ["appHandle"],
|
||||
"applicationIcons": ["paths"],
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
/** Generate macOS admission artifacts from the shared CUA operation definitions. */
|
||||
import { readFile, writeFile } from 'node:fs/promises'
|
||||
import { z } from 'zod'
|
||||
import { AX_TRAVERSAL_MAX } from '../src/ax-limits.ts'
|
||||
import { NativeResourceParameters } from '@deepseek-ai/dsh-cua/protocol'
|
||||
import { nativeProtocol } from '@deepseek-ai/dsh-cua/native-schema'
|
||||
import { PermissionGuideParameters, PermissionSettingsParameters } from '../src/permission-guide.ts'
|
||||
|
||||
@@ -9,7 +11,13 @@ import { PermissionGuideParameters, PermissionSettingsParameters } from '../src/
|
||||
* @returns Native JSON Schema and Swift field/enum tables, including private continuations.
|
||||
*/
|
||||
export function renderNativeProtocol(): { json: string; swift: string } {
|
||||
const traversal = {
|
||||
maxElements: z.number().int().min(1).max(AX_TRAVERSAL_MAX.maxElements),
|
||||
maxDepth: z.number().int().min(1).max(AX_TRAVERSAL_MAX.maxDepth),
|
||||
}
|
||||
const { fields, enums, json } = nativeProtocol({
|
||||
getState: NativeResourceParameters.getState.extend(traversal),
|
||||
paste: NativeResourceParameters.paste.extend(traversal),
|
||||
sendKeyEvent: z.object({ runId: z.string().min(1) }),
|
||||
mouseMove: z.object({ useSystemPointer: z.boolean().optional() }),
|
||||
requestPermission: PermissionGuideParameters, openPermissionSettings: PermissionSettingsParameters,
|
||||
@@ -20,6 +28,8 @@ export function renderNativeProtocol(): { json: string; swift: string } {
|
||||
const swift = [
|
||||
'// Generated by native/generate-protocol.ts from @deepseek-ai/dsh-cua/native-schema. Do not edit.',
|
||||
'enum NativeProtocol {',
|
||||
` static let elementRange = 1...${AX_TRAVERSAL_MAX.maxElements}`,
|
||||
` static let depthRange = 1...${AX_TRAVERSAL_MAX.maxDepth}`,
|
||||
' static let fields: [String: Set<String>] = [',
|
||||
...[...fields].sort(([a], [b]) => a.localeCompare(b)).map(([method, names]) => ` ${JSON.stringify(method)}: [${quoted(names)}],`),
|
||||
' ]',
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$comment": "Generated from the shared CUA operation schema. Do not edit.",
|
||||
"title": "dsh-cua native request",
|
||||
"description": "Host resource fields and private continuations are not model tool parameters. App handles and snapshots belong to one helper; coordinates use window-local native units.",
|
||||
"description": "The Host resolves resource limits before dispatch. Private continuations are available only to the Host. App handles and snapshots belong to one helper; coordinates use window-local native units.",
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
@@ -250,12 +250,12 @@
|
||||
"maxElements": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 10000
|
||||
"maximum": 100000
|
||||
},
|
||||
"maxDepth": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 100
|
||||
"maximum": 512
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -944,12 +944,12 @@
|
||||
"maxElements": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 10000
|
||||
"maximum": 100000
|
||||
},
|
||||
"maxDepth": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 100
|
||||
"maximum": 512
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
|
||||
@@ -200,7 +200,9 @@ def main():
|
||||
helper.eof()
|
||||
print("PASS: coordinate observation, application snapshot, and selected-window rejection")
|
||||
return
|
||||
first = call("getState", primary_params)
|
||||
limited = call("getState", {**primary_params, "maxElements": 1})
|
||||
assert limited["truncated"] and "maxElements=1" in limited["tree"], limited
|
||||
first = call("getState", {**primary_params, "maxElements": 100000, "maxDepth": 512})
|
||||
second = call("getState", secondary_params)
|
||||
assert "Increment secondary" not in first["tree"], first
|
||||
assert "Increment primary" not in second["tree"], second
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
/** Largest traversal budgets accepted by the macOS helper. */
|
||||
export const AX_TRAVERSAL_MAX = { maxElements: 100_000, maxDepth: 512 } as const
|
||||
@@ -5,7 +5,7 @@ import { COMMON_GUIDANCE } from '@deepseek-ai/dsh-cua/guidance'
|
||||
export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.'
|
||||
|
||||
const methodDetails = new Map<string, string>([
|
||||
['getAccessibilityTree', 'If the tree reports maxElements or maxDepth truncation, select a narrower window or ask the user to raise the named macOS provider setting and read again. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.'],
|
||||
['getAccessibilityTree', 'macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.'],
|
||||
['mouseMove', 'Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without that route, HOVER_UNAVAILABLE is returned before sending input. Physical movement requires an unobscured foreground target. Explain any necessary activation and observe the target again.'],
|
||||
['click', 'click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying.'],
|
||||
])
|
||||
|
||||
@@ -9,6 +9,7 @@ import { createPermissionParameters } from './permission-guide.ts'
|
||||
import { nativeHostArchitecture, prepareNativeExecutable, type SourceSigning } from './native-build.ts'
|
||||
import { createSharedNativeChannel } from './shared-native.ts'
|
||||
import { installHeldKeys } from './held-keys.ts'
|
||||
import { AX_TRAVERSAL_MAX } from './ax-limits.ts'
|
||||
import { GUIDANCE } from './guidance.ts'
|
||||
export { GUIDANCE } from './guidance.ts'
|
||||
|
||||
@@ -43,6 +44,8 @@ export interface Config extends CuaConfig {
|
||||
/** Validate shared operation bounds and macOS source signing before preparing the helper. */
|
||||
export const Config = Schema.object({
|
||||
...CuaConfigFields,
|
||||
maxElements: CuaConfigFields.maxElements.max(AX_TRAVERSAL_MAX.maxElements).default(5000),
|
||||
maxDepth: CuaConfigFields.maxDepth.max(AX_TRAVERSAL_MAX.maxDepth).default(50),
|
||||
permissionPollIntervalMs: Schema.number().step(1).min(100).max(2 ** 31 - 1).default(1000),
|
||||
sourceSigning: Schema.union([
|
||||
Schema.const('ad-hoc'),
|
||||
|
||||
@@ -63,7 +63,7 @@ it('enumerates every available method once in its catalog entry', () => {
|
||||
})
|
||||
|
||||
it('documents option fields and return fields', () => {
|
||||
for (const token of ['emit', 'includeMenus', 'details', 'relative', 'mouseButton', 'clickCount', 'holdMs', 'modifiers', 'format', 'prefix', 'suffix', 'selectionType']) expect(COMMON_GUIDANCE).toContain(token)
|
||||
for (const token of ['emit', 'includeMenus', 'details', 'maxElements', 'maxDepth', 'relative', 'mouseButton', 'clickCount', 'holdMs', 'modifiers', 'format', 'prefix', 'suffix', 'selectionType']) expect(COMMON_GUIDANCE).toContain(token)
|
||||
for (const token of ['dispatched', 'foregroundChanged', 'hoverRoute', 'systemPointerMoved', 'bytes', 'mimeType', 'pixelWidth', 'pixelHeight', 'windowId']) expect(COMMON_GUIDANCE).toContain(token)
|
||||
})
|
||||
|
||||
|
||||
@@ -134,7 +134,7 @@ it.skipIf(process.platform === 'win32')('shares one application between retained
|
||||
const toolCalls = agent.session.snapshotEvents().filter(event => event.type === 'tool/call')
|
||||
expect(events.filter(event => event.event === 'call').map(event => event.requestId)).toEqual(toolCalls.map(event => event.data.callId))
|
||||
expect(events.find(event => event.event === 'state')?.count).toBe(1)
|
||||
expect(events.find(event => event.method === 'getState')?.params).toMatchObject({ maxElements: 2000, maxDepth: 25 })
|
||||
expect(events.find(event => event.method === 'getState')?.params).toMatchObject({ maxElements: 5000, maxDepth: 50 })
|
||||
expect(events.find(event => event.method === 'click')?.params).toMatchObject({ holdMs: 800 })
|
||||
expect(toolCalls[3]?.data.arguments).toBe(JSON.stringify(calls[3]))
|
||||
|
||||
@@ -173,7 +173,15 @@ it.skipIf(process.platform === 'win32')('shares one application between retained
|
||||
})
|
||||
const retainedApp = await objectApi(agent).getApp('test.fixture')
|
||||
const retainedWindow = await retainedApp.getWindow(7)
|
||||
expect(await retainedWindow.getAccessibilityTree({ emit: false })).toContain('Count: 1')
|
||||
await retainedApp.getAccessibilityTree({ maxDepth: 512 })
|
||||
expect(await retainedWindow.getAccessibilityTree({ maxElements: 100000, emit: false })).toContain('Count: 1')
|
||||
await retainedWindow.getAccessibilityTree({ emit: false })
|
||||
const reads = (await readEvents()).filter(event => event.method === 'getState').slice(-3)
|
||||
expect(reads.map(event => event.params)).toEqual([
|
||||
{ appHandle: retainedApp.appHandle, maxElements: 5000, maxDepth: 512 },
|
||||
{ appHandle: retainedApp.appHandle, windowId: 7, maxElements: 100000, maxDepth: 50 },
|
||||
{ appHandle: retainedApp.appHandle, windowId: 7, maxElements: 5000, maxDepth: 50 },
|
||||
])
|
||||
const peer = await context.agents.create({ sessionId: SessionId('native-loader-peer'), agentOptions: { provider: 'native-fixture', model: 'vision' } })
|
||||
const peerApp = await objectApi(peer.agent).getApp('test.fixture')
|
||||
const peerWindow = await peerApp.getWindow(7)
|
||||
|
||||
@@ -21,6 +21,16 @@ describe('native protocol generation', () => {
|
||||
expect(validate({ id: 'invalid', method: 'setValue', params: target })).toBe(false)
|
||||
})
|
||||
|
||||
it('enforces macOS tree budgets at native admission', () => {
|
||||
const validate = new Ajv2020().compile(JSON.parse(renderNativeProtocol().json))
|
||||
const read = (maxElements: number, maxDepth: number) => ({
|
||||
id: 'tree', method: 'getState', params: { appHandle: 'app', maxElements, maxDepth },
|
||||
})
|
||||
expect(validate(read(100000, 512))).toBe(true)
|
||||
expect(validate(read(100001, 512))).toBe(false)
|
||||
expect(validate(read(100000, 513))).toBe(false)
|
||||
})
|
||||
|
||||
it('rejects a key array at both model and native pressKey admission', () => {
|
||||
const request = { method: 'pressKey', params: { appHandle: 'app', key: ['w', 'd'] } }
|
||||
expect(Request.safeParse(request).success).toBe(false)
|
||||
@@ -29,7 +39,6 @@ describe('native protocol generation', () => {
|
||||
})
|
||||
|
||||
it.each([
|
||||
{ method: 'getState', params: { appHandle: 'app', maxElements: 1 } },
|
||||
{ method: 'drag', params: { appHandle: 'app', windowId: 1, fromX: 1, fromY: 1, toX: 2, toY: 2, steps: 1 } },
|
||||
{ method: 'paste', params: { appHandle: 'app', text: 'text', maxClipboardBytes: 1024 } },
|
||||
{ method: 'paste', params: { appHandle: 'app', text: 'text', pasteReadTimeoutMs: 1000 } },
|
||||
|
||||
@@ -94,6 +94,12 @@ it('projects shared controls and macOS signing without adding signing to the sha
|
||||
for (const field of ['sourceSigning', 'permissionPollIntervalMs', 'agentPreset', 'helperPath']) {
|
||||
expect(projected.schema.anyOf?.[0]).toHaveProperty(`properties.${field}`)
|
||||
}
|
||||
expect(CuaConfig({})).toMatchObject({ maxElements: 2000, maxDepth: 25 })
|
||||
expect(NativeProvider.Config({})).toMatchObject({ maxElements: 5000, maxDepth: 50 })
|
||||
const limits = { maxElements: 100000, maxDepth: 512 }
|
||||
expect(NativeProvider.Config(limits)).toMatchObject(limits)
|
||||
expect(() => NativeProvider.Config({ maxElements: 100001 })).toThrow()
|
||||
expect(() => NativeProvider.Config({ maxDepth: 513 })).toThrow()
|
||||
expect(CuaConfig({})).not.toHaveProperty('sourceSigning')
|
||||
expect(CuaConfig.dict).not.toHaveProperty('sourceSigning')
|
||||
expect(NativeProvider.Config({ sourceSigning: 'ad-hoc', timeoutMs: 1234 })).toMatchObject({
|
||||
@@ -768,6 +774,7 @@ it('prepares the selected native component before registering CUA and creates no
|
||||
it('adds an installation-owned bridge URL when persistent PTC is available', async () => {
|
||||
const f = await setup()
|
||||
const text = async () => (await f.ctx.systemPrompt.assemble()).sections.find(section => section.name === 'computer-use:dsh-cua')?.text
|
||||
expect(await text()).toContain('AX read defaults: maxElements=64, maxDepth=8.')
|
||||
expect(await text()).not.toContain('Persistent PTC initialization:')
|
||||
await f.ctx.plugin(FileSystem)
|
||||
await f.ctx.plugin(Sandbox)
|
||||
|
||||
@@ -15,8 +15,8 @@
|
||||
en: 5db4d4fea25493b9
|
||||
zh: 65ab64fc24eb0d09
|
||||
/deepseek-ai-dsh-cua/use-this-package:
|
||||
en: 7835281e581aa269
|
||||
zh: f23550c0ea6c6d63
|
||||
en: a16d6b7e0d6d13a1
|
||||
zh: a3fb53cffc92b706
|
||||
/deepseek-ai-dsh-cua/use-this-package/permission-controls:
|
||||
en: d92d5948a7941e8d
|
||||
zh: d43491ae6521b87e
|
||||
|
||||
@@ -31,7 +31,7 @@ Application consent supplies English, Chinese, and Japanese display text from `s
|
||||
|
||||
Mount the [macOS provider](../cua-macos/README.md) to enable native operations on macOS. External PTC hosts import `connectCua` from `@deepseek-ai/dsh-cua/bridge` and supply the current `tools.cua` binding. Source compositions and native preparation belong to the provider.
|
||||
|
||||
Use `cua.getApp(selector)` to select an application; it obtains consent and optionally launches a stopped application in the background, without reading its accessibility tree. `cua.listApps` and `cua.getState` list running applications; `cua.findApps(query)` searches installed applications, including stopped ones. `app.activate()`, `app.getWindows()`, and `app.getWindow(windowId)` operate on the selected application; `getWindow` returns the object without a native request; window operations reject closed or foreign windows. Select an application and one of its windows before any capture or pointer input. Read `app.getAccessibilityTree()` or `window.getAccessibilityTree()` explicitly before element operations. Screenshots remain available when AX reading fails. Within one API instance, the facade keeps one current accessibility snapshot per application handle across app and window objects, including repeated selections; each new read invalidates the previous snapshot. Element operations use the latest snapshot ID, so callers use IDs from the latest tree and do not assume an older ID still identifies the same control. Element IDs are opaque strings.
|
||||
Use `cua.getApp(selector)` to select an application; it obtains consent and optionally launches a stopped application in the background, without reading its accessibility tree. `cua.listApps` and `cua.getState` list running applications; `cua.findApps(query)` searches installed applications, including stopped ones. `app.activate()`, `app.getWindows()`, and `app.getWindow(windowId)` operate on the selected application; `getWindow` returns the object without a native request; window operations reject closed or foreign windows. Select an application and one of its windows before any capture or pointer input. Read `app.getAccessibilityTree()` or `window.getAccessibilityTree()` explicitly before element operations. Screenshots remain available when AX reading fails. App and window reads accept `maxElements` and `maxDepth` independently. Omitted limits use the provider configuration. Within one API instance, the facade keeps one current accessibility snapshot per application handle across app and window objects, including repeated selections; each new read invalidates the previous snapshot. Element operations use the latest snapshot ID, so callers use IDs from the latest tree and do not assume an older ID still identifies the same control. Element IDs are opaque strings.
|
||||
|
||||
Keyboard methods are available in the scopes a provider declares; the provider owns how input is delivered within each scope. A lost connection invalidates its handles and coordinates. Failed cleanup is reported before that connection is retired; select the application again and inspect fresh state, because earlier input may have partially executed. Screenshot image errors retain the failing stage and attachment error code. Invalid request arguments reject before dispatch with a readable method, parameter and requirement.
|
||||
|
||||
|
||||
@@ -31,7 +31,7 @@ kind: "package-reference"
|
||||
|
||||
挂载 [macOS 提供方](../cua-macos/README.zh.md)以在 macOS 上启用原生操作。外部 PTC 宿主从 `@deepseek-ai/dsh-cua/bridge` 导入 `connectCua`，并提供当前 `tools.cua` 绑定。源码组合与原生组件准备由提供方负责。
|
||||
|
||||
使用 `cua.getApp(selector)` 选择应用；它会获取授权、按需在后台启动已停止的应用，不读取辅助功能树。`cua.listApps` 和 `cua.getState` 列出运行中的应用；`cua.findApps(query)` 搜索已安装应用，包括未运行的应用。`app.activate()`、`app.getWindows()` 和 `app.getWindow(windowId)` 操作选定的应用；`getWindow` 返回对象时不发送原生请求；窗口操作会拒绝已关闭或属于其他应用的窗口。进行任何截图或指针输入前，必须选择应用及其一个窗口。元素操作前须显式读取 `app.getAccessibilityTree()` 或 `window.getAccessibilityTree()`。AX 读取失败时仍可截图。在单个 API 实例内，门面按应用句柄保存一份当前辅助功能快照，该句柄的所有应用对象和窗口对象共享它，包括重复选择；每次新读取都会使上一份快照失效。元素操作使用最新快照 ID，因此调用方应使用最新树中的元素 ID，不应假定旧 ID 仍指向同一控件。元素 ID 是不透明字符串。
|
||||
使用 `cua.getApp(selector)` 选择应用；它会获取授权、按需在后台启动已停止的应用，不读取辅助功能树。`cua.listApps` 和 `cua.getState` 列出运行中的应用；`cua.findApps(query)` 搜索已安装应用，包括未运行的应用。`app.activate()`、`app.getWindows()` 和 `app.getWindow(windowId)` 操作选定的应用；`getWindow` 返回对象时不发送原生请求；窗口操作会拒绝已关闭或属于其他应用的窗口。进行任何截图或指针输入前，必须选择应用及其一个窗口。元素操作前须显式读取 `app.getAccessibilityTree()` 或 `window.getAccessibilityTree()`。AX 读取失败时仍可截图。读取应用或窗口时，可以单独传入 `maxElements` 或 `maxDepth`。省略的参数使用当前平台配置。在单个 API 实例内，门面按应用句柄保存一份当前辅助功能快照，该句柄的所有应用对象和窗口对象共享它，包括重复选择；每次新读取都会使上一份快照失效。元素操作使用最新快照 ID，因此调用方应使用最新树中的元素 ID，不应假定旧 ID 仍指向同一控件。元素 ID 是不透明字符串。
|
||||
|
||||
键盘方法可在 provider 声明的作用域中使用；每个作用域内的输入投递由 provider 负责。连接丢失会使其句柄和坐标失效。清理失败会先报告，再释放该连接；应重新选择应用并读取新状态，因为先前的输入可能已部分执行。截图图像错误会保留失败阶段和附件错误码。请求参数校验失败时，在分派前返回包含方法、参数和要求的可读消息。
|
||||
|
||||
|
||||
@@ -124,6 +124,8 @@ function createCuaObjects(call: ReturnType<typeof validatedCall>, sink: CuaObser
|
||||
requireAccessibility()
|
||||
observed.delete(target.appHandle)
|
||||
const state = StateResult.parse(await scopedCall('getState', {
|
||||
...options?.maxElements === undefined ? {} : { maxElements: options.maxElements },
|
||||
...options?.maxDepth === undefined ? {} : { maxDepth: options.maxDepth },
|
||||
...options?.includeMenus === undefined ? {} : { includeMenus: options.includeMenus },
|
||||
...options?.details === undefined ? {} : { details: options.details },
|
||||
}))
|
||||
|
||||
@@ -24,7 +24,7 @@ Setup and discovery:
|
||||
Available at app or window level depending on the platform.
|
||||
|
||||
Read getAccessibilityTree explicitly before using element ids. Selection and screenshots do not read or replace the current AX snapshot. Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the tree within the provider limits; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names.
|
||||
- listActions(elementId: string) -> string[] — list the actions an element advertises.
|
||||
- invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none.
|
||||
- performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument.
|
||||
|
||||
@@ -77,9 +77,9 @@ export interface CuaConfig {
|
||||
textEventIntervalMs: number
|
||||
/** Native connection cleanup, socket-write and launcher termination budget in milliseconds. */
|
||||
graceMs: number
|
||||
/** Maximum nodes admitted by a native accessibility snapshot. */
|
||||
/** Default node limit when an accessibility read omits maxElements. */
|
||||
maxElements: number
|
||||
/** Maximum depth traversed by a native accessibility snapshot. */
|
||||
/** Default depth limit when an accessibility read omits maxDepth. */
|
||||
maxDepth: number
|
||||
}
|
||||
|
||||
@@ -155,7 +155,7 @@ export function installCua(ctx: Context, config: CuaConfig, backend: CuaBackend)
|
||||
const mouseControl = systemPointer ? new MouseControl(ctx, config.requireSystemPointerApproval) : undefined
|
||||
const requestSchema = systemPointer ? SystemPointerRequest : Request
|
||||
const instructions = systemPointer ? withSystemPointerGuidance(backend.guidance) : backend.guidance
|
||||
const guidance = `${instructions}\n\nMaximum click holdMs: ${config.maxClickHoldMs} ms.`
|
||||
const guidance = `${instructions}\n\nAX read defaults: maxElements=${config.maxElements}, maxDepth=${config.maxDepth}.\n\nMaximum click holdMs: ${config.maxClickHoldMs} ms.`
|
||||
const native: NativeClientConfig = {
|
||||
helperPath, cwd: process.cwd(), timeoutMs: config.timeoutMs, axTimeoutMs: config.axTimeoutMs,
|
||||
connectTimeoutMs: config.connectTimeoutMs, screenshotTimeoutMs: config.screenshotTimeoutMs,
|
||||
@@ -230,7 +230,7 @@ export function installCua(ctx: Context, config: CuaConfig, backend: CuaBackend)
|
||||
? { ...params, maxClipboardBytes: config.maxClipboardBytes, pasteReadTimeoutMs: config.pasteReadTimeoutMs,
|
||||
maxElements: config.maxElements, maxDepth: config.maxDepth }
|
||||
: method === 'getState'
|
||||
? { ...params, maxElements: config.maxElements, maxDepth: config.maxDepth }
|
||||
? { maxElements: config.maxElements, maxDepth: config.maxDepth, ...params }
|
||||
: method === 'drag' ? { ...params, durationMs: config.dragDurationMs, steps: config.dragSteps }
|
||||
: params
|
||||
const requestNative = () => client.request(method, nativeParams, signal, execution.callId)
|
||||
|
||||
@@ -78,7 +78,7 @@ export function nativeProtocol(
|
||||
$schema: 'https://json-schema.org/draft/2020-12/schema',
|
||||
$comment: 'Generated from the shared CUA operation schema. Do not edit.',
|
||||
title: 'dsh-cua native request',
|
||||
description: 'Host resource fields and private continuations are not model tool parameters. App handles and snapshots belong to one helper; coordinates use window-local native units.',
|
||||
description: 'The Host resolves resource limits before dispatch. Private continuations are available only to the Host. App handles and snapshots belong to one helper; coordinates use window-local native units.',
|
||||
oneOf: schemas,
|
||||
}, null, 2) + '\n' }
|
||||
}
|
||||
@@ -7,6 +7,11 @@ const id = z.number().int().positive().max(2 ** 32 - 1)
|
||||
const pid = z.number().int().positive().max(2 ** 31 - 1)
|
||||
const point = z.number().nonnegative()
|
||||
const relativePoint = z.number().int().min(0).max(999)
|
||||
const traversalLimit = z.number().min(1).multipleOf(1)
|
||||
const nativeTraversalLimits = z.object({
|
||||
maxElements: z.number().int().min(1).max(10000),
|
||||
maxDepth: z.number().int().min(1).max(100),
|
||||
}).strict()
|
||||
const app = { appHandle: z.string().min(1) }
|
||||
const pointer = { mouseButton: z.enum(['left', 'right', 'middle']).optional(),
|
||||
modifiers: z.array(z.enum(['command', 'control', 'option', 'shift', 'fn'])).optional() }
|
||||
@@ -50,6 +55,8 @@ function requestSchema<T extends z.ZodType>(hover: T) {
|
||||
operation('getState', { ...app, windowId: id.optional(),
|
||||
includeMenus: z.boolean().optional().describe('Expand menu contents; omitted or false shows menu-bar entries only.'),
|
||||
details: z.boolean().optional().describe('Include identifiers, global native bounds and complete action names.'),
|
||||
maxElements: traversalLimit.optional().describe('Maximum nodes to visit on this read. Omission uses the provider setting.'),
|
||||
maxDepth: traversalLimit.optional().describe('Maximum depth to traverse on this read. Omission uses the provider setting.'),
|
||||
}),
|
||||
operation('listActions', snapshot),
|
||||
operation('performAction', { ...snapshot, action: z.string().min(1), argument: jsonValue.optional() }),
|
||||
@@ -91,12 +98,12 @@ export const SystemPointerRequest = requestSchema(z.union(MouseMoveTargets.map(t
|
||||
useSystemPointer: z.boolean().optional().describe('Request use of the system pointer. The Host checks permission before dispatch.'),
|
||||
}))))
|
||||
|
||||
/** Resource parameters supplied by the Host, never accepted from model tool arguments. */
|
||||
/** Native resource fields supplied or completed by the Host before dispatch. */
|
||||
export const NativeResourceParameters = {
|
||||
getState: z.object({ maxElements: z.number().int().min(1).max(10000), maxDepth: z.number().int().min(1).max(100) }).strict(),
|
||||
getState: nativeTraversalLimits,
|
||||
drag: z.object({ durationMs: z.number().int().min(1).max(5000), steps: z.number().int().min(1).max(240) }).strict(),
|
||||
paste: z.object({ maxClipboardBytes: z.number().int().min(1024), pasteReadTimeoutMs: z.number().int().min(1).max(2 ** 31 - 1),
|
||||
maxElements: z.number().int().min(1).max(10000), maxDepth: z.number().int().min(1).max(100),
|
||||
...nativeTraversalLimits.shape,
|
||||
}).strict(),
|
||||
findApps: z.object({ maxAppMatches: z.number().int().min(1).max(1000) }).strict(),
|
||||
}
|
||||
|
||||
@@ -42,6 +42,10 @@ export interface CuaAxOptions extends CuaObservationOptions {
|
||||
includeMenus?: boolean
|
||||
/** Include identifiers, global native bounds and complete action names. */
|
||||
details?: boolean
|
||||
/** Maximum nodes visited for this read. Omission uses the provider setting. */
|
||||
maxElements?: number
|
||||
/** Maximum traversal depth for this read. Omission uses the provider setting. */
|
||||
maxDepth?: number
|
||||
}
|
||||
/** Reported permissions and operations; null sessionLocked means no usable console session was verified. */
|
||||
export type CuaCapabilities = z.output<typeof CapabilitiesResult>
|
||||
@@ -103,8 +107,8 @@ export interface CuaSurface {
|
||||
/**
|
||||
* Read the full accessibility tree, with menu contents collapsed by default.
|
||||
* The returned tree becomes the application's current snapshot; use element ids from its latest successful read.
|
||||
* @param options - emission, menu expansion and field detail options.
|
||||
* @returns Full accessibility tree text with element ids, subject to the provider's traversal limits.
|
||||
* @param options - Output, traversal limits, menu expansion and field detail options.
|
||||
* @returns Accessibility tree text with element ids, up to the limits for this read.
|
||||
*/
|
||||
getAccessibilityTree(options?: CuaAxOptions): Promise<string>
|
||||
/**
|
||||
|
||||
@@ -57,8 +57,8 @@ it('adds and removes CUA for existing and new PTC Agents without changing other
|
||||
const mounted = await ctx.plugin(provider, {})
|
||||
await expect.poll(() => ctx.tools.get('cua', existing.agent)).toBeDefined()
|
||||
expect(ctx.tools.get('cua', standard.agent)).toBeUndefined()
|
||||
expect((await ctx.systemPrompt.assemble({ scope: existing.agent })).sections)
|
||||
.toContainEqual(expect.objectContaining({ name: 'computer-use:dsh-cua', text: 'Fixture CUA guidance\n\nMaximum click holdMs: 5000 ms.' }))
|
||||
const { sections } = await ctx.systemPrompt.assemble({ scope: existing.agent })
|
||||
expect(sections.find(section => section.name === 'computer-use:dsh-cua')?.text).toContain('Fixture CUA guidance')
|
||||
await ctx.agentPresets.select(existing.agent, 'ptc')
|
||||
expect(ctx.tools.schemas(existing.agent).filter(tool => tool.name === 'cua')).toHaveLength(1)
|
||||
const fresh = await create('fresh', 'ptc')
|
||||
|
||||
@@ -1028,3 +1028,18 @@ it('forwards provider options while retaining the selected target and coordinate
|
||||
appHandle: 'app-17', snapshotId: 'snapshot-1', windowId: 23, x: 250, y: 750, relative: true, trackingMode: 'coarse',
|
||||
} })
|
||||
})
|
||||
|
||||
it('forwards bounded traversal limits on a window-only provider', async () => {
|
||||
const f = fixture()
|
||||
f.setCapabilities({ ...capabilities, platform: 'windows', appLevelAccessibility: false })
|
||||
const app = await f.api.getApp('Editor')
|
||||
const win = await app.getWindow(23)
|
||||
const before = f.dispatch.mock.calls.length
|
||||
await expect(win.getAccessibilityTree({ maxDepth: 0 })).rejects.toThrow('maxDepth')
|
||||
await expect(win.getAccessibilityTree({ maxElements: 1.5 })).rejects.toThrow('maxElements')
|
||||
expect(f.dispatch).toHaveBeenCalledTimes(before)
|
||||
await win.getAccessibilityTree({ maxDepth: 1, maxElements: 1 })
|
||||
expect(f.dispatch).toHaveBeenLastCalledWith({
|
||||
method: 'getState', params: { appHandle: app.appHandle, windowId: 23, maxDepth: 1, maxElements: 1 },
|
||||
})
|
||||
})
|
||||
@@ -6,7 +6,7 @@ import { CapabilitiesResult, Request, ScreenshotResult, SystemPointerRequest } f
|
||||
const rejectedRequests: Array<{ reason: string; request: JsonValue }> = [
|
||||
{ reason: 'raw process ID instead of a selected app handle', request: { method: 'typeText', params: { pid: 17, text: 'hello' } } },
|
||||
{ reason: 'unknown native method', request: { method: 'shutdown', params: {} } },
|
||||
{ reason: 'host-owned traversal limit', request: { method: 'getState', params: { appHandle: 'app', maxElements: 1000000 } } },
|
||||
{ reason: 'zero node budget', request: { method: 'getState', params: { appHandle: 'app', maxElements: 0 } } },
|
||||
{ reason: 'extra tool argument', request: { method: 'listApps', params: {}, helperPath: '/untrusted/helper' } },
|
||||
{ reason: 'element click without an observed snapshot', request: { method: 'click', params: { appHandle: 'app', elementId: '0' } } },
|
||||
{ reason: 'numeric element id', request: { method: 'click', params: { appHandle: 'app', snapshotId: 'snapshot', elementId: 0 } } },
|
||||
@@ -66,11 +66,11 @@ it.each([
|
||||
}).success).toBe(false)
|
||||
})
|
||||
|
||||
it('accepts disclosure options without accepting host traversal limits', () => {
|
||||
const params = { appHandle: 'app', includeMenus: true, details: false }
|
||||
it('accepts disclosure options and bounded per-read traversal limits', () => {
|
||||
const params = { appHandle: 'app', includeMenus: true, details: false, maxDepth: 513, maxElements: 100001 }
|
||||
expect(Request.parse({ method: 'getState', params })).toEqual({ method: 'getState', params })
|
||||
expect(Request.safeParse({ method: 'getState', params: { ...params, details: 'all' } }).success).toBe(false)
|
||||
expect(Request.safeParse({ method: 'getState', params: { ...params, maxDepth: 100 } }).success).toBe(false)
|
||||
expect(Request.safeParse({ method: 'getState', params: { ...params, maxDepth: -1 } }).success).toBe(false)
|
||||
})
|
||||
|
||||
it.each([
|
||||
|
||||
@@ -57,7 +57,7 @@ Setup and discovery:
|
||||
Available at app or window level depending on the platform.
|
||||
|
||||
Read getAccessibilityTree explicitly before using element ids. Selection and screenshots do not read or replace the current AX snapshot. Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the tree within the provider limits; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. If the tree reports maxElements or maxDepth truncation, select a narrower window or ask the user to raise the named macOS provider setting and read again. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- listActions(elementId: string) -> string[] — list the actions an element advertises.
|
||||
- invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none.
|
||||
- performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument.
|
||||
@@ -85,4 +85,6 @@ Raw tools use { method, params }: include appHandle for app calls and windowId f
|
||||
|
||||
Only if the task cannot be completed without moving the user’s pointer, call await win.mouseMove(target, { useSystemPointer: true }). Prefer an advertised accessibility action, a click on a visible control, or browser tools when possible. The question interrupts the user. When approval is disabled an explicit move can act immediately. Pass true on every physical move, even after approval. Omitting the option or passing false uses the default hover route without prompting. Approval or refusal lasts until this session closes or DSH restarts. A refusal sends no input. If the user dismisses the request, ask again only if they request it. Wait for an outstanding request to finish. Subagents cannot request physical movement.
|
||||
|
||||
AX read defaults: maxElements=5000, maxDepth=50.
|
||||
|
||||
Maximum click holdMs: 5000 ms.
|
||||
@@ -304,6 +304,18 @@
|
||||
"details": {
|
||||
"description": "Include identifiers, global native bounds and complete action names.",
|
||||
"type": "boolean"
|
||||
},
|
||||
"maxElements": {
|
||||
"description": "Maximum nodes to visit on this read. Omission uses the provider setting.",
|
||||
"type": "number",
|
||||
"minimum": 1,
|
||||
"multipleOf": 1
|
||||
},
|
||||
"maxDepth": {
|
||||
"description": "Maximum depth to traverse on this read. Omission uses the provider setting.",
|
||||
"type": "number",
|
||||
"minimum": 1,
|
||||
"multipleOf": 1
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
|
||||
@@ -178,6 +178,9 @@ function createApplicationState(socket, record, application, captureBroker, conn
|
||||
snapshots.set(params.windowId ?? null, { snapshotId, windowId: params.windowId ?? null, axless })
|
||||
if (axless) return { appHandle: app.appHandle, windowId: 8, snapshotId,
|
||||
tree: '', elements: [], windows: [axlessWindow], truncated: false }
|
||||
if (params.maxElements === 1) return { appHandle: app.appHandle, windowId: params.windowId ?? null, snapshotId,
|
||||
tree: `AX scope=${params.windowId === undefined ? 'application' : 'window'} offsets=UTF-16\n- [0] AXButton "Increment"\nAccessibility tree truncated (maxElements=1).`,
|
||||
elements: [], windows: [window], truncated: true }
|
||||
return { appHandle: app.appHandle, windowId: params.windowId ?? null,
|
||||
snapshotId: `${scope}-snapshot-${generation}`, tree: `AX scope=${params.windowId === undefined ? 'application' : 'window'} offsets=UTF-16${params.details ? ' bounds=global-points' : ''}\n- [0] AXButton "Increment"${params.details ? ' id="increment" actions=["AXPress"] bounds=[60,80,100,20]' : ''}\n- [1] AXTextField valueWritable=${writableFailure === 'AXValue' ? 'unknown' : 'true'} selectionWritable=${writableFailure === 'AXSelectedTextRange' ? 'unknown' : 'true'} value=${JSON.stringify(application.value)} placeholder="Fixture text"\n- AXStaticText "Count: ${application.count}"${application.hovering ? '\n- AXStaticText "Hover information"' : ''}\n- [2] AXMenuBarItem "File"${params.includeMenus ? '\n - [3] AXMenuItem "Fixture command"' : '\nMenu contents omitted; request includeMenus=true to expand.'}`,
|
||||
elements: [{ elementId: '0', role: 'AXButton', title: 'Increment', value: null, description: null,
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
"type": "tool-call",
|
||||
"id": "cua-ptc-native-result",
|
||||
"name": "run_code",
|
||||
"arguments": "{\"code\":\"const { Buffer } = await import('node:buffer');\\nconst app = await tools.cua({ method: 'getApp', params: { selector: 'test.fixture' } });\\nconst state = await tools.cua({ method: 'getState', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst shot = await tools.cua({ method: 'screenshot', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst bytes = Buffer.from(shot.data, 'base64');\\nreturn { appHandle: app.appHandle, hasIncrement: state.tree.includes('Increment'), pngBytes: bytes.length, signature: bytes.subarray(0, 8).toString('hex'), imageWidth: shot.attachment.width, imageHeight: shot.attachment.height };\",\"description\":\"Read native CUA values and receive one screenshot\"}"
|
||||
"arguments": "{\"code\":\"const { Buffer } = await import('node:buffer');\\nconst app = await tools.cua({ method: 'getApp', params: { selector: 'test.fixture' } });\\nconst target = { appHandle: app.appHandle, windowId: 7 };\\nconst first = await tools.cua({ method: 'getState', params: { ...target, maxElements: 1 } });\\nconst state = first.truncated ? await tools.cua({ method: 'getState', params: { ...target, maxElements: 16 } }) : first;\\nconst shot = await tools.cua({ method: 'screenshot', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst bytes = Buffer.from(shot.data, 'base64');\\nreturn { appHandle: app.appHandle, initiallyTruncated: first.truncated, recoveredTextField: !first.tree.includes('AXTextField') && state.tree.includes('AXTextField'), hasIncrement: state.tree.includes('Increment'), pngBytes: bytes.length, signature: bytes.subarray(0, 8).toString('hex'), imageWidth: shot.attachment.width, imageHeight: shot.attachment.height };\",\"description\":\"Read native CUA values and receive one screenshot\"}"
|
||||
}
|
||||
},
|
||||
{
|
||||
|
||||
@@ -12,21 +12,23 @@
|
||||
{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"tools":"{{tools}}"},"reason":"initial"}}
|
||||
{"type":"request/context","data":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"}}
|
||||
{"type":"session/title","data":{"title":"Use tools.cua from Node PTC","messageSeqs":[8],"source":{"kind":"fallback"}}}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-ptc-native-result","name":"run_code","arguments":"{\"code\":\"const { Buffer } = await import('node:buffer');\\nconst app = await tools.cua({ method: 'getApp', params: { selector: 'test.fixture' } });\\nconst state = await tools.cua({ method: 'getState', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst shot = await tools.cua({ method: 'screenshot', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst bytes = Buffer.from(shot.data, 'base64');\\nreturn { appHandle: app.appHandle, hasIncrement: state.tree.includes('Increment'), pngBytes: bytes.length, signature: bytes.subarray(0, 8).toString('hex'), imageWidth: shot.attachment.width, imageHeight: shot.attachment.height };\",\"description\":\"Read native CUA values and receive one screenshot\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:4}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1794213719496,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1794213719496,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-ptc-native-result","name":"run_code","arguments":"{\"code\":\"const { Buffer } = await import('node:buffer');\\nconst app = await tools.cua({ method: 'getApp', params: { selector: 'test.fixture' } });\\nconst state = await tools.cua({ method: 'getState', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst shot = await tools.cua({ method: 'screenshot', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst bytes = Buffer.from(shot.data, 'base64');\\nreturn { appHandle: app.appHandle, hasIncrement: state.tree.includes('Increment'), pngBytes: bytes.length, signature: bytes.subarray(0, 8).toString('hex'), imageWidth: shot.attachment.width, imageHeight: shot.attachment.height };\",\"description\":\"Read native CUA values and receive one screenshot\"}"}}},{"type":"chunk","time":1794213719496,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1794213719496,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"}
|
||||
{"type":"tool/call","data":{"turn":1,"step":1,"callId":"cua-ptc-native-result","name":"run_code","arguments":"{\"code\":\"const { Buffer } = await import('node:buffer');\\nconst app = await tools.cua({ method: 'getApp', params: { selector: 'test.fixture' } });\\nconst state = await tools.cua({ method: 'getState', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst shot = await tools.cua({ method: 'screenshot', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst bytes = Buffer.from(shot.data, 'base64');\\nreturn { appHandle: app.appHandle, hasIncrement: state.tree.includes('Increment'), pngBytes: bytes.length, signature: bytes.subarray(0, 8).toString('hex'), imageWidth: shot.attachment.width, imageHeight: shot.attachment.height };\",\"description\":\"Read native CUA values and receive one screenshot\"}"}}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-ptc-native-result","name":"run_code","arguments":"{\"code\":\"const { Buffer } = await import('node:buffer');\\nconst app = await tools.cua({ method: 'getApp', params: { selector: 'test.fixture' } });\\nconst target = { appHandle: app.appHandle, windowId: 7 };\\nconst first = await tools.cua({ method: 'getState', params: { ...target, maxElements: 1 } });\\nconst state = first.truncated ? await tools.cua({ method: 'getState', params: { ...target, maxElements: 16 } }) : first;\\nconst shot = await tools.cua({ method: 'screenshot', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst bytes = Buffer.from(shot.data, 'base64');\\nreturn { appHandle: app.appHandle, initiallyTruncated: first.truncated, recoveredTextField: !first.tree.includes('AXTextField') && state.tree.includes('AXTextField'), hasIncrement: state.tree.includes('Increment'), pngBytes: bytes.length, signature: bytes.subarray(0, 8).toString('hex'), imageWidth: shot.attachment.width, imageHeight: shot.attachment.height };\",\"description\":\"Read native CUA values and receive one screenshot\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:4}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1794214206474,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1794214206475,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-ptc-native-result","name":"run_code","arguments":"{\"code\":\"const { Buffer } = await import('node:buffer');\\nconst app = await tools.cua({ method: 'getApp', params: { selector: 'test.fixture' } });\\nconst target = { appHandle: app.appHandle, windowId: 7 };\\nconst first = await tools.cua({ method: 'getState', params: { ...target, maxElements: 1 } });\\nconst state = first.truncated ? await tools.cua({ method: 'getState', params: { ...target, maxElements: 16 } }) : first;\\nconst shot = await tools.cua({ method: 'screenshot', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst bytes = Buffer.from(shot.data, 'base64');\\nreturn { appHandle: app.appHandle, initiallyTruncated: first.truncated, recoveredTextField: !first.tree.includes('AXTextField') && state.tree.includes('AXTextField'), hasIncrement: state.tree.includes('Increment'), pngBytes: bytes.length, signature: bytes.subarray(0, 8).toString('hex'), imageWidth: shot.attachment.width, imageHeight: shot.attachment.height };\",\"description\":\"Read native CUA values and receive one screenshot\"}"}}},{"type":"chunk","time":1794214206475,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1794214206475,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"}
|
||||
{"type":"tool/call","data":{"turn":1,"step":1,"callId":"cua-ptc-native-result","name":"run_code","arguments":"{\"code\":\"const { Buffer } = await import('node:buffer');\\nconst app = await tools.cua({ method: 'getApp', params: { selector: 'test.fixture' } });\\nconst target = { appHandle: app.appHandle, windowId: 7 };\\nconst first = await tools.cua({ method: 'getState', params: { ...target, maxElements: 1 } });\\nconst state = first.truncated ? await tools.cua({ method: 'getState', params: { ...target, maxElements: 16 } }) : first;\\nconst shot = await tools.cua({ method: 'screenshot', params: { appHandle: app.appHandle, windowId: 7 } });\\nconst bytes = Buffer.from(shot.data, 'base64');\\nreturn { appHandle: app.appHandle, initiallyTruncated: first.truncated, recoveredTextField: !first.tree.includes('AXTextField') && state.tree.includes('AXTextField'), hasIncrement: state.tree.includes('Increment'), pngBytes: bytes.length, signature: bytes.subarray(0, 8).toString('hex'), imageWidth: shot.attachment.width, imageHeight: shot.attachment.height };\",\"description\":\"Read native CUA values and receive one screenshot\"}"}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:1","name":"cua","arguments":{"method":"getApp","params":{"selector":"test.fixture"}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:1","name":"cua","arguments":{"method":"getApp","params":{"selector":"test.fixture"}},"isError":false,"content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"name\":\"Fixture\",\"bundleId\":\"test.fixture\",\"running\":true}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:2","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app","windowId":7}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:2","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app","windowId":7}},"isError":false,"content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-1\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 0\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:3","name":"cua","arguments":{"method":"screenshot","params":{"appHandle":"fixture-app","windowId":7}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:3","name":"cua","arguments":{"method":"screenshot","params":{"appHandle":"fixture-app","windowId":7}},"isError":false,"content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}]}}
|
||||
{"type":"tool/result","data":{"turn":1,"step":1,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-ptc-native-result"},"toolCallId":"cua-ptc-native-result","content":[{"type":"text","text":"{\n \"appHandle\": \"fixture-app\",\n \"hasIncrement\": true,\n \"pngBytes\": 90,\n \"signature\": \"89504e470d0a1a0a\",\n \"imageWidth\": 1,\n \"imageHeight\": 1\n}"}],"isError":false,"id":"{{message:5}}"}},"sourceEventSeqs":[14],"surfaceOp":"append"}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:2","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app","windowId":7,"maxElements":1}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:2","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app","windowId":7,"maxElements":1}},"isError":false,"content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-1\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\nAccessibility tree truncated (maxElements=1).\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":true}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:3","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app","windowId":7,"maxElements":16}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:3","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app","windowId":7,"maxElements":16}},"isError":false,"content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-2\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 0\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:4","name":"cua","arguments":{"method":"screenshot","params":{"appHandle":"fixture-app","windowId":7}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-native-result","parentCallId":"cua-ptc-native-result","subCallId":"cua-ptc-native-result:ptc:4","name":"cua","arguments":{"method":"screenshot","params":{"appHandle":"fixture-app","windowId":7}},"isError":false,"content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}]}}
|
||||
{"type":"tool/result","data":{"turn":1,"step":1,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-ptc-native-result"},"toolCallId":"cua-ptc-native-result","content":[{"type":"text","text":"{\n \"appHandle\": \"fixture-app\",\n \"initiallyTruncated\": true,\n \"recoveredTextField\": true,\n \"hasIncrement\": true,\n \"pngBytes\": 90,\n \"signature\": \"89504e470d0a1a0a\",\n \"imageWidth\": 1,\n \"imageHeight\": 1\n}"}],"isError":false,"id":"{{message:5}}"}},"sourceEventSeqs":[14],"surfaceOp":"append"}
|
||||
{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"inserted":[{"content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}],"source":{"kind":"ptc-mode"},"role":"user","id":"{{message:6}}"}]}}
|
||||
{"type":"step/end","data":{"turn":1,"step":1}}
|
||||
{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"removedCount":1,"inserted":[]}}
|
||||
{"type":"step/start","data":{"turn":1,"step":2}}
|
||||
{"type":"user/message","data":{"content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}],"source":{"kind":"ptc-mode"},"role":"user","id":"{{message:6}}"},"surfaceOp":"append"}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-ptc-ax-failure","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nawait writeFile('fixture-ax-failure', '');\\ntry {\\n const cua = connectCua(() => tools.cua);\\n const app = await cua.getApp('test.fixture');\\n const windows = await app.getWindows({ emit: false });\\n const window = await app.getWindow(windows[0].windowId);\\n const failures = [];\\n for (const surface of [app, window]) {\\n try { await surface.getAccessibilityTree(); throw new Error('Expected AX failure'); }\\n catch (error) { if (!error.message.includes('Fixture accessibility read failed')) throw error; failures.push(error.message); }\\n }\\n let coordinateRefused = false;\\n try { await window.click([100, 100], { relative: true }); }\\n catch (error) { if (!error.message.includes('Window has no observation')) throw error; coordinateRefused = true; }\\n if (!coordinateRefused) throw new Error('Coordinate input accepted without observation');\\n const shot = await window.getScreenshot();\\n const coordinateDispatched = (await window.click([100, 100], { relative: true })).dispatched;\\n let elementRefused = false;\\n try { await window.invoke('0'); }\\n catch (error) { if (!error.message.includes('Read getAccessibilityTree')) throw error; elementRefused = true; }\\n if (!elementRefused) throw new Error('Element input accepted without an AX snapshot');\\n return { axFailures: failures.length, screenshotWindow: shot.windowId, pngBytes: shot.bytes.length, elementRefused, coordinateRefused, coordinateDispatched };\\n} finally { await unlink('fixture-ax-failure'); }\",\"description\":\"Capture through the object API despite application and window AX failures\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:7}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1794213720632,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1794213720632,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-ptc-ax-failure","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nawait writeFile('fixture-ax-failure', '');\\ntry {\\n const cua = connectCua(() => tools.cua);\\n const app = await cua.getApp('test.fixture');\\n const windows = await app.getWindows({ emit: false });\\n const window = await app.getWindow(windows[0].windowId);\\n const failures = [];\\n for (const surface of [app, window]) {\\n try { await surface.getAccessibilityTree(); throw new Error('Expected AX failure'); }\\n catch (error) { if (!error.message.includes('Fixture accessibility read failed')) throw error; failures.push(error.message); }\\n }\\n let coordinateRefused = false;\\n try { await window.click([100, 100], { relative: true }); }\\n catch (error) { if (!error.message.includes('Window has no observation')) throw error; coordinateRefused = true; }\\n if (!coordinateRefused) throw new Error('Coordinate input accepted without observation');\\n const shot = await window.getScreenshot();\\n const coordinateDispatched = (await window.click([100, 100], { relative: true })).dispatched;\\n let elementRefused = false;\\n try { await window.invoke('0'); }\\n catch (error) { if (!error.message.includes('Read getAccessibilityTree')) throw error; elementRefused = true; }\\n if (!elementRefused) throw new Error('Element input accepted without an AX snapshot');\\n return { axFailures: failures.length, screenshotWindow: shot.windowId, pngBytes: shot.bytes.length, elementRefused, coordinateRefused, coordinateDispatched };\\n} finally { await unlink('fixture-ax-failure'); }\",\"description\":\"Capture through the object API despite application and window AX failures\"}"}}},{"type":"chunk","time":1794213720632,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1794213720632,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-ptc-ax-failure","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nawait writeFile('fixture-ax-failure', '');\\ntry {\\n const cua = connectCua(() => tools.cua);\\n const app = await cua.getApp('test.fixture');\\n const windows = await app.getWindows({ emit: false });\\n const window = await app.getWindow(windows[0].windowId);\\n const failures = [];\\n for (const surface of [app, window]) {\\n try { await surface.getAccessibilityTree(); throw new Error('Expected AX failure'); }\\n catch (error) { if (!error.message.includes('Fixture accessibility read failed')) throw error; failures.push(error.message); }\\n }\\n let coordinateRefused = false;\\n try { await window.click([100, 100], { relative: true }); }\\n catch (error) { if (!error.message.includes('Window has no observation')) throw error; coordinateRefused = true; }\\n if (!coordinateRefused) throw new Error('Coordinate input accepted without observation');\\n const shot = await window.getScreenshot();\\n const coordinateDispatched = (await window.click([100, 100], { relative: true })).dispatched;\\n let elementRefused = false;\\n try { await window.invoke('0'); }\\n catch (error) { if (!error.message.includes('Read getAccessibilityTree')) throw error; elementRefused = true; }\\n if (!elementRefused) throw new Error('Element input accepted without an AX snapshot');\\n return { axFailures: failures.length, screenshotWindow: shot.windowId, pngBytes: shot.bytes.length, elementRefused, coordinateRefused, coordinateDispatched };\\n} finally { await unlink('fixture-ax-failure'); }\",\"description\":\"Capture through the object API despite application and window AX failures\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:7}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1794214207288,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1794214207288,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-ptc-ax-failure","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nawait writeFile('fixture-ax-failure', '');\\ntry {\\n const cua = connectCua(() => tools.cua);\\n const app = await cua.getApp('test.fixture');\\n const windows = await app.getWindows({ emit: false });\\n const window = await app.getWindow(windows[0].windowId);\\n const failures = [];\\n for (const surface of [app, window]) {\\n try { await surface.getAccessibilityTree(); throw new Error('Expected AX failure'); }\\n catch (error) { if (!error.message.includes('Fixture accessibility read failed')) throw error; failures.push(error.message); }\\n }\\n let coordinateRefused = false;\\n try { await window.click([100, 100], { relative: true }); }\\n catch (error) { if (!error.message.includes('Window has no observation')) throw error; coordinateRefused = true; }\\n if (!coordinateRefused) throw new Error('Coordinate input accepted without observation');\\n const shot = await window.getScreenshot();\\n const coordinateDispatched = (await window.click([100, 100], { relative: true })).dispatched;\\n let elementRefused = false;\\n try { await window.invoke('0'); }\\n catch (error) { if (!error.message.includes('Read getAccessibilityTree')) throw error; elementRefused = true; }\\n if (!elementRefused) throw new Error('Element input accepted without an AX snapshot');\\n return { axFailures: failures.length, screenshotWindow: shot.windowId, pngBytes: shot.bytes.length, elementRefused, coordinateRefused, coordinateDispatched };\\n} finally { await unlink('fixture-ax-failure'); }\",\"description\":\"Capture through the object API despite application and window AX failures\"}"}}},{"type":"chunk","time":1794214207288,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1794214207288,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"}
|
||||
{"type":"tool/call","data":{"turn":1,"step":2,"callId":"cua-ptc-ax-failure","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nawait writeFile('fixture-ax-failure', '');\\ntry {\\n const cua = connectCua(() => tools.cua);\\n const app = await cua.getApp('test.fixture');\\n const windows = await app.getWindows({ emit: false });\\n const window = await app.getWindow(windows[0].windowId);\\n const failures = [];\\n for (const surface of [app, window]) {\\n try { await surface.getAccessibilityTree(); throw new Error('Expected AX failure'); }\\n catch (error) { if (!error.message.includes('Fixture accessibility read failed')) throw error; failures.push(error.message); }\\n }\\n let coordinateRefused = false;\\n try { await window.click([100, 100], { relative: true }); }\\n catch (error) { if (!error.message.includes('Window has no observation')) throw error; coordinateRefused = true; }\\n if (!coordinateRefused) throw new Error('Coordinate input accepted without observation');\\n const shot = await window.getScreenshot();\\n const coordinateDispatched = (await window.click([100, 100], { relative: true })).dispatched;\\n let elementRefused = false;\\n try { await window.invoke('0'); }\\n catch (error) { if (!error.message.includes('Read getAccessibilityTree')) throw error; elementRefused = true; }\\n if (!elementRefused) throw new Error('Element input accepted without an AX snapshot');\\n return { axFailures: failures.length, screenshotWindow: shot.windowId, pngBytes: shot.bytes.length, elementRefused, coordinateRefused, coordinateDispatched };\\n} finally { await unlink('fixture-ax-failure'); }\",\"description\":\"Capture through the object API despite application and window AX failures\"}"}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-ax-failure","parentCallId":"cua-ptc-ax-failure","subCallId":"cua-ptc-ax-failure:ptc:1","name":"cua","arguments":{"method":"getCapabilities","params":{}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-ax-failure","parentCallId":"cua-ptc-ax-failure","subCallId":"cua-ptc-ax-failure:ptc:1","name":"cua","arguments":{"method":"getCapabilities","params":{}},"isError":false,"content":[{"type":"text","text":"{\"sessionLocked\":false,\"permissions\":{\"accessibility\":true,\"screenRecording\":true},\"platform\":\"macos\",\"systemPointer\":true,\"methods\":[\"activate\",\"click\",\"drag\",\"findApps\",\"getApp\",\"getCapabilities\",\"getState\",\"invoke\",\"listActions\",\"listApps\",\"listWindows\",\"mouseMove\",\"paste\",\"performAction\",\"pressKey\",\"screenshot\",\"scroll\",\"selectText\",\"sendKeyEvent\",\"setValue\",\"show\",\"typeText\"],\"appLevelAccessibility\":true,\"keyboard\":[\"app\"],\"browser\":false,\"coordinateSpace\":\"screenshot-pixels\"}"}]}}
|
||||
@@ -44,29 +46,29 @@
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-ax-failure","parentCallId":"cua-ptc-ax-failure","subCallId":"cua-ptc-ax-failure:ptc:7","name":"cua","arguments":{"method":"screenshot","params":{"appHandle":"fixture-app","windowId":7}},"isError":false,"content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-ax-failure","parentCallId":"cua-ptc-ax-failure","subCallId":"cua-ptc-ax-failure:ptc:8","name":"cua","arguments":{"method":"click","params":{"appHandle":"fixture-app","windowId":7,"x":100,"y":100,"relative":true,"mouseButton":"left","clickCount":1}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-ax-failure","parentCallId":"cua-ptc-ax-failure","subCallId":"cua-ptc-ax-failure:ptc:8","name":"cua","arguments":{"method":"click","params":{"appHandle":"fixture-app","windowId":7,"x":100,"y":100,"relative":true,"mouseButton":"left","clickCount":1}},"isError":false,"content":[{"type":"text","text":"{\"dispatched\":true}"}]}}
|
||||
{"type":"tool/result","data":{"turn":1,"step":2,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-ptc-ax-failure"},"toolCallId":"cua-ptc-ax-failure","content":[{"type":"text","text":"{\n \"axFailures\": 2,\n \"screenshotWindow\": 7,\n \"pngBytes\": 90,\n \"elementRefused\": true,\n \"coordinateRefused\": true,\n \"coordinateDispatched\": true\n}"}],"isError":false,"id":"{{message:8}}"}},"sourceEventSeqs":[28],"surfaceOp":"append"}
|
||||
{"type":"tool/result","data":{"turn":1,"step":2,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-ptc-ax-failure"},"toolCallId":"cua-ptc-ax-failure","content":[{"type":"text","text":"{\n \"axFailures\": 2,\n \"screenshotWindow\": 7,\n \"pngBytes\": 90,\n \"elementRefused\": true,\n \"coordinateRefused\": true,\n \"coordinateDispatched\": true\n}"}],"isError":false,"id":"{{message:8}}"}},"sourceEventSeqs":[30],"surfaceOp":"append"}
|
||||
{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"inserted":[{"content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}],"source":{"kind":"ptc-mode"},"role":"user","id":"{{message:9}}"}]}}
|
||||
{"type":"step/end","data":{"turn":1,"step":2}}
|
||||
{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"removedCount":1,"inserted":[]}}
|
||||
{"type":"step/start","data":{"turn":1,"step":3}}
|
||||
{"type":"user/message","data":{"content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}],"source":{"kind":"ptc-mode"},"role":"user","id":"{{message:9}}"},"surfaceOp":"append"}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":3,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-ptc-writability","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nconst cua = connectCua(() => tools.cua);\\nconst app = await cua.getApp('test.fixture');\\nconst results = [];\\nfor (const attribute of ['AXValue', 'AXSelectedTextRange']) {\\n await writeFile('fixture-ax-writability-failure', attribute);\\n try {\\n const tree = await app.getAccessibilityTree({ emit: false });\\n const field = attribute === 'AXValue' ? 'valueWritable' : 'selectionWritable';\\n if (!tree.includes(field + '=unknown') || tree.includes(field + '=true')) throw new Error('Unknown writability was lost');\\n if (!tree.includes('Increment') || !tree.includes('Count:')) throw new Error('Other nodes were lost');\\n let writeRefused = false;\\n try {\\n if (attribute === 'AXValue') await app.setValue('1', 'must not be written');\\n else await app.selectText('1', 'Fixture');\\n } catch (error) {\\n if (!error.message.includes('writability of ' + attribute + ' failed with error -25200')) throw error;\\n writeRefused = true;\\n }\\n if (!writeRefused) throw new Error('Write accepted with unknown writability');\\n results.push({ attribute, unknown: true, otherNodesRetained: true, writeRefused });\\n } finally { await unlink('fixture-ax-writability-failure'); }\\n}\\nreturn results;\",\"description\":\"Retain AX trees with unknown writability and reject writes\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:10}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1794213720832,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1794213720832,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-ptc-writability","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nconst cua = connectCua(() => tools.cua);\\nconst app = await cua.getApp('test.fixture');\\nconst results = [];\\nfor (const attribute of ['AXValue', 'AXSelectedTextRange']) {\\n await writeFile('fixture-ax-writability-failure', attribute);\\n try {\\n const tree = await app.getAccessibilityTree({ emit: false });\\n const field = attribute === 'AXValue' ? 'valueWritable' : 'selectionWritable';\\n if (!tree.includes(field + '=unknown') || tree.includes(field + '=true')) throw new Error('Unknown writability was lost');\\n if (!tree.includes('Increment') || !tree.includes('Count:')) throw new Error('Other nodes were lost');\\n let writeRefused = false;\\n try {\\n if (attribute === 'AXValue') await app.setValue('1', 'must not be written');\\n else await app.selectText('1', 'Fixture');\\n } catch (error) {\\n if (!error.message.includes('writability of ' + attribute + ' failed with error -25200')) throw error;\\n writeRefused = true;\\n }\\n if (!writeRefused) throw new Error('Write accepted with unknown writability');\\n results.push({ attribute, unknown: true, otherNodesRetained: true, writeRefused });\\n } finally { await unlink('fixture-ax-writability-failure'); }\\n}\\nreturn results;\",\"description\":\"Retain AX trees with unknown writability and reject writes\"}"}}},{"type":"chunk","time":1794213720832,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1794213720832,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":3,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-ptc-writability","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nconst cua = connectCua(() => tools.cua);\\nconst app = await cua.getApp('test.fixture');\\nconst results = [];\\nfor (const attribute of ['AXValue', 'AXSelectedTextRange']) {\\n await writeFile('fixture-ax-writability-failure', attribute);\\n try {\\n const tree = await app.getAccessibilityTree({ emit: false });\\n const field = attribute === 'AXValue' ? 'valueWritable' : 'selectionWritable';\\n if (!tree.includes(field + '=unknown') || tree.includes(field + '=true')) throw new Error('Unknown writability was lost');\\n if (!tree.includes('Increment') || !tree.includes('Count:')) throw new Error('Other nodes were lost');\\n let writeRefused = false;\\n try {\\n if (attribute === 'AXValue') await app.setValue('1', 'must not be written');\\n else await app.selectText('1', 'Fixture');\\n } catch (error) {\\n if (!error.message.includes('writability of ' + attribute + ' failed with error -25200')) throw error;\\n writeRefused = true;\\n }\\n if (!writeRefused) throw new Error('Write accepted with unknown writability');\\n results.push({ attribute, unknown: true, otherNodesRetained: true, writeRefused });\\n } finally { await unlink('fixture-ax-writability-failure'); }\\n}\\nreturn results;\",\"description\":\"Retain AX trees with unknown writability and reject writes\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:10}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1794214207512,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1794214207512,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-ptc-writability","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nconst cua = connectCua(() => tools.cua);\\nconst app = await cua.getApp('test.fixture');\\nconst results = [];\\nfor (const attribute of ['AXValue', 'AXSelectedTextRange']) {\\n await writeFile('fixture-ax-writability-failure', attribute);\\n try {\\n const tree = await app.getAccessibilityTree({ emit: false });\\n const field = attribute === 'AXValue' ? 'valueWritable' : 'selectionWritable';\\n if (!tree.includes(field + '=unknown') || tree.includes(field + '=true')) throw new Error('Unknown writability was lost');\\n if (!tree.includes('Increment') || !tree.includes('Count:')) throw new Error('Other nodes were lost');\\n let writeRefused = false;\\n try {\\n if (attribute === 'AXValue') await app.setValue('1', 'must not be written');\\n else await app.selectText('1', 'Fixture');\\n } catch (error) {\\n if (!error.message.includes('writability of ' + attribute + ' failed with error -25200')) throw error;\\n writeRefused = true;\\n }\\n if (!writeRefused) throw new Error('Write accepted with unknown writability');\\n results.push({ attribute, unknown: true, otherNodesRetained: true, writeRefused });\\n } finally { await unlink('fixture-ax-writability-failure'); }\\n}\\nreturn results;\",\"description\":\"Retain AX trees with unknown writability and reject writes\"}"}}},{"type":"chunk","time":1794214207512,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1794214207512,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"}
|
||||
{"type":"tool/call","data":{"turn":1,"step":3,"callId":"cua-ptc-writability","name":"run_code","arguments":"{\"code\":\"const { writeFile, unlink } = await import('node:fs/promises');\\nconst { resolve } = await import('node:path');\\nconst { pathToFileURL } = await import('node:url');\\nconst { connectCua } = await import(pathToFileURL(resolve('cua-bridge.mjs')).href);\\nconst cua = connectCua(() => tools.cua);\\nconst app = await cua.getApp('test.fixture');\\nconst results = [];\\nfor (const attribute of ['AXValue', 'AXSelectedTextRange']) {\\n await writeFile('fixture-ax-writability-failure', attribute);\\n try {\\n const tree = await app.getAccessibilityTree({ emit: false });\\n const field = attribute === 'AXValue' ? 'valueWritable' : 'selectionWritable';\\n if (!tree.includes(field + '=unknown') || tree.includes(field + '=true')) throw new Error('Unknown writability was lost');\\n if (!tree.includes('Increment') || !tree.includes('Count:')) throw new Error('Other nodes were lost');\\n let writeRefused = false;\\n try {\\n if (attribute === 'AXValue') await app.setValue('1', 'must not be written');\\n else await app.selectText('1', 'Fixture');\\n } catch (error) {\\n if (!error.message.includes('writability of ' + attribute + ' failed with error -25200')) throw error;\\n writeRefused = true;\\n }\\n if (!writeRefused) throw new Error('Write accepted with unknown writability');\\n results.push({ attribute, unknown: true, otherNodesRetained: true, writeRefused });\\n } finally { await unlink('fixture-ax-writability-failure'); }\\n}\\nreturn results;\",\"description\":\"Retain AX trees with unknown writability and reject writes\"}"}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:1","name":"cua","arguments":{"method":"getCapabilities","params":{}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:1","name":"cua","arguments":{"method":"getCapabilities","params":{}},"isError":false,"content":[{"type":"text","text":"{\"sessionLocked\":false,\"permissions\":{\"accessibility\":true,\"screenRecording\":true},\"platform\":\"macos\",\"systemPointer\":true,\"methods\":[\"activate\",\"click\",\"drag\",\"findApps\",\"getApp\",\"getCapabilities\",\"getState\",\"invoke\",\"listActions\",\"listApps\",\"listWindows\",\"mouseMove\",\"paste\",\"performAction\",\"pressKey\",\"screenshot\",\"scroll\",\"selectText\",\"sendKeyEvent\",\"setValue\",\"show\",\"typeText\"],\"appLevelAccessibility\":true,\"keyboard\":[\"app\"],\"browser\":false,\"coordinateSpace\":\"screenshot-pixels\"}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:2","name":"cua","arguments":{"method":"getApp","params":{"selector":"test.fixture"}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:2","name":"cua","arguments":{"method":"getApp","params":{"selector":"test.fixture"}},"isError":false,"content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"name\":\"Fixture\",\"bundleId\":\"test.fixture\",\"running\":true}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:3","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app"}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:3","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app"}},"isError":false,"content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":null,\"snapshotId\":\"fixture-snapshot-2\",\"tree\":\"AX scope=application offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=unknown selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 1\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:4","name":"cua","arguments":{"method":"setValue","params":{"appHandle":"fixture-app","snapshotId":"fixture-snapshot-2","elementId":"1","value":"must not be written"}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:4","name":"cua","arguments":{"method":"setValue","params":{"appHandle":"fixture-app","snapshotId":"fixture-snapshot-2","elementId":"1","value":"must not be written"}},"isError":true,"content":[{"type":"text","text":"Error: NATIVE_ERROR: Accessibility read writability of AXValue failed with error -25200"}]}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:3","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app"}},"isError":false,"content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":null,\"snapshotId\":\"fixture-snapshot-3\",\"tree\":\"AX scope=application offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=unknown selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 1\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:4","name":"cua","arguments":{"method":"setValue","params":{"appHandle":"fixture-app","snapshotId":"fixture-snapshot-3","elementId":"1","value":"must not be written"}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:4","name":"cua","arguments":{"method":"setValue","params":{"appHandle":"fixture-app","snapshotId":"fixture-snapshot-3","elementId":"1","value":"must not be written"}},"isError":true,"content":[{"type":"text","text":"Error: NATIVE_ERROR: Accessibility read writability of AXValue failed with error -25200"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:5","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app"}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:5","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app"}},"isError":false,"content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":null,\"snapshotId\":\"fixture-snapshot-3\",\"tree\":\"AX scope=application offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=unknown value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 1\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:6","name":"cua","arguments":{"method":"selectText","params":{"appHandle":"fixture-app","snapshotId":"fixture-snapshot-3","elementId":"1","text":"Fixture"}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:6","name":"cua","arguments":{"method":"selectText","params":{"appHandle":"fixture-app","snapshotId":"fixture-snapshot-3","elementId":"1","text":"Fixture"}},"isError":true,"content":[{"type":"text","text":"Error: NATIVE_ERROR: Accessibility read writability of AXSelectedTextRange failed with error -25200"}]}}
|
||||
{"type":"tool/result","data":{"turn":1,"step":3,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-ptc-writability"},"toolCallId":"cua-ptc-writability","content":[{"type":"text","text":"[\n {\n \"attribute\": \"AXValue\",\n \"unknown\": true,\n \"otherNodesRetained\": true,\n \"writeRefused\": true\n },\n {\n \"attribute\": \"AXSelectedTextRange\",\n \"unknown\": true,\n \"otherNodesRetained\": true,\n \"writeRefused\": true\n }\n]"}],"isError":false,"id":"{{message:11}}"}},"sourceEventSeqs":[52],"surfaceOp":"append"}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:5","name":"cua","arguments":{"method":"getState","params":{"appHandle":"fixture-app"}},"isError":false,"content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":null,\"snapshotId\":\"fixture-snapshot-4\",\"tree\":\"AX scope=application offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=unknown value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 1\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}]}}
|
||||
{"type":"tool/ptc-dispatch-start","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:6","name":"cua","arguments":{"method":"selectText","params":{"appHandle":"fixture-app","snapshotId":"fixture-snapshot-4","elementId":"1","text":"Fixture"}}}}
|
||||
{"type":"tool/ptc-dispatch","data":{"rootCallId":"cua-ptc-writability","parentCallId":"cua-ptc-writability","subCallId":"cua-ptc-writability:ptc:6","name":"cua","arguments":{"method":"selectText","params":{"appHandle":"fixture-app","snapshotId":"fixture-snapshot-4","elementId":"1","text":"Fixture"}},"isError":true,"content":[{"type":"text","text":"Error: NATIVE_ERROR: Accessibility read writability of AXSelectedTextRange failed with error -25200"}]}}
|
||||
{"type":"tool/result","data":{"turn":1,"step":3,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-ptc-writability"},"toolCallId":"cua-ptc-writability","content":[{"type":"text","text":"[\n {\n \"attribute\": \"AXValue\",\n \"unknown\": true,\n \"otherNodesRetained\": true,\n \"writeRefused\": true\n },\n {\n \"attribute\": \"AXSelectedTextRange\",\n \"unknown\": true,\n \"otherNodesRetained\": true,\n \"writeRefused\": true\n }\n]"}],"isError":false,"id":"{{message:11}}"}},"sourceEventSeqs":[54],"surfaceOp":"append"}
|
||||
{"type":"step/end","data":{"turn":1,"step":3}}
|
||||
{"type":"step/start","data":{"turn":1,"step":4}}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":4,"message":{"role":"assistant","content":[{"type":"text","text":"Received native CUA values and PNG screenshots through PTC. The object API captured a 90-byte PNG after two AX read failures and refused element input without an accessibility snapshot. Both writability probe failures left the tree readable with unknown fields and kept writes strict."}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:12}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1794213721002,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"chunk","time":1794213721002,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"Received native CUA values and PNG screenshots through PTC. The object API captured a 90-byte PNG after two AX read failures and refused element input without an accessibility snapshot. Both writability probe failures left the tree readable with unknown fields and kept writes strict."}}},{"type":"chunk","time":1794213721002,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1794213721002,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":4,"message":{"role":"assistant","content":[{"type":"text","text":"Received native CUA values and PNG screenshots through PTC. The object API captured a 90-byte PNG after two AX read failures and refused element input without an accessibility snapshot. Both writability probe failures left the tree readable with unknown fields and kept writes strict."}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:12}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1794214207718,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"chunk","time":1794214207718,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"Received native CUA values and PNG screenshots through PTC. The object API captured a 90-byte PNG after two AX read failures and refused element input without an accessibility snapshot. Both writability probe failures left the tree readable with unknown fields and kept writes strict."}}},{"type":"chunk","time":1794214207718,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1794214207718,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"}
|
||||
{"type":"step/end","data":{"turn":1,"step":4}}
|
||||
{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}}
|
||||
@@ -59,7 +59,7 @@ Setup and discovery:
|
||||
Available at app or window level depending on the platform.
|
||||
|
||||
Read getAccessibilityTree explicitly before using element ids. Selection and screenshots do not read or replace the current AX snapshot. Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the tree within the provider limits; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. If the tree reports maxElements or maxDepth truncation, select a narrower window or ask the user to raise the named macOS provider setting and read again. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- listActions(elementId: string) -> string[] — list the actions an element advertises.
|
||||
- invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none.
|
||||
- performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument.
|
||||
@@ -87,6 +87,8 @@ Raw tools use { method, params }: include appHandle for app calls and windowId f
|
||||
|
||||
Only if the task cannot be completed without moving the user’s pointer, call await win.mouseMove(target, { useSystemPointer: true }). Prefer an advertised accessibility action, a click on a visible control, or browser tools when possible. The question interrupts the user. When approval is disabled an explicit move can act immediately. Pass true on every physical move, even after approval. Omitting the option or passing false uses the default hover route without prompting. Approval or refusal lasts until this session closes or DSH restarts. A refusal sends no input. If the user dismisses the request, ask again only if they request it. Wait for an outstanding request to finish. Subagents cannot request physical movement.
|
||||
|
||||
AX read defaults: maxElements=5000, maxDepth=50.
|
||||
|
||||
Maximum click holdMs: 5000 ms.
|
||||
|
||||
## Writing code for run_code
|
||||
|
||||
@@ -59,7 +59,7 @@ Setup and discovery:
|
||||
Available at app or window level depending on the platform.
|
||||
|
||||
Read getAccessibilityTree explicitly before using element ids. Selection and screenshots do not read or replace the current AX snapshot. Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the tree within the provider limits; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. If the tree reports maxElements or maxDepth truncation, select a narrower window or ask the user to raise the named macOS provider setting and read again. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean, maxElements?: number, maxDepth?: number }) -> string — set the node count or depth for one read. Omitted limits use the provider settings. includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. macOS accepts maxElements up to 100000 and maxDepth up to 512. If the tree is truncated, select a narrower window or raise the reported limit on the next read. In persistent PTC, capture a large tree with const tree = await win.getAccessibilityTree({ emit: false }). Filter its lines in memory, or save it with await tools.write({ file_path: path, content: tree }) to a new workspace file. Search its contents with tools.grep({ path, pattern }) and use tools.read around a matching line. Print only the matches you need. Use element ids from the file only while that observation is current.
|
||||
- listActions(elementId: string) -> string[] — list the actions an element advertises.
|
||||
- invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none.
|
||||
- performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument.
|
||||
@@ -85,6 +85,8 @@ Available at app or window level depending on the platform.
|
||||
|
||||
Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree maps to getState; getWindow has no raw method; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name.
|
||||
|
||||
AX read defaults: maxElements=2000, maxDepth=25.
|
||||
|
||||
Maximum click holdMs: 5000 ms.
|
||||
|
||||
Persistent PTC initialization:
|
||||
|
||||
Reference in new issue
Block a user