feat(cua): add an experimental opt-in with on-demand native preparation

This commit is contained in:
creatixchu committed 2026-09-25 18:26:24 +08:00
1 parent 3b7e28a873
commit 504b6df830
131 files changed
+1293 -329

No files matched your search

@@ -9,11 +9,11 @@
en: 2ba6c43b20a3d347
zh: 5647179ddaa38ad9
/agent-note-ship-experimental-capabilities-as-optional-bundles/decision:
en: 1c5a43b3ce2d3056
zh: b59f549a12cf67d1
en: 6366670f925c9506
zh: 060b96718162262c
/agent-note-ship-experimental-capabilities-as-optional-bundles/alternatives-considered:
en: a65d84004e48b3ef
zh: 17221f08e7a8a89b
/agent-note-ship-experimental-capabilities-as-optional-bundles/consequences:
en: d52ed0a248131a37
zh: 779b8d415281c5da
en: 4357d3f0e8312778
zh: f509a517221fe2d1
@@ -10,9 +10,9 @@ The Web plugin page offered two optional bundles, Agent Teams and voice input. A
## Decision
`OPTIONAL_BUNDLES` lists Agent Teams, voice input, and Auto review. Every optional bundle declares `icon` and exports `./locale/*.json` with `meta.title` and `meta.description`, so the Official group renders a localized title, description, and image; `verify-default-product-isolation` rejects an optional bundle without them. The Inspector's `cordis.patch.yml` names the package instead of a built-artifact path, so the same file is the bundle patch and the `--patch` overlay for a built Web launch.
`OPTIONAL_BUNDLES` lists Agent Teams, voice input, Auto review, and Computer Use. Computer Use lives in the computer-use group and declares `dsh.bundle.experimental: true`; the UI also retains experimental labels inferred from existing experimental package names. Every optional bundle declares `icon` and exports `./locale/*.json` with `meta.title` and `meta.description`, so the Official group renders a localized title, description, and image; `verify-default-product-isolation` rejects an optional bundle without them. The Inspector's `cordis.patch.yml` names the package instead of a built-artifact path, so the same file is the bundle patch and the `--patch` overlay for a built Web launch.
An optional bundle is a runtime dependency of the installation, so its dependency graph is downloaded by every `dsh` install. The list therefore admits only packages whose graph is already in the installation's closure; Auto review adds nothing. The Inspector stays available through explicit installation and does not appear in the default plugin list. The browser-use and computer-use providers stay explicit compositions: Playwright MCP, Chrome DevTools MCP, and the native Cua Driver runtime binaries add about 85 MB and 21 packages to every install whether or not the bundle is enabled, and a shipped Cua Driver MCP switch would offer a capability whose executable the installation does not carry. Those provider packages keep locale display metadata for their component rows. Two more packages stay out for other reasons: `ptc-runtime-python` replaces the PTC runtime, and `workflow-ptc` rejects a non-TypeScript runtime at load while the Web presets carry `workflow-ptc` rows a bundle patch cannot reach; `browser-use-stagehand-native` requires a native model name and API key at schema validation with no configuration form on the page.
An optional bundle is a runtime dependency of the installation, so its dependency graph is downloaded by every `dsh` install. Auto review adds no dependencies. Computer Use adds repository-owned provider and bundle code; its source build is prepared only for an enabled profile and brings no external Cua Driver runtime. The Inspector stays available through explicit installation and does not appear in the default plugin list. The external browser-use and Cua Driver providers stay explicit compositions: Playwright MCP, Chrome DevTools MCP, and the native Cua Driver runtime binaries add about 85 MB and 21 packages to every install whether or not the bundle is enabled, and a shipped Cua Driver MCP switch would offer a capability whose executable the installation does not carry. Those provider packages keep locale display metadata for their component rows. Two more packages stay out for other reasons: `ptc-runtime-python` replaces the PTC runtime, and `workflow-ptc` rejects a non-TypeScript runtime at load while the Web presets carry `workflow-ptc` rows a bundle patch cannot reach; `browser-use-stagehand-native` requires a native model name and API key at schema validation with no configuration form on the page.
## Alternatives considered
@@ -22,4 +22,4 @@ An optional bundle is a runtime dependency of the installation, so its dependenc
## Consequences
The Official group grows from two to three entries, each tagged experimental by its experimental name. The runtime dependency closure of the installation is unchanged. A browser-use or computer-use provider still needs its Service Definition and the provider in a profile patch or composition.
The Official group offers four disabled-by-default features. Experimental labels can be explicit bundle metadata. Computer Use composes its registration service, native provider and persistent PTC; enabling it changes the current profile’s execution mode and its description states that effect.
@@ -10,9 +10,9 @@ Web 插件页只提供两个可选 bundle:Agent Teams 与语音输入。Auto r
## 决策
`OPTIONAL_BUNDLES` 列出 Agent Teams、语音输入与 Auto review。每个可选 bundle 都声明 `icon` 并导出带 `meta.title` 与 `meta.description` 的 `./locale/*.json`,官方分组因此能渲染本地化标题、描述和图片;`verify-default-product-isolation` 会拒绝缺少这些声明的可选 bundle。Inspector 的 `cordis.patch.yml` 直接写包名而非构建产物路径,所以同一文件既是 bundle patch,也是已构建 Web 启动的 `--patch` overlay。
`OPTIONAL_BUNDLES` 列出 Agent Teams、语音输入、Auto review 和 Computer Use。Computer Use 位于 computer-use 包组,通过 `dsh.bundle.experimental: true` 声明实验性;界面也保留按既有实验包名推导的标签。每个可选 bundle 都声明 `icon` 并导出带 `meta.title` 与 `meta.description` 的 `./locale/*.json`,官方分组因此能渲染本地化标题、描述和图片;`verify-default-product-isolation` 会拒绝缺少这些声明的可选 bundle。Inspector 的 `cordis.patch.yml` 直接写包名而非构建产物路径,所以同一文件既是 bundle patch,也是已构建 Web 启动的 `--patch` overlay。
可选 bundle 是安装的运行时依赖,其依赖图会随每次 `dsh` 安装一起下载。因此名单只接纳依赖图已在安装闭包内的包;Auto review 不增加任何依赖。Inspector 保留显式安装方式,不在默认插件列表中显示。浏览器操作与电脑操作提供方仍保持显式组合:Playwright MCP、Chrome DevTools MCP 与原生 Cua Driver 运行时二进制会让每次安装多出约 85 MB、21 个包,无论 bundle 是否启用;而随附的 Cua Driver MCP 开关会提供一个安装本身并不携带其可执行文件的能力。这些提供方包保留 locale 显示元数据,供组件行使用。另有两个包因其他原因不纳入:`ptc-runtime-python` 会替换 PTC 运行时,而 `workflow-ptc` 在加载时拒绝非 TypeScript 运行时,且 Web preset 内含 bundle patch 无法触及的 `workflow-ptc` 行;`browser-use-stagehand-native` 在 schema 校验时就要求原生模型名称与 API 密钥,而插件页没有对应的配置表单。
可选 bundle 是安装的运行时依赖,其依赖图会随每次 `dsh` 安装一起下载。Auto review 不增加依赖。Computer Use 增加仓库自有提供方和组合包代码;源码构建只为已启用的 profile 准备,不引入外部 Cua Driver 运行时。Inspector 保留显式安装方式,不在默认插件列表中显示。外部浏览器操作与 Cua Driver 提供方仍保持显式组合:Playwright MCP、Chrome DevTools MCP 与原生 Cua Driver 运行时二进制会让每次安装多出约 85 MB、21 个包,无论 bundle 是否启用;而随附的 Cua Driver MCP 开关会提供一个安装本身并不携带其可执行文件的能力。这些提供方包保留 locale 显示元数据,供组件行使用。另有两个包因其他原因不纳入:`ptc-runtime-python` 会替换 PTC 运行时,而 `workflow-ptc` 在加载时拒绝非 TypeScript 运行时,且 Web preset 内含 bundle patch 无法触及的 `workflow-ptc` 行;`browser-use-stagehand-native` 在 schema 校验时就要求原生模型名称与 API 密钥,而插件页没有对应的配置表单。
## 考虑过的替代方案
@@ -22,4 +22,4 @@ Web 插件页只提供两个可选 bundle:Agent Teams 与语音输入。Auto r
## 后果
官方分组从两项增至三项,每项都因实验包名而带实验性标签。安装的运行时依赖闭包保持不变。浏览器操作或电脑操作提供方仍需在 profile patch 或组合中同时挂载其 Service Definition 与提供方。
官方分组提供四个默认关闭的功能。实验性标签可以由组合包元数据显式声明。Computer Use 组合注册服务、原生提供方和 persistent PTC;开启会改变当前 profile 的执行模式,描述中会说明该影响。
+2 -1
View File
@@ -100,7 +100,8 @@
"@deepseek-ai/dsh-atomic-write": "workspace:*",
"@deepseek-ai/dsh-agent-preset": "workspace:*",
"@deepseek-ai/dsh-experimental-voice-input-bundle": "workspace:*",
"@deepseek-ai/dsh-experimental-auto-review": "workspace:*"
"@deepseek-ai/dsh-experimental-auto-review": "workspace:*",
"@deepseek-ai/dsh-cua-bundle": "workspace:*"
},
"devDependencies": {
"@agentclientprotocol/sdk": "1.4.0",
+59
View File
@@ -0,0 +1,59 @@
/** Plugin selection, persistent execution and restoration through the existing Web switch. */
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { chromium, type Browser } from 'playwright'
import { expect, it } from 'vitest'
import { launchWebScaffold, type WebScaffold } from './scaffold.ts'
import { ZH_BROWSER_LOCALE } from './support.ts'
it('keeps Computer Use off by default and persists its explicit profile-wide PTC switch', async () => {
const directory = await mkdtemp(join(tmpdir(), 'cua-optin-web-'))
let ownedScaffold: WebScaffold | undefined
let browser: Browser | undefined
try {
const patch = join(directory, 'native-fixture.yml')
// Native execution is outside this UI scenario; the provider must not spawn it while merely enabled.
await writeFile(patch, JSON.stringify([{ id: 'cua', config: { helperPath: process.execPath } }]))
const scaffold = ownedScaffold = await launchWebScaffold({ extraOverlayPath: [
fileURLToPath(new URL('./pin-browse-picker.overlay.yml', import.meta.url)), patch,
] })
browser = await chromium.launch()
const page = await browser.newPage({ viewport: { width: 1680, height: 1000 }, locale: ZH_BROWSER_LOCALE })
await page.goto(scaffold.authenticatedUrl, { waitUntil: 'load' })
await page.getByRole('navigation', { name: '全局面板' }).getByRole('button', { name: '插件', exact: true }).click()
const card = page.locator('[data-plugin-package="@deepseek-ai/dsh-cua-bundle"]')
const toggle = card.getByRole('switch', { name: '启用 Computer Use', exact: true })
await toggle.waitFor()
expect(await toggle.getAttribute('aria-checked')).toBe('false')
expect(await card.getByText('实验性', { exact: true }).count()).toBe(1)
expect(await card.textContent()).toContain('persistent PTC')
expect(await card.textContent()).not.toContain('Swift')
expect(scaffold.ctx.ptcRuntime.programFormat).toBe('function-body')
expect(scaffold.ctx.get('computerUse')).toBeUndefined()
const selected = async () => JSON.parse(await readFile(join(scaffold.harnessHome, 'profiles/scaffold/package.json'), 'utf8')).dsh.profile.bundles as string[]
expect(await selected()).not.toContain('@deepseek-ai/dsh-cua-bundle')
await toggle.click()
await expect.poll(() => toggle.getAttribute('aria-checked')).toBe('true')
await expect.poll(() => scaffold.ctx.ptcRuntime.programFormat).toBe('repl')
expect(await selected()).toContain('@deepseek-ai/dsh-cua-bundle')
expect(scaffold.ctx.get('computerUse')).toBeDefined()
const runtime = scaffold.ctx.ptcRuntime
const stateOwner = {}
const program = 'globalThis.counter = (globalThis.counter ?? 0) + 1'
for (const value of [1, 2]) {
expect((await runtime.run(runtime.resolve({ program, bindings: [], stateOwner }))).value).toBe(value)
}
await toggle.click()
await expect.poll(() => toggle.getAttribute('aria-checked')).toBe('false')
await expect.poll(() => scaffold.ctx.ptcRuntime.programFormat).toBe('function-body')
expect(await selected()).not.toContain('@deepseek-ai/dsh-cua-bundle')
expect(scaffold.ctx.get('computerUse')).toBeUndefined()
} finally {
try { await browser?.close() } finally {
try { await ownedScaffold?.close() } finally { await rm(directory, { recursive: true, force: true }) }
}
}
})
+1 -1
View File
@@ -130,7 +130,7 @@ describe('web e2e: plugin manager', () => {
expect(await panel.locator('[data-plugin-group="official"] [data-plugin-package]').count()).toBe(OPTIONAL_BUNDLES.length)
expect(await panel.locator('[data-plugin-group="official"] [data-plugin-item]').count()).toBe(4)
expect(await panel.getByText('实验性', { exact: true }).count())
.toBe(OPTIONAL_BUNDLES.filter(name => name.startsWith('@deepseek-ai/dsh-experimental-')).length)
.toBe((await scaffold.ctx.pluginManager.listBundles()).filter(bundle => bundle.optional && (bundle.experimental === true || bundle.name.startsWith('@deepseek-ai/dsh-experimental-'))).length)
expect(await panel.locator('[data-plugin-package="@deepseek-ai/dsh-experimental-inspector"]').count()).toBe(0)
expect(await panel.getByRole('switch', { name: '启用 语音输入', exact: true }).getAttribute('aria-checked')).toBe('false')
// A bundle that is off still shows the rows its patch declares, without switches.
+40 -37
View File
@@ -760,6 +760,45 @@ export interface Config {
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-credentials-local -->
<!-- BEGIN GENERATED config-catalog:@deepseek-ai/dsh-cua -->
<a id="deepseek-aidsh-cua"></a>
## `@deepseek-ai/dsh-cua`
- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents`
- `source`: [`packages/computer-use/cua/src/index.ts:24`](../packages/computer-use/cua/src/index.ts)
```ts config-catalog
/** Native executable and per-operation resource bounds. */
export interface Config {
/** Absolute native helper override; omission prepares the package-owned executable. */
helperPath?: string
/** Maximum time in milliseconds for native component preparation. */
buildTimeoutMs: number
/** Deadline for one native operation, including capture and AX traversal. */
timeoutMs: number
/** Deadline for each individual Accessibility system call. */
axTimeoutMs: number
/** Maximum bytes retained for one helper response frame. */
maxResponseBytes: number
/** Maximum UTF-8 bytes admitted for one helper request frame. */
maxRequestBytes: number
/** Duration in milliseconds of one same-window drag. */
dragDurationMs: number
/** Number of movement segments dispatched during a drag. */
dragSteps: number
/** Delay in milliseconds after each Unicode key-down or key-up event. */
textEventIntervalMs: number
/** Managed process termination grace in milliseconds. */
graceMs: number
/** Maximum nodes admitted by a native accessibility snapshot. */
maxElements: number
/** Maximum depth traversed by a native accessibility snapshot. */
maxDepth: number
}
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-cua -->
<!-- BEGIN GENERATED config-catalog:@deepseek-ai/dsh-deepseek-account-platform -->
<a id="deepseek-aidsh-deepseek-account-platform"></a>
@@ -941,43 +980,6 @@ export interface Config {
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-experimental-computer-use-cua-driver-mcp -->
<!-- BEGIN GENERATED config-catalog:@deepseek-ai/dsh-experimental-cua -->
<a id="deepseek-aidsh-experimental-cua"></a>
## `@deepseek-ai/dsh-experimental-cua`
- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents`
- `source`: [`packages/experimental/cua/src/index.ts:22`](../packages/experimental/cua/src/index.ts)
```ts config-catalog
/** Native executable and per-operation resource bounds. */
export interface Config {
/** Absolute path to the locally built macOS native helper. */
helperPath: string
/** Deadline for one native operation, including capture and AX traversal. */
timeoutMs: number
/** Deadline for each individual Accessibility system call. */
axTimeoutMs: number
/** Maximum bytes retained for one helper response frame. */
maxResponseBytes: number
/** Maximum UTF-8 bytes admitted for one helper request frame. */
maxRequestBytes: number
/** Duration in milliseconds of one same-window drag. */
dragDurationMs: number
/** Number of movement segments dispatched during a drag. */
dragSteps: number
/** Delay in milliseconds after each Unicode key-down or key-up event. */
textEventIntervalMs: number
/** Managed process termination grace in milliseconds. */
graceMs: number
/** Maximum nodes admitted by a native accessibility snapshot. */
maxElements: number
/** Maximum depth traversed by a native accessibility snapshot. */
maxDepth: number
}
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-experimental-cua -->
<!-- BEGIN GENERATED config-catalog:@deepseek-ai/dsh-experimental-inspector -->
<a id="deepseek-aidsh-experimental-inspector"></a>
@@ -4458,6 +4460,7 @@ Imported as libraries by other packages; a `cordis.yml` cannot load them.
| `@deepseek-ai/dsh-client-ui-slots` | — | [`packages/client/ui-slots/src/index.ts`](../packages/client/ui-slots/src/index.ts) |
| `@deepseek-ai/dsh-client-web` | — | [`packages/client/web/src/index.ts`](../packages/client/web/src/index.ts) |
| `@deepseek-ai/dsh-cmdline` | — | [`packages/boot/cmdline/src/index.ts`](../packages/boot/cmdline/src/index.ts) |
| `@deepseek-ai/dsh-cua-bundle` | — | [`packages/computer-use/cua-bundle/src/index.ts`](../packages/computer-use/cua-bundle/src/index.ts) |
| `@deepseek-ai/dsh-deque` | — | [`packages/util/deque/src/index.ts`](../packages/util/deque/src/index.ts) |
| `@deepseek-ai/dsh-experimental-agent-team-profile` | — | [`packages/experimental/agent-team-profile/src/index.ts`](../packages/experimental/agent-team-profile/src/index.ts) |
| `@deepseek-ai/dsh-experimental-browser-use-runtime` | — | [`packages/experimental/browser-use-runtime/src/index.ts`](../packages/experimental/browser-use-runtime/src/index.ts) |
+40 -37
View File
@@ -762,6 +762,45 @@ export interface Config {
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-credentials-local -->
<!-- BEGIN GENERATED config-catalog:@deepseek-ai/dsh-cua -->
<a id="deepseek-aidsh-cua"></a>
## `@deepseek-ai/dsh-cua`
- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents`
- `source`: [`packages/computer-use/cua/src/index.ts:24`](../packages/computer-use/cua/src/index.ts)
```ts config-catalog
/** Native executable and per-operation resource bounds. */
export interface Config {
/** Absolute native helper override; omission prepares the package-owned executable. */
helperPath?: string
/** Maximum time in milliseconds for native component preparation. */
buildTimeoutMs: number
/** Deadline for one native operation, including capture and AX traversal. */
timeoutMs: number
/** Deadline for each individual Accessibility system call. */
axTimeoutMs: number
/** Maximum bytes retained for one helper response frame. */
maxResponseBytes: number
/** Maximum UTF-8 bytes admitted for one helper request frame. */
maxRequestBytes: number
/** Duration in milliseconds of one same-window drag. */
dragDurationMs: number
/** Number of movement segments dispatched during a drag. */
dragSteps: number
/** Delay in milliseconds after each Unicode key-down or key-up event. */
textEventIntervalMs: number
/** Managed process termination grace in milliseconds. */
graceMs: number
/** Maximum nodes admitted by a native accessibility snapshot. */
maxElements: number
/** Maximum depth traversed by a native accessibility snapshot. */
maxDepth: number
}
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-cua -->
<!-- BEGIN GENERATED config-catalog:@deepseek-ai/dsh-deepseek-account-platform -->
<a id="deepseek-aidsh-deepseek-account-platform"></a>
@@ -943,43 +982,6 @@ export interface Config {
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-experimental-computer-use-cua-driver-mcp -->
<!-- BEGIN GENERATED config-catalog:@deepseek-ai/dsh-experimental-cua -->
<a id="deepseek-aidsh-experimental-cua"></a>
## `@deepseek-ai/dsh-experimental-cua`
- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents`
- `source`: [`packages/experimental/cua/src/index.ts:22`](../packages/experimental/cua/src/index.ts)
```ts config-catalog
/** Native executable and per-operation resource bounds. */
export interface Config {
/** Absolute path to the locally built macOS native helper. */
helperPath: string
/** Deadline for one native operation, including capture and AX traversal. */
timeoutMs: number
/** Deadline for each individual Accessibility system call. */
axTimeoutMs: number
/** Maximum bytes retained for one helper response frame. */
maxResponseBytes: number
/** Maximum UTF-8 bytes admitted for one helper request frame. */
maxRequestBytes: number
/** Duration in milliseconds of one same-window drag. */
dragDurationMs: number
/** Number of movement segments dispatched during a drag. */
dragSteps: number
/** Delay in milliseconds after each Unicode key-down or key-up event. */
textEventIntervalMs: number
/** Managed process termination grace in milliseconds. */
graceMs: number
/** Maximum nodes admitted by a native accessibility snapshot. */
maxElements: number
/** Maximum depth traversed by a native accessibility snapshot. */
maxDepth: number
}
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-experimental-cua -->
<!-- BEGIN GENERATED config-catalog:@deepseek-ai/dsh-experimental-inspector -->
<a id="deepseek-aidsh-experimental-inspector"></a>
@@ -4460,6 +4462,7 @@ export interface Config {
| `@deepseek-ai/dsh-client-ui-slots` | — | [`packages/client/ui-slots/src/index.ts`](../packages/client/ui-slots/src/index.ts) |
| `@deepseek-ai/dsh-client-web` | — | [`packages/client/web/src/index.ts`](../packages/client/web/src/index.ts) |
| `@deepseek-ai/dsh-cmdline` | — | [`packages/boot/cmdline/src/index.ts`](../packages/boot/cmdline/src/index.ts) |
| `@deepseek-ai/dsh-cua-bundle` | — | [`packages/computer-use/cua-bundle/src/index.ts`](../packages/computer-use/cua-bundle/src/index.ts) |
| `@deepseek-ai/dsh-deque` | — | [`packages/util/deque/src/index.ts`](../packages/util/deque/src/index.ts) |
| `@deepseek-ai/dsh-experimental-agent-team-profile` | — | [`packages/experimental/agent-team-profile/src/index.ts`](../packages/experimental/agent-team-profile/src/index.ts) |
| `@deepseek-ai/dsh-experimental-browser-use-runtime` | — | [`packages/experimental/browser-use-runtime/src/index.ts`](../packages/experimental/browser-use-runtime/src/index.ts) |
+3 -3
View File
@@ -56,9 +56,9 @@ This matrix shows which packages dispatch each harness-owned event and which pac
| `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | [`acp`](../packages/acp/acp), [`llm`](../packages/llm/llm), `remotes` |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:75`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
| `permission-presets/catalog-changed` | `emit` | [`packages/interaction/permission-presets/src/types.ts:48`](../packages/interaction/permission-presets/src/types.ts) | [`permission-presets`](../packages/interaction/permission-presets) (`events.dispatch`) | `remotes` |
| `plugin-manager/changed` | `emit` | [`packages/boot/plugin-manager/src/types.ts:243`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/install-log` | `emit` | [`packages/boot/plugin-manager/src/types.ts:249`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/install-state` | `emit` | [`packages/boot/plugin-manager/src/types.ts:256`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/changed` | `emit` | [`packages/boot/plugin-manager/src/types.ts:245`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/install-log` | `emit` | [`packages/boot/plugin-manager/src/types.ts:251`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/install-state` | `emit` | [`packages/boot/plugin-manager/src/types.ts:258`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `schedule/changed` | `emit` | [`packages/schedule/schedule/src/types.ts:471`](../packages/schedule/schedule/src/types.ts) | [`schedule`](../packages/schedule/schedule) (`emit`) | `remotes` |
| `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - |
| `session/created` | `emit` | [`packages/core/session/src/index.ts:55`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), `session-controller`, [`session-log-deepseek`](../packages/session/session-log-deepseek), [`session-projection`](../packages/session/session-projection), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title), [`time-context`](../packages/context/time-context), [`tool-todo`](../packages/todo/tool-todo), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
+3 -3
View File
@@ -58,9 +58,9 @@
| `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | [`acp`](../packages/acp/acp), [`llm`](../packages/llm/llm), `remotes` |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:75`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
| `permission-presets/catalog-changed` | `emit` | [`packages/interaction/permission-presets/src/types.ts:48`](../packages/interaction/permission-presets/src/types.ts) | [`permission-presets`](../packages/interaction/permission-presets) (`events.dispatch`) | `remotes` |
| `plugin-manager/changed` | `emit` | [`packages/boot/plugin-manager/src/types.ts:243`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/install-log` | `emit` | [`packages/boot/plugin-manager/src/types.ts:249`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/install-state` | `emit` | [`packages/boot/plugin-manager/src/types.ts:256`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/changed` | `emit` | [`packages/boot/plugin-manager/src/types.ts:245`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/install-log` | `emit` | [`packages/boot/plugin-manager/src/types.ts:251`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `plugin-manager/install-state` | `emit` | [`packages/boot/plugin-manager/src/types.ts:258`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` |
| `schedule/changed` | `emit` | [`packages/schedule/schedule/src/types.ts:471`](../packages/schedule/schedule/src/types.ts) | [`schedule`](../packages/schedule/schedule) (`emit`) | `remotes` |
| `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - |
| `session/created` | `emit` | [`packages/core/session/src/index.ts:55`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), `session-controller`, [`session-log-deepseek`](../packages/session/session-log-deepseek), [`session-projection`](../packages/session/session-projection), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title), [`time-context`](../packages/context/time-context), [`tool-todo`](../packages/todo/tool-todo), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
+1 -1
View File
@@ -1612,7 +1612,7 @@ flowchart TD
| [`experimental-browser-use-playwright-mcp`](../packages/experimental/browser-use-playwright-mcp) | `experimental` | [`agent`](../packages/core/agent), [`browser-use`](../packages/browser-use/browser-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`experimental-browser-use-stagehand-native`](../packages/experimental/browser-use-stagehand-native) | `experimental` | [`agent`](../packages/core/agent), [`browser-use`](../packages/browser-use/browser-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`experimental-computer-use-cua-driver-native`](../packages/experimental/computer-use-cua-driver-native) | `experimental` | [`computer-use`](../packages/computer-use/computer-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`experimental-cua`](../packages/experimental/cua) | `experimental` | [`agent`](../packages/core/agent), [`computer-use`](../packages/computer-use/computer-use), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`experimental-cua`](../packages/computer-use/cua) | `experimental` | [`agent`](../packages/core/agent), [`computer-use`](../packages/computer-use/computer-use), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`cordis-host-runner`](../packages/extensions/cordis-host-runner) | `extensions` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`tools`](../packages/core/tools), [`typert-protocol`](../packages/typert/protocol) |
| [`message-feedback`](../packages/feedback/message-feedback) | `feedback` | [`brand`](../packages/util/brand), [`command-feedback`](../packages/feedback/command-feedback), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence), [`typert-protocol`](../packages/typert/protocol) |
| [`repeat-tool-reminder`](../packages/guard/repeat-tool-reminder) | `guard` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools) |
+1 -1
View File
@@ -1614,7 +1614,7 @@ flowchart TD
| [`experimental-browser-use-playwright-mcp`](../packages/experimental/browser-use-playwright-mcp) | `experimental` | [`agent`](../packages/core/agent), [`browser-use`](../packages/browser-use/browser-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`experimental-browser-use-stagehand-native`](../packages/experimental/browser-use-stagehand-native) | `experimental` | [`agent`](../packages/core/agent), [`browser-use`](../packages/browser-use/browser-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`experimental-computer-use-cua-driver-native`](../packages/experimental/computer-use-cua-driver-native) | `experimental` | [`computer-use`](../packages/computer-use/computer-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`experimental-cua`](../packages/experimental/cua) | `experimental` | [`agent`](../packages/core/agent), [`computer-use`](../packages/computer-use/computer-use), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`experimental-cua`](../packages/computer-use/cua) | `experimental` | [`agent`](../packages/core/agent), [`computer-use`](../packages/computer-use/computer-use), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
| [`cordis-host-runner`](../packages/extensions/cordis-host-runner) | `extensions` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`tools`](../packages/core/tools), [`typert-protocol`](../packages/typert/protocol) |
| [`message-feedback`](../packages/feedback/message-feedback) | `feedback` | [`brand`](../packages/util/brand), [`command-feedback`](../packages/feedback/command-feedback), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence), [`typert-protocol`](../packages/typert/protocol) |
| [`repeat-tool-reminder`](../packages/guard/repeat-tool-reminder) | `guard` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools) |
+2 -2
View File
@@ -3,8 +3,8 @@
# After editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/computer-use.md
/computer-use:
en: f216bcec22b775b6
zh: 3f673bdbef20146e
en: 5cff3fb08cb08666
zh: 4730c09637628efc
/computer-use/choose-a-provider:
en: fa2e0d7b41c0450b
zh: af9a799711f901e6
+2
View File
@@ -4,6 +4,8 @@ English | [中文](computer-use.zh.md)
Computer use lets a model observe and operate the local desktop through a configured provider. The shared DSH capability is called **computer use**; **Cua Driver** names the upstream implementation.
[Computer Use](../../packages/computer-use/cua-bundle/README.md) provides an optional experimental switch for macOS CUA and persistent PTC.
## Choose a provider
Mount [`dsh-computer-use`](../../packages/computer-use/computer-use/README.md) and one provider in the same composition. Both Cua Driver providers are experimental public npm packages and require explicit activation.
+2
View File
@@ -4,6 +4,8 @@
计算机操作让模型通过配置的提供方观察并操作本地桌面。DSH 的共享能力称为 **computer use(计算机操作)**;**Cua Driver** 是上游实现的名称。
[Computer Use](../../packages/computer-use/cua-bundle/README.zh.md) 提供 macOS CUA 和 persistent PTC 的可选实验性开关。
## 选择提供方
在同一组合中挂载 [`dsh-computer-use`](../../packages/computer-use/computer-use/README.zh.md) 和一个提供方。两个 Cua Driver 提供方都是公开发布到 npm 的实验性包,均需显式启用。
+5 -3
View File
@@ -200,7 +200,8 @@
"mock:llm": "node --import tsx packages/test-support/llm-mock-server/src/bin.ts",
"start:web": "node --import tsx/esm apps/cli/src/bin.ts web",
"dev:web": "tsx scripts/dev-web.ts --poll",
"postinstall": "node scripts/install-lefthook.mjs"
"postinstall": "node scripts/install-lefthook.mjs",
"build:cua": "tsx scripts/build-cua.ts"
},
"devDependencies": {
"@deepseek-ai/dsh-agent": "workspace:*",
@@ -212,7 +213,7 @@
"@deepseek-ai/dsh-computer-use": "workspace:*",
"@deepseek-ai/dsh-experimental-computer-use-cua-driver-mcp": "workspace:*",
"@deepseek-ai/dsh-experimental-computer-use-cua-driver-native": "workspace:*",
"@deepseek-ai/dsh-experimental-cua": "workspace:*",
"@deepseek-ai/dsh-cua": "workspace:*",
"@deepseek-ai/dsh-llm-replay": "workspace:*",
"@deepseek-ai/dsh-mcp-client": "workspace:*",
"@deepseek-ai/dsh-tool-session-query": "workspace:*",
@@ -256,6 +257,7 @@
"vite-tsconfig-paths": "^6.1.1",
"vitest": "^4.1.8",
"@deepseek-ai/dsh-skill-office": "workspace:*",
"@deepseek-ai/dsh-tool-workspace-dependencies": "workspace:*"
"@deepseek-ai/dsh-tool-workspace-dependencies": "workspace:*",
"@deepseek-ai/dsh-cua-bundle": "workspace:*"
}
}
+1
View File
@@ -211,6 +211,7 @@ export const DEFAULT_PROFILE_BUNDLES: readonly string[] = ['@deepseek-ai/dsh-bas
* [admission](../../../../.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.md)).
*/
export const OPTIONAL_BUNDLES: readonly string[] = [
'@deepseek-ai/dsh-cua-bundle',
'@deepseek-ai/dsh-experimental-agent-team-profile',
'@deepseek-ai/dsh-experimental-voice-input-bundle',
'@deepseek-ai/dsh-experimental-auto-review',
+4 -1
View File
@@ -298,11 +298,14 @@ export class PluginManager extends TypertRemoteService {
...(readOnlyReason === undefined ? {} : { readOnlyReason }), error: { code: 'not-bundle' }, rows: [], overrides: [] })
continue
}
const experimental = info.dsh?.bundle?.experimental
if (experimental !== undefined && typeof experimental !== 'boolean') throw new Error('bundle experimental must be a boolean')
const compatibility = evaluatePluginCompatibility(info, exemptions)
if (compatibility !== undefined && !compatibility.exempted) throw new ManagementFailure('incompatible-version', [incompatiblePlugin(compatibility)])
const dir = resolveBundleDir('dsh', name, this.profile.installAnchor, this.profile.dir)
const meta = readPluginMeta(info.name ?? name, pathToFileURL(join(dir, 'package.json')).href)
bundles.push({ name, ...(info.version === undefined ? {} : { version: info.version }),
bundles.push({ name, ...experimental === undefined ? {} : { experimental },
...(info.version === undefined ? {} : { version: info.version }),
...(info.description === undefined || info.description === '' ? {} : { description: info.description }),
...meta === undefined ? {} : { meta },
enabled, installed, optional, removable: removable && readOnlyReason === undefined,
@@ -46,6 +46,8 @@ export interface BundleRowInfo {
/** One installed or installation-provided bundle. */
export interface BundleInfo {
name: string
/** Explicit experimental feature status declared by the bundle. */
experimental?: boolean
version?: string
/** Local display text with available translations or literal fallbacks, or a metadata diagnostic. */
meta?: PluginLocalizedMeta
@@ -1394,3 +1394,16 @@ it.each([false, true])('rechecks installed bundle peers before accepting a disab
}
expect(readProfileManifest('test', dir).dsh?.profile?.bundles).toEqual(['core', 'extra'])
})
it.each([true, false])('projects the explicit experimental flag %s independently of package naming', async (experimental) => {
const { manager, dir } = await fixture()
const path = join(dir, 'node_modules/extra/package.json')
const manifest = JSON.parse(readFileSync(path, 'utf8'))
manifest.dsh.bundle.experimental = experimental
writeFileSync(path, JSON.stringify(manifest))
expect((await manager.listBundles()).find(row => row.name === 'extra')).toMatchObject({ experimental })
manifest.dsh.bundle.experimental = 'yes'
writeFileSync(path, JSON.stringify(manifest))
expect((await manager.listBundles()).find(row => row.name === 'extra')?.error)
.toMatchObject({ code: 'operation-error', diagnostic: expect.stringContaining('must be a boolean') })
})
@@ -21,8 +21,8 @@
en: 130b84453fcb0f8c
zh: f03976813657b35b
/deepseek-ai-dsh-client-ui-plugin-manager/use-this-package/switching-a-bundle:
en: abc90531efa889d5
zh: 0562570f85e69c35
en: 258331f20921521d
zh: df87c43fd27746e7
/deepseek-ai-dsh-client-ui-plugin-manager/use-this-package/switching-one-row-of-a-bundle:
en: 620c6a3ee20848bd
zh: a9429f047a12bb4c
@@ -43,6 +43,8 @@ When the Host attributes a network failure or timeout to a GitHub address and of
### Switching a bundle
A bundle can declare `dsh.bundle.experimental: true` to display the experimental tag independently of its directory or package name; existing experimental package names retain that tag.
A bundle's page shows its full package name under the title, the spec that installs it elsewhere. A bundle's switch changes its layer selection. A profile with HMR recomposes before the operation completes; one without HMR, and a bundle a higher layer overrides, say so in a toast. A bundle the Host cannot read carries a problem tag and its reason on its page and cannot be switched on; one that provides the management components stays locked. The Host answers with error codes, which the page's dictionary words; pnpm's and the Loader's own diagnostics are shown as they are. The page excludes built-in profile bundles from cards and counts even when the profile holds them as dependencies or the Host reports an error. The Host inventory remains complete; the Settings Plugins section's Plugin list tab inspects their plugins.
### Switching one row of a bundle
@@ -43,6 +43,8 @@ Host 将网络失败或超时归因于 GitHub 地址,且提供 npmmirror 时
### 切换一个组合包
组合包可通过 `dsh.bundle.experimental: true` 显示实验性标签,代码目录和包名不必包含 experimental;已有实验性包名也保留标签。
组合包页面在标题下方显示完整包名,也就是在别处安装它所需的 spec。组合包开关改变其层选择。启用了 HMR 的 profile 在操作完成前重组;没有 HMR 的 profile,以及被更高层覆盖的组合包,会以 toast 说明。Host 读不了的组合包带异常标签,其页面给出原因,且不能打开;提供管理组件的组合包保持锁定。Host 以错误码作答,由页面字典措辞;pnpm 与 Loader 自己的诊断原样显示。页面从卡片与数量中排除内置 profile 组合包,即使 profile 将它们列为依赖或 Host 报告了异常。Host 清单仍保留完整数据;设置中「插件」分区的「插件列表」标签页负责查看它们的插件。
### 切换组合包里的一行
@@ -293,7 +293,7 @@ function EnableSwitch({ pkg, title, t, busy, onSetEnabled }: {
/** The status one card carries: running, off, or a problem the Host reported. */
function packageStatus(pkg: PackageView): 'running' | 'disabled' | 'problem' {
if (pkg.error !== undefined) return 'problem'
if (pkg.error !== undefined || (pkg.enabled && pkg.rows.some(row => row.phase === 'failed'))) return 'problem'
return pkg.enabled ? 'running' : 'disabled'
}
@@ -77,6 +77,8 @@ export interface PackageRow {
/** One bundle as the page shows it: the Host's bundle joined with the entries its rows run as. */
export interface PackageView {
readonly name: string
/** Experimental feature marker supplied by the bundle manifest. */
readonly experimental?: boolean
readonly version?: string
readonly description?: string
/** Local package display text and metadata diagnostics supplied by the Host. */
@@ -410,6 +412,7 @@ export function packageView(bundle: BundleInfo, plugins: readonly PluginInfo[]):
name: bundle.name,
installed: bundle.installed,
optional: bundle.optional,
...bundle.experimental === undefined ? {} : { experimental: bundle.experimental },
enabled: bundle.enabled,
rows,
...bundle.version === undefined ? {} : { version: bundle.version },
@@ -108,12 +108,12 @@ export function shortName(name: string): string {
* @returns localized copy with a technical-name fallback and the independent beta status.
*/
export function packageText(
pkg: Pick<PackageView, 'name' | 'meta'>, resolveText: PluginManagerFace['resolveText'],
pkg: Pick<PackageView, 'name' | 'meta' | 'experimental'>, resolveText: PluginManagerFace['resolveText'],
): { title: string; description: string | undefined; beta: boolean } {
return {
title: pkg.meta?.title === undefined ? pkg.name : resolveText(pkg.meta.title),
description: pkg.meta?.description === undefined ? undefined : resolveText(pkg.meta.description) || undefined,
beta: pkg.name.startsWith('@deepseek-ai/dsh-experimental-'),
beta: pkg.experimental === true || pkg.name.startsWith('@deepseek-ai/dsh-experimental-'),
}
}
@@ -314,6 +314,24 @@ describe('PluginManagerPage', () => {
expect(actions.setEnabled).toHaveBeenCalledExactlyOnceWith('@deepseek-ai/dsh-experimental-agent-team-profile', true)
})
it('keeps the persisted selection while marking a failed component as a problem', () => {
renderTab({ packages: [pkg({ name: 'cua', enabled: true, rows: [row({ phase: 'failed' })] })] })
expect(document.querySelector('[data-plugin-package="cua"]')?.getAttribute('data-plugin-status')).toBe('problem')
expect(screen.getByRole('switch', { name: en.enableToggle.replace('{name}', 'cua') }).getAttribute('aria-checked')).toBe('true')
})
it('marks a normally named Computer Use bundle experimental and explains persistent PTC before enabling', () => {
const description = 'Enabling switches this profile to persistent PTC.'
const { actions } = renderTab({ packages: [pkg({
name: '@deepseek-ai/dsh-cua-bundle', experimental: true, enabled: false, optional: true, installed: false,
meta: { title: 'Computer Use', description },
})] })
expect(screen.getByText(en.statusBeta)).toBeTruthy()
expect(screen.getByText(description)).toBeTruthy()
fireEvent.click(screen.getByRole('switch', { name: en.enableToggle.replace('{name}', 'Computer Use') }))
expect(actions.setEnabled).toHaveBeenCalledExactlyOnceWith('@deepseek-ai/dsh-cua-bundle', true)
})
it.each([
'@deepseek-ai/dsh-experimental-agent-team-profile',
'@deepseek-ai/dsh-experimental-auto-review',
@@ -1412,3 +1412,7 @@ it('recognizes the official registry without a trailing slash and with uppercase
face.openInstall()
await vi.waitFor(() => { expect(state().install.registry).toEqual({ kind: 'offered', registry: MIRROR }) })
})
it('retains the bundle experimental marker in its client projection', () => {
expect(packageView({ ...BUNDLE, experimental: true }, PLUGINS)).toMatchObject({ experimental: true })
})
+2 -2
View File
@@ -15,8 +15,8 @@
en: 1eb35b015a6c423f
zh: a8a3dceee2c9e943
/packages-computer-use/packages:
en: 9aa86e6451f60c3d
zh: c5e6816e90f2d74b
en: ef528c66e71d7501
zh: 14175b4ac5a75c25
/packages-computer-use/related-documentation:
en: ad08d3417a0d4630
zh: cfe8e03033c2fede
+2
View File
@@ -27,6 +27,8 @@ Choose one provider and mount the shared registration service.
| Package | Role | ctx key |
|---|---|---|
| [`computer-use`](computer-use/README.md) | Exclusive named provider registration | `ctx.computerUse` |
| [`cua`](cua/README.md) | Native macOS application and window operations | `ctx.computerUse` |
| [`cua-bundle`](cua-bundle/README.md) | Optional Computer Use switch with persistent PTC | — |
<a id="related-documentation"></a>
## Related documentation
+2
View File
@@ -27,6 +27,8 @@ kind: "package-group"
| 包 | 职责 | ctx 键 |
|---|---|---|
| [`computer-use`](computer-use/README.zh.md) | 按名称独占注册提供方 | `ctx.computerUse` |
| [`cua`](cua/README.zh.md) | 原生 macOS 应用与窗口操作 | `ctx.computerUse` |
| [`cua-bundle`](cua-bundle/README.zh.md) | 默认关闭的 Computer Use 开关与 persistent PTC 组合 | — |
<a id="related-documentation"></a>
## 相关文档
@@ -0,0 +1,46 @@
# Bilingual-pair consistency record for README.md (docs/i18n/README.md): per heading
# section, a hash of its English and Chinese blocks outside code blocks and generated regions.
# After editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/computer-use/cua-bundle/README.md
/:
en: 6a4efed7f053f52c
zh: 9135d38515947b44
/deepseek-ai-dsh-cua-bundle:
en: 86711192bd44ca25
zh: 4709726fe53e51a9
/deepseek-ai-dsh-cua-bundle/summary:
en: c88c9461995c0718
zh: 7e7ba3f2e159ca5f
/deepseek-ai-dsh-cua-bundle/table-of-contents:
en: b563e4ec4727d0de
zh: d0228da978238e05
/deepseek-ai-dsh-cua-bundle/use-this-package:
en: 3efb9862b6071b01
zh: dbbbdc33477d27ec
/deepseek-ai-dsh-cua-bundle/understand-the-implementation:
en: 86e0dd2fb9d6c485
zh: 221055b0c0659290
/deepseek-ai-dsh-cua-bundle/further-exploration:
en: 9f15d56b5b16e1ca
zh: 75eb08e8828c8877
/deepseek-ai-dsh-cua-bundle/model-experience:
en: 215e7ba838619b7b
zh: a311da3843709f90
/deepseek-ai-dsh-cua-bundle/model-experience/computer-use-and-persistent-ptc:
en: b932589e03b6ff18
zh: b932589e03b6ff18
/deepseek-ai-dsh-cua-bundle/model-experience/computer-use-and-persistent-ptc/what-the-model-sees:
en: e0ec4ec0a2683bf8
zh: 31f52853554fc9cb
/deepseek-ai-dsh-cua-bundle/model-experience/computer-use-and-persistent-ptc/token-impact:
en: 8b2d8de0b7a602f1
zh: baf88cb87a0125a3
/deepseek-ai-dsh-cua-bundle/model-experience/computer-use-and-persistent-ptc/kv-cache-effect:
en: 531bbb60e3628734
zh: 66781f28c4980f77
/deepseek-ai-dsh-cua-bundle/known-limitations-and-deferred-work:
en: 786708046aea6f31
zh: b1879da465dd27c3
/deepseek-ai-dsh-cua-bundle/known-limitations-and-deferred-work/dev-note:
en: 0a1253c6f7b5e8b9
zh: ce96645b3e75874e
@@ -0,0 +1,86 @@
---
description: "Enable experimental Computer Use and persistent PTC from the plugin page."
kind: "package-bundle"
---
# @deepseek-ai/dsh-cua-bundle
English | [中文](README.zh.md)
## Summary
Enable macOS Computer Use from Plugins and use persistent PTC in the current profile. It is off by default; disabled profiles prepare no native components and register neither CUA tools nor their model guidance.
## Table of Contents
- [Use this package](#use-this-package)
- [Understand the implementation](#understand-the-implementation)
- [Further Exploration](#further-exploration)
- [Model Experience](#model-experience)
- [Known Limitations and Deferred Work](#known-limitations-and-deferred-work)
-----
<a id="use-this-package"></a>
## Use this package
Run the following commands from the repository root, then open Plugins → Official → Computer Use. Its card marks the feature experimental and explains that this profile switches to persistent PTC, retaining variables across calls without a file sandbox.
```sh
pnpm run build
pnpm dsh web
```
The switch reuses `dsh.profile.bundles` in the profile's `package.json`. Enabling prepares native components; source startup also checks an already enabled profile. Subsequent `pnpm run build` reads the same selection in the existing Web profile and checks the incremental build only when selected. First preparation requires local compiler tools and takes machine-dependent time; failures retain their diagnostics and publish no new executable.
Disabling unloads CUA and its persistent runtime, cancels and awaits owned operations, clears retained variables, and restores the preceding tool/runtime configuration. After a project build updates the executable, re-enable or restart to replace running helpers.
-----
<a id="understand-the-implementation"></a>
## Understand the implementation
<details>
<summary>Maintainer details</summary>
`cordis.patch.yml` inserts the registration service, CUA provider and persistent PTC, overriding the default runtime and tool mode. Plugin management persists bundle selection and performs ordinary recomposition; there is no second enabled flag. This configuration-only bundle has no independently mutable projection, so no runtime invariant companion is published.
</details>
-----
<a id="further-exploration"></a>
## Further Exploration
[Computer use](../../../docs/subsystems/computer-use.md)
-----
<a id="model-experience"></a>
## Model Experience
### Computer Use and persistent PTC
#### What the model sees
Enabling contributes the CUA tool, guidance and persistent PTC execution instructions; disabling withdraws them. [CUA](../cua/README.md) and [PTC](../../ptc-runtime/ptc-runtime-node/README.md#persistent-repl) own the full interfaces and output semantics.
#### Token impact
Enabling adds CUA guidance and changes the `run_code` description. Disabling withdraws these contributions.
#### KV Cache effect
Enabling and disabling change tools and system guidance; subsequent requests use the updated prefix.
## Known Limitations and Deferred Work
<a id="known-limitations-and-deferred-work"></a>
- Qualification currently covers repository Web launches on macOS. Desktop installation integration and system-permission onboarding are separate work.
- Enabling changes tool presentation and the PTC runtime for the current profile; it is not a per-task mode selector.
<a id="dev-note"></a>
### Dev Note
None.
@@ -0,0 +1,86 @@
---
description: "从插件页开启实验性 Computer Use 和 persistent PTC。"
kind: "package-bundle"
---
# @deepseek-ai/dsh-cua-bundle
[English](README.md) | 中文
## 概述
从插件页开启 macOS Computer Use,并在当前 profile 使用 persistent PTC。默认关闭,关闭时不准备本机组件、不注册 CUA 工具或添加其模型指引。
## 目录
- [使用本包](#use-this-package)
- [理解实现](#understand-the-implementation)
- [进一步探索](#further-exploration)
- [模型体验](#model-experience)
- [已知限制与延后工作](#known-limitations-and-deferred-work)
-----
<a id="use-this-package"></a>
## 使用本包
在仓库根目录运行下面的命令,再打开侧栏「插件 → 官方 → Computer Use」。卡片标注实验性,并说明当前配置档会切换为 persistent PTC、变量跨调用保留且不提供文件沙箱。
```sh
pnpm run build
pnpm dsh web
```
开关复用 profile 的 `package.json` 中 `dsh.profile.bundles`。开启时先准备本机组件;源码启动恢复已开启状态时也检查。后续 `pnpm run build` 读取现有 Web profile 的同一列表,选中才检查增量构建。首次准备需要本机编译工具,耗时取决于机器;失败保留原始诊断,不发布新的可执行文件。
关闭会卸载 CUA 和持久执行器,取消并等待自有操作结束;保留的代码变量随之清除,原有工具与执行器配置重新生效。项目构建更新可执行文件后,重新启用或重启,运行中的 helper 才会换用新版。
-----
<a id="understand-the-implementation"></a>
## 理解实现
<details>
<summary>维护者说明</summary>
`cordis.patch.yml` 插入注册表、CUA 提供方和 persistent PTC,覆盖默认执行器与工具模式。插件管理器保存组合包选择并触发正常重组;没有第二份启用标记。此配置组合没有独立可变投影,因此不发布 runtime invariant 配套模块。
</details>
-----
<a id="further-exploration"></a>
## 进一步探索
[Computer use](../../../docs/subsystems/computer-use.zh.md)
-----
<a id="model-experience"></a>
## 模型体验
### Computer Use and persistent PTC
#### What the model sees
开启时模型获得 CUA 操作工具、使用指引和 persistent PTC 的执行说明;关闭时撤回这些贡献。完整接口与输出约定由 [CUA](../cua/README.zh.md) 和 [PTC](../../ptc-runtime/ptc-runtime-node/README.zh.md#persistent-repl) 说明。
#### Token impact
启用时添加 CUA 指引,并更新 `run_code` 说明。关闭时撤回这些贡献。
#### KV Cache effect
启停会改变模型工具与系统指引;后续请求使用更新后的前缀。
## 已知限制与延后工作
<a id="known-limitations-and-deferred-work"></a>
- 目前验收范围是仓库中的 macOS Web 启动。桌面安装包集成与系统授权引导另行处理。
- 开启影响当前 profile 的工具呈现和 PTC 执行器;它不是逐任务的模式开关。
<a id="dev-note"></a>
### 开发备注
None.
@@ -0,0 +1,12 @@
- id: ptc-runtime
disabled: true
- id: tools
config:
mode: ptc
- insert:
- id: cua-provider-registry
name: '@deepseek-ai/dsh-computer-use'
- id: cua
name: '@deepseek-ai/dsh-cua'
- id: ptc-runtime-persistent
name: '@deepseek-ai/dsh-ptc-runtime-node/persistent'
+12
View File
@@ -0,0 +1,12 @@
<svg width="36" height="36" viewBox="0 0 36 36" fill="none" xmlns="http://www.w3.org/2000/svg">
<circle cx="15.5" cy="15.5" r="8.5" fill="url(#a)" />
<circle cx="15.5" cy="15.5" r="5" fill="#FFFFFF" />
<path d="M13.2 15.5L14.9 17.2L18.2 13.9" stroke="#145AF3" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" />
<path d="M22 22L29 29" stroke="#145AF3" stroke-width="3.2" stroke-linecap="round" />
<defs>
<linearGradient id="a" x1="7" y1="7" x2="24" y2="24" gradientUnits="userSpaceOnUse">
<stop stop-color="#7CB7FF" />
<stop offset="1" stop-color="#45D9E7" />
</linearGradient>
</defs>
</svg>

After

Width:  |  Height:  |  Size: 642 B

@@ -0,0 +1,6 @@
{
"meta": {
"title": "Computer Use",
"description": "Operate macOS apps. Enabling switches this profile to persistent PTC, retaining code variables across calls without a file sandbox. First activation prepares local components; later project builds check for updates. Disabling clears retained variables."
}
}
@@ -0,0 +1,6 @@
{
"meta": {
"title": "Computer Use",
"description": "操作 macOS 应用。启用后当前配置档切换为 persistent PTC,代码变量跨调用保留,不提供文件沙箱。首次开启会准备本机组件,后续项目构建会检查更新。关闭会清除保留的变量。"
}
}
@@ -0,0 +1,54 @@
{
"name": "@deepseek-ai/dsh-cua-bundle",
"description": "Optional macOS Computer Use with persistent PTC",
"version": "0.1.7-rc.2",
"icon": "./icon.svg",
"publishConfig": {
"access": "public"
},
"repository": {
"type": "git",
"url": "git+https://github.com/deepseek-ai/deepseek-harness.git",
"directory": "packages/computer-use/cua-bundle"
},
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./cordis.patch.yml": "./cordis.patch.yml",
"./locale/*.json": "./locale/*.json",
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"icon.svg",
"locale/*.json",
"lib/index.js",
"cordis.patch.yml",
"lib/types/**/*.d.ts"
],
"license": "MIT",
"dsh": {
"bundle": {
"patch": "./cordis.patch.yml",
"experimental": true
}
},
"dependencies": {
"@deepseek-ai/dsh-cua": "workspace:*",
"@deepseek-ai/dsh-computer-use": "workspace:*",
"@deepseek-ai/dsh-ptc-runtime-node": "workspace:*"
},
"peerDependencies": {
"@deepseek-ai/cordis": "workspace:~"
},
"devDependencies": {
"@deepseek-ai/cordis": "workspace:~",
"@deepseek-ai/cordis-plugin-include": "workspace:~",
"js-yaml": "^4.2.0"
}
}
@@ -0,0 +1,2 @@
/** Optional Computer Use composition; configuration rows live in cordis.patch.yml. */
export {}
@@ -0,0 +1,15 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": [
"src"
],
"references": [
{
"path": "../../../vendor/cordis"
}
]
}
@@ -0,0 +1,52 @@
# Bilingual-pair consistency record for README.md (docs/i18n/README.md): per heading
# section, a hash of its English and Chinese blocks outside code blocks and generated regions.
# After editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/computer-use/cua/README.md
/:
en: a1503331d3f35516
zh: fcb8e5220ff732e1
/deepseek-ai-dsh-cua:
en: 10b9bc038b8de707
zh: 60d3eaf0b4349754
/deepseek-ai-dsh-cua/summary:
en: 4549f02fc53854a5
zh: 6f74f1130b6623a8
/deepseek-ai-dsh-cua/table-of-contents:
en: d152484eb41ac6b4
zh: 7b30b973a849247d
/deepseek-ai-dsh-cua/use-this-package:
en: 4978d663fc761c08
zh: f39f8469cdf12004
/deepseek-ai-dsh-cua/use-this-package/connect-an-external-execution-environment:
en: 59b551986b1da1df
zh: 0dd69fc4633492df
/deepseek-ai-dsh-cua/understand-the-implementation:
en: ee9fd0ddf93d496d
zh: 38d8d5f8b58e6f50
/deepseek-ai-dsh-cua/further-exploration:
en: b1047318879cd6f4
zh: e4ce4a5ad2a9c4dc
/deepseek-ai-dsh-cua/model-experience:
en: 215e7ba838619b7b
zh: a311da3843709f90
/deepseek-ai-dsh-cua/model-experience/native-application-operations:
en: ab7cba4a42334ab9
zh: f77cd9d84624b6c2
/deepseek-ai-dsh-cua/model-experience/native-application-operations/what-the-model-sees:
en: c221fa5ce647b1dd
zh: 9388c91a9f89e029
/deepseek-ai-dsh-cua/model-experience/native-application-operations/what-the-model-sees/operation-guidance-verbatim:
en: 2dbc22ffd1f8a717
zh: d4458872e75305d9
/deepseek-ai-dsh-cua/model-experience/native-application-operations/token-effect:
en: ab52b46d874fa8b9
zh: e87d3620ef77cb06
/deepseek-ai-dsh-cua/model-experience/native-application-operations/kv-cache-effect:
en: a2c9acb5a4320dda
zh: e8d1586af60215c4
/deepseek-ai-dsh-cua/known-limitations-and-deferred-work:
en: 0fd701796488ac03
zh: 9ce1ed68bc3cd624
/deepseek-ai-dsh-cua/known-limitations-and-deferred-work/dev-note:
en: b836f01bd0bceb4a
zh: 8122fa08b2aa95fa
@@ -3,7 +3,7 @@ description: "Inspect and operate native macOS applications through scoped app a
kind: "package-reference"
---
# @deepseek-ai/dsh-experimental-cua
# @deepseek-ai/dsh-cua
English | [中文](README.zh.md)
@@ -25,20 +25,21 @@ Inspect macOS permissions, select running applications and windows, read accessi
<a id="use-this-package"></a>
## Use this package
Build with Swift 6 and Xcode command-line tools on macOS 14 or later, then mount the source overlay from the repository root:
For repository Web use, enable [Computer Use](../cua-bundle/README.md) in Plugins. This prepares the local executable on macOS 14 or later; subsequent builds check it while the existing Web profile keeps the bundle selected. Disabled profiles do not prepare native components. Explicit source composition is also available:
```sh
swift build --package-path packages/experimental/cua/native --configuration release
cua_native_root="$PWD/packages/experimental/cua/native"
swift build --package-path packages/computer-use/cua/native --configuration release
cua_native_root="$PWD/packages/computer-use/cua/native"
export DSH_CUA_HELPER="$cua_native_root/.build/release/dsh-cua-native"
pnpm dsh headless --patch packages/experimental/cua/examples/macos.cordis.yml --dump-config
pnpm dsh headless --patch packages/computer-use/cua/examples/macos.cordis.yml --dump-config
```
The dump command checks profile composition without a model request. For a real task, replace `--dump-config` with a prompt and configure the profile’s model credentials. The overlay mounts the computer-use registry and this provider; remove an existing competing provider before using it. The responsible host/helper needs Accessibility permission for AX/input and Screen Recording permission for capture. `getCapabilities` reports permissions and session lock state without opening a system dialog. Locked or unavailable desktop sessions reject UI observations and input before dispatch; the user must unlock macOS before resuming. Start the target application yourself. Select it by name, bundle ID, absolute application path, or PID. `getApp` reads the selected application without activating it. Model guidance starts with background observation and input, without activation during initialization or before each action. If a hidden window, another Space or a Stage Manager thumbnail prevents observation, explain the foreground-dependent step before `app.activate()` or an observed window’s `raise` action, then read fresh state. Applications may change focus after input; background delivery does not guarantee an undisturbed user session.
| Field | Default | Meaning |
| --- | --- | --- |
| `helperPath` | required | Absolute native executable path. |
| `helperPath` | package-owned executable | Optional absolute native executable override. |
| `buildTimeoutMs` | `300000` | Maximum native preparation time. |
| `timeoutMs` | `30000` | Per-operation deadline; cancellation can leave partial input. |
| `axTimeoutMs` | `3000` | Deadline per Accessibility system call, configured independently of the complete operation. |
| `maxRequestBytes` / `maxResponseBytes` | `1048576` / `33554432` | UTF-8 request and response limits. |
@@ -54,12 +55,14 @@ Agent disposal and plugin unload cancel and await the complete tool call, includ
Raw `getState` exposes the validated state fields and its complete `tree`. The native helper also returns structured element diagnostics for native checks; the tool projection omits that duplicate array from model text and structured tool values.
Native preparation serializes compiler checks and executable publication with the shared file lock, retains the native build cache, and replaces the executable atomically only when its bytes change. Cancellation and preparation failure finish managed process cleanup before releasing the lock.
### Connect an external execution environment
To run through persistent PTC, add `--patch packages/experimental/cua/examples/persistent-ptc.cordis.yml` after the macOS overlay. This provider keeps variables for the live Agent and runs without a file sandbox. The caller imports the bridge and supplies the current authorized `tools.cua` binding. This integration snippet assumes an execution host has supplied `tools`; the package does not install a global object or keep variables alive:
To run through persistent PTC, add `--patch packages/computer-use/cua/examples/persistent-ptc.cordis.yml` after the macOS overlay. This provider keeps variables for the live Agent and runs without a file sandbox. The caller imports the bridge and supplies the current authorized `tools.cua` binding. This integration snippet assumes an execution host has supplied `tools`; the package does not install a global object or keep variables alive:
```javascript
import { connectCua } from '@deepseek-ai/dsh-experimental-cua/bridge'
import { connectCua } from '@deepseek-ai/dsh-cua/bridge'
const cua = connectCua(() => tools.cua, text => console.log(text))
const app = await cua.getApp("com.apple.TextEdit")
const windows = await app.getWindows()
@@ -100,7 +103,7 @@ Node roles and metadata share one AX batch per observation. Group and scroll-are
<a id="further-exploration"></a>
## Further Exploration
- [Computer-use registration](../../computer-use/computer-use/README.md)
- [Computer-use registration](../computer-use/README.md)
- [Native protocol](native/protocol.schema.json)
- [Object interfaces](src/types.ts)
@@ -148,7 +151,7 @@ Raw operation tools use { method, params }, including params: {} for parameterle
Await operations sequentially. After input, read fresh state to verify the effect. A successful action means dispatched, not verified completion. Cancellation may leave partial effects; never retry uncertain input without observing. STALE_APP requires getApp again; STALE_SNAPSHOT requires getState again. WINDOW_CHANGED requires fresh state before any retry; use an advertised AX action or explicitly restore the window before screenshot or coordinate input, then observe again. PERMISSION_DENIED requires the user to grant macOS Accessibility or Screen Recording permission.
Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-experimental-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
```
#### Token effect
@@ -3,7 +3,7 @@ description: "通过按应用和窗口限定的对象,观察和操作原生 ma
kind: "package-reference"
---
# @deepseek-ai/dsh-experimental-cua
# @deepseek-ai/dsh-cua
[English](README.md) | 中文
@@ -25,20 +25,21 @@ kind: "package-reference"
<a id="use-this-package"></a>
## 使用本包
在 macOS 14 或更高版本上使用 Swift 6 和 Xcode 命令行工具构建,然后从仓库根目录挂载源码配置:
从仓库运行 Web 时,在插件页开启 [Computer Use](../cua-bundle/README.zh.md)。首次启用会在 macOS 14 及以上准备本机组件;现有 Web profile 保持选中时,后续构建继续检查更新。未开启的 profile 不准备原生组件。也可以显式加载源码覆盖层:
```sh
swift build --package-path packages/experimental/cua/native --configuration release
cua_native_root="$PWD/packages/experimental/cua/native"
swift build --package-path packages/computer-use/cua/native --configuration release
cua_native_root="$PWD/packages/computer-use/cua/native"
export DSH_CUA_HELPER="$cua_native_root/.build/release/dsh-cua-native"
pnpm dsh headless --patch packages/experimental/cua/examples/macos.cordis.yml --dump-config
pnpm dsh headless --patch packages/computer-use/cua/examples/macos.cordis.yml --dump-config
```
配置输出命令不发起模型请求。执行真实任务时,将 `--dump-config` 替换为任务提示,并配置该 profile 的模型凭据。配置挂载 computer-use 注册表和本 provider;使用前需移除已有的竞争 provider。负责执行的宿主或辅助进程需要辅助功能权限来读取 AX 和输入,需要屏幕录制权限来截图。`getCapabilities` 查询权限和锁屏状态,不弹出系统授权窗口。桌面会话锁定或不可用时,界面观察与输入会在分派前拒绝,需由用户解锁 macOS 后继续。目标应用需提前启动,可按名称、bundle ID、应用绝对路径或 PID 选择。`getApp` 读取选中的应用,不自动激活它。模型指引要求先在后台观察和输入,不在初始化或每次动作前激活应用。窗口隐藏、位于其他桌面空间或被台前调度缩为缩略图而无法观察时,先说明需要前台的步骤,再显式调用 `app.activate()` 或窗口的 `raise` 动作,之后重新读取状态。应用收到输入后仍可能改变焦点,后台投递不保证用户操作始终不受干扰。
| 字段 | 默认值 | 含义 |
| --- | --- | --- |
| `helperPath` | 必填 | 原生可执行文件的绝对路径。 |
| `helperPath` | 包内可执行文件 | 可选的原生可执行文件绝对路径覆盖。 |
| `buildTimeoutMs` | `300000` | 原生组件准备的最长时间。 |
| `timeoutMs` | `30000` | 单次操作截止时间,取消后可能已发生部分输入。 |
| `axTimeoutMs` | `3000` | 每次 Accessibility 系统调用的截止时间,与完整操作截止独立配置。 |
| `maxRequestBytes` / `maxResponseBytes` | `1048576` / `33554432` | UTF-8 请求与响应字节限制。 |
@@ -54,12 +55,14 @@ Agent 销毁和插件卸载会取消并等待完整工具调用,包括图像
原始 `getState` 操作返回经过校验的状态字段和完整 `tree`。原生 helper 还返回供原生检查使用的结构化元素诊断数组;工具投影会从模型文本和结构化工具值中移除这份重复数组。
原生准备复用文件锁,串行检查编译和发布可执行文件,保留构建缓存,只在字节变化时原子替换可执行文件。取消或准备失败后,先完成受管进程清理再释放锁。
### 接入外部执行环境
使用 persistent PTC 时,在 macOS 配置后追加 `--patch packages/experimental/cua/examples/persistent-ptc.cordis.yml`。该 provider 为存活的 Agent 保留变量,不启用文件沙箱。调用方导入适配模块,并提供当前执行中已授权的 `tools.cua` 绑定。下面的接入代码假设执行宿主已提供 `tools`;本包不安装全局对象,也不负责保留变量:
使用 persistent PTC 时,在 macOS 配置后追加 `--patch packages/computer-use/cua/examples/persistent-ptc.cordis.yml`。该 provider 为存活的 Agent 保留变量,不启用文件沙箱。调用方导入适配模块,并提供当前执行中已授权的 `tools.cua` 绑定。下面的接入代码假设执行宿主已提供 `tools`;本包不安装全局对象,也不负责保留变量:
```javascript
import { connectCua } from '@deepseek-ai/dsh-experimental-cua/bridge'
import { connectCua } from '@deepseek-ai/dsh-cua/bridge'
const cua = connectCua(() => tools.cua, text => console.log(text))
const app = await cua.getApp("com.apple.TextEdit")
const windows = await app.getWindows()
@@ -100,7 +103,7 @@ AppKit sheet 可以通过其窗口 ID 单独选择,也会出现在所属文档
<a id="further-exploration"></a>
## 进一步阅读
- [Computer-use 注册](../../computer-use/computer-use/README.zh.md)
- [Computer-use 注册](../computer-use/README.zh.md)
- [原生协议](native/protocol.schema.json)
- [对象接口](src/types.ts)
@@ -148,7 +151,7 @@ Raw operation tools use { method, params }, including params: {} for parameterle
Await operations sequentially. After input, read fresh state to verify the effect. A successful action means dispatched, not verified completion. Cancellation may leave partial effects; never retry uncertain input without observing. STALE_APP requires getApp again; STALE_SNAPSHOT requires getState again. WINDOW_CHANGED requires fresh state before any retry; use an advertised AX action or explicitly restore the window before screenshot or coordinate input, then observe again. PERMISSION_DENIED requires the user to grant macOS Accessibility or Screen Recording permission.
Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-experimental-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
```
#### Token 影响
@@ -1,7 +1,7 @@
- insert:
- id: cua-provider-registry
name: '../../../computer-use/computer-use/src/index.ts'
- id: experimental-cua
name: '../../computer-use/src/index.ts'
- id: cua
name: '../src/index.ts'
config:
helperPath: !!js process.env.DSH_CUA_HELPER
@@ -1,3 +1,4 @@
target/
.build/
.swiftpm/
bin/
@@ -2,7 +2,7 @@
This macOS 14+ JSON-lines helper uses Swift 6, AppKit, Accessibility, CoreGraphics and ScreenCaptureKit. It adapts selected MIT-licensed Cua Driver behavior without loading an external driver. `upstream.json` records the fixed upstream version, adapted Swift files and local modules; `LICENSE.cua-driver` preserves the MIT notice.
Build with `swift build --package-path packages/experimental/cua/native --configuration release`. The binary is `.build/release/dsh-cua-native` beneath this directory. The DSH experimental plugin owns its process lifetime. EOF drains an accepted final request, releases retained AX references and exits. The `.build/` directory is not committed.
Build with `swift build --package-path packages/computer-use/cua/native --configuration release`. The binary is `.build/release/dsh-cua-native` beneath this directory. The DSH CUA plugin owns its process lifetime. EOF drains an accepted final request, releases retained AX references and exits. The `.build/` directory is not committed.
Each stdin line is `{ "id": "opaque request id", "method": "method name", "params": {} }`. Each stdout line contains the same ID and exactly one of `result` or `error: {code, message}`. Malformed envelopes return `INVALID_REQUEST` with a null ID. Requests execute serially. Diagnostics must not be written to stdout. `protocol.schema.json` defines request fields.
@@ -32,25 +32,25 @@ Capture and pointer delivery read CG bounds before and after an exact-window AX
The responsible host/helper needs macOS Accessibility permission for AX state and input, and Screen Recording permission for screenshots. Capability preflight and permission errors do not grant permissions. Capture refuses a window that moves or resizes while the image is being read. Unassociated system panels and accessory-only applications are not selectable.
Use `python3 packages/experimental/cua/native/tests/test_swift.py` for offline unit checks with the selected Apple toolchain’s Swift Testing framework/runtime paths and `python3 packages/experimental/cua/native/tests/protocol_e2e.py` for JSONL admission, ordered replies and EOF shutdown without desktop operations. All Python native checks accept `--helper` to select a separately built comparison binary.
Use `python3 packages/computer-use/cua/native/tests/test_swift.py` for offline unit checks with the selected Apple toolchain’s Swift Testing framework/runtime paths and `python3 packages/computer-use/cua/native/tests/protocol_e2e.py` for JSONL admission, ordered replies and EOF shutdown without desktop operations. All Python native checks accept `--helper` to select a separately built comparison binary.
Run `python3 packages/experimental/cua/native/tests/axless_e2e.py` after building and unlocking the Mac to verify an owned window whose application removes and later restores its AX roots. The check requires existing Accessibility/Screen Recording grants, rejects stale element indices, captures the AXless window, verifies a coordinate click in the fixture state file, and checks restoration. It never changes system permission settings.
Run `python3 packages/computer-use/cua/native/tests/axless_e2e.py` after building and unlocking the Mac to verify an owned window whose application removes and later restores its AX roots. The check requires existing Accessibility/Screen Recording grants, rejects stale element indices, captures the AXless window, verifies a coordinate click in the fixture state file, and checks restoration. It never changes system permission settings.
Stable error codes are `INVALID_REQUEST` (malformed envelope), `INVALID_ARGUMENT` (invalid parameters), `UNSUPPORTED` (unknown operation), `APP_UNAVAILABLE`, `AMBIGUOUS_APP`, `STALE_APP`, `STALE_SNAPSHOT`, `WINDOW_UNAVAILABLE`, `WINDOW_CHANGED`, `PERMISSION_DENIED`, `PANEL_NOT_READY` (system panel recipient unavailable before input), and `NATIVE_ERROR` (remaining operating-system failures). An error never promises that a partially dispatched input was rolled back. The host must not retry input automatically.
Observation reads each non-layout node’s descriptive properties and children through `AXUIElementCopyMultipleAttributeValues`. Per-attribute errors retain the individual-read rules; unsupported batch APIs use individual reads. Values are retained only during that observation, and input reads live attributes. Application state can change between nodes; a batch does not make the whole tree atomic.
Run `python3 packages/experimental/cua/native/tests/observation_benchmark.py --reference-helper /absolute/reference --candidate-helper /absolute/candidate --artifacts /private/evidence` to compare complete observations of the same owned AppKit window with 35 and 400 synthetic rows. Each helper receives one warmup and three alternating measured requests. All result fields must match except helper-generated app and snapshot IDs. The report keeps raw timings, byte counts, hashes and replies; it sets no CI speed threshold and makes no memory claim. Fixture startup can activate the application. Run without other desktop automation or compilation jobs.
Run `python3 packages/computer-use/cua/native/tests/observation_benchmark.py --reference-helper /absolute/reference --candidate-helper /absolute/candidate --artifacts /private/evidence` to compare complete observations of the same owned AppKit window with 35 and 400 synthetic rows. Each helper receives one warmup and three alternating measured requests. All result fields must match except helper-generated app and snapshot IDs. The report keeps raw timings, byte counts, hashes and replies; it sets no CI speed threshold and makes no memory claim. Fixture startup can activate the application. Run without other desktop automation or compilation jobs.
Use `--panels-only` to isolate the system file-panel checks. Fixture windows can join application sets so Stage Manager does not shrink them during background input.
Run `python3 packages/experimental/cua/native/tests/e2e.py` after building to exercise an isolated two-window AppKit fixture. It independently records button counts, Unicode text, actual scroll position, right-click callbacks, and drag release position. Scroll, right-button down/up, and drag down/up counters must each increase exactly once per requested gesture. The check also verifies a real NSOpenPanel sheet, path-entry keyboard routing and the selected file URL. It covers scoped snapshot replacement, screenshot dimensions, stale app cleanup and EOF shutdown. `--artifacts DIRECTORY` writes fixture-only screenshots and a private protocol transcript; the transcript contains the running-app inventory and is not a publication artifact.
Run `python3 packages/computer-use/cua/native/tests/e2e.py` after building to exercise an isolated two-window AppKit fixture. It independently records button counts, Unicode text, actual scroll position, right-click callbacks, and drag release position. Scroll, right-button down/up, and drag down/up counters must each increase exactly once per requested gesture. The check also verifies a real NSOpenPanel sheet, path-entry keyboard routing and the selected file URL. It covers scoped snapshot replacement, screenshot dimensions, stale app cleanup and EOF shutdown. `--artifacts DIRECTORY` writes fixture-only screenshots and a private protocol transcript; the transcript contains the running-app inventory and is not a publication artifact.
For a Blender keyboard check, run `python3 packages/experimental/cua/native/tests/blender_keyboard_e2e.py --blender /absolute/path/to/Blender.app/Contents/MacOS/Blender`. It launches one owned factory-startup GUI with private user directories, observes actual Text Editor text, selection and received modifier flags, and checks Command+A replacement, Shift+Left selection and ordinary input after modifier release. It requires an unlocked macOS session and Accessibility permission. The test activates its own instance; run it without concurrent desktop automation. `--helper` selects a previous compatible binary for a negative control; `--artifacts` selects the parent of a new private evidence directory. No Blender runtime is downloaded.
For a Blender keyboard check, run `python3 packages/computer-use/cua/native/tests/blender_keyboard_e2e.py --blender /absolute/path/to/Blender.app/Contents/MacOS/Blender`. It launches one owned factory-startup GUI with private user directories, observes actual Text Editor text, selection and received modifier flags, and checks Command+A replacement, Shift+Left selection and ordinary input after modifier release. It requires an unlocked macOS session and Accessibility permission. The test activates its own instance; run it without concurrent desktop automation. `--helper` selects a previous compatible binary for a negative control; `--artifacts` selects the parent of a new private evidence directory. No Blender runtime is downloaded.
For text pacing comparisons, use `native/tests/text_input_benchmark.py` with `--reference-helper`, `--candidate-helper`, `--blender` and `--artifacts`. It types identical synthetic 200/2000-scalar strings into fresh Blender Text Editors and requires full text equality, including Chinese, emoji and newlines. Repeated `--reference-arg` / `--candidate-arg` options select helper startup flags. Raw request and verified-effect timings include observer publication latency and exclude startup; these manual measurements have no CI speed threshold.
For an independently isolated Electron/Chromium check, run `python3 packages/experimental/cua/native/tests/electron_e2e.py --electron /absolute/path/to/Electron.app/Contents/MacOS/Electron`. Omitting `--electron` uses an already installed desktop workspace runtime. The check never downloads Electron. Its renderer runs with Node integration disabled, context isolation and Chromium sandboxing enabled, private userData/sessionData/log directories, and a local fixture page. Renderer click/input events write independent observations through the main process; all test actions still use the native helper. AX polling waits for actual controls after Chromium accessibility activation. Native and Electron subprocess groups are owned and stopped before temporary files are removed. Optional artifacts include a fixture-only screenshot and private request transcript.
For an independently isolated Electron/Chromium check, run `python3 packages/computer-use/cua/native/tests/electron_e2e.py --electron /absolute/path/to/Electron.app/Contents/MacOS/Electron`. Omitting `--electron` uses an already installed desktop workspace runtime. The check never downloads Electron. Its renderer runs with Node integration disabled, context isolation and Chromium sandboxing enabled, private userData/sessionData/log directories, and a local fixture page. Renderer click/input events write independent observations through the main process; all test actions still use the native helper. AX polling waits for actual controls after Chromium accessibility activation. Native and Electron subprocess groups are owned and stopped before temporary files are removed. Optional artifacts include a fixture-only screenshot and private request transcript.
System file-panel keyboard routing resolves the remote recipient through the selected app’s focused AX sheet. Only Apple’s Open and Save Panel Service is accepted, and its focused ancestry must still contain the identified remote window. Element pointer fallbacks resolve an app-owned window or sheet through AX parents when AXWindow is unavailable. AX actions retain priority over pointer input; visual click markers need no system-cursor movement.
@@ -1,6 +1,6 @@
{
"name": "@deepseek-ai/dsh-experimental-cua",
"description": "Experimental Codex-style macOS computer use",
"name": "@deepseek-ai/dsh-cua",
"description": "macOS computer use through application and window objects",
"version": "0.1.7-rc.2",
"publishConfig": {
"access": "public"
@@ -8,7 +8,7 @@
"repository": {
"type": "git",
"url": "git+https://github.com/deepseek-ai/deepseek-harness.git",
"directory": "packages/experimental/cua"
"directory": "packages/computer-use/cua"
},
"type": "module",
"main": "lib/index.js",
@@ -27,6 +27,10 @@
"./bridge": {
"types": "./lib/types/bridge.d.ts",
"default": "./lib/bridge.js"
},
"./native-build": {
"types": "./lib/types/native-build.d.ts",
"default": "./lib/native-build.js"
}
},
"files": [
@@ -41,7 +45,9 @@
"native/upstream.json",
"native/README.txt",
"native/protocol.schema.json",
"lib/types/**/*.d.ts"
"lib/types/**/*.d.ts",
"lib/native-build.js",
"native/bin"
],
"license": "MIT",
"peerDependencies": {
@@ -57,7 +63,8 @@
"@deepseek-ai/dsh-util-values": "workspace:*",
"@deepseek-ai/schemastery": "workspace:~",
"zod": "^4.4.3",
"@deepseek-ai/dsh-brand": "workspace:*"
"@deepseek-ai/dsh-brand": "workspace:*",
"@deepseek-ai/dsh-atomic-write": "workspace:*"
},
"devDependencies": {
"@deepseek-ai/cordis": "workspace:~",
File renamed without changes.
@@ -1,5 +1,6 @@
/** Experimental Codex-style macOS computer use over locally built native source. */
import { isAbsolute } from 'node:path'
/** macOS computer use over a managed native helper. */
import { isAbsolute, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import type { Context } from '@deepseek-ai/cordis'
import Schema from '@deepseek-ai/schemastery'
import type { Agent } from '@deepseek-ai/dsh-agent'
@@ -7,6 +8,7 @@ import { ComputerUseProviderName } from '@deepseek-ai/dsh-computer-use/brand'
import { createMcpToolDefinition } from '@deepseek-ai/dsh-mcp-client'
import { z } from 'zod'
import { NativeClient } from './native-client.ts'
import { nativeHostArchitecture, prepareNativeExecutable } from './native-build.ts'
import { Request, ScreenshotResult, StateResult } from './protocol.ts'
import type {} from '@deepseek-ai/dsh-system-prompt'
import type { ToolExecution } from '@deepseek-ai/dsh-tools'
@@ -14,14 +16,16 @@ import type {} from '@deepseek-ai/dsh-subprocess'
import type {} from '@deepseek-ai/dsh-computer-use'
/** Loader plugin identity. */
export const name = 'experimental-cua'
export const name = 'cua'
/** Services required for logged tools and owned native process lifetime. */
export const inject = ['computerUse', 'tools', 'systemPrompt', 'subprocess', 'agents']
/** Native executable and per-operation resource bounds. */
export interface Config {
/** Absolute path to the locally built macOS native helper. */
helperPath: string
/** Absolute native helper override; omission prepares the package-owned executable. */
helperPath?: string
/** Maximum time in milliseconds for native component preparation. */
buildTimeoutMs: number
/** Deadline for one native operation, including capture and AX traversal. */
timeoutMs: number
/** Deadline for each individual Accessibility system call. */
@@ -46,7 +50,8 @@ export interface Config {
/** Validated host configuration; no driver is downloaded at runtime. */
export const Config: Schema<Config> = Schema.object({
helperPath: Schema.string().required(),
helperPath: Schema.string(),
buildTimeoutMs: Schema.number().step(1).min(1).max(2 ** 31 - 1).default(300_000),
timeoutMs: Schema.number().step(1).min(1).max(2 ** 31 - 1).default(30_000),
axTimeoutMs: Schema.number().step(1).min(1).max(2 ** 31 - 1).default(3000),
maxRequestBytes: Schema.number().step(1).min(1024).default(1024 * 1024),
@@ -90,7 +95,7 @@ Raw operation tools use { method, params }, including params: {} for parameterle
Await operations sequentially. After input, read fresh state to verify the effect. A successful action means dispatched, not verified completion. Cancellation may leave partial effects; never retry uncertain input without observing. STALE_APP requires getApp again; STALE_SNAPSHOT requires getState again. WINDOW_CHANGED requires fresh state before any retry; use an advertised AX action or explicitly restore the window before screenshot or coordinate input, then observe again. PERMISSION_DENIED requires the user to grant macOS Accessibility or Screen Recording permission.
Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-experimental-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.`
Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.`
function executionAgent(execution: Readonly<ToolExecution>): Agent {
if (!execution.agent) throw new Error('dsh-cua requires an agent-owned execution')
@@ -100,10 +105,33 @@ function executionAgent(execution: Readonly<ToolExecution>): Agent {
/**
* Mount logged native operations with a separate helper per agent.
* @param ctx - Host context with computer-use, subprocess and tool services.
* @param config - Validated macOS helper path and bounds.
* @param config - Validated native preparation and operation bounds.
* @returns Preparation completion when the package-owned native executable is selected.
*/
export function apply(ctx: Context, config: Config): void {
if (!isAbsolute(config.helperPath)) throw new Error('dsh-cua helperPath must be absolute')
export function apply(ctx: Context, config: Config): void | Promise<void> {
if (config.helperPath !== undefined) {
if (!isAbsolute(config.helperPath)) throw new Error('dsh-cua helperPath must be absolute')
return mount(ctx, { ...config, helperPath: config.helperPath })
}
const architecture = nativeHostArchitecture(process.platform, process.arch)
const controller = new AbortController()
const prepared = prepareNativeExecutable({
packageRoot: resolve(fileURLToPath(new URL('..', import.meta.url))),
architecture, subprocess: ctx.subprocess,
timeoutMs: config.buildTimeoutMs, graceMs: config.graceMs, signal: controller.signal,
})
ctx.effect(() => async () => {
controller.abort()
// The Loader receives preparation failures through apply; disposal only awaits quiescence.
await prepared.catch(() => {})
}, 'dsh-cua.native-preparation')
return prepared.then((helperPath) => {
controller.signal.throwIfAborted()
mount(ctx, { ...config, helperPath })
})
}
function mount(ctx: Context, config: Config & { helperPath: string }): void {
const clients = new Map<Agent, NativeClient>()
const lifetime = new AbortController()
const pending = new Map<Promise<unknown>, Agent>()
@@ -0,0 +1,116 @@
/** Source-checkout preparation and installed native executable resolution. */
import { constants } from 'node:fs'
import { access, mkdir, readFile } from 'node:fs/promises'
import { dirname, join, resolve } from 'node:path'
import { withFileLock, writeFileAtomic } from '@deepseek-ai/dsh-atomic-write'
import type { SubprocessRuntime } from '@deepseek-ai/dsh-subprocess'
/** Architecture names used by the platform-specific native payload. */
export type NativeArchitecture = 'arm64' | 'x64'
/**
* Validate the host before preparing a platform-specific executable.
* @param platform - Node host platform.
* @param architecture - Node host architecture.
* @returns Supported macOS architecture.
* @throws When the current host has no native implementation.
*/
export function nativeHostArchitecture(platform: string, architecture: string): NativeArchitecture {
if (platform !== 'darwin' || (architecture !== 'arm64' && architecture !== 'x64')) throw new Error('Computer Use currently requires macOS on Apple silicon or Intel')
return architecture
}
/** Explicit compiler ownership and resource limits for one preparation. */
export interface NativeBuildSpec {
packageRoot: string
architecture: NativeArchitecture
subprocess: Pick<SubprocessRuntime, 'spawn'>
signal: AbortSignal
timeoutMs: number
graceMs: number
}
/**
* Locate the platform-specific executable owned by the CUA package.
* @param packageRoot - Directory containing the CUA package manifest.
* @param architecture - Architecture of the process that will execute the helper.
* @returns Absolute executable path.
*/
export function nativeExecutablePath(packageRoot: string, architecture: NativeArchitecture): string {
return join(packageRoot, 'native', 'bin', `darwin-${architecture}`, 'dsh-cua-native')
}
/**
* Recognize the repository-owned package location before allowing local compilation.
* @param packageRoot - Real directory containing this package.
* @returns True only for a CUA package in a DSH source checkout.
*/
export async function isSourcePackage(packageRoot: string): Promise<boolean> {
const root = resolve(packageRoot, '../../..')
if (join(root, 'packages/computer-use/cua') !== packageRoot) return false
try {
const manifest: unknown = JSON.parse(await readFile(join(root, 'package.json'), 'utf8'))
return typeof manifest === 'object' && manifest !== null && 'name' in manifest && manifest.name === '@deepseek-ai/dsh-root'
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return false
throw error
}
}
/**
* Let the native build system check sources and its retained compiler cache, then publish atomically.
* @param spec - Checkout, target, managed process provider and cancellation/deadline limits.
* @returns Executable path after compilation and complete process cleanup.
* @throws When compilation, cancellation, resource cleanup or artifact publication fails.
*/
export async function buildNativeExecutable(spec: NativeBuildSpec): Promise<string> {
spec.signal.throwIfAborted()
const executable = nativeExecutablePath(spec.packageRoot, spec.architecture)
await mkdir(dirname(executable), { recursive: true })
const signal = AbortSignal.any([spec.signal, AbortSignal.timeout(spec.timeoutMs)])
return withFileLock(executable, () => compileNative(spec, signal, executable), { waitMs: spec.timeoutMs, signal })
}
async function compileNative(spec: NativeBuildSpec, signal: AbortSignal, executable: string): Promise<string> {
const nativeRoot = join(spec.packageRoot, 'native')
const scratch = join(nativeRoot, '.build', `runtime-${spec.architecture}`)
const handle = spec.subprocess.spawn({
argv: ['/usr/bin/xcrun', 'swift', 'build', '--package-path', nativeRoot,
'--scratch-path', scratch, '--configuration', 'release', '--product', 'dsh-cua-native',
'--arch', spec.architecture === 'x64' ? 'x86_64' : 'arm64'],
cwd: spec.packageRoot, signal, graceMs: spec.graceMs,
stdio: { stdin: 'ignore', stdout: { maxBytes: 65_536 }, stderr: { maxBytes: 65_536 } },
})
const outcome = await handle.done.finally(async () => {
try { handle.terminate() } finally {
if (!await handle.waitForExit()) throw new Error('Computer Use preparation process did not exit cleanly')
}
})
signal.throwIfAborted()
if (outcome.exitCode !== 0 || outcome.signal !== null) {
const diagnostics = `${handle.collected.stdout?.readFrom(0).text ?? ''}${handle.collected.stderr?.readFrom(0).text ?? ''}`
throw new Error(`Computer Use native preparation failed (${String(outcome.exitCode)}, ${String(outcome.signal)}): ${diagnostics}`)
}
const built = await readFile(join(scratch, 'release', 'dsh-cua-native'))
let previous: Buffer | undefined
try { previous = await readFile(executable) } catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error
}
signal.throwIfAborted()
if (!previous?.equals(built)) await writeFileAtomic(executable, built, { mode: 0o755 })
await access(executable, constants.X_OK)
return executable
}
/**
* Prepare source builds on activation; installed packages must already carry their native payload.
* @param spec - Native preparation inputs for this host.
* @returns Ready native executable without requiring a compiler for an installed package.
*/
export async function prepareNativeExecutable(spec: NativeBuildSpec): Promise<string> {
if (await isSourcePackage(spec.packageRoot)) return buildNativeExecutable(spec)
const executable = nativeExecutablePath(spec.packageRoot, spec.architecture)
await access(executable, constants.X_OK)
spec.signal.throwIfAborted()
return executable
}
@@ -0,0 +1,135 @@
/** Preparation uses the existing compiler cache and publishes only successful build output. */
import { mkdtemp, mkdir, readFile, rm, stat, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { expect, it, onTestFinished, vi } from 'vitest'
import type { SubprocessHandle, SubprocessOutcome, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess'
import { buildNativeExecutable, isSourcePackage, nativeExecutablePath, nativeHostArchitecture, prepareNativeExecutable } from '../src/native-build.ts'
async function fixture() {
const root = await mkdtemp(join(tmpdir(), 'cua-native-build-'))
onTestFinished(() => rm(root, { recursive: true, force: true }))
const packageRoot = join(root, 'packages/computer-use/cua')
await mkdir(packageRoot, { recursive: true })
const controller = new AbortController()
const output = Buffer.from([0xcf, 0xfa, 0xed, 0xfe, 0, 255, 128])
let outcome: SubprocessOutcome = { exitCode: 0, signal: null }
let beforeExit = async () => {}
const terminate = vi.fn()
const waitForExit = vi.fn(async () => true)
const spawn = vi.fn((request: SubprocessSpawnSpec): SubprocessHandle => {
const scratch = request.argv[request.argv.indexOf('--scratch-path') + 1]!
return {
stdin: undefined, stdout: undefined, stderr: undefined, control: undefined,
collected: { stderr: { readFrom: () => ({ text: 'compiler diagnostic', nextOffset: 19, lossy: false }) } },
done: (async () => {
await beforeExit()
if (outcome.exitCode === 0) {
await mkdir(join(scratch, 'release'), { recursive: true })
await writeFile(join(scratch, 'release/dsh-cua-native'), output)
}
return outcome
})(), terminate, waitForExit,
}
})
const spec = { packageRoot, architecture: 'arm64' as const, subprocess: { spawn }, signal: controller.signal, timeoutMs: 60_000, graceMs: 100 }
return { root, spec, spawn, output, controller, terminate, waitForExit,
setOutcome: (value: SubprocessOutcome) => { outcome = value },
beforeExit: (callback: () => Promise<void>) => { beforeExit = callback },
}
}
it('recognizes only the repository-owned package location as a source build', async () => {
const f = await fixture()
expect(await isSourcePackage(f.spec.packageRoot)).toBe(false)
for (const value of [null, 2, {}, { name: 'another-project' }, { name: '@deepseek-ai/dsh-root' }]) {
await writeFile(join(f.root, 'package.json'), JSON.stringify(value))
expect(await isSourcePackage(f.spec.packageRoot)).toBe(typeof value === 'object' && value !== null && 'name' in value && value.name === '@deepseek-ai/dsh-root')
}
expect(await isSourcePackage(join(f.root, 'node_modules/@deepseek-ai/dsh-cua'))).toBe(false)
await writeFile(join(f.root, 'package.json'), '{')
await expect(isSourcePackage(f.spec.packageRoot)).rejects.toThrow()
})
it.each([['darwin', 'arm64'], ['darwin', 'x64']])('selects the %s/%s executable', (platform, architecture) => {
expect(nativeHostArchitecture(platform, architecture)).toBe(architecture)
})
it.each([['linux', 'arm64'], ['win32', 'x64'], ['darwin', 'ia32']])('refuses unsupported %s/%s hosts', (platform, architecture) => {
expect(() => nativeHostArchitecture(platform, architecture)).toThrow('requires macOS')
})
it('publishes exact bytes and leaves an unchanged executable inode intact', async () => {
const f = await fixture()
const path = await buildNativeExecutable(f.spec)
const first = await stat(path)
expect(await readFile(path)).toEqual(f.output)
await buildNativeExecutable(f.spec)
expect((await stat(path)).ino).toBe(first.ino)
expect(f.spawn).toHaveBeenCalledTimes(2)
expect(f.spawn.mock.calls[0]![0].argv).toContain('arm64')
expect(f.terminate).toHaveBeenCalledTimes(2)
expect(f.waitForExit).toHaveBeenCalledTimes(2)
})
it('checks source builds on activation and maps Intel architecture to the compiler target', async () => {
const f = await fixture()
await writeFile(join(f.root, 'package.json'), '{"name":"@deepseek-ai/dsh-root"}')
const path = await prepareNativeExecutable({ ...f.spec, architecture: 'x64' })
expect(f.spawn.mock.calls[0]![0].argv).toContain('x86_64')
expect(path).toBe(nativeExecutablePath(f.spec.packageRoot, 'x64'))
})
it('uses an installed executable without starting a compiler', async () => {
const f = await fixture()
const path = nativeExecutablePath(f.spec.packageRoot, 'arm64')
await mkdir(join(path, '..'), { recursive: true })
await writeFile(path, f.output, { mode: 0o755 })
expect(await prepareNativeExecutable(f.spec)).toBe(path)
expect(f.spawn).not.toHaveBeenCalled()
})
it('keeps the previous executable when a build fails and reports compiler output', async () => {
const f = await fixture()
const path = await buildNativeExecutable(f.spec)
f.setOutcome({ exitCode: 1, signal: null })
await expect(buildNativeExecutable(f.spec)).rejects.toThrow('compiler diagnostic')
expect(await readFile(path)).toEqual(f.output)
f.setOutcome({ exitCode: null, signal: 'SIGTERM' })
await expect(buildNativeExecutable(f.spec)).rejects.toThrow('SIGTERM')
})
it('awaits compiler cleanup and refuses to publish after cancellation', async () => {
const f = await fixture()
f.beforeExit(async () => { f.controller.abort() })
await expect(buildNativeExecutable(f.spec)).rejects.toThrow()
expect(f.waitForExit).toHaveBeenCalledTimes(1)
await expect(readFile(nativeExecutablePath(f.spec.packageRoot, 'arm64'))).rejects.toMatchObject({ code: 'ENOENT' })
await expect(buildNativeExecutable(f.spec)).rejects.toThrow()
expect(f.spawn).toHaveBeenCalledTimes(1)
})
it('reports incomplete compiler process cleanup before publishing', async () => {
const f = await fixture()
f.waitForExit.mockResolvedValue(false)
await expect(buildNativeExecutable(f.spec)).rejects.toThrow('did not exit cleanly')
await expect(readFile(nativeExecutablePath(f.spec.packageRoot, 'arm64'))).rejects.toMatchObject({ code: 'ENOENT' })
})
it('serializes publication and cancels a competing preparation without starting another compiler', async () => {
const f = await fixture()
const release: PromiseWithResolvers<void> = Promise.withResolvers()
const entered: PromiseWithResolvers<void> = Promise.withResolvers()
onTestFinished(() => { release.resolve() })
f.beforeExit(async () => { entered.resolve(); await release.promise })
const first = buildNativeExecutable(f.spec)
await entered.promise
const competing = new AbortController()
const second = buildNativeExecutable({ ...f.spec, signal: competing.signal })
const rejected = expect(second).rejects.toThrow()
competing.abort()
try {
await rejected
expect(f.spawn).toHaveBeenCalledTimes(1)
} finally { release.resolve(); await first }
expect(await readFile(nativeExecutablePath(f.spec.packageRoot, 'arm64'))).toEqual(f.output)
})
Loaded 100 of 131 files, more files were not shown because too many files have changed in this diff. Show more