diff --git a/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.i18n.yaml b/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.i18n.yaml index e8573b2e72..3150530935 100644 --- a/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.i18n.yaml @@ -9,11 +9,11 @@ en: 2ba6c43b20a3d347 zh: 5647179ddaa38ad9 /agent-note-ship-experimental-capabilities-as-optional-bundles/decision: - en: 1c5a43b3ce2d3056 - zh: b59f549a12cf67d1 + en: 6366670f925c9506 + zh: 060b96718162262c /agent-note-ship-experimental-capabilities-as-optional-bundles/alternatives-considered: en: a65d84004e48b3ef zh: 17221f08e7a8a89b /agent-note-ship-experimental-capabilities-as-optional-bundles/consequences: - en: d52ed0a248131a37 - zh: 779b8d415281c5da + en: 4357d3f0e8312778 + zh: f509a517221fe2d1 diff --git a/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.md b/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.md index a2a83645c9..1ce39ecbb8 100644 --- a/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.md +++ b/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.md @@ -10,9 +10,9 @@ The Web plugin page offered two optional bundles, Agent Teams and voice input. A ## Decision -`OPTIONAL_BUNDLES` lists Agent Teams, voice input, and Auto review. Every optional bundle declares `icon` and exports `./locale/*.json` with `meta.title` and `meta.description`, so the Official group renders a localized title, description, and image; `verify-default-product-isolation` rejects an optional bundle without them. The Inspector's `cordis.patch.yml` names the package instead of a built-artifact path, so the same file is the bundle patch and the `--patch` overlay for a built Web launch. +`OPTIONAL_BUNDLES` lists Agent Teams, voice input, Auto review, and Computer Use. Computer Use lives in the computer-use group and declares `dsh.bundle.experimental: true`; the UI also retains experimental labels inferred from existing experimental package names. Every optional bundle declares `icon` and exports `./locale/*.json` with `meta.title` and `meta.description`, so the Official group renders a localized title, description, and image; `verify-default-product-isolation` rejects an optional bundle without them. The Inspector's `cordis.patch.yml` names the package instead of a built-artifact path, so the same file is the bundle patch and the `--patch` overlay for a built Web launch. -An optional bundle is a runtime dependency of the installation, so its dependency graph is downloaded by every `dsh` install. The list therefore admits only packages whose graph is already in the installation's closure; Auto review adds nothing. The Inspector stays available through explicit installation and does not appear in the default plugin list. The browser-use and computer-use providers stay explicit compositions: Playwright MCP, Chrome DevTools MCP, and the native Cua Driver runtime binaries add about 85 MB and 21 packages to every install whether or not the bundle is enabled, and a shipped Cua Driver MCP switch would offer a capability whose executable the installation does not carry. Those provider packages keep locale display metadata for their component rows. Two more packages stay out for other reasons: `ptc-runtime-python` replaces the PTC runtime, and `workflow-ptc` rejects a non-TypeScript runtime at load while the Web presets carry `workflow-ptc` rows a bundle patch cannot reach; `browser-use-stagehand-native` requires a native model name and API key at schema validation with no configuration form on the page. +An optional bundle is a runtime dependency of the installation, so its dependency graph is downloaded by every `dsh` install. Auto review adds no dependencies. Computer Use adds repository-owned provider and bundle code; its source build is prepared only for an enabled profile and brings no external Cua Driver runtime. The Inspector stays available through explicit installation and does not appear in the default plugin list. The external browser-use and Cua Driver providers stay explicit compositions: Playwright MCP, Chrome DevTools MCP, and the native Cua Driver runtime binaries add about 85 MB and 21 packages to every install whether or not the bundle is enabled, and a shipped Cua Driver MCP switch would offer a capability whose executable the installation does not carry. Those provider packages keep locale display metadata for their component rows. Two more packages stay out for other reasons: `ptc-runtime-python` replaces the PTC runtime, and `workflow-ptc` rejects a non-TypeScript runtime at load while the Web presets carry `workflow-ptc` rows a bundle patch cannot reach; `browser-use-stagehand-native` requires a native model name and API key at schema validation with no configuration form on the page. ## Alternatives considered @@ -22,4 +22,4 @@ An optional bundle is a runtime dependency of the installation, so its dependenc ## Consequences -The Official group grows from two to three entries, each tagged experimental by its experimental name. The runtime dependency closure of the installation is unchanged. A browser-use or computer-use provider still needs its Service Definition and the provider in a profile patch or composition. +The Official group offers four disabled-by-default features. Experimental labels can be explicit bundle metadata. Computer Use composes its registration service, native provider and persistent PTC; enabling it changes the current profile’s execution mode and its description states that effect. diff --git a/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.zh.md b/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.zh.md index 62e248e5ae..6c608f1327 100644 --- a/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.zh.md +++ b/.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.zh.md @@ -10,9 +10,9 @@ Web 插件页只提供两个可选 bundle：Agent Teams 与语音输入。Auto r ## 决策 -`OPTIONAL_BUNDLES` 列出 Agent Teams、语音输入与 Auto review。每个可选 bundle 都声明 `icon` 并导出带 `meta.title` 与 `meta.description` 的 `./locale/*.json`，官方分组因此能渲染本地化标题、描述和图片；`verify-default-product-isolation` 会拒绝缺少这些声明的可选 bundle。Inspector 的 `cordis.patch.yml` 直接写包名而非构建产物路径，所以同一文件既是 bundle patch，也是已构建 Web 启动的 `--patch` overlay。 +`OPTIONAL_BUNDLES` 列出 Agent Teams、语音输入、Auto review 和 Computer Use。Computer Use 位于 computer-use 包组，通过 `dsh.bundle.experimental: true` 声明实验性；界面也保留按既有实验包名推导的标签。每个可选 bundle 都声明 `icon` 并导出带 `meta.title` 与 `meta.description` 的 `./locale/*.json`，官方分组因此能渲染本地化标题、描述和图片；`verify-default-product-isolation` 会拒绝缺少这些声明的可选 bundle。Inspector 的 `cordis.patch.yml` 直接写包名而非构建产物路径，所以同一文件既是 bundle patch，也是已构建 Web 启动的 `--patch` overlay。 -可选 bundle 是安装的运行时依赖，其依赖图会随每次 `dsh` 安装一起下载。因此名单只接纳依赖图已在安装闭包内的包；Auto review 不增加任何依赖。Inspector 保留显式安装方式，不在默认插件列表中显示。浏览器操作与电脑操作提供方仍保持显式组合：Playwright MCP、Chrome DevTools MCP 与原生 Cua Driver 运行时二进制会让每次安装多出约 85 MB、21 个包，无论 bundle 是否启用；而随附的 Cua Driver MCP 开关会提供一个安装本身并不携带其可执行文件的能力。这些提供方包保留 locale 显示元数据，供组件行使用。另有两个包因其他原因不纳入：`ptc-runtime-python` 会替换 PTC 运行时，而 `workflow-ptc` 在加载时拒绝非 TypeScript 运行时，且 Web preset 内含 bundle patch 无法触及的 `workflow-ptc` 行；`browser-use-stagehand-native` 在 schema 校验时就要求原生模型名称与 API 密钥，而插件页没有对应的配置表单。 +可选 bundle 是安装的运行时依赖，其依赖图会随每次 `dsh` 安装一起下载。Auto review 不增加依赖。Computer Use 增加仓库自有提供方和组合包代码；源码构建只为已启用的 profile 准备，不引入外部 Cua Driver 运行时。Inspector 保留显式安装方式，不在默认插件列表中显示。外部浏览器操作与 Cua Driver 提供方仍保持显式组合：Playwright MCP、Chrome DevTools MCP 与原生 Cua Driver 运行时二进制会让每次安装多出约 85 MB、21 个包，无论 bundle 是否启用；而随附的 Cua Driver MCP 开关会提供一个安装本身并不携带其可执行文件的能力。这些提供方包保留 locale 显示元数据，供组件行使用。另有两个包因其他原因不纳入：`ptc-runtime-python` 会替换 PTC 运行时，而 `workflow-ptc` 在加载时拒绝非 TypeScript 运行时，且 Web preset 内含 bundle patch 无法触及的 `workflow-ptc` 行；`browser-use-stagehand-native` 在 schema 校验时就要求原生模型名称与 API 密钥，而插件页没有对应的配置表单。 ## 考虑过的替代方案 @@ -22,4 +22,4 @@ Web 插件页只提供两个可选 bundle：Agent Teams 与语音输入。Auto r ## 后果 -官方分组从两项增至三项，每项都因实验包名而带实验性标签。安装的运行时依赖闭包保持不变。浏览器操作或电脑操作提供方仍需在 profile patch 或组合中同时挂载其 Service Definition 与提供方。 +官方分组提供四个默认关闭的功能。实验性标签可以由组合包元数据显式声明。Computer Use 组合注册服务、原生提供方和 persistent PTC；开启会改变当前 profile 的执行模式，描述中会说明该影响。 diff --git a/apps/cli/package.json b/apps/cli/package.json index 8b9dbbac17..b5f74d14d2 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -100,7 +100,8 @@ "@deepseek-ai/dsh-atomic-write": "workspace:*", "@deepseek-ai/dsh-agent-preset": "workspace:*", "@deepseek-ai/dsh-experimental-voice-input-bundle": "workspace:*", - "@deepseek-ai/dsh-experimental-auto-review": "workspace:*" + "@deepseek-ai/dsh-experimental-auto-review": "workspace:*", + "@deepseek-ai/dsh-cua-bundle": "workspace:*" }, "devDependencies": { "@agentclientprotocol/sdk": "1.4.0", diff --git a/apps/web/tests/cua-optin.e2e.ts b/apps/web/tests/cua-optin.e2e.ts new file mode 100644 index 0000000000..708b05369d --- /dev/null +++ b/apps/web/tests/cua-optin.e2e.ts @@ -0,0 +1,59 @@ +/** Plugin selection, persistent execution and restoration through the existing Web switch. */ +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { chromium, type Browser } from 'playwright' +import { expect, it } from 'vitest' +import { launchWebScaffold, type WebScaffold } from './scaffold.ts' +import { ZH_BROWSER_LOCALE } from './support.ts' + +it('keeps Computer Use off by default and persists its explicit profile-wide PTC switch', async () => { + const directory = await mkdtemp(join(tmpdir(), 'cua-optin-web-')) + let ownedScaffold: WebScaffold | undefined + let browser: Browser | undefined + try { + const patch = join(directory, 'native-fixture.yml') + // Native execution is outside this UI scenario; the provider must not spawn it while merely enabled. + await writeFile(patch, JSON.stringify([{ id: 'cua', config: { helperPath: process.execPath } }])) + const scaffold = ownedScaffold = await launchWebScaffold({ extraOverlayPath: [ + fileURLToPath(new URL('./pin-browse-picker.overlay.yml', import.meta.url)), patch, + ] }) + browser = await chromium.launch() + const page = await browser.newPage({ viewport: { width: 1680, height: 1000 }, locale: ZH_BROWSER_LOCALE }) + await page.goto(scaffold.authenticatedUrl, { waitUntil: 'load' }) + await page.getByRole('navigation', { name: '全局面板' }).getByRole('button', { name: '插件', exact: true }).click() + const card = page.locator('[data-plugin-package="@deepseek-ai/dsh-cua-bundle"]') + const toggle = card.getByRole('switch', { name: '启用 Computer Use', exact: true }) + await toggle.waitFor() + expect(await toggle.getAttribute('aria-checked')).toBe('false') + expect(await card.getByText('实验性', { exact: true }).count()).toBe(1) + expect(await card.textContent()).toContain('persistent PTC') + expect(await card.textContent()).not.toContain('Swift') + expect(scaffold.ctx.ptcRuntime.programFormat).toBe('function-body') + expect(scaffold.ctx.get('computerUse')).toBeUndefined() + const selected = async () => JSON.parse(await readFile(join(scaffold.harnessHome, 'profiles/scaffold/package.json'), 'utf8')).dsh.profile.bundles as string[] + expect(await selected()).not.toContain('@deepseek-ai/dsh-cua-bundle') + + await toggle.click() + await expect.poll(() => toggle.getAttribute('aria-checked')).toBe('true') + await expect.poll(() => scaffold.ctx.ptcRuntime.programFormat).toBe('repl') + expect(await selected()).toContain('@deepseek-ai/dsh-cua-bundle') + expect(scaffold.ctx.get('computerUse')).toBeDefined() + const runtime = scaffold.ctx.ptcRuntime + const stateOwner = {} + const program = 'globalThis.counter = (globalThis.counter ?? 0) + 1' + for (const value of [1, 2]) { + expect((await runtime.run(runtime.resolve({ program, bindings: [], stateOwner }))).value).toBe(value) + } + await toggle.click() + await expect.poll(() => toggle.getAttribute('aria-checked')).toBe('false') + await expect.poll(() => scaffold.ctx.ptcRuntime.programFormat).toBe('function-body') + expect(await selected()).not.toContain('@deepseek-ai/dsh-cua-bundle') + expect(scaffold.ctx.get('computerUse')).toBeUndefined() + } finally { + try { await browser?.close() } finally { + try { await ownedScaffold?.close() } finally { await rm(directory, { recursive: true, force: true }) } + } + } +}) diff --git a/apps/web/tests/plugin-manager.e2e.ts b/apps/web/tests/plugin-manager.e2e.ts index cfa6a9a763..d6689eeb78 100644 --- a/apps/web/tests/plugin-manager.e2e.ts +++ b/apps/web/tests/plugin-manager.e2e.ts @@ -130,7 +130,7 @@ describe('web e2e: plugin manager', () => { expect(await panel.locator('[data-plugin-group="official"] [data-plugin-package]').count()).toBe(OPTIONAL_BUNDLES.length) expect(await panel.locator('[data-plugin-group="official"] [data-plugin-item]').count()).toBe(4) expect(await panel.getByText('实验性', { exact: true }).count()) - .toBe(OPTIONAL_BUNDLES.filter(name => name.startsWith('@deepseek-ai/dsh-experimental-')).length) + .toBe((await scaffold.ctx.pluginManager.listBundles()).filter(bundle => bundle.optional && (bundle.experimental === true || bundle.name.startsWith('@deepseek-ai/dsh-experimental-'))).length) expect(await panel.locator('[data-plugin-package="@deepseek-ai/dsh-experimental-inspector"]').count()).toBe(0) expect(await panel.getByRole('switch', { name: '启用 语音输入', exact: true }).getAttribute('aria-checked')).toBe('false') // A bundle that is off still shows the rows its patch declares, without switches. diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 15f0be12d5..1545ba8e96 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -760,6 +760,45 @@ export interface Config { ``` + + + +## `@deepseek-ai/dsh-cua` + +- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents` +- `source`: [`packages/computer-use/cua/src/index.ts:24`](../packages/computer-use/cua/src/index.ts) + +```ts config-catalog +/** Native executable and per-operation resource bounds. */ +export interface Config { + /** Absolute native helper override; omission prepares the package-owned executable. */ + helperPath?: string + /** Maximum time in milliseconds for native component preparation. */ + buildTimeoutMs: number + /** Deadline for one native operation, including capture and AX traversal. */ + timeoutMs: number + /** Deadline for each individual Accessibility system call. */ + axTimeoutMs: number + /** Maximum bytes retained for one helper response frame. */ + maxResponseBytes: number + /** Maximum UTF-8 bytes admitted for one helper request frame. */ + maxRequestBytes: number + /** Duration in milliseconds of one same-window drag. */ + dragDurationMs: number + /** Number of movement segments dispatched during a drag. */ + dragSteps: number + /** Delay in milliseconds after each Unicode key-down or key-up event. */ + textEventIntervalMs: number + /** Managed process termination grace in milliseconds. */ + graceMs: number + /** Maximum nodes admitted by a native accessibility snapshot. */ + maxElements: number + /** Maximum depth traversed by a native accessibility snapshot. */ + maxDepth: number +} +``` + + @@ -941,43 +980,6 @@ export interface Config { ``` - - - -## `@deepseek-ai/dsh-experimental-cua` - -- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents` -- `source`: [`packages/experimental/cua/src/index.ts:22`](../packages/experimental/cua/src/index.ts) - -```ts config-catalog -/** Native executable and per-operation resource bounds. */ -export interface Config { - /** Absolute path to the locally built macOS native helper. */ - helperPath: string - /** Deadline for one native operation, including capture and AX traversal. */ - timeoutMs: number - /** Deadline for each individual Accessibility system call. */ - axTimeoutMs: number - /** Maximum bytes retained for one helper response frame. */ - maxResponseBytes: number - /** Maximum UTF-8 bytes admitted for one helper request frame. */ - maxRequestBytes: number - /** Duration in milliseconds of one same-window drag. */ - dragDurationMs: number - /** Number of movement segments dispatched during a drag. */ - dragSteps: number - /** Delay in milliseconds after each Unicode key-down or key-up event. */ - textEventIntervalMs: number - /** Managed process termination grace in milliseconds. */ - graceMs: number - /** Maximum nodes admitted by a native accessibility snapshot. */ - maxElements: number - /** Maximum depth traversed by a native accessibility snapshot. */ - maxDepth: number -} -``` - - @@ -4458,6 +4460,7 @@ Imported as libraries by other packages; a `cordis.yml` cannot load them. | `@deepseek-ai/dsh-client-ui-slots` | — | [`packages/client/ui-slots/src/index.ts`](../packages/client/ui-slots/src/index.ts) | | `@deepseek-ai/dsh-client-web` | — | [`packages/client/web/src/index.ts`](../packages/client/web/src/index.ts) | | `@deepseek-ai/dsh-cmdline` | — | [`packages/boot/cmdline/src/index.ts`](../packages/boot/cmdline/src/index.ts) | +| `@deepseek-ai/dsh-cua-bundle` | — | [`packages/computer-use/cua-bundle/src/index.ts`](../packages/computer-use/cua-bundle/src/index.ts) | | `@deepseek-ai/dsh-deque` | — | [`packages/util/deque/src/index.ts`](../packages/util/deque/src/index.ts) | | `@deepseek-ai/dsh-experimental-agent-team-profile` | — | [`packages/experimental/agent-team-profile/src/index.ts`](../packages/experimental/agent-team-profile/src/index.ts) | | `@deepseek-ai/dsh-experimental-browser-use-runtime` | — | [`packages/experimental/browser-use-runtime/src/index.ts`](../packages/experimental/browser-use-runtime/src/index.ts) | diff --git a/docs/config-catalog.zh.md b/docs/config-catalog.zh.md index f22d4dc09d..f45adcc553 100644 --- a/docs/config-catalog.zh.md +++ b/docs/config-catalog.zh.md @@ -762,6 +762,45 @@ export interface Config { ``` + + + +## `@deepseek-ai/dsh-cua` + +- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents` +- `source`: [`packages/computer-use/cua/src/index.ts:24`](../packages/computer-use/cua/src/index.ts) + +```ts config-catalog +/** Native executable and per-operation resource bounds. */ +export interface Config { + /** Absolute native helper override; omission prepares the package-owned executable. */ + helperPath?: string + /** Maximum time in milliseconds for native component preparation. */ + buildTimeoutMs: number + /** Deadline for one native operation, including capture and AX traversal. */ + timeoutMs: number + /** Deadline for each individual Accessibility system call. */ + axTimeoutMs: number + /** Maximum bytes retained for one helper response frame. */ + maxResponseBytes: number + /** Maximum UTF-8 bytes admitted for one helper request frame. */ + maxRequestBytes: number + /** Duration in milliseconds of one same-window drag. */ + dragDurationMs: number + /** Number of movement segments dispatched during a drag. */ + dragSteps: number + /** Delay in milliseconds after each Unicode key-down or key-up event. */ + textEventIntervalMs: number + /** Managed process termination grace in milliseconds. */ + graceMs: number + /** Maximum nodes admitted by a native accessibility snapshot. */ + maxElements: number + /** Maximum depth traversed by a native accessibility snapshot. */ + maxDepth: number +} +``` + + @@ -943,43 +982,6 @@ export interface Config { ``` - - - -## `@deepseek-ai/dsh-experimental-cua` - -- `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents` -- `source`: [`packages/experimental/cua/src/index.ts:22`](../packages/experimental/cua/src/index.ts) - -```ts config-catalog -/** Native executable and per-operation resource bounds. */ -export interface Config { - /** Absolute path to the locally built macOS native helper. */ - helperPath: string - /** Deadline for one native operation, including capture and AX traversal. */ - timeoutMs: number - /** Deadline for each individual Accessibility system call. */ - axTimeoutMs: number - /** Maximum bytes retained for one helper response frame. */ - maxResponseBytes: number - /** Maximum UTF-8 bytes admitted for one helper request frame. */ - maxRequestBytes: number - /** Duration in milliseconds of one same-window drag. */ - dragDurationMs: number - /** Number of movement segments dispatched during a drag. */ - dragSteps: number - /** Delay in milliseconds after each Unicode key-down or key-up event. */ - textEventIntervalMs: number - /** Managed process termination grace in milliseconds. */ - graceMs: number - /** Maximum nodes admitted by a native accessibility snapshot. */ - maxElements: number - /** Maximum depth traversed by a native accessibility snapshot. */ - maxDepth: number -} -``` - - @@ -4460,6 +4462,7 @@ export interface Config { | `@deepseek-ai/dsh-client-ui-slots` | — | [`packages/client/ui-slots/src/index.ts`](../packages/client/ui-slots/src/index.ts) | | `@deepseek-ai/dsh-client-web` | — | [`packages/client/web/src/index.ts`](../packages/client/web/src/index.ts) | | `@deepseek-ai/dsh-cmdline` | — | [`packages/boot/cmdline/src/index.ts`](../packages/boot/cmdline/src/index.ts) | +| `@deepseek-ai/dsh-cua-bundle` | — | [`packages/computer-use/cua-bundle/src/index.ts`](../packages/computer-use/cua-bundle/src/index.ts) | | `@deepseek-ai/dsh-deque` | — | [`packages/util/deque/src/index.ts`](../packages/util/deque/src/index.ts) | | `@deepseek-ai/dsh-experimental-agent-team-profile` | — | [`packages/experimental/agent-team-profile/src/index.ts`](../packages/experimental/agent-team-profile/src/index.ts) | | `@deepseek-ai/dsh-experimental-browser-use-runtime` | — | [`packages/experimental/browser-use-runtime/src/index.ts`](../packages/experimental/browser-use-runtime/src/index.ts) | diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index 5312354d60..d836b76219 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -56,9 +56,9 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | [`acp`](../packages/acp/acp), [`llm`](../packages/llm/llm), `remotes` | | `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:75`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) | | `permission-presets/catalog-changed` | `emit` | [`packages/interaction/permission-presets/src/types.ts:48`](../packages/interaction/permission-presets/src/types.ts) | [`permission-presets`](../packages/interaction/permission-presets) (`events.dispatch`) | `remotes` | -| `plugin-manager/changed` | `emit` | [`packages/boot/plugin-manager/src/types.ts:243`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | -| `plugin-manager/install-log` | `emit` | [`packages/boot/plugin-manager/src/types.ts:249`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | -| `plugin-manager/install-state` | `emit` | [`packages/boot/plugin-manager/src/types.ts:256`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | +| `plugin-manager/changed` | `emit` | [`packages/boot/plugin-manager/src/types.ts:245`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | +| `plugin-manager/install-log` | `emit` | [`packages/boot/plugin-manager/src/types.ts:251`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | +| `plugin-manager/install-state` | `emit` | [`packages/boot/plugin-manager/src/types.ts:258`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | | `schedule/changed` | `emit` | [`packages/schedule/schedule/src/types.ts:471`](../packages/schedule/schedule/src/types.ts) | [`schedule`](../packages/schedule/schedule) (`emit`) | `remotes` | | `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - | | `session/created` | `emit` | [`packages/core/session/src/index.ts:55`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), `session-controller`, [`session-log-deepseek`](../packages/session/session-log-deepseek), [`session-projection`](../packages/session/session-projection), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title), [`time-context`](../packages/context/time-context), [`tool-todo`](../packages/todo/tool-todo), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) | diff --git a/docs/event-producer-consumer.zh.md b/docs/event-producer-consumer.zh.md index 6baa07d2d2..42ccdd24af 100644 --- a/docs/event-producer-consumer.zh.md +++ b/docs/event-producer-consumer.zh.md @@ -58,9 +58,9 @@ | `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | [`acp`](../packages/acp/acp), [`llm`](../packages/llm/llm), `remotes` | | `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:75`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) | | `permission-presets/catalog-changed` | `emit` | [`packages/interaction/permission-presets/src/types.ts:48`](../packages/interaction/permission-presets/src/types.ts) | [`permission-presets`](../packages/interaction/permission-presets) (`events.dispatch`) | `remotes` | -| `plugin-manager/changed` | `emit` | [`packages/boot/plugin-manager/src/types.ts:243`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | -| `plugin-manager/install-log` | `emit` | [`packages/boot/plugin-manager/src/types.ts:249`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | -| `plugin-manager/install-state` | `emit` | [`packages/boot/plugin-manager/src/types.ts:256`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | +| `plugin-manager/changed` | `emit` | [`packages/boot/plugin-manager/src/types.ts:245`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | +| `plugin-manager/install-log` | `emit` | [`packages/boot/plugin-manager/src/types.ts:251`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | +| `plugin-manager/install-state` | `emit` | [`packages/boot/plugin-manager/src/types.ts:258`](../packages/boot/plugin-manager/src/types.ts) | [`plugin-manager`](../packages/boot/plugin-manager) (`emit`) | `remotes` | | `schedule/changed` | `emit` | [`packages/schedule/schedule/src/types.ts:471`](../packages/schedule/schedule/src/types.ts) | [`schedule`](../packages/schedule/schedule) (`emit`) | `remotes` | | `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - | | `session/created` | `emit` | [`packages/core/session/src/index.ts:55`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), `session-controller`, [`session-log-deepseek`](../packages/session/session-log-deepseek), [`session-projection`](../packages/session/session-projection), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title), [`time-context`](../packages/context/time-context), [`tool-todo`](../packages/todo/tool-todo), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) | diff --git a/docs/module-graph.md b/docs/module-graph.md index 40f183c389..7fc3077986 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -1612,7 +1612,7 @@ flowchart TD | [`experimental-browser-use-playwright-mcp`](../packages/experimental/browser-use-playwright-mcp) | `experimental` | [`agent`](../packages/core/agent), [`browser-use`](../packages/browser-use/browser-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`experimental-browser-use-stagehand-native`](../packages/experimental/browser-use-stagehand-native) | `experimental` | [`agent`](../packages/core/agent), [`browser-use`](../packages/browser-use/browser-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`experimental-computer-use-cua-driver-native`](../packages/experimental/computer-use-cua-driver-native) | `experimental` | [`computer-use`](../packages/computer-use/computer-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | -| [`experimental-cua`](../packages/experimental/cua) | `experimental` | [`agent`](../packages/core/agent), [`computer-use`](../packages/computer-use/computer-use), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | +| [`experimental-cua`](../packages/computer-use/cua) | `experimental` | [`agent`](../packages/core/agent), [`computer-use`](../packages/computer-use/computer-use), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`cordis-host-runner`](../packages/extensions/cordis-host-runner) | `extensions` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`tools`](../packages/core/tools), [`typert-protocol`](../packages/typert/protocol) | | [`message-feedback`](../packages/feedback/message-feedback) | `feedback` | [`brand`](../packages/util/brand), [`command-feedback`](../packages/feedback/command-feedback), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence), [`typert-protocol`](../packages/typert/protocol) | | [`repeat-tool-reminder`](../packages/guard/repeat-tool-reminder) | `guard` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools) | diff --git a/docs/module-graph.zh.md b/docs/module-graph.zh.md index 3cd090576c..1ec8f1efad 100644 --- a/docs/module-graph.zh.md +++ b/docs/module-graph.zh.md @@ -1614,7 +1614,7 @@ flowchart TD | [`experimental-browser-use-playwright-mcp`](../packages/experimental/browser-use-playwright-mcp) | `experimental` | [`agent`](../packages/core/agent), [`browser-use`](../packages/browser-use/browser-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`experimental-browser-use-stagehand-native`](../packages/experimental/browser-use-stagehand-native) | `experimental` | [`agent`](../packages/core/agent), [`browser-use`](../packages/browser-use/browser-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`experimental-computer-use-cua-driver-native`](../packages/experimental/computer-use-cua-driver-native) | `experimental` | [`computer-use`](../packages/computer-use/computer-use), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | -| [`experimental-cua`](../packages/experimental/cua) | `experimental` | [`agent`](../packages/core/agent), [`computer-use`](../packages/computer-use/computer-use), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | +| [`experimental-cua`](../packages/computer-use/cua) | `experimental` | [`agent`](../packages/core/agent), [`computer-use`](../packages/computer-use/computer-use), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`cordis-host-runner`](../packages/extensions/cordis-host-runner) | `extensions` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`tools`](../packages/core/tools), [`typert-protocol`](../packages/typert/protocol) | | [`message-feedback`](../packages/feedback/message-feedback) | `feedback` | [`brand`](../packages/util/brand), [`command-feedback`](../packages/feedback/command-feedback), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence), [`typert-protocol`](../packages/typert/protocol) | | [`repeat-tool-reminder`](../packages/guard/repeat-tool-reminder) | `guard` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools) | diff --git a/docs/subsystems/computer-use.i18n.yaml b/docs/subsystems/computer-use.i18n.yaml index c0dfa781ec..9281851e27 100644 --- a/docs/subsystems/computer-use.i18n.yaml +++ b/docs/subsystems/computer-use.i18n.yaml @@ -3,8 +3,8 @@ # After editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/computer-use.md /computer-use: - en: f216bcec22b775b6 - zh: 3f673bdbef20146e + en: 5cff3fb08cb08666 + zh: 4730c09637628efc /computer-use/choose-a-provider: en: fa2e0d7b41c0450b zh: af9a799711f901e6 diff --git a/docs/subsystems/computer-use.md b/docs/subsystems/computer-use.md index 290df7f02e..61200a648f 100644 --- a/docs/subsystems/computer-use.md +++ b/docs/subsystems/computer-use.md @@ -4,6 +4,8 @@ English | [中文](computer-use.zh.md) Computer use lets a model observe and operate the local desktop through a configured provider. The shared DSH capability is called **computer use**; **Cua Driver** names the upstream implementation. +[Computer Use](../../packages/computer-use/cua-bundle/README.md) provides an optional experimental switch for macOS CUA and persistent PTC. + ## Choose a provider Mount [`dsh-computer-use`](../../packages/computer-use/computer-use/README.md) and one provider in the same composition. Both Cua Driver providers are experimental public npm packages and require explicit activation. diff --git a/docs/subsystems/computer-use.zh.md b/docs/subsystems/computer-use.zh.md index 4c6db29487..f878ad4534 100644 --- a/docs/subsystems/computer-use.zh.md +++ b/docs/subsystems/computer-use.zh.md @@ -4,6 +4,8 @@ 计算机操作让模型通过配置的提供方观察并操作本地桌面。DSH 的共享能力称为 **computer use（计算机操作）**；**Cua Driver** 是上游实现的名称。 +[Computer Use](../../packages/computer-use/cua-bundle/README.zh.md) 提供 macOS CUA 和 persistent PTC 的可选实验性开关。 + ## 选择提供方 在同一组合中挂载 [`dsh-computer-use`](../../packages/computer-use/computer-use/README.zh.md) 和一个提供方。两个 Cua Driver 提供方都是公开发布到 npm 的实验性包，均需显式启用。 diff --git a/package.json b/package.json index 1664f21ea4..1e13f12f66 100644 --- a/package.json +++ b/package.json @@ -200,7 +200,8 @@ "mock:llm": "node --import tsx packages/test-support/llm-mock-server/src/bin.ts", "start:web": "node --import tsx/esm apps/cli/src/bin.ts web", "dev:web": "tsx scripts/dev-web.ts --poll", - "postinstall": "node scripts/install-lefthook.mjs" + "postinstall": "node scripts/install-lefthook.mjs", + "build:cua": "tsx scripts/build-cua.ts" }, "devDependencies": { "@deepseek-ai/dsh-agent": "workspace:*", @@ -212,7 +213,7 @@ "@deepseek-ai/dsh-computer-use": "workspace:*", "@deepseek-ai/dsh-experimental-computer-use-cua-driver-mcp": "workspace:*", "@deepseek-ai/dsh-experimental-computer-use-cua-driver-native": "workspace:*", - "@deepseek-ai/dsh-experimental-cua": "workspace:*", + "@deepseek-ai/dsh-cua": "workspace:*", "@deepseek-ai/dsh-llm-replay": "workspace:*", "@deepseek-ai/dsh-mcp-client": "workspace:*", "@deepseek-ai/dsh-tool-session-query": "workspace:*", @@ -256,6 +257,7 @@ "vite-tsconfig-paths": "^6.1.1", "vitest": "^4.1.8", "@deepseek-ai/dsh-skill-office": "workspace:*", - "@deepseek-ai/dsh-tool-workspace-dependencies": "workspace:*" + "@deepseek-ai/dsh-tool-workspace-dependencies": "workspace:*", + "@deepseek-ai/dsh-cua-bundle": "workspace:*" } } diff --git a/packages/boot/app-boot/src/profile.ts b/packages/boot/app-boot/src/profile.ts index da580c762e..7f03f2c89f 100644 --- a/packages/boot/app-boot/src/profile.ts +++ b/packages/boot/app-boot/src/profile.ts @@ -211,6 +211,7 @@ export const DEFAULT_PROFILE_BUNDLES: readonly string[] = ['@deepseek-ai/dsh-bas * [admission](../../../../.agents/notes/implemented/architecture/2026-09-21-experimental-capabilities-as-optional-bundles.md)). */ export const OPTIONAL_BUNDLES: readonly string[] = [ + '@deepseek-ai/dsh-cua-bundle', '@deepseek-ai/dsh-experimental-agent-team-profile', '@deepseek-ai/dsh-experimental-voice-input-bundle', '@deepseek-ai/dsh-experimental-auto-review', diff --git a/packages/boot/plugin-manager/src/index.ts b/packages/boot/plugin-manager/src/index.ts index ea8a94f548..191a1231e4 100644 --- a/packages/boot/plugin-manager/src/index.ts +++ b/packages/boot/plugin-manager/src/index.ts @@ -298,11 +298,14 @@ export class PluginManager extends TypertRemoteService { ...(readOnlyReason === undefined ? {} : { readOnlyReason }), error: { code: 'not-bundle' }, rows: [], overrides: [] }) continue } + const experimental = info.dsh?.bundle?.experimental + if (experimental !== undefined && typeof experimental !== 'boolean') throw new Error('bundle experimental must be a boolean') const compatibility = evaluatePluginCompatibility(info, exemptions) if (compatibility !== undefined && !compatibility.exempted) throw new ManagementFailure('incompatible-version', [incompatiblePlugin(compatibility)]) const dir = resolveBundleDir('dsh', name, this.profile.installAnchor, this.profile.dir) const meta = readPluginMeta(info.name ?? name, pathToFileURL(join(dir, 'package.json')).href) - bundles.push({ name, ...(info.version === undefined ? {} : { version: info.version }), + bundles.push({ name, ...experimental === undefined ? {} : { experimental }, + ...(info.version === undefined ? {} : { version: info.version }), ...(info.description === undefined || info.description === '' ? {} : { description: info.description }), ...meta === undefined ? {} : { meta }, enabled, installed, optional, removable: removable && readOnlyReason === undefined, diff --git a/packages/boot/plugin-manager/src/types.ts b/packages/boot/plugin-manager/src/types.ts index b87f231e60..bb18752179 100644 --- a/packages/boot/plugin-manager/src/types.ts +++ b/packages/boot/plugin-manager/src/types.ts @@ -46,6 +46,8 @@ export interface BundleRowInfo { /** One installed or installation-provided bundle. */ export interface BundleInfo { name: string + /** Explicit experimental feature status declared by the bundle. */ + experimental?: boolean version?: string /** Local display text with available translations or literal fallbacks, or a metadata diagnostic. */ meta?: PluginLocalizedMeta diff --git a/packages/boot/plugin-manager/tests/manager.spec.ts b/packages/boot/plugin-manager/tests/manager.spec.ts index caaf31f25b..9057e34d6a 100644 --- a/packages/boot/plugin-manager/tests/manager.spec.ts +++ b/packages/boot/plugin-manager/tests/manager.spec.ts @@ -1394,3 +1394,16 @@ it.each([false, true])('rechecks installed bundle peers before accepting a disab } expect(readProfileManifest('test', dir).dsh?.profile?.bundles).toEqual(['core', 'extra']) }) + +it.each([true, false])('projects the explicit experimental flag %s independently of package naming', async (experimental) => { + const { manager, dir } = await fixture() + const path = join(dir, 'node_modules/extra/package.json') + const manifest = JSON.parse(readFileSync(path, 'utf8')) + manifest.dsh.bundle.experimental = experimental + writeFileSync(path, JSON.stringify(manifest)) + expect((await manager.listBundles()).find(row => row.name === 'extra')).toMatchObject({ experimental }) + manifest.dsh.bundle.experimental = 'yes' + writeFileSync(path, JSON.stringify(manifest)) + expect((await manager.listBundles()).find(row => row.name === 'extra')?.error) + .toMatchObject({ code: 'operation-error', diagnostic: expect.stringContaining('must be a boolean') }) +}) diff --git a/packages/client/ui-plugin-manager/README.i18n.yaml b/packages/client/ui-plugin-manager/README.i18n.yaml index 893f4e8405..11438f8ca6 100644 --- a/packages/client/ui-plugin-manager/README.i18n.yaml +++ b/packages/client/ui-plugin-manager/README.i18n.yaml @@ -21,8 +21,8 @@ en: 130b84453fcb0f8c zh: f03976813657b35b /deepseek-ai-dsh-client-ui-plugin-manager/use-this-package/switching-a-bundle: - en: abc90531efa889d5 - zh: 0562570f85e69c35 + en: 258331f20921521d + zh: df87c43fd27746e7 /deepseek-ai-dsh-client-ui-plugin-manager/use-this-package/switching-one-row-of-a-bundle: en: 620c6a3ee20848bd zh: a9429f047a12bb4c diff --git a/packages/client/ui-plugin-manager/README.md b/packages/client/ui-plugin-manager/README.md index 09580f5cbe..5bfd7a282c 100644 --- a/packages/client/ui-plugin-manager/README.md +++ b/packages/client/ui-plugin-manager/README.md @@ -43,6 +43,8 @@ When the Host attributes a network failure or timeout to a GitHub address and of ### Switching a bundle +A bundle can declare `dsh.bundle.experimental: true` to display the experimental tag independently of its directory or package name; existing experimental package names retain that tag. + A bundle's page shows its full package name under the title, the spec that installs it elsewhere. A bundle's switch changes its layer selection. A profile with HMR recomposes before the operation completes; one without HMR, and a bundle a higher layer overrides, say so in a toast. A bundle the Host cannot read carries a problem tag and its reason on its page and cannot be switched on; one that provides the management components stays locked. The Host answers with error codes, which the page's dictionary words; pnpm's and the Loader's own diagnostics are shown as they are. The page excludes built-in profile bundles from cards and counts even when the profile holds them as dependencies or the Host reports an error. The Host inventory remains complete; the Settings Plugins section's Plugin list tab inspects their plugins. ### Switching one row of a bundle diff --git a/packages/client/ui-plugin-manager/README.zh.md b/packages/client/ui-plugin-manager/README.zh.md index 93f764df98..66d32be103 100644 --- a/packages/client/ui-plugin-manager/README.zh.md +++ b/packages/client/ui-plugin-manager/README.zh.md @@ -43,6 +43,8 @@ Host 将网络失败或超时归因于 GitHub 地址，且提供 npmmirror 时 ### 切换一个组合包 +组合包可通过 `dsh.bundle.experimental: true` 显示实验性标签，代码目录和包名不必包含 experimental；已有实验性包名也保留标签。 + 组合包页面在标题下方显示完整包名，也就是在别处安装它所需的 spec。组合包开关改变其层选择。启用了 HMR 的 profile 在操作完成前重组；没有 HMR 的 profile，以及被更高层覆盖的组合包，会以 toast 说明。Host 读不了的组合包带异常标签，其页面给出原因，且不能打开；提供管理组件的组合包保持锁定。Host 以错误码作答，由页面字典措辞；pnpm 与 Loader 自己的诊断原样显示。页面从卡片与数量中排除内置 profile 组合包，即使 profile 将它们列为依赖或 Host 报告了异常。Host 清单仍保留完整数据；设置中「插件」分区的「插件列表」标签页负责查看它们的插件。 ### 切换组合包里的一行 diff --git a/packages/client/ui-plugin-manager/src/client/PluginManagerPage.tsx b/packages/client/ui-plugin-manager/src/client/PluginManagerPage.tsx index 183ed40fae..c3cb90d3e0 100644 --- a/packages/client/ui-plugin-manager/src/client/PluginManagerPage.tsx +++ b/packages/client/ui-plugin-manager/src/client/PluginManagerPage.tsx @@ -293,7 +293,7 @@ function EnableSwitch({ pkg, title, t, busy, onSetEnabled }: { /** The status one card carries: running, off, or a problem the Host reported. */ function packageStatus(pkg: PackageView): 'running' | 'disabled' | 'problem' { - if (pkg.error !== undefined) return 'problem' + if (pkg.error !== undefined || (pkg.enabled && pkg.rows.some(row => row.phase === 'failed'))) return 'problem' return pkg.enabled ? 'running' : 'disabled' } diff --git a/packages/client/ui-plugin-manager/src/client/manager-store.ts b/packages/client/ui-plugin-manager/src/client/manager-store.ts index ab1c3aa038..90c97f094c 100644 --- a/packages/client/ui-plugin-manager/src/client/manager-store.ts +++ b/packages/client/ui-plugin-manager/src/client/manager-store.ts @@ -77,6 +77,8 @@ export interface PackageRow { /** One bundle as the page shows it: the Host's bundle joined with the entries its rows run as. */ export interface PackageView { readonly name: string + /** Experimental feature marker supplied by the bundle manifest. */ + readonly experimental?: boolean readonly version?: string readonly description?: string /** Local package display text and metadata diagnostics supplied by the Host. */ @@ -410,6 +412,7 @@ export function packageView(bundle: BundleInfo, plugins: readonly PluginInfo[]): name: bundle.name, installed: bundle.installed, optional: bundle.optional, + ...bundle.experimental === undefined ? {} : { experimental: bundle.experimental }, enabled: bundle.enabled, rows, ...bundle.version === undefined ? {} : { version: bundle.version }, diff --git a/packages/client/ui-plugin-manager/src/client/presentation.ts b/packages/client/ui-plugin-manager/src/client/presentation.ts index 292d604386..38c75e2431 100644 --- a/packages/client/ui-plugin-manager/src/client/presentation.ts +++ b/packages/client/ui-plugin-manager/src/client/presentation.ts @@ -108,12 +108,12 @@ export function shortName(name: string): string { * @returns localized copy with a technical-name fallback and the independent beta status. */ export function packageText( - pkg: Pick, resolveText: PluginManagerFace['resolveText'], + pkg: Pick, resolveText: PluginManagerFace['resolveText'], ): { title: string; description: string | undefined; beta: boolean } { return { title: pkg.meta?.title === undefined ? pkg.name : resolveText(pkg.meta.title), description: pkg.meta?.description === undefined ? undefined : resolveText(pkg.meta.description) || undefined, - beta: pkg.name.startsWith('@deepseek-ai/dsh-experimental-'), + beta: pkg.experimental === true || pkg.name.startsWith('@deepseek-ai/dsh-experimental-'), } } diff --git a/packages/client/ui-plugin-manager/tests/components.client.spec.tsx b/packages/client/ui-plugin-manager/tests/components.client.spec.tsx index 09303b9e7a..fa0be51e38 100644 --- a/packages/client/ui-plugin-manager/tests/components.client.spec.tsx +++ b/packages/client/ui-plugin-manager/tests/components.client.spec.tsx @@ -314,6 +314,24 @@ describe('PluginManagerPage', () => { expect(actions.setEnabled).toHaveBeenCalledExactlyOnceWith('@deepseek-ai/dsh-experimental-agent-team-profile', true) }) + it('keeps the persisted selection while marking a failed component as a problem', () => { + renderTab({ packages: [pkg({ name: 'cua', enabled: true, rows: [row({ phase: 'failed' })] })] }) + expect(document.querySelector('[data-plugin-package="cua"]')?.getAttribute('data-plugin-status')).toBe('problem') + expect(screen.getByRole('switch', { name: en.enableToggle.replace('{name}', 'cua') }).getAttribute('aria-checked')).toBe('true') + }) + + it('marks a normally named Computer Use bundle experimental and explains persistent PTC before enabling', () => { + const description = 'Enabling switches this profile to persistent PTC.' + const { actions } = renderTab({ packages: [pkg({ + name: '@deepseek-ai/dsh-cua-bundle', experimental: true, enabled: false, optional: true, installed: false, + meta: { title: 'Computer Use', description }, + })] }) + expect(screen.getByText(en.statusBeta)).toBeTruthy() + expect(screen.getByText(description)).toBeTruthy() + fireEvent.click(screen.getByRole('switch', { name: en.enableToggle.replace('{name}', 'Computer Use') })) + expect(actions.setEnabled).toHaveBeenCalledExactlyOnceWith('@deepseek-ai/dsh-cua-bundle', true) + }) + it.each([ '@deepseek-ai/dsh-experimental-agent-team-profile', '@deepseek-ai/dsh-experimental-auto-review', diff --git a/packages/client/ui-plugin-manager/tests/manager-store.client.spec.ts b/packages/client/ui-plugin-manager/tests/manager-store.client.spec.ts index 0cd5683e0e..eee61391bd 100644 --- a/packages/client/ui-plugin-manager/tests/manager-store.client.spec.ts +++ b/packages/client/ui-plugin-manager/tests/manager-store.client.spec.ts @@ -1412,3 +1412,7 @@ it('recognizes the official registry without a trailing slash and with uppercase face.openInstall() await vi.waitFor(() => { expect(state().install.registry).toEqual({ kind: 'offered', registry: MIRROR }) }) }) + +it('retains the bundle experimental marker in its client projection', () => { + expect(packageView({ ...BUNDLE, experimental: true }, PLUGINS)).toMatchObject({ experimental: true }) +}) diff --git a/packages/computer-use/README.i18n.yaml b/packages/computer-use/README.i18n.yaml index c6e98fc6e5..ef8628a94d 100644 --- a/packages/computer-use/README.i18n.yaml +++ b/packages/computer-use/README.i18n.yaml @@ -15,8 +15,8 @@ en: 1eb35b015a6c423f zh: a8a3dceee2c9e943 /packages-computer-use/packages: - en: 9aa86e6451f60c3d - zh: c5e6816e90f2d74b + en: ef528c66e71d7501 + zh: 14175b4ac5a75c25 /packages-computer-use/related-documentation: en: ad08d3417a0d4630 zh: cfe8e03033c2fede diff --git a/packages/computer-use/README.md b/packages/computer-use/README.md index 1b07c668ad..65737cd0b0 100644 --- a/packages/computer-use/README.md +++ b/packages/computer-use/README.md @@ -27,6 +27,8 @@ Choose one provider and mount the shared registration service. | Package | Role | ctx key | |---|---|---| | [`computer-use`](computer-use/README.md) | Exclusive named provider registration | `ctx.computerUse` | +| [`cua`](cua/README.md) | Native macOS application and window operations | `ctx.computerUse` | +| [`cua-bundle`](cua-bundle/README.md) | Optional Computer Use switch with persistent PTC | — | ## Related documentation diff --git a/packages/computer-use/README.zh.md b/packages/computer-use/README.zh.md index d1aa763117..9f0de44dd4 100644 --- a/packages/computer-use/README.zh.md +++ b/packages/computer-use/README.zh.md @@ -27,6 +27,8 @@ kind: "package-group" | 包 | 职责 | ctx 键 | |---|---|---| | [`computer-use`](computer-use/README.zh.md) | 按名称独占注册提供方 | `ctx.computerUse` | +| [`cua`](cua/README.zh.md) | 原生 macOS 应用与窗口操作 | `ctx.computerUse` | +| [`cua-bundle`](cua-bundle/README.zh.md) | 默认关闭的 Computer Use 开关与 persistent PTC 组合 | — | ## 相关文档 diff --git a/packages/computer-use/cua-bundle/README.i18n.yaml b/packages/computer-use/cua-bundle/README.i18n.yaml new file mode 100644 index 0000000000..5f6e45a6c3 --- /dev/null +++ b/packages/computer-use/cua-bundle/README.i18n.yaml @@ -0,0 +1,46 @@ +# Bilingual-pair consistency record for README.md (docs/i18n/README.md): per heading +# section, a hash of its English and Chinese blocks outside code blocks and generated regions. +# After editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/computer-use/cua-bundle/README.md +/: + en: 6a4efed7f053f52c + zh: 9135d38515947b44 +/deepseek-ai-dsh-cua-bundle: + en: 86711192bd44ca25 + zh: 4709726fe53e51a9 +/deepseek-ai-dsh-cua-bundle/summary: + en: c88c9461995c0718 + zh: 7e7ba3f2e159ca5f +/deepseek-ai-dsh-cua-bundle/table-of-contents: + en: b563e4ec4727d0de + zh: d0228da978238e05 +/deepseek-ai-dsh-cua-bundle/use-this-package: + en: 3efb9862b6071b01 + zh: dbbbdc33477d27ec +/deepseek-ai-dsh-cua-bundle/understand-the-implementation: + en: 86e0dd2fb9d6c485 + zh: 221055b0c0659290 +/deepseek-ai-dsh-cua-bundle/further-exploration: + en: 9f15d56b5b16e1ca + zh: 75eb08e8828c8877 +/deepseek-ai-dsh-cua-bundle/model-experience: + en: 215e7ba838619b7b + zh: a311da3843709f90 +/deepseek-ai-dsh-cua-bundle/model-experience/computer-use-and-persistent-ptc: + en: b932589e03b6ff18 + zh: b932589e03b6ff18 +/deepseek-ai-dsh-cua-bundle/model-experience/computer-use-and-persistent-ptc/what-the-model-sees: + en: e0ec4ec0a2683bf8 + zh: 31f52853554fc9cb +/deepseek-ai-dsh-cua-bundle/model-experience/computer-use-and-persistent-ptc/token-impact: + en: 8b2d8de0b7a602f1 + zh: baf88cb87a0125a3 +/deepseek-ai-dsh-cua-bundle/model-experience/computer-use-and-persistent-ptc/kv-cache-effect: + en: 531bbb60e3628734 + zh: 66781f28c4980f77 +/deepseek-ai-dsh-cua-bundle/known-limitations-and-deferred-work: + en: 786708046aea6f31 + zh: b1879da465dd27c3 +/deepseek-ai-dsh-cua-bundle/known-limitations-and-deferred-work/dev-note: + en: 0a1253c6f7b5e8b9 + zh: ce96645b3e75874e diff --git a/packages/computer-use/cua-bundle/README.md b/packages/computer-use/cua-bundle/README.md new file mode 100644 index 0000000000..75ec2d00f5 --- /dev/null +++ b/packages/computer-use/cua-bundle/README.md @@ -0,0 +1,86 @@ +--- +description: "Enable experimental Computer Use and persistent PTC from the plugin page." +kind: "package-bundle" +--- + +# @deepseek-ai/dsh-cua-bundle + +English | [中文](README.zh.md) + +## Summary + +Enable macOS Computer Use from Plugins and use persistent PTC in the current profile. It is off by default; disabled profiles prepare no native components and register neither CUA tools nor their model guidance. + +## Table of Contents + +- [Use this package](#use-this-package) +- [Understand the implementation](#understand-the-implementation) +- [Further Exploration](#further-exploration) +- [Model Experience](#model-experience) +- [Known Limitations and Deferred Work](#known-limitations-and-deferred-work) + +----- + + +## Use this package + +Run the following commands from the repository root, then open Plugins → Official → Computer Use. Its card marks the feature experimental and explains that this profile switches to persistent PTC, retaining variables across calls without a file sandbox. + +```sh +pnpm run build +pnpm dsh web +``` + +The switch reuses `dsh.profile.bundles` in the profile's `package.json`. Enabling prepares native components; source startup also checks an already enabled profile. Subsequent `pnpm run build` reads the same selection in the existing Web profile and checks the incremental build only when selected. First preparation requires local compiler tools and takes machine-dependent time; failures retain their diagnostics and publish no new executable. + +Disabling unloads CUA and its persistent runtime, cancels and awaits owned operations, clears retained variables, and restores the preceding tool/runtime configuration. After a project build updates the executable, re-enable or restart to replace running helpers. + +----- + + +## Understand the implementation + +
+Maintainer details + +`cordis.patch.yml` inserts the registration service, CUA provider and persistent PTC, overriding the default runtime and tool mode. Plugin management persists bundle selection and performs ordinary recomposition; there is no second enabled flag. This configuration-only bundle has no independently mutable projection, so no runtime invariant companion is published. + +
+ +----- + + +## Further Exploration + +[Computer use](../../../docs/subsystems/computer-use.md) + +----- + + +## Model Experience + +### Computer Use and persistent PTC + +#### What the model sees + +Enabling contributes the CUA tool, guidance and persistent PTC execution instructions; disabling withdraws them. [CUA](../cua/README.md) and [PTC](../../ptc-runtime/ptc-runtime-node/README.md#persistent-repl) own the full interfaces and output semantics. + +#### Token impact + +Enabling adds CUA guidance and changes the `run_code` description. Disabling withdraws these contributions. + +#### KV Cache effect + +Enabling and disabling change tools and system guidance; subsequent requests use the updated prefix. + +## Known Limitations and Deferred Work + + + +- Qualification currently covers repository Web launches on macOS. Desktop installation integration and system-permission onboarding are separate work. +- Enabling changes tool presentation and the PTC runtime for the current profile; it is not a per-task mode selector. + + +### Dev Note + +None. diff --git a/packages/computer-use/cua-bundle/README.zh.md b/packages/computer-use/cua-bundle/README.zh.md new file mode 100644 index 0000000000..b3e331cd91 --- /dev/null +++ b/packages/computer-use/cua-bundle/README.zh.md @@ -0,0 +1,86 @@ +--- +description: "从插件页开启实验性 Computer Use 和 persistent PTC。" +kind: "package-bundle" +--- + +# @deepseek-ai/dsh-cua-bundle + +[English](README.md) | 中文 + +## 概述 + +从插件页开启 macOS Computer Use，并在当前 profile 使用 persistent PTC。默认关闭，关闭时不准备本机组件、不注册 CUA 工具或添加其模型指引。 + +## 目录 + +- [使用本包](#use-this-package) +- [理解实现](#understand-the-implementation) +- [进一步探索](#further-exploration) +- [模型体验](#model-experience) +- [已知限制与延后工作](#known-limitations-and-deferred-work) + +----- + + +## 使用本包 + +在仓库根目录运行下面的命令，再打开侧栏「插件 → 官方 → Computer Use」。卡片标注实验性，并说明当前配置档会切换为 persistent PTC、变量跨调用保留且不提供文件沙箱。 + +```sh +pnpm run build +pnpm dsh web +``` + +开关复用 profile 的 `package.json` 中 `dsh.profile.bundles`。开启时先准备本机组件；源码启动恢复已开启状态时也检查。后续 `pnpm run build` 读取现有 Web profile 的同一列表，选中才检查增量构建。首次准备需要本机编译工具，耗时取决于机器；失败保留原始诊断，不发布新的可执行文件。 + +关闭会卸载 CUA 和持久执行器，取消并等待自有操作结束；保留的代码变量随之清除，原有工具与执行器配置重新生效。项目构建更新可执行文件后，重新启用或重启，运行中的 helper 才会换用新版。 + +----- + + +## 理解实现 + +
+维护者说明 + +`cordis.patch.yml` 插入注册表、CUA 提供方和 persistent PTC，覆盖默认执行器与工具模式。插件管理器保存组合包选择并触发正常重组；没有第二份启用标记。此配置组合没有独立可变投影，因此不发布 runtime invariant 配套模块。 + +
+ +----- + + +## 进一步探索 + +[Computer use](../../../docs/subsystems/computer-use.zh.md) + +----- + + +## 模型体验 + +### Computer Use and persistent PTC + +#### What the model sees + +开启时模型获得 CUA 操作工具、使用指引和 persistent PTC 的执行说明；关闭时撤回这些贡献。完整接口与输出约定由 [CUA](../cua/README.zh.md) 和 [PTC](../../ptc-runtime/ptc-runtime-node/README.zh.md#persistent-repl) 说明。 + +#### Token impact + +启用时添加 CUA 指引，并更新 `run_code` 说明。关闭时撤回这些贡献。 + +#### KV Cache effect + +启停会改变模型工具与系统指引；后续请求使用更新后的前缀。 + +## 已知限制与延后工作 + + + +- 目前验收范围是仓库中的 macOS Web 启动。桌面安装包集成与系统授权引导另行处理。 +- 开启影响当前 profile 的工具呈现和 PTC 执行器；它不是逐任务的模式开关。 + + +### 开发备注 + +None. diff --git a/packages/computer-use/cua-bundle/cordis.patch.yml b/packages/computer-use/cua-bundle/cordis.patch.yml new file mode 100644 index 0000000000..c4b0d5961c --- /dev/null +++ b/packages/computer-use/cua-bundle/cordis.patch.yml @@ -0,0 +1,12 @@ +- id: ptc-runtime + disabled: true +- id: tools + config: + mode: ptc +- insert: + - id: cua-provider-registry + name: '@deepseek-ai/dsh-computer-use' + - id: cua + name: '@deepseek-ai/dsh-cua' + - id: ptc-runtime-persistent + name: '@deepseek-ai/dsh-ptc-runtime-node/persistent' diff --git a/packages/computer-use/cua-bundle/icon.svg b/packages/computer-use/cua-bundle/icon.svg new file mode 100644 index 0000000000..5d50e0fa5a --- /dev/null +++ b/packages/computer-use/cua-bundle/icon.svg @@ -0,0 +1,12 @@ + + + + + + + + + + + + diff --git a/packages/computer-use/cua-bundle/locale/en.json b/packages/computer-use/cua-bundle/locale/en.json new file mode 100644 index 0000000000..770d339d8e --- /dev/null +++ b/packages/computer-use/cua-bundle/locale/en.json @@ -0,0 +1,6 @@ +{ + "meta": { + "title": "Computer Use", + "description": "Operate macOS apps. Enabling switches this profile to persistent PTC, retaining code variables across calls without a file sandbox. First activation prepares local components; later project builds check for updates. Disabling clears retained variables." + } +} diff --git a/packages/computer-use/cua-bundle/locale/zh.json b/packages/computer-use/cua-bundle/locale/zh.json new file mode 100644 index 0000000000..2ebdee0123 --- /dev/null +++ b/packages/computer-use/cua-bundle/locale/zh.json @@ -0,0 +1,6 @@ +{ + "meta": { + "title": "Computer Use", + "description": "操作 macOS 应用。启用后当前配置档切换为 persistent PTC，代码变量跨调用保留，不提供文件沙箱。首次开启会准备本机组件，后续项目构建会检查更新。关闭会清除保留的变量。" + } +} diff --git a/packages/computer-use/cua-bundle/package.json b/packages/computer-use/cua-bundle/package.json new file mode 100644 index 0000000000..ac420907c9 --- /dev/null +++ b/packages/computer-use/cua-bundle/package.json @@ -0,0 +1,54 @@ +{ + "name": "@deepseek-ai/dsh-cua-bundle", + "description": "Optional macOS Computer Use with persistent PTC", + "version": "0.1.7-rc.2", + "icon": "./icon.svg", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/computer-use/cua-bundle" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./cordis.patch.yml": "./cordis.patch.yml", + "./locale/*.json": "./locale/*.json", + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "icon.svg", + "locale/*.json", + "lib/index.js", + "cordis.patch.yml", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "dsh": { + "bundle": { + "patch": "./cordis.patch.yml", + "experimental": true + } + }, + "dependencies": { + "@deepseek-ai/dsh-cua": "workspace:*", + "@deepseek-ai/dsh-computer-use": "workspace:*", + "@deepseek-ai/dsh-ptc-runtime-node": "workspace:*" + }, + "peerDependencies": { + "@deepseek-ai/cordis": "workspace:~" + }, + "devDependencies": { + "@deepseek-ai/cordis": "workspace:~", + "@deepseek-ai/cordis-plugin-include": "workspace:~", + "js-yaml": "^4.2.0" + } +} diff --git a/packages/computer-use/cua-bundle/src/index.ts b/packages/computer-use/cua-bundle/src/index.ts new file mode 100644 index 0000000000..f04783f7c0 --- /dev/null +++ b/packages/computer-use/cua-bundle/src/index.ts @@ -0,0 +1,2 @@ +/** Optional Computer Use composition; configuration rows live in cordis.patch.yml. */ +export {} diff --git a/packages/computer-use/cua-bundle/tsconfig.json b/packages/computer-use/cua-bundle/tsconfig.json new file mode 100644 index 0000000000..f1a449634c --- /dev/null +++ b/packages/computer-use/cua-bundle/tsconfig.json @@ -0,0 +1,15 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cordis" + } + ] +} diff --git a/packages/computer-use/cua/README.i18n.yaml b/packages/computer-use/cua/README.i18n.yaml new file mode 100644 index 0000000000..fef7888943 --- /dev/null +++ b/packages/computer-use/cua/README.i18n.yaml @@ -0,0 +1,52 @@ +# Bilingual-pair consistency record for README.md (docs/i18n/README.md): per heading +# section, a hash of its English and Chinese blocks outside code blocks and generated regions. +# After editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/computer-use/cua/README.md +/: + en: a1503331d3f35516 + zh: fcb8e5220ff732e1 +/deepseek-ai-dsh-cua: + en: 10b9bc038b8de707 + zh: 60d3eaf0b4349754 +/deepseek-ai-dsh-cua/summary: + en: 4549f02fc53854a5 + zh: 6f74f1130b6623a8 +/deepseek-ai-dsh-cua/table-of-contents: + en: d152484eb41ac6b4 + zh: 7b30b973a849247d +/deepseek-ai-dsh-cua/use-this-package: + en: 4978d663fc761c08 + zh: f39f8469cdf12004 +/deepseek-ai-dsh-cua/use-this-package/connect-an-external-execution-environment: + en: 59b551986b1da1df + zh: 0dd69fc4633492df +/deepseek-ai-dsh-cua/understand-the-implementation: + en: ee9fd0ddf93d496d + zh: 38d8d5f8b58e6f50 +/deepseek-ai-dsh-cua/further-exploration: + en: b1047318879cd6f4 + zh: e4ce4a5ad2a9c4dc +/deepseek-ai-dsh-cua/model-experience: + en: 215e7ba838619b7b + zh: a311da3843709f90 +/deepseek-ai-dsh-cua/model-experience/native-application-operations: + en: ab7cba4a42334ab9 + zh: f77cd9d84624b6c2 +/deepseek-ai-dsh-cua/model-experience/native-application-operations/what-the-model-sees: + en: c221fa5ce647b1dd + zh: 9388c91a9f89e029 +/deepseek-ai-dsh-cua/model-experience/native-application-operations/what-the-model-sees/operation-guidance-verbatim: + en: 2dbc22ffd1f8a717 + zh: d4458872e75305d9 +/deepseek-ai-dsh-cua/model-experience/native-application-operations/token-effect: + en: ab52b46d874fa8b9 + zh: e87d3620ef77cb06 +/deepseek-ai-dsh-cua/model-experience/native-application-operations/kv-cache-effect: + en: a2c9acb5a4320dda + zh: e8d1586af60215c4 +/deepseek-ai-dsh-cua/known-limitations-and-deferred-work: + en: 0fd701796488ac03 + zh: 9ce1ed68bc3cd624 +/deepseek-ai-dsh-cua/known-limitations-and-deferred-work/dev-note: + en: b836f01bd0bceb4a + zh: 8122fa08b2aa95fa diff --git a/packages/experimental/cua/README.md b/packages/computer-use/cua/README.md similarity index 91% rename from packages/experimental/cua/README.md rename to packages/computer-use/cua/README.md index 0267bb2e17..72c8d411e2 100644 --- a/packages/experimental/cua/README.md +++ b/packages/computer-use/cua/README.md @@ -3,7 +3,7 @@ description: "Inspect and operate native macOS applications through scoped app a kind: "package-reference" --- -# @deepseek-ai/dsh-experimental-cua +# @deepseek-ai/dsh-cua English | [中文](README.zh.md) @@ -25,20 +25,21 @@ Inspect macOS permissions, select running applications and windows, read accessi ## Use this package -Build with Swift 6 and Xcode command-line tools on macOS 14 or later, then mount the source overlay from the repository root: +For repository Web use, enable [Computer Use](../cua-bundle/README.md) in Plugins. This prepares the local executable on macOS 14 or later; subsequent builds check it while the existing Web profile keeps the bundle selected. Disabled profiles do not prepare native components. Explicit source composition is also available: ```sh -swift build --package-path packages/experimental/cua/native --configuration release -cua_native_root="$PWD/packages/experimental/cua/native" +swift build --package-path packages/computer-use/cua/native --configuration release +cua_native_root="$PWD/packages/computer-use/cua/native" export DSH_CUA_HELPER="$cua_native_root/.build/release/dsh-cua-native" -pnpm dsh headless --patch packages/experimental/cua/examples/macos.cordis.yml --dump-config +pnpm dsh headless --patch packages/computer-use/cua/examples/macos.cordis.yml --dump-config ``` The dump command checks profile composition without a model request. For a real task, replace `--dump-config` with a prompt and configure the profile’s model credentials. The overlay mounts the computer-use registry and this provider; remove an existing competing provider before using it. The responsible host/helper needs Accessibility permission for AX/input and Screen Recording permission for capture. `getCapabilities` reports permissions and session lock state without opening a system dialog. Locked or unavailable desktop sessions reject UI observations and input before dispatch; the user must unlock macOS before resuming. Start the target application yourself. Select it by name, bundle ID, absolute application path, or PID. `getApp` reads the selected application without activating it. Model guidance starts with background observation and input, without activation during initialization or before each action. If a hidden window, another Space or a Stage Manager thumbnail prevents observation, explain the foreground-dependent step before `app.activate()` or an observed window’s `raise` action, then read fresh state. Applications may change focus after input; background delivery does not guarantee an undisturbed user session. | Field | Default | Meaning | | --- | --- | --- | -| `helperPath` | required | Absolute native executable path. | +| `helperPath` | package-owned executable | Optional absolute native executable override. | +| `buildTimeoutMs` | `300000` | Maximum native preparation time. | | `timeoutMs` | `30000` | Per-operation deadline; cancellation can leave partial input. | | `axTimeoutMs` | `3000` | Deadline per Accessibility system call, configured independently of the complete operation. | | `maxRequestBytes` / `maxResponseBytes` | `1048576` / `33554432` | UTF-8 request and response limits. | @@ -54,12 +55,14 @@ Agent disposal and plugin unload cancel and await the complete tool call, includ Raw `getState` exposes the validated state fields and its complete `tree`. The native helper also returns structured element diagnostics for native checks; the tool projection omits that duplicate array from model text and structured tool values. +Native preparation serializes compiler checks and executable publication with the shared file lock, retains the native build cache, and replaces the executable atomically only when its bytes change. Cancellation and preparation failure finish managed process cleanup before releasing the lock. + ### Connect an external execution environment -To run through persistent PTC, add `--patch packages/experimental/cua/examples/persistent-ptc.cordis.yml` after the macOS overlay. This provider keeps variables for the live Agent and runs without a file sandbox. The caller imports the bridge and supplies the current authorized `tools.cua` binding. This integration snippet assumes an execution host has supplied `tools`; the package does not install a global object or keep variables alive: +To run through persistent PTC, add `--patch packages/computer-use/cua/examples/persistent-ptc.cordis.yml` after the macOS overlay. This provider keeps variables for the live Agent and runs without a file sandbox. The caller imports the bridge and supplies the current authorized `tools.cua` binding. This integration snippet assumes an execution host has supplied `tools`; the package does not install a global object or keep variables alive: ```javascript -import { connectCua } from '@deepseek-ai/dsh-experimental-cua/bridge' +import { connectCua } from '@deepseek-ai/dsh-cua/bridge' const cua = connectCua(() => tools.cua, text => console.log(text)) const app = await cua.getApp("com.apple.TextEdit") const windows = await app.getWindows() @@ -100,7 +103,7 @@ Node roles and metadata share one AX batch per observation. Group and scroll-are ## Further Exploration -- [Computer-use registration](../../computer-use/computer-use/README.md) +- [Computer-use registration](../computer-use/README.md) - [Native protocol](native/protocol.schema.json) - [Object interfaces](src/types.ts) @@ -148,7 +151,7 @@ Raw operation tools use { method, params }, including params: {} for parameterle Await operations sequentially. After input, read fresh state to verify the effect. A successful action means dispatched, not verified completion. Cancellation may leave partial effects; never retry uncertain input without observing. STALE_APP requires getApp again; STALE_SNAPSHOT requires getState again. WINDOW_CHANGED requires fresh state before any retry; use an advertised AX action or explicitly restore the window before screenshot or coordinate input, then observe again. PERMISSION_DENIED requires the user to grant macOS Accessibility or Screen Recording permission. -Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-experimental-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value. +Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value. ``` #### Token effect diff --git a/packages/experimental/cua/README.zh.md b/packages/computer-use/cua/README.zh.md similarity index 91% rename from packages/experimental/cua/README.zh.md rename to packages/computer-use/cua/README.zh.md index 40a57a9c91..14eff6e828 100644 --- a/packages/experimental/cua/README.zh.md +++ b/packages/computer-use/cua/README.zh.md @@ -3,7 +3,7 @@ description: "通过按应用和窗口限定的对象，观察和操作原生 ma kind: "package-reference" --- -# @deepseek-ai/dsh-experimental-cua +# @deepseek-ai/dsh-cua [English](README.md) | 中文 @@ -25,20 +25,21 @@ kind: "package-reference" ## 使用本包 -在 macOS 14 或更高版本上使用 Swift 6 和 Xcode 命令行工具构建，然后从仓库根目录挂载源码配置： +从仓库运行 Web 时，在插件页开启 [Computer Use](../cua-bundle/README.zh.md)。首次启用会在 macOS 14 及以上准备本机组件；现有 Web profile 保持选中时，后续构建继续检查更新。未开启的 profile 不准备原生组件。也可以显式加载源码覆盖层： ```sh -swift build --package-path packages/experimental/cua/native --configuration release -cua_native_root="$PWD/packages/experimental/cua/native" +swift build --package-path packages/computer-use/cua/native --configuration release +cua_native_root="$PWD/packages/computer-use/cua/native" export DSH_CUA_HELPER="$cua_native_root/.build/release/dsh-cua-native" -pnpm dsh headless --patch packages/experimental/cua/examples/macos.cordis.yml --dump-config +pnpm dsh headless --patch packages/computer-use/cua/examples/macos.cordis.yml --dump-config ``` 配置输出命令不发起模型请求。执行真实任务时，将 `--dump-config` 替换为任务提示，并配置该 profile 的模型凭据。配置挂载 computer-use 注册表和本 provider；使用前需移除已有的竞争 provider。负责执行的宿主或辅助进程需要辅助功能权限来读取 AX 和输入，需要屏幕录制权限来截图。`getCapabilities` 查询权限和锁屏状态，不弹出系统授权窗口。桌面会话锁定或不可用时，界面观察与输入会在分派前拒绝，需由用户解锁 macOS 后继续。目标应用需提前启动，可按名称、bundle ID、应用绝对路径或 PID 选择。`getApp` 读取选中的应用，不自动激活它。模型指引要求先在后台观察和输入，不在初始化或每次动作前激活应用。窗口隐藏、位于其他桌面空间或被台前调度缩为缩略图而无法观察时，先说明需要前台的步骤，再显式调用 `app.activate()` 或窗口的 `raise` 动作，之后重新读取状态。应用收到输入后仍可能改变焦点，后台投递不保证用户操作始终不受干扰。 | 字段 | 默认值 | 含义 | | --- | --- | --- | -| `helperPath` | 必填 | 原生可执行文件的绝对路径。 | +| `helperPath` | 包内可执行文件 | 可选的原生可执行文件绝对路径覆盖。 | +| `buildTimeoutMs` | `300000` | 原生组件准备的最长时间。 | | `timeoutMs` | `30000` | 单次操作截止时间，取消后可能已发生部分输入。 | | `axTimeoutMs` | `3000` | 每次 Accessibility 系统调用的截止时间，与完整操作截止独立配置。 | | `maxRequestBytes` / `maxResponseBytes` | `1048576` / `33554432` | UTF-8 请求与响应字节限制。 | @@ -54,12 +55,14 @@ Agent 销毁和插件卸载会取消并等待完整工具调用，包括图像 原始 `getState` 操作返回经过校验的状态字段和完整 `tree`。原生 helper 还返回供原生检查使用的结构化元素诊断数组；工具投影会从模型文本和结构化工具值中移除这份重复数组。 +原生准备复用文件锁，串行检查编译和发布可执行文件，保留构建缓存，只在字节变化时原子替换可执行文件。取消或准备失败后，先完成受管进程清理再释放锁。 + ### 接入外部执行环境 -使用 persistent PTC 时，在 macOS 配置后追加 `--patch packages/experimental/cua/examples/persistent-ptc.cordis.yml`。该 provider 为存活的 Agent 保留变量，不启用文件沙箱。调用方导入适配模块，并提供当前执行中已授权的 `tools.cua` 绑定。下面的接入代码假设执行宿主已提供 `tools`；本包不安装全局对象，也不负责保留变量： +使用 persistent PTC 时，在 macOS 配置后追加 `--patch packages/computer-use/cua/examples/persistent-ptc.cordis.yml`。该 provider 为存活的 Agent 保留变量，不启用文件沙箱。调用方导入适配模块，并提供当前执行中已授权的 `tools.cua` 绑定。下面的接入代码假设执行宿主已提供 `tools`；本包不安装全局对象，也不负责保留变量： ```javascript -import { connectCua } from '@deepseek-ai/dsh-experimental-cua/bridge' +import { connectCua } from '@deepseek-ai/dsh-cua/bridge' const cua = connectCua(() => tools.cua, text => console.log(text)) const app = await cua.getApp("com.apple.TextEdit") const windows = await app.getWindows() @@ -100,7 +103,7 @@ AppKit sheet 可以通过其窗口 ID 单独选择，也会出现在所属文档 ## 进一步阅读 -- [Computer-use 注册](../../computer-use/computer-use/README.zh.md) +- [Computer-use 注册](../computer-use/README.zh.md) - [原生协议](native/protocol.schema.json) - [对象接口](src/types.ts) @@ -148,7 +151,7 @@ Raw operation tools use { method, params }, including params: {} for parameterle Await operations sequentially. After input, read fresh state to verify the effect. A successful action means dispatched, not verified completion. Cancellation may leave partial effects; never retry uncertain input without observing. STALE_APP requires getApp again; STALE_SNAPSHOT requires getState again. WINDOW_CHANGED requires fresh state before any retry; use an advertised AX action or explicitly restore the window before screenshot or coordinate input, then observe again. PERMISSION_DENIED requires the user to grant macOS Accessibility or Screen Recording permission. -Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-experimental-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value. +Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value. ``` #### Token 影响 diff --git a/packages/experimental/cua/examples/macos.cordis.yml b/packages/computer-use/cua/examples/macos.cordis.yml similarity index 60% rename from packages/experimental/cua/examples/macos.cordis.yml rename to packages/computer-use/cua/examples/macos.cordis.yml index da90ec37ef..af6b19ce4c 100644 --- a/packages/experimental/cua/examples/macos.cordis.yml +++ b/packages/computer-use/cua/examples/macos.cordis.yml @@ -1,7 +1,7 @@ - insert: - id: cua-provider-registry - name: '../../../computer-use/computer-use/src/index.ts' - - id: experimental-cua + name: '../../computer-use/src/index.ts' + - id: cua name: '../src/index.ts' config: helperPath: !!js process.env.DSH_CUA_HELPER diff --git a/packages/experimental/cua/examples/persistent-ptc.cordis.yml b/packages/computer-use/cua/examples/persistent-ptc.cordis.yml similarity index 100% rename from packages/experimental/cua/examples/persistent-ptc.cordis.yml rename to packages/computer-use/cua/examples/persistent-ptc.cordis.yml diff --git a/packages/experimental/cua/native/.gitignore b/packages/computer-use/cua/native/.gitignore similarity index 83% rename from packages/experimental/cua/native/.gitignore rename to packages/computer-use/cua/native/.gitignore index 3168a3629c..7befc780e8 100644 --- a/packages/experimental/cua/native/.gitignore +++ b/packages/computer-use/cua/native/.gitignore @@ -1,3 +1,4 @@ target/ .build/ .swiftpm/ +bin/ diff --git a/packages/experimental/cua/native/LICENSE.cua-driver b/packages/computer-use/cua/native/LICENSE.cua-driver similarity index 100% rename from packages/experimental/cua/native/LICENSE.cua-driver rename to packages/computer-use/cua/native/LICENSE.cua-driver diff --git a/packages/experimental/cua/native/Package.swift b/packages/computer-use/cua/native/Package.swift similarity index 100% rename from packages/experimental/cua/native/Package.swift rename to packages/computer-use/cua/native/Package.swift diff --git a/packages/experimental/cua/native/README.txt b/packages/computer-use/cua/native/README.txt similarity index 94% rename from packages/experimental/cua/native/README.txt rename to packages/computer-use/cua/native/README.txt index c003c8ced6..9981e3491a 100644 --- a/packages/experimental/cua/native/README.txt +++ b/packages/computer-use/cua/native/README.txt @@ -2,7 +2,7 @@ This macOS 14+ JSON-lines helper uses Swift 6, AppKit, Accessibility, CoreGraphics and ScreenCaptureKit. It adapts selected MIT-licensed Cua Driver behavior without loading an external driver. `upstream.json` records the fixed upstream version, adapted Swift files and local modules; `LICENSE.cua-driver` preserves the MIT notice. -Build with `swift build --package-path packages/experimental/cua/native --configuration release`. The binary is `.build/release/dsh-cua-native` beneath this directory. The DSH experimental plugin owns its process lifetime. EOF drains an accepted final request, releases retained AX references and exits. The `.build/` directory is not committed. +Build with `swift build --package-path packages/computer-use/cua/native --configuration release`. The binary is `.build/release/dsh-cua-native` beneath this directory. The DSH CUA plugin owns its process lifetime. EOF drains an accepted final request, releases retained AX references and exits. The `.build/` directory is not committed. Each stdin line is `{ "id": "opaque request id", "method": "method name", "params": {} }`. Each stdout line contains the same ID and exactly one of `result` or `error: {code, message}`. Malformed envelopes return `INVALID_REQUEST` with a null ID. Requests execute serially. Diagnostics must not be written to stdout. `protocol.schema.json` defines request fields. @@ -32,25 +32,25 @@ Capture and pointer delivery read CG bounds before and after an exact-window AX The responsible host/helper needs macOS Accessibility permission for AX state and input, and Screen Recording permission for screenshots. Capability preflight and permission errors do not grant permissions. Capture refuses a window that moves or resizes while the image is being read. Unassociated system panels and accessory-only applications are not selectable. -Use `python3 packages/experimental/cua/native/tests/test_swift.py` for offline unit checks with the selected Apple toolchain’s Swift Testing framework/runtime paths and `python3 packages/experimental/cua/native/tests/protocol_e2e.py` for JSONL admission, ordered replies and EOF shutdown without desktop operations. All Python native checks accept `--helper` to select a separately built comparison binary. +Use `python3 packages/computer-use/cua/native/tests/test_swift.py` for offline unit checks with the selected Apple toolchain’s Swift Testing framework/runtime paths and `python3 packages/computer-use/cua/native/tests/protocol_e2e.py` for JSONL admission, ordered replies and EOF shutdown without desktop operations. All Python native checks accept `--helper` to select a separately built comparison binary. -Run `python3 packages/experimental/cua/native/tests/axless_e2e.py` after building and unlocking the Mac to verify an owned window whose application removes and later restores its AX roots. The check requires existing Accessibility/Screen Recording grants, rejects stale element indices, captures the AXless window, verifies a coordinate click in the fixture state file, and checks restoration. It never changes system permission settings. +Run `python3 packages/computer-use/cua/native/tests/axless_e2e.py` after building and unlocking the Mac to verify an owned window whose application removes and later restores its AX roots. The check requires existing Accessibility/Screen Recording grants, rejects stale element indices, captures the AXless window, verifies a coordinate click in the fixture state file, and checks restoration. It never changes system permission settings. Stable error codes are `INVALID_REQUEST` (malformed envelope), `INVALID_ARGUMENT` (invalid parameters), `UNSUPPORTED` (unknown operation), `APP_UNAVAILABLE`, `AMBIGUOUS_APP`, `STALE_APP`, `STALE_SNAPSHOT`, `WINDOW_UNAVAILABLE`, `WINDOW_CHANGED`, `PERMISSION_DENIED`, `PANEL_NOT_READY` (system panel recipient unavailable before input), and `NATIVE_ERROR` (remaining operating-system failures). An error never promises that a partially dispatched input was rolled back. The host must not retry input automatically. Observation reads each non-layout node’s descriptive properties and children through `AXUIElementCopyMultipleAttributeValues`. Per-attribute errors retain the individual-read rules; unsupported batch APIs use individual reads. Values are retained only during that observation, and input reads live attributes. Application state can change between nodes; a batch does not make the whole tree atomic. -Run `python3 packages/experimental/cua/native/tests/observation_benchmark.py --reference-helper /absolute/reference --candidate-helper /absolute/candidate --artifacts /private/evidence` to compare complete observations of the same owned AppKit window with 35 and 400 synthetic rows. Each helper receives one warmup and three alternating measured requests. All result fields must match except helper-generated app and snapshot IDs. The report keeps raw timings, byte counts, hashes and replies; it sets no CI speed threshold and makes no memory claim. Fixture startup can activate the application. Run without other desktop automation or compilation jobs. +Run `python3 packages/computer-use/cua/native/tests/observation_benchmark.py --reference-helper /absolute/reference --candidate-helper /absolute/candidate --artifacts /private/evidence` to compare complete observations of the same owned AppKit window with 35 and 400 synthetic rows. Each helper receives one warmup and three alternating measured requests. All result fields must match except helper-generated app and snapshot IDs. The report keeps raw timings, byte counts, hashes and replies; it sets no CI speed threshold and makes no memory claim. Fixture startup can activate the application. Run without other desktop automation or compilation jobs. Use `--panels-only` to isolate the system file-panel checks. Fixture windows can join application sets so Stage Manager does not shrink them during background input. -Run `python3 packages/experimental/cua/native/tests/e2e.py` after building to exercise an isolated two-window AppKit fixture. It independently records button counts, Unicode text, actual scroll position, right-click callbacks, and drag release position. Scroll, right-button down/up, and drag down/up counters must each increase exactly once per requested gesture. The check also verifies a real NSOpenPanel sheet, path-entry keyboard routing and the selected file URL. It covers scoped snapshot replacement, screenshot dimensions, stale app cleanup and EOF shutdown. `--artifacts DIRECTORY` writes fixture-only screenshots and a private protocol transcript; the transcript contains the running-app inventory and is not a publication artifact. +Run `python3 packages/computer-use/cua/native/tests/e2e.py` after building to exercise an isolated two-window AppKit fixture. It independently records button counts, Unicode text, actual scroll position, right-click callbacks, and drag release position. Scroll, right-button down/up, and drag down/up counters must each increase exactly once per requested gesture. The check also verifies a real NSOpenPanel sheet, path-entry keyboard routing and the selected file URL. It covers scoped snapshot replacement, screenshot dimensions, stale app cleanup and EOF shutdown. `--artifacts DIRECTORY` writes fixture-only screenshots and a private protocol transcript; the transcript contains the running-app inventory and is not a publication artifact. -For a Blender keyboard check, run `python3 packages/experimental/cua/native/tests/blender_keyboard_e2e.py --blender /absolute/path/to/Blender.app/Contents/MacOS/Blender`. It launches one owned factory-startup GUI with private user directories, observes actual Text Editor text, selection and received modifier flags, and checks Command+A replacement, Shift+Left selection and ordinary input after modifier release. It requires an unlocked macOS session and Accessibility permission. The test activates its own instance; run it without concurrent desktop automation. `--helper` selects a previous compatible binary for a negative control; `--artifacts` selects the parent of a new private evidence directory. No Blender runtime is downloaded. +For a Blender keyboard check, run `python3 packages/computer-use/cua/native/tests/blender_keyboard_e2e.py --blender /absolute/path/to/Blender.app/Contents/MacOS/Blender`. It launches one owned factory-startup GUI with private user directories, observes actual Text Editor text, selection and received modifier flags, and checks Command+A replacement, Shift+Left selection and ordinary input after modifier release. It requires an unlocked macOS session and Accessibility permission. The test activates its own instance; run it without concurrent desktop automation. `--helper` selects a previous compatible binary for a negative control; `--artifacts` selects the parent of a new private evidence directory. No Blender runtime is downloaded. For text pacing comparisons, use `native/tests/text_input_benchmark.py` with `--reference-helper`, `--candidate-helper`, `--blender` and `--artifacts`. It types identical synthetic 200/2000-scalar strings into fresh Blender Text Editors and requires full text equality, including Chinese, emoji and newlines. Repeated `--reference-arg` / `--candidate-arg` options select helper startup flags. Raw request and verified-effect timings include observer publication latency and exclude startup; these manual measurements have no CI speed threshold. -For an independently isolated Electron/Chromium check, run `python3 packages/experimental/cua/native/tests/electron_e2e.py --electron /absolute/path/to/Electron.app/Contents/MacOS/Electron`. Omitting `--electron` uses an already installed desktop workspace runtime. The check never downloads Electron. Its renderer runs with Node integration disabled, context isolation and Chromium sandboxing enabled, private userData/sessionData/log directories, and a local fixture page. Renderer click/input events write independent observations through the main process; all test actions still use the native helper. AX polling waits for actual controls after Chromium accessibility activation. Native and Electron subprocess groups are owned and stopped before temporary files are removed. Optional artifacts include a fixture-only screenshot and private request transcript. +For an independently isolated Electron/Chromium check, run `python3 packages/computer-use/cua/native/tests/electron_e2e.py --electron /absolute/path/to/Electron.app/Contents/MacOS/Electron`. Omitting `--electron` uses an already installed desktop workspace runtime. The check never downloads Electron. Its renderer runs with Node integration disabled, context isolation and Chromium sandboxing enabled, private userData/sessionData/log directories, and a local fixture page. Renderer click/input events write independent observations through the main process; all test actions still use the native helper. AX polling waits for actual controls after Chromium accessibility activation. Native and Electron subprocess groups are owned and stopped before temporary files are removed. Optional artifacts include a fixture-only screenshot and private request transcript. System file-panel keyboard routing resolves the remote recipient through the selected app’s focused AX sheet. Only Apple’s Open and Save Panel Service is accepted, and its focused ancestry must still contain the identified remote window. Element pointer fallbacks resolve an app-owned window or sheet through AX parents when AXWindow is unavailable. AX actions retain priority over pointer input; visual click markers need no system-cursor movement. diff --git a/packages/experimental/cua/native/Sources/CuaNative/AXAccess.swift b/packages/computer-use/cua/native/Sources/CuaNative/AXAccess.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/AXAccess.swift rename to packages/computer-use/cua/native/Sources/CuaNative/AXAccess.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/AXWorker.swift b/packages/computer-use/cua/native/Sources/CuaNative/AXWorker.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/AXWorker.swift rename to packages/computer-use/cua/native/Sources/CuaNative/AXWorker.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/KeyboardInput.swift b/packages/computer-use/cua/native/Sources/CuaNative/KeyboardInput.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/KeyboardInput.swift rename to packages/computer-use/cua/native/Sources/CuaNative/KeyboardInput.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/Main.swift b/packages/computer-use/cua/native/Sources/CuaNative/Main.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/Main.swift rename to packages/computer-use/cua/native/Sources/CuaNative/Main.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/NativePlatform.swift b/packages/computer-use/cua/native/Sources/CuaNative/NativePlatform.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/NativePlatform.swift rename to packages/computer-use/cua/native/Sources/CuaNative/NativePlatform.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/PIDEventPoster.swift b/packages/computer-use/cua/native/Sources/CuaNative/PIDEventPoster.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/PIDEventPoster.swift rename to packages/computer-use/cua/native/Sources/CuaNative/PIDEventPoster.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/Protocol.swift b/packages/computer-use/cua/native/Sources/CuaNative/Protocol.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/Protocol.swift rename to packages/computer-use/cua/native/Sources/CuaNative/Protocol.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/RequestAdmission.swift b/packages/computer-use/cua/native/Sources/CuaNative/RequestAdmission.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/RequestAdmission.swift rename to packages/computer-use/cua/native/Sources/CuaNative/RequestAdmission.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/RequestCoordinator.swift b/packages/computer-use/cua/native/Sources/CuaNative/RequestCoordinator.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/RequestCoordinator.swift rename to packages/computer-use/cua/native/Sources/CuaNative/RequestCoordinator.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/Runtime.swift b/packages/computer-use/cua/native/Sources/CuaNative/Runtime.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/Runtime.swift rename to packages/computer-use/cua/native/Sources/CuaNative/Runtime.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/WindowCapture.swift b/packages/computer-use/cua/native/Sources/CuaNative/WindowCapture.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/WindowCapture.swift rename to packages/computer-use/cua/native/Sources/CuaNative/WindowCapture.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/WindowGeometry.swift b/packages/computer-use/cua/native/Sources/CuaNative/WindowGeometry.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/WindowGeometry.swift rename to packages/computer-use/cua/native/Sources/CuaNative/WindowGeometry.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/WindowPointerInput.swift b/packages/computer-use/cua/native/Sources/CuaNative/WindowPointerInput.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/WindowPointerInput.swift rename to packages/computer-use/cua/native/Sources/CuaNative/WindowPointerInput.swift diff --git a/packages/experimental/cua/native/Sources/CuaNative/WindowSpaces.swift b/packages/computer-use/cua/native/Sources/CuaNative/WindowSpaces.swift similarity index 100% rename from packages/experimental/cua/native/Sources/CuaNative/WindowSpaces.swift rename to packages/computer-use/cua/native/Sources/CuaNative/WindowSpaces.swift diff --git a/packages/experimental/cua/native/protocol.schema.json b/packages/computer-use/cua/native/protocol.schema.json similarity index 100% rename from packages/experimental/cua/native/protocol.schema.json rename to packages/computer-use/cua/native/protocol.schema.json diff --git a/packages/experimental/cua/native/tests/AxlessFixture.swift b/packages/computer-use/cua/native/tests/AxlessFixture.swift similarity index 100% rename from packages/experimental/cua/native/tests/AxlessFixture.swift rename to packages/computer-use/cua/native/tests/AxlessFixture.swift diff --git a/packages/experimental/cua/native/tests/Fixture.swift b/packages/computer-use/cua/native/tests/Fixture.swift similarity index 100% rename from packages/experimental/cua/native/tests/Fixture.swift rename to packages/computer-use/cua/native/tests/Fixture.swift diff --git a/packages/experimental/cua/native/tests/axless_e2e.py b/packages/computer-use/cua/native/tests/axless_e2e.py similarity index 100% rename from packages/experimental/cua/native/tests/axless_e2e.py rename to packages/computer-use/cua/native/tests/axless_e2e.py diff --git a/packages/experimental/cua/native/tests/blender_keyboard_e2e.py b/packages/computer-use/cua/native/tests/blender_keyboard_e2e.py similarity index 100% rename from packages/experimental/cua/native/tests/blender_keyboard_e2e.py rename to packages/computer-use/cua/native/tests/blender_keyboard_e2e.py diff --git a/packages/experimental/cua/native/tests/blender_keyboard_fixture.py b/packages/computer-use/cua/native/tests/blender_keyboard_fixture.py similarity index 100% rename from packages/experimental/cua/native/tests/blender_keyboard_fixture.py rename to packages/computer-use/cua/native/tests/blender_keyboard_fixture.py diff --git a/packages/experimental/cua/native/tests/e2e.py b/packages/computer-use/cua/native/tests/e2e.py similarity index 100% rename from packages/experimental/cua/native/tests/e2e.py rename to packages/computer-use/cua/native/tests/e2e.py diff --git a/packages/experimental/cua/native/tests/electron-fixture.html b/packages/computer-use/cua/native/tests/electron-fixture.html similarity index 100% rename from packages/experimental/cua/native/tests/electron-fixture.html rename to packages/computer-use/cua/native/tests/electron-fixture.html diff --git a/packages/experimental/cua/native/tests/electron-fixture.mjs b/packages/computer-use/cua/native/tests/electron-fixture.mjs similarity index 100% rename from packages/experimental/cua/native/tests/electron-fixture.mjs rename to packages/computer-use/cua/native/tests/electron-fixture.mjs diff --git a/packages/experimental/cua/native/tests/electron_e2e.py b/packages/computer-use/cua/native/tests/electron_e2e.py similarity index 100% rename from packages/experimental/cua/native/tests/electron_e2e.py rename to packages/computer-use/cua/native/tests/electron_e2e.py diff --git a/packages/experimental/cua/native/tests/native_test_support.py b/packages/computer-use/cua/native/tests/native_test_support.py similarity index 100% rename from packages/experimental/cua/native/tests/native_test_support.py rename to packages/computer-use/cua/native/tests/native_test_support.py diff --git a/packages/experimental/cua/native/tests/observation_benchmark.py b/packages/computer-use/cua/native/tests/observation_benchmark.py similarity index 100% rename from packages/experimental/cua/native/tests/observation_benchmark.py rename to packages/computer-use/cua/native/tests/observation_benchmark.py diff --git a/packages/experimental/cua/native/tests/protocol_e2e.py b/packages/computer-use/cua/native/tests/protocol_e2e.py similarity index 100% rename from packages/experimental/cua/native/tests/protocol_e2e.py rename to packages/computer-use/cua/native/tests/protocol_e2e.py diff --git a/packages/experimental/cua/native/tests/swift/AXFailureTests.swift b/packages/computer-use/cua/native/tests/swift/AXFailureTests.swift similarity index 100% rename from packages/experimental/cua/native/tests/swift/AXFailureTests.swift rename to packages/computer-use/cua/native/tests/swift/AXFailureTests.swift diff --git a/packages/experimental/cua/native/tests/swift/ForegroundTests.swift b/packages/computer-use/cua/native/tests/swift/ForegroundTests.swift similarity index 100% rename from packages/experimental/cua/native/tests/swift/ForegroundTests.swift rename to packages/computer-use/cua/native/tests/swift/ForegroundTests.swift diff --git a/packages/experimental/cua/native/tests/swift/PointerTimingTests.swift b/packages/computer-use/cua/native/tests/swift/PointerTimingTests.swift similarity index 100% rename from packages/experimental/cua/native/tests/swift/PointerTimingTests.swift rename to packages/computer-use/cua/native/tests/swift/PointerTimingTests.swift diff --git a/packages/experimental/cua/native/tests/swift/ProtocolTests.swift b/packages/computer-use/cua/native/tests/swift/ProtocolTests.swift similarity index 100% rename from packages/experimental/cua/native/tests/swift/ProtocolTests.swift rename to packages/computer-use/cua/native/tests/swift/ProtocolTests.swift diff --git a/packages/experimental/cua/native/tests/swift/WindowGeometryTests.swift b/packages/computer-use/cua/native/tests/swift/WindowGeometryTests.swift similarity index 100% rename from packages/experimental/cua/native/tests/swift/WindowGeometryTests.swift rename to packages/computer-use/cua/native/tests/swift/WindowGeometryTests.swift diff --git a/packages/experimental/cua/native/tests/swift/WindowSpacesTests.swift b/packages/computer-use/cua/native/tests/swift/WindowSpacesTests.swift similarity index 100% rename from packages/experimental/cua/native/tests/swift/WindowSpacesTests.swift rename to packages/computer-use/cua/native/tests/swift/WindowSpacesTests.swift diff --git a/packages/experimental/cua/native/tests/test_cleanup.py b/packages/computer-use/cua/native/tests/test_cleanup.py similarity index 100% rename from packages/experimental/cua/native/tests/test_cleanup.py rename to packages/computer-use/cua/native/tests/test_cleanup.py diff --git a/packages/experimental/cua/native/tests/test_swift.py b/packages/computer-use/cua/native/tests/test_swift.py similarity index 100% rename from packages/experimental/cua/native/tests/test_swift.py rename to packages/computer-use/cua/native/tests/test_swift.py diff --git a/packages/experimental/cua/native/tests/text_input_benchmark.py b/packages/computer-use/cua/native/tests/text_input_benchmark.py similarity index 100% rename from packages/experimental/cua/native/tests/text_input_benchmark.py rename to packages/computer-use/cua/native/tests/text_input_benchmark.py diff --git a/packages/experimental/cua/native/upstream.json b/packages/computer-use/cua/native/upstream.json similarity index 100% rename from packages/experimental/cua/native/upstream.json rename to packages/computer-use/cua/native/upstream.json diff --git a/packages/experimental/cua/package.json b/packages/computer-use/cua/package.json similarity index 85% rename from packages/experimental/cua/package.json rename to packages/computer-use/cua/package.json index 75f1e9dbf7..6d4f388f4c 100644 --- a/packages/experimental/cua/package.json +++ b/packages/computer-use/cua/package.json @@ -1,6 +1,6 @@ { - "name": "@deepseek-ai/dsh-experimental-cua", - "description": "Experimental Codex-style macOS computer use", + "name": "@deepseek-ai/dsh-cua", + "description": "macOS computer use through application and window objects", "version": "0.1.7-rc.2", "publishConfig": { "access": "public" @@ -8,7 +8,7 @@ "repository": { "type": "git", "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", - "directory": "packages/experimental/cua" + "directory": "packages/computer-use/cua" }, "type": "module", "main": "lib/index.js", @@ -27,6 +27,10 @@ "./bridge": { "types": "./lib/types/bridge.d.ts", "default": "./lib/bridge.js" + }, + "./native-build": { + "types": "./lib/types/native-build.d.ts", + "default": "./lib/native-build.js" } }, "files": [ @@ -41,7 +45,9 @@ "native/upstream.json", "native/README.txt", "native/protocol.schema.json", - "lib/types/**/*.d.ts" + "lib/types/**/*.d.ts", + "lib/native-build.js", + "native/bin" ], "license": "MIT", "peerDependencies": { @@ -57,7 +63,8 @@ "@deepseek-ai/dsh-util-values": "workspace:*", "@deepseek-ai/schemastery": "workspace:~", "zod": "^4.4.3", - "@deepseek-ai/dsh-brand": "workspace:*" + "@deepseek-ai/dsh-brand": "workspace:*", + "@deepseek-ai/dsh-atomic-write": "workspace:*" }, "devDependencies": { "@deepseek-ai/cordis": "workspace:~", diff --git a/packages/experimental/cua/src/api.ts b/packages/computer-use/cua/src/api.ts similarity index 100% rename from packages/experimental/cua/src/api.ts rename to packages/computer-use/cua/src/api.ts diff --git a/packages/experimental/cua/src/bridge.ts b/packages/computer-use/cua/src/bridge.ts similarity index 100% rename from packages/experimental/cua/src/bridge.ts rename to packages/computer-use/cua/src/bridge.ts diff --git a/packages/experimental/cua/src/index.ts b/packages/computer-use/cua/src/index.ts similarity index 86% rename from packages/experimental/cua/src/index.ts rename to packages/computer-use/cua/src/index.ts index 4c2f863f43..2545661a7e 100644 --- a/packages/experimental/cua/src/index.ts +++ b/packages/computer-use/cua/src/index.ts @@ -1,5 +1,6 @@ -/** Experimental Codex-style macOS computer use over locally built native source. */ -import { isAbsolute } from 'node:path' +/** macOS computer use over a managed native helper. */ +import { isAbsolute, resolve } from 'node:path' +import { fileURLToPath } from 'node:url' import type { Context } from '@deepseek-ai/cordis' import Schema from '@deepseek-ai/schemastery' import type { Agent } from '@deepseek-ai/dsh-agent' @@ -7,6 +8,7 @@ import { ComputerUseProviderName } from '@deepseek-ai/dsh-computer-use/brand' import { createMcpToolDefinition } from '@deepseek-ai/dsh-mcp-client' import { z } from 'zod' import { NativeClient } from './native-client.ts' +import { nativeHostArchitecture, prepareNativeExecutable } from './native-build.ts' import { Request, ScreenshotResult, StateResult } from './protocol.ts' import type {} from '@deepseek-ai/dsh-system-prompt' import type { ToolExecution } from '@deepseek-ai/dsh-tools' @@ -14,14 +16,16 @@ import type {} from '@deepseek-ai/dsh-subprocess' import type {} from '@deepseek-ai/dsh-computer-use' /** Loader plugin identity. */ -export const name = 'experimental-cua' +export const name = 'cua' /** Services required for logged tools and owned native process lifetime. */ export const inject = ['computerUse', 'tools', 'systemPrompt', 'subprocess', 'agents'] /** Native executable and per-operation resource bounds. */ export interface Config { - /** Absolute path to the locally built macOS native helper. */ - helperPath: string + /** Absolute native helper override; omission prepares the package-owned executable. */ + helperPath?: string + /** Maximum time in milliseconds for native component preparation. */ + buildTimeoutMs: number /** Deadline for one native operation, including capture and AX traversal. */ timeoutMs: number /** Deadline for each individual Accessibility system call. */ @@ -46,7 +50,8 @@ export interface Config { /** Validated host configuration; no driver is downloaded at runtime. */ export const Config: Schema = Schema.object({ - helperPath: Schema.string().required(), + helperPath: Schema.string(), + buildTimeoutMs: Schema.number().step(1).min(1).max(2 ** 31 - 1).default(300_000), timeoutMs: Schema.number().step(1).min(1).max(2 ** 31 - 1).default(30_000), axTimeoutMs: Schema.number().step(1).min(1).max(2 ** 31 - 1).default(3000), maxRequestBytes: Schema.number().step(1).min(1024).default(1024 * 1024), @@ -90,7 +95,7 @@ Raw operation tools use { method, params }, including params: {} for parameterle Await operations sequentially. After input, read fresh state to verify the effect. A successful action means dispatched, not verified completion. Cancellation may leave partial effects; never retry uncertain input without observing. STALE_APP requires getApp again; STALE_SNAPSHOT requires getState again. WINDOW_CHANGED requires fresh state before any retry; use an advertised AX action or explicitly restore the window before screenshot or coordinate input, then observe again. PERMISSION_DENIED requires the user to grant macOS Accessibility or Screen Recording permission. -Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-experimental-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.` +Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.` function executionAgent(execution: Readonly): Agent { if (!execution.agent) throw new Error('dsh-cua requires an agent-owned execution') @@ -100,10 +105,33 @@ function executionAgent(execution: Readonly): Agent { /** * Mount logged native operations with a separate helper per agent. * @param ctx - Host context with computer-use, subprocess and tool services. - * @param config - Validated macOS helper path and bounds. + * @param config - Validated native preparation and operation bounds. + * @returns Preparation completion when the package-owned native executable is selected. */ -export function apply(ctx: Context, config: Config): void { - if (!isAbsolute(config.helperPath)) throw new Error('dsh-cua helperPath must be absolute') +export function apply(ctx: Context, config: Config): void | Promise { + if (config.helperPath !== undefined) { + if (!isAbsolute(config.helperPath)) throw new Error('dsh-cua helperPath must be absolute') + return mount(ctx, { ...config, helperPath: config.helperPath }) + } + const architecture = nativeHostArchitecture(process.platform, process.arch) + const controller = new AbortController() + const prepared = prepareNativeExecutable({ + packageRoot: resolve(fileURLToPath(new URL('..', import.meta.url))), + architecture, subprocess: ctx.subprocess, + timeoutMs: config.buildTimeoutMs, graceMs: config.graceMs, signal: controller.signal, + }) + ctx.effect(() => async () => { + controller.abort() + // The Loader receives preparation failures through apply; disposal only awaits quiescence. + await prepared.catch(() => {}) + }, 'dsh-cua.native-preparation') + return prepared.then((helperPath) => { + controller.signal.throwIfAborted() + mount(ctx, { ...config, helperPath }) + }) +} + +function mount(ctx: Context, config: Config & { helperPath: string }): void { const clients = new Map() const lifetime = new AbortController() const pending = new Map, Agent>() diff --git a/packages/computer-use/cua/src/native-build.ts b/packages/computer-use/cua/src/native-build.ts new file mode 100644 index 0000000000..0feb25e5db --- /dev/null +++ b/packages/computer-use/cua/src/native-build.ts @@ -0,0 +1,116 @@ +/** Source-checkout preparation and installed native executable resolution. */ +import { constants } from 'node:fs' +import { access, mkdir, readFile } from 'node:fs/promises' +import { dirname, join, resolve } from 'node:path' +import { withFileLock, writeFileAtomic } from '@deepseek-ai/dsh-atomic-write' +import type { SubprocessRuntime } from '@deepseek-ai/dsh-subprocess' + +/** Architecture names used by the platform-specific native payload. */ +export type NativeArchitecture = 'arm64' | 'x64' + +/** + * Validate the host before preparing a platform-specific executable. + * @param platform - Node host platform. + * @param architecture - Node host architecture. + * @returns Supported macOS architecture. + * @throws When the current host has no native implementation. + */ +export function nativeHostArchitecture(platform: string, architecture: string): NativeArchitecture { + if (platform !== 'darwin' || (architecture !== 'arm64' && architecture !== 'x64')) throw new Error('Computer Use currently requires macOS on Apple silicon or Intel') + return architecture +} + +/** Explicit compiler ownership and resource limits for one preparation. */ +export interface NativeBuildSpec { + packageRoot: string + architecture: NativeArchitecture + subprocess: Pick + signal: AbortSignal + timeoutMs: number + graceMs: number +} + +/** + * Locate the platform-specific executable owned by the CUA package. + * @param packageRoot - Directory containing the CUA package manifest. + * @param architecture - Architecture of the process that will execute the helper. + * @returns Absolute executable path. + */ +export function nativeExecutablePath(packageRoot: string, architecture: NativeArchitecture): string { + return join(packageRoot, 'native', 'bin', `darwin-${architecture}`, 'dsh-cua-native') +} + +/** + * Recognize the repository-owned package location before allowing local compilation. + * @param packageRoot - Real directory containing this package. + * @returns True only for a CUA package in a DSH source checkout. + */ +export async function isSourcePackage(packageRoot: string): Promise { + const root = resolve(packageRoot, '../../..') + if (join(root, 'packages/computer-use/cua') !== packageRoot) return false + try { + const manifest: unknown = JSON.parse(await readFile(join(root, 'package.json'), 'utf8')) + return typeof manifest === 'object' && manifest !== null && 'name' in manifest && manifest.name === '@deepseek-ai/dsh-root' + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return false + throw error + } +} + +/** + * Let the native build system check sources and its retained compiler cache, then publish atomically. + * @param spec - Checkout, target, managed process provider and cancellation/deadline limits. + * @returns Executable path after compilation and complete process cleanup. + * @throws When compilation, cancellation, resource cleanup or artifact publication fails. + */ +export async function buildNativeExecutable(spec: NativeBuildSpec): Promise { + spec.signal.throwIfAborted() + const executable = nativeExecutablePath(spec.packageRoot, spec.architecture) + await mkdir(dirname(executable), { recursive: true }) + const signal = AbortSignal.any([spec.signal, AbortSignal.timeout(spec.timeoutMs)]) + return withFileLock(executable, () => compileNative(spec, signal, executable), { waitMs: spec.timeoutMs, signal }) +} + +async function compileNative(spec: NativeBuildSpec, signal: AbortSignal, executable: string): Promise { + const nativeRoot = join(spec.packageRoot, 'native') + const scratch = join(nativeRoot, '.build', `runtime-${spec.architecture}`) + const handle = spec.subprocess.spawn({ + argv: ['/usr/bin/xcrun', 'swift', 'build', '--package-path', nativeRoot, + '--scratch-path', scratch, '--configuration', 'release', '--product', 'dsh-cua-native', + '--arch', spec.architecture === 'x64' ? 'x86_64' : 'arm64'], + cwd: spec.packageRoot, signal, graceMs: spec.graceMs, + stdio: { stdin: 'ignore', stdout: { maxBytes: 65_536 }, stderr: { maxBytes: 65_536 } }, + }) + const outcome = await handle.done.finally(async () => { + try { handle.terminate() } finally { + if (!await handle.waitForExit()) throw new Error('Computer Use preparation process did not exit cleanly') + } + }) + signal.throwIfAborted() + if (outcome.exitCode !== 0 || outcome.signal !== null) { + const diagnostics = `${handle.collected.stdout?.readFrom(0).text ?? ''}${handle.collected.stderr?.readFrom(0).text ?? ''}` + throw new Error(`Computer Use native preparation failed (${String(outcome.exitCode)}, ${String(outcome.signal)}): ${diagnostics}`) + } + const built = await readFile(join(scratch, 'release', 'dsh-cua-native')) + let previous: Buffer | undefined + try { previous = await readFile(executable) } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error + } + signal.throwIfAborted() + if (!previous?.equals(built)) await writeFileAtomic(executable, built, { mode: 0o755 }) + await access(executable, constants.X_OK) + return executable +} + +/** + * Prepare source builds on activation; installed packages must already carry their native payload. + * @param spec - Native preparation inputs for this host. + * @returns Ready native executable without requiring a compiler for an installed package. + */ +export async function prepareNativeExecutable(spec: NativeBuildSpec): Promise { + if (await isSourcePackage(spec.packageRoot)) return buildNativeExecutable(spec) + const executable = nativeExecutablePath(spec.packageRoot, spec.architecture) + await access(executable, constants.X_OK) + spec.signal.throwIfAborted() + return executable +} diff --git a/packages/experimental/cua/src/native-client.ts b/packages/computer-use/cua/src/native-client.ts similarity index 100% rename from packages/experimental/cua/src/native-client.ts rename to packages/computer-use/cua/src/native-client.ts diff --git a/packages/experimental/cua/src/protocol.ts b/packages/computer-use/cua/src/protocol.ts similarity index 100% rename from packages/experimental/cua/src/protocol.ts rename to packages/computer-use/cua/src/protocol.ts diff --git a/packages/experimental/cua/src/types.ts b/packages/computer-use/cua/src/types.ts similarity index 100% rename from packages/experimental/cua/src/types.ts rename to packages/computer-use/cua/src/types.ts diff --git a/packages/experimental/cua/tests/api.spec.ts b/packages/computer-use/cua/tests/api.spec.ts similarity index 100% rename from packages/experimental/cua/tests/api.spec.ts rename to packages/computer-use/cua/tests/api.spec.ts diff --git a/packages/experimental/cua/tests/bridge.spec.ts b/packages/computer-use/cua/tests/bridge.spec.ts similarity index 100% rename from packages/experimental/cua/tests/bridge.spec.ts rename to packages/computer-use/cua/tests/bridge.spec.ts diff --git a/packages/experimental/cua/tests/large-image-ptc.spec.ts b/packages/computer-use/cua/tests/large-image-ptc.spec.ts similarity index 100% rename from packages/experimental/cua/tests/large-image-ptc.spec.ts rename to packages/computer-use/cua/tests/large-image-ptc.spec.ts diff --git a/packages/experimental/cua/tests/loader-composition.spec.ts b/packages/computer-use/cua/tests/loader-composition.spec.ts similarity index 100% rename from packages/experimental/cua/tests/loader-composition.spec.ts rename to packages/computer-use/cua/tests/loader-composition.spec.ts diff --git a/packages/computer-use/cua/tests/native-build.spec.ts b/packages/computer-use/cua/tests/native-build.spec.ts new file mode 100644 index 0000000000..4bedc37c82 --- /dev/null +++ b/packages/computer-use/cua/tests/native-build.spec.ts @@ -0,0 +1,135 @@ +/** Preparation uses the existing compiler cache and publishes only successful build output. */ +import { mkdtemp, mkdir, readFile, rm, stat, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { expect, it, onTestFinished, vi } from 'vitest' +import type { SubprocessHandle, SubprocessOutcome, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import { buildNativeExecutable, isSourcePackage, nativeExecutablePath, nativeHostArchitecture, prepareNativeExecutable } from '../src/native-build.ts' + +async function fixture() { + const root = await mkdtemp(join(tmpdir(), 'cua-native-build-')) + onTestFinished(() => rm(root, { recursive: true, force: true })) + const packageRoot = join(root, 'packages/computer-use/cua') + await mkdir(packageRoot, { recursive: true }) + const controller = new AbortController() + const output = Buffer.from([0xcf, 0xfa, 0xed, 0xfe, 0, 255, 128]) + let outcome: SubprocessOutcome = { exitCode: 0, signal: null } + let beforeExit = async () => {} + const terminate = vi.fn() + const waitForExit = vi.fn(async () => true) + const spawn = vi.fn((request: SubprocessSpawnSpec): SubprocessHandle => { + const scratch = request.argv[request.argv.indexOf('--scratch-path') + 1]! + return { + stdin: undefined, stdout: undefined, stderr: undefined, control: undefined, + collected: { stderr: { readFrom: () => ({ text: 'compiler diagnostic', nextOffset: 19, lossy: false }) } }, + done: (async () => { + await beforeExit() + if (outcome.exitCode === 0) { + await mkdir(join(scratch, 'release'), { recursive: true }) + await writeFile(join(scratch, 'release/dsh-cua-native'), output) + } + return outcome + })(), terminate, waitForExit, + } + }) + const spec = { packageRoot, architecture: 'arm64' as const, subprocess: { spawn }, signal: controller.signal, timeoutMs: 60_000, graceMs: 100 } + return { root, spec, spawn, output, controller, terminate, waitForExit, + setOutcome: (value: SubprocessOutcome) => { outcome = value }, + beforeExit: (callback: () => Promise) => { beforeExit = callback }, + } +} + +it('recognizes only the repository-owned package location as a source build', async () => { + const f = await fixture() + expect(await isSourcePackage(f.spec.packageRoot)).toBe(false) + for (const value of [null, 2, {}, { name: 'another-project' }, { name: '@deepseek-ai/dsh-root' }]) { + await writeFile(join(f.root, 'package.json'), JSON.stringify(value)) + expect(await isSourcePackage(f.spec.packageRoot)).toBe(typeof value === 'object' && value !== null && 'name' in value && value.name === '@deepseek-ai/dsh-root') + } + expect(await isSourcePackage(join(f.root, 'node_modules/@deepseek-ai/dsh-cua'))).toBe(false) + await writeFile(join(f.root, 'package.json'), '{') + await expect(isSourcePackage(f.spec.packageRoot)).rejects.toThrow() +}) + +it.each([['darwin', 'arm64'], ['darwin', 'x64']])('selects the %s/%s executable', (platform, architecture) => { + expect(nativeHostArchitecture(platform, architecture)).toBe(architecture) +}) +it.each([['linux', 'arm64'], ['win32', 'x64'], ['darwin', 'ia32']])('refuses unsupported %s/%s hosts', (platform, architecture) => { + expect(() => nativeHostArchitecture(platform, architecture)).toThrow('requires macOS') +}) + +it('publishes exact bytes and leaves an unchanged executable inode intact', async () => { + const f = await fixture() + const path = await buildNativeExecutable(f.spec) + const first = await stat(path) + expect(await readFile(path)).toEqual(f.output) + await buildNativeExecutable(f.spec) + expect((await stat(path)).ino).toBe(first.ino) + expect(f.spawn).toHaveBeenCalledTimes(2) + expect(f.spawn.mock.calls[0]![0].argv).toContain('arm64') + expect(f.terminate).toHaveBeenCalledTimes(2) + expect(f.waitForExit).toHaveBeenCalledTimes(2) +}) + +it('checks source builds on activation and maps Intel architecture to the compiler target', async () => { + const f = await fixture() + await writeFile(join(f.root, 'package.json'), '{"name":"@deepseek-ai/dsh-root"}') + const path = await prepareNativeExecutable({ ...f.spec, architecture: 'x64' }) + expect(f.spawn.mock.calls[0]![0].argv).toContain('x86_64') + expect(path).toBe(nativeExecutablePath(f.spec.packageRoot, 'x64')) +}) + +it('uses an installed executable without starting a compiler', async () => { + const f = await fixture() + const path = nativeExecutablePath(f.spec.packageRoot, 'arm64') + await mkdir(join(path, '..'), { recursive: true }) + await writeFile(path, f.output, { mode: 0o755 }) + expect(await prepareNativeExecutable(f.spec)).toBe(path) + expect(f.spawn).not.toHaveBeenCalled() +}) + +it('keeps the previous executable when a build fails and reports compiler output', async () => { + const f = await fixture() + const path = await buildNativeExecutable(f.spec) + f.setOutcome({ exitCode: 1, signal: null }) + await expect(buildNativeExecutable(f.spec)).rejects.toThrow('compiler diagnostic') + expect(await readFile(path)).toEqual(f.output) + f.setOutcome({ exitCode: null, signal: 'SIGTERM' }) + await expect(buildNativeExecutable(f.spec)).rejects.toThrow('SIGTERM') +}) + +it('awaits compiler cleanup and refuses to publish after cancellation', async () => { + const f = await fixture() + f.beforeExit(async () => { f.controller.abort() }) + await expect(buildNativeExecutable(f.spec)).rejects.toThrow() + expect(f.waitForExit).toHaveBeenCalledTimes(1) + await expect(readFile(nativeExecutablePath(f.spec.packageRoot, 'arm64'))).rejects.toMatchObject({ code: 'ENOENT' }) + await expect(buildNativeExecutable(f.spec)).rejects.toThrow() + expect(f.spawn).toHaveBeenCalledTimes(1) +}) + +it('reports incomplete compiler process cleanup before publishing', async () => { + const f = await fixture() + f.waitForExit.mockResolvedValue(false) + await expect(buildNativeExecutable(f.spec)).rejects.toThrow('did not exit cleanly') + await expect(readFile(nativeExecutablePath(f.spec.packageRoot, 'arm64'))).rejects.toMatchObject({ code: 'ENOENT' }) +}) + +it('serializes publication and cancels a competing preparation without starting another compiler', async () => { + const f = await fixture() + const release: PromiseWithResolvers = Promise.withResolvers() + const entered: PromiseWithResolvers = Promise.withResolvers() + onTestFinished(() => { release.resolve() }) + f.beforeExit(async () => { entered.resolve(); await release.promise }) + const first = buildNativeExecutable(f.spec) + await entered.promise + const competing = new AbortController() + const second = buildNativeExecutable({ ...f.spec, signal: competing.signal }) + const rejected = expect(second).rejects.toThrow() + competing.abort() + try { + await rejected + expect(f.spawn).toHaveBeenCalledTimes(1) + } finally { release.resolve(); await first } + expect(await readFile(nativeExecutablePath(f.spec.packageRoot, 'arm64'))).toEqual(f.output) +}) diff --git a/packages/experimental/cua/tests/native-client.spec.ts b/packages/computer-use/cua/tests/native-client.spec.ts similarity index 100% rename from packages/experimental/cua/tests/native-client.spec.ts rename to packages/computer-use/cua/tests/native-client.spec.ts diff --git a/packages/experimental/cua/tests/protocol.spec.ts b/packages/computer-use/cua/tests/protocol.spec.ts similarity index 100% rename from packages/experimental/cua/tests/protocol.spec.ts rename to packages/computer-use/cua/tests/protocol.spec.ts diff --git a/packages/experimental/cua/tests/provider-lifecycle.spec.ts b/packages/computer-use/cua/tests/provider-lifecycle.spec.ts similarity index 84% rename from packages/experimental/cua/tests/provider-lifecycle.spec.ts rename to packages/computer-use/cua/tests/provider-lifecycle.spec.ts index 3aeee1b7c6..e57bd0aa3d 100644 --- a/packages/experimental/cua/tests/provider-lifecycle.spec.ts +++ b/packages/computer-use/cua/tests/provider-lifecycle.spec.ts @@ -12,10 +12,18 @@ import { SessionId } from '@deepseek-ai/dsh-session' import Subprocess from '@deepseek-ai/dsh-subprocess' import { expect, it, onTestFinished, vi } from 'vitest' import * as NativeProvider from '../src/index.ts' +import type { NativeBuildSpec } from '../src/native-build.ts' const native = vi.hoisted(() => ({ + prepare: vi.fn<(spec: NativeBuildSpec) => Promise>(), + architecture: vi.fn(() => 'arm64' as const), request: vi.fn(), dispose: vi.fn(), instances: [] as Array<{ disposed: boolean }>, })) +vi.mock('../src/native-build.ts', async importOriginal => ({ + ...await importOriginal(), + nativeHostArchitecture: native.architecture, + prepareNativeExecutable: native.prepare, +})) vi.mock('../src/native-client.ts', async importOriginal => ({ ...await importOriginal(), NativeClient: class { @@ -27,7 +35,7 @@ vi.mock('../src/native-client.ts', async importOriginal => ({ })) const config = { - helperPath: resolve('fixture-helper'), timeoutMs: 10_000, axTimeoutMs: 3000, maxRequestBytes: 1024, maxResponseBytes: 4096, + helperPath: resolve('fixture-helper'), buildTimeoutMs: 300_000, timeoutMs: 10_000, axTimeoutMs: 3000, maxRequestBytes: 1024, maxResponseBytes: 4096, graceMs: 10, textEventIntervalMs: 8, maxElements: 64, maxDepth: 8, dragDurationMs: 200, dragSteps: 10, } satisfies NativeProvider.Config @@ -332,3 +340,57 @@ it.each([ storage.mockRestore() } }) + +it('prepares the selected native component before registering CUA and creates no helper until a tool call', async () => { + const f = await setup() + await f.provider.dispose() + native.prepare.mockReset().mockResolvedValue(resolve('prepared-helper')) + await NativeProvider.apply(f.ctx, { ...config, helperPath: undefined }) + expect(native.prepare).toHaveBeenCalledTimes(1) + expect(native.instances).toHaveLength(0) + await f.execute() + expect(native.instances).toHaveLength(1) + const signal = native.prepare.mock.calls[0]![0].signal + await f.ctx.fiber.dispose() + expect(signal.aborted).toBe(true) +}) + +it('does not register operations after native preparation fails', async () => { + const f = await setup() + await f.provider.dispose() + native.prepare.mockReset().mockRejectedValue(new Error('compiler unavailable')) + await expect(NativeProvider.apply(f.ctx, { ...config, helperPath: undefined })).rejects.toThrow('compiler unavailable') + expect(f.ctx.tools.schemas().some(tool => tool.name === 'cua')).toBe(false) + expect(native.instances).toHaveLength(0) +}) + +it('waits for canceled preparation and never publishes its late success', async () => { + const f = await setup() + await f.provider.dispose() + const ready = Promise.withResolvers() + onTestFinished(() => { ready.resolve(resolve('late-helper')) }) + native.prepare.mockReset().mockReturnValue(ready.promise) + const starting = Promise.resolve(NativeProvider.apply(f.ctx, { ...config, helperPath: undefined })) + const failed = expect(starting).rejects.toThrow() + const signal = native.prepare.mock.calls[0]![0].signal + const aborted = new Promise(resolve => signal.addEventListener('abort', () => resolve(), { once: true })) + let closed = false + const closing = f.ctx.fiber.dispose().then(() => { closed = true }) + try { + await aborted + expect(closed).toBe(false) + } finally { + ready.resolve(resolve('late-helper')) + await Promise.all([closing, failed]) + } + expect(native.instances).toHaveLength(0) +}) + +it('rejects an unsupported native host before starting preparation', async () => { + const f = await setup() + await f.provider.dispose() + native.architecture.mockImplementationOnce(() => { throw new Error('unsupported host') }) + native.prepare.mockReset() + expect(() => NativeProvider.apply(f.ctx, { ...config, helperPath: undefined })).toThrow('unsupported host') + expect(native.prepare).not.toHaveBeenCalled() +}) diff --git a/packages/experimental/cua/tsconfig.json b/packages/computer-use/cua/tsconfig.json similarity index 88% rename from packages/experimental/cua/tsconfig.json rename to packages/computer-use/cua/tsconfig.json index 4d9a6e588f..0f282350b7 100644 --- a/packages/experimental/cua/tsconfig.json +++ b/packages/computer-use/cua/tsconfig.json @@ -15,7 +15,7 @@ "path": "../../../vendor/schemastery" }, { - "path": "../../computer-use/computer-use" + "path": "../computer-use" }, { "path": "../../core/tools" @@ -37,6 +37,9 @@ }, { "path": "../../util/brand" + }, + { + "path": "../../util/atomic-write" } ] } diff --git a/packages/experimental/cua/tsdown.config.ts b/packages/computer-use/cua/tsdown.config.ts similarity index 82% rename from packages/experimental/cua/tsdown.config.ts rename to packages/computer-use/cua/tsdown.config.ts index 6f91243efb..6b5e654d3a 100644 --- a/packages/experimental/cua/tsdown.config.ts +++ b/packages/computer-use/cua/tsdown.config.ts @@ -1,7 +1,7 @@ import { defineConfig } from 'tsdown' export default defineConfig(Object.entries({ - index: 'lib/types/index.js', api: 'lib/types/api.js', bridge: 'lib/types/bridge.js', + index: 'lib/types/index.js', api: 'lib/types/api.js', bridge: 'lib/types/bridge.js', 'native-build': 'lib/types/native-build.js', }).map(([name, path]) => ({ entry: { [name]: path }, outDir: 'lib', format: ['esm'], platform: 'node', target: 'es2024', diff --git a/packages/experimental/README.i18n.yaml b/packages/experimental/README.i18n.yaml index d286bcceab..c3c5329e44 100644 --- a/packages/experimental/README.i18n.yaml +++ b/packages/experimental/README.i18n.yaml @@ -15,8 +15,8 @@ en: 1eb35b015a6c423f zh: a8a3dceee2c9e943 /packages-experimental/packages: - en: 6fa578a62a864262 - zh: 1f8ca5a4250e7b54 + en: ad10945dac9dbb51 + zh: 245b454406ce99f8 /packages-experimental/related-documentation: en: 31282a28dd828c81 zh: ebb1d3cacdc1bd6d diff --git a/packages/experimental/README.md b/packages/experimental/README.md index 428934bd38..a49181f7de 100644 --- a/packages/experimental/README.md +++ b/packages/experimental/README.md @@ -36,7 +36,6 @@ The experimental group contains prototype capabilities whose contracts can chang | [`ptc-runtime-python`](ptc-runtime-python/README.md) | CPython subprocess backend for the PTC execution seam | `ctx.ptcRuntime` | | [`computer-use-cua-driver-mcp`](computer-use-cua-driver-mcp/README.md) | Use an installed Cua Driver through MCP | `ctx.computerUse` | | [`computer-use-cua-driver-native`](computer-use-cua-driver-native/README.md) | Embed the Cua Driver native npm runtime | `ctx.computerUse` | -| [`cua`](cua/README.md) | Codex-style macOS app and window operations | `ctx.computerUse` | | [`browser-use-playwright-mcp`](browser-use-playwright-mcp/README.md) | Playwright browser tools over MCP | `ctx.browserUse` | | [`browser-use-chrome-devtools-mcp`](browser-use-chrome-devtools-mcp/README.md) | Chrome DevTools inspection and browser control over MCP | `ctx.browserUse` | | [`browser-use-stagehand-native`](browser-use-stagehand-native/README.md) | Stagehand browser operations with explicitly configured native models | `ctx.browserUse` | diff --git a/packages/experimental/README.zh.md b/packages/experimental/README.zh.md index edc7000462..68b905673f 100644 --- a/packages/experimental/README.zh.md +++ b/packages/experimental/README.zh.md @@ -36,7 +36,6 @@ kind: "package-group" | [`ptc-runtime-python`](ptc-runtime-python/README.zh.md) | PTC 执行 seam 的 CPython 子进程后端 | `ctx.ptcRuntime` | | [`computer-use-cua-driver-mcp`](computer-use-cua-driver-mcp/README.zh.md) | 通过 MCP 使用已安装的 Cua Driver | `ctx.computerUse` | | [`computer-use-cua-driver-native`](computer-use-cua-driver-native/README.zh.md) | 嵌入 Cua Driver 原生 npm 运行时 | `ctx.computerUse` | -| [`cua`](cua/README.zh.md) | Codex 风格 macOS 应用与窗口操作 | `ctx.computerUse` | | [`browser-use-playwright-mcp`](browser-use-playwright-mcp/README.zh.md) | 通过 MCP 提供 Playwright 浏览器工具 | `ctx.browserUse` | | [`browser-use-chrome-devtools-mcp`](browser-use-chrome-devtools-mcp/README.zh.md) | 通过 MCP 提供 Chrome DevTools 检查与浏览器控制 | `ctx.browserUse` | | [`browser-use-stagehand-native`](browser-use-stagehand-native/README.zh.md) | Stagehand 浏览器操作与显式配置的原生模型 | `ctx.browserUse` | diff --git a/packages/experimental/cua/README.i18n.yaml b/packages/experimental/cua/README.i18n.yaml deleted file mode 100644 index 069b5bee8b..0000000000 --- a/packages/experimental/cua/README.i18n.yaml +++ /dev/null @@ -1,52 +0,0 @@ -# Bilingual-pair consistency record for README.md (docs/i18n/README.md): per heading -# section, a hash of its English and Chinese blocks outside code blocks and generated regions. -# After editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/experimental/cua/README.md -/: - en: a1503331d3f35516 - zh: fcb8e5220ff732e1 -/deepseek-ai-dsh-experimental-cua: - en: fff802c0fccb9353 - zh: 9d5436065b8ca835 -/deepseek-ai-dsh-experimental-cua/summary: - en: 4549f02fc53854a5 - zh: 6f74f1130b6623a8 -/deepseek-ai-dsh-experimental-cua/table-of-contents: - en: d152484eb41ac6b4 - zh: 7b30b973a849247d -/deepseek-ai-dsh-experimental-cua/use-this-package: - en: caff9cfbb38fddc8 - zh: cf470484c6133654 -/deepseek-ai-dsh-experimental-cua/use-this-package/connect-an-external-execution-environment: - en: dc2f1cd930a6b6ba - zh: 12b4f57fa745afbf -/deepseek-ai-dsh-experimental-cua/understand-the-implementation: - en: ee9fd0ddf93d496d - zh: 38d8d5f8b58e6f50 -/deepseek-ai-dsh-experimental-cua/further-exploration: - en: b1047318879cd6f4 - zh: e4ce4a5ad2a9c4dc -/deepseek-ai-dsh-experimental-cua/model-experience: - en: 215e7ba838619b7b - zh: a311da3843709f90 -/deepseek-ai-dsh-experimental-cua/model-experience/native-application-operations: - en: ab7cba4a42334ab9 - zh: f77cd9d84624b6c2 -/deepseek-ai-dsh-experimental-cua/model-experience/native-application-operations/what-the-model-sees: - en: c221fa5ce647b1dd - zh: 9388c91a9f89e029 -/deepseek-ai-dsh-experimental-cua/model-experience/native-application-operations/what-the-model-sees/operation-guidance-verbatim: - en: 2dbc22ffd1f8a717 - zh: d4458872e75305d9 -/deepseek-ai-dsh-experimental-cua/model-experience/native-application-operations/token-effect: - en: ab52b46d874fa8b9 - zh: e87d3620ef77cb06 -/deepseek-ai-dsh-experimental-cua/model-experience/native-application-operations/kv-cache-effect: - en: a2c9acb5a4320dda - zh: e8d1586af60215c4 -/deepseek-ai-dsh-experimental-cua/known-limitations-and-deferred-work: - en: 0fd701796488ac03 - zh: 9ce1ed68bc3cd624 -/deepseek-ai-dsh-experimental-cua/known-limitations-and-deferred-work/dev-note: - en: b836f01bd0bceb4a - zh: 8122fa08b2aa95fa diff --git a/packages/extensions/tool-cordis/src/api-catalog.ts b/packages/extensions/tool-cordis/src/api-catalog.ts index 657135d93f..7404e926b2 100644 --- a/packages/extensions/tool-cordis/src/api-catalog.ts +++ b/packages/extensions/tool-cordis/src/api-catalog.ts @@ -4638,7 +4638,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'BundleInfo', - declaration: 'export interface BundleInfo {\n name: string;\n version?: string;\n meta?: PluginLocalizedMeta;\n description?: string;\n enabled: boolean;\n installed: boolean;\n optional: boolean;\n removable: boolean;\n readOnlyReason?: ReadOnlyReason;\n error?: ManagementError;\n rows: BundleRowInfo[];\n overrides: string[];\n}', + declaration: 'export interface BundleInfo {\n name: string;\n experimental?: boolean;\n version?: string;\n meta?: PluginLocalizedMeta;\n description?: string;\n enabled: boolean;\n installed: boolean;\n optional: boolean;\n removable: boolean;\n readOnlyReason?: ReadOnlyReason;\n error?: ManagementError;\n rows: BundleRowInfo[];\n overrides: string[];\n}', }, { name: 'BundleRowInfo', diff --git a/packages/preset/agent-preset/skills/cordis-composition-reference/references/packages.md b/packages/preset/agent-preset/skills/cordis-composition-reference/references/packages.md index 5172db250e..9a1221c1d2 100644 --- a/packages/preset/agent-preset/skills/cordis-composition-reference/references/packages.md +++ b/packages/preset/agent-preset/skills/cordis-composition-reference/references/packages.md @@ -186,7 +186,7 @@ Every package below exports a Cordis plugin that a bundle patch can name in a Lo | `@deepseek-ai/dsh-experimental-client-ui-voice-input` | no | Record speech and insert editable text into the conversation draft | | `@deepseek-ai/dsh-experimental-computer-use-cua-driver-mcp` | yes | Experimental computer use through an installed Cua Driver MCP executable | | `@deepseek-ai/dsh-experimental-computer-use-cua-driver-native` | no | Experimental computer-use provider embedding the Cua Driver native npm SDK | -| `@deepseek-ai/dsh-experimental-cua` | yes | Experimental Codex-style macOS computer use | +| `@deepseek-ai/dsh-cua` | yes | Experimental Codex-style macOS computer use | | `@deepseek-ai/dsh-experimental-inspector` | yes | Experimental cross-realm CDP hub for Host debugging and Client Runtime inspection | | `@deepseek-ai/dsh-experimental-ptc-runtime-python` | yes | CPython subprocess implementation of the DeepSeek Harness PTC execution seam | | `@deepseek-ai/dsh-experimental-speech-to-text` | yes | Experimental speech recognition with independently selectable providers | diff --git a/packages/util/atomic-write/README.i18n.yaml b/packages/util/atomic-write/README.i18n.yaml index cb293fdbb8..982e689e9f 100644 --- a/packages/util/atomic-write/README.i18n.yaml +++ b/packages/util/atomic-write/README.i18n.yaml @@ -9,14 +9,14 @@ en: ed491d62812ad8cc zh: 653e184dd035ec71 /deepseek-ai-dsh-atomic-write/summary: - en: 6c4c49bda04ed05f - zh: 1080ddd179d0a3d2 + en: d2d40e759179f047 + zh: c53f56de74b26e5a /deepseek-ai-dsh-atomic-write/table-of-contents: en: d152484eb41ac6b4 zh: 09388d293f9be9cb /deepseek-ai-dsh-atomic-write/use-this-package: - en: 9cc27a0ee5ae0808 - zh: d0ee06b54452f600 + en: 30fbc2c61904195d + zh: 09c2ac3f44aa3f08 /deepseek-ai-dsh-atomic-write/use-this-package/writing-a-file-atomically: en: 7b757dc3c3b27335 zh: a097d90a2561475d @@ -24,8 +24,8 @@ en: 77c865c1f78d2a0c zh: 5a91874c352c6f18 /deepseek-ai-dsh-atomic-write/use-this-package/failures-to-plan-for: - en: 59308dc757e25273 - zh: 01113ed577a413cb + en: a8292aa69d7f909c + zh: 60d8403fa6e04d17 /deepseek-ai-dsh-atomic-write/understand-the-implementation: en: 93b537308f7c88c7 zh: 282e0d2042aeb1be @@ -48,8 +48,8 @@ en: 815e93723a98919c zh: 921da870f67e4033 /deepseek-ai-dsh-atomic-write/known-limitations-and-deferred-work: - en: 5d0b395da8833bbb - zh: 6191bab46d288be7 + en: fa413fe1ee23e01f + zh: 8563b57f91ea194b /deepseek-ai-dsh-atomic-write/known-limitations-and-deferred-work/dev-note: en: d3461ceeaf041974 zh: f59df2adbf52fed9 diff --git a/packages/util/atomic-write/README.md b/packages/util/atomic-write/README.md index bddff7852e..3be8910116 100644 --- a/packages/util/atomic-write/README.md +++ b/packages/util/atomic-write/README.md @@ -9,7 +9,7 @@ English | [中文](README.zh.md) ## Summary -Use `dsh-atomic-write` to replace a file without exposing partial content or following a symlinked temporary path. Its writer lock serializes read-modify-write cycles across processes so concurrent writers cannot overwrite one another with stale state. Each replacement uses caller-selected permission bits on a fresh inode, which safely narrows an existing file's permissions. This zero-dependency library accepts strings; it does not provide a `cordis.yml` plugin or crash durability because it does not call `fsync`. +Use `dsh-atomic-write` to replace a file without exposing partial content or following a symlinked temporary path. Its writer lock serializes read-modify-write cycles across processes so concurrent writers cannot overwrite one another with stale state. Each replacement uses caller-selected permission bits on a fresh inode, which safely narrows an existing file's permissions. This zero-dependency library accepts strings and binary byte arrays; it does not provide a `cordis.yml` plugin or crash durability because it does not call `fsync`. ## Table of Contents @@ -25,7 +25,7 @@ Use `dsh-atomic-write` to replace a file without exposing partial content or fol ## Use this package -Use `writeFileAtomic` when a file-backed store must replace one already-rendered string without ever exposing a partial, symlink-hijacked, or wider-permission state, and `withFileLock` when several processes read-modify-write the same file. The smallest path is one call with the final content and the replacement's permission bits. +Use `writeFileAtomic` when a file-backed store must replace complete text or binary content without ever exposing a partial, symlink-hijacked, or wider-permission state, and `withFileLock` when several processes read-modify-write the same file. The smallest path is one call with the final content and the replacement's permission bits. ### Writing a file atomically @@ -60,7 +60,7 @@ Only writers contend — readers never take the lock — and a contender backs o Windows retries one `EPERM` when the lock cannot be observed, because its holder can release between exclusive creation and the existence check. A repeated unconfirmed `EPERM` is rethrown without running the operation. -The lock's parent directory must already exist, so `withFileLock` rejects an invalid parent hierarchy before running the operation. A process that exits while holding the lock leaves the lock sibling behind, and the next writer takes it over. A lock whose record is empty or incomplete, or names a PID that a live process reused, is not taken over; later writers time out, and an operator removes it only after verifying that no writer still owns it. +The lock's parent directory must already exist, so `withFileLock` rejects an invalid parent hierarchy before running the operation. A process that exits while holding the lock leaves the lock sibling behind, and the next writer takes it over. A lock whose record is empty or incomplete, or names a PID that a live process reused, is not taken over; later writers time out, and an operator removes it only after verifying that no writer still owns it. An optional `signal` cancels acquisition and contention waits; the operation owns cancellation after it starts. ----- @@ -123,7 +123,7 @@ Nothing here enters a request prefix, so provider cache reuse is unaffected. These limits define where the package is not the right tool. They are current package constraints, not a task backlog. - **Atomic, not durable** — no `fsync` of the file or its directory, so after a crash the rename may be observed unwound. The file-backed stores here re-read and republish on boot, keeping durability the caller's policy. -- **String content only** — no `Buffer` or stream form until a consumer needs one. +- **Callers supply complete content** — strings and `Uint8Array` (including `Buffer`) are accepted; streaming writes are unsupported. - **Some orphaned locks require operator recovery** — a lock whose record is empty or incomplete, or names a PID a live process reused, stays in place; later writers time out without deleting it. A contender that ends between creating a claim and removing the lock leaves both `.lock` and `.lock.takeover-`, and the operator removes both. - **One host and one PID namespace** — the probe runs on the contender's host. Writers on several hosts sharing a network filesystem, or containers sharing a volume, can see a live holder as exited, take over its lock, and write at the same time as it. diff --git a/packages/util/atomic-write/README.zh.md b/packages/util/atomic-write/README.zh.md index ef60663f84..f416dc0dab 100644 --- a/packages/util/atomic-write/README.zh.md +++ b/packages/util/atomic-write/README.zh.md @@ -9,7 +9,7 @@ kind: "package-library" ## 概述 -使用 `dsh-atomic-write` 替换文件时，不会暴露部分内容，也不会跟随临时路径上的符号链接。它的写锁会跨进程串行化读-修改-写入循环，因此并发写入方不会用陈旧状态相互覆盖。每次替换都会在全新 inode 上使用调用方选择的权限位，从而安全地收窄现有文件的权限。这个零依赖库只接受字符串；它不提供 `cordis.yml` 插件，也不保证崩溃持久性，因为它不调用 `fsync`。 +使用 `dsh-atomic-write` 替换文件时，不会暴露部分内容，也不会跟随临时路径上的符号链接。它的写锁会跨进程串行化读-修改-写入循环，因此并发写入方不会用陈旧状态相互覆盖。每次替换都会在全新 inode 上使用调用方选择的权限位，从而安全地收窄现有文件的权限。这个零依赖库接受字符串和二进制字节数组；它不提供 `cordis.yml` 插件，也不保证崩溃持久性，因为它不调用 `fsync`。 ## 目录 @@ -25,7 +25,7 @@ kind: "package-library" ## 使用本包 -当文件型存储必须替换一份已渲染好的字符串、且绝不允许暴露部分写入、符号链接劫持或权限过宽状态时，使用 `writeFileAtomic`；当多个进程对同一文件执行读-修改-写入循环时，使用 `withFileLock`。最小路径是一次调用，传入最终内容与替换 inode 的权限位。 +当文件型存储必须替换完整文本或二进制内容、且绝不允许暴露部分写入、符号链接劫持或权限过宽状态时，使用 `writeFileAtomic`；当多个进程对同一文件执行读-修改-写入循环时，使用 `withFileLock`。最小路径是一次调用，传入最终内容与替换 inode 的权限位。 ### 原子写入文件 @@ -60,7 +60,7 @@ await withFileLock('/home/u/.dsh/cordis.patch.yml', async () => { Windows 在无法观察到锁时会对 `EPERM` 重试一次，因为持锁方可能在独占创建与存在性检查之间释放锁。再次出现无法确认锁存在的 `EPERM` 时，会重新抛出错误且不运行操作。 -锁的父目录必须已经存在，因此 `withFileLock` 会在运行操作之前拒绝无效的父目录层级。持锁进程退出时会把锁文件留在原地，下一个写入方会接管它。锁记录为空或不完整，或其 PID 已被存活进程复用时，锁不会被接管；后续写入方超时失败，操作者只有在确认没有写入方仍持有该锁后才会移除它。 +锁的父目录必须已经存在，因此 `withFileLock` 会在运行操作之前拒绝无效的父目录层级。持锁进程退出时会把锁文件留在原地，下一个写入方会接管它。锁记录为空或不完整，或其 PID 已被存活进程复用时，锁不会被接管；后续写入方超时失败，操作者只有在确认没有写入方仍持有该锁后才会移除它。 `signal` 可取消获取锁和等待竞争；操作开始后由操作自身处理取消。 ----- @@ -123,7 +123,7 @@ Windows 在无法观察到锁时会对 `EPERM` 重试一次，因为持锁方可 这些限制说明本包何时不是合适的工具。它们是当前包约束，不是任务积压。 - **原子但不保证持久**——不对文件或其所在目录做 `fsync`，因此崩溃后可能观察到 rename 被回退。此处的文件型存储在启动时重新读取并重新发布，把持久性留作调用方的策略。 -- **仅支持字符串内容**——在有消费方需要之前，不提供 `Buffer` 或流式形态。 +- **调用方提供完整内容**——接受字符串或 `Uint8Array`（包括 `Buffer`）；不支持流式写入。 - **部分遗留锁需要操作者恢复**——锁记录为空或不完整，或其 PID 已被存活进程复用时，锁会留在原地；后续写入方超时也不会删除它。竞争者若在创建认领文件与移除锁之间结束，会同时留下 `.lock` 和 `.lock.takeover-<记录哈希>`，由操作者删除两者。 - **单一主机与单一 PID 命名空间**——探测在竞争者所在主机上进行。多台主机通过网络文件系统共享、或多个容器共享同一卷时，写入方可能把存活持有者视为已退出，接管其锁并与之同时写入。 diff --git a/packages/util/atomic-write/src/index.ts b/packages/util/atomic-write/src/index.ts index c567a2b2fc..60e2c67b8c 100644 --- a/packages/util/atomic-write/src/index.ts +++ b/packages/util/atomic-write/src/index.ts @@ -14,6 +14,7 @@ import { createHash, randomBytes } from 'node:crypto' import { lstat, mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises' import { dirname } from 'node:path' +import { setTimeout as wait } from 'node:timers/promises' const WINDOWS_TRANSIENT_RENAME_ERRORS: ReadonlySet = new Set(['EACCES', 'EBUSY', 'EPERM']) const WINDOWS_RENAME_RETRY_INITIAL_MS = 20 @@ -76,7 +77,7 @@ export interface WriteFileAtomicOptions { * @param content - complete next file content. * @param options - permission bits for the replacement inode. */ -export async function writeFileAtomic(filename: string, content: string, options: WriteFileAtomicOptions): Promise { +export async function writeFileAtomic(filename: string, content: string | Uint8Array, options: WriteFileAtomicOptions): Promise { await mkdir(dirname(filename), { recursive: true, ...options.dirMode === undefined ? {} : { mode: options.dirMode }, @@ -197,6 +198,8 @@ const DEFAULT_LOCK_WAIT_MS = 2_000 /** Options for one {@link withFileLock} acquisition. */ export interface FileLockOptions { + /** Cancel acquisition and contention waits; the operation owns cancellation after it starts. */ + signal?: AbortSignal /** * Maximum time to wait for the lock, in milliseconds. State one when the * holder's operation legitimately runs longer than file work — a credential @@ -242,6 +245,7 @@ export async function withFileLock( let delay = LOCK_RETRY_INITIAL_MS let retriedUnconfirmedPermissionError = false for (;;) { + options?.signal?.throwIfAborted() try { await writeFile(lockPath, `${process.pid}\n`, { mode: 0o600, flag: 'wx' }) break @@ -257,10 +261,11 @@ export async function withFileLock( if (Date.now() >= deadline) { throw new Error(`atomic-write: timed out waiting for the writer lock at ${lockPath}`) } - await new Promise(resolve => setTimeout(resolve, delay)) + await wait(delay, undefined, { signal: options?.signal }) delay = Math.min(delay * 2, LOCK_RETRY_MAX_MS) } try { + options?.signal?.throwIfAborted() return await operation() } finally { await rm(lockPath, { force: true }) diff --git a/packages/util/atomic-write/tests/atomic-write.spec.ts b/packages/util/atomic-write/tests/atomic-write.spec.ts index 0f97822753..13cfd18a7d 100644 --- a/packages/util/atomic-write/tests/atomic-write.spec.ts +++ b/packages/util/atomic-write/tests/atomic-write.spec.ts @@ -122,6 +122,13 @@ async function waitForLock(lockPath: string): Promise { } describe('writeFileAtomic', () => { + it('preserves non-UTF-8 binary content during atomic replacement', async () => { + const path = join(await scratch(), 'native-helper') + const bytes = new Uint8Array([0, 255, 128, 0xcf, 0xfa, 0xed, 0xfe]) + await writeFileAtomic(path, bytes, { mode: 0o700 }) + expect(await readFile(path)).toEqual(Buffer.from(bytes)) + }) + it('creates the file and its parents with exactly the stated mode', async () => { const dir = await scratch() const target = join(dir, 'nested', 'deep', 'doc.yaml') @@ -213,6 +220,26 @@ describe('writeFileAtomic', () => { }) describe('withFileLock', () => { + it('cancels a waiting writer without releasing the current holder', async () => { + const target = join(await scratch(), 'document') + const release: PromiseWithResolvers = Promise.withResolvers() + const entered: PromiseWithResolvers = Promise.withResolvers() + const holder = withFileLock(target, async () => { entered.resolve(); await release.promise }) + await entered.promise + const controller = new AbortController() + const operation = vi.fn(async () => {}) + try { + const waiter = withFileLock(target, operation, { waitMs: 10_000, signal: controller.signal }) + const rejected = expect(waiter).rejects.toThrow() + controller.abort() + await rejected + expect(operation).not.toHaveBeenCalled() + expect(await readFile(`${target}.lock`, 'utf8')).toBe(record(process.pid)) + await expect(withFileLock(target, operation, { signal: controller.signal })).rejects.toThrow() + } finally { release.resolve(); await holder } + await expect(stat(`${target}.lock`)).rejects.toMatchObject({ code: 'ENOENT' }) + }) + it('retries EPERM only when the lock path currently exists', async () => { const dir = await scratch() const target = join(dir, 'document') diff --git a/packages/util/package-manifest/README.i18n.yaml b/packages/util/package-manifest/README.i18n.yaml index 41446bb257..d4a4275ccc 100644 --- a/packages/util/package-manifest/README.i18n.yaml +++ b/packages/util/package-manifest/README.i18n.yaml @@ -15,8 +15,8 @@ en: d152484eb41ac6b4 zh: 1db86285abf6c499 /deepseek-ai-dsh-package-manifest/use-this-package: - en: 43fb882676bf22dd - zh: 04e512bc4059e5d4 + en: f5409ff623381ff1 + zh: c7d1a7c52c02bd10 /deepseek-ai-dsh-package-manifest/understand-the-implementation: en: 64e1e57d1c3761e0 zh: 9fc039770ce43c63 diff --git a/packages/util/package-manifest/README.md b/packages/util/package-manifest/README.md index 7c4c493541..d6fa3f49e1 100644 --- a/packages/util/package-manifest/README.md +++ b/packages/util/package-manifest/README.md @@ -50,6 +50,7 @@ The following metadata fields are optional. Omitting them leaves the format vers | Field | Meaning | |---|---| | `dsh.manifestVersion` | Manifest format identifier; the declared format is `1`, independent of the npm package version and Session format version. | +| `dsh.bundle.experimental` | Explicit experimental feature label in plugin management; independent of the package name and enabled state. | | `engines.dsh` | Author-declared compatible DSH versions as a SemVer range, including exact prerelease versions. This field sits beside `engines.node` and `engines.npm`; an engines object may omit `dsh`. | `LocalizedText` carries literal text or a language map with a required English fallback. `PluginLocalizedMeta` carries optional display title, description, an image data URL resolved from `package.json.icon`, and metadata diagnostics for installed plugins. [App boot](../../boot/app-boot/README.md) reads these values; this package only supplies their types. diff --git a/packages/util/package-manifest/README.zh.md b/packages/util/package-manifest/README.zh.md index 58a150ae32..d8ad41c960 100644 --- a/packages/util/package-manifest/README.zh.md +++ b/packages/util/package-manifest/README.zh.md @@ -50,6 +50,7 @@ const manifest: DshPackageManifest = { | 字段 | 含义 | |---|---| | `dsh.manifestVersion` | manifest（元数据清单）格式标识；声明的格式为 `1`，独立于 npm 包版本和 Session 格式版本。 | +| `dsh.bundle.experimental` | 在插件页显示实验性标签，不改变包名或启用状态。 | | `engines.dsh` | 作者声明的兼容 DSH 版本，使用 SemVer 范围，也可填写精确的预发布版本。此字段与 `engines.node`、`engines.npm` 并列；engines 对象可省略 `dsh`。 | `LocalizedText` 携带字面文本或带必需英文回退值的语言映射。`PluginLocalizedMeta` 携带已安装插件的可选展示标题、描述、从 `package.json.icon` 解析的图片 data URL，以及元信息诊断。[App boot](../../boot/app-boot/README.zh.md) 读取这些值；本包只提供其类型。 diff --git a/packages/util/package-manifest/src/types.ts b/packages/util/package-manifest/src/types.ts index cda212f5a8..974cad9dac 100644 --- a/packages/util/package-manifest/src/types.ts +++ b/packages/util/package-manifest/src/types.ts @@ -67,6 +67,8 @@ export interface DshEnginesManifest { /** The configuration layer exported by a bundle package. */ export interface DshBundleManifest { + /** Mark this optional feature as experimental in plugin management. */ + experimental?: boolean /** One patch file path, or an ordered list applied in sequence, each relative to the declaring package root. */ patch: string | string[] } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 8d07e5fba4..4d0afcc962 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -31,6 +31,12 @@ importers: '@deepseek-ai/dsh-computer-use': specifier: workspace:* version: link:packages/computer-use/computer-use + '@deepseek-ai/dsh-cua': + specifier: workspace:* + version: link:packages/computer-use/cua + '@deepseek-ai/dsh-cua-bundle': + specifier: workspace:* + version: link:packages/computer-use/cua-bundle '@deepseek-ai/dsh-experimental-browser-use-chrome-devtools-mcp': specifier: workspace:* version: link:packages/experimental/browser-use-chrome-devtools-mcp @@ -49,9 +55,6 @@ importers: '@deepseek-ai/dsh-experimental-computer-use-cua-driver-native': specifier: workspace:* version: link:packages/experimental/computer-use-cua-driver-native - '@deepseek-ai/dsh-experimental-cua': - specifier: workspace:* - version: link:packages/experimental/cua '@deepseek-ai/dsh-llm-replay': specifier: workspace:* version: link:packages/test-support/llm-replay @@ -244,6 +247,9 @@ importers: '@deepseek-ai/dsh-cordis-client-runner': specifier: workspace:* version: link:../../packages/extensions/cordis-client-runner + '@deepseek-ai/dsh-cua-bundle': + specifier: workspace:* + version: link:../../packages/computer-use/cua-bundle '@deepseek-ai/dsh-experimental-agent-team-profile': specifier: workspace:* version: link:../../packages/experimental/agent-team-profile @@ -6351,6 +6357,116 @@ importers: specifier: workspace:* version: link:../../util/brand + packages/computer-use/cua: + dependencies: + '@deepseek-ai/dsh-atomic-write': + specifier: workspace:* + version: link:../../util/atomic-write + '@deepseek-ai/dsh-brand': + specifier: workspace:* + version: link:../../util/brand + '@deepseek-ai/dsh-mcp-client': + specifier: workspace:* + version: link:../../mcp/mcp-client + '@deepseek-ai/dsh-util-values': + specifier: workspace:* + version: link:../../util/values + '@deepseek-ai/schemastery': + specifier: link:../../../vendor/schemastery + version: link:../../../vendor/schemastery + zod: + specifier: ^4.4.3 + version: 4.4.3 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:~ + version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:~ + version: link:../../../vendor/include + '@deepseek-ai/cordis-plugin-loader': + specifier: workspace:~ + version: link:../../../vendor/loader + '@deepseek-ai/dsh-agent': + specifier: workspace:* + version: link:../../core/agent + '@deepseek-ai/dsh-agent-loop': + specifier: workspace:* + version: link:../../core/agent-loop + '@deepseek-ai/dsh-agent-loop-testkit': + specifier: workspace:* + version: link:../../test-support/agent-loop-testkit + '@deepseek-ai/dsh-attachment': + specifier: workspace:* + version: link:../../attachment/attachment + '@deepseek-ai/dsh-attachment-local': + specifier: workspace:* + version: link:../../attachment/attachment-local + '@deepseek-ai/dsh-computer-use': + specifier: workspace:* + version: link:../computer-use + '@deepseek-ai/dsh-fs-local': + specifier: workspace:* + version: link:../../fs/fs-local + '@deepseek-ai/dsh-llm': + specifier: workspace:* + version: link:../../llm/llm + '@deepseek-ai/dsh-llm-deepseek': + specifier: workspace:* + version: link:../../llm/llm-deepseek + '@deepseek-ai/dsh-ptc-runtime-node': + specifier: workspace:* + version: link:../../ptc-runtime/ptc-runtime-node + '@deepseek-ai/dsh-sandbox-local': + specifier: workspace:* + version: link:../../sandbox/sandbox-local + '@deepseek-ai/dsh-sandbox-policy': + specifier: workspace:* + version: link:../../sandbox/sandbox-policy + '@deepseek-ai/dsh-session': + specifier: workspace:* + version: link:../../core/session + '@deepseek-ai/dsh-session-projection': + specifier: workspace:* + version: link:../../session/session-projection + '@deepseek-ai/dsh-subprocess': + specifier: workspace:* + version: link:../../subprocess/subprocess + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:* + version: link:../../subprocess/subprocess-local + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:* + version: link:../../core/system-prompt + '@deepseek-ai/dsh-tools': + specifier: workspace:* + version: link:../../core/tools + sharp: + specifier: ^0.35.3 + version: 0.35.3(@types/node@22.20.0) + + packages/computer-use/cua-bundle: + dependencies: + '@deepseek-ai/dsh-computer-use': + specifier: workspace:* + version: link:../computer-use + '@deepseek-ai/dsh-cua': + specifier: workspace:* + version: link:../cua + '@deepseek-ai/dsh-ptc-runtime-node': + specifier: workspace:* + version: link:../../ptc-runtime/ptc-runtime-node + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:~ + version: link:../../../vendor/cordis + '@deepseek-ai/cordis-plugin-include': + specifier: workspace:~ + version: link:../../../vendor/include + js-yaml: + specifier: ^4.2.0 + version: 4.3.1 + packages/context/agent-instructions: dependencies: '@deepseek-ai/dsh-util-values': @@ -7710,91 +7826,6 @@ importers: specifier: workspace:* version: link:../../core/tools - packages/experimental/cua: - dependencies: - '@deepseek-ai/dsh-brand': - specifier: workspace:* - version: link:../../util/brand - '@deepseek-ai/dsh-mcp-client': - specifier: workspace:* - version: link:../../mcp/mcp-client - '@deepseek-ai/dsh-util-values': - specifier: workspace:* - version: link:../../util/values - '@deepseek-ai/schemastery': - specifier: link:../../../vendor/schemastery - version: link:../../../vendor/schemastery - zod: - specifier: ^4.4.3 - version: 4.4.3 - devDependencies: - '@deepseek-ai/cordis': - specifier: workspace:~ - version: link:../../../vendor/cordis - '@deepseek-ai/cordis-plugin-include': - specifier: workspace:~ - version: link:../../../vendor/include - '@deepseek-ai/cordis-plugin-loader': - specifier: workspace:~ - version: link:../../../vendor/loader - '@deepseek-ai/dsh-agent': - specifier: workspace:* - version: link:../../core/agent - '@deepseek-ai/dsh-agent-loop': - specifier: workspace:* - version: link:../../core/agent-loop - '@deepseek-ai/dsh-agent-loop-testkit': - specifier: workspace:* - version: link:../../test-support/agent-loop-testkit - '@deepseek-ai/dsh-attachment': - specifier: workspace:* - version: link:../../attachment/attachment - '@deepseek-ai/dsh-attachment-local': - specifier: workspace:* - version: link:../../attachment/attachment-local - '@deepseek-ai/dsh-computer-use': - specifier: workspace:* - version: link:../../computer-use/computer-use - '@deepseek-ai/dsh-fs-local': - specifier: workspace:* - version: link:../../fs/fs-local - '@deepseek-ai/dsh-llm': - specifier: workspace:* - version: link:../../llm/llm - '@deepseek-ai/dsh-llm-deepseek': - specifier: workspace:* - version: link:../../llm/llm-deepseek - '@deepseek-ai/dsh-ptc-runtime-node': - specifier: workspace:* - version: link:../../ptc-runtime/ptc-runtime-node - '@deepseek-ai/dsh-sandbox-local': - specifier: workspace:* - version: link:../../sandbox/sandbox-local - '@deepseek-ai/dsh-sandbox-policy': - specifier: workspace:* - version: link:../../sandbox/sandbox-policy - '@deepseek-ai/dsh-session': - specifier: workspace:* - version: link:../../core/session - '@deepseek-ai/dsh-session-projection': - specifier: workspace:* - version: link:../../session/session-projection - '@deepseek-ai/dsh-subprocess': - specifier: workspace:* - version: link:../../subprocess/subprocess - '@deepseek-ai/dsh-subprocess-local': - specifier: workspace:* - version: link:../../subprocess/subprocess-local - '@deepseek-ai/dsh-system-prompt': - specifier: workspace:* - version: link:../../core/system-prompt - '@deepseek-ai/dsh-tools': - specifier: workspace:* - version: link:../../core/tools - sharp: - specifier: ^0.35.3 - version: 0.35.3(@types/node@22.20.0) - packages/experimental/inspector: dependencies: '@deepseek-ai/dsh-app-boot': diff --git a/scripts/build-cua.spec.ts b/scripts/build-cua.spec.ts new file mode 100644 index 0000000000..42e385c813 --- /dev/null +++ b/scripts/build-cua.spec.ts @@ -0,0 +1,31 @@ +/** Native preparation follows the same persisted bundle selection as the Web switch. */ +import { mkdtemp, mkdir, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { expect, it, onTestFinished } from 'vitest' +import { computerUseSelected } from './build-cua.ts' + +it('does not create or select a profile when Computer Use has not been enabled', async () => { + const home = await mkdtemp(join(tmpdir(), 'cua-selection-')) + onTestFinished(() => rm(home, { recursive: true, force: true })) + expect(computerUseSelected(home, 'web')).toBe(false) + const directory = join(home, 'profiles/web') + await mkdir(directory, { recursive: true }) + await writeFile(join(directory, 'package.json'), '{}') + expect(computerUseSelected(home, 'web')).toBe(false) +}) + +it('reads enable and disable changes from the existing profile manifest', async () => { + const home = await mkdtemp(join(tmpdir(), 'cua-selection-')) + onTestFinished(() => rm(home, { recursive: true, force: true })) + const directory = join(home, 'profiles/web') + await mkdir(directory, { recursive: true }) + for (const enabled of [false, true, false]) { + const bundles = ['@deepseek-ai/dsh-base', ...(enabled ? ['@deepseek-ai/dsh-cua-bundle'] : [])] + await writeFile(join(directory, 'package.json'), JSON.stringify({ dsh: { profile: { bundles } } })) + expect(computerUseSelected(home, 'web')).toBe(enabled) + expect(computerUseSelected(home, 'another-profile')).toBe(false) + } + await writeFile(join(directory, 'package.json'), JSON.stringify({ dsh: { profile: { bundles: 'cua' } } })) + expect(() => computerUseSelected(home, 'web')).toThrow('string array') +}) diff --git a/scripts/build-cua.ts b/scripts/build-cua.ts new file mode 100644 index 0000000000..79afbdd722 --- /dev/null +++ b/scripts/build-cua.ts @@ -0,0 +1,57 @@ +/** Prepare native components only when selected by the existing Web profile. */ +import { existsSync } from 'node:fs' +import { join, resolve } from 'node:path' +import { parseArgs } from 'node:util' +import { Context } from '@deepseek-ai/cordis' +import { readProfileManifest } from '@deepseek-ai/dsh-app-boot' +import { resolveDshHome } from '@deepseek-ai/dsh-home-paths' +import Subprocess from '@deepseek-ai/dsh-subprocess-local' +import { buildNativeExecutable } from '../packages/computer-use/cua/src/native-build.ts' + +/** + * Read the existing plugin selection without creating or mutating a profile. + * @param home - Harness home used by the application launch. + * @param profile - Profile whose optional bundle selection controls preparation. + * @returns Whether Computer Use is selected in that profile. + */ +export function computerUseSelected(home: string, profile: string): boolean { + const directory = join(home, 'profiles', profile) + if (!existsSync(join(directory, 'package.json'))) return false + const manifest = readProfileManifest('build:cua', directory) + const bundles = manifest.dsh?.profile?.bundles + if (bundles === undefined) return false + if (!Array.isArray(bundles) || !bundles.every(name => typeof name === 'string')) throw new Error('build:cua: profile bundles must be a string array') + return bundles.includes('@deepseek-ai/dsh-cua-bundle') +} + +async function main(): Promise { + const { values } = parseArgs({ options: { + force: { type: 'boolean', default: false }, profile: { type: 'string', default: 'web' }, + arch: { type: 'string' }, + }, allowPositionals: false }) + const selected = values.force || computerUseSelected(resolveDshHome(), values.profile) + if (!selected || process.platform !== 'darwin') return + const architecture = values.arch ?? process.arch + if (architecture !== 'arm64' && architecture !== 'x64') throw new Error(`build:cua: unsupported architecture ${architecture}`) + const ctx = new Context() + const controller = new AbortController() + const stop = () => { controller.abort() } + process.on('SIGINT', stop) + process.on('SIGTERM', stop) + try { + await ctx.plugin(Subprocess) + const helper = await buildNativeExecutable({ + packageRoot: resolve(import.meta.dirname, '../packages/computer-use/cua'), + architecture, subprocess: ctx.subprocess, signal: controller.signal, + timeoutMs: 600_000, graceMs: 3000, + }) + console.log(`build:cua: ready ${helper}`) + } finally { + try { await ctx.fiber.dispose() } finally { + process.removeListener('SIGINT', stop) + process.removeListener('SIGTERM', stop) + } + } +} + +if (import.meta.main) await main() diff --git a/scripts/build.ts b/scripts/build.ts index e64fd1486e..8350286cb6 100644 --- a/scripts/build.ts +++ b/scripts/build.ts @@ -43,6 +43,7 @@ function main(): void { rmSync(resolve(root, CLIENT_BUILD_RECORD_PATH), { force: true }) runScript('build:native-system', buildEnvironment) runScript('build:lib', buildEnvironment) + runScript('build:cua', buildEnvironment) runScript('build:web', buildEnvironment) const record = writeClientBuildRecord(root, clientEnvironment) console.log( diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index d560c66c30..2199634946 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -175,7 +175,7 @@ export function readWorkspaceManifests(repositoryRoot: string): WorkspaceManifes const packageFileExtras: Readonly> = { // CUA publishes its client API and attributed native source for local macOS builds. - '@deepseek-ai/dsh-experimental-cua': ['lib/api.js', 'lib/bridge.js', 'native/Package.swift', 'native/Sources', 'native/tests/swift', 'native/tests/test_swift.py', 'native/LICENSE.cua-driver', 'native/upstream.json', 'native/README.txt', 'native/protocol.schema.json'], + '@deepseek-ai/dsh-cua': ['lib/native-build.js', 'native/bin', 'lib/api.js', 'lib/bridge.js', 'native/Package.swift', 'native/Sources', 'native/tests/swift', 'native/tests/test_swift.py', 'native/LICENSE.cua-driver', 'native/upstream.json', 'native/README.txt', 'native/protocol.schema.json'], // Owned Worker bundles import this public bootstrap before their business entry. '@deepseek-ai/dsh-app-boot': ['lib/worker/profile-resolution-bootstrap.js'], // Statically linked client libraries keep their stylesheets next to the emitted diff --git a/scripts/release/families.spec.ts b/scripts/release/families.spec.ts index cf97f5dd4e..a54d7c3fb2 100644 --- a/scripts/release/families.spec.ts +++ b/scripts/release/families.spec.ts @@ -60,7 +60,6 @@ describe('release families', () => { '@deepseek-ai/dsh-experimental-client-ui-voice-input', '@deepseek-ai/dsh-experimental-computer-use-cua-driver-mcp', '@deepseek-ai/dsh-experimental-computer-use-cua-driver-native', - '@deepseek-ai/dsh-experimental-cua', '@deepseek-ai/dsh-experimental-inspector', '@deepseek-ai/dsh-experimental-ptc-runtime-python', '@deepseek-ai/dsh-experimental-speech-to-text-sensevoice', diff --git a/snapshots/session/cua-native/cordis.snapshot.yml b/snapshots/session/cua-native/cordis.snapshot.yml index ed1638a2fa..5a4a3357bf 100644 --- a/snapshots/session/cua-native/cordis.snapshot.yml +++ b/snapshots/session/cua-native/cordis.snapshot.yml @@ -48,6 +48,6 @@ - id: computer-use name: '@deepseek-ai/dsh-computer-use' - id: cua-native - name: '@deepseek-ai/dsh-experimental-cua' + name: '@deepseek-ai/dsh-cua' config: helperPath: !!js process.cwd() + '/native-helper.mjs' diff --git a/snapshots/session/cua-native/cordis.yml b/snapshots/session/cua-native/cordis.yml index 3873713667..bea3c3d679 100644 --- a/snapshots/session/cua-native/cordis.yml +++ b/snapshots/session/cua-native/cordis.yml @@ -30,6 +30,6 @@ - id: computer-use name: '@deepseek-ai/dsh-computer-use' - id: cua-native - name: '@deepseek-ai/dsh-experimental-cua' + name: '@deepseek-ai/dsh-cua' config: helperPath: !!js process.cwd() + '/native-helper.mjs' diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index 3e4f66d84f..1c67ff66be 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -59,4 +59,4 @@ Raw operation tools use { method, params }, including params: {} for parameterle Await operations sequentially. After input, read fresh state to verify the effect. A successful action means dispatched, not verified completion. Cancellation may leave partial effects; never retry uncertain input without observing. STALE_APP requires getApp again; STALE_SNAPSHOT requires getState again. WINDOW_CHANGED requires fresh state before any retry; use an advertised AX action or explicitly restore the window before screenshot or coordinate input, then observe again. PERMISSION_DENIED requires the user to grant macOS Accessibility or Screen Recording permission. -Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-experimental-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value. +Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste inserts plain text using native Unicode input without changing the clipboard; rich-text formats are unavailable. Browser applications can be controlled with getApp like other native apps; browser tabs and app launching are unavailable. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, initialize once with var { connectCua } = await import("@deepseek-ai/dsh-cua/bridge"); var cua = connectCua(() => tools.cua, text => console.log(text)). Retain app objects between calls. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value. diff --git a/tsconfig.base.json b/tsconfig.base.json index 5ba5bea5a8..0ec95cee8b 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -68,9 +68,10 @@ "@deepseek-ai/dsh-computer-use": ["./packages/computer-use/computer-use/src/index.ts"], "@deepseek-ai/dsh-computer-use/brand": ["./packages/computer-use/computer-use/src/brand.ts"], "@deepseek-ai/dsh-experimental-computer-use-cua-driver-mcp": ["./packages/experimental/computer-use-cua-driver-mcp/src/index.ts"], - "@deepseek-ai/dsh-experimental-cua": ["./packages/experimental/cua/src/index.ts"], - "@deepseek-ai/dsh-experimental-cua/api": ["./packages/experimental/cua/src/api.ts"], - "@deepseek-ai/dsh-experimental-cua/bridge": ["./packages/experimental/cua/src/bridge.ts"], + "@deepseek-ai/dsh-cua": ["./packages/computer-use/cua/src/index.ts"], + "@deepseek-ai/dsh-cua/api": ["./packages/computer-use/cua/src/api.ts"], + "@deepseek-ai/dsh-cua/bridge": ["./packages/computer-use/cua/src/bridge.ts"], + "@deepseek-ai/dsh-cua/native-build": ["./packages/computer-use/cua/src/native-build.ts"], "@deepseek-ai/dsh-experimental-computer-use-cua-driver-native": ["./packages/experimental/computer-use-cua-driver-native/src/index.ts"], "@deepseek-ai/cordis": ["./vendor/cordis/src"], "@deepseek-ai/cosmokit": ["./vendor/cosmokit/src"], @@ -374,6 +375,7 @@ "@deepseek-ai/dsh-credentials": ["./packages/credentials/credentials/src"], "@deepseek-ai/dsh-credentials/invariant": ["./packages/credentials/credentials/src/invariant.ts"], "@deepseek-ai/dsh-credentials-local": ["./packages/credentials/credentials-local/src"], + "@deepseek-ai/dsh-cua-bundle": ["./packages/computer-use/cua-bundle/src"], "@deepseek-ai/dsh-deepseek-llm-api-extensions": ["./packages/llm/deepseek-llm-api-extensions/src"], "@deepseek-ai/dsh-deque": ["./packages/util/deque/src"], "@deepseek-ai/dsh-file-reference": ["./packages/context/file-reference/src"], diff --git a/tsconfig.host.json b/tsconfig.host.json index 16d315e964..258914e54e 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -62,6 +62,7 @@ "apps/web/tests/plugin-config.e2e.ts", "apps/web/tests/input-plugin-extensions.e2e.ts", "apps/web/tests/plugin-manager.e2e.ts", + "apps/web/tests/cua-optin.e2e.ts", "apps/web/tests/plugin-install-cancel.e2e.ts", "apps/web/tests/plugin-install-approve.e2e.ts", "apps/web/tests/plugin-install-registry.e2e.ts", @@ -223,7 +224,8 @@ { "path": "./packages/computer-use/computer-use" }, { "path": "./packages/experimental/computer-use-cua-driver-mcp" }, { "path": "./packages/experimental/computer-use-cua-driver-native" }, - { "path": "./packages/experimental/cua" }, + { "path": "./packages/computer-use/cua" }, + { "path": "./packages/computer-use/cua-bundle" }, { "path": "./packages/client/ui-sidebar-documentpreview/tsconfig.host.json" }, { "path": "./packages/client/ui-sidebar-browser/tsconfig.host.json" }, { "path": "./packages/client/ui-settings-general/tsconfig.host.json" },