From feeb88466c0353f3b8340d9da1f08c355ade841c Mon Sep 17 00:00:00 2001 From: yudshj Date: Thu, 5 Nov 2026 23:22:47 +0800 Subject: [PATCH 01/44] =?UTF-8?q?perf(open-in-app):=20=E6=96=87=E4=BB=B6?= =?UTF-8?q?=E5=8F=B3=E9=94=AE=E8=8F=9C=E5=8D=95=E4=B8=8D=E5=86=8D=E7=AD=89?= =?UTF-8?q?=E5=BE=85=E5=85=B3=E8=81=94=E5=BA=94=E7=94=A8=E6=9F=A5=E8=AF=A2?= =?UTF-8?q?=EF=BC=8C=E6=8C=89=E6=96=87=E4=BB=B6=E7=B1=BB=E5=9E=8B=E7=BC=93?= =?UTF-8?q?=E5=AD=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The file context menu now names the application recorded for the file's type without waiting for the Host's native association query (osascript / PowerShell / gio), then re-queries in the background and updates the open menu in place when the answer changes. Before the first answer for a type, the row opens the file through its OS association. --- .../client/ui-open-in-app/README.i18n.yaml | 12 +-- packages/client/ui-open-in-app/README.md | 7 +- packages/client/ui-open-in-app/README.zh.md | 7 +- .../src/client/context-menus.tsx | 87 +++++++++------- .../client/ui-open-in-app/src/client/index.ts | 1 + .../ui-open-in-app/src/client/open-path.ts | 98 +++++++++++++++++-- .../tests/browser-plugin.client.spec.ts | 23 ++++- .../tests/context-menu-actions.client.spec.ts | 81 +++++++++++++-- .../tests/open-path.client.spec.ts | 89 +++++++++++++++++ 9 files changed, 343 insertions(+), 62 deletions(-) diff --git a/packages/client/ui-open-in-app/README.i18n.yaml b/packages/client/ui-open-in-app/README.i18n.yaml index cfc7291505..5c3f8aa198 100644 --- a/packages/client/ui-open-in-app/README.i18n.yaml +++ b/packages/client/ui-open-in-app/README.i18n.yaml @@ -18,11 +18,11 @@ en: b149da9963eb9837 zh: 2426ea37fbd0f8b7 /deepseek-ai-dsh-client-ui-open-in-app/use-this-package/what-to-expect: - en: e07d3a607d54cb47 - zh: 9a7342f05b64db38 + en: 23fe56e2d62462ac + zh: e703585eac610e47 /deepseek-ai-dsh-client-ui-open-in-app/understand-the-implementation: - en: 22da59203ca974e6 - zh: f288bd6a193bccef + en: 32c651cd37edf401 + zh: c62dff20876e41fe /deepseek-ai-dsh-client-ui-open-in-app/further-exploration: en: 720527877e452a97 zh: b2241c7a336a3eeb @@ -33,8 +33,8 @@ en: ca75c51c89c2c9b0 zh: c2241315dc0c89c9 /deepseek-ai-dsh-client-ui-open-in-app/known-limitations-and-deferred-work: - en: 59c434b70679fc16 - zh: 7fe9e52b0bb82df1 + en: bf22c831463d67b6 + zh: 78965275b1bc1179 /deepseek-ai-dsh-client-ui-open-in-app/known-limitations-and-deferred-work/dev-note: en: 84f92568fc74adf1 zh: 286b47e6b5b1a556 diff --git a/packages/client/ui-open-in-app/README.md b/packages/client/ui-open-in-app/README.md index 9b0f75963b..5faf096be6 100644 --- a/packages/client/ui-open-in-app/README.md +++ b/packages/client/ui-open-in-app/README.md @@ -39,11 +39,11 @@ The **Open locally** shortcut captures the main Session’s directory and rememb File menus list the discovered associated applications without a separate “Open in default app” row. “Show file location” stays in a fixed footer separated from the scrolling applications. A successful query with only one available action renders a single button without a dropdown. Initial file-association loading uses a gray skeleton icon. When the Host identifies a default application the menu marks it “(default)” and the main button opens it; otherwise the first associated application takes that place, so the main button reveals the file only when no application is registered. Directory menus have no reveal row. Choosing a file application does not change the system default. An unpreviewable file uses the same menu in a 40px-high split button with an icon and action text; its label is “Open” or “Show file location” according to the default action. -Mounted controls with the same association reader and file share one query and its results; opening either menu refreshes all of them. Releasing the last control cancels the query and discards its state. Cancelled queries cannot replace another file’s results. A failed query displays a menu message and leaves reveal available as the default action. File-association discovery follows [native-command](../../util/native-command/README.md); an empty result, including a platform without a discovery adapter, uses the same reveal fallback without platform branches in the control. +Mounted controls with the same association reader and file share one query and its results; opening either menu refreshes all of them. Releasing the last control cancels the query and discards its state. Cancelled queries cannot replace another file’s results. Every successful answer also replaces the page’s record for the file’s association class: the final name extension, or the whole path when the name has none. A failed query displays a menu message and leaves reveal available as the default action. File-association discovery follows [native-command](../../util/native-command/README.md); an empty result, including a platform without a discovery adapter, uses the same reveal fallback without platform branches in the control. Workspace context menus reveal the captured folder through the shared directory application catalog. File-tree directories expose native reveal and Copy folder path; they do not offer file preview or save. -Context menus contribute file preview, native application opening, original-file reveal, and URL navigation and copying through [ui-context-menu](../ui-context-menu/README.md). Session submenus resolve the captured Session ID’s directory rather than the main view, and observe application, operation, Session-list, and locale changes. Native operations report through the feature’s global toast after menu dismissal or target changes, without a second registry toast. Native file actions resolve Session paths through authorized metadata before querying applications. Missing files disable path actions; image reveal requires an actual Host file and always targets that original. [ui-file-actions](../ui-file-actions/README.md) owns copy, download, save-as, and their completion feedback, including revealing the saved destination from its toast. +Context menus contribute file preview, native application opening, original-file reveal, and URL navigation and copying through [ui-context-menu](../ui-context-menu/README.md). Session submenus resolve the captured Session ID’s directory rather than the main view, and observe application, operation, Session-list, and locale changes. Native operations report through the feature’s global toast after menu dismissal or target changes, without a second registry toast. Native file actions resolve Session paths through authorized metadata, then name the application recorded for the file’s type without waiting for the Host. Until the Host first answers for that type, the row reads “Open in default application” and opens the file through its OS association. Each opening re-queries the Host in the background; a changed answer updates the open menu in place, and a failed query keeps the recorded answer. Missing files disable path actions; image reveal requires an actual Host file and always targets that original. [ui-file-actions](../ui-file-actions/README.md) owns copy, download, save-as, and their completion feedback, including revealing the saved destination from its toast. File menus copy absolute paths without line numbers; URL copying preserves query and fragment. Image files also expose image copy and original download, alongside the existing file actions. Successful downloads and saves show the shared Toast; picker cancellation is silent. @@ -59,7 +59,7 @@ A file menu offers “Open in sidebar” and “Open in floating panel”, which [`session-open-with.ts`](src/client/session-open-with.ts) shares one resolver across the shared context-menu package’s `session` and `sessionHeader` targets; other contributions do not cross between those targets. The file-tree slot uses `OpenInAppAction` with its owner’s `absolutePath`; there is no independent Session-header opener slot. A page-lifetime [`controller`](src/client/controller.ts) owns availability, persisted choice, and launch POST state. Menu observers and injected component hooks read those same sources. Document-relative routes and payload types come from the Host’s browser-safe `@deepseek-ai/dsh-host-open-in-app/shared` subpath. Adapters resolve native and transport failures to named outcomes; skipped requests announce nothing. -The directory and file adapters supply application metadata and operations to [`OpenTargetButton`](src/client/OpenTargetButton.tsx), which owns menu ordering, default markers, icons, and sizing. Its injected reporter sends outcomes to one feature-owned `shell.overlay` toast, so switching Sessions or closing the initiating preview cannot discard the result. The file header and empty state share `FileOpenTarget`, while `OpenPathInjected.applications` queries `session.workspacePathApplications` through [`open-path.ts`](src/client/open-path.ts). Opening uses `session.openWorkspacePath`; the Host revalidates an explicitly selected handler before launch. The directory adapter keeps the existing catalog routes. `FileRouteAction` supplies the same control to delivery cards and change review through `deliverables.file.actions` and `deliverables.review.file.actions`; their authenticated routes retain Session file authorization. A failed or unavailable file query therefore needs no platform-specific UI implementation. +The directory and file adapters supply application metadata and operations to [`OpenTargetButton`](src/client/OpenTargetButton.tsx), which owns menu ordering, default markers, icons, and sizing. Its injected reporter sends outcomes to one feature-owned `shell.overlay` toast, so switching Sessions or closing the initiating preview cannot discard the result. The file header and empty state share `FileOpenTarget`, while `OpenPathInjected.applications` queries `session.workspacePathApplications` through [`open-path.ts`](src/client/open-path.ts). The same controller keeps the page’s answers by association class, runs one background refresh per class, and publishes a new snapshot only when an answer changes. The file menu observes that snapshot and re-resolves from its verified path synchronously, so the context-menu registry keeps the other rows while the application row changes. Opening uses `session.openWorkspacePath`; the Host revalidates an explicitly selected handler before launch. The directory adapter keeps the existing catalog routes. `FileRouteAction` supplies the same control to delivery cards and change review through `deliverables.file.actions` and `deliverables.review.file.actions`; their authenticated routes retain Session file authorization. A failed or unavailable file query therefore needs no platform-specific UI implementation. @@ -93,6 +93,7 @@ The Host verifies the path through the composed filesystem before opening or rev - **The dictionaries gate the menu.** A host catalog extension without a matching `app.` entry in both dictionaries stays invisible instead of showing a raw id; extending the catalog means extending [`dsh-host-open-in-app`](../../host/open-in-app/README.md) and this package's locales together. - **Availability is read once per page.** An application installed while the page is open appears after a reload (and, host-side, after a host restart); the desktop answer behind the file controls is read once per page as well. +- **File associations are cached by type for the page.** Files sharing a final name extension share one answer, so a per-document override, such as a macOS “Open with” choice for one file, or Linux content sniffing and multi-part extensions such as `.tar.gz` can show the type’s previous answer until the background query corrects the open menu. Names without an extension are recorded per path. Records grow with the file types seen and end with the page. - **One reveal label for every platform.** The Session Remote reports whether a desktop exists, not which file manager it runs, so the menu says "Show file location" rather than naming Finder or File Explorer as the delivery cards do. - **The delivery cards keep their own opener.** [`ui-deliverables`](../ui-deliverables/README.md) still opens declared files through its own Session-and-event routes; folding those cards onto the file controls here remains deferred. diff --git a/packages/client/ui-open-in-app/README.zh.md b/packages/client/ui-open-in-app/README.zh.md index 7bac1ae79f..b0a6177b8f 100644 --- a/packages/client/ui-open-in-app/README.zh.md +++ b/packages/client/ui-open-in-app/README.zh.md @@ -39,11 +39,11 @@ kind: "package-reference" 文件菜单列出已发现的关联应用，不单列“用默认应用打开”。“显示文件位置”固定在菜单底部，通过分隔线与滚动的应用列表分开。查询成功且只有一个可用操作时显示单按钮，不再显示下拉箭头。文件关联首次加载时使用灰色骨架图标。Host 识别出默认应用时，菜单将该项标注为“（默认）”，主按钮打开该应用；否则第一个关联应用占据该位置，只有在一个关联应用都没有时主按钮才执行文件定位。目录菜单不包含定位项。选择文件应用不修改系统默认应用。无法预览文件时，空态使用同一菜单，按钮增大到 40px 高并显示图标和动作文字；根据默认动作显示“打开”或“显示文件位置”。 -使用同一查询函数和文件的已挂载控件共用查询与结果，打开任一菜单会刷新所有相关控件。最后一个控件释放后取消查询并清除状态。已取消的查询不会覆盖其他文件的结果。查询失败时菜单显示提示，并以文件定位作为默认动作。文件关联查询的平台支持范围见 [native-command](../../util/native-command/README.zh.md)；查询结果为空时，包括平台尚无查询适配器的情况，控件统一使用定位动作，不按操作系统分支处理。 +使用同一查询函数和文件的已挂载控件共用查询与结果，打开任一菜单会刷新所有相关控件。最后一个控件释放后取消查询并清除状态。已取消的查询不会覆盖其他文件的结果。每次查询成功后，结果还会替换本页面中该文件关联类别的记录：类别按文件名的最后一个扩展名划分，没有扩展名的文件按完整路径划分。查询失败时菜单显示提示，并以文件定位作为默认动作。文件关联查询的平台支持范围见 [native-command](../../util/native-command/README.zh.md)；查询结果为空时，包括平台尚无查询适配器的情况，控件统一使用定位动作，不按操作系统分支处理。 工作区右键菜单通过共享的目录应用列表定位被点击的文件夹。文件树目录提供原生定位与复制文件夹路径，不提供文件预览或保存。 -右键菜单通过 [ui-context-menu](../ui-context-menu/README.zh.md) 贡献文件预览、原生应用打开、原文件定位以及链接导航与复制。会话子菜单解析被捕获会话 ID 的目录，不使用主视图，并观察应用、操作、会话列表及语言变化。原生操作在菜单关闭或目标切换后仍通过本功能的全局 toast 报告，不产生第二个注册表提示。原生文件操作先通过授权元数据解析会话路径，再查询关联应用。文件不存在时禁用路径操作；图片定位要求实际 Host 文件，并始终针对原文件。[ui-file-actions](../ui-file-actions/README.zh.md) 负责复制、下载、另存为及完成提示，包括通过提示定位保存目标。 +右键菜单通过 [ui-context-menu](../ui-context-menu/README.zh.md) 贡献文件预览、原生应用打开、原文件定位以及链接导航与复制。会话子菜单解析被捕获会话 ID 的目录，不使用主视图，并观察应用、操作、会话列表及语言变化。原生操作在菜单关闭或目标切换后仍通过本功能的全局 toast 报告，不产生第二个注册表提示。原生文件操作先通过授权元数据解析会话路径，然后直接显示该文件类型已记录的应用，不等待 Host。Host 首次回答该类型之前，这一项显示“使用默认应用打开”，通过系统的文件关联打开文件。每次打开菜单都会在后台重新查询 Host；回答有变化时，已打开的菜单原地更新，查询失败则保留已记录的回答。文件不存在时禁用路径操作；图片定位要求实际 Host 文件，并始终针对原文件。[ui-file-actions](../ui-file-actions/README.zh.md) 负责复制、下载、另存为及完成提示，包括通过提示定位保存目标。 文件菜单复制不含行号的绝对路径；网址复制保留查询参数与锚点。图片文件在既有文件操作之外，还提供复制图片与下载原图。下载和保存成功后显示共享 Toast，取消选择器时不提示。 @@ -59,7 +59,7 @@ kind: "package-reference" [`session-open-with.ts`](src/client/session-open-with.ts) 为共享右键菜单包声明的 `session` 和 `sessionHeader` 目标共用一个解析器；其他贡献不会在这两种目标间扩散。文件树插槽通过所属组件的 `absolutePath` 使用 `OpenInAppAction`；会话头部没有独立打开按钮插槽。页面生命周期的 [`controller`](src/client/controller.ts) 持有可用性、持久化偏好与启动 POST 状态。菜单观察器和组件注入 hooks 读取同一组源。文档相对路由与载荷类型来自 Host 的浏览器安全子路径 `@deepseek-ai/dsh-host-open-in-app/shared`。适配器将原生操作及传输失败解析为具名结果；跳过的请求不播报。 -目录和文件适配器把应用信息与操作交给 [`OpenTargetButton`](src/client/OpenTargetButton.tsx)，由它统一管理菜单顺序、默认标记、图标和尺寸。注入的回调将结果交给本功能在 `shell.overlay` 中持有的唯一 toast，因此切换会话或关闭发起操作的预览不会丢失结果。文件标题栏和空态共用 `FileOpenTarget`，`OpenPathInjected.applications` 通过 [`open-path.ts`](src/client/open-path.ts) 查询 `session.workspacePathApplications`。打开操作使用 `session.openWorkspacePath`，Host 在启动前重新验证指定的关联应用。`FileRouteAction` 通过 `deliverables.file.actions` 和 `deliverables.review.file.actions` 为交付卡片和变更对比页提供同一控件，其认证路由保留会话文件校验。目录适配器继续使用已有的应用列表路由，文件查询失败或不可用时无需增加平台专用的界面实现。 +目录和文件适配器把应用信息与操作交给 [`OpenTargetButton`](src/client/OpenTargetButton.tsx)，由它统一管理菜单顺序、默认标记、图标和尺寸。注入的回调将结果交给本功能在 `shell.overlay` 中持有的唯一 toast，因此切换会话或关闭发起操作的预览不会丢失结果。文件标题栏和空态共用 `FileOpenTarget`，`OpenPathInjected.applications` 通过 [`open-path.ts`](src/client/open-path.ts) 查询 `session.workspacePathApplications`。同一控制器按关联类别保存本页面的回答，每个类别同一时间只运行一次后台刷新，只在回答变化时发布新快照。文件菜单观察该快照，并基于已验证的路径同步重新解析，因此右键菜单注册表保留其他各项，只有应用这一项原地变化。打开操作使用 `session.openWorkspacePath`，Host 在启动前重新验证指定的关联应用。`FileRouteAction` 通过 `deliverables.file.actions` 和 `deliverables.review.file.actions` 为交付卡片和变更对比页提供同一控件，其认证路由保留会话文件校验。目录适配器继续使用已有的应用列表路由，文件查询失败或不可用时无需增加平台专用的界面实现。 @@ -93,6 +93,7 @@ Host 在打开或定位前通过当前文件系统验证路径。没有对应 Ho - **词典把守菜单。** 主机目录的新条目若在两份词典中没有对应的 `app.` 条目，将保持不可见而不是显示裸 id；扩展目录意味着同时扩展 [`dsh-host-open-in-app`](../../host/open-in-app/README.zh.md) 与本包的 locale。 - **可用性每页只读一次。** 页面打开期间安装的应用要重新加载页面后才出现（主机侧还需主机重启）；文件控件背后的桌面回答同样每页只读一次。 +- **文件关联按类型在页面内缓存。** 最后一个扩展名相同的文件共用一个回答，因此单个文件的单独设置（例如 macOS 中只针对某个文件的“打开方式”），或 Linux 的内容嗅探与 `.tar.gz` 这类多段扩展名，可能先显示该类型之前的回答，再由后台查询更正已打开的菜单。没有扩展名的文件按路径记录。记录随出现过的文件类型增长，并在页面关闭时释放。 - **所有平台共用一个定位标签。** Session Remote 只报告有没有桌面，不报告它跑的是哪个文件管理器，所以菜单写「显示文件位置」，而不像交付卡片那样点名访达或文件资源管理器。 - **交付卡片保留自己的打开器。** [`ui-deliverables`](../ui-deliverables/README.zh.md) 仍通过自己按 Session 与事件定位的路由打开声明过的文件；把这些卡片并到这里的文件控件上仍属延期工作。 diff --git a/packages/client/ui-open-in-app/src/client/context-menus.tsx b/packages/client/ui-open-in-app/src/client/context-menus.tsx index 5810a3aab7..6ee7d48cca 100644 --- a/packages/client/ui-open-in-app/src/client/context-menus.tsx +++ b/packages/client/ui-open-in-app/src/client/context-menus.tsx @@ -3,7 +3,7 @@ import { useState } from 'react' import { IconRightUpOutlineRegular } from '@deepseek-ai/dsh-client-ui-primitives' import type { PropsRuntime } from '@deepseek-ai/dsh-client-ui-slots' import type { Context } from '@deepseek-ai/cordis' -import type {} from '@deepseek-ai/dsh-client-ui-context-menu/client' +import type { ContextMenuTargetMap, MenuAction } from '@deepseek-ai/dsh-client-ui-context-menu/client' import type {} from '@deepseek-ai/dsh-client-ui-workspace/client' import type {} from '@deepseek-ai/dsh-client-ui-sidebar-right/client' import type {} from '@deepseek-ai/dsh-api-workspace-files/remote' @@ -76,40 +76,61 @@ export function registerContextMenus( ] }, }), 'open-in-app: directory context menu') + const fileMenu = (target: ContextMenuTargetMap['file'], absolutePath: string | undefined): MenuAction[] => { + const { sessionId, path, line } = target + const available = absolutePath !== undefined + const desktop = paths.desktop.getSnapshot() === true && available + // Undefined until the Host first answers for this file type; the row then opens through the association. + const apps = desktop ? paths.cachedApplications(absolutePath) : [] + const application = apps?.find(app => app.default) + const openApplication = absolutePath === undefined || !desktop ? undefined + : application !== undefined ? () => open(absolutePath, 'open', application.id) + : apps === undefined ? () => open(absolutePath, 'open') : undefined + // The menu names a host, not the pane it was raised in: a floating pane is never a docked landing place. + const openIn = (destination: 'sidebar' | 'float') => () => { + const cwd = ctx.sessions.list.getSnapshot().byId[sessionId]?.cwd + ctx.sidebarRight.openResourceIn(sessionId, fileAddressFor(sessionId, cwd, path), { + destination, ...(line === undefined ? {} : { params: { line } }), + }) + } + return [ + { id: 'sidebar', icon: { key: 'open-in-app.sidebar' }, label: t('menu.openInSidebar'), disabled: !available, run: openIn('sidebar') }, + { id: 'float', icon: { key: 'open-in-app.float' }, label: t('menu.openInFloat'), disabled: !available, run: openIn('float') }, + { id: 'reveal', icon: { key: 'open-in-app.reveal' }, label: fileManager(), disabled: !desktop, + ...absolutePath === undefined ? {} : { run: () => open(absolutePath, 'reveal') } }, + { id: 'open', label: application ? t('open.title', { app: application.name }) : t('menu.defaultApp'), + icon: { key: 'open-in-app.application', ...application?.icon ? { src: application.icon } : {} }, + disabled: openApplication === undefined, ...openApplication === undefined ? {} : { run: openApplication } }, + { id: 'copyPath', icon: { key: 'open-in-app.copy' }, failureLabel: t('menu.copyFailed'), label: t('menu.copyPath'), + run: async () => { + const cwd = ctx.sessions.list.getSnapshot().byId[sessionId]?.cwd + await navigator.clipboard.writeText(absolutePath ?? resolveWorkspacePath(cwd, path)) + ctx.contextMenus.notify(t('menu.copied'), true) + } }, + ] + } + // The verified Host path of each open file menu. Association answers re-resolve the menu from it + // synchronously, so the registry keeps the other rows while the application row changes in place. + const verified = new WeakMap() ctx.effect(() => ctx.contextMenus.register({ id: 'open-in-app.file', target: 'file', order: 100, - resolve: async (target, signal) => { - const { sessionId, path, line } = target - await paths.load() - const absolutePath = await resolvePath(sessionId, path, signal) - const available = absolutePath !== undefined - const desktop = paths.desktop.getSnapshot() === true && available - const apps = desktop ? await paths.applications(absolutePath, signal) : [] - signal.throwIfAborted() - const application = apps?.find(app => app.default) - // The menu names a host, not the pane it was raised in: a floating pane is never a docked landing place. - const openIn = (destination: 'sidebar' | 'float') => () => { - const cwd = ctx.sessions.list.getSnapshot().byId[sessionId]?.cwd - ctx.sidebarRight.openResourceIn(sessionId, fileAddressFor(sessionId, cwd, path), { - destination, ...(line === undefined ? {} : { params: { line } }), - }) - } - return [ - { id: 'sidebar', icon: { key: 'open-in-app.sidebar' }, label: t('menu.openInSidebar'), disabled: !available, run: openIn('sidebar') }, - { id: 'float', icon: { key: 'open-in-app.float' }, label: t('menu.openInFloat'), disabled: !available, run: openIn('float') }, - { id: 'reveal', icon: { key: 'open-in-app.reveal' }, label: fileManager(), disabled: !desktop, - ...absolutePath === undefined ? {} : { run: () => open(absolutePath, 'reveal') } }, - { id: 'open', label: application ? t('open.title', { app: application.name }) : t('menu.defaultApp'), - icon: { key: 'open-in-app.application', ...application?.icon ? { src: application.icon } : {} }, - disabled: application === undefined, - ...absolutePath === undefined || application === undefined ? {} : { run: () => open(absolutePath, 'open', application.id) } }, - { id: 'copyPath', icon: { key: 'open-in-app.copy' }, failureLabel: t('menu.copyFailed'), label: t('menu.copyPath'), - run: async () => { - const cwd = ctx.sessions.list.getSnapshot().byId[sessionId]?.cwd - await navigator.clipboard.writeText(absolutePath ?? resolveWorkspacePath(cwd, path)) - ctx.contextMenus.notify(t('menu.copied'), true) - } }, - ] + observe: (target) => { + // Each occurrence subscribes before its first resolution, so a reopened target is verified again. + verified.set(target, null) + return [paths.associations] + }, + resolve: (target, signal) => { + const settled = verified.get(target) + if (settled) return fileMenu(target, settled.absolutePath) + return (async () => { + await paths.load() + const absolutePath = await resolvePath(target.sessionId, target.path, signal) + signal.throwIfAborted() + verified.set(target, { absolutePath }) + // A cached answer shows at once; every opening re-queries so a changed association reaches this menu. + if (absolutePath !== undefined && paths.desktop.getSnapshot() === true) paths.refreshApplications(absolutePath) + return fileMenu(target, absolutePath) + })() }, }), 'open-in-app: file context menu') ctx.effect(() => ctx.contextMenus.register({ diff --git a/packages/client/ui-open-in-app/src/client/index.ts b/packages/client/ui-open-in-app/src/client/index.ts index e706f9eb25..84bdc9168e 100644 --- a/packages/client/ui-open-in-app/src/client/index.ts +++ b/packages/client/ui-open-in-app/src/client/index.ts @@ -56,6 +56,7 @@ export function apply(ctx: ClientContext): void { const controller = new OpenInAppController() void controller.load() const paths = new OpenInAppPathController(ctx.remote) + ctx.effect(() => () => { paths.dispose() }, 'open-in-app: association refreshes') ctx.effect(() => ctx.locale.register(NS, { zh, en, ja, es, fr, de, ru, pt }), 'open-in-app: dictionaries') const t = ctx.locale.bind(NS) // Keep operation feedback visible after its initiating panel closes. diff --git a/packages/client/ui-open-in-app/src/client/open-path.ts b/packages/client/ui-open-in-app/src/client/open-path.ts index 4484bb29b9..bc9fb4c808 100644 --- a/packages/client/ui-open-in-app/src/client/open-path.ts +++ b/packages/client/ui-open-in-app/src/client/open-path.ts @@ -1,12 +1,34 @@ /** * Host desktop availability, file associations, and open/reveal actions over the Session Remote. - * The desktop answer is read once per page; a failed read renders no control. + * The desktop answer is read once per page; a failed read renders no control. Association answers + * are recorded per association class so menus can show them without waiting for the Host. */ -import { createSnapshotStore, type SnapshotStore } from '@deepseek-ai/dsh-client-store' +import { createSnapshotStore, type ObservableSnapshot, type SnapshotStore } from '@deepseek-ai/dsh-client-store' import type { RemoteResult } from '@deepseek-ai/dsh-api-remotes/client' import type { SessionOpenWorkspacePathRequest, SessionWorkspacePathApplication } from '@deepseek-ai/dsh-api-session-controller/types' import type { OpenInAppOutcome, OpenInAppPathFailure } from './outcome.ts' +type Applications = readonly SessionWorkspacePathApplication[] + +/** + * Name the association class of a Host path: desktops look handlers up by the final name extension, so + * files sharing it share one answer; a name without one (`Makefile`, `.env`) is its own class. + */ +function associationKey(path: string): string { + const name = path.slice(Math.max(path.lastIndexOf('/'), path.lastIndexOf('\\')) + 1) + const dot = name.lastIndexOf('.') + return dot > 0 && dot < name.length - 1 ? `extension:${name.slice(dot)}` : `path:${path}` +} + +/** Compare two answers field by field, including icon data. */ +function sameApplications(left: Applications, right: Applications): boolean { + return left.length === right.length && left.every((app, index) => { + const other = right[index] + return other !== undefined && app.id === other.id && app.name === other.name + && app.default === other.default && app.icon === other.icon + }) +} + /** What a path gesture asks of the Host desktop: the default application, or the file manager showing the file. */ export type OpenInAppPathAction = 'open' | 'reveal' @@ -24,11 +46,24 @@ export interface OpenInAppPathRemote { } } -/** Page-lifetime desktop availability and the open/reveal carrier shared by every path control. */ +/** + * Page-lifetime desktop availability, association answers, and the open/reveal carrier shared by every path control. + * Answers are kept per association class for the page's life and grow with the classes queried. + */ export class OpenInAppPathController { /** Whether the Host desktop can open paths; null until the Host answered, false also after a failed read. */ readonly desktop: SnapshotStore = createSnapshotStore(null) + private readonly answers = createSnapshotStore>(new Map()) + /** Latest successful answer per association class; the snapshot keeps its identity until an answer changes. */ + readonly associations: ObservableSnapshot> = this.answers private loading: Promise | undefined + private readonly lifetime = new AbortController() + private readonly refreshing = new Set() + /** Start order of the query whose answer each class holds; an earlier query never replaces a later one. */ + private readonly answeredBy = new Map() + /** One icon string per application id, reused by every class that lists the application. */ + private readonly icons = new Map() + private queries = 0 /** * @param remote - the Session Remote namespace answering availability and running gestures. @@ -66,20 +101,67 @@ export class OpenInAppPathController { } /** - * Refresh the file's OS associations; failures remain distinct from an empty handler list. + * Query the file's current OS associations and record a successful answer for its association class. + * Failures remain distinct from an empty handler list and leave the recorded answer in place. * @param path - file path reported by the Host. - * @param signal - lifetime of the requesting preview. + * @param signal - lifetime of the requesting control. * @returns application metadata, or null when the query fails. */ - async applications(path: string, signal: AbortSignal): Promise { - let result: RemoteResult + async applications(path: string, signal: AbortSignal): Promise { + const order = ++this.queries + let result: RemoteResult try { result = await this.remote.session.workspacePathApplications({ path }, signal) } catch (_error) { // The control reports query failure and keeps file reveal available. return null } - return result.ok ? result.value : null + if (!result.ok) return null + this.record(associationKey(path), order, result.value) + return result.value + } + + /** + * Read the recorded answer of a file's association class without querying the Host. + * @param path - absolute Host path of the file. + * @returns the latest successful answer, or undefined before a query of this class succeeded. + */ + cachedApplications(path: string): Applications | undefined { + return this.answers.getSnapshot().get(associationKey(path)) + } + + /** + * Re-query a file's associations in the background. One refresh per association class runs at a time; + * it outlives the requesting menu and ends with its answer or {@link dispose}. + * @param path - absolute Host path of the file. + */ + refreshApplications(path: string): void { + const key = associationKey(path) + if (this.lifetime.signal.aborted || this.refreshing.has(key)) return + this.refreshing.add(key) + void this.applications(path, this.lifetime.signal).finally(() => { this.refreshing.delete(key) }) + } + + /** Cancel background refreshes and refuse new ones; recorded answers stay readable. */ + dispose(): void { + this.lifetime.abort() + } + + private record(key: string, order: number, applications: Applications): void { + if ((this.answeredBy.get(key) ?? 0) > order) return + this.answeredBy.set(key, order) + const answers = this.answers.getSnapshot() + const previous = answers.get(key) + if (previous !== undefined && sameApplications(previous, applications)) return + const next = new Map(answers) + next.set(key, applications.map((app) => { + if (app.icon === null) return app + const icon = this.icons.get(app.id) + if (icon === app.icon) return { ...app, icon } + this.icons.set(app.id, app.icon) + return app + })) + this.answers.set(next) } private async run(): Promise { diff --git a/packages/client/ui-open-in-app/tests/browser-plugin.client.spec.ts b/packages/client/ui-open-in-app/tests/browser-plugin.client.spec.ts index ca27f69e50..274d379207 100644 --- a/packages/client/ui-open-in-app/tests/browser-plugin.client.spec.ts +++ b/packages/client/ui-open-in-app/tests/browser-plugin.client.spec.ts @@ -49,10 +49,13 @@ afterEach(() => { /** The Session Remote slice the path controls call; answers a desktop and acknowledges every gesture. */ const remote = { session: { - workspacePathApplications: vi.fn(async () => ({ ok: true as const, value: [] })), + workspacePathApplications: vi.fn(async (_request: { path: string }, _signal?: AbortSignal) => ({ ok: true as const, value: [] })), canOpenWorkspacePath: vi.fn(async () => ({ ok: true as const, value: true })), openWorkspacePath: vi.fn(async () => ({ ok: true as const, value: { opened: true as const } })), }, + workspaceFiles: { + stat: vi.fn(async (_sessionId: SessionId, path: string) => ({ ok: true as const, value: { absolutePath: `/w/${path}` } })), + }, } /** Boot the browser half over a real slot tree that declares the header list and the document-preview seats. */ @@ -335,6 +338,24 @@ describe('open-in-app browser half', () => { expect(ctx.slots.entries('deliverables.file.actions')).toHaveLength(0) }) + it('lets a file-menu association refresh outlive the menu and cancels it with the plugin', async () => { + vi.stubGlobal('fetch', vi.fn(async () => new Response(JSON.stringify({ apps: [] }), { status: 200 }))) + const { ctx, fiber } = await bench() + let refresh: AbortSignal | undefined + remote.session.workspacePathApplications.mockImplementationOnce((_request, signal) => { + refresh = signal + return new Promise(() => {}) + }) + const menu = ctx.contextMenus.resolve({ kind: 'file', value: { sessionId: 'files' as SessionId, path: 'notes.txt' } }) + await menu.ready + expect(remote.workspaceFiles.stat).toHaveBeenCalledWith('files', 'notes.txt', expect.any(AbortSignal)) + expect(remote.session.workspacePathApplications).toHaveBeenCalledWith({ path: '/w/notes.txt' }, refresh) + menu.dispose() + expect(refresh?.aborted).toBe(false) + await fiber.dispose() + expect(refresh?.aborted).toBe(true) + }) + it('injects the announcement seat into the delivery entries and releases every entry with the fiber', async () => { vi.stubGlobal('fetch', vi.fn(async () => new Response(JSON.stringify({ apps: [] }), { status: 200 }))) const { ctx, fiber } = await bench() diff --git a/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts b/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts index 10407f9f54..b3d7d5a319 100644 --- a/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts +++ b/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts @@ -7,6 +7,8 @@ import { createSnapshotStore } from '@deepseek-ai/dsh-client-store' import { SessionId } from '@deepseek-ai/dsh-session' import { MenuRegistry } from '@deepseek-ai/dsh-client-ui-context-menu/src/client/registry.ts' import type { ContextMenus, ContextMenuTarget } from '@deepseek-ai/dsh-client-ui-context-menu/client' +import type { RemoteResult } from '@deepseek-ai/dsh-api-remotes/client' +import type { SessionWorkspacePathApplication } from '@deepseek-ai/dsh-api-session-controller/types' import type { FileActionSource, FileSaveResult } from '@deepseek-ai/dsh-client-ui-file-actions/client' import { OpenInAppPathController } from '../src/client/open-path.ts' import { OpenInAppController } from '../src/client/controller.ts' @@ -15,6 +17,7 @@ import { registerContextMenus } from '../src/client/context-menus.tsx' import type { OpenInAppToastSource } from '../src/client/open-toast.tsx' const sessionId = SessionId('menu-actions') +type Applications = readonly SessionWorkspacePathApplication[] afterEach(() => { vi.restoreAllMocks(); vi.unstubAllGlobals() }) async function boot() { @@ -27,13 +30,13 @@ async function boot() { const remote = { session: { canOpenWorkspacePath: vi.fn(async () => ({ ok: true as const, value: true })), - workspacePathApplications: vi.fn(async () => ({ ok: true as const, value: [ - { id: 'preview', name: 'Preview', default: true, icon: 'data:image/png;base64,a' }, - ] })), + workspacePathApplications: vi.fn(async (_request: { path: string }, _signal?: AbortSignal): Promise> => ({ + ok: true, value: [{ id: 'preview', name: 'Preview', default: true, icon: 'data:image/png;base64,a' }], + })), openWorkspacePath: vi.fn(async () => ({ ok: true as const, value: undefined })), }, workspaceFiles: { - stat: vi.fn(async () => ({ ok: true as const, value: { absolutePath: '/workspace/file.svg' } })), + stat: vi.fn(async (_sessionId: SessionId, path: string) => ({ ok: true as const, value: { absolutePath: path.startsWith('/') ? path : `/workspace/${path}` } })), readBytes: vi.fn(async () => ({ ok: true as const, value: { data: [...new TextEncoder().encode('')] } })), }, } @@ -82,14 +85,17 @@ it('routes file actions through an explicit host, the line, native association a const h = await boot() // A menu raised inside a floating panel names that panel; the open names a host instead, never a pane. const file = { kind: 'file', value: { sessionId, path: 'file.svg', paneId: 'float7' as never, line: 24 } } as const - const items = (await h.resolve(file)).source.getSnapshot().filter(item => item.id.startsWith('open-in-app.file/')) - expect(items.map(item => [item.id, item.label, item.icon?.key])).toEqual([ + const first = await h.resolve(file) + const items = () => first.source.getSnapshot().filter(item => item.id.startsWith('open-in-app.file/')) + await vi.waitFor(() => { expect(items().find(item => item.id === 'open-in-app.file/open')?.label).toBe('open.title') }) + expect(items().map(item => [item.id, item.label, item.icon?.key])).toEqual([ ['open-in-app.file/sidebar', 'menu.openInSidebar', 'open-in-app.sidebar'], ['open-in-app.file/float', 'menu.openInFloat', 'open-in-app.float'], ['open-in-app.file/reveal', 'menu.reveal', 'open-in-app.reveal'], ['open-in-app.file/open', 'open.title', 'open-in-app.application'], ['open-in-app.file/copyPath', 'menu.copyPath', 'open-in-app.copy'], ]) + first.dispose() await h.choose(file, 'open-in-app.file/sidebar') expect(h.sidebar.openResourceIn).toHaveBeenLastCalledWith(sessionId, expect.stringContaining('file.svg'), { destination: 'sidebar', params: { line: 24 } }) @@ -114,7 +120,8 @@ it('routes file actions through an explicit host, the line, native association a const simple = { kind: 'file', value: { sessionId: SessionId('absent'), path: '/file.unknown' } } as const h.remote.session.workspacePathApplications.mockResolvedValueOnce({ ok: true, value: [] }) const menu = await h.resolve(simple) - expect(menu.source.getSnapshot().find(item => item.id.endsWith('/open'))?.disabled).toBe(true) + // Without an answer the row opens through the association; the Host's empty answer then disables it in place. + await vi.waitFor(() => { expect(menu.source.getSnapshot().find(item => item.id.endsWith('/open'))?.disabled).toBe(true) }) expect(menu.source.getSnapshot().some(item => item.id.endsWith('/copyImage'))).toBe(false) expect(menu.source.getSnapshot().filter(item => item.id.startsWith('file-actions.')).map(item => item.id)) .toEqual(['file-actions.file/saveAs']) @@ -122,10 +129,59 @@ it('routes file actions through an explicit host, the line, native association a expect(h.sidebar.openResourceIn).toHaveBeenLastCalledWith(SessionId('absent'), expect.any(String), { destination: 'sidebar' }) h.remote.session.workspacePathApplications.mockResolvedValueOnce({ ok: true, value: [] }) const floating = await h.resolve(simple) + expect(floating.source.getSnapshot().find(item => item.id.endsWith('/open'))?.disabled).toBe(true) floating.select('open-in-app.file/float') expect(h.sidebar.openResourceIn).toHaveBeenLastCalledWith(SessionId('absent'), expect.any(String), { destination: 'float' }) }) +it('opens file menus before the Host answers and changes only the application row in place', async () => { + const h = await boot() + const answers: ((answer: RemoteResult) => void)[] = [] + h.remote.session.workspacePathApplications.mockImplementation(() => new Promise((answer) => { answers.push(answer) })) + const file = { kind: 'file', value: { sessionId, path: 'notes.txt' } } as const + const menu = await h.resolve(file) + const rows = () => menu.source.getSnapshot().filter(item => item.id.startsWith('open-in-app.file/')) + const application = () => rows().find(item => item.id === 'open-in-app.file/open') + expect(answers).toHaveLength(1) + expect(application()).toMatchObject({ label: 'menu.defaultApp', disabled: false }) + expect(application()?.icon).toEqual({ key: 'open-in-app.application' }) + const published: number[] = [] + const stop = menu.source.subscribe(() => { published.push(rows().length) }) + answers[0]!({ ok: true, value: [ + { id: '/Applications/TextEdit.app', name: 'TextEdit', default: false, icon: null }, + { id: '/Applications/Sublime Text.app', name: 'Sublime Text', default: true, icon: 'data:image/png;base64,c3Vi' }, + ] }) + await vi.waitFor(() => { expect(application()).toMatchObject({ label: 'open.title', icon: { src: 'data:image/png;base64,c3Vi' } }) }) + stop() + expect(published.length).toBeGreaterThan(0) + expect(published.every(count => count === 5)).toBe(true) + menu.select('open-in-app.file/open') + await vi.waitFor(() => { + expect(h.remote.session.openWorkspacePath).toHaveBeenLastCalledWith({ path: '/workspace/notes.txt', application: '/Applications/Sublime Text.app' }) + }) + // Another file of the same type shows the recorded answer at once and asks the Host again. + const other = await h.resolve({ kind: 'file', value: { sessionId, path: 'todo.txt' } }) + const otherApplication = () => other.source.getSnapshot().find(item => item.id === 'open-in-app.file/open') + expect(otherApplication()?.label).toBe('open.title') + expect(answers).toHaveLength(2) + answers[1]!({ ok: true, value: [] }) + await vi.waitFor(() => { expect(otherApplication()).toMatchObject({ label: 'menu.defaultApp', disabled: true }) }) + other.dispose() + expect(h.failure).not.toHaveBeenCalled() +}) + +it('opens through the OS association until the Host first answers, including after failed queries', async () => { + const h = await boot() + h.remote.session.workspacePathApplications.mockResolvedValue({ ok: false } as never) + const menu = await h.resolve({ kind: 'file', value: { sessionId, path: 'notes.md' } }) + await vi.waitFor(() => { expect(h.remote.session.workspacePathApplications).toHaveBeenCalledOnce() }) + const application = menu.source.getSnapshot().find(item => item.id === 'open-in-app.file/open') + expect(application).toMatchObject({ label: 'menu.defaultApp', disabled: false }) + menu.select('open-in-app.file/open') + await vi.waitFor(() => { expect(h.remote.session.openWorkspacePath).toHaveBeenLastCalledWith({ path: '/workspace/notes.md' }) }) + await vi.waitFor(() => { expect(h.report).toHaveBeenLastCalledWith({ kind: 'opened' }) }) +}) + it('keeps copy-path available on failed metadata and reports native action failures', async () => { const h = await boot() const file = { kind: 'file', value: { sessionId, path: 'file.svg' } } as const @@ -139,14 +195,23 @@ it('keeps copy-path available on failed metadata and reports native action failu h.remote.session.openWorkspacePath.mockResolvedValueOnce({ ok: false } as never) await h.choose(file, 'open-in-app.file/reveal') await vi.waitFor(() => { expect(h.report).toHaveBeenLastCalledWith({ kind: 'failed', failure: 'revealError' }) }) + await vi.waitFor(() => { expect(h.paths.cachedApplications('/workspace/file.svg')).toHaveLength(1) }) + // A failed query keeps the recorded answer. h.remote.session.workspacePathApplications.mockResolvedValueOnce({ ok: false } as never) - expect((await h.resolve(file)).source.getSnapshot().find(item => item.id.endsWith('/open'))?.disabled).toBe(true) + const kept = await h.resolve(file) + await vi.waitFor(() => { + expect(h.remote.session.workspacePathApplications.mock.settledResults.every(result => result.type !== 'incomplete')).toBe(true) + }) + expect(kept.source.getSnapshot().find(item => item.id.endsWith('/open'))).toMatchObject({ label: 'open.title', disabled: false }) + kept.dispose() h.paths.desktop.set(false) h.fileActions.canSaveAs = false + const queried = h.remote.session.workspacePathApplications.mock.calls.length const web = await h.resolve(file) expect(web.source.getSnapshot().some(item => item.id.endsWith('/saveAs'))).toBe(false) expect(web.source.getSnapshot().filter(item => ['/open', '/reveal'].some(suffix => item.id.endsWith(suffix))) .every(item => item.disabled)).toBe(true) + expect(h.remote.session.workspacePathApplications).toHaveBeenCalledTimes(queried) }) it('opens links in the registered browser, externally, and on the clipboard', async () => { diff --git a/packages/client/ui-open-in-app/tests/open-path.client.spec.ts b/packages/client/ui-open-in-app/tests/open-path.client.spec.ts index e542b9164c..8eb4362032 100644 --- a/packages/client/ui-open-in-app/tests/open-path.client.spec.ts +++ b/packages/client/ui-open-in-app/tests/open-path.client.spec.ts @@ -94,3 +94,92 @@ it('distinguishes failed association queries from an empty application list', as const rejecting = new OpenInAppPathController(remoteOf({ workspacePathApplications: async () => { throw new Error('disconnected') } })) expect(await rejecting.applications('/file.mp3', new AbortController().signal)).toBeNull() }) + +describe('OpenInAppPathController association answers', () => { + type Answer = Awaited> + const app = (name: string, icon: string | null = null) => ({ id: `/Applications/${name}.app`, name, default: true, icon }) + const answered = (...apps: ReturnType[]): Answer => ({ ok: true, value: apps }) + /** A Remote whose association queries settle when the test answers them. */ + function deferredRemote() { + const queries: { path: string; signal: AbortSignal | undefined; answer: (result: Answer) => void }[] = [] + const workspacePathApplications = vi.fn((request: { path: string }, signal?: AbortSignal) => new Promise((answer) => { + queries.push({ path: request.path, signal, answer }) + })) + return { queries, controller: new OpenInAppPathController(remoteOf({ workspacePathApplications })) } + } + + it('shares an answer across files with the same final extension and keeps extensionless names apart', async () => { + const answers: Record = { + '/w/notes.txt': answered(app('TextEdit')), '/w/Makefile': answered(app('Xcode')), '/w/.env': answered(app('Zed')), + } + const controller = new OpenInAppPathController(remoteOf({ workspacePathApplications: async ({ path }) => answers[path]! })) + const signal = new AbortController().signal + expect(controller.cachedApplications('/w/notes.txt')).toBeUndefined() + for (const path of Object.keys(answers)) await controller.applications(path, signal) + expect(controller.cachedApplications('/elsewhere/todo.txt')).toEqual([app('TextEdit')]) + expect(controller.cachedApplications('C:\\work\\archive.tar.txt')).toEqual([app('TextEdit')]) + expect(controller.cachedApplications('/w/NOTES.TXT')).toBeUndefined() + expect(controller.cachedApplications('/w/Makefile')).toEqual([app('Xcode')]) + expect(controller.cachedApplications('/other/Makefile')).toBeUndefined() + expect(controller.cachedApplications('/w/.env')).toEqual([app('Zed')]) + expect(controller.cachedApplications('/w/draft.')).toBeUndefined() + }) + + it('publishes changed answers only, and an earlier query never replaces a later answer', async () => { + const { queries, controller } = deferredRemote() + const signal = new AbortController().signal + const earlier = controller.applications('/w/a.txt', signal) + const later = controller.applications('/w/b.txt', signal) + queries[1]!.answer(answered(app('Sublime Text', 'data:image/png;base64,c3Vi'))) + expect(await later).toEqual([app('Sublime Text', 'data:image/png;base64,c3Vi')]) + const published = controller.associations.getSnapshot() + queries[0]!.answer(answered(app('TextEdit'))) + expect(await earlier).toEqual([app('TextEdit')]) + expect(controller.associations.getSnapshot()).toBe(published) + expect(controller.cachedApplications('/w/a.txt')).toEqual([app('Sublime Text', 'data:image/png;base64,c3Vi')]) + const unchanged = controller.applications('/w/c.txt', signal) + queries[2]!.answer(answered(app('Sublime Text', 'data:image/png;base64,c3Vi'))) + await unchanged + expect(controller.associations.getSnapshot()).toBe(published) + const changed = controller.applications('/w/c.txt', signal) + queries[3]!.answer(answered(app('Sublime Text', 'data:image/png;base64,bmV3'), app('TextEdit'))) + await changed + expect(controller.associations.getSnapshot()).not.toBe(published) + expect(controller.cachedApplications('/w/a.txt')).toEqual([app('Sublime Text', 'data:image/png;base64,bmV3'), app('TextEdit')]) + const other = controller.applications('/w/d.md', signal) + queries[4]!.answer(answered(app('Sublime Text', 'data:image/png;base64,bmV3'))) + await other + expect(controller.cachedApplications('/w/e.md')).toEqual([app('Sublime Text', 'data:image/png;base64,bmV3')]) + expect(controller.cachedApplications('/w/e.txt')).toHaveLength(2) + }) + + it('keeps the recorded answer through refused and rejected queries', async () => { + const workspacePathApplications = vi.fn() + .mockResolvedValueOnce(answered(app('Preview'))) + .mockResolvedValueOnce({ ok: false, error: new RemoteError('gateway/internal', 'boom', {}) }) + .mockRejectedValueOnce(new Error('down')) + const controller = new OpenInAppPathController(remoteOf({ workspacePathApplications })) + const signal = new AbortController().signal + await controller.applications('/w/a.png', signal) + expect(await controller.applications('/w/a.png', signal)).toBeNull() + expect(await controller.applications('/w/a.png', signal)).toBeNull() + expect(controller.cachedApplications('/w/b.png')).toEqual([app('Preview')]) + }) + + it('runs one background refresh per association class until it settles, and disposal cancels it', async () => { + const { queries, controller } = deferredRemote() + controller.refreshApplications('/w/a.txt') + controller.refreshApplications('/w/b.txt') + controller.refreshApplications('/w/c.md') + expect(queries.map(query => query.path)).toEqual(['/w/a.txt', '/w/c.md']) + queries[0]!.answer(answered(app('TextEdit'))) + await vi.waitFor(() => { expect(controller.cachedApplications('/w/b.txt')).toEqual([app('TextEdit')]) }) + controller.refreshApplications('/w/b.txt') + expect(queries.map(query => query.path)).toEqual(['/w/a.txt', '/w/c.md', '/w/b.txt']) + controller.dispose() + expect(queries.every(query => query.signal?.aborted)).toBe(true) + controller.refreshApplications('/w/d.json') + expect(queries).toHaveLength(3) + expect(controller.cachedApplications('/w/a.txt')).toEqual([app('TextEdit')]) + }) +}) From 0eda288f13d124932378cb47bccb08f974d787de Mon Sep 17 00:00:00 2001 From: yudshj Date: Thu, 5 Nov 2026 23:47:41 +0800 Subject: [PATCH 02/44] =?UTF-8?q?fix(open-in-app):=20=E5=90=8E=E5=8F=B0?= =?UTF-8?q?=E5=88=B7=E6=96=B0=E5=90=8C=E4=B8=80=E6=97=B6=E9=97=B4=E5=8F=AA?= =?UTF-8?q?=E8=B7=91=E4=B8=80=E4=B8=AA=EF=BC=8C=E9=AA=8C=E8=AF=81=E6=9C=9F?= =?UTF-8?q?=E9=97=B4=E7=9A=84=E5=9B=9E=E7=AD=94=E4=B8=8D=E5=86=8D=E8=A7=A6?= =?UTF-8?q?=E5=8F=91=E9=87=8D=E5=A4=8D=20stat?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A newer file-menu opening cancels the unfinished background refresh and queries its own file, so a stuck native query cannot block later refreshes and quick openings no longer fan out concurrent queries. Answers published while a menu is still verifying its target no longer restart that verification. README wording now matches the refresh order and documents the remaining cache limitations. --- .../client/ui-open-in-app/README.i18n.yaml | 12 +++--- packages/client/ui-open-in-app/README.md | 6 +-- packages/client/ui-open-in-app/README.zh.md | 6 +-- .../src/client/context-menus.tsx | 9 ++++- .../ui-open-in-app/src/client/open-path.ts | 20 +++++----- .../tests/context-menu-actions.client.spec.ts | 37 +++++++++++++++++-- .../tests/open-path.client.spec.ts | 21 +++++------ 7 files changed, 73 insertions(+), 38 deletions(-) diff --git a/packages/client/ui-open-in-app/README.i18n.yaml b/packages/client/ui-open-in-app/README.i18n.yaml index 5c3f8aa198..1524a8857f 100644 --- a/packages/client/ui-open-in-app/README.i18n.yaml +++ b/packages/client/ui-open-in-app/README.i18n.yaml @@ -18,11 +18,11 @@ en: b149da9963eb9837 zh: 2426ea37fbd0f8b7 /deepseek-ai-dsh-client-ui-open-in-app/use-this-package/what-to-expect: - en: 23fe56e2d62462ac - zh: e703585eac610e47 + en: 4e40e41d89fe99a9 + zh: 818b488e997132f5 /deepseek-ai-dsh-client-ui-open-in-app/understand-the-implementation: - en: 32c651cd37edf401 - zh: c62dff20876e41fe + en: 020fda5b8360fb3d + zh: 7c33f3f45c41e64e /deepseek-ai-dsh-client-ui-open-in-app/further-exploration: en: 720527877e452a97 zh: b2241c7a336a3eeb @@ -33,8 +33,8 @@ en: ca75c51c89c2c9b0 zh: c2241315dc0c89c9 /deepseek-ai-dsh-client-ui-open-in-app/known-limitations-and-deferred-work: - en: bf22c831463d67b6 - zh: 78965275b1bc1179 + en: 2fd6d93cc66f3b42 + zh: 9199fbf041fe2ad4 /deepseek-ai-dsh-client-ui-open-in-app/known-limitations-and-deferred-work/dev-note: en: 84f92568fc74adf1 zh: 286b47e6b5b1a556 diff --git a/packages/client/ui-open-in-app/README.md b/packages/client/ui-open-in-app/README.md index 5faf096be6..a5eecd9344 100644 --- a/packages/client/ui-open-in-app/README.md +++ b/packages/client/ui-open-in-app/README.md @@ -39,7 +39,7 @@ The **Open locally** shortcut captures the main Session’s directory and rememb File menus list the discovered associated applications without a separate “Open in default app” row. “Show file location” stays in a fixed footer separated from the scrolling applications. A successful query with only one available action renders a single button without a dropdown. Initial file-association loading uses a gray skeleton icon. When the Host identifies a default application the menu marks it “(default)” and the main button opens it; otherwise the first associated application takes that place, so the main button reveals the file only when no application is registered. Directory menus have no reveal row. Choosing a file application does not change the system default. An unpreviewable file uses the same menu in a 40px-high split button with an icon and action text; its label is “Open” or “Show file location” according to the default action. -Mounted controls with the same association reader and file share one query and its results; opening either menu refreshes all of them. Releasing the last control cancels the query and discards its state. Cancelled queries cannot replace another file’s results. Every successful answer also replaces the page’s record for the file’s association class: the final name extension, or the whole path when the name has none. A failed query displays a menu message and leaves reveal available as the default action. File-association discovery follows [native-command](../../util/native-command/README.md); an empty result, including a platform without a discovery adapter, uses the same reveal fallback without platform branches in the control. +Mounted controls with the same association reader and file share one query and its results; opening either menu refreshes all of them. Releasing the last control cancels the query and discards its state. Cancelled queries cannot replace another file’s results. A successful answer also replaces the page’s record for the file’s association class unless a later-started query has already answered it; the class is the final name extension, compared case-sensitively, or the whole path when the name has none. A failed query displays a menu message and leaves reveal available as the default action. File-association discovery follows [native-command](../../util/native-command/README.md); an empty result, including a platform without a discovery adapter, uses the same reveal fallback without platform branches in the control. Workspace context menus reveal the captured folder through the shared directory application catalog. File-tree directories expose native reveal and Copy folder path; they do not offer file preview or save. @@ -59,7 +59,7 @@ A file menu offers “Open in sidebar” and “Open in floating panel”, which [`session-open-with.ts`](src/client/session-open-with.ts) shares one resolver across the shared context-menu package’s `session` and `sessionHeader` targets; other contributions do not cross between those targets. The file-tree slot uses `OpenInAppAction` with its owner’s `absolutePath`; there is no independent Session-header opener slot. A page-lifetime [`controller`](src/client/controller.ts) owns availability, persisted choice, and launch POST state. Menu observers and injected component hooks read those same sources. Document-relative routes and payload types come from the Host’s browser-safe `@deepseek-ai/dsh-host-open-in-app/shared` subpath. Adapters resolve native and transport failures to named outcomes; skipped requests announce nothing. -The directory and file adapters supply application metadata and operations to [`OpenTargetButton`](src/client/OpenTargetButton.tsx), which owns menu ordering, default markers, icons, and sizing. Its injected reporter sends outcomes to one feature-owned `shell.overlay` toast, so switching Sessions or closing the initiating preview cannot discard the result. The file header and empty state share `FileOpenTarget`, while `OpenPathInjected.applications` queries `session.workspacePathApplications` through [`open-path.ts`](src/client/open-path.ts). The same controller keeps the page’s answers by association class, runs one background refresh per class, and publishes a new snapshot only when an answer changes. The file menu observes that snapshot and re-resolves from its verified path synchronously, so the context-menu registry keeps the other rows while the application row changes. Opening uses `session.openWorkspacePath`; the Host revalidates an explicitly selected handler before launch. The directory adapter keeps the existing catalog routes. `FileRouteAction` supplies the same control to delivery cards and change review through `deliverables.file.actions` and `deliverables.review.file.actions`; their authenticated routes retain Session file authorization. A failed or unavailable file query therefore needs no platform-specific UI implementation. +The directory and file adapters supply application metadata and operations to [`OpenTargetButton`](src/client/OpenTargetButton.tsx), which owns menu ordering, default markers, icons, and sizing. Its injected reporter sends outcomes to one feature-owned `shell.overlay` toast, so switching Sessions or closing the initiating preview cannot discard the result. The file header and empty state share `FileOpenTarget`, while `OpenPathInjected.applications` queries `session.workspacePathApplications` through [`open-path.ts`](src/client/open-path.ts). The same controller keeps the page’s answers by association class, runs one background refresh at a time, which the next opening cancels, and publishes a new snapshot only when an answer changes. The file menu observes that snapshot and re-resolves from its verified path synchronously, so the context-menu registry keeps the other rows while the application row changes. Opening uses `session.openWorkspacePath`; the Host revalidates an explicitly selected handler before launch. The directory adapter keeps the existing catalog routes. `FileRouteAction` supplies the same control to delivery cards and change review through `deliverables.file.actions` and `deliverables.review.file.actions`; their authenticated routes retain Session file authorization. A failed or unavailable file query therefore needs no platform-specific UI implementation. @@ -93,7 +93,7 @@ The Host verifies the path through the composed filesystem before opening or rev - **The dictionaries gate the menu.** A host catalog extension without a matching `app.` entry in both dictionaries stays invisible instead of showing a raw id; extending the catalog means extending [`dsh-host-open-in-app`](../../host/open-in-app/README.md) and this package's locales together. - **Availability is read once per page.** An application installed while the page is open appears after a reload (and, host-side, after a host restart); the desktop answer behind the file controls is read once per page as well. -- **File associations are cached by type for the page.** Files sharing a final name extension share one answer, so a per-document override, such as a macOS “Open with” choice for one file, or Linux content sniffing and multi-part extensions such as `.tar.gz` can show the type’s previous answer until the background query corrects the open menu. Names without an extension are recorded per path. Records grow with the file types seen and end with the page. +- **File associations are cached by type for the page.** Files sharing a final name extension share one answer. A per-document override, such as a macOS “Open with” choice for one file, can therefore show the type’s previous answer until the background query corrects the open menu; Linux content sniffing and multi-part extensions such as `.tar.gz` have the same effect. Choosing the row before the correction opens the application that answer named. Names without an extension are recorded per path. Records, with one icon per application, grow with the file types seen and end with the page. - **One reveal label for every platform.** The Session Remote reports whether a desktop exists, not which file manager it runs, so the menu says "Show file location" rather than naming Finder or File Explorer as the delivery cards do. - **The delivery cards keep their own opener.** [`ui-deliverables`](../ui-deliverables/README.md) still opens declared files through its own Session-and-event routes; folding those cards onto the file controls here remains deferred. diff --git a/packages/client/ui-open-in-app/README.zh.md b/packages/client/ui-open-in-app/README.zh.md index b0a6177b8f..d4fb2fe9eb 100644 --- a/packages/client/ui-open-in-app/README.zh.md +++ b/packages/client/ui-open-in-app/README.zh.md @@ -39,7 +39,7 @@ kind: "package-reference" 文件菜单列出已发现的关联应用，不单列“用默认应用打开”。“显示文件位置”固定在菜单底部，通过分隔线与滚动的应用列表分开。查询成功且只有一个可用操作时显示单按钮，不再显示下拉箭头。文件关联首次加载时使用灰色骨架图标。Host 识别出默认应用时，菜单将该项标注为“（默认）”，主按钮打开该应用；否则第一个关联应用占据该位置，只有在一个关联应用都没有时主按钮才执行文件定位。目录菜单不包含定位项。选择文件应用不修改系统默认应用。无法预览文件时，空态使用同一菜单，按钮增大到 40px 高并显示图标和动作文字；根据默认动作显示“打开”或“显示文件位置”。 -使用同一查询函数和文件的已挂载控件共用查询与结果，打开任一菜单会刷新所有相关控件。最后一个控件释放后取消查询并清除状态。已取消的查询不会覆盖其他文件的结果。每次查询成功后，结果还会替换本页面中该文件关联类别的记录：类别按文件名的最后一个扩展名划分，没有扩展名的文件按完整路径划分。查询失败时菜单显示提示，并以文件定位作为默认动作。文件关联查询的平台支持范围见 [native-command](../../util/native-command/README.zh.md)；查询结果为空时，包括平台尚无查询适配器的情况，控件统一使用定位动作，不按操作系统分支处理。 +使用同一查询函数和文件的已挂载控件共用查询与结果，打开任一菜单会刷新所有相关控件。最后一个控件释放后取消查询并清除状态。已取消的查询不会覆盖其他文件的结果。查询成功后，结果还会替换本页面中该文件关联类别的记录，除非更晚发起的查询已经给出回答；类别按文件名的最后一个扩展名划分（区分大小写），没有扩展名的文件按完整路径划分。查询失败时菜单显示提示，并以文件定位作为默认动作。文件关联查询的平台支持范围见 [native-command](../../util/native-command/README.zh.md)；查询结果为空时，包括平台尚无查询适配器的情况，控件统一使用定位动作，不按操作系统分支处理。 工作区右键菜单通过共享的目录应用列表定位被点击的文件夹。文件树目录提供原生定位与复制文件夹路径，不提供文件预览或保存。 @@ -59,7 +59,7 @@ kind: "package-reference" [`session-open-with.ts`](src/client/session-open-with.ts) 为共享右键菜单包声明的 `session` 和 `sessionHeader` 目标共用一个解析器；其他贡献不会在这两种目标间扩散。文件树插槽通过所属组件的 `absolutePath` 使用 `OpenInAppAction`；会话头部没有独立打开按钮插槽。页面生命周期的 [`controller`](src/client/controller.ts) 持有可用性、持久化偏好与启动 POST 状态。菜单观察器和组件注入 hooks 读取同一组源。文档相对路由与载荷类型来自 Host 的浏览器安全子路径 `@deepseek-ai/dsh-host-open-in-app/shared`。适配器将原生操作及传输失败解析为具名结果；跳过的请求不播报。 -目录和文件适配器把应用信息与操作交给 [`OpenTargetButton`](src/client/OpenTargetButton.tsx)，由它统一管理菜单顺序、默认标记、图标和尺寸。注入的回调将结果交给本功能在 `shell.overlay` 中持有的唯一 toast，因此切换会话或关闭发起操作的预览不会丢失结果。文件标题栏和空态共用 `FileOpenTarget`，`OpenPathInjected.applications` 通过 [`open-path.ts`](src/client/open-path.ts) 查询 `session.workspacePathApplications`。同一控制器按关联类别保存本页面的回答，每个类别同一时间只运行一次后台刷新，只在回答变化时发布新快照。文件菜单观察该快照，并基于已验证的路径同步重新解析，因此右键菜单注册表保留其他各项，只有应用这一项原地变化。打开操作使用 `session.openWorkspacePath`，Host 在启动前重新验证指定的关联应用。`FileRouteAction` 通过 `deliverables.file.actions` 和 `deliverables.review.file.actions` 为交付卡片和变更对比页提供同一控件，其认证路由保留会话文件校验。目录适配器继续使用已有的应用列表路由，文件查询失败或不可用时无需增加平台专用的界面实现。 +目录和文件适配器把应用信息与操作交给 [`OpenTargetButton`](src/client/OpenTargetButton.tsx)，由它统一管理菜单顺序、默认标记、图标和尺寸。注入的回调将结果交给本功能在 `shell.overlay` 中持有的唯一 toast，因此切换会话或关闭发起操作的预览不会丢失结果。文件标题栏和空态共用 `FileOpenTarget`，`OpenPathInjected.applications` 通过 [`open-path.ts`](src/client/open-path.ts) 查询 `session.workspacePathApplications`。同一控制器按关联类别保存本页面的回答，同一时间只运行一次后台刷新（下一次打开菜单会取消尚未完成的刷新），只在回答变化时发布新快照。文件菜单观察该快照，并基于已验证的路径同步重新解析，因此右键菜单注册表保留其他各项，只有应用这一项原地变化。打开操作使用 `session.openWorkspacePath`，Host 在启动前重新验证指定的关联应用。`FileRouteAction` 通过 `deliverables.file.actions` 和 `deliverables.review.file.actions` 为交付卡片和变更对比页提供同一控件，其认证路由保留会话文件校验。目录适配器继续使用已有的应用列表路由，文件查询失败或不可用时无需增加平台专用的界面实现。 @@ -93,7 +93,7 @@ Host 在打开或定位前通过当前文件系统验证路径。没有对应 Ho - **词典把守菜单。** 主机目录的新条目若在两份词典中没有对应的 `app.` 条目，将保持不可见而不是显示裸 id；扩展目录意味着同时扩展 [`dsh-host-open-in-app`](../../host/open-in-app/README.zh.md) 与本包的 locale。 - **可用性每页只读一次。** 页面打开期间安装的应用要重新加载页面后才出现（主机侧还需主机重启）；文件控件背后的桌面回答同样每页只读一次。 -- **文件关联按类型在页面内缓存。** 最后一个扩展名相同的文件共用一个回答，因此单个文件的单独设置（例如 macOS 中只针对某个文件的“打开方式”），或 Linux 的内容嗅探与 `.tar.gz` 这类多段扩展名，可能先显示该类型之前的回答，再由后台查询更正已打开的菜单。没有扩展名的文件按路径记录。记录随出现过的文件类型增长，并在页面关闭时释放。 +- **文件关联按类型在页面内缓存。** 最后一个扩展名相同的文件共用一个回答。因此单个文件的单独设置（例如 macOS 中只针对某个文件的“打开方式”）可能先显示该类型之前的回答，再由后台查询更正已打开的菜单；Linux 的内容嗅探和 `.tar.gz` 这类多段扩展名也有同样的效果。在更正之前选择这一项，会用该回答中的应用打开。没有扩展名的文件按路径记录。记录以及每个应用的一份图标随出现过的文件类型增长，并在页面关闭时释放。 - **所有平台共用一个定位标签。** Session Remote 只报告有没有桌面，不报告它跑的是哪个文件管理器，所以菜单写「显示文件位置」，而不像交付卡片那样点名访达或文件资源管理器。 - **交付卡片保留自己的打开器。** [`ui-deliverables`](../ui-deliverables/README.zh.md) 仍通过自己按 Session 与事件定位的路由打开声明过的文件；把这些卡片并到这里的文件控件上仍属延期工作。 diff --git a/packages/client/ui-open-in-app/src/client/context-menus.tsx b/packages/client/ui-open-in-app/src/client/context-menus.tsx index 6ee7d48cca..2ec13942e7 100644 --- a/packages/client/ui-open-in-app/src/client/context-menus.tsx +++ b/packages/client/ui-open-in-app/src/client/context-menus.tsx @@ -111,13 +111,18 @@ export function registerContextMenus( } // The verified Host path of each open file menu. Association answers re-resolve the menu from it // synchronously, so the registry keeps the other rows while the application row changes in place. + // This relies on the registry subscribing once per occurrence before its first resolution, and on a + // synchronous resolution replacing rows without clearing them first. const verified = new WeakMap() ctx.effect(() => ctx.contextMenus.register({ id: 'open-in-app.file', target: 'file', order: 100, observe: (target) => { - // Each occurrence subscribes before its first resolution, so a reopened target is verified again. verified.set(target, null) - return [paths.associations] + // The first resolution reads the newest answers once verified; answers arriving earlier need no re-resolution. + return [{ + getSnapshot: paths.associations.getSnapshot.bind(paths.associations), + subscribe: (listener: () => void) => paths.associations.subscribe(() => { if (verified.get(target)) listener() }), + }] }, resolve: (target, signal) => { const settled = verified.get(target) diff --git a/packages/client/ui-open-in-app/src/client/open-path.ts b/packages/client/ui-open-in-app/src/client/open-path.ts index bc9fb4c808..b0c70ba38f 100644 --- a/packages/client/ui-open-in-app/src/client/open-path.ts +++ b/packages/client/ui-open-in-app/src/client/open-path.ts @@ -20,7 +20,7 @@ function associationKey(path: string): string { return dot > 0 && dot < name.length - 1 ? `extension:${name.slice(dot)}` : `path:${path}` } -/** Compare two answers field by field, including icon data. */ +/** Equal answers keep the published snapshot, so observers re-resolve only when an answer changes. */ function sameApplications(left: Applications, right: Applications): boolean { return left.length === right.length && left.every((app, index) => { const other = right[index] @@ -58,10 +58,11 @@ export class OpenInAppPathController { readonly associations: ObservableSnapshot> = this.answers private loading: Promise | undefined private readonly lifetime = new AbortController() - private readonly refreshing = new Set() + /** The background refresh in flight; a newer refresh cancels it. */ + private refresh: AbortController | undefined /** Start order of the query whose answer each class holds; an earlier query never replaces a later one. */ private readonly answeredBy = new Map() - /** One icon string per application id, reused by every class that lists the application. */ + /** One icon string per application id, reused by every class that lists the application so its data URL is held once. */ private readonly icons = new Map() private queries = 0 @@ -131,15 +132,16 @@ export class OpenInAppPathController { } /** - * Re-query a file's associations in the background. One refresh per association class runs at a time; - * it outlives the requesting menu and ends with its answer or {@link dispose}. + * Re-query a file's associations in the background. One refresh runs at a time: it outlives the + * requesting menu and ends with its answer, the next refresh, which cancels it, or {@link dispose}. * @param path - absolute Host path of the file. */ refreshApplications(path: string): void { - const key = associationKey(path) - if (this.lifetime.signal.aborted || this.refreshing.has(key)) return - this.refreshing.add(key) - void this.applications(path, this.lifetime.signal).finally(() => { this.refreshing.delete(key) }) + if (this.lifetime.signal.aborted) return + this.refresh?.abort() + const refresh = new AbortController() + this.refresh = refresh + void this.applications(path, AbortSignal.any([this.lifetime.signal, refresh.signal])) } /** Cancel background refreshes and refuse new ones; recorded answers stay readable. */ diff --git a/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts b/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts index b3d7d5a319..2ea3d500a2 100644 --- a/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts +++ b/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts @@ -18,6 +18,8 @@ import type { OpenInAppToastSource } from '../src/client/open-toast.tsx' const sessionId = SessionId('menu-actions') type Applications = readonly SessionWorkspacePathApplication[] +/** Let settled Remote answers finish their queued continuations. */ +const settle = () => new Promise((resolve) => { setTimeout(resolve, 0) }) afterEach(() => { vi.restoreAllMocks(); vi.unstubAllGlobals() }) async function boot() { @@ -172,16 +174,41 @@ it('opens file menus before the Host answers and changes only the application ro it('opens through the OS association until the Host first answers, including after failed queries', async () => { const h = await boot() - h.remote.session.workspacePathApplications.mockResolvedValue({ ok: false } as never) + let refuse = () => {} + h.remote.session.workspacePathApplications.mockImplementationOnce(() => new Promise((answer) => { + refuse = () => { answer({ ok: false } as never) } + })) const menu = await h.resolve({ kind: 'file', value: { sessionId, path: 'notes.md' } }) - await vi.waitFor(() => { expect(h.remote.session.workspacePathApplications).toHaveBeenCalledOnce() }) - const application = menu.source.getSnapshot().find(item => item.id === 'open-in-app.file/open') - expect(application).toMatchObject({ label: 'menu.defaultApp', disabled: false }) + const application = () => menu.source.getSnapshot().find(item => item.id === 'open-in-app.file/open') + expect(application()).toMatchObject({ label: 'menu.defaultApp', disabled: false }) + refuse() + await settle() + expect(h.paths.cachedApplications('/workspace/notes.md')).toBeUndefined() + expect(application()).toMatchObject({ label: 'menu.defaultApp', disabled: false }) menu.select('open-in-app.file/open') await vi.waitFor(() => { expect(h.remote.session.openWorkspacePath).toHaveBeenLastCalledWith({ path: '/workspace/notes.md' }) }) await vi.waitFor(() => { expect(h.report).toHaveBeenLastCalledWith({ kind: 'opened' }) }) }) +it('verifies a file menu once when answers for other files arrive during verification', async () => { + const h = await boot() + let release = () => {} + const verifying = new Promise((resolve) => { release = resolve }) + h.remote.workspaceFiles.stat.mockImplementation(async (_sessionId: SessionId, path: string) => { + await verifying + return { ok: true as const, value: { absolutePath: `/workspace/${path}` } } + }) + const menu = h.registry.resolve({ kind: 'file', value: { sessionId, path: 'notes.txt' } }) + await h.paths.applications('/workspace/other.md', new AbortController().signal) + expect(h.paths.cachedApplications('/workspace/other.md')).toHaveLength(1) + release() + await menu.ready + expect(h.remote.workspaceFiles.stat).toHaveBeenCalledTimes(2) + await vi.waitFor(() => { expect(menu.source.getSnapshot().find(item => item.id === 'open-in-app.file/open')?.label).toBe('open.title') }) + expect(h.remote.workspaceFiles.stat).toHaveBeenCalledTimes(2) + menu.dispose() +}) + it('keeps copy-path available on failed metadata and reports native action failures', async () => { const h = await boot() const file = { kind: 'file', value: { sessionId, path: 'file.svg' } } as const @@ -202,6 +229,8 @@ it('keeps copy-path available on failed metadata and reports native action failu await vi.waitFor(() => { expect(h.remote.session.workspacePathApplications.mock.settledResults.every(result => result.type !== 'incomplete')).toBe(true) }) + await settle() + expect(h.paths.cachedApplications('/workspace/file.svg')).toHaveLength(1) expect(kept.source.getSnapshot().find(item => item.id.endsWith('/open'))).toMatchObject({ label: 'open.title', disabled: false }) kept.dispose() h.paths.desktop.set(false) diff --git a/packages/client/ui-open-in-app/tests/open-path.client.spec.ts b/packages/client/ui-open-in-app/tests/open-path.client.spec.ts index 8eb4362032..af6f3475b7 100644 --- a/packages/client/ui-open-in-app/tests/open-path.client.spec.ts +++ b/packages/client/ui-open-in-app/tests/open-path.client.spec.ts @@ -99,11 +99,12 @@ describe('OpenInAppPathController association answers', () => { type Answer = Awaited> const app = (name: string, icon: string | null = null) => ({ id: `/Applications/${name}.app`, name, default: true, icon }) const answered = (...apps: ReturnType[]): Answer => ({ ok: true, value: apps }) - /** A Remote whose association queries settle when the test answers them. */ + /** A Remote whose association queries settle when the test answers them, and reject once cancelled. */ function deferredRemote() { const queries: { path: string; signal: AbortSignal | undefined; answer: (result: Answer) => void }[] = [] - const workspacePathApplications = vi.fn((request: { path: string }, signal?: AbortSignal) => new Promise((answer) => { + const workspacePathApplications = vi.fn((request: { path: string }, signal?: AbortSignal) => new Promise((answer, reject) => { queries.push({ path: request.path, signal, answer }) + signal?.addEventListener('abort', () => { reject(new Error('cancelled')) }, { once: true }) })) return { queries, controller: new OpenInAppPathController(remoteOf({ workspacePathApplications })) } } @@ -166,19 +167,17 @@ describe('OpenInAppPathController association answers', () => { expect(controller.cachedApplications('/w/b.png')).toEqual([app('Preview')]) }) - it('runs one background refresh per association class until it settles, and disposal cancels it', async () => { + it('runs one background refresh at a time, cancelling the older for the newer file, and disposal cancels it', async () => { const { queries, controller } = deferredRemote() controller.refreshApplications('/w/a.txt') controller.refreshApplications('/w/b.txt') - controller.refreshApplications('/w/c.md') - expect(queries.map(query => query.path)).toEqual(['/w/a.txt', '/w/c.md']) - queries[0]!.answer(answered(app('TextEdit'))) - await vi.waitFor(() => { expect(controller.cachedApplications('/w/b.txt')).toEqual([app('TextEdit')]) }) - controller.refreshApplications('/w/b.txt') - expect(queries.map(query => query.path)).toEqual(['/w/a.txt', '/w/c.md', '/w/b.txt']) + expect(queries.map(query => [query.path, query.signal?.aborted])).toEqual([['/w/a.txt', true], ['/w/b.txt', false]]) + queries[1]!.answer(answered(app('TextEdit'))) + await vi.waitFor(() => { expect(controller.cachedApplications('/w/c.txt')).toEqual([app('TextEdit')]) }) + controller.refreshApplications('/w/d.md') controller.dispose() - expect(queries.every(query => query.signal?.aborted)).toBe(true) - controller.refreshApplications('/w/d.json') + expect(queries[2]).toMatchObject({ path: '/w/d.md', signal: { aborted: true } }) + controller.refreshApplications('/w/e.json') expect(queries).toHaveLength(3) expect(controller.cachedApplications('/w/a.txt')).toEqual([app('TextEdit')]) }) From ea7c21c8a47dba0814ef3e6e13af8b4316a8d261 Mon Sep 17 00:00:00 2001 From: yudshj Date: Fri, 6 Nov 2026 09:01:29 +0800 Subject: [PATCH 03/44] =?UTF-8?q?fix(open-in-app):=20=E6=AF=8F=E6=AC=A1?= =?UTF-8?q?=E6=89=93=E5=BC=80=E6=96=87=E4=BB=B6=E8=8F=9C=E5=8D=95=E9=83=BD?= =?UTF-8?q?=E6=A0=A1=E9=AA=8C=E8=B7=AF=E5=BE=84=EF=BC=8C=E5=B7=B2=E6=89=93?= =?UTF-8?q?=E5=BC=80=E7=9A=84=E8=8F=9C=E5=8D=95=E5=8F=AA=E5=93=8D=E5=BA=94?= =?UTF-8?q?=E6=9C=AC=E7=B1=BB=E5=9E=8B=E7=9A=84=E5=9B=9E=E7=AD=94?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A file menu now reuses its verified Host path only while its own association subscription re-resolves it synchronously, and the verification record is cleared when a stat starts instead of when the registry observes. Every opening therefore runs its own stat whatever order the context-menu registry observes and resolves in; a registry change can at most cost another stat or a flicker. An open menu re-resolves only when its own file type's answer changes. The answer comparison lists every application field and fails to compile when the type gains one. README documents the remaining cache limitations. --- .../client/ui-open-in-app/README.i18n.yaml | 12 ++--- packages/client/ui-open-in-app/README.md | 7 +-- packages/client/ui-open-in-app/README.zh.md | 7 +-- .../src/client/context-menus.tsx | 51 ++++++++++++------- .../ui-open-in-app/src/client/open-path.ts | 10 ++-- .../tests/context-menu-actions.client.spec.ts | 22 ++++++++ 6 files changed, 76 insertions(+), 33 deletions(-) diff --git a/packages/client/ui-open-in-app/README.i18n.yaml b/packages/client/ui-open-in-app/README.i18n.yaml index 1524a8857f..a38fe57e45 100644 --- a/packages/client/ui-open-in-app/README.i18n.yaml +++ b/packages/client/ui-open-in-app/README.i18n.yaml @@ -18,11 +18,11 @@ en: b149da9963eb9837 zh: 2426ea37fbd0f8b7 /deepseek-ai-dsh-client-ui-open-in-app/use-this-package/what-to-expect: - en: 4e40e41d89fe99a9 - zh: 818b488e997132f5 + en: 53d5d74d5b1d9b66 + zh: bdd9806dca51e957 /deepseek-ai-dsh-client-ui-open-in-app/understand-the-implementation: - en: 020fda5b8360fb3d - zh: 7c33f3f45c41e64e + en: c26d8937f5d38119 + zh: d353c2377b9747aa /deepseek-ai-dsh-client-ui-open-in-app/further-exploration: en: 720527877e452a97 zh: b2241c7a336a3eeb @@ -33,8 +33,8 @@ en: ca75c51c89c2c9b0 zh: c2241315dc0c89c9 /deepseek-ai-dsh-client-ui-open-in-app/known-limitations-and-deferred-work: - en: 2fd6d93cc66f3b42 - zh: 9199fbf041fe2ad4 + en: f206d2fefa52f81d + zh: 51025a6e19d01295 /deepseek-ai-dsh-client-ui-open-in-app/known-limitations-and-deferred-work/dev-note: en: 84f92568fc74adf1 zh: 286b47e6b5b1a556 diff --git a/packages/client/ui-open-in-app/README.md b/packages/client/ui-open-in-app/README.md index a5eecd9344..5ab8a3894f 100644 --- a/packages/client/ui-open-in-app/README.md +++ b/packages/client/ui-open-in-app/README.md @@ -39,7 +39,7 @@ The **Open locally** shortcut captures the main Session’s directory and rememb File menus list the discovered associated applications without a separate “Open in default app” row. “Show file location” stays in a fixed footer separated from the scrolling applications. A successful query with only one available action renders a single button without a dropdown. Initial file-association loading uses a gray skeleton icon. When the Host identifies a default application the menu marks it “(default)” and the main button opens it; otherwise the first associated application takes that place, so the main button reveals the file only when no application is registered. Directory menus have no reveal row. Choosing a file application does not change the system default. An unpreviewable file uses the same menu in a 40px-high split button with an icon and action text; its label is “Open” or “Show file location” according to the default action. -Mounted controls with the same association reader and file share one query and its results; opening either menu refreshes all of them. Releasing the last control cancels the query and discards its state. Cancelled queries cannot replace another file’s results. A successful answer also replaces the page’s record for the file’s association class unless a later-started query has already answered it; the class is the final name extension, compared case-sensitively, or the whole path when the name has none. A failed query displays a menu message and leaves reveal available as the default action. File-association discovery follows [native-command](../../util/native-command/README.md); an empty result, including a platform without a discovery adapter, uses the same reveal fallback without platform branches in the control. +Mounted controls with the same association reader and file share one query and its results; opening either menu refreshes all of them. Releasing the last control cancels the query and discards its state. Cancelled queries cannot replace another file’s results. A successful answer also replaces the page’s record for the file’s association class unless a later-started query has already answered it; the class is the final name extension, compared case-sensitively, or the whole path when the name has none. Mounted controls do not read that record and keep their own answer until their menu opens again, so the file header and a context menu can name different applications for one file meanwhile. A failed query displays a menu message and leaves reveal available as the default action. File-association discovery follows [native-command](../../util/native-command/README.md); an empty result, including a platform without a discovery adapter, uses the same reveal fallback without platform branches in the control. Workspace context menus reveal the captured folder through the shared directory application catalog. File-tree directories expose native reveal and Copy folder path; they do not offer file preview or save. @@ -59,7 +59,7 @@ A file menu offers “Open in sidebar” and “Open in floating panel”, which [`session-open-with.ts`](src/client/session-open-with.ts) shares one resolver across the shared context-menu package’s `session` and `sessionHeader` targets; other contributions do not cross between those targets. The file-tree slot uses `OpenInAppAction` with its owner’s `absolutePath`; there is no independent Session-header opener slot. A page-lifetime [`controller`](src/client/controller.ts) owns availability, persisted choice, and launch POST state. Menu observers and injected component hooks read those same sources. Document-relative routes and payload types come from the Host’s browser-safe `@deepseek-ai/dsh-host-open-in-app/shared` subpath. Adapters resolve native and transport failures to named outcomes; skipped requests announce nothing. -The directory and file adapters supply application metadata and operations to [`OpenTargetButton`](src/client/OpenTargetButton.tsx), which owns menu ordering, default markers, icons, and sizing. Its injected reporter sends outcomes to one feature-owned `shell.overlay` toast, so switching Sessions or closing the initiating preview cannot discard the result. The file header and empty state share `FileOpenTarget`, while `OpenPathInjected.applications` queries `session.workspacePathApplications` through [`open-path.ts`](src/client/open-path.ts). The same controller keeps the page’s answers by association class, runs one background refresh at a time, which the next opening cancels, and publishes a new snapshot only when an answer changes. The file menu observes that snapshot and re-resolves from its verified path synchronously, so the context-menu registry keeps the other rows while the application row changes. Opening uses `session.openWorkspacePath`; the Host revalidates an explicitly selected handler before launch. The directory adapter keeps the existing catalog routes. `FileRouteAction` supplies the same control to delivery cards and change review through `deliverables.file.actions` and `deliverables.review.file.actions`; their authenticated routes retain Session file authorization. A failed or unavailable file query therefore needs no platform-specific UI implementation. +The directory and file adapters supply application metadata and operations to [`OpenTargetButton`](src/client/OpenTargetButton.tsx), which owns menu ordering, default markers, icons, and sizing. Its injected reporter sends outcomes to one feature-owned `shell.overlay` toast, so switching Sessions or closing the initiating preview cannot discard the result. The file header and empty state share `FileOpenTarget`, while `OpenPathInjected.applications` queries `session.workspacePathApplications` through [`open-path.ts`](src/client/open-path.ts). The same controller keeps the page’s answers by association class, runs one background refresh at a time, which the next opening or plugin unload cancels, and publishes a new snapshot only when an answer changes. The file menu observes that snapshot; when its own file type’s answer changes, it re-resolves synchronously from the path its opening verified, so the context-menu registry keeps the other rows while the application row changes. Opening uses `session.openWorkspacePath`; the Host revalidates an explicitly selected handler before launch. The directory adapter keeps the existing catalog routes. `FileRouteAction` supplies the same control to delivery cards and change review through `deliverables.file.actions` and `deliverables.review.file.actions`; their authenticated routes retain Session file authorization. A failed or unavailable file query therefore needs no platform-specific UI implementation. @@ -93,7 +93,8 @@ The Host verifies the path through the composed filesystem before opening or rev - **The dictionaries gate the menu.** A host catalog extension without a matching `app.` entry in both dictionaries stays invisible instead of showing a raw id; extending the catalog means extending [`dsh-host-open-in-app`](../../host/open-in-app/README.md) and this package's locales together. - **Availability is read once per page.** An application installed while the page is open appears after a reload (and, host-side, after a host restart); the desktop answer behind the file controls is read once per page as well. -- **File associations are cached by type for the page.** Files sharing a final name extension share one answer. A per-document override, such as a macOS “Open with” choice for one file, can therefore show the type’s previous answer until the background query corrects the open menu; Linux content sniffing and multi-part extensions such as `.tar.gz` have the same effect. Choosing the row before the correction opens the application that answer named. Names without an extension are recorded per path. Records, with one icon per application, grow with the file types seen and end with the page. +- **File associations are cached by type for the page.** Files sharing a final name extension share one answer; extensions that differ only in case, such as `.PNG` and `.png`, are separate types. A per-document override, such as a macOS “Open with” choice for one file, can therefore show the type’s previous answer until the background query corrects the open menu. Linux content sniffing and multi-part extensions such as `.tar.gz` have the same effect, and one file’s empty answer shows the row disabled for the type’s other files the same way. Choosing the row before the correction opens the application that answer named. Names without an extension are recorded per path. Records, with one icon per application, grow with the file types seen and end with the page. +- **One background refresh at a time.** Opening another file menu cancels the previous menu’s unanswered query, so that type keeps its earlier answer, or none, until its next opening. - **One reveal label for every platform.** The Session Remote reports whether a desktop exists, not which file manager it runs, so the menu says "Show file location" rather than naming Finder or File Explorer as the delivery cards do. - **The delivery cards keep their own opener.** [`ui-deliverables`](../ui-deliverables/README.md) still opens declared files through its own Session-and-event routes; folding those cards onto the file controls here remains deferred. diff --git a/packages/client/ui-open-in-app/README.zh.md b/packages/client/ui-open-in-app/README.zh.md index d4fb2fe9eb..dd65ffdb58 100644 --- a/packages/client/ui-open-in-app/README.zh.md +++ b/packages/client/ui-open-in-app/README.zh.md @@ -39,7 +39,7 @@ kind: "package-reference" 文件菜单列出已发现的关联应用，不单列“用默认应用打开”。“显示文件位置”固定在菜单底部，通过分隔线与滚动的应用列表分开。查询成功且只有一个可用操作时显示单按钮，不再显示下拉箭头。文件关联首次加载时使用灰色骨架图标。Host 识别出默认应用时，菜单将该项标注为“（默认）”，主按钮打开该应用；否则第一个关联应用占据该位置，只有在一个关联应用都没有时主按钮才执行文件定位。目录菜单不包含定位项。选择文件应用不修改系统默认应用。无法预览文件时，空态使用同一菜单，按钮增大到 40px 高并显示图标和动作文字；根据默认动作显示“打开”或“显示文件位置”。 -使用同一查询函数和文件的已挂载控件共用查询与结果，打开任一菜单会刷新所有相关控件。最后一个控件释放后取消查询并清除状态。已取消的查询不会覆盖其他文件的结果。查询成功后，结果还会替换本页面中该文件关联类别的记录，除非更晚发起的查询已经给出回答；类别按文件名的最后一个扩展名划分（区分大小写），没有扩展名的文件按完整路径划分。查询失败时菜单显示提示，并以文件定位作为默认动作。文件关联查询的平台支持范围见 [native-command](../../util/native-command/README.zh.md)；查询结果为空时，包括平台尚无查询适配器的情况，控件统一使用定位动作，不按操作系统分支处理。 +使用同一查询函数和文件的已挂载控件共用查询与结果，打开任一菜单会刷新所有相关控件。最后一个控件释放后取消查询并清除状态。已取消的查询不会覆盖其他文件的结果。查询成功后，结果还会替换本页面中该文件关联类别的记录，除非更晚发起的查询已经给出回答；类别按文件名的最后一个扩展名划分（区分大小写），没有扩展名的文件按完整路径划分。已挂载控件不读取这份记录，会保留自己的回答，直到再次打开其菜单，因此在此期间同一个文件在文件标题栏和右键菜单中可能显示不同的应用。查询失败时菜单显示提示，并以文件定位作为默认动作。文件关联查询的平台支持范围见 [native-command](../../util/native-command/README.zh.md)；查询结果为空时，包括平台尚无查询适配器的情况，控件统一使用定位动作，不按操作系统分支处理。 工作区右键菜单通过共享的目录应用列表定位被点击的文件夹。文件树目录提供原生定位与复制文件夹路径，不提供文件预览或保存。 @@ -59,7 +59,7 @@ kind: "package-reference" [`session-open-with.ts`](src/client/session-open-with.ts) 为共享右键菜单包声明的 `session` 和 `sessionHeader` 目标共用一个解析器；其他贡献不会在这两种目标间扩散。文件树插槽通过所属组件的 `absolutePath` 使用 `OpenInAppAction`；会话头部没有独立打开按钮插槽。页面生命周期的 [`controller`](src/client/controller.ts) 持有可用性、持久化偏好与启动 POST 状态。菜单观察器和组件注入 hooks 读取同一组源。文档相对路由与载荷类型来自 Host 的浏览器安全子路径 `@deepseek-ai/dsh-host-open-in-app/shared`。适配器将原生操作及传输失败解析为具名结果；跳过的请求不播报。 -目录和文件适配器把应用信息与操作交给 [`OpenTargetButton`](src/client/OpenTargetButton.tsx)，由它统一管理菜单顺序、默认标记、图标和尺寸。注入的回调将结果交给本功能在 `shell.overlay` 中持有的唯一 toast，因此切换会话或关闭发起操作的预览不会丢失结果。文件标题栏和空态共用 `FileOpenTarget`，`OpenPathInjected.applications` 通过 [`open-path.ts`](src/client/open-path.ts) 查询 `session.workspacePathApplications`。同一控制器按关联类别保存本页面的回答，同一时间只运行一次后台刷新（下一次打开菜单会取消尚未完成的刷新），只在回答变化时发布新快照。文件菜单观察该快照，并基于已验证的路径同步重新解析，因此右键菜单注册表保留其他各项，只有应用这一项原地变化。打开操作使用 `session.openWorkspacePath`，Host 在启动前重新验证指定的关联应用。`FileRouteAction` 通过 `deliverables.file.actions` 和 `deliverables.review.file.actions` 为交付卡片和变更对比页提供同一控件，其认证路由保留会话文件校验。目录适配器继续使用已有的应用列表路由，文件查询失败或不可用时无需增加平台专用的界面实现。 +目录和文件适配器把应用信息与操作交给 [`OpenTargetButton`](src/client/OpenTargetButton.tsx)，由它统一管理菜单顺序、默认标记、图标和尺寸。注入的回调将结果交给本功能在 `shell.overlay` 中持有的唯一 toast，因此切换会话或关闭发起操作的预览不会丢失结果。文件标题栏和空态共用 `FileOpenTarget`，`OpenPathInjected.applications` 通过 [`open-path.ts`](src/client/open-path.ts) 查询 `session.workspacePathApplications`。同一控制器按关联类别保存本页面的回答，同一时间只运行一次后台刷新（下一次打开菜单或插件卸载会取消尚未完成的刷新），只在回答变化时发布新快照。文件菜单观察该快照；只有本文件类型的回答变化时，才基于本次打开时验证的路径同步重新解析，因此右键菜单注册表保留其他各项，只有应用这一项原地变化。打开操作使用 `session.openWorkspacePath`，Host 在启动前重新验证指定的关联应用。`FileRouteAction` 通过 `deliverables.file.actions` 和 `deliverables.review.file.actions` 为交付卡片和变更对比页提供同一控件，其认证路由保留会话文件校验。目录适配器继续使用已有的应用列表路由，文件查询失败或不可用时无需增加平台专用的界面实现。 @@ -93,7 +93,8 @@ Host 在打开或定位前通过当前文件系统验证路径。没有对应 Ho - **词典把守菜单。** 主机目录的新条目若在两份词典中没有对应的 `app.` 条目，将保持不可见而不是显示裸 id；扩展目录意味着同时扩展 [`dsh-host-open-in-app`](../../host/open-in-app/README.zh.md) 与本包的 locale。 - **可用性每页只读一次。** 页面打开期间安装的应用要重新加载页面后才出现（主机侧还需主机重启）；文件控件背后的桌面回答同样每页只读一次。 -- **文件关联按类型在页面内缓存。** 最后一个扩展名相同的文件共用一个回答。因此单个文件的单独设置（例如 macOS 中只针对某个文件的“打开方式”）可能先显示该类型之前的回答，再由后台查询更正已打开的菜单；Linux 的内容嗅探和 `.tar.gz` 这类多段扩展名也有同样的效果。在更正之前选择这一项，会用该回答中的应用打开。没有扩展名的文件按路径记录。记录以及每个应用的一份图标随出现过的文件类型增长，并在页面关闭时释放。 +- **文件关联按类型在页面内缓存。** 最后一个扩展名相同的文件共用一个回答；只有大小写不同的扩展名（如 `.PNG` 和 `.png`）算作不同类型。因此单个文件的单独设置（例如 macOS 中只针对某个文件的“打开方式”）可能先显示该类型之前的回答，再由后台查询更正已打开的菜单。Linux 的内容嗅探和 `.tar.gz` 这类多段扩展名也有同样的效果；某个文件的空回答也会以同样的方式，让同类型其他文件的这一项先显示为禁用。在更正之前选择这一项，会用该回答中的应用打开。没有扩展名的文件按路径记录。记录以及每个应用的一份图标随出现过的文件类型增长，并在页面关闭时释放。 +- **同一时间只有一次后台刷新。** 打开另一个文件的菜单会取消前一个菜单尚未返回的查询，该类型保留之前的回答（或仍没有回答），直到下次打开。 - **所有平台共用一个定位标签。** Session Remote 只报告有没有桌面，不报告它跑的是哪个文件管理器，所以菜单写「显示文件位置」，而不像交付卡片那样点名访达或文件资源管理器。 - **交付卡片保留自己的打开器。** [`ui-deliverables`](../ui-deliverables/README.zh.md) 仍通过自己按 Session 与事件定位的路由打开声明过的文件；把这些卡片并到这里的文件控件上仍属延期工作。 diff --git a/packages/client/ui-open-in-app/src/client/context-menus.tsx b/packages/client/ui-open-in-app/src/client/context-menus.tsx index 2ec13942e7..76e518ae3d 100644 --- a/packages/client/ui-open-in-app/src/client/context-menus.tsx +++ b/packages/client/ui-open-in-app/src/client/context-menus.tsx @@ -109,31 +109,46 @@ export function registerContextMenus( } }, ] } - // The verified Host path of each open file menu. Association answers re-resolve the menu from it - // synchronously, so the registry keeps the other rows while the application row changes in place. - // This relies on the registry subscribing once per occurrence before its first resolution, and on a - // synchronous resolution replacing rows without clearing them first. - const verified = new WeakMap() + // The Host path from each file target's latest successful stat, shared by menus open on one target object at once. + // A stat clears it while running; answers arriving meanwhile need no re-resolution, as that stat's resolution reads + // the newest answers. + const verified = new WeakMap() + // The target whose changed association answer is being applied; only that re-resolution reuses the verified path, + // so every opening runs its own stat. The in-place update relies on the registry re-resolving synchronously from + // the notification, without which the menu stats again, and on a synchronous resolution replacing rows without + // clearing them first, without which the rows flicker. + let answering: ContextMenuTargetMap['file'] | undefined ctx.effect(() => ctx.contextMenus.register({ id: 'open-in-app.file', target: 'file', order: 100, - observe: (target) => { - verified.set(target, null) - // The first resolution reads the newest answers once verified; answers arriving earlier need no re-resolution. - return [{ - getSnapshot: paths.associations.getSnapshot.bind(paths.associations), - subscribe: (listener: () => void) => paths.associations.subscribe(() => { if (verified.get(target)) listener() }), - }] - }, + observe: target => [{ + getSnapshot: paths.associations.getSnapshot.bind(paths.associations), + subscribe: (listener: () => void) => { + // Answers for other file types leave this menu unchanged. + let shown: ReturnType + return paths.associations.subscribe(() => { + const absolutePath = verified.get(target) + if (absolutePath === undefined) return + const applications = paths.cachedApplications(absolutePath) + if (applications === shown) return + shown = applications + answering = target + try { listener() } finally { answering = undefined } + }) + }, + }], resolve: (target, signal) => { - const settled = verified.get(target) - if (settled) return fileMenu(target, settled.absolutePath) + const verifiedPath = answering === target ? verified.get(target) : undefined + if (verifiedPath !== undefined) return fileMenu(target, verifiedPath) + verified.delete(target) return (async () => { await paths.load() const absolutePath = await resolvePath(target.sessionId, target.path, signal) signal.throwIfAborted() - verified.set(target, { absolutePath }) - // A cached answer shows at once; every opening re-queries so a changed association reaches this menu. - if (absolutePath !== undefined && paths.desktop.getSnapshot() === true) paths.refreshApplications(absolutePath) + if (absolutePath !== undefined) { + verified.set(target, absolutePath) + // A cached answer shows at once; every opening re-queries so a changed association reaches this menu. + if (paths.desktop.getSnapshot() === true) paths.refreshApplications(absolutePath) + } return fileMenu(target, absolutePath) })() }, diff --git a/packages/client/ui-open-in-app/src/client/open-path.ts b/packages/client/ui-open-in-app/src/client/open-path.ts index b0c70ba38f..ba17321fca 100644 --- a/packages/client/ui-open-in-app/src/client/open-path.ts +++ b/packages/client/ui-open-in-app/src/client/open-path.ts @@ -20,12 +20,16 @@ function associationKey(path: string): string { return dot > 0 && dot < name.length - 1 ? `extension:${name.slice(dot)}` : `path:${path}` } +/** The compared application fields; a field added to the type fails to compile until it is listed here. */ +const APPLICATION_FIELDS = Object.keys({ + id: true, name: true, default: true, icon: true, +} satisfies Record) as readonly (keyof SessionWorkspacePathApplication)[] + /** Equal answers keep the published snapshot, so observers re-resolve only when an answer changes. */ function sameApplications(left: Applications, right: Applications): boolean { return left.length === right.length && left.every((app, index) => { const other = right[index] - return other !== undefined && app.id === other.id && app.name === other.name - && app.default === other.default && app.icon === other.icon + return other !== undefined && APPLICATION_FIELDS.every(field => app[field] === other[field]) }) } @@ -58,7 +62,7 @@ export class OpenInAppPathController { readonly associations: ObservableSnapshot> = this.answers private loading: Promise | undefined private readonly lifetime = new AbortController() - /** The background refresh in flight; a newer refresh cancels it. */ + /** The latest background refresh; a newer refresh aborts it. */ private refresh: AbortController | undefined /** Start order of the query whose answer each class holds; an earlier query never replaces a later one. */ private readonly answeredBy = new Map() diff --git a/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts b/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts index 2ea3d500a2..85a92ccac3 100644 --- a/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts +++ b/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts @@ -209,6 +209,28 @@ it('verifies a file menu once when answers for other files arrive during verific menu.dispose() }) +it('verifies every opening of a file target, whatever order the registry observes and resolves in', async () => { + const register = vi.spyOn(MenuRegistry.prototype, 'register') + const h = await boot() + const contribution = register.mock.calls.map(([input]) => input).find(input => input.id === 'open-in-app.file')! + const target = { sessionId, path: 'notes.txt' } + // Resolving without observing first, as a registry that subscribes after its first resolution would. + await contribution.resolve(target, new AbortController().signal) + await contribution.resolve(target, new AbortController().signal) + expect(h.remote.workspaceFiles.stat).toHaveBeenCalledTimes(2) +}) + +it('re-resolves an open file menu only when its own file type answers differently', async () => { + const h = await boot() + const menu = await h.resolve({ kind: 'file', value: { sessionId, path: 'notes.txt' } }) + await vi.waitFor(() => { expect(menu.source.getSnapshot().find(item => item.id === 'open-in-app.file/open')?.label).toBe('open.title') }) + const shown = menu.source.getSnapshot() + await h.paths.applications('/workspace/other.md', new AbortController().signal) + expect(h.paths.cachedApplications('/workspace/other.md')).toHaveLength(1) + expect(menu.source.getSnapshot()).toBe(shown) + menu.dispose() +}) + it('keeps copy-path available on failed metadata and reports native action failures', async () => { const h = await boot() const file = { kind: 'file', value: { sessionId, path: 'file.svg' } } as const From a0ae65fa355438b04859843cff248399267ccc43 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 13:18:42 +0800 Subject: [PATCH 04/44] Honor the selected macOS window for element clicks and hover --- .../native/Sources/CuaNative/AXAccess.swift | 4 ++-- .../native/Sources/CuaNative/AXWorker.swift | 13 +++++++++++-- packages/experimental/cua-macos/src/guidance.ts | 2 +- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift index 8376f1a312..50c8bd3a64 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift @@ -639,7 +639,7 @@ final class AXAccess { return AXProjection(tree: lines.joined(separator: "\n"), elements: elements, indexed: indexed, focusedWindowID: focusedID, truncated: truncated) } - func enclosingWindow(_ element: AXUIElement, owner: ProcessIdentity) throws -> WindowInfo { + func enclosingWindow(_ element: AXUIElement) throws -> (element: AXUIElement, id: UInt32) { var current: AXUIElement? = element var visited: [AXUIElement] = [] for _ in 0..<100 { @@ -648,7 +648,7 @@ final class AXAccess { visited.append(node) let role = try string(node, "AXRole") if role == "AXWindow" || role == "AXSheet", let id = try windowID(node) { - return try WindowGeometry.resolve(owner: owner, windowID: id) { try bounds(node) } + return (node, id) } current = try self.element(node, "AXParent") } diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift index dea2eacb7a..ad079bedac 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift @@ -350,8 +350,16 @@ actor AXWorker { let point: WindowPoint var hoverElement: AXUIElement? if params.contains("elementId") { - guard !params.contains("x"), !params.contains("y"), !params.contains("windowId") else { throw RequestParameters.invalid("Element and coordinate targets cannot be combined") } + guard !params.contains("x"), !params.contains("y"), !params.contains("relative") else { throw RequestParameters.invalid("Element and coordinate targets cannot be combined") } let selected = try element(handle, params) + let scopedWindow = try params.optionalWindowID().map { id in + try coordinateSnapshot(handle, params, id) + let enclosing = try ax().enclosingWindow(selected) + guard enclosing.id == id else { + throw NativeFailure(code: "STALE_SNAPSHOT", message: "Element belongs to another window; read the selected window again") + } + return enclosing + } hoverElement = selected if method == "click" { let names = try ax().actions(selected) @@ -364,7 +372,8 @@ actor AXWorker { try ax().requireEnabled(selected) } guard let bounds = try ax().bounds(selected), bounds.width > 0, bounds.height > 0 else { throw NativeFailure(code: "NATIVE_ERROR", message: "Element has no pointer bounds") } - window = try ax().enclosingWindow(selected, owner: identity) + let enclosing = try scopedWindow ?? ax().enclosingWindow(selected) + window = try WindowGeometry.resolve(owner: identity, windowID: enclosing.id) { try ax().bounds(enclosing.element) } point = WindowPoint(x: bounds.x + bounds.width / 2 - window.bounds.x, y: bounds.y + bounds.height / 2 - window.bounds.y) } else { let id = try params.windowID() diff --git a/packages/experimental/cua-macos/src/guidance.ts b/packages/experimental/cua-macos/src/guidance.ts index 3f062dfbc7..d50011eb06 100644 --- a/packages/experimental/cua-macos/src/guidance.ts +++ b/packages/experimental/cua-macos/src/guidance.ts @@ -2,7 +2,7 @@ import { COMMON_GUIDANCE } from '@deepseek-ai/dsh-cua/guidance' /** macOS operation levels, permissions, selectors, and native measurement units. */ -export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Use window coordinates for click and mouseMove on macOS. The helper rejects element-targeted pointer requests carrying the window ID supplied by these object methods. For an element’s default action, use invoke with its latest ID. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' +export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' /** Complete model instructions for this provider. */ export const GUIDANCE = `${PLATFORM_GUIDANCE} From 10d4199270a6445945caf33622bdd4360b9f7faf Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 13:22:18 +0800 Subject: [PATCH 05/44] Document and exercise window-scoped macOS element input --- packages/experimental/cua-macos/README.i18n.yaml | 8 ++++---- packages/experimental/cua-macos/README.md | 5 +++-- packages/experimental/cua-macos/README.zh.md | 5 +++-- packages/experimental/cua-macos/native/tests/e2e.py | 13 +++++++++++-- .../session/cua-native/system-prompt.expected.md | 2 +- snapshots/session/cua-ptc/system-prompt.expected.md | 2 +- .../ptc-script-retained/system-prompt.expected.md | 2 +- 7 files changed, 24 insertions(+), 13 deletions(-) diff --git a/packages/experimental/cua-macos/README.i18n.yaml b/packages/experimental/cua-macos/README.i18n.yaml index f5c5b0ad0e..88d127489f 100644 --- a/packages/experimental/cua-macos/README.i18n.yaml +++ b/packages/experimental/cua-macos/README.i18n.yaml @@ -18,8 +18,8 @@ en: 7fdb7c0462275417 zh: 0a4a9562f4a82945 /deepseek-ai-dsh-experimental-cua-macos/use-this-package/connect-an-external-execution-environment: - en: 8bf27cf5a258b1f4 - zh: dba694e92bc6c091 + en: d80ab0362946144c + zh: 2cb9fc12bb473300 /deepseek-ai-dsh-experimental-cua-macos/understand-the-implementation: en: 4e82b85a5aa72eee zh: 4e004b7a205d8688 @@ -45,8 +45,8 @@ en: a2c9acb5a4320dda zh: e8d1586af60215c4 /deepseek-ai-dsh-experimental-cua-macos/known-limitations-and-deferred-work: - en: dfedb97af676b4f4 - zh: 1c910e047dbb04ff + en: 4fefaf88ea9eefc6 + zh: d3d18cfbccf8c20f /deepseek-ai-dsh-experimental-cua-macos/known-limitations-and-deferred-work/dev-note: en: b836f01bd0bceb4a zh: 8122fa08b2aa95fa diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index 4cc1f4b2b6..311c7cce7a 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -122,6 +122,8 @@ Use the `cua` API to discover and select the application (`listApps`, `getState` Screenshot capture requires an attachment store and a resolved model route declaring image input. The shared attachment admission validates canonical Base64, the raster and its configured capacity, then stores the image once. Native data remains available to programmatic callers; model content carries coordinate metadata and the durable image reference. Response byte limits apply before parsing. +Window clicks and mouse moves accept an element ID from the latest tree. The element must belong to the selected window. To trigger the default action of a menu item outside any window, use `app.invoke(id)`. + Absolute coordinates for `mouseMove`, `click`, and `drag` use `[x, y]` tuples; `scroll` uses named `x` and `y`. All use pixels in the latest returned screenshot. Pass `relative: true` for integer thousandths of window width and height, from 0 to 999; omission or `false` uses pixels. Relative input can use a successful AX read or screenshot, while absolute input requires a screenshot. The Host maps the delivered image to native points, including attachment and model-request resizing. A value of 999 remains inside the window edge. Coordinate options do not apply to element IDs. A width or height change since the latest observation returns `WINDOW_CHANGED` before input, including raw coordinate calls without a snapshot ID; read fresh AX or take a new screenshot before proceeding. Window movement alone preserves window-local coordinates. Capture and pointer input reject known disagreement between the selected window’s accessibility frame and displayed bounds with `WINDOW_CHANGED`, before posting pointer events. This includes Stage Manager transformations when AX retains the full window frame. Read fresh state; any activation or window raise remains explicit. Advertised AX actions remain available. Missing AX geometry retains the AXless path and does not establish that a transformed window can be operated. Retained objects resolve the current tool binding on every call. The external runtime owns persistence, imports, cancellation and completion-value serialization. Screenshots return encoded bytes with their MIME type, pixel dimensions and window ID; the standard tool pipeline admits the image attachment exactly once. `emit: false` suppresses facade output only. A host that uses another dispatcher can build objects through `createCuaApi` from `./api`. @@ -173,7 +175,7 @@ The plugin contributes one `cua` operation tool with method-specific parameters. ##### Operation guidance (verbatim) ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Use window coordinates for click and mouseMove on macOS. The helper rejects element-targeted pointer requests carrying the window ID supplied by these object methods. For an element’s default action, use invoke with its latest ID. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -241,7 +243,6 @@ The queue orders individual native requests. Holding an app object does not rese - The guide closes when its selected settings window leaves the visible-window list, including when a Space switch or minimization removes it. Request permission again after returning to reopen the guide. - Only the macOS native implementation is provided. Windows support is in development; this package remains in `experimental` until that support is complete, then moves to the `computer-use` group. Browser tabs, AX diffs, unassociated system panels and accessory-only apps are unsupported. - Element IDs belong to the application's current accessibility snapshot; a later read replaces them. Screenshots and pointer input require an explicit window object. -- On macOS, the window object includes its window ID on element-targeted `click` and `mouseMove` calls. The helper rejects that combination with `INVALID_ARGUMENT`. Use window coordinates for pointer input or `invoke(elementId)` for a control’s default action. - Coordinates use the returned screenshot pixels, including the title bar; `relative: true` selects integers from 0 to 999. Native point conversion is Host-owned. Window movement, application changes and other desktop users can race observations and input. - macOS Stage Manager can reduce background window bounds and screenshots to thumbnails. Full-window observation and coordinate input in this state are not guaranteed; model guidance requires explaining foreground needs and avoiding repeated focus reclamation. - Background input depends on private SkyLight symbols; macOS updates can make them unavailable. Validate each target OS version. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 9bb74015f4..629c99c357 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -122,6 +122,8 @@ void await window.getScreenshot() 截图要求附件服务可用，并确认当前模型支持图片输入。共享附件接入校验规范 Base64、图像数据及配置容量，然后保存一次图片。程序调用方仍可取得原生数据，模型内容包含坐标元数据和持久图片引用。响应字节上限在解析前执行。 +窗口的 `click` 和 `mouseMove` 接受最新树中的元素 ID。元素必须属于所选窗口。菜单项如果不属于任何窗口，可用 `app.invoke(id)` 触发默认动作。 + `mouseMove`、`click` 和 `drag` 的绝对坐标使用 `[x, y]` 元组；`scroll` 使用命名的 `x` 和 `y`。它们都使用最近返回截图中的像素。传入 `relative: true` 时使用 0～999 的千分比整数；省略或传入 `false` 时使用像素。相对坐标可基于成功的 AX 读取或截图，绝对坐标必须先截图。Host 根据实际送入模型的图片尺寸换算原生 point，包含附件归一化和模型请求阶段的缩放。999 映射到窗口边缘以内。坐标选项不适用于元素 ID。窗口宽高与最近一次观察不同时，在输入前返回 `WINDOW_CHANGED`，即使原始坐标调用不传快照 ID 也会检查；重新读取 AX 或截图后再操作。单纯移动窗口不使窗口内坐标失效。截图和鼠标输入会核对所选窗口的辅助功能框与显示边界；已知两者不一致时，在发送任何鼠标事件前返回 `WINDOW_CHANGED`。当 AX 保留完整窗口框时，这也会拦住 Stage Manager 变换后的窗口。需要重新读取状态；激活应用或抬起窗口仍须显式调用。元素声明的 AX 动作保持可用。缺少 AX 几何时保留无 AX 路径，不代表变换后的窗口一定可以操作。 保留的对象会在每次调用时获取当前工具绑定。外部运行时负责持久化、导入、取消和最终表达式值的序列化。截图返回编码字节及其 MIME 类型、像素尺寸和窗口 ID，标准工具链负责将图片作为附件传递一次。`emit: false` 只控制对象层的输出。使用其他分派器的宿主可以通过 `./api` 的 `createCuaApi` 创建对象。 @@ -173,7 +175,7 @@ AppKit sheet 可以通过其窗口 ID 单独选择，也会出现在所属文档 ##### 操作指导文本（原文） ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Use window coordinates for click and mouseMove on macOS. The helper rejects element-targeted pointer requests carrying the window ID supplied by these object methods. For an element’s default action, use invoke with its latest ID. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -241,7 +243,6 @@ Raw tools use { method, params }: include appHandle for app calls and windowId f - 所选设置窗口离开可见窗口列表时，引导框会关闭，包括切换桌面空间或最小化导致窗口不再可见的情况。返回后再次申请权限可重新打开引导。 - 只提供 macOS 原生实现，Windows 支持正在开发中。本包暂放在 `experimental`，待 Windows 支持完成后迁回 `computer-use` 组。不支持浏览器标签、启动应用、AX 差分、无法确认应用归属的系统面板，以及仅后台辅助类型的应用。 - 元素 ID 只对应用当前的辅助功能快照有效；之后的读取会替换它们。截图和指针输入都要求显式的窗口对象。 -- 在 macOS 上，窗口对象调用 `click(elementId)` 或 `mouseMove(elementId)` 时会附带窗口 ID，helper 会以 `INVALID_ARGUMENT` 拒绝该组合。鼠标操作可传窗口坐标，执行控件的默认动作可调用 `invoke(elementId)`。 - 坐标使用包含标题栏的返回截图像素；`relative: true` 使用 0～999 的整数，由 Host 换算原生 point。窗口移动、应用变化及其他桌面使用者的操作可能与观察和输入交错。 - macOS 台前调度可能将后台窗口的尺寸与截图变为缩略图，当前无法保证这种状态下的完整画面与坐标输入；模型指引要求说明前台需求并避免反复抢回焦点。 - 后台输入依赖私有 SkyLight 符号，macOS 升级可能使其不可用，需在目标系统版本验证。 diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index 11c266387b..017a5dec75 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -158,7 +158,16 @@ def main(): assert len(matches) == 1, state return {**owner, "snapshotId": state["snapshotId"], "elementId": matches[0]["elementId"]} - target = button_target(first, "Increment primary") + target = {**button_target(first, "Increment primary"), "windowId": primary["windowId"]} + app_state = call("getState", {**owner, "maxElements": 300, "maxDepth": 20}) + other_button = button_target(app_state, "Increment secondary") + for method in ("click", "mouseMove"): + unknown = call(method, {**target, "elementId": "9999"}, "INVALID_ARGUMENT") + assert "elementId is absent" in unknown["message"], unknown + call(method, {**target, "x": 0, "y": 0}, "INVALID_ARGUMENT") + call(method, {**target, "relative": True}, "INVALID_ARGUMENT") + call(method, {**target, "windowId": secondary["windowId"]}, "STALE_SNAPSHOT") + call(method, {**other_button, "windowId": primary["windowId"]}, "STALE_SNAPSHOT") call("click", target) wait_state(lambda state: state["count"] == 1, "primary click was not received exactly once") call("click", button_target(second, "Increment secondary")) @@ -203,7 +212,7 @@ def main(): hover_state = call("getState", primary_params) capture(primary, "02-native-hover.png") assert json.loads(state_path.read_text())["hoverVisible"], "Observation dismissed native hover controls" - hover_target = button_target(hover_state, "Use native hover details") + hover_target = {**button_target(hover_state, "Use native hover details"), "windowId": primary["windowId"]} mouse_move(hover_target) call("click", hover_target) wait_state(lambda actual: actual["hoverClicks"] == 1, "Native hover control did not activate") diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index d736617f95..cd615392cd 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -33,7 +33,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Use window coordinates for click and mouseMove on macOS. The helper rejects element-targeted pointer requests carrying the window ID supplied by these object methods. For an element’s default action, use invoke with its latest ID. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index 1e4c496e13..cf3c4797fb 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Use window coordinates for click and mouseMove on macOS. The helper rejects element-targeted pointer requests carrying the window ID supplied by these object methods. For an element’s default action, use invoke with its latest ID. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index da1e87c182..a2b412b5ff 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Use window coordinates for click and mouseMove on macOS. The helper rejects element-targeted pointer requests carrying the window ID supplied by these object methods. For an element’s default action, use invoke with its latest ID. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. From d4f0efa260b3f8d4a3b02b6e00a6097c845a7609 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 13:23:45 +0800 Subject: [PATCH 06/44] Exercise mouse delivery to the center of a scoped element --- packages/experimental/cua-macos/native/tests/e2e.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index 017a5dec75..e7e40f5c73 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -170,6 +170,8 @@ def main(): call(method, {**other_button, "windowId": primary["windowId"]}, "STALE_SNAPSHOT") call("click", target) wait_state(lambda state: state["count"] == 1, "primary click was not received exactly once") + call("click", {**target, "holdMs": 50}) + wait_state(lambda state: state["count"] == 2, "element-center mouse click did not reach the primary button") call("click", button_target(second, "Increment secondary")) wait_state(lambda state: state["secondaryCount"] == 1, "second window snapshot was invalidated by primary action") first = call("getState", primary_params) From 97ec654f44a0743988b282d85c4e0dd07a877ae4 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 14:08:05 +0800 Subject: [PATCH 07/44] Send mouse events for every macOS click --- .../native/Sources/CuaNative/AXWorker.swift | 26 +++++-------------- .../experimental/cua-macos/src/guidance.ts | 2 +- 2 files changed, 8 insertions(+), 20 deletions(-) diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift index ad079bedac..ba16630e6a 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift @@ -350,29 +350,17 @@ actor AXWorker { let point: WindowPoint var hoverElement: AXUIElement? if params.contains("elementId") { - guard !params.contains("x"), !params.contains("y"), !params.contains("relative") else { throw RequestParameters.invalid("Element and coordinate targets cannot be combined") } + guard !params.contains("x"), !params.contains("y") else { throw RequestParameters.invalid("Element and coordinate targets cannot be combined") } + let requestedWindow = try params.optionalWindowID() + if let requestedWindow { try coordinateSnapshot(handle, params, requestedWindow) } let selected = try element(handle, params) - let scopedWindow = try params.optionalWindowID().map { id in - try coordinateSnapshot(handle, params, id) - let enclosing = try ax().enclosingWindow(selected) - guard enclosing.id == id else { - throw NativeFailure(code: "STALE_SNAPSHOT", message: "Element belongs to another window; read the selected window again") - } - return enclosing + let enclosing = try ax().enclosingWindow(selected) + if let requestedWindow, enclosing.id != requestedWindow { + throw NativeFailure(code: "STALE_SNAPSHOT", message: "Element belongs to window \(enclosing.id); select that window and read its tree") } + if method == "click" { try ax().requireEnabled(selected) } hoverElement = selected - if method == "click" { - let names = try ax().actions(selected) - let action: String? = holdMs != nil || !modifiers.isEmpty ? nil : button == "right" && names.contains("AXShowMenu") ? "AXShowMenu" - : (button == "left" && count == 1 ? (names.contains("AXPress") ? "AXPress" : names.contains("AXPick") ? "AXPick" : nil) : nil) - if let action { - try ax().perform(selected, action) - return .result(Self.actionResult) - } - try ax().requireEnabled(selected) - } guard let bounds = try ax().bounds(selected), bounds.width > 0, bounds.height > 0 else { throw NativeFailure(code: "NATIVE_ERROR", message: "Element has no pointer bounds") } - let enclosing = try scopedWindow ?? ax().enclosingWindow(selected) window = try WindowGeometry.resolve(owner: identity, windowID: enclosing.id) { try ax().bounds(enclosing.element) } point = WindowPoint(x: bounds.x + bounds.width / 2 - window.bounds.x, y: bounds.y + bounds.height / 2 - window.bounds.y) } else { diff --git a/packages/experimental/cua-macos/src/guidance.ts b/packages/experimental/cua-macos/src/guidance.ts index d50011eb06..dd01a43b6e 100644 --- a/packages/experimental/cua-macos/src/guidance.ts +++ b/packages/experimental/cua-macos/src/guidance.ts @@ -2,7 +2,7 @@ import { COMMON_GUIDANCE } from '@deepseek-ai/dsh-cua/guidance' /** macOS operation levels, permissions, selectors, and native measurement units. */ -export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' +export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' /** Complete model instructions for this provider. */ export const GUIDANCE = `${PLATFORM_GUIDANCE} From ecd56a18d3c8a7128772d1d4ff0e6cd2c2df0f7d Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 14:08:06 +0800 Subject: [PATCH 08/44] Verify and document mouse clicks separately from AX actions --- .../cua-native-platform/guide.i18n.yaml | 4 +-- .../v0.2.2-rc.1/cua-native-platform/guide.md | 2 +- .../cua-native-platform/guide.zh.md | 2 +- .../experimental/cua-macos/README.i18n.yaml | 4 +-- packages/experimental/cua-macos/README.md | 6 ++-- packages/experimental/cua-macos/README.zh.md | 6 ++-- .../cua-macos/native/README.i18n.yaml | 4 +-- .../experimental/cua-macos/native/README.md | 6 ++-- .../cua-macos/native/README.zh.md | 6 ++-- .../native/tests/CapabilitiesFixture.swift | 6 +++- .../native/tests/application_consent_e2e.py | 4 +-- .../native/tests/capabilities_e2e.py | 31 +++++++++++-------- .../cua-macos/native/tests/electron_e2e.py | 2 +- .../cua-native/system-prompt.expected.md | 2 +- .../session/cua-ptc/system-prompt.expected.md | 2 +- .../system-prompt.expected.md | 2 +- 16 files changed, 49 insertions(+), 40 deletions(-) diff --git a/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.i18n.yaml b/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.i18n.yaml index 8d9654901c..a76d4de320 100644 --- a/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.i18n.yaml +++ b/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.i18n.yaml @@ -12,5 +12,5 @@ en: e5d1027faa55adba zh: 4b88ad6a70977fb3 /native-cua-application-and-window-api/migration: - en: 6cabae699a340845 - zh: f2256751518c5e36 + en: b09e3dde32391786 + zh: f702e1e25b260b29 diff --git a/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.md b/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.md index 5023ac9ff0..3179dae0d9 100644 --- a/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.md +++ b/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.md @@ -15,7 +15,7 @@ Providers supply startup capabilities alongside their platform guidance. The nat ## Migration -1. After `cua.getApp(selector)`, select the target from `app.getWindows()` with `app.getWindow(windowId)`. Use `window.getScreenshot()`, `window.click()`, `window.scroll()`, and `window.drag()`. Replace `hover` with `window.mouseMove()`. +1. After `cua.getApp(selector)`, select the target from `app.getWindows()` with `app.getWindow(windowId)`. Use `window.getScreenshot()`, `window.click()`, `window.scroll()`, and `window.drag()`. Replace `hover` with `window.mouseMove()`. On macOS, `window.click(id)` sends mouse events at the element center. Switch to `invoke(id)` for the default accessibility action, or `performAction(id, "AXShowMenu")` to request an advertised menu action. 2. Replace `getAXState()` with `getAccessibilityTree()`. Replace `getAXStateAndScreenshot()` with separate tree and screenshot reads. Selection reads the initial tree silently for a window, and for an app when `appLevelAccessibility` is true. Read the tree explicitly to display it. 3. Pass the opaque string ID from the latest tree wherever you passed a numeric element index. In raw requests use `elementId` with `snapshotId`. Replace `performSecondaryAction` with `listActions(id)` and `performAction(id, exactName, argument?)`, or use `invoke(id)` for the default action. `setValue` accepts the provider’s supported JSON values. On macOS, `performAction(id, 'AXRaise')` performs the advertised raise action. Call `app.activate()` first when the application must become active. 4. Replace a packed chord such as `pressKey('shift+Tab')` with `pressKey('Tab', ['shift'])`. Check `(await cua.getCapabilities()).methods` before using `sendKeyEvent`: macOS supports complete keystrokes and rejects directional events with `NOT_IMPLEMENTED`. diff --git a/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.zh.md b/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.zh.md index 880034fbd8..f0a4a905e0 100644 --- a/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.zh.md +++ b/docs/upgrade-guide/v0.2.2-rc.1/cua-native-platform/guide.zh.md @@ -15,7 +15,7 @@ CUA 脚本在指定窗口上截图和操作鼠标。App 和 Window 对象共享 ## 迁移 -1. 调用 `cua.getApp(selector)` 后，从 `app.getWindows()` 返回的列表选定目标，再调用 `app.getWindow(windowId)`。在窗口上调用 `getScreenshot()`、`click()`、`scroll()` 和 `drag()`，将原有的 `hover` 调用改为 `window.mouseMove()`。 +1. 调用 `cua.getApp(selector)` 后，从 `app.getWindows()` 返回的列表选定目标，再调用 `app.getWindow(windowId)`。在窗口上调用 `getScreenshot()`、`click()`、`scroll()` 和 `drag()`，将原有的 `hover` 调用改为 `window.mouseMove()`。在 macOS 上，`window.click(id)` 在元素中心发送鼠标事件。需要执行默认辅助功能动作时，改用 `invoke(id)`。需要打开控件菜单且控件声明了 `AXShowMenu` 时，调用 `performAction(id, "AXShowMenu")`。 2. 将 `getAXState()` 改为 `getAccessibilityTree()`，将 `getAXStateAndScreenshot()` 拆成读取树和截图两次调用。选择窗口时会静默读取初始树。选择应用时，只有 `appLevelAccessibility` 为真才读取。要显示树，请显式读取。 3. 将数字元素索引改为从最新树中取得的字符串 ID。直接发送请求时使用 `elementId` 和 `snapshotId`。原先调用 `performSecondaryAction` 的地方，先用 `listActions(id)` 取得完整动作名，再调用 `performAction(id, exactName, argument?)`。要执行默认动作，可调用 `invoke(id)`。`setValue` 接受提供方支持的 JSON 值。 在 macOS 上，`performAction(id, 'AXRaise')` 只执行元素声明的抬起窗口动作。需要把应用切到前台时先调用 `app.activate()`。 4. 将 `pressKey('shift+Tab')` 之类的组合键写法改为 `pressKey('Tab', ['shift'])`。调用 `sendKeyEvent` 前检查 `(await cua.getCapabilities()).methods`。macOS 支持完整按键，单独发送按下或松开事件会返回 `NOT_IMPLEMENTED`。 diff --git a/packages/experimental/cua-macos/README.i18n.yaml b/packages/experimental/cua-macos/README.i18n.yaml index 88d127489f..42e23f1df1 100644 --- a/packages/experimental/cua-macos/README.i18n.yaml +++ b/packages/experimental/cua-macos/README.i18n.yaml @@ -18,8 +18,8 @@ en: 7fdb7c0462275417 zh: 0a4a9562f4a82945 /deepseek-ai-dsh-experimental-cua-macos/use-this-package/connect-an-external-execution-environment: - en: d80ab0362946144c - zh: 2cb9fc12bb473300 + en: 6054576016409f5d + zh: 5dc25764c92b8db2 /deepseek-ai-dsh-experimental-cua-macos/understand-the-implementation: en: 4e82b85a5aa72eee zh: 4e004b7a205d8688 diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index 311c7cce7a..4fc614c260 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -122,7 +122,7 @@ Use the `cua` API to discover and select the application (`listApps`, `getState` Screenshot capture requires an attachment store and a resolved model route declaring image input. The shared attachment admission validates canonical Base64, the raster and its configured capacity, then stores the image once. Native data remains available to programmatic callers; model content carries coordinate metadata and the durable image reference. Response byte limits apply before parsing. -Window clicks and mouse moves accept an element ID from the latest tree. The element must belong to the selected window. To trigger the default action of a menu item outside any window, use `app.invoke(id)`. +Use `win.click(id)` to send mouse events at the center of an element, or `win.click([x, y])` to click a point. For both `click(id)` and `mouseMove(id)`, the element must belong to the selected window. To trigger the default action of a menu item outside any window, use `app.invoke(id)`. Absolute coordinates for `mouseMove`, `click`, and `drag` use `[x, y]` tuples; `scroll` uses named `x` and `y`. All use pixels in the latest returned screenshot. Pass `relative: true` for integer thousandths of window width and height, from 0 to 999; omission or `false` uses pixels. Relative input can use a successful AX read or screenshot, while absolute input requires a screenshot. The Host maps the delivered image to native points, including attachment and model-request resizing. A value of 999 remains inside the window edge. Coordinate options do not apply to element IDs. A width or height change since the latest observation returns `WINDOW_CHANGED` before input, including raw coordinate calls without a snapshot ID; read fresh AX or take a new screenshot before proceeding. Window movement alone preserves window-local coordinates. Capture and pointer input reject known disagreement between the selected window’s accessibility frame and displayed bounds with `WINDOW_CHANGED`, before posting pointer events. This includes Stage Manager transformations when AX retains the full window frame. Read fresh state; any activation or window raise remains explicit. Advertised AX actions remain available. Missing AX geometry retains the AXless path and does not establish that a transformed window can be operated. @@ -130,7 +130,7 @@ Retained objects resolve the current tool binding on every call. The external ru AppKit sheets are selectable by their window IDs and remain visible in their document window’s tree. Keyboard input follows a system Open/Save panel only when the application’s focused sheet identifies one Apple panel-service window and that service still focuses the same panel. Unrelated or ambiguous remote recipients fail before dispatch. -Element clicks use opaque element IDs from the current snapshot. Left clicks support a single or double click; right and middle clicks support one. Click and drag options accept a button and modifier names; a modified click uses pointer input. Drags hold that button and carry its modifiers through every movement and release. AX text marks disabled controls, `valueWritable=true`, and available `selected` state. The helper uses an advertised accessibility action for unmodified clicks when available; pointer delivery resolves the element’s current bounds and verifies its owning window. Actions report dispatch, not application success. A visual pointer or click marker can be rendered independently of input; AX actions do not require moving the system cursor. Window-directed mouse input can still change application focus, selection or dialogs. Read state after input. `STALE_APP` requires acquiring the application again; `STALE_SNAPSHOT` requires a fresh AX read. Request timeout/cancellation waits for native acknowledgement and preserves usable handles. Missing cleanup acknowledgement closes that Agent’s connection. A screenshot RPC still awaiting its system callback occupies the single capture slot; further screenshots return `CAPTURE_BUSY`, while AX and input remain available. A late capture result is discarded, and its real completion releases that slot. Only an unresponsive operation or failed cleanup can invalidate the shared process and all its handles. Never retry uncertain input automatically. Closing an Agent releases its connection and AX references, without undoing changes to applications. +Element clicks use opaque element IDs from the current snapshot. Left clicks support a single or double click; right and middle clicks support one. Click and drag options accept a button and modifier names. Drags hold that button and carry its modifiers through every movement and release. AX text marks disabled controls, `valueWritable=true`, and available `selected` state. Actions report dispatch, not application success. A visual pointer or click marker can be rendered independently of input; AX actions do not require moving the system cursor. Window-directed mouse input can still change application focus, selection or dialogs. Read state after input. `STALE_APP` requires acquiring the application again; `STALE_SNAPSHOT` requires a fresh AX read. Request timeout/cancellation waits for native acknowledgement and preserves usable handles. Missing cleanup acknowledgement closes that Agent’s connection. A screenshot RPC still awaiting its system callback occupies the single capture slot; further screenshots return `CAPTURE_BUSY`, while AX and input remain available. A late capture result is discarded, and its real completion releases that slot. Only an unresponsive operation or failed cleanup can invalidate the shared process and all its handles. Never retry uncertain input automatically. Closing an Agent releases its connection and AX references, without undoing changes to applications. ----- @@ -175,7 +175,7 @@ The plugin contributes one `cua` operation tool with method-specific parameters. ##### Operation guidance (verbatim) ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 629c99c357..5e2e721984 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -122,7 +122,7 @@ void await window.getScreenshot() 截图要求附件服务可用，并确认当前模型支持图片输入。共享附件接入校验规范 Base64、图像数据及配置容量，然后保存一次图片。程序调用方仍可取得原生数据，模型内容包含坐标元数据和持久图片引用。响应字节上限在解析前执行。 -窗口的 `click` 和 `mouseMove` 接受最新树中的元素 ID。元素必须属于所选窗口。菜单项如果不属于任何窗口，可用 `app.invoke(id)` 触发默认动作。 +`win.click(id)` 向元素中心发送鼠标事件，`win.click([x, y])` 向给定位置发送鼠标事件。`click(id)` 和 `mouseMove(id)` 使用的元素须属于所选窗口。菜单项如果不属于任何窗口，可用 `app.invoke(id)` 触发默认动作。 `mouseMove`、`click` 和 `drag` 的绝对坐标使用 `[x, y]` 元组；`scroll` 使用命名的 `x` 和 `y`。它们都使用最近返回截图中的像素。传入 `relative: true` 时使用 0～999 的千分比整数；省略或传入 `false` 时使用像素。相对坐标可基于成功的 AX 读取或截图，绝对坐标必须先截图。Host 根据实际送入模型的图片尺寸换算原生 point，包含附件归一化和模型请求阶段的缩放。999 映射到窗口边缘以内。坐标选项不适用于元素 ID。窗口宽高与最近一次观察不同时，在输入前返回 `WINDOW_CHANGED`，即使原始坐标调用不传快照 ID 也会检查；重新读取 AX 或截图后再操作。单纯移动窗口不使窗口内坐标失效。截图和鼠标输入会核对所选窗口的辅助功能框与显示边界；已知两者不一致时，在发送任何鼠标事件前返回 `WINDOW_CHANGED`。当 AX 保留完整窗口框时，这也会拦住 Stage Manager 变换后的窗口。需要重新读取状态；激活应用或抬起窗口仍须显式调用。元素声明的 AX 动作保持可用。缺少 AX 几何时保留无 AX 路径，不代表变换后的窗口一定可以操作。 @@ -130,7 +130,7 @@ void await window.getScreenshot() AppKit sheet 可以通过其窗口 ID 单独选择，也会出现在所属文档窗口的控件树中。只有应用当前获得焦点的 sheet 指向唯一的 Apple 面板服务窗口，且该服务仍聚焦于同一面板时，键盘输入才转交系统打开或保存面板。无法确认或存在歧义的远端接收方会在输入前报错。 -元素点击使用当前快照中的不透明元素 ID。左键支持单击或双击；右键和中键支持单击。点击和拖动选项可指定按键与修饰键名称；带修饰键的点击使用指针输入。拖动期间保持指定按键，并在移动和释放事件中携带修饰键。AX 文本标明禁用控件、`valueWritable=true` 和可读取的 `selected` 状态。不带修饰键的点击优先使用元素声明的辅助功能动作；需要发送鼠标事件时，会重新读取元素位置并核实所属窗口。动作返回只表示输入已分派。可视指针或点击标记可以独立绘制；AX 动作无需移动系统指针。窗口定向鼠标输入仍可能改变应用焦点、选区或对话框。输入后需要读取状态确认结果。`STALE_APP` 需要重新获取应用，`STALE_SNAPSHOT` 需要重新读取 AX。请求超时或取消会等待原生停止确认，保留仍可用的句柄；无法确认清理完成时关闭该 Agent 的连接。未返回系统回调的截图 RPC 占用唯一捕获槽位，新截图立即返回 `CAPTURE_BUSY`，AX 和输入仍可使用。迟到图片被丢弃，真实回调结束后释放槽位。只有操作无法停止或清理失败时，才可能使共享进程及所有旧句柄失效。不能自动重试结果不确定的输入。关闭 Agent 会释放其连接和 AX 引用，但不会撤销应用中的修改。 +元素点击使用当前快照中的不透明元素 ID。左键支持单击或双击；右键和中键支持单击。点击和拖动选项可指定按键与修饰键名称。拖动期间保持指定按键，并在移动和释放事件中携带修饰键。AX 文本标明禁用控件、`valueWritable=true` 和可读取的 `selected` 状态。动作返回只表示输入已分派。可视指针或点击标记可以独立绘制；AX 动作无需移动系统指针。窗口定向鼠标输入仍可能改变应用焦点、选区或对话框。输入后需要读取状态确认结果。`STALE_APP` 需要重新获取应用，`STALE_SNAPSHOT` 需要重新读取 AX。请求超时或取消会等待原生停止确认，保留仍可用的句柄；无法确认清理完成时关闭该 Agent 的连接。未返回系统回调的截图 RPC 占用唯一捕获槽位，新截图立即返回 `CAPTURE_BUSY`，AX 和输入仍可使用。迟到图片被丢弃，真实回调结束后释放槽位。只有操作无法停止或清理失败时，才可能使共享进程及所有旧句柄失效。不能自动重试结果不确定的输入。关闭 Agent 会释放其连接和 AX 引用，但不会撤销应用中的修改。 ----- @@ -175,7 +175,7 @@ AppKit sheet 可以通过其窗口 ID 单独选择，也会出现在所属文档 ##### 操作指导文本（原文） ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/packages/experimental/cua-macos/native/README.i18n.yaml b/packages/experimental/cua-macos/native/README.i18n.yaml index baff1bc950..6ae5afb709 100644 --- a/packages/experimental/cua-macos/native/README.i18n.yaml +++ b/packages/experimental/cua-macos/native/README.i18n.yaml @@ -21,8 +21,8 @@ en: 20be674c0592d66d zh: e26112b908187acc /dsh-cua-native-helper/observation-and-input: - en: bf1fa9dbe1cb6dfe - zh: 14097def08a240a4 + en: 8c8ec56eeb88dccf + zh: 4a11a9682e4caec2 /dsh-cua-native-helper/verification: en: 77b266136d5e3631 zh: a5f135e1b12f7a52 diff --git a/packages/experimental/cua-macos/native/README.md b/packages/experimental/cua-macos/native/README.md index d6a1ba141e..57891d69b4 100644 --- a/packages/experimental/cua-macos/native/README.md +++ b/packages/experimental/cua-macos/native/README.md @@ -97,9 +97,9 @@ Protocol declarations are generated from [the shared protocol](../../cua/src/pro `getState` optionally scopes AX traversal to an owned window or nested sheet `windowId`. A new or failed refresh invalidates the previous snapshot for that app/window scope, while other window scopes remain usable. When an owned CG window has no matching AX surface, getState returns a new snapshot with empty tree/elements and no truncation; its coordinates and screenshots remain available. It rechecks process lifetime, window ownership, the unlocked session and Accessibility permission after traversal. Window-discovery APIDisabled and communication errors remain failures; unsupported or absent optional AX attributes can produce an empty observation. Missing or foreign-owned windows never produce an empty success. An AX element invalidated during traversal rejects the whole read with `STALE_SNAPSHOT`; no partial snapshot is published. Callers may read again after the application changes, without repeating input. Raw element targets require `appHandle`, `snapshotId`, and an opaque string `elementId`; coordinate actions require `windowId` and may include a snapshot ID from that window or an app-wide observation. Native references never cross JSON. Window-list refresh releases snapshots for closed windows. Native process identity, snapshot generation, and window ownership checks do not lock the desktop against simultaneous user or application changes. -Element clicks without holdMs prefer advertised AX actions and otherwise use pointer delivery. Coordinate click, scroll and drag validate window ownership and bounds, then use background PID/window-directed primitives. Each event uses SkyLight when available, otherwise the public PID-targeted API. A posted event is never repeated through the other API. Coordinates are macOS points from the top-left of the entire window, including its title bar. Positive `deltaLinesY` reveals content above; negative reveals below. Positive `deltaLinesX` reveals content left; negative reveals right. Each axis accepts an integer from -10 to 10, and at least one must be nonzero. The helper sends line-unit scroll events directly. Click and drag accept left/right/middle buttons and command/control/option/shift/fn modifiers. Only left clicks support double-click. Modified element clicks use pointer delivery. Drag uses one button press/move/release in one window; callers explicitly supply durationMs (1..5000) and steps (1..240). The movement delay budget is divided with fractional precision across the segments; fixed press/release settling and scheduler latency are additional. No implicit fallback sends global desktop input. +Clicks send mouse-down and mouse-up events at the element center or the supplied coordinates. Use `invoke` for the default AX action or `performAction` for a named action. Click, scroll and drag validate window ownership and bounds, then use background PID/window-directed primitives. Each event uses SkyLight when available, otherwise the public PID-targeted API. A posted event is never repeated through the other API. Coordinates are macOS points from the top-left of the entire window, including its title bar. Positive `deltaLinesY` reveals content above; negative reveals below. Positive `deltaLinesX` reveals content left; negative reveals right. Each axis accepts an integer from -10 to 10, and at least one must be nonzero. The helper sends line-unit scroll events directly. Click and drag accept left/right/middle buttons and command/control/option/shift/fn modifiers. Only left clicks support double-click. Drag uses one button press/move/release in one window; callers explicitly supply durationMs (1..5000) and steps (1..240). The movement delay budget is divided with fractional precision across the segments; fixed press/release settling and scheduler latency are additional. No implicit fallback sends global desktop input. -`click.holdMs` is an optional nonnegative integer in milliseconds per press; zero adds no hold delay. Each connection supplies `maxClickHoldMs` to bound explicit holds. The diagnostic helper option `--max-click-hold-ms` defaults to 5000. It bypasses AX actions and uses the element center or supplied coordinates. Omission retains ordinary click timing. The complete gesture holds the request scheduler slot; cancellation, timeout and disconnect interrupt the hold and post the pending mouse-up before releasing that slot. Received events remain effective, and forced process termination cannot guarantee cleanup. +`click.holdMs` is an optional nonnegative integer in milliseconds per press; zero adds no hold delay. Each connection supplies `maxClickHoldMs` to bound explicit holds. The diagnostic helper option `--max-click-hold-ms` defaults to 5000. Omission retains ordinary click timing. The complete gesture holds the request scheduler slot; cancellation, timeout and disconnect interrupt the hold and post the pending mouse-up before releasing that slot. Received events remain effective, and forced process termination cannot guarantee cleanup. Capture and pointer delivery read CG bounds before and after an exact-window AX frame query. Movement during the query returns WINDOW_CHANGED. Valid AX/CG frames must have equal outward-rounded integral point rectangles; a known mismatch returns WINDOW_CHANGED before mouse movement, priming or button delivery. The helper preserves both coordinate sources rather than substituting AX geometry for CG geometry. Missing or unusable AX geometry remains unknown, while AX communication, permission and invalid-element errors propagate. Capture without Accessibility permission retains its CG/SCK checks. Advertised AX actions bypass this pointer preflight; an AX action error does not trigger pointer input. These reads do not lock the window against later movement, and no geometry check activates or raises it. @@ -111,7 +111,7 @@ Host-only permission controls are `permissionStatus {}`, `requestPermission` and Observation reads each non-layout node’s descriptive properties and children through `AXUIElementCopyMultipleAttributeValues`. Per-attribute errors retain the individual-read rules; unsupported batch APIs use individual reads. Values are retained only during that observation, and input reads live attributes. Application state can change between nodes; a batch does not make the whole tree atomic. -System file-panel keyboard routing resolves the remote recipient through the selected app’s focused AX sheet. Only Apple’s Open and Save Panel Service is accepted, and its focused ancestry must still contain the identified remote window. Element pointer fallbacks resolve an app-owned window or sheet through AX parents when AXWindow is unavailable. Ordinary clicks retain AX action priority over pointer input; visual click markers need no system-cursor movement. +System file-panel keyboard routing resolves the remote recipient through the selected app’s focused AX sheet. Only Apple’s Open and Save Panel Service is accepted, and its focused ancestry must still contain the identified remote window. Element-targeted pointer input follows AX parents to find the enclosing window or sheet. Keyboard modifier names are plain strings. The shared object API passes the names through unchanged; the macOS helper accepts `command`/`cmd`, `shift`, `option`/`alt`, `control`/`ctrl`, and `fn`, and rejects an unknown name with `INVALID_ARGUMENT`. It does not parse chord strings. diff --git a/packages/experimental/cua-macos/native/README.zh.md b/packages/experimental/cua-macos/native/README.zh.md index 2863988e66..4d27f100c5 100644 --- a/packages/experimental/cua-macos/native/README.zh.md +++ b/packages/experimental/cua-macos/native/README.zh.md @@ -97,9 +97,9 @@ AppKit 在主事件循环上运行。阻塞式 socket 接收与握手使用后 `getState` 可将 AX 遍历限定到应用所属窗口或嵌套 sheet 的 `windowId`。新的刷新或失败的刷新使同一应用/窗口范围的旧快照失效，其他窗口范围仍可使用。应用所属 CG 窗口没有匹配 AX 界面时，getState 返回空 tree/elements、truncated 为 false 的新快照，仍可截图和坐标输入。遍历后重新检查进程生命期、窗口归属、会话未锁定及辅助功能权限。窗口发现中的 APIDisabled 和通信错误仍会失败；不支持或缺失的可选 AX 属性可以产生空观察。不存在或属于其他应用的窗口不会返回空成功结果。遍历中 AX 元素失效会以 `STALE_SNAPSHOT` 拒绝整次读取，不发布部分树。应用变化后可以重新观察，但不重复输入。原始元素目标要求 `appHandle`、`snapshotId` 和不透明字符串 `elementId`；坐标动作要求 `windowId`，并可携带属于该窗口或应用级观察的快照 ID。原生引用不跨 JSON 传递。刷新窗口列表会释放已关闭窗口的快照。进程身份、快照代次和窗口归属检查不会锁住桌面，用户或应用仍可同时改变状态。 -未指定 holdMs 的元素点击优先使用声明的 AX 动作，否则通过鼠标事件发送。坐标点击、滚动和拖动核实窗口归属和边界后，使用后台 PID/窗口定向输入。事件优先使用 SkyLight，不可用时使用公开的 PID 定向 API；已发送的事件不通过另一 API 重发。坐标是整个窗口左上角起算的 macOS point，包含标题栏。`deltaLinesY` 为正时显示上方内容，为负时显示下方内容；`deltaLinesX` 为正时显示左侧内容，为负时显示右侧内容。每个轴接受 -10 到 10 的整数，至少一个轴非零。helper 直接发送按行计量的滚轮事件。点击和拖动支持左、右、中键及 command/control/option/shift/fn 修饰键，只有左键支持双击。带修饰键的元素点击使用鼠标输入。拖动在一个窗口中完成一次按下、移动、抬起，调用方显式提供 durationMs（1..5000）和 steps（1..240）。移动延迟按小数精度分配到各段，按下和抬起的固定等待及调度延迟另计。不存在隐式改用全局桌面输入的后备路径。 +点击在元素中心或给定坐标发送鼠标按下和抬起事件。执行默认 AX 动作用 `invoke`，执行指定动作用 `performAction`。点击、滚动和拖动核实窗口归属和边界后，使用后台 PID/窗口定向输入。事件优先使用 SkyLight，不可用时使用公开的 PID 定向 API；已发送的事件不通过另一 API 重发。坐标是整个窗口左上角起算的 macOS point，包含标题栏。`deltaLinesY` 为正时显示上方内容，为负时显示下方内容；`deltaLinesX` 为正时显示左侧内容，为负时显示右侧内容。每个轴接受 -10 到 10 的整数，至少一个轴非零。helper 直接发送按行计量的滚轮事件。点击和拖动支持左、右、中键及 command/control/option/shift/fn 修饰键，只有左键支持双击。拖动在一个窗口中完成一次按下、移动、抬起，调用方显式提供 durationMs（1..5000）和 steps（1..240）。移动延迟按小数精度分配到各段，按下和抬起的固定等待及调度延迟另计。不存在隐式改用全局桌面输入的后备路径。 -`click.holdMs` 是可选的每次按住时长，单位为毫秒，接受非负整数；0 表示不额外等待。每个连接通过 `maxClickHoldMs` 限制显式按住时长。诊断 helper 的 `--max-click-hold-ms` 选项默认 5000。指定后使用元素中心或给定坐标发送鼠标事件，跳过 AX 动作。省略时保留普通点击时长。整个手势占用请求调度槽位；取消、超时和断开连接会中断等待，在释放槽位前发送对应的松开事件。已接收的事件仍然生效，进程被强制终止时无法保证清理。 +`click.holdMs` 是可选的每次按住时长，单位为毫秒，接受非负整数；0 表示不额外等待。每个连接通过 `maxClickHoldMs` 限制显式按住时长。诊断 helper 的 `--max-click-hold-ms` 选项默认 5000。省略时保留普通点击时长。整个手势占用请求调度槽位；取消、超时和断开连接会中断等待，在释放槽位前发送对应的松开事件。已接收的事件仍然生效，进程被强制终止时无法保证清理。 截图和鼠标输入在精确窗口的 AX 边界查询前后读取 CG 边界，查询期间移动会返回 WINDOW_CHANGED。有效的 AX/CG 边界向外取整后必须一致；已知不一致会在移动、预备事件或按键发送前返回 WINDOW_CHANGED。helper 保留两个坐标来源，不用 AX 几何替换 CG 几何。缺失或无效的 AX 几何仍表示未知，AX 通信、权限和无效元素错误继续传递。无辅助功能权限的截图保留 CG/SCK 检查。声明的 AX 动作不经过鼠标预检，AX 动作失败也不改用鼠标输入。这些读取不会锁住窗口以阻止后续移动，几何检查不激活或抬起窗口。 @@ -111,7 +111,7 @@ AppKit 在主事件循环上运行。阻塞式 socket 接收与握手使用后 观察通过 `AXUIElementCopyMultipleAttributeValues` 读取每个非布局节点的描述属性和子节点。逐属性错误保留单独读取的规则；不支持批量 API 时使用单独读取。值只在本次观察期间保留，输入读取即时属性。应用状态可能在节点之间变化，批量读取不使整棵树具备原子性。 -系统文件面板的键盘路由通过所选应用的 AX 焦点 sheet 解析远端接收方。只接受 Apple 的 Open and Save Panel Service，且其焦点祖先链仍须包含已识别的远端窗口。AXWindow 不可用时，元素鼠标后备路径沿 AX 父节点解析应用所属窗口或 sheet。普通点击保持 AX 动作优先，可视点击标记不需要系统鼠标移动。 +系统文件面板的键盘路由通过所选应用的 AX 焦点 sheet 解析远端接收方。只接受 Apple 的 Open and Save Panel Service，且其焦点祖先链仍须包含已识别的远端窗口。按元素发送鼠标事件时，helper 沿 AX 父节点查找所属窗口或 sheet。 键盘修饰键名称是普通字符串。共享对象 API 原样传递名称；macOS helper 接受 `command`/`cmd`、`shift`、`option`/`alt`、`control`/`ctrl` 和 `fn`，未知名称返回 `INVALID_ARGUMENT`。它不解析组合键字符串。 diff --git a/packages/experimental/cua-macos/native/tests/CapabilitiesFixture.swift b/packages/experimental/cua-macos/native/tests/CapabilitiesFixture.swift index 8e621fd014..5382155260 100644 --- a/packages/experimental/cua-macos/native/tests/CapabilitiesFixture.swift +++ b/packages/experimental/cua-macos/native/tests/CapabilitiesFixture.swift @@ -4,9 +4,11 @@ import AppKit final class CapabilityPad: NSView { var received: ((NSEvent) -> Void)? var pressed: (() -> Void)? + var showMenu: (() -> Void)? override func isAccessibilityElement() -> Bool { true } override func accessibilityRole() -> NSAccessibility.Role? { .button } override func accessibilityPerformPress() -> Bool { pressed?(); return true } + override func accessibilityPerformShowMenu() -> Bool { showMenu?(); return true } override var isFlipped: Bool { true } override var acceptsFirstResponder: Bool { true } override func acceptsFirstMouse(for event: NSEvent?) -> Bool { true } @@ -46,6 +48,7 @@ final class CapabilitiesFixture: NSObject, NSApplicationDelegate, NSWindowDelega var pad: CapabilityPad! var events: [[String: Any]] = [] var axPresses = 0 + var axMenus = 0 init(statePath: String) { self.statePath = statePath } func applicationDidFinishLaunching(_ notification: Notification) { let bar = NSMenu() @@ -77,6 +80,7 @@ final class CapabilitiesFixture: NSObject, NSApplicationDelegate, NSWindowDelega pad = CapabilityPad(frame: NSRect(x: 20, y: 25, width: 600, height: 170)) pad.setAccessibilityTitle("Input receiver") pad.pressed = { [weak self] in self?.axPresses += 1; self?.persist() } + pad.showMenu = { [weak self] in self?.axMenus += 1; self?.persist() } pad.received = { [weak self] event in guard let self else { return } var value: [String: Any] = ["type": event.type.rawValue, "timestamp": event.timestamp, "flags": event.modifierFlags.rawValue, @@ -119,7 +123,7 @@ final class CapabilitiesFixture: NSObject, NSApplicationDelegate, NSWindowDelega let result: [String: Any] = ["pid": ProcessInfo.processInfo.processIdentifier, "windowId": window.windowNumber, "active": NSApplication.shared.isActive, "keyWindow": window.isKeyWindow, "textFocused": window.firstResponder === text, "pasteAttempts": pasteAttempts, - "text": text.string, "selection": [selection.location, selection.length], "runs": runs, "events": events, "axPresses": axPresses, + "text": text.string, "selection": [selection.location, selection.length], "runs": runs, "events": events, "axPresses": axPresses, "axMenus": axMenus, "pad": [frame.minX, window.frame.height - frame.maxY, frame.width, frame.height]] try! JSONSerialization.data(withJSONObject: result).write(to: URL(fileURLWithPath: statePath), options: .atomic) } diff --git a/packages/experimental/cua-macos/native/tests/application_consent_e2e.py b/packages/experimental/cua-macos/native/tests/application_consent_e2e.py index f2579584b1..10a193d5f2 100644 --- a/packages/experimental/cua-macos/native/tests/application_consent_e2e.py +++ b/packages/experimental/cua-macos/native/tests/application_consent_e2e.py @@ -44,7 +44,7 @@ def main(): owner = {"appHandle": app["appHandle"], "windowId": state["windowId"]} observed = helper.call("getState", {**owner, "maxElements": 2000, "maxDepth": 25}) pad = next(item for item in observed["elements"] if item["role"] == "AXButton" and item["title"] == "Input receiver") - helper.call("click", {"appHandle": app["appHandle"], "snapshotId": observed["snapshotId"], "elementId": pad["elementId"]}) + helper.call("invoke", {**owner, "snapshotId": observed["snapshotId"], "elementId": pad["elementId"]}) wait_for_state(state_path, lambda value: value["axPresses"] == state["axPresses"] + 1) captured = helper.call("screenshot", owner) assert base64.b64decode(captured["data"], validate=True).startswith(b"\x89PNG\r\n\x1a\n") @@ -184,7 +184,7 @@ def main(): os.kill(pid, 0) print(f"Deleted executable: bundle_id={bundle_id is not None}, proc_pidpath_bytes={copied}, errno={path_error}") print("PASS: changed bundles cannot launch; cold/running and direct-bundle identities agree; copies share consent; other IDs stay separate; missing IDs and standalone GUIs use persistent paths across restarts" - + (f"; {checked_windows} GUI instances expose controls, receive an AX click and produce a PNG" if args.desktop else "")) + + (f"; {checked_windows} GUI instances expose controls, receive a default AX action and produce a PNG" if args.desktop else "")) if __name__ == "__main__": diff --git a/packages/experimental/cua-macos/native/tests/capabilities_e2e.py b/packages/experimental/cua-macos/native/tests/capabilities_e2e.py index aaf9cd03e7..650aff14e9 100644 --- a/packages/experimental/cua-macos/native/tests/capabilities_e2e.py +++ b/packages/experimental/cua-macos/native/tests/capabilities_e2e.py @@ -46,19 +46,24 @@ def stop_application(pid): raise AssertionError("fixture did not exit after SIGTERM") -def check_click_holds(helper, owner, window, point, state_file): - observed = helper.call("getState", {**owner, "maxElements": 2000, "maxDepth": 25}) +def check_clicks(helper, window, point, state_file): + observed = helper.call("getState", {**window, "maxElements": 2000, "maxDepth": 25}) pad = next(item for item in observed["elements"] if item["role"] == "AXButton" and item["title"] == "Input receiver") - target = {**owner, "snapshotId": observed["snapshotId"], "elementId": pad["elementId"]} + target = {**window, "snapshotId": observed["snapshotId"], "elementId": pad["elementId"]} state = json.loads(state_file.read_text()) - presses = state["axPresses"] - helper.call("click", target) - state = wait_for_state(state_file, lambda value: value["axPresses"] == presses + 1) count = len(state["events"]) + presses, menus = state["axPresses"], state["axMenus"] + assert {"AXPress", "AXShowMenu"}.issubset(helper.call("listActions", target)) + helper.call("invoke", target) + state = wait_for_state(state_file, lambda value: value["axPresses"] == presses + 1) + helper.call("performAction", {**target, "action": "AXShowMenu"}) + state = wait_for_state(state_file, lambda value: value["axMenus"] == menus + 1) + assert len(state["events"]) == count, state for invalid in [-1, 1.5, 5001, "800", None]: helper.call("click", {**target, "holdMs": invalid}, "INVALID_ARGUMENT") - assert json.loads(state_file.read_text())["axPresses"] == presses + 1 - for params, types in [({**target, "holdMs": 0}, [1, 2]), + for params, types in [(target, [1, 2]), ({**target, "mouseButton": "right"}, [3, 4]), + ({**window, **point}, [1, 2]), + ({**target, "holdMs": 0}, [1, 2]), ({**target, "holdMs": 150}, [1, 2]), ({**window, **point, "holdMs": 150, "mouseButton": "right"}, [3, 4]), ({**window, **point, "holdMs": 150, "mouseButton": "middle"}, [25, 26]), @@ -68,8 +73,8 @@ def check_click_holds(helper, owner, window, point, state_file): events = state["events"][count:] assert [event["type"] for event in events] == types, events for index in range(0, len(events), 2): - assert events[index + 1]["timestamp"] - events[index]["timestamp"] >= params["holdMs"] / 1000, events - assert state["axPresses"] == presses + 1, state + assert events[index + 1]["timestamp"] - events[index]["timestamp"] >= params.get("holdMs", 0) / 1000, events + assert (state["axPresses"], state["axMenus"]) == (presses + 1, menus + 1), state count += len(types) # Production cancellation travels directly over the socket; the diagnostic stdio proxy exchanges whole requests. key = hashlib.sha256(Path(helper.process.args[0]).read_bytes()).hexdigest() @@ -77,7 +82,7 @@ def check_click_holds(helper, owner, window, point, state_file): for ending in ["cancel", "timeout", "disconnect"]: connection = Connection(socket_path, key, maxClickHoldMs=7000, timeoutMs=2000 if ending == "timeout" else 30000) try: - app = connection.call("getApp", {"selector": str(observed["pid"])}) + app = connection.call("getApp", {"selector": str(state["pid"])}) connection.call("getState", {"appHandle": app["appHandle"], "windowId": window["windowId"], "maxElements": 2000, "maxDepth": 25}) params = {**window, **point, "appHandle": app["appHandle"], "holdMs": 6000} @@ -104,7 +109,7 @@ def check_click_holds(helper, owner, window, point, state_file): connection.close() connection = Connection(socket_path, key, maxClickHoldMs=0) try: - app = connection.call("getApp", {"selector": str(observed["pid"])}) + app = connection.call("getApp", {"selector": str(state["pid"])}) connection.call("getState", {"appHandle": app["appHandle"], "windowId": window["windowId"], "maxElements": 2000, "maxDepth": 25}) count = len(state["events"]) @@ -189,7 +194,7 @@ def main(): call("pressKey", {**owner, "key": "KP_0"}) call("pressKey", {**owner, "key": "KP_Enter"}) state = wait_for_state(state_file, lambda value: {82, 76}.issubset({event.get("keyCode") for event in value["events"]})) - state = check_click_holds(helper, owner, window, point, state_file) + state = check_clicks(helper, window, point, state_file) if args.artifacts: args.artifacts.mkdir(parents=True, exist_ok=True) shot = call("screenshot", window) diff --git a/packages/experimental/cua-macos/native/tests/electron_e2e.py b/packages/experimental/cua-macos/native/tests/electron_e2e.py index d23eb17ebd..0e2b966516 100644 --- a/packages/experimental/cua-macos/native/tests/electron_e2e.py +++ b/packages/experimental/cua-macos/native/tests/electron_e2e.py @@ -99,7 +99,7 @@ def main(): window = next(window for window in state['windows'] if window['windowId'] == window_id) if not args.hover_only: - call('click', {**owner, 'snapshotId': state['snapshotId'], 'elementId': buttons[0]['elementId']}) + call('invoke', {**owner, 'snapshotId': state['snapshotId'], 'elementId': buttons[0]['elementId']}) wait_state(lambda actual: actual['clicks'] == 1 and actual['lastClickTrusted'] is True, 'Native AX press did not produce one trusted DOM click') state = observe() diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index cd615392cd..9aed9a4161 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -33,7 +33,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index cf3c4797fb..677eb6406d 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index a2b412b5ff..a138bad2ce 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For click and mouseMove, use window coordinates or an element ID from the latest tree that belongs to the selected window. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. From 5d0ffe3c995b5fdd9c3696c68c2f44e8b7b2ce47 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 14:10:51 +0800 Subject: [PATCH 09/44] Invoke the system Open panel button through its default action --- packages/experimental/cua-macos/README.i18n.yaml | 4 ++-- packages/experimental/cua-macos/README.md | 2 +- packages/experimental/cua-macos/README.zh.md | 2 +- .../cua-macos/native/tests/application_consent_e2e.py | 6 +++--- packages/experimental/cua-macos/native/tests/e2e.py | 2 +- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/packages/experimental/cua-macos/README.i18n.yaml b/packages/experimental/cua-macos/README.i18n.yaml index 42e23f1df1..af699d02ba 100644 --- a/packages/experimental/cua-macos/README.i18n.yaml +++ b/packages/experimental/cua-macos/README.i18n.yaml @@ -18,8 +18,8 @@ en: 7fdb7c0462275417 zh: 0a4a9562f4a82945 /deepseek-ai-dsh-experimental-cua-macos/use-this-package/connect-an-external-execution-environment: - en: 6054576016409f5d - zh: 5dc25764c92b8db2 + en: 7a9e7a28f05a6937 + zh: c4d2494da7cf5d08 /deepseek-ai-dsh-experimental-cua-macos/understand-the-implementation: en: 4e82b85a5aa72eee zh: 4e004b7a205d8688 diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index 4fc614c260..339ac3a448 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -122,7 +122,7 @@ Use the `cua` API to discover and select the application (`listApps`, `getState` Screenshot capture requires an attachment store and a resolved model route declaring image input. The shared attachment admission validates canonical Base64, the raster and its configured capacity, then stores the image once. Native data remains available to programmatic callers; model content carries coordinate metadata and the durable image reference. Response byte limits apply before parsing. -Use `win.click(id)` to send mouse events at the center of an element, or `win.click([x, y])` to click a point. For both `click(id)` and `mouseMove(id)`, the element must belong to the selected window. To trigger the default action of a menu item outside any window, use `app.invoke(id)`. +Use `win.click(id)` to send mouse events at the center of an element, or `win.click([x, y])` to click a point. For both `click(id)` and `mouseMove(id)`, the element must belong to the selected window. Use `invoke(id)` to activate a control through accessibility, for example the Open button in a system file panel or a menu item in the app tree. Absolute coordinates for `mouseMove`, `click`, and `drag` use `[x, y]` tuples; `scroll` uses named `x` and `y`. All use pixels in the latest returned screenshot. Pass `relative: true` for integer thousandths of window width and height, from 0 to 999; omission or `false` uses pixels. Relative input can use a successful AX read or screenshot, while absolute input requires a screenshot. The Host maps the delivered image to native points, including attachment and model-request resizing. A value of 999 remains inside the window edge. Coordinate options do not apply to element IDs. A width or height change since the latest observation returns `WINDOW_CHANGED` before input, including raw coordinate calls without a snapshot ID; read fresh AX or take a new screenshot before proceeding. Window movement alone preserves window-local coordinates. Capture and pointer input reject known disagreement between the selected window’s accessibility frame and displayed bounds with `WINDOW_CHANGED`, before posting pointer events. This includes Stage Manager transformations when AX retains the full window frame. Read fresh state; any activation or window raise remains explicit. Advertised AX actions remain available. Missing AX geometry retains the AXless path and does not establish that a transformed window can be operated. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 5e2e721984..52b12e5c39 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -122,7 +122,7 @@ void await window.getScreenshot() 截图要求附件服务可用，并确认当前模型支持图片输入。共享附件接入校验规范 Base64、图像数据及配置容量，然后保存一次图片。程序调用方仍可取得原生数据，模型内容包含坐标元数据和持久图片引用。响应字节上限在解析前执行。 -`win.click(id)` 向元素中心发送鼠标事件，`win.click([x, y])` 向给定位置发送鼠标事件。`click(id)` 和 `mouseMove(id)` 使用的元素须属于所选窗口。菜单项如果不属于任何窗口，可用 `app.invoke(id)` 触发默认动作。 +`win.click(id)` 向元素中心发送鼠标事件，`win.click([x, y])` 向给定位置发送鼠标事件。`click(id)` 和 `mouseMove(id)` 使用的元素须属于所选窗口。要通过辅助功能触发控件的默认动作，可用 `invoke(id)`，例如系统文件面板的“打开”按钮，或应用树中的菜单项。 `mouseMove`、`click` 和 `drag` 的绝对坐标使用 `[x, y]` 元组；`scroll` 使用命名的 `x` 和 `y`。它们都使用最近返回截图中的像素。传入 `relative: true` 时使用 0～999 的千分比整数；省略或传入 `false` 时使用像素。相对坐标可基于成功的 AX 读取或截图，绝对坐标必须先截图。Host 根据实际送入模型的图片尺寸换算原生 point，包含附件归一化和模型请求阶段的缩放。999 映射到窗口边缘以内。坐标选项不适用于元素 ID。窗口宽高与最近一次观察不同时，在输入前返回 `WINDOW_CHANGED`，即使原始坐标调用不传快照 ID 也会检查；重新读取 AX 或截图后再操作。单纯移动窗口不使窗口内坐标失效。截图和鼠标输入会核对所选窗口的辅助功能框与显示边界；已知两者不一致时，在发送任何鼠标事件前返回 `WINDOW_CHANGED`。当 AX 保留完整窗口框时，这也会拦住 Stage Manager 变换后的窗口。需要重新读取状态；激活应用或抬起窗口仍须显式调用。元素声明的 AX 动作保持可用。缺少 AX 几何时保留无 AX 路径，不代表变换后的窗口一定可以操作。 diff --git a/packages/experimental/cua-macos/native/tests/application_consent_e2e.py b/packages/experimental/cua-macos/native/tests/application_consent_e2e.py index 10a193d5f2..e8b2b25be8 100644 --- a/packages/experimental/cua-macos/native/tests/application_consent_e2e.py +++ b/packages/experimental/cua-macos/native/tests/application_consent_e2e.py @@ -14,13 +14,13 @@ import tempfile import uuid from capabilities_e2e import stop_application, wait_for_state -from native_test_support import BINARY, ROOT, NativeHelper, stop +from native_test_support import ROOT, NativeHelper, stop def main(): parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--helper", type=Path, default=BINARY) - parser.add_argument("--desktop", action="store_true", help="Read, click and capture owned fixture windows; requires existing system grants") + parser.add_argument("--helper", type=Path) + parser.add_argument("--desktop", action="store_true", help="Read owned fixture windows, invoke their buttons and capture them; requires existing system grants") args = parser.parse_args() transcript = [] with tempfile.TemporaryDirectory(prefix="dsh-cua-consent-") as temporary, ExitStack() as cleanup: diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index e7e40f5c73..94eb893f74 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -76,7 +76,7 @@ def check_file_panel(call, owner, primary_params, primary, directory, wait_state and any(e["identifier"] == "OKButton" and e["enabled"] for e in observed["elements"]), "Go to folder did not select the requested file") open_button = next(e for e in ready["elements"] if e["identifier"] == "OKButton") - call("click", {**owner, "snapshotId": ready["snapshotId"], "elementId": open_button["elementId"]}) + assert call("invoke", {**owner, "snapshotId": ready["snapshotId"], "elementId": open_button["elementId"]})["dispatched"] wait_state(lambda state: bool(state["selectedFile"]) and Path(state["selectedFile"]).samefile(selected_file), "File picker did not return the intended file") # AppKit can retain the dismissed panel's hidden WindowServer window after From 84c760f2050a993be3ec7057672fc597a8187633 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 14:33:59 +0800 Subject: [PATCH 10/44] Describe macOS mouse dispatch on the click method itself --- packages/experimental/cua-macos/README.md | 4 ++-- packages/experimental/cua-macos/README.zh.md | 4 ++-- packages/experimental/cua-macos/src/guidance.ts | 2 +- packages/experimental/cua/README.i18n.yaml | 4 ++-- packages/experimental/cua/README.md | 2 +- packages/experimental/cua/README.zh.md | 2 +- packages/experimental/cua/src/guidance.ts | 2 +- packages/experimental/cua/src/types.ts | 6 +++--- snapshots/session/cua-native/system-prompt.expected.md | 4 ++-- snapshots/session/cua-ptc/system-prompt.expected.md | 4 ++-- .../session/ptc-script-retained/system-prompt.expected.md | 4 ++-- 11 files changed, 19 insertions(+), 19 deletions(-) diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index 339ac3a448..2d5e828514 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -175,7 +175,7 @@ The plugin contributes one `cua` operation tool with method-specific parameters. ##### Operation guidance (verbatim) ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -210,7 +210,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 52b12e5c39..f0303b499c 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -175,7 +175,7 @@ AppKit sheet 可以通过其窗口 ID 单独选择，也会出现在所属文档 ##### 操作指导文本（原文） ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -210,7 +210,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua-macos/src/guidance.ts b/packages/experimental/cua-macos/src/guidance.ts index dd01a43b6e..6624bde4b7 100644 --- a/packages/experimental/cua-macos/src/guidance.ts +++ b/packages/experimental/cua-macos/src/guidance.ts @@ -2,7 +2,7 @@ import { COMMON_GUIDANCE } from '@deepseek-ai/dsh-cua/guidance' /** macOS operation levels, permissions, selectors, and native measurement units. */ -export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' +export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' /** Complete model instructions for this provider. */ export const GUIDANCE = `${PLATFORM_GUIDANCE} diff --git a/packages/experimental/cua/README.i18n.yaml b/packages/experimental/cua/README.i18n.yaml index f4abb13c43..dd3c6d217a 100644 --- a/packages/experimental/cua/README.i18n.yaml +++ b/packages/experimental/cua/README.i18n.yaml @@ -15,8 +15,8 @@ en: 5db4d4fea25493b9 zh: 65ab64fc24eb0d09 /deepseek-ai-dsh-cua/use-this-package: - en: ba1f8e6ae51f69d7 - zh: 8f4fde9857bae319 + en: d0088d2ec1ab8cf2 + zh: d927289a63d7a564 /deepseek-ai-dsh-cua/use-this-package/permission-controls: en: d92d5948a7941e8d zh: d43491ae6521b87e diff --git a/packages/experimental/cua/README.md b/packages/experimental/cua/README.md index c95e4ae8ec..365c549650 100644 --- a/packages/experimental/cua/README.md +++ b/packages/experimental/cua/README.md @@ -43,7 +43,7 @@ Application consent precedes launch, observation, capture, and input. Session ac Actions return a `dispatched` boolean; `mouseMove` also reports `hoverRoute` and `systemPointerMoved`, and `show()` adds `windowId` and `foregroundChanged`. Read fresh state to verify the result. Screenshots and pointer actions are available only on explicit window objects. Coordinate actions default to pixels in the latest returned screenshot; `relative: true` accepts integers from 0 to 999 after a successful window observation. Values are divided by 1000 and scaled to the window dimensions, then clamped inside the window. The Host resolves attachment normalization and the selected model route’s image projection before returning the screenshot, then converts input to the provider’s native window coordinates. Resizing invalidates the screenshot mapping; `SCREENSHOT_REQUIRED` requires a new screenshot for absolute coordinates. Resizing rejects old coordinates with `WINDOW_CHANGED`; movement alone preserves window-local coordinates. -`click(target, { holdMs: 800 })` holds the mouse button for 800 milliseconds before release. `holdMs` applies only to `click`; it accepts nonnegative integers and applies to each press of a double-click; zero releases without an additional hold. The provider config `maxClickHoldMs` limits explicit holds and defaults to 5000 ms. Explicit holds use pointer input; omission preserves ordinary click behavior. The operation deadline includes the hold, and cancellation or timeout releases the button. Received input cannot be undone; forced helper termination cannot guarantee release. +`click(target, { holdMs: 800 })` holds the mouse button for 800 milliseconds before release. `holdMs` applies only to `click`; it accepts nonnegative integers and applies to each press of a double-click; zero releases without an additional hold. The provider config `maxClickHoldMs` limits explicit holds and defaults to 5000 ms. On macOS, both ordinary and held clicks send mouse events at the element center or the supplied point. Use `invoke(id)` for the default accessibility action. The operation deadline includes the hold, and cancellation or timeout releases the button. Received input cannot be undone; forced helper termination cannot guarantee release. ### Permission controls diff --git a/packages/experimental/cua/README.zh.md b/packages/experimental/cua/README.zh.md index c1c5ed863c..b64c445c44 100644 --- a/packages/experimental/cua/README.zh.md +++ b/packages/experimental/cua/README.zh.md @@ -43,7 +43,7 @@ kind: "package-reference" 动作返回 `dispatched` 布尔值；`mouseMove` 还会返回 `hoverRoute` 和 `systemPointerMoved`，`show()` 还会返回 `windowId` 和 `foregroundChanged`。请读取新状态以验证结果。截图和指针操作只能通过显式窗口对象使用。坐标操作默认使用最近返回截图中的像素；成功观察窗口后，传入 `relative: true` 可使用 0～999 的整数坐标。数值除以 1000 后按窗口尺寸缩放，并限制在窗口内。Host 在返回截图前解析附件归一化和当前模型的图片缩放规则，再将输入换算为提供方的原生窗口坐标。窗口尺寸变化会使截图映射失效，绝对坐标遇到 `SCREENSHOT_REQUIRED` 时需要重新截图。窗口缩放后以 `WINDOW_CHANGED` 拒绝旧坐标；单纯移动窗口不改变窗口内坐标。 -`click(target, { holdMs: 800 })` 按住鼠标按钮 800 毫秒后松开。`holdMs` 仅适用于 `click`；它接受非负整数，双击时应用于每次按下；0 表示按下后立即松开，不额外等待。提供方配置 `maxClickHoldMs` 限制显式按住时长，默认 5000 毫秒。指定时长后使用鼠标事件；省略时保留普通点击行为。按住时间计入操作超时，取消或超时会松开按钮。已接收的输入无法撤销；辅助进程被强制终止时无法保证松开。 +`click(target, { holdMs: 800 })` 按住鼠标按钮 800 毫秒后松开。`holdMs` 仅适用于 `click`；它接受非负整数，双击时应用于每次按下；0 表示按下后立即松开，不额外等待。提供方配置 `maxClickHoldMs` 限制显式按住时长，默认 5000 毫秒。在 macOS 上，无论是否指定按住时长，点击都会向元素中心或给定坐标发送鼠标事件。要执行控件的默认辅助功能动作，请用 `invoke(id)`。按住时间计入操作超时，取消或超时会松开按钮。已接收的输入无法撤销；辅助进程被强制终止时无法保证松开。 ### 权限控件 diff --git a/packages/experimental/cua/src/guidance.ts b/packages/experimental/cua/src/guidance.ts index 93947ad32c..adf62b8b5e 100644 --- a/packages/experimental/cua/src/guidance.ts +++ b/packages/experimental/cua/src/guidance.ts @@ -33,7 +33,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua/src/types.ts b/packages/experimental/cua/src/types.ts index c059a546f2..b44e8f4ef6 100644 --- a/packages/experimental/cua/src/types.ts +++ b/packages/experimental/cua/src/types.ts @@ -79,7 +79,6 @@ export type CuaMouseButton = NonNullable['mouseButton']> /** * Left clicks support one or two presses; right and middle clicks support one. * holdMs is a nonnegative integer in milliseconds, limited by maxClickHoldMs; zero releases immediately. - * Explicit holds use pointer input; omission preserves ordinary clicks. */ export type CuaClickOptions = Pick, 'mouseButton' | 'clickCount' | 'modifiers' | 'holdMs'> & CuaCoordinateOptions /** One same-window drag; omitted button is left and omitted modifiers are empty. */ @@ -201,9 +200,10 @@ export interface CuaWindow extends CuaSurface { */ mouseMove(target: CuaElementId | readonly [number, number], options?: CuaCoordinateOptions): Promise /** - * Click a current element id or screenshot-local point. + * Click an element or a point in this window. On macOS both forms send mouse-down and mouse-up events. + * Use {@link CuaSurface.invoke} for the element's default accessibility action. * Cancellation or timeout releases the held button; received input cannot be undone. The operation deadline includes holdMs. - * @param target - element id or [x,y]. + * @param target - On macOS, an id from the latest tree targets the element center. A coordinate pair targets the given point. * @param options - button, click count, modifiers, optional hold duration and coordinate mode. * @returns dispatch evidence. */ diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index 9aed9a4161..b34c3def7c 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -33,7 +33,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -68,7 +68,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index 677eb6406d..00451355c7 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -70,7 +70,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index a138bad2ce..b2abdbb7e7 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. click sends mouse events at window coordinates or at the center of an element from the latest tree in that window. Use invoke for the default accessibility action, including on menu items. mouseMove accepts either target. getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -70,7 +70,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. From 7557dda6fffe6215c62b6c1eca8fdc6fdcf99f4a Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 14:43:03 +0800 Subject: [PATCH 11/44] Explain how to recover from macOS pointer target mismatches --- .../experimental/cua-macos/README.i18n.yaml | 4 +-- packages/experimental/cua-macos/README.md | 4 +-- packages/experimental/cua-macos/README.zh.md | 4 +-- .../cua-macos/native/README.i18n.yaml | 8 ++--- .../experimental/cua-macos/native/README.md | 4 +-- .../cua-macos/native/README.zh.md | 4 +-- .../native/Sources/CuaNative/AXAccess.swift | 2 +- .../native/Sources/CuaNative/AXWorker.swift | 4 +-- .../cua-macos/native/tests/e2e.py | 30 +++++++++++++------ .../experimental/cua-macos/src/guidance.ts | 2 +- .../cua-native/system-prompt.expected.md | 2 +- .../cua-native/workspace/native-helper.mjs | 2 +- .../session/cua-ptc/system-prompt.expected.md | 2 +- .../system-prompt.expected.md | 2 +- 14 files changed, 43 insertions(+), 31 deletions(-) diff --git a/packages/experimental/cua-macos/README.i18n.yaml b/packages/experimental/cua-macos/README.i18n.yaml index af699d02ba..76c65f408d 100644 --- a/packages/experimental/cua-macos/README.i18n.yaml +++ b/packages/experimental/cua-macos/README.i18n.yaml @@ -18,8 +18,8 @@ en: 7fdb7c0462275417 zh: 0a4a9562f4a82945 /deepseek-ai-dsh-experimental-cua-macos/use-this-package/connect-an-external-execution-environment: - en: 7a9e7a28f05a6937 - zh: c4d2494da7cf5d08 + en: 740031b2bf73b350 + zh: 30c4929c5bb6baeb /deepseek-ai-dsh-experimental-cua-macos/understand-the-implementation: en: 4e82b85a5aa72eee zh: 4e004b7a205d8688 diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index 2d5e828514..7a96cac17e 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -128,7 +128,7 @@ Absolute coordinates for `mouseMove`, `click`, and `drag` use `[x, y]` tuples; ` Retained objects resolve the current tool binding on every call. The external runtime owns persistence, imports, cancellation and completion-value serialization. Screenshots return encoded bytes with their MIME type, pixel dimensions and window ID; the standard tool pipeline admits the image attachment exactly once. `emit: false` suppresses facade output only. A host that uses another dispatcher can build objects through `createCuaApi` from `./api`. -AppKit sheets are selectable by their window IDs and remain visible in their document window’s tree. Keyboard input follows a system Open/Save panel only when the application’s focused sheet identifies one Apple panel-service window and that service still focuses the same panel. Unrelated or ambiguous remote recipients fail before dispatch. +An AppKit sheet can appear in its document window’s tree with a separate window ID. For pointer input on the sheet, use `app.getWindow(sheetId)` and an element ID from that window’s tree. Keyboard input follows a system Open/Save panel only when the application’s focused sheet identifies one Apple panel-service window and that service still focuses the same panel. Unrelated or ambiguous remote recipients fail before dispatch. Element clicks use opaque element IDs from the current snapshot. Left clicks support a single or double click; right and middle clicks support one. Click and drag options accept a button and modifier names. Drags hold that button and carry its modifiers through every movement and release. AX text marks disabled controls, `valueWritable=true`, and available `selected` state. Actions report dispatch, not application success. A visual pointer or click marker can be rendered independently of input; AX actions do not require moving the system cursor. Window-directed mouse input can still change application focus, selection or dialogs. Read state after input. `STALE_APP` requires acquiring the application again; `STALE_SNAPSHOT` requires a fresh AX read. Request timeout/cancellation waits for native acknowledgement and preserves usable handles. Missing cleanup acknowledgement closes that Agent’s connection. A screenshot RPC still awaiting its system callback occupies the single capture slot; further screenshots return `CAPTURE_BUSY`, while AX and input remain available. A late capture result is discarded, and its real completion releases that slot. Only an unresponsive operation or failed cleanup can invalidate the shared process and all its handles. Never retry uncertain input automatically. Closing an Agent releases its connection and AX references, without undoing changes to applications. @@ -175,7 +175,7 @@ The plugin contributes one `cua` operation tool with method-specific parameters. ##### Operation guidance (verbatim) ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index f0303b499c..ca39bab33c 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -128,7 +128,7 @@ void await window.getScreenshot() 保留的对象会在每次调用时获取当前工具绑定。外部运行时负责持久化、导入、取消和最终表达式值的序列化。截图返回编码字节及其 MIME 类型、像素尺寸和窗口 ID，标准工具链负责将图片作为附件传递一次。`emit: false` 只控制对象层的输出。使用其他分派器的宿主可以通过 `./api` 的 `createCuaApi` 创建对象。 -AppKit sheet 可以通过其窗口 ID 单独选择，也会出现在所属文档窗口的控件树中。只有应用当前获得焦点的 sheet 指向唯一的 Apple 面板服务窗口，且该服务仍聚焦于同一面板时，键盘输入才转交系统打开或保存面板。无法确认或存在歧义的远端接收方会在输入前报错。 +AppKit 的 sheet 可以出现在文档窗口的树里，并有独立的窗口 ID。对 sheet 发送鼠标输入时，先用 `app.getWindow(sheetId)` 选择它，再使用该窗口树中的元素 ID。只有应用当前获得焦点的 sheet 指向唯一的 Apple 面板服务窗口，且该服务仍聚焦于同一面板时，键盘输入才转交系统打开或保存面板。无法确认或存在歧义的远端接收方会在输入前报错。 元素点击使用当前快照中的不透明元素 ID。左键支持单击或双击；右键和中键支持单击。点击和拖动选项可指定按键与修饰键名称。拖动期间保持指定按键，并在移动和释放事件中携带修饰键。AX 文本标明禁用控件、`valueWritable=true` 和可读取的 `selected` 状态。动作返回只表示输入已分派。可视指针或点击标记可以独立绘制；AX 动作无需移动系统指针。窗口定向鼠标输入仍可能改变应用焦点、选区或对话框。输入后需要读取状态确认结果。`STALE_APP` 需要重新获取应用，`STALE_SNAPSHOT` 需要重新读取 AX。请求超时或取消会等待原生停止确认，保留仍可用的句柄；无法确认清理完成时关闭该 Agent 的连接。未返回系统回调的截图 RPC 占用唯一捕获槽位，新截图立即返回 `CAPTURE_BUSY`，AX 和输入仍可使用。迟到图片被丢弃，真实回调结束后释放槽位。只有操作无法停止或清理失败时，才可能使共享进程及所有旧句柄失效。不能自动重试结果不确定的输入。关闭 Agent 会释放其连接和 AX 引用，但不会撤销应用中的修改。 @@ -175,7 +175,7 @@ AppKit sheet 可以通过其窗口 ID 单独选择，也会出现在所属文档 ##### 操作指导文本（原文） ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/packages/experimental/cua-macos/native/README.i18n.yaml b/packages/experimental/cua-macos/native/README.i18n.yaml index 6ae5afb709..89135ddfbe 100644 --- a/packages/experimental/cua-macos/native/README.i18n.yaml +++ b/packages/experimental/cua-macos/native/README.i18n.yaml @@ -21,11 +21,11 @@ en: 20be674c0592d66d zh: e26112b908187acc /dsh-cua-native-helper/observation-and-input: - en: 8c8ec56eeb88dccf - zh: 4a11a9682e4caec2 + en: 2408e0c0e9c873a2 + zh: e06236fc32efc4fa /dsh-cua-native-helper/verification: - en: 77b266136d5e3631 - zh: a5f135e1b12f7a52 + en: 49f4cce2a4cc3114 + zh: e9d65c92ef35fe24 /dsh-cua-native-helper/further-exploration: en: 265e1927952ce6d7 zh: 7c35d9fd98b6e6c8 diff --git a/packages/experimental/cua-macos/native/README.md b/packages/experimental/cua-macos/native/README.md index 57891d69b4..c01b273d1b 100644 --- a/packages/experimental/cua-macos/native/README.md +++ b/packages/experimental/cua-macos/native/README.md @@ -95,7 +95,7 @@ Protocol declarations are generated from [the shared protocol](../../cua/src/pro ## Observation and input -`getState` optionally scopes AX traversal to an owned window or nested sheet `windowId`. A new or failed refresh invalidates the previous snapshot for that app/window scope, while other window scopes remain usable. When an owned CG window has no matching AX surface, getState returns a new snapshot with empty tree/elements and no truncation; its coordinates and screenshots remain available. It rechecks process lifetime, window ownership, the unlocked session and Accessibility permission after traversal. Window-discovery APIDisabled and communication errors remain failures; unsupported or absent optional AX attributes can produce an empty observation. Missing or foreign-owned windows never produce an empty success. An AX element invalidated during traversal rejects the whole read with `STALE_SNAPSHOT`; no partial snapshot is published. Callers may read again after the application changes, without repeating input. Raw element targets require `appHandle`, `snapshotId`, and an opaque string `elementId`; coordinate actions require `windowId` and may include a snapshot ID from that window or an app-wide observation. Native references never cross JSON. Window-list refresh releases snapshots for closed windows. Native process identity, snapshot generation, and window ownership checks do not lock the desktop against simultaneous user or application changes. +`getState` optionally scopes AX traversal to an owned window or nested sheet `windowId`. A new or failed refresh invalidates the previous snapshot for that app/window scope, while other window scopes remain usable. When an owned CG window has no matching AX surface, getState returns a new snapshot with empty tree/elements and no truncation; its coordinates and screenshots remain available. It rechecks process lifetime, window ownership, the unlocked session and Accessibility permission after traversal. Window-discovery APIDisabled and communication errors remain failures; unsupported or absent optional AX attributes can produce an empty observation. Missing or foreign-owned windows never produce an empty success. An AX element invalidated during traversal rejects the whole read with `STALE_SNAPSHOT`; no partial snapshot is published. Callers may read again after the application changes, without repeating input. Raw element targets require `appHandle`, `snapshotId` and an opaque string `elementId`. For `click` and `mouseMove`, the optional `windowId` must match both the snapshot scope (unless app-wide) and the element’s enclosing window or sheet. A mismatch returns `STALE_SNAPSHOT` before dispatch. Coordinate targets use `windowId`, `x` and `y` and may include a snapshot from that window or the app. Mixing an element ID with coordinates returns `INVALID_ARGUMENT`. The Host converts relative coordinates before dispatch, so the native JSONL protocol rejects the `relative` field. Native references never cross JSON. Window-list refresh releases snapshots for closed windows. Native process identity, snapshot generation, and window ownership checks do not lock the desktop against simultaneous user or application changes. Clicks send mouse-down and mouse-up events at the element center or the supplied coordinates. Use `invoke` for the default AX action or `performAction` for a named action. Click, scroll and drag validate window ownership and bounds, then use background PID/window-directed primitives. Each event uses SkyLight when available, otherwise the public PID-targeted API. A posted event is never repeated through the other API. Coordinates are macOS points from the top-left of the entire window, including its title bar. Positive `deltaLinesY` reveals content above; negative reveals below. Positive `deltaLinesX` reveals content left; negative reveals right. Each axis accepts an integer from -10 to 10, and at least one must be nonzero. The helper sends line-unit scroll events directly. Click and drag accept left/right/middle buttons and command/control/option/shift/fn modifiers. Only left clicks support double-click. Drag uses one button press/move/release in one window; callers explicitly supply durationMs (1..5000) and steps (1..240). The movement delay budget is divided with fractional precision across the segments; fixed press/release settling and scheduler latency are additional. No implicit fallback sends global desktop input. @@ -147,7 +147,7 @@ Run `python3 packages/experimental/cua-macos/native/tests/axless_e2e.py` after b Run `python3 packages/experimental/cua-macos/native/tests/observation_benchmark.py --reference-helper /absolute/reference --candidate-helper /absolute/candidate --artifacts /private/evidence` to compare complete observations of the same owned AppKit window with 35 and 400 synthetic rows. Each helper receives one warmup and three alternating measured requests. All result fields must match except helper-generated app and snapshot IDs. The report keeps raw timings, byte counts, hashes and replies; it sets no CI speed threshold and makes no memory claim. Fixture startup can activate the application. Run without other desktop automation or compilation jobs. -Use `--panels-only` to isolate the system file-panel checks. Fixture windows can join application sets so Stage Manager does not shrink them during background input. Use `--hover-only` with `e2e.py` or `electron_e2e.py` to verify hover independently of keyboard and file-panel scenarios. Add `--alternate-hover` to `e2e.py` for AX reveal, descendant entry, click and exit with sampled system-pointer stability. `--input-overlays` verifies that pass-through overlays allow hover and input-blocking overlays reject it without moving the pointer. +Use `--panels-only` to verify that a document-window request for a sheet control reports its sheet ID and that the control can be read from the selected sheet. Fixture windows can join application sets so Stage Manager does not shrink them during background input. Use `--hover-only` with `e2e.py` or `electron_e2e.py` to verify hover independently of keyboard and file-panel scenarios. Add `--alternate-hover` to `e2e.py` for AX reveal, descendant entry, click and exit with sampled system-pointer stability. `--input-overlays` verifies that pass-through overlays allow hover and input-blocking overlays reject it without moving the pointer. Run `python3 packages/experimental/cua-macos/native/tests/e2e.py` after building to exercise an isolated two-window AppKit fixture. It independently records button counts, Unicode text, actual scroll position, right-click callbacks, and drag release position. Scroll, right-button down/up, and drag down/up counters must each increase exactly once per requested gesture. The check also verifies a real NSOpenPanel sheet, path-entry keyboard routing and the selected file URL. It covers scoped snapshot replacement, screenshot dimensions, stale app cleanup and EOF shutdown. `--artifacts DIRECTORY` writes fixture-only screenshots and a private protocol transcript; the transcript contains the running-app inventory and is not a publication artifact. diff --git a/packages/experimental/cua-macos/native/README.zh.md b/packages/experimental/cua-macos/native/README.zh.md index 4d27f100c5..307d2eaee8 100644 --- a/packages/experimental/cua-macos/native/README.zh.md +++ b/packages/experimental/cua-macos/native/README.zh.md @@ -95,7 +95,7 @@ AppKit 在主事件循环上运行。阻塞式 socket 接收与握手使用后 ## 观察与输入 -`getState` 可将 AX 遍历限定到应用所属窗口或嵌套 sheet 的 `windowId`。新的刷新或失败的刷新使同一应用/窗口范围的旧快照失效，其他窗口范围仍可使用。应用所属 CG 窗口没有匹配 AX 界面时，getState 返回空 tree/elements、truncated 为 false 的新快照，仍可截图和坐标输入。遍历后重新检查进程生命期、窗口归属、会话未锁定及辅助功能权限。窗口发现中的 APIDisabled 和通信错误仍会失败；不支持或缺失的可选 AX 属性可以产生空观察。不存在或属于其他应用的窗口不会返回空成功结果。遍历中 AX 元素失效会以 `STALE_SNAPSHOT` 拒绝整次读取，不发布部分树。应用变化后可以重新观察，但不重复输入。原始元素目标要求 `appHandle`、`snapshotId` 和不透明字符串 `elementId`；坐标动作要求 `windowId`，并可携带属于该窗口或应用级观察的快照 ID。原生引用不跨 JSON 传递。刷新窗口列表会释放已关闭窗口的快照。进程身份、快照代次和窗口归属检查不会锁住桌面，用户或应用仍可同时改变状态。 +`getState` 可将 AX 遍历限定到应用所属窗口或嵌套 sheet 的 `windowId`。新的刷新或失败的刷新使同一应用/窗口范围的旧快照失效，其他窗口范围仍可使用。应用所属 CG 窗口没有匹配 AX 界面时，getState 返回空 tree/elements、truncated 为 false 的新快照，仍可截图和坐标输入。遍历后重新检查进程生命期、窗口归属、会话未锁定及辅助功能权限。窗口发现中的 APIDisabled 和通信错误仍会失败；不支持或缺失的可选 AX 属性可以产生空观察。不存在或属于其他应用的窗口不会返回空成功结果。遍历中 AX 元素失效会以 `STALE_SNAPSHOT` 拒绝整次读取，不发布部分树。应用变化后可以重新观察，但不重复输入。原生元素目标需要 `appHandle`、`snapshotId` 和字符串 `elementId`。`click` 和 `mouseMove` 可附带 `windowId`。它必须与元素所属窗口或 sheet 一致，对窗口级快照还必须与快照的窗口一致，否则在发送输入前返回 `STALE_SNAPSHOT`。坐标目标使用 `windowId`、`x`、`y`，可以附带来自该窗口或应用的快照。混用元素 ID 和坐标会返回 `INVALID_ARGUMENT`。Host 在发送请求前换算相对坐标，原生 JSONL 协议会拒绝 `relative` 字段。原生引用不跨 JSON 传递。刷新窗口列表会释放已关闭窗口的快照。进程身份、快照代次和窗口归属检查不会锁住桌面，用户或应用仍可同时改变状态。 点击在元素中心或给定坐标发送鼠标按下和抬起事件。执行默认 AX 动作用 `invoke`，执行指定动作用 `performAction`。点击、滚动和拖动核实窗口归属和边界后，使用后台 PID/窗口定向输入。事件优先使用 SkyLight，不可用时使用公开的 PID 定向 API；已发送的事件不通过另一 API 重发。坐标是整个窗口左上角起算的 macOS point，包含标题栏。`deltaLinesY` 为正时显示上方内容，为负时显示下方内容；`deltaLinesX` 为正时显示左侧内容，为负时显示右侧内容。每个轴接受 -10 到 10 的整数，至少一个轴非零。helper 直接发送按行计量的滚轮事件。点击和拖动支持左、右、中键及 command/control/option/shift/fn 修饰键，只有左键支持双击。拖动在一个窗口中完成一次按下、移动、抬起，调用方显式提供 durationMs（1..5000）和 steps（1..240）。移动延迟按小数精度分配到各段，按下和抬起的固定等待及调度延迟另计。不存在隐式改用全局桌面输入的后备路径。 @@ -147,7 +147,7 @@ AppKit 在主事件循环上运行。阻塞式 socket 接收与握手使用后 运行 `python3 packages/experimental/cua-macos/native/tests/observation_benchmark.py --reference-helper /absolute/reference --candidate-helper /absolute/candidate --artifacts /private/evidence`，对比同一自建 AppKit 窗口中 35 行和 400 行合成内容的完整观察。每个 helper 先预热一次，再交替测量三次。除 helper 生成的应用和快照 ID 外，结果字段必须一致。报告保留原始耗时、字节数、哈希和回复，不设置 CI 性能阈值，也不声称内存表现。fixture 启动可能激活应用。运行时避免其他桌面自动化或编译任务。 -使用 `--panels-only` 单独检查系统文件面板。fixture 窗口可加入应用集合，避免 Stage Manager 在后台输入时缩小它们。使用 `e2e.py` 或 `electron_e2e.py` 的 `--hover-only`，可独立于键盘和文件面板场景验证悬停。为 `e2e.py` 加上 `--alternate-hover` 可验证 AX 展开、进入子控件、点击和移出，并采样确认系统指针不变。`--input-overlays` 验证穿透鼠标的覆盖窗口允许悬停、拦截输入的覆盖窗口拒绝悬停且不移动指针。 +使用 `--panels-only` 验证通过文档窗口操作 sheet 控件时会返回该 sheet 的窗口 ID，并验证选择 sheet 后能读取该控件。fixture 窗口可加入应用集合，避免 Stage Manager 在后台输入时缩小它们。使用 `e2e.py` 或 `electron_e2e.py` 的 `--hover-only`，可独立于键盘和文件面板场景验证悬停。为 `e2e.py` 加上 `--alternate-hover` 可验证 AX 展开、进入子控件、点击和移出，并采样确认系统指针不变。`--input-overlays` 验证穿透鼠标的覆盖窗口允许悬停、拦截输入的覆盖窗口拒绝悬停且不移动指针。 构建后运行 `python3 packages/experimental/cua-macos/native/tests/e2e.py`，操作一个隔离的双窗口 AppKit fixture。fixture 独立记录按钮计数、Unicode 文本、实际滚动位置、右键回调和拖动释放位置。滚动、右键按下/抬起和拖动按下/抬起计数每次手势必须恰好增加一次。检查还验证真实 NSOpenPanel sheet、路径输入的键盘路由和选中文件 URL，并覆盖范围内快照替换、截图尺寸、过期应用清理和 EOF 退出。`--artifacts DIRECTORY` 写出仅包含 fixture 的截图和私有协议记录；协议记录包含运行中应用清单，不用于公开发布。 diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift index 50c8bd3a64..1a1e4f595b 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift @@ -652,7 +652,7 @@ final class AXAccess { } current = try self.element(node, "AXParent") } - throw NativeFailure(code: "WINDOW_UNAVAILABLE", message: "Element has no app-owned window or sheet; refresh getAccessibilityTree") + throw NativeFailure(code: "WINDOW_UNAVAILABLE", message: "Pointer input requires an enclosing window or sheet. For a menu item’s default action, use app.invoke(id)") } func keyboardRecipient(_ owner: ProcessIdentity, catalog: ApplicationCatalogSnapshot) throws -> ProcessIdentity { var current = try element(AXUIElementCreateApplication(owner.pid), "AXFocusedWindow") diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift index ba16630e6a..ea58f27f7e 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift @@ -155,8 +155,8 @@ actor AXWorker { private func coordinateSnapshot(_ handle: String, _ params: RequestParameters, _ windowID: UInt32) throws { guard let id = try params.optionalString("snapshotId") else { return } let value = try snapshot(handle, id) - guard value.scope.windowID == nil || value.scope.windowID == windowID else { - throw NativeFailure(code: "STALE_SNAPSHOT", message: "Snapshot belongs to another window") + if let source = value.scope.windowID, source != windowID { + throw NativeFailure(code: "STALE_SNAPSHOT", message: "Snapshot belongs to window \(source); read getAccessibilityTree on window \(windowID)") } } private func coordinateWindow(_ identity: ProcessIdentity, _ windowID: UInt32, trace: ScreenshotTrace? = nil) throws -> WindowInfo { diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index 94eb893f74..92fd5bed9f 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -43,12 +43,18 @@ def check_file_panel(call, owner, primary_params, primary, directory, wait_state raise AssertionError(description + ": " + observed["tree"]) time.sleep(0.025) - sheet = wait_observation(lambda observed: any(e["identifier"] == "open-panel" for e in observed["elements"]), - "File panel did not appear") - sheet_id = sheet["focusedWindowId"] - assert sheet_id != primary["windowId"], sheet + document = wait_observation(lambda observed: any(e["identifier"] == "OKButton" for e in observed["elements"]), + "File panel control did not appear in the document tree", primary_params) + sheet_id = document["focusedWindowId"] + assert sheet_id is not None and sheet_id != primary["windowId"], document + button = next(e for e in document["elements"] if e["identifier"] == "OKButton") + for method in ("click", "mouseMove"): + error = call(method, {**owner, "windowId": primary["windowId"], "snapshotId": document["snapshotId"], + "elementId": button["elementId"]}, "STALE_SNAPSHOT") + assert f"window {sheet_id}; select that window" in error["message"], error scoped = call("getState", {**primary_params, "windowId": sheet_id}) assert any(e["role"] == "AXSheet" for e in scoped["elements"]), scoped + assert any(e["identifier"] == "OKButton" for e in scoped["elements"]), scoped assert "Increment primary" not in scoped["tree"], scoped panel = next(e for e in scoped["elements"] if e["identifier"] == "open-panel") call("performAction", {**owner, "snapshotId": scoped["snapshotId"], "elementId": panel["elementId"], "action": "AXRaise"}) @@ -74,9 +80,9 @@ def check_file_panel(call, owner, primary_params, primary, directory, wait_state call("pressKey", {**owner, "key": "Return"}) ready = wait_observation(lambda observed: not any(e["identifier"] == "PathTextField" for e in observed["elements"]) and any(e["identifier"] == "OKButton" and e["enabled"] for e in observed["elements"]), - "Go to folder did not select the requested file") + "Go to folder did not select the requested file", {**primary_params, "windowId": sheet_id}) open_button = next(e for e in ready["elements"] if e["identifier"] == "OKButton") - assert call("invoke", {**owner, "snapshotId": ready["snapshotId"], "elementId": open_button["elementId"]})["dispatched"] + assert call("invoke", {**owner, "windowId": sheet_id, "snapshotId": ready["snapshotId"], "elementId": open_button["elementId"]})["dispatched"] wait_state(lambda state: bool(state["selectedFile"]) and Path(state["selectedFile"]).samefile(selected_file), "File picker did not return the intended file") # AppKit can retain the dismissed panel's hidden WindowServer window after @@ -161,12 +167,18 @@ def main(): target = {**button_target(first, "Increment primary"), "windowId": primary["windowId"]} app_state = call("getState", {**owner, "maxElements": 300, "maxDepth": 20}) other_button = button_target(app_state, "Increment secondary") + menu_item = next(e for e in first["elements"] if e["role"] == "AXMenuBarItem") for method in ("click", "mouseMove"): + windowless = call(method, {**target, "elementId": menu_item["elementId"]}, "WINDOW_UNAVAILABLE") + assert "app.invoke(id)" in windowless["message"], windowless unknown = call(method, {**target, "elementId": "9999"}, "INVALID_ARGUMENT") assert "elementId is absent" in unknown["message"], unknown - call(method, {**target, "x": 0, "y": 0}, "INVALID_ARGUMENT") - call(method, {**target, "relative": True}, "INVALID_ARGUMENT") - call(method, {**target, "windowId": secondary["windowId"]}, "STALE_SNAPSHOT") + combined = call(method, {**target, "x": 0, "y": 0}, "INVALID_ARGUMENT") + assert combined["message"] == "Element and coordinate targets cannot be combined", combined + raw_relative = call(method, {**target, "relative": True}, "INVALID_ARGUMENT") + assert raw_relative["message"] == "Unknown operation parameter", raw_relative + stale = call(method, {**target, "windowId": secondary["windowId"]}, "STALE_SNAPSHOT") + assert f'read getAccessibilityTree on window {secondary["windowId"]}' in stale["message"], stale call(method, {**other_button, "windowId": primary["windowId"]}, "STALE_SNAPSHOT") call("click", target) wait_state(lambda state: state["count"] == 1, "primary click was not received exactly once") diff --git a/packages/experimental/cua-macos/src/guidance.ts b/packages/experimental/cua-macos/src/guidance.ts index 6624bde4b7..e2127a78ab 100644 --- a/packages/experimental/cua-macos/src/guidance.ts +++ b/packages/experimental/cua-macos/src/guidance.ts @@ -2,7 +2,7 @@ import { COMMON_GUIDANCE } from '@deepseek-ai/dsh-cua/guidance' /** macOS operation levels, permissions, selectors, and native measurement units. */ -export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' +export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' /** Complete model instructions for this provider. */ export const GUIDANCE = `${PLATFORM_GUIDANCE} diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index b34c3def7c..6e32a6a9d3 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -33,7 +33,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/snapshots/session/cua-native/workspace/native-helper.mjs b/snapshots/session/cua-native/workspace/native-helper.mjs index 494e7d2c1a..b10d4793e0 100755 --- a/snapshots/session/cua-native/workspace/native-helper.mjs +++ b/snapshots/session/cua-native/workspace/native-helper.mjs @@ -197,7 +197,7 @@ function createApplicationState(socket, record, application, captureBroker, conn ? snapshots.get(params.windowId) ?? snapshots.get(null) : [...snapshots.values()].find(snapshot => snapshot.snapshotId === params.snapshotId) if (!observed) refuse('STALE_SNAPSHOT', 'Read fresh state before acting') - if (params.windowId !== undefined && observed.windowId !== null && observed.windowId !== params.windowId) refuse('STALE_SNAPSHOT', 'Snapshot belongs to another window') + if (params.windowId !== undefined && observed.windowId !== null && observed.windowId !== params.windowId) refuse('STALE_SNAPSHOT', `Snapshot belongs to window ${observed.windowId}; read getAccessibilityTree on window ${params.windowId}`) if (observed.axless && params.elementId !== undefined) refuse('INVALID_ARGUMENT', 'Element id is absent from the selected snapshot') if (method === 'mouseMove') application.hovering = params.elementId === '0' || (params.x >= 20 && params.x <= 40) else if (method === 'click' && params.windowId === 8) application.visualCount++ diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index 00451355c7..81450a3782 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index b2abdbb7e7..743dd9cc1f 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. From 3b677fdf1a448dc184779a095790170ead64405d Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 14:44:04 +0800 Subject: [PATCH 12/44] Exercise stale window recovery through the CUA object API --- .../experimental/cua-macos/tests/loader-composition.spec.ts | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/packages/experimental/cua-macos/tests/loader-composition.spec.ts b/packages/experimental/cua-macos/tests/loader-composition.spec.ts index cfd7fe4883..e92f138d87 100644 --- a/packages/experimental/cua-macos/tests/loader-composition.spec.ts +++ b/packages/experimental/cua-macos/tests/loader-composition.spec.ts @@ -181,6 +181,11 @@ it.skipIf(process.platform === 'win32')('shares one application between retained const screenshot = await peerWindow.getScreenshot({ emit: false }) expect(screenshot).toMatchObject({ mimeType: 'image/png', pixelWidth: 1, pixelHeight: 1, windowId: 7 }) expect(Buffer.from(screenshot.bytes).toString('base64')).toBe(png) + await peerApp.getWindow(8) + const recovery = 'Snapshot belongs to window 8; read getAccessibilityTree on window 7' + await expect(peerWindow.click(element('0'))).rejects.toThrow(recovery) + await expect(peerWindow.mouseMove(element('0'))).rejects.toThrow(recovery) + expect(await peerWindow.getAccessibilityTree({ emit: false })).toContain('Count: 1') await peerWindow.click(element('0')) expect(await retainedWindow.getAccessibilityTree({ emit: false })).toContain('Count: 2') const sharedEvents = await readEvents() From 84bd8fd7bafb0bfaaa00515c4a9be6686b03e954 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 14:46:50 +0800 Subject: [PATCH 13/44] Check the file panel after its Open button becomes actionable --- .../cua-macos/native/tests/e2e.py | 31 ++++++++++--------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index 92fd5bed9f..830f767d80 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -43,18 +43,12 @@ def check_file_panel(call, owner, primary_params, primary, directory, wait_state raise AssertionError(description + ": " + observed["tree"]) time.sleep(0.025) - document = wait_observation(lambda observed: any(e["identifier"] == "OKButton" for e in observed["elements"]), - "File panel control did not appear in the document tree", primary_params) - sheet_id = document["focusedWindowId"] - assert sheet_id is not None and sheet_id != primary["windowId"], document - button = next(e for e in document["elements"] if e["identifier"] == "OKButton") - for method in ("click", "mouseMove"): - error = call(method, {**owner, "windowId": primary["windowId"], "snapshotId": document["snapshotId"], - "elementId": button["elementId"]}, "STALE_SNAPSHOT") - assert f"window {sheet_id}; select that window" in error["message"], error + sheet = wait_observation(lambda observed: any(e["identifier"] == "open-panel" for e in observed["elements"]), + "File panel did not appear") + sheet_id = sheet["focusedWindowId"] + assert sheet_id is not None and sheet_id != primary["windowId"], sheet scoped = call("getState", {**primary_params, "windowId": sheet_id}) assert any(e["role"] == "AXSheet" for e in scoped["elements"]), scoped - assert any(e["identifier"] == "OKButton" for e in scoped["elements"]), scoped assert "Increment primary" not in scoped["tree"], scoped panel = next(e for e in scoped["elements"] if e["identifier"] == "open-panel") call("performAction", {**owner, "snapshotId": scoped["snapshotId"], "elementId": panel["elementId"], "action": "AXRaise"}) @@ -78,10 +72,19 @@ def check_file_panel(call, owner, primary_params, primary, directory, wait_state wait_observation(lambda observed: any(e["identifier"] == "PathTextField" and e["value"] == str(selected_file) for e in observed["elements"]), "Panel text input did not arrive") call("pressKey", {**owner, "key": "Return"}) - ready = wait_observation(lambda observed: not any(e["identifier"] == "PathTextField" for e in observed["elements"]) - and any(e["identifier"] == "OKButton" and e["enabled"] for e in observed["elements"]), - "Go to folder did not select the requested file", {**primary_params, "windowId": sheet_id}) - open_button = next(e for e in ready["elements"] if e["identifier"] == "OKButton") + def open_button_ready(observed): + return (not any(e["identifier"] == "PathTextField" for e in observed["elements"]) + and any(e["identifier"] == "OKButton" and e["enabled"] for e in observed["elements"])) + + document = wait_observation(open_button_ready, "Selected file did not enable Open in the document tree", + {**primary_params, "maxElements": 2000, "maxDepth": 25}) + button = next(e for e in document["elements"] if e["identifier"] == "OKButton") + for method in ("click", "mouseMove"): + error = call(method, {**owner, "windowId": primary["windowId"], "snapshotId": document["snapshotId"], + "elementId": button["elementId"]}, "STALE_SNAPSHOT") + assert f"window {sheet_id}; select that window" in error["message"], error + ready = call("getState", {**owner, "windowId": sheet_id, "maxElements": 2000, "maxDepth": 25}) + open_button = next(e for e in ready["elements"] if e["identifier"] == "OKButton" and e["enabled"]) assert call("invoke", {**owner, "windowId": sheet_id, "snapshotId": ready["snapshotId"], "elementId": open_button["elementId"]})["dispatched"] wait_state(lambda state: bool(state["selectedFile"]) and Path(state["selectedFile"]).samefile(selected_file), "File picker did not return the intended file") From eba321d9185bd7d5d3c4dddd64744c0a0feb8860 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 14:48:28 +0800 Subject: [PATCH 14/44] Open the fixture sheet through its button action before testing pointer scope --- packages/experimental/cua-macos/native/tests/e2e.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index 830f767d80..95776b4031 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -20,7 +20,7 @@ from native_test_support import ROOT, NativeHelper, line, run_cleanup, stop def check_file_panel(call, owner, primary_params, primary, directory, wait_state, button_target): selected_file = Path(directory) / "sheet-target.txt" selected_file.write_text("Owned file picker fixture\n") - call("click", button_target(call("getState", primary_params), "Open fixture file")) + assert call("invoke", button_target(call("getState", primary_params), "Open fixture file"))["dispatched"] # A remote AX traversal can consume the helper's 15-second response budget. # Allow two completed observations rather than timing out after the first. From 09b90ed11272465df53a5aa7f988ba41e515b7c8 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 15:20:30 +0800 Subject: [PATCH 15/44] Keep macOS hover off the system pointer by default and route file panel clicks --- .../2026-10-13-cua-continuous-hover.i18n.yaml | 4 +- .../2026-10-13-cua-continuous-hover.md | 2 +- .../2026-10-13-cua-continuous-hover.zh.md | 2 +- ...cua-hover-routes-and-coordinates.i18n.yaml | 4 +- ...-10-13-cua-hover-routes-and-coordinates.md | 2 +- ...-13-cua-hover-routes-and-coordinates.zh.md | 2 +- docs/config-catalog.md | 2 + docs/config-catalog.zh.md | 2 + .../cua-hover-ax-only/guide.i18n.yaml | 16 ++++++ .../v0.2.2-rc.1/cua-hover-ax-only/guide.md | 18 ++++++ .../v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md | 18 ++++++ .../experimental/cua-macos/README.i18n.yaml | 8 +-- packages/experimental/cua-macos/README.md | 13 +++-- packages/experimental/cua-macos/README.zh.md | 13 +++-- .../cua-macos/native/README.i18n.yaml | 8 +-- .../experimental/cua-macos/native/README.md | 8 +-- .../cua-macos/native/README.zh.md | 8 +-- .../native/Sources/CuaNative/AXAccess.swift | 56 +++++++++++-------- .../native/Sources/CuaNative/AXHover.swift | 6 ++ .../native/Sources/CuaNative/AXWorker.swift | 17 +++++- .../CuaNative/NativeProtocol.generated.swift | 2 +- .../CuaNative/RequestCoordinator.swift | 2 +- .../cua-macos/native/generate-protocol.ts | 2 + .../cua-macos/native/protocol.schema.json | 6 ++ .../cua-macos/native/tests/e2e.py | 16 +++++- .../cua-macos/native/tests/electron_e2e.py | 8 ++- .../native/tests/swift/SystemHoverTests.swift | 11 ++++ .../experimental/cua-macos/src/guidance.ts | 2 +- packages/experimental/cua-macos/src/index.ts | 4 ++ .../cua-macos/tests/guidance.spec.ts | 3 + .../tests/protocol-generation.spec.ts | 1 + .../tests/provider-lifecycle.spec.ts | 14 ++++- packages/experimental/cua/src/guidance.ts | 6 +- packages/experimental/cua/src/index.ts | 3 + .../experimental/cua/src/native-schema.ts | 6 +- packages/experimental/cua/src/types.ts | 12 ++-- .../session/cua-native/replay.override.json | 33 +++++++++++ .../cua-native/system-prompt.expected.md | 8 +-- .../cua-native/workspace/native-helper.mjs | 12 +++- .../session/cua-ptc/system-prompt.expected.md | 8 +-- .../system-prompt.expected.md | 8 +-- 41 files changed, 278 insertions(+), 98 deletions(-) create mode 100644 docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.i18n.yaml create mode 100644 docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.md create mode 100644 docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md diff --git a/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.i18n.yaml b/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.i18n.yaml index dba92660c8..04a78f70d3 100644 --- a/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.i18n.yaml @@ -9,8 +9,8 @@ en: cdc04139c8d69415 zh: a3fe3e78ada08d95 /agent-note-continuous-cua-hover-without-cross-call-ownership/decision: - en: d1ea8fdc05927896 - zh: d3039bbb82bc949c + en: 125c860c1c568960 + zh: 8a2ece1101358200 /agent-note-continuous-cua-hover-without-cross-call-ownership/alternatives-considered: en: 18d917ee2e909b93 zh: 5b5ad0530f138cda diff --git a/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.md b/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.md index 1cc8db5f81..2cce428cff 100644 --- a/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.md +++ b/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.md @@ -10,7 +10,7 @@ Hover interactions span movement, delayed content, observation and input inside ## Decision -An explicit Window exposes `mouseMove` for coordinates or an observed element ID. The result reports `hoverRoute` and `systemPointerMoved`. It never presses a mouse button; disabled controls can use system-pointer movement. Completion, AX reads and screenshots do not restore a previous pointer position. Successive calls can visit intermediate points before entering a popup. Clicks deliver button events at their target without an off-window priming click. The [input-routing decision](2026-10-13-cua-hover-routes-and-coordinates.md) selects AX alternate UI before falling back to system movement with foreground and mouse-hit checks. It never activates or raises the app implicitly. Other input or window changes can race that check. +An explicit Window exposes `mouseMove` for coordinates or an observed element ID. The result reports `hoverRoute` and `systemPointerMoved`. It never presses a mouse button; disabled controls can use system-pointer movement with `systemPointerHover` enabled. Completion, AX reads and screenshots do not restore a previous pointer position. Successive calls can visit intermediate points before entering a popup. Clicks deliver button events at their target without an off-window priming click. The [input-routing decision](2026-10-13-cua-hover-routes-and-coordinates.md) selects AX alternate UI before allowing opt-in system movement with foreground and mouse-hit checks. It never activates or raises the app implicitly. Other input or window changes can race that check. The [shared helper](2026-10-09-cua-shared-native-helper.md) serializes complete individual operations. It grants no application ownership across calls. Earlier arrival, a retained app object and Session lifetime do not establish task priority. Waiting for the model does not reserve the application. Explicit task coordination determines any required execution order. diff --git a/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.zh.md b/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.zh.md index 2f5ef0ce36..cc2457a70a 100644 --- a/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.zh.md +++ b/.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.zh.md @@ -10,7 +10,7 @@ Status: implemented ## 决策 -显式窗口提供 `mouseMove`，接受窗口坐标或已观察元素的字符串 ID，结果用 `hoverRoute` 和 `systemPointerMoved` 报告执行路径。悬停不按下鼠标按钮，禁用控件可使用系统鼠标移动路径。调用结束、AX 读取和截图不恢复先前的鼠标位置。进入浮层前可连续调用并经过中间位置。点击只在目标位置发送按键事件，不先在窗口外点击。 [输入路径决定](2026-10-13-cua-hover-routes-and-coordinates.zh.md)优先选择 AX 替代界面，再回退到经过前台和鼠标命中检查的系统移动，不隐式激活或抬起应用。其他输入或窗口变化仍可能与检查交错。 +显式窗口提供 `mouseMove`，接受窗口坐标或已观察元素的字符串 ID，结果用 `hoverRoute` 和 `systemPointerMoved` 报告执行路径。悬停不按下鼠标按钮，启用 `systemPointerHover` 后可通过系统鼠标悬停禁用控件。调用结束、AX 读取和截图不恢复先前的鼠标位置。进入浮层前可连续调用并经过中间位置。点击只在目标位置发送按键事件，不先在窗口外点击。 [输入路径决定](2026-10-13-cua-hover-routes-and-coordinates.zh.md)优先选择 AX 替代界面，只有启用该配置时才回退到经过前台和鼠标命中检查的系统移动，不隐式激活或抬起应用。其他输入或窗口变化仍可能与检查交错。 [共享 helper](2026-10-09-cua-shared-native-helper.zh.md)串行执行完整的单次操作，不分配跨调用的应用所有权。先发起请求、持有 app 对象和 Session 寿命均不代表任务优先级。等待模型期间不占用应用。需要确定执行顺序时，由明确的任务协调决定。 diff --git a/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.i18n.yaml b/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.i18n.yaml index 8259c024fb..9775baf3a4 100644 --- a/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.i18n.yaml @@ -9,8 +9,8 @@ en: f96bf1ebc0719fb0 zh: e9a1144f0e76572d /agent-note-background-hover-and-screenshot-coordinates/decision: - en: 6ac0b77bea25fcf7 - zh: e50fec9049926fdc + en: bdf4d3386521ce00 + zh: 96db93177ad96a76 /agent-note-background-hover-and-screenshot-coordinates/alternatives-considered: en: 5fc4beaf79761c50 zh: 4948333b0020be28 diff --git a/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.md b/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.md index 1091fcaf5e..c385525731 100644 --- a/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.md +++ b/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.md @@ -10,7 +10,7 @@ Hover-only controls can sometimes be revealed without moving the user's pointer. ## Decision -Native hover first uses a same-window element or ancestor advertising both `AXShowAlternateUI` and `AXShowDefaultUI`. Entering its revealed descendants preserves the parent UI; leaving restores it. An unavailable route falls back to the real system pointer. An AX action failure propagates because received effects may be uncertain. The result records the route and whether the system pointer moved. System movement requires the selected foreground app and AppKit's mouse hit test, which excludes input-transparent overlays. No route grants [cross-call ownership](2026-10-13-cua-continuous-hover.md). +Native hover first uses a same-window element or ancestor advertising both `AXShowAlternateUI` and `AXShowDefaultUI`. Entering its revealed descendants preserves the parent UI; leaving restores it. An unavailable route returns `HOVER_UNAVAILABLE` by default. Enabling `systemPointerHover` permits the real system pointer fallback. An AX action failure propagates because received effects may be uncertain. The result records the route and whether the system pointer moved. System movement requires the selected foreground app and AppKit's mouse hit test, which excludes input-transparent overlays. No route grants [cross-call ownership](2026-10-13-cua-continuous-hover.md). Model-facing coordinate input defaults to pixels in the returned screenshot. `relative: true` accepts integer thousandths from 0 to 999 for `click`, `mouseMove`, `scroll` and `drag`; it does not apply to opaque element IDs. The Host owns the mapping to native window points. An application-wide observation refreshes its inventory; a scoped observation preserves other windows' image mappings. Window resizing invalidates the old image, while movement preserves local coordinates. Relative endpoints remain inside the window and do not depend on screenshot resolution. diff --git a/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.zh.md b/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.zh.md index 2344d61c95..0d3db32400 100644 --- a/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.zh.md +++ b/.agents/notes/implemented/architecture/2026-10-13-cua-hover-routes-and-coordinates.zh.md @@ -10,7 +10,7 @@ Status: implemented ## 决策 -原生悬停优先寻找同一窗口内同时声明 `AXShowAlternateUI` 和 `AXShowDefaultUI` 的元素或祖先。进入其展开区域的子控件时保持父控件展开，离开后恢复。没有可用路径时回退到真实系统鼠标。AX 动作失败直接返回，因为已产生的效果可能不确定。结果记录实际路径及是否移动系统鼠标。系统移动要求所选应用处于前台，并使用 AppKit 的鼠标命中查询排除不接收输入的覆盖窗口。所有路径都遵守[跨调用不分配所有权](2026-10-13-cua-continuous-hover.zh.md)的决定。 +原生悬停优先寻找同一窗口内同时声明 `AXShowAlternateUI` 和 `AXShowDefaultUI` 的元素或祖先。进入其展开区域的子控件时保持父控件展开，离开后恢复。没有可用路径时默认返回 `HOVER_UNAVAILABLE`。启用 `systemPointerHover` 后才允许移动系统光标。AX 动作失败直接返回，因为已产生的效果可能不确定。结果记录实际路径及是否移动系统鼠标。系统移动要求所选应用处于前台，并使用 AppKit 的鼠标命中查询排除不接收输入的覆盖窗口。所有路径都遵守[跨调用不分配所有权](2026-10-13-cua-continuous-hover.zh.md)的决定。 面向模型的坐标默认使用返回截图中的像素。`click`、`mouseMove`、`scroll` 和 `drag` 传入 `relative: true` 时接受 0～999 的千分比整数，仅用于坐标目标。Host 负责换算原生窗口 point。应用级观察刷新窗口清单，单窗口观察保留其他窗口的图片映射。窗口尺寸变化使旧图片失效，移动窗口保留局部坐标。相对坐标的端点保持在窗口内，不随截图分辨率改变。 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 7562c8bf96..e6fafb2ae4 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1119,6 +1119,8 @@ export interface Config { export interface Config extends CuaConfig { /** Interval in milliseconds between permission and position checks while the native guide is open. */ permissionPollIntervalMs: number + /** Allow mouseMove to move the visible system pointer when AX hover is unavailable. Off by default to avoid interrupting the user. */ + systemPointerHover: boolean /** * Source-build signing: `ad-hoc` (default) binds grants to the code hash; * an object selects an existing certificate in an explicitly configured keychain. diff --git a/docs/config-catalog.zh.md b/docs/config-catalog.zh.md index 0d75b6e523..7aa0c7245c 100644 --- a/docs/config-catalog.zh.md +++ b/docs/config-catalog.zh.md @@ -1121,6 +1121,8 @@ export interface Config { export interface Config extends CuaConfig { /** Interval in milliseconds between permission and position checks while the native guide is open. */ permissionPollIntervalMs: number + /** Allow mouseMove to move the visible system pointer when AX hover is unavailable. Off by default to avoid interrupting the user. */ + systemPointerHover: boolean /** * Source-build signing: `ad-hoc` (default) binds grants to the code hash; * an object selects an existing certificate in an explicitly configured keychain. diff --git a/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.i18n.yaml b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.i18n.yaml new file mode 100644 index 0000000000..69471af235 --- /dev/null +++ b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.i18n.yaml @@ -0,0 +1,16 @@ +# Bilingual-pair consistency record for guide.md (docs/i18n/README.md): per heading +# section, a hash of its English and Chinese blocks outside code blocks and generated regions. +# After editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.md +/: + en: 5ebb4c8a6b20b644 + zh: 22a8983d06fa5789 +/macos-hover-keeps-the-system-pointer-in-place-by-default: + en: 6833a7851486f10d + zh: 22983f326fb725ce +/macos-hover-keeps-the-system-pointer-in-place-by-default/change: + en: 55c3070b24b4ad39 + zh: 2555d127b0f4e4a0 +/macos-hover-keeps-the-system-pointer-in-place-by-default/migration: + en: bfa15519fd2afb89 + zh: b350e3a7aa2aaa7a diff --git a/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.md b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.md new file mode 100644 index 0000000000..fdd3d811eb --- /dev/null +++ b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.md @@ -0,0 +1,18 @@ +--- +kind: upgrade-guide +description: "macOS mouseMove requires AX alternate UI by default and returns HOVER_UNAVAILABLE instead of moving the user's cursor." +--- + +# macOS hover keeps the system pointer in place by default + +English | [中文](guide.zh.md) + +## Change + +On macOS, `win.mouseMove(target)` uses an enabled element or ancestor that advertises both `AXShowAlternateUI` and `AXShowDefaultUI`. Without that route, previous versions moved the visible system cursor. The default now returns `HOVER_UNAVAILABLE` before sending input. + +## Migration + +1. For an available control, use its advertised action or click it directly. Use browser tools for web content that requires page hover. +2. To restore visible pointer movement, set `systemPointerHover: true` in the `config` of the macOS provider entry (`id: cua`) in your profile's `cordis.patch.yml`. Preserve the entry's other settings. Restart the profile. +3. Call `win.mouseMove` on a foreground, unobscured control without AX alternate UI. The opt-in result reports `hoverRoute: "system-pointer"` and `systemPointerMoved: true`. Keep the default to avoid disturbing the user's cursor. diff --git a/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md new file mode 100644 index 0000000000..7202988db2 --- /dev/null +++ b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md @@ -0,0 +1,18 @@ +--- +kind: upgrade-guide +description: "macOS 的 mouseMove 默认要求控件支持 AX 备用界面，否则返回 HOVER_UNAVAILABLE，保留用户的光标位置。" +--- + +# macOS 悬停默认保留系统光标位置 + +[English](guide.md) | 中文 + +## 变更 + +在 macOS 上，`win.mouseMove(target)` 会查找已启用且同时声明 `AXShowAlternateUI` 和 `AXShowDefaultUI` 的控件或祖先。找不到时，旧版会移动系统光标，新版默认在发送输入前返回 `HOVER_UNAVAILABLE`。 + +## 迁移 + +1. 对已经可用的控件，可执行它声明的动作或直接点击。需要网页悬停时可用浏览器工具。 +2. 要恢复可见光标移动，在 profile 的 `cordis.patch.yml` 中，给 macOS 提供方条目（`id: cua`）的 `config` 添加 `systemPointerHover: true`。保留该条目的其他设置，重启 profile。 +3. 对前台未受遮挡且不提供 AX 备用界面的控件调用 `win.mouseMove`。启用配置时，结果为 `hoverRoute: "system-pointer"` 和 `systemPointerMoved: true`。要保留用户的光标位置，使用默认配置。 diff --git a/packages/experimental/cua-macos/README.i18n.yaml b/packages/experimental/cua-macos/README.i18n.yaml index 76c65f408d..37458210d5 100644 --- a/packages/experimental/cua-macos/README.i18n.yaml +++ b/packages/experimental/cua-macos/README.i18n.yaml @@ -15,11 +15,11 @@ en: 1777e7bf6785f986 zh: e3daac1074ebb74d /deepseek-ai-dsh-experimental-cua-macos/use-this-package: - en: 7fdb7c0462275417 - zh: 0a4a9562f4a82945 + en: 533ed4dd11af5126 + zh: 03036e8c949d6aca /deepseek-ai-dsh-experimental-cua-macos/use-this-package/connect-an-external-execution-environment: - en: 740031b2bf73b350 - zh: 30c4929c5bb6baeb + en: 0da6d36bdf117089 + zh: 0d3692ab25ac2923 /deepseek-ai-dsh-experimental-cua-macos/understand-the-implementation: en: 4e82b85a5aa72eee zh: 4e004b7a205d8688 diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index 7a96cac17e..ce128bab4f 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -64,6 +64,7 @@ Source builds default to `sourceSigning: ad-hoc`, without creating certificates | `buildTimeoutMs` | `300000` | Maximum native preparation time. | | `timeoutMs` | `30000` | Per-operation deadline; cancellation can leave partial input. | | `connectTimeoutMs` | `5000` | Deadline for connecting to or starting the shared helper. | +| `systemPointerHover` | `false` | Allow `mouseMove` to move the visible cursor when AX hover is unavailable. | | `permissionPollIntervalMs` | `1000` | Interval in milliseconds between permission and position checks while the native guide is open. | | `screenshotTimeoutMs` | `5000` | Screenshot deadline including queue time, capped by `timeoutMs`. | | `axTimeoutMs` | `3000` | Deadline per Accessibility system call, configured independently of the complete operation. | @@ -114,7 +115,7 @@ Use the `cua` API to discover and select the application (`listApps`, `getState` `getWindow` reads its initial tree silently. An owned window without an AX surface returns an empty snapshot and remains available for screenshots and coordinate input. Within one API instance, the facade keeps one current accessibility snapshot per application handle, shared by every app and window object for that handle, including repeated selections. Reading any window replaces that snapshot; element operations use the latest snapshot ID, so use IDs from the latest tree and do not assume an older ID still identifies the same control. A failed read publishes no partial snapshot; read again without repeating the preceding input. Window objects own screenshots and pointer input. Keyboard methods stay on `app` because input follows the app’s current focus. -`mouseMove(target)` first reveals advertised AX alternate UI without moving the system cursor. Entering a revealed descendant preserves its parent UI, and leaving restores the default UI. When unavailable, the call moves the visible system cursor to an AX element center or window-local point without pressing a button, including over disabled controls. It requires the selected app to be foreground and the target point unobscured; otherwise `FOREGROUND_REQUIRED` requires explicit activation or raising and a fresh observation. Completion, AX reads and screenshots do not restore the previous point. Wait for the application’s hover delay, observe the revealed content, then hover into it and click; successive calls can visit intermediate points. Other input or application changes can interrupt the interaction. [Continuous-interaction rationale](../../../.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.md). +`mouseMove(target)` defaults to AX hover. An enabled target or ancestor must advertise both `AXShowAlternateUI` and `AXShowDefaultUI`. Moving into its descendants keeps the UI open, and leaving restores it. Without an AX route the helper returns `HOVER_UNAVAILABLE` before sending input. Click a visible control or use `performAction` with an advertised action. For browser content without these AX actions, use browser tools. Enable `systemPointerHover` only when moving the user’s visible cursor is acceptable. That route requires a foreground, unobscured target and leaves the cursor at the destination. `paste(text, { format: "text" | "html" })` temporarily publishes the requested clipboard format and invokes the application’s enabled standard Paste menu command. It keeps the temporary format until a pasteboard reader requests its data, then restores every captured original format while it still owns the clipboard, preserving a later copy. Paste operations share a cross-process lock and cancellation joins helper cleanup. A forced process exit can prevent restoration. Unsupported Paste commands fail without typing fallback; `typeText` remains separate key-event input. @@ -122,7 +123,7 @@ Use the `cua` API to discover and select the application (`listApps`, `getState` Screenshot capture requires an attachment store and a resolved model route declaring image input. The shared attachment admission validates canonical Base64, the raster and its configured capacity, then stores the image once. Native data remains available to programmatic callers; model content carries coordinate metadata and the durable image reference. Response byte limits apply before parsing. -Use `win.click(id)` to send mouse events at the center of an element, or `win.click([x, y])` to click a point. For both `click(id)` and `mouseMove(id)`, the element must belong to the selected window. Use `invoke(id)` to activate a control through accessibility, for example the Open button in a system file panel or a menu item in the app tree. +Use `win.click(id)` to send mouse events at the center of an element, or `win.click([x, y])` to click a point. For both `click(id)` and `mouseMove(id)`, the element must belong to the selected window. Prefer `invoke(id)` for a background control’s default action. For mouse input, activate the application and observe the target first. Mouse clicks on system Open and Save sheets are routed to their verified panel-service window. Absolute coordinates for `mouseMove`, `click`, and `drag` use `[x, y]` tuples; `scroll` uses named `x` and `y`. All use pixels in the latest returned screenshot. Pass `relative: true` for integer thousandths of window width and height, from 0 to 999; omission or `false` uses pixels. Relative input can use a successful AX read or screenshot, while absolute input requires a screenshot. The Host maps the delivered image to native points, including attachment and model-request resizing. A value of 999 remains inside the window edge. Coordinate options do not apply to element IDs. A width or height change since the latest observation returns `WINDOW_CHANGED` before input, including raw coordinate calls without a snapshot ID; read fresh AX or take a new screenshot before proceeding. Window movement alone preserves window-local coordinates. Capture and pointer input reject known disagreement between the selected window’s accessibility frame and displayed bounds with `WINDOW_CHANGED`, before posting pointer events. This includes Stage Manager transformations when AX retains the full window frame. Read fresh state; any activation or window raise remains explicit. Advertised AX actions remain available. Missing AX geometry retains the AXless path and does not establish that a transformed window can be operated. @@ -175,7 +176,7 @@ The plugin contributes one `cua` operation tool with method-specific parameters. ##### Operation guidance (verbatim) ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -209,12 +210,12 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then call app.activate() and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index ca39bab33c..2801e0ccbc 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -64,6 +64,7 @@ pnpm dsh headless --patch packages/experimental/cua-macos/examples/macos.cordis. | `buildTimeoutMs` | `300000` | 原生组件准备的最长时间。 | | `timeoutMs` | `30000` | 单次操作截止时间，取消后可能已发生部分输入。 | | `connectTimeoutMs` | `5000` | 连接或启动共享 helper 的截止时间。 | +| `systemPointerHover` | `false` | AX 悬停不可用时，允许 `mouseMove` 移动可见光标。 | | `permissionPollIntervalMs` | `1000` | 原生引导打开期间检查权限和窗口位置的间隔，单位为毫秒。 | | `screenshotTimeoutMs` | `5000` | 截图总期限，包含排队时间，且不超过 `timeoutMs`。 | | `axTimeoutMs` | `3000` | 每次 Accessibility 系统调用的截止时间，与完整操作截止独立配置。 | @@ -114,7 +115,7 @@ void await window.getScreenshot() `getWindow` 静默读取初始树。已确认归属但没有 AX 界面的窗口返回空快照，仍可截图和发送坐标输入。在单个 API 实例内，门面按应用句柄保存一份当前辅助功能快照，该句柄的所有应用对象和窗口对象共享它，包括重复选择。读取任一窗口都会替换该快照；元素操作使用最新快照 ID，因此应使用最新树中的元素 ID，不应假定旧 ID 仍指向同一控件。读取失败不会发布部分快照；可以重新读取，但不要重复此前的输入。窗口对象负责截图和指针输入。键盘方法保留在 `app` 上，因为输入跟随应用的当前焦点。 -`mouseMove(target)` 优先通过控件声明的 AX 动作展开替代界面，不移动系统鼠标。进入展开区域的子控件时保持父控件展开，离开后恢复默认界面。没有这条路径时，该方法将可见系统鼠标移到 AX 元素中心或窗口内坐标，不按鼠标按钮，也可用于禁用控件。所选应用必须在前台，且目标点未被遮挡，否则返回 `FOREGROUND_REQUIRED`，调用方需显式激活或抬起窗口并重新观察。调用结束、AX 读取和截图不恢复先前的位置。等待应用的悬停延迟后，观察出现的内容，再移入并点击；连续调用可经过中间位置。其他输入或应用自身变化可能中断交互。[连续交互的设计理由](../../../.agents/notes/implemented/architecture/2026-10-13-cua-continuous-hover.zh.md)。 +`mouseMove(target)` 默认通过 AX 悬停。目标或其祖先必须已启用，并同时声明 `AXShowAlternateUI` 和 `AXShowDefaultUI`。进入子控件会保持展开，离开后恢复默认界面。如果没有可用的 AX 路径，helper 返回 `HOVER_UNAVAILABLE`，不发送输入。可点击已经可见的控件，或用 `performAction` 执行控件声明的动作。浏览器内容缺少这些 AX 动作时，请使用浏览器工具。只有允许移动用户可见光标时才开启 `systemPointerHover`。这条路径要求目标在前台且未被遮挡，操作后光标停在目标位置。 `paste(text, { format: "text" | "html" })` 临时提供指定格式的剪贴板内容，并调用应用已启用的标准 Paste 菜单命令。临时格式保留到剪贴板读取方请求其数据，然后只在仍持有剪贴板时恢复保存的全部原格式，保留期间产生的新复制内容。粘贴操作共用跨进程锁，取消会等待辅助进程清理。强制终止进程可能阻止恢复。应用不支持 Paste 命令时明确失败，不自动改成逐字输入；`typeText` 仍单独提供按键事件输入。 @@ -122,7 +123,7 @@ void await window.getScreenshot() 截图要求附件服务可用，并确认当前模型支持图片输入。共享附件接入校验规范 Base64、图像数据及配置容量，然后保存一次图片。程序调用方仍可取得原生数据，模型内容包含坐标元数据和持久图片引用。响应字节上限在解析前执行。 -`win.click(id)` 向元素中心发送鼠标事件，`win.click([x, y])` 向给定位置发送鼠标事件。`click(id)` 和 `mouseMove(id)` 使用的元素须属于所选窗口。要通过辅助功能触发控件的默认动作，可用 `invoke(id)`，例如系统文件面板的“打开”按钮，或应用树中的菜单项。 +`win.click(id)` 向元素中心发送鼠标事件，`win.click([x, y])` 向给定位置发送鼠标事件。`click(id)` 和 `mouseMove(id)` 使用的元素须属于所选窗口。后台控件优先用 `invoke(id)` 执行默认动作。需要鼠标输入时，先激活应用并重新观察目标。系统打开或保存面板的鼠标点击投递给已核实的文件面板服务窗口。 `mouseMove`、`click` 和 `drag` 的绝对坐标使用 `[x, y]` 元组；`scroll` 使用命名的 `x` 和 `y`。它们都使用最近返回截图中的像素。传入 `relative: true` 时使用 0～999 的千分比整数；省略或传入 `false` 时使用像素。相对坐标可基于成功的 AX 读取或截图，绝对坐标必须先截图。Host 根据实际送入模型的图片尺寸换算原生 point，包含附件归一化和模型请求阶段的缩放。999 映射到窗口边缘以内。坐标选项不适用于元素 ID。窗口宽高与最近一次观察不同时，在输入前返回 `WINDOW_CHANGED`，即使原始坐标调用不传快照 ID 也会检查；重新读取 AX 或截图后再操作。单纯移动窗口不使窗口内坐标失效。截图和鼠标输入会核对所选窗口的辅助功能框与显示边界；已知两者不一致时，在发送任何鼠标事件前返回 `WINDOW_CHANGED`。当 AX 保留完整窗口框时，这也会拦住 Stage Manager 变换后的窗口。需要重新读取状态；激活应用或抬起窗口仍须显式调用。元素声明的 AX 动作保持可用。缺少 AX 几何时保留无 AX 路径，不代表变换后的窗口一定可以操作。 @@ -175,7 +176,7 @@ AppKit 的 sheet 可以出现在文档窗口的树里，并有独立的窗口 ID ##### 操作指导文本（原文） ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -209,12 +210,12 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then call app.activate() and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. diff --git a/packages/experimental/cua-macos/native/README.i18n.yaml b/packages/experimental/cua-macos/native/README.i18n.yaml index 89135ddfbe..25d5fb8952 100644 --- a/packages/experimental/cua-macos/native/README.i18n.yaml +++ b/packages/experimental/cua-macos/native/README.i18n.yaml @@ -18,11 +18,11 @@ en: 1435672513328c05 zh: 11e7a1e672410c10 /dsh-cua-native-helper/request-protocol: - en: 20be674c0592d66d - zh: e26112b908187acc + en: 88a59904ca4ebedc + zh: 14f2b666d7382145 /dsh-cua-native-helper/observation-and-input: - en: 2408e0c0e9c873a2 - zh: e06236fc32efc4fa + en: da6a1c45fd166818 + zh: 7f378d5409a6085d /dsh-cua-native-helper/verification: en: 49f4cce2a4cc3114 zh: e9d65c92ef35fe24 diff --git a/packages/experimental/cua-macos/native/README.md b/packages/experimental/cua-macos/native/README.md index c01b273d1b..bdb844e806 100644 --- a/packages/experimental/cua-macos/native/README.md +++ b/packages/experimental/cua-macos/native/README.md @@ -83,7 +83,7 @@ Successful `activate` returns `dispatched` and `foregroundChanged`. A known chan Every app-targeted operation requires `appHandle` from `getApp` in the same helper. The helper pins the PID and kernel process start time, and invalidates the handle and all of its snapshots after process exit or PID reuse. Calling `getApp` can acquire the replacement process without reactivating old handles. A random per-helper namespace prevents handles from a terminated helper matching those of its replacement. Repeated acquisition of the same live process shares its handle. A post-launch inspection failure invalidates the pending handle and preserves the original error. It does not terminate a process already started; launchApp cannot reuse the failed handle. -Stable error codes are `INVALID_REQUEST` (malformed envelope), `INVALID_ARGUMENT` (invalid parameters), `UNSUPPORTED` (unknown operation), `APP_UNAVAILABLE`, `APP_PATH_UNAVAILABLE`, `PROCESS_INSPECTION_FAILED`, `AMBIGUOUS_APP`, `STALE_APP`, `STALE_SNAPSHOT`, `WINDOW_UNAVAILABLE`, `WINDOW_CHANGED`, `PERMISSION_DENIED`, `FOREGROUND_REQUIRED` (hover target is not the unobscured foreground window), `PANEL_NOT_READY` (system panel recipient unavailable before input), and `NATIVE_ERROR` (remaining operating-system failures). An error never promises that a partially dispatched input was rolled back. The host must not retry input automatically. +Stable error codes are `INVALID_REQUEST` (malformed envelope), `INVALID_ARGUMENT` (invalid parameters), `UNSUPPORTED` (unknown operation), `APP_UNAVAILABLE`, `APP_PATH_UNAVAILABLE`, `PROCESS_INSPECTION_FAILED`, `AMBIGUOUS_APP`, `STALE_APP`, `STALE_SNAPSHOT`, `WINDOW_UNAVAILABLE`, `WINDOW_CHANGED`, `PERMISSION_DENIED`, `HOVER_UNAVAILABLE`, `FOREGROUND_REQUIRED` (hover target is not the unobscured foreground window), `PANEL_NOT_READY` (system panel recipient unavailable before input), and `NATIVE_ERROR` (remaining operating-system failures). An error never promises that a partially dispatched input was rolled back. The host must not retry input automatically. Application selection returns an authorization record to the Host: a SHA-256 digest of the case-sensitive bundle identifier prefixed with `macos:bundle-id:`, display name, optional executable path, and persistable flag. All installations with that identifier share consent, including unsigned and ad-hoc-signed applications. Without a nonempty bundle identifier, both running and stopped applications use SHA-256 of `macos:executable-path:` followed by the symlink-resolved executable path and remain eligible for saved consent. A running bundle identifier remains usable when the process path query fails; the record omits executablePath. Available paths must still match the selection. Without either identity, APP_PATH_UNAVAILABLE reports the path query cause, including its errno, before any target operation. PROCESS_INSPECTION_FAILED means the process instance could not be verified; only ESRCH from process inspection is reported as a missing process. Neither failure triggers an automatic retry. Process IDs and start times validate live handles but do not enter the saved identity. The Host strips this record from model results and obtains human consent before launch, observation or input. This pipe protocol is trusted Host transport, not a sandbox against a separately authorized shell process. @@ -111,7 +111,7 @@ Host-only permission controls are `permissionStatus {}`, `requestPermission` and Observation reads each non-layout node’s descriptive properties and children through `AXUIElementCopyMultipleAttributeValues`. Per-attribute errors retain the individual-read rules; unsupported batch APIs use individual reads. Values are retained only during that observation, and input reads live attributes. Application state can change between nodes; a batch does not make the whole tree atomic. -System file-panel keyboard routing resolves the remote recipient through the selected app’s focused AX sheet. Only Apple’s Open and Save Panel Service is accepted, and its focused ancestry must still contain the identified remote window. Element-targeted pointer input follows AX parents to find the enclosing window or sheet. +System file-panel keyboard routing resolves the remote recipient through the selected app’s focused AX sheet. Only Apple’s Open and Save Panel Service is accepted, and its focused ancestry must still contain the identified remote window. For element and coordinate clicks on an Open or Save panel, the recipient is the verified panel-service window. Its frame must match the selected host window. Keyboard modifier names are plain strings. The shared object API passes the names through unchanged; the macOS helper accepts `command`/`cmd`, `shift`, `option`/`alt`, `control`/`ctrl`, and `fn`, and rejects an unknown name with `INVALID_ARGUMENT`. It does not parse chord strings. @@ -121,9 +121,9 @@ Tree observation and action discovery treat a generic AX failure reading one ele `paste` publishes one plain-text or HTML clipboard format, dispatches the application's enabled standard Paste menu command, and waits for a clipboard data request. It then restores the preserved formats only while its ownership marker and change count still match. The Host serializes its pastes across helpers with a file lock. maxClipboardBytes bounds both input and preserved contents; unsupported menus and oversized data fail before dispatch. A later copy is preserved. Cancellation restores owned clipboard data when possible; forced termination cannot guarantee restoration, and a data request does not prove the target completed editing. After dispatch, `pasteReadTimeoutMs` bounds waiting for clipboard consumption. Expiry returns `PASTE_NOT_CONSUMED`, closes the promised data provider, and restores the original clipboard only while still owned. The failure does not prove the app will never process its queued Paste command; inspect the target before deciding whether to retry. -`mouseMove` accepts an observed element ID or a window-local point and never presses a mouse button. Disabled controls may be hovered. No request completion, AX read or screenshot restores the previous point; subsequent mouseMove calls can enter revealed content. Click sends its button events at the target without an off-window priming click. Individual requests are serialized, but no application is reserved between calls. Other input or application changes can interrupt the interaction. +`mouseMove` reveals or restores alternate UI through an enabled element or ancestor advertising both `AXShowAlternateUI` and `AXShowDefaultUI`. Without an AX route it returns `HOVER_UNAVAILABLE` before sending input. The Host supplies the native boolean `allowSystemPointer` from its `systemPointerHover` setting. Omission means false. Only true permits posting a system pointer move after checking the foreground process and window hit. -`mouseMove` first uses a same-window element or ancestor advertising both `AXShowAlternateUI` and `AXShowDefaultUI`. Revealed descendants retain the parent UI; leaving restores it once. An AX action failure propagates without trying another input route. Without an applicable AX route, the method moves the visible system cursor through the system input queue after checking the foreground PID and AppKit mouse hit testing at the point. Input-transparent windows and transparent regions do not block the target. A failed check returns `FOREGROUND_REQUIRED` without moving the cursor. The result identifies `hoverRoute` as `accessibility` or `system-pointer` and reports `systemPointerMoved`. The method does not activate, raise or restore focus. Other input or window changes can race the check and dispatch. +The opt-in system pointer route returns `FOREGROUND_REQUIRED` if the target is obscured or belongs to a background app. The result reports the route and whether the visible cursor moved in `hoverRoute` and `systemPointerMoved`. It never presses a button or restores the cursor after a read. Later input may interrupt the hover. Clicks send their button events at the target without an off-window priming click. ----- diff --git a/packages/experimental/cua-macos/native/README.zh.md b/packages/experimental/cua-macos/native/README.zh.md index 307d2eaee8..51d6c13c29 100644 --- a/packages/experimental/cua-macos/native/README.zh.md +++ b/packages/experimental/cua-macos/native/README.zh.md @@ -83,7 +83,7 @@ AppKit 在主事件循环上运行。阻塞式 socket 接收与握手使用后 每个面向应用的操作都要求使用同一 helper 的 `getApp` 返回的 `appHandle`。helper 固定 PID 和内核进程启动时间，进程退出或 PID 被复用后使句柄及其全部快照失效。再次调用 `getApp` 可取得替代进程，不会重新激活旧句柄。随机的 helper 命名空间防止已终止 helper 的句柄与替代实例重合。重复获取同一存活进程会共享句柄。启动后检查失败会使待启动句柄失效，并保留原错误；已经启动的进程不会被终止，launchApp 也不能复用失败的句柄。 -稳定错误码包括 `INVALID_REQUEST`（信封无效）、`INVALID_ARGUMENT`（参数无效）、`UNSUPPORTED`（操作未知）、`APP_UNAVAILABLE`、`APP_PATH_UNAVAILABLE`、`PROCESS_INSPECTION_FAILED`、`AMBIGUOUS_APP`、`STALE_APP`、`STALE_SNAPSHOT`、`WINDOW_UNAVAILABLE`、`WINDOW_CHANGED`、`PERMISSION_DENIED`、`FOREGROUND_REQUIRED`（悬停目标不是未被遮挡的前台窗口）、`PANEL_NOT_READY`（输入前系统面板接收方尚不可用）及 `NATIVE_ERROR`（其余操作系统失败）。错误不保证已分派的部分输入被回滚，Host 不得自动重试输入。 +稳定错误码包括 `INVALID_REQUEST`（信封无效）、`INVALID_ARGUMENT`（参数无效）、`UNSUPPORTED`（操作未知）、`APP_UNAVAILABLE`、`APP_PATH_UNAVAILABLE`、`PROCESS_INSPECTION_FAILED`、`AMBIGUOUS_APP`、`STALE_APP`、`STALE_SNAPSHOT`、`WINDOW_UNAVAILABLE`、`WINDOW_CHANGED`、`PERMISSION_DENIED`、`HOVER_UNAVAILABLE`、`FOREGROUND_REQUIRED`（悬停目标不是未被遮挡的前台窗口）、`PANEL_NOT_READY`（输入前系统面板接收方尚不可用）及 `NATIVE_ERROR`（其余操作系统失败）。错误不保证已分派的部分输入被回滚，Host 不得自动重试输入。 应用选择向 Host 返回授权记录：区分大小写的 Bundle ID 加上 `macos:bundle-id:` 前缀后计算的 SHA-256 摘要、显示名称、可选的可执行路径和可持久化标记。相同标识的所有安装副本共用授权，包括未签名和临时签名的应用。缺少非空 Bundle ID 时，运行中和未启动应用都对 `macos:executable-path:` 与解析符号链接后的可执行路径计算 SHA-256，仍可保存长期授权。进程路径查询失败时，运行中的有效 Bundle ID 仍可用于授权，记录省略 executablePath；路径可取得时仍须与所选目标一致。两种身份均不可用时，在执行目标操作前返回 APP_PATH_UNAVAILABLE，保留路径查询原因和 errno。PROCESS_INSPECTION_FAILED 表示无法验证进程实例，只有进程检查返回 ESRCH 时才报告进程不存在；两种失败都不触发自动重试。进程号和启动时间只校验存活句柄，不进入已保存身份。Host 从模型结果中移除此记录，并在启动、观察和输入前取得用户许可。此管道协议用于可信 Host 传输，无法隔离另一个已获授权的 shell 进程。 @@ -111,7 +111,7 @@ AppKit 在主事件循环上运行。阻塞式 socket 接收与握手使用后 观察通过 `AXUIElementCopyMultipleAttributeValues` 读取每个非布局节点的描述属性和子节点。逐属性错误保留单独读取的规则；不支持批量 API 时使用单独读取。值只在本次观察期间保留，输入读取即时属性。应用状态可能在节点之间变化，批量读取不使整棵树具备原子性。 -系统文件面板的键盘路由通过所选应用的 AX 焦点 sheet 解析远端接收方。只接受 Apple 的 Open and Save Panel Service，且其焦点祖先链仍须包含已识别的远端窗口。按元素发送鼠标事件时，helper 沿 AX 父节点查找所属窗口或 sheet。 +系统文件面板的键盘路由通过所选应用的 AX 焦点 sheet 解析远端接收方。只接受 Apple 的 Open and Save Panel Service，且其焦点祖先链仍须包含已识别的远端窗口。面板的元素点击和坐标点击发送至已核实的系统面板服务窗口。服务窗口的边界必须和所选宿主窗口一致。 键盘修饰键名称是普通字符串。共享对象 API 原样传递名称；macOS helper 接受 `command`/`cmd`、`shift`、`option`/`alt`、`control`/`ctrl` 和 `fn`，未知名称返回 `INVALID_ARGUMENT`。它不解析组合键字符串。 @@ -121,9 +121,9 @@ AppKit 在主事件循环上运行。阻塞式 socket 接收与握手使用后 `paste` 提供一种纯文本或 HTML 剪贴板格式，调用应用已启用的标准 Paste 菜单命令，并等待剪贴板数据请求。随后仅在所有权标记和修改计数仍匹配时恢复原有格式。Host 通过文件锁串行化不同 helper 的粘贴。maxClipboardBytes 同时限制输入和保留内容，不支持的菜单和过大的数据在分派前失败。后续复制会被保留。取消会尽可能恢复仍持有的剪贴板数据，强制结束不保证恢复，数据请求也不证明目标已完成编辑。 分派完成后，`pasteReadTimeoutMs` 限制等待剪贴板读取的时间。到期返回 `PASTE_NOT_CONSUMED`，关闭延迟数据提供方，并仅在仍持有剪贴板时恢复原内容。该失败不能证明应用以后不会处理已排队的 Paste 命令；决定是否重试前先检查目标内容。 -`mouseMove` 接受已观察的元素 ID 或窗口内坐标，不按下鼠标按钮，也可用于禁用控件。调用结束、AX 读取和截图都不恢复原位置，后续 mouseMove 调用可进入新出现的内容。点击只在目标位置发送按键事件，不先在窗口外点击。单次请求串行执行，不在调用之间保留应用。其他输入或应用变化可能中断交互。 +默认情况下，`mouseMove` 通过同一窗口内启用的控件或祖先声明的 `AXShowAlternateUI` 和 `AXShowDefaultUI` 展开或恢复界面。缺少这条路径时返回 `HOVER_UNAVAILABLE`，不发送输入。Host 根据部署配置 `systemPointerHover` 传入原生布尔参数 `allowSystemPointer`，省略时为 false。只有值为 true 才允许移动全局光标，且须检查前台进程和鼠标命中的窗口。 -`mouseMove` 优先查找同一窗口内声明了 `AXShowAlternateUI` 和 `AXShowDefaultUI` 的元素或祖先。进入展开区域的子控件时保留父控件界面，离开时恢复一次。AX 动作失败直接返回错误，不尝试其他输入路径。没有可用 AX 路径时，该方法核实前台 PID，并通过 AppKit 查询目标点实际接收鼠标输入的窗口后，使用系统输入队列移动可见鼠标。不接收鼠标输入的窗口和透明区域不会阻挡目标。检查失败返回 `FOREGROUND_REQUIRED`，不移动鼠标。结果中的 `hoverRoute` 标明 `accessibility` 或 `system-pointer`，`systemPointerMoved` 标明是否移动系统鼠标。该方法不激活、不抬起窗口，也不恢复焦点。其他输入或窗口变化可能与检查和分派交错。 +开启后，系统指针路线会检查前台应用和坐标命中的窗口，不满足条件时返回 `FOREGROUND_REQUIRED`。返回值 `hoverRoute` 和 `systemPointerMoved` 说明走了哪条路线以及是否移动了可见光标。悬停不会按下鼠标，也不会在读取后还原光标。后续输入可能中断悬停。点击只在目标点发送按键事件，不先在窗口外点击。 ----- diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift index 1a1e4f595b..3b39ab1f02 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXAccess.swift @@ -419,6 +419,10 @@ final class AXAccess { } /// Resolve an exact AX window or sheet without enabling accessibility or traversing its controls. func windowBounds(_ id: UInt32, owner: ProcessIdentity) throws -> WindowBounds? { + guard let node = try windowElement(id, owner: owner) else { return nil } + return try bounds(node) + } + func windowElement(_ id: UInt32, owner: ProcessIdentity) throws -> AXUIElement? { let app = AXUIElementCreateApplication(owner.pid) var windows = try children(app, "AXWindows") if let focused = try element(app, "AXFocusedWindow"), !windows.contains(where: { CFEqual($0, focused) }) { @@ -427,9 +431,9 @@ final class AXAccess { for window in windows { let role = try string(window, "AXRole") guard role == "AXWindow" || role == "AXSheet" else { continue } - if try windowID(window) == id { return try bounds(window) } + if try windowID(window) == id { return window } for sheet in try children(window) where try string(sheet, "AXRole") == "AXSheet" { - if try windowID(sheet) == id { return try bounds(sheet) } + if try windowID(sheet) == id { return sheet } } } return nil @@ -654,6 +658,31 @@ final class AXAccess { } throw NativeFailure(code: "WINDOW_UNAVAILABLE", message: "Pointer input requires an enclosing window or sheet. For a menu item’s default action, use app.invoke(id)") } + /// A system file panel renders in its host window but receives input in the panel service's window. + func filePanelRecipient(_ node: AXUIElement, owner: ProcessIdentity, catalog: ApplicationCatalogSnapshot) throws -> (process: ProcessIdentity, windowID: UInt32)? { + guard ["open-panel", "save-panel"].contains(try string(node, "AXIdentifier") ?? "") else { return nil } + guard let hostID = try windowID(node) else { throw NativeFailure(code: "PANEL_NOT_READY", message: "File panel has no window ID") } + _ = try NativePlatform.ownedWindow(owner.pid, hostID) + var recipient: (ProcessIdentity, UInt32)? + for child in try children(node) { + guard let id = try windowID(child), let window = NativePlatform.windows(allLayers: true).first(where: { $0.windowID == id }), window.pid != owner.pid else { continue } + guard catalog.application(window.pid)?.bundleID == "com.apple.appkit.xpc.openAndSavePanelService" else { + throw NativeFailure(code: "PANEL_NOT_READY", message: "File panel refers to an unsupported remote process") + } + let process = try NativePlatform.identity(window.pid) + if let old = recipient, old.0 != process || old.1 != id { throw NativeFailure(code: "PANEL_NOT_READY", message: "File panel has ambiguous remote recipients") } + recipient = (process, id) + } + guard let (process, id) = recipient else { throw NativeFailure(code: "PANEL_NOT_READY", message: "System file panel has no ready input recipient") } + var focus = try element(AXUIElementCreateApplication(process.pid), "AXFocusedWindow") + for _ in 0..<100 { + guard let focused = focus else { break } + if try windowID(focused) == id { try NativePlatform.verify(process); return (process, id) } + focus = try element(focused, "AXParent") + } + throw NativeFailure(code: "PANEL_NOT_READY", message: "System file panel focus changed; observe fresh state") + } + func keyboardRecipient(_ owner: ProcessIdentity, catalog: ApplicationCatalogSnapshot) throws -> ProcessIdentity { var current = try element(AXUIElementCreateApplication(owner.pid), "AXFocusedWindow") var visited: [AXUIElement] = [] @@ -661,28 +690,7 @@ final class AXAccess { guard let node = current else { return owner } if visited.contains(where: { CFEqual($0, node) }) { break } visited.append(node) - if ["open-panel", "save-panel"].contains(try string(node, "AXIdentifier") ?? "") { - guard let hostID = try windowID(node) else { throw NativeFailure(code: "PANEL_NOT_READY", message: "File panel has no window ID") } - _ = try NativePlatform.ownedWindow(owner.pid, hostID) - var recipient: (ProcessIdentity, UInt32)? - for child in try children(node) { - guard let id = try windowID(child), let window = NativePlatform.windows(allLayers: true).first(where: { $0.windowID == id }), window.pid != owner.pid else { continue } - guard catalog.application(window.pid)?.bundleID == "com.apple.appkit.xpc.openAndSavePanelService" else { - throw NativeFailure(code: "PANEL_NOT_READY", message: "File panel refers to an unsupported remote process") - } - let process = try NativePlatform.identity(window.pid) - if let old = recipient, old.0 != process || old.1 != id { throw NativeFailure(code: "PANEL_NOT_READY", message: "File panel has ambiguous remote recipients") } - recipient = (process, id) - } - guard let (process, id) = recipient else { throw NativeFailure(code: "PANEL_NOT_READY", message: "System file panel has no ready input recipient") } - var focus = try element(AXUIElementCreateApplication(process.pid), "AXFocusedWindow") - for _ in 0..<100 { - guard let focused = focus else { break } - if try windowID(focused) == id { try NativePlatform.verify(process); return process } - focus = try element(focused, "AXParent") - } - throw NativeFailure(code: "PANEL_NOT_READY", message: "System file panel focus changed; observe fresh state") - } + if let panel = try filePanelRecipient(node, owner: owner, catalog: catalog) { return panel.process } current = try element(node, "AXParent") } throw NativeFailure(code: "PANEL_NOT_READY", message: "Focused accessibility ancestry exceeds the traversal limit") diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift index ff398914c7..656aa97a07 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift @@ -2,6 +2,12 @@ import ApplicationServices final class AXHover { + static func requireSystemPointer(allowed: Bool) throws { + guard allowed else { + throw NativeFailure(code: "HOVER_UNAVAILABLE", message: "No enabled element at this target or in its ancestors offers both AXShowAlternateUI and AXShowDefaultUI. System-pointer hover is disabled; this provider has no other pointer-free hover route. No input was sent and the pointer did not move. Use click on an available control or performAction with an action from listActions.") + } + } + private var revealed: (element: AXUIElement, windowID: UInt32)? /// Entering revealed descendants preserves their parent UI; leaving restores it once. diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift index ea58f27f7e..f00e63b622 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift @@ -349,6 +349,7 @@ actor AXWorker { let window: WindowInfo let point: WindowPoint var hoverElement: AXUIElement? + var elementWindow: AXUIElement? if params.contains("elementId") { guard !params.contains("x"), !params.contains("y") else { throw RequestParameters.invalid("Element and coordinate targets cannot be combined") } let requestedWindow = try params.optionalWindowID() @@ -359,6 +360,7 @@ actor AXWorker { throw NativeFailure(code: "STALE_SNAPSHOT", message: "Element belongs to window \(enclosing.id); select that window and read its tree") } if method == "click" { try ax().requireEnabled(selected) } + elementWindow = enclosing.element hoverElement = selected guard let bounds = try ax().bounds(selected), bounds.width > 0, bounds.height > 0 else { throw NativeFailure(code: "NATIVE_ERROR", message: "Element has no pointer bounds") } window = try WindowGeometry.resolve(owner: identity, windowID: enclosing.id) { try ax().bounds(enclosing.element) } @@ -371,6 +373,7 @@ actor AXWorker { } let route = WindowRoute(owner: identity, windowID: window.windowID, bounds: window.bounds) if method == "mouseMove" { + let allowSystemPointer = try params.optionalBoolean("allowSystemPointer") ?? false try WindowPointerInput.validate(point, in: route) if hoverElement == nil { hoverElement = try ax().element(at: CGPoint(x: window.bounds.x + point.x, y: window.bounds.y + point.y), owner: identity) @@ -381,9 +384,21 @@ actor AXWorker { if try hover.move(to: hoverElement, in: route, access: ax()) { return .result(Self.mouseMoveResult(route: "accessibility", pointerMoved: false)) } + try AXHover.requireSystemPointer(allowed: allowSystemPointer) return .mouseMove(route, point) } - try await inputs().1.click(in: route, at: point, button: button, count: count, modifiers: modifiers, holdMs: holdMs) + var clickRoute = route + if AXIsProcessTrusted(), let node = try elementWindow ?? ax().windowElement(window.windowID, owner: identity), + let panel = try ax().filePanelRecipient(node, owner: identity, catalog: catalog) { + let serviceWindow = try NativePlatform.ownedWindow(panel.process.pid, panel.windowID) + guard serviceWindow.bounds == window.bounds else { + throw NativeFailure(code: "WINDOW_CHANGED", message: "File panel and its input window have different frames; read fresh window state") + } + try NativePlatform.verify(identity) + try NativePlatform.verify(panel.process) + clickRoute = WindowRoute(owner: panel.process, windowID: panel.windowID, bounds: serviceWindow.bounds) + } + try await inputs().1.click(in: clickRoute, at: point, button: button, count: count, modifiers: modifiers, holdMs: holdMs) case "scroll", "drag": let id = try params.windowID() try coordinateSnapshot(handle, params, id) diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/NativeProtocol.generated.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/NativeProtocol.generated.swift index a13ca0ab35..0da4e5cb96 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/NativeProtocol.generated.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/NativeProtocol.generated.swift @@ -14,7 +14,7 @@ enum NativeProtocol { "listActions": ["appHandle", "elementId", "snapshotId", "windowId"], "listApps": [], "listWindows": ["appHandle"], - "mouseMove": ["appHandle", "elementId", "snapshotId", "windowId", "x", "y"], + "mouseMove": ["allowSystemPointer", "appHandle", "elementId", "snapshotId", "windowId", "x", "y"], "openPermissionSettings": ["permission", "settingsMessage"], "paste": ["appHandle", "format", "maxClipboardBytes", "maxDepth", "maxElements", "pasteReadTimeoutMs", "text", "windowId"], "performAction": ["action", "appHandle", "argument", "elementId", "snapshotId", "windowId"], diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/RequestCoordinator.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/RequestCoordinator.swift index 00a1d709a1..3a778cb8ae 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/RequestCoordinator.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/RequestCoordinator.swift @@ -165,7 +165,7 @@ import Darwin let params = try RequestParameters(request) try cancellation.check() if request.method == "screenshot" { try capture.requireAvailable() } - let requiresCatalog = ["getApp", "listApps", "findApps", "typeText", "pressKey"].contains(request.method) + let requiresCatalog = ["getApp", "listApps", "findApps", "typeText", "pressKey", "click"].contains(request.method) let catalog = requiresCatalog ? ApplicationCatalog.snapshot() : ApplicationCatalogSnapshot(applications: [], frontPID: nil) let result: JSONValue if request.method == "permissionStatus" { diff --git a/packages/experimental/cua-macos/native/generate-protocol.ts b/packages/experimental/cua-macos/native/generate-protocol.ts index 49bfeea38b..398999cc2c 100644 --- a/packages/experimental/cua-macos/native/generate-protocol.ts +++ b/packages/experimental/cua-macos/native/generate-protocol.ts @@ -1,5 +1,6 @@ /** Generate macOS admission artifacts from the shared CUA operation definitions. */ import { readFile, writeFile } from 'node:fs/promises' +import { z } from 'zod' import { nativeProtocol } from '@deepseek-ai/dsh-cua/native-schema' import { PermissionGuideParameters, PermissionSettingsParameters } from '../src/permission-guide.ts' @@ -9,6 +10,7 @@ import { PermissionGuideParameters, PermissionSettingsParameters } from '../src/ */ export function renderNativeProtocol(): { json: string; swift: string } { const { fields, enums, json } = nativeProtocol({ + mouseMove: z.object({ allowSystemPointer: z.boolean().optional() }), requestPermission: PermissionGuideParameters, openPermissionSettings: PermissionSettingsParameters, }) const quoted = (values: Iterable) => [...values].sort().map(value => JSON.stringify(value)).join(', ') diff --git a/packages/experimental/cua-macos/native/protocol.schema.json b/packages/experimental/cua-macos/native/protocol.schema.json index 4dda3c8511..b0d74a84dc 100644 --- a/packages/experimental/cua-macos/native/protocol.schema.json +++ b/packages/experimental/cua-macos/native/protocol.schema.json @@ -488,6 +488,9 @@ "type": "string", "minLength": 1, "description": "Opaque string defined by the platform provider, valid for at least the lifetime of one snapshot." + }, + "allowSystemPointer": { + "type": "boolean" } }, "required": [ @@ -520,6 +523,9 @@ "y": { "type": "number", "minimum": 0 + }, + "allowSystemPointer": { + "type": "boolean" } }, "required": [ diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index 95776b4031..03ae9ac2ac 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -85,7 +85,7 @@ def check_file_panel(call, owner, primary_params, primary, directory, wait_state assert f"window {sheet_id}; select that window" in error["message"], error ready = call("getState", {**owner, "windowId": sheet_id, "maxElements": 2000, "maxDepth": 25}) open_button = next(e for e in ready["elements"] if e["identifier"] == "OKButton" and e["enabled"]) - assert call("invoke", {**owner, "windowId": sheet_id, "snapshotId": ready["snapshotId"], "elementId": open_button["elementId"]})["dispatched"] + assert call("click", {**owner, "windowId": sheet_id, "snapshotId": ready["snapshotId"], "elementId": open_button["elementId"]})["dispatched"] wait_state(lambda state: bool(state["selectedFile"]) and Path(state["selectedFile"]).samefile(selected_file), "File picker did not return the intended file") # AppKit can retain the dismissed panel's hidden WindowServer window after @@ -217,8 +217,18 @@ def main(): call("performAction", {**owner, "snapshotId": current["snapshotId"], "elementId": window_element["elementId"], "action": "AXRaise"}) call("getState", primary_params) + current = call("getState", primary_params) + no_hover = {**button_target(current, "Increment primary"), "windowId": primary["windowId"]} + before = json.loads(state_path.read_text()) + denied = call("mouseMove", no_hover, "HOVER_UNAVAILABLE") + assert "systemPointerMoved" not in denied and "No input was sent" in denied["message"], denied + after = wait_state(lambda s: s["cursorSamples"] > before["cursorSamples"], "Cursor sampling stopped") + assert after["cursorChanges"] == before["cursorChanges"] and not after["hoverVisible"], after + def mouse_move(target): - result = call("mouseMove", target) + result = call("mouseMove", target if args.alternate_hover else {**target, "allowSystemPointer": True}) + if not args.alternate_hover: + assert result["hoverRoute"] == "system-pointer" and result["systemPointerMoved"] is True, result if args.alternate_hover: assert result["hoverRoute"] == "accessibility" and result["systemPointerMoved"] is False, result return result @@ -251,7 +261,7 @@ def main(): mouse_move(target) wait_state(lambda actual: actual["hoverVisible"], "Input-transparent overlay blocked hover") before = cycle_overlay(2) - call("mouseMove", target, "FOREGROUND_REQUIRED") + call("mouseMove", {**target, "allowSystemPointer": True}, "FOREGROUND_REQUIRED") after = wait_state(lambda actual: actual["cursorSamples"] >= before["cursorSamples"] + 2, "Pointer observation did not advance") assert after["cursorChanges"] == before["cursorChanges"], "Rejected hover moved the system pointer" diff --git a/packages/experimental/cua-macos/native/tests/electron_e2e.py b/packages/experimental/cua-macos/native/tests/electron_e2e.py index 0e2b966516..403bb21eba 100644 --- a/packages/experimental/cua-macos/native/tests/electron_e2e.py +++ b/packages/experimental/cua-macos/native/tests/electron_e2e.py @@ -134,7 +134,9 @@ def main(): call('mouseMove', {**owner, 'windowId': window_id, 'x': -1, 'y': 0}, 'INVALID_ARGUMENT') call('mouseMove', {**owner, 'windowId': window_id, 'x': 100000, 'y': 0}, 'INVALID_ARGUMENT') call('mouseMove', {**owner, 'snapshotId': state['snapshotId'], 'elementId': trigger['elementId'], 'x': 0}, 'INVALID_ARGUMENT') - call('mouseMove', {**owner, 'snapshotId': state['snapshotId'], 'elementId': trigger['elementId']}) + hover_target = {**owner, 'snapshotId': state['snapshotId'], 'elementId': trigger['elementId']} + call('mouseMove', hover_target, 'HOVER_UNAVAILABLE') + call('mouseMove', {**hover_target, 'allowSystemPointer': True}) actual = wait_state(lambda actual: actual['popupOpen'], 'Hover did not reveal delayed information') assert actual['pointerDowns'] == baseline and actual['lastMove'] == {'buttons': 0, 'trusted': True}, actual state = observe() @@ -149,13 +151,13 @@ def main(): target = {**owner, 'windowId': window_id, 'x': bounds[0] + bounds[2] / 2 - window['bounds']['x'], 'y': bounds[1] + bounds[3] / 2 - window['bounds']['y']} - call('mouseMove', target) + call('mouseMove', {**target, 'allowSystemPointer': True}) state = observe() assert 'Hover information' in state['tree'], 'Moving into the popup dismissed it' call('click', target) actual = wait_state(lambda actual: actual['popupClicks'] == expected_clicks, 'Popup action did not receive its click') assert actual['popupOpen'] and actual['pointerDowns'] == baseline + 1 and actual['popupClickTrusted'], actual - call('mouseMove', {**owner, 'windowId': window_id, 'x': 10, 'y': 40}) + call('mouseMove', {**owner, 'windowId': window_id, 'x': 10, 'y': 40, 'allowSystemPointer': True}) wait_state(lambda actual: not actual['popupOpen'], 'Leaving the hover region did not close it') hover_sequence() helper.eof() diff --git a/packages/experimental/cua-macos/native/tests/swift/SystemHoverTests.swift b/packages/experimental/cua-macos/native/tests/swift/SystemHoverTests.swift index 38f82ac4a8..8bb26dfdb9 100644 --- a/packages/experimental/cua-macos/native/tests/swift/SystemHoverTests.swift +++ b/packages/experimental/cua-macos/native/tests/swift/SystemHoverTests.swift @@ -3,6 +3,17 @@ import Testing import CoreGraphics @testable import CuaNative +@Test func unavailableAXHoverRequiresExplicitSystemPointerPermission() throws { + do { + try AXHover.requireSystemPointer(allowed: false) + Issue.record("A default hover without an AX route was allowed to move the pointer") + } catch let failure as NativeFailure { + #expect(failure.code == "HOVER_UNAVAILABLE") + #expect(failure.message.contains("No input was sent")) + } + try AXHover.requireSystemPointer(allowed: true) +} + @Test func systemHoverRequiresTheForegroundUnobscuredTarget() throws { let owner = ProcessIdentity(pid: 42, startedSeconds: 1, startedMicroseconds: 0) let bounds = WindowBounds(x: 100, y: 100, width: 400, height: 300) diff --git a/packages/experimental/cua-macos/src/guidance.ts b/packages/experimental/cua-macos/src/guidance.ts index e2127a78ab..9da4e5f85a 100644 --- a/packages/experimental/cua-macos/src/guidance.ts +++ b/packages/experimental/cua-macos/src/guidance.ts @@ -2,7 +2,7 @@ import { COMMON_GUIDANCE } from '@deepseek-ai/dsh-cua/guidance' /** macOS operation levels, permissions, selectors, and native measurement units. */ -export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' +export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' /** Complete model instructions for this provider. */ export const GUIDANCE = `${PLATFORM_GUIDANCE} diff --git a/packages/experimental/cua-macos/src/index.ts b/packages/experimental/cua-macos/src/index.ts index 8d88d9c21e..52788c0752 100644 --- a/packages/experimental/cua-macos/src/index.ts +++ b/packages/experimental/cua-macos/src/index.ts @@ -32,6 +32,8 @@ export const inject = ['computerUse', 'tools', 'systemPrompt', 'subprocess', 'ag export interface Config extends CuaConfig { /** Interval in milliseconds between permission and position checks while the native guide is open. */ permissionPollIntervalMs: number + /** Allow mouseMove to move the visible system pointer when AX hover is unavailable. Off by default to avoid interrupting the user. */ + systemPointerHover: boolean /** * Source-build signing: `ad-hoc` (default) binds grants to the code hash; * an object selects an existing certificate in an explicitly configured keychain. @@ -42,6 +44,7 @@ export interface Config extends CuaConfig { /** Validate shared operation bounds and macOS source signing before preparing the helper. */ export const Config = Schema.object({ ...CuaConfigFields, + systemPointerHover: Schema.boolean().default(false), permissionPollIntervalMs: Schema.number().step(1).min(100).max(2 ** 31 - 1).default(1000), sourceSigning: Schema.union([ Schema.const('ad-hoc'), @@ -61,6 +64,7 @@ export const Config = Schema.object({ */ export async function apply(ctx: Context, config: Config): Promise { const backend = { createChannel: createSharedNativeChannel, guidance: GUIDANCE, capabilities: CAPABILITIES, + systemPointerHover: config.systemPointerHover, permissionParameters: createPermissionParameters(config.permissionPollIntervalMs) } if (config.helperPath !== undefined) { if (!isAbsolute(config.helperPath)) throw new Error('dsh-cua helperPath must be absolute') diff --git a/packages/experimental/cua-macos/tests/guidance.spec.ts b/packages/experimental/cua-macos/tests/guidance.spec.ts index f0f5679623..7afb53b84c 100644 --- a/packages/experimental/cua-macos/tests/guidance.spec.ts +++ b/packages/experimental/cua-macos/tests/guidance.spec.ts @@ -27,6 +27,9 @@ it('states the macOS accessibility, keyboard, and mouse levels', () => { expect(PLATFORM_GUIDANCE).toContain('accessibility tree operations are available at app or window level') expect(PLATFORM_GUIDANCE).toContain('Simulated keyboard input is available at app level') expect(PLATFORM_GUIDANCE).toContain('Simulated mouse input is available at window level') + const hover = COMMON_GUIDANCE.split('\n').find(line => line.startsWith('- mouseMove(')) + for (const term of ['AXShowAlternateUI', 'AXShowDefaultUI', 'HOVER_UNAVAILABLE', 'systemPointerHover']) expect(hover).toContain(term) + expect(COMMON_GUIDANCE.split('\n').find(line => line.startsWith('- click('))).toContain('call app.activate() first') }) it('enumerates every available method once in its catalog entry', () => { diff --git a/packages/experimental/cua-macos/tests/protocol-generation.spec.ts b/packages/experimental/cua-macos/tests/protocol-generation.spec.ts index dd5a3911c6..aefc6fbee6 100644 --- a/packages/experimental/cua-macos/tests/protocol-generation.spec.ts +++ b/packages/experimental/cua-macos/tests/protocol-generation.spec.ts @@ -26,6 +26,7 @@ describe('native protocol generation', () => { { method: 'drag', params: { appHandle: 'app', windowId: 1, fromX: 1, fromY: 1, toX: 2, toY: 2, steps: 1 } }, { method: 'paste', params: { appHandle: 'app', text: 'text', maxClipboardBytes: 1024 } }, { method: 'paste', params: { appHandle: 'app', text: 'text', pasteReadTimeoutMs: 1000 } }, + { method: 'mouseMove', params: { appHandle: 'app', windowId: 1, x: 0, y: 0, allowSystemPointer: true } }, { method: 'findApps', params: { query: 'Editor', maxAppMatches: 1 } }, { method: 'launchApp', params: { appHandle: 'pending' } }, { method: 'applicationIcons', params: { paths: ['/Applications/Fixture.app'] } }, diff --git a/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts b/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts index 04f5b0dd80..38e632e6ae 100644 --- a/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts +++ b/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts @@ -61,7 +61,7 @@ vi.mock('@deepseek-ai/dsh-cua/native-client', async importOriginal => ({ })) const config = { - permissionPollIntervalMs: 1000, + permissionPollIntervalMs: 1000, systemPointerHover: false, agentPreset: false, helperPath: resolve('fixture-helper'), sourceSigning: 'ad-hoc', buildTimeoutMs: 300_000, timeoutMs: 10_000, connectTimeoutMs: 5000, screenshotTimeoutMs: 5000, axTimeoutMs: 3000, maxRequestBytes: 1024, maxResponseBytes: 4096, maxClipboardBytes: 8192, pasteReadTimeoutMs: 1000, maxAppMatches: 20, graceMs: 10, textEventIntervalMs: 8, maxClickHoldMs: 7000, maxElements: 64, maxDepth: 8, dragDurationMs: 200, dragSteps: 10, @@ -109,7 +109,7 @@ class ImageAdapter extends LlmAdapter { const imageData = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAADElEQVQImWNgZGIGAAAOAAeCcsnOAAAAAElFTkSuQmCC' const imageMetadata = { pixelWidth: 1, pixelHeight: 1, bounds: { x: 2, y: 3, width: 1, height: 1 }, coordinateSpace: 'window-points', windowId: 7 } -async function setup(model?: { provider: string; model: string }) { +async function setup(model?: { provider: string; model: string }, systemPointerHover = false) { native.request.mockReset().mockResolvedValue([]) native.dispose.mockReset() native.instances.length = 0 @@ -119,7 +119,7 @@ async function setup(model?: { provider: string; model: string }) { await mountAgentLoopTestDependencies(ctx) await ctx.plugin(ComputerUseRegistry) await ctx.plugin(Subprocess, []) - const provider = await ctx.plugin(NativeProvider, config) + const provider = await ctx.plugin(NativeProvider, { ...config, systemPointerHover }) await mountAgentLoopTestHarness(ctx) const handle = await ctx.agents.create({ sessionId: SessionId('native-lifecycle'), ...model ? { agentOptions: model } : {} }) let index = 0 @@ -398,6 +398,14 @@ it.each([false, true])('rejects an ended Agent with prior helper acquisition %s' expect(native.request).toHaveBeenCalledTimes(requests) }) +it.each([false, true])('passes the deployment hover policy (%s) to native input', async (allowed) => { + const f = await setup(undefined, allowed) + await selectApplication(f) + const params = { appHandle: 'app-one', windowId: 7, snapshotId: 's', elementId: '0' } + await f.execute('mouseMove', params) + expect(native.request).toHaveBeenLastCalledWith('mouseMove', { ...params, allowSystemPointer: allowed }, expect.any(AbortSignal), expect.any(String)) +}) + it('supplies configured resource bounds and clears registrations on unload', async () => { const f = await setup() await selectApplication(f) diff --git a/packages/experimental/cua/src/guidance.ts b/packages/experimental/cua/src/guidance.ts index adf62b8b5e..1d6d0266d1 100644 --- a/packages/experimental/cua/src/guidance.ts +++ b/packages/experimental/cua/src/guidance.ts @@ -32,12 +32,12 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then call app.activate() and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. diff --git a/packages/experimental/cua/src/index.ts b/packages/experimental/cua/src/index.ts index d6c75f46bc..7cb5177f71 100644 --- a/packages/experimental/cua/src/index.ts +++ b/packages/experimental/cua/src/index.ts @@ -126,6 +126,8 @@ export interface CuaBackend { createChannel: NativeChannelFactory guidance: string capabilities: CuaStartupCapabilities + /** macOS deployment policy sent only to the native mouseMove operation. */ + systemPointerHover?: boolean /** * Build native permission parameters for the selected action and Client language. * @param method Native permission action. @@ -211,6 +213,7 @@ export function installCua(ctx: Context, config: CuaConfig, backend: CuaBackend) const signal = execution.signal signal.throwIfAborted() const client = execution.nativeClient + if (method === 'mouseMove' && backend.systemPointerHover !== undefined) params.allowSystemPointer = backend.systemPointerHover const nativeParams = method === 'findApps' ? { ...params, maxAppMatches: config.maxAppMatches } : method === 'paste' diff --git a/packages/experimental/cua/src/native-schema.ts b/packages/experimental/cua/src/native-schema.ts index c62a5e6b42..bbd52e2591 100644 --- a/packages/experimental/cua/src/native-schema.ts +++ b/packages/experimental/cua/src/native-schema.ts @@ -13,15 +13,15 @@ function variants(schema: Schema): Schema[] { return schema.anyOf ?? schema.oneO /** * Derive native request schemas and admission fields from the shared operation definitions. - * @param permissionFields Platform-specific parameters for user-facing permission controls. + * @param platformFields Platform-specific parameters added by the Host before native dispatch. * @returns Native field and enum tables and the complete wire schema. */ -export function nativeProtocol(permissionFields: Partial> = {}): { +export function nativeProtocol(platformFields: Partial> = {}): { fields: Map> enums: Map>> json: string } { - const resources = new Map(Object.entries({ ...NativeResourceParameters, ...permissionFields })) + const resources = new Map(Object.entries({ ...NativeResourceParameters, ...platformFields })) const schemas = [...Request.options, ...NativeOnlyOperations] .map(schema => z.toJSONSchema(schema)) const fields = new Map>() diff --git a/packages/experimental/cua/src/types.ts b/packages/experimental/cua/src/types.ts index b44e8f4ef6..4e5f1ef0d9 100644 --- a/packages/experimental/cua/src/types.ts +++ b/packages/experimental/cua/src/types.ts @@ -190,10 +190,11 @@ export interface CuaWindow extends CuaSurface { */ getScreenshot(options?: CuaObservationOptions): Promise /** - * Reveal advertised AX alternate UI, otherwise move the system cursor without pressing a button. - * The system-pointer route requires the app to be foreground and the point unobscured; never activates or raises it. - * Returning and subsequent observations do not restore the previous pointer position. - * Other input and application changes can interrupt hover; observe before continuing. + * On macOS, prefer an available control's click or advertised accessibility action over hover. + * By default, mouseMove reveals or restores alternate UI on an enabled target or ancestor that advertises + * both AXShowAlternateUI and AXShowDefaultUI. Without an AX route it throws HOVER_UNAVAILABLE before sending input. + * Only the macOS provider's systemPointerHover setting permits moving the visible cursor as a fallback. + * That route requires a foreground app and an unobscured target and leaves the cursor at the destination. * @param target - current element id or screenshot-local [x,y]. * @param options - coordinate mode; applies only to [x,y]. * @returns dispatch evidence and the selected route; observe the resulting UI after its hover delay. @@ -201,7 +202,8 @@ export interface CuaWindow extends CuaSurface { mouseMove(target: CuaElementId | readonly [number, number], options?: CuaCoordinateOptions): Promise /** * Click an element or a point in this window. On macOS both forms send mouse-down and mouse-up events. - * Use {@link CuaSurface.invoke} for the element's default accessibility action. + * For a background control, prefer {@link CuaSurface.invoke} for its default action. When a mouse click is needed, + * call app.activate() first and observe the target again. Background apps can ignore dispatched mouse events. * Cancellation or timeout releases the held button; received input cannot be undone. The operation deadline includes holdMs. * @param target - On macOS, an id from the latest tree targets the element center. A coordinate pair targets the given point. * @param options - button, click count, modifiers, optional hold duration and coordinate mode. diff --git a/snapshots/session/cua-native/replay.override.json b/snapshots/session/cua-native/replay.override.json index 42d5c9d7af..b43a6e20f0 100644 --- a/snapshots/session/cua-native/replay.override.json +++ b/snapshots/session/cua-native/replay.override.json @@ -1385,6 +1385,39 @@ } ] }, + { + "kind": "chunks", + "chunks": [ + { + "type": "block-start", + "index": 0, + "blockType": "tool-call" + }, + { + "type": "block-end", + "index": 0, + "block": { + "type": "tool-call", + "id": "cua-hover-unavailable", + "name": "cua", + "arguments": "{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":0,\"y\":200,\"relative\":true}}" + } + }, + { + "type": "usage", + "usage": { + "inputTokens": 3, + "outputTokens": 3 + } + }, + { + "type": "finish", + "reason": { + "kind": "tool-calls" + } + } + ] + }, { "kind": "chunks", "chunks": [ diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index 6e32a6a9d3..1738793701 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -33,7 +33,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -67,12 +67,12 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then call app.activate() and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. diff --git a/snapshots/session/cua-native/workspace/native-helper.mjs b/snapshots/session/cua-native/workspace/native-helper.mjs index b10d4793e0..0786eeefe6 100755 --- a/snapshots/session/cua-native/workspace/native-helper.mjs +++ b/snapshots/session/cua-native/workspace/native-helper.mjs @@ -199,12 +199,20 @@ function createApplicationState(socket, record, application, captureBroker, conn if (!observed) refuse('STALE_SNAPSHOT', 'Read fresh state before acting') if (params.windowId !== undefined && observed.windowId !== null && observed.windowId !== params.windowId) refuse('STALE_SNAPSHOT', `Snapshot belongs to window ${observed.windowId}; read getAccessibilityTree on window ${params.windowId}`) if (observed.axless && params.elementId !== undefined) refuse('INVALID_ARGUMENT', 'Element id is absent from the selected snapshot') - if (method === 'mouseMove') application.hovering = params.elementId === '0' || (params.x >= 20 && params.x <= 40) + let systemPointerMoved = false + if (method === 'mouseMove') { + const alternate = params.elementId === '0' || (params.x >= 20 && params.x <= 40) + if (!alternate && !application.hovering) { + if (!params.allowSystemPointer) refuse('HOVER_UNAVAILABLE', 'No AX alternate UI at this target. No input was sent and the pointer did not move.') + systemPointerMoved = true + } + application.hovering = alternate + } else if (method === 'click' && params.windowId === 8) application.visualCount++ else if (method === 'click') application.count++ else application.value = params.value record('state', { count: application.count, value: application.value, visualCount: application.visualCount }) - return { dispatched: true, ...method === 'mouseMove' ? { hoverRoute: 'accessibility', systemPointerMoved: false } : {} } + return { dispatched: true, ...method === 'mouseMove' ? { hoverRoute: systemPointerMoved ? 'system-pointer' : 'accessibility', systemPointerMoved } : {} } } if (method === 'pressKey' && params.key === 'Return') observationChanged = true // Fixture-only keys simulate a pending system callback and its eventual completion. diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index 81450a3782..ff29eddb7f 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -69,12 +69,12 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then call app.activate() and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index 743dd9cc1f..434e92e3f7 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -69,12 +69,12 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — reveal AX alternate UI or move the system pointer without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Use invoke(id) for the default accessibility action. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then call app.activate() and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. From ca163024ab80f64f58532e47235b4bd3aaddd4c3 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 15:25:22 +0800 Subject: [PATCH 16/44] Replay a refused pointer-free hover after dismissing alternate UI --- snapshots/session/cua-native/session.v4.jsonl | 169 +++++++++--------- 1 file changed, 87 insertions(+), 82 deletions(-) diff --git a/snapshots/session/cua-native/session.v4.jsonl b/snapshots/session/cua-native/session.v4.jsonl index 29c6f99a65..10a71dc794 100644 --- a/snapshots/session/cua-native/session.v4.jsonl +++ b/snapshots/session/cua-native/session.v4.jsonl @@ -12,281 +12,286 @@ {"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"tools":"{{tools}}"},"reason":"initial"}} {"type":"request/context","data":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"}} {"type":"session/title","data":{"title":"Inspect CUA capabilities, select the","messageSeqs":[8],"source":{"kind":"fallback"}}} -{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-1","name":"cua","arguments":"{\"method\":\"getCapabilities\",\"params\":{}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:4}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947688973,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947688973,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-1","name":"cua","arguments":"{\"method\":\"getCapabilities\",\"params\":{}}"}}},{"type":"chunk","time":1791947688973,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947688973,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-1","name":"cua","arguments":"{\"method\":\"getCapabilities\",\"params\":{}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:4}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949731596,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949731597,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-1","name":"cua","arguments":"{\"method\":\"getCapabilities\",\"params\":{}}"}}},{"type":"chunk","time":1793949731597,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949731597,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":1,"callId":"cua-native-1","name":"cua","arguments":"{\"method\":\"getCapabilities\",\"params\":{}}"}} {"type":"tool/result","data":{"turn":1,"step":1,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-1"},"toolCallId":"cua-native-1","content":[{"type":"text","text":"{\"sessionLocked\":false,\"permissions\":{\"accessibility\":true,\"screenRecording\":true},\"platform\":\"macos\",\"methods\":[\"activate\",\"click\",\"drag\",\"findApps\",\"getApp\",\"getCapabilities\",\"getState\",\"invoke\",\"listActions\",\"listApps\",\"listWindows\",\"mouseMove\",\"paste\",\"performAction\",\"pressKey\",\"screenshot\",\"scroll\",\"selectText\",\"setValue\",\"show\",\"typeText\"],\"appLevelAccessibility\":true,\"keyboard\":[\"app\"],\"browser\":false,\"coordinateSpace\":\"screenshot-pixels\"}"}],"isError":false,"id":"{{message:5}}"}},"sourceEventSeqs":[14],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":1}} {"type":"step/start","data":{"turn":1,"step":2}} -{"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-unauthorized-getState","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:6}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689573,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689573,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-unauthorized-getState","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1791947689573,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689573,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-unauthorized-getState","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:6}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732188,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732188,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-unauthorized-getState","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1793949732188,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732188,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":2,"callId":"cua-unauthorized-getState","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\"}}"}} {"type":"tool/result","data":{"turn":1,"step":2,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-unauthorized-getState"},"toolCallId":"cua-unauthorized-getState","content":[{"type":"text","text":"Error: APP_ACCESS_REQUIRED: Select the application with getApp and obtain user consent before reading or controlling it"}],"isError":true,"id":"{{message:7}}"}},"sourceEventSeqs":[19],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":2}} {"type":"step/start","data":{"turn":1,"step":3}} -{"type":"assistant/message","data":{"turn":1,"step":3,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-unauthorized-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:8}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689596,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689596,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-unauthorized-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947689596,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689596,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":3,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-unauthorized-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:8}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732206,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732206,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-unauthorized-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949732206,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732206,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":3,"callId":"cua-unauthorized-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":3,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-unauthorized-screenshot"},"toolCallId":"cua-unauthorized-screenshot","content":[{"type":"text","text":"Error: APP_ACCESS_REQUIRED: Select the application with getApp and obtain user consent before reading or controlling it"}],"isError":true,"id":"{{message:9}}"}},"sourceEventSeqs":[24],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":3}} {"type":"step/start","data":{"turn":1,"step":4}} -{"type":"assistant/message","data":{"turn":1,"step":4,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-unauthorized-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:10}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689615,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689615,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-unauthorized-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1791947689615,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689615,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":4,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-unauthorized-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:10}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732224,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732224,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-unauthorized-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1793949732224,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732224,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":4,"callId":"cua-unauthorized-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\"}}"}} {"type":"tool/result","data":{"turn":1,"step":4,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-unauthorized-click"},"toolCallId":"cua-unauthorized-click","content":[{"type":"text","text":"Error: APP_ACCESS_REQUIRED: Select the application with getApp and obtain user consent before reading or controlling it"}],"isError":true,"id":"{{message:11}}"}},"sourceEventSeqs":[29],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":4}} {"type":"step/start","data":{"turn":1,"step":5}} -{"type":"assistant/message","data":{"turn":1,"step":5,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-2","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"test.fixture\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:12}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689634,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689634,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-2","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"test.fixture\"}}"}}},{"type":"chunk","time":1791947689634,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689634,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":5,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-2","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"test.fixture\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:12}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732241,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732241,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-2","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"test.fixture\"}}"}}},{"type":"chunk","time":1793949732241,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732241,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":5,"callId":"cua-native-2","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"test.fixture\"}}"}} {"type":"tool/result","data":{"turn":1,"step":5,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-2"},"toolCallId":"cua-native-2","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"name\":\"Fixture\",\"bundleId\":\"test.fixture\",\"running\":true}"}],"isError":false,"id":"{{message:13}}"}},"sourceEventSeqs":[34],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":5}} {"type":"step/start","data":{"turn":1,"step":6}} -{"type":"assistant/message","data":{"turn":1,"step":6,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-3","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"includeMenus\":true,\"details\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:14}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689657,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689657,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-3","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"includeMenus\":true,\"details\":true}}"}}},{"type":"chunk","time":1791947689657,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689657,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":6,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-3","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"includeMenus\":true,\"details\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:14}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732276,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732276,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-3","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"includeMenus\":true,\"details\":true}}"}}},{"type":"chunk","time":1793949732276,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732276,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":6,"callId":"cua-native-3","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"includeMenus\":true,\"details\":true}}"}} {"type":"tool/result","data":{"turn":1,"step":6,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-3"},"toolCallId":"cua-native-3","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-1\",\"tree\":\"AX scope=window offsets=UTF-16 bounds=global-points\\n- [0] AXButton \\\"Increment\\\" id=\\\"increment\\\" actions=[\\\"AXPress\\\"] bounds=[60,80,100,20]\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 0\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\n - [3] AXMenuItem \\\"Fixture command\\\"\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:15}}"}},"sourceEventSeqs":[39],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":6}} {"type":"step/start","data":{"turn":1,"step":7}} -{"type":"assistant/message","data":{"turn":1,"step":7,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-4","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\",\"holdMs\":800}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:16}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689683,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689683,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-4","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\",\"holdMs\":800}}"}}},{"type":"chunk","time":1791947689683,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689683,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":7,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-4","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\",\"holdMs\":800}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:16}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732298,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732298,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-4","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\",\"holdMs\":800}}"}}},{"type":"chunk","time":1793949732298,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732298,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":7,"callId":"cua-native-4","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementId\":\"0\",\"holdMs\":800}}"}} {"type":"tool/result","data":{"turn":1,"step":7,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-4"},"toolCallId":"cua-native-4","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:17}}"}},"sourceEventSeqs":[44],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":7}} {"type":"step/start","data":{"turn":1,"step":8}} -{"type":"assistant/message","data":{"turn":1,"step":8,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-5","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:18}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689706,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689706,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-5","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947689706,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689706,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":8,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-5","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:18}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732316,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732316,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-5","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949732316,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732316,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":8,"callId":"cua-native-5","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":8,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-5"},"toolCallId":"cua-native-5","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-2\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 1\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:19}}"}},"sourceEventSeqs":[49],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":8}} {"type":"step/start","data":{"turn":1,"step":9}} -{"type":"assistant/message","data":{"turn":1,"step":9,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-6","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:20}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689729,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689729,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-6","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947689729,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689729,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":9,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-6","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:20}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732333,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732333,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-6","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949732333,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732333,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":9,"callId":"cua-native-6","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":9,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-6"},"toolCallId":"cua-native-6","content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}],"isError":false,"id":"{{message:21}}"}},"sourceEventSeqs":[54],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":9}} {"type":"step/start","data":{"turn":1,"step":10}} -{"type":"assistant/message","data":{"turn":1,"step":10,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-7","name":"cua","arguments":"{\"method\":\"listWindows\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:22}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689792,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689792,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-7","name":"cua","arguments":"{\"method\":\"listWindows\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1791947689792,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689792,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":10,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-7","name":"cua","arguments":"{\"method\":\"listWindows\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:22}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732405,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732405,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-7","name":"cua","arguments":"{\"method\":\"listWindows\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1793949732405,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732405,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":10,"callId":"cua-native-7","name":"cua","arguments":"{\"method\":\"listWindows\",\"params\":{\"appHandle\":\"fixture-app\"}}"}} {"type":"tool/result","data":{"turn":1,"step":10,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-7"},"toolCallId":"cua-native-7","content":[{"type":"text","text":"[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true},{\"windowId\":8,\"appName\":\"Fixture\",\"title\":\"AXless canvas\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}]"}],"isError":false,"id":"{{message:23}}"}},"sourceEventSeqs":[59],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":10}} {"type":"step/start","data":{"turn":1,"step":11}} -{"type":"assistant/message","data":{"turn":1,"step":11,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-8","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:24}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689834,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689834,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-8","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}}},{"type":"chunk","time":1791947689834,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689834,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":11,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-8","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:24}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732424,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732424,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-8","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}}},{"type":"chunk","time":1793949732424,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732424,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":11,"callId":"cua-native-8","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}} {"type":"tool/result","data":{"turn":1,"step":11,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-8"},"toolCallId":"cua-native-8","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-3\",\"tree\":\"\",\"windows\":[{\"windowId\":8,\"appName\":\"Fixture\",\"title\":\"AXless canvas\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:25}}"}},"sourceEventSeqs":[64],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":11}} {"type":"step/start","data":{"turn":1,"step":12}} -{"type":"assistant/message","data":{"turn":1,"step":12,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-9","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:26}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689861,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689861,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-9","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}}},{"type":"chunk","time":1791947689861,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689861,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":12,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-9","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:26}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732445,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732445,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-9","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}}},{"type":"chunk","time":1793949732445,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732445,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":12,"callId":"cua-native-9","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}} {"type":"tool/result","data":{"turn":1,"step":12,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-9"},"toolCallId":"cua-native-9","content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":8,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}],"isError":false,"id":"{{message:27}}"}},"sourceEventSeqs":[69],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":12}} {"type":"step/start","data":{"turn":1,"step":13}} -{"type":"assistant/message","data":{"turn":1,"step":13,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-10","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-3\",\"x\":0,\"y\":0}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:28}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689896,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689896,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-10","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-3\",\"x\":0,\"y\":0}}"}}},{"type":"chunk","time":1791947689896,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689896,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":13,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-10","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-3\",\"x\":0,\"y\":0}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:28}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732481,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732481,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-10","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-3\",\"x\":0,\"y\":0}}"}}},{"type":"chunk","time":1793949732481,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732481,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":13,"callId":"cua-native-10","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-3\",\"x\":0,\"y\":0}}"}} {"type":"tool/result","data":{"turn":1,"step":13,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-10"},"toolCallId":"cua-native-10","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:29}}"}},"sourceEventSeqs":[74],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":13}} {"type":"step/start","data":{"turn":1,"step":14}} -{"type":"assistant/message","data":{"turn":1,"step":14,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-11","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:30}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689918,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689918,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-11","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}}},{"type":"chunk","time":1791947689918,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689918,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":14,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-11","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:30}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732502,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732502,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-11","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}}},{"type":"chunk","time":1793949732502,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732502,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":14,"callId":"cua-native-11","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8}}"}} {"type":"tool/result","data":{"turn":1,"step":14,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-11"},"toolCallId":"cua-native-11","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-4\",\"tree\":\"\",\"windows\":[{\"windowId\":8,\"appName\":\"Fixture\",\"title\":\"AXless canvas\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:31}}"}},"sourceEventSeqs":[79],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":14}} {"type":"step/start","data":{"turn":1,"step":15}} -{"type":"assistant/message","data":{"turn":1,"step":15,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-show","name":"cua","arguments":"{\"method\":\"show\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:32}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689940,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689940,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-show","name":"cua","arguments":"{\"method\":\"show\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947689940,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689940,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":15,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-show","name":"cua","arguments":"{\"method\":\"show\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:32}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732523,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732523,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-show","name":"cua","arguments":"{\"method\":\"show\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949732523,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732523,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":15,"callId":"cua-native-show","name":"cua","arguments":"{\"method\":\"show\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":15,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-show"},"toolCallId":"cua-native-show","content":[{"type":"text","text":"{\"dispatched\":true,\"windowId\":7,\"foregroundChanged\":false}"}],"isError":false,"id":"{{message:33}}"}},"sourceEventSeqs":[84],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":15}} {"type":"step/start","data":{"turn":1,"step":16}} -{"type":"assistant/message","data":{"turn":1,"step":16,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-activate-first","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:34}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689961,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689961,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-activate-first","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1791947689961,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689961,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":16,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-activate-first","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:34}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732546,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732546,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-activate-first","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1793949732546,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732546,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":16,"callId":"cua-native-activate-first","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}} {"type":"tool/result","data":{"turn":1,"step":16,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-activate-first"},"toolCallId":"cua-native-activate-first","content":[{"type":"text","text":"{\"dispatched\":true,\"foregroundChanged\":true,\"foreground\":{\"requestId\":\"cua-native-activate-first\",\"targetPid\":9,\"before\":{\"frontPid\":42,\"focusedWindowId\":7,\"frontWindowId\":99,\"frontWindowPid\":42},\"after\":{\"frontPid\":9,\"focusedWindowId\":7,\"frontWindowId\":7,\"frontWindowPid\":9},\"activationRequested\":true}}"}],"isError":false,"id":"{{message:35}}"}},"sourceEventSeqs":[89],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":16}} {"type":"step/start","data":{"turn":1,"step":17}} -{"type":"assistant/message","data":{"turn":1,"step":17,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-activate-repeat","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:36}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947689983,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947689983,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-activate-repeat","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1791947689983,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947689983,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":17,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-activate-repeat","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:36}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732573,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732573,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-activate-repeat","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1793949732573,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732573,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":17,"callId":"cua-native-activate-repeat","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}} {"type":"tool/result","data":{"turn":1,"step":17,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-activate-repeat"},"toolCallId":"cua-native-activate-repeat","content":[{"type":"text","text":"{\"dispatched\":true,\"foregroundChanged\":false,\"foreground\":{\"requestId\":\"cua-native-activate-repeat\",\"targetPid\":9,\"before\":{\"frontPid\":9,\"focusedWindowId\":7,\"frontWindowId\":7,\"frontWindowPid\":9},\"after\":{\"frontPid\":9,\"focusedWindowId\":7,\"frontWindowId\":7,\"frontWindowPid\":9},\"activationRequested\":false}}"}],"isError":false,"id":"{{message:37}}"}},"sourceEventSeqs":[94],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":17}} {"type":"step/start","data":{"turn":1,"step":18}} -{"type":"assistant/message","data":{"turn":1,"step":18,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-transition","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"Return\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:38}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690002,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690002,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-transition","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"Return\"}}"}}},{"type":"chunk","time":1791947690002,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690002,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":18,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-transition","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"Return\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:38}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732598,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732598,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-transition","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"Return\"}}"}}},{"type":"chunk","time":1793949732598,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732598,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":18,"callId":"cua-native-transition","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"Return\"}}"}} {"type":"tool/result","data":{"turn":1,"step":18,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-transition"},"toolCallId":"cua-native-transition","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:39}}"}},"sourceEventSeqs":[99],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":18}} {"type":"step/start","data":{"turn":1,"step":19}} -{"type":"assistant/message","data":{"turn":1,"step":19,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-stale-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:40}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690019,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690019,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-stale-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690019,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690019,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":19,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-stale-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:40}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732622,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732622,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-stale-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949732622,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732622,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":19,"callId":"cua-native-stale-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":19,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-stale-read"},"toolCallId":"cua-native-stale-read","content":[{"type":"text","text":"Error: STALE_SNAPSHOT: Accessibility tree changed during observation; read fresh getAccessibilityTree"}],"isError":true,"id":"{{message:41}}"}},"sourceEventSeqs":[104],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":19}} {"type":"step/start","data":{"turn":1,"step":20}} -{"type":"assistant/message","data":{"turn":1,"step":20,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-stale-index","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-2\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:42}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690040,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690040,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-stale-index","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-2\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1791947690040,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690040,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":20,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-stale-index","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-2\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:42}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732643,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732643,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-stale-index","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-2\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1793949732643,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732643,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":20,"callId":"cua-native-stale-index","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-2\",\"elementId\":\"0\"}}"}} {"type":"tool/result","data":{"turn":1,"step":20,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-stale-index"},"toolCallId":"cua-native-stale-index","content":[{"type":"text","text":"Error: STALE_SNAPSHOT: Read fresh state before acting"}],"isError":true,"id":"{{message:43}}"}},"sourceEventSeqs":[109],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":20}} {"type":"step/start","data":{"turn":1,"step":21}} -{"type":"assistant/message","data":{"turn":1,"step":21,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-other-window","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-4\",\"x\":0,\"y\":0,\"holdMs\":800}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:44}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690064,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690064,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-other-window","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-4\",\"x\":0,\"y\":0,\"holdMs\":800}}"}}},{"type":"chunk","time":1791947690064,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690064,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":21,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-other-window","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-4\",\"x\":0,\"y\":0,\"holdMs\":800}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:44}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732663,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732663,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-other-window","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-4\",\"x\":0,\"y\":0,\"holdMs\":800}}"}}},{"type":"chunk","time":1793949732663,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732663,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":21,"callId":"cua-native-other-window","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-4\",\"x\":0,\"y\":0,\"holdMs\":800}}"}} {"type":"tool/result","data":{"turn":1,"step":21,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-other-window"},"toolCallId":"cua-native-other-window","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:45}}"}},"sourceEventSeqs":[114],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":21}} {"type":"step/start","data":{"turn":1,"step":22}} -{"type":"assistant/message","data":{"turn":1,"step":22,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-fresh-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:46}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690087,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690087,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-fresh-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690087,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690087,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":22,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-fresh-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:46}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732687,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732687,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-fresh-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949732687,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732687,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":22,"callId":"cua-native-fresh-read","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":22,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-fresh-read"},"toolCallId":"cua-native-fresh-read","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-5\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 1\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:47}}"}},"sourceEventSeqs":[119],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":22}} {"type":"step/start","data":{"turn":1,"step":23}} -{"type":"assistant/message","data":{"turn":1,"step":23,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-fresh-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-5\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:48}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690111,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690111,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-fresh-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-5\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1791947690111,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690111,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":23,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-fresh-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-5\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:48}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732710,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732710,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-fresh-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-5\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1793949732710,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732710,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":23,"callId":"cua-native-fresh-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-5\",\"elementId\":\"0\"}}"}} {"type":"tool/result","data":{"turn":1,"step":23,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-fresh-click"},"toolCallId":"cua-native-fresh-click","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:49}}"}},"sourceEventSeqs":[124],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":23}} {"type":"step/start","data":{"turn":1,"step":24}} -{"type":"assistant/message","data":{"turn":1,"step":24,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:50}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690141,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690141,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690141,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690141,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":24,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:50}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732733,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732733,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949732733,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732733,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":24,"callId":"cua-native-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":24,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-verify"},"toolCallId":"cua-native-verify","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-6\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 2\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:51}}"}},"sourceEventSeqs":[129],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":24}} {"type":"step/start","data":{"turn":1,"step":25}} -{"type":"assistant/message","data":{"turn":1,"step":25,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:52}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690162,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690162,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}}},{"type":"chunk","time":1791947690162,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690162,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":25,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:52}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732756,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732756,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}}},{"type":"chunk","time":1793949732756,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732756,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":25,"callId":"cua-native-geometry-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}} {"type":"tool/result","data":{"turn":1,"step":25,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-geometry-observe"},"toolCallId":"cua-native-geometry-observe","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":9,\"snapshotId\":\"fixture-snapshot-7\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 2\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":9,\"appName\":\"Fixture\",\"title\":\"Transformed window\",\"bounds\":{\"x\":16,\"y\":512,\"width\":131,\"height\":145},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:53}}"}},"sourceEventSeqs":[134],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":25}} {"type":"step/start","data":{"turn":1,"step":26}} -{"type":"assistant/message","data":{"turn":1,"step":26,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:54}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690186,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690186,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}}},{"type":"chunk","time":1791947690186,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690186,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":26,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:54}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732777,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732777,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}}},{"type":"chunk","time":1793949732777,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732777,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":26,"callId":"cua-native-geometry-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}} {"type":"tool/result","data":{"turn":1,"step":26,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-geometry-capture"},"toolCallId":"cua-native-geometry-capture","content":[{"type":"text","text":"Error: WINDOW_CHANGED: Window display and accessibility bounds disagree. Read fresh state; if unchanged, use AX actions or explicitly restore the window before capture or coordinate input"}],"isError":true,"id":"{{message:55}}"}},"sourceEventSeqs":[139],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":26}} {"type":"step/start","data":{"turn":1,"step":27}} -{"type":"assistant/message","data":{"turn":1,"step":27,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"snapshotId\":\"fixture-snapshot-7\",\"x\":20,\"y\":20,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:56}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690221,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690221,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"snapshotId\":\"fixture-snapshot-7\",\"x\":20,\"y\":20,\"relative\":true}}"}}},{"type":"chunk","time":1791947690221,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690221,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":27,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"snapshotId\":\"fixture-snapshot-7\",\"x\":20,\"y\":20,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:56}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732798,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732798,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"snapshotId\":\"fixture-snapshot-7\",\"x\":20,\"y\":20,\"relative\":true}}"}}},{"type":"chunk","time":1793949732798,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732798,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":27,"callId":"cua-native-geometry-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"snapshotId\":\"fixture-snapshot-7\",\"x\":20,\"y\":20,\"relative\":true}}"}} {"type":"tool/result","data":{"turn":1,"step":27,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-geometry-click"},"toolCallId":"cua-native-geometry-click","content":[{"type":"text","text":"Error: WINDOW_CHANGED: Window display and accessibility bounds disagree. Read fresh state; if unchanged, use AX actions or explicitly restore the window before capture or coordinate input"}],"isError":true,"id":"{{message:57}}"}},"sourceEventSeqs":[144],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":27}} {"type":"step/start","data":{"turn":1,"step":28}} -{"type":"assistant/message","data":{"turn":1,"step":28,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-scroll","name":"cua","arguments":"{\"method\":\"scroll\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"x\":20,\"y\":20,\"deltaLinesX\":0,\"deltaLinesY\":-1,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:58}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690248,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690248,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-scroll","name":"cua","arguments":"{\"method\":\"scroll\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"x\":20,\"y\":20,\"deltaLinesX\":0,\"deltaLinesY\":-1,\"relative\":true}}"}}},{"type":"chunk","time":1791947690248,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690248,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":28,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-scroll","name":"cua","arguments":"{\"method\":\"scroll\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"x\":20,\"y\":20,\"deltaLinesX\":0,\"deltaLinesY\":-1,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:58}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732822,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732822,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-scroll","name":"cua","arguments":"{\"method\":\"scroll\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"x\":20,\"y\":20,\"deltaLinesX\":0,\"deltaLinesY\":-1,\"relative\":true}}"}}},{"type":"chunk","time":1793949732822,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732822,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":28,"callId":"cua-native-geometry-scroll","name":"cua","arguments":"{\"method\":\"scroll\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"x\":20,\"y\":20,\"deltaLinesX\":0,\"deltaLinesY\":-1,\"relative\":true}}"}} {"type":"tool/result","data":{"turn":1,"step":28,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-geometry-scroll"},"toolCallId":"cua-native-geometry-scroll","content":[{"type":"text","text":"Error: WINDOW_CHANGED: Window display and accessibility bounds disagree. Read fresh state; if unchanged, use AX actions or explicitly restore the window before capture or coordinate input"}],"isError":true,"id":"{{message:59}}"}},"sourceEventSeqs":[149],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":28}} {"type":"step/start","data":{"turn":1,"step":29}} -{"type":"assistant/message","data":{"turn":1,"step":29,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-drag","name":"cua","arguments":"{\"method\":\"drag\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"fromX\":20,\"fromY\":20,\"toX\":30,\"toY\":30,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:60}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690267,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690267,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-drag","name":"cua","arguments":"{\"method\":\"drag\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"fromX\":20,\"fromY\":20,\"toX\":30,\"toY\":30,\"relative\":true}}"}}},{"type":"chunk","time":1791947690267,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690267,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":29,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-drag","name":"cua","arguments":"{\"method\":\"drag\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"fromX\":20,\"fromY\":20,\"toX\":30,\"toY\":30,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:60}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732845,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732845,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-drag","name":"cua","arguments":"{\"method\":\"drag\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"fromX\":20,\"fromY\":20,\"toX\":30,\"toY\":30,\"relative\":true}}"}}},{"type":"chunk","time":1793949732845,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732845,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":29,"callId":"cua-native-geometry-drag","name":"cua","arguments":"{\"method\":\"drag\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"fromX\":20,\"fromY\":20,\"toX\":30,\"toY\":30,\"relative\":true}}"}} {"type":"tool/result","data":{"turn":1,"step":29,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-geometry-drag"},"toolCallId":"cua-native-geometry-drag","content":[{"type":"text","text":"Error: WINDOW_CHANGED: Window display and accessibility bounds disagree. Read fresh state; if unchanged, use AX actions or explicitly restore the window before capture or coordinate input"}],"isError":true,"id":"{{message:61}}"}},"sourceEventSeqs":[154],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":29}} {"type":"step/start","data":{"turn":1,"step":30}} -{"type":"assistant/message","data":{"turn":1,"step":30,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-ax","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-7\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:62}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690294,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690294,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-ax","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-7\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1791947690294,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690294,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":30,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-ax","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-7\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:62}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732870,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732870,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-ax","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-7\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1793949732870,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732870,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":30,"callId":"cua-native-geometry-ax","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-7\",\"elementId\":\"0\"}}"}} {"type":"tool/result","data":{"turn":1,"step":30,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-geometry-ax"},"toolCallId":"cua-native-geometry-ax","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:63}}"}},"sourceEventSeqs":[159],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":30}} {"type":"step/start","data":{"turn":1,"step":31}} -{"type":"assistant/message","data":{"turn":1,"step":31,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:64}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690314,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690314,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}}},{"type":"chunk","time":1791947690314,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690314,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":31,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-native-geometry-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:64}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732900,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732900,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-native-geometry-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}}},{"type":"chunk","time":1793949732900,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732900,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":31,"callId":"cua-native-geometry-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9}}"}} {"type":"tool/result","data":{"turn":1,"step":31,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-native-geometry-verify"},"toolCallId":"cua-native-geometry-verify","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":9,\"snapshotId\":\"fixture-snapshot-8\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 3\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":9,\"appName\":\"Fixture\",\"title\":\"Transformed window\",\"bounds\":{\"x\":16,\"y\":512,\"width\":131,\"height\":145},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:65}}"}},"sourceEventSeqs":[164],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":31}} {"type":"step/start","data":{"turn":1,"step":32}} -{"type":"assistant/message","data":{"turn":1,"step":32,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-busy-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F11\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:66}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690340,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690340,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-busy-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F11\"}}"}}},{"type":"chunk","time":1791947690340,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690340,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":32,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-busy-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F11\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:66}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732924,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732924,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-busy-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F11\"}}"}}},{"type":"chunk","time":1793949732924,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732924,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":32,"callId":"cua-capture-busy-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F11\"}}"}} {"type":"tool/result","data":{"turn":1,"step":32,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-capture-busy-arm"},"toolCallId":"cua-capture-busy-arm","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:67}}"}},"sourceEventSeqs":[169],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":32}} {"type":"step/start","data":{"turn":1,"step":33}} -{"type":"assistant/message","data":{"turn":1,"step":33,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-busy","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:68}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690365,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690365,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-busy","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690365,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690365,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":33,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-busy","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:68}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732947,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732947,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-busy","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949732947,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732947,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":33,"callId":"cua-capture-busy","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":33,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-capture-busy"},"toolCallId":"cua-capture-busy","content":[{"type":"text","text":"Error: CAPTURE_BUSY: A previous system capture is still pending; other native operations remain available\nScreenshot diagnostics:\n[cua-screenshot] {\"requestId\":\"cua-capture-busy\",\"stage\":\"helper.failed\",\"elapsedMs\":1,\"errorCode\":\"CAPTURE_BUSY\"}"}],"isError":true,"id":"{{message:69}}"}},"sourceEventSeqs":[174],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":33}} {"type":"step/start","data":{"turn":1,"step":34}} -{"type":"assistant/message","data":{"turn":1,"step":34,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-busy-ax","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:70}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690394,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690394,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-busy-ax","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690394,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690394,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":34,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-busy-ax","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:70}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732969,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732969,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-busy-ax","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949732969,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732969,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":34,"callId":"cua-capture-busy-ax","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":34,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-capture-busy-ax"},"toolCallId":"cua-capture-busy-ax","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-9\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 3\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:71}}"}},"sourceEventSeqs":[179],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":34}} {"type":"step/start","data":{"turn":1,"step":35}} -{"type":"assistant/message","data":{"turn":1,"step":35,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-callback","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F10\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:72}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690414,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690414,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-callback","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F10\"}}"}}},{"type":"chunk","time":1791947690414,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690414,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":35,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-callback","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F10\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:72}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949732988,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949732988,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-callback","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F10\"}}"}}},{"type":"chunk","time":1793949732988,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949732988,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":35,"callId":"cua-capture-callback","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F10\"}}"}} {"type":"tool/result","data":{"turn":1,"step":35,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-capture-callback"},"toolCallId":"cua-capture-callback","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:73}}"}},"sourceEventSeqs":[184],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":35}} {"type":"step/start","data":{"turn":1,"step":36}} -{"type":"assistant/message","data":{"turn":1,"step":36,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-recovered","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:74}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690434,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690434,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-recovered","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690434,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690434,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":36,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-capture-recovered","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:74}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733007,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733007,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-capture-recovered","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949733007,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733007,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":36,"callId":"cua-capture-recovered","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":36,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-capture-recovered"},"toolCallId":"cua-capture-recovered","content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}],"isError":false,"id":"{{message:75}}"}},"sourceEventSeqs":[189],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":36}} {"type":"step/start","data":{"turn":1,"step":37}} -{"type":"assistant/message","data":{"turn":1,"step":37,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-enter","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-9\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:76}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690466,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690466,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-enter","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-9\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1791947690466,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690466,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":37,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-enter","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-9\",\"elementId\":\"0\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:76}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733041,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733041,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-enter","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-9\",\"elementId\":\"0\"}}"}}},{"type":"chunk","time":1793949733041,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733041,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":37,"callId":"cua-hover-enter","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-9\",\"elementId\":\"0\"}}"}} {"type":"tool/result","data":{"turn":1,"step":37,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-hover-enter"},"toolCallId":"cua-hover-enter","content":[{"type":"text","text":"{\"dispatched\":true,\"hoverRoute\":\"accessibility\",\"systemPointerMoved\":false}"}],"isError":false,"id":"{{message:77}}"}},"sourceEventSeqs":[194],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":37}} {"type":"step/start","data":{"turn":1,"step":38}} -{"type":"assistant/message","data":{"turn":1,"step":38,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:78}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690495,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690495,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690495,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690495,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":38,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:78}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733060,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733060,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949733060,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733060,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":38,"callId":"cua-hover-observe","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":38,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-hover-observe"},"toolCallId":"cua-hover-observe","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-10\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 3\\\"\\n- AXStaticText \\\"Hover information\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:79}}"}},"sourceEventSeqs":[199],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":38}} {"type":"step/start","data":{"turn":1,"step":39}} -{"type":"assistant/message","data":{"turn":1,"step":39,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:80}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690517,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690517,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690517,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690517,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":39,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:80}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733087,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733087,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949733087,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733087,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":39,"callId":"cua-hover-capture","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} {"type":"tool/result","data":{"turn":1,"step":39,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-hover-capture"},"toolCallId":"cua-hover-capture","content":[{"type":"text","text":"{\"pixelWidth\":1,\"pixelHeight\":1,\"windowId\":7,\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"coordinateSpace\":\"screenshot-pixels\"}"},{"type":"image","attachment":{"attachmentId":"sha256:999f1d1527ee7e79266f16add5430fff76b1225d742464a5b1ff1f02971bb8ee","mediaType":"image/png","width":1,"height":1,"bytes":90}}],"isError":false,"id":"{{message:81}}"}},"sourceEventSeqs":[204],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":39}} {"type":"step/start","data":{"turn":1,"step":40}} -{"type":"assistant/message","data":{"turn":1,"step":40,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-popup","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":100,\"y\":200,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:82}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690565,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690565,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-popup","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":100,\"y\":200,\"relative\":true}}"}}},{"type":"chunk","time":1791947690565,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690565,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":40,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-popup","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":100,\"y\":200,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:82}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733130,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733130,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-popup","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":100,\"y\":200,\"relative\":true}}"}}},{"type":"chunk","time":1793949733130,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733130,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":40,"callId":"cua-hover-popup","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":100,\"y\":200,\"relative\":true}}"}} {"type":"tool/result","data":{"turn":1,"step":40,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-hover-popup"},"toolCallId":"cua-hover-popup","content":[{"type":"text","text":"{\"dispatched\":true,\"hoverRoute\":\"accessibility\",\"systemPointerMoved\":false}"}],"isError":false,"id":"{{message:83}}"}},"sourceEventSeqs":[209],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":40}} {"type":"step/start","data":{"turn":1,"step":41}} -{"type":"assistant/message","data":{"turn":1,"step":41,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-10\",\"x\":100,\"y\":200,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:84}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690590,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690590,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-10\",\"x\":100,\"y\":200,\"relative\":true}}"}}},{"type":"chunk","time":1791947690590,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690590,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":41,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-10\",\"x\":100,\"y\":200,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:84}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733147,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733147,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-10\",\"x\":100,\"y\":200,\"relative\":true}}"}}},{"type":"chunk","time":1793949733147,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733147,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":41,"callId":"cua-hover-click","name":"cua","arguments":"{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-10\",\"x\":100,\"y\":200,\"relative\":true}}"}} {"type":"tool/result","data":{"turn":1,"step":41,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-hover-click"},"toolCallId":"cua-hover-click","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:85}}"}},"sourceEventSeqs":[214],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":41}} {"type":"step/start","data":{"turn":1,"step":42}} -{"type":"assistant/message","data":{"turn":1,"step":42,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-exit","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":0,\"y\":200,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:86}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690613,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690613,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-exit","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":0,\"y\":200,\"relative\":true}}"}}},{"type":"chunk","time":1791947690613,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690613,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":42,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-exit","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":0,\"y\":200,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:86}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733164,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733164,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-exit","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":0,\"y\":200,\"relative\":true}}"}}},{"type":"chunk","time":1793949733164,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733164,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} {"type":"tool/call","data":{"turn":1,"step":42,"callId":"cua-hover-exit","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":0,\"y\":200,\"relative\":true}}"}} {"type":"tool/result","data":{"turn":1,"step":42,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-hover-exit"},"toolCallId":"cua-hover-exit","content":[{"type":"text","text":"{\"dispatched\":true,\"hoverRoute\":\"accessibility\",\"systemPointerMoved\":false}"}],"isError":false,"id":"{{message:87}}"}},"sourceEventSeqs":[219],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":42}} {"type":"step/start","data":{"turn":1,"step":43}} -{"type":"assistant/message","data":{"turn":1,"step":43,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:88}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690633,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690633,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690633,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690633,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":43,"callId":"cua-hover-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} -{"type":"tool/result","data":{"turn":1,"step":43,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-hover-verify"},"toolCallId":"cua-hover-verify","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-11\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 4\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:89}}"}},"sourceEventSeqs":[224],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":43,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-unavailable","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":0,\"y\":200,\"relative\":true}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:88}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733183,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733183,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-unavailable","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":0,\"y\":200,\"relative\":true}}"}}},{"type":"chunk","time":1793949733183,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733183,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":43,"callId":"cua-hover-unavailable","name":"cua","arguments":"{\"method\":\"mouseMove\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"x\":0,\"y\":200,\"relative\":true}}"}} +{"type":"tool/result","data":{"turn":1,"step":43,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-hover-unavailable"},"toolCallId":"cua-hover-unavailable","content":[{"type":"text","text":"Error: HOVER_UNAVAILABLE: No AX alternate UI at this target. No input was sent and the pointer did not move."}],"isError":true,"id":"{{message:89}}"}},"sourceEventSeqs":[224],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":43}} {"type":"step/start","data":{"turn":1,"step":44}} -{"type":"assistant/message","data":{"turn":1,"step":44,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-background","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F9\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:90}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690653,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690653,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-background","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F9\"}}"}}},{"type":"chunk","time":1791947690653,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690653,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":44,"callId":"cua-paste-background","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F9\"}}"}} -{"type":"tool/result","data":{"turn":1,"step":44,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-background"},"toolCallId":"cua-paste-background","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:91}}"}},"sourceEventSeqs":[229],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":44,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-hover-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:90}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733201,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733201,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-hover-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949733201,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733201,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":44,"callId":"cua-hover-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} +{"type":"tool/result","data":{"turn":1,"step":44,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-hover-verify"},"toolCallId":"cua-hover-verify","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-11\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 4\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:91}}"}},"sourceEventSeqs":[229],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":44}} {"type":"step/start","data":{"turn":1,"step":45}} -{"type":"assistant/message","data":{"turn":1,"step":45,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-unread","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:92}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690672,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690672,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-unread","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}}},{"type":"chunk","time":1791947690672,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690672,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":45,"callId":"cua-paste-unread","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}} -{"type":"tool/result","data":{"turn":1,"step":45,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-unread"},"toolCallId":"cua-paste-unread","content":[{"type":"text","text":"Error: PASTE_NOT_CONSUMED: Paste command was dispatched, but clipboard data was not requested within the read deadline. Bring the target app to the foreground and focus its text input. Inspect the contents before deciding whether to retry; a delayed paste may still run."}],"isError":true,"id":"{{message:93}}"}},"sourceEventSeqs":[234],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":45,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-background","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F9\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:92}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733220,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733220,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-background","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F9\"}}"}}},{"type":"chunk","time":1793949733220,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733220,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":45,"callId":"cua-paste-background","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F9\"}}"}} +{"type":"tool/result","data":{"turn":1,"step":45,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-background"},"toolCallId":"cua-paste-background","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:93}}"}},"sourceEventSeqs":[234],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":45}} {"type":"step/start","data":{"turn":1,"step":46}} -{"type":"assistant/message","data":{"turn":1,"step":46,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-inspect","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:94}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690696,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690696,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-inspect","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690696,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690696,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":46,"callId":"cua-paste-inspect","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} -{"type":"tool/result","data":{"turn":1,"step":46,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-inspect"},"toolCallId":"cua-paste-inspect","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-12\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 4\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:95}}"}},"sourceEventSeqs":[239],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":46,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-unread","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:94}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733238,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733238,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-unread","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}}},{"type":"chunk","time":1793949733238,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733238,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":46,"callId":"cua-paste-unread","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}} +{"type":"tool/result","data":{"turn":1,"step":46,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-unread"},"toolCallId":"cua-paste-unread","content":[{"type":"text","text":"Error: PASTE_NOT_CONSUMED: Paste command was dispatched, but clipboard data was not requested within the read deadline. Bring the target app to the foreground and focus its text input. Inspect the contents before deciding whether to retry; a delayed paste may still run."}],"isError":true,"id":"{{message:95}}"}},"sourceEventSeqs":[239],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":46}} {"type":"step/start","data":{"turn":1,"step":47}} -{"type":"assistant/message","data":{"turn":1,"step":47,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-activate","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:96}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690714,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690714,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-activate","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1791947690714,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690714,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":47,"callId":"cua-paste-activate","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}} -{"type":"tool/result","data":{"turn":1,"step":47,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-activate"},"toolCallId":"cua-paste-activate","content":[{"type":"text","text":"{\"dispatched\":true,\"foregroundChanged\":true,\"foreground\":{\"requestId\":\"cua-paste-activate\",\"targetPid\":9,\"before\":{\"frontPid\":42,\"focusedWindowId\":7,\"frontWindowId\":99,\"frontWindowPid\":42},\"after\":{\"frontPid\":9,\"focusedWindowId\":7,\"frontWindowId\":7,\"frontWindowPid\":9},\"activationRequested\":true}}"}],"isError":false,"id":"{{message:97}}"}},"sourceEventSeqs":[244],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":47,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-inspect","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:96}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733260,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733260,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-inspect","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949733260,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733260,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":47,"callId":"cua-paste-inspect","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} +{"type":"tool/result","data":{"turn":1,"step":47,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-inspect"},"toolCallId":"cua-paste-inspect","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-12\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 4\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:97}}"}},"sourceEventSeqs":[244],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":47}} {"type":"step/start","data":{"turn":1,"step":48}} -{"type":"assistant/message","data":{"turn":1,"step":48,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-focus","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:98}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690734,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690734,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-focus","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690734,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690734,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":48,"callId":"cua-paste-focus","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} -{"type":"tool/result","data":{"turn":1,"step":48,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-focus"},"toolCallId":"cua-paste-focus","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-13\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 4\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:99}}"}},"sourceEventSeqs":[249],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":48,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-activate","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:98}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733280,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733280,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-activate","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}}},{"type":"chunk","time":1793949733280,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733280,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":48,"callId":"cua-paste-activate","name":"cua","arguments":"{\"method\":\"activate\",\"params\":{\"appHandle\":\"fixture-app\"}}"}} +{"type":"tool/result","data":{"turn":1,"step":48,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-activate"},"toolCallId":"cua-paste-activate","content":[{"type":"text","text":"{\"dispatched\":true,\"foregroundChanged\":true,\"foreground\":{\"requestId\":\"cua-paste-activate\",\"targetPid\":9,\"before\":{\"frontPid\":42,\"focusedWindowId\":7,\"frontWindowId\":99,\"frontWindowPid\":42},\"after\":{\"frontPid\":9,\"focusedWindowId\":7,\"frontWindowId\":7,\"frontWindowPid\":9},\"activationRequested\":true}}"}],"isError":false,"id":"{{message:99}}"}},"sourceEventSeqs":[249],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":48}} {"type":"step/start","data":{"turn":1,"step":49}} -{"type":"assistant/message","data":{"turn":1,"step":49,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-consumed","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:100}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690757,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690757,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-consumed","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}}},{"type":"chunk","time":1791947690757,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690757,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":49,"callId":"cua-paste-consumed","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}} -{"type":"tool/result","data":{"turn":1,"step":49,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-consumed"},"toolCallId":"cua-paste-consumed","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:101}}"}},"sourceEventSeqs":[254],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":49,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-focus","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:100}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733300,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733300,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-focus","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949733300,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733300,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":49,"callId":"cua-paste-focus","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} +{"type":"tool/result","data":{"turn":1,"step":49,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-focus"},"toolCallId":"cua-paste-focus","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-13\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 4\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:101}}"}},"sourceEventSeqs":[254],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":49}} {"type":"step/start","data":{"turn":1,"step":50}} -{"type":"assistant/message","data":{"turn":1,"step":50,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:102}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690778,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690778,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690778,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690778,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":50,"callId":"cua-paste-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} -{"type":"tool/result","data":{"turn":1,"step":50,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-verify"},"toolCallId":"cua-paste-verify","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-14\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"fixture paste\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 4\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:103}}"}},"sourceEventSeqs":[259],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":50,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-consumed","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:102}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733320,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733320,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-consumed","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}}},{"type":"chunk","time":1793949733320,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733320,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":50,"callId":"cua-paste-consumed","name":"cua","arguments":"{\"method\":\"paste\",\"params\":{\"appHandle\":\"fixture-app\",\"text\":\"fixture paste\"}}"}} +{"type":"tool/result","data":{"turn":1,"step":50,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-consumed"},"toolCallId":"cua-paste-consumed","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:103}}"}},"sourceEventSeqs":[259],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":50}} {"type":"step/start","data":{"turn":1,"step":51}} -{"type":"assistant/message","data":{"turn":1,"step":51,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-debug-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F12\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:104}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690801,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690801,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-debug-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F12\"}}"}}},{"type":"chunk","time":1791947690801,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690801,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":51,"callId":"cua-debug-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F12\"}}"}} -{"type":"tool/result","data":{"turn":1,"step":51,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-debug-arm"},"toolCallId":"cua-debug-arm","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:105}}"}},"sourceEventSeqs":[264],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":51,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-paste-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:104}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733340,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733340,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-paste-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949733340,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733340,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":51,"callId":"cua-paste-verify","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} +{"type":"tool/result","data":{"turn":1,"step":51,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-paste-verify"},"toolCallId":"cua-paste-verify","content":[{"type":"text","text":"{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-14\",\"tree\":\"AX scope=window offsets=UTF-16\\n- [0] AXButton \\\"Increment\\\"\\n- [1] AXTextField valueWritable=true selectionWritable=true value=\\\"fixture paste\\\" placeholder=\\\"Fixture text\\\"\\n- AXStaticText \\\"Count: 4\\\"\\n- [2] AXMenuBarItem \\\"File\\\"\\nMenu contents omitted; request includeMenus=true to expand.\",\"windows\":[{\"windowId\":7,\"appName\":\"Fixture\",\"title\":\"CUA fixture\",\"bounds\":{\"x\":40,\"y\":60,\"width\":320,\"height\":200},\"isOnScreen\":true,\"onCurrentSpace\":true}],\"truncated\":false}"}],"isError":false,"id":"{{message:105}}"}},"sourceEventSeqs":[264],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":51}} {"type":"step/start","data":{"turn":1,"step":52}} -{"type":"assistant/message","data":{"turn":1,"step":52,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-debug-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:106}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690821,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690821,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-debug-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1791947690821,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690821,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":52,"callId":"cua-debug-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} -{"type":"tool/result","data":{"turn":1,"step":52,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-debug-screenshot"},"toolCallId":"cua-debug-screenshot","content":[{"type":"text","text":"Error: NATIVE_PROTOCOL: Helper returned an invalid response; acquire fresh application handles\nScreenshot diagnostics:\n[cua-screenshot] {\"requestId\":\"cua-debug-screenshot\",\"stage\":\"capture-image.begin\",\"elapsedMs\":1}"}],"isError":true,"id":"{{message:107}}"}},"sourceEventSeqs":[269],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":52,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-debug-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F12\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:106}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733361,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733361,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-debug-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F12\"}}"}}},{"type":"chunk","time":1793949733361,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733361,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":52,"callId":"cua-debug-arm","name":"cua","arguments":"{\"method\":\"pressKey\",\"params\":{\"appHandle\":\"fixture-app\",\"key\":\"F12\"}}"}} +{"type":"tool/result","data":{"turn":1,"step":52,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-debug-arm"},"toolCallId":"cua-debug-arm","content":[{"type":"text","text":"{\"dispatched\":true}"}],"isError":false,"id":"{{message:107}}"}},"sourceEventSeqs":[269],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":52}} {"type":"step/start","data":{"turn":1,"step":53}} -{"type":"assistant/message","data":{"turn":1,"step":53,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-consent-denied","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Denied Fixture\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:108}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947690852,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947690852,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-consent-denied","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Denied Fixture\"}}"}}},{"type":"chunk","time":1791947690852,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947690852,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":53,"callId":"cua-consent-denied","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Denied Fixture\"}}"}} -{"type":"tool/result","data":{"turn":1,"step":53,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-consent-denied"},"toolCallId":"cua-consent-denied","content":[{"type":"text","text":"Error: APP_ACCESS_DENIED: The user denied access to Denied Fixture. Do not retry or use another tool to bypass this decision."}],"isError":true,"id":"{{message:109}}"}},"sourceEventSeqs":[274],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":53,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-debug-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:108}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733379,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733379,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-debug-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}}},{"type":"chunk","time":1793949733379,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733379,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":53,"callId":"cua-debug-screenshot","name":"cua","arguments":"{\"method\":\"screenshot\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7}}"}} +{"type":"tool/result","data":{"turn":1,"step":53,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-debug-screenshot"},"toolCallId":"cua-debug-screenshot","content":[{"type":"text","text":"Error: NATIVE_PROTOCOL: Helper returned an invalid response; acquire fresh application handles\nScreenshot diagnostics:\n[cua-screenshot] {\"requestId\":\"cua-debug-screenshot\",\"stage\":\"capture-image.begin\",\"elapsedMs\":1}"}],"isError":true,"id":"{{message:109}}"}},"sourceEventSeqs":[274],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":53}} {"type":"step/start","data":{"turn":1,"step":54}} -{"type":"assistant/message","data":{"turn":1,"step":54,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-consent-revocable","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Revocable Fixture\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:110}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947691022,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947691022,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-consent-revocable","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Revocable Fixture\"}}"}}},{"type":"chunk","time":1791947691022,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947691022,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":54,"callId":"cua-consent-revocable","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Revocable Fixture\"}}"}} -{"type":"tool/result","data":{"turn":1,"step":54,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-consent-revocable"},"toolCallId":"cua-consent-revocable","content":[{"type":"text","text":"{\"appHandle\":\"revoked-app\",\"name\":\"Revocable Fixture\",\"running\":true}"}],"isError":false,"id":"{{message:111}}"}},"sourceEventSeqs":[279],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":54,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-consent-denied","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Denied Fixture\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:110}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733403,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733403,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-consent-denied","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Denied Fixture\"}}"}}},{"type":"chunk","time":1793949733403,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733403,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":54,"callId":"cua-consent-denied","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Denied Fixture\"}}"}} +{"type":"tool/result","data":{"turn":1,"step":54,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-consent-denied"},"toolCallId":"cua-consent-denied","content":[{"type":"text","text":"Error: APP_ACCESS_DENIED: The user denied access to Denied Fixture. Do not retry or use another tool to bypass this decision."}],"isError":true,"id":"{{message:111}}"}},"sourceEventSeqs":[279],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":54}} {"type":"step/start","data":{"turn":1,"step":55}} -{"type":"assistant/message","data":{"turn":1,"step":55,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-consent-revoked","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"revoked-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:112}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947691192,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1791947691192,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-consent-revoked","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"revoked-app\"}}"}}},{"type":"chunk","time":1791947691192,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947691192,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} -{"type":"tool/call","data":{"turn":1,"step":55,"callId":"cua-consent-revoked","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"revoked-app\"}}"}} -{"type":"tool/result","data":{"turn":1,"step":55,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-consent-revoked"},"toolCallId":"cua-consent-revoked","content":[{"type":"text","text":"Error: APP_ACCESS_REVOKED: Application access was revoked; stop using this application"}],"isError":true,"id":"{{message:113}}"}},"sourceEventSeqs":[284],"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":55,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-consent-revocable","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Revocable Fixture\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:112}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733577,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733577,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-consent-revocable","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Revocable Fixture\"}}"}}},{"type":"chunk","time":1793949733577,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733577,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":55,"callId":"cua-consent-revocable","name":"cua","arguments":"{\"method\":\"getApp\",\"params\":{\"selector\":\"Revocable Fixture\"}}"}} +{"type":"tool/result","data":{"turn":1,"step":55,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-consent-revocable"},"toolCallId":"cua-consent-revocable","content":[{"type":"text","text":"{\"appHandle\":\"revoked-app\",\"name\":\"Revocable Fixture\",\"running\":true}"}],"isError":false,"id":"{{message:113}}"}},"sourceEventSeqs":[284],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":55}} {"type":"step/start","data":{"turn":1,"step":56}} -{"type":"assistant/message","data":{"turn":1,"step":56,"message":{"role":"assistant","content":[{"type":"text","text":"Observed Count: 4 after hovering, observing and clicking the revealed control. Moving away dismissed the hover information. Geometry conflicts rejected capture and coordinate gestures; the advertised AX button action remained available. No automatic activation or input retry was requested."}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:114}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1791947691212,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"chunk","time":1791947691212,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"Observed Count: 4 after hovering, observing and clicking the revealed control. Moving away dismissed the hover information. Geometry conflicts rejected capture and coordinate gestures; the advertised AX button action remained available. No automatic activation or input retry was requested."}}},{"type":"chunk","time":1791947691212,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1791947691212,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":1,"step":56,"message":{"role":"assistant","content":[{"type":"tool-call","id":"cua-consent-revoked","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"revoked-app\"}}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:114}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733745,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}},{"type":"chunk","time":1793949733745,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"cua-consent-revoked","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"revoked-app\"}}"}}},{"type":"chunk","time":1793949733745,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733745,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}]},"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":56,"callId":"cua-consent-revoked","name":"cua","arguments":"{\"method\":\"getState\",\"params\":{\"appHandle\":\"revoked-app\"}}"}} +{"type":"tool/result","data":{"turn":1,"step":56,"message":{"role":"tool","source":{"kind":"tool","callId":"cua-consent-revoked"},"toolCallId":"cua-consent-revoked","content":[{"type":"text","text":"Error: APP_ACCESS_REVOKED: Application access was revoked; stop using this application"}],"isError":true,"id":"{{message:115}}"}},"sourceEventSeqs":[289],"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":56}} +{"type":"step/start","data":{"turn":1,"step":57}} +{"type":"assistant/message","data":{"turn":1,"step":57,"message":{"role":"assistant","content":[{"type":"text","text":"Observed Count: 4 after hovering, observing and clicking the revealed control. Moving away dismissed the hover information. Geometry conflicts rejected capture and coordinate gestures; the advertised AX button action remained available. No automatic activation or input retry was requested."}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:116}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1793949733763,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"chunk","time":1793949733763,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"Observed Count: 4 after hovering, observing and clicking the revealed control. Moving away dismissed the hover information. Geometry conflicts rejected capture and coordinate gestures; the advertised AX button action remained available. No automatic activation or input retry was requested."}}},{"type":"chunk","time":1793949733763,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1793949733763,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":57}} {"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} From 54387e9278a71fb5a441ed3ec5fa27dbeffb2ae9 Mon Sep 17 00:00:00 2001 From: yudshj Date: Fri, 6 Nov 2026 15:30:04 +0800 Subject: [PATCH 17/44] =?UTF-8?q?feat(open-in-app):=20=E6=9F=90=E7=B1=BB?= =?UTF-8?q?=E5=9E=8B=E7=AC=AC=E4=B8=80=E6=AC=A1=E5=8F=B3=E9=94=AE=E6=97=B6?= =?UTF-8?q?=E6=9C=80=E5=A4=9A=E7=AD=89=20300=20ms=20=E5=85=B3=E8=81=94?= =?UTF-8?q?=E5=BA=94=E7=94=A8=E5=86=8D=E5=BC=B9=E5=87=BA=E8=8F=9C=E5=8D=95?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A file type without a recorded association answer now holds the file menu until the background query settles, 300 ms pass, or the menu closes, so the application row usually opens already named instead of changing after the menu appears. Later openings of the type still show the recorded answer at once and refresh in the background. Open menus track the answer they last applied per verification, so answers for other types never re-resolve them. README documents the wait and its first-opening delay. --- .../client/ui-open-in-app/README.i18n.yaml | 12 ++-- packages/client/ui-open-in-app/README.md | 5 +- packages/client/ui-open-in-app/README.zh.md | 5 +- .../src/client/context-menus.tsx | 67 ++++++++++++++----- .../ui-open-in-app/src/client/open-path.ts | 8 ++- .../tests/context-menu-actions.client.spec.ts | 51 +++++++++++++- 6 files changed, 116 insertions(+), 32 deletions(-) diff --git a/packages/client/ui-open-in-app/README.i18n.yaml b/packages/client/ui-open-in-app/README.i18n.yaml index a38fe57e45..60b0cb137d 100644 --- a/packages/client/ui-open-in-app/README.i18n.yaml +++ b/packages/client/ui-open-in-app/README.i18n.yaml @@ -18,11 +18,11 @@ en: b149da9963eb9837 zh: 2426ea37fbd0f8b7 /deepseek-ai-dsh-client-ui-open-in-app/use-this-package/what-to-expect: - en: 53d5d74d5b1d9b66 - zh: bdd9806dca51e957 + en: c9022effccebd157 + zh: 54e76ec13b5c1029 /deepseek-ai-dsh-client-ui-open-in-app/understand-the-implementation: - en: c26d8937f5d38119 - zh: d353c2377b9747aa + en: 423b66c7f36f7c40 + zh: 9b891fb7c8a8d240 /deepseek-ai-dsh-client-ui-open-in-app/further-exploration: en: 720527877e452a97 zh: b2241c7a336a3eeb @@ -33,8 +33,8 @@ en: ca75c51c89c2c9b0 zh: c2241315dc0c89c9 /deepseek-ai-dsh-client-ui-open-in-app/known-limitations-and-deferred-work: - en: f206d2fefa52f81d - zh: 51025a6e19d01295 + en: 31af13d29ff76a1c + zh: 6a08bd80d71c6434 /deepseek-ai-dsh-client-ui-open-in-app/known-limitations-and-deferred-work/dev-note: en: 84f92568fc74adf1 zh: 286b47e6b5b1a556 diff --git a/packages/client/ui-open-in-app/README.md b/packages/client/ui-open-in-app/README.md index 5ab8a3894f..5664376801 100644 --- a/packages/client/ui-open-in-app/README.md +++ b/packages/client/ui-open-in-app/README.md @@ -43,7 +43,7 @@ Mounted controls with the same association reader and file share one query and i Workspace context menus reveal the captured folder through the shared directory application catalog. File-tree directories expose native reveal and Copy folder path; they do not offer file preview or save. -Context menus contribute file preview, native application opening, original-file reveal, and URL navigation and copying through [ui-context-menu](../ui-context-menu/README.md). Session submenus resolve the captured Session ID’s directory rather than the main view, and observe application, operation, Session-list, and locale changes. Native operations report through the feature’s global toast after menu dismissal or target changes, without a second registry toast. Native file actions resolve Session paths through authorized metadata, then name the application recorded for the file’s type without waiting for the Host. Until the Host first answers for that type, the row reads “Open in default application” and opens the file through its OS association. Each opening re-queries the Host in the background; a changed answer updates the open menu in place, and a failed query keeps the recorded answer. Missing files disable path actions; image reveal requires an actual Host file and always targets that original. [ui-file-actions](../ui-file-actions/README.md) owns copy, download, save-as, and their completion feedback, including revealing the saved destination from its toast. +Context menus contribute file preview, native application opening, original-file reveal, and URL navigation and copying through [ui-context-menu](../ui-context-menu/README.md). Session submenus resolve the captured Session ID’s directory rather than the main view, and observe application, operation, Session-list, and locale changes. Native operations report through the feature’s global toast after menu dismissal or target changes, without a second registry toast. Native file actions resolve Session paths through authorized metadata, then name the application recorded for the file’s type without waiting for the Host. A type’s first opening on the page waits up to 300 ms for the Host’s answer; if none arrives by then, the row reads “Open in default application” and opens the file through its OS association. Each opening re-queries the Host in the background; a changed answer updates the open menu in place, and a failed query keeps the recorded answer. Missing files disable path actions; image reveal requires an actual Host file and always targets that original. [ui-file-actions](../ui-file-actions/README.md) owns copy, download, save-as, and their completion feedback, including revealing the saved destination from its toast. File menus copy absolute paths without line numbers; URL copying preserves query and fragment. Image files also expose image copy and original download, alongside the existing file actions. Successful downloads and saves show the shared Toast; picker cancellation is silent. @@ -59,7 +59,7 @@ A file menu offers “Open in sidebar” and “Open in floating panel”, which [`session-open-with.ts`](src/client/session-open-with.ts) shares one resolver across the shared context-menu package’s `session` and `sessionHeader` targets; other contributions do not cross between those targets. The file-tree slot uses `OpenInAppAction` with its owner’s `absolutePath`; there is no independent Session-header opener slot. A page-lifetime [`controller`](src/client/controller.ts) owns availability, persisted choice, and launch POST state. Menu observers and injected component hooks read those same sources. Document-relative routes and payload types come from the Host’s browser-safe `@deepseek-ai/dsh-host-open-in-app/shared` subpath. Adapters resolve native and transport failures to named outcomes; skipped requests announce nothing. -The directory and file adapters supply application metadata and operations to [`OpenTargetButton`](src/client/OpenTargetButton.tsx), which owns menu ordering, default markers, icons, and sizing. Its injected reporter sends outcomes to one feature-owned `shell.overlay` toast, so switching Sessions or closing the initiating preview cannot discard the result. The file header and empty state share `FileOpenTarget`, while `OpenPathInjected.applications` queries `session.workspacePathApplications` through [`open-path.ts`](src/client/open-path.ts). The same controller keeps the page’s answers by association class, runs one background refresh at a time, which the next opening or plugin unload cancels, and publishes a new snapshot only when an answer changes. The file menu observes that snapshot; when its own file type’s answer changes, it re-resolves synchronously from the path its opening verified, so the context-menu registry keeps the other rows while the application row changes. Opening uses `session.openWorkspacePath`; the Host revalidates an explicitly selected handler before launch. The directory adapter keeps the existing catalog routes. `FileRouteAction` supplies the same control to delivery cards and change review through `deliverables.file.actions` and `deliverables.review.file.actions`; their authenticated routes retain Session file authorization. A failed or unavailable file query therefore needs no platform-specific UI implementation. +The directory and file adapters supply application metadata and operations to [`OpenTargetButton`](src/client/OpenTargetButton.tsx), which owns menu ordering, default markers, icons, and sizing. Its injected reporter sends outcomes to one feature-owned `shell.overlay` toast, so switching Sessions or closing the initiating preview cannot discard the result. The file header and empty state share `FileOpenTarget`, while `OpenPathInjected.applications` queries `session.workspacePathApplications` through [`open-path.ts`](src/client/open-path.ts). The same controller keeps the page’s answers by association class, runs one background refresh at a time, which the next opening or plugin unload cancels, and publishes a new snapshot only when an answer changes. The file menu observes that snapshot; when its own file type’s answer changes, it re-resolves synchronously from the path its opening verified, so the context-menu registry keeps the other rows while the application row changes. A type without a recorded answer holds the file contribution until its refresh settles, 300 ms pass, or the menu closes; the registry opens the menu once every contribution is ready, so the whole menu waits. Opening uses `session.openWorkspacePath`; the Host revalidates an explicitly selected handler before launch. The directory adapter keeps the existing catalog routes. `FileRouteAction` supplies the same control to delivery cards and change review through `deliverables.file.actions` and `deliverables.review.file.actions`; their authenticated routes retain Session file authorization. A failed or unavailable file query therefore needs no platform-specific UI implementation. @@ -94,6 +94,7 @@ The Host verifies the path through the composed filesystem before opening or rev - **The dictionaries gate the menu.** A host catalog extension without a matching `app.` entry in both dictionaries stays invisible instead of showing a raw id; extending the catalog means extending [`dsh-host-open-in-app`](../../host/open-in-app/README.md) and this package's locales together. - **Availability is read once per page.** An application installed while the page is open appears after a reload (and, host-side, after a host restart); the desktop answer behind the file controls is read once per page as well. - **File associations are cached by type for the page.** Files sharing a final name extension share one answer; extensions that differ only in case, such as `.PNG` and `.png`, are separate types. A per-document override, such as a macOS “Open with” choice for one file, can therefore show the type’s previous answer until the background query corrects the open menu. Linux content sniffing and multi-part extensions such as `.tar.gz` have the same effect, and one file’s empty answer shows the row disabled for the type’s other files the same way. Choosing the row before the correction opens the application that answer named. Names without an extension are recorded per path. Records, with one icon per application, grow with the file types seen and end with the page. +- **A type’s first file menu can open late.** The whole menu waits up to 300 ms for that type’s first answer on the page, and a slower answer still changes the row after opening. - **One background refresh at a time.** Opening another file menu cancels the previous menu’s unanswered query, so that type keeps its earlier answer, or none, until its next opening. - **One reveal label for every platform.** The Session Remote reports whether a desktop exists, not which file manager it runs, so the menu says "Show file location" rather than naming Finder or File Explorer as the delivery cards do. - **The delivery cards keep their own opener.** [`ui-deliverables`](../ui-deliverables/README.md) still opens declared files through its own Session-and-event routes; folding those cards onto the file controls here remains deferred. diff --git a/packages/client/ui-open-in-app/README.zh.md b/packages/client/ui-open-in-app/README.zh.md index dd65ffdb58..69598cbc60 100644 --- a/packages/client/ui-open-in-app/README.zh.md +++ b/packages/client/ui-open-in-app/README.zh.md @@ -43,7 +43,7 @@ kind: "package-reference" 工作区右键菜单通过共享的目录应用列表定位被点击的文件夹。文件树目录提供原生定位与复制文件夹路径，不提供文件预览或保存。 -右键菜单通过 [ui-context-menu](../ui-context-menu/README.zh.md) 贡献文件预览、原生应用打开、原文件定位以及链接导航与复制。会话子菜单解析被捕获会话 ID 的目录，不使用主视图，并观察应用、操作、会话列表及语言变化。原生操作在菜单关闭或目标切换后仍通过本功能的全局 toast 报告，不产生第二个注册表提示。原生文件操作先通过授权元数据解析会话路径，然后直接显示该文件类型已记录的应用，不等待 Host。Host 首次回答该类型之前，这一项显示“使用默认应用打开”，通过系统的文件关联打开文件。每次打开菜单都会在后台重新查询 Host；回答有变化时，已打开的菜单原地更新，查询失败则保留已记录的回答。文件不存在时禁用路径操作；图片定位要求实际 Host 文件，并始终针对原文件。[ui-file-actions](../ui-file-actions/README.zh.md) 负责复制、下载、另存为及完成提示，包括通过提示定位保存目标。 +右键菜单通过 [ui-context-menu](../ui-context-menu/README.zh.md) 贡献文件预览、原生应用打开、原文件定位以及链接导航与复制。会话子菜单解析被捕获会话 ID 的目录，不使用主视图，并观察应用、操作、会话列表及语言变化。原生操作在菜单关闭或目标切换后仍通过本功能的全局 toast 报告，不产生第二个注册表提示。原生文件操作先通过授权元数据解析会话路径，然后直接显示该文件类型已记录的应用，不等待 Host。某个类型在本页面第一次打开菜单时，最多等待 Host 回答 300 毫秒；到时仍未回答，这一项显示“使用默认应用打开”，通过系统的文件关联打开文件。每次打开菜单都会在后台重新查询 Host；回答有变化时，已打开的菜单原地更新，查询失败则保留已记录的回答。文件不存在时禁用路径操作；图片定位要求实际 Host 文件，并始终针对原文件。[ui-file-actions](../ui-file-actions/README.zh.md) 负责复制、下载、另存为及完成提示，包括通过提示定位保存目标。 文件菜单复制不含行号的绝对路径；网址复制保留查询参数与锚点。图片文件在既有文件操作之外，还提供复制图片与下载原图。下载和保存成功后显示共享 Toast，取消选择器时不提示。 @@ -59,7 +59,7 @@ kind: "package-reference" [`session-open-with.ts`](src/client/session-open-with.ts) 为共享右键菜单包声明的 `session` 和 `sessionHeader` 目标共用一个解析器；其他贡献不会在这两种目标间扩散。文件树插槽通过所属组件的 `absolutePath` 使用 `OpenInAppAction`；会话头部没有独立打开按钮插槽。页面生命周期的 [`controller`](src/client/controller.ts) 持有可用性、持久化偏好与启动 POST 状态。菜单观察器和组件注入 hooks 读取同一组源。文档相对路由与载荷类型来自 Host 的浏览器安全子路径 `@deepseek-ai/dsh-host-open-in-app/shared`。适配器将原生操作及传输失败解析为具名结果；跳过的请求不播报。 -目录和文件适配器把应用信息与操作交给 [`OpenTargetButton`](src/client/OpenTargetButton.tsx)，由它统一管理菜单顺序、默认标记、图标和尺寸。注入的回调将结果交给本功能在 `shell.overlay` 中持有的唯一 toast，因此切换会话或关闭发起操作的预览不会丢失结果。文件标题栏和空态共用 `FileOpenTarget`，`OpenPathInjected.applications` 通过 [`open-path.ts`](src/client/open-path.ts) 查询 `session.workspacePathApplications`。同一控制器按关联类别保存本页面的回答，同一时间只运行一次后台刷新（下一次打开菜单或插件卸载会取消尚未完成的刷新），只在回答变化时发布新快照。文件菜单观察该快照；只有本文件类型的回答变化时，才基于本次打开时验证的路径同步重新解析，因此右键菜单注册表保留其他各项，只有应用这一项原地变化。打开操作使用 `session.openWorkspacePath`，Host 在启动前重新验证指定的关联应用。`FileRouteAction` 通过 `deliverables.file.actions` 和 `deliverables.review.file.actions` 为交付卡片和变更对比页提供同一控件，其认证路由保留会话文件校验。目录适配器继续使用已有的应用列表路由，文件查询失败或不可用时无需增加平台专用的界面实现。 +目录和文件适配器把应用信息与操作交给 [`OpenTargetButton`](src/client/OpenTargetButton.tsx)，由它统一管理菜单顺序、默认标记、图标和尺寸。注入的回调将结果交给本功能在 `shell.overlay` 中持有的唯一 toast，因此切换会话或关闭发起操作的预览不会丢失结果。文件标题栏和空态共用 `FileOpenTarget`，`OpenPathInjected.applications` 通过 [`open-path.ts`](src/client/open-path.ts) 查询 `session.workspacePathApplications`。同一控制器按关联类别保存本页面的回答，同一时间只运行一次后台刷新（下一次打开菜单或插件卸载会取消尚未完成的刷新），只在回答变化时发布新快照。文件菜单观察该快照；只有本文件类型的回答变化时，才基于本次打开时验证的路径同步重新解析，因此右键菜单注册表保留其他各项，只有应用这一项原地变化。尚无记录的类型会让文件菜单项等到这次刷新结束、满 300 毫秒或菜单关闭；注册表在所有菜单项都就绪后才打开菜单，所以整个菜单都会等待。打开操作使用 `session.openWorkspacePath`，Host 在启动前重新验证指定的关联应用。`FileRouteAction` 通过 `deliverables.file.actions` 和 `deliverables.review.file.actions` 为交付卡片和变更对比页提供同一控件，其认证路由保留会话文件校验。目录适配器继续使用已有的应用列表路由，文件查询失败或不可用时无需增加平台专用的界面实现。 @@ -94,6 +94,7 @@ Host 在打开或定位前通过当前文件系统验证路径。没有对应 Ho - **词典把守菜单。** 主机目录的新条目若在两份词典中没有对应的 `app.` 条目，将保持不可见而不是显示裸 id；扩展目录意味着同时扩展 [`dsh-host-open-in-app`](../../host/open-in-app/README.zh.md) 与本包的 locale。 - **可用性每页只读一次。** 页面打开期间安装的应用要重新加载页面后才出现（主机侧还需主机重启）；文件控件背后的桌面回答同样每页只读一次。 - **文件关联按类型在页面内缓存。** 最后一个扩展名相同的文件共用一个回答；只有大小写不同的扩展名（如 `.PNG` 和 `.png`）算作不同类型。因此单个文件的单独设置（例如 macOS 中只针对某个文件的“打开方式”）可能先显示该类型之前的回答，再由后台查询更正已打开的菜单。Linux 的内容嗅探和 `.tar.gz` 这类多段扩展名也有同样的效果；某个文件的空回答也会以同样的方式，让同类型其他文件的这一项先显示为禁用。在更正之前选择这一项，会用该回答中的应用打开。没有扩展名的文件按路径记录。记录以及每个应用的一份图标随出现过的文件类型增长，并在页面关闭时释放。 +- **某个类型的第一次文件菜单可能打开得较慢。** 整个菜单最多等待该类型在本页面的第一个回答 300 毫秒；回答更慢时，菜单打开后这一项仍会原地变化。 - **同一时间只有一次后台刷新。** 打开另一个文件的菜单会取消前一个菜单尚未返回的查询，该类型保留之前的回答（或仍没有回答），直到下次打开。 - **所有平台共用一个定位标签。** Session Remote 只报告有没有桌面，不报告它跑的是哪个文件管理器，所以菜单写「显示文件位置」，而不像交付卡片那样点名访达或文件资源管理器。 - **交付卡片保留自己的打开器。** [`ui-deliverables`](../ui-deliverables/README.zh.md) 仍通过自己按 Session 与事件定位的路由打开声明过的文件；把这些卡片并到这里的文件控件上仍属延期工作。 diff --git a/packages/client/ui-open-in-app/src/client/context-menus.tsx b/packages/client/ui-open-in-app/src/client/context-menus.tsx index 76e518ae3d..aa69108391 100644 --- a/packages/client/ui-open-in-app/src/client/context-menus.tsx +++ b/packages/client/ui-open-in-app/src/client/context-menus.tsx @@ -9,12 +9,38 @@ import type {} from '@deepseek-ai/dsh-client-ui-sidebar-right/client' import type {} from '@deepseek-ai/dsh-api-workspace-files/remote' import type {} from '@deepseek-ai/dsh-client-ui-sidebar-browser/client' import type { SessionId } from '@deepseek-ai/dsh-session/types' +import type { SessionWorkspacePathApplication } from '@deepseek-ai/dsh-api-session-controller/types' import type { OpenInAppController } from './controller.ts' import { fileAddressFor, resolveWorkspacePath } from '@deepseek-ai/dsh-util-workspace-path' import type { OpenInAppPathController } from './open-path.ts' import type { OpenInAppKey } from './locales.ts' import type { OpenInAppToastSource } from './open-toast.tsx' +type Applications = readonly SessionWorkspacePathApplication[] +/** A file menu's verified Host path and the association answer its resolution showed. */ +interface Verification { readonly absolutePath: string; readonly shown: Applications | undefined } + +/** + * How long a file menu waits for the first association answer of its file type before opening with the + * unnamed default-application row; most first answers arrive sooner, so the row rarely changes after opening. + */ +const FIRST_ANSWER_WAIT_MS = 300 + +/** + * @param pending - settles without rejecting. + * @param ms - longest wait. + * @param signal - ends the wait early. + * @returns settles when `pending` settles, after `ms`, or on abort, whichever comes first; the timer never outlives it. + */ +function settleWithin(pending: Promise, ms: number, signal: AbortSignal): Promise { + return new Promise((settle) => { + const done = (): void => { clearTimeout(timer); signal.removeEventListener('abort', done); settle() } + const timer = setTimeout(done, ms) + signal.addEventListener('abort', done, { once: true }) + void pending.then(done) + }) +} + /** * Register native opening, original-path reveal, and URL actions. * @param ctx - declared plugin dependencies. @@ -109,10 +135,10 @@ export function registerContextMenus( } }, ] } - // The Host path from each file target's latest successful stat, shared by menus open on one target object at once. - // A stat clears it while running; answers arriving meanwhile need no re-resolution, as that stat's resolution reads - // the newest answers. - const verified = new WeakMap() + // The Host path from each file target's latest successful stat and the answer its menu showed, shared by menus open + // on one target object at once. A stat clears it while running; answers arriving meanwhile need no re-resolution, as + // that stat's resolution reads the newest answers. + const verified = new WeakMap() // The target whose changed association answer is being applied; only that re-resolution reuses the verified path, // so every opening runs its own stat. The in-place update relies on the registry re-resolving synchronously from // the notification, without which the menu stats again, and on a synchronous resolution replacing rows without @@ -123,32 +149,39 @@ export function registerContextMenus( observe: target => [{ getSnapshot: paths.associations.getSnapshot.bind(paths.associations), subscribe: (listener: () => void) => { - // Answers for other file types leave this menu unchanged. - let shown: ReturnType + // The answer this subscription last applied for a verification; answers for other file types leave it unchanged. + let applied: { readonly verification: Verification; readonly applications: Applications | undefined } | undefined return paths.associations.subscribe(() => { - const absolutePath = verified.get(target) - if (absolutePath === undefined) return - const applications = paths.cachedApplications(absolutePath) - if (applications === shown) return - shown = applications + const verification = verified.get(target) + if (verification === undefined) return + const applications = paths.cachedApplications(verification.absolutePath) + const current = applied?.verification === verification ? applied.applications : verification.shown + if (applications === current) return + applied = { verification, applications } answering = target try { listener() } finally { answering = undefined } }) }, }], resolve: (target, signal) => { - const verifiedPath = answering === target ? verified.get(target) : undefined - if (verifiedPath !== undefined) return fileMenu(target, verifiedPath) + const reused = answering === target ? verified.get(target) : undefined + if (reused !== undefined) return fileMenu(target, reused.absolutePath) verified.delete(target) return (async () => { await paths.load() const absolutePath = await resolvePath(target.sessionId, target.path, signal) signal.throwIfAborted() - if (absolutePath !== undefined) { - verified.set(target, absolutePath) - // A cached answer shows at once; every opening re-queries so a changed association reaches this menu. - if (paths.desktop.getSnapshot() === true) paths.refreshApplications(absolutePath) + if (absolutePath === undefined) return fileMenu(target, undefined) + if (paths.desktop.getSnapshot() === true) { + // Every opening re-queries so a changed association reaches this menu; a type's first opening waits for it briefly. + const unanswered = paths.cachedApplications(absolutePath) === undefined + const refreshed = paths.refreshApplications(absolutePath) + if (unanswered) { + await settleWithin(refreshed, FIRST_ANSWER_WAIT_MS, signal) + signal.throwIfAborted() + } } + verified.set(target, { absolutePath, shown: paths.cachedApplications(absolutePath) }) return fileMenu(target, absolutePath) })() }, diff --git a/packages/client/ui-open-in-app/src/client/open-path.ts b/packages/client/ui-open-in-app/src/client/open-path.ts index ba17321fca..85ea703b5d 100644 --- a/packages/client/ui-open-in-app/src/client/open-path.ts +++ b/packages/client/ui-open-in-app/src/client/open-path.ts @@ -139,13 +139,15 @@ export class OpenInAppPathController { * Re-query a file's associations in the background. One refresh runs at a time: it outlives the * requesting menu and ends with its answer, the next refresh, which cancels it, or {@link dispose}. * @param path - absolute Host path of the file. + * @returns settles without rejecting when this refresh's query settles, after any answer is recorded; + * already settled after {@link dispose}. */ - refreshApplications(path: string): void { - if (this.lifetime.signal.aborted) return + refreshApplications(path: string): Promise { + if (this.lifetime.signal.aborted) return Promise.resolve() this.refresh?.abort() const refresh = new AbortController() this.refresh = refresh - void this.applications(path, AbortSignal.any([this.lifetime.signal, refresh.signal])) + return this.applications(path, AbortSignal.any([this.lifetime.signal, refresh.signal])).then(() => {}) } /** Cancel background refreshes and refuse new ones; recorded answers stay readable. */ diff --git a/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts b/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts index 85a92ccac3..4a0e8963ae 100644 --- a/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts +++ b/packages/client/ui-open-in-app/tests/context-menu-actions.client.spec.ts @@ -122,7 +122,7 @@ it('routes file actions through an explicit host, the line, native association a const simple = { kind: 'file', value: { sessionId: SessionId('absent'), path: '/file.unknown' } } as const h.remote.session.workspacePathApplications.mockResolvedValueOnce({ ok: true, value: [] }) const menu = await h.resolve(simple) - // Without an answer the row opens through the association; the Host's empty answer then disables it in place. + // The Host's empty answer arrives within the first-answer wait, so the row opens disabled. await vi.waitFor(() => { expect(menu.source.getSnapshot().find(item => item.id.endsWith('/open'))?.disabled).toBe(true) }) expect(menu.source.getSnapshot().some(item => item.id.endsWith('/copyImage'))).toBe(false) expect(menu.source.getSnapshot().filter(item => item.id.startsWith('file-actions.')).map(item => item.id)) @@ -136,7 +136,47 @@ it('routes file actions through an explicit host, the line, native association a expect(h.sidebar.openResourceIn).toHaveBeenLastCalledWith(SessionId('absent'), expect.any(String), { destination: 'float' }) }) -it('opens file menus before the Host answers and changes only the application row in place', async () => { +it('names the application when the first answer of a file type arrives within the wait', async () => { + const h = await boot() + // The answer arrives a macrotask later, after a resolution that did not wait would have published its rows. + h.remote.session.workspacePathApplications.mockImplementationOnce(() => new Promise((answer) => { + setTimeout(() => { answer({ ok: true, value: [{ id: 'preview', name: 'Preview', default: true, icon: null }] }) }, 10) + })) + const menu = h.registry.resolve({ kind: 'file', value: { sessionId, path: 'notes.txt' } }) + const labels: (string | undefined)[] = [] + const stop = menu.source.subscribe(() => { + labels.push(menu.source.getSnapshot().find(item => item.id === 'open-in-app.file/open')?.label) + }) + await menu.ready + stop() + expect(labels.filter(label => label !== undefined)).toEqual(['open.title']) + menu.dispose() +}) + +it('waits for the first answer of a file type at most 300 ms, and a dismissal ends the wait', async () => { + vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] }) + onTestFinished(() => { vi.useRealTimers() }) + const h = await boot() + h.remote.session.workspacePathApplications.mockImplementation(() => new Promise(() => {})) + const menu = h.registry.resolve({ kind: 'file', value: { sessionId, path: 'notes.txt' } }) + let ready = false + void menu.ready.then(() => { ready = true }) + await vi.advanceTimersByTimeAsync(299) + expect(ready).toBe(false) + await vi.advanceTimersByTimeAsync(1) + expect(ready).toBe(true) + expect(menu.source.getSnapshot().find(item => item.id === 'open-in-app.file/open')).toMatchObject({ label: 'menu.defaultApp', disabled: false }) + menu.dispose() + const dismissed = h.registry.resolve({ kind: 'file', value: { sessionId, path: 'todo.md' } }) + await vi.advanceTimersByTimeAsync(100) + expect(vi.getTimerCount()).toBe(1) + dismissed.dispose() + await dismissed.ready + expect(vi.getTimerCount()).toBe(0) + expect(h.failure).not.toHaveBeenCalled() +}) + +it('opens file menus after the first-answer wait and changes only the application row in place', async () => { const h = await boot() const answers: ((answer: RemoteResult) => void)[] = [] h.remote.session.workspacePathApplications.mockImplementation(() => new Promise((answer) => { answers.push(answer) })) @@ -228,6 +268,13 @@ it('re-resolves an open file menu only when its own file type answers differentl await h.paths.applications('/workspace/other.md', new AbortController().signal) expect(h.paths.cachedApplications('/workspace/other.md')).toHaveLength(1) expect(menu.source.getSnapshot()).toBe(shown) + h.remote.session.workspacePathApplications.mockResolvedValueOnce({ ok: true, value: [] }) + await h.paths.applications('/workspace/todo.txt', new AbortController().signal) + const changed = menu.source.getSnapshot() + expect(changed.find(item => item.id === 'open-in-app.file/open')).toMatchObject({ label: 'menu.defaultApp', disabled: true }) + h.remote.session.workspacePathApplications.mockResolvedValueOnce({ ok: true, value: [] }) + await h.paths.applications('/workspace/other.md', new AbortController().signal) + expect(menu.source.getSnapshot()).toBe(changed) menu.dispose() }) From 0c2fbb031dd6b1e96a1b4ad971567342db34c85a Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 15:31:55 +0800 Subject: [PATCH 18/44] Capture the selected macOS window without its attached window group --- .../cua-macos/native/Sources/CuaNative/WindowCapture.swift | 2 ++ 1 file changed, 2 insertions(+) diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/WindowCapture.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/WindowCapture.swift index a61960d2ea..0d167f4cab 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/WindowCapture.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/WindowCapture.swift @@ -115,6 +115,8 @@ final class WindowCapture: @unchecked Sendable { configuration.width = pixelsWide configuration.height = pixelsHigh configuration.showsCursor = false + // Attached sheets can otherwise capture the entire document window scaled to the sheet's dimensions. + if #available(macOS 14.2, *) { configuration.includeChildWindows = false } configuration.ignoreShadowsSingleWindow = true configuration.ignoreGlobalClipSingleWindow = true trace?.mark("capture-config.end") From c07035b4ef6211be5ebf4529d701eb3936f33d23 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 15:34:32 +0800 Subject: [PATCH 19/44] Check the rendered file panel button against its AX bounds --- .../experimental/cua-macos/README.i18n.yaml | 4 ++-- packages/experimental/cua-macos/README.md | 2 +- packages/experimental/cua-macos/README.zh.md | 2 +- .../native/tests/CheckScreenshotLabel.swift | 19 +++++++++++++++++++ .../cua-macos/native/tests/e2e.py | 9 +++++++++ 5 files changed, 32 insertions(+), 4 deletions(-) create mode 100644 packages/experimental/cua-macos/native/tests/CheckScreenshotLabel.swift diff --git a/packages/experimental/cua-macos/README.i18n.yaml b/packages/experimental/cua-macos/README.i18n.yaml index 37458210d5..0d83c6e538 100644 --- a/packages/experimental/cua-macos/README.i18n.yaml +++ b/packages/experimental/cua-macos/README.i18n.yaml @@ -18,8 +18,8 @@ en: 533ed4dd11af5126 zh: 03036e8c949d6aca /deepseek-ai-dsh-experimental-cua-macos/use-this-package/connect-an-external-execution-environment: - en: 0da6d36bdf117089 - zh: 0d3692ab25ac2923 + en: b734a0907747580e + zh: 8110c5951b70e100 /deepseek-ai-dsh-experimental-cua-macos/understand-the-implementation: en: 4e82b85a5aa72eee zh: 4e004b7a205d8688 diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index ce128bab4f..742e218285 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -123,7 +123,7 @@ Use the `cua` API to discover and select the application (`listApps`, `getState` Screenshot capture requires an attachment store and a resolved model route declaring image input. The shared attachment admission validates canonical Base64, the raster and its configured capacity, then stores the image once. Native data remains available to programmatic callers; model content carries coordinate metadata and the durable image reference. Response byte limits apply before parsing. -Use `win.click(id)` to send mouse events at the center of an element, or `win.click([x, y])` to click a point. For both `click(id)` and `mouseMove(id)`, the element must belong to the selected window. Prefer `invoke(id)` for a background control’s default action. For mouse input, activate the application and observe the target first. Mouse clicks on system Open and Save sheets are routed to their verified panel-service window. +Use `win.click(id)` to send mouse events at the center of an element, or `win.click([x, y])` to click a point. For both `click(id)` and `mouseMove(id)`, the element must belong to the selected window. Prefer `invoke(id)` for a background control’s default action. For mouse input, activate the application and observe the target first. Mouse clicks on system Open and Save sheets are routed to their verified panel-service window. On macOS 14.2 and later, a window screenshot excludes its attached window group so the image uses the selected window’s bounds. Absolute coordinates for `mouseMove`, `click`, and `drag` use `[x, y]` tuples; `scroll` uses named `x` and `y`. All use pixels in the latest returned screenshot. Pass `relative: true` for integer thousandths of window width and height, from 0 to 999; omission or `false` uses pixels. Relative input can use a successful AX read or screenshot, while absolute input requires a screenshot. The Host maps the delivered image to native points, including attachment and model-request resizing. A value of 999 remains inside the window edge. Coordinate options do not apply to element IDs. A width or height change since the latest observation returns `WINDOW_CHANGED` before input, including raw coordinate calls without a snapshot ID; read fresh AX or take a new screenshot before proceeding. Window movement alone preserves window-local coordinates. Capture and pointer input reject known disagreement between the selected window’s accessibility frame and displayed bounds with `WINDOW_CHANGED`, before posting pointer events. This includes Stage Manager transformations when AX retains the full window frame. Read fresh state; any activation or window raise remains explicit. Advertised AX actions remain available. Missing AX geometry retains the AXless path and does not establish that a transformed window can be operated. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 2801e0ccbc..276baf2131 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -123,7 +123,7 @@ void await window.getScreenshot() 截图要求附件服务可用，并确认当前模型支持图片输入。共享附件接入校验规范 Base64、图像数据及配置容量，然后保存一次图片。程序调用方仍可取得原生数据，模型内容包含坐标元数据和持久图片引用。响应字节上限在解析前执行。 -`win.click(id)` 向元素中心发送鼠标事件，`win.click([x, y])` 向给定位置发送鼠标事件。`click(id)` 和 `mouseMove(id)` 使用的元素须属于所选窗口。后台控件优先用 `invoke(id)` 执行默认动作。需要鼠标输入时，先激活应用并重新观察目标。系统打开或保存面板的鼠标点击投递给已核实的文件面板服务窗口。 +`win.click(id)` 向元素中心发送鼠标事件，`win.click([x, y])` 向给定位置发送鼠标事件。`click(id)` 和 `mouseMove(id)` 使用的元素须属于所选窗口。后台控件优先用 `invoke(id)` 执行默认动作。需要鼠标输入时，先激活应用并重新观察目标。系统打开或保存面板的鼠标点击投递给已核实的文件面板服务窗口。在 macOS 14.2 及以上版本，截图只包含所选窗口，图像按该窗口的边界对齐。 `mouseMove`、`click` 和 `drag` 的绝对坐标使用 `[x, y]` 元组；`scroll` 使用命名的 `x` 和 `y`。它们都使用最近返回截图中的像素。传入 `relative: true` 时使用 0～999 的千分比整数；省略或传入 `false` 时使用像素。相对坐标可基于成功的 AX 读取或截图，绝对坐标必须先截图。Host 根据实际送入模型的图片尺寸换算原生 point，包含附件归一化和模型请求阶段的缩放。999 映射到窗口边缘以内。坐标选项不适用于元素 ID。窗口宽高与最近一次观察不同时，在输入前返回 `WINDOW_CHANGED`，即使原始坐标调用不传快照 ID 也会检查；重新读取 AX 或截图后再操作。单纯移动窗口不使窗口内坐标失效。截图和鼠标输入会核对所选窗口的辅助功能框与显示边界；已知两者不一致时，在发送任何鼠标事件前返回 `WINDOW_CHANGED`。当 AX 保留完整窗口框时，这也会拦住 Stage Manager 变换后的窗口。需要重新读取状态；激活应用或抬起窗口仍须显式调用。元素声明的 AX 动作保持可用。缺少 AX 几何时保留无 AX 路径，不代表变换后的窗口一定可以操作。 diff --git a/packages/experimental/cua-macos/native/tests/CheckScreenshotLabel.swift b/packages/experimental/cua-macos/native/tests/CheckScreenshotLabel.swift new file mode 100644 index 0000000000..4b6b566c0f --- /dev/null +++ b/packages/experimental/cua-macos/native/tests/CheckScreenshotLabel.swift @@ -0,0 +1,19 @@ +// Verify a screenshot against an AX control using text recognized from the captured pixels. +import Foundation +import Vision + +let args = Array(CommandLine.arguments.dropFirst()) +guard args.count == 6, let x = Double(args[2]), let y = Double(args[3]), + let width = Double(args[4]), let height = Double(args[5]) else { + fatalError("Expected PNG path, label and normalized top-left AX bounds") +} +let expected = CGRect(x: x, y: y, width: width, height: height) +let request = VNRecognizeTextRequest() +request.recognitionLevel = .accurate +request.automaticallyDetectsLanguage = true +try VNImageRequestHandler(url: URL(fileURLWithPath: args[0]), options: [:]).perform([request]) +let matches = (request.results ?? []).filter { $0.topCandidates(1).first?.string.caseInsensitiveCompare(args[1]) == .orderedSame } +guard matches.contains(where: { expected.contains(CGPoint(x: $0.boundingBox.midX, y: 1 - $0.boundingBox.midY)) }) else { + fputs("The screenshot does not show the control label inside its AX bounds.\n", stderr) + exit(1) +} diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index 03ae9ac2ac..8697b6d781 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -85,6 +85,15 @@ def check_file_panel(call, owner, primary_params, primary, directory, wait_state assert f"window {sheet_id}; select that window" in error["message"], error ready = call("getState", {**owner, "windowId": sheet_id, "maxElements": 2000, "maxDepth": 25}) open_button = next(e for e in ready["elements"] if e["identifier"] == "OKButton" and e["enabled"]) + image = call("screenshot", {**owner, "windowId": sheet_id}) + png = Path(directory) / "open-sheet.png" + png.write_bytes(base64.b64decode(image["data"], validate=True)) + x, y, width, height = open_button["bounds"] + frame = image["bounds"] + expected = ((x - frame["x"]) / frame["width"], (y - frame["y"]) / frame["height"], + width / frame["width"], height / frame["height"]) + subprocess.run(["swift", str(ROOT / "tests/CheckScreenshotLabel.swift"), str(png), + open_button["title"], *map(str, expected)], check=True) assert call("click", {**owner, "windowId": sheet_id, "snapshotId": ready["snapshotId"], "elementId": open_button["elementId"]})["dispatched"] wait_state(lambda state: bool(state["selectedFile"]) and Path(state["selectedFile"]).samefile(selected_file), "File picker did not return the intended file") From 344e8a91a58ab8cda5d5f7debff27de6169b2d6a Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 15:35:37 +0800 Subject: [PATCH 20/44] Resolve element hover before checking geometry for pointer delivery --- .../native/Sources/CuaNative/AXHover.swift | 6 ++-- .../native/Sources/CuaNative/AXWorker.swift | 35 ++++++++++++------- 2 files changed, 25 insertions(+), 16 deletions(-) diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift index 656aa97a07..ecab67f6d7 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift @@ -11,7 +11,7 @@ final class AXHover { private var revealed: (element: AXUIElement, windowID: UInt32)? /// Entering revealed descendants preserves their parent UI; leaving restores it once. - func move(to target: AXUIElement?, in route: WindowRoute, access: AXAccess) throws -> Bool { + func move(to target: AXUIElement?, in windowID: UInt32, access: AXAccess) throws -> Bool { var current = target var visited: [AXUIElement] = [] var alternate: AXUIElement? @@ -20,7 +20,7 @@ final class AXHover { throw NativeFailure(code: "NATIVE_ERROR", message: "Accessibility hover ancestry contains a cycle; read fresh state") } visited.append(node) - if let revealed, revealed.windowID == route.windowID, CFEqual(revealed.element, node) { return true } + if let revealed, revealed.windowID == windowID, CFEqual(revealed.element, node) { return true } let actions = try access.actions(node) if alternate == nil, actions.contains("AXShowAlternateUI"), actions.contains("AXShowDefaultUI"), try access.boolean(node, "AXEnabled") != false { alternate = node } @@ -35,7 +35,7 @@ final class AXHover { } if let alternate { try access.perform(alternate, "AXShowAlternateUI") - revealed = (alternate, route.windowID) + revealed = (alternate, windowID) return true } return leaving diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift index f00e63b622..73e8e78fd9 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift @@ -104,6 +104,14 @@ actor AXWorker { access = created return created } + private func moveUsingAX(_ element: AXUIElement?, windowID: UInt32, handle: String, params: RequestParameters) throws -> Bool { + let allowSystemPointer = try params.optionalBoolean("allowSystemPointer") ?? false + let hover = hovers[handle] ?? AXHover() + hovers[handle] = hover + if try hover.move(to: element, in: windowID, access: ax()) { return true } + try AXHover.requireSystemPointer(allowed: allowSystemPointer) + return false + } private func inputs() -> (KeyboardInput, WindowPointerInput) { if let keyboard, let pointer { return (keyboard, pointer) } let poster = PIDEventPoster() @@ -348,7 +356,6 @@ actor AXWorker { let modifiers = try params.modifiers() let window: WindowInfo let point: WindowPoint - var hoverElement: AXUIElement? var elementWindow: AXUIElement? if params.contains("elementId") { guard !params.contains("x"), !params.contains("y") else { throw RequestParameters.invalid("Element and coordinate targets cannot be combined") } @@ -359,9 +366,15 @@ actor AXWorker { if let requestedWindow, enclosing.id != requestedWindow { throw NativeFailure(code: "STALE_SNAPSHOT", message: "Element belongs to window \(enclosing.id); select that window and read its tree") } - if method == "click" { try ax().requireEnabled(selected) } + if method == "mouseMove" { + _ = try NativePlatform.ownedWindow(identity.pid, enclosing.id) + if try moveUsingAX(selected, windowID: enclosing.id, handle: handle, params: params) { + return .result(Self.mouseMoveResult(route: "accessibility", pointerMoved: false)) + } + } else { + try ax().requireEnabled(selected) + } elementWindow = enclosing.element - hoverElement = selected guard let bounds = try ax().bounds(selected), bounds.width > 0, bounds.height > 0 else { throw NativeFailure(code: "NATIVE_ERROR", message: "Element has no pointer bounds") } window = try WindowGeometry.resolve(owner: identity, windowID: enclosing.id) { try ax().bounds(enclosing.element) } point = WindowPoint(x: bounds.x + bounds.width / 2 - window.bounds.x, y: bounds.y + bounds.height / 2 - window.bounds.y) @@ -373,18 +386,14 @@ actor AXWorker { } let route = WindowRoute(owner: identity, windowID: window.windowID, bounds: window.bounds) if method == "mouseMove" { - let allowSystemPointer = try params.optionalBoolean("allowSystemPointer") ?? false try WindowPointerInput.validate(point, in: route) - if hoverElement == nil { - hoverElement = try ax().element(at: CGPoint(x: window.bounds.x + point.x, y: window.bounds.y + point.y), owner: identity) - if let hit = hoverElement, try ax().windowID(hit) != window.windowID { hoverElement = nil } + if elementWindow == nil { + var hit = try ax().element(at: CGPoint(x: window.bounds.x + point.x, y: window.bounds.y + point.y), owner: identity) + if let target = hit, try ax().windowID(target) != window.windowID { hit = nil } + if try moveUsingAX(hit, windowID: window.windowID, handle: handle, params: params) { + return .result(Self.mouseMoveResult(route: "accessibility", pointerMoved: false)) + } } - let hover = hovers[handle] ?? AXHover() - hovers[handle] = hover - if try hover.move(to: hoverElement, in: route, access: ax()) { - return .result(Self.mouseMoveResult(route: "accessibility", pointerMoved: false)) - } - try AXHover.requireSystemPointer(allowed: allowSystemPointer) return .mouseMove(route, point) } var clickRoute = route From eba9e768e65e9c578d71b1663227afb076b605ec Mon Sep 17 00:00:00 2001 From: Turtle Date: Fri, 6 Nov 2026 15:39:00 +0800 Subject: [PATCH 21/44] fix(ui-goal): order goal actions edit, clear, pause and tighten icon gaps Refs #8024 Co-authored-by: Yifffan <133629972+Yifffan@users.noreply.github.com> --- .../expected/goal-bar/active.expected.md | 2 +- .../expected/goal-bar/blocked.expected.md | 2 +- .../expected/goal-bar/inactive.expected.md | 2 +- .../src/client/queue/QueueDock.module.css | 2 +- packages/client/ui-goal/README.i18n.yaml | 4 +-- packages/client/ui-goal/README.md | 2 +- packages/client/ui-goal/README.zh.md | 2 +- .../ui-goal/src/client/GoalBar.module.css | 2 +- .../client/ui-goal/src/client/GoalBar.tsx | 34 +++++++++---------- .../ui-goal/tests/goalbar.client.spec.tsx | 6 ++-- .../web/queue-actions/goal-stop.expected.md | 2 +- .../web/queue-actions/layout.expected.md | 2 +- 12 files changed, 32 insertions(+), 30 deletions(-) diff --git a/apps/web/tests/expected/goal-bar/active.expected.md b/apps/web/tests/expected/goal-bar/active.expected.md index cf3227ee51..423676465d 100644 --- a/apps/web/tests/expected/goal-bar/active.expected.md +++ b/apps/web/tests/expected/goal-bar/active.expected.md @@ -1,4 +1,4 @@ - text: Ongoing Goal guard rapid clear clicks -- button "Pause goal" - button "Edit goal" - button "Clear goal" +- button "Pause goal" diff --git a/apps/web/tests/expected/goal-bar/blocked.expected.md b/apps/web/tests/expected/goal-bar/blocked.expected.md index dbfcf5563b..b61a46559d 100644 --- a/apps/web/tests/expected/goal-bar/blocked.expected.md +++ b/apps/web/tests/expected/goal-bar/blocked.expected.md @@ -1,4 +1,4 @@ - text: Blocked Goal retry with the edited plan -- button "Resume goal" - button "Edit goal" - button "Clear goal" +- button "Resume goal" diff --git a/apps/web/tests/expected/goal-bar/inactive.expected.md b/apps/web/tests/expected/goal-bar/inactive.expected.md index 729dbd1dd9..64258bf74b 100644 --- a/apps/web/tests/expected/goal-bar/inactive.expected.md +++ b/apps/web/tests/expected/goal-bar/inactive.expected.md @@ -1,4 +1,4 @@ - text: Inactive Goal guard rapid clear clicks -- button "Resume goal" - button "Edit goal" - button "Clear goal" +- button "Resume goal" diff --git a/packages/client/ui-conversation/src/client/queue/QueueDock.module.css b/packages/client/ui-conversation/src/client/queue/QueueDock.module.css index d80240c6b2..1783694789 100644 --- a/packages/client/ui-conversation/src/client/queue/QueueDock.module.css +++ b/packages/client/ui-conversation/src/client/queue/QueueDock.module.css @@ -206,7 +206,7 @@ display: flex; flex: none; align-items: center; - gap: 10px; + gap: 8px; } .status { diff --git a/packages/client/ui-goal/README.i18n.yaml b/packages/client/ui-goal/README.i18n.yaml index 815b11e50f..00694a134b 100644 --- a/packages/client/ui-goal/README.i18n.yaml +++ b/packages/client/ui-goal/README.i18n.yaml @@ -15,8 +15,8 @@ en: d152484eb41ac6b4 zh: 09388d293f9be9cb /deepseek-ai-dsh-client-ui-goal/use-this-package: - en: 7f6620a84a087d07 - zh: 80071126cc64b3e5 + en: a70a68194ca0c799 + zh: 3830bcebfa1879a9 /deepseek-ai-dsh-client-ui-goal/use-this-package/the-command-input-bubble: en: 05f0360b6c6dd3e9 zh: d14c3f15dc4cea5b diff --git a/packages/client/ui-goal/README.md b/packages/client/ui-goal/README.md index 986e5c6811..bbada420cd 100644 --- a/packages/client/ui-goal/README.md +++ b/packages/client/ui-goal/README.md @@ -25,7 +25,7 @@ The Web GUI goal surface shows both the durable goal state and its current proce ## Use this package -Mount this plugin alongside `ui-conversation` and the goal domain package; the strip then appears as the second card in the composer-context stack (after Todo, before Queue) whenever the session has a goal. Todo and Goal use the same panel elevation above the composer. An armed active goal offers pause; an active-but-disarmed, paused, or blocked goal offers resume; edit rewrites the objective in the ui-primitives `InlineEditor`, so multi-line objectives survive editing; clear removes the goal and suppresses the strip until the projection catches up. +Mount this plugin alongside `ui-conversation` and the goal domain package; the strip then appears as the second card in the composer-context stack (after Todo, before Queue) whenever the session has a goal. Todo and Goal use the same panel elevation above the composer. The strip orders its actions edit, clear, then pause or resume, matching the Queue rows' edit and remove positions. An armed active goal offers pause; an active-but-disarmed, paused, or blocked goal offers resume; edit rewrites the objective in the ui-primitives `InlineEditor`, so multi-line objectives survive editing; clear removes the goal and suppresses the strip until the projection catches up. ### The command-input bubble diff --git a/packages/client/ui-goal/README.zh.md b/packages/client/ui-goal/README.zh.md index 36e47153d9..9cece977c8 100644 --- a/packages/client/ui-goal/README.zh.md +++ b/packages/client/ui-goal/README.zh.md @@ -25,7 +25,7 @@ Web GUI 的 goal 界面同时显示持久 goal 状态及当前的进程本地激 ## 使用本包 -与 `ui-conversation` 及 goal 领域包一起挂载本插件；只要会话存在目标，条带就会作为 composer 上下文堆栈的第二张卡片出现（位于 Todo 之后、Queue 之前）。Todo 与 Goal 使用相同的面板 elevation，使其位于 composer 层级之上。已 armed 的 active goal 提供暂停动作；active-but-disarmed、paused 或 blocked 的 goal 提供恢复；编辑在 ui-primitives 的 `InlineEditor` 中重写目标文本，多行目标编辑后仍保留换行；清除移除目标，并在投影追上之前抑制条带。 +与 `ui-conversation` 及 goal 领域包一起挂载本插件；只要会话存在目标，条带就会作为 composer 上下文堆栈的第二张卡片出现（位于 Todo 之后、Queue 之前）。Todo 与 Goal 使用相同的面板 elevation，使其位于 composer 层级之上。条带的动作按编辑、清除、暂停或恢复的顺序排列，与 Queue 行的编辑和移除位置一致。已 armed 的 active goal 提供暂停动作；active-but-disarmed、paused 或 blocked 的 goal 提供恢复；编辑在 ui-primitives 的 `InlineEditor` 中重写目标文本，多行目标编辑后仍保留换行；清除移除目标，并在投影追上之前抑制条带。 ### 指令输入气泡 diff --git a/packages/client/ui-goal/src/client/GoalBar.module.css b/packages/client/ui-goal/src/client/GoalBar.module.css index 798cd0d9f6..9d7c99716c 100644 --- a/packages/client/ui-goal/src/client/GoalBar.module.css +++ b/packages/client/ui-goal/src/client/GoalBar.module.css @@ -100,7 +100,7 @@ .actions { display: flex; align-items: center; - gap: 10px; + gap: 8px; flex: none; } diff --git a/packages/client/ui-goal/src/client/GoalBar.tsx b/packages/client/ui-goal/src/client/GoalBar.tsx index 120538c3c4..025bf51b8a 100644 --- a/packages/client/ui-goal/src/client/GoalBar.tsx +++ b/packages/client/ui-goal/src/client/GoalBar.tsx @@ -1,9 +1,9 @@ /** * GoalBar: the goal indicator docked above the message composer (input dock * strip). A present goal shows a goal glyph, a phase label, the truncated - * objective, and icon actions — resume when active-disarmed, paused, or - * blocked, edit (inline form in the same strip), and clear. Goal creation - * lives on the `/goal` command, not here: loading (undefined), no goal + * objective, and icon actions — edit (inline form in the same strip), clear, + * then pause when armed or resume when active-disarmed, paused, or blocked. + * Goal creation lives on the `/goal` command, not here: loading (undefined), no goal * (null), and complete goals render nothing. Durable state arrives as the projected whole snapshot; * process-local activation arrives through the injected activation hook. */ @@ -137,20 +137,6 @@ export function GoalBar({ goal, activation, onEdit, onPause, onResume, onClear, {goal.objective} {actionError !== null && {actionError}}
- {goal.phase === 'active' && activation === 'armed' && ( - - - - )} - {showResume && ( - - - - )} + {goal.phase === 'active' && activation === 'armed' && ( + + + + )} + {showResume && ( + + + + )}
diff --git a/packages/client/ui-goal/tests/goalbar.client.spec.tsx b/packages/client/ui-goal/tests/goalbar.client.spec.tsx index cc9dda1c67..32947a6b9d 100644 --- a/packages/client/ui-goal/tests/goalbar.client.spec.tsx +++ b/packages/client/ui-goal/tests/goalbar.client.spec.tsx @@ -146,9 +146,10 @@ describe('GoalBar', () => { expect(screen.getByRole('textbox', { name: '目标内容' })).toBeTruthy() }) - it('active goal: the pause action pauses', () => { + it('active goal: the pause action follows edit and clear, and pauses', () => { const actions = makeActions() render() + expect(screen.getAllByRole('button').map(b => b.getAttribute('aria-label'))).toEqual(['编辑目标', '清除目标', '暂停目标']) fireEvent.click(screen.getByRole('button', { name: '暂停目标' })) expect(actions.onPause).toHaveBeenCalledTimes(1) }) @@ -163,10 +164,11 @@ describe('GoalBar', () => { expect(screen.queryByRole('button', { name: '暂停目标' })).toBeNull() }) - it('paused goal: "已暂停的目标" with a resume action before edit', () => { + it('paused goal: "已暂停的目标" with a resume action after edit and clear', () => { const actions = makeActions() render() expect(screen.getByText('已暂停的目标')).toBeTruthy() + expect(screen.getAllByRole('button').map(b => b.getAttribute('aria-label'))).toEqual(['编辑目标', '清除目标', '恢复目标']) fireEvent.click(screen.getByRole('button', { name: '恢复目标' })) expect(actions.onResume).toHaveBeenCalledTimes(1) }) diff --git a/snapshots/web/queue-actions/goal-stop.expected.md b/snapshots/web/queue-actions/goal-stop.expected.md index 92a11037e0..19018fd989 100644 --- a/snapshots/web/queue-actions/goal-stop.expected.md +++ b/snapshots/web/queue-actions/goal-stop.expected.md @@ -45,9 +45,9 @@ - button "Bad response" - button "Branch into a new conversation" - text: {{clock}} Inactive Goal Keep working after Stop -- button "Resume goal" - button "Edit goal" - button "Clear goal" +- button "Resume goal" - textbox "Message or run a task, / commands, @ files or sessions" - button "Add files or run commands" - 'button "Access mode, current: Workspace Write"': Workspace Write diff --git a/snapshots/web/queue-actions/layout.expected.md b/snapshots/web/queue-actions/layout.expected.md index b79770e640..9c4352daf7 100644 --- a/snapshots/web/queue-actions/layout.expected.md +++ b/snapshots/web/queue-actions/layout.expected.md @@ -21,9 +21,9 @@ - region "To-dos": - button "To-dos 1 completed · 1 in progress" - text: Ongoing Goal Keep the composer context panels aligned -- button "Pause goal" - button "Edit goal" - button "Clear goal" +- button "Pause goal" - button "2 queued messages" - textbox "Cmd/Ctrl+Enter steers all queued messages" - button "Add files or run commands" From 0d6c614164c2f0835b5fba5af0243e2a923900f0 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 15:40:54 +0800 Subject: [PATCH 22/44] Establish the file panel keyboard target and compare hover state across refusal --- packages/experimental/cua-macos/native/tests/e2e.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index 8697b6d781..4e30375a81 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -18,6 +18,7 @@ from native_test_support import ROOT, NativeHelper, line, run_cleanup, stop def check_file_panel(call, owner, primary_params, primary, directory, wait_state, button_target): + call("activate", owner) selected_file = Path(directory) / "sheet-target.txt" selected_file.write_text("Owned file picker fixture\n") assert call("invoke", button_target(call("getState", primary_params), "Open fixture file"))["dispatched"] @@ -232,7 +233,7 @@ def main(): denied = call("mouseMove", no_hover, "HOVER_UNAVAILABLE") assert "systemPointerMoved" not in denied and "No input was sent" in denied["message"], denied after = wait_state(lambda s: s["cursorSamples"] > before["cursorSamples"], "Cursor sampling stopped") - assert after["cursorChanges"] == before["cursorChanges"] and not after["hoverVisible"], after + assert (after["cursorChanges"], after["hoverVisible"]) == (before["cursorChanges"], before["hoverVisible"]), after def mouse_move(target): result = call("mouseMove", target if args.alternate_hover else {**target, "allowSystemPointer": True}) From 1fd66fd75f3dcaf03c92b712fe1dd288628db6cb Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 15:43:06 +0800 Subject: [PATCH 23/44] Measure refused hover in the cursor-stability fixture --- .../experimental/cua-macos/native/tests/e2e.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/packages/experimental/cua-macos/native/tests/e2e.py b/packages/experimental/cua-macos/native/tests/e2e.py index 4e30375a81..2a16b54b49 100644 --- a/packages/experimental/cua-macos/native/tests/e2e.py +++ b/packages/experimental/cua-macos/native/tests/e2e.py @@ -226,14 +226,14 @@ def main(): window_element = next(element for element in current["elements"] if element["role"] == "AXWindow") call("performAction", {**owner, "snapshotId": current["snapshotId"], "elementId": window_element["elementId"], "action": "AXRaise"}) - call("getState", primary_params) current = call("getState", primary_params) - no_hover = {**button_target(current, "Increment primary"), "windowId": primary["windowId"]} - before = json.loads(state_path.read_text()) - denied = call("mouseMove", no_hover, "HOVER_UNAVAILABLE") - assert "systemPointerMoved" not in denied and "No input was sent" in denied["message"], denied - after = wait_state(lambda s: s["cursorSamples"] > before["cursorSamples"], "Cursor sampling stopped") - assert (after["cursorChanges"], after["hoverVisible"]) == (before["cursorChanges"], before["hoverVisible"]), after + if args.alternate_hover: + no_hover = {**button_target(current, "Increment primary"), "windowId": primary["windowId"]} + before = json.loads(state_path.read_text()) + denied = call("mouseMove", no_hover, "HOVER_UNAVAILABLE") + assert "systemPointerMoved" not in denied and "No input was sent" in denied["message"], denied + after = wait_state(lambda s: s["cursorSamples"] > before["cursorSamples"], "Cursor sampling stopped") + assert (after["cursorChanges"], after["hoverVisible"]) == (before["cursorChanges"], before["hoverVisible"]), after def mouse_move(target): result = call("mouseMove", target if args.alternate_hover else {**target, "allowSystemPointer": True}) From a8fb5eb9fa2e64b97ed474098c0275836cec6899 Mon Sep 17 00:00:00 2001 From: Turtle Date: Fri, 6 Nov 2026 16:07:17 +0800 Subject: [PATCH 24/44] fix(ui-goal): keep goal action positions while activation loads Co-authored-by: Yifffan <133629972+Yifffan@users.noreply.github.com> --- apps/web/tests/goal-bar.e2e.ts | 8 ++--- packages/client/ui-goal/README.i18n.yaml | 4 +-- packages/client/ui-goal/README.md | 2 +- packages/client/ui-goal/README.zh.md | 2 +- .../client/ui-goal/src/client/GoalBar.tsx | 32 +++++++++---------- .../ui-goal/tests/goalbar.client.spec.tsx | 7 ++++ 6 files changed, 31 insertions(+), 24 deletions(-) diff --git a/apps/web/tests/goal-bar.e2e.ts b/apps/web/tests/goal-bar.e2e.ts index 132d44569b..348a7520a6 100644 --- a/apps/web/tests/goal-bar.e2e.ts +++ b/apps/web/tests/goal-bar.e2e.ts @@ -53,9 +53,9 @@ describe('web e2e: goal bar clear convergence', () => { const bar = page.locator('[data-goal-bar]') await bar.waitFor({ timeout: 10_000 }) const pause = bar.getByRole('button', { name: 'Pause goal' }) - await expect.poll(() => pause.count(), { + await expect.poll(() => pause.isEnabled(), { timeout: 10_000, - }).toBe(1) + }).toBe(true) await pause.hover() const pauseTooltip = page.getByRole('tooltip', { name: 'Pause goal', exact: true }) await pauseTooltip.waitFor() @@ -120,7 +120,7 @@ describe('web e2e: goal bar clear convergence', () => { await input.fill('/goal retry after a blocker') await input.press('Enter') const bar = page.locator('[data-goal-bar]') - await bar.getByRole('button', { name: 'Pause goal' }).waitFor({ timeout: 10_000 }) + await expect.poll(() => bar.getByRole('button', { name: 'Pause goal' }).isEnabled(), { timeout: 10_000 }).toBe(true) const agents = scaffold.ctx.agents.list() expect(agents).toHaveLength(1) @@ -140,7 +140,7 @@ describe('web e2e: goal bar clear convergence', () => { await compareOrRefreshGolden(BLOCKED_EXPECTED, blocked, MODE) await bar.getByRole('button', { name: 'Resume goal' }).click() - await bar.getByRole('button', { name: 'Pause goal' }).waitFor({ timeout: 10_000 }) + await expect.poll(() => bar.getByRole('button', { name: 'Pause goal' }).isEnabled(), { timeout: 10_000 }).toBe(true) expect(scaffold.ctx.goals.get(agent)).toMatchObject({ phase: 'active', activation: 'armed', objective: 'retry with the edited plan', }) diff --git a/packages/client/ui-goal/README.i18n.yaml b/packages/client/ui-goal/README.i18n.yaml index 00694a134b..809bbef816 100644 --- a/packages/client/ui-goal/README.i18n.yaml +++ b/packages/client/ui-goal/README.i18n.yaml @@ -15,8 +15,8 @@ en: d152484eb41ac6b4 zh: 09388d293f9be9cb /deepseek-ai-dsh-client-ui-goal/use-this-package: - en: a70a68194ca0c799 - zh: 3830bcebfa1879a9 + en: 796ed31dda615e5a + zh: b797d7bd3d3f34bc /deepseek-ai-dsh-client-ui-goal/use-this-package/the-command-input-bubble: en: 05f0360b6c6dd3e9 zh: d14c3f15dc4cea5b diff --git a/packages/client/ui-goal/README.md b/packages/client/ui-goal/README.md index bbada420cd..cadad96470 100644 --- a/packages/client/ui-goal/README.md +++ b/packages/client/ui-goal/README.md @@ -25,7 +25,7 @@ The Web GUI goal surface shows both the durable goal state and its current proce ## Use this package -Mount this plugin alongside `ui-conversation` and the goal domain package; the strip then appears as the second card in the composer-context stack (after Todo, before Queue) whenever the session has a goal. Todo and Goal use the same panel elevation above the composer. The strip orders its actions edit, clear, then pause or resume, matching the Queue rows' edit and remove positions. An armed active goal offers pause; an active-but-disarmed, paused, or blocked goal offers resume; edit rewrites the objective in the ui-primitives `InlineEditor`, so multi-line objectives survive editing; clear removes the goal and suppresses the strip until the projection catches up. +Mount this plugin alongside `ui-conversation` and the goal domain package; the strip then appears as the second card in the composer-context stack (after Todo, before Queue) whenever the session has a goal. Todo and Goal use the same panel elevation above the composer. The strip orders its actions edit, clear, then pause or resume, matching the Queue rows' edit and remove positions. An armed active goal offers pause, shown disabled until its activation is known; an active-but-disarmed, paused, or blocked goal offers resume; edit rewrites the objective in the ui-primitives `InlineEditor`, so multi-line objectives survive editing; clear removes the goal and suppresses the strip until the projection catches up. ### The command-input bubble diff --git a/packages/client/ui-goal/README.zh.md b/packages/client/ui-goal/README.zh.md index 9cece977c8..9d96d3fa0b 100644 --- a/packages/client/ui-goal/README.zh.md +++ b/packages/client/ui-goal/README.zh.md @@ -25,7 +25,7 @@ Web GUI 的 goal 界面同时显示持久 goal 状态及当前的进程本地激 ## 使用本包 -与 `ui-conversation` 及 goal 领域包一起挂载本插件；只要会话存在目标，条带就会作为 composer 上下文堆栈的第二张卡片出现（位于 Todo 之后、Queue 之前）。Todo 与 Goal 使用相同的面板 elevation，使其位于 composer 层级之上。条带的动作按编辑、清除、暂停或恢复的顺序排列，与 Queue 行的编辑和移除位置一致。已 armed 的 active goal 提供暂停动作；active-but-disarmed、paused 或 blocked 的 goal 提供恢复；编辑在 ui-primitives 的 `InlineEditor` 中重写目标文本，多行目标编辑后仍保留换行；清除移除目标，并在投影追上之前抑制条带。 +与 `ui-conversation` 及 goal 领域包一起挂载本插件；只要会话存在目标，条带就会作为 composer 上下文堆栈的第二张卡片出现（位于 Todo 之后、Queue 之前）。Todo 与 Goal 使用相同的面板 elevation，使其位于 composer 层级之上。条带的动作按编辑、清除、暂停或恢复的顺序排列，与 Queue 行的编辑和移除位置一致。已 armed 的 active goal 提供暂停动作，激活状态未知时暂停按钮显示为禁用；active-but-disarmed、paused 或 blocked 的 goal 提供恢复；编辑在 ui-primitives 的 `InlineEditor` 中重写目标文本，多行目标编辑后仍保留换行；清除移除目标，并在投影追上之前抑制条带。 ### 指令输入气泡 diff --git a/packages/client/ui-goal/src/client/GoalBar.tsx b/packages/client/ui-goal/src/client/GoalBar.tsx index 025bf51b8a..c6bb31711b 100644 --- a/packages/client/ui-goal/src/client/GoalBar.tsx +++ b/packages/client/ui-goal/src/client/GoalBar.tsx @@ -2,7 +2,8 @@ * GoalBar: the goal indicator docked above the message composer (input dock * strip). A present goal shows a goal glyph, a phase label, the truncated * objective, and icon actions — edit (inline form in the same strip), clear, - * then pause when armed or resume when active-disarmed, paused, or blocked. + * then pause when armed (disabled until activation is known) or resume when + * active-disarmed, paused, or blocked. * Goal creation lives on the `/goal` command, not here: loading (undefined), no goal * (null), and complete goals render nothing. Durable state arrives as the projected whole snapshot; * process-local activation arrives through the injected activation hook. @@ -128,7 +129,8 @@ export function GoalBar({ goal, activation, onEdit, onPause, onResume, onClear, const title = goal.phase === 'blocked' ? goal.blockedReason?.message : undefined const label = goal.phase === 'active' ? activeLabel(activation, t) : t(PHASE_LABELS[goal.phase]) - const showResume = goal.phase !== 'active' || activation === 'disarmed' + const resume = goal.phase !== 'active' || activation === 'disarmed' + const toggleLabel = t(resume ? 'action.resume' : 'action.pause') return (
@@ -153,20 +155,18 @@ export function GoalBar({ goal, activation, onEdit, onPause, onResume, onClear, - {goal.phase === 'active' && activation === 'armed' && ( - - - - )} - {showResume && ( - - - - )} + {/* Until activation arrives, an active goal keeps a disabled pause so edit and clear stay in place. */} + + +
diff --git a/packages/client/ui-goal/tests/goalbar.client.spec.tsx b/packages/client/ui-goal/tests/goalbar.client.spec.tsx index 32947a6b9d..0016781fbb 100644 --- a/packages/client/ui-goal/tests/goalbar.client.spec.tsx +++ b/packages/client/ui-goal/tests/goalbar.client.spec.tsx @@ -154,6 +154,13 @@ describe('GoalBar', () => { expect(actions.onPause).toHaveBeenCalledTimes(1) }) + it('active goal with unknown activation: keeps a disabled pause after edit and clear', () => { + const actions = makeActions() + render() + expect(screen.getAllByRole('button').map(b => b.getAttribute('aria-label'))).toEqual(['编辑目标', '清除目标', '暂停目标']) + expect((screen.getByRole('button', { name: '暂停目标' }) as HTMLButtonElement).disabled).toBe(true) + }) + it('active disarmed goal: "待继续的目标" with a resume action instead of pause', () => { const actions = makeActions() render() From 94336c6b3ddcfca9c4562183978dd308414a198f Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Fri, 6 Nov 2026 16:15:24 +0800 Subject: [PATCH 25/44] fix(forge): pin change request diffs to the polled head and count every review thread ForgeProvider.diff now takes the head commit it must describe. The GitHub provider reads headRefOid before and after `gh pr diff` and returns the new ForgeFailure code `stale` when either read names another commit, so the forge panel never caches a patch under a head it does not describe. A stale read polls the change request at once, and the changes view waits for the next status before reading again. Status polling pages reviewThreads up to the new `maxReviewThreads` setting (default 1,000) instead of counting only the first 100, and sets ChangeRequestSnapshot.threadsTruncated when the count is a lower bound. --- apps/web/tests/forge-panel.e2e.ts | 14 +- docs/config-catalog.md | 4 +- docs/config-catalog.zh.md | 4 +- docs/subsystems/forge.i18n.yaml | 4 +- docs/subsystems/forge.md | 29 ++- docs/subsystems/forge.zh.md | 29 ++- .../forge-pinned-diff/guide.i18n.yaml | 16 ++ .../v0.2.2-rc.1/forge-pinned-diff/guide.md | 26 +++ .../v0.2.2-rc.1/forge-pinned-diff/guide.zh.md | 26 +++ .../client-ui-forge/README.i18n.yaml | 8 +- .../experimental/client-ui-forge/README.md | 6 +- .../experimental/client-ui-forge/README.zh.md | 6 +- .../src/client/ChangeRequestCard.tsx | 9 +- .../client-ui-forge/src/client/ChangesTab.tsx | 16 +- .../experimental/client-ui-forge/src/index.ts | 17 +- .../experimental/client-ui-forge/src/types.ts | 6 +- .../tests/change-request-card.client.spec.tsx | 2 + .../tests/changes-tab.client.spec.tsx | 36 ++++ .../tests/client-fixtures.client.ts | 1 + .../tests/controller.host.spec.ts | 33 +++- .../client-ui-forge/tests/harness.ts | 6 +- .../forge-github/README.i18n.yaml | 8 +- packages/experimental/forge-github/README.md | 10 +- .../experimental/forge-github/README.zh.md | 10 +- .../experimental/forge-github/src/graphql.ts | 175 ++++++++++++++---- .../experimental/forge-github/src/index.ts | 9 +- .../experimental/forge-github/src/provider.ts | 119 +++++++++--- .../forge-github/tests/fixtures.ts | 14 +- .../forge-github/tests/graphql.spec.ts | 133 +++++++++++-- .../forge-github/tests/plugin.spec.ts | 2 + .../forge-github/tests/provider.spec.ts | 125 +++++++++++-- .../forge-github/tests/review.spec.ts | 4 +- .../forge-session/tests/fixtures.ts | 1 + .../tests/loader-composition.spec.ts | 3 +- packages/experimental/forge/README.i18n.yaml | 4 +- packages/experimental/forge/README.md | 2 +- packages/experimental/forge/README.zh.md | 2 +- packages/experimental/forge/src/index.ts | 15 +- packages/experimental/forge/src/types.ts | 17 +- .../experimental/forge/tests/fake-provider.ts | 5 +- .../forge/tests/lifecycle.spec.ts | 4 +- .../experimental/forge/tests/registry.spec.ts | 6 +- .../experimental/forge/tests/watch.spec.ts | 21 +++ .../extensions/tool-cordis/src/api-catalog.ts | 16 +- 44 files changed, 812 insertions(+), 191 deletions(-) create mode 100644 docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.i18n.yaml create mode 100644 docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.md create mode 100644 docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.zh.md diff --git a/apps/web/tests/forge-panel.e2e.ts b/apps/web/tests/forge-panel.e2e.ts index 44f24a6a58..656f5b91e4 100644 --- a/apps/web/tests/forge-panel.e2e.ts +++ b/apps/web/tests/forge-panel.e2e.ts @@ -120,25 +120,31 @@ if (args[0] === 'api' && args[1] === 'graphql') { }) const pulls = { 7: { - number: 7, url: 'https://github.com/octo/demo/pull/7', title: 'Add the pull request panel', state: 'OPEN', isDraft: false, + id: 'pull-7', number: 7, url: 'https://github.com/octo/demo/pull/7', title: 'Add the pull request panel', state: 'OPEN', isDraft: false, updatedAt: '2026-10-09T00:00:00Z', closedAt: null, author: { login: 'octo' }, headRefName: 'feat', headRefOid: 'a'.repeat(40), baseRefName: 'main', additions: 128, deletions: 14, mergeable: 'MERGEABLE', mergeStateStatus: 'BLOCKED', reviewDecision: 'REVIEW_REQUIRED', - reviewThreads: { nodes: [{ isResolved: false }] }, + reviewThreads: { pageInfo: { hasNextPage: false, endCursor: null }, nodes: [{ isResolved: false }] }, commits: { nodes: [{ commit: { statusCheckRollup: { id: 'rollup-7', state: 'FAILURE', contexts: { pageInfo: { hasNextPage: true, endCursor: 'checks-100' }, nodes: Array.from({ length: 100 }, (_, i) => check('lint', 'FAILURE', '2026-10-08T00:00:42Z', i + 1)), } } } }] }, }, 9: { - number: 9, url: 'https://github.com/octo/demo/pull/9', title: 'Add the merge button', state: state.mergedAt === null ? 'OPEN' : 'MERGED', + id: 'pull-9', number: 9, url: 'https://github.com/octo/demo/pull/9', title: 'Add the merge button', state: state.mergedAt === null ? 'OPEN' : 'MERGED', isDraft: !state.ready, updatedAt: '2026-10-09T00:00:00Z', closedAt: state.mergedAt, author: { login: 'octo' }, headRefName: 'feat/merge', headRefOid: state.head ?? 'b'.repeat(40), baseRefName: 'main', additions: 30, deletions: 2, mergeable: 'MERGEABLE', - mergeStateStatus: state.ready ? 'CLEAN' : 'BLOCKED', reviewDecision: 'APPROVED', reviewThreads: { nodes: [] }, + mergeStateStatus: state.ready ? 'CLEAN' : 'BLOCKED', reviewDecision: 'APPROVED', + reviewThreads: { pageInfo: { hasNextPage: false, endCursor: null }, nodes: [] }, commits: { nodes: [{ commit: { statusCheckRollup: { id: 'rollup-9', state: 'SUCCESS', contexts: { pageInfo: { hasNextPage: false, endCursor: null }, nodes: [check('lint', 'SUCCESS', '2026-10-09T00:00:42Z')], } } } }] }, }, } + if (query.includes('pullRequest(number: $number)')) { + const number = Number(args.find(arg => arg.startsWith('number=')).slice('number='.length)) + process.stdout.write(JSON.stringify({ data: { repository: { pullRequest: { headRefOid: pulls[number].headRefOid } } } })) + process.exit(0) + } if (query.includes('node(id: $id)')) { process.stdout.write(JSON.stringify({ data: { node: { contexts: { pageInfo: { hasNextPage: false, endCursor: null }, diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 7562c8bf96..0e37afca2b 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1142,7 +1142,7 @@ export type SourceSigning = 'ad-hoc' | { ## `@deepseek-ai/dsh-experimental-forge` -- `source`: [`packages/experimental/forge/src/types.ts:232`](../packages/experimental/forge/src/types.ts) +- `source`: [`packages/experimental/forge/src/types.ts:239`](../packages/experimental/forge/src/types.ts) ```ts config-catalog /** Poll cadence and fetch deadline for watched change requests. */ @@ -1178,6 +1178,8 @@ export interface Config { timeoutMs?: number /** Maximum captured bytes per `gh` output stream and combined JSON bytes of one complete check read. */ maxOutputBytes?: number + /** Most review threads counted per pull request in one status read; a pull request with more reports a lower bound. */ + maxReviewThreads?: number } ``` diff --git a/docs/config-catalog.zh.md b/docs/config-catalog.zh.md index 0d75b6e523..ccf14207de 100644 --- a/docs/config-catalog.zh.md +++ b/docs/config-catalog.zh.md @@ -1144,7 +1144,7 @@ export type SourceSigning = 'ad-hoc' | { ## `@deepseek-ai/dsh-experimental-forge` -- `source`: [`packages/experimental/forge/src/types.ts:232`](../packages/experimental/forge/src/types.ts) +- `source`: [`packages/experimental/forge/src/types.ts:239`](../packages/experimental/forge/src/types.ts) ```ts config-catalog /** Poll cadence and fetch deadline for watched change requests. */ @@ -1180,6 +1180,8 @@ export interface Config { timeoutMs?: number /** Maximum captured bytes per `gh` output stream and combined JSON bytes of one complete check read. */ maxOutputBytes?: number + /** Most review threads counted per pull request in one status read; a pull request with more reports a lower bound. */ + maxReviewThreads?: number } ``` diff --git a/docs/subsystems/forge.i18n.yaml b/docs/subsystems/forge.i18n.yaml index 3f4994aa39..f60794a506 100644 --- a/docs/subsystems/forge.i18n.yaml +++ b/docs/subsystems/forge.i18n.yaml @@ -18,5 +18,5 @@ en: 13dfd36f2166bba0 zh: a369434f52f04d57 /forges/panel: - en: dbc28e731f357edf - zh: a1977fafc0651427 + en: 8182a3120b2d4aa3 + zh: 68b6631047cb0f48 diff --git a/docs/subsystems/forge.md b/docs/subsystems/forge.md index f36141ea1b..591e9d47f4 100644 --- a/docs/subsystems/forge.md +++ b/docs/subsystems/forge.md @@ -68,8 +68,12 @@ interface ForgeProvider { markReady(ref: ChangeRequestRef, signal: AbortSignal): Promise /** Merge a change request while its head is still `request.headSha`, resolving to undefined once merged. */ merge(ref: ChangeRequestRef, request: MergeRequest, signal: AbortSignal): Promise - /** Read the changes of a change request as the host computes them, as one unified patch with `a/` and `b/` path prefixes. */ - diff(ref: ChangeRequestRef, signal: AbortSignal): Promise + /** + * Read the changes of a change request at head commit `headSha` as the host computes them, as one unified + * patch with `a/` and `b/` path prefixes. A patch is returned only when it describes `headSha`; once the + * head is another commit, the read resolves to a `stale` failure. + */ + diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise /** Read the text of `path` at `commit` in the change request's repository. */ fileText(ref: ChangeRequestRef, commit: string, path: string, signal: AbortSignal): Promise /** Build the web page of `path` at `commit` in the change request's repository. */ @@ -176,8 +180,10 @@ interface ChangeRequestSnapshot { readonly checksRollup: ChecksRollup /** Whether the host reported more checks than `checks` contains. */ readonly checksTruncated: boolean - /** Number of review threads that are not resolved. */ + /** Number of counted review threads that are not resolved. */ readonly unresolvedThreads: number + /** Whether the change request has more review threads than the provider counted, so `unresolvedThreads` is a lower bound. */ + readonly threadsTruncated: boolean /** ISO 8601 time of the host's last update to the change request. */ readonly updatedAt: string /** ISO 8601 time the change request was merged or closed, or null while it is open. */ @@ -191,9 +197,10 @@ interface ForgeFailure { /** * `not-found`: the host does not expose it; `unauthenticated`: credentials are missing or rejected; * `too-large`: the host or the provider refuses an answer over its size limit, so reading again fails - * the same way; `unavailable`: any other failure. + * the same way; `stale`: the head is no longer the commit the read named, so reading again at the head of + * a later status can succeed; `unavailable`: any other failure. */ - readonly code: 'not-found' | 'unauthenticated' | 'too-large' | 'unavailable' + readonly code: 'not-found' | 'unauthenticated' | 'too-large' | 'stale' | 'unavailable' /** Human-readable cause. */ readonly message: string } @@ -568,7 +575,7 @@ type ForgeChangeRequestStatesFrame = | { readonly type: 'changed'; readonly states: readonly ForgeChangeRequestListState[] } ``` -The changes view reads through the same Remote. `localDiff` reads one `ForgeLocalScope` of the Session's checkout, and `changeRequestDiff` reads a bound change request at the head of its latest status through the provider's `diff`; both return files that the Host parsed into `ForgeDiffFile` hunks. `localText` and `changeRequestText` read the new side of one file, which supplies the unchanged lines between hunks. Each inspection also sizes the `branch` scope as `ForgePanelState.local` for the panel's local changes card, always against the automatic fork point. A `branch` scope whose `base` names a ref compares with the merge base of HEAD and that branch instead; `diffBases` lists the branches the view offers, with the automatic fork point. +The changes view reads through the same Remote. `localDiff` reads one `ForgeLocalScope` of the Session's checkout, and `changeRequestDiff` reads a bound change request at the head of its latest status through the provider's `diff`, which returns a patch only for that head commit; both return files that the Host parsed into `ForgeDiffFile` hunks. `localText` and `changeRequestText` read the new side of one file, which supplies the unchanged lines between hunks. Each inspection also sizes the `branch` scope as `ForgePanelState.local` for the panel's local changes card, always against the automatic fork point. A `branch` scope whose `base` names a ref compares with the merge base of HEAD and that branch instead; `diffBases` lists the branches the view offers, with the automatic fork point. When the head moved past the latest status, the provider returns `stale` and `ctx.forge.diff` polls the change request at once; `changeRequestDiff` rejects with `forgePanel/stale-head` and caches nothing, and the changes view keeps loading until a later status arrives, then reads again at that status's head. Review comments therefore always address the commit whose lines the view shows. ```ts type-equiv /** Which local changes a diff covers; each reads the Session's checkout through Git. */ @@ -814,13 +821,16 @@ async merge(ref: ChangeRequestRef, request: MergeRequest, signal: AbortSignal): async fetchChecks(ref: ChangeRequestRef, signal: AbortSignal): Promise /** - * Read the changes of a change request as its host computes them. + * Read the changes of a change request at one head commit as its host + * computes them. A `stale` failure polls the change request again at once + * when it is watched, so its next status names the current head. * @param ref - change request served by a registered provider. + * @param headSha - head commit the patch must describe, usually the head of the latest status. * @param signal - cancellation of the read. - * @returns one unified patch with `a/` and `b/` path prefixes, or why the host did not provide it. + * @returns one unified patch with `a/` and `b/` path prefixes, or why the host did not provide it at `headSha`. * @throws when no provider serves the change request's forge, or with `signal.reason` when it aborts. */ -async diff(ref: ChangeRequestRef, signal: AbortSignal): Promise +async diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise /** * Publish one comment or a completed review, then refresh the watched status. @@ -1015,6 +1025,7 @@ Serves the forge panel. Following a Session inspects the checkout of its current * @param signal - request cancellation. * @returns the files, the head commit, each file's web page, and the commits the checkout has beyond the head. * @throws `forgePanel/not-bound` unless the URL is a visible binding of the Session that a provider serves; + * `forgePanel/stale-head` when the head moved past the latest status, which is then polled again; * `forgePanel/diff-too-large` when the host or its provider refuses the diff over a size limit; * `forgePanel/diff-failed` before its status was read and when the host does not provide the diff otherwise. */ diff --git a/docs/subsystems/forge.zh.md b/docs/subsystems/forge.zh.md index 61c53fbd6c..e73f9a6713 100644 --- a/docs/subsystems/forge.zh.md +++ b/docs/subsystems/forge.zh.md @@ -68,8 +68,12 @@ interface ForgeProvider { markReady(ref: ChangeRequestRef, signal: AbortSignal): Promise /** Merge a change request while its head is still `request.headSha`, resolving to undefined once merged. */ merge(ref: ChangeRequestRef, request: MergeRequest, signal: AbortSignal): Promise - /** Read the changes of a change request as the host computes them, as one unified patch with `a/` and `b/` path prefixes. */ - diff(ref: ChangeRequestRef, signal: AbortSignal): Promise + /** + * Read the changes of a change request at head commit `headSha` as the host computes them, as one unified + * patch with `a/` and `b/` path prefixes. A patch is returned only when it describes `headSha`; once the + * head is another commit, the read resolves to a `stale` failure. + */ + diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise /** Read the text of `path` at `commit` in the change request's repository. */ fileText(ref: ChangeRequestRef, commit: string, path: string, signal: AbortSignal): Promise /** Build the web page of `path` at `commit` in the change request's repository. */ @@ -176,8 +180,10 @@ interface ChangeRequestSnapshot { readonly checksRollup: ChecksRollup /** Whether the host reported more checks than `checks` contains. */ readonly checksTruncated: boolean - /** Number of review threads that are not resolved. */ + /** Number of counted review threads that are not resolved. */ readonly unresolvedThreads: number + /** Whether the change request has more review threads than the provider counted, so `unresolvedThreads` is a lower bound. */ + readonly threadsTruncated: boolean /** ISO 8601 time of the host's last update to the change request. */ readonly updatedAt: string /** ISO 8601 time the change request was merged or closed, or null while it is open. */ @@ -191,9 +197,10 @@ interface ForgeFailure { /** * `not-found`: the host does not expose it; `unauthenticated`: credentials are missing or rejected; * `too-large`: the host or the provider refuses an answer over its size limit, so reading again fails - * the same way; `unavailable`: any other failure. + * the same way; `stale`: the head is no longer the commit the read named, so reading again at the head of + * a later status can succeed; `unavailable`: any other failure. */ - readonly code: 'not-found' | 'unauthenticated' | 'too-large' | 'unavailable' + readonly code: 'not-found' | 'unauthenticated' | 'too-large' | 'stale' | 'unavailable' /** Human-readable cause. */ readonly message: string } @@ -568,7 +575,7 @@ type ForgeChangeRequestStatesFrame = | { readonly type: 'changed'; readonly states: readonly ForgeChangeRequestListState[] } ``` -变更视图经同一个 Remote 读取。`localDiff` 读取 Session 检出的某个 `ForgeLocalScope`；`changeRequestDiff` 经提供方的 `diff` 读取已绑定变更请求在其最新状态 head 处的变更；两者返回的文件都已在宿主上解析为 `ForgeDiffFile` 的 hunk。`localText` 与 `changeRequestText` 读取一个文件的新版本，用来补出 hunk 之间未变更的行。每次检查还会把 `branch` 范围的规模统计为 `ForgePanelState.local`，供面板的变更卡片使用，始终以自动分叉点为准。`base` 指定了引用的 `branch` 范围改为与 HEAD 和该分支的合并基准比较；`diffBases` 列出视图可选的分支以及自动分叉点。 +变更视图经同一个 Remote 读取。`localDiff` 读取 Session 检出的某个 `ForgeLocalScope`；`changeRequestDiff` 经提供方的 `diff` 读取已绑定变更请求在其最新状态 head 处的变更，提供方只返回描述该 head 提交的补丁；两者返回的文件都已在宿主上解析为 `ForgeDiffFile` 的 hunk。`localText` 与 `changeRequestText` 读取一个文件的新版本，用来补出 hunk 之间未变更的行。每次检查还会把 `branch` 范围的规模统计为 `ForgePanelState.local`，供面板的变更卡片使用，始终以自动分叉点为准。`base` 指定了引用的 `branch` 范围改为与 HEAD 和该分支的合并基准比较；`diffBases` 列出视图可选的分支以及自动分叉点。head 已越过最新状态时，提供方返回 `stale`，`ctx.forge.diff` 立即重新轮询该变更请求；`changeRequestDiff` 以 `forgePanel/stale-head` 拒绝且不缓存任何内容，变更视图保持加载，直到更新的状态到达后按该状态的 head 重新读取。因此评审评论始终指向视图所显示行的那个提交。 ```ts type-equiv /** Which local changes a diff covers; each reads the Session's checkout through Git. */ @@ -814,13 +821,16 @@ async merge(ref: ChangeRequestRef, request: MergeRequest, signal: AbortSignal): async fetchChecks(ref: ChangeRequestRef, signal: AbortSignal): Promise /** - * Read the changes of a change request as its host computes them. + * Read the changes of a change request at one head commit as its host + * computes them. A `stale` failure polls the change request again at once + * when it is watched, so its next status names the current head. * @param ref - change request served by a registered provider. + * @param headSha - head commit the patch must describe, usually the head of the latest status. * @param signal - cancellation of the read. - * @returns one unified patch with `a/` and `b/` path prefixes, or why the host did not provide it. + * @returns one unified patch with `a/` and `b/` path prefixes, or why the host did not provide it at `headSha`. * @throws when no provider serves the change request's forge, or with `signal.reason` when it aborts. */ -async diff(ref: ChangeRequestRef, signal: AbortSignal): Promise +async diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise /** * Publish one comment or a completed review, then refresh the watched status. @@ -1015,6 +1025,7 @@ Serves the forge panel. Following a Session inspects the checkout of its current * @param signal - request cancellation. * @returns the files, the head commit, each file's web page, and the commits the checkout has beyond the head. * @throws `forgePanel/not-bound` unless the URL is a visible binding of the Session that a provider serves; + * `forgePanel/stale-head` when the head moved past the latest status, which is then polled again; * `forgePanel/diff-too-large` when the host or its provider refuses the diff over a size limit; * `forgePanel/diff-failed` before its status was read and when the host does not provide the diff otherwise. */ diff --git a/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.i18n.yaml b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.i18n.yaml new file mode 100644 index 0000000000..ffe417704a --- /dev/null +++ b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.i18n.yaml @@ -0,0 +1,16 @@ +# Bilingual-pair consistency record for guide.md (docs/i18n/README.md): per heading +# section, a hash of its English and Chinese blocks outside code blocks and generated regions. +# After editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.md +/: + en: ad1c3c4e42227656 + zh: 286eac73122ddbef +/forge-diffs-read-one-head-commit: + en: c017d657e5d9eb08 + zh: d0eb2aef480e7753 +/forge-diffs-read-one-head-commit/change: + en: 669f43ccd2364c5a + zh: 97190faaa338da51 +/forge-diffs-read-one-head-commit/migration: + en: 04cf41940f9fa126 + zh: edf52bd1d8882bf9 diff --git a/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.md b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.md new file mode 100644 index 0000000000..e1d3c890ef --- /dev/null +++ b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.md @@ -0,0 +1,26 @@ +--- +kind: upgrade-guide +description: "ForgeProvider.diff and ctx.forge.diff take the head commit to read, ForgeFailure gains the stale code, and ChangeRequestSnapshot gains threadsTruncated." +--- + +# Forge diffs read one head commit + +English | [中文](guide.zh.md) + +## Change + +`ForgeProvider.diff(ref, signal)` becomes `diff(ref, headSha, signal)`, and `ctx.forge.diff(ref, signal)` becomes `ctx.forge.diff(ref, headSha, signal)`. A provider returns a patch only when it describes `headSha`; when the change request's head is another commit, it resolves to a `ForgeFailure` whose code is the new `stale`. `ctx.forge.diff` then polls the change request again at once when it is watched. The GitHub provider reads the head before and after `gh pr diff` and returns `stale` when either read names another commit. + +`ForgeFailure['code']` gains `stale`. `ChangeRequestSnapshot` gains the required `threadsTruncated: boolean`, which marks `unresolvedThreads` as a lower bound when the provider counted only some review threads. + +The `forgePanel.changeRequestDiff` Remote rejects with `forgePanel/stale-head` instead of returning a patch read at another head. + +This affects third-party `ForgeProvider` implementations registered with `ctx.forge.register`, code that calls `ctx.forge.diff` or `forgePanel.changeRequestDiff`, code that builds `ChangeRequestSnapshot` values, and code that switches exhaustively over `ForgeFailure['code']`. All types come from `@deepseek-ai/dsh-experimental-forge`. + +## Migration + +1. In a `ForgeProvider`, add the `headSha: string` parameter to `diff` before `signal`. Return the patch only when the host computed it at `headSha`; otherwise return `{ code: 'stale', message }`. A host whose diff endpoint takes no commit can read the head before and after the diff, as [`GitHubProvider.diff`](../../../../packages/experimental/forge-github/src/provider.ts) does. +2. Pass the head commit to every `ctx.forge.diff` call, usually `ctx.forge.latest(ref)`'s `snapshot.headSha`. On `stale`, wait for a later status and read again at its head; do not cache the result under the old head. +3. Set `threadsTruncated` in every `ChangeRequestSnapshot` a provider builds: `false` when `unresolvedThreads` counts every review thread. +4. Handle `stale` wherever code switches exhaustively over `ForgeFailure['code']`, and `forgePanel/stale-head` wherever a caller of `forgePanel.changeRequestDiff` distinguishes Remote error codes. +5. Run your type check; it succeeds when every `diff` implementation and call passes a head commit, every snapshot sets `threadsTruncated`, and no `ForgeFailure` code is unhandled. diff --git a/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.zh.md b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.zh.md new file mode 100644 index 0000000000..e5549364ec --- /dev/null +++ b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.zh.md @@ -0,0 +1,26 @@ +--- +kind: upgrade-guide +description: "ForgeProvider.diff 与 ctx.forge.diff 需要传入要读取的 head 提交，ForgeFailure 新增 stale 代码，ChangeRequestSnapshot 新增 threadsTruncated。" +--- + +# Forge diff 读取指定的 head 提交 + +[English](guide.md) | 中文 + +## 变更 + +`ForgeProvider.diff(ref, signal)` 改为 `diff(ref, headSha, signal)`，`ctx.forge.diff(ref, signal)` 改为 `ctx.forge.diff(ref, headSha, signal)`。提供方只在补丁描述 `headSha` 时返回补丁；变更请求的 head 已是其他提交时，resolve 为代码为新增 `stale` 的 `ForgeFailure`。此时若该变更请求正被 watch，`ctx.forge.diff` 会立即重新轮询它。GitHub 提供方在 `gh pr diff` 前后各读一次 head，任一次不是该提交时返回 `stale`。 + +`ForgeFailure['code']` 新增 `stale`。`ChangeRequestSnapshot` 新增必填字段 `threadsTruncated: boolean`；提供方只统计了部分评审讨论时，它表示 `unresolvedThreads` 只是下限。 + +`forgePanel.changeRequestDiff` Remote 以 `forgePanel/stale-head` 拒绝，不再返回在其他 head 读到的补丁。 + +受影响的是用 `ctx.forge.register` 注册的第三方 `ForgeProvider` 实现、调用 `ctx.forge.diff` 或 `forgePanel.changeRequestDiff` 的代码、构造 `ChangeRequestSnapshot` 的代码，以及对 `ForgeFailure['code']` 做穷尽分支的代码。这些类型都来自 `@deepseek-ai/dsh-experimental-forge`。 + +## 迁移 + +1. 在 `ForgeProvider` 的 `diff` 中，于 `signal` 之前加入 `headSha: string` 参数。只有托管平台在 `headSha` 处计算出补丁时才返回补丁，否则返回 `{ code: 'stale', message }`。若托管平台的 diff 接口不接受提交参数，可以像 [`GitHubProvider.diff`](../../../../packages/experimental/forge-github/src/provider.ts) 那样在读取 diff 前后各读一次 head。 +2. 每次调用 `ctx.forge.diff` 都传入 head 提交，通常是 `ctx.forge.latest(ref)` 的 `snapshot.headSha`。收到 `stale` 时，等待更新的状态并按其 head 重新读取；不要把结果缓存在旧 head 下。 +3. 提供方构造的每个 `ChangeRequestSnapshot` 都要设置 `threadsTruncated`：`unresolvedThreads` 统计了全部评审讨论时为 `false`。 +4. 凡对 `ForgeFailure['code']` 做穷尽分支的地方都处理 `stale`；凡区分 `forgePanel.changeRequestDiff` Remote 错误代码的调用方都处理 `forgePanel/stale-head`。 +5. 运行类型检查；所有 `diff` 实现与调用都传入 head 提交、所有快照都设置 `threadsTruncated`、没有未处理的 `ForgeFailure` 代码时，检查通过。 diff --git a/packages/experimental/client-ui-forge/README.i18n.yaml b/packages/experimental/client-ui-forge/README.i18n.yaml index 203cd3e907..9c2c5d288b 100644 --- a/packages/experimental/client-ui-forge/README.i18n.yaml +++ b/packages/experimental/client-ui-forge/README.i18n.yaml @@ -15,11 +15,11 @@ en: d152484eb41ac6b4 zh: 2f8a621b991260a1 /deepseek-ai-dsh-experimental-client-ui-forge/use-this-package: - en: 1101418b5621ba9d - zh: 4d66052d4708d02c + en: 458a6d3447c33aa2 + zh: 5af3eb9439840b73 /deepseek-ai-dsh-experimental-client-ui-forge/understand-the-implementation: - en: 0c2dd86b5f6b489d - zh: 2f0fce0affa48f59 + en: 04b6519156ba47de + zh: 84c4683f3762c22c /deepseek-ai-dsh-experimental-client-ui-forge/further-exploration: en: fa4d8c579ffc49ff zh: 5856ed7469e4585b diff --git a/packages/experimental/client-ui-forge/README.md b/packages/experimental/client-ui-forge/README.md index 29afc7d4e6..84f9b90c78 100644 --- a/packages/experimental/client-ui-forge/README.md +++ b/packages/experimental/client-ui-forge/README.md @@ -36,9 +36,9 @@ Panel text follows the application language: Chinese, English, Japanese, Spanish - **Code review and changes** — on the right-sidebar start page, opens the same change request tab as the Session header button. - **Header button** — appears when the Session has bound change requests or its directory is inside a Git checkout, and shows or hides the tab. A red dot marks an open change request with failed checks, requested changes, or conflicts. - **Session list** — after a Session row's time, a branch glyph shows while the Session works in the worktree the panel created for it, and a glyph shows the state of its most urgent change request: one needing attention first, then open, draft, not yet polled, merged, and closed. The glyph of an open change request keeps the row's subdued color, with its head branch end filled in the color of running, passed, or failed checks; merged and closed ones keep their state colors. The row's hover card names the worktree branch and what it started from, then up to three change requests with their number, title, state, and checks, and how many more there are. A restart shows the states known before it until the next poll. -- **Tab** — the first row names the checked-out branch, or the abbreviated commit on a detached HEAD, after a folder glyph for the project directory or a branch glyph for a worktree, and the kind of checkout, which is the worktree the Session created with its base, another worktree, or the project directory; its tooltip adds the path with the Host home shortened to `~`, its button copies the full path, and for a removed worktree it offers Restore. Below it, each change request card shows a glyph for its state, its title, which opens it in the browser, and its number; the glyph tells open, draft, merged, and closed apart, and marks an open one that is not a draft with failed checks, requested changes, or conflicts as needing attention, as the header button's red dot does. The card's header also offers Open in browser, Copy change request link, and Remove from this Session, whose toast offers Undo. Cancelled checks have a separate count; a collapsed card names cancellations when no listed check failed. A collapsed card adds the checks of an open change request and the most urgent of requested changes, conflicts, or draft, or when a merged or closed one ended. An expanded card shows the head and base branches, then one row per fact: for an open change request the checks, whose row opens the list of checks with failed ones first and their durations and links, the review decision with unresolved threads, and the merge state, or when a merged or closed one ended; and the changed lines, formatted in the selected UI language, whose row opens the changes view. An open draft also offers Ready for review; a change request the host would merge now offers Merge, whose menu lists the methods the repository allows with the user's usual one chosen and pins the displayed head when opened. Confirm merge uses that head; a polled head change disables confirmation until the menu is reopened after review. A toast reports each outcome, and new commits since that head stop the merge. When exactly one bound change request is open, its card starts expanded. Refresh inspects the checkout, checks sign-in, and polls every change request again; its icon turns until the inspection and the sign-in check end. A banner shows the `gh auth login` command when sign-in is missing. +- **Tab** — the first row names the checked-out branch, or the abbreviated commit on a detached HEAD, after a folder glyph for the project directory or a branch glyph for a worktree, and the kind of checkout, which is the worktree the Session created with its base, another worktree, or the project directory; its tooltip adds the path with the Host home shortened to `~`, its button copies the full path, and for a removed worktree it offers Restore. Below it, each change request card shows a glyph for its state, its title, which opens it in the browser, and its number; the glyph tells open, draft, merged, and closed apart, and marks an open one that is not a draft with failed checks, requested changes, or conflicts as needing attention, as the header button's red dot does. The card's header also offers Open in browser, Copy change request link, and Remove from this Session, whose toast offers Undo. Cancelled checks have a separate count; a collapsed card names cancellations when no listed check failed. A collapsed card adds the checks of an open change request and the most urgent of requested changes, conflicts, or draft, or when a merged or closed one ended. An expanded card shows the head and base branches, then one row per fact: for an open change request the checks, whose row opens the list of checks with failed ones first and their durations and links, the review decision with unresolved threads, whose count ends in `+` when the provider counted only some threads, and the merge state, or when a merged or closed one ended; and the changed lines, formatted in the selected UI language, whose row opens the changes view. An open draft also offers Ready for review; a change request the host would merge now offers Merge, whose menu lists the methods the repository allows with the user's usual one chosen and pins the displayed head when opened. Confirm merge uses that head; a polled head change disables confirmation until the menu is reopened after review. A toast reports each outcome, and new commits since that head stop the merge. When exactly one bound change request is open, its card starts expanded. Refresh inspects the checkout, checks sign-in, and polls every change request again; its icon turns until the inspection and the sign-in check end. A banner shows the `gh auth login` command when sign-in is missing. - **Local changes card** — whenever the `branch` scope changes files, independently of bound change requests, a card above the change requests names the ref it compares with, such as `HEAD → origin/main`, which is always the default base below, and shows the added and deleted lines and the number of files, which open the changes view; its file list, which names files as the changes view does, opens the view at one file. Create change request, offered when the code host repository is recognized and no open change request (including drafts) has the checked-out branch as its head, sends the Session a user message in the UI language that asks the Agent to create a change request for these changes, and a toast reports whether it was sent. -- **Changes view** — The Changes row of an expanded change request card and the local changes card open a right-sidebar tab that lists the changed files. The local tab is named Local changes and shows Branch by default: the working tree and untracked files against the fork point. Its header names the checked-out branch, or the abbreviated commit on a detached HEAD, and the base branch, which opens a searchable menu: first Default, the base commit of the worktree the Session created, else the merge base of HEAD with the default branch of the code host remote, then remote-tracking and then local branches, each most recently committed first, without the checked-out branch and, outside a created worktree, without the remote-tracking branch Default already names. Typing filters the listed rows at once and lists the branches whose names contain the text, ignoring case, after a short pause; Enter chooses the first row. Choosing another branch compares with the merge base of HEAD and that branch. The tab keeps the choice while other scopes show; closing the tab or reloading the application returns it to Default. When the chosen branch is gone, its name is struck through, and when it shares no history with HEAD, the view says so as well; both offer Use the default base branch instead of Retry. A narrow view hides the note that uncommitted changes count, then the checked-out branch. Uncommitted compares them with HEAD, Unstaged with the index, and Staged compares the index with HEAD. A change request view shows the host's diff at its head and says how many commits the checked-out branch has beyond that head when it is the change request's head branch. The toolbar jumps to a file, totals the changed lines, refreshes, and switches between two columns and one and between wrapped and cut long lines. Visited changes tabs stay mounted across tab and Session switches, hiding the sidebar, and docking changes, preserving file folds, revealed large files, expanded context, layout, and scroll position while the diff is unchanged. Hidden changes views release their panel subscription and defer diff reads; returning reads again only when the latest head or local summary differs. A refresh that replaces the diff discards expanded context, and changed content may move the reading position. Temporary menus close when the view is hidden. Closing the tab or reloading the application discards this progress. Files are named before their directories, which use smaller subdued type and shorten first; hovering one shows its complete path, and both paths for a renamed file. Each file shows badges for a new, deleted, or renamed file, the last naming the old file name, or the old path when only the directory changed, and, in the Branch scope, an untracked one, and a menu that copies its path and opens the whole file in the sidebar for local changes or its page at the head in the browser for a change request. Unchanged lines between hunks collapse into rows that expand from the file's new side: the working tree, the index for Staged, or the head commit through the code host; a file that changed after the diff was read stays collapsed until the view reads again. A binary file and an untracked file over `maxDiffBytes` show a note instead of their lines, a file whose hunks draw more than 1,000 lines starts as one collapsed strip that names its changed lines and draws them when opened, and a file below files that change more than 5,000 lines starts collapsed. The view reads again on Refresh and the page refresh shortcut, on each navigation, when the change request's head moves, and when the Branch scope's summary changes; a change request's view first waits for the status that names its head, unless no provider serves it or that poll failed. While a read replaces changes already shown, the Refresh icon turns. A change request whose diff the code host or its provider refuses over a size limit, such as a GitHub diff over 300 files, says its changes are too large to show here and offers View in browser; another failed read offers Retry and, for a change request, View in browser. A change request's diff is downloaded once per head commit and base branch while it stays cached, so reopening, refreshing, and navigating at the same head read nothing from the host. +- **Changes view** — The Changes row of an expanded change request card and the local changes card open a right-sidebar tab that lists the changed files. The local tab is named Local changes and shows Branch by default: the working tree and untracked files against the fork point. Its header names the checked-out branch, or the abbreviated commit on a detached HEAD, and the base branch, which opens a searchable menu: first Default, the base commit of the worktree the Session created, else the merge base of HEAD with the default branch of the code host remote, then remote-tracking and then local branches, each most recently committed first, without the checked-out branch and, outside a created worktree, without the remote-tracking branch Default already names. Typing filters the listed rows at once and lists the branches whose names contain the text, ignoring case, after a short pause; Enter chooses the first row. Choosing another branch compares with the merge base of HEAD and that branch. The tab keeps the choice while other scopes show; closing the tab or reloading the application returns it to Default. When the chosen branch is gone, its name is struck through, and when it shares no history with HEAD, the view says so as well; both offer Use the default base branch instead of Retry. A narrow view hides the note that uncommitted changes count, then the checked-out branch. Uncommitted compares them with HEAD, Unstaged with the index, and Staged compares the index with HEAD. A change request view shows the host's diff at its head and says how many commits the checked-out branch has beyond that head when it is the change request's head branch. The toolbar jumps to a file, totals the changed lines, refreshes, and switches between two columns and one and between wrapped and cut long lines. Visited changes tabs stay mounted across tab and Session switches, hiding the sidebar, and docking changes, preserving file folds, revealed large files, expanded context, layout, and scroll position while the diff is unchanged. Hidden changes views release their panel subscription and defer diff reads; returning reads again only when the latest head or local summary differs. A refresh that replaces the diff discards expanded context, and changed content may move the reading position. Temporary menus close when the view is hidden. Closing the tab or reloading the application discards this progress. Files are named before their directories, which use smaller subdued type and shorten first; hovering one shows its complete path, and both paths for a renamed file. Each file shows badges for a new, deleted, or renamed file, the last naming the old file name, or the old path when only the directory changed, and, in the Branch scope, an untracked one, and a menu that copies its path and opens the whole file in the sidebar for local changes or its page at the head in the browser for a change request. Unchanged lines between hunks collapse into rows that expand from the file's new side: the working tree, the index for Staged, or the head commit through the code host; a file that changed after the diff was read stays collapsed until the view reads again. A binary file and an untracked file over `maxDiffBytes` show a note instead of their lines, a file whose hunks draw more than 1,000 lines starts as one collapsed strip that names its changed lines and draws them when opened, and a file below files that change more than 5,000 lines starts collapsed. The view reads again on Refresh and the page refresh shortcut, on each navigation, when the change request's head moves, and when the Branch scope's summary changes; a change request's view first waits for the status that names its head, unless no provider serves it or that poll failed. When the head moved past the latest status, the view keeps loading and reads again once a later status arrives, at its head. While a read replaces changes already shown, the Refresh icon turns. A change request whose diff the code host or its provider refuses over a size limit, such as a GitHub diff over 300 files, says its changes are too large to show here and offers View in browser; another failed read offers Retry and, for a change request, View in browser. A change request's diff is downloaded once per head commit and base branch while it stays cached, so reopening, refreshing, and navigating at the same head read nothing from the host. The checks row initially shows the polling preview. While it is incomplete, the card uses the host’s aggregate outcome without a failure count and labels the loaded check count as partial. Opening the list loads the remaining pages for that PR through `changeRequestChecks`; its deadline is independent of polling. A failed read keeps the preview and offers Retry. Closing the list, changing the head, or unmounting cancels the read; late results are ignored. Reopening reads current details. The Remote requires a visible Session binding and rejects a response for a different head commit. @@ -67,7 +67,7 @@ Restoration records `restoring` before mutation and `restored` only after succes Each inspection also decides whether the worktree is missing: the Session's directory lies inside the created worktree, `ctx.fs.lstat` finds no directory there, and `show-ref` in the source repository tells whether the branch still exists. -`localDiff` takes the fork point from the created worktree's `baseCommit` when the Session works in it, else, for each remote in the code host order, from `symbolic-ref --quiet --short refs/remotes//HEAD` and `merge-base HEAD`; without one the `branch` scope refuses with `forgePanel/no-fork-point`. A `branch` scope that names a base ref under `refs/heads/` or `refs/remotes/` instead takes `merge-base HEAD` with it once `show-ref --verify` finds the ref, refusing with `forgePanel/base-missing` when the ref does not exist and with `forgePanel/no-fork-point` when HEAD shares no history with it; another ref name is refused before Git runs. It runs `diff --no-ext-diff --no-textconv --submodule=short --ignore-submodules=dirty --no-color -M` with fixed `a/` and `b/` prefixes and `core.quotePath=false` against the fork point, against HEAD (the empty tree before the first commit), against the index, or with `--cached` against HEAD. Before patches and statistics, Git configuration is read to apply command-local clean/process filter overrides. A failed or truncated configuration read stops the comparison: `localDiff` reports `forgePanel/filter-config-failed`, with a retry after correcting Git configuration or the output limit; failed statistics leave the local changes card hidden. Filter enumeration uses `maxDiffBytes` for patches and `maxOutputBytes` for statistics. Untracked files, at most `maxUntrackedFiles` with the rest counted as omitted, come from `ls-files --others --exclude-standard -z` and are read through `ctx.fs` as new files, binary when their first 8,000 bytes hold a NUL byte or they are not UTF-8. A patch, file list, or file over `maxDiffBytes` refuses with `forgePanel/diff-too-large`, and a directory outside a checkout with `forgePanel/not-a-checkout`. The Host parses patches with jsdiff into files and hunks, so the browser receives no raw patch. `localText` reads the new side of one file, from the index through `show :` for Staged and through `ctx.fs` otherwise, and refuses a path that leaves the checkout. `changeRequestDiff` and `changeRequestText` act only on a visible binding a provider serves, else they refuse with `forgePanel/not-bound`; they read the head commit of the binding's latest status through the provider's `diff` and `fileText`, refuse a `too-large` failure with `forgePanel/diff-too-large`, and refuse with `forgePanel/diff-failed` before a status was read or when the host refuses otherwise. The Host keeps the patches `changeRequestDiff` read, one per change request at its head commit and base branch, and drops the least recently read once they exceed `maxCachedDiffBytes`; a later read at the same head and base answers from memory for any Session, and a failed read is not kept. The commits the checkout has beyond the head come from `symbolic-ref`, `cat-file -e`, and `rev-list --count`, and are null when the checked-out branch is not the head branch, the head commit is absent, or Git fails. +`localDiff` takes the fork point from the created worktree's `baseCommit` when the Session works in it, else, for each remote in the code host order, from `symbolic-ref --quiet --short refs/remotes//HEAD` and `merge-base HEAD`; without one the `branch` scope refuses with `forgePanel/no-fork-point`. A `branch` scope that names a base ref under `refs/heads/` or `refs/remotes/` instead takes `merge-base HEAD` with it once `show-ref --verify` finds the ref, refusing with `forgePanel/base-missing` when the ref does not exist and with `forgePanel/no-fork-point` when HEAD shares no history with it; another ref name is refused before Git runs. It runs `diff --no-ext-diff --no-textconv --submodule=short --ignore-submodules=dirty --no-color -M` with fixed `a/` and `b/` prefixes and `core.quotePath=false` against the fork point, against HEAD (the empty tree before the first commit), against the index, or with `--cached` against HEAD. Before patches and statistics, Git configuration is read to apply command-local clean/process filter overrides. A failed or truncated configuration read stops the comparison: `localDiff` reports `forgePanel/filter-config-failed`, with a retry after correcting Git configuration or the output limit; failed statistics leave the local changes card hidden. Filter enumeration uses `maxDiffBytes` for patches and `maxOutputBytes` for statistics. Untracked files, at most `maxUntrackedFiles` with the rest counted as omitted, come from `ls-files --others --exclude-standard -z` and are read through `ctx.fs` as new files, binary when their first 8,000 bytes hold a NUL byte or they are not UTF-8. A patch, file list, or file over `maxDiffBytes` refuses with `forgePanel/diff-too-large`, and a directory outside a checkout with `forgePanel/not-a-checkout`. The Host parses patches with jsdiff into files and hunks, so the browser receives no raw patch. `localText` reads the new side of one file, from the index through `show :` for Staged and through `ctx.fs` otherwise, and refuses a path that leaves the checkout. `changeRequestDiff` and `changeRequestText` act only on a visible binding a provider serves, else they refuse with `forgePanel/not-bound`; they read the head commit of the binding's latest status through the provider's `diff` and `fileText`, refuse a `too-large` failure with `forgePanel/diff-too-large`, a `stale` failure, meaning the head moved past that status, with `forgePanel/stale-head`, and refuse with `forgePanel/diff-failed` before a status was read or when the host refuses otherwise. A `stale` diff polls the change request at once and is not cached, so no patch is ever kept under a head it does not describe. The Host keeps the patches `changeRequestDiff` read, one per change request at its head commit and base branch, and drops the least recently read once they exceed `maxCachedDiffBytes`; a later read at the same head and base answers from memory for any Session, and a failed read is not kept. The commits the checkout has beyond the head come from `symbolic-ref`, `cat-file -e`, and `rev-list --count`, and are null when the checked-out branch is not the head branch, the head commit is absent, or Git fails. `changeRequestStates` streams, for the Session list, the last known number, title, state, draft flag, checks rollup, review decision, and mergeability of every change request that a listed Session the caller may read binds: a `baseline` frame, then `changed` frames. One run serves every stream. Cancellation is observed before initialization and between frames; the last stream and plugin disposal await initialization and accepted writes, then close the run’s storage domain. A later run waits for that closure before reopening the domain and reading its stored states. Binding updates received during listing override its initial snapshot. It lists Sessions through `sessionQuery` without loading them, takes the visible bindings of a loaded Session from `ctx.forgeSession` and of another from its cached `forgeChangeRequests` summary, follows binding changes of loaded Sessions, and subscribes to each bound change request through `ctx.forge.watch`; closed requests use the idle polling cadence, and merged requests remain subscribed for explicit refresh. It stores every changed state in the `forge_panel_change_requests` storage domain under the SHA-256 digest of its URL and drops unbound stored states only when every listed Session has known bindings; a failed write is logged and the state stays in memory. The stream fails when the Host lacks the Session query, projection cache, storage, or Session store service. diff --git a/packages/experimental/client-ui-forge/README.zh.md b/packages/experimental/client-ui-forge/README.zh.md index f45d0c2568..4c6cc8f5ed 100644 --- a/packages/experimental/client-ui-forge/README.zh.md +++ b/packages/experimental/client-ui-forge/README.zh.md @@ -36,9 +36,9 @@ kind: "package-reference" - **代码审查和变更**——右侧栏起始页中的入口，与会话标题栏按钮打开同一个变更请求标签页。 - **标题栏按钮**——Session 有已绑定的变更请求、或其目录位于 Git 检出中时出现，用来显示或隐藏标签页。打开状态的变更请求存在检查失败、被要求修改或冲突时，按钮带红点。 - **会话列表**——Session 在面板为它创建的 worktree 中工作时，会话行的时间之后显示分支图标；另有一个图标显示它最需要关注的变更请求的状态，依次为需要处理、打开、草稿、尚未轮询、已合并、已关闭。打开中的变更请求图标保持会话行的浅色，head 分支端点填充为检查进行中、已通过或失败对应的颜色；已合并与已关闭的保留各自的状态颜色。会话行的悬浮卡片写出 worktree 分支及其起点，再列出最多三个变更请求的编号、标题、状态与检查，并说明还有多少个。重启后，下次轮询之前显示重启前已知的状态。 -- **标签页**——顶部一行显示当前检出的分支（分离 HEAD 时为缩写提交），项目目录前显示目录图标、worktree 前显示分支图标，并说明检出的类型（Session 创建的 worktree 及其基准、其他 worktree 或项目目录）；它的提示给出路径，宿主主目录缩写为 `~`，旁边的按钮复制完整路径；worktree 已被删除时提供“恢复”。其下每张变更请求卡片显示状态图标、标题（点击在浏览器中打开）与编号；图标区分打开、草稿、已合并与已关闭，打开中的变更请求（草稿除外）检查失败、被要求修改或有冲突时显示为“需要处理”，与标题栏按钮的红点一致。卡片右上角还提供“在浏览器中打开”“复制变更请求链接”与“从本会话移除”，移除后的 toast 提供“撤销”。已取消的检查单独计数；列出的检查没有失败项时，折叠卡片显示取消数量。折叠的卡片还显示打开的变更请求的检查，以及被要求修改、冲突、草稿中最需要注意的一项；已合并或已关闭的则显示结束时间。展开的卡片显示源分支与目标分支，下面每项一行：打开的变更请求依次是检查（点击该行弹出检查列表，失败的排在前面，带耗时与链接）、评审结论与未解决讨论数、合并状态，已合并或已关闭的显示结束时间；最后是按所选界面语言格式化的变更行数，点击该行打开变更视图。打开中的草稿还提供“标记为可评审”；主机现在就会合并的变更请求提供“合并”，其菜单列出仓库允许的合并方式并选中用户常用的一种，打开时固定卡片显示的 head。“确认合并”使用这个 head；轮询发现 head 变化后禁用确认，需重新检查并打开菜单。每次操作的结果都以 toast 报告；该 head 之后有新提交时不会合并。恰好有一个已绑定的变更请求处于打开状态时，它的卡片默认展开。刷新会重新检查检出与登录状态，并立即再次轮询每个变更请求；检查与登录状态确认结束前，刷新图标持续旋转。缺少登录时，横幅显示 `gh auth login` 命令。 +- **标签页**——顶部一行显示当前检出的分支（分离 HEAD 时为缩写提交），项目目录前显示目录图标、worktree 前显示分支图标，并说明检出的类型（Session 创建的 worktree 及其基准、其他 worktree 或项目目录）；它的提示给出路径，宿主主目录缩写为 `~`，旁边的按钮复制完整路径；worktree 已被删除时提供“恢复”。其下每张变更请求卡片显示状态图标、标题（点击在浏览器中打开）与编号；图标区分打开、草稿、已合并与已关闭，打开中的变更请求（草稿除外）检查失败、被要求修改或有冲突时显示为“需要处理”，与标题栏按钮的红点一致。卡片右上角还提供“在浏览器中打开”“复制变更请求链接”与“从本会话移除”，移除后的 toast 提供“撤销”。已取消的检查单独计数；列出的检查没有失败项时，折叠卡片显示取消数量。折叠的卡片还显示打开的变更请求的检查，以及被要求修改、冲突、草稿中最需要注意的一项；已合并或已关闭的则显示结束时间。展开的卡片显示源分支与目标分支，下面每项一行：打开的变更请求依次是检查（点击该行弹出检查列表，失败的排在前面，带耗时与链接）、评审结论与未解决讨论数（提供方只统计了部分讨论时数字后带 `+`）、合并状态，已合并或已关闭的显示结束时间；最后是按所选界面语言格式化的变更行数，点击该行打开变更视图。打开中的草稿还提供“标记为可评审”；主机现在就会合并的变更请求提供“合并”，其菜单列出仓库允许的合并方式并选中用户常用的一种，打开时固定卡片显示的 head。“确认合并”使用这个 head；轮询发现 head 变化后禁用确认，需重新检查并打开菜单。每次操作的结果都以 toast 报告；该 head 之后有新提交时不会合并。恰好有一个已绑定的变更请求处于打开状态时，它的卡片默认展开。刷新会重新检查检出与登录状态，并立即再次轮询每个变更请求；检查与登录状态确认结束前，刷新图标持续旋转。缺少登录时，横幅显示 `gh auth login` 命令。 - **本地变更卡片**——`branch` 范围有文件变更时始终显示，不受已关联变更请求影响。变更请求列表上方的卡片显示比较的基准（例如 `HEAD → origin/main`，始终是下文的默认基准）以及新增、删除行数与文件数，点击打开变更视图；展开的文件列表像变更视图一样标出各个文件，点击某个文件时，变更视图定位到该文件。识别出代码托管仓库、且没有以当前检出分支为源分支的打开中变更请求（含草稿）时，卡片提供“创建变更请求”：它以界面语言向 Session 发送一条用户消息，请 Agent 为这些变更创建变更请求，并以 toast 报告是否已发送。 -- **变更视图**——展开的变更请求卡片上的“变更”一行与本地变更卡片会打开一个右侧边栏标签页，列出所有变更的文件。本地标签页名为“本地变更”，默认显示“分支”：工作树与未跟踪文件相对于分叉点的变更。它的标题栏写出当前检出的分支（分离 HEAD 时为缩写提交）与基准分支；点击基准分支打开可搜索的菜单：最前面是“默认”，即 Session 创建的 worktree 的基准提交，否则是 HEAD 与代码托管远端默认分支的合并基准；其后依次是远程跟踪分支与本地分支，各按最近提交排序，不含当前检出的分支，不在所创建的 worktree 中时也不含“默认”已经指向的远程跟踪分支。输入文字会立即筛选已列出的行，短暂停顿后列出名称包含该文字（不区分大小写）的分支；按 Enter 选择第一行。选择其他分支后，与 HEAD 和该分支的合并基准比较。标签页切换到其他范围时仍保留这个选择；关闭标签页或重新加载应用后回到“默认”。所选分支已不存在时，它的名称显示删除线；它与 HEAD 没有共同历史时，视图同样给出说明；两种情况都提供“使用默认基准分支”而不是“重试”。视图较窄时，先隐藏“包含未提交的修改”，再隐藏当前分支。“未提交”与 HEAD 比较，“未暂存”与暂存区比较，“已暂存”比较暂存区与 HEAD。变更请求的视图显示托管平台在其 head 处的 diff；当前检出分支就是该变更请求的源分支时，还会说明本地分支比该 head 多出几个提交。工具栏可以跳到某个文件、汇总变更行数、刷新，并在左右对比与单栏对比、自动换行与截断长行之间切换。访问过的变更标签页会在切换标签页与 Session、隐藏侧边栏及调整停靠位置时保持挂载，在 diff 未改变时保留文件折叠、大文件展开、上下文展开、布局与滚动位置。隐藏的变更视图释放面板订阅并推迟 diff 读取；返回时仅在最新 head 或本地摘要不同时重新读取。刷新替换 diff 后会丢弃展开的上下文，内容变化也可能移动阅读位置。视图隐藏时关闭临时菜单。关闭标签页或重新加载应用后不保留这些进度。文件名在前，目录在后，用更小的浅色字显示，空间不够时目录先缩短；悬停时显示完整路径，重命名的文件显示新旧两条路径。每个文件带有新文件、已删除、重命名的标记（重命名标记写出原文件名，只换了目录时写出原路径），“分支”范围下的未跟踪文件还标“未提交”；文件菜单可以复制路径，本地变更可在侧边栏打开整个文件，变更请求可在浏览器中查看该文件在 head 处的页面。hunk 之间未变更的行折叠成一行，点击后从文件的新版本展开：工作树、“已暂存”时为暂存区，或经托管平台读取的 head 提交；读取 diff 之后文件又有修改时不会展开，直到视图重新读取。二进制文件与超过 `maxDiffBytes` 的未跟踪文件只显示一条说明；hunk 超过 1,000 行的文件先显示为一条写出变更行数的折叠条，点击后才绘制；前面的文件累计变更超过 5,000 行后，其后的文件默认收起。刷新与页面刷新快捷键、每次导航、变更请求的 head 变化，以及“分支”范围的摘要变化时，视图都会重新读取；变更请求的视图会先等到给出 head 的状态再读取，除非没有提供方服务它或那次轮询失败。重新读取已显示的变更时，刷新图标会旋转。托管平台或其提供方因超过大小上限拒绝给出 diff 的变更请求（例如 GitHub 上超过 300 个文件的 diff）显示“变更太大，无法在这里显示”并提供“在浏览器中查看”；其他读取失败提供“重试”，变更请求还提供“在浏览器中查看”。变更请求的 diff 在缓存期间对每个 head 提交与目标分支只下载一次，因此在同一 head 下重新打开、刷新与导航都不会再从托管平台读取。 +- **变更视图**——展开的变更请求卡片上的“变更”一行与本地变更卡片会打开一个右侧边栏标签页，列出所有变更的文件。本地标签页名为“本地变更”，默认显示“分支”：工作树与未跟踪文件相对于分叉点的变更。它的标题栏写出当前检出的分支（分离 HEAD 时为缩写提交）与基准分支；点击基准分支打开可搜索的菜单：最前面是“默认”，即 Session 创建的 worktree 的基准提交，否则是 HEAD 与代码托管远端默认分支的合并基准；其后依次是远程跟踪分支与本地分支，各按最近提交排序，不含当前检出的分支，不在所创建的 worktree 中时也不含“默认”已经指向的远程跟踪分支。输入文字会立即筛选已列出的行，短暂停顿后列出名称包含该文字（不区分大小写）的分支；按 Enter 选择第一行。选择其他分支后，与 HEAD 和该分支的合并基准比较。标签页切换到其他范围时仍保留这个选择；关闭标签页或重新加载应用后回到“默认”。所选分支已不存在时，它的名称显示删除线；它与 HEAD 没有共同历史时，视图同样给出说明；两种情况都提供“使用默认基准分支”而不是“重试”。视图较窄时，先隐藏“包含未提交的修改”，再隐藏当前分支。“未提交”与 HEAD 比较，“未暂存”与暂存区比较，“已暂存”比较暂存区与 HEAD。变更请求的视图显示托管平台在其 head 处的 diff；当前检出分支就是该变更请求的源分支时，还会说明本地分支比该 head 多出几个提交。工具栏可以跳到某个文件、汇总变更行数、刷新，并在左右对比与单栏对比、自动换行与截断长行之间切换。访问过的变更标签页会在切换标签页与 Session、隐藏侧边栏及调整停靠位置时保持挂载，在 diff 未改变时保留文件折叠、大文件展开、上下文展开、布局与滚动位置。隐藏的变更视图释放面板订阅并推迟 diff 读取；返回时仅在最新 head 或本地摘要不同时重新读取。刷新替换 diff 后会丢弃展开的上下文，内容变化也可能移动阅读位置。视图隐藏时关闭临时菜单。关闭标签页或重新加载应用后不保留这些进度。文件名在前，目录在后，用更小的浅色字显示，空间不够时目录先缩短；悬停时显示完整路径，重命名的文件显示新旧两条路径。每个文件带有新文件、已删除、重命名的标记（重命名标记写出原文件名，只换了目录时写出原路径），“分支”范围下的未跟踪文件还标“未提交”；文件菜单可以复制路径，本地变更可在侧边栏打开整个文件，变更请求可在浏览器中查看该文件在 head 处的页面。hunk 之间未变更的行折叠成一行，点击后从文件的新版本展开：工作树、“已暂存”时为暂存区，或经托管平台读取的 head 提交；读取 diff 之后文件又有修改时不会展开，直到视图重新读取。二进制文件与超过 `maxDiffBytes` 的未跟踪文件只显示一条说明；hunk 超过 1,000 行的文件先显示为一条写出变更行数的折叠条，点击后才绘制；前面的文件累计变更超过 5,000 行后，其后的文件默认收起。刷新与页面刷新快捷键、每次导航、变更请求的 head 变化，以及“分支”范围的摘要变化时，视图都会重新读取；变更请求的视图会先等到给出 head 的状态再读取，除非没有提供方服务它或那次轮询失败。head 已越过最新状态时，视图保持加载，等更新的状态到达后按其 head 重新读取。重新读取已显示的变更时，刷新图标会旋转。托管平台或其提供方因超过大小上限拒绝给出 diff 的变更请求（例如 GitHub 上超过 300 个文件的 diff）显示“变更太大，无法在这里显示”并提供“在浏览器中查看”；其他读取失败提供“重试”，变更请求还提供“在浏览器中查看”。变更请求的 diff 在缓存期间对每个 head 提交与目标分支只下载一次，因此在同一 head 下重新打开、刷新与导航都不会再从托管平台读取。 检查行先显示轮询预览。不完整时，卡片使用托管平台的总体状态而不显示失败总数，并注明已加载的检查数量尚不完整。打开列表后，经 `changeRequestChecks` 单独读取该 PR 的后续页，时限独立于轮询。读取失败时保留预览并提供重试。关闭列表、head 变化或组件卸载会取消读取，忽略迟到结果；重新打开会读取当前详情。Remote 要求 URL 是该 Session 可见的绑定，并拒绝 head 提交不一致的响应。 @@ -67,7 +67,7 @@ worktree 请求是一条 `plugin:client-ui-forge/worktree` 记录，由 `forgeWo 每次检查还会判断 worktree 是否缺失：Session 的目录位于所创建的 worktree 内，`ctx.fs.lstat` 在那里找不到目录，再由源仓库中的 `show-ref` 判断分支是否还在。 -Session 在所创建的 worktree 中工作时，`localDiff` 以它的 `baseCommit` 为分叉点；否则按代码托管远端的顺序，对每个远端依次运行 `symbolic-ref --quiet --short refs/remotes//HEAD` 与 `merge-base HEAD` 找出分叉点；找不到时 `branch` 范围以 `forgePanel/no-fork-point` 拒绝。`branch` 范围指定了 `refs/heads/` 或 `refs/remotes/` 下的基准引用时，`show-ref --verify` 找到该引用后改用 `merge-base HEAD` 与它的结果；引用不存在时以 `forgePanel/base-missing` 拒绝，HEAD 与它没有共同历史时以 `forgePanel/no-fork-point` 拒绝；其他引用名在运行 Git 之前即被拒绝。它以固定的 `a/`、`b/` 前缀和 `core.quotePath=false` 运行 `diff --no-ext-diff --no-textconv --submodule=short --ignore-submodules=dirty --no-color -M`，分别与分叉点、HEAD（第一次提交之前为空树）、暂存区比较，或带 `--cached` 与 HEAD 比较。读取补丁与统计前会查询 Git 配置，应用命令级 clean/process filter 覆盖。配置读取失败或被截断时停止比较：`localDiff` 报告 `forgePanel/filter-config-failed`，修正 Git 配置或读取上限后可重试；统计失败时本地变更卡片保持隐藏。过滤器枚举在读取补丁时使用 `maxDiffBytes` 上限，统计时使用 `maxOutputBytes` 上限。未跟踪文件由 `ls-files --others --exclude-standard -z` 列出，最多 `maxUntrackedFiles` 个，其余计为未列出；它们经 `ctx.fs` 读取并作为新文件加入，前 8,000 字节含 NUL 字节或不是 UTF-8 时视为二进制。补丁、文件列表或文件超过 `maxDiffBytes` 时以 `forgePanel/diff-too-large` 拒绝，目录不在检出中时以 `forgePanel/not-a-checkout` 拒绝。宿主用 jsdiff 把补丁解析为文件与 hunk，浏览器不会收到原始补丁。`localText` 读取一个文件的新版本：“已暂存”经 `show :` 从暂存区读取，其余经 `ctx.fs` 读取；离开检出的路径会被拒绝。`changeRequestDiff` 与 `changeRequestText` 只作用于该 Session 中由提供方服务的可见绑定，否则以 `forgePanel/not-bound` 拒绝；它们经提供方的 `diff` 与 `fileText` 读取绑定最新状态中的 head 提交，`too-large` 失败以 `forgePanel/diff-too-large` 拒绝，尚未读到状态或托管平台以其他原因拒绝时以 `forgePanel/diff-failed` 拒绝。Host 保留 `changeRequestDiff` 读到的补丁，每个变更请求一份，按 head 提交与目标分支区分，总量超过 `maxCachedDiffBytes` 时丢弃最久未读的；之后在同一 head 与目标分支下的读取不论来自哪个 Session 都从内存返回，读取失败的结果不保留。本地分支比 head 多出的提交数由 `symbolic-ref`、`cat-file -e` 与 `rev-list --count` 得出；当前检出分支不是源分支、本地没有 head 提交或 Git 失败时为 null。 +Session 在所创建的 worktree 中工作时，`localDiff` 以它的 `baseCommit` 为分叉点；否则按代码托管远端的顺序，对每个远端依次运行 `symbolic-ref --quiet --short refs/remotes//HEAD` 与 `merge-base HEAD` 找出分叉点；找不到时 `branch` 范围以 `forgePanel/no-fork-point` 拒绝。`branch` 范围指定了 `refs/heads/` 或 `refs/remotes/` 下的基准引用时，`show-ref --verify` 找到该引用后改用 `merge-base HEAD` 与它的结果；引用不存在时以 `forgePanel/base-missing` 拒绝，HEAD 与它没有共同历史时以 `forgePanel/no-fork-point` 拒绝；其他引用名在运行 Git 之前即被拒绝。它以固定的 `a/`、`b/` 前缀和 `core.quotePath=false` 运行 `diff --no-ext-diff --no-textconv --submodule=short --ignore-submodules=dirty --no-color -M`，分别与分叉点、HEAD（第一次提交之前为空树）、暂存区比较，或带 `--cached` 与 HEAD 比较。读取补丁与统计前会查询 Git 配置，应用命令级 clean/process filter 覆盖。配置读取失败或被截断时停止比较：`localDiff` 报告 `forgePanel/filter-config-failed`，修正 Git 配置或读取上限后可重试；统计失败时本地变更卡片保持隐藏。过滤器枚举在读取补丁时使用 `maxDiffBytes` 上限，统计时使用 `maxOutputBytes` 上限。未跟踪文件由 `ls-files --others --exclude-standard -z` 列出，最多 `maxUntrackedFiles` 个，其余计为未列出；它们经 `ctx.fs` 读取并作为新文件加入，前 8,000 字节含 NUL 字节或不是 UTF-8 时视为二进制。补丁、文件列表或文件超过 `maxDiffBytes` 时以 `forgePanel/diff-too-large` 拒绝，目录不在检出中时以 `forgePanel/not-a-checkout` 拒绝。宿主用 jsdiff 把补丁解析为文件与 hunk，浏览器不会收到原始补丁。`localText` 读取一个文件的新版本：“已暂存”经 `show :` 从暂存区读取，其余经 `ctx.fs` 读取；离开检出的路径会被拒绝。`changeRequestDiff` 与 `changeRequestText` 只作用于该 Session 中由提供方服务的可见绑定，否则以 `forgePanel/not-bound` 拒绝；它们经提供方的 `diff` 与 `fileText` 读取绑定最新状态中的 head 提交，`too-large` 失败以 `forgePanel/diff-too-large` 拒绝，`stale` 失败（head 已越过该状态）以 `forgePanel/stale-head` 拒绝，尚未读到状态或托管平台以其他原因拒绝时以 `forgePanel/diff-failed` 拒绝。`stale` 的 diff 会立即重新轮询该变更请求且不缓存，因此不会有补丁被存放在它不描述的 head 下。Host 保留 `changeRequestDiff` 读到的补丁，每个变更请求一份，按 head 提交与目标分支区分，总量超过 `maxCachedDiffBytes` 时丢弃最久未读的；之后在同一 head 与目标分支下的读取不论来自哪个 Session 都从内存返回，读取失败的结果不保留。本地分支比 head 多出的提交数由 `symbolic-ref`、`cat-file -e` 与 `rev-list --count` 得出；当前检出分支不是源分支、本地没有 head 提交或 Git 失败时为 null。 `changeRequestStates` 为会话列表推送调用方可读的、已列出的 Session 所绑定的每个变更请求的最近编号、标题、状态、草稿标记、检查汇总、评审结论与可合并性：先是一个 `baseline` 帧，之后是 `changed` 帧。所有流共用一次运行。初始化之前和帧之间均响应取消；最后一条流结束及插件卸载会等待初始化与已接受的写入完成，再关闭本次运行的存储域。后续运行会等待该域关闭后重新打开，并读取已存状态。列举期间收到的绑定更新优先于初始快照。它经 `sessionQuery` 列出 Session 而不加载它们，已加载 Session 的可见绑定取自 `ctx.forgeSession`，其他 Session 的取自缓存的 `forgeChangeRequests` 摘要；它跟随已加载 Session 的绑定变化，并经 `ctx.forge.watch` 订阅每个已绑定的变更请求；已关闭的按空闲间隔轮询，已合并的保留订阅以接收显式刷新结果。每个变化了的状态以其 URL 的 SHA-256 摘要为键存入 `forge_panel_change_requests` 存储域，仅在所有已列出 Session 的绑定均已知时删除无人绑定的已存状态；写入失败时记录日志，状态仍保留在内存中。宿主缺少 Session 查询、投影缓存、存储或 Session 存储服务时，流以失败结束。 diff --git a/packages/experimental/client-ui-forge/src/client/ChangeRequestCard.tsx b/packages/experimental/client-ui-forge/src/client/ChangeRequestCard.tsx index 9795e3558c..0397991ad0 100644 --- a/packages/experimental/client-ui-forge/src/client/ChangeRequestCard.tsx +++ b/packages/experimental/client-ui-forge/src/client/ChangeRequestCard.tsx @@ -75,6 +75,7 @@ const FAILURE: Record = { 'not-found': 'change.failure.notFound', 'unauthenticated': 'change.failure.unauthenticated', 'too-large': 'change.failure.unavailable', + 'stale': 'change.failure.unavailable', 'unavailable': 'change.failure.unavailable', } @@ -89,6 +90,12 @@ const CHECK_RANK: Record = { failure: 0, pending: 1, success: 2, neutral: 3, skipped: 3, cancelled: 3, } +/** The unresolved thread count, with `+` when the provider counted only some threads. */ +function threadCount(snapshot: ChangeRequestSnapshot, locale: string): string { + const count = snapshot.unresolvedThreads.toLocaleString(locale) + return snapshot.threadsTruncated ? `${count}+` : count +} + function failureOf(entry: ForgePanelChangeRequest): ForgeFailure | undefined { return entry.update !== null && 'failure' in entry.update ? entry.update.failure : undefined } @@ -434,7 +441,7 @@ function Details({ entry, snapshot, name, now, actions, locale, t }: { {t('change.row.review')} - {snapshot.unresolvedThreads > 0 ? t('change.review.threads', { review, count: snapshot.unresolvedThreads }) : review} + {snapshot.unresolvedThreads > 0 ? t('change.review.threads', { review, count: threadCount(snapshot, locale) }) : review}
diff --git a/packages/experimental/client-ui-forge/src/client/ChangesTab.tsx b/packages/experimental/client-ui-forge/src/client/ChangesTab.tsx index 983c6b58a3..a65edc8570 100644 --- a/packages/experimental/client-ui-forge/src/client/ChangesTab.tsx +++ b/packages/experimental/client-ui-forge/src/client/ChangesTab.tsx @@ -462,11 +462,23 @@ export function ChangesTab({ // Names the read the inputs ask for; it is in flight until `finished` matches it. const read = version === null ? null : `${dataKey}\n${navigation.revision}\n${attempt}\n${version}` const [finished, setFinished] = useState(null) + // A change request read refused because the head moved past its status stays in flight until a later status arrives. + const statusAt = entry?.update?.fetchedAt ?? null + const latestStatusAt = useRef(statusAt) + useEffect(() => { latestStatusAt.current = statusAt }, [statusAt]) + const [refused, setRefused] = useState<{ readonly read: string; readonly statusAt: number | null } | null>(null) + const waiting = refused !== null && refused.read === read && refused.statusAt === statusAt useEffect(() => { - if (!tab.visible || read === null || read === finished) return + if (!tab.visible || read === null || read === finished || waiting) return const controller = new AbortController() + const readAt = latestStatusAt.current void readChanges(source, localScope, { localDiff, changeRequestDiff }, controller.signal).then((result) => { if (controller.signal.aborted) return + if (!result.ok && result.error.code === 'forgePanel/stale-head') { + setRefused({ read, statusAt: readAt }) + setFailure(null) + return + } setFinished(read) if (result.ok) { setLoaded({ key: dataKey, data: result.value }) @@ -476,7 +488,7 @@ export function ChangesTab({ } }) return () => { controller.abort() } - }, [tab.visible, read, source, localScope, dataKey, localDiff, changeRequestDiff]) + }, [tab.visible, read, waiting, source, localScope, dataKey, localDiff, changeRequestDiff]) const data = loaded !== null && loaded.key === dataKey ? loaded.data : null const files = useMemo(() => filesInTreeOrder(changedFileTree(data?.files ?? [])), [data]) // The centered loader marks a first read; the toolbar's glyph turns only while shown changes are read again. diff --git a/packages/experimental/client-ui-forge/src/index.ts b/packages/experimental/client-ui-forge/src/index.ts index 6800a70b34..bb2dfdc334 100644 --- a/packages/experimental/client-ui-forge/src/index.ts +++ b/packages/experimental/client-ui-forge/src/index.ts @@ -15,6 +15,7 @@ import type {} from '@deepseek-ai/dsh-session-projection-cache' import type {} from '@deepseek-ai/dsh-session-query' import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { Remote, RemoteError, TypertRemoteService } from '@deepseek-ai/dsh-typert-protocol' +import { assertNever } from '@deepseek-ai/dsh-util-values' import { gitIn, inspectCheckout, listBranches } from './git.ts' import type { Inspection } from './git.ts' import { parseDiff } from './diff.ts' @@ -334,6 +335,7 @@ export class ForgePanelController extends TypertRemoteService { * @param signal - request cancellation. * @returns the files, the head commit, each file's web page, and the commits the checkout has beyond the head. * @throws `forgePanel/not-bound` unless the URL is a visible binding of the Session that a provider serves; + * `forgePanel/stale-head` when the head moved past the latest status, which is then polled again; * `forgePanel/diff-too-large` when the host or its provider refuses the diff over a size limit; * `forgePanel/diff-failed` before its status was read and when the host does not provide the diff otherwise. */ @@ -349,7 +351,7 @@ export class ForgePanelController extends TypertRemoteService { const version = JSON.stringify([headSha, baseBranch]) let patch = this.patches.get(key, version) if (patch === undefined) { - const read = await this.home.forge.diff(ref, signal) + const read = await this.home.forge.diff(ref, headSha, signal) if (typeof read !== 'string') throw readError(read) this.patches.set(key, version, read) patch = read @@ -719,10 +721,19 @@ export class ForgePanelController extends TypertRemoteService { /** * Report why the code host did not provide a diff or file. * @param failure - the provider's failure. - * @returns `forgePanel/diff-too-large` for an answer over a size limit, else `forgePanel/diff-failed`, with the provider's message. + * @returns `forgePanel/diff-too-large` for an answer over a size limit, `forgePanel/stale-head` for a head that moved, + * else `forgePanel/diff-failed`, with the provider's message. */ function readError(failure: ForgeFailure): RemoteError { - return new RemoteError(failure.code === 'too-large' ? 'forgePanel/diff-too-large' : 'forgePanel/diff-failed', failure.message, {}) + switch (failure.code) { + case 'too-large': return new RemoteError('forgePanel/diff-too-large', failure.message, {}) + case 'stale': return new RemoteError('forgePanel/stale-head', failure.message, {}) + case 'not-found': + case 'unauthenticated': + case 'unavailable': return new RemoteError('forgePanel/diff-failed', failure.message, {}) + /* v8 ignore next -- closed ForgeFailure code union exhaustiveness guard */ + default: return assertNever(failure.code, 'ForgeFailure.code') + } } /** diff --git a/packages/experimental/client-ui-forge/src/types.ts b/packages/experimental/client-ui-forge/src/types.ts index 2a11e67ac6..ace12522bb 100644 --- a/packages/experimental/client-ui-forge/src/types.ts +++ b/packages/experimental/client-ui-forge/src/types.ts @@ -388,7 +388,11 @@ declare module '@deepseek-ai/dsh-typert-protocol' { 'forgePanel/restore-failed': Record /** The URL is not a visible binding of the Session that a provider serves, so the panel does not act on it. */ 'forgePanel/not-bound': Record - /** The change request has new commits since the head the user confirmed, so it was not merged. */ + /** + * The change request's head is no longer the commit the request named: the head the user confirmed for a merge, + * the head displayed for complete checks, or the head of the latest status for a diff. A diff read polls the change + * request again at once, so a later status names the current head. + */ 'forgePanel/stale-head': Record /** The code host refused the change or could not be reached; the message gives the host's reason. */ 'forgePanel/action-failed': Record diff --git a/packages/experimental/client-ui-forge/tests/change-request-card.client.spec.tsx b/packages/experimental/client-ui-forge/tests/change-request-card.client.spec.tsx index 7ef34076a5..e7305ff6d1 100644 --- a/packages/experimental/client-ui-forge/tests/change-request-card.client.spec.tsx +++ b/packages/experimental/client-ui-forge/tests/change-request-card.client.spec.tsx @@ -261,6 +261,8 @@ describe('expanded ChangeRequestCard', () => { ['an approval with unresolved threads', { reviewDecision: 'approved', unresolvedThreads: 2 }, 'Approved · 2 unresolved', 'success'], ['requested changes with an unresolved thread', { reviewDecision: 'changes-requested', unresolvedThreads: 1 }, 'Changes requested · 1 unresolved', 'error'], + ['more threads than the provider counted', { reviewDecision: 'review-required', unresolvedThreads: 1200, threadsTruncated: true }, + 'Review required · 1,200+ unresolved', null], ] as const)('states the review of %s', (_label, overrides, text, tone) => { const value = rowValue(card(entry(1, snapshot(1, overrides)), { expanded: true }).item, en['change.row.review']) expect(value.textContent).toBe(text) diff --git a/packages/experimental/client-ui-forge/tests/changes-tab.client.spec.tsx b/packages/experimental/client-ui-forge/tests/changes-tab.client.spec.tsx index eaa90722ce..1abfc93d1a 100644 --- a/packages/experimental/client-ui-forge/tests/changes-tab.client.spec.tsx +++ b/packages/experimental/client-ui-forge/tests/changes-tab.client.spec.tsx @@ -124,6 +124,7 @@ const REFUSED = { tooLarge: { ok: false, error: new RemoteError('forgePanel/diff-too-large', 'too large', {}) }, filterConfig: { ok: false, error: new RemoteError('forgePanel/filter-config-failed', 'filter configuration unavailable', {}) }, gone: { ok: false, error: new RemoteError('forgePanel/diff-failed', 'gone', {}) }, + staleHead: { ok: false, error: new RemoteError('forgePanel/stale-head', 'the head moved', {}) }, hostRefused: { ok: false, error: new RemoteError('forgePanel/diff-failed', 'diff too large', {}) }, failed: { ok: false, error: new RemoteError('gateway/bad-request', 'failed', {}) }, } satisfies Record> @@ -1094,6 +1095,41 @@ describe('ChangesTab change requests', () => { expect(m.faces.changeRequestDiff).toHaveBeenCalledTimes(2) }) + it('keeps loading after the head moved past its status and reads again once a later status arrives', async () => { + const m = mount({ source, setup: (faces) => { faces.changeRequestDiff.mockResolvedValueOnce(REFUSED.staleHead) } }) + await settle() + expect(m.faces.changeRequestDiff).toHaveBeenCalledOnce() + expect(m.view.queryByRole('status')).toBeNull() + expect(m.paths()).toEqual([]) + + // A later status with the same head reads again, as when the host's answers disagreed for a moment. + const later = (fetchedAt: number, headSha: string) => { + const polled = entry(7, snapshot(7, { headSha })) + return ready({ changeRequests: [{ ...polled, update: { ...polled.update!, fetchedAt } }] }) + } + m.faces.changeRequestDiff.mockResolvedValueOnce(REFUSED.staleHead) + act(() => { m.panel.set({ state: later(2, HEAD), disconnected: false }) }) + await settle() + expect(m.faces.changeRequestDiff).toHaveBeenCalledTimes(2) + expect(m.view.queryByRole('status')).toBeNull() + + act(() => { m.panel.set({ state: later(3, 'b'.repeat(40)), disconnected: false }) }) + await settle() + expect(m.faces.changeRequestDiff).toHaveBeenCalledTimes(3) + expect(m.paths()).toEqual(['docs/gone.txt', 'docs/notes.txt', 'src/app.ts']) + }) + + it('keeps shown changes while a read waits for a later status, turning the refresh glyph', async () => { + const m = mount({ source }) + await settle() + m.faces.changeRequestDiff.mockResolvedValueOnce(REFUSED.staleHead) + fireEvent.click(m.view.getByRole('button', { name: en['diff.refresh'] })) + await settle() + expect(m.paths()).toEqual(['docs/gone.txt', 'docs/notes.txt', 'src/app.ts']) + expect(m.view.queryByText(en['diff.error'])).toBeNull() + expect(m.view.getByRole('button', { name: en['diff.refresh'] }).querySelector('[data-turning]')).not.toBeNull() + }) + it.each([ ['unpushed commits it cannot count', null], ['no unpushed commits', 0], diff --git a/packages/experimental/client-ui-forge/tests/client-fixtures.client.ts b/packages/experimental/client-ui-forge/tests/client-fixtures.client.ts index 55bcc141af..dad26b57b0 100644 --- a/packages/experimental/client-ui-forge/tests/client-fixtures.client.ts +++ b/packages/experimental/client-ui-forge/tests/client-fixtures.client.ts @@ -39,6 +39,7 @@ export function snapshot(number: number, overrides: Partial { expect(opened.provider.diffs).toEqual([7, 7, 7, 7]) }) + it('never labels a diff read after a push with the head of the earlier status', async () => { + const { repo } = await project() + const opened = await open(repo) + const [polledHead, pushedHead] = ['a'.repeat(40), 'b'.repeat(40)] + const patches = new Map([[polledHead, PATCH], [pushedHead, PATCH.replaceAll('src/a.ts', 'src/b.ts')]]) + // The host serves the head of the moment it answers, as GitHub's pull request diff does. + let hostHead = polledHead + const reads: string[] = [] + Object.assign(opened.provider, { + diff: async (_ref: unknown, headSha: string): Promise => { + reads.push(headSha) + return headSha === hostHead ? patches.get(hostHead)! : { code: 'stale', message: `head moved to ${hostHead}` } + }, + }) + shellTurn(opened.session, 1, [{ callId: 'create', command: 'gh pr create --fill', output: `${pullUrl(7)}\n` }]) + const follower = reader(opened) + await follower.until(state => polled(state, pullUrl(7))) + hostHead = pushedHead + opened.provider.overrides.set(7, { headSha: pushedHead }) + const fetches = opened.provider.fetches.length + + await expect(opened.ctx.forgePanel.changeRequestDiff(opened.session, pullUrl(7), signal)) + .rejects.toMatchObject({ code: 'forgePanel/stale-head', message: `head moved to ${pushedHead}` }) + // The refused read polls again at once, without waiting for the idle interval. + await follower.until(state => state.changeRequests[0]?.snapshot?.headSha === pushedHead) + expect(opened.provider.fetches.length).toBe(fetches + 1) + const diff = await opened.ctx.forgePanel.changeRequestDiff(opened.session, pullUrl(7), signal) + expect([diff.headSha, diff.files.map(file => file.path)]).toEqual([pushedHead, ['src/b.ts']]) + expect(reads).toEqual([polledHead, pushedHead]) + }) + it('reports a diff or file over a size limit as too large', async () => { const { repo } = await project() const opened = await open(repo) diff --git a/packages/experimental/client-ui-forge/tests/harness.ts b/packages/experimental/client-ui-forge/tests/harness.ts index 652de1a153..02f1c8cb79 100644 --- a/packages/experimental/client-ui-forge/tests/harness.ts +++ b/packages/experimental/client-ui-forge/tests/harness.ts @@ -262,6 +262,7 @@ export function snapshot(ref: ChangeRequestRef): ChangeRequestSnapshot { checksRollup: 'success', checksTruncated: false, unresolvedThreads: 0, + threadsTruncated: false, updatedAt: '2026-10-09T00:00:00Z', closedAt: null, } @@ -285,6 +286,8 @@ export class FakeProvider implements ForgeProvider { readonly patches = new Map() /** Change request numbers in the order their diffs were read. */ readonly diffs: number[] = [] + /** Head commits the diff reads named, in the same order. */ + readonly diffHeads: string[] = [] /** File texts or failures by `:`; others return a not-found failure. */ readonly texts = new Map() /** Open change request numbers that a head lookup returns, by head branch. */ @@ -333,8 +336,9 @@ export class FakeProvider implements ForgeProvider { return Promise.resolve(this.actionResult) } - diff(ref: ChangeRequestRef): Promise { + diff(ref: ChangeRequestRef, headSha: string): Promise { this.diffs.push(ref.number) + this.diffHeads.push(headSha) return Promise.resolve(this.patches.get(ref.number) ?? { code: 'not-found', message: `no diff of #${ref.number}` }) } diff --git a/packages/experimental/forge-github/README.i18n.yaml b/packages/experimental/forge-github/README.i18n.yaml index b096b71898..5b14a27a65 100644 --- a/packages/experimental/forge-github/README.i18n.yaml +++ b/packages/experimental/forge-github/README.i18n.yaml @@ -15,8 +15,8 @@ en: d152484eb41ac6b4 zh: 2f8a621b991260a1 /deepseek-ai-dsh-experimental-forge-github/use-this-package: - en: f2bd06d91b744305 - zh: ad956b539a43b880 + en: 3eece595e10bc32e + zh: fe3c5cf59f9f17d4 /deepseek-ai-dsh-experimental-forge-github/understand-the-implementation: en: 90f3c10096be642a zh: 4e1b663c52019ac2 @@ -30,8 +30,8 @@ en: 26b946af4b742e17 zh: 00b34f0721ac72f6 /deepseek-ai-dsh-experimental-forge-github/known-limitations-and-deferred-work: - en: 37308d9458bb9316 - zh: 3d4188f3e94af282 + en: 627b11577406b029 + zh: 6596e545c1c5a6fd /deepseek-ai-dsh-experimental-forge-github/known-limitations-and-deferred-work/dev-note: en: e67f633dc6fe7773 zh: 3bb0c20c93f16f98 diff --git a/packages/experimental/forge-github/README.md b/packages/experimental/forge-github/README.md index 2d9e2ce57a..643563baa4 100644 --- a/packages/experimental/forge-github/README.md +++ b/packages/experimental/forge-github/README.md @@ -34,17 +34,19 @@ Mount the provider after the forge registry and list every GitHub Enterprise hos hosts: [github.com, ghe.example.com] ``` -The provider runs `gh` as a host process, even when Sessions execute commands in another execution world. Sign in once per host with `gh auth login --hostname `. The child receives the host environment without `DSH_*` variables and without variables whose names contain `KEY`, `PASSWORD`, `SECRET`, or `TOKEN`, so `GH_TOKEN` and `GITHUB_TOKEN` are not forwarded and `gh` uses its own credential store. Prompts, update notices, pagers, and colors are disabled. `maxOutputBytes` must be a positive safe integer; only `timeoutMs` is bounded by the runtime timer limit. `ghCommand`, `hosts`, `timeoutMs`, and `maxOutputBytes` are listed in the [configuration catalog](../../../docs/config-catalog.md#deepseek-aidsh-experimental-forge-github). +The provider runs `gh` as a host process, even when Sessions execute commands in another execution world. Sign in once per host with `gh auth login --hostname `. The child receives the host environment without `DSH_*` variables and without variables whose names contain `KEY`, `PASSWORD`, `SECRET`, or `TOKEN`, so `GH_TOKEN` and `GITHUB_TOKEN` are not forwarded and `gh` uses its own credential store. Prompts, update notices, pagers, and colors are disabled. `maxOutputBytes` and `maxReviewThreads` must be positive safe integers; only `timeoutMs` is bounded by the runtime timer limit. `ghCommand`, `hosts`, `timeoutMs`, `maxOutputBytes`, and `maxReviewThreads` are listed in the [configuration catalog](../../../docs/config-catalog.md#deepseek-aidsh-experimental-forge-github). The provider serves a remote URL in `https`, `http`, `ssh`, or `git` form, or in scp form such as `git@github.com:owner/name.git`, when its host is configured. HTTP(S) remotes retain custom ports in the host identity; SSH and Git transport ports are excluded. It serves pull request URLs of the form `https://///pull/` with any trailing path, query, or fragment, and builds that form without the suffix as the canonical URL. A readiness probe runs `gh auth status --hostname `: exit status 0 is `ready`, another exit status is `unauthenticated` with the first diagnostic line, and a `gh` that cannot start is `unavailable`. A head lookup returns up to 20 open pull requests whose head is the branch, newest first, excluding pull requests from forks; a lookup of the default branch returns none. Polling returns the first 100 head-commit check contexts without following pages. An incomplete preview sets `checksTruncated` and keeps GitHub’s aggregate state; its counts do not describe the full list. `fetchChecks(ref, signal)` reads complete details only for the requested PR, pinning continuation pages to the first response’s rollup. The combined UTF-8 JSON responses must fit `maxOutputBytes`; excess returns `too-large`. Failed or non-advancing pages fail only this detail read and leave polling results intact. Runs with the same application, workflow, event, and check name keep only the newest run id; a queued replacement supersedes an older completed run. Runs without a numeric database id are retained by node id because their relative age is unknown. Commit status contexts keep the newest result per context name. Complete lists derive their indicator from these results; unknown context kinds retain GitHub’s aggregate state. +Every status read counts review threads in pages of up to 100 until it has counted `maxReviewThreads` (default 1,000) of them; each page after the first is one more GraphQL query. A pull request with more threads sets `threadsTruncated`, and its `unresolvedThreads` counts only the threads read. A thread page that fails or does not advance fails that pull request's status read and leaves the other pull requests of the batch intact. + Unfinished check runs and completed runs with stale, missing, or unrecognized conclusions map to `pending`. Other completed checks map to `success`, `failure` (failure, timed out, startup failure, or action required), `cancelled`, `skipped`, or `neutral`. Cancelled checks keep the rollup non-passing. Check runs keep their start and completion times. Commit status contexts map `SUCCESS` to `success`, `FAILURE` and `ERROR` to `failure`, and every other state to `pending`, and carry no times. A GraphQL `NOT_FOUND` error becomes a `not-found` failure. A response without GraphQL data or errors, including an HTTP error JSON body, uses the first stderr diagnostic or the body's message. Authentication diagnostics become `unauthenticated`; oversized check pages become `too-large`; rate limits and other failures become `unavailable`. Head lookups preserve the same diagnostic in their rejection. A pull request is `mergeReady` when it is open, not a draft, GitHub reports its merge state as `CLEAN`, `HAS_HOOKS`, or `UNSTABLE`, and the signed-in user has write, maintain, or admin permission. `mergeMethods` lists the squash, merge-commit, and rebase methods the repository allows, the user's default merge method first. Marking ready runs `gh pr ready --repo //`. Merging calls `PUT repos///pulls//merge` through `gh api` with the method and the expected head SHA, so GitHub refuses once new commits arrived, which is reported as `stale`. Unlike `gh pr merge`, it never enables auto-merge, never joins a merge queue, and never deletes the head branch. A refused action reports GitHub's message, and a diagnostic that mentions authentication becomes `unauthenticated`. -`diff` runs `gh pr diff --repo // --color never`. `fileText` reads `repos///contents/?ref=` through `gh api` with the raw media type and accepts only a full 40- or 64-digit hexadecimal commit. `fileUrl` returns `https://///blob//`. Both paths percent-encode each segment. A refused read becomes `not-found` for HTTP 404, `unauthenticated` for a diagnostic that mentions authentication, `too-large` for GitHub's `too_large` error, which a diff over its own size limit draws, and `unavailable` otherwise, with GitHub's message. A read whose output exceeds `maxOutputBytes` becomes `too-large`, and one that outlives `timeoutMs` becomes `unavailable`. +`diff` reads the head commit through GraphQL, runs `gh pr diff --repo // --color never`, and reads the head commit again; it returns the patch only when both reads name the requested commit, and `stale` otherwise. The requested commit must be a full 40- or 64-digit hexadecimal commit. `gh pr diff` takes no commit argument and answers at the head of the moment it runs, so the two reads bracket it. The REST compare endpoint, which names commits explicitly, is not used: compared with the base branch it is empty for a merged pull request, compared with `baseRefOid` it starts from the base commit GitHub recorded at the pull request's last update instead of the current base tip, and it returns diffs over 300 files that the pull request diff refuses. `fileText` reads `repos///contents/?ref=` through `gh api` with the raw media type and accepts only a full 40- or 64-digit hexadecimal commit. `fileUrl` returns `https://///blob//`. Both paths percent-encode each segment. A refused read becomes `not-found` for HTTP 404, `unauthenticated` for a diagnostic that mentions authentication, `too-large` for GitHub's `too_large` error, which a diff over its own size limit draws, and `unavailable` otherwise, with GitHub's message. A read whose output exceeds `maxOutputBytes` becomes `too-large`, and one that outlives `timeoutMs` becomes `unavailable`. Review submission checks the current pull request head, then posts a single inline comment or one completed review through the REST API with an explicit `commit_id`. Multiline comments carry their side and inclusive start and end lines. Review decisions are Comment, Approve, and Request changes. JSON request bodies travel on stdin. The provider never retries a mutation: a timeout, unexpected response, or server error after dispatch returns an unknown outcome that the caller must reconcile on GitHub. A commit arriving between the check and the write can make the explicitly addressed review outdated; GitHub does not offer an atomic head precondition on these endpoints. @@ -93,9 +95,9 @@ None. Provider output reaches only host-side watchers. - **GitHub CLI on the host** — `gh` must be installed and signed in for each configured host; tokens in environment variables are not forwarded. Hosts missing from `hosts` are never read, including references retained across configuration changes. A status fetch returns `unavailable` for those references; a head lookup rejects before starting `gh`. -- **First 100 review threads** — unresolved threads after the first 100 are not counted. +- **Review thread bound** — a status read counts at most `maxReviewThreads` review threads per pull request; beyond it `unresolvedThreads` is a lower bound that `threadsTruncated` marks. - **No fork discovery** — head lookups exclude pull requests whose head is in another repository. -- **Rate limits** — every poll costs one GraphQL query per host, plus one query per additional page of 100 check records, against the signed-in account’s rate limit. +- **Rate limits** — every poll costs one GraphQL query per host, plus one query per further page of 100 review threads, and a complete check read one more query per further page of 100 check records; every diff read adds two head reads. All count against the signed-in account’s rate limit. - **No merge queue** — a base branch that requires a merge queue refuses a direct merge, and the provider never queues one. diff --git a/packages/experimental/forge-github/README.zh.md b/packages/experimental/forge-github/README.zh.md index 1ac57ffa7d..404567565d 100644 --- a/packages/experimental/forge-github/README.zh.md +++ b/packages/experimental/forge-github/README.zh.md @@ -34,17 +34,19 @@ kind: "package-reference" hosts: [github.com, ghe.example.com] ``` -即使 Session 在其他执行环境中运行命令，提供方也以宿主进程的形式运行 `gh`。每个主机执行一次 `gh auth login --hostname ` 完成登录。子进程继承的宿主环境去掉了 `DSH_*` 变量以及名称包含 `KEY`、`PASSWORD`、`SECRET` 或 `TOKEN` 的变量，因此 `GH_TOKEN` 与 `GITHUB_TOKEN` 不会被转发，`gh` 使用自己的凭据存储。交互提示、更新提醒、分页器与颜色输出均被关闭。`maxOutputBytes` 必须是正安全整数；只有 `timeoutMs` 受运行时定时器上限约束。`ghCommand`、`hosts`、`timeoutMs` 与 `maxOutputBytes` 见[配置目录](../../../docs/config-catalog.zh.md#deepseek-aidsh-experimental-forge-github)。 +即使 Session 在其他执行环境中运行命令，提供方也以宿主进程的形式运行 `gh`。每个主机执行一次 `gh auth login --hostname ` 完成登录。子进程继承的宿主环境去掉了 `DSH_*` 变量以及名称包含 `KEY`、`PASSWORD`、`SECRET` 或 `TOKEN` 的变量，因此 `GH_TOKEN` 与 `GITHUB_TOKEN` 不会被转发，`gh` 使用自己的凭据存储。交互提示、更新提醒、分页器与颜色输出均被关闭。`maxOutputBytes` 与 `maxReviewThreads` 必须是正安全整数；只有 `timeoutMs` 受运行时定时器上限约束。`ghCommand`、`hosts`、`timeoutMs`、`maxOutputBytes` 与 `maxReviewThreads` 见[配置目录](../../../docs/config-catalog.zh.md#deepseek-aidsh-experimental-forge-github)。 当主机在配置中时，提供方服务 `https`、`http`、`ssh`、`git` 形式的远端 URL，以及 `git@github.com:owner/name.git` 这类 scp 形式。HTTP(S) 远端在主机标识中保留自定义端口；SSH 和 Git 传输端口不计入主机标识。它服务 `https://///pull/` 形式的 pull request URL，允许其后附带任意路径、查询或片段，并以去掉这些后缀的形式作为规范 URL。就绪探测运行 `gh auth status --hostname `：退出码为 0 时为 `ready`，其他退出码为 `unauthenticated` 并带第一行诊断，`gh` 无法启动时为 `unavailable`。按 head 分支查找时，最多返回 20 个以该分支为 head 的打开状态 pull request，按创建时间从新到旧排列，不包含来自 fork 的 pull request；查找默认分支时返回空结果。 轮询只返回 head 提交的前 100 条检查上下文，不追读后续页。预览不完整时设置 `checksTruncated` 并保留 GitHub 的汇总状态，局部计数不代表完整列表。`fetchChecks(ref, signal)` 仅按需读取指定 PR 的完整详情，后续页固定使用首次响应的 rollup。所有 JSON 响应的 UTF-8 字节总量受 `maxOutputBytes` 限制，超出时返回 `too-large`。分页失败或游标不前进只使此次详情读取失败，保留轮询结果。同一应用、工作流、触发事件与检查名称的运行只保留运行 id 最新的一条；新的排队运行会替代旧的已完成运行。缺少数字数据库 id 的运行无法确定先后，因此按节点 id 保留。提交状态上下文按名称保留最新结果。完整列表从这些结果计算总体指示；未知上下文类型保留 GitHub 的汇总状态。 +每次状态读取按每页最多 100 个统计评审讨论，直到已统计 `maxReviewThreads`（默认 1,000）个为止；第一页之后每读一页多一次 GraphQL 查询。讨论更多的 pull request 设置 `threadsTruncated`，其 `unresolvedThreads` 只统计已读取的讨论。讨论分页失败或游标不前进时，该 pull request 的状态读取失败，同批其他 pull request 不受影响。 + 未完成的检查运行（check run），以及结论已过期、缺失或无法识别的已完成运行，均映射为 `pending`。其他已完成的检查映射为 `success`、`failure`（失败、超时、启动失败或需要操作）、`cancelled`、`skipped` 或 `neutral`。已取消的检查使汇总保持未通过状态。检查运行保留其开始与完成时间。提交状态上下文把 `SUCCESS` 映射为 `success`，把 `FAILURE` 与 `ERROR` 映射为 `failure`，其余状态均映射为 `pending`，且不带时间。GraphQL 的 `NOT_FOUND` 错误转为 `not-found` 失败。响应不含 GraphQL data 或 errors 时（包括 HTTP 错误 JSON），使用 stderr 的首条诊断或响应体的 message。认证诊断转为 `unauthenticated`；超出输出大小上限的检查分页转为 `too-large`；限流与其他失败转为 `unavailable`。head 查找在拒绝时保留相同诊断。 pull request 处于打开状态、不是草稿、GitHub 报告其合并状态为 `CLEAN`、`HAS_HOOKS` 或 `UNSTABLE`，且已登录用户具有 write、maintain 或 admin 权限时，`mergeReady` 为真。`mergeMethods` 列出仓库允许的压缩合并、合并提交与变基合并方式，用户默认的合并方式排在最前。标记为可评审运行 `gh pr ready --repo //`。合并经 `gh api` 调用 `PUT repos///pulls//merge`，带上合并方式与预期的 head SHA，因此有新提交到达后 GitHub 会拒绝，报告为 `stale`。与 `gh pr merge` 不同，它从不启用自动合并、从不加入合并队列，也从不删除 head 分支。被拒绝的操作报告 GitHub 的原因，提到身份验证的诊断转为 `unauthenticated`。 -`diff` 运行 `gh pr diff --repo // --color never`。`fileText` 经 `gh api` 以 raw 媒体类型读取 `repos///contents/?ref=`，只接受完整的 40 位或 64 位十六进制提交。`fileUrl` 返回 `https://///blob//`。两者的路径都逐段做百分号编码。被拒绝的读取在 HTTP 404 时为 `not-found`，诊断提到身份验证时为 `unauthenticated`，GitHub 报告 `too_large` 错误时（diff 超过其自身大小上限即会如此）为 `too-large`，其余为 `unavailable`，并带有 GitHub 的原因。输出超过 `maxOutputBytes` 的读取为 `too-large`，超过 `timeoutMs` 的为 `unavailable`。 +`diff` 先经 GraphQL 读取 head 提交，再运行 `gh pr diff --repo // --color never`，然后再读一次 head 提交；两次读取都是请求的提交时才返回补丁，否则返回 `stale`。请求的提交必须是完整的 40 位或 64 位十六进制提交。`gh pr diff` 不接受提交参数，返回运行那一刻 head 的结果，因此用前后两次读取把它夹住。不使用显式指定提交的 REST compare 接口：与基准分支比较时，已合并的 pull request 得到空结果；与 `baseRefOid` 比较时，起点是 GitHub 在 pull request 最后一次更新时记录的基准提交，而不是当前的基准分支末端；它还会返回 pull request diff 拒绝的超过 300 个文件的 diff。`fileText` 经 `gh api` 以 raw 媒体类型读取 `repos///contents/?ref=`，只接受完整的 40 位或 64 位十六进制提交。`fileUrl` 返回 `https://///blob//`。两者的路径都逐段做百分号编码。被拒绝的读取在 HTTP 404 时为 `not-found`，诊断提到身份验证时为 `unauthenticated`，GitHub 报告 `too_large` 错误时（diff 超过其自身大小上限即会如此）为 `too-large`，其余为 `unavailable`，并带有 GitHub 的原因。输出超过 `maxOutputBytes` 的读取为 `too-large`，超过 `timeoutMs` 的为 `unavailable`。 提交评审时先检查 PR 当前的 head，再通过 REST API 发布单条行内评论或一次完整评审，并显式指定 `commit_id`。多行评论携带所属侧及包含端点的起止行号。评审结论为“评论”“批准”或“请求更改”。JSON 请求正文通过标准输入传递。provider 不会重试写操作；发送后的超时、异常响应或服务端错误会返回结果不明，由调用方到 GitHub 核对。检查与写入之间到来的新提交可能让已明确定位到旧提交的评审过时；GitHub 的这些接口不提供原子的 head 前置条件。 @@ -93,9 +95,9 @@ pull request 处于打开状态、不是草稿、GitHub 报告其合并状态为 - **宿主需要 GitHub CLI**——必须安装 `gh` 并为每个已配置主机登录；环境变量中的令牌不会被转发。不在 `hosts` 中的主机永远不会被读取，包括配置变更后保留的引用。状态读取为这些引用返回 `unavailable`；head 查找会在启动 `gh` 前拒绝。 -- **前 100 个评审讨论**——前 100 个之后的未解决讨论不计入。 +- **评审讨论上限**——每次状态读取每个 pull request 最多统计 `maxReviewThreads` 个评审讨论；超出时 `unresolvedThreads` 只是下限，由 `threadsTruncated` 标出。 - **不发现 fork**——按 head 分支查找时，不包含 head 位于其他仓库的 pull request。 -- **速率限制**——每次轮询以已登录账号的速率配额为每个主机消耗一次 GraphQL 查询；检查记录每多一页（100 条），再消耗一次查询。 +- **速率限制**——每次轮询为每个主机消耗一次 GraphQL 查询，评审讨论每多一页（100 个）再消耗一次；完整检查读取时检查记录每多一页（100 条）再消耗一次；每次 diff 读取另加两次 head 读取。这些都计入已登录账号的速率配额。 - **不支持合并队列**——要求合并队列的基准分支会拒绝直接合并，提供方也不会把变更请求加入队列。 diff --git a/packages/experimental/forge-github/src/graphql.ts b/packages/experimental/forge-github/src/graphql.ts index a7d0a8d16b..cea358e1a0 100644 --- a/packages/experimental/forge-github/src/graphql.ts +++ b/packages/experimental/forge-github/src/graphql.ts @@ -19,17 +19,34 @@ const CHECK_FIELDS = `pageInfo { hasNextPage endCursor } ... on StatusContext { context state targetUrl createdAt } }` -const PULL_REQUEST_FIELDS = `number url title state isDraft updatedAt closedAt +/** Most nodes GitHub returns in one connection page. */ +const PAGE_SIZE = 100 + +const THREAD_FIELDS = 'pageInfo { hasNextPage endCursor } nodes { isResolved }' + +/** + * Pull request fields of one snapshot. + * @param threads - review threads the first page holds, at most {@link PAGE_SIZE}. + * @returns the selection of the snapshot fragment. + */ +function pullRequestFields(threads: number): string { + return `id number url title state isDraft updatedAt closedAt author { login } headRefName headRefOid baseRefName additions deletions mergeable mergeStateStatus reviewDecision - reviewThreads(first: 100) { nodes { isResolved } } - commits(last: 1) { nodes { commit { statusCheckRollup { id state contexts(first: 100) { + reviewThreads(first: ${threads}) { ${THREAD_FIELDS} } + commits(last: 1) { nodes { commit { statusCheckRollup { id state contexts(first: ${PAGE_SIZE}) { ${CHECK_FIELDS} } } } } }` +} /** Continue reading the same commit's rollup even if the pull request head moves. */ export const CHECKS_QUERY = `query($id: ID!, $cursor: String!) { - node(id: $id) { ... on StatusCheckRollup { contexts(first: 100, after: $cursor) { ${CHECK_FIELDS} } } } + node(id: $id) { ... on StatusCheckRollup { contexts(first: ${PAGE_SIZE}, after: $cursor) { ${CHECK_FIELDS} } } } +}` + +/** Continue counting a pull request's review threads; variables `id`, `cursor`, and the integer `first`. */ +export const THREADS_QUERY = `query($id: ID!, $cursor: String!, $first: Int!) { + node(id: $id) { ... on PullRequest { reviewThreads(first: $first, after: $cursor) { ${THREAD_FIELDS} } } } }` const REPOSITORY_FIELDS = 'squashMergeAllowed mergeCommitAllowed rebaseMergeAllowed viewerPermission viewerDefaultMergeMethod' @@ -42,6 +59,11 @@ export const HEAD_QUERY = `query($owner: String!, $name: String!, $branch: Strin } }` +/** Head commit of one pull request; variables `owner` and `name` are strings, `number` is an integer. */ +export const HEAD_COMMIT_QUERY = `query($owner: String!, $name: String!, $number: Int!) { + repository(owner: $owner, name: $name) { pullRequest(number: $number) { headRefOid } } +}` + const checkRunSchema = z.object({ __typename: z.literal('CheckRun'), id: z.string(), @@ -66,19 +88,40 @@ const statusContextSchema = z.object({ targetUrl: z.string().nullable(), }) -const checkPageSchema = z.object({ - pageInfo: z.object({ hasNextPage: z.boolean(), endCursor: z.string().nullable() }), - nodes: z.array(z.unknown()), -}) +const pageInfoSchema = z.object({ hasNextPage: z.boolean(), endCursor: z.string().nullable() }) + +/** Position of a connection page within its connection. */ +type PageInfo = z.infer + +const checkPageSchema = z.object({ pageInfo: pageInfoSchema, nodes: z.array(z.unknown()) }) + +const threadPageSchema = z.object({ pageInfo: pageInfoSchema, nodes: z.array(z.object({ isResolved: z.boolean() })) }) + +/** One validated connection page or a failure that prevents publishing a partial count. */ +export type PageResult = { readonly ok: true; readonly page: T } | { readonly ok: false; readonly failure: ForgeFailure } /** One validated page of check contexts or a failure that prevents publishing a partial count. */ -export type CheckPageResult = { readonly ok: true; readonly page: z.infer } - | { readonly ok: false; readonly failure: ForgeFailure } +export type CheckPageResult = PageResult> + +/** One validated page of review threads or a failure that prevents publishing a partial count. */ +export type ThreadPageResult = PageResult> /** Opaque GitHub node id that pins check pagination to one commit's rollup. */ type RollupId = Branded<'GitHubStatusCheckRollupId'> +/** Opaque GitHub node id of a pull request. */ +type PullRequestId = Branded<'GitHubPullRequestId'> + +/** Continuation reads that complete one snapshot response. */ +export interface SnapshotPages { + /** Read the next page of a pull request's review threads, holding at most `first` threads. */ + readonly threads: (id: PullRequestId, cursor: string, first: number) => Promise + /** Read the next check page of a rollup; omission keeps the first check page with its truncation flag. */ + readonly checks?: (id: RollupId, cursor: string) => Promise +} + const pullRequestSchema = z.object({ + id: z.string().transform(value => value as PullRequestId), url: z.string(), title: z.string(), state: z.enum(['OPEN', 'CLOSED', 'MERGED']), @@ -94,7 +137,7 @@ const pullRequestSchema = z.object({ mergeable: z.string(), mergeStateStatus: z.string(), reviewDecision: z.string().nullable(), - reviewThreads: z.object({ nodes: z.array(z.object({ isResolved: z.boolean() })) }), + reviewThreads: threadPageSchema, commits: z.object({ nodes: z.array(z.object({ commit: z.object({ @@ -143,6 +186,8 @@ const headSchema = z.object({ }), }) +const headCommitSchema = z.object({ data: z.object({ repository: z.object({ pullRequest: z.object({ headRefOid: z.string() }) }) }) }) + type PullRequestNode = z.infer type RepositoryNode = z.infer @@ -169,16 +214,19 @@ export interface SnapshotQuery { readonly document: string /** Change requests keyed by alias path `r.p`. */ readonly targets: ReadonlyMap + /** Most review threads counted per change request. */ + readonly maxReviewThreads: number } /** * Build one query that reads every change request in `refs`, grouping them by * repository. Owners, names, and numbers are validated before they are inlined. * @param refs - change requests on one host. + * @param maxReviewThreads - positive integer bound on the review threads counted per change request. * @returns the document and its alias targets. * @throws when an owner, name, or number cannot be inlined safely. */ -export function snapshotQuery(refs: readonly ChangeRequestRef[]): SnapshotQuery { +export function snapshotQuery(refs: readonly ChangeRequestRef[], maxReviewThreads: number): SnapshotQuery { const repositories = new Map }>() for (const ref of refs) { const { owner, name } = checkedRepository(ref) @@ -201,22 +249,26 @@ export function snapshotQuery(refs: readonly ChangeRequestRef[]): SnapshotQuery const owner = JSON.stringify(repository.owner) fields.push(`r${r}: repository(owner: ${owner}, name: ${JSON.stringify(repository.name)}) { ${REPOSITORY_FIELDS} ${pulls.join(' ')} }`) } - return { document: `query { ${fields.join(' ')} }\nfragment ChangeRequestFields on PullRequest { ${PULL_REQUEST_FIELDS} }`, targets } + const document = `query { ${fields.join(' ')} } +fragment ChangeRequestFields on PullRequest { ${pullRequestFields(Math.min(PAGE_SIZE, maxReviewThreads))} }` + return { document, targets, maxReviewThreads } } /** * Map a snapshot response, including GitHub's partial-error form, into one * result per alias target. Path-scoped errors apply to their repository or * pull request; errors without a path apply to every unresolved target. A - * repository without readable merge settings offers no merge. Polling returns + * repository without readable merge settings offers no merge. Review threads + * are counted across pages up to `query.maxReviewThreads`. Polling returns * the first check page; an explicit detail read can complete its pagination. + * A continuation page that fails or does not advance fails its target. * @param body - parsed JSON printed by `gh api graphql`. * @param query - the query that produced the response. - * @param readChecks - optional detail reader for the same commit; omission returns the first page with its truncation flag. - * @returns a snapshot with its check truncation flag, or a failure, for every target. + * @param pages - continuation reads of review threads and, for a detail read, of checks on the same commit. + * @returns a snapshot with its check and thread truncation flags, or a failure, for every target. */ export async function mapSnapshotResponse( - body: unknown, query: SnapshotQuery, readChecks?: (id: RollupId, cursor: string) => Promise, + body: unknown, query: SnapshotQuery, pages: SnapshotPages, ): Promise> { const results = new Map() const parsed = responseSchema.safeParse(body) @@ -235,23 +287,14 @@ export async function mapSnapshotResponse( const node = pullRequestSchema.safeParse(raw) const settings = repositorySchema.safeParse(parsed.data.data?.[repositoryAlias]) if (node.success) { + const { id, reviewThreads } = node.data const rollup = node.data.commits.nodes[0]?.commit.statusCheckRollup - if (rollup !== undefined && rollup !== null) { - const cursors = new Set() - while (readChecks !== undefined && rollup.contexts.pageInfo.hasNextPage) { - const cursor = rollup.contexts.pageInfo.endCursor - if (cursor === null || cursors.has(cursor)) { - results.set(key, { code: 'unavailable', message: 'GitHub check pagination did not advance' }) - break - } - cursors.add(cursor) - const next = await readChecks(rollup.id, cursor) - if (!next.ok) { results.set(key, next.failure); break } - rollup.contexts.nodes.push(...next.page.nodes) - rollup.contexts.pageInfo = next.page.pageInfo - } - } - if (!results.has(key)) results.set(key, toSnapshot(ref, node.data, settings.success ? settings.data : undefined)) + const { checks } = pages + const failure = await readPages(reviewThreads, query.maxReviewThreads, 'review thread', + async (cursor, first) => await pages.threads(id, cursor, first)) + ?? (rollup === undefined || rollup === null || checks === undefined ? undefined + : await readPages(rollup.contexts, Number.POSITIVE_INFINITY, 'check', async cursor => await checks(rollup.id, cursor))) + results.set(key, failure ?? toSnapshot(ref, node.data, settings.success ? settings.data : undefined)) } else if (error !== undefined) results.set(key, { code: error.type === 'NOT_FOUND' ? 'not-found' : 'unavailable', message: error.message }) else if (raw === null || raw === undefined) results.set(key, { code: 'not-found', message: 'pull request not found' }) else results.set(key, { code: 'unavailable', message: 'unexpected pull request fields' }) @@ -259,21 +302,76 @@ export async function mapSnapshotResponse( return results } +/** + * Append the following pages of a connection until it ends or holds `limit` nodes. + * @param connection - first page, extended in place. + * @param limit - most nodes to hold. + * @param label - what the connection lists, for failure messages. + * @param read - reads the page after `cursor`, holding at most `first` nodes. + * @returns a failure that prevents publishing the partial connection, or undefined once it ended or is full. + */ +async function readPages( + connection: { nodes: T[]; pageInfo: PageInfo }, + limit: number, + label: string, + read: (cursor: string, first: number) => Promise>, +): Promise { + const cursors = new Set() + while (connection.pageInfo.hasNextPage && connection.nodes.length < limit) { + const cursor = connection.pageInfo.endCursor + if (cursor === null || cursors.has(cursor)) return { code: 'unavailable', message: `GitHub ${label} pagination did not advance` } + cursors.add(cursor) + const next = await read(cursor, Math.min(PAGE_SIZE, limit - connection.nodes.length)) + if (!next.ok) return next.failure + connection.nodes.push(...next.page.nodes) + connection.pageInfo = next.page.pageInfo + } + return undefined +} + /** * Validate a continuation response before adding its checks to the snapshot. * @param body - parsed GraphQL response for a rollup node. * @returns the complete page or a failure; partial-error pages are not accepted. */ export function mapCheckPageResponse(body: unknown): CheckPageResult { + return mapPageResponse(body, z.object({ contexts: checkPageSchema }).transform(node => node.contexts), 'check') +} + +/** + * Validate a continuation response of {@link THREADS_QUERY} before counting its review threads. + * @param body - parsed GraphQL response for a pull request node. + * @returns the complete page or a failure; partial-error pages are not accepted. + */ +export function mapThreadPageResponse(body: unknown): ThreadPageResult { + return mapPageResponse(body, z.object({ reviewThreads: threadPageSchema }).transform(node => node.reviewThreads), 'review thread') +} + +/** + * Validate one continuation response of a `node(id:)` query. + * @param body - parsed GraphQL response. + * @param page - schema that maps the node to its connection page. + * @param label - what the connection lists, for failure messages. + * @returns the page, or a failure for an error, a missing node, or unexpected fields. + */ +function mapPageResponse(body: unknown, page: z.ZodType, label: string): PageResult { const parsed = z.object({ - data: z.object({ node: z.object({ contexts: checkPageSchema }).nullable() }).nullable().optional(), + data: z.object({ node: page.nullable() }).nullable().optional(), errors: z.array(z.object({ message: z.string() })).optional(), }).safeParse(body) - if (parsed.success && !parsed.data.errors?.length && parsed.data.data?.node != null) { - return { ok: true, page: parsed.data.data.node.contexts } - } + if (parsed.success && !parsed.data.errors?.length && parsed.data.data?.node != null) return { ok: true, page: parsed.data.data.node } return { ok: false, failure: { code: 'unavailable', - message: parsed.success ? parsed.data.errors?.[0]?.message ?? 'GitHub check page is unavailable' : 'unexpected GitHub check page' } } + message: parsed.success ? parsed.data.errors?.[0]?.message ?? `GitHub ${label} page is unavailable` : `unexpected GitHub ${label} page` } } +} + +/** + * Map a head commit read of {@link HEAD_COMMIT_QUERY}. + * @param body - parsed JSON printed by `gh api graphql`. + * @returns the head commit object name, or undefined when the response does not name one. + */ +export function mapHeadCommitResponse(body: unknown): string | undefined { + const parsed = headCommitSchema.safeParse(body) + return parsed.success ? parsed.data.data.repository.pullRequest.headRefOid : undefined } /** @@ -366,6 +464,7 @@ function toSnapshot( checksRollup: summary, checksTruncated: rollup?.contexts.pageInfo.hasNextPage ?? false, unresolvedThreads: node.reviewThreads.nodes.filter(thread => !thread.isResolved).length, + threadsTruncated: node.reviewThreads.pageInfo.hasNextPage, updatedAt: node.updatedAt, closedAt: node.closedAt, } diff --git a/packages/experimental/forge-github/src/index.ts b/packages/experimental/forge-github/src/index.ts index a0bc66126a..c061b57091 100644 --- a/packages/experimental/forge-github/src/index.ts +++ b/packages/experimental/forge-github/src/index.ts @@ -20,6 +20,8 @@ export interface Config { timeoutMs?: number /** Maximum captured bytes per `gh` output stream and combined JSON bytes of one complete check read. */ maxOutputBytes?: number + /** Most review threads counted per pull request in one status read; a pull request with more reports a lower bound. */ + maxReviewThreads?: number } export const Config: z = z.object({ @@ -27,6 +29,7 @@ export const Config: z = z.object({ hosts: z.array(z.string()).default(['github.com']), timeoutMs: z.number().default(30_000), maxOutputBytes: z.number().default(8_388_608), + maxReviewThreads: z.number().default(1_000), }) const HOST = /^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*(:\d+)?$/i @@ -41,8 +44,8 @@ export function apply(ctx: Context, config: Config): void { if (!Number.isSafeInteger(resolved.timeoutMs) || resolved.timeoutMs <= 0 || resolved.timeoutMs > MAX_TIMER_DELAY_MS) { throw new Error(`forge-github: timeoutMs must be a positive integer no greater than ${MAX_TIMER_DELAY_MS}`) } - if (!Number.isSafeInteger(resolved.maxOutputBytes) || resolved.maxOutputBytes <= 0) { - throw new Error('forge-github: maxOutputBytes must be a positive safe integer') + for (const key of ['maxOutputBytes', 'maxReviewThreads'] as const) { + if (!Number.isSafeInteger(resolved[key]) || resolved[key] <= 0) throw new Error(`forge-github: ${key} must be a positive safe integer`) } - ctx.forge.register(new GitHubProvider(ghRunner(resolved), resolved.hosts, resolved.maxOutputBytes)) + ctx.forge.register(new GitHubProvider(ghRunner(resolved), resolved.hosts, resolved)) } diff --git a/packages/experimental/forge-github/src/provider.ts b/packages/experimental/forge-github/src/provider.ts index ca56473956..16c93acfbc 100644 --- a/packages/experimental/forge-github/src/provider.ts +++ b/packages/experimental/forge-github/src/provider.ts @@ -7,7 +7,11 @@ import type { } from '@deepseek-ai/dsh-experimental-forge' import { GhOutputLimitError, GhTimeoutError, GhUnavailableError } from './gh.ts' import type { GhOutput, GhRunner } from './gh.ts' -import { CHECKS_QUERY, hasGraphqlFields, HEAD_QUERY, mapCheckPageResponse, mapHeadResponse, mapSnapshotResponse, snapshotQuery } from './graphql.ts' +import { + CHECKS_QUERY, hasGraphqlFields, HEAD_COMMIT_QUERY, HEAD_QUERY, mapCheckPageResponse, mapHeadCommitResponse, mapHeadResponse, + mapSnapshotResponse, mapThreadPageResponse, snapshotQuery, THREADS_QUERY, +} from './graphql.ts' +import type { SnapshotPages } from './graphql.ts' import { checkedRepository, parsePullRequestUrl, parseRemoteUrl } from './urls.ts' import { submitGitHubReview } from './review.ts' @@ -25,6 +29,14 @@ const TOO_LARGE = /\btoo_large\b/ /** A full SHA-1 or SHA-256 object name. */ const COMMIT = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/i +/** Bounds on what one read retains or counts. */ +export interface GitHubLimits { + /** Maximum combined JSON bytes retained by an explicit check read. */ + readonly maxOutputBytes: number + /** Most review threads counted per pull request in one status read. */ + readonly maxReviewThreads: number +} + /** Reads GitHub and GitHub Enterprise pull requests through a `gh` runner. */ export class GitHubProvider implements ForgeProvider { readonly forge = 'github' @@ -39,9 +51,9 @@ export class GitHubProvider implements ForgeProvider { /** * @param gh - runner that executes `gh` on the host. * @param hosts - host names this provider serves, such as `github.com` and Enterprise hosts. - * @param maxOutputBytes - maximum combined JSON bytes retained by an explicit check read. + * @param limits - bounds on retained check bytes and counted review threads. */ - constructor(private readonly gh: GhRunner, hosts: readonly string[], private readonly maxOutputBytes: number) { + constructor(private readonly gh: GhRunner, hosts: readonly string[], private readonly limits: GitHubLimits) { this.hosts = new Set(hosts.map(host => host.toLowerCase())) } @@ -93,10 +105,7 @@ export class GitHubProvider implements ForgeProvider { return results.get(changeRequestKey(ref))! } catch (error) { if (signal.aborted) throw signal.reason - if (error instanceof GhUnavailableError || error instanceof GhTimeoutError || error instanceof GhOutputLimitError) { - return { code: error instanceof GhOutputLimitError ? 'too-large' : 'unavailable', message: error.message } - } - throw error + return processFailure(error) } } @@ -140,12 +149,22 @@ export class GitHubProvider implements ForgeProvider { ], signal) } - /** Read the pull request's diff with `gh pr diff`, which GitHub computes against the merge base. */ - async diff(ref: ChangeRequestRef, signal: AbortSignal): Promise { + /** + * Read the pull request's diff with `gh pr diff`, which GitHub computes + * against the merge base at the head of the moment it answers. The head is + * read before and after the diff; the patch is returned only when both reads + * name `headSha`, and a different head returns `stale` without the patch. + */ + async diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise { const { host, owner, name } = checkedRepository(ref) const failure = this.hostFailure(host) if (failure !== undefined) return failure - return await this.read(['pr', 'diff', String(ref.number), '--repo', `${host}/${owner}/${name}`, '--color', 'never'], signal) + if (!COMMIT.test(headSha)) throw new Error(`forge-github: invalid head commit ${JSON.stringify(headSha)}`) + const before = await this.headMismatch(ref, headSha, signal) + if (before !== undefined) return before + const patch = await this.read(['pr', 'diff', String(ref.number), '--repo', `${host}/${owner}/${name}`, '--color', 'never'], signal) + if (typeof patch !== 'string') return patch + return await this.headMismatch(ref, headSha, signal) ?? patch } /** Read one file at a commit through the contents API in its raw form. */ @@ -178,6 +197,23 @@ export class GitHubProvider implements ForgeProvider { } } + /** + * Read the head commit of a pull request. + * @returns undefined when it is `headSha`, `stale` when it is another commit, or why it could not be read. + */ + private async headMismatch(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise { + const { host, owner, name } = ref.repository + const output = await this.read([ + 'api', 'graphql', '--hostname', host, + '-f', `query=${HEAD_COMMIT_QUERY}`, '-f', `owner=${owner}`, '-f', `name=${name}`, '-F', `number=${ref.number}`, + ], signal) + if (typeof output !== 'string') return output + const head = mapHeadCommitResponse(parseJson(output)) + if (head === undefined) return { code: 'unavailable', message: 'unexpected GitHub response to the head commit read' } + if (head.toLowerCase() === headSha.toLowerCase()) return undefined + return { code: 'stale', message: `the pull request head moved from ${headSha} to ${head}` } + } + /** * Run one reading `gh` command, returning its output. A refusal, a `gh` that * cannot start, output over `maxOutputBytes`, and a run past `timeoutMs` become failures. @@ -232,34 +268,61 @@ export class GitHubProvider implements ForgeProvider { signal: AbortSignal, completeChecks = false, ): Promise> { - const query = snapshotQuery(refs) + const query = snapshotQuery(refs, this.limits.maxReviewThreads) const output = await this.gh(['api', 'graphql', '--hostname', host, '-f', `query=${query.document}`], signal) const body = parseJson(output.stdout) - if (!completeChecks && hasGraphqlFields(body)) return await mapSnapshotResponse(body, query) + const threads: SnapshotPages['threads'] = async (id, cursor, first) => { + const page = await this.continuation(host, + ['-f', `query=${THREADS_QUERY}`, '-f', `id=${id}`, '-f', `cursor=${cursor}`, '-F', `first=${first}`], signal) + if (!('stdout' in page)) return { ok: false, failure: page } + const value = parseJson(page.stdout) + return hasGraphqlFields(value) ? mapThreadPageResponse(value) : { ok: false, failure: responseFailure(page, value) } + } + if (!completeChecks && hasGraphqlFields(body)) return await mapSnapshotResponse(body, query, { threads }) let bytes = Buffer.byteLength(output.stdout) const tooLarge: ForgeFailure = { code: 'too-large', message: 'GitHub check details exceed the output size limit' } - if (bytes > this.maxOutputBytes) return new Map([...query.targets.values()].map(ref => [changeRequestKey(ref), tooLarge])) - if (hasGraphqlFields(body)) return await mapSnapshotResponse(body, query, async (id, cursor) => { - signal.throwIfAborted() - let page: GhOutput - try { - page = await this.gh(['api', 'graphql', '--hostname', host, - '-f', `query=${CHECKS_QUERY}`, '-f', `id=${id}`, '-f', `cursor=${cursor}`], signal) - } catch (error) { - if (signal.aborted) throw signal.reason - if (error instanceof GhUnavailableError || error instanceof GhTimeoutError || error instanceof GhOutputLimitError) { - return { ok: false, failure: { code: error instanceof GhOutputLimitError ? 'too-large' : 'unavailable', message: error.message } } - } - throw error - } + if (bytes > this.limits.maxOutputBytes) return new Map([...query.targets.values()].map(ref => [changeRequestKey(ref), tooLarge])) + if (hasGraphqlFields(body)) return await mapSnapshotResponse(body, query, { threads, checks: async (id, cursor) => { + const page = await this.continuation(host, ['-f', `query=${CHECKS_QUERY}`, '-f', `id=${id}`, '-f', `cursor=${cursor}`], signal) + if (!('stdout' in page)) return { ok: false, failure: page } bytes += Buffer.byteLength(page.stdout) - if (bytes > this.maxOutputBytes) return { ok: false, failure: tooLarge } + if (bytes > this.limits.maxOutputBytes) return { ok: false, failure: tooLarge } const value = parseJson(page.stdout) return hasGraphqlFields(value) ? mapCheckPageResponse(value) : { ok: false, failure: responseFailure(page, value) } - }) + } }) const failure = responseFailure(output, body) return new Map([...query.targets.values()].map(ref => [changeRequestKey(ref), failure] as const)) } + + /** + * Run one continuation query of a snapshot read. + * @param host - host the snapshot was read from. + * @param variables - `gh api graphql` field flags, the query first. + * @param signal - cancellation of the snapshot read. + * @returns the output, or the failure of a `gh` that cannot start, times out, or prints over its limit. + * @throws with `signal.reason` when the read is cancelled. + */ + private async continuation(host: string, variables: readonly string[], signal: AbortSignal): Promise { + signal.throwIfAborted() + try { + return await this.gh(['api', 'graphql', '--hostname', host, ...variables], signal) + } catch (error) { + if (signal.aborted) throw signal.reason + return processFailure(error) + } + } +} + +/** + * Report a `gh` invocation that did not run to an exit code. + * @param error - what the runner threw. + * @returns `too-large` for output over its limit, `unavailable` for a `gh` that cannot start or times out. + * @throws `error` when it is any other failure. + */ +function processFailure(error: unknown): ForgeFailure { + if (error instanceof GhOutputLimitError) return { code: 'too-large', message: error.message } + if (error instanceof GhUnavailableError || error instanceof GhTimeoutError) return { code: 'unavailable', message: error.message } + throw error } /** Why `gh` refused: GitHub's message, else its first diagnostic line, with the HTTP status it names. */ diff --git a/packages/experimental/forge-github/tests/fixtures.ts b/packages/experimental/forge-github/tests/fixtures.ts index 5eacc54a6b..66f2f47652 100644 --- a/packages/experimental/forge-github/tests/fixtures.ts +++ b/packages/experimental/forge-github/tests/fixtures.ts @@ -42,6 +42,7 @@ export function repositorySettings(overrides: Record = {}): Rec /** A pull request node with the fields the snapshot query selects. */ export function pullRequestNode(number: number, overrides: Record = {}, contexts: unknown[] = [], rollupState: string | null = 'SUCCESS'): Record { return { + id: `pull-${number}`, number, url: `https://github.com/octo/demo/pull/${number}`, title: `Change ${number}`, @@ -58,7 +59,7 @@ export function pullRequestNode(number: number, overrides: Record ({ isResolved })) } +} diff --git a/packages/experimental/forge-github/tests/graphql.spec.ts b/packages/experimental/forge-github/tests/graphql.spec.ts index 80a96bae20..0181602326 100644 --- a/packages/experimental/forge-github/tests/graphql.spec.ts +++ b/packages/experimental/forge-github/tests/graphql.spec.ts @@ -1,17 +1,25 @@ import { changeRequestKey } from '@deepseek-ai/dsh-experimental-forge' import type { ChangeRequestSnapshot } from '@deepseek-ai/dsh-experimental-forge' import { describe, expect, it } from 'vitest' -import { mapCheckPageResponse, mapHeadResponse, mapSnapshotResponse, snapshotQuery, type CheckPageResult } from '../src/graphql.ts' -import { checkRun, demo, pull, pullRequestNode, repositorySettings, statusContext } from './fixtures.ts' +import { + mapCheckPageResponse, mapHeadCommitResponse, mapHeadResponse, mapSnapshotResponse, mapThreadPageResponse, snapshotQuery, +} from '../src/graphql.ts' +import type { SnapshotPages, ThreadPageResult } from '../src/graphql.ts' +import { checkRun, demo, pull, pullRequestNode, repositorySettings, statusContext, threadPage } from './fixtures.ts' -const noMoreChecks = async (): Promise => { throw new Error('Unexpected check page') } +const MAX_THREADS = 1_000 + +const noMorePages: SnapshotPages = { + threads: async () => { throw new Error('Unexpected review thread page') }, + checks: async () => { throw new Error('Unexpected check page') }, +} const other = { ...demo, name: 'other' } /** Map one pull request node of a repository with `settings`, or without readable merge settings when it is null. */ async function single(node: unknown, settings: Record | null = repositorySettings()): Promise { - const query = snapshotQuery([pull(1)]) - const results = await mapSnapshotResponse({ data: { r0: { ...settings, p0: node } } }, query, noMoreChecks) + const query = snapshotQuery([pull(1)], MAX_THREADS) + const results = await mapSnapshotResponse({ data: { r0: { ...settings, p0: node } } }, query, noMorePages) const result = results.get(changeRequestKey(pull(1))) if (result === undefined || 'code' in result) throw new Error(`expected a snapshot, received ${JSON.stringify(result)}`) return result @@ -20,7 +28,7 @@ async function single(node: unknown, settings: Record | null = describe('snapshot query', () => { it('groups change requests by repository and aliases each one', () => { const upper = { ...demo, owner: 'Octo', name: 'Demo' } - const query = snapshotQuery([pull(1), pull(2, upper), pull(1, upper), pull(7, other)]) + const query = snapshotQuery([pull(1), pull(2, upper), pull(1, upper), pull(7, other)], MAX_THREADS) expect([...query.targets]).toEqual([['r0.p0', pull(1)], ['r0.p1', pull(2, upper)], ['r1.p0', pull(7, other)]]) const settings = 'squashMergeAllowed mergeCommitAllowed rebaseMergeAllowed viewerPermission viewerDefaultMergeMethod' @@ -30,9 +38,9 @@ describe('snapshot query', () => { }) it('refuses owners, names, and numbers that cannot be inlined', () => { - expect(() => snapshotQuery([pull(1, { ...demo, owner: 'oc"to' })])).toThrow('forge-github: invalid repository "oc\\"to/demo"') - expect(() => snapshotQuery([pull(1.5)])).toThrow('forge-github: invalid pull request number 1.5') - expect(() => snapshotQuery([pull(0)])).toThrow('forge-github: invalid pull request number 0') + expect(() => snapshotQuery([pull(1, { ...demo, owner: 'oc"to' })], MAX_THREADS)).toThrow('forge-github: invalid repository "oc\\"to/demo"') + expect(() => snapshotQuery([pull(1.5)], MAX_THREADS)).toThrow('forge-github: invalid pull request number 1.5') + expect(() => snapshotQuery([pull(0)], MAX_THREADS)).toThrow('forge-github: invalid pull request number 0') }) }) @@ -65,6 +73,7 @@ describe('snapshot mapping', () => { checksRollup: 'success', checksTruncated: false, unresolvedThreads: 2, + threadsTruncated: false, updatedAt: '2026-10-09T01:02:03Z', closedAt: null, }) @@ -260,7 +269,7 @@ describe('snapshot mapping', () => { }) it('maps path-scoped and global errors from the partial-error response', async () => { - const query = snapshotQuery([pull(1), pull(2), pull(7, other)]) + const query = snapshotQuery([pull(1), pull(2), pull(7, other)], MAX_THREADS) const body = { data: { r0: { p0: pullRequestNode(1), p1: null }, r1: null }, errors: [ @@ -269,7 +278,7 @@ describe('snapshot mapping', () => { ], } - const results = await mapSnapshotResponse(body, query, noMoreChecks) + const results = await mapSnapshotResponse(body, query, noMorePages) expect(results.get(changeRequestKey(pull(1)))).toMatchObject({ state: 'open' }) expect(results.get(changeRequestKey(pull(2)))).toEqual({ code: 'not-found', message: 'Could not resolve to a PullRequest with the number of 2.' }) @@ -277,8 +286,8 @@ describe('snapshot mapping', () => { }) it('applies an error without a path to every unresolved change request', async () => { - const query = snapshotQuery([pull(1), pull(2)]) - const results = (await mapSnapshotResponse({ data: null, errors: [{ message: 'API rate limit exceeded' }] }, query, noMoreChecks)) + const query = snapshotQuery([pull(1), pull(2)], MAX_THREADS) + const results = (await mapSnapshotResponse({ data: null, errors: [{ message: 'API rate limit exceeded' }] }, query, noMorePages)) expect([...results.values()]).toEqual([ { code: 'unavailable', message: 'API rate limit exceeded' }, @@ -287,30 +296,93 @@ describe('snapshot mapping', () => { }) it('reports a query validation error whose path names a fragment instead of a response alias', async () => { - const query = snapshotQuery([pull(1)]) + const query = snapshotQuery([pull(1)], MAX_THREADS) const results = await mapSnapshotResponse({ errors: [{ path: ['fragment ChangeRequestFields', 'invalidField'], - message: 'Field invalidField does not exist' }] }, query, noMoreChecks) + message: 'Field invalidField does not exist' }] }, query, noMorePages) expect([...results.values()]).toEqual([{ code: 'unavailable', message: 'Field invalidField does not exist' }]) }) it('reports missing nodes, malformed nodes, and malformed bodies', async () => { - const query = snapshotQuery([pull(1), pull(2)]) + const query = snapshotQuery([pull(1), pull(2)], MAX_THREADS) - expect([...(await mapSnapshotResponse({ data: { r0: { p0: null, p1: { title: 7 } } } }, query, noMoreChecks)).values()]).toEqual([ + expect([...(await mapSnapshotResponse({ data: { r0: { p0: null, p1: { title: 7 } } } }, query, noMorePages)).values()]).toEqual([ { code: 'not-found', message: 'pull request not found' }, { code: 'unavailable', message: 'unexpected pull request fields' }, ]) - expect([...(await mapSnapshotResponse({}, query, noMoreChecks)).values()]).toEqual([ + expect([...(await mapSnapshotResponse({}, query, noMorePages)).values()]).toEqual([ { code: 'unavailable', message: 'unexpected GitHub response' }, { code: 'unavailable', message: 'unexpected GitHub response' }, ]) - expect([...(await mapSnapshotResponse('not json', query, noMoreChecks)).values()]).toEqual([ + expect([...(await mapSnapshotResponse('not json', query, noMorePages)).values()]).toEqual([ { code: 'unavailable', message: 'unexpected GitHub response' }, { code: 'unavailable', message: 'unexpected GitHub response' }, ]) }) }) +describe('review thread counting', () => { + /** Map pull request 1, whose first page holds `first` and whose following pages hold `rest`, counting at most `max` threads. */ + async function counted(max: number, first: readonly boolean[], rest: readonly (readonly boolean[])[]) { + const reads: [string, string, number][] = [] + const threads: SnapshotPages['threads'] = async (id, cursor, size) => { + reads.push([id, cursor, size]) + const index = Number(cursor.slice('page-'.length)) + const resolved = rest[index - 1] + if (resolved === undefined) throw new Error(`Unexpected review thread page ${cursor}`) + const more = index < rest.length || size < resolved.length + return { ok: true, page: threadPage(resolved.slice(0, size), more ? `page-${index + 1}` : null) } + } + const query = snapshotQuery([pull(1)], max) + const node = pullRequestNode(1, { reviewThreads: threadPage(first, rest.length > 0 ? 'page-1' : null) }) + const results = await mapSnapshotResponse({ data: { r0: { ...repositorySettings(), p0: node } } }, query, { threads }) + const result = results.get(changeRequestKey(pull(1))) + return { result, reads, document: query.document } + } + const hundred = (unresolved: number): boolean[] => Array.from({ length: 100 }, (_, index) => index >= unresolved) + + it('counts unresolved threads across every page of a pull request with more than 100 threads', async () => { + const { result, reads } = await counted(MAX_THREADS, hundred(1), [hundred(2), [false, true, false]]) + + expect(result).toMatchObject({ unresolvedThreads: 5, threadsTruncated: false }) + expect(reads).toEqual([['pull-1', 'page-1', 100], ['pull-1', 'page-2', 100]]) + }) + + it.each([ + ['exactly the bound', 203, 5, false, [100, 3]], + ['one thread below the total', 202, 4, true, [100, 2]], + ['a bound inside the second page', 150, 3, true, [50]], + ] as const)('stops counting at %s and marks the count as a lower bound past it', async (_label, max, unresolved, truncated, sizes) => { + const { result, reads } = await counted(max, hundred(1), [hundred(2), [false, true, false]]) + + expect(result).toMatchObject({ unresolvedThreads: unresolved, threadsTruncated: truncated }) + expect(reads.map(([, , size]) => size)).toEqual(sizes) + }) + + it('reads a first page no larger than a bound below 100', async () => { + const { result, reads, document } = await counted(2, [false, false], [[false]]) + + expect(document).toContain('reviewThreads(first: 2)') + expect(result).toMatchObject({ unresolvedThreads: 2, threadsTruncated: true }) + expect(reads).toEqual([]) + expect((await counted(MAX_THREADS, [false], [])).document).toContain('reviewThreads(first: 100)') + }) + + it.each([ + ['a failed page', { ok: false, failure: { code: 'unavailable', message: 'Rate limited' } }, 'Rate limited'], + ['a page that repeats its cursor', { ok: true, page: threadPage([false], 'page-1') }, 'GitHub review thread pagination did not advance'], + ['a page without a cursor', { ok: true, page: { pageInfo: { hasNextPage: true, endCursor: null }, nodes: [] } }, + 'GitHub review thread pagination did not advance'], + ] satisfies [string, ThreadPageResult, string][])('fails the pull request for %s without losing another one', async (_label, page, message) => { + const query = snapshotQuery([pull(1), pull(2)], MAX_THREADS) + const body = { data: { r0: { ...repositorySettings(), p0: pullRequestNode(1, { reviewThreads: threadPage(hundred(0), 'page-1') }), p1: pullRequestNode(2) } } } + + const results = await mapSnapshotResponse(body, query, { threads: async () => page }) + + expect(results.get(changeRequestKey(pull(1)))).toEqual({ code: 'unavailable', message }) + expect(results.get(changeRequestKey(pull(2)))).toMatchObject({ unresolvedThreads: 2, threadsTruncated: false }) + }) +}) + describe('check page mapping', () => { it('validates the next page', () => { const page = { pageInfo: { hasNextPage: false, endCursor: null }, nodes: [statusContext('ci', 'SUCCESS')] } @@ -326,6 +398,29 @@ describe('check page mapping', () => { }) }) +describe('review thread page mapping', () => { + it('validates the next page', () => { + const page = threadPage([true, false], 'next') + expect(mapThreadPageResponse({ data: { node: { reviewThreads: page } } })).toEqual({ ok: true, page }) + }) + + it.each([ + [{ data: { node: null } }, 'GitHub review thread page is unavailable'], + [{ data: { node: { reviewThreads: threadPage([true]) } }, errors: [{ message: 'Rate limited' }] }, 'Rate limited'], + [{ data: { node: { reviewThreads: { nodes: [] } } } }, 'unexpected GitHub review thread page'], + ])('rejects unavailable, partial, or malformed pages', (body, message) => { + expect(mapThreadPageResponse(body)).toEqual({ ok: false, failure: { code: 'unavailable', message } }) + }) +}) + +describe('head commit mapping', () => { + it('returns the head commit, or nothing for a response that names none', () => { + expect(mapHeadCommitResponse({ data: { repository: { pullRequest: { headRefOid: 'a'.repeat(40) } } } })).toBe('a'.repeat(40)) + expect(mapHeadCommitResponse({ data: { repository: { pullRequest: null } } })).toBeUndefined() + expect(mapHeadCommitResponse('not json')).toBeUndefined() + }) +}) + describe('head branch lookup mapping', () => { const body = (defaultBranch: string | null, nodes: { number: number; isCrossRepository: boolean }[]) => ({ data: { repository: { defaultBranchRef: defaultBranch === null ? null : { name: defaultBranch }, pullRequests: { nodes } } }, diff --git a/packages/experimental/forge-github/tests/plugin.spec.ts b/packages/experimental/forge-github/tests/plugin.spec.ts index a214182694..1385e6094e 100644 --- a/packages/experimental/forge-github/tests/plugin.spec.ts +++ b/packages/experimental/forge-github/tests/plugin.spec.ts @@ -56,6 +56,8 @@ describe('forge-github plugin', () => { [{ maxOutputBytes: 1.5 }, 'forge-github: maxOutputBytes must be a positive safe integer'], [{ maxOutputBytes: 0 }, 'forge-github: maxOutputBytes must be a positive safe integer'], [{ maxOutputBytes: Number.MAX_SAFE_INTEGER + 1 }, 'forge-github: maxOutputBytes must be a positive safe integer'], + [{ maxReviewThreads: 0 }, 'forge-github: maxReviewThreads must be a positive safe integer'], + [{ maxReviewThreads: 2.5 }, 'forge-github: maxReviewThreads must be a positive safe integer'], ])('rejects %j at load', async (config, message) => { const context = await forge() diff --git a/packages/experimental/forge-github/tests/provider.spec.ts b/packages/experimental/forge-github/tests/provider.spec.ts index ce513e0ab8..bc48a26b30 100644 --- a/packages/experimental/forge-github/tests/provider.spec.ts +++ b/packages/experimental/forge-github/tests/provider.spec.ts @@ -2,8 +2,10 @@ import { changeRequestKey } from '@deepseek-ai/dsh-experimental-forge' import { describe, expect, it } from 'vitest' import { GhOutputLimitError, GhTimeoutError, GhUnavailableError } from '../src/gh.ts' import type { GhOutput, GhRunner } from '../src/gh.ts' +import { HEAD_COMMIT_QUERY, THREADS_QUERY } from '../src/graphql.ts' import { GitHubProvider } from '../src/provider.ts' -import { checkRun, demo, pull, pullRequestNode } from './fixtures.ts' +import type { GitHubLimits } from '../src/provider.ts' +import { checkRun, demo, pull, pullRequestNode, threadPage } from './fixtures.ts' const enterprise = { forge: 'github', host: 'ghe.corp.example', owner: 'team', name: 'svc' } const signal = new AbortController().signal @@ -23,8 +25,10 @@ function output(stdout: unknown, exitCode = 0, stderr = ''): GhOutput { return { exitCode, stdout: typeof stdout === 'string' ? stdout : JSON.stringify(stdout), stderr } } +const limits: GitHubLimits = { maxOutputBytes: 8_388_608, maxReviewThreads: 1_000 } + function provider(run: GhRunner): GitHubProvider { - return new GitHubProvider(run, ['github.com', 'GHE.corp.example'], 8_388_608) + return new GitHubProvider(run, ['github.com', 'GHE.corp.example'], limits) } describe('GitHub provider addressing', () => { @@ -46,7 +50,7 @@ describe('GitHub provider addressing', () => { it('uses a configured HTTP port for URL mapping and GitHub queries', async () => { const gh = fakeGh(() => output({ data: { r0: { p0: pullRequestNode(1) } } })) - const subject = new GitHubProvider(gh.run, ['ghe.example.com:8443'], 8_388_608) + const subject = new GitHubProvider(gh.run, ['ghe.example.com:8443'], limits) const repository = subject.repositoryFromRemote('https://ghe.example.com:8443/team/repo.git')! const ref = { repository, number: 1 } expect(subject.changeRequestFromUrl(subject.changeRequestUrl(ref))).toEqual(ref) @@ -69,6 +73,47 @@ describe('GitHub provider fetch', () => { expect(gh.calls).toHaveLength(1) }) + it('counts review threads past the first page while polling, bounded by maxReviewThreads', async () => { + const many = pullRequestNode(1, { reviewThreads: threadPage(Array.from({ length: 100 }, () => true), 'threads-1') }) + const gh = fakeGh(args => args.includes(`query=${THREADS_QUERY}`) + ? output({ data: { node: { reviewThreads: threadPage([false, false], 'threads-2') } } }) + : output({ data: { r0: { p0: many, p1: pullRequestNode(2) } } })) + + const results = await new GitHubProvider(gh.run, ['github.com'], { ...limits, maxReviewThreads: 102 }).fetch([pull(1), pull(2)], signal) + + expect(results.get(changeRequestKey(pull(1)))).toMatchObject({ unresolvedThreads: 2, threadsTruncated: true }) + expect(results.get(changeRequestKey(pull(2)))).toMatchObject({ unresolvedThreads: 2, threadsTruncated: false }) + expect(gh.calls).toHaveLength(2) + expect(gh.calls[1]).toEqual(['api', 'graphql', '--hostname', 'github.com', + '-f', `query=${THREADS_QUERY}`, '-f', 'id=pull-1', '-f', 'cursor=threads-1', '-F', 'first=2']) + }) + + it.each([ + [new GhTimeoutError('timed out'), 'unavailable'], + [new GhOutputLimitError('too much output'), 'too-large'], + [new GhUnavailableError('gh missing'), 'unavailable'], + ] as const)('fails only the pull request whose review thread page cannot be read', async (error, code) => { + const many = pullRequestNode(1, { reviewThreads: threadPage([true], 'threads-1') }) + const gh = fakeGh((args) => { + if (args.includes(`query=${THREADS_QUERY}`)) throw error + return output({ data: { r0: { p0: many, p1: pullRequestNode(2) } } }) + }) + + const results = await provider(gh.run).fetch([pull(1), pull(2)], signal) + + expect(results.get(changeRequestKey(pull(1)))).toEqual({ code, message: error.message }) + expect(results.get(changeRequestKey(pull(2)))).toMatchObject({ title: 'Change 2' }) + }) + + it('reports a review thread page that GitHub refuses', async () => { + const gh = fakeGh(args => args.includes(`query=${THREADS_QUERY}`) + ? output({ message: 'API rate limit exceeded' }, 1, 'gh: API rate limit exceeded (HTTP 403)') + : output({ data: { r0: { p0: pullRequestNode(1, { reviewThreads: threadPage([true], 'threads-1') }) } } })) + + await expect(provider(gh.run).fetchChecks(pull(1), signal)) + .resolves.toEqual({ code: 'unavailable', message: 'gh: API rate limit exceeded (HTTP 403)' }) + }) + it.each([ [new GhTimeoutError('timed out'), 'unavailable'], [new GhOutputLimitError('too much output'), 'too-large'], @@ -92,7 +137,7 @@ describe('GitHub provider fetch', () => { const total = Buffer.byteLength(first.stdout) + Buffer.byteLength(last.stdout) for (const limit of [total, total - 1, Buffer.byteLength(first.stdout) - 1]) { const gh = fakeGh(args => args.includes('id=fixed-rollup') ? last : first) - const result = await new GitHubProvider(gh.run, ['github.com'], limit).fetchChecks(pull(1), signal) + const result = await new GitHubProvider(gh.run, ['github.com'], { ...limits, maxOutputBytes: limit }).fetchChecks(pull(1), signal) if (limit === total) expect(result).toMatchObject({ checksTruncated: false }) else expect(result).toMatchObject({ code: 'too-large' }) expect(gh.calls).toHaveLength(limit < Buffer.byteLength(first.stdout) ? 1 : 2) @@ -192,7 +237,7 @@ describe('GitHub provider fetch', () => { it('rejects references retained after their host was removed from configuration', async () => { const gh = fakeGh(() => output('')) const retained = provider(gh.run).changeRequestFromUrl('https://ghe.corp.example/team/svc/pull/2')! - const replacement = new GitHubProvider(gh.run, ['github.com'], 8_388_608) + const replacement = new GitHubProvider(gh.run, ['github.com'], limits) const results = await replacement.fetch([retained], signal) await expect(replacement.findByHead(retained.repository, 'topic', signal)).rejects.toThrow('is not configured') expect(results.get(changeRequestKey(retained))).toMatchObject({ code: 'unavailable' }) @@ -354,12 +399,66 @@ describe('GitHub provider actions', () => { describe('GitHub provider reads', () => { const head = 'c'.repeat(40) + const pushed = 'd'.repeat(40) - it('reads the pull request diff through gh pr diff on its host', async () => { - const gh = fakeGh(() => output('diff --git a/x b/x\n')) + /** Answer head commit reads with `heads` in turn, the last one repeating, and every other command with `other`. */ + function pinned(heads: readonly (string | GhOutput)[], other: (args: readonly string[]) => GhOutput = () => output('diff --git a/x b/x\n')) { + let read = 0 + return fakeGh((args) => { + if (!args.includes(`query=${HEAD_COMMIT_QUERY}`)) return other(args) + const answer = heads[Math.min(read++, heads.length - 1)] + return typeof answer === 'string' ? output({ data: { repository: { pullRequest: { headRefOid: answer } } } }) : answer! + }) + } - await expect(provider(gh.run).diff(pull(7, enterprise), signal)).resolves.toBe('diff --git a/x b/x\n') - expect(gh.calls).toEqual([['pr', 'diff', '7', '--repo', 'ghe.corp.example/team/svc', '--color', 'never']]) + it('reads the pull request diff through gh pr diff on its host between two reads of the requested head', async () => { + const gh = pinned([head]) + + await expect(provider(gh.run).diff(pull(7, enterprise), head, signal)).resolves.toBe('diff --git a/x b/x\n') + const headRead = ['api', 'graphql', '--hostname', 'ghe.corp.example', + '-f', `query=${HEAD_COMMIT_QUERY}`, '-f', 'owner=team', '-f', 'name=svc', '-F', 'number=7'] + expect(gh.calls).toEqual([headRead, ['pr', 'diff', '7', '--repo', 'ghe.corp.example/team/svc', '--color', 'never'], headRead]) + }) + + it('compares the head commit without regard to case', async () => { + await expect(provider(pinned([head]).run).diff(pull(7), head.toUpperCase(), signal)).resolves.toBe('diff --git a/x b/x\n') + }) + + it('reports a head that moved before the read as stale without reading the diff', async () => { + const gh = pinned([pushed]) + + await expect(provider(gh.run).diff(pull(7), head, signal)) + .resolves.toEqual({ code: 'stale', message: `the pull request head moved from ${head} to ${pushed}` }) + expect(gh.calls).toHaveLength(1) + }) + + it('discards a diff read while the head moved', async () => { + const gh = pinned([head, pushed]) + + await expect(provider(gh.run).diff(pull(7), head, signal)) + .resolves.toEqual({ code: 'stale', message: `the pull request head moved from ${head} to ${pushed}` }) + expect(gh.calls.map(call => call[0] === 'pr' ? 'diff' : 'head')).toEqual(['head', 'diff', 'head']) + }) + + it.each([ + ['before the diff', [output({ data: { repository: { pullRequest: null } }, errors: [{ type: 'NOT_FOUND', message: 'Could not resolve' }] }, 1, + 'gh: Could not resolve to a PullRequest with the number of 7.')], 1, + { code: 'not-found', message: 'gh: Could not resolve to a PullRequest with the number of 7.' }], + ['after the diff', [head, output({ message: 'Server Error' }, 1, 'gh: Server Error (HTTP 502)')], 3, { code: 'unavailable', message: 'Server Error' }], + ['without a head commit in the answer', [output({ data: { repository: {} } })], 1, + { code: 'unavailable', message: 'unexpected GitHub response to the head commit read' }], + ] as const)('reports a head read that fails %s', async (_label, heads, calls, failure) => { + const gh = pinned(heads) + + await expect(provider(gh.run).diff(pull(7), head, signal)).resolves.toEqual(failure) + expect(gh.calls).toHaveLength(calls) + }) + + it('refuses a head that is not a full commit object name before starting gh', async () => { + const gh = pinned([head]) + + await expect(provider(gh.run).diff(pull(7), 'HEAD', signal)).rejects.toThrow('forge-github: invalid head commit "HEAD"') + expect(gh.calls).toEqual([]) }) it('reads a file at a commit through the raw contents API, encoding each path segment', async () => { @@ -401,18 +500,18 @@ describe('GitHub provider reads', () => { { code: 'unauthenticated', message: 'To get started with GitHub CLI, please run: gh auth login' }], ['no output', '', '', { code: 'unavailable', message: 'gh exited with 1' }], ])('reports a read refused for %s', async (_label, body, stderr, failure) => { - await expect(provider(fakeGh(() => output(body, 1, stderr)).run).diff(pull(7), signal)).resolves.toEqual(failure) + await expect(provider(pinned([head], () => output(body, 1, stderr)).run).diff(pull(7), head, signal)).resolves.toEqual(failure) }) it('reports a read as unavailable when gh cannot start, rethrows other failures, and refuses an invalid commit', async () => { - await expect(provider(() => Promise.reject(new GhUnavailableError('gh could not start: spawn gh ENOENT'))).diff(pull(7), signal)) + await expect(provider(() => Promise.reject(new GhUnavailableError('gh could not start: spawn gh ENOENT'))).diff(pull(7), head, signal)) .resolves.toEqual({ code: 'unavailable', message: 'gh could not start: spawn gh ENOENT' }) await expect(provider(() => Promise.reject(new Error('aborted'))).fileText(pull(7), head, 'a', signal)).rejects.toThrow('aborted') await expect(provider(fakeGh(() => output('')).run).fileText(pull(7), 'main', 'a', signal)).rejects.toThrow('forge-github: invalid commit "main"') }) it('reports output over maxOutputBytes as too large and a run past timeoutMs as unavailable', async () => { - await expect(provider(() => Promise.reject(new GhOutputLimitError('gh pr output exceeds maxOutputBytes'))).diff(pull(7), signal)) + await expect(provider(() => Promise.reject(new GhOutputLimitError('gh pr output exceeds maxOutputBytes'))).diff(pull(7), head, signal)) .resolves.toEqual({ code: 'too-large', message: 'gh pr output exceeds maxOutputBytes' }) await expect(provider(() => Promise.reject(new GhTimeoutError('gh api timed out after 30000ms'))).fileText(pull(7), head, 'a', signal)) .resolves.toEqual({ code: 'unavailable', message: 'gh api timed out after 30000ms' }) @@ -427,7 +526,7 @@ it('rejects readiness, actions, and content reads for an unconfigured host witho await expect(subject.readiness(target.repository.host, signal)).resolves.toEqual({ state: 'unavailable', message }) await expect(subject.markReady(target, signal)).resolves.toEqual({ code: 'failed', message }) await expect(subject.merge(target, { method: 'squash', headSha: 'a'.repeat(40) }, signal)).resolves.toEqual({ code: 'failed', message }) - await expect(subject.diff(target, signal)).resolves.toEqual({ code: 'unavailable', message }) + await expect(subject.diff(target, 'a'.repeat(40), signal)).resolves.toEqual({ code: 'unavailable', message }) await expect(subject.fileText(target, 'a'.repeat(40), 'file.txt', signal)).resolves.toEqual({ code: 'unavailable', message }) expect(gh.calls).toEqual([]) }) diff --git a/packages/experimental/forge-github/tests/review.spec.ts b/packages/experimental/forge-github/tests/review.spec.ts index e128508994..002d018f6a 100644 --- a/packages/experimental/forge-github/tests/review.spec.ts +++ b/packages/experimental/forge-github/tests/review.spec.ts @@ -11,7 +11,7 @@ const single: ForgeReviewRequest = { kind: 'comment', headSha: head, comment } const success = { exitCode: 0, stdout: JSON.stringify({ html_url: 'https://github.com/octo/demo/pull/7#discussion_r1' }), stderr: '' } function fixture() { const gh = vi.fn().mockResolvedValueOnce({ exitCode: 0, stdout: JSON.stringify({ head: { sha: head }, state: 'open' }), stderr: '' }).mockResolvedValue(success) - return { gh, provider: new GitHubProvider(gh, ['github.com'], 8_388_608), signal: new AbortController().signal } + return { gh, provider: new GitHubProvider(gh, ['github.com'], { maxOutputBytes: 8_388_608, maxReviewThreads: 1_000 }), signal: new AbortController().signal } } describe('review submission', () => { @@ -91,7 +91,7 @@ describe('review submission', () => { }) it('rejects a host outside the provider configuration without invoking gh', async () => { const { gh, signal } = fixture() - expect(await new GitHubProvider(gh, ['enterprise.test'], 8_388_608).submitReview(ref, single, signal)).toMatchObject({ ok: false, code: 'failed' }) + expect(await new GitHubProvider(gh, ['enterprise.test'], { maxOutputBytes: 8_388_608, maxReviewThreads: 1_000 }).submitReview(ref, single, signal)).toMatchObject({ ok: false, code: 'failed' }) expect(gh).not.toHaveBeenCalled() }) }) diff --git a/packages/experimental/forge-session/tests/fixtures.ts b/packages/experimental/forge-session/tests/fixtures.ts index d176c97101..2c857ad28e 100644 --- a/packages/experimental/forge-session/tests/fixtures.ts +++ b/packages/experimental/forge-session/tests/fixtures.ts @@ -46,6 +46,7 @@ export function snapshot(ref: ChangeRequestRef, overrides: Partial ({ + id: 'pull-' + number, number, url: 'https://github.com/octo/demo/pull/' + number, title: 'Change ' + number, @@ -84,7 +85,7 @@ const node = (number) => ({ mergeable: 'MERGEABLE', mergeStateStatus: number === 7 ? 'BLOCKED' : 'CLEAN', reviewDecision: null, - reviewThreads: { nodes: [] }, + reviewThreads: { pageInfo: { hasNextPage: false, endCursor: null }, nodes: [] }, commits: { nodes: [{ commit: { statusCheckRollup: { id: 'rollup-' + number, state: number === 7 ? 'PENDING' : 'SUCCESS', contexts: { pageInfo: { hasNextPage: false, endCursor: null }, nodes: [{ diff --git a/packages/experimental/forge/README.i18n.yaml b/packages/experimental/forge/README.i18n.yaml index a51d57819e..a60d0a4d78 100644 --- a/packages/experimental/forge/README.i18n.yaml +++ b/packages/experimental/forge/README.i18n.yaml @@ -15,8 +15,8 @@ en: d152484eb41ac6b4 zh: 2f8a621b991260a1 /deepseek-ai-dsh-experimental-forge/use-this-package: - en: 9ca86ee541048ff2 - zh: 7aef4feb76542dd6 + en: 897de781e679f5f2 + zh: 3d0c589742a4dac0 /deepseek-ai-dsh-experimental-forge/understand-the-implementation: en: e193cb547c47069b zh: faa517522402ea41 diff --git a/packages/experimental/forge/README.md b/packages/experimental/forge/README.md index 5d2bb55ec6..e04f46a256 100644 --- a/packages/experimental/forge/README.md +++ b/packages/experimental/forge/README.md @@ -36,7 +36,7 @@ Mount the registry with at least one provider package, such as the GitHub provid `markReady(ref, signal)` marks a draft ready for review, and `merge(ref, { method, headSha }, signal)` merges a change request only while its head is still `headSha`. Both delegate to the provider of the reference's forge, throw when none is registered, and resolve to undefined once the host accepted the change or to a `ForgeActionFailure`: `unauthenticated`, `stale` when the head moved past `headSha`, or `failed` with the host's reason. After either settles, a watched change request is polled at once. Snapshots carry `mergeReady`, whether the host would merge the change request now for the signed-in user, and `mergeMethods`, the methods the repository accepts with that user's usual one first. -`diff(ref, signal)` reads the change request's unified patch as the host computes it, with `a/` and `b/` path prefixes, and `fileText(ref, commit, path, signal)` reads one file of its repository at a commit. Both delegate to the provider of the reference's forge, throw when none is registered, and resolve to the text or to a `ForgeFailure`, which is `too-large` when the host or the provider refuses an answer over its size limit. `fileUrl(ref, commit, path)` returns the web page of that file at that commit. +`diff(ref, headSha, signal)` reads the change request's unified patch at head commit `headSha` as the host computes it, with `a/` and `b/` path prefixes, and `fileText(ref, commit, path, signal)` reads one file of its repository at a commit. Both delegate to the provider of the reference's forge, throw when none is registered, and resolve to the text or to a `ForgeFailure`, which is `too-large` when the host or the provider refuses an answer over its size limit. A provider returns a patch only when it describes `headSha`; once the head is another commit, `diff` resolves to `stale`, and a watched change request is polled at once so its next status names the current head. Snapshots carry `unresolvedThreads` with `threadsTruncated`, which marks the count as a lower bound when the provider counted only some review threads. `fileUrl(ref, commit, path)` returns the web page of that file at that commit. `fetchChecks(ref, signal)` reads one change request’s complete checks on demand. It has its own `fetchTimeoutMs` deadline and does not occupy the polling batch or publish to its watchers. Failure leaves previously polled snapshots intact. A provider without `fetchChecks` returns `unavailable`. Caller cancellation and provider or service disposal abort the read; disposal waits for it to settle. diff --git a/packages/experimental/forge/README.zh.md b/packages/experimental/forge/README.zh.md index 8961c8d31d..9e61402f5b 100644 --- a/packages/experimental/forge/README.zh.md +++ b/packages/experimental/forge/README.zh.md @@ -36,7 +36,7 @@ kind: "package-reference" `markReady(ref, signal)` 把草稿标记为可评审；`merge(ref, { method, headSha }, signal)` 只在 head 仍是 `headSha` 时合并变更请求。两者都委托给引用所属 forge 的提供方，没有注册提供方时抛错；主机接受时 resolve 为 undefined，否则为 `ForgeActionFailure`：`unauthenticated`、head 已越过 `headSha` 时的 `stale`，或带主机原因的 `failed`。两者结束后，被监听的变更请求会立即重新轮询。快照带有 `mergeReady`（主机现在是否会为已登录用户合并该变更请求）与 `mergeMethods`（仓库接受的合并方式，该用户常用的方式排在最前）。 -`diff(ref, signal)` 读取托管平台计算出的变更请求统一补丁，路径带 `a/` 与 `b/` 前缀；`fileText(ref, commit, path, signal)` 读取其仓库中某个提交下的一个文件。两者都委托给引用所属 forge 的提供方，没有注册提供方时抛错，resolve 为文本或 `ForgeFailure`；托管平台或提供方因结果超过其大小上限而拒绝时，失败为 `too-large`。`fileUrl(ref, commit, path)` 返回该文件在该提交下的网页地址。 +`diff(ref, headSha, signal)` 读取托管平台计算出的变更请求在 head 提交 `headSha` 处的统一补丁，路径带 `a/` 与 `b/` 前缀；`fileText(ref, commit, path, signal)` 读取其仓库中某个提交下的一个文件。两者都委托给引用所属 forge 的提供方，没有注册提供方时抛错，resolve 为文本或 `ForgeFailure`；托管平台或提供方因结果超过其大小上限而拒绝时，失败为 `too-large`。提供方只在补丁描述 `headSha` 时返回补丁；head 已是其他提交时，`diff` resolve 为 `stale`，被 watch 的变更请求随即重新轮询，使下一次状态给出当前 head。快照携带 `unresolvedThreads` 与 `threadsTruncated`；提供方只统计了部分评审讨论时，后者表示该计数只是下限。`fileUrl(ref, commit, path)` 返回该文件在该提交下的网页地址。 `fetchChecks(ref, signal)` 按需读取单个变更请求的完整检查，使用独立的 `fetchTimeoutMs` 时限，不占用轮询批次，也不向其监听方发布结果。读取失败时保留已有轮询快照。提供方未实现 `fetchChecks` 时返回 `unavailable`。调用方取消、提供方或服务销毁都会中止读取；销毁会等待读取结束。 diff --git a/packages/experimental/forge/src/index.ts b/packages/experimental/forge/src/index.ts index d5204d988a..df05ef9796 100644 --- a/packages/experimental/forge/src/index.ts +++ b/packages/experimental/forge/src/index.ts @@ -236,15 +236,20 @@ export class ForgeService extends Service { } /** - * Read the changes of a change request as its host computes them. + * Read the changes of a change request at one head commit as its host + * computes them. A `stale` failure polls the change request again at once + * when it is watched, so its next status names the current head. * @param ref - change request served by a registered provider. + * @param headSha - head commit the patch must describe, usually the head of the latest status. * @param signal - cancellation of the read. - * @returns one unified patch with `a/` and `b/` path prefixes, or why the host did not provide it. + * @returns one unified patch with `a/` and `b/` path prefixes, or why the host did not provide it at `headSha`. * @throws when no provider serves the change request's forge, or with `signal.reason` when it aborts. */ - async diff(ref: ChangeRequestRef, signal: AbortSignal): Promise { - return await this.invoke(this.registration(ref.repository.forge), signal, - (provider, operation) => provider.diff(ref, operation)) + async diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise { + const result = await this.invoke(this.registration(ref.repository.forge), signal, + (provider, operation) => provider.diff(ref, headSha, operation)) + if (typeof result !== 'string' && result.code === 'stale') this.refresh(ref) + return result } /** diff --git a/packages/experimental/forge/src/types.ts b/packages/experimental/forge/src/types.ts index 35a977dcfe..7d124dc80f 100644 --- a/packages/experimental/forge/src/types.ts +++ b/packages/experimental/forge/src/types.ts @@ -100,8 +100,10 @@ export interface ChangeRequestSnapshot { readonly checksRollup: ChecksRollup /** Whether the host reported more checks than `checks` contains. */ readonly checksTruncated: boolean - /** Number of review threads that are not resolved. */ + /** Number of counted review threads that are not resolved. */ readonly unresolvedThreads: number + /** Whether the change request has more review threads than the provider counted, so `unresolvedThreads` is a lower bound. */ + readonly threadsTruncated: boolean /** ISO 8601 time of the host's last update to the change request. */ readonly updatedAt: string /** ISO 8601 time the change request was merged or closed, or null while it is open. */ @@ -113,9 +115,10 @@ export interface ForgeFailure { /** * `not-found`: the host does not expose it; `unauthenticated`: credentials are missing or rejected; * `too-large`: the host or the provider refuses an answer over its size limit, so reading again fails - * the same way; `unavailable`: any other failure. + * the same way; `stale`: the head is no longer the commit the read named, so reading again at the head of + * a later status can succeed; `unavailable`: any other failure. */ - readonly code: 'not-found' | 'unauthenticated' | 'too-large' | 'unavailable' + readonly code: 'not-found' | 'unauthenticated' | 'too-large' | 'stale' | 'unavailable' /** Human-readable cause. */ readonly message: string } @@ -214,8 +217,12 @@ export interface ForgeProvider { markReady(ref: ChangeRequestRef, signal: AbortSignal): Promise /** Merge a change request while its head is still `request.headSha`, resolving to undefined once merged. */ merge(ref: ChangeRequestRef, request: MergeRequest, signal: AbortSignal): Promise - /** Read the changes of a change request as the host computes them, as one unified patch with `a/` and `b/` path prefixes. */ - diff(ref: ChangeRequestRef, signal: AbortSignal): Promise + /** + * Read the changes of a change request at head commit `headSha` as the host computes them, as one unified + * patch with `a/` and `b/` path prefixes. A patch is returned only when it describes `headSha`; once the + * head is another commit, the read resolves to a `stale` failure. + */ + diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise /** Read the text of `path` at `commit` in the change request's repository. */ fileText(ref: ChangeRequestRef, commit: string, path: string, signal: AbortSignal): Promise /** Build the web page of `path` at `commit` in the change request's repository. */ diff --git a/packages/experimental/forge/tests/fake-provider.ts b/packages/experimental/forge/tests/fake-provider.ts index 6f04abdc00..d82f6e7b65 100644 --- a/packages/experimental/forge/tests/fake-provider.ts +++ b/packages/experimental/forge/tests/fake-provider.ts @@ -39,6 +39,7 @@ export function snapshot(target: ChangeRequestRef, overrides: Partial { - this.actions.push(`diff ${target.number}`) + async diff(target: ChangeRequestRef, headSha: string): Promise { + this.actions.push(`diff ${target.number} ${headSha}`) return this.readResult } diff --git a/packages/experimental/forge/tests/lifecycle.spec.ts b/packages/experimental/forge/tests/lifecycle.spec.ts index 26e779eb87..1021b2fa1f 100644 --- a/packages/experimental/forge/tests/lifecycle.spec.ts +++ b/packages/experimental/forge/tests/lifecycle.spec.ts @@ -137,7 +137,7 @@ it.each(['readiness', 'markReady', 'merge', 'diff', 'fileText', 'submitReview', }) vi.spyOn(provider, 'markReady').mockImplementation(async (_ref, signal) => { await hold(signal); return undefined }) vi.spyOn(provider, 'merge').mockImplementation(async (_ref, _request, signal) => { await hold(signal); return undefined }) - vi.spyOn(provider, 'diff').mockImplementation(async (_ref, signal) => { await hold(signal); return 'diff' }) + vi.spyOn(provider, 'diff').mockImplementation(async (_ref, _headSha, signal) => { await hold(signal); return 'diff' }) vi.spyOn(provider, 'fileText').mockImplementation(async (_ref, _commit, _path, signal) => { await hold(signal); return 'file' }) provider.submitReview = async (_ref, _request, signal) => { await hold(signal); return { ok: true, url: 'https://fake.test/review' } } provider.fetchChecks = async (target, signal) => { await hold(signal); return snapshot(target) } @@ -150,7 +150,7 @@ it.each(['readiness', 'markReady', 'merge', 'diff', 'fileText', 'submitReview', readiness: () => service.readiness(demo, signal), markReady: () => service.markReady(ref(1), signal), merge: () => service.merge(ref(1), { method: 'squash', headSha: 'a'.repeat(40) }, signal), - diff: () => service.diff(ref(1), signal), + diff: () => service.diff(ref(1), 'a'.repeat(40), signal), fileText: () => service.fileText(ref(1), 'a'.repeat(40), 'file.txt', signal), } const rejected = expect(calls[method]()).rejects.toThrow('was unregistered') diff --git a/packages/experimental/forge/tests/registry.spec.ts b/packages/experimental/forge/tests/registry.spec.ts index c70a07057b..30af40c019 100644 --- a/packages/experimental/forge/tests/registry.spec.ts +++ b/packages/experimental/forge/tests/registry.spec.ts @@ -71,7 +71,7 @@ describe('forge registry', () => { await expect(ctx.forge.markReady(ref(1, elsewhere), signal)).rejects.toThrow('forge: no provider is registered for "missing"') await expect(ctx.forge.merge(ref(1, elsewhere), { method: 'squash', headSha: 'a'.repeat(40) }, signal)) .rejects.toThrow('forge: no provider is registered for "missing"') - await expect(ctx.forge.diff(ref(1, elsewhere), signal)).rejects.toThrow('forge: no provider is registered for "missing"') + await expect(ctx.forge.diff(ref(1, elsewhere), 'a'.repeat(40), signal)).rejects.toThrow('forge: no provider is registered for "missing"') await expect(ctx.forge.fileText(ref(1, elsewhere), 'a'.repeat(40), 'x', signal)).rejects.toThrow('forge: no provider is registered for "missing"') expect(() => ctx!.forge.fileUrl(ref(1, elsewhere), 'a'.repeat(40), 'x')).toThrow('forge: no provider is registered for "missing"') }) @@ -97,11 +97,11 @@ describe('forge registry', () => { await provide(ctx, provider) const signal = new AbortController().signal - await expect(ctx.forge.diff(ref(3), signal)).resolves.toBe('diff --git a/x b/x') + await expect(ctx.forge.diff(ref(3), 'a'.repeat(40), signal)).resolves.toBe('diff --git a/x b/x') provider.readResult = { code: 'not-found', message: 'gone' } await expect(ctx.forge.fileText(ref(3), 'b'.repeat(40), 'src/a.ts', signal)).resolves.toEqual({ code: 'not-found', message: 'gone' }) expect(ctx.forge.fileUrl(ref(3), 'b'.repeat(40), 'src/a.ts')).toBe(`https://fake.test/Octo/Demo/file/${'b'.repeat(40)}/src/a.ts`) - expect(provider.actions).toEqual(['diff 3', `text 3 ${'b'.repeat(40)} src/a.ts`]) + expect(provider.actions).toEqual([`diff 3 ${'a'.repeat(40)}`, `text 3 ${'b'.repeat(40)} src/a.ts`]) }) it('delegates change request actions to the provider of their forge and returns its outcome', async () => { diff --git a/packages/experimental/forge/tests/watch.spec.ts b/packages/experimental/forge/tests/watch.spec.ts index 5760417c26..2685e0d904 100644 --- a/packages/experimental/forge/tests/watch.spec.ts +++ b/packages/experimental/forge/tests/watch.spec.ts @@ -223,6 +223,27 @@ describe('forge watch', () => { expect(provider.fetches).toHaveLength(3) }) + it('polls a watched change request at once after a diff read finds a newer head, and only then', async () => { + const provider = new FakeProvider() + provider.results.set(key, snapshot(ref(1))) + const context = await setup(provider) + collect(context) + await vi.advanceTimersByTimeAsync(0) + const signal = new AbortController().signal + + provider.readResult = 'diff --git a/x b/x' + await context.forge.diff(ref(1), 'a'.repeat(40), signal) + provider.readResult = { code: 'not-found', message: 'gone' } + await context.forge.diff(ref(1), 'a'.repeat(40), signal) + await vi.advanceTimersByTimeAsync(0) + expect(provider.fetches).toHaveLength(1) + + provider.readResult = { code: 'stale', message: 'the head moved' } + await expect(context.forge.diff(ref(1), 'a'.repeat(40), signal)).resolves.toEqual({ code: 'stale', message: 'the head moved' }) + await vi.advanceTimersByTimeAsync(0) + expect(provider.fetches).toHaveLength(2) + }) + it('coalesces refreshes during a fetch into one immediate subsequent fetch', async () => { const provider = new FakeProvider() const release = held(provider, target => snapshot(target)) diff --git a/packages/extensions/tool-cordis/src/api-catalog.ts b/packages/extensions/tool-cordis/src/api-catalog.ts index 74a5703129..9e5da268f3 100644 --- a/packages/extensions/tool-cordis/src/api-catalog.ts +++ b/packages/extensions/tool-cordis/src/api-catalog.ts @@ -1246,10 +1246,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ throws: ['on cancellation, provider disposal, or expiry of this read\'s own `fetchTimeoutMs` deadline.'], }, { - signature: 'async diff(ref: ChangeRequestRef, signal: AbortSignal): Promise', - description: 'Read the changes of a change request as its host computes them.', - parameters: [{ name: 'ref', description: 'change request served by a registered provider.' }, { name: 'signal', description: 'cancellation of the read.' }], - returns: 'one unified patch with `a/` and `b/` path prefixes, or why the host did not provide it.', + signature: 'async diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise', + description: 'Read the changes of a change request at one head commit as its host computes them. A `stale` failure polls the change request again at once when it is watched, so its next status names the current head.', + parameters: [{ name: 'ref', description: 'change request served by a registered provider.' }, { name: 'headSha', description: 'head commit the patch must describe, usually the head of the latest status.' }, { name: 'signal', description: 'cancellation of the read.' }], + returns: 'one unified patch with `a/` and `b/` path prefixes, or why the host did not provide it at `headSha`.', throws: ['when no provider serves the change request\'s forge, or with `signal.reason` when it aborts.'], }, { @@ -1372,7 +1372,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ description: 'Read the changes of a change request of a Session as its code host computes them, at the head commit of its latest status. A patch already read at the same head commit and base branch is reused from memory, for any Session, until `maxCachedDiffBytes` evicts it.', parameters: [{ name: 'session', description: 'Session the change request is bound to.' }, { name: 'url', description: 'binding URL of the change request.' }, { name: 'signal', description: 'request cancellation.' }], returns: 'the files, the head commit, each file\'s web page, and the commits the checkout has beyond the head.', - throws: ['`forgePanel/not-bound` unless the URL is a visible binding of the Session that a provider serves; `forgePanel/diff-too-large` when the host or its provider refuses the diff over a size limit; `forgePanel/diff-failed` before its status was read and when the host does not provide the diff otherwise.'], + throws: ['`forgePanel/not-bound` unless the URL is a visible binding of the Session that a provider serves; `forgePanel/stale-head` when the head moved past the latest status, which is then polled again; `forgePanel/diff-too-large` when the host or its provider refuses the diff over a size limit; `forgePanel/diff-failed` before its status was read and when the host does not provide the diff otherwise.'], }, { signature: '@Remote async changeRequestText(session: Session, url: string, commit: string, path: string, signal: AbortSignal): Promise', @@ -5956,7 +5956,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'ChangeRequestSnapshot', - declaration: 'export interface ChangeRequestSnapshot {\n readonly ref: ChangeRequestRef;\n readonly url: string;\n readonly title: string;\n readonly state: ChangeRequestState;\n readonly draft: boolean;\n readonly author: string | null;\n readonly headBranch: string;\n readonly headSha: string;\n readonly baseBranch: string;\n readonly additions: number;\n readonly deletions: number;\n readonly mergeability: Mergeability;\n readonly mergeReady: boolean;\n readonly mergeMethods: readonly MergeMethod[];\n readonly reviewDecision: ReviewDecision;\n readonly checks: readonly ChangeRequestCheck[];\n readonly checksRollup: ChecksRollup;\n readonly checksTruncated: boolean;\n readonly unresolvedThreads: number;\n readonly updatedAt: string;\n readonly closedAt: string | null;\n}', + declaration: 'export interface ChangeRequestSnapshot {\n readonly ref: ChangeRequestRef;\n readonly url: string;\n readonly title: string;\n readonly state: ChangeRequestState;\n readonly draft: boolean;\n readonly author: string | null;\n readonly headBranch: string;\n readonly headSha: string;\n readonly baseBranch: string;\n readonly additions: number;\n readonly deletions: number;\n readonly mergeability: Mergeability;\n readonly mergeReady: boolean;\n readonly mergeMethods: readonly MergeMethod[];\n readonly reviewDecision: ReviewDecision;\n readonly checks: readonly ChangeRequestCheck[];\n readonly checksRollup: ChecksRollup;\n readonly checksTruncated: boolean;\n readonly unresolvedThreads: number;\n readonly threadsTruncated: boolean;\n readonly updatedAt: string;\n readonly closedAt: string | null;\n}', }, { name: 'ChangeRequestState', @@ -6604,7 +6604,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'ForgeFailure', - declaration: 'export interface ForgeFailure {\n readonly code: \'not-found\' | \'unauthenticated\' | \'too-large\' | \'unavailable\';\n readonly message: string;\n}', + declaration: 'export interface ForgeFailure {\n readonly code: \'not-found\' | \'unauthenticated\' | \'too-large\' | \'stale\' | \'unavailable\';\n readonly message: string;\n}', }, { name: 'ForgeLineComment', @@ -6640,7 +6640,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'ForgeProvider', - declaration: 'export interface ForgeProvider {\n readonly forge: string;\n submitReview?(ref: ChangeRequestRef, request: ForgeReviewRequest, signal: AbortSignal): Promise;\n repositoryFromRemote(url: string): ForgeRepository | undefined;\n changeRequestFromUrl(url: string): ChangeRequestRef | undefined;\n changeRequestUrl(ref: ChangeRequestRef): string;\n fetch(refs: readonly ChangeRequestRef[], signal: AbortSignal): Promise>;\n fetchChecks?(ref: ChangeRequestRef, signal: AbortSignal): Promise;\n findByHead(repository: ForgeRepository, branch: string, signal: AbortSignal): Promise;\n readiness(host: string, signal: AbortSignal): Promise;\n markReady(ref: ChangeRequestRef, signal: AbortSignal): Promise;\n merge(ref: ChangeRequestRef, request: MergeRequest, signal: AbortSignal): Promise;\n diff(ref: ChangeRequestRef, signal: AbortSignal): Promise;\n fileText(ref: ChangeRequestRef, commit: string, path: string, signal: AbortSignal): Promise;\n fileUrl(ref: ChangeRequestRef, commit: string, path: string): string;\n}', + declaration: 'export interface ForgeProvider {\n readonly forge: string;\n submitReview?(ref: ChangeRequestRef, request: ForgeReviewRequest, signal: AbortSignal): Promise;\n repositoryFromRemote(url: string): ForgeRepository | undefined;\n changeRequestFromUrl(url: string): ChangeRequestRef | undefined;\n changeRequestUrl(ref: ChangeRequestRef): string;\n fetch(refs: readonly ChangeRequestRef[], signal: AbortSignal): Promise>;\n fetchChecks?(ref: ChangeRequestRef, signal: AbortSignal): Promise;\n findByHead(repository: ForgeRepository, branch: string, signal: AbortSignal): Promise;\n readiness(host: string, signal: AbortSignal): Promise;\n markReady(ref: ChangeRequestRef, signal: AbortSignal): Promise;\n merge(ref: ChangeRequestRef, request: MergeRequest, signal: AbortSignal): Promise;\n diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise;\n fileText(ref: ChangeRequestRef, commit: string, path: string, signal: AbortSignal): Promise;\n fileUrl(ref: ChangeRequestRef, commit: string, path: string): string;\n}', }, { name: 'ForgeReadiness', From 3430de353188d0f27554cd75f424cc94b5fc313c Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 16:16:47 +0800 Subject: [PATCH 26/44] Keep platform-specific CUA instructions and hover policy in the macOS provider --- packages/experimental/cua-macos/README.md | 28 +++++++++---------- packages/experimental/cua-macos/README.zh.md | 28 +++++++++---------- .../cua-macos/native/README.i18n.yaml | 4 +-- .../experimental/cua-macos/native/README.md | 2 +- .../cua-macos/native/README.zh.md | 2 +- .../native/Sources/CuaNative/AXWorker.swift | 8 ++++-- .../CuaNative/RequestCoordinator.swift | 3 +- .../native/Sources/CuaNative/Runtime.swift | 7 ++++- .../tests/swift/SharedHelperTests.swift | 10 ++++++- .../experimental/cua-macos/src/guidance.ts | 18 ++++++++++-- packages/experimental/cua-macos/src/index.ts | 5 ++-- .../cua-macos/src/shared-native.ts | 11 +++++--- .../cua-macos/tests/guidance.spec.ts | 13 ++++++--- .../tests/provider-lifecycle.spec.ts | 25 ++--------------- .../cua-macos/tests/shared-native.spec.ts | 14 ++++++---- packages/experimental/cua/README.i18n.yaml | 4 +-- packages/experimental/cua/README.md | 2 +- packages/experimental/cua/README.zh.md | 2 +- packages/experimental/cua/src/guidance.ts | 27 +++++++++--------- packages/experimental/cua/src/index.ts | 3 -- .../experimental/cua/src/native-schema.ts | 4 +-- packages/experimental/cua/src/types.ts | 22 ++++++--------- .../cua-native/system-prompt.expected.md | 28 +++++++++---------- .../cua-native/workspace/native-helper.mjs | 8 ++++-- .../session/cua-ptc/system-prompt.expected.md | 28 +++++++++---------- .../system-prompt.expected.md | 28 +++++++++---------- 26 files changed, 177 insertions(+), 157 deletions(-) diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index 742e218285..6abe3c08de 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -176,7 +176,7 @@ The plugin contributes one `cua` operation tool with method-specific parameters. ##### Operation guidance (verbatim) ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -187,20 +187,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. -- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. +- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. +- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -210,17 +210,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. +- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 276baf2131..13f4fa1d06 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -176,7 +176,7 @@ AppKit 的 sheet 可以出现在文档窗口的树里，并有独立的窗口 ID ##### 操作指导文本（原文） ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -187,20 +187,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. -- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. +- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. +- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -210,17 +210,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. +- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. diff --git a/packages/experimental/cua-macos/native/README.i18n.yaml b/packages/experimental/cua-macos/native/README.i18n.yaml index 25d5fb8952..8d43fbd6b8 100644 --- a/packages/experimental/cua-macos/native/README.i18n.yaml +++ b/packages/experimental/cua-macos/native/README.i18n.yaml @@ -21,8 +21,8 @@ en: 88a59904ca4ebedc zh: 14f2b666d7382145 /dsh-cua-native-helper/observation-and-input: - en: da6a1c45fd166818 - zh: 7f378d5409a6085d + en: 2d380e942a4057e4 + zh: fe7c9339ec63a6e5 /dsh-cua-native-helper/verification: en: 49f4cce2a4cc3114 zh: e9d65c92ef35fe24 diff --git a/packages/experimental/cua-macos/native/README.md b/packages/experimental/cua-macos/native/README.md index bdb844e806..bc80110951 100644 --- a/packages/experimental/cua-macos/native/README.md +++ b/packages/experimental/cua-macos/native/README.md @@ -121,7 +121,7 @@ Tree observation and action discovery treat a generic AX failure reading one ele `paste` publishes one plain-text or HTML clipboard format, dispatches the application's enabled standard Paste menu command, and waits for a clipboard data request. It then restores the preserved formats only while its ownership marker and change count still match. The Host serializes its pastes across helpers with a file lock. maxClipboardBytes bounds both input and preserved contents; unsupported menus and oversized data fail before dispatch. A later copy is preserved. Cancellation restores owned clipboard data when possible; forced termination cannot guarantee restoration, and a data request does not prove the target completed editing. After dispatch, `pasteReadTimeoutMs` bounds waiting for clipboard consumption. Expiry returns `PASTE_NOT_CONSUMED`, closes the promised data provider, and restores the original clipboard only while still owned. The failure does not prove the app will never process its queued Paste command; inspect the target before deciding whether to retry. -`mouseMove` reveals or restores alternate UI through an enabled element or ancestor advertising both `AXShowAlternateUI` and `AXShowDefaultUI`. Without an AX route it returns `HOVER_UNAVAILABLE` before sending input. The Host supplies the native boolean `allowSystemPointer` from its `systemPointerHover` setting. Omission means false. Only true permits posting a system pointer move after checking the foreground process and window hit. +`mouseMove` reveals or restores alternate UI through an enabled element or ancestor advertising both `AXShowAlternateUI` and `AXShowDefaultUI`. Without an AX route it returns `HOVER_UNAVAILABLE` before sending input. The macOS provider sets `systemPointerHover` in the connection handshake. Diagnostic clients that omit this setting may opt in per request with `allowSystemPointer: true`. The default is false. Only true permits posting a system pointer move after checking the foreground process and window hit. The opt-in system pointer route returns `FOREGROUND_REQUIRED` if the target is obscured or belongs to a background app. The result reports the route and whether the visible cursor moved in `hoverRoute` and `systemPointerMoved`. It never presses a button or restores the cursor after a read. Later input may interrupt the hover. Clicks send their button events at the target without an off-window priming click. diff --git a/packages/experimental/cua-macos/native/README.zh.md b/packages/experimental/cua-macos/native/README.zh.md index 51d6c13c29..0927a82bce 100644 --- a/packages/experimental/cua-macos/native/README.zh.md +++ b/packages/experimental/cua-macos/native/README.zh.md @@ -121,7 +121,7 @@ AppKit 在主事件循环上运行。阻塞式 socket 接收与握手使用后 `paste` 提供一种纯文本或 HTML 剪贴板格式，调用应用已启用的标准 Paste 菜单命令，并等待剪贴板数据请求。随后仅在所有权标记和修改计数仍匹配时恢复原有格式。Host 通过文件锁串行化不同 helper 的粘贴。maxClipboardBytes 同时限制输入和保留内容，不支持的菜单和过大的数据在分派前失败。后续复制会被保留。取消会尽可能恢复仍持有的剪贴板数据，强制结束不保证恢复，数据请求也不证明目标已完成编辑。 分派完成后，`pasteReadTimeoutMs` 限制等待剪贴板读取的时间。到期返回 `PASTE_NOT_CONSUMED`，关闭延迟数据提供方，并仅在仍持有剪贴板时恢复原内容。该失败不能证明应用以后不会处理已排队的 Paste 命令；决定是否重试前先检查目标内容。 -默认情况下，`mouseMove` 通过同一窗口内启用的控件或祖先声明的 `AXShowAlternateUI` 和 `AXShowDefaultUI` 展开或恢复界面。缺少这条路径时返回 `HOVER_UNAVAILABLE`，不发送输入。Host 根据部署配置 `systemPointerHover` 传入原生布尔参数 `allowSystemPointer`，省略时为 false。只有值为 true 才允许移动全局光标，且须检查前台进程和鼠标命中的窗口。 +默认情况下，`mouseMove` 通过同一窗口内启用的控件或祖先声明的 `AXShowAlternateUI` 和 `AXShowDefaultUI` 展开或恢复界面。缺少这条路径时返回 `HOVER_UNAVAILABLE`，不发送输入。macOS 提供方在连接握手中设置 `systemPointerHover`。未在握手中指定该配置的诊断客户端，可以在单次请求中传入 `allowSystemPointer: true`。默认值为 false。只有值为 true 才允许移动全局光标，且须检查前台进程和鼠标命中的窗口。 开启后，系统指针路线会检查前台应用和坐标命中的窗口，不满足条件时返回 `FOREGROUND_REQUIRED`。返回值 `hoverRoute` 和 `systemPointerMoved` 说明走了哪条路线以及是否移动了可见光标。悬停不会按下鼠标，也不会在读取后还原光标。后续输入可能中断悬停。点击只在目标点发送按键事件，不先在窗口外点击。 diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift index 73e8e78fd9..96a6054525 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift @@ -78,6 +78,7 @@ actor AXWorker { private let axTimeout: Float private let textEventInterval: Double private let maxClickHoldMs: Int + private let systemPointerHover: Bool? private let sessionState: @Sendable () -> Bool? private var access: AXAccess? private var keyboard: KeyboardInput? @@ -85,12 +86,13 @@ actor AXWorker { private var hovers: [String: AXHover] = [:] private var closed = false - init(cancellation: CancellationLatch, axTimeout: Float, textEventInterval: Double, maxClickHoldMs: Int, + init(cancellation: CancellationLatch, axTimeout: Float, textEventInterval: Double, maxClickHoldMs: Int, systemPointerHover: Bool? = nil, sessionState: @escaping @Sendable () -> Bool? = NativePlatform.sessionLocked) { self.cancellation = cancellation self.axTimeout = axTimeout self.textEventInterval = textEventInterval self.maxClickHoldMs = maxClickHoldMs + self.systemPointerHover = systemPointerHover self.sessionState = sessionState } private func identifier(_ kind: String) throws -> String { @@ -105,7 +107,9 @@ actor AXWorker { return created } private func moveUsingAX(_ element: AXUIElement?, windowID: UInt32, handle: String, params: RequestParameters) throws -> Bool { - let allowSystemPointer = try params.optionalBoolean("allowSystemPointer") ?? false + let requestSetting = try params.optionalBoolean("allowSystemPointer") + // Diagnostic JSONL clients without a deployment setting may opt in per request. + let allowSystemPointer = systemPointerHover ?? requestSetting ?? false let hover = hovers[handle] ?? AXHover() hovers[handle] = hover if try hover.move(to: element, in: windowID, access: ax()) { return true } diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/RequestCoordinator.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/RequestCoordinator.swift index 3a778cb8ae..000a69e848 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/RequestCoordinator.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/RequestCoordinator.swift @@ -42,7 +42,8 @@ import Darwin self.onClose = onClose self.onFatal = onFatal worker = AXWorker(cancellation: cancellation, axTimeout: configuration.axTimeoutSeconds, - textEventInterval: configuration.textEventIntervalSeconds, maxClickHoldMs: configuration.maxClickHoldMs) + textEventInterval: configuration.textEventIntervalSeconds, maxClickHoldMs: configuration.maxClickHoldMs, + systemPointerHover: configuration.systemPointerHover) input = JSONLInput(limit: configuration.maxRequestBytes, cancellation: cancellation, channel: channel) self.writer = writer } diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/Runtime.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/Runtime.swift index d61f0803ab..c31a07ebef 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/Runtime.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/Runtime.swift @@ -21,6 +21,7 @@ struct HelperConfiguration: Sendable { var shutdownGraceSeconds: Double = 3 var textEventIntervalSeconds: Double = 0.00025 var maxClickHoldMs = 5000 + var systemPointerHover: Bool? var timeoutSeconds: Double = 30 var screenshotTimeoutSeconds: Double = 5 init(arguments: [String]) throws { @@ -64,7 +65,7 @@ struct HelperConfiguration: Sendable { guard let fields = decoded.object, Set(fields.keys) == ["type", "protocol", "runtimeKey", "config"], fields["type"] == .string("hello"), fields["protocol"] == .number(1), fields["runtimeKey"] == .string(runtimeKey), let config = fields["config"]?.object, - Set(config.keys) == ["maxRequestBytes", "maxResponseBytes", "axTimeoutMs", "textEventIntervalMs", "maxClickHoldMs", "screenshotTimeoutMs", "timeoutMs", "graceMs"] else { + Set(config.keys).subtracting(["systemPointerHover"]) == ["maxRequestBytes", "maxResponseBytes", "axTimeoutMs", "textEventIntervalMs", "maxClickHoldMs", "screenshotTimeoutMs", "timeoutMs", "graceMs"] else { throw NativeFailure(code: "NATIVE_PROTOCOL", message: "Expected shared helper protocol 1 hello for this runtime") } func number(_ name: String) throws -> String { @@ -82,6 +83,10 @@ struct HelperConfiguration: Sendable { "--screenshot-timeout-ms", number("screenshotTimeoutMs"), "--timeout-ms", number("timeoutMs"), "--shutdown-grace-ms", number("graceMs"), ]) + if let setting = config["systemPointerHover"] { + guard case .bool(let allowed) = setting else { throw RequestParameters.invalid("systemPointerHover must be a boolean") } + systemPointerHover = allowed + } } } diff --git a/packages/experimental/cua-macos/native/tests/swift/SharedHelperTests.swift b/packages/experimental/cua-macos/native/tests/swift/SharedHelperTests.swift index 9b43bc330e..52f7f0f224 100644 --- a/packages/experimental/cua-macos/native/tests/swift/SharedHelperTests.swift +++ b/packages/experimental/cua-macos/native/tests/swift/SharedHelperTests.swift @@ -33,6 +33,14 @@ private func hello(overrides: [String: JSONValue] = [:]) throws -> Data { } } +@Test func readsSystemPointerHoverFromEachConnection() throws { + #expect(try HelperConfiguration(hello: hello(), runtimeKey: runtimeKey).systemPointerHover == nil) + for allowed in [true, false] { + let configuration = try HelperConfiguration(hello: hello(overrides: ["systemPointerHover": .bool(allowed)]), runtimeKey: runtimeKey) + #expect(configuration.systemPointerHover == allowed) + } +} + @Test func resolvesClickHoldLimitFromHelperArguments() throws { #expect(try HelperConfiguration(arguments: []).maxClickHoldMs == 5000) for limit in [0, 7000] { @@ -48,7 +56,7 @@ private func hello(overrides: [String: JSONValue] = [:]) throws -> Data { ("axTimeoutMs", .number(0)), ("textEventIntervalMs", .number(-1)), ("screenshotTimeoutMs", .number(0)), ("timeoutMs", .string("8000")), ("graceMs", .number(-1)), ("maxClickHoldMs", .number(-1)), ("maxClickHoldMs", .number(0.5)), - ("maxClickHoldMs", .number(2147483648)), ("unexpected", .bool(true)), + ("maxClickHoldMs", .number(2147483648)), ("unexpected", .bool(true)), ("systemPointerHover", .string("true")), ]) func refusesInvalidSharedConfiguration(_ name: String, _ value: JSONValue) throws { #expect(throws: NativeFailure.self) { try HelperConfiguration(hello: hello(overrides: [name: value]), runtimeKey: runtimeKey) } diff --git a/packages/experimental/cua-macos/src/guidance.ts b/packages/experimental/cua-macos/src/guidance.ts index 9da4e5f85a..0379d91120 100644 --- a/packages/experimental/cua-macos/src/guidance.ts +++ b/packages/experimental/cua-macos/src/guidance.ts @@ -2,9 +2,23 @@ import { COMMON_GUIDANCE } from '@deepseek-ai/dsh-cua/guidance' /** macOS operation levels, permissions, selectors, and native measurement units. */ -export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' +export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' + +const methodDetails = new Map([ + ['cua.getApp', 'Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background.'], + ['cua.findApps', 'Searches /Applications, /System/Applications, and ~/Applications.'], + ['app.show', 'Unhides the app and restores its focused, main, or sole window without activating it.'], + ['window.show', 'Unhides the app and restores this window without activating it.'], + ['mouseMove', 'Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured.'], + ['click', 'Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click.'], + ['typeText', 'Line endings send Return and tabs send Tab.'], + ['paste', 'Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying.'], +]) /** Complete model instructions for this provider. */ export const GUIDANCE = `${PLATFORM_GUIDANCE} -${COMMON_GUIDANCE}` +${COMMON_GUIDANCE.replace(/^- ([\w.]+)\(.*$/gm, (entry, method: string) => { + const detail = methodDetails.get(method) + return detail ? `${entry} ${detail}` : entry +})}` diff --git a/packages/experimental/cua-macos/src/index.ts b/packages/experimental/cua-macos/src/index.ts index 52788c0752..4b07e9c8b7 100644 --- a/packages/experimental/cua-macos/src/index.ts +++ b/packages/experimental/cua-macos/src/index.ts @@ -4,6 +4,7 @@ import { fileURLToPath } from 'node:url' import type { Context } from '@deepseek-ai/cordis' import Schema from '@deepseek-ai/schemastery' import { installCua, type CuaConfig, type CuaStartupCapabilities } from '@deepseek-ai/dsh-cua' +import type { NativeChannelFactory } from '@deepseek-ai/dsh-cua/native-client' import { CuaConfigFields } from '@deepseek-ai/dsh-cua/config' import { createPermissionParameters } from './permission-guide.ts' import { nativeHostArchitecture, prepareNativeExecutable, type SourceSigning } from './native-build.ts' @@ -63,8 +64,8 @@ export const Config = Schema.object({ * @returns Completion after native preparation and tool registration. */ export async function apply(ctx: Context, config: Config): Promise { - const backend = { createChannel: createSharedNativeChannel, guidance: GUIDANCE, capabilities: CAPABILITIES, - systemPointerHover: config.systemPointerHover, + const createChannel: NativeChannelFactory = (owner, native) => createSharedNativeChannel(owner, native, config.systemPointerHover) + const backend = { createChannel, guidance: GUIDANCE, capabilities: CAPABILITIES, permissionParameters: createPermissionParameters(config.permissionPollIntervalMs) } if (config.helperPath !== undefined) { if (!isAbsolute(config.helperPath)) throw new Error('dsh-cua helperPath must be absolute') diff --git a/packages/experimental/cua-macos/src/shared-native.ts b/packages/experimental/cua-macos/src/shared-native.ts index 0fe2cf6436..7be638a96b 100644 --- a/packages/experimental/cua-macos/src/shared-native.ts +++ b/packages/experimental/cua-macos/src/shared-native.ts @@ -56,7 +56,8 @@ class SharedNativeChannel implements NativeChannel { private cleanupTimer: ReturnType | undefined private cleanupFailure: Error | undefined - constructor(private readonly owner: { subprocess: Pick }, private readonly config: NativeClientConfig) { + constructor(private readonly owner: { subprocess: Pick }, private readonly config: NativeClientConfig, + private readonly systemPointerHover: boolean) { this.stdin = new Writable({ write: (chunk: Buffer, encoding, callback) => { void this.ready.promise.then((socket) => { @@ -214,7 +215,8 @@ class SharedNativeChannel implements NativeChannel { const { maxRequestBytes, maxResponseBytes, axTimeoutMs, textEventIntervalMs, maxClickHoldMs, screenshotTimeoutMs, timeoutMs, graceMs } = this.config socket.write(JSON.stringify({ type: 'hello', protocol: 1, runtimeKey, - config: { maxRequestBytes, maxResponseBytes, axTimeoutMs, textEventIntervalMs, maxClickHoldMs, screenshotTimeoutMs, timeoutMs, graceMs } }) + '\n') + config: { maxRequestBytes, maxResponseBytes, axTimeoutMs, textEventIntervalMs, maxClickHoldMs, + screenshotTimeoutMs, timeoutMs, graceMs, systemPointerHover: this.systemPointerHover } }) + '\n') }) const receive = (chunk: Buffer): void => { const newline = chunk.indexOf(10) @@ -286,8 +288,9 @@ class SharedNativeChannel implements NativeChannel { * Prepare or join the native process; writes wait for identity-checked readiness. * @param owner - Provider owning the short-lived native startup command. * @param config - Native runtime identity, operation limits and connection deadlines. + * @param systemPointerHover - Whether this connection may move the visible pointer when AX hover is unavailable. * @returns One independently disposable Agent connection, without operation replay. */ -export function createSharedNativeChannel(owner: { subprocess: Pick }, config: NativeClientConfig): NativeChannel { - return new SharedNativeChannel(owner, config) +export function createSharedNativeChannel(owner: { subprocess: Pick }, config: NativeClientConfig, systemPointerHover: boolean): NativeChannel { + return new SharedNativeChannel(owner, config, systemPointerHover) } diff --git a/packages/experimental/cua-macos/tests/guidance.spec.ts b/packages/experimental/cua-macos/tests/guidance.spec.ts index 7afb53b84c..c00f600a79 100644 --- a/packages/experimental/cua-macos/tests/guidance.spec.ts +++ b/packages/experimental/cua-macos/tests/guidance.spec.ts @@ -19,7 +19,7 @@ it('keeps exactly three ordered operation headings', () => { }) it('prefixes concise platform guidance before the shared catalog', () => { - expect(GUIDANCE).toBe(`${PLATFORM_GUIDANCE}\n\n${COMMON_GUIDANCE}`) + expect(GUIDANCE.startsWith(`${PLATFORM_GUIDANCE}\n\nUse cua`)).toBe(true) expect(PLATFORM_GUIDANCE).not.toContain('##') }) @@ -27,9 +27,12 @@ it('states the macOS accessibility, keyboard, and mouse levels', () => { expect(PLATFORM_GUIDANCE).toContain('accessibility tree operations are available at app or window level') expect(PLATFORM_GUIDANCE).toContain('Simulated keyboard input is available at app level') expect(PLATFORM_GUIDANCE).toContain('Simulated mouse input is available at window level') - const hover = COMMON_GUIDANCE.split('\n').find(line => line.startsWith('- mouseMove(')) + const hover = GUIDANCE.split('\n').find(line => line.startsWith('- mouseMove(')) for (const term of ['AXShowAlternateUI', 'AXShowDefaultUI', 'HOVER_UNAVAILABLE', 'systemPointerHover']) expect(hover).toContain(term) - expect(COMMON_GUIDANCE.split('\n').find(line => line.startsWith('- click('))).toContain('call app.activate() first') + expect(GUIDANCE.split('\n').find(line => line.startsWith('- click('))).toContain('call app.activate() first') + for (const platformDetail of ['macOS', 'AXShowAlternateUI', 'AXRaise', 'HOVER_UNAVAILABLE', 'systemPointerHover', 'mouse-down', 'bundle ID', 'Paste command']) { + expect(COMMON_GUIDANCE).not.toContain(platformDetail) + } }) it('enumerates every available method once in its catalog entry', () => { @@ -41,7 +44,9 @@ it('enumerates every available method once in its catalog entry', () => { ...windowNames.filter(name => name !== 'windowId').map(name => `${name}(`), ...['typeText', 'paste', 'pressKey'].map(name => `${name}(`), ] - for (const signature of signatures) expect(COMMON_GUIDANCE.split('\n').filter(line => line.startsWith(`- ${signature}`))).toHaveLength(1) + for (const catalog of [COMMON_GUIDANCE, GUIDANCE]) { + for (const signature of signatures) expect(catalog.split('\n').filter(line => line.startsWith(`- ${signature}`))).toHaveLength(1) + } expect(signatures).toHaveLength(24) expect(new Set(signatures.map(signature => signature.slice(signature.indexOf('.') + 1, -1))).size).toBe(23) expect(COMMON_GUIDANCE).toContain('app.appHandle') diff --git a/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts b/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts index 38e632e6ae..017a605ce1 100644 --- a/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts +++ b/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts @@ -71,6 +71,7 @@ const { helperPath: _helperPath, ...automaticConfig } = config it('defaults to ad-hoc signing and requires a keychain path for certificate signing', () => { expect(NativeProvider.Config({}).sourceSigning).toBe('ad-hoc') + expect(NativeProvider.Config({}).systemPointerHover).toBe(false) expect(NativeProvider.Config({ sourceSigning: 'ad-hoc' }).sourceSigning).toBe('ad-hoc') for (const sourceSigning of [{ identity: 'Apple Development: Example', keychain: '/keychains/custom.keychain-db' }, { identity: 'ABC123', keychain: '/keychains/custom.keychain-db' }]) { expect(NativeProvider.Config({ sourceSigning }).sourceSigning).toEqual(sourceSigning) @@ -109,7 +110,7 @@ class ImageAdapter extends LlmAdapter { const imageData = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAADElEQVQImWNgZGIGAAAOAAeCcsnOAAAAAElFTkSuQmCC' const imageMetadata = { pixelWidth: 1, pixelHeight: 1, bounds: { x: 2, y: 3, width: 1, height: 1 }, coordinateSpace: 'window-points', windowId: 7 } -async function setup(model?: { provider: string; model: string }, systemPointerHover = false) { +async function setup(model?: { provider: string; model: string }) { native.request.mockReset().mockResolvedValue([]) native.dispose.mockReset() native.instances.length = 0 @@ -119,7 +120,7 @@ async function setup(model?: { provider: string; model: string }, systemPointerH await mountAgentLoopTestDependencies(ctx) await ctx.plugin(ComputerUseRegistry) await ctx.plugin(Subprocess, []) - const provider = await ctx.plugin(NativeProvider, { ...config, systemPointerHover }) + const provider = await ctx.plugin(NativeProvider, config) await mountAgentLoopTestHarness(ctx) const handle = await ctx.agents.create({ sessionId: SessionId('native-lifecycle'), ...model ? { agentOptions: model } : {} }) let index = 0 @@ -386,26 +387,6 @@ it('rejects malformed and unowned operations before acquiring a helper', async ( expect(native.request).not.toHaveBeenCalled() }) -it.each([false, true])('rejects an ended Agent with prior helper acquisition %s', async (acquired) => { - const f = await setup() - if (acquired) await f.execute() - await f.handle.dispose() - const requests = native.request.mock.calls.length - const result = await f.execute() - expect(result.isError).toBe(true) - expect(JSON.stringify(result.content)).toContain('agent has been disposed') - expect(native.instances).toHaveLength(acquired ? 1 : 0) - expect(native.request).toHaveBeenCalledTimes(requests) -}) - -it.each([false, true])('passes the deployment hover policy (%s) to native input', async (allowed) => { - const f = await setup(undefined, allowed) - await selectApplication(f) - const params = { appHandle: 'app-one', windowId: 7, snapshotId: 's', elementId: '0' } - await f.execute('mouseMove', params) - expect(native.request).toHaveBeenLastCalledWith('mouseMove', { ...params, allowSystemPointer: allowed }, expect.any(AbortSignal), expect.any(String)) -}) - it('supplies configured resource bounds and clears registrations on unload', async () => { const f = await setup() await selectApplication(f) diff --git a/packages/experimental/cua-macos/tests/shared-native.spec.ts b/packages/experimental/cua-macos/tests/shared-native.spec.ts index cef346ab9b..871475fb53 100644 --- a/packages/experimental/cua-macos/tests/shared-native.spec.ts +++ b/packages/experimental/cua-macos/tests/shared-native.spec.ts @@ -133,8 +133,8 @@ async function fixture(options: { timeoutMs: 30_000, graceMs: options.graceMs ?? 1000, axTimeoutMs: 3000, maxRequestBytes: 65536, maxResponseBytes: 1048576, textEventIntervalMs: 0.5, maxClickHoldMs: 7000, } - function open(): NativeChannel { - const channel = createSharedNativeChannel({ subprocess: { spawn } }, config) + function open(systemPointerHover = false): NativeChannel { + const channel = createSharedNativeChannel({ subprocess: { spawn } }, config, systemPointerHover) channels.push(channel) return channel } @@ -172,7 +172,7 @@ describe.skipIf(process.platform === 'win32')('shared native Unix connection', ( it('keeps concurrent Agent scopes independent and leaves other sockets alive when one closes', async () => { const f = await fixture() const first = f.open() - const second = f.open() + const second = f.open(true) const firstReply = nextBytes(first) const secondReply = nextBytes(second) await Promise.all([write(first, 'one\n'), write(second, 'two\n')]) @@ -186,10 +186,14 @@ describe.skipIf(process.platform === 'win32')('shared native Unix connection', ( expect(f.spawn).toHaveBeenCalledTimes(2) // oxlint-disable-next-line typescript/unbound-method -- Fixture termination methods are spies inspected without invocation. for (const handle of f.handles) expect(handle.terminate).not.toHaveBeenCalled() - expect(f.hellos).toEqual([expect.objectContaining({ type: 'hello', protocol: 1, config: { + const limits = { axTimeoutMs: 3000, timeoutMs: 30000, screenshotTimeoutMs: 5000, graceMs: 1000, maxRequestBytes: 65536, maxResponseBytes: 1048576, textEventIntervalMs: 0.5, maxClickHoldMs: 7000, - } }), expect.anything()]) + } + expect(f.hellos).toHaveLength(2) + for (const systemPointerHover of [true, false]) { + expect(f.hellos).toContainEqual(expect.objectContaining({ type: 'hello', protocol: 1, config: { ...limits, systemPointerHover } })) + } }) it('waits for both launcher outcome and managed range release before sending any bytes', async () => { diff --git a/packages/experimental/cua/README.i18n.yaml b/packages/experimental/cua/README.i18n.yaml index dd3c6d217a..f4abb13c43 100644 --- a/packages/experimental/cua/README.i18n.yaml +++ b/packages/experimental/cua/README.i18n.yaml @@ -15,8 +15,8 @@ en: 5db4d4fea25493b9 zh: 65ab64fc24eb0d09 /deepseek-ai-dsh-cua/use-this-package: - en: d0088d2ec1ab8cf2 - zh: d927289a63d7a564 + en: ba1f8e6ae51f69d7 + zh: 8f4fde9857bae319 /deepseek-ai-dsh-cua/use-this-package/permission-controls: en: d92d5948a7941e8d zh: d43491ae6521b87e diff --git a/packages/experimental/cua/README.md b/packages/experimental/cua/README.md index 365c549650..c95e4ae8ec 100644 --- a/packages/experimental/cua/README.md +++ b/packages/experimental/cua/README.md @@ -43,7 +43,7 @@ Application consent precedes launch, observation, capture, and input. Session ac Actions return a `dispatched` boolean; `mouseMove` also reports `hoverRoute` and `systemPointerMoved`, and `show()` adds `windowId` and `foregroundChanged`. Read fresh state to verify the result. Screenshots and pointer actions are available only on explicit window objects. Coordinate actions default to pixels in the latest returned screenshot; `relative: true` accepts integers from 0 to 999 after a successful window observation. Values are divided by 1000 and scaled to the window dimensions, then clamped inside the window. The Host resolves attachment normalization and the selected model route’s image projection before returning the screenshot, then converts input to the provider’s native window coordinates. Resizing invalidates the screenshot mapping; `SCREENSHOT_REQUIRED` requires a new screenshot for absolute coordinates. Resizing rejects old coordinates with `WINDOW_CHANGED`; movement alone preserves window-local coordinates. -`click(target, { holdMs: 800 })` holds the mouse button for 800 milliseconds before release. `holdMs` applies only to `click`; it accepts nonnegative integers and applies to each press of a double-click; zero releases without an additional hold. The provider config `maxClickHoldMs` limits explicit holds and defaults to 5000 ms. On macOS, both ordinary and held clicks send mouse events at the element center or the supplied point. Use `invoke(id)` for the default accessibility action. The operation deadline includes the hold, and cancellation or timeout releases the button. Received input cannot be undone; forced helper termination cannot guarantee release. +`click(target, { holdMs: 800 })` holds the mouse button for 800 milliseconds before release. `holdMs` applies only to `click`; it accepts nonnegative integers and applies to each press of a double-click; zero releases without an additional hold. The provider config `maxClickHoldMs` limits explicit holds and defaults to 5000 ms. Explicit holds use pointer input; omission preserves ordinary click behavior. The operation deadline includes the hold, and cancellation or timeout releases the button. Received input cannot be undone; forced helper termination cannot guarantee release. ### Permission controls diff --git a/packages/experimental/cua/README.zh.md b/packages/experimental/cua/README.zh.md index b64c445c44..c1c5ed863c 100644 --- a/packages/experimental/cua/README.zh.md +++ b/packages/experimental/cua/README.zh.md @@ -43,7 +43,7 @@ kind: "package-reference" 动作返回 `dispatched` 布尔值；`mouseMove` 还会返回 `hoverRoute` 和 `systemPointerMoved`，`show()` 还会返回 `windowId` 和 `foregroundChanged`。请读取新状态以验证结果。截图和指针操作只能通过显式窗口对象使用。坐标操作默认使用最近返回截图中的像素；成功观察窗口后，传入 `relative: true` 可使用 0～999 的整数坐标。数值除以 1000 后按窗口尺寸缩放，并限制在窗口内。Host 在返回截图前解析附件归一化和当前模型的图片缩放规则，再将输入换算为提供方的原生窗口坐标。窗口尺寸变化会使截图映射失效，绝对坐标遇到 `SCREENSHOT_REQUIRED` 时需要重新截图。窗口缩放后以 `WINDOW_CHANGED` 拒绝旧坐标；单纯移动窗口不改变窗口内坐标。 -`click(target, { holdMs: 800 })` 按住鼠标按钮 800 毫秒后松开。`holdMs` 仅适用于 `click`；它接受非负整数，双击时应用于每次按下；0 表示按下后立即松开，不额外等待。提供方配置 `maxClickHoldMs` 限制显式按住时长，默认 5000 毫秒。在 macOS 上，无论是否指定按住时长，点击都会向元素中心或给定坐标发送鼠标事件。要执行控件的默认辅助功能动作，请用 `invoke(id)`。按住时间计入操作超时，取消或超时会松开按钮。已接收的输入无法撤销；辅助进程被强制终止时无法保证松开。 +`click(target, { holdMs: 800 })` 按住鼠标按钮 800 毫秒后松开。`holdMs` 仅适用于 `click`；它接受非负整数，双击时应用于每次按下；0 表示按下后立即松开，不额外等待。提供方配置 `maxClickHoldMs` 限制显式按住时长，默认 5000 毫秒。指定时长后使用鼠标事件；省略时保留普通点击行为。按住时间计入操作超时，取消或超时会松开按钮。已接收的输入无法撤销；辅助进程被强制终止时无法保证松开。 ### 权限控件 diff --git a/packages/experimental/cua/src/guidance.ts b/packages/experimental/cua/src/guidance.ts index 1d6d0266d1..27be40b4c7 100644 --- a/packages/experimental/cua/src/guidance.ts +++ b/packages/experimental/cua/src/guidance.ts @@ -1,5 +1,4 @@ /** Shared model instructions for native application and window operations. */ -/** Async method catalog with consent and observation rules, grouped by observation, mouse, and keyboard. */ export const COMMON_GUIDANCE = `Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. In persistent PTC, use the host-provided bridge initialization once and retain the returned objects; cua comes from that bridge and is not a preinstalled global. Use void await for observations that emit, and never stringify screenshot image bytes. @@ -9,20 +8,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. -- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. +- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. +- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -32,17 +31,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. +- typeText(text: string) -> ActionResult — type text at the current input focus. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name.` diff --git a/packages/experimental/cua/src/index.ts b/packages/experimental/cua/src/index.ts index 7cb5177f71..d6c75f46bc 100644 --- a/packages/experimental/cua/src/index.ts +++ b/packages/experimental/cua/src/index.ts @@ -126,8 +126,6 @@ export interface CuaBackend { createChannel: NativeChannelFactory guidance: string capabilities: CuaStartupCapabilities - /** macOS deployment policy sent only to the native mouseMove operation. */ - systemPointerHover?: boolean /** * Build native permission parameters for the selected action and Client language. * @param method Native permission action. @@ -213,7 +211,6 @@ export function installCua(ctx: Context, config: CuaConfig, backend: CuaBackend) const signal = execution.signal signal.throwIfAborted() const client = execution.nativeClient - if (method === 'mouseMove' && backend.systemPointerHover !== undefined) params.allowSystemPointer = backend.systemPointerHover const nativeParams = method === 'findApps' ? { ...params, maxAppMatches: config.maxAppMatches } : method === 'paste' diff --git a/packages/experimental/cua/src/native-schema.ts b/packages/experimental/cua/src/native-schema.ts index bbd52e2591..a6122ed71e 100644 --- a/packages/experimental/cua/src/native-schema.ts +++ b/packages/experimental/cua/src/native-schema.ts @@ -13,10 +13,10 @@ function variants(schema: Schema): Schema[] { return schema.anyOf ?? schema.oneO /** * Derive native request schemas and admission fields from the shared operation definitions. - * @param platformFields Platform-specific parameters added by the Host before native dispatch. + * @param platformFields Additional native protocol fields accepted by the provider. * @returns Native field and enum tables and the complete wire schema. */ -export function nativeProtocol(platformFields: Partial> = {}): { +export function nativeProtocol(platformFields: Partial['method'] | 'requestPermission' | 'openPermissionSettings', z.ZodObject>> = {}): { fields: Map> enums: Map>> json: string diff --git a/packages/experimental/cua/src/types.ts b/packages/experimental/cua/src/types.ts index 4e5f1ef0d9..4c6b27b78e 100644 --- a/packages/experimental/cua/src/types.ts +++ b/packages/experimental/cua/src/types.ts @@ -79,6 +79,7 @@ export type CuaMouseButton = NonNullable['mouseButton']> /** * Left clicks support one or two presses; right and middle clicks support one. * holdMs is a nonnegative integer in milliseconds, limited by maxClickHoldMs; zero releases immediately. + * Explicit holds use pointer input; omission preserves ordinary clicks. */ export type CuaClickOptions = Pick, 'mouseButton' | 'clickCount' | 'modifiers' | 'holdMs'> & CuaCoordinateOptions /** One same-window drag; omitted button is left and omitted modifiers are empty. */ @@ -95,7 +96,7 @@ export type CuaScrollOptions = Pick, 'x' | 'y' | 'deltaL */ export interface CuaSurface { /** - * Unhide the app and restore a minimized selected, focused, main or sole window without requesting activation. + * Request that the provider show the selected application or window. * @returns Dispatch evidence, the selected window when available, and sampled foreground change; read fresh state before input. */ show(): Promise @@ -190,22 +191,17 @@ export interface CuaWindow extends CuaSurface { */ getScreenshot(options?: CuaObservationOptions): Promise /** - * On macOS, prefer an available control's click or advertised accessibility action over hover. - * By default, mouseMove reveals or restores alternate UI on an enabled target or ancestor that advertises - * both AXShowAlternateUI and AXShowDefaultUI. Without an AX route it throws HOVER_UNAVAILABLE before sending input. - * Only the macOS provider's systemPointerHover setting permits moving the visible cursor as a fallback. - * That route requires a foreground app and an unobscured target and leaves the cursor at the destination. + * Request hover at an element or a point in this window. The provider reports the route used and whether + * the visible system pointer moved. Observe the resulting UI after its hover delay. * @param target - current element id or screenshot-local [x,y]. * @param options - coordinate mode; applies only to [x,y]. * @returns dispatch evidence and the selected route; observe the resulting UI after its hover delay. */ mouseMove(target: CuaElementId | readonly [number, number], options?: CuaCoordinateOptions): Promise /** - * Click an element or a point in this window. On macOS both forms send mouse-down and mouse-up events. - * For a background control, prefer {@link CuaSurface.invoke} for its default action. When a mouse click is needed, - * call app.activate() first and observe the target again. Background apps can ignore dispatched mouse events. + * Click a current element id or screenshot-local point. * Cancellation or timeout releases the held button; received input cannot be undone. The operation deadline includes holdMs. - * @param target - On macOS, an id from the latest tree targets the element center. A coordinate pair targets the given point. + * @param target - element id or [x,y]. * @param options - button, click count, modifiers, optional hold duration and coordinate mode. * @returns dispatch evidence. */ @@ -274,9 +270,9 @@ export interface CuaApi { */ getState(options?: CuaObservationOptions): Promise<{ apps: CuaAppInfo[]; browsers: [] }> /** - * Select an app, obtain consent, launch it in the background if necessary, and silently retain initial AX state. - * @param selector - Exact name, bundle ID, absolute application path or running PID string; ambiguous names reject. - * @returns An application handle tied to this process lifetime; its initial tree read becomes the current snapshot. + * Select an app and obtain consent. Providers with app-level accessibility silently read the initial tree. + * @param selector - A selector accepted by the provider; ambiguous matches reject. + * @returns An application handle tied to this process lifetime. */ getApp(selector: string): Promise } diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index 1738793701..5a793421cf 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -33,7 +33,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -44,20 +44,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. -- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. +- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. +- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -67,17 +67,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. +- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. diff --git a/snapshots/session/cua-native/workspace/native-helper.mjs b/snapshots/session/cua-native/workspace/native-helper.mjs index 0786eeefe6..1e148d7ab5 100755 --- a/snapshots/session/cua-native/workspace/native-helper.mjs +++ b/snapshots/session/cua-native/workspace/native-helper.mjs @@ -47,6 +47,7 @@ function serve() { const server = createServer({ allowHalfOpen: true }, socket => { sockets.add(socket) let connected = false + let systemPointerHover const connectionId = ++connection const input = createInterface({ input: socket }) const scopedRecord = (event, data = {}) => record(event, { connection: connectionId, ...data }) @@ -58,6 +59,7 @@ function serve() { socket.destroy() return } + systemPointerHover = request.config.systemPointerHover connected = true hadClient = true clearTimeout(startupDeadline) @@ -71,7 +73,7 @@ function serve() { } let response try { - const result = call(request.id, request.method, request.params) + const result = call(request.id, request.method, request.params, systemPointerHover) if (result === undefined) return response = { id: request.id, result } } catch (error) { @@ -122,7 +124,7 @@ function createApplicationState(socket, record, application, captureBroker, conn let observationChanged = false let malformedScreenshot = false function refuse(code, message) { throw Object.assign(new Error(message), { code }) } - function call(requestId, method, params) { + function call(requestId, method, params, systemPointerHover) { record('call', { requestId, method, params }) if (method === 'applicationIcons') return Object.fromEntries(params.paths.map(path => [path, `data:image/png;base64,${png}`])) if (method === 'openPermissionSettings') return { opened: true } @@ -203,7 +205,7 @@ function createApplicationState(socket, record, application, captureBroker, conn if (method === 'mouseMove') { const alternate = params.elementId === '0' || (params.x >= 20 && params.x <= 40) if (!alternate && !application.hovering) { - if (!params.allowSystemPointer) refuse('HOVER_UNAVAILABLE', 'No AX alternate UI at this target. No input was sent and the pointer did not move.') + if (!(systemPointerHover ?? params.allowSystemPointer)) refuse('HOVER_UNAVAILABLE', 'No AX alternate UI at this target. No input was sent and the pointer did not move.') systemPointerMoved = true } application.hovering = alternate diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index ff29eddb7f..8e3daa7253 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -46,20 +46,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. -- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. +- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. +- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -69,17 +69,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. +- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index 434e92e3f7..fcd315e68c 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. Default hover uses AX alternate UI and keeps the system cursor in place. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). getApp also accepts bundle IDs and .app paths, and findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -46,20 +46,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. -- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. +- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. +- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -69,17 +69,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — on macOS, prefer acting on an available control. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on the element or an enabled ancestor. Without an AX route it returns HOVER_UNAVAILABLE and sends no input. For browser content without those actions, use browser tools. Use click for a visible target or performAction for an advertised AXShowAlternateUI, AXShowMenu or AXPress. Only systemPointerHover: true permits moving the visible cursor; this requires a foreground, unobscured target. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — on macOS, send mouse-down and mouse-up events at the center of an element in this window (id) or at the given point ([x, y]). Prefer invoke(id) for a background control’s default action. When a mouse click is needed, call app.activate() first and observe the target again: a background app can ignore a dispatched click. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. For hover-only content, prefer an available click or advertised accessibility action. With system-pointer hover enabled, FOREGROUND_REQUIRED means activate the app explicitly and check that the point is unobscured. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. +- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. From 0f15e8766abf883ba0566d876782aeb55b1885be Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 16:19:01 +0800 Subject: [PATCH 27/44] Refresh the configuration catalog and retain Agent disposal coverage --- docs/config-catalog.md | 2 +- docs/config-catalog.zh.md | 2 +- .../cua-macos/tests/provider-lifecycle.spec.ts | 12 ++++++++++++ 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/docs/config-catalog.md b/docs/config-catalog.md index e6fafb2ae4..230dcfd070 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1112,7 +1112,7 @@ export interface Config { - `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents` - `refs`: [`CuaConfig`](../packages/experimental/cua/src/index.ts) -- `source`: [`packages/experimental/cua-macos/src/index.ts:32`](../packages/experimental/cua-macos/src/index.ts) +- `source`: [`packages/experimental/cua-macos/src/index.ts:33`](../packages/experimental/cua-macos/src/index.ts) ```ts config-catalog /** Native preparation, source-build signing, operation bounds, and profile-local application grants. */ diff --git a/docs/config-catalog.zh.md b/docs/config-catalog.zh.md index 7aa0c7245c..663c726c2a 100644 --- a/docs/config-catalog.zh.md +++ b/docs/config-catalog.zh.md @@ -1114,7 +1114,7 @@ export interface Config { - `inject`: `computerUse` · `tools` · `systemPrompt` · `subprocess` · `agents` - `refs`: [`CuaConfig`](../packages/experimental/cua/src/index.ts) -- `source`: [`packages/experimental/cua-macos/src/index.ts:32`](../packages/experimental/cua-macos/src/index.ts) +- `source`: [`packages/experimental/cua-macos/src/index.ts:33`](../packages/experimental/cua-macos/src/index.ts) ```ts config-catalog /** Native preparation, source-build signing, operation bounds, and profile-local application grants. */ diff --git a/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts b/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts index 017a605ce1..48b9eb99e3 100644 --- a/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts +++ b/packages/experimental/cua-macos/tests/provider-lifecycle.spec.ts @@ -387,6 +387,18 @@ it('rejects malformed and unowned operations before acquiring a helper', async ( expect(native.request).not.toHaveBeenCalled() }) +it.each([false, true])('rejects an ended Agent with prior helper acquisition %s', async (acquired) => { + const f = await setup() + if (acquired) await f.execute() + await f.handle.dispose() + const requests = native.request.mock.calls.length + const result = await f.execute() + expect(result.isError).toBe(true) + expect(JSON.stringify(result.content)).toContain('agent has been disposed') + expect(native.instances).toHaveLength(acquired ? 1 : 0) + expect(native.request).toHaveBeenCalledTimes(requests) +}) + it('supplies configured resource bounds and clears registrations on unload', async () => { const f = await setup() await selectApplication(f) From 5e7532f8fafe6b6685b04babf2d6ad85d9a449b1 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 16:21:51 +0800 Subject: [PATCH 28/44] Verify macOS hover permission stays with its connection --- .../cua-hover-ax-only/guide.i18n.yaml | 4 ++-- .../v0.2.2-rc.1/cua-hover-ax-only/guide.md | 2 +- .../v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md | 2 +- .../native/Sources/CuaNative/AXHover.swift | 5 +++-- .../native/Sources/CuaNative/AXWorker.swift | 4 +--- .../native/tests/swift/SystemHoverTests.swift | 19 +++++++++++-------- 6 files changed, 19 insertions(+), 17 deletions(-) diff --git a/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.i18n.yaml b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.i18n.yaml index 69471af235..9a48f7428c 100644 --- a/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.i18n.yaml +++ b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.i18n.yaml @@ -12,5 +12,5 @@ en: 55c3070b24b4ad39 zh: 2555d127b0f4e4a0 /macos-hover-keeps-the-system-pointer-in-place-by-default/migration: - en: bfa15519fd2afb89 - zh: b350e3a7aa2aaa7a + en: 1f0ab7e8741e821f + zh: f59adc3a23ebcb43 diff --git a/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.md b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.md index fdd3d811eb..f7191cc254 100644 --- a/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.md +++ b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.md @@ -14,5 +14,5 @@ On macOS, `win.mouseMove(target)` uses an enabled element or ancestor that adver ## Migration 1. For an available control, use its advertised action or click it directly. Use browser tools for web content that requires page hover. -2. To restore visible pointer movement, set `systemPointerHover: true` in the `config` of the macOS provider entry (`id: cua`) in your profile's `cordis.patch.yml`. Preserve the entry's other settings. Restart the profile. +2. To restore visible pointer movement, set `systemPointerHover: true` in the `config` of the macOS provider entry (`id: cua`) in your profile's `cordis.patch.yml`. Preserve the entry's other settings. If you set `helperPath`, rebuild the custom helper so it accepts `systemPointerHover` in the connection handshake. Restart the profile. 3. Call `win.mouseMove` on a foreground, unobscured control without AX alternate UI. The opt-in result reports `hoverRoute: "system-pointer"` and `systemPointerMoved: true`. Keep the default to avoid disturbing the user's cursor. diff --git a/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md index 7202988db2..713ae43fb3 100644 --- a/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md +++ b/docs/upgrade-guide/v0.2.2-rc.1/cua-hover-ax-only/guide.zh.md @@ -14,5 +14,5 @@ description: "macOS 的 mouseMove 默认要求控件支持 AX 备用界面，否 ## 迁移 1. 对已经可用的控件，可执行它声明的动作或直接点击。需要网页悬停时可用浏览器工具。 -2. 要恢复可见光标移动，在 profile 的 `cordis.patch.yml` 中，给 macOS 提供方条目（`id: cua`）的 `config` 添加 `systemPointerHover: true`。保留该条目的其他设置，重启 profile。 +2. 要恢复可见光标移动，在 profile 的 `cordis.patch.yml` 中，给 macOS 提供方条目（`id: cua`）的 `config` 添加 `systemPointerHover: true`。保留该条目的其他设置。如果指定了 `helperPath`，先重新构建自定义 helper，使其在建立连接时能读取 `systemPointerHover`，然后重启 profile。 3. 对前台未受遮挡且不提供 AX 备用界面的控件调用 `win.mouseMove`。启用配置时，结果为 `hoverRoute: "system-pointer"` 和 `systemPointerMoved: true`。要保留用户的光标位置，使用默认配置。 diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift index ecab67f6d7..17b142d113 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXHover.swift @@ -2,8 +2,9 @@ import ApplicationServices final class AXHover { - static func requireSystemPointer(allowed: Bool) throws { - guard allowed else { + /// Only diagnostic clients without a connection policy may opt in per request. + static func requireSystemPointer(connectionSetting: Bool?, requestSetting: Bool?) throws { + guard connectionSetting ?? requestSetting ?? false else { throw NativeFailure(code: "HOVER_UNAVAILABLE", message: "No enabled element at this target or in its ancestors offers both AXShowAlternateUI and AXShowDefaultUI. System-pointer hover is disabled; this provider has no other pointer-free hover route. No input was sent and the pointer did not move. Use click on an available control or performAction with an action from listActions.") } } diff --git a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift index 96a6054525..c4cc6066c8 100644 --- a/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift +++ b/packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift @@ -108,12 +108,10 @@ actor AXWorker { } private func moveUsingAX(_ element: AXUIElement?, windowID: UInt32, handle: String, params: RequestParameters) throws -> Bool { let requestSetting = try params.optionalBoolean("allowSystemPointer") - // Diagnostic JSONL clients without a deployment setting may opt in per request. - let allowSystemPointer = systemPointerHover ?? requestSetting ?? false let hover = hovers[handle] ?? AXHover() hovers[handle] = hover if try hover.move(to: element, in: windowID, access: ax()) { return true } - try AXHover.requireSystemPointer(allowed: allowSystemPointer) + try AXHover.requireSystemPointer(connectionSetting: systemPointerHover, requestSetting: requestSetting) return false } private func inputs() -> (KeyboardInput, WindowPointerInput) { diff --git a/packages/experimental/cua-macos/native/tests/swift/SystemHoverTests.swift b/packages/experimental/cua-macos/native/tests/swift/SystemHoverTests.swift index 8bb26dfdb9..df48871c4c 100644 --- a/packages/experimental/cua-macos/native/tests/swift/SystemHoverTests.swift +++ b/packages/experimental/cua-macos/native/tests/swift/SystemHoverTests.swift @@ -3,15 +3,18 @@ import Testing import CoreGraphics @testable import CuaNative -@Test func unavailableAXHoverRequiresExplicitSystemPointerPermission() throws { - do { - try AXHover.requireSystemPointer(allowed: false) - Issue.record("A default hover without an AX route was allowed to move the pointer") - } catch let failure as NativeFailure { - #expect(failure.code == "HOVER_UNAVAILABLE") - #expect(failure.message.contains("No input was sent")) +@Test func unavailableAXHoverHonorsConnectionPolicyBeforeRequestFlags() throws { + for (connection, request): (Bool?, Bool?) in [(nil, nil), (nil, false), (false, nil), (false, true)] { + do { + try AXHover.requireSystemPointer(connectionSetting: connection, requestSetting: request) + Issue.record("An unapproved request was allowed to move the pointer") + } catch let failure as NativeFailure { + #expect(failure.code == "HOVER_UNAVAILABLE") + #expect(failure.message.contains("No input was sent")) + } } - try AXHover.requireSystemPointer(allowed: true) + try AXHover.requireSystemPointer(connectionSetting: true, requestSetting: nil) + try AXHover.requireSystemPointer(connectionSetting: nil, requestSetting: true) } @Test func systemHoverRequiresTheForegroundUnobscuredTarget() throws { From 4736af3b0066b0f1eb9cc2cdba994d84d34b3a56 Mon Sep 17 00:00:00 2001 From: Turtle Date: Fri, 6 Nov 2026 16:22:06 +0800 Subject: [PATCH 29/44] test(ui-goal): drop unnecessary button cast Co-authored-by: Yifffan <133629972+Yifffan@users.noreply.github.com> --- packages/client/ui-goal/tests/goalbar.client.spec.tsx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/client/ui-goal/tests/goalbar.client.spec.tsx b/packages/client/ui-goal/tests/goalbar.client.spec.tsx index 0016781fbb..aa86379524 100644 --- a/packages/client/ui-goal/tests/goalbar.client.spec.tsx +++ b/packages/client/ui-goal/tests/goalbar.client.spec.tsx @@ -158,7 +158,7 @@ describe('GoalBar', () => { const actions = makeActions() render() expect(screen.getAllByRole('button').map(b => b.getAttribute('aria-label'))).toEqual(['编辑目标', '清除目标', '暂停目标']) - expect((screen.getByRole('button', { name: '暂停目标' }) as HTMLButtonElement).disabled).toBe(true) + expect(screen.getByRole('button', { name: '暂停目标' }).hasAttribute('disabled')).toBe(true) }) it('active disarmed goal: "待继续的目标" with a resume action instead of pause', () => { From 7dc864a931d6a42286ebb7b75a3f717475ed36f0 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Fri, 6 Nov 2026 17:05:05 +0800 Subject: [PATCH 30/44] fix(forge-github): read pinned diffs through the compare API The GitHub provider now reads headRefOid and baseRefOid in one GraphQL query, returns `stale` when the head moved, and reads the patch from `compare/...` instead of bracketing `gh pr diff` with two head reads. Both ends are fixed commits. The compare read lives in the internal `readComparison` helper so commit-range diffs can reuse it. For every open and merged pull request measured, the compare patch is byte-identical to `gh pr diff`. Unlike the pull request diff, it returns diffs over 300 files, so such a pull request now shows its complete diff; only output over `maxOutputBytes` or GitHub's HTTP 406 / `too_large` refusal is `too-large`. The read passes `--allow-escape-sequences`, which gh 2.97 and newer require to print patches whose files contain terminal escape sequences. --- apps/web/tests/forge-panel.e2e.ts | 84 +++++++++++----- .../forge-pinned-diff/guide.i18n.yaml | 8 +- .../v0.2.2-rc.1/forge-pinned-diff/guide.md | 4 +- .../v0.2.2-rc.1/forge-pinned-diff/guide.zh.md | 4 +- .../client-ui-forge/README.i18n.yaml | 4 +- .../experimental/client-ui-forge/README.md | 2 +- .../experimental/client-ui-forge/README.zh.md | 2 +- .../forge-github/README.i18n.yaml | 12 +-- packages/experimental/forge-github/README.md | 7 +- .../experimental/forge-github/README.zh.md | 7 +- .../experimental/forge-github/src/compare.ts | 35 +++++++ .../experimental/forge-github/src/graphql.ts | 33 +++++-- .../experimental/forge-github/src/provider.ts | 65 ++++++------- .../experimental/forge-github/src/urls.ts | 12 +++ .../forge-github/tests/compare.spec.ts | 30 ++++++ .../forge-github/tests/graphql.spec.ts | 16 ++-- .../forge-github/tests/provider.spec.ts | 96 ++++++++++--------- 17 files changed, 276 insertions(+), 145 deletions(-) create mode 100644 packages/experimental/forge-github/src/compare.ts create mode 100644 packages/experimental/forge-github/tests/compare.spec.ts diff --git a/apps/web/tests/forge-panel.e2e.ts b/apps/web/tests/forge-panel.e2e.ts index 656f5b91e4..bb1ea51c5a 100644 --- a/apps/web/tests/forge-panel.e2e.ts +++ b/apps/web/tests/forge-panel.e2e.ts @@ -43,8 +43,10 @@ const run = promisify(execFile) * 7 with one failed and one passed check run, one unresolved thread, and a * required review, and pull request 9, an approved draft whose checks passed. * `gh pr ready 9` and the REST merge of pull request 9 change what later - * queries report, through `state.json`. `gh pr diff N` prints `pr-N.diff`, or - * fails with `pr-N.refused` on standard error when that file exists, and the raw + * queries report, through `state.json`, which can also move the head of pull + * request 9. The compare API between a pull request's recorded base and its + * head prints `pr-N.diff`, or, when `pr-N.refused` exists, fails with that JSON + * error body and the matching diagnostic as GitHub's HTTP 406 refusal. The raw * contents API serves `contents.json`, keyed by `path@commit`. Every invocation * is appended to `calls.jsonl`. */ @@ -87,13 +89,24 @@ if (args[0] === 'api' && args[4] === 'POST' && ['repos/octo/demo/pulls/7/comment process.stdout.write(JSON.stringify({ html_url: 'https://github.com/octo/demo/pull/7#pullrequestreview-1' })) process.exit(0) } -if (args[0] === 'pr' && args[1] === 'diff') { - const refused = join(here, 'pr-' + args[2] + '.refused') - if (existsSync(refused)) { - process.stderr.write(readFileSync(refused, 'utf8')) +const BASE = 'e'.repeat(40) +const heads = () => ({ 7: 'a'.repeat(40), 9: state.head ?? 'b'.repeat(40) }) +if (args[0] === 'api' && args.includes('Accept: application/vnd.github.diff')) { + const [, range] = args.at(-1).split('/compare/') + const [base, head] = range.split('...') + const number = Object.entries(heads()).find(([, oid]) => oid === head)?.[0] + if (base !== BASE || number === undefined) { + process.stderr.write('gh: Not Found (HTTP 404)\\n') process.exit(1) } - await new Promise(resolve => process.stdout.write(readFileSync(join(here, 'pr-' + args[2] + '.diff'), 'utf8'), resolve)) + const refused = join(here, 'pr-' + number + '.refused') + if (existsSync(refused)) { + const message = readFileSync(refused, 'utf8') + process.stdout.write(JSON.stringify({ message, errors: [{ resource: 'PullRequest', field: 'diff', code: 'too_large' }], status: '406' })) + process.stderr.write('gh: ' + message + ' (HTTP 406)\\n') + process.exit(1) + } + await new Promise(resolve => process.stdout.write(readFileSync(join(here, 'pr-' + number + '.diff'), 'utf8'), resolve)) process.exit(0) } if (args[0] === 'api' && args[3] === '-H' && args[4] === 'Accept: application/vnd.github.raw+json') { @@ -132,7 +145,7 @@ if (args[0] === 'api' && args[1] === 'graphql') { 9: { id: 'pull-9', number: 9, url: 'https://github.com/octo/demo/pull/9', title: 'Add the merge button', state: state.mergedAt === null ? 'OPEN' : 'MERGED', isDraft: !state.ready, updatedAt: '2026-10-09T00:00:00Z', closedAt: state.mergedAt, author: { login: 'octo' }, - headRefName: 'feat/merge', headRefOid: state.head ?? 'b'.repeat(40), baseRefName: 'main', additions: 30, deletions: 2, mergeable: 'MERGEABLE', + headRefName: 'feat/merge', headRefOid: heads()[9], baseRefName: 'main', additions: 30, deletions: 2, mergeable: 'MERGEABLE', mergeStateStatus: state.ready ? 'CLEAN' : 'BLOCKED', reviewDecision: 'APPROVED', reviewThreads: { pageInfo: { hasNextPage: false, endCursor: null }, nodes: [] }, commits: { nodes: [{ commit: { statusCheckRollup: { id: 'rollup-9', state: 'SUCCESS', contexts: { @@ -142,7 +155,7 @@ if (args[0] === 'api' && args[1] === 'graphql') { } if (query.includes('pullRequest(number: $number)')) { const number = Number(args.find(arg => arg.startsWith('number=')).slice('number='.length)) - process.stdout.write(JSON.stringify({ data: { repository: { pullRequest: { headRefOid: pulls[number].headRefOid } } } })) + process.stdout.write(JSON.stringify({ data: { repository: { pullRequest: { headRefOid: pulls[number].headRefOid, baseRefOid: BASE } } } })) process.exit(0) } if (query.includes('node(id: $id)')) { @@ -171,13 +184,14 @@ process.exit(1) /** Head commit of pull request 7 in the GitHub CLI double. */ const HEAD7 = 'a'.repeat(40) -/** What gh 2.100 prints for the diff of pull request 9 when GitHub refuses it for having over 300 files. */ -const PR9_REFUSAL = [ - 'could not find pull request diff: HTTP 406: Sorry, the diff exceeded the maximum number of files (300). Consider using \'List pull ' - + 'requests files\' API or locally cloning the repository instead. (https://api.github.com/repos/octo/demo/pulls/9)', - 'PullRequest.diff too_large', - '', -].join('\n') +/** Patch of pull request 9: one line added to each of 301 files, more than the pull request diff endpoint accepts. */ +const PR9_DIFF = Array.from({ length: 301 }, (_, index) => [ + `diff --git a/merge/m${index}.ts b/merge/m${index}.ts`, 'index 1111111..2222222 100644', `--- a/merge/m${index}.ts`, `+++ b/merge/m${index}.ts`, + '@@ -1 +1,2 @@', ' export {}', `+export const m${index} = ${index}`, '', +].join('\n')).join('') + +/** Message of the double's HTTP 406 refusal, which it sends in the error body format GitHub uses for a diff over its size limit. */ +const PR9_REFUSAL = 'Sorry, the diff exceeded the size limit.' /** Line `n` of `src/panel.ts` before (`old`) or after (`new`) pull request 7, which changes lines 2 and 15 of its 20. */ function panelLine(n: number, side: 'old' | 'new'): string { @@ -249,7 +263,7 @@ async function launchPanel(failureShot: string, scenario: 'normal' | 'follow-up' await writeFile(gh, GH) await chmod(gh, 0o755) await writeFile(join(root, 'pr-7.diff'), PR7_DIFF) - await writeFile(join(root, 'pr-9.refused'), PR9_REFUSAL) + await writeFile(join(root, 'pr-9.diff'), PR9_DIFF) const panelText = `${Array.from({ length: 20 }, (_, index) => panelLine(index + 1, 'new')).join('\n')}\n` await writeFile(join(root, 'contents.json'), JSON.stringify({ [`src/panel.ts@${HEAD7}`]: panelText })) const overlay = join(root, 'forge.patch.yml') @@ -1011,21 +1025,43 @@ it.skipIf(MODE === 'record')('shows the local changes of the Session by scope an expect(await reviewDialog.getByRole('radio', { name: '评论', exact: true }).isChecked()).toBe(true) await page.keyboard.press('Escape') - // GitHub refuses the diff of pull request 9 as too large, so its view offers the web page instead of a retry. + // Pull request 9 changes 301 files, more than the pull request diff endpoint accepts; the compare read shows every file. scaffold.ctx.forgeSession.track(session, DRAFT) await expect.poll(() => header.getAttribute('aria-pressed'), { timeout: 20_000 }).toBe('false') await header.click() const draft = page.locator(`[data-forge-change="${DRAFT}"]`) await draft.getByRole('button', { name: '展开 #9', exact: true }).click() await draft.getByRole('button', { name: '查看 #9 的变更：新增 30 行，删除 2 行', exact: true }).click({ timeout: 20_000 }) - const refused = page.locator('[data-forge-changes="change"]').filter({ hasText: 'Add the merge button' }) - await refused.getByRole('status').filter({ hasText: '变更太大，无法在这里显示' }).waitFor({ timeout: 20_000 }) - await refused.getByRole('button', { name: '在浏览器中查看', exact: true }).waitFor() - expect(await refused.getByRole('button', { name: '重试', exact: true }).count()).toBe(0) + const wide = page.locator('[data-forge-changes="change"]').filter({ hasText: 'Add the merge button' }) + await wide.getByRole('region', { name: '全部 301 个文件', exact: true }).waitFor({ timeout: 20_000 }) + expect(await wide.locator('[data-forge-diff-file]').count()).toBe(301) + + // A push moves the head to a commit whose diff GitHub refuses over its size limit, so a newly opened view offers the web page + // instead of a retry. + await page.locator('[data-dockkit-tab]').filter({ hasText: '#9 的变更' }).locator('[data-dockkit-tab-close]').click() + await expect.poll(() => wide.count()).toBe(0) + await writeFile(join(root, 'state.json'), JSON.stringify({ ready: false, mergedAt: null, head: 'd'.repeat(40) })) + await writeFile(join(root, 'pr-9.refused'), PR9_REFUSAL) + scaffold.ctx.forge.refresh({ repository: { forge: 'github', host: 'github.com', owner: 'octo', name: 'demo' }, number: 9 }) + await expect.poll(() => scaffold.ctx.forge.latest({ repository: { forge: 'github', host: 'github.com', owner: 'octo', name: 'demo' }, number: 9 }), + { timeout: 20_000 }).toMatchObject({ snapshot: { headSha: 'd'.repeat(40) } }) + if (await header.getAttribute('aria-pressed') !== 'true') await header.click() + const expand = draft.getByRole('button', { name: '展开 #9', exact: true }) + if (await expand.count() > 0) await expand.click() + await draft.getByRole('button', { name: '查看 #9 的变更：新增 30 行，删除 2 行', exact: true }).click({ timeout: 20_000 }) + await wide.getByRole('status').filter({ hasText: '变更太大，无法在这里显示' }).waitFor({ timeout: 20_000 }) + await wide.getByRole('button', { name: '在浏览器中查看', exact: true }).waitFor() + expect(await wide.getByRole('button', { name: '重试', exact: true }).count()).toBe(0) const calls = (await readFile(join(root, 'calls.jsonl'), 'utf8')).trim().split('\n').map(line => JSON.parse(line) as string[]) - expect(calls).toContainEqual(['pr', 'diff', '7', '--repo', 'github.com/octo/demo', '--color', 'never']) - expect(calls).toContainEqual(['pr', 'diff', '9', '--repo', 'github.com/octo/demo', '--color', 'never']) + const compared = (head: string) => [ + 'api', '--hostname', 'github.com', '--allow-escape-sequences', '-H', 'Accept: application/vnd.github.diff', + `repos/octo/demo/compare/${'e'.repeat(40)}...${head}`, + ] + expect(calls).toContainEqual(compared(HEAD7)) + expect(calls).toContainEqual(compared('b'.repeat(40))) + expect(calls).toContainEqual(compared('d'.repeat(40))) + expect(calls.some(call => call[0] === 'pr' && call[1] === 'diff')).toBe(false) expect(calls).toContainEqual([ 'api', '--hostname', 'github.com', '-H', 'Accept: application/vnd.github.raw+json', `repos/octo/demo/contents/src/panel.ts?ref=${HEAD7}`, ]) diff --git a/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.i18n.yaml b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.i18n.yaml index ffe417704a..3e10d3d26b 100644 --- a/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.i18n.yaml +++ b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.i18n.yaml @@ -9,8 +9,8 @@ en: c017d657e5d9eb08 zh: d0eb2aef480e7753 /forge-diffs-read-one-head-commit/change: - en: 669f43ccd2364c5a - zh: 97190faaa338da51 + en: 6b6465c90a0f893e + zh: fb265bd706a77196 /forge-diffs-read-one-head-commit/migration: - en: 04cf41940f9fa126 - zh: edf52bd1d8882bf9 + en: 147b16ca7c30624c + zh: ce246be22d7c378a diff --git a/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.md b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.md index e1d3c890ef..e31ee72f57 100644 --- a/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.md +++ b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.md @@ -9,7 +9,7 @@ English | [中文](guide.zh.md) ## Change -`ForgeProvider.diff(ref, signal)` becomes `diff(ref, headSha, signal)`, and `ctx.forge.diff(ref, signal)` becomes `ctx.forge.diff(ref, headSha, signal)`. A provider returns a patch only when it describes `headSha`; when the change request's head is another commit, it resolves to a `ForgeFailure` whose code is the new `stale`. `ctx.forge.diff` then polls the change request again at once when it is watched. The GitHub provider reads the head before and after `gh pr diff` and returns `stale` when either read names another commit. +`ForgeProvider.diff(ref, signal)` becomes `diff(ref, headSha, signal)`, and `ctx.forge.diff(ref, signal)` becomes `ctx.forge.diff(ref, headSha, signal)`. A provider returns a patch only when it describes `headSha`; when the change request's head is another commit, it resolves to a `ForgeFailure` whose code is the new `stale`. `ctx.forge.diff` then polls the change request again at once when it is watched. The GitHub provider reads the head and base commits in one GraphQL query, returns `stale` when the head is another commit, and otherwise reads the patch through the compare API between those two commits instead of `gh pr diff`. A GitHub pull request over 300 files, which `gh pr diff` refused as too large, now shows its complete diff; only output over `maxOutputBytes` or GitHub's own refusal remains `too-large`. `ForgeFailure['code']` gains `stale`. `ChangeRequestSnapshot` gains the required `threadsTruncated: boolean`, which marks `unresolvedThreads` as a lower bound when the provider counted only some review threads. @@ -19,7 +19,7 @@ This affects third-party `ForgeProvider` implementations registered with `ctx.fo ## Migration -1. In a `ForgeProvider`, add the `headSha: string` parameter to `diff` before `signal`. Return the patch only when the host computed it at `headSha`; otherwise return `{ code: 'stale', message }`. A host whose diff endpoint takes no commit can read the head before and after the diff, as [`GitHubProvider.diff`](../../../../packages/experimental/forge-github/src/provider.ts) does. +1. In a `ForgeProvider`, add the `headSha: string` parameter to `diff` before `signal`. Return the patch only when the host computed it at `headSha`; otherwise return `{ code: 'stale', message }`. [`GitHubProvider.diff`](../../../../packages/experimental/forge-github/src/provider.ts) reads the current head first and then a diff addressed by commit. 2. Pass the head commit to every `ctx.forge.diff` call, usually `ctx.forge.latest(ref)`'s `snapshot.headSha`. On `stale`, wait for a later status and read again at its head; do not cache the result under the old head. 3. Set `threadsTruncated` in every `ChangeRequestSnapshot` a provider builds: `false` when `unresolvedThreads` counts every review thread. 4. Handle `stale` wherever code switches exhaustively over `ForgeFailure['code']`, and `forgePanel/stale-head` wherever a caller of `forgePanel.changeRequestDiff` distinguishes Remote error codes. diff --git a/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.zh.md b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.zh.md index e5549364ec..cb4de638a2 100644 --- a/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.zh.md +++ b/docs/upgrade-guide/v0.2.2-rc.1/forge-pinned-diff/guide.zh.md @@ -9,7 +9,7 @@ description: "ForgeProvider.diff 与 ctx.forge.diff 需要传入要读取的 hea ## 变更 -`ForgeProvider.diff(ref, signal)` 改为 `diff(ref, headSha, signal)`，`ctx.forge.diff(ref, signal)` 改为 `ctx.forge.diff(ref, headSha, signal)`。提供方只在补丁描述 `headSha` 时返回补丁；变更请求的 head 已是其他提交时，resolve 为代码为新增 `stale` 的 `ForgeFailure`。此时若该变更请求正被 watch，`ctx.forge.diff` 会立即重新轮询它。GitHub 提供方在 `gh pr diff` 前后各读一次 head，任一次不是该提交时返回 `stale`。 +`ForgeProvider.diff(ref, signal)` 改为 `diff(ref, headSha, signal)`，`ctx.forge.diff(ref, signal)` 改为 `ctx.forge.diff(ref, headSha, signal)`。提供方只在补丁描述 `headSha` 时返回补丁；变更请求的 head 已是其他提交时，resolve 为代码为新增 `stale` 的 `ForgeFailure`。此时若该变更请求正被 watch，`ctx.forge.diff` 会立即重新轮询它。GitHub 提供方在一次 GraphQL 查询中读取 head 与基准提交，head 是其他提交时返回 `stale`，否则不再使用 `gh pr diff`，改经 compare API 读取这两个提交之间的补丁。超过 300 个文件、此前被 `gh pr diff` 判为过大的 GitHub pull request 现在显示完整 diff；只有输出超过 `maxOutputBytes` 或 GitHub 自身拒绝时才仍为 `too-large`。 `ForgeFailure['code']` 新增 `stale`。`ChangeRequestSnapshot` 新增必填字段 `threadsTruncated: boolean`；提供方只统计了部分评审讨论时，它表示 `unresolvedThreads` 只是下限。 @@ -19,7 +19,7 @@ description: "ForgeProvider.diff 与 ctx.forge.diff 需要传入要读取的 hea ## 迁移 -1. 在 `ForgeProvider` 的 `diff` 中，于 `signal` 之前加入 `headSha: string` 参数。只有托管平台在 `headSha` 处计算出补丁时才返回补丁，否则返回 `{ code: 'stale', message }`。若托管平台的 diff 接口不接受提交参数，可以像 [`GitHubProvider.diff`](../../../../packages/experimental/forge-github/src/provider.ts) 那样在读取 diff 前后各读一次 head。 +1. 在 `ForgeProvider` 的 `diff` 中，于 `signal` 之前加入 `headSha: string` 参数。只有托管平台在 `headSha` 处计算出补丁时才返回补丁，否则返回 `{ code: 'stale', message }`。[`GitHubProvider.diff`](../../../../packages/experimental/forge-github/src/provider.ts) 先读取当前 head，再读取按提交寻址的 diff。 2. 每次调用 `ctx.forge.diff` 都传入 head 提交，通常是 `ctx.forge.latest(ref)` 的 `snapshot.headSha`。收到 `stale` 时，等待更新的状态并按其 head 重新读取；不要把结果缓存在旧 head 下。 3. 提供方构造的每个 `ChangeRequestSnapshot` 都要设置 `threadsTruncated`：`unresolvedThreads` 统计了全部评审讨论时为 `false`。 4. 凡对 `ForgeFailure['code']` 做穷尽分支的地方都处理 `stale`；凡区分 `forgePanel.changeRequestDiff` Remote 错误代码的调用方都处理 `forgePanel/stale-head`。 diff --git a/packages/experimental/client-ui-forge/README.i18n.yaml b/packages/experimental/client-ui-forge/README.i18n.yaml index 9c2c5d288b..a8f761a92e 100644 --- a/packages/experimental/client-ui-forge/README.i18n.yaml +++ b/packages/experimental/client-ui-forge/README.i18n.yaml @@ -15,8 +15,8 @@ en: d152484eb41ac6b4 zh: 2f8a621b991260a1 /deepseek-ai-dsh-experimental-client-ui-forge/use-this-package: - en: 458a6d3447c33aa2 - zh: 5af3eb9439840b73 + en: 837dfbd63ed8d0f2 + zh: 5bb94cf667e9e902 /deepseek-ai-dsh-experimental-client-ui-forge/understand-the-implementation: en: 04b6519156ba47de zh: 84c4683f3762c22c diff --git a/packages/experimental/client-ui-forge/README.md b/packages/experimental/client-ui-forge/README.md index 84f9b90c78..e9062907d1 100644 --- a/packages/experimental/client-ui-forge/README.md +++ b/packages/experimental/client-ui-forge/README.md @@ -38,7 +38,7 @@ Panel text follows the application language: Chinese, English, Japanese, Spanish - **Session list** — after a Session row's time, a branch glyph shows while the Session works in the worktree the panel created for it, and a glyph shows the state of its most urgent change request: one needing attention first, then open, draft, not yet polled, merged, and closed. The glyph of an open change request keeps the row's subdued color, with its head branch end filled in the color of running, passed, or failed checks; merged and closed ones keep their state colors. The row's hover card names the worktree branch and what it started from, then up to three change requests with their number, title, state, and checks, and how many more there are. A restart shows the states known before it until the next poll. - **Tab** — the first row names the checked-out branch, or the abbreviated commit on a detached HEAD, after a folder glyph for the project directory or a branch glyph for a worktree, and the kind of checkout, which is the worktree the Session created with its base, another worktree, or the project directory; its tooltip adds the path with the Host home shortened to `~`, its button copies the full path, and for a removed worktree it offers Restore. Below it, each change request card shows a glyph for its state, its title, which opens it in the browser, and its number; the glyph tells open, draft, merged, and closed apart, and marks an open one that is not a draft with failed checks, requested changes, or conflicts as needing attention, as the header button's red dot does. The card's header also offers Open in browser, Copy change request link, and Remove from this Session, whose toast offers Undo. Cancelled checks have a separate count; a collapsed card names cancellations when no listed check failed. A collapsed card adds the checks of an open change request and the most urgent of requested changes, conflicts, or draft, or when a merged or closed one ended. An expanded card shows the head and base branches, then one row per fact: for an open change request the checks, whose row opens the list of checks with failed ones first and their durations and links, the review decision with unresolved threads, whose count ends in `+` when the provider counted only some threads, and the merge state, or when a merged or closed one ended; and the changed lines, formatted in the selected UI language, whose row opens the changes view. An open draft also offers Ready for review; a change request the host would merge now offers Merge, whose menu lists the methods the repository allows with the user's usual one chosen and pins the displayed head when opened. Confirm merge uses that head; a polled head change disables confirmation until the menu is reopened after review. A toast reports each outcome, and new commits since that head stop the merge. When exactly one bound change request is open, its card starts expanded. Refresh inspects the checkout, checks sign-in, and polls every change request again; its icon turns until the inspection and the sign-in check end. A banner shows the `gh auth login` command when sign-in is missing. - **Local changes card** — whenever the `branch` scope changes files, independently of bound change requests, a card above the change requests names the ref it compares with, such as `HEAD → origin/main`, which is always the default base below, and shows the added and deleted lines and the number of files, which open the changes view; its file list, which names files as the changes view does, opens the view at one file. Create change request, offered when the code host repository is recognized and no open change request (including drafts) has the checked-out branch as its head, sends the Session a user message in the UI language that asks the Agent to create a change request for these changes, and a toast reports whether it was sent. -- **Changes view** — The Changes row of an expanded change request card and the local changes card open a right-sidebar tab that lists the changed files. The local tab is named Local changes and shows Branch by default: the working tree and untracked files against the fork point. Its header names the checked-out branch, or the abbreviated commit on a detached HEAD, and the base branch, which opens a searchable menu: first Default, the base commit of the worktree the Session created, else the merge base of HEAD with the default branch of the code host remote, then remote-tracking and then local branches, each most recently committed first, without the checked-out branch and, outside a created worktree, without the remote-tracking branch Default already names. Typing filters the listed rows at once and lists the branches whose names contain the text, ignoring case, after a short pause; Enter chooses the first row. Choosing another branch compares with the merge base of HEAD and that branch. The tab keeps the choice while other scopes show; closing the tab or reloading the application returns it to Default. When the chosen branch is gone, its name is struck through, and when it shares no history with HEAD, the view says so as well; both offer Use the default base branch instead of Retry. A narrow view hides the note that uncommitted changes count, then the checked-out branch. Uncommitted compares them with HEAD, Unstaged with the index, and Staged compares the index with HEAD. A change request view shows the host's diff at its head and says how many commits the checked-out branch has beyond that head when it is the change request's head branch. The toolbar jumps to a file, totals the changed lines, refreshes, and switches between two columns and one and between wrapped and cut long lines. Visited changes tabs stay mounted across tab and Session switches, hiding the sidebar, and docking changes, preserving file folds, revealed large files, expanded context, layout, and scroll position while the diff is unchanged. Hidden changes views release their panel subscription and defer diff reads; returning reads again only when the latest head or local summary differs. A refresh that replaces the diff discards expanded context, and changed content may move the reading position. Temporary menus close when the view is hidden. Closing the tab or reloading the application discards this progress. Files are named before their directories, which use smaller subdued type and shorten first; hovering one shows its complete path, and both paths for a renamed file. Each file shows badges for a new, deleted, or renamed file, the last naming the old file name, or the old path when only the directory changed, and, in the Branch scope, an untracked one, and a menu that copies its path and opens the whole file in the sidebar for local changes or its page at the head in the browser for a change request. Unchanged lines between hunks collapse into rows that expand from the file's new side: the working tree, the index for Staged, or the head commit through the code host; a file that changed after the diff was read stays collapsed until the view reads again. A binary file and an untracked file over `maxDiffBytes` show a note instead of their lines, a file whose hunks draw more than 1,000 lines starts as one collapsed strip that names its changed lines and draws them when opened, and a file below files that change more than 5,000 lines starts collapsed. The view reads again on Refresh and the page refresh shortcut, on each navigation, when the change request's head moves, and when the Branch scope's summary changes; a change request's view first waits for the status that names its head, unless no provider serves it or that poll failed. When the head moved past the latest status, the view keeps loading and reads again once a later status arrives, at its head. While a read replaces changes already shown, the Refresh icon turns. A change request whose diff the code host or its provider refuses over a size limit, such as a GitHub diff over 300 files, says its changes are too large to show here and offers View in browser; another failed read offers Retry and, for a change request, View in browser. A change request's diff is downloaded once per head commit and base branch while it stays cached, so reopening, refreshing, and navigating at the same head read nothing from the host. +- **Changes view** — The Changes row of an expanded change request card and the local changes card open a right-sidebar tab that lists the changed files. The local tab is named Local changes and shows Branch by default: the working tree and untracked files against the fork point. Its header names the checked-out branch, or the abbreviated commit on a detached HEAD, and the base branch, which opens a searchable menu: first Default, the base commit of the worktree the Session created, else the merge base of HEAD with the default branch of the code host remote, then remote-tracking and then local branches, each most recently committed first, without the checked-out branch and, outside a created worktree, without the remote-tracking branch Default already names. Typing filters the listed rows at once and lists the branches whose names contain the text, ignoring case, after a short pause; Enter chooses the first row. Choosing another branch compares with the merge base of HEAD and that branch. The tab keeps the choice while other scopes show; closing the tab or reloading the application returns it to Default. When the chosen branch is gone, its name is struck through, and when it shares no history with HEAD, the view says so as well; both offer Use the default base branch instead of Retry. A narrow view hides the note that uncommitted changes count, then the checked-out branch. Uncommitted compares them with HEAD, Unstaged with the index, and Staged compares the index with HEAD. A change request view shows the host's diff at its head and says how many commits the checked-out branch has beyond that head when it is the change request's head branch. The toolbar jumps to a file, totals the changed lines, refreshes, and switches between two columns and one and between wrapped and cut long lines. Visited changes tabs stay mounted across tab and Session switches, hiding the sidebar, and docking changes, preserving file folds, revealed large files, expanded context, layout, and scroll position while the diff is unchanged. Hidden changes views release their panel subscription and defer diff reads; returning reads again only when the latest head or local summary differs. A refresh that replaces the diff discards expanded context, and changed content may move the reading position. Temporary menus close when the view is hidden. Closing the tab or reloading the application discards this progress. Files are named before their directories, which use smaller subdued type and shorten first; hovering one shows its complete path, and both paths for a renamed file. Each file shows badges for a new, deleted, or renamed file, the last naming the old file name, or the old path when only the directory changed, and, in the Branch scope, an untracked one, and a menu that copies its path and opens the whole file in the sidebar for local changes or its page at the head in the browser for a change request. Unchanged lines between hunks collapse into rows that expand from the file's new side: the working tree, the index for Staged, or the head commit through the code host; a file that changed after the diff was read stays collapsed until the view reads again. A binary file and an untracked file over `maxDiffBytes` show a note instead of their lines, a file whose hunks draw more than 1,000 lines starts as one collapsed strip that names its changed lines and draws them when opened, and a file below files that change more than 5,000 lines starts collapsed. The view reads again on Refresh and the page refresh shortcut, on each navigation, when the change request's head moves, and when the Branch scope's summary changes; a change request's view first waits for the status that names its head, unless no provider serves it or that poll failed. When the head moved past the latest status, the view keeps loading and reads again once a later status arrives, at its head. While a read replaces changes already shown, the Refresh icon turns. A change request whose diff the code host or its provider refuses over a size limit, such as a GitHub diff whose output exceeds the provider's `maxOutputBytes`, says its changes are too large to show here and offers View in browser; another failed read offers Retry and, for a change request, View in browser. A change request's diff is downloaded once per head commit and base branch while it stays cached, so reopening, refreshing, and navigating at the same head read nothing from the host. The checks row initially shows the polling preview. While it is incomplete, the card uses the host’s aggregate outcome without a failure count and labels the loaded check count as partial. Opening the list loads the remaining pages for that PR through `changeRequestChecks`; its deadline is independent of polling. A failed read keeps the preview and offers Retry. Closing the list, changing the head, or unmounting cancels the read; late results are ignored. Reopening reads current details. The Remote requires a visible Session binding and rejects a response for a different head commit. diff --git a/packages/experimental/client-ui-forge/README.zh.md b/packages/experimental/client-ui-forge/README.zh.md index 4c6cc8f5ed..6dc88dcfdf 100644 --- a/packages/experimental/client-ui-forge/README.zh.md +++ b/packages/experimental/client-ui-forge/README.zh.md @@ -38,7 +38,7 @@ kind: "package-reference" - **会话列表**——Session 在面板为它创建的 worktree 中工作时，会话行的时间之后显示分支图标；另有一个图标显示它最需要关注的变更请求的状态，依次为需要处理、打开、草稿、尚未轮询、已合并、已关闭。打开中的变更请求图标保持会话行的浅色，head 分支端点填充为检查进行中、已通过或失败对应的颜色；已合并与已关闭的保留各自的状态颜色。会话行的悬浮卡片写出 worktree 分支及其起点，再列出最多三个变更请求的编号、标题、状态与检查，并说明还有多少个。重启后，下次轮询之前显示重启前已知的状态。 - **标签页**——顶部一行显示当前检出的分支（分离 HEAD 时为缩写提交），项目目录前显示目录图标、worktree 前显示分支图标，并说明检出的类型（Session 创建的 worktree 及其基准、其他 worktree 或项目目录）；它的提示给出路径，宿主主目录缩写为 `~`，旁边的按钮复制完整路径；worktree 已被删除时提供“恢复”。其下每张变更请求卡片显示状态图标、标题（点击在浏览器中打开）与编号；图标区分打开、草稿、已合并与已关闭，打开中的变更请求（草稿除外）检查失败、被要求修改或有冲突时显示为“需要处理”，与标题栏按钮的红点一致。卡片右上角还提供“在浏览器中打开”“复制变更请求链接”与“从本会话移除”，移除后的 toast 提供“撤销”。已取消的检查单独计数；列出的检查没有失败项时，折叠卡片显示取消数量。折叠的卡片还显示打开的变更请求的检查，以及被要求修改、冲突、草稿中最需要注意的一项；已合并或已关闭的则显示结束时间。展开的卡片显示源分支与目标分支，下面每项一行：打开的变更请求依次是检查（点击该行弹出检查列表，失败的排在前面，带耗时与链接）、评审结论与未解决讨论数（提供方只统计了部分讨论时数字后带 `+`）、合并状态，已合并或已关闭的显示结束时间；最后是按所选界面语言格式化的变更行数，点击该行打开变更视图。打开中的草稿还提供“标记为可评审”；主机现在就会合并的变更请求提供“合并”，其菜单列出仓库允许的合并方式并选中用户常用的一种，打开时固定卡片显示的 head。“确认合并”使用这个 head；轮询发现 head 变化后禁用确认，需重新检查并打开菜单。每次操作的结果都以 toast 报告；该 head 之后有新提交时不会合并。恰好有一个已绑定的变更请求处于打开状态时，它的卡片默认展开。刷新会重新检查检出与登录状态，并立即再次轮询每个变更请求；检查与登录状态确认结束前，刷新图标持续旋转。缺少登录时，横幅显示 `gh auth login` 命令。 - **本地变更卡片**——`branch` 范围有文件变更时始终显示，不受已关联变更请求影响。变更请求列表上方的卡片显示比较的基准（例如 `HEAD → origin/main`，始终是下文的默认基准）以及新增、删除行数与文件数，点击打开变更视图；展开的文件列表像变更视图一样标出各个文件，点击某个文件时，变更视图定位到该文件。识别出代码托管仓库、且没有以当前检出分支为源分支的打开中变更请求（含草稿）时，卡片提供“创建变更请求”：它以界面语言向 Session 发送一条用户消息，请 Agent 为这些变更创建变更请求，并以 toast 报告是否已发送。 -- **变更视图**——展开的变更请求卡片上的“变更”一行与本地变更卡片会打开一个右侧边栏标签页，列出所有变更的文件。本地标签页名为“本地变更”，默认显示“分支”：工作树与未跟踪文件相对于分叉点的变更。它的标题栏写出当前检出的分支（分离 HEAD 时为缩写提交）与基准分支；点击基准分支打开可搜索的菜单：最前面是“默认”，即 Session 创建的 worktree 的基准提交，否则是 HEAD 与代码托管远端默认分支的合并基准；其后依次是远程跟踪分支与本地分支，各按最近提交排序，不含当前检出的分支，不在所创建的 worktree 中时也不含“默认”已经指向的远程跟踪分支。输入文字会立即筛选已列出的行，短暂停顿后列出名称包含该文字（不区分大小写）的分支；按 Enter 选择第一行。选择其他分支后，与 HEAD 和该分支的合并基准比较。标签页切换到其他范围时仍保留这个选择；关闭标签页或重新加载应用后回到“默认”。所选分支已不存在时，它的名称显示删除线；它与 HEAD 没有共同历史时，视图同样给出说明；两种情况都提供“使用默认基准分支”而不是“重试”。视图较窄时，先隐藏“包含未提交的修改”，再隐藏当前分支。“未提交”与 HEAD 比较，“未暂存”与暂存区比较，“已暂存”比较暂存区与 HEAD。变更请求的视图显示托管平台在其 head 处的 diff；当前检出分支就是该变更请求的源分支时，还会说明本地分支比该 head 多出几个提交。工具栏可以跳到某个文件、汇总变更行数、刷新，并在左右对比与单栏对比、自动换行与截断长行之间切换。访问过的变更标签页会在切换标签页与 Session、隐藏侧边栏及调整停靠位置时保持挂载，在 diff 未改变时保留文件折叠、大文件展开、上下文展开、布局与滚动位置。隐藏的变更视图释放面板订阅并推迟 diff 读取；返回时仅在最新 head 或本地摘要不同时重新读取。刷新替换 diff 后会丢弃展开的上下文，内容变化也可能移动阅读位置。视图隐藏时关闭临时菜单。关闭标签页或重新加载应用后不保留这些进度。文件名在前，目录在后，用更小的浅色字显示，空间不够时目录先缩短；悬停时显示完整路径，重命名的文件显示新旧两条路径。每个文件带有新文件、已删除、重命名的标记（重命名标记写出原文件名，只换了目录时写出原路径），“分支”范围下的未跟踪文件还标“未提交”；文件菜单可以复制路径，本地变更可在侧边栏打开整个文件，变更请求可在浏览器中查看该文件在 head 处的页面。hunk 之间未变更的行折叠成一行，点击后从文件的新版本展开：工作树、“已暂存”时为暂存区，或经托管平台读取的 head 提交；读取 diff 之后文件又有修改时不会展开，直到视图重新读取。二进制文件与超过 `maxDiffBytes` 的未跟踪文件只显示一条说明；hunk 超过 1,000 行的文件先显示为一条写出变更行数的折叠条，点击后才绘制；前面的文件累计变更超过 5,000 行后，其后的文件默认收起。刷新与页面刷新快捷键、每次导航、变更请求的 head 变化，以及“分支”范围的摘要变化时，视图都会重新读取；变更请求的视图会先等到给出 head 的状态再读取，除非没有提供方服务它或那次轮询失败。head 已越过最新状态时，视图保持加载，等更新的状态到达后按其 head 重新读取。重新读取已显示的变更时，刷新图标会旋转。托管平台或其提供方因超过大小上限拒绝给出 diff 的变更请求（例如 GitHub 上超过 300 个文件的 diff）显示“变更太大，无法在这里显示”并提供“在浏览器中查看”；其他读取失败提供“重试”，变更请求还提供“在浏览器中查看”。变更请求的 diff 在缓存期间对每个 head 提交与目标分支只下载一次，因此在同一 head 下重新打开、刷新与导航都不会再从托管平台读取。 +- **变更视图**——展开的变更请求卡片上的“变更”一行与本地变更卡片会打开一个右侧边栏标签页，列出所有变更的文件。本地标签页名为“本地变更”，默认显示“分支”：工作树与未跟踪文件相对于分叉点的变更。它的标题栏写出当前检出的分支（分离 HEAD 时为缩写提交）与基准分支；点击基准分支打开可搜索的菜单：最前面是“默认”，即 Session 创建的 worktree 的基准提交，否则是 HEAD 与代码托管远端默认分支的合并基准；其后依次是远程跟踪分支与本地分支，各按最近提交排序，不含当前检出的分支，不在所创建的 worktree 中时也不含“默认”已经指向的远程跟踪分支。输入文字会立即筛选已列出的行，短暂停顿后列出名称包含该文字（不区分大小写）的分支；按 Enter 选择第一行。选择其他分支后，与 HEAD 和该分支的合并基准比较。标签页切换到其他范围时仍保留这个选择；关闭标签页或重新加载应用后回到“默认”。所选分支已不存在时，它的名称显示删除线；它与 HEAD 没有共同历史时，视图同样给出说明；两种情况都提供“使用默认基准分支”而不是“重试”。视图较窄时，先隐藏“包含未提交的修改”，再隐藏当前分支。“未提交”与 HEAD 比较，“未暂存”与暂存区比较，“已暂存”比较暂存区与 HEAD。变更请求的视图显示托管平台在其 head 处的 diff；当前检出分支就是该变更请求的源分支时，还会说明本地分支比该 head 多出几个提交。工具栏可以跳到某个文件、汇总变更行数、刷新，并在左右对比与单栏对比、自动换行与截断长行之间切换。访问过的变更标签页会在切换标签页与 Session、隐藏侧边栏及调整停靠位置时保持挂载，在 diff 未改变时保留文件折叠、大文件展开、上下文展开、布局与滚动位置。隐藏的变更视图释放面板订阅并推迟 diff 读取；返回时仅在最新 head 或本地摘要不同时重新读取。刷新替换 diff 后会丢弃展开的上下文，内容变化也可能移动阅读位置。视图隐藏时关闭临时菜单。关闭标签页或重新加载应用后不保留这些进度。文件名在前，目录在后，用更小的浅色字显示，空间不够时目录先缩短；悬停时显示完整路径，重命名的文件显示新旧两条路径。每个文件带有新文件、已删除、重命名的标记（重命名标记写出原文件名，只换了目录时写出原路径），“分支”范围下的未跟踪文件还标“未提交”；文件菜单可以复制路径，本地变更可在侧边栏打开整个文件，变更请求可在浏览器中查看该文件在 head 处的页面。hunk 之间未变更的行折叠成一行，点击后从文件的新版本展开：工作树、“已暂存”时为暂存区，或经托管平台读取的 head 提交；读取 diff 之后文件又有修改时不会展开，直到视图重新读取。二进制文件与超过 `maxDiffBytes` 的未跟踪文件只显示一条说明；hunk 超过 1,000 行的文件先显示为一条写出变更行数的折叠条，点击后才绘制；前面的文件累计变更超过 5,000 行后，其后的文件默认收起。刷新与页面刷新快捷键、每次导航、变更请求的 head 变化，以及“分支”范围的摘要变化时，视图都会重新读取；变更请求的视图会先等到给出 head 的状态再读取，除非没有提供方服务它或那次轮询失败。head 已越过最新状态时，视图保持加载，等更新的状态到达后按其 head 重新读取。重新读取已显示的变更时，刷新图标会旋转。托管平台或其提供方因超过大小上限拒绝给出 diff 的变更请求（例如 GitHub 上输出超过提供方 `maxOutputBytes` 的 diff）显示“变更太大，无法在这里显示”并提供“在浏览器中查看”；其他读取失败提供“重试”，变更请求还提供“在浏览器中查看”。变更请求的 diff 在缓存期间对每个 head 提交与目标分支只下载一次，因此在同一 head 下重新打开、刷新与导航都不会再从托管平台读取。 检查行先显示轮询预览。不完整时，卡片使用托管平台的总体状态而不显示失败总数，并注明已加载的检查数量尚不完整。打开列表后，经 `changeRequestChecks` 单独读取该 PR 的后续页，时限独立于轮询。读取失败时保留预览并提供重试。关闭列表、head 变化或组件卸载会取消读取，忽略迟到结果；重新打开会读取当前详情。Remote 要求 URL 是该 Session 可见的绑定，并拒绝 head 提交不一致的响应。 diff --git a/packages/experimental/forge-github/README.i18n.yaml b/packages/experimental/forge-github/README.i18n.yaml index 5b14a27a65..8859923642 100644 --- a/packages/experimental/forge-github/README.i18n.yaml +++ b/packages/experimental/forge-github/README.i18n.yaml @@ -15,11 +15,11 @@ en: d152484eb41ac6b4 zh: 2f8a621b991260a1 /deepseek-ai-dsh-experimental-forge-github/use-this-package: - en: 3eece595e10bc32e - zh: fe3c5cf59f9f17d4 + en: 4f8cd0d3b2df749c + zh: 8d398c088900a535 /deepseek-ai-dsh-experimental-forge-github/understand-the-implementation: - en: 90f3c10096be642a - zh: 4e1b663c52019ac2 + en: f98e77171cba809f + zh: 1b330cb342d69efe /deepseek-ai-dsh-experimental-forge-github/further-exploration: en: d377d5fd9d9f190e zh: 920bcbfdb8c993f2 @@ -30,8 +30,8 @@ en: 26b946af4b742e17 zh: 00b34f0721ac72f6 /deepseek-ai-dsh-experimental-forge-github/known-limitations-and-deferred-work: - en: 627b11577406b029 - zh: 6596e545c1c5a6fd + en: 3dba4fd8b8375b39 + zh: 79190c7cb7b68656 /deepseek-ai-dsh-experimental-forge-github/known-limitations-and-deferred-work/dev-note: en: e67f633dc6fe7773 zh: 3bb0c20c93f16f98 diff --git a/packages/experimental/forge-github/README.md b/packages/experimental/forge-github/README.md index 643563baa4..9b290e605a 100644 --- a/packages/experimental/forge-github/README.md +++ b/packages/experimental/forge-github/README.md @@ -46,7 +46,7 @@ Unfinished check runs and completed runs with stale, missing, or unrecognized co A pull request is `mergeReady` when it is open, not a draft, GitHub reports its merge state as `CLEAN`, `HAS_HOOKS`, or `UNSTABLE`, and the signed-in user has write, maintain, or admin permission. `mergeMethods` lists the squash, merge-commit, and rebase methods the repository allows, the user's default merge method first. Marking ready runs `gh pr ready --repo //`. Merging calls `PUT repos///pulls//merge` through `gh api` with the method and the expected head SHA, so GitHub refuses once new commits arrived, which is reported as `stale`. Unlike `gh pr merge`, it never enables auto-merge, never joins a merge queue, and never deletes the head branch. A refused action reports GitHub's message, and a diagnostic that mentions authentication becomes `unauthenticated`. -`diff` reads the head commit through GraphQL, runs `gh pr diff --repo // --color never`, and reads the head commit again; it returns the patch only when both reads name the requested commit, and `stale` otherwise. The requested commit must be a full 40- or 64-digit hexadecimal commit. `gh pr diff` takes no commit argument and answers at the head of the moment it runs, so the two reads bracket it. The REST compare endpoint, which names commits explicitly, is not used: compared with the base branch it is empty for a merged pull request, compared with `baseRefOid` it starts from the base commit GitHub recorded at the pull request's last update instead of the current base tip, and it returns diffs over 300 files that the pull request diff refuses. `fileText` reads `repos///contents/?ref=` through `gh api` with the raw media type and accepts only a full 40- or 64-digit hexadecimal commit. `fileUrl` returns `https://///blob//`. Both paths percent-encode each segment. A refused read becomes `not-found` for HTTP 404, `unauthenticated` for a diagnostic that mentions authentication, `too-large` for GitHub's `too_large` error, which a diff over its own size limit draws, and `unavailable` otherwise, with GitHub's message. A read whose output exceeds `maxOutputBytes` becomes `too-large`, and one that outlives `timeoutMs` becomes `unavailable`. +`diff` reads `headRefOid` and `baseRefOid` in one GraphQL query and returns `stale` when the head is not the requested commit, which must be a full 40- or 64-digit hexadecimal commit. Otherwise it reads `repos///compare/...` through `gh api --allow-escape-sequences` with the diff media type, the patch from the merge base of both commits, so neither end follows a moving branch. For every open and merged pull request measured, this patch is byte-identical to `gh pr diff`, which takes no commit argument. Unlike the pull request diff, the compare endpoint returns diffs over 300 files, so only output over `maxOutputBytes` or GitHub's own refusal makes such a diff `too-large`. `--allow-escape-sequences` keeps terminal escape sequences that file contents hold, which `gh` 2.97 and newer otherwise refuse to print. `fileText` reads `repos///contents/?ref=` through `gh api` with the raw media type and accepts only a full 40- or 64-digit hexadecimal commit. `fileUrl` returns `https://///blob//`. Both paths percent-encode each segment. A refused read becomes `not-found` for HTTP 404, `unauthenticated` for a diagnostic that mentions authentication, `too-large` for GitHub's HTTP 406 refusal or `too_large` error, which a diff or file over its size limit draws, and `unavailable` otherwise, with GitHub's message. A read whose output exceeds `maxOutputBytes` becomes `too-large`, and one that outlives `timeoutMs` becomes `unavailable`. Review submission checks the current pull request head, then posts a single inline comment or one completed review through the REST API with an explicit `commit_id`. Multiline comments carry their side and inclusive start and end lines. Review decisions are Comment, Approve, and Request changes. JSON request bodies travel on stdin. The provider never retries a mutation: a timeout, unexpected response, or server error after dispatch returns an unknown outcome that the caller must reconcile on GitHub. A commit arriving between the check and the write can make the explicitly addressed review outdated; GitHub does not offer an atomic head precondition on these endpoints. @@ -65,8 +65,9 @@ A fetch groups references by host and sends one initial `gh api graphql` query p | [`src/index.ts`](src/index.ts) | Configuration validation and provider registration | | [`src/provider.ts`](src/provider.ts) | URL mapping, host grouping, and failure classification | | [`src/graphql.ts`](src/graphql.ts) | Query construction and response validation and mapping | +| [`src/compare.ts`](src/compare.ts) | Patch between two commits through the compare API, shared by every diff read | | [`src/gh.ts`](src/gh.ts) | Host `gh` execution with environment scrubbing and limits | -| [`src/urls.ts`](src/urls.ts) | Remote URL and pull request URL parsing | +| [`src/urls.ts`](src/urls.ts) | Remote URL and pull request URL parsing, and validation of inlined names and commits | @@ -97,7 +98,7 @@ None. Provider output reaches only host-side watchers. - **GitHub CLI on the host** — `gh` must be installed and signed in for each configured host; tokens in environment variables are not forwarded. Hosts missing from `hosts` are never read, including references retained across configuration changes. A status fetch returns `unavailable` for those references; a head lookup rejects before starting `gh`. - **Review thread bound** — a status read counts at most `maxReviewThreads` review threads per pull request; beyond it `unresolvedThreads` is a lower bound that `threadsTruncated` marks. - **No fork discovery** — head lookups exclude pull requests whose head is in another repository. -- **Rate limits** — every poll costs one GraphQL query per host, plus one query per further page of 100 review threads, and a complete check read one more query per further page of 100 check records; every diff read adds two head reads. All count against the signed-in account’s rate limit. +- **Rate limits** — every poll costs one GraphQL query per host, plus one query per further page of 100 review threads, and a complete check read one more query per further page of 100 check records; every diff read costs one GraphQL query and one REST request. All count against the signed-in account’s rate limit. - **No merge queue** — a base branch that requires a merge queue refuses a direct merge, and the provider never queues one. diff --git a/packages/experimental/forge-github/README.zh.md b/packages/experimental/forge-github/README.zh.md index 404567565d..291ef3d085 100644 --- a/packages/experimental/forge-github/README.zh.md +++ b/packages/experimental/forge-github/README.zh.md @@ -46,7 +46,7 @@ kind: "package-reference" pull request 处于打开状态、不是草稿、GitHub 报告其合并状态为 `CLEAN`、`HAS_HOOKS` 或 `UNSTABLE`，且已登录用户具有 write、maintain 或 admin 权限时，`mergeReady` 为真。`mergeMethods` 列出仓库允许的压缩合并、合并提交与变基合并方式，用户默认的合并方式排在最前。标记为可评审运行 `gh pr ready --repo //`。合并经 `gh api` 调用 `PUT repos///pulls//merge`，带上合并方式与预期的 head SHA，因此有新提交到达后 GitHub 会拒绝，报告为 `stale`。与 `gh pr merge` 不同，它从不启用自动合并、从不加入合并队列，也从不删除 head 分支。被拒绝的操作报告 GitHub 的原因，提到身份验证的诊断转为 `unauthenticated`。 -`diff` 先经 GraphQL 读取 head 提交，再运行 `gh pr diff --repo // --color never`，然后再读一次 head 提交；两次读取都是请求的提交时才返回补丁，否则返回 `stale`。请求的提交必须是完整的 40 位或 64 位十六进制提交。`gh pr diff` 不接受提交参数，返回运行那一刻 head 的结果，因此用前后两次读取把它夹住。不使用显式指定提交的 REST compare 接口：与基准分支比较时，已合并的 pull request 得到空结果；与 `baseRefOid` 比较时，起点是 GitHub 在 pull request 最后一次更新时记录的基准提交，而不是当前的基准分支末端；它还会返回 pull request diff 拒绝的超过 300 个文件的 diff。`fileText` 经 `gh api` 以 raw 媒体类型读取 `repos///contents/?ref=`，只接受完整的 40 位或 64 位十六进制提交。`fileUrl` 返回 `https://///blob//`。两者的路径都逐段做百分号编码。被拒绝的读取在 HTTP 404 时为 `not-found`，诊断提到身份验证时为 `unauthenticated`，GitHub 报告 `too_large` 错误时（diff 超过其自身大小上限即会如此）为 `too-large`，其余为 `unavailable`，并带有 GitHub 的原因。输出超过 `maxOutputBytes` 的读取为 `too-large`，超过 `timeoutMs` 的为 `unavailable`。 +`diff` 在一次 GraphQL 查询中读取 `headRefOid` 与 `baseRefOid`，head 不是请求的提交时返回 `stale`；请求的提交必须是完整的 40 位或 64 位十六进制提交。否则经 `gh api --allow-escape-sequences` 以 diff 媒体类型读取 `repos///compare/...`，即两个提交的合并基准到 head 的补丁，两端都不跟随移动的分支。实测的所有 open 与已合并 pull request 上，该补丁与不接受提交参数的 `gh pr diff` 逐字节一致。与 pull request diff 不同，compare 接口会返回超过 300 个文件的 diff，因此这类 diff 只有在输出超过 `maxOutputBytes` 或 GitHub 自身拒绝时才为 `too-large`。`--allow-escape-sequences` 保留文件内容中的终端转义序列；否则 `gh` 2.97 及以上版本拒绝输出。`fileText` 经 `gh api` 以 raw 媒体类型读取 `repos///contents/?ref=`，只接受完整的 40 位或 64 位十六进制提交。`fileUrl` 返回 `https://///blob//`。两者的路径都逐段做百分号编码。被拒绝的读取在 HTTP 404 时为 `not-found`，诊断提到身份验证时为 `unauthenticated`，GitHub 以 HTTP 406 拒绝或报告 `too_large` 错误时（diff 或文件超过其大小上限即会如此）为 `too-large`，其余为 `unavailable`，并带有 GitHub 的原因。输出超过 `maxOutputBytes` 的读取为 `too-large`，超过 `timeoutMs` 的为 `unavailable`。 提交评审时先检查 PR 当前的 head，再通过 REST API 发布单条行内评论或一次完整评审，并显式指定 `commit_id`。多行评论携带所属侧及包含端点的起止行号。评审结论为“评论”“批准”或“请求更改”。JSON 请求正文通过标准输入传递。provider 不会重试写操作；发送后的超时、异常响应或服务端错误会返回结果不明，由调用方到 GitHub 核对。检查与写入之间到来的新提交可能让已明确定位到旧提交的评审过时；GitHub 的这些接口不提供原子的 head 前置条件。 @@ -65,8 +65,9 @@ pull request 处于打开状态、不是草稿、GitHub 报告其合并状态为 | [`src/index.ts`](src/index.ts) | 配置校验与提供方注册 | | [`src/provider.ts`](src/provider.ts) | URL 映射、按主机分组与失败分类 | | [`src/graphql.ts`](src/graphql.ts) | 构造查询，并校验和映射响应 | +| [`src/compare.ts`](src/compare.ts) | 经 compare API 读取两个提交之间的补丁，供所有 diff 读取共用 | | [`src/gh.ts`](src/gh.ts) | 在宿主上执行 `gh`，清理环境变量并施加限制 | -| [`src/urls.ts`](src/urls.ts) | 解析远端 URL 与 pull request URL | +| [`src/urls.ts`](src/urls.ts) | 解析远端 URL 与 pull request URL，并校验写入请求的名称与提交 | @@ -97,7 +98,7 @@ pull request 处于打开状态、不是草稿、GitHub 报告其合并状态为 - **宿主需要 GitHub CLI**——必须安装 `gh` 并为每个已配置主机登录；环境变量中的令牌不会被转发。不在 `hosts` 中的主机永远不会被读取，包括配置变更后保留的引用。状态读取为这些引用返回 `unavailable`；head 查找会在启动 `gh` 前拒绝。 - **评审讨论上限**——每次状态读取每个 pull request 最多统计 `maxReviewThreads` 个评审讨论；超出时 `unresolvedThreads` 只是下限，由 `threadsTruncated` 标出。 - **不发现 fork**——按 head 分支查找时，不包含 head 位于其他仓库的 pull request。 -- **速率限制**——每次轮询为每个主机消耗一次 GraphQL 查询，评审讨论每多一页（100 个）再消耗一次；完整检查读取时检查记录每多一页（100 条）再消耗一次；每次 diff 读取另加两次 head 读取。这些都计入已登录账号的速率配额。 +- **速率限制**——每次轮询为每个主机消耗一次 GraphQL 查询，评审讨论每多一页（100 个）再消耗一次；完整检查读取时检查记录每多一页（100 条）再消耗一次；每次 diff 读取消耗一次 GraphQL 查询与一次 REST 请求。这些都计入已登录账号的速率配额。 - **不支持合并队列**——要求合并队列的基准分支会拒绝直接合并，提供方也不会把变更请求加入队列。 diff --git a/packages/experimental/forge-github/src/compare.ts b/packages/experimental/forge-github/src/compare.ts new file mode 100644 index 0000000000..f6c9ef4aec --- /dev/null +++ b/packages/experimental/forge-github/src/compare.ts @@ -0,0 +1,35 @@ +/** Reads GitHub's patch between two commits through the REST compare endpoint. */ + +import type { ForgeFailure, ForgeRepository } from '@deepseek-ai/dsh-experimental-forge' +import { isCommit } from './urls.ts' + +/** Runs one reading `gh` command and maps a refusal to a failure, as the provider's reads do. */ +export type GhTextReader = (args: readonly string[], signal: AbortSignal) => Promise + +/** + * Read the unified patch from the merge base of `base` and `head` to `head`, as + * GitHub computes the diff of a pull request with that base and head commit. + * Both ends are commit object names, so the answer never follows a moving + * branch. The patch keeps terminal escape sequences that file contents hold; + * `gh` 2.97 and newer refuses to print them without `--allow-escape-sequences`. + * GitHub returns the patch however many files it changes; `read` bounds its size. + * @param read - runs `gh` with the provider's output limit and failure mapping. + * @param repository - repository that holds both commits, with an owner and name already checked for inlining. + * @param base - commit whose merge base with `head` starts the patch. + * @param head - commit the patch ends at. + * @param signal - cancellation of the read. + * @returns the patch with `a/` and `b/` path prefixes, or why GitHub did not provide it. + * @throws when `base` or `head` is not a full commit object name. + */ +export async function readComparison( + read: GhTextReader, repository: ForgeRepository, base: string, head: string, signal: AbortSignal, +): Promise { + for (const commit of [base, head]) { + if (!isCommit(commit)) throw new Error(`forge-github: invalid commit ${JSON.stringify(commit)}`) + } + const { host, owner, name } = repository + return await read([ + 'api', '--hostname', host, '--allow-escape-sequences', '-H', 'Accept: application/vnd.github.diff', + `repos/${owner}/${name}/compare/${base}...${head}`, + ], signal) +} diff --git a/packages/experimental/forge-github/src/graphql.ts b/packages/experimental/forge-github/src/graphql.ts index cea358e1a0..75922515eb 100644 --- a/packages/experimental/forge-github/src/graphql.ts +++ b/packages/experimental/forge-github/src/graphql.ts @@ -7,7 +7,7 @@ import type { Mergeability, MergeMethod, ReviewDecision, } from '@deepseek-ai/dsh-experimental-forge' import { z } from 'zod' -import { checkedRepository } from './urls.ts' +import { checkedRepository, isCommit } from './urls.ts' const CHECK_FIELDS = `pageInfo { hasNextPage endCursor } nodes { @@ -59,9 +59,9 @@ export const HEAD_QUERY = `query($owner: String!, $name: String!, $branch: Strin } }` -/** Head commit of one pull request; variables `owner` and `name` are strings, `number` is an integer. */ -export const HEAD_COMMIT_QUERY = `query($owner: String!, $name: String!, $number: Int!) { - repository(owner: $owner, name: $name) { pullRequest(number: $number) { headRefOid } } +/** Head and base commits of one pull request; variables `owner` and `name` are strings, `number` is an integer. */ +export const PULL_REQUEST_COMMITS_QUERY = `query($owner: String!, $name: String!, $number: Int!) { + repository(owner: $owner, name: $name) { pullRequest(number: $number) { headRefOid baseRefOid } } }` const checkRunSchema = z.object({ @@ -186,7 +186,10 @@ const headSchema = z.object({ }), }) -const headCommitSchema = z.object({ data: z.object({ repository: z.object({ pullRequest: z.object({ headRefOid: z.string() }) }) }) }) +const pullRequestCommitsSchema = z.object({ data: z.object({ repository: z.object({ pullRequest: z.object({ + headRefOid: z.string().refine(isCommit), + baseRefOid: z.string().refine(isCommit), +}) }) }) }) type PullRequestNode = z.infer type RepositoryNode = z.infer @@ -364,14 +367,24 @@ function mapPageResponse(body: unknown, page: z.ZodType, label: string): P message: parsed.success ? parsed.data.errors?.[0]?.message ?? `GitHub ${label} page is unavailable` : `unexpected GitHub ${label} page` } } } +/** Head and base commits of a pull request as one read of {@link PULL_REQUEST_COMMITS_QUERY} named them. */ +export interface PullRequestCommits { + /** Commit at the head of the pull request. */ + readonly head: string + /** Commit of the base branch GitHub recorded with that head, whose merge base with it starts the pull request's diff. */ + readonly base: string +} + /** - * Map a head commit read of {@link HEAD_COMMIT_QUERY}. + * Map a read of {@link PULL_REQUEST_COMMITS_QUERY}. * @param body - parsed JSON printed by `gh api graphql`. - * @returns the head commit object name, or undefined when the response does not name one. + * @returns both commits, or undefined when the response does not name them as full commit object names. */ -export function mapHeadCommitResponse(body: unknown): string | undefined { - const parsed = headCommitSchema.safeParse(body) - return parsed.success ? parsed.data.data.repository.pullRequest.headRefOid : undefined +export function mapPullRequestCommitsResponse(body: unknown): PullRequestCommits | undefined { + const parsed = pullRequestCommitsSchema.safeParse(body) + if (!parsed.success) return undefined + const { headRefOid, baseRefOid } = parsed.data.data.repository.pullRequest + return { head: headRefOid, base: baseRefOid } } /** diff --git a/packages/experimental/forge-github/src/provider.ts b/packages/experimental/forge-github/src/provider.ts index 16c93acfbc..f48f98a07b 100644 --- a/packages/experimental/forge-github/src/provider.ts +++ b/packages/experimental/forge-github/src/provider.ts @@ -7,12 +7,13 @@ import type { } from '@deepseek-ai/dsh-experimental-forge' import { GhOutputLimitError, GhTimeoutError, GhUnavailableError } from './gh.ts' import type { GhOutput, GhRunner } from './gh.ts' +import { readComparison } from './compare.ts' import { - CHECKS_QUERY, hasGraphqlFields, HEAD_COMMIT_QUERY, HEAD_QUERY, mapCheckPageResponse, mapHeadCommitResponse, mapHeadResponse, - mapSnapshotResponse, mapThreadPageResponse, snapshotQuery, THREADS_QUERY, + CHECKS_QUERY, hasGraphqlFields, HEAD_QUERY, mapCheckPageResponse, mapHeadResponse, mapPullRequestCommitsResponse, + mapSnapshotResponse, mapThreadPageResponse, PULL_REQUEST_COMMITS_QUERY, snapshotQuery, THREADS_QUERY, } from './graphql.ts' import type { SnapshotPages } from './graphql.ts' -import { checkedRepository, parsePullRequestUrl, parseRemoteUrl } from './urls.ts' +import { checkedRepository, isCommit, parsePullRequestUrl, parseRemoteUrl } from './urls.ts' import { submitGitHubReview } from './review.ts' const AUTHENTICATION = /gh auth login|authentication|bad credentials|HTTP 401/i @@ -21,13 +22,13 @@ const AUTHENTICATION = /gh auth login|authentication|bad credentials|HTTP 401/i const HTTP_STATUS = /\(HTTP (\d{3})\)/ /** - * GitHub's error code for an answer over its size limits, such as a diff over - * 300 files: `gh pr diff` prints it after the field, and `gh api` returns it in the JSON error body. + * GitHub's error code for an answer over its size limits, such as a file over + * the contents API limit: `gh api` returns it in the JSON error body. */ const TOO_LARGE = /\btoo_large\b/ -/** A full SHA-1 or SHA-256 object name. */ -const COMMIT = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/i +/** HTTP status GitHub answers when it refuses to render a diff over its limits. */ +const NOT_ACCEPTABLE = '406' /** Bounds on what one read retains or counts. */ export interface GitHubLimits { @@ -142,7 +143,7 @@ export class GitHubProvider implements ForgeProvider { const { host, owner, name } = checkedRepository(ref) const failure = this.hostFailure(host) if (failure !== undefined) return { code: 'failed', message: failure.message } - if (!COMMIT.test(request.headSha)) throw new Error(`forge-github: invalid head commit ${JSON.stringify(request.headSha)}`) + if (!isCommit(request.headSha)) throw new Error(`forge-github: invalid head commit ${JSON.stringify(request.headSha)}`) return await this.act([ 'api', '--hostname', host, '--method', 'PUT', `repos/${owner}/${name}/pulls/${ref.number}/merge`, '-f', `merge_method=${request.method}`, '-f', `sha=${request.headSha}`, @@ -150,21 +151,28 @@ export class GitHubProvider implements ForgeProvider { } /** - * Read the pull request's diff with `gh pr diff`, which GitHub computes - * against the merge base at the head of the moment it answers. The head is - * read before and after the diff; the patch is returned only when both reads - * name `headSha`, and a different head returns `stale` without the patch. + * Read the pull request's diff at `headSha`. One GraphQL read names the + * current head and the base commit GitHub recorded with it; a head other + * than `headSha` returns `stale`. Otherwise the compare endpoint reads the + * patch from the merge base of that base commit and `headSha`, as the pull + * request's own diff does, with both ends fixed. */ async diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise { const { host, owner, name } = checkedRepository(ref) const failure = this.hostFailure(host) if (failure !== undefined) return failure - if (!COMMIT.test(headSha)) throw new Error(`forge-github: invalid head commit ${JSON.stringify(headSha)}`) - const before = await this.headMismatch(ref, headSha, signal) - if (before !== undefined) return before - const patch = await this.read(['pr', 'diff', String(ref.number), '--repo', `${host}/${owner}/${name}`, '--color', 'never'], signal) - if (typeof patch !== 'string') return patch - return await this.headMismatch(ref, headSha, signal) ?? patch + if (!isCommit(headSha)) throw new Error(`forge-github: invalid head commit ${JSON.stringify(headSha)}`) + const output = await this.read([ + 'api', 'graphql', '--hostname', host, + '-f', `query=${PULL_REQUEST_COMMITS_QUERY}`, '-f', `owner=${owner}`, '-f', `name=${name}`, '-F', `number=${ref.number}`, + ], signal) + if (typeof output !== 'string') return output + const commits = mapPullRequestCommitsResponse(parseJson(output)) + if (commits === undefined) return { code: 'unavailable', message: 'unexpected GitHub response to the pull request commit read' } + if (commits.head.toLowerCase() !== headSha.toLowerCase()) { + return { code: 'stale', message: `the pull request head moved from ${headSha} to ${commits.head}` } + } + return await readComparison(async (args, operation) => await this.read(args, operation), ref.repository, commits.base, headSha, signal) } /** Read one file at a commit through the contents API in its raw form. */ @@ -172,7 +180,7 @@ export class GitHubProvider implements ForgeProvider { const { host, owner, name } = checkedRepository(ref) const failure = this.hostFailure(host) if (failure !== undefined) return failure - if (!COMMIT.test(commit)) throw new Error(`forge-github: invalid commit ${JSON.stringify(commit)}`) + if (!isCommit(commit)) throw new Error(`forge-github: invalid commit ${JSON.stringify(commit)}`) return await this.read([ 'api', '--hostname', host, '-H', 'Accept: application/vnd.github.raw+json', `repos/${owner}/${name}/contents/${encodedPath(path)}?ref=${commit}`, @@ -197,23 +205,6 @@ export class GitHubProvider implements ForgeProvider { } } - /** - * Read the head commit of a pull request. - * @returns undefined when it is `headSha`, `stale` when it is another commit, or why it could not be read. - */ - private async headMismatch(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise { - const { host, owner, name } = ref.repository - const output = await this.read([ - 'api', 'graphql', '--hostname', host, - '-f', `query=${HEAD_COMMIT_QUERY}`, '-f', `owner=${owner}`, '-f', `name=${name}`, '-F', `number=${ref.number}`, - ], signal) - if (typeof output !== 'string') return output - const head = mapHeadCommitResponse(parseJson(output)) - if (head === undefined) return { code: 'unavailable', message: 'unexpected GitHub response to the head commit read' } - if (head.toLowerCase() === headSha.toLowerCase()) return undefined - return { code: 'stale', message: `the pull request head moved from ${headSha} to ${head}` } - } - /** * Run one reading `gh` command, returning its output. A refusal, a `gh` that * cannot start, output over `maxOutputBytes`, and a run past `timeoutMs` become failures. @@ -231,7 +222,7 @@ export class GitHubProvider implements ForgeProvider { if (output.exitCode === 0) return output.stdout const { message, status } = refusal(output) if (AUTHENTICATION.test(output.stderr)) return { code: 'unauthenticated', message } - if (TOO_LARGE.test(output.stderr) || TOO_LARGE.test(output.stdout)) return { code: 'too-large', message } + if (status === NOT_ACCEPTABLE || TOO_LARGE.test(output.stderr) || TOO_LARGE.test(output.stdout)) return { code: 'too-large', message } return { code: status === '404' || /could not resolve to a pullrequest/i.test(output.stderr) ? 'not-found' : 'unavailable', message } } diff --git a/packages/experimental/forge-github/src/urls.ts b/packages/experimental/forge-github/src/urls.ts index 539abe3a58..a3a2a7f3c3 100644 --- a/packages/experimental/forge-github/src/urls.ts +++ b/packages/experimental/forge-github/src/urls.ts @@ -5,6 +5,9 @@ import type { ChangeRequestRef, ForgeRepository } from '@deepseek-ai/dsh-experim /** Characters GitHub permits in owner and repository names. */ const SEGMENT = /^[A-Za-z0-9_.-]+$/ +/** A full SHA-1 or SHA-256 object name. */ +const COMMIT = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/i + /** Host, owner, and repository name addressed by a URL. */ export interface GitHubLocation { /** Lowercase host name. */ @@ -38,6 +41,15 @@ export function checkedRepository(ref: ChangeRequestRef): ForgeRepository { return ref.repository } +/** + * Whether a value is a full commit object name that can be inlined into a request. + * @param value - candidate commit. + * @returns whether it is 40 or 64 hexadecimal digits. + */ +export function isCommit(value: string): boolean { + return COMMIT.test(value) +} + /** * Parse a Git remote URL. Accepts `http(s)://[user@]host[:port]/owner/name`, * `ssh://[user@]host[:port]/owner/name`, `git://host/owner/name`, and the diff --git a/packages/experimental/forge-github/tests/compare.spec.ts b/packages/experimental/forge-github/tests/compare.spec.ts new file mode 100644 index 0000000000..878de35501 --- /dev/null +++ b/packages/experimental/forge-github/tests/compare.spec.ts @@ -0,0 +1,30 @@ +import type { ForgeFailure } from '@deepseek-ai/dsh-experimental-forge' +import { describe, expect, it } from 'vitest' +import { readComparison } from '../src/compare.ts' +import { demo } from './fixtures.ts' + +const signal = new AbortController().signal +const [base, head] = ['a'.repeat(40), 'b'.repeat(64)] + +describe('compare read', () => { + it('reads the diff media type of the three-dot comparison, keeping escape sequences, and returns what the reader returns', async () => { + const calls: (readonly string[])[] = [] + const refused: ForgeFailure = { code: 'too-large', message: 'Not Acceptable' } + let answer: string | ForgeFailure = 'diff --git a/x b/x\n' + const read = async (args: readonly string[]) => { calls.push(args); return answer } + + await expect(readComparison(read, demo, base, head, signal)).resolves.toBe('diff --git a/x b/x\n') + answer = refused + await expect(readComparison(read, demo, base, head, signal)).resolves.toBe(refused) + expect(calls[0]).toEqual([ + 'api', '--hostname', 'github.com', '--allow-escape-sequences', '-H', 'Accept: application/vnd.github.diff', `repos/octo/demo/compare/${base}...${head}`, + ]) + }) + + it.each([['base', 'main', head], ['head', base, 'HEAD']])('refuses a %s that is not a full commit object name before reading', async (_label, from, to) => { + const calls: (readonly string[])[] = [] + await expect(readComparison(async (args) => { calls.push(args); return '' }, demo, from, to, signal)) + .rejects.toThrow(`forge-github: invalid commit ${JSON.stringify(from === base ? to : from)}`) + expect(calls).toEqual([]) + }) +}) diff --git a/packages/experimental/forge-github/tests/graphql.spec.ts b/packages/experimental/forge-github/tests/graphql.spec.ts index 0181602326..6084784a73 100644 --- a/packages/experimental/forge-github/tests/graphql.spec.ts +++ b/packages/experimental/forge-github/tests/graphql.spec.ts @@ -2,7 +2,7 @@ import { changeRequestKey } from '@deepseek-ai/dsh-experimental-forge' import type { ChangeRequestSnapshot } from '@deepseek-ai/dsh-experimental-forge' import { describe, expect, it } from 'vitest' import { - mapCheckPageResponse, mapHeadCommitResponse, mapHeadResponse, mapSnapshotResponse, mapThreadPageResponse, snapshotQuery, + mapCheckPageResponse, mapHeadResponse, mapPullRequestCommitsResponse, mapSnapshotResponse, mapThreadPageResponse, snapshotQuery, } from '../src/graphql.ts' import type { SnapshotPages, ThreadPageResult } from '../src/graphql.ts' import { checkRun, demo, pull, pullRequestNode, repositorySettings, statusContext, threadPage } from './fixtures.ts' @@ -413,11 +413,15 @@ describe('review thread page mapping', () => { }) }) -describe('head commit mapping', () => { - it('returns the head commit, or nothing for a response that names none', () => { - expect(mapHeadCommitResponse({ data: { repository: { pullRequest: { headRefOid: 'a'.repeat(40) } } } })).toBe('a'.repeat(40)) - expect(mapHeadCommitResponse({ data: { repository: { pullRequest: null } } })).toBeUndefined() - expect(mapHeadCommitResponse('not json')).toBeUndefined() +describe('pull request commit mapping', () => { + const answer = (headRefOid: string, baseRefOid: string) => ({ data: { repository: { pullRequest: { headRefOid, baseRefOid } } } }) + + it('returns the head and base commits, or nothing for a response that does not name both as commit object names', () => { + expect(mapPullRequestCommitsResponse(answer('a'.repeat(40), 'b'.repeat(64)))).toEqual({ head: 'a'.repeat(40), base: 'b'.repeat(64) }) + expect(mapPullRequestCommitsResponse(answer('a'.repeat(40), 'main'))).toBeUndefined() + expect(mapPullRequestCommitsResponse(answer('HEAD', 'b'.repeat(40)))).toBeUndefined() + expect(mapPullRequestCommitsResponse({ data: { repository: { pullRequest: null } } })).toBeUndefined() + expect(mapPullRequestCommitsResponse('not json')).toBeUndefined() }) }) diff --git a/packages/experimental/forge-github/tests/provider.spec.ts b/packages/experimental/forge-github/tests/provider.spec.ts index bc48a26b30..4ffe4e411f 100644 --- a/packages/experimental/forge-github/tests/provider.spec.ts +++ b/packages/experimental/forge-github/tests/provider.spec.ts @@ -2,7 +2,7 @@ import { changeRequestKey } from '@deepseek-ai/dsh-experimental-forge' import { describe, expect, it } from 'vitest' import { GhOutputLimitError, GhTimeoutError, GhUnavailableError } from '../src/gh.ts' import type { GhOutput, GhRunner } from '../src/gh.ts' -import { HEAD_COMMIT_QUERY, THREADS_QUERY } from '../src/graphql.ts' +import { PULL_REQUEST_COMMITS_QUERY, THREADS_QUERY } from '../src/graphql.ts' import { GitHubProvider } from '../src/provider.ts' import type { GitHubLimits } from '../src/provider.ts' import { checkRun, demo, pull, pullRequestNode, threadPage } from './fixtures.ts' @@ -400,62 +400,64 @@ describe('GitHub provider actions', () => { describe('GitHub provider reads', () => { const head = 'c'.repeat(40) const pushed = 'd'.repeat(40) + const base = 'b'.repeat(40) - /** Answer head commit reads with `heads` in turn, the last one repeating, and every other command with `other`. */ - function pinned(heads: readonly (string | GhOutput)[], other: (args: readonly string[]) => GhOutput = () => output('diff --git a/x b/x\n')) { - let read = 0 + /** Answer the pull request commit read with `commits`, and every other command, the compare read among them, with `other`. */ + function pinned( + commits: { readonly head: string; readonly base: string } | GhOutput = { head, base }, + other: (args: readonly string[]) => GhOutput = () => output('diff --git a/x b/x\n'), + ) { return fakeGh((args) => { - if (!args.includes(`query=${HEAD_COMMIT_QUERY}`)) return other(args) - const answer = heads[Math.min(read++, heads.length - 1)] - return typeof answer === 'string' ? output({ data: { repository: { pullRequest: { headRefOid: answer } } } }) : answer! + if (!args.includes(`query=${PULL_REQUEST_COMMITS_QUERY}`)) return other(args) + return 'exitCode' in commits ? commits : output({ data: { repository: { pullRequest: { headRefOid: commits.head, baseRefOid: commits.base } } } }) }) } - it('reads the pull request diff through gh pr diff on its host between two reads of the requested head', async () => { - const gh = pinned([head]) + it('reads the diff between the recorded base and the requested head through the compare API after one commit read', async () => { + const gh = pinned() await expect(provider(gh.run).diff(pull(7, enterprise), head, signal)).resolves.toBe('diff --git a/x b/x\n') - const headRead = ['api', 'graphql', '--hostname', 'ghe.corp.example', - '-f', `query=${HEAD_COMMIT_QUERY}`, '-f', 'owner=team', '-f', 'name=svc', '-F', 'number=7'] - expect(gh.calls).toEqual([headRead, ['pr', 'diff', '7', '--repo', 'ghe.corp.example/team/svc', '--color', 'never'], headRead]) + expect(gh.calls).toEqual([ + ['api', 'graphql', '--hostname', 'ghe.corp.example', + '-f', `query=${PULL_REQUEST_COMMITS_QUERY}`, '-f', 'owner=team', '-f', 'name=svc', '-F', 'number=7'], + ['api', '--hostname', 'ghe.corp.example', '--allow-escape-sequences', '-H', 'Accept: application/vnd.github.diff', + `repos/team/svc/compare/${base}...${head}`], + ]) }) - it('compares the head commit without regard to case', async () => { - await expect(provider(pinned([head]).run).diff(pull(7), head.toUpperCase(), signal)).resolves.toBe('diff --git a/x b/x\n') + it('compares the head commit without regard to case and addresses the requested spelling', async () => { + const gh = pinned() + + await expect(provider(gh.run).diff(pull(7), head.toUpperCase(), signal)).resolves.toBe('diff --git a/x b/x\n') + expect(gh.calls[1]?.at(-1)).toBe(`repos/octo/demo/compare/${base}...${head.toUpperCase()}`) }) - it('reports a head that moved before the read as stale without reading the diff', async () => { - const gh = pinned([pushed]) + it('reports a head that moved past the requested commit as stale without reading the diff', async () => { + const gh = pinned({ head: pushed, base }) await expect(provider(gh.run).diff(pull(7), head, signal)) .resolves.toEqual({ code: 'stale', message: `the pull request head moved from ${head} to ${pushed}` }) expect(gh.calls).toHaveLength(1) }) - it('discards a diff read while the head moved', async () => { - const gh = pinned([head, pushed]) - - await expect(provider(gh.run).diff(pull(7), head, signal)) - .resolves.toEqual({ code: 'stale', message: `the pull request head moved from ${head} to ${pushed}` }) - expect(gh.calls.map(call => call[0] === 'pr' ? 'diff' : 'head')).toEqual(['head', 'diff', 'head']) - }) - it.each([ - ['before the diff', [output({ data: { repository: { pullRequest: null } }, errors: [{ type: 'NOT_FOUND', message: 'Could not resolve' }] }, 1, - 'gh: Could not resolve to a PullRequest with the number of 7.')], 1, + ['a missing pull request', output({ data: { repository: { pullRequest: null } }, errors: [{ type: 'NOT_FOUND', message: 'Could not resolve' }] }, 1, + 'gh: Could not resolve to a PullRequest with the number of 7.'), { code: 'not-found', message: 'gh: Could not resolve to a PullRequest with the number of 7.' }], - ['after the diff', [head, output({ message: 'Server Error' }, 1, 'gh: Server Error (HTTP 502)')], 3, { code: 'unavailable', message: 'Server Error' }], - ['without a head commit in the answer', [output({ data: { repository: {} } })], 1, - { code: 'unavailable', message: 'unexpected GitHub response to the head commit read' }], - ] as const)('reports a head read that fails %s', async (_label, heads, calls, failure) => { - const gh = pinned(heads) + ['a refusal', output({ message: 'Server Error' }, 1, 'gh: Server Error (HTTP 502)'), { code: 'unavailable', message: 'Server Error' }], + ['an answer without commits', output({ data: { repository: {} } }), + { code: 'unavailable', message: 'unexpected GitHub response to the pull request commit read' }], + ['a base that is not a commit object name', output({ data: { repository: { pullRequest: { headRefOid: head, baseRefOid: 'main' } } } }), + { code: 'unavailable', message: 'unexpected GitHub response to the pull request commit read' }], + ] as const)('reports a commit read that fails with %s without reading the diff', async (_label, answer, failure) => { + const gh = pinned(answer) await expect(provider(gh.run).diff(pull(7), head, signal)).resolves.toEqual(failure) - expect(gh.calls).toHaveLength(calls) + expect(gh.calls).toHaveLength(1) }) it('refuses a head that is not a full commit object name before starting gh', async () => { - const gh = pinned([head]) + const gh = pinned() await expect(provider(gh.run).diff(pull(7), 'HEAD', signal)).rejects.toThrow('forge-github: invalid head commit "HEAD"') expect(gh.calls).toEqual([]) @@ -479,16 +481,20 @@ describe('GitHub provider reads', () => { ['a missing pull request', '', 'GraphQL: Could not resolve to a PullRequest with the number of 7. (repository.pullRequest)', { code: 'not-found', message: 'GraphQL: Could not resolve to a PullRequest with the number of 7. (repository.pullRequest)' }], ['a missing file', { message: 'Not Found' }, 'gh: Not Found (HTTP 404)', { code: 'not-found', message: 'Not Found' }], - // What gh 2.100 prints for a pull request over 300 files. - ['a diff over GitHub\'s size limit', '', [ - 'could not find pull request diff: HTTP 406: Sorry, the diff exceeded the maximum number of files (300). Consider using \'List pull ' - + 'requests files\' API or locally cloning the repository instead. (https://api.github.com/repos/octo/demo/pulls/7)', - 'PullRequest.diff too_large', - ].join('\n'), { + // What gh 2.100 prints when GitHub refuses a diff over its size limit, as the pull request diff endpoint does past 300 files. + ['a diff over GitHub\'s size limit', { + message: 'Sorry, the diff exceeded the maximum number of files (300). Consider using \'List pull requests files\' API or locally ' + + 'cloning the repository instead.', + errors: [{ resource: 'PullRequest', field: 'diff', code: 'too_large' }], + status: '406', + }, 'gh: Sorry, the diff exceeded the maximum number of files (300). Consider using \'List pull requests files\' API or locally ' + + 'cloning the repository instead. (HTTP 406)', { code: 'too-large', - message: 'could not find pull request diff: HTTP 406: Sorry, the diff exceeded the maximum number of files (300). Consider using ' - + '\'List pull requests files\' API or locally cloning the repository instead. (https://api.github.com/repos/octo/demo/pulls/7)', + message: 'Sorry, the diff exceeded the maximum number of files (300). Consider using \'List pull requests files\' API or locally ' + + 'cloning the repository instead.', }], + ['a diff GitHub refuses to render without an error code', { message: 'Not Acceptable' }, 'gh: Not Acceptable (HTTP 406)', + { code: 'too-large', message: 'Not Acceptable' }], ['a file over the contents API limit', { message: 'This API returns blobs up to 1 MB in size. The requested blob is too large to fetch via the API.', errors: [{ resource: 'Blob', field: 'data', code: 'too_large' }], @@ -500,7 +506,7 @@ describe('GitHub provider reads', () => { { code: 'unauthenticated', message: 'To get started with GitHub CLI, please run: gh auth login' }], ['no output', '', '', { code: 'unavailable', message: 'gh exited with 1' }], ])('reports a read refused for %s', async (_label, body, stderr, failure) => { - await expect(provider(pinned([head], () => output(body, 1, stderr)).run).diff(pull(7), head, signal)).resolves.toEqual(failure) + await expect(provider(pinned(undefined, () => output(body, 1, stderr)).run).diff(pull(7), head, signal)).resolves.toEqual(failure) }) it('reports a read as unavailable when gh cannot start, rethrows other failures, and refuses an invalid commit', async () => { @@ -511,8 +517,10 @@ describe('GitHub provider reads', () => { }) it('reports output over maxOutputBytes as too large and a run past timeoutMs as unavailable', async () => { - await expect(provider(() => Promise.reject(new GhOutputLimitError('gh pr output exceeds maxOutputBytes'))).diff(pull(7), head, signal)) - .resolves.toEqual({ code: 'too-large', message: 'gh pr output exceeds maxOutputBytes' }) + const overLimit = pinned(undefined, () => { throw new GhOutputLimitError('gh api output exceeds maxOutputBytes') }) + await expect(provider(overLimit.run).diff(pull(7), head, signal)) + .resolves.toEqual({ code: 'too-large', message: 'gh api output exceeds maxOutputBytes' }) + expect(overLimit.calls).toHaveLength(2) await expect(provider(() => Promise.reject(new GhTimeoutError('gh api timed out after 30000ms'))).fileText(pull(7), head, 'a', signal)) .resolves.toEqual({ code: 'unavailable', message: 'gh api timed out after 30000ms' }) }) From af1a520b5a23dfd10910f0ffb696130719753515 Mon Sep 17 00:00:00 2001 From: yudshj Date: Fri, 6 Nov 2026 17:21:56 +0800 Subject: [PATCH 31/44] =?UTF-8?q?test(open-in-app):=20=E6=98=BE=E5=BC=8F?= =?UTF-8?q?=E5=BF=BD=E7=95=A5=E5=90=8E=E5=8F=B0=E5=88=B7=E6=96=B0=E8=BF=94?= =?UTF-8?q?=E5=9B=9E=E7=9A=84=20Promise?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit refreshApplications now returns a Promise; the refresh lifecycle test starts refreshes without awaiting them, so it marks them with void for the type-aware no-floating-promises lint. --- .../client/ui-open-in-app/tests/open-path.client.spec.ts | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/packages/client/ui-open-in-app/tests/open-path.client.spec.ts b/packages/client/ui-open-in-app/tests/open-path.client.spec.ts index af6f3475b7..3222bbb738 100644 --- a/packages/client/ui-open-in-app/tests/open-path.client.spec.ts +++ b/packages/client/ui-open-in-app/tests/open-path.client.spec.ts @@ -169,15 +169,15 @@ describe('OpenInAppPathController association answers', () => { it('runs one background refresh at a time, cancelling the older for the newer file, and disposal cancels it', async () => { const { queries, controller } = deferredRemote() - controller.refreshApplications('/w/a.txt') - controller.refreshApplications('/w/b.txt') + void controller.refreshApplications('/w/a.txt') + void controller.refreshApplications('/w/b.txt') expect(queries.map(query => [query.path, query.signal?.aborted])).toEqual([['/w/a.txt', true], ['/w/b.txt', false]]) queries[1]!.answer(answered(app('TextEdit'))) await vi.waitFor(() => { expect(controller.cachedApplications('/w/c.txt')).toEqual([app('TextEdit')]) }) - controller.refreshApplications('/w/d.md') + void controller.refreshApplications('/w/d.md') controller.dispose() expect(queries[2]).toMatchObject({ path: '/w/d.md', signal: { aborted: true } }) - controller.refreshApplications('/w/e.json') + void controller.refreshApplications('/w/e.json') expect(queries).toHaveLength(3) expect(controller.cachedApplications('/w/a.txt')).toEqual([app('TextEdit')]) }) From a2dc561cb9c6109d4135c76fbff84dbe18aea237 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 17:28:12 +0800 Subject: [PATCH 32/44] Limit shared CUA guidance changes to hover and foreground recovery --- packages/experimental/cua-macos/README.md | 28 +++++++++---------- packages/experimental/cua-macos/README.zh.md | 28 +++++++++---------- .../experimental/cua-macos/src/guidance.ts | 8 +----- .../cua-macos/tests/guidance.spec.ts | 2 +- packages/experimental/cua/src/guidance.ts | 27 +++++++++--------- packages/experimental/cua/src/types.ts | 13 +++++---- .../cua-native/system-prompt.expected.md | 28 +++++++++---------- .../session/cua-ptc/system-prompt.expected.md | 28 +++++++++---------- .../system-prompt.expected.md | 28 +++++++++---------- 9 files changed, 93 insertions(+), 97 deletions(-) diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index 6abe3c08de..c03e5a8d1b 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -176,7 +176,7 @@ The plugin contributes one `cua` operation tool with method-specific parameters. ##### Operation guidance (verbatim) ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -187,20 +187,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. -- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. +- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. +- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -210,17 +210,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then explicitly activate the target and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. +- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 13f4fa1d06..82de79da81 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -176,7 +176,7 @@ AppKit 的 sheet 可以出现在文档窗口的树里，并有独立的窗口 ID ##### 操作指导文本（原文） ```markdown -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -187,20 +187,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. -- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. +- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. +- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -210,17 +210,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then explicitly activate the target and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. +- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. diff --git a/packages/experimental/cua-macos/src/guidance.ts b/packages/experimental/cua-macos/src/guidance.ts index 0379d91120..1739be9639 100644 --- a/packages/experimental/cua-macos/src/guidance.ts +++ b/packages/experimental/cua-macos/src/guidance.ts @@ -2,17 +2,11 @@ import { COMMON_GUIDANCE } from '@deepseek-ai/dsh-cua/guidance' /** macOS operation levels, permissions, selectors, and native measurement units. */ -export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' +export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action.' const methodDetails = new Map([ - ['cua.getApp', 'Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background.'], - ['cua.findApps', 'Searches /Applications, /System/Applications, and ~/Applications.'], - ['app.show', 'Unhides the app and restores its focused, main, or sole window without activating it.'], - ['window.show', 'Unhides the app and restores this window without activating it.'], ['mouseMove', 'Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured.'], ['click', 'Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click.'], - ['typeText', 'Line endings send Return and tabs send Tab.'], - ['paste', 'Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying.'], ]) /** Complete model instructions for this provider. */ diff --git a/packages/experimental/cua-macos/tests/guidance.spec.ts b/packages/experimental/cua-macos/tests/guidance.spec.ts index c00f600a79..3ad190d5c8 100644 --- a/packages/experimental/cua-macos/tests/guidance.spec.ts +++ b/packages/experimental/cua-macos/tests/guidance.spec.ts @@ -30,7 +30,7 @@ it('states the macOS accessibility, keyboard, and mouse levels', () => { const hover = GUIDANCE.split('\n').find(line => line.startsWith('- mouseMove(')) for (const term of ['AXShowAlternateUI', 'AXShowDefaultUI', 'HOVER_UNAVAILABLE', 'systemPointerHover']) expect(hover).toContain(term) expect(GUIDANCE.split('\n').find(line => line.startsWith('- click('))).toContain('call app.activate() first') - for (const platformDetail of ['macOS', 'AXShowAlternateUI', 'AXRaise', 'HOVER_UNAVAILABLE', 'systemPointerHover', 'mouse-down', 'bundle ID', 'Paste command']) { + for (const platformDetail of ['macOS', 'AXShowAlternateUI', 'AXRaise', 'HOVER_UNAVAILABLE', 'systemPointerHover', 'mouse-down']) { expect(COMMON_GUIDANCE).not.toContain(platformDetail) } }) diff --git a/packages/experimental/cua/src/guidance.ts b/packages/experimental/cua/src/guidance.ts index 27be40b4c7..0ce5da8944 100644 --- a/packages/experimental/cua/src/guidance.ts +++ b/packages/experimental/cua/src/guidance.ts @@ -1,4 +1,5 @@ /** Shared model instructions for native application and window operations. */ +/** Async method catalog with consent and observation rules, grouped by observation, mouse, and keyboard. */ export const COMMON_GUIDANCE = `Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. In persistent PTC, use the host-provided bridge initialization once and retain the returned objects; cua comes from that bridge and is not a preinstalled global. Use void await for observations that emit, and never stringify screenshot image bytes. @@ -8,20 +9,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. -- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. +- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. +- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -31,17 +32,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then explicitly activate the target and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type text at the current input focus. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. +- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name.` diff --git a/packages/experimental/cua/src/types.ts b/packages/experimental/cua/src/types.ts index 4c6b27b78e..9f13f5b3c4 100644 --- a/packages/experimental/cua/src/types.ts +++ b/packages/experimental/cua/src/types.ts @@ -96,7 +96,7 @@ export type CuaScrollOptions = Pick, 'x' | 'y' | 'deltaL */ export interface CuaSurface { /** - * Request that the provider show the selected application or window. + * Unhide the app and restore a minimized selected, focused, main or sole window without requesting activation. * @returns Dispatch evidence, the selected window when available, and sampled foreground change; read fresh state before input. */ show(): Promise @@ -191,8 +191,9 @@ export interface CuaWindow extends CuaSurface { */ getScreenshot(options?: CuaObservationOptions): Promise /** - * Request hover at an element or a point in this window. The provider reports the route used and whether - * the visible system pointer moved. Observe the resulting UI after its hover delay. + * Hover over an element or a point without pressing a button. The provider reports the route and whether the system pointer moved. + * Returning and subsequent observations do not restore the previous pointer position. + * Other input and application changes can interrupt hover; observe before continuing. * @param target - current element id or screenshot-local [x,y]. * @param options - coordinate mode; applies only to [x,y]. * @returns dispatch evidence and the selected route; observe the resulting UI after its hover delay. @@ -270,9 +271,9 @@ export interface CuaApi { */ getState(options?: CuaObservationOptions): Promise<{ apps: CuaAppInfo[]; browsers: [] }> /** - * Select an app and obtain consent. Providers with app-level accessibility silently read the initial tree. - * @param selector - A selector accepted by the provider; ambiguous matches reject. - * @returns An application handle tied to this process lifetime. + * Select an app, obtain consent, launch it in the background if necessary, and silently retain initial AX state. + * @param selector - Exact name, bundle ID, absolute application path or running PID string; ambiguous names reject. + * @returns An application handle tied to this process lifetime; its initial tree read becomes the current snapshot. */ getApp(selector: string): Promise } diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index 5a793421cf..3d9737ffcf 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -33,7 +33,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -44,20 +44,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. -- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. +- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. +- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -67,17 +67,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then explicitly activate the target and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. +- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index 8e3daa7253..e40ae8bdd0 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -46,20 +46,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. -- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. +- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. +- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -69,17 +69,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then explicitly activate the target and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. +- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index fcd315e68c..c357fcc58d 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -35,7 +35,7 @@ Start independent delegations with `subagent` or `subagent_fork` together in one Bash, formatters, code generators, and scripts can overwrite concurrent edits without conflict detection; coordinate their changes explicitly. The agent integrating the work must review the final diff and run relevant tests. -macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. +macOS: accessibility tree operations are available at app or window level. Simulated keyboard input is available at app level. Simulated mouse input is available at window level. For pointer input on a sheet, select its window ID with app.getWindow(id). For menu items outside a window, use app.invoke(id). findApps searches /Applications, /System/Applications, and ~/Applications. Use the Computer Use permission page to grant the helper application Accessibility and Screen Recording permissions in System Settings. Native window and AX bounds use macOS points, while delivered screenshots carry their own pixel dimensions. show does not switch Spaces, and Stage Manager can present reduced window geometry; explain any required foreground step before calling app.activate() or performing an advertised AXRaise action. Use cua to observe and operate native applications. Always use cua.getCapabilities() to confirm whether a specific API is available on the current platform; its reported capabilities take precedence over availability statements in these instructions. All methods are async; observation options default to { emit: true }, and emit: false suppresses sink output only, leaving returned values unchanged. Actions report dispatch evidence, not verified completion. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI actions and ask the user to unlock the desktop session; null or unknown means verify a usable unlocked desktop session first. cua.getApp requests user consent; if access is denied or revoked, stop using that application until the user explicitly permits another request, and never bypass a refusal or switch tools to avoid it. Never replay uncertain input without a fresh observation. @@ -46,20 +46,20 @@ Signatures show resolved return values after await. ActionResult = { dispatched: Setup and discovery: - cua.getCapabilities(options?: { emit?: boolean }) -> { platform: string, permissions: { accessibility: boolean, screenRecording: boolean }, appLevelAccessibility: boolean, keyboard: ('app' | 'window')[], sessionLocked: boolean | null, methods: string[], coordinateSpace: 'screenshot-pixels', browser: false } — inspect reported permissions, sessionLocked, methods, and keyboard scopes without requesting permissions. - cua.listApps(options?: { emit?: boolean }) -> { id: string, displayName: string, isRunning: true, pid: number }[] — enumerate running applications. -- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search for applications that match the query. Searches /Applications, /System/Applications, and ~/Applications. +- cua.findApps(query: string, options?: { emit?: boolean }) -> ({ id: string, displayName: string, path: string } & ({ isRunning: true, pid: number } | { isRunning: false }))[] — search installed applications, including stopped matches; only running matches carry pid. - cua.getState(options?: { emit?: boolean }) -> { apps: { id: string, displayName: string, isRunning: true, pid: number }[], browsers: [] } — summarize running applications; this is not an accessibility observation. -- cua.getApp(selector: string) -> App — select an application using a provider-supported selector and obtain consent. When app-level accessibility is available, selection reads the initial tree silently. Accepts a name, bundle ID, .app path, or running PID string. A stopped app launches in the background. -- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds. A null onCurrentSpace means desktop membership is unknown. -- app.getWindow(windowId: number) -> Window — select one window and silently read its initial accessibility tree. Select a window before screenshots or pointer input. +- cua.getApp(selector: string) -> App — select an application by name, bundle ID, absolute path, or running PID string, obtain consent, launch in the background if needed, and read initial AX state silently; it does not activate the app. +- app.getWindows(options?: { emit?: boolean }) -> { windowId: number, appName: string, title: string, bounds: { x: number, y: number, width: number, height: number }, isOnScreen: boolean, onCurrentSpace: boolean | null }[] — enumerate windows with global native bounds; null onCurrentSpace means unknown Space membership. +- app.getWindow(windowId: number) -> Window — select one window and read its initial AX state silently; do this before any window screenshot or coordinate input. - app.activate() -> ActionResult — request explicit foreground activation; use only when foreground access is required. -- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the application be shown. Unhides the app and restores its focused, main, or sole window without activating it. -- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — request that the selected window be shown. Unhides the app and restores this window without activating it. +- app.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide the app and restore its selected, focused, main, or sole window without activation. +- window.show() -> { dispatched: boolean, windowId: number | null, foregroundChanged: boolean | null } — unhide and restore that explicit window without activation. ## Accessibility tree Available at app or window level depending on the platform. -Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so use IDs from the most recent observation. -- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text. includeMenus requests menu contents and details requests additional element information. +Element ids are opaque strings from the latest successful tree; the object facade keeps one current snapshot per app across its repeated app and window objects, a new read replaces it, and a failed read invalidates the previous one. Repeated strings can appear for different elements, so do not reuse an old id blindly, and element operations do not fall back to pointer input or typing. +- getAccessibilityTree(options?: { emit?: boolean, includeMenus?: boolean, details?: boolean }) -> string — read the full tree text; includeMenus expands menu contents and details adds identifiers, native bounds, and complete action names. - listActions(elementId: string) -> string[] — list the actions an element advertises. - invoke(elementId: string) -> ActionResult — invoke the element's default action; dispatched is false when it has none. - performAction(elementId: string, action: string, argument?: JsonValue) -> ActionResult — perform an observed action with an optional direct JSON argument. @@ -69,17 +69,17 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request hover at an observed element or window point. Inspect hoverRoute and systemPointerMoved to see how the provider dispatched it. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an observed element or a window point. Set clickCount to 2 to perform a left-button double-click. holdMs controls each press duration. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. -A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current window observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. +A string target is an element id; coordinate targets are [x, y] and are pixels in the latest returned screenshot by default. With relative: true they are integer thousandths of the window width and height from 0 to 999, based on the current AX observation or screenshot. Left clicks support one or two presses; right and middle clicks support one. FOREGROUND_REQUIRED means explain the need, then explicitly activate the target and observe again; activation is never automatic. Read fresh state after actions, and treat WINDOW_CHANGED, STALE_SNAPSHOT, and SCREENSHOT_REQUIRED as requiring the stated fresh observation or capture before any retry. ## Simulated keyboard Available at app or window level depending on the platform. -- typeText(text: string) -> ActionResult — type text at the current input focus. Line endings send Return and tabs send Tab. -- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — paste text or HTML at the current input focus. Uses the app’s Paste command and restores the clipboard unless another writer changes it. For PASTE_NOT_CONSUMED, inspect the target before retrying. +- typeText(text: string) -> ActionResult — type literal text at the app's current focus; line endings send Return and tabs send Tab. +- paste(text: string, options?: { format?: 'text' | 'html' }) -> ActionResult — deliver text or HTML through the app's Paste command, restoring prior clipboard contents unless another writer changes them; a Paste failure is surfaced without substituting typeText, and PASTE_NOT_CONSUMED means inspect before any retry. - pressKey(key: string, modifiers?: string[]) -> ActionResult — press and release one complete keystroke. For example pressKey('Tab', ['shift']) or pressKey('+'). Key and modifier names are platform-defined strings. Pass key and modifiers separately; do not pack them into a chord string. Raw tools use { method, params }: include appHandle for app calls and windowId for window calls, and use snapshotId with elementId for element routes. The emit option is facade-only: emit is local output control, so raw callers must follow the raw operation schema for admitted params. getAccessibilityTree and getWindow read through raw getState; getWindows maps to listWindows; getScreenshot maps to screenshot; cua.getState maps to listApps; every other named operation uses the same method name. From 24477680bd3bfa6e978964a45d6d70ff30545d09 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 17:46:22 +0800 Subject: [PATCH 33/44] Guide macOS clicks on clipped elements and verify their effects --- packages/experimental/cua-macos/README.md | 2 +- packages/experimental/cua-macos/README.zh.md | 2 +- packages/experimental/cua-macos/src/guidance.ts | 2 +- packages/experimental/cua-macos/tests/guidance.spec.ts | 3 ++- snapshots/session/cua-native/system-prompt.expected.md | 2 +- snapshots/session/cua-ptc/system-prompt.expected.md | 2 +- .../session/ptc-script-retained/system-prompt.expected.md | 2 +- 7 files changed, 8 insertions(+), 7 deletions(-) diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index c03e5a8d1b..db6dcd39a7 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -211,7 +211,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 82de79da81..567ae15da7 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -211,7 +211,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua-macos/src/guidance.ts b/packages/experimental/cua-macos/src/guidance.ts index 1739be9639..d57592e9db 100644 --- a/packages/experimental/cua-macos/src/guidance.ts +++ b/packages/experimental/cua-macos/src/guidance.ts @@ -6,7 +6,7 @@ export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are avail const methodDetails = new Map([ ['mouseMove', 'Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured.'], - ['click', 'Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click.'], + ['click', 'Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying.'], ]) /** Complete model instructions for this provider. */ diff --git a/packages/experimental/cua-macos/tests/guidance.spec.ts b/packages/experimental/cua-macos/tests/guidance.spec.ts index 3ad190d5c8..d16958a36a 100644 --- a/packages/experimental/cua-macos/tests/guidance.spec.ts +++ b/packages/experimental/cua-macos/tests/guidance.spec.ts @@ -29,7 +29,8 @@ it('states the macOS accessibility, keyboard, and mouse levels', () => { expect(PLATFORM_GUIDANCE).toContain('Simulated mouse input is available at window level') const hover = GUIDANCE.split('\n').find(line => line.startsWith('- mouseMove(')) for (const term of ['AXShowAlternateUI', 'AXShowDefaultUI', 'HOVER_UNAVAILABLE', 'systemPointerHover']) expect(hover).toContain(term) - expect(GUIDANCE.split('\n').find(line => line.startsWith('- click('))).toContain('call app.activate() first') + const click = GUIDANCE.split('\n').find(line => line.startsWith('- click(')) + for (const step of ['call app.activate() first', 'outside the window', 'screenshot pixels', 'before typing or retrying']) expect(click).toContain(step) for (const platformDetail of ['macOS', 'AXShowAlternateUI', 'AXRaise', 'HOVER_UNAVAILABLE', 'systemPointerHover', 'mouse-down']) { expect(COMMON_GUIDANCE).not.toContain(platformDetail) } diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index 3d9737ffcf..c9a19ee43f 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -68,7 +68,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index e40ae8bdd0..525f7cbd80 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -70,7 +70,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index c357fcc58d..84b37ec0bc 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -70,7 +70,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Both target forms send mouse-down and mouse-up events: an ID targets the element center and coordinates target the given point. Prefer invoke(id) for a background control’s default action. For a mouse click, call app.activate() first and observe the target again. A background app can ignore a dispatched click. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. From b9b6fccd6c3a8091bf8ac760c7bce802a9113a6b Mon Sep 17 00:00:00 2001 From: creatixchu Date: Fri, 6 Nov 2026 17:52:25 +0800 Subject: [PATCH 34/44] Name the window receiver in macOS click guidance --- packages/experimental/cua-macos/README.md | 2 +- packages/experimental/cua-macos/README.zh.md | 2 +- packages/experimental/cua-macos/src/guidance.ts | 2 +- packages/experimental/cua-macos/tests/guidance.spec.ts | 2 ++ snapshots/session/cua-native/system-prompt.expected.md | 2 +- snapshots/session/cua-ptc/system-prompt.expected.md | 2 +- snapshots/session/ptc-script-retained/system-prompt.expected.md | 2 +- 7 files changed, 8 insertions(+), 6 deletions(-) diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index db6dcd39a7..9a388e5d0c 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -211,7 +211,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 567ae15da7..1ae598bdc9 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -211,7 +211,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua-macos/src/guidance.ts b/packages/experimental/cua-macos/src/guidance.ts index d57592e9db..f8779651a9 100644 --- a/packages/experimental/cua-macos/src/guidance.ts +++ b/packages/experimental/cua-macos/src/guidance.ts @@ -6,7 +6,7 @@ export const PLATFORM_GUIDANCE = 'macOS: accessibility tree operations are avail const methodDetails = new Map([ ['mouseMove', 'Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured.'], - ['click', 'Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying.'], + ['click', 'click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying.'], ]) /** Complete model instructions for this provider. */ diff --git a/packages/experimental/cua-macos/tests/guidance.spec.ts b/packages/experimental/cua-macos/tests/guidance.spec.ts index d16958a36a..c4a6aeec7d 100644 --- a/packages/experimental/cua-macos/tests/guidance.spec.ts +++ b/packages/experimental/cua-macos/tests/guidance.spec.ts @@ -30,6 +30,8 @@ it('states the macOS accessibility, keyboard, and mouse levels', () => { const hover = GUIDANCE.split('\n').find(line => line.startsWith('- mouseMove(')) for (const term of ['AXShowAlternateUI', 'AXShowDefaultUI', 'HOVER_UNAVAILABLE', 'systemPointerHover']) expect(hover).toContain(term) const click = GUIDANCE.split('\n').find(line => line.startsWith('- click(')) + expect(click).toContain('await app.getWindow(windowId), then call win.click(target)') + expect(click).toContain('use app.invoke(id)') for (const step of ['call app.activate() first', 'outside the window', 'screenshot pixels', 'before typing or retrying']) expect(click).toContain(step) for (const platformDetail of ['macOS', 'AXShowAlternateUI', 'AXRaise', 'HOVER_UNAVAILABLE', 'systemPointerHover', 'mouse-down']) { expect(COMMON_GUIDANCE).not.toContain(platformDetail) diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index c9a19ee43f..a67bc45042 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -68,7 +68,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index 525f7cbd80..000bbe3322 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -70,7 +70,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index 84b37ec0bc..09031991fc 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -70,7 +70,7 @@ Element ids are opaque strings from the latest successful tree; the object facad Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. - mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. -- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. Prefer invoke(id) for a background control’s default action. For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. +- click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. From e07b170d7780d38ac01296e2e1ecfc6198fb6001 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Fri, 6 Nov 2026 18:17:04 +0800 Subject: [PATCH 35/44] fix(forge): address review of pinned diffs and thread counting - forge-github: repeat a compare read without --allow-escape-sequences when a gh older than 2.97 rejects the option; such a gh prints escape sequences unchanged, so every version reads every diff (checked with gh 2.96.0 and 2.100.0 on a pull request whose patch holds an ESC byte). - client-ui-forge: a stale refusal now re-runs the read effect, so a status that arrived during the read, including a failed poll that keeps the earlier head, starts the next read at once instead of leaving the changes view loading. - forge: ctx.forge.diff polls at once only for the first stale read at a head commit; later stale reads at it wait for the regular poll, which bounds the read-poll loop while GitHub's answers briefly disagree. - forge-github: review thread pages share the byte budget of a complete check read. - client-ui-forge: a truncated count shows its "+" even when it is zero. - READMEs: document the batch deadline and rewritten-base limitations. --- docs/subsystems/forge.i18n.yaml | 4 +- docs/subsystems/forge.md | 8 ++-- docs/subsystems/forge.zh.md | 8 ++-- .../client-ui-forge/README.i18n.yaml | 8 ++-- .../experimental/client-ui-forge/README.md | 4 +- .../experimental/client-ui-forge/README.zh.md | 4 +- .../src/client/ChangeRequestCard.tsx | 3 +- .../client-ui-forge/src/client/ChangesTab.tsx | 5 ++- .../tests/change-request-card.client.spec.tsx | 2 + .../tests/changes-tab.client.spec.tsx | 25 +++++++++++- .../forge-github/README.i18n.yaml | 8 ++-- packages/experimental/forge-github/README.md | 8 ++-- .../experimental/forge-github/README.zh.md | 8 ++-- .../experimental/forge-github/src/compare.ts | 20 +++++++--- .../experimental/forge-github/src/provider.ts | 38 +++++++++++-------- .../forge-github/tests/compare.spec.ts | 14 +++++++ .../forge-github/tests/provider.spec.ts | 24 ++++++++++++ packages/experimental/forge/README.i18n.yaml | 4 +- packages/experimental/forge/README.md | 2 +- packages/experimental/forge/README.zh.md | 2 +- packages/experimental/forge/src/index.ts | 16 ++++++-- .../experimental/forge/tests/watch.spec.ts | 22 +++++++++++ .../extensions/tool-cordis/src/api-catalog.ts | 2 +- 23 files changed, 178 insertions(+), 61 deletions(-) diff --git a/docs/subsystems/forge.i18n.yaml b/docs/subsystems/forge.i18n.yaml index f60794a506..d8c9c3ada9 100644 --- a/docs/subsystems/forge.i18n.yaml +++ b/docs/subsystems/forge.i18n.yaml @@ -18,5 +18,5 @@ en: 13dfd36f2166bba0 zh: a369434f52f04d57 /forges/panel: - en: 8182a3120b2d4aa3 - zh: 68b6631047cb0f48 + en: 1d3400c14260e62c + zh: 191e9bae2d753244 diff --git a/docs/subsystems/forge.md b/docs/subsystems/forge.md index 591e9d47f4..94de4749bf 100644 --- a/docs/subsystems/forge.md +++ b/docs/subsystems/forge.md @@ -575,7 +575,7 @@ type ForgeChangeRequestStatesFrame = | { readonly type: 'changed'; readonly states: readonly ForgeChangeRequestListState[] } ``` -The changes view reads through the same Remote. `localDiff` reads one `ForgeLocalScope` of the Session's checkout, and `changeRequestDiff` reads a bound change request at the head of its latest status through the provider's `diff`, which returns a patch only for that head commit; both return files that the Host parsed into `ForgeDiffFile` hunks. `localText` and `changeRequestText` read the new side of one file, which supplies the unchanged lines between hunks. Each inspection also sizes the `branch` scope as `ForgePanelState.local` for the panel's local changes card, always against the automatic fork point. A `branch` scope whose `base` names a ref compares with the merge base of HEAD and that branch instead; `diffBases` lists the branches the view offers, with the automatic fork point. When the head moved past the latest status, the provider returns `stale` and `ctx.forge.diff` polls the change request at once; `changeRequestDiff` rejects with `forgePanel/stale-head` and caches nothing, and the changes view keeps loading until a later status arrives, then reads again at that status's head. Review comments therefore always address the commit whose lines the view shows. +The changes view reads through the same Remote. `localDiff` reads one `ForgeLocalScope` of the Session's checkout, and `changeRequestDiff` reads a bound change request at the head of its latest status through the provider's `diff`, which returns a patch only for that head commit; both return files that the Host parsed into `ForgeDiffFile` hunks. `localText` and `changeRequestText` read the new side of one file, which supplies the unchanged lines between hunks. Each inspection also sizes the `branch` scope as `ForgePanelState.local` for the panel's local changes card, always against the automatic fork point. A `branch` scope whose `base` names a ref compares with the merge base of HEAD and that branch instead; `diffBases` lists the branches the view offers, with the automatic fork point. When the head moved past the latest status, the provider returns `stale` and `ctx.forge.diff` polls the change request at once, once per head commit; `changeRequestDiff` rejects with `forgePanel/stale-head` and caches nothing, and the changes view keeps loading until a later status arrives, then reads again at that status's head. Review comments therefore always address the commit whose lines the view shows. ```ts type-equiv /** Which local changes a diff covers; each reads the Session's checkout through Git. */ @@ -822,8 +822,10 @@ async fetchChecks(ref: ChangeRequestRef, signal: AbortSignal): Promise
diff --git a/packages/experimental/client-ui-forge/src/client/ChangesTab.tsx b/packages/experimental/client-ui-forge/src/client/ChangesTab.tsx index a65edc8570..7a66da2466 100644 --- a/packages/experimental/client-ui-forge/src/client/ChangesTab.tsx +++ b/packages/experimental/client-ui-forge/src/client/ChangesTab.tsx @@ -462,7 +462,8 @@ export function ChangesTab({ // Names the read the inputs ask for; it is in flight until `finished` matches it. const read = version === null ? null : `${dataKey}\n${navigation.revision}\n${attempt}\n${version}` const [finished, setFinished] = useState(null) - // A change request read refused because the head moved past its status stays in flight until a later status arrives. + // A change request read refused because the head moved past its status stays in flight until a status later than + // the one the read started with arrives; one that arrived during the read starts the next read at once. const statusAt = entry?.update?.fetchedAt ?? null const latestStatusAt = useRef(statusAt) useEffect(() => { latestStatusAt.current = statusAt }, [statusAt]) @@ -488,7 +489,7 @@ export function ChangesTab({ } }) return () => { controller.abort() } - }, [tab.visible, read, waiting, source, localScope, dataKey, localDiff, changeRequestDiff]) + }, [tab.visible, read, refused, waiting, source, localScope, dataKey, localDiff, changeRequestDiff]) const data = loaded !== null && loaded.key === dataKey ? loaded.data : null const files = useMemo(() => filesInTreeOrder(changedFileTree(data?.files ?? [])), [data]) // The centered loader marks a first read; the toolbar's glyph turns only while shown changes are read again. diff --git a/packages/experimental/client-ui-forge/tests/change-request-card.client.spec.tsx b/packages/experimental/client-ui-forge/tests/change-request-card.client.spec.tsx index e7305ff6d1..8e11e4c0d8 100644 --- a/packages/experimental/client-ui-forge/tests/change-request-card.client.spec.tsx +++ b/packages/experimental/client-ui-forge/tests/change-request-card.client.spec.tsx @@ -263,6 +263,8 @@ describe('expanded ChangeRequestCard', () => { 'Changes requested · 1 unresolved', 'error'], ['more threads than the provider counted', { reviewDecision: 'review-required', unresolvedThreads: 1200, threadsTruncated: true }, 'Review required · 1,200+ unresolved', null], + ['uncounted threads after resolved ones', { reviewDecision: 'approved', unresolvedThreads: 0, threadsTruncated: true }, + 'Approved · 0+ unresolved', 'success'], ] as const)('states the review of %s', (_label, overrides, text, tone) => { const value = rowValue(card(entry(1, snapshot(1, overrides)), { expanded: true }).item, en['change.row.review']) expect(value.textContent).toBe(text) diff --git a/packages/experimental/client-ui-forge/tests/changes-tab.client.spec.tsx b/packages/experimental/client-ui-forge/tests/changes-tab.client.spec.tsx index 1abfc93d1a..e6a2912325 100644 --- a/packages/experimental/client-ui-forge/tests/changes-tab.client.spec.tsx +++ b/packages/experimental/client-ui-forge/tests/changes-tab.client.spec.tsx @@ -20,7 +20,7 @@ import type { PanelSnapshot } from '../src/client/source.ts' import css from '../src/client/ChangesTab.module.css' import baseCss from '../src/client/BaseMenu.module.css' import type { - ForgeBaseBranch, ForgeBaseBranches, ForgeChangeRequestDiff, ForgeDiffFile, ForgeLocalDiff, ForgePanelState, + ForgeBaseBranch, ForgeBaseBranches, ForgeChangeRequestDiff, ForgeDiffFile, ForgeLocalDiff, ForgePanelChangeRequest, ForgePanelState, } from '../src/types.ts' import { entry, ready, snapshot } from './client-fixtures.client.ts' @@ -1119,6 +1119,29 @@ describe('ChangesTab change requests', () => { expect(m.paths()).toEqual(['docs/gone.txt', 'docs/notes.txt', 'src/app.ts']) }) + it.each([ + ['a later poll with the same head', (polled: ForgePanelChangeRequest) => polled], + ['a failed poll that keeps the earlier status', (polled: ForgePanelChangeRequest) => ({ + ...polled, update: { key: polled.update!.key, fetchedAt: 2, failure: { code: 'unavailable', message: 'down' } }, + } as const)], + ] as const)('reads again at once when %s arrived while a read the host refused as stale was in flight', async (_label, arrive) => { + const refused = Promise.withResolvers() + const m = mount({ source, setup: (faces) => { faces.changeRequestDiff.mockReturnValueOnce(refused.promise) } }) + await settle() + expect(m.faces.changeRequestDiff).toHaveBeenCalledOnce() + + const polled = entry(7, snapshot(7)) + const later = arrive({ ...polled, update: { ...polled.update!, fetchedAt: 2 } }) + act(() => { m.panel.set({ state: ready({ changeRequests: [later] }), disconnected: false }) }) + await settle() + expect(m.faces.changeRequestDiff).toHaveBeenCalledOnce() + refused.resolve(REFUSED.staleHead) + await settle() + + expect(m.faces.changeRequestDiff).toHaveBeenCalledTimes(2) + expect(m.paths()).toEqual(['docs/gone.txt', 'docs/notes.txt', 'src/app.ts']) + }) + it('keeps shown changes while a read waits for a later status, turning the refresh glyph', async () => { const m = mount({ source }) await settle() diff --git a/packages/experimental/forge-github/README.i18n.yaml b/packages/experimental/forge-github/README.i18n.yaml index 8859923642..655506d0dc 100644 --- a/packages/experimental/forge-github/README.i18n.yaml +++ b/packages/experimental/forge-github/README.i18n.yaml @@ -15,8 +15,8 @@ en: d152484eb41ac6b4 zh: 2f8a621b991260a1 /deepseek-ai-dsh-experimental-forge-github/use-this-package: - en: 4f8cd0d3b2df749c - zh: 8d398c088900a535 + en: 4ae3c0d7fd03b0d5 + zh: bb11975bc30b80a8 /deepseek-ai-dsh-experimental-forge-github/understand-the-implementation: en: f98e77171cba809f zh: 1b330cb342d69efe @@ -30,8 +30,8 @@ en: 26b946af4b742e17 zh: 00b34f0721ac72f6 /deepseek-ai-dsh-experimental-forge-github/known-limitations-and-deferred-work: - en: 3dba4fd8b8375b39 - zh: 79190c7cb7b68656 + en: 9f13ae6d928e4ef7 + zh: 09a549b1655f4de3 /deepseek-ai-dsh-experimental-forge-github/known-limitations-and-deferred-work/dev-note: en: e67f633dc6fe7773 zh: 3bb0c20c93f16f98 diff --git a/packages/experimental/forge-github/README.md b/packages/experimental/forge-github/README.md index 9b290e605a..37cb8dce5e 100644 --- a/packages/experimental/forge-github/README.md +++ b/packages/experimental/forge-github/README.md @@ -38,15 +38,15 @@ The provider runs `gh` as a host process, even when Sessions execute commands in The provider serves a remote URL in `https`, `http`, `ssh`, or `git` form, or in scp form such as `git@github.com:owner/name.git`, when its host is configured. HTTP(S) remotes retain custom ports in the host identity; SSH and Git transport ports are excluded. It serves pull request URLs of the form `https://///pull/` with any trailing path, query, or fragment, and builds that form without the suffix as the canonical URL. A readiness probe runs `gh auth status --hostname `: exit status 0 is `ready`, another exit status is `unauthenticated` with the first diagnostic line, and a `gh` that cannot start is `unavailable`. A head lookup returns up to 20 open pull requests whose head is the branch, newest first, excluding pull requests from forks; a lookup of the default branch returns none. -Polling returns the first 100 head-commit check contexts without following pages. An incomplete preview sets `checksTruncated` and keeps GitHub’s aggregate state; its counts do not describe the full list. `fetchChecks(ref, signal)` reads complete details only for the requested PR, pinning continuation pages to the first response’s rollup. The combined UTF-8 JSON responses must fit `maxOutputBytes`; excess returns `too-large`. Failed or non-advancing pages fail only this detail read and leave polling results intact. Runs with the same application, workflow, event, and check name keep only the newest run id; a queued replacement supersedes an older completed run. Runs without a numeric database id are retained by node id because their relative age is unknown. Commit status contexts keep the newest result per context name. Complete lists derive their indicator from these results; unknown context kinds retain GitHub’s aggregate state. +Polling returns the first 100 head-commit check contexts without following pages. An incomplete preview sets `checksTruncated` and keeps GitHub’s aggregate state; its counts do not describe the full list. `fetchChecks(ref, signal)` reads complete details only for the requested PR, pinning continuation pages to the first response’s rollup. The combined UTF-8 JSON responses, review thread pages included, must fit `maxOutputBytes`; excess returns `too-large`. Failed or non-advancing pages fail only this detail read and leave polling results intact. Runs with the same application, workflow, event, and check name keep only the newest run id; a queued replacement supersedes an older completed run. Runs without a numeric database id are retained by node id because their relative age is unknown. Commit status contexts keep the newest result per context name. Complete lists derive their indicator from these results; unknown context kinds retain GitHub’s aggregate state. -Every status read counts review threads in pages of up to 100 until it has counted `maxReviewThreads` (default 1,000) of them; each page after the first is one more GraphQL query. A pull request with more threads sets `threadsTruncated`, and its `unresolvedThreads` counts only the threads read. A thread page that fails or does not advance fails that pull request's status read and leaves the other pull requests of the batch intact. +Every status read counts review threads in pages of up to 100 until it has counted `maxReviewThreads` (default 1,000) of them; each page after the first is one more GraphQL query. A pull request with more threads sets `threadsTruncated`, and its `unresolvedThreads` counts only the threads read. A thread page that fails or does not advance fails that pull request's status read and leaves the other pull requests of the batch intact; a partial count is not published. Unfinished check runs and completed runs with stale, missing, or unrecognized conclusions map to `pending`. Other completed checks map to `success`, `failure` (failure, timed out, startup failure, or action required), `cancelled`, `skipped`, or `neutral`. Cancelled checks keep the rollup non-passing. Check runs keep their start and completion times. Commit status contexts map `SUCCESS` to `success`, `FAILURE` and `ERROR` to `failure`, and every other state to `pending`, and carry no times. A GraphQL `NOT_FOUND` error becomes a `not-found` failure. A response without GraphQL data or errors, including an HTTP error JSON body, uses the first stderr diagnostic or the body's message. Authentication diagnostics become `unauthenticated`; oversized check pages become `too-large`; rate limits and other failures become `unavailable`. Head lookups preserve the same diagnostic in their rejection. A pull request is `mergeReady` when it is open, not a draft, GitHub reports its merge state as `CLEAN`, `HAS_HOOKS`, or `UNSTABLE`, and the signed-in user has write, maintain, or admin permission. `mergeMethods` lists the squash, merge-commit, and rebase methods the repository allows, the user's default merge method first. Marking ready runs `gh pr ready --repo //`. Merging calls `PUT repos///pulls//merge` through `gh api` with the method and the expected head SHA, so GitHub refuses once new commits arrived, which is reported as `stale`. Unlike `gh pr merge`, it never enables auto-merge, never joins a merge queue, and never deletes the head branch. A refused action reports GitHub's message, and a diagnostic that mentions authentication becomes `unauthenticated`. -`diff` reads `headRefOid` and `baseRefOid` in one GraphQL query and returns `stale` when the head is not the requested commit, which must be a full 40- or 64-digit hexadecimal commit. Otherwise it reads `repos///compare/...` through `gh api --allow-escape-sequences` with the diff media type, the patch from the merge base of both commits, so neither end follows a moving branch. For every open and merged pull request measured, this patch is byte-identical to `gh pr diff`, which takes no commit argument. Unlike the pull request diff, the compare endpoint returns diffs over 300 files, so only output over `maxOutputBytes` or GitHub's own refusal makes such a diff `too-large`. `--allow-escape-sequences` keeps terminal escape sequences that file contents hold, which `gh` 2.97 and newer otherwise refuse to print. `fileText` reads `repos///contents/?ref=` through `gh api` with the raw media type and accepts only a full 40- or 64-digit hexadecimal commit. `fileUrl` returns `https://///blob//`. Both paths percent-encode each segment. A refused read becomes `not-found` for HTTP 404, `unauthenticated` for a diagnostic that mentions authentication, `too-large` for GitHub's HTTP 406 refusal or `too_large` error, which a diff or file over its size limit draws, and `unavailable` otherwise, with GitHub's message. A read whose output exceeds `maxOutputBytes` becomes `too-large`, and one that outlives `timeoutMs` becomes `unavailable`. +`diff` reads `headRefOid` and `baseRefOid` in one GraphQL query and returns `stale` when the head is not the requested commit, which must be a full 40- or 64-digit hexadecimal commit. Otherwise it reads `repos///compare/...` through `gh api --allow-escape-sequences` with the diff media type, the patch from the merge base of both commits, so neither end follows a moving branch. For every open and merged pull request measured, this patch is byte-identical to `gh pr diff`, which takes no commit argument. Unlike the pull request diff, the compare endpoint returns diffs over 300 files, so only output over `maxOutputBytes` or GitHub's own refusal makes such a diff `too-large`. `--allow-escape-sequences` keeps terminal escape sequences that file contents hold, which `gh` 2.97 and newer otherwise refuse to print; an older `gh` rejects the option with `unknown flag` before sending a request and prints such sequences unchanged, so that rejection repeats the read without it. `fileText` reads `repos///contents/?ref=` through `gh api` with the raw media type and accepts only a full 40- or 64-digit hexadecimal commit. `fileUrl` returns `https://///blob//`. Both paths percent-encode each segment. A refused read becomes `not-found` for HTTP 404, `unauthenticated` for a diagnostic that mentions authentication, `too-large` for GitHub's HTTP 406 refusal or `too_large` error, which a diff or file over its size limit draws, and `unavailable` otherwise, with GitHub's message. A read whose output exceeds `maxOutputBytes` becomes `too-large`, and one that outlives `timeoutMs` becomes `unavailable`. Review submission checks the current pull request head, then posts a single inline comment or one completed review through the REST API with an explicit `commit_id`. Multiline comments carry their side and inclusive start and end lines. Review decisions are Comment, Approve, and Request changes. JSON request bodies travel on stdin. The provider never retries a mutation: a timeout, unexpected response, or server error after dispatch returns an unknown outcome that the caller must reconcile on GitHub. A commit arriving between the check and the write can make the explicitly addressed review outdated; GitHub does not offer an atomic head precondition on these endpoints. @@ -97,6 +97,8 @@ None. Provider output reaches only host-side watchers. - **GitHub CLI on the host** — `gh` must be installed and signed in for each configured host; tokens in environment variables are not forwarded. Hosts missing from `hosts` are never read, including references retained across configuration changes. A status fetch returns `unavailable` for those references; a head lookup rejects before starting `gh`. - **Review thread bound** — a status read counts at most `maxReviewThreads` review threads per pull request; beyond it `unresolvedThreads` is a lower bound that `threadsTruncated` marks. +- **Batch deadline** — continuation pages of a pull request with many review threads lengthen its host's polling batch; a batch that outlives `ctx.forge`'s `fetchTimeoutMs` reports `unavailable` for every pull request in it, not only for the one with many threads. +- **Rewritten base branch** — the diff starts from the merge base of the head and the base commit GitHub recorded at the pull request's last update. After the base branch is force-pushed, that merge base can differ from the one GitHub's pull request view uses, so the patch can include or omit commits until the pull request is updated. - **No fork discovery** — head lookups exclude pull requests whose head is in another repository. - **Rate limits** — every poll costs one GraphQL query per host, plus one query per further page of 100 review threads, and a complete check read one more query per further page of 100 check records; every diff read costs one GraphQL query and one REST request. All count against the signed-in account’s rate limit. - **No merge queue** — a base branch that requires a merge queue refuses a direct merge, and the provider never queues one. diff --git a/packages/experimental/forge-github/README.zh.md b/packages/experimental/forge-github/README.zh.md index 291ef3d085..836781e92a 100644 --- a/packages/experimental/forge-github/README.zh.md +++ b/packages/experimental/forge-github/README.zh.md @@ -38,15 +38,15 @@ kind: "package-reference" 当主机在配置中时，提供方服务 `https`、`http`、`ssh`、`git` 形式的远端 URL，以及 `git@github.com:owner/name.git` 这类 scp 形式。HTTP(S) 远端在主机标识中保留自定义端口；SSH 和 Git 传输端口不计入主机标识。它服务 `https://///pull/` 形式的 pull request URL，允许其后附带任意路径、查询或片段，并以去掉这些后缀的形式作为规范 URL。就绪探测运行 `gh auth status --hostname `：退出码为 0 时为 `ready`，其他退出码为 `unauthenticated` 并带第一行诊断，`gh` 无法启动时为 `unavailable`。按 head 分支查找时，最多返回 20 个以该分支为 head 的打开状态 pull request，按创建时间从新到旧排列，不包含来自 fork 的 pull request；查找默认分支时返回空结果。 -轮询只返回 head 提交的前 100 条检查上下文，不追读后续页。预览不完整时设置 `checksTruncated` 并保留 GitHub 的汇总状态，局部计数不代表完整列表。`fetchChecks(ref, signal)` 仅按需读取指定 PR 的完整详情，后续页固定使用首次响应的 rollup。所有 JSON 响应的 UTF-8 字节总量受 `maxOutputBytes` 限制，超出时返回 `too-large`。分页失败或游标不前进只使此次详情读取失败，保留轮询结果。同一应用、工作流、触发事件与检查名称的运行只保留运行 id 最新的一条；新的排队运行会替代旧的已完成运行。缺少数字数据库 id 的运行无法确定先后，因此按节点 id 保留。提交状态上下文按名称保留最新结果。完整列表从这些结果计算总体指示；未知上下文类型保留 GitHub 的汇总状态。 +轮询只返回 head 提交的前 100 条检查上下文，不追读后续页。预览不完整时设置 `checksTruncated` 并保留 GitHub 的汇总状态，局部计数不代表完整列表。`fetchChecks(ref, signal)` 仅按需读取指定 PR 的完整详情，后续页固定使用首次响应的 rollup。所有 JSON 响应（包括评审讨论分页）的 UTF-8 字节总量受 `maxOutputBytes` 限制，超出时返回 `too-large`。分页失败或游标不前进只使此次详情读取失败，保留轮询结果。同一应用、工作流、触发事件与检查名称的运行只保留运行 id 最新的一条；新的排队运行会替代旧的已完成运行。缺少数字数据库 id 的运行无法确定先后，因此按节点 id 保留。提交状态上下文按名称保留最新结果。完整列表从这些结果计算总体指示；未知上下文类型保留 GitHub 的汇总状态。 -每次状态读取按每页最多 100 个统计评审讨论，直到已统计 `maxReviewThreads`（默认 1,000）个为止；第一页之后每读一页多一次 GraphQL 查询。讨论更多的 pull request 设置 `threadsTruncated`，其 `unresolvedThreads` 只统计已读取的讨论。讨论分页失败或游标不前进时，该 pull request 的状态读取失败，同批其他 pull request 不受影响。 +每次状态读取按每页最多 100 个统计评审讨论，直到已统计 `maxReviewThreads`（默认 1,000）个为止；第一页之后每读一页多一次 GraphQL 查询。讨论更多的 pull request 设置 `threadsTruncated`，其 `unresolvedThreads` 只统计已读取的讨论。讨论分页失败或游标不前进时，该 pull request 的状态读取失败，同批其他 pull request 不受影响；不发布不完整的计数。 未完成的检查运行（check run），以及结论已过期、缺失或无法识别的已完成运行，均映射为 `pending`。其他已完成的检查映射为 `success`、`failure`（失败、超时、启动失败或需要操作）、`cancelled`、`skipped` 或 `neutral`。已取消的检查使汇总保持未通过状态。检查运行保留其开始与完成时间。提交状态上下文把 `SUCCESS` 映射为 `success`，把 `FAILURE` 与 `ERROR` 映射为 `failure`，其余状态均映射为 `pending`，且不带时间。GraphQL 的 `NOT_FOUND` 错误转为 `not-found` 失败。响应不含 GraphQL data 或 errors 时（包括 HTTP 错误 JSON），使用 stderr 的首条诊断或响应体的 message。认证诊断转为 `unauthenticated`；超出输出大小上限的检查分页转为 `too-large`；限流与其他失败转为 `unavailable`。head 查找在拒绝时保留相同诊断。 pull request 处于打开状态、不是草稿、GitHub 报告其合并状态为 `CLEAN`、`HAS_HOOKS` 或 `UNSTABLE`，且已登录用户具有 write、maintain 或 admin 权限时，`mergeReady` 为真。`mergeMethods` 列出仓库允许的压缩合并、合并提交与变基合并方式，用户默认的合并方式排在最前。标记为可评审运行 `gh pr ready --repo //`。合并经 `gh api` 调用 `PUT repos///pulls//merge`，带上合并方式与预期的 head SHA，因此有新提交到达后 GitHub 会拒绝，报告为 `stale`。与 `gh pr merge` 不同，它从不启用自动合并、从不加入合并队列，也从不删除 head 分支。被拒绝的操作报告 GitHub 的原因，提到身份验证的诊断转为 `unauthenticated`。 -`diff` 在一次 GraphQL 查询中读取 `headRefOid` 与 `baseRefOid`，head 不是请求的提交时返回 `stale`；请求的提交必须是完整的 40 位或 64 位十六进制提交。否则经 `gh api --allow-escape-sequences` 以 diff 媒体类型读取 `repos///compare/...`，即两个提交的合并基准到 head 的补丁，两端都不跟随移动的分支。实测的所有 open 与已合并 pull request 上，该补丁与不接受提交参数的 `gh pr diff` 逐字节一致。与 pull request diff 不同，compare 接口会返回超过 300 个文件的 diff，因此这类 diff 只有在输出超过 `maxOutputBytes` 或 GitHub 自身拒绝时才为 `too-large`。`--allow-escape-sequences` 保留文件内容中的终端转义序列；否则 `gh` 2.97 及以上版本拒绝输出。`fileText` 经 `gh api` 以 raw 媒体类型读取 `repos///contents/?ref=`，只接受完整的 40 位或 64 位十六进制提交。`fileUrl` 返回 `https://///blob//`。两者的路径都逐段做百分号编码。被拒绝的读取在 HTTP 404 时为 `not-found`，诊断提到身份验证时为 `unauthenticated`，GitHub 以 HTTP 406 拒绝或报告 `too_large` 错误时（diff 或文件超过其大小上限即会如此）为 `too-large`，其余为 `unavailable`，并带有 GitHub 的原因。输出超过 `maxOutputBytes` 的读取为 `too-large`，超过 `timeoutMs` 的为 `unavailable`。 +`diff` 在一次 GraphQL 查询中读取 `headRefOid` 与 `baseRefOid`，head 不是请求的提交时返回 `stale`；请求的提交必须是完整的 40 位或 64 位十六进制提交。否则经 `gh api --allow-escape-sequences` 以 diff 媒体类型读取 `repos///compare/...`，即两个提交的合并基准到 head 的补丁，两端都不跟随移动的分支。实测的所有 open 与已合并 pull request 上，该补丁与不接受提交参数的 `gh pr diff` 逐字节一致。与 pull request diff 不同，compare 接口会返回超过 300 个文件的 diff，因此这类 diff 只有在输出超过 `maxOutputBytes` 或 GitHub 自身拒绝时才为 `too-large`。`--allow-escape-sequences` 保留文件内容中的终端转义序列；否则 `gh` 2.97 及以上版本拒绝输出。更早的 `gh` 在发出请求前就以 `unknown flag` 拒绝该选项，且原样输出这类序列，因此遇到这一拒绝时去掉该选项重读一次。`fileText` 经 `gh api` 以 raw 媒体类型读取 `repos///contents/?ref=`，只接受完整的 40 位或 64 位十六进制提交。`fileUrl` 返回 `https://///blob//`。两者的路径都逐段做百分号编码。被拒绝的读取在 HTTP 404 时为 `not-found`，诊断提到身份验证时为 `unauthenticated`，GitHub 以 HTTP 406 拒绝或报告 `too_large` 错误时（diff 或文件超过其大小上限即会如此）为 `too-large`，其余为 `unavailable`，并带有 GitHub 的原因。输出超过 `maxOutputBytes` 的读取为 `too-large`，超过 `timeoutMs` 的为 `unavailable`。 提交评审时先检查 PR 当前的 head，再通过 REST API 发布单条行内评论或一次完整评审，并显式指定 `commit_id`。多行评论携带所属侧及包含端点的起止行号。评审结论为“评论”“批准”或“请求更改”。JSON 请求正文通过标准输入传递。provider 不会重试写操作；发送后的超时、异常响应或服务端错误会返回结果不明，由调用方到 GitHub 核对。检查与写入之间到来的新提交可能让已明确定位到旧提交的评审过时；GitHub 的这些接口不提供原子的 head 前置条件。 @@ -97,6 +97,8 @@ pull request 处于打开状态、不是草稿、GitHub 报告其合并状态为 - **宿主需要 GitHub CLI**——必须安装 `gh` 并为每个已配置主机登录；环境变量中的令牌不会被转发。不在 `hosts` 中的主机永远不会被读取，包括配置变更后保留的引用。状态读取为这些引用返回 `unavailable`；head 查找会在启动 `gh` 前拒绝。 - **评审讨论上限**——每次状态读取每个 pull request 最多统计 `maxReviewThreads` 个评审讨论；超出时 `unresolvedThreads` 只是下限，由 `threadsTruncated` 标出。 +- **批次时限**——评审讨论很多的 pull request 的续页会拉长其主机的轮询批次；批次超过 `ctx.forge` 的 `fetchTimeoutMs` 时，批次内每个 pull request 都报告 `unavailable`，而不只是讨论很多的那个。 +- **基准分支被改写**——diff 从 head 与 GitHub 在 pull request 最后一次更新时记录的基准提交的合并基准开始。基准分支被强制推送后，该合并基准可能不同于 GitHub pull request 页面使用的合并基准，因此在 pull request 更新前，补丁可能多出或缺少提交。 - **不发现 fork**——按 head 分支查找时，不包含 head 位于其他仓库的 pull request。 - **速率限制**——每次轮询为每个主机消耗一次 GraphQL 查询，评审讨论每多一页（100 个）再消耗一次；完整检查读取时检查记录每多一页（100 条）再消耗一次；每次 diff 读取消耗一次 GraphQL 查询与一次 REST 请求。这些都计入已登录账号的速率配额。 - **不支持合并队列**——要求合并队列的基准分支会拒绝直接合并，提供方也不会把变更请求加入队列。 diff --git a/packages/experimental/forge-github/src/compare.ts b/packages/experimental/forge-github/src/compare.ts index f6c9ef4aec..3598b4f807 100644 --- a/packages/experimental/forge-github/src/compare.ts +++ b/packages/experimental/forge-github/src/compare.ts @@ -6,12 +6,20 @@ import { isCommit } from './urls.ts' /** Runs one reading `gh` command and maps a refusal to a failure, as the provider's reads do. */ export type GhTextReader = (args: readonly string[], signal: AbortSignal) => Promise +/** `gh` option that prints a response containing terminal escape sequences; `gh` 2.97 added it. */ +const ALLOW_ESCAPES = '--allow-escape-sequences' + +/** What a `gh` older than 2.97 reports for {@link ALLOW_ESCAPES}, before it sends any request. */ +const UNKNOWN_ALLOW_ESCAPES = `unknown flag: ${ALLOW_ESCAPES}` + /** * Read the unified patch from the merge base of `base` and `head` to `head`, as * GitHub computes the diff of a pull request with that base and head commit. * Both ends are commit object names, so the answer never follows a moving - * branch. The patch keeps terminal escape sequences that file contents hold; - * `gh` 2.97 and newer refuses to print them without `--allow-escape-sequences`. + * branch. The patch keeps terminal escape sequences that file contents hold: + * `gh` 2.97 and newer refuses to print them without `--allow-escape-sequences`, + * and an older `gh`, which rejects that option, prints them unchanged, so a + * rejection of the option alone repeats the read without it. * GitHub returns the patch however many files it changes; `read` bounds its size. * @param read - runs `gh` with the provider's output limit and failure mapping. * @param repository - repository that holds both commits, with an owner and name already checked for inlining. @@ -28,8 +36,8 @@ export async function readComparison( if (!isCommit(commit)) throw new Error(`forge-github: invalid commit ${JSON.stringify(commit)}`) } const { host, owner, name } = repository - return await read([ - 'api', '--hostname', host, '--allow-escape-sequences', '-H', 'Accept: application/vnd.github.diff', - `repos/${owner}/${name}/compare/${base}...${head}`, - ], signal) + const request = ['-H', 'Accept: application/vnd.github.diff', `repos/${owner}/${name}/compare/${base}...${head}`] + const patch = await read(['api', '--hostname', host, ALLOW_ESCAPES, ...request], signal) + if (typeof patch === 'string' || !patch.message.startsWith(UNKNOWN_ALLOW_ESCAPES)) return patch + return await read(['api', '--hostname', host, ...request], signal) } diff --git a/packages/experimental/forge-github/src/provider.ts b/packages/experimental/forge-github/src/provider.ts index f48f98a07b..1cd66bad30 100644 --- a/packages/experimental/forge-github/src/provider.ts +++ b/packages/experimental/forge-github/src/provider.ts @@ -12,7 +12,7 @@ import { CHECKS_QUERY, hasGraphqlFields, HEAD_QUERY, mapCheckPageResponse, mapHeadResponse, mapPullRequestCommitsResponse, mapSnapshotResponse, mapThreadPageResponse, PULL_REQUEST_COMMITS_QUERY, snapshotQuery, THREADS_QUERY, } from './graphql.ts' -import type { SnapshotPages } from './graphql.ts' +import type { PageResult, SnapshotPages } from './graphql.ts' import { checkedRepository, isCommit, parsePullRequestUrl, parseRemoteUrl } from './urls.ts' import { submitGitHubReview } from './review.ts' @@ -262,25 +262,31 @@ export class GitHubProvider implements ForgeProvider { const query = snapshotQuery(refs, this.limits.maxReviewThreads) const output = await this.gh(['api', 'graphql', '--hostname', host, '-f', `query=${query.document}`], signal) const body = parseJson(output.stdout) - const threads: SnapshotPages['threads'] = async (id, cursor, first) => { - const page = await this.continuation(host, - ['-f', `query=${THREADS_QUERY}`, '-f', `id=${id}`, '-f', `cursor=${cursor}`, '-F', `first=${first}`], signal) - if (!('stdout' in page)) return { ok: false, failure: page } - const value = parseJson(page.stdout) - return hasGraphqlFields(value) ? mapThreadPageResponse(value) : { ok: false, failure: responseFailure(page, value) } - } - if (!completeChecks && hasGraphqlFields(body)) return await mapSnapshotResponse(body, query, { threads }) + // A complete check read retains every response it reads, so all of them share one byte budget. let bytes = Buffer.byteLength(output.stdout) const tooLarge: ForgeFailure = { code: 'too-large', message: 'GitHub check details exceed the output size limit' } - if (bytes > this.limits.maxOutputBytes) return new Map([...query.targets.values()].map(ref => [changeRequestKey(ref), tooLarge])) - if (hasGraphqlFields(body)) return await mapSnapshotResponse(body, query, { threads, checks: async (id, cursor) => { - const page = await this.continuation(host, ['-f', `query=${CHECKS_QUERY}`, '-f', `id=${id}`, '-f', `cursor=${cursor}`], signal) - if (!('stdout' in page)) return { ok: false, failure: page } + const overBudget = (page: GhOutput): boolean => { + if (!completeChecks) return false bytes += Buffer.byteLength(page.stdout) - if (bytes > this.limits.maxOutputBytes) return { ok: false, failure: tooLarge } + return bytes > this.limits.maxOutputBytes + } + const continuation = async ( + variables: readonly string[], map: (value: unknown) => PageResult, + ): Promise> => { + const page = await this.continuation(host, variables, signal) + if (!('stdout' in page)) return { ok: false, failure: page } + if (overBudget(page)) return { ok: false, failure: tooLarge } const value = parseJson(page.stdout) - return hasGraphqlFields(value) ? mapCheckPageResponse(value) : { ok: false, failure: responseFailure(page, value) } - } }) + return hasGraphqlFields(value) ? map(value) : { ok: false, failure: responseFailure(page, value) } + } + const threads: SnapshotPages['threads'] = async (id, cursor, first) => await continuation( + ['-f', `query=${THREADS_QUERY}`, '-f', `id=${id}`, '-f', `cursor=${cursor}`, '-F', `first=${first}`], mapThreadPageResponse) + if (!completeChecks && hasGraphqlFields(body)) return await mapSnapshotResponse(body, query, { threads }) + if (bytes > this.limits.maxOutputBytes) return new Map([...query.targets.values()].map(ref => [changeRequestKey(ref), tooLarge])) + if (hasGraphqlFields(body)) { + return await mapSnapshotResponse(body, query, { threads, checks: async (id, cursor) => await continuation( + ['-f', `query=${CHECKS_QUERY}`, '-f', `id=${id}`, '-f', `cursor=${cursor}`], mapCheckPageResponse) }) + } const failure = responseFailure(output, body) return new Map([...query.targets.values()].map(ref => [changeRequestKey(ref), failure] as const)) } diff --git a/packages/experimental/forge-github/tests/compare.spec.ts b/packages/experimental/forge-github/tests/compare.spec.ts index 878de35501..271a898715 100644 --- a/packages/experimental/forge-github/tests/compare.spec.ts +++ b/packages/experimental/forge-github/tests/compare.spec.ts @@ -21,6 +21,20 @@ describe('compare read', () => { ]) }) + it('repeats a read that an older gh rejects for the escape option without that option, and no other failure', async () => { + const calls: (readonly string[])[] = [] + const answers: (string | ForgeFailure)[] = [ + { code: 'unavailable', message: 'unknown flag: --allow-escape-sequences' }, 'diff --git a/x b/x\n', + { code: 'unavailable', message: 'Server Error' }, + ] + const read = async (args: readonly string[]) => { calls.push(args); return answers.shift()! } + + await expect(readComparison(read, demo, base, head, signal)).resolves.toBe('diff --git a/x b/x\n') + expect(calls[1]).toEqual(['api', '--hostname', 'github.com', '-H', 'Accept: application/vnd.github.diff', `repos/octo/demo/compare/${base}...${head}`]) + await expect(readComparison(read, demo, base, head, signal)).resolves.toEqual({ code: 'unavailable', message: 'Server Error' }) + expect(calls).toHaveLength(3) + }) + it.each([['base', 'main', head], ['head', base, 'HEAD']])('refuses a %s that is not a full commit object name before reading', async (_label, from, to) => { const calls: (readonly string[])[] = [] await expect(readComparison(async (args) => { calls.push(args); return '' }, demo, from, to, signal)) diff --git a/packages/experimental/forge-github/tests/provider.spec.ts b/packages/experimental/forge-github/tests/provider.spec.ts index 4ffe4e411f..38fa865e7e 100644 --- a/packages/experimental/forge-github/tests/provider.spec.ts +++ b/packages/experimental/forge-github/tests/provider.spec.ts @@ -144,6 +144,20 @@ describe('GitHub provider fetch', () => { } }) + it('counts review thread pages toward the byte budget of a complete detail read, but not of polling', async () => { + const first = output({ data: { r0: { p0: pullRequestNode(1, { reviewThreads: threadPage([true], 'threads-1') }) } } }) + const page = output({ data: { node: { reviewThreads: threadPage([false, false]) } } }) + const total = Buffer.byteLength(first.stdout) + Buffer.byteLength(page.stdout) + for (const limit of [total, total - 1]) { + const gh = fakeGh(args => args.includes(`query=${THREADS_QUERY}`) ? page : first) + const subject = new GitHubProvider(gh.run, ['github.com'], { ...limits, maxOutputBytes: limit }) + const result = await subject.fetchChecks(pull(1), signal) + if (limit === total) expect(result).toMatchObject({ unresolvedThreads: 2, threadsTruncated: false }) + else expect(result).toEqual({ code: 'too-large', message: 'GitHub check details exceed the output size limit' }) + expect((await subject.fetch([pull(1)], signal)).get(changeRequestKey(pull(1)))).toMatchObject({ unresolvedThreads: 2 }) + } + }) + it('reads past 100 historical runs from the pinned rollup and counts only the latest run', async () => { const gh = fakeGh(args => output(args.includes('id=fixed-rollup') ? { data: { node: { contexts: { pageInfo: { hasNextPage: false, endCursor: null }, nodes: [ @@ -456,6 +470,16 @@ describe('GitHub provider reads', () => { expect(gh.calls).toHaveLength(1) }) + it('reads the diff with a gh older than 2.97, which rejects the escape option, by repeating the read without it', async () => { + // What gh 2.96.0 prints for an option it does not know. + const gh = pinned(undefined, args => args.includes('--allow-escape-sequences') + ? output('', 1, 'unknown flag: --allow-escape-sequences\n\nUsage: gh api [flags]\n') + : output('diff --git a/x b/x\n')) + + await expect(provider(gh.run).diff(pull(7), head, signal)).resolves.toBe('diff --git a/x b/x\n') + expect(gh.calls.map(call => call.includes('--allow-escape-sequences'))).toEqual([false, true, false]) + }) + it('refuses a head that is not a full commit object name before starting gh', async () => { const gh = pinned() diff --git a/packages/experimental/forge/README.i18n.yaml b/packages/experimental/forge/README.i18n.yaml index a60d0a4d78..7e6bbba230 100644 --- a/packages/experimental/forge/README.i18n.yaml +++ b/packages/experimental/forge/README.i18n.yaml @@ -15,8 +15,8 @@ en: d152484eb41ac6b4 zh: 2f8a621b991260a1 /deepseek-ai-dsh-experimental-forge/use-this-package: - en: 897de781e679f5f2 - zh: 3d0c589742a4dac0 + en: e51a2f965613ecaf + zh: 609baadb4bb090e4 /deepseek-ai-dsh-experimental-forge/understand-the-implementation: en: e193cb547c47069b zh: faa517522402ea41 diff --git a/packages/experimental/forge/README.md b/packages/experimental/forge/README.md index e04f46a256..368eab3c2f 100644 --- a/packages/experimental/forge/README.md +++ b/packages/experimental/forge/README.md @@ -36,7 +36,7 @@ Mount the registry with at least one provider package, such as the GitHub provid `markReady(ref, signal)` marks a draft ready for review, and `merge(ref, { method, headSha }, signal)` merges a change request only while its head is still `headSha`. Both delegate to the provider of the reference's forge, throw when none is registered, and resolve to undefined once the host accepted the change or to a `ForgeActionFailure`: `unauthenticated`, `stale` when the head moved past `headSha`, or `failed` with the host's reason. After either settles, a watched change request is polled at once. Snapshots carry `mergeReady`, whether the host would merge the change request now for the signed-in user, and `mergeMethods`, the methods the repository accepts with that user's usual one first. -`diff(ref, headSha, signal)` reads the change request's unified patch at head commit `headSha` as the host computes it, with `a/` and `b/` path prefixes, and `fileText(ref, commit, path, signal)` reads one file of its repository at a commit. Both delegate to the provider of the reference's forge, throw when none is registered, and resolve to the text or to a `ForgeFailure`, which is `too-large` when the host or the provider refuses an answer over its size limit. A provider returns a patch only when it describes `headSha`; once the head is another commit, `diff` resolves to `stale`, and a watched change request is polled at once so its next status names the current head. Snapshots carry `unresolvedThreads` with `threadsTruncated`, which marks the count as a lower bound when the provider counted only some review threads. `fileUrl(ref, commit, path)` returns the web page of that file at that commit. +`diff(ref, headSha, signal)` reads the change request's unified patch at head commit `headSha` as the host computes it, with `a/` and `b/` path prefixes, and `fileText(ref, commit, path, signal)` reads one file of its repository at a commit. Both delegate to the provider of the reference's forge, throw when none is registered, and resolve to the text or to a `ForgeFailure`, which is `too-large` when the host or the provider refuses an answer over its size limit. A provider returns a patch only when it describes `headSha`; once the head is another commit, `diff` resolves to `stale`. The first stale read at a head commit polls a watched change request at once so its next status names the current head; later stale reads at that commit wait for the regular poll, which bounds reads while the host's answers briefly disagree. Snapshots carry `unresolvedThreads` with `threadsTruncated`, which marks the count as a lower bound when the provider counted only some review threads. `fileUrl(ref, commit, path)` returns the web page of that file at that commit. `fetchChecks(ref, signal)` reads one change request’s complete checks on demand. It has its own `fetchTimeoutMs` deadline and does not occupy the polling batch or publish to its watchers. Failure leaves previously polled snapshots intact. A provider without `fetchChecks` returns `unavailable`. Caller cancellation and provider or service disposal abort the read; disposal waits for it to settle. diff --git a/packages/experimental/forge/README.zh.md b/packages/experimental/forge/README.zh.md index 9e61402f5b..b992ef1f4d 100644 --- a/packages/experimental/forge/README.zh.md +++ b/packages/experimental/forge/README.zh.md @@ -36,7 +36,7 @@ kind: "package-reference" `markReady(ref, signal)` 把草稿标记为可评审；`merge(ref, { method, headSha }, signal)` 只在 head 仍是 `headSha` 时合并变更请求。两者都委托给引用所属 forge 的提供方，没有注册提供方时抛错；主机接受时 resolve 为 undefined，否则为 `ForgeActionFailure`：`unauthenticated`、head 已越过 `headSha` 时的 `stale`，或带主机原因的 `failed`。两者结束后，被监听的变更请求会立即重新轮询。快照带有 `mergeReady`（主机现在是否会为已登录用户合并该变更请求）与 `mergeMethods`（仓库接受的合并方式，该用户常用的方式排在最前）。 -`diff(ref, headSha, signal)` 读取托管平台计算出的变更请求在 head 提交 `headSha` 处的统一补丁，路径带 `a/` 与 `b/` 前缀；`fileText(ref, commit, path, signal)` 读取其仓库中某个提交下的一个文件。两者都委托给引用所属 forge 的提供方，没有注册提供方时抛错，resolve 为文本或 `ForgeFailure`；托管平台或提供方因结果超过其大小上限而拒绝时，失败为 `too-large`。提供方只在补丁描述 `headSha` 时返回补丁；head 已是其他提交时，`diff` resolve 为 `stale`，被 watch 的变更请求随即重新轮询，使下一次状态给出当前 head。快照携带 `unresolvedThreads` 与 `threadsTruncated`；提供方只统计了部分评审讨论时，后者表示该计数只是下限。`fileUrl(ref, commit, path)` 返回该文件在该提交下的网页地址。 +`diff(ref, headSha, signal)` 读取托管平台计算出的变更请求在 head 提交 `headSha` 处的统一补丁，路径带 `a/` 与 `b/` 前缀；`fileText(ref, commit, path, signal)` 读取其仓库中某个提交下的一个文件。两者都委托给引用所属 forge 的提供方，没有注册提供方时抛错，resolve 为文本或 `ForgeFailure`；托管平台或提供方因结果超过其大小上限而拒绝时，失败为 `too-large`。提供方只在补丁描述 `headSha` 时返回补丁；head 已是其他提交时，`diff` resolve 为 `stale`。同一 head 提交的第一次 stale 读取会让被 watch 的变更请求立即重新轮询，使下一次状态给出当前 head；之后在该提交上的 stale 读取等待常规轮询，从而在托管平台的应答短暂不一致时限制读取次数。快照携带 `unresolvedThreads` 与 `threadsTruncated`；提供方只统计了部分评审讨论时，后者表示该计数只是下限。`fileUrl(ref, commit, path)` 返回该文件在该提交下的网页地址。 `fetchChecks(ref, signal)` 按需读取单个变更请求的完整检查，使用独立的 `fetchTimeoutMs` 时限，不占用轮询批次，也不向其监听方发布结果。读取失败时保留已有轮询快照。提供方未实现 `fetchChecks` 时返回 `unavailable`。调用方取消、提供方或服务销毁都会中止读取；销毁会等待读取结束。 diff --git a/packages/experimental/forge/src/index.ts b/packages/experimental/forge/src/index.ts index df05ef9796..24e403b460 100644 --- a/packages/experimental/forge/src/index.ts +++ b/packages/experimental/forge/src/index.ts @@ -44,6 +44,8 @@ interface Watch { /** Epoch milliseconds of the next poll; undefined while in flight or after a terminal state. */ dueAt: number | undefined failures: number + /** Head commit whose `stale` diff read last polled at once, so further stale reads at it wait for the regular poll. */ + staleHead: string | undefined } /** @@ -237,8 +239,10 @@ export class ForgeService extends Service { /** * Read the changes of a change request at one head commit as its host - * computes them. A `stale` failure polls the change request again at once - * when it is watched, so its next status names the current head. + * computes them. The first `stale` failure at a head commit polls a watched + * change request again at once, so its next status names the current head; + * later stale reads at the same commit wait for the regular poll, which + * bounds the reads while the host's answers disagree. * @param ref - change request served by a registered provider. * @param headSha - head commit the patch must describe, usually the head of the latest status. * @param signal - cancellation of the read. @@ -248,7 +252,11 @@ export class ForgeService extends Service { async diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise { const result = await this.invoke(this.registration(ref.repository.forge), signal, (provider, operation) => provider.diff(ref, headSha, operation)) - if (typeof result !== 'string' && result.code === 'stale') this.refresh(ref) + const watch = this.watches.get(changeRequestKey(ref)) + if (typeof result !== 'string' && result.code === 'stale' && watch !== undefined && watch.staleHead !== headSha) { + watch.staleHead = headSha + this.refresh(ref) + } return result } @@ -307,7 +315,7 @@ export class ForgeService extends Service { const dispose = this.ctx.effect(() => { let watch = this.watches.get(key) if (watch === undefined) { - watch = { key, ref, listeners: new Set(), latest: undefined, dueAt: Date.now(), failures: 0 } + watch = { key, ref, listeners: new Set(), latest: undefined, dueAt: Date.now(), failures: 0, staleHead: undefined } this.watches.set(key, watch) this.schedule() } diff --git a/packages/experimental/forge/tests/watch.spec.ts b/packages/experimental/forge/tests/watch.spec.ts index 2685e0d904..4537d82670 100644 --- a/packages/experimental/forge/tests/watch.spec.ts +++ b/packages/experimental/forge/tests/watch.spec.ts @@ -244,6 +244,28 @@ describe('forge watch', () => { expect(provider.fetches).toHaveLength(2) }) + it('polls at once only for the first stale diff read at a head, leaving later ones to the regular cadence', async () => { + const provider = new FakeProvider() + provider.results.set(key, snapshot(ref(1))) + const context = await setup(provider) + collect(context) + await vi.advanceTimersByTimeAsync(0) + const signal = new AbortController().signal + provider.readResult = { code: 'stale', message: 'the head moved' } + + await context.forge.diff(ref(1), 'a'.repeat(40), signal) + await vi.advanceTimersByTimeAsync(0) + await context.forge.diff(ref(1), 'a'.repeat(40), signal) + await vi.advanceTimersByTimeAsync(0) + expect(provider.fetches).toHaveLength(2) + await vi.advanceTimersByTimeAsync(config.idleIntervalMs) + expect(provider.fetches).toHaveLength(3) + + await context.forge.diff(ref(1), 'b'.repeat(40), signal) + await vi.advanceTimersByTimeAsync(0) + expect(provider.fetches).toHaveLength(4) + }) + it('coalesces refreshes during a fetch into one immediate subsequent fetch', async () => { const provider = new FakeProvider() const release = held(provider, target => snapshot(target)) diff --git a/packages/extensions/tool-cordis/src/api-catalog.ts b/packages/extensions/tool-cordis/src/api-catalog.ts index f5771a8141..3d3e0ff983 100644 --- a/packages/extensions/tool-cordis/src/api-catalog.ts +++ b/packages/extensions/tool-cordis/src/api-catalog.ts @@ -1247,7 +1247,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ }, { signature: 'async diff(ref: ChangeRequestRef, headSha: string, signal: AbortSignal): Promise', - description: 'Read the changes of a change request at one head commit as its host computes them. A `stale` failure polls the change request again at once when it is watched, so its next status names the current head.', + description: 'Read the changes of a change request at one head commit as its host computes them. The first `stale` failure at a head commit polls a watched change request again at once, so its next status names the current head; later stale reads at the same commit wait for the regular poll, which bounds the reads while the host\'s answers disagree.', parameters: [{ name: 'ref', description: 'change request served by a registered provider.' }, { name: 'headSha', description: 'head commit the patch must describe, usually the head of the latest status.' }, { name: 'signal', description: 'cancellation of the read.' }], returns: 'one unified patch with `a/` and `b/` path prefixes, or why the host did not provide it at `headSha`.', throws: ['when no provider serves the change request\'s forge, or with `signal.reason` when it aborts.'], From 0a228f0a6affd854133bd02a2c1cf5657cf13532 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 6 Nov 2026 14:53:11 +0800 Subject: [PATCH 36/44] feat(ui-chat): load earlier history with explicit consumption intent --- ...chat-history-consumption-details.i18n.yaml | 34 +++ ...-11-06-chat-history-consumption-details.md | 101 +++++++++ ...-06-chat-history-consumption-details.zh.md | 101 +++++++++ ...6-11-06-chat-history-consumption.i18n.yaml | 31 +++ .../2026-11-06-chat-history-consumption.md | 74 ++++++ .../2026-11-06-chat-history-consumption.zh.md | 74 ++++++ .../api/session-controller/README.i18n.yaml | 4 +- packages/api/session-controller/README.md | 2 +- packages/api/session-controller/README.zh.md | 2 +- .../src/client/contract/session.ts | 18 +- .../src/client/contract/snapshot.ts | 5 + .../src/client/sessions/session.ts | 151 ++++++++++--- .../tests/session.client.spec.ts | 212 +++++++++++++++++- packages/client/ui-chat/README.i18n.yaml | 4 +- packages/client/ui-chat/README.md | 4 +- packages/client/ui-chat/README.zh.md | 4 +- packages/client/ui-chat/src/client/apply.ts | 3 +- .../src/client/chat/ChatView.module.css | 17 ++ .../ui-chat/src/client/chat/ChatView.tsx | 31 ++- .../ui-chat/src/client/chat/flow-motion.ts | 3 + .../src/client/chat/use-chat-navigation.ts | 137 +++++++++-- .../src/client/chat/use-chat-scroll.ts | 54 +++-- .../src/client/chat/use-chat-viewport.ts | 209 +++++++++++++++-- .../ui-chat/src/client/contract/slots.ts | 3 + packages/client/ui-chat/src/client/locale.ts | 32 +++ .../tests/apply-inject.client.spec.tsx | 2 +- .../tests/chat-history-intent.client.spec.ts | 117 ++++++++++ .../ui-chat/tests/chat-scroll.client.spec.tsx | 18 +- .../ui-chat/tests/chat-view.client.spec.tsx | 173 ++++++++++---- .../tests/chat-viewport.client.spec.ts | 130 +++++++++++ .../conversation-registry.client.spec.ts | 3 +- .../tests/queue-dock.client.spec.tsx | 2 +- .../schedule-catalog-action.client.spec.tsx | 2 +- .../tests/ui-session.client.spec.ts | 2 +- .../ui-trajectory/tests/views.client.spec.tsx | 2 +- .../tests/plan-review-panel.client.spec.tsx | 2 +- .../user-questions-composer.client.spec.tsx | 2 +- .../src/client/api-catalog.ts | 4 +- .../src/client/slot-catalog.ts | 24 +- .../tests/quote-selection.client.spec.tsx | 2 +- .../client-runtime/src/fixtures.ts | 2 +- .../client-runtime/src/sessions.ts | 8 + 42 files changed, 1625 insertions(+), 180 deletions(-) create mode 100644 .agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml create mode 100644 .agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md create mode 100644 .agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md create mode 100644 .agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml create mode 100644 .agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md create mode 100644 .agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md create mode 100644 packages/client/ui-chat/tests/chat-history-intent.client.spec.ts diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml new file mode 100644 index 0000000000..4d1f5caf20 --- /dev/null +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml @@ -0,0 +1,34 @@ +# Bilingual-pair consistency record for 2026-11-06-chat-history-consumption-details.md (docs/i18n/README.md): per heading +# section, a hash of its English and Chinese blocks outside code blocks and generated regions. +# After editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md +/agent-note-chat-history-consumption-classes-and-interactions: + en: 69ce0866b2608208 + zh: 55fdeded5d7a4779 +/agent-note-chat-history-consumption-classes-and-interactions/problem: + en: edaa7ca0e55a9c33 + zh: 33d2b5ac07760988 +/agent-note-chat-history-consumption-classes-and-interactions/proposal: + en: 5e5553981c48d0e2 + zh: 1195ac8d64180451 +/agent-note-chat-history-consumption-classes-and-interactions/proposal/owners-and-interfaces: + en: 58f6f76fd263cafb + zh: 13ac0a6259d30cda +/agent-note-chat-history-consumption-classes-and-interactions/proposal/data-lifetimes: + en: ffdbe0e796463624 + zh: f10bd2ced5748017 +/agent-note-chat-history-consumption-classes-and-interactions/proposal/consumption-intent-and-wakeups: + en: 993c2985bf6f3c8f + zh: 8f88fb46584fb272 +/agent-note-chat-history-consumption-classes-and-interactions/proposal/shared-anchors-and-layout-reads: + en: 2439cd4cb6a81bc4 + zh: 63b8c83897a88d2c +/agent-note-chat-history-consumption-classes-and-interactions/alternatives-considered: + en: 1b9b8eb12ee5dbd7 + zh: e4ecc3cdc3375438 +/agent-note-chat-history-consumption-classes-and-interactions/acceptance-criteria: + en: 8a51c549570a5e0d + zh: eb7b3c27b9dcd98c +/agent-note-chat-history-consumption-classes-and-interactions/risks: + en: aee9f541d058818b + zh: 74e4b6098b9f4b46 diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md new file mode 100644 index 0000000000..ccc137d86b --- /dev/null +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md @@ -0,0 +1,101 @@ +# Agent Note: Chat history consumption classes and interactions + +Status: proposed + +English | [中文](2026-11-06-chat-history-consumption-details.zh.md) + +## Problem + +Session owns a history page's network lifetime, while Chat owns whether that page can safely enter the current view. Two-phase loading must reuse existing publication and scrolling capabilities without copying business data, consumption intent, and DOM preservation state into one owner. + +## Proposal + +[History prefetch and safe consumption](2026-11-06-chat-history-consumption.md) defines user-visible behavior and trade-offs. This note defines class interactions and lifetimes without adding an independent scroll animation or React commit manager. + +### Owners and interfaces + +| Owner | Data and operations | +|---|---| +| [Session](../../../../packages/api/session-controller/src/client/sessions/session.ts) | `loadOlder({ deferConsumption?: boolean })` prepares and optionally consumes a page; `consumeOlder()` synchronously publishes a valid staged page; `historyLoadStatus` is idle/loading/pendingConsumption/error, and `loadingOlder` derives from loading, including Turn jumps | +| Session request record | Holds generation, starting cursor, page entries, hasMore, Promise, and consumption/invalidation flags; staging does not advance the visible window cursor | +| [ChatNavigation](../../../../packages/client/ui-chat/src/client/chat/use-chat-navigation.ts) | Owns none/auto/manual consumption intent and request exclusion; waits or falls back by blocker; only manual consumption registers a fold-completion callback | +| [ChatViewport](../../../../packages/client/ui-chat/src/client/chat/use-chat-viewport.ts) | Reads internal input/motion state and scrollTop, returns a blocker or native/preserve/prepend, and performs distance measurement and existing anchor algorithms | +| [useChatScroll](../../../../packages/client/ui-chat/src/client/chat/use-chat-scroll.ts) | Connects existing reading, scroll, content, and size notifications to consumption checks and settles a pending one-shot correction during commit handling | +| [ChatView](../../../../packages/client/ui-chat/src/client/chat/ChatView.tsx) | Requests and consumes through injected callbacks and reads state through useSession; renders a fixed-height history control without retaining Session history data or accumulating historyWasPartial | + +```text +ChatNavigation.checkHistory + ├─ pendingConsumption + intent=none → retain page until clicked + ├─ pendingConsumption + intent=auto/manual → viewport.historyConsumption + │ ├─ touch/inner-scroll → retain consumption intent + │ ├─ motion/pointer/unavailable → auto falls back, manual waits + │ ├─ native → consumeOlder + │ ├─ prepend → capture paging position → consumeOlder → correct once after commit + │ └─ preserve → beginPreserving → consumeOlder + ├─ loading → no duplicate request + ├─ error → wait for explicit retry + └─ idle + hasMore + (manual intent or near top without following) → request a page + +Session.loadOlder + → prepareOlder reuses or creates the request + → await the captured request Promise + → consumeOlder when request identity remains valid and consumption is not deferred + +Session.prepareOlder + → network prepend → capture the page without publishing eventSource + → validate progress → pendingConsumption or error + +Session.consumeOlder + → check generation/cursor/consuming + → synchronous eventSource.prepend → existing Conversation merge and flush + → clear the request record and publish final status +``` + +### Data lifetimes + +`isOpen` means only that Session `openState === 'open'`; it is independent of whether a page is `pendingConsumption`. `historyWasPartial` is monotonic for one Client Session instance: once its visible window has `hasMore=true`, the fact remains until the instance ends, across component remounts and reconnects, without entering the persistent log. + +`prepareOlder()` returns an existing request or creates one, and `loadOlder()` rechecks identity after awaiting that captured request Promise. An obsolete Promise cannot consume a newer page. A default call consumes the prepared page; `deferConsumption` retains a valid nonempty page, while an empty terminal page can immediately confirm exhaustion. Consumption retains the request record and sets `consuming` to prevent synchronous subscriber reentry without borrowing the network loading state. + +An ordinary request captures the transport page first. It can be consumed only after the request succeeds and its cursor advances, or an empty page explicitly confirms exhaustion. Failures and empty pages that still claim more history enter error without automatic retry. Window replacement, reconnect, and disposal invalidate old pages; when an in-flight request may have advanced the transport cursor, the existing stream restart restores a baseline before another page is allowed. + +`loadThrough(seq)` is explicit navigation: it first adopts a staged page and then uses existing Turn-aligned paging. An ordinary in-flight request remains exclusive, and the caller continues after it settles. A jump publishes its successful prefix once at completion or a later failure, without delaying live appends. `loadingOlder` always equals `historyLoadStatus === 'loading'`. + +### Consumption intent and wakeups + +An automatic request records auto intent; a click records manual intent. Successful consumption, failure, exhaustion, or navigation cancellation clears the intent. A cached page with none intent is neither measured nor consumed by ordinary events; its button records manual intent without requesting it again. Motion, pointer, or unavailable positioning returns auto to none; touch and inner-scroll retain auto. Manual intent waits through these blockers rather than treating deferral as success. + +Prefetch reuses the existing 500ms reading sample, scrollend, content commits, and size notifications, coalescing them into one rAF check. It uses no IntersectionObserver or additional sample timer and measures nothing in the raw scroll event. Only a ResizeObserver entry for the scroller itself refreshes cached clientHeight during the callback's read phase; column growth does not reread viewport height. Once internal state permits an automatic request, it reads the history control's distance from the viewport and compares it with two cached viewport heights. + +`historyConsumption()` does not query DOM nodes or measure sizes. `FlowMotion.active()` exposes existing running, batchScheduled, batch, and settling state as a read-only query; passive listeners maintain input state. Size observation removes detached scrolling targets. Finger release, inner scroll completion, transition completion/cancellation, and existing notifications can wake retained intent; fold-completion callbacks are registered only for manual intent. Without consumption intent, these notifications do not revive an abandoned automatic page. + +### Shared anchors and layout reads + +At `scrollTop ≤ 0.5`, selection uses `beginPaging`'s first paging-anchor rule without a reading-position hit test. `measurePaging()` only captures a `PagingPosition`; `retainPaging()` separately owns persistent preservation and inner-size observation. Stationary preservation and one-shot correction share remount resolution in `resolvePaging()` and inner/outer calculations in `applyPaging()`. + +Persistent preservation restores the captured screen position. One-shot correction retains inner and outer scrolling since capture and compensates only layout displacement; it neither enters paging state nor subscribes to inner-size changes, and releases after one application. `finishHistoryPrepend()` returns the actual landing, and commit handling immediately saves it through `reading.preservePosition()` rather than waiting for a later scroll event. The actual conversation scrollport uses `overscroll-behavior-y: none`, without creating smooth scrolling or a new momentum animation. + +Distance confirmation still needs current geometry, and one-shot correction after prepend still needs the new position. These reads serve correctness, but placement inside rAF, ResizeObserver, or commit handling alone does not prove they add no layout. Read frequency, read/write ordering, and actual Layout/Paint costs require separate verification; position stability is not performance evidence. + +## Alternatives considered + +**Pause publication globally in Conversation.** That also blocks live tail output and expands the cross-view impact. + +**Retain a complete window snapshot in Chat.** That copies business data and can overwrite messages received while waiting. + +**Store separate coordinates and continuously track the first row for one-shot correction.** That duplicates existing paging anchors and can expand scroll ownership; share the position record and calculations while keeping lifetimes separate. + +## Acceptance criteria + +- Default single-stage calls, deferred consumption, and Turn jumps coexist; an old request cannot consume or overwrite a new one. +- Ordinary events do not revive automatic fallback; touch/inner-scroll continuation, manual completion, and unmount cleanup have focused verification. +- One-shot correction does not register persistent observation or undo intervening reader movement; existing persistent and inner preservation do not regress. +- Distance checks use cached viewport height; raw scroll listeners and consumption admission perform no layout reads. + +## Risks + +**Known issue: `preserveMovement` cannot distinguish native anchoring from reader input.** The shared correction adds changes in the outer and inner scroll positions since capture. A deterministic geometry model confirms duplicate compensation when the browser has already anchored the same prepend: an outer `scrollTop` of `0.5`, or outer `0` with inner `40`, enters the one-shot path and treats the native adjustment as movement to preserve. The resulting additional displacement is an algorithmic limitation, not evidence that every browser exhibits it. Code remains unchanged for this report-only finding; independent actual-browser verification is pending. + +Explicit navigation and multiple views share a Session, so Chat's automatic waiting policy must not move into the global event source. Manual intent can still wait if usable positioning never becomes available; actual views need cancellation, reconnect, and anchorless-structure verification. One-shot geometry measurement, native inner anchoring, and asynchronous content growth need browser evidence from the current build. + +Reports must identify source/artifact versions, real or synthetic inputs, missing conditions, and cleanup results. V8 function or range hits are not TypeScript line/branch percentages; earlier iteration recordings, Chrome/Safari manual feedback, or a successful build do not establish a complete matrix pass for this proposal. diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md new file mode 100644 index 0000000000..104b65e5a6 --- /dev/null +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md @@ -0,0 +1,101 @@ +# Agent Note: Chat 历史消费的类与交互 + +Status: proposed + +[English](2026-11-06-chat-history-consumption-details.md) | 中文 + +## Problem + +历史页的网络生命周期属于 Session，能否安全进入当前页面属于 Chat。两阶段加载需要复用现有事件发布与滚动能力，不能把业务数据、消费意图和 DOM 保位状态复制到同一个所有者。 + +## Proposal + +用户可见行为与取舍由[历史预取与安全消费](2026-11-06-chat-history-consumption.zh.md)定义。本文规定类之间的调用与生命周期，不增加独立的滚动动画或 React 提交管理。 + +### 所有者与接口 + +| 所有者 | 数据与操作 | +|---|---| +| [Session](../../../../packages/api/session-controller/src/client/sessions/session.ts) | `loadOlder({ deferConsumption?: boolean })` 准备并按选项消费一页；`consumeOlder()` 同步发布有效暂存页；`historyLoadStatus` 表达 idle/loading/pendingConsumption/error，`loadingOlder` 从 loading 派生并包含 Turn 跳转 | +| Session 请求记录 | 保存 generation、请求前游标、页内容、hasMore、Promise 与消费/失效标记；缓存页不提前改变可见窗口游标 | +| [ChatNavigation](../../../../packages/client/ui-chat/src/client/chat/use-chat-navigation.ts) | 保存 none/auto/manual 消费意图和请求互斥；按阻塞原因等待或退回按钮；仅手动消费注册折叠结束通知 | +| [ChatViewport](../../../../packages/client/ui-chat/src/client/chat/use-chat-viewport.ts) | 读取内部输入/动画状态与 scrollTop，报告阻塞原因或 native/preserve/prepend；执行距离测量和已有锚点算法 | +| [useChatScroll](../../../../packages/client/ui-chat/src/client/chat/use-chat-scroll.ts) | 把既有阅读、滚动、内容及尺寸通知接到消费检查；在提交阶段结算待执行的一次补偿 | +| [ChatView](../../../../packages/client/ui-chat/src/client/chat/ChatView.tsx) | 通过注入回调请求/消费，通过 useSession 读取状态；固定高度显示历史控件，不保存 Session 历史数据或累计 historyWasPartial | + +```text +ChatNavigation.checkHistory + ├─ pendingConsumption + intent=none → retain page until clicked + ├─ pendingConsumption + intent=auto/manual → viewport.historyConsumption + │ ├─ touch/inner-scroll → retain consumption intent + │ ├─ motion/pointer/unavailable → auto falls back, manual waits + │ ├─ native → consumeOlder + │ ├─ prepend → capture paging position → consumeOlder → correct once after commit + │ └─ preserve → beginPreserving → consumeOlder + ├─ loading → no duplicate request + ├─ error → wait for explicit retry + └─ idle + hasMore + (manual intent or near top without following) → request a page + +Session.loadOlder + → prepareOlder reuses or creates the request + → await the captured request Promise + → consumeOlder when request identity remains valid and consumption is not deferred + +Session.prepareOlder + → network prepend → capture the page without publishing eventSource + → validate progress → pendingConsumption or error + +Session.consumeOlder + → check generation/cursor/consuming + → synchronous eventSource.prepend → existing Conversation merge and flush + → clear the request record and publish final status +``` + +### 数据生命周期 + +`isOpen` 只表达会话 `openState === 'open'`，与某页是否 `pendingConsumption` 无关。`historyWasPartial` 是客户端 Session 实例的单调事实：可见窗口曾有 `hasMore=true` 后保留到实例结束，不随组件重挂载或重连清除，不写持久化日志。 + +`prepareOlder()` 返回已有请求或创建一个请求，`loadOlder()` 等待捕获的请求 Promise 后再核对身份。旧 Promise 完成不能消费随后创建的新页。默认调用消费准备好的页；`deferConsumption` 保留有效非空页，终态空页可以直接确认耗尽。消费期间保持请求记录并设置 `consuming`，阻止同步订阅者重入，不借用网络 loading 状态。 + +普通请求先捕获传输返回页，只有请求成功且游标前进，或空页明确表示耗尽时才能消费。失败或空页仍声称有更多则进入 error，不自动重试。窗口替换、重连和销毁使旧页失效；如果在途请求可能已推进传输游标，现有流重启先恢复基线，再允许下一页。 + +`loadThrough(seq)` 是显式导航：先采用已有待消费页，再沿既有 Turn 对齐规则加载；普通请求在途时保持互斥，由调用方在其结算后继续。跳转的成功前缀在完成或后续失败时一次发布，实时追加不等待它。`loadingOlder` 始终等价于 `historyLoadStatus === 'loading'`。 + +### 消费意图与唤醒 + +自动请求开始时登记 auto，点击登记 manual。消费成功、失败、耗尽或取消导航时清除意图。缓存存在但意图为 none 时，普通事件不测量或消费该缓存；按钮重新登记 manual，不重复取页。motion、pointer 或 unavailable 使 auto 退回 none；touch 与 inner-scroll 保留 auto。manual 在这些阻塞下继续等待，不把暂缓当成成功。 + +历史预取复用已有 500ms 阅读采样、scrollend、内容提交和尺寸通知，并合并为一次 rAF 检查。不使用 IntersectionObserver，不新增采样计时器，不在原始 scroll 事件中测量。仅 scroller 自身的 ResizeObserver 条目在回调读阶段更新 clientHeight 缓存，column 增长不重新读取视口高度。内部状态允许自动请求时，读取历史控件相对视口的位置，与缓存的两屏高度比较。 + +`historyConsumption()` 不查询 DOM 或测量尺寸。`FlowMotion.active()` 从已有 running、batchScheduled、batch、settling 状态提供只读查询；输入状态来自被动事件监听。滚动目标失效的清理在尺寸观察处进行。手指释放、内层滚动结束、过渡结束/取消和既有通知可以唤醒保留的意图；折叠结束回调只为 manual 注册。没有消费意图时这些通知不恢复已经放弃的自动页。 + +### 锚点复用与布局读取 + +顶部 `scrollTop ≤ 0.5` 使用 `beginPaging` 的首条分页锚点规则，不先执行阅读位置命中检测。`measurePaging()` 只采集 `PagingPosition`；`retainPaging()` 单独拥有持续保位和内层尺寸观察。普通静止保位与一次补偿共用 `resolvePaging()` 的重挂载查找及 `applyPaging()` 的内外层计算。 + +持续保位恢复采集时的屏幕位置。一次补偿保留采集后发生的内外层滚动，只抵消布局位移；它不进入 paging 状态、不注册内层尺寸观察，应用一次后释放。`finishHistoryPrepend()` 返回实际落点，提交处理立即通过 `reading.preservePosition()` 保存阅读位置，不依赖后续 scroll 事件。会话实际滚动容器设置 `overscroll-behavior-y: none`，不创建平滑滚动或新的惯性动画。 + +距离检查仍需要当前几何；前插后的一次补偿仍需要读取新位置。这些读数有正确性用途，但不能仅凭位于 rAF、ResizeObserver 或提交回调中，就认定不会增加布局。读取频率、读写顺序及实际 Layout/Paint 成本分别验收；位置不漂移不能替代性能证明。 + +## Alternatives considered + +**在 Conversation 全局暂停发布。** 会同时阻止尾部实时输出，扩大跨视图影响。 + +**在 Chat 保存完整窗口快照。** 复制业务数据，消费时可能覆盖等待期间的新消息。 + +**为一次补偿另存坐标并持续追踪首行。** 与既有分页锚点重复，并可能扩大滚动控制；共享位置记录和计算，只区分生命周期。 + +## Acceptance criteria + +- 默认单阶段调用、延迟消费及 Turn 跳转共存，旧请求不能消费或覆盖新请求。 +- auto 回退不会被普通事件恢复；触摸与内层滚动续接、manual 完成承诺和卸载清理都有定向验证。 +- 一次补偿不注册持续观察、不撤销期间的读者滚动；既有持续保位和内层补偿不退化。 +- 距离读取使用缓存视口高度；原始滚动监听与消费准入不读取布局。 + +## Risks + +**已知问题：`preserveMovement` 无法区分原生锚定与读者输入。** 共享补偿计算会加上采集后内外层滚动位置的变化。确定性的几何模型已确认：浏览器已为同一次前插完成锚定时，外层 `scrollTop` 为 `0.5`，或外层为 `0`、内层为 `40`，会进入一次补偿路径，把原生调整当成需要保留的移动，从而重复补偿。产生的额外位移是算法限制，不代表所有浏览器都会复现。本问题按仅报告处理，代码不变；独立真实浏览器验证仍待完成。 + +显式导航与多个视图共享 Session，不能把 Chat 的自动等待条件放进全局事件源。手动操作若永远得不到可用定位条件仍可能等待；实际页面应验证取消、重连和无锚点结构。一次补偿的布局测量、内层浏览器锚定及异步内容增长需要当前构建的浏览器证据。 + +验收报告应列明源码/产物版本、真实或合成输入、未覆盖条件以及清理结果。V8 函数或区间命中不等于 TypeScript 行/分支百分比；早期迭代的录像、Chrome/Safari 手工体验和单个构建通过不能证明本提案的完整矩阵已通过。 diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml new file mode 100644 index 0000000000..85c5633b69 --- /dev/null +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml @@ -0,0 +1,31 @@ +# Bilingual-pair consistency record for 2026-11-06-chat-history-consumption.md (docs/i18n/README.md): per heading +# section, a hash of its English and Chinese blocks outside code blocks and generated regions. +# After editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md +/agent-note-chat-history-prefetch-and-safe-consumption: + en: b0bb85f8780edc41 + zh: f3576c234a2f1a99 +/agent-note-chat-history-prefetch-and-safe-consumption/problem: + en: fbdf714be9ea6c90 + zh: d9b6c07347812fc1 +/agent-note-chat-history-prefetch-and-safe-consumption/proposal: + en: bda6c4b3dfff143e + zh: 4316f6dec6adaa50 +/agent-note-chat-history-prefetch-and-safe-consumption/proposal/manual-and-automatic-loading: + en: 2cbfd8cc05037895 + zh: 024968506e2ac994 +/agent-note-chat-history-prefetch-and-safe-consumption/proposal/position-and-status-presentation: + en: 1afb940b9b2a2fcc + zh: 8201614a80598e5b +/agent-note-chat-history-prefetch-and-safe-consumption/proposal/scope-and-retained-decisions: + en: dd4eeadd2fccab4b + zh: e05d53e63cc44ddf +/agent-note-chat-history-prefetch-and-safe-consumption/alternatives-considered: + en: 7c6d9d470c5b5c93 + zh: dc12846ee0f56aae +/agent-note-chat-history-prefetch-and-safe-consumption/acceptance-criteria: + en: b3005052df98ab10 + zh: c4bdf2bed8d23550 +/agent-note-chat-history-prefetch-and-safe-consumption/risks: + en: 2016c51019518572 + zh: e31e4e4ac09a2c5d diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md new file mode 100644 index 0000000000..bfe932f77b --- /dev/null +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md @@ -0,0 +1,74 @@ +# Agent Note: Chat history prefetch and safe consumption + +Status: proposed + +English | [中文](2026-11-06-chat-history-consumption.zh.md) + +## Problem + +With only a Load earlier button, readers must reach the beginning of loaded history before clicking. Prefetch can reduce interruption, but a request that starts in a suitable state can return during touch dragging, a navigation change, or a fold animation. History prepending must not change the existing fold driver or simulate momentum. + +## Proposal + +Separate requesting history from consuming it. The Client Session retains at most one unconsumed page, and Chat decides whether and when to publish it. It retains the new page rather than an old window snapshot, so waiting does not block streaming at the tail. Class ownership and calls are described in the [technical details](2026-11-06-chat-history-consumption-details.md). + +### Manual and automatic loading + +A manual click retains one loading intent: consume an existing page, join an in-flight request, or request a page. A temporary blocker does not complete the click; the operation ends with effective consumption, confirmed exhaustion, or an explicit failure. A newer Turn jump, return to bottom, view closure, or other navigation or lifecycle change can cancel the earlier reading intent. + +Automatic loading requests history after the reader leaves tail following and comes within two viewport heights of the history control. It reuses the existing 500ms reading sample, scroll completion, content commits, and size notifications, without another polling loop or IntersectionObserver. Detection can therefore wait until the next existing sample. + +A cached page and an intent to continue automatically are separate facts. Automatic loading falls back to a button on occasional conflicts instead of waiting for every state to recover. Touch and actual inner scrolling retain continuation so mobile readers need not click after every finger release. + +| State when data is ready | Automatic intent | Manual intent | +|---|---|---| +| Ordinary outer momentum, with other conditions satisfied | Consume with native browser anchoring | Consume with native browser anchoring | +| Stationary reading away from the top | Consume with existing reading preservation | Consume with existing reading preservation | +| `scrollTop ≤ 0.5`, with other conditions satisfied | Reuse the paging anchor and correct once after prepend | Reuse the paging anchor and correct once after prepend | +| Folding, revealing, or an existing follow animation | Keep the page and return to the button | Retain intent until completion | +| A finger held down or actual inner scrolling | Retain intent and continue afterward | Retain intent and continue afterward | +| A held mouse pointer, scrollbar dragging, or temporarily unavailable positioning | Keep the page and return to the button | Retain intent until usable | +| Tail following resumes after automatic prefetch | Keep the page and cancel automatic consumption | The click has paused tail following | +| Streaming continues at the tail | Does not itself block consumption | Does not itself block consumption | + +After fallback, scrolling, animation completion, and content notifications cannot reactivate that cached page; only a click or explicit navigation can adopt it. Requests and unconsumed pages remain mutually exclusive: no duplicate request for a cached page and no timeout that forces insertion. + +### Position and status presentation + +The actual conversation scrollport uses `overscroll-behavior-y: none` to suppress vertical boundary bounce and scroll chaining, without implementing a different momentum animation. A prepend at the top reuses the existing paging-position record and correction calculation, applies it once, and releases it; persistent preservation elsewhere retains its existing lifetime. Reader movement during the correction interval is preserved rather than restoring the position from request time. + +The history control keeps the same height across states. A network request shows Loading earlier; an active consumption intent without a held input shows Processing; automatic fallback or a held input shows Load earlier; failure shows Retry loading; consuming the final page shows Start of conversation. The control is excluded from native anchoring. Session `historyWasPartial` records that this instance has displayed incomplete history and survives component remounts and reconnects; a short Session that opens complete has no history row. + +### Scope and retained decisions + +Do not add `flushSync` or change Conversation publication cadence, fold-batch targets, or ScrollFollow policy. Consumption admission reads only internal state and `scrollTop`; DOM geometry reads belong to distance confirmation and position correction. Do not add an independent persistent head-anchor controller, `batchShift`, or a per-frame compensator. + +[One fold motion](../../implemented/architecture/2026-10-13-chat-single-fold-motion.md) and [group-tree exchanges](../../implemented/architecture/2026-10-11-chat-group-tree-exchanges.md) retain ownership of folding and grouping decisions; this proposal does not supersede them. + +## Alternatives considered + +**Prepend every response immediately and compensate continuously.** That makes pagination take over existing motion; this design corrects once at the top and waits or falls back on conflicts. + +**Automatically wait for every unsafe state to recover.** That adds coordination obligations for occasional animations, navigation, and automatic continuation; retain touch continuation and manual completion, but return other automatic conflicts to the button. + +**Remove staging entirely or discard conflicting pages.** Conflicts can start during a request, and the transport history cursor may already have advanced; retaining one page is simpler than rolling back that cursor. + +**Disable loading whenever a Turn is running.** Readers still need history during long replies, so execution state cannot replace the current presentation conditions. + +**Separate IO prefetch wakeups and React synchronous-commit control.** Reuse existing sampling and publication, accepting sample latency; do not add commit scheduling without evidence of a concrete failure. + +## Acceptance criteria + +- Requests, staging, and consumption never duplicate a page request; tail streaming continues. +- Ordinary events do not revive automatic fallback; touch, inner scrolling, and manual clicks retain their specified continuation. +- Stationary and top-edge prepends preserve old content position; ordinary momentum consumption adds no outer JS scroll write. +- Failure, no progress, exhaustion, reconnect, and explicit navigation have defined outcomes without request loops or stale-page consumption. +- Control states have equal height; existing fold-driver behavior is retained, and browser evidence identifies the current build, simulated inputs, and missing cases. + +## Risks + +**Known issue: duplicate anchoring compensation.** A deterministic geometry-model reproduction shows that one-shot correction can count native anchoring for the same page as reader movement and compensate twice: this occurs with an outer `scrollTop` of `0.5`, or with outer `0` and inner `40` when native anchoring has already moved the inner position. The algorithm is therefore not universally safe. This issue is recorded without a code fix; independent verification in actual browsers is pending. The mechanism is detailed in the [technical risks](2026-11-06-chat-history-consumption-details.md#risks). + +CSS feature support alone does not establish that native anchoring can act. Automatic prefetch waits for the first size observation, and the 500ms sample can let fast scrolling reach the top first. The synchronous geometry read after insertion is a cost of one-shot correction; its call site alone does not prove that it adds no layout. A stationary manual operation that can never obtain an anchor retains its completion intent and needs coverage for that structure. + +Audit and measurement are ongoing, so this record remains proposed. Earlier Chrome/Safari manual feedback and headless results from different iterations do not establish a complete matrix pass for the final source; actual scrollbar dragging, same-document reconnect, cross-platform input, and exceptional cleanup paths require separately attributed evidence. diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md new file mode 100644 index 0000000000..31787a81b7 --- /dev/null +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md @@ -0,0 +1,74 @@ +# Agent Note: Chat 历史预取与安全消费 + +Status: proposed + +[English](2026-11-06-chat-history-consumption.md) | 中文 + +## Problem + +只提供“加载更早”按钮时，读者必须到达已加载历史的开头再点击。提前请求可以减少中断，但请求发出时适合加载，不代表返回时适合前插：读者可能开始触摸拖动、切换导航目标，或者遇到折叠动画。分页不应为了前插历史改变已有折叠驱动或模拟惯性。 + +## Proposal + +将历史请求与消费分开。客户端 Session 最多保留一页未消费的数据，Chat 决定是否以及何时发布它。数据保存为新页而不是旧窗口快照，所以等待不阻止尾部流式输出。类与调用关系见[技术细节](2026-11-06-chat-history-consumption-details.zh.md)。 + +### 手动与自动加载 + +手动点击保留一次加载意图：已有缓存就消费缓存，有在途请求就等待该请求，否则请求一页。暂时不安全不算点击完成；该操作以有效消费、确认到头或明确失败结束。显式 Turn 跳转、返回底部、会话关闭等新的导航或生命周期变化可以取消原阅读意图。 + +自动加载在读者离开底部跟随、进入距历史控件两屏以内时请求。它复用已有 500ms 阅读采样及滚动结束、内容提交、尺寸通知，不新增轮询或 IntersectionObserver。代价是发现进入预取范围可能延后至下一次已有采样。 + +缓存是否存在与是否需要自动续接是两个事实。自动操作在低频冲突下退回按钮，不承诺等所有状态恢复；触摸与实际内层滚动仍保留续接，避免要求手机用户每次松手后再点击。 + +| 数据就绪时的状态 | 自动意图 | 手动意图 | +|---|---|---| +| 普通外层惯性，其他条件允许 | 浏览器原生锚定，消费 | 浏览器原生锚定，消费 | +| 非顶部静止阅读 | 既有阅读保位，消费 | 既有阅读保位，消费 | +| `scrollTop ≤ 0.5`，其他条件允许 | 复用分页锚点，前插后补偿一次 | 复用分页锚点，前插后补偿一次 | +| 折叠、展开或已有跟随动画 | 保留缓存，退回按钮 | 保留意图，等待结束 | +| 手指按住或实际内层滚动 | 保留意图，结束后继续 | 保留意图，结束后继续 | +| 鼠标按住、滚动条拖动或定位能力暂不可用 | 保留缓存，退回按钮 | 保留意图，等待可用 | +| 自动预取后恢复底部跟随 | 保留缓存，取消自动消费 | 点击已暂停底部跟随 | +| 底部继续流式输出 | 本身不阻止消费 | 本身不阻止消费 | + +退回按钮后，滚动、动画结束和内容通知都不能重新激活该缓存页；只有点击或显式导航可以采用它。请求和待消费页仍然互斥，不为缓存页重复请求，也不超时强行插入。 + +### 位置与状态展示 + +会话实际滚动容器设置 `overscroll-behavior-y: none`，关闭纵向边界回弹和滚动传递，不实现另一套惯性动画。顶部前插复用既有分页位置记录和补偿计算，只应用一次并释放；非顶部持续保位保持原生命周期。补偿期间发生的用户滚动保留，不把读者拉回请求发出时的位置。 + +历史控件各状态保持同高。网络请求显示“正在加载”；仍有消费意图且未按住输入时显示“正在处理”；自动回退或输入按住时显示“加载更早”；失败显示“重新加载”；最后一页消费完成后显示“已到会话开头”。控件不参与原生锚定。Session 的 `historyWasPartial` 记录本实例曾显示不完整历史，组件重挂载和重连保留此事实；首次打开即完整的短会话不添加历史行。 + +### 范围与既有决策 + +不新增 `flushSync`，不改变 Conversation 发布节奏、折叠批次目标或 ScrollFollow 的跟随策略。消费准入只读内部状态和 `scrollTop`；DOM 几何读取留在距离确认和位置补偿处。不新增独立的持续开头锚点控制器、`batchShift` 或每帧补偿器。 + +[单一折叠运动](../../implemented/architecture/2026-10-13-chat-single-fold-motion.zh.md)与[分组树交互段](../../implemented/architecture/2026-10-11-chat-group-tree-exchanges.zh.md)仍拥有折叠和分组决策，本提案不取代它们。 + +## Alternatives considered + +**任意状态下数据到达即前插，再持续补偿。** 需要分页接管已有运动；本方案只在顶部补偿一次，冲突时等待或退回按钮。 + +**所有不安全状态都自动等待恢复。** 增加低频动画、导航与自动续接的协调义务；保留触摸续接和手动完成，其他自动冲突退回按钮。 + +**完全取消暂存或直接丢弃冲突页。** 请求途中仍可能发生冲突，而且传输历史游标可能已经前移；保留一页比回滚游标更直接。 + +**运行中的 Turn 禁止加载。** 长回复期间仍需阅读历史，运行状态不能代替当前界面的消费条件。 + +**独立 IO 预取唤醒与 React 同步提交控制。** 复用已有采样和发布路径，接受采样延迟；没有具体失败证据时不增加提交调度机制。 + +## Acceptance criteria + +- 请求、暂存与消费期间不重复取页；尾部流式继续更新。 +- 自动退回按钮后不会被普通事件恢复；触摸、内层滚动和手动点击保留约定的续接。 +- 静止和顶部前插保持旧内容位置；普通惯性消费不新增 JS 外层滚动写入。 +- 页面失败、无进展、到头、重连和显式导航均有明确结果，不循环请求或消费过期页。 +- 控件各状态同高；既有折叠驱动保持行为，真实浏览器验收区分当前构建、输入模拟和未覆盖情况。 + +## Risks + +**已知问题：锚定重复补偿。** 确定性的几何模型复现表明，一次补偿可能把同一页已经发生的原生锚定当作读者移动，再补偿一次：外层 `scrollTop` 为 `0.5`，或外层为 `0`、内层为 `40` 且原生锚定已移动内层位置时会出现。该算法因此不能视为普遍安全。本问题仅记录，不修改代码；独立的真实浏览器验证仍待完成。机制见[技术风险](2026-11-06-chat-history-consumption-details.zh.md#risks)。 + +原生锚定适用性不能只由 CSS 支持检查保证。首次尺寸观察前不会自动预取；500ms 采样可能使高速滚动先到顶部。前插后的同步几何读取是一次补偿的成本，不能仅凭调用位置保证没有额外布局。静止手动消费若始终无法获得锚点，完成意图仍会等待，需要覆盖该特殊结构。 + +审计与测量尚在进行，本记录保持 proposed。此前 Chrome/Safari 手工反馈和不同迭代的 headless 结果不等同于最终源码的全矩阵通过；真实滚动条拖动、同页面重连、多平台输入和异常清理分支应按实际证据单独报告。 diff --git a/packages/api/session-controller/README.i18n.yaml b/packages/api/session-controller/README.i18n.yaml index 3d31867b67..94482498be 100644 --- a/packages/api/session-controller/README.i18n.yaml +++ b/packages/api/session-controller/README.i18n.yaml @@ -15,8 +15,8 @@ en: 1fdf707322fb6996 zh: 0204f7e9d4aa3cce /session-controller/use-this-package: - en: 9cac36e3c0a32959 - zh: 37c42cc3edafa798 + en: 50fa3dd0a2ce79c8 + zh: 88bd7ea6058d2122 /session-controller/client-references: en: e8979a803ec3e194 zh: c567efcd971728ba diff --git a/packages/api/session-controller/README.md b/packages/api/session-controller/README.md index b430594b0e..f414ce78ef 100644 --- a/packages/api/session-controller/README.md +++ b/packages/api/session-controller/README.md @@ -70,7 +70,7 @@ Fork copies the exact inclusive event prefix selected by `atSeq`, including a cu A resume blocked by an existing write handle returns `session/writer-held` with the Session id; other resume failures retain `gateway/internal`. -`loadThrough(seq)` retains older pages privately until its shared target is covered or loading ends, then publishes the successful pages as one ordered prepend. Live events remain visible while history loads. A later page failure retains the successful prefix; replacing the history window discards buffered pages from the replaced window. Ordinary `loadOlder()` publishes one Host-selected page directly. +`loadThrough(seq)` retains older pages privately until its shared target is covered or loading ends, then publishes the successful pages as one ordered prepend. Live events remain visible while history loads. A later page failure retains the successful prefix; replacing the history window discards buffered pages from the replaced window. Ordinary `loadOlder()` prepares and consumes one Host-selected page; `{ deferConsumption: true }` retains that page until `consumeOlder()` publishes it synchronously. A retained page blocks another ordinary request without blocking live events. `historyLoadStatus` distinguishes `idle`, `loading`, `pendingConsumption`, and `error`; `loadingOlder` is derived from `loading`, including Turn jumps. Failures and pages without progress permit explicit retry rather than claiming exhaustion. `historyWasPartial` remembers an incomplete visible window for the Client Session lifetime, including reconnects, without persisting presentation state. The Client's initial `follow`, reconnect openings, and `loadOlder()` request at least 50 append-origin `user/message` and `assistant/message` events and cross at least two `turn/start` events, including the partial Turn at the loaded window's beginning. The Host stops at the first Turn start satisfying both minima, or at 500 counted messages or history exhaustion, whichever comes first. Steering does not add a Turn boundary. Intervening events accompany the page without counting toward the message budget. The optional `turnWindow` on page and follow requests supplies these minima under `maxMessages`. `loadThrough()` uses the same rules with a 200-message minimum per page; the 500-message cap does not limit the whole jump. Requests without `turnWindow` retain message-aligned paging. diff --git a/packages/api/session-controller/README.zh.md b/packages/api/session-controller/README.zh.md index 314ec58840..2a83f4504d 100644 --- a/packages/api/session-controller/README.zh.md +++ b/packages/api/session-controller/README.zh.md @@ -70,7 +70,7 @@ Fork 将 `atSeq` 选定的含边界事件前缀（包括开放轮次内的截点 恢复会话时若已有写句柄占用，返回 `session/writer-held`，并携带会话 id；其他恢复失败仍返回 `gateway/internal`。 -`loadThrough(seq)` 在共享目标被覆盖或加载结束前私下保留较早页面，随后把成功取得的页面按顺序作为一次前插发布。历史加载期间实时事件仍然可见。后续页面失败时保留已成功取得的部分；历史窗口被替换时丢弃被替换窗口的暂存页面。普通 `loadOlder()` 直接发布 Host 选取的一页结果。 +`loadThrough(seq)` 在共享目标被覆盖或加载结束前私下保留较早页面，随后把成功取得的页面按顺序作为一次前插发布。历史加载期间实时事件仍然可见。后续页面失败时保留已成功取得的部分；历史窗口被替换时丢弃被替换窗口的暂存页面。普通 `loadOlder()` 准备并消费 Host 选取的一页；传入 `{ deferConsumption: true }` 时保留这一页，直到 `consumeOlder()` 同步发布。暂存页阻止重复普通请求，但不阻止实时事件。`historyLoadStatus` 区分 `idle`、`loading`、`pendingConsumption` 和 `error`；`loadingOlder` 从 `loading` 派生，包含 Turn 跳转。失败和无进展页面允许明确重试，不伪装成历史耗尽。`historyWasPartial` 在客户端 Session 生命周期内记住可见窗口曾不完整，重连保留，但不持久化呈现状态。 Client 的首次 `follow`、重连首屏与 `loadOlder()` 至少请求 50 条以 append 方式追加的 `user/message` 和 `assistant/message` 事件，并向前跨过至少两个 `turn/start`，其中包含已加载窗口开头尚未补齐的轮次。Host 在首次同时满足两个下限的轮次开头停止；若先达到 500 条计数消息或历史已耗尽，则立即返回。Steering 不增加轮次分界。区间内的其他事件随页返回，但不计入消息预算。分页和 follow 请求可选的 `turnWindow` 在 `maxMessages` 上限内指定这两个下限。`loadThrough()` 复用相同规则，仅将每页消息数下限设为 200；500 条上限不限制整次跳转。不带 `turnWindow` 的请求保留按消息对齐的分页方式。 diff --git a/packages/api/session-controller/src/client/contract/session.ts b/packages/api/session-controller/src/client/contract/session.ts index 9233ac84cb..ce060ec4aa 100644 --- a/packages/api/session-controller/src/client/contract/session.ts +++ b/packages/api/session-controller/src/client/contract/session.ts @@ -4,8 +4,8 @@ * ObservableSnapshot half), and orchestration code calls the behavior verbs * below — nothing else. Widening this interface is the explicit act of * widening what features may do to a session (and what every test fixture - * must stub); implementation-internal entry points (history staging, wire-frame - * dispatch) stay on the class, invisible out here. + * must stub); implementation-internal wire-frame dispatch stays on the class, + * invisible out here. */ import type { FileAttachmentRef, ImageAttachmentRef } from '@deepseek-ai/dsh-attachment' import type { MessageId } from '@deepseek-ai/dsh-llm/brand' @@ -132,10 +132,18 @@ export interface ISession { /** * Extend history by at least 50 messages and two Turn starts, including a * partial Turn at the window's beginning. Stop at 500 messages or history - * exhaustion even when those minima cannot be met. Publish one prepend. - * @returns completion; failures land in snapshot.openState/loadingOlder. + * exhaustion even when those minima cannot be met. At most one page may be + * loading or awaiting consumption; live events remain independently visible. + * @param options - deferConsumption keeps the page unpublished until consumeOlder; omitted consumes before completion. + * @returns request completion; failures and no progress set snapshot.historyLoadStatus to error. */ - loadOlder(): Promise + loadOlder(options?: { deferConsumption?: boolean }): Promise + /** + * Publish the prepared page synchronously if its generation and starting cursor still match. + * A terminal empty page may publish hasMore=false; an invalidated page is discarded. + * @returns whether a prepared page was published; false while loading, absent, or invalidated. + */ + consumeOlder(): boolean /** * Page history backwards until the window covers `seq` (inclusive) — the * turn-jump loader. Repeated calls while a jump is paging lower its shared diff --git a/packages/api/session-controller/src/client/contract/snapshot.ts b/packages/api/session-controller/src/client/contract/snapshot.ts index 4912d19b16..b6ba71b907 100644 --- a/packages/api/session-controller/src/client/contract/snapshot.ts +++ b/packages/api/session-controller/src/client/contract/snapshot.ts @@ -78,7 +78,12 @@ export interface SessionSnapshot { readonly openState: OpenState readonly openError: RemoteFailure | null readonly hasMore: boolean + /** This Client Session has exposed an incomplete history window; remains true across replacements. */ + readonly historyWasPartial: boolean + /** Whether historyLoadStatus is loading, including Turn-jump requests. */ readonly loadingOlder: boolean + /** History request status; pendingConsumption retains an unpublished ordinary page. */ + readonly historyLoadStatus: 'idle' | 'loading' | 'pendingConsumption' | 'error' readonly promptError: PromptError | null readonly blank: boolean readonly lastAgentError: string | null diff --git a/packages/api/session-controller/src/client/sessions/session.ts b/packages/api/session-controller/src/client/sessions/session.ts index e74dd84422..60194de5eb 100644 --- a/packages/api/session-controller/src/client/sessions/session.ts +++ b/packages/api/session-controller/src/client/sessions/session.ts @@ -67,6 +67,16 @@ interface PendingHistory { readonly pages: (readonly SessionEventLikeEntry[])[] } +interface OlderHistoryRequest { + readonly generation: number + readonly beforeSeq: SessionLogOffset + page: { readonly entries: readonly SessionEventLikeEntry[]; readonly hasMore: boolean } | null + invalidated: boolean + recovering: boolean + consuming: boolean + promise?: Promise +} + /** Manager-owned observers of a Session object's local state edges. */ export interface SessionOptions { /** Catalog-discovered address selecting non-activating subagent transport. */ @@ -95,13 +105,16 @@ export class Session implements SessionFace { // ---- Window and derived state (all private; the snapshot is the only read API) ---- private baseSeq = SessionLogOffset(0) private hasMore = false + private historyWasPartial = false private openState: OpenState = 'cold' private openError: RemoteFailure | null = null private openPromise: Promise | null = null /** Bumped by stream replacement to invalidate an in-flight doOpen. Stale * passes drop all writes once the generation moves on. */ private openGeneration = 0 - private loadingOlder = false + private get loadingOlder(): boolean { return this.historyLoadStatus === 'loading' } + private historyLoadStatus: SessionSnapshot['historyLoadStatus'] = 'idle' + private olderRequest: OlderHistoryRequest | null = null /** Shared low-water target of the running jump loop; null when no jump is paging. */ private jumpTargetSeq: SessionSeq | null = null /** The running jump loop's completion, shared by retargeting callers. */ @@ -415,30 +428,95 @@ export class Session implements SessionFace { return promise } - /** Prepend one Turn-aligned page: at least 50 messages and two Turn starts, capped at 500 messages. */ - async loadOlder(): Promise { - if (this.openState !== 'open' || !this.hasMore || this.loadingOlder) return + /** @inheritdoc */ + async loadOlder(options?: { deferConsumption?: boolean }): Promise { + const request = this.prepareOlder() + await request?.promise + if (request !== null && this.olderRequest === request + && (!options?.deferConsumption || request.page?.entries.length === 0)) this.consumeOlder() + } + + private prepareOlder(): OlderHistoryRequest | null { + if (this.olderRequest !== null) return this.olderRequest const events = this.events - if (events === undefined) return - this.loadingOlder = true + if (this.openState !== 'open' || !this.hasMore || this.loadingOlder || events === undefined) return null + const request: OlderHistoryRequest = { + generation: this.openGeneration, + beforeSeq: this.baseSeq, + page: null, + invalidated: false, + recovering: false, + consuming: false, + } + this.olderRequest = request + this.historyLoadStatus = 'loading' this.notifier.markDirty() - try { - await events.prepend({ - beforeSeq: this.baseSeq, - ...HISTORY_PAGE_OPTIONS, - }) - } catch (error) { - if (!isRemoteFailure(error)) { - console.error('[session-controller] loadOlder failed:', error) + request.promise = (async () => { + try { + await events.prepend({ beforeSeq: request.beforeSeq, ...HISTORY_PAGE_OPTIONS }) + if (this.olderRequest !== request) return + if (request.invalidated || request.generation !== this.openGeneration || request.beforeSeq !== this.baseSeq) return + const page = request.page + const first = page?.entries[0] + this.historyLoadStatus = page === null || (first === undefined ? page.hasMore : first.event.seq >= request.beforeSeq) + ? 'error' : 'pendingConsumption' + } catch (error) { + if (this.olderRequest !== request) return + request.page = null + this.historyLoadStatus = 'error' + if (!isRemoteFailure(error)) console.error('[session-controller] loadOlder failed:', error) + } finally { + if (this.olderRequest === request) { + if (request.invalidated && this.events === events) { + // A replacement restores the transport cursor before another page uses the visible window. + request.recovering = true + this.historyLoadStatus = 'loading' + events.restart() + } else if (this.historyLoadStatus === 'error') this.olderRequest = null + this.notifier.markDirty() + } } + })() + return request + } + + /** @inheritdoc */ + consumeOlder(): boolean { + const request = this.olderRequest + if (request === null || request.consuming || this.historyLoadStatus !== 'pendingConsumption' || request.page === null) return false + if (this.openState !== 'open' || request.invalidated || request.generation !== this.openGeneration + || request.beforeSeq !== this.baseSeq) { + this.clearOlderHistory() + return false + } + const page = request.page + // Retain the request through synchronous feed notification so subscribers cannot reenter consumption. + request.consuming = true + try { + this.prependWindow(page.entries, page.hasMore) + return true + } catch (error) { + if (this.olderRequest === request) this.historyLoadStatus = 'error' + throw error } finally { - this.loadingOlder = false + if (this.olderRequest === request) { + this.olderRequest = null + if (this.historyLoadStatus !== 'error') this.historyLoadStatus = 'idle' + } this.notifier.markDirty() } } + private clearOlderHistory(): void { + this.olderRequest = null + this.historyLoadStatus = this.pendingHistory === null ? 'idle' : 'loading' + this.notifier.markDirty() + } + /** Jump loader: page backwards until the window covers seq (see ISession.loadThrough). */ loadThrough(seq: SessionSeq): Promise { + this.consumeOlder() + if (this.olderRequest !== null) return Promise.resolve() if (this.openState !== 'open' || !this.hasMore || this.baseSeq <= seq) return Promise.resolve() if (this.jumpPromise !== null) { // Retarget the running loop to the lowest requested seq. @@ -459,7 +537,7 @@ export class Session implements SessionFace { } this.pendingHistory = pending this.jumpTargetSeq = seq - this.loadingOlder = true + this.historyLoadStatus = 'loading' this.notifier.markDirty() // Stale-pass guard (the doOpen pattern): a resync mid-loop replaces the // stream generation; this pass then stops instead of paging the new @@ -467,27 +545,35 @@ export class Session implements SessionFace { const generation = this.openGeneration this.jumpPromise = (async () => { try { - while (pending.hasMore && this.jumpTargetSeq !== null && pending.beforeSeq > this.jumpTargetSeq) { + while (this.jumpTargetSeq !== null && pending.beforeSeq > this.jumpTargetSeq) { if (generation !== this.openGeneration) return const before = pending.beforeSeq await events.prepend({ beforeSeq: before, ...JUMP_PAGE_OPTIONS }) + if (generation !== this.openGeneration || this.pendingHistory !== pending) return // No-progress guard: an empty or dropped page that still claims more // history must end the loop, not spin it. - if (pending.beforeSeq >= before) return + if (pending.beforeSeq >= before) { + if (pending.hasMore) this.historyLoadStatus = 'error' + return + } + if (!pending.hasMore) return } } catch (error) { + if (generation === this.openGeneration && this.pendingHistory === pending) this.historyLoadStatus = 'error' if (!isRemoteFailure(error)) { console.error('[session-controller] loadThrough failed:', error) } } finally { - this.jumpTargetSeq = null - this.jumpPromise = null - this.pendingHistory = null - this.loadingOlder = false - if (generation === this.openGeneration && pending.pages.length > 0) { - this.prependWindow(pending.pages.reverse().flat(), pending.hasMore) + if (this.pendingHistory === pending) { + this.jumpTargetSeq = null + this.jumpPromise = null + this.pendingHistory = null + if (this.historyLoadStatus === 'loading') this.historyLoadStatus = 'idle' + if (generation === this.openGeneration && pending.pages.length > 0) { + this.prependWindow(pending.pages.reverse().flat(), pending.hasMore) + } + this.notifier.markDirty() } - this.notifier.markDirty() } })() return this.jumpPromise @@ -499,6 +585,7 @@ export class Session implements SessionFace { async resync(): Promise { if (this.openState === 'cold') return // never opened: no window to rebuild (doOpen flips to 'loading' synchronously, so cold implies no in-flight open) this.openGeneration++ + this.clearOlderHistory() const events = this.events this.events = undefined await events?.dispose() @@ -606,6 +693,7 @@ export class Session implements SessionFace { else this.retireFailedSubmission(requestId) } this.openGeneration++ + this.clearOlderHistory() const events = this.events this.events = undefined await events?.dispose() @@ -655,7 +743,9 @@ export class Session implements SessionFace { ) return case 'prepend': - this.prependWindow(change.entries, change.hasMore) + if (this.olderRequest !== null) { + this.olderRequest.page = { entries: change.entries, hasMore: change.hasMore } + } else this.prependWindow(change.entries, change.hasMore) return case 'append': this.publishAssistantEntry(this.assistantStream.acceptDurable(change.entry)) @@ -676,8 +766,13 @@ export class Session implements SessionFace { // attempts makes a held notification reopen follow once for an atomic // page/baseline pair instead of applying it to an unrelated repair cut. const visible = this.assistantStream.replace(entries, assistantStream) + if (this.olderRequest !== null && this.loadingOlder && !this.olderRequest.recovering) { + this.olderRequest.invalidated = true + this.olderRequest.page = null + } else this.clearOlderHistory() this.baseSeq = SessionLogOffset(entries[0]?.event.seq ?? 0) this.hasMore = hasMore + this.historyWasPartial ||= hasMore if (this.pendingHistory !== null) { this.pendingHistory.beforeSeq = this.baseSeq this.pendingHistory.hasMore = hasMore @@ -745,6 +840,7 @@ export class Session implements SessionFace { } this.baseSeq = entries[0] === undefined ? this.baseSeq : SessionLogOffset(entries[0].event.seq) this.hasMore = hasMore + this.historyWasPartial ||= hasMore this.eventSource.prepend(entries, hasMore) } @@ -892,6 +988,7 @@ export class Session implements SessionFace { if (generation !== this.openGeneration || this.events !== events) return if (!isRemoteFailure(error)) throw error this.openGeneration++ + this.clearOlderHistory() this.events = undefined this.openPromise = null this.openState = 'error' @@ -917,7 +1014,9 @@ export class Session implements SessionFace { openState: this.openState, openError: this.openError, hasMore: this.hasMore, + historyWasPartial: this.historyWasPartial, loadingOlder: this.loadingOlder, + historyLoadStatus: this.historyLoadStatus, promptError: this.promptError, blank: this.blankBit, lastAgentError: this.lastAgentError, diff --git a/packages/api/session-controller/tests/session.client.spec.ts b/packages/api/session-controller/tests/session.client.spec.ts index e118ef29e5..6c6d308fe9 100644 --- a/packages/api/session-controller/tests/session.client.spec.ts +++ b/packages/api/session-controller/tests/session.client.spec.ts @@ -199,6 +199,194 @@ describe('paging', () => { expect(eventSeqs(session)).toEqual([...older, ...newer].map(event => event.seq)) }) + it('stages one page without hiding live events and consumes it once', async ({ mock, start }) => { + const older = plainTurn(SessionSeq(0), 0, 'old', 'answer') + const newer = plainTurn(SessionSeq(6), 1, 'new', 'answer') + const session = await sessionBench(mock, start, SID) + mock.stream(FOLLOW, followScript(history(newer, true))) + mock.remote.session.page.mockImplementation(pageRule(history(older))) + await session.open() + await session.loadOlder({ deferConsumption: true }) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'pendingConsumption', loadingOlder: false, hasMore: true }) + expect(eventSeqs(session)).toEqual(newer.map(event => event.seq)) + await session.loadOlder({ deferConsumption: true }) + expect(mock.log.requests(PAGE)).toHaveLength(1) + await pushEvent(mock, ev.turnStart(SessionSeq(12), 2)) + const consumeStates: { status: string; loading: boolean }[] = [] + const stop = session.eventSource.subscribe(() => { + session.handleRunning(true) + const snapshot = session.getSnapshot() + consumeStates.push({ status: snapshot.historyLoadStatus, loading: snapshot.loadingOlder }) + expect(session.consumeOlder()).toBe(false) + void session.loadOlder({ deferConsumption: true }) + }) + try { + expect(session.consumeOlder()).toBe(true) + } finally { + stop() + } + expect(consumeStates).toEqual([{ status: 'pendingConsumption', loading: false }]) + expect(session.consumeOlder()).toBe(false) + expect(mock.log.requests(PAGE)).toHaveLength(1) + expect(eventSeqs(session)).toEqual([...older, ...newer].map(event => event.seq).concat(SessionSeq(12))) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'idle', hasMore: false }) + }) + + it('joins a deferred request for default consumption without requesting another page', async ({ mock, start }) => { + const older = plainTurn(SessionSeq(0), 0, 'old', 'answer') + const newer = plainTurn(SessionSeq(6), 1, 'new', 'answer') + const session = await sessionBench(mock, start, SID) + mock.stream(FOLLOW, followScript(history(newer, true))) + const gate = Promise.withResolvers>() + mock.remote.session.page.mockReturnValueOnce(gate.promise) + await session.open() + const preparing = session.loadOlder({ deferConsumption: true }) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'loading', loadingOlder: true }) + const consuming = session.loadOlder() + gate.resolve(history(older)) + await Promise.all([preparing, consuming]) + expect(mock.log.requests(PAGE)).toHaveLength(1) + expect(eventSeqs(session)).toEqual([...older, ...newer].map(event => event.seq)) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'idle', loadingOlder: false }) + }) + + it('default loading consumes an already prepared page without another request', async ({ mock, start }) => { + const older = plainTurn(SessionSeq(0), 0, 'old', 'answer') + const newer = plainTurn(SessionSeq(6), 1, 'new', 'answer') + const session = await sessionBench(mock, start, SID) + mock.stream(FOLLOW, followScript(history(newer, true))) + mock.remote.session.page.mockImplementation(pageRule(history(older))) + await session.open() + await session.loadOlder({ deferConsumption: true }) + await session.loadOlder() + expect(mock.log.requests(PAGE)).toHaveLength(1) + expect(eventSeqs(session)).toEqual([...older, ...newer].map(event => event.seq)) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'idle', loadingOlder: false }) + }) + + it('does not consume a newer prepared page when an obsolete default request completes', async ({ mock, start }) => { + const older = plainTurn(SessionSeq(0), 0, 'old', 'answer') + const newer = plainTurn(SessionSeq(6), 1, 'new', 'answer') + const session = await sessionBench(mock, start, SID) + mock.stream(FOLLOW, followScript(history(newer, true))) + mock.remote.session.page.mockImplementation(pageRule(history(older))) + await session.open() + const obsolete = Promise.withResolvers() + const prepend = vi.spyOn(SessionEventStream.prototype, 'prepend').mockImplementationOnce(() => obsolete.promise) + const loading = session.loadOlder() + try { + await session.resync() + await session.loadOlder({ deferConsumption: true }) + obsolete.resolve(undefined) + await loading + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'pendingConsumption', loadingOlder: false }) + expect(eventSeqs(session)).toEqual(newer.map(event => event.seq)) + expect(session.consumeOlder()).toBe(true) + } finally { + obsolete.resolve(undefined) + await loading + prepend.mockRestore() + } + }) + + it('retains partial-history knowledge across exhaustion and replacement only for the same Session instance', async ({ mock, start }) => { + const older = plainTurn(SessionSeq(0), 0, 'old', 'answer') + const newer = plainTurn(SessionSeq(6), 1, 'new', 'answer') + const complete = [...older, ...newer] + const session = await sessionBench(mock, start, SID) + expect(session.getSnapshot().historyWasPartial).toBe(false) + mock.stream(FOLLOW, followScript(history(newer, true))) + mock.remote.session.page.mockImplementation(pageRule(history(older))) + await session.open() + expect(session.getSnapshot().historyWasPartial).toBe(true) + await session.loadOlder() + expect(session.getSnapshot()).toMatchObject({ hasMore: false, historyWasPartial: true }) + mock.stream(FOLLOW, followScript(history(complete, false))) + await session.resync() + expect(session.getSnapshot()).toMatchObject({ hasMore: false, historyWasPartial: true }) + mock.streams.fail(FOLLOW, new RemoteStreamCarrierError('replace complete window')) + await vi.waitFor(() => { expect(mock.log.requests(FOLLOW)).toHaveLength(3) }) + await mock.streams.drained(FOLLOW) + expect(session.getSnapshot()).toMatchObject({ hasMore: false, historyWasPartial: true }) + const client = await start() + const fresh = new Session(SID, client.ctx.remote) + onTestFinished(() => fresh.dispose()) + expect(fresh.getSnapshot().historyWasPartial).toBe(false) + await fresh.open() + expect(fresh.getSnapshot()).toMatchObject({ hasMore: false, historyWasPartial: false }) + }) + + it('reports no progress and allows an explicit retry to confirm exhaustion', async ({ mock, start }) => { + const session = await sessionBench(mock, start, SID) + mock.stream(FOLLOW, followScript(history(plainTurn(SessionSeq(6), 1, 'new', 'answer'), true))) + mock.remote.session.page.mockImplementation(pageRule(history([], true))) + await session.open() + await session.loadOlder({ deferConsumption: true }) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'error', loadingOlder: false, hasMore: true }) + expect(session.consumeOlder()).toBe(false) + mock.remote.session.page.mockImplementation(pageRule(history([], false))) + await session.loadOlder({ deferConsumption: true }) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'idle', loadingOlder: false, hasMore: false }) + expect(mock.log.requests(PAGE)).toHaveLength(2) + }) + + it('consumes a staged page before a turn jump fetches the remaining history', async ({ mock, start }) => { + const oldest = plainTurn(SessionSeq(0), 0, 'oldest', 'answer') + const middle = plainTurn(SessionSeq(6), 1, 'middle', 'answer') + const newest = plainTurn(SessionSeq(12), 2, 'newest', 'answer') + const session = await sessionBench(mock, start, SID) + mock.stream(FOLLOW, followScript(history(newest, true))) + mock.remote.session.page.mockImplementation(pageRule(request => history( + request.beforeSeq === 12 ? middle : oldest, request.beforeSeq === 12, + ))) + await session.open() + await session.loadOlder({ deferConsumption: true }) + await session.loadThrough(SessionSeq(0)) + expect(eventSeqs(session)).toEqual([...oldest, ...middle, ...newest].map(event => event.seq)) + expect((mock.log.requests(PAGE) as SessionPageRequest[]).map(request => request.beforeSeq)).toEqual([12, 6]) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'idle', loadingOlder: false, hasMore: false }) + }) + + it('invalidates a staged page when the event window is replaced', async ({ mock, start }) => { + const session = await sessionBench(mock, start, SID) + const older = plainTurn(SessionSeq(0), 0, 'old', 'answer') + const newer = plainTurn(SessionSeq(6), 1, 'new', 'answer') + mock.stream(FOLLOW, followScript(history(newer, true))) + mock.remote.session.page.mockImplementation(pageRule(history(older))) + await session.open() + await session.loadOlder({ deferConsumption: true }) + await session.resync() + expect(session.consumeOlder()).toBe(false) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'idle', loadingOlder: false, hasMore: true }) + expect(eventSeqs(session)).toEqual(newer.map(event => event.seq)) + }) + + it('repairs the transport cursor when follow replaces a window during an older request', async ({ mock, start }) => { + const session = await sessionBench(mock, start, SID) + const older = plainTurn(SessionSeq(0), 0, 'old', 'answer') + const newer = plainTurn(SessionSeq(6), 1, 'new', 'answer') + mock.stream(FOLLOW, followScript(history(newer, true))) + const page = Promise.withResolvers>() + mock.remote.session.page.mockReturnValueOnce(page.promise) + await session.open() + const loading = session.loadOlder({ deferConsumption: true }) + await vi.waitFor(() => { expect(mock.log.requests(PAGE)).toHaveLength(1) }) + mock.streams.fail(FOLLOW, new RemoteStreamCarrierError('replace while paging')) + await vi.waitFor(() => { expect(mock.log.requests(FOLLOW)).toHaveLength(2) }) + await mock.streams.drained(FOLLOW) + page.resolve(history(older)) + await loading + await vi.waitFor(() => { + expect(mock.log.requests(FOLLOW)).toHaveLength(3) + expect(session.getSnapshot().historyLoadStatus).toBe('idle') + }) + expect(session.consumeOlder()).toBe(false) + expect(eventSeqs(session)).toEqual(newer.map(event => event.seq)) + mock.remote.session.page.mockImplementation(pageRule(history(older))) + await session.loadOlder() + expect(eventSeqs(session)).toEqual([...older, ...newer].map(event => event.seq)) + }) + it('installs a page without interpreting business replacement metadata', async ({ mock, start }) => { const session = await sessionBench(mock, start, SID) mock.stream(FOLLOW, followScript(history([ @@ -218,7 +406,7 @@ describe('paging', () => { } }) - it('drops a discontinuous older page fail-soft (window unchanged, hasMore cleared)', async ({ mock, start }) => { + it('reports a discontinuous older page without claiming history exhaustion', async ({ mock, start }) => { const session = await sessionBench(mock, start, SID) mock.stream(FOLLOW, followScript(history(plainTurn(SessionSeq(10), 1, '新', '页'), true))) mock.remote.session.page.mockImplementation(pageRule(history(plainTurn(SessionSeq(0), 0, '断', '层'), true))) // tail seq 5, but baseSeq is 10 → hole @@ -229,7 +417,7 @@ describe('paging', () => { await session.loadOlder() const snapshot = session.getSnapshot() expect(session.eventSource.getSnapshot().entries).toEqual(windowBefore.entries) - expect(snapshot.hasMore).toBe(false) + expect(snapshot).toMatchObject({ hasMore: true, historyLoadStatus: 'error' }) } finally { errorSpy.mockRestore() } @@ -247,11 +435,11 @@ describe('paging', () => { mock.remote.session.page.mockImplementation(pageRule(request => (request.beforeSeq === 12 ? history(middle, true) : history(oldest)))) mock.remote.session.page.mockReturnValueOnce(gate.promise) const jump = session.loadThrough(SessionSeq(0)) - expect(session.getSnapshot().loadingOlder).toBe(true) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'loading', loadingOlder: true }) gate.resolve(history(middle, true)) await jump const snapshot = session.getSnapshot() - expect(snapshot.loadingOlder).toBe(false) + expect(snapshot).toMatchObject({ historyLoadStatus: 'idle', loadingOlder: false }) expect(eventSeqs(session)).toEqual([...oldest, ...middle, ...newest].map(event => event.seq)) expect(mock.log.requests(PAGE)).toMatchObject([ { beforeSeq: 12, maxMessages: 500, turnWindow: { minMessages: 200, minTurns: 2 } }, @@ -325,7 +513,7 @@ describe('paging', () => { expect(changed).toHaveBeenCalledOnce() expect(eventSeqs(session)).toEqual([...middle, ...newest].map(event => event.seq)) - expect(session.getSnapshot()).toMatchObject({ loadingOlder: false, hasMore: true }) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'error', loadingOlder: false, hasMore: true }) }) it('discards buffered jump pages when the Session replaces its stream', async ({ mock, start }) => { @@ -430,7 +618,19 @@ describe('paging', () => { mock.remote.session.page.mockImplementation(pageRule(history([], true))) // empty page still claiming more history await session.open() await session.loadThrough(SessionSeq(0)) - expect(session.getSnapshot().loadingOlder).toBe(false) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'error', loadingOlder: false }) + expect(mock.log.requests(PAGE)).toHaveLength(1) + }) + + it('loadThrough treats a terminal empty page as exhaustion rather than an error', async ({ mock, start }) => { + const session = await sessionBench(mock, start, SID) + const newest = plainTurn(SessionSeq(12), 2, 'x', 'y') + mock.stream(FOLLOW, followScript(history(newest, true))) + mock.remote.session.page.mockImplementation(pageRule(history([], false))) + await session.open() + await session.loadThrough(SessionSeq(0)) + expect(session.getSnapshot()).toMatchObject({ historyLoadStatus: 'idle', loadingOlder: false, hasMore: false }) + expect(eventSeqs(session)).toEqual(newest.map(event => event.seq)) expect(mock.log.requests(PAGE)).toHaveLength(1) }) diff --git a/packages/client/ui-chat/README.i18n.yaml b/packages/client/ui-chat/README.i18n.yaml index 2e047aefad..b17f85460e 100644 --- a/packages/client/ui-chat/README.i18n.yaml +++ b/packages/client/ui-chat/README.i18n.yaml @@ -36,8 +36,8 @@ en: e2b02942a55fb669 zh: a38bbee0cc69ae4e /deepseek-ai-dsh-client-ui-chat/scroll-ownership: - en: dd4e69828096c3a0 - zh: 2a9bc370f44efcd7 + en: a8e5c51345ef61b9 + zh: 0e65169b79504da8 /deepseek-ai-dsh-client-ui-chat/model-experience: en: c17b4dd43a61c569 zh: f7414309810c3a08 diff --git a/packages/client/ui-chat/README.md b/packages/client/ui-chat/README.md index 562ed7ebef..8df3db6b32 100644 --- a/packages/client/ui-chat/README.md +++ b/packages/client/ui-chat/README.md @@ -184,7 +184,9 @@ Within a process group, wheel, touchstart, and any pointerdown interrupt an acti `useChatViewport` owns turn-aware DOM reads, clamped writes, native events, and one retained paging anchor. For Load older, Node and Group seats mark eligible anchors from their existing disclosure state. The viewport selects the first nonempty, unhidden marker in transcript order without hit testing or geometry-based search, then measures that element and its scroll containers. It compensates the anchor's capped group first, then gives the remaining displacement to the transcript scrollport. An inner compensation write cancels that group's animation through its bound controller and pauses following; reader scrolling back to the bottom resumes it. Commits and later content resizes reuse that anchor; a remounted row is resolved by the same semantic key. Compensation stays within the actual scroll ranges without adding bottom space. -`useChatReading` owns follow policy, sampled reader input, and semantic memory; `useChatNavigation` owns turn jumps and requests preservation from the viewport. Reading gestures release the paging anchor, but composer clicks, typing, and non-scrolling keys retain it; while a page is still loading, `scrollend` captures the reader's new position. `useChatScroll` coordinates their committed inputs. Explicit navigation carries its measured landing into reading policy, so it does not rediscover the known target with a hit test. +`useChatReading` owns follow policy, sampled reader input, and semantic memory; `useChatNavigation` owns turn jumps and requests preservation from the viewport. Reading gestures release the paging anchor, but composer clicks, typing, and non-scrolling keys retain it. A deferred history request captures its anchor at consumption rather than at request time. `useChatScroll` coordinates their committed inputs. Explicit navigation carries its measured landing into reading policy, so it does not rediscover the known target with a hit test. + +History prefetch uses the existing 500ms reading sample and content/size notifications, with one distance check before requesting within two viewport heights. At most one page waits for consumption. Touch and inner scrolling retain automatic intent until release; active motion, pointer holds, or an unavailable anchor return automatic loading to the button without retrying the cached page. Explicit clicks retain intent until consumption, exhaustion, failure, or superseding navigation. Stationary consumption uses retained positioning; ordinary outer scrolling uses browser anchoring. At scrollTop ≤ 0.5, the first paging anchor supplies one content-displacement correction that retains intervening reader movement and then releases immediately. The actual Chat scrollport disables vertical overscroll bounce, without adding a scrolling animation. The history row keeps its height through loading, processing, retry, and exhaustion; initially complete Sessions omit it. Active-Turn highlighting is approximate: `readVisibleTurn` binary-searches the flow slot's direct Node/Group boxes (or the column's in standalone rendering) and retains the preceding candidate in gaps. It neither hit-tests the document nor searches Group members or all Turn markers. Empty Seats retain zero-height in-flow boxes so outer positions remain ordered without extra spacing. This lookup does not change semantic position capture or paging compensation. diff --git a/packages/client/ui-chat/README.zh.md b/packages/client/ui-chat/README.zh.md index 7696d53951..c2144fd861 100644 --- a/packages/client/ui-chat/README.zh.md +++ b/packages/client/ui-chat/README.zh.md @@ -184,7 +184,9 @@ Chat 会在历史前插与 renderer 重新挂载时恢复语义锚点，并且 `useChatViewport` 负责识别轮次的 DOM 读取、限制范围后的写入、原生事件及一个持续保留的分页锚点。加载更早时，Node 与 Group 容器根据已有开合状态标记可选锚点。视口按正文顺序选择第一个非空、未隐藏的标记，不做命中测试或按几何位置搜索，再测量该元素及其滚动容器。先补偿锚点所在的限高组，再把剩余位移交给整段文本的滚动区域。内层补偿写入通过该组绑定的控制器取消动画并暂停跟随，读者滚回底部后恢复。提交和后续内容尺寸变化复用该锚点；消息行重挂载时按同一语义 key 重新定位。补偿限于实际可滚动范围，不增加底部空白。 -`useChatReading` 负责跟随策略、读者输入采样和语义位置记忆；`useChatNavigation` 负责轮次跳转，并向视口请求位置保持。阅读手势释放分页锚点，composer 内的点击、打字和非滚动按键则保留锚点；分页仍在加载时，`scrollend` 会记录读者的新位置。`useChatScroll` 协调已提交的输入。显式导航把测得的落点交给阅读策略，因此无需通过命中测试重新寻找已知目标。 +`useChatReading` 负责跟随策略、读者输入采样和语义位置记忆；`useChatNavigation` 负责轮次跳转，并向视口请求位置保持。阅读手势释放分页锚点，composer 内的点击、打字和非滚动按键则保留锚点。延迟消费的历史请求在消费时采集锚点，不在发请求时采集。`useChatScroll` 协调已提交的输入。显式导航把测得的落点交给阅读策略，因此无需通过命中测试重新寻找已知目标。 + +历史预取复用已有 500ms 阅读采样及内容／尺寸通知，在请求前确认距顶部两屏以内，最多一页等待消费。触摸和内层滚动保留自动意图至释放；动画、指针按住或无法取得锚点使自动加载退回按钮，不自动重试缓存页。明确点击保留意图，直到消费、到头、失败或被新导航取代。静止消费使用保位，普通外层滚动使用浏览器锚定；scrollTop ≤ 0.5 时用首条分页锚点补偿一次内容位移，保留期间的读者滚动，随后立即释放。实际 Chat 滚动区关闭纵向越界回弹，不新增滚动动画。历史行在加载、处理、重试和到头时保持高度，首次打开就完整的 Session 不显示该行。 活跃轮次高亮采用近似定位：`readVisibleTurn` 对 flow 插槽的直属 Node/Group 布局盒二分（独立渲染时使用正文容器的直属布局盒），落在间隙时保留前方候选。它不做文档命中测试，也不查询 Group 成员或全部 Turn 标记。空 Seat 保留零高度的流内布局盒，使外层位置有序且不增加间距。这种定位不改变语义位置捕获或分页补偿。 diff --git a/packages/client/ui-chat/src/client/apply.ts b/packages/client/ui-chat/src/client/apply.ts index 084e45e414..7e4a3a5f01 100644 --- a/packages/client/ui-chat/src/client/apply.ts +++ b/packages/client/ui-chat/src/client/apply.ts @@ -287,7 +287,8 @@ export function apply(ctx: Context): void { window.open(url, '_blank', 'noopener,noreferrer') } }, - loadOlder: () => { void session.loadOlder() }, + loadOlder: () => { void session.loadOlder({ deferConsumption: true }) }, + consumeOlder: () => session.consumeOlder(), loadThrough: seq => session.loadThrough(seq), loadImage: Object.assign( (attachment: ImageAttachmentRef) => ctx.uiConversation.imageUrl(sessionId, attachment), diff --git a/packages/client/ui-chat/src/client/chat/ChatView.module.css b/packages/client/ui-chat/src/client/chat/ChatView.module.css index 9929770b35..1f84e97842 100644 --- a/packages/client/ui-chat/src/client/chat/ChatView.module.css +++ b/packages/client/ui-chat/src/client/chat/ChatView.module.css @@ -37,6 +37,11 @@ container-type: inline-size; } +.scroll, +:global([data-conversation-scroll]):has(.root) { + overscroll-behavior-y: none; +} + .root[data-chat-following-tail] .scroll, :global([data-conversation-scroll]):has(.root[data-chat-following-tail]) { overflow-anchor: none; @@ -318,11 +323,15 @@ } .older { + overflow-anchor: none; display: flex; justify-content: center; } .older button { + display: grid; + line-height: 18px; + white-space: nowrap; border: none; border-radius: var(--dsw-radius-sm); padding: 4px 12px; @@ -332,6 +341,14 @@ cursor: pointer; } +.older button > span { + grid-area: 1 / 1; +} + +.historyHidden { + visibility: hidden; +} + .older button:disabled { cursor: default; opacity: 0.6; diff --git a/packages/client/ui-chat/src/client/chat/ChatView.tsx b/packages/client/ui-chat/src/client/chat/ChatView.tsx index 48e081bfa8..8fa10ca0ec 100644 --- a/packages/client/ui-chat/src/client/chat/ChatView.tsx +++ b/packages/client/ui-chat/src/client/chat/ChatView.tsx @@ -63,7 +63,7 @@ const MOTION_TAIL_MS = 800 export function ChatView({ useSession, useChat, useConversation, useSessions, renderSlot, sessionId, openFile, openExternalLink, resolveFilePath, useContextMenu, - loadOlder, loadThrough, loadImage, inspectCall, chatScroll, forkAt, + loadOlder, consumeOlder, loadThrough, loadImage, inspectCall, chatScroll, forkAt, usePresentation, useProjection, useInteractionIds, dispatchInteraction, t, fileDrag, }: ChatViewSlotProps) { const { getTriggerProps } = useContextMenu() @@ -136,6 +136,8 @@ export function ChatView({ const openError = useSession(s => s.openError) const hasMore = useSession(s => s.hasMore) const loadingOlder = useSession(s => s.loadingOlder) + const historyLoadStatus = useSession(s => s.historyLoadStatus) + const historyWasPartial = useSession(s => s.historyWasPartial) const [fileOpenError, setFileOpenError] = useState<{ path: string; message: string } | null>(null) const [fileOpenBusy, setFileOpenBusy] = useState(false) // Close/retry must ignore a settlement that started before the latest @@ -211,8 +213,9 @@ export function ChatView({ const steeringId = latestSteering?.source.kind === 'user' && 'rpcId' in latestSteering.source ? latestSteering.source.rpcId : latestSteering?.id ?? null const scroll = useChatScroll({ - ready: openState === 'open', - order, firstSeq, lastKey, running, loadingOlder, hasMore, chatScroll, loadOlder, loadThrough, deferCompletedTurns, + isOpen: openState === 'open', + order, firstSeq, lastKey, running, loadingOlder, historyLoadStatus, hasMore, chatScroll, + loadOlder, consumeOlder, loadThrough, deferCompletedTurns, lastIsUser: lastKey !== null && nodeStore.get(lastKey)?.kind === 'user', steeringId, submissionId: visibleSubmissions.at(-1)?.requestId ?? null, @@ -247,10 +250,24 @@ export function ChatView({ {t('chat.loadError', { message: openError.message, code: openError.code })}
)} - {hasMore && ( -
-
)} diff --git a/packages/client/ui-chat/src/client/chat/flow-motion.ts b/packages/client/ui-chat/src/client/chat/flow-motion.ts index b34c1f2645..7deddd0e9a 100644 --- a/packages/client/ui-chat/src/client/chat/flow-motion.ts +++ b/packages/client/ui-chat/src/client/chat/flow-motion.ts @@ -72,6 +72,8 @@ export interface InputFoldDriver { /** Animation lifetime and fold coordination for one viewport. */ export interface FlowMotion extends FlowMotionRows { + /** @returns whether owned transitions or a scheduled, running, or settling batch remain active. */ + active(): boolean /** @returns whether a row owned by this viewport is closing. */ foldActive(): boolean /** @param listener - callback after the last fold. @returns cancellation for a pending callback. */ @@ -494,6 +496,7 @@ export function createFlowMotion( return { collapse, reveal, cancelMotion, foldActive, onFoldIdle, cancelInput, resumeInput, clear, hideNow, + active: () => running.size > 0 || batchScheduled || batch || settling, inputActive: () => batch || (batchScheduled && [...boxes.values()].some(box => box.closing && batchMember(box))), } } diff --git a/packages/client/ui-chat/src/client/chat/use-chat-navigation.ts b/packages/client/ui-chat/src/client/chat/use-chat-navigation.ts index 84e2b3ec77..3d5dbed544 100644 --- a/packages/client/ui-chat/src/client/chat/use-chat-navigation.ts +++ b/packages/client/ui-chat/src/client/chat/use-chat-navigation.ts @@ -1,5 +1,6 @@ /** Turn jumps and history-prepend anchoring, independent of DOM measurement. */ import { useLayoutEffect, useState } from 'react' +import type { SessionSnapshot } from '@deepseek-ai/dsh-api-session-controller/client' import type { SessionSeq } from '@deepseek-ai/dsh-session/types' import type { ChatNode } from '../contract/chat-nodes.ts' import type { ChatViewSlotProps } from '../contract/slots.ts' @@ -8,10 +9,12 @@ import type { ChatReading, ReadingSample } from './use-chat-reading.ts' import type { ChatViewport } from './use-chat-viewport.ts' /** History availability and loading operations for the committed Chat window. */ -export interface ChatNavigationInput extends Pick { +export interface ChatNavigationInput extends Pick { readonly firstSeq: ChatNode['anchorSeq'] | null readonly hasMore: boolean readonly loadingOlder: boolean + readonly historyLoadStatus: SessionSnapshot['historyLoadStatus'] + readonly isOpen: boolean } interface TurnJump { @@ -22,23 +25,34 @@ interface TurnJump { repageHead: ChatNavigationInput['firstSeq'] } +type HistoryIntent = 'none' | 'auto' | 'manual' + /** Owns one replaceable turn jump and the anchor retained while history loads. */ export class ChatNavigation { private jump: TurnJump | null = null private settleFrame: number | null = null + private historyFrame: number | null = null + private stopFoldWait: (() => void) | null = null + private historyIntent: HistoryIntent = 'none' + private historyRequested = false constructor( private readonly viewport: ChatViewport, private readonly reading: ChatReading, private input: ChatNavigationInput, private readonly onBusyTurn: (turn: number | null) => void, + private readonly onHistoryIntent: (intent: HistoryIntent) => void = () => {}, ) {} /** * Adopt committed history availability without starting a request. * @param input - history state from the latest committed render. */ - setInput(input: ChatNavigationInput): void { this.input = input } + setInput(input: ChatNavigationInput): void { + this.input = input + this.historyRequested = false + if (!input.isOpen || input.historyLoadStatus === 'error' || !input.hasMore) this.setHistoryIntent('none') + } /** Cancel navigation when opening a Chat view. */ reset(): void { @@ -48,10 +62,21 @@ export class ChatNavigation { /** Cancel local callbacks; late history completions cannot revive a task. */ dispose(): void { this.clearTask() + if (this.historyFrame !== null) cancelAnimationFrame(this.historyFrame) + this.historyFrame = null + this.stopFoldWait?.() + this.stopFoldWait = null } /** Release the jump, paging anchor, and busy indicator without cancelling shared history I/O. */ cancel(): void { + this.setHistoryIntent('none') + this.clearTask() + this.onBusyTurn(null) + } + + /** Release a jump after reader input without cancelling an accepted history click. */ + readerInteracted(): void { this.clearTask() this.onBusyTurn(null) } @@ -60,6 +85,7 @@ export class ChatNavigation { this.cancelFrame() this.jump = null this.viewport.stopPreserving() + this.viewport.cancelHistoryPrepend() } /** @@ -90,12 +116,99 @@ export class ChatNavigation { this.request(jump) } - /** Request one older page while retaining the current semantic position. */ + /** Request and consume one page, retaining the click until success, exhaustion, or failure. */ readonly loadEarlier = (): void => { this.cancel() - this.viewport.beginPaging() + this.setHistoryIntent('manual') this.reading.pauseFollowing() - this.input.loadOlder() + if (this.input.historyLoadStatus === 'error') { + this.historyRequested = true + this.input.loadOlder() + } else this.requestHistoryCheck() + } + + private setHistoryIntent(intent: HistoryIntent): void { + if (intent === this.historyIntent) return + this.historyIntent = intent + if (intent !== 'manual') { + this.stopFoldWait?.() + this.stopFoldWait = null + } + this.onHistoryIntent(intent) + } + + /** Coalesce event-driven checks; waiting never schedules its own next frame. */ + requestHistoryCheck(): void { + if (!this.input.isOpen || this.input.historyLoadStatus === 'error') return + if (this.historyIntent === 'none' && (this.input.historyLoadStatus !== 'idle' + || !this.input.hasMore || this.reading.followingTail)) return + if (this.historyFrame !== null || typeof requestAnimationFrame !== 'function') return + this.historyFrame = requestAnimationFrame(() => { + this.historyFrame = null + this.checkHistory() + }) + } + + private checkHistory(): void { + const input = this.input + if (!input.isOpen || this.jump !== null || this.historyRequested || input.loadingOlder) return + if (input.historyLoadStatus === 'error') return + if (input.historyLoadStatus === 'pendingConsumption') { + if (this.historyIntent === 'none') return + if (this.reading.followingTail && this.historyIntent === 'auto') { + this.setHistoryIntent('none') + return + } + const mode = this.viewport.historyConsumption() + switch (mode) { + case 'touch': + case 'inner-scroll': + return + case 'motion': + case 'pointer': + case 'unavailable': + if (this.historyIntent === 'auto') this.setHistoryIntent('none') + else if (mode === 'motion' && this.viewport.motion.foldActive()) { + this.stopFoldWait ??= this.viewport.motion.onFoldIdle(() => { + this.stopFoldWait = null + this.requestHistoryCheck() + }) + } + return + case 'prepend': + if (!this.viewport.beginHistoryPrepend() && input.firstSeq !== null) { + if (this.historyIntent === 'auto') this.setHistoryIntent('none') + return + } + break + case 'preserve': + case 'native': + break + } + if (mode === 'preserve') { + this.viewport.beginPreserving() + if (!this.viewport.preserving) this.viewport.beginPaging() + if (!this.viewport.preserving && input.firstSeq !== null) { + if (this.historyIntent === 'auto') this.setHistoryIntent('none') + return + } + this.reading.pauseFollowing() + } + this.historyRequested = true + const consumed = input.consumeOlder() + if (consumed) this.setHistoryIntent('none') + else { + this.historyRequested = false + if (mode === 'preserve') this.viewport.stopPreserving() + if (mode === 'prepend') this.viewport.cancelHistoryPrepend() + } + return + } + if (!input.hasMore || this.historyIntent !== 'manual' && (this.reading.followingTail + || !this.viewport.confirmHistoryStart())) return + if (this.historyIntent === 'none') this.setHistoryIntent('auto') + this.historyRequested = true + input.loadOlder() } /** @@ -120,13 +233,6 @@ export class ChatNavigation { return true } - /** Retarget a still-loading page only after inner or outer reader scrolling ends. */ - readerSettled(): void { - if (this.input.loadingOlder && this.jump === null && !this.viewport.preserving && !this.reading.followingTail) { - this.viewport.beginPreserving() - } - } - /** Land, retry, or complete the current jump against the committed window. */ reconcile(): void { const jump = this.jump @@ -197,10 +303,11 @@ export class ChatNavigation { */ export function useChatNavigation( viewport: ChatViewport, reading: ChatReading, input: ChatNavigationInput, -): { navigation: ChatNavigation; busyTurn: number | null } { +): { navigation: ChatNavigation; busyTurn: number | null; historyPending: boolean; historyProcessing: boolean } { const [busyTurn, setBusyTurn] = useState(null) - const [navigation] = useState(() => new ChatNavigation(viewport, reading, input, setBusyTurn)) + const [historyIntent, setHistoryIntent] = useState('none') + const [navigation] = useState(() => new ChatNavigation(viewport, reading, input, setBusyTurn, setHistoryIntent)) useLayoutEffect(() => { navigation.setInput(input) }, [navigation, input]) useLayoutEffect(() => () => { navigation.dispose() }, [navigation]) - return { navigation, busyTurn } + return { navigation, busyTurn, historyPending: historyIntent === 'manual', historyProcessing: historyIntent !== 'none' } } diff --git a/packages/client/ui-chat/src/client/chat/use-chat-scroll.ts b/packages/client/ui-chat/src/client/chat/use-chat-scroll.ts index 7924a33e3f..c8254dc626 100644 --- a/packages/client/ui-chat/src/client/chat/use-chat-scroll.ts +++ b/packages/client/ui-chat/src/client/chat/use-chat-scroll.ts @@ -1,5 +1,5 @@ /** Composes viewport operations, reading policy, and history navigation for Chat. */ -import { useCallback, useLayoutEffect, useMemo, useRef, type RefObject } from 'react' +import { useCallback, useLayoutEffect, useMemo, useRef, useState, type RefObject } from 'react' import type { TranscriptViewMode } from '../../chat-settings.ts' import type { ChatSnapshot } from '../contract/snapshot.ts' import type { ChatViewSlotProps } from '../contract/slots.ts' @@ -11,7 +11,6 @@ import type { FlowMotionRows } from './flow-motion.ts' /** Committed content and Session operations used to reconcile scroll ownership. */ export interface ChatScrollInput extends ChatNavigationInput { readonly chatScroll: ChatViewSlotProps['chatScroll'] - readonly ready: boolean readonly order: readonly string[] readonly lastKey: string | null readonly lastIsUser: boolean @@ -33,12 +32,16 @@ interface ChatScrollState extends ChatReadingState { readonly busyTurn: number | null readonly navigateToTurn: ChatNavigation['navigateToTurn'] readonly loadEarlier: ChatNavigation['loadEarlier'] + readonly historyInputHeld: boolean + readonly historyProcessing: boolean + readonly historyPending: boolean + readonly historyEdgeRef: (element: HTMLDivElement | null) => void readonly returnToBottom: () => void readonly motion: FlowMotionRows } /** - * Coordinate scroll policy after Chat content commits. Initial history must be ready + * Coordinate scroll policy after Chat content commits. Initial history must be available * before automatic scroll or resize handling can initialize reading state and fold motion. * New submitted input supersedes pending reader sampling. * @param input - current Chat content, scroll memory, and history operations. @@ -46,15 +49,17 @@ interface ChatScrollState extends ChatReadingState { */ export function useChatScroll(input: ChatScrollInput): ChatScrollState { const { - ready, order, firstSeq, lastKey, lastIsUser, steeringId, submissionId, submissionSteers, running, - deferCompletedTurns, loadedTurns, chatScroll, hasMore, loadingOlder, loadOlder, loadThrough, transcriptView, + isOpen, order, firstSeq, lastKey, lastIsUser, steeringId, submissionId, submissionSteers, running, + deferCompletedTurns, loadedTurns, chatScroll, hasMore, loadingOlder, historyLoadStatus, + loadOlder, consumeOlder, loadThrough, transcriptView, } = input const { viewport, listRef, columnRef } = useChatViewport() + const [historyInputHeld, setHistoryInputHeld] = useState(false) const { reading, state } = useChatReading(viewport, chatScroll, loadedTurns.at(-1)?.turn ?? null) const navigationInput = useMemo(() => ({ - firstSeq, loadingOlder, hasMore, loadOlder, loadThrough, - }), [firstSeq, loadingOlder, hasMore, loadOlder, loadThrough]) - const { navigation, busyTurn } = useChatNavigation(viewport, reading, navigationInput) + firstSeq, loadingOlder, hasMore, historyLoadStatus, isOpen, loadOlder, consumeOlder, loadThrough, + }), [firstSeq, loadingOlder, hasMore, historyLoadStatus, isOpen, loadOlder, consumeOlder, loadThrough]) + const { navigation, busyTurn, historyPending, historyProcessing } = useChatNavigation(viewport, reading, navigationInput) const content = useRef<{ input: ChatScrollInput; applied: ChatScrollInput | null; opened: boolean }>({ input, applied: null, opened: false, }) @@ -64,7 +69,7 @@ export function useChatScroll(input: ChatScrollInput): ChatScrollState { const processContent = useCallback(() => { const current = content.current.input - if (!current.ready && !content.current.opened) return + if (!current.isOpen && !content.current.opened) return const previous = content.current.applied const steeringInput = current.steeringId !== null && current.steeringId !== previous?.steeringId && current.steeringId !== previous?.submissionId @@ -73,7 +78,7 @@ export function useChatScroll(input: ChatScrollInput): ChatScrollState { const ownInput = (current.lastIsUser && current.lastKey !== previous?.lastKey) || steeringInput || submissionInput if (reading.pending && !ownInput) return content.current.applied = current - if (current.ready && !content.current.opened) { + if (current.isOpen && !content.current.opened) { content.current.opened = true navigation.reset() reading.restore() @@ -92,7 +97,7 @@ export function useChatScroll(input: ChatScrollInput): ChatScrollState { navigation.reconcile() return } - const tipChanged = previous === null || current.ready !== previous.ready + const tipChanged = previous === null || current.isOpen !== previous.isOpen || current.firstSeq !== previous.firstSeq || current.lastKey !== previous.lastKey || current.order.length !== previous.order.length || current.running !== previous.running || current.steeringId !== previous.steeringId || current.submissionId !== previous.submissionId @@ -112,12 +117,17 @@ export function useChatScroll(input: ChatScrollInput): ChatScrollState { }, scrollEnd: (outer) => { reading.onScrollEnd(outer) - navigation.readerSettled() + navigation.requestHistoryCheck() + }, + interact: () => { navigation.readerInteracted() }, + history: () => { + setHistoryInputHeld(viewport.historyInputHeld) + if (content.current.opened) navigation.requestHistoryCheck() }, - interact: () => { navigation.cancel() }, intent: () => { cancelPendingFollow(); reading.interruptFollow() }, resize: () => { if (!content.current.opened) return + navigation.requestHistoryCheck() if (navigation.contentCommitted()) { navigation.reconcile() return @@ -129,6 +139,7 @@ export function useChatScroll(input: ChatScrollInput): ChatScrollState { const disconnectReading = reading.connect((sample) => { navigation.readerSampled(sample) processContent() + navigation.requestHistoryCheck() }) return () => { disconnectViewport() @@ -141,7 +152,7 @@ export function useChatScroll(input: ChatScrollInput): ChatScrollState { useLayoutEffect(() => { const previous = content.current.input content.current.input = { - ready, order, lastKey, lastIsUser, steeringId, submissionId, submissionSteers, running, loadedTurns, + order, lastKey, lastIsUser, steeringId, submissionId, submissionSteers, running, loadedTurns, chatScroll, deferCompletedTurns, transcriptView, ...navigationInput, } if (!deferCompletedTurns) { @@ -149,13 +160,18 @@ export function useChatScroll(input: ChatScrollInput): ChatScrollState { reading.interruptFollow() } viewport.updateTurns(loadedTurns) - const layoutChanged = previous.order !== order || previous.ready !== ready - if (layoutChanged) viewport.invalidate() + const layoutChanged = previous.order !== order || previous.isOpen !== isOpen + if (layoutChanged) { + const landing = viewport.finishHistoryPrepend() + if (landing !== null) reading.preservePosition(landing) + viewport.invalidate() + } viewport.reclaimBelow() processContent() if (layoutChanged && content.current.opened) reading.refreshActiveTurn() + if (content.current.opened) navigation.requestHistoryCheck() }, [ - viewport, reading, processContent, navigationInput, ready, order, lastKey, lastIsUser, + viewport, reading, processContent, navigationInput, isOpen, order, lastKey, lastIsUser, steeringId, submissionId, submissionSteers, running, loadedTurns, chatScroll, deferCompletedTurns, transcriptView, cancelPendingFollow, ]) @@ -182,6 +198,10 @@ export function useChatScroll(input: ChatScrollInput): ChatScrollState { listRef, columnRef, ...state, busyTurn, motion: viewport.motion, navigateToTurn: navigation.navigateToTurn, loadEarlier: navigation.loadEarlier, + historyPending, + historyProcessing, + historyInputHeld, + historyEdgeRef: viewport.setHistoryEdge, returnToBottom, } } diff --git a/packages/client/ui-chat/src/client/chat/use-chat-viewport.ts b/packages/client/ui-chat/src/client/chat/use-chat-viewport.ts index b0dbf7528a..98c6ac2186 100644 --- a/packages/client/ui-chat/src/client/chat/use-chat-viewport.ts +++ b/packages/client/ui-chat/src/client/chat/use-chat-viewport.ts @@ -26,6 +26,8 @@ interface ViewportEvents { interact: () => void /** Any reading gesture, before paging ownership is consulted. */ intent?: () => void + /** Recheck a prepared history page after an input or transition settles. */ + history?: () => void } interface ViewportElements { @@ -65,7 +67,7 @@ interface ClosingLayout { interface PagingPosition { readonly row: HTMLElement readonly position: ChatScrollPosition - readonly group: { readonly body: HTMLElement; readonly content: HTMLElement; readonly top: number } | null + readonly group: { readonly body: HTMLElement; readonly content: HTMLElement; readonly top: number; readonly scrollTop: number } | null } const READING_INTENTS = ['wheel', 'touchstart', 'pointerdown', 'keydown', 'beforematch'] as const @@ -79,7 +81,13 @@ export class ChatViewport { private turns: ReturnType = [] private observation: { top: number; landing: ViewportLanding | null } = { top: 0, landing: null } private paging: PagingPosition | null = null + private historyPrepend: PagingPosition | null = null private reader: FoldReader | null = null + private historyEdge: HTMLElement | null = null + private historyHeight = 0 + private historyScrolling = new Set() + private historyPointer = false + private historyTouch = false /** Rows measured before an instant close; see {@link ChatViewport.keepAboveClosing}. */ private closing: ClosingLayout | null = null /** Discards a closing measurement that no size observation consumed by the end of its frame. */ @@ -114,18 +122,26 @@ export class ChatViewport { const elements = { list, column, scroller, composer, spacer } this.elements = elements if (spacer !== null) spacer.style.height = '' + scroller.addEventListener('scroll', this.onHistoryScroll, { passive: true, capture: true }) + scroller.addEventListener('pointerdown', this.onHistoryPointerDown, { passive: true, capture: true }) + scroller.addEventListener('touchstart', this.onHistoryTouch, { passive: true, capture: true }) + for (const type of ['pointerup', 'pointercancel', 'touchend', 'touchcancel', 'transitionend', 'transitioncancel']) { + scroller.ownerDocument.addEventListener(type, this.onHistoryRelease, true) + } scroller.addEventListener('scroll', this.onScroll, { passive: true }) scroller.addEventListener('scrollend', this.onScrollEnd, { passive: true, capture: true }) for (const type of READING_INTENTS) scroller.addEventListener(type, this.onIntent, { passive: true, capture: true }) if (typeof ResizeObserver !== 'undefined') { - this.observer = new ResizeObserver(() => { + this.observer = new ResizeObserver((entries) => { if (this.elements !== elements) return // Read while layout is still clean, before this callback writes room or scroll positions. Those writes // leave the content height unchanged, so it is also the content this frame paints. + if (entries.some(entry => entry.target === elements.scroller)) this.historyHeight = elements.scroller.clientHeight this.settledContent = elements.scroller.scrollHeight - this.spacerHeight() this.keepAboveClosing() this.reclaimBelow() this.invalidate() + this.pruneHistoryScrolling() this.events?.resize() }) this.observer.observe(column) @@ -136,10 +152,22 @@ export class ChatViewport { /** Disconnect DOM resources and clear observations for the detached view. */ detach(): void { + this.historyPrepend = null this.stopPreserving() this.motion.clear() if (this.elements !== null) this.elements.scroller.style.overflowAnchor = '' this.dropClosing() + const scroller = this.elements?.scroller + scroller?.removeEventListener('scroll', this.onHistoryScroll, true) + scroller?.removeEventListener('pointerdown', this.onHistoryPointerDown, true) + scroller?.removeEventListener('touchstart', this.onHistoryTouch, true) + for (const type of ['pointerup', 'pointercancel', 'touchend', 'touchcancel', 'transitionend', 'transitioncancel']) { + scroller?.ownerDocument.removeEventListener(type, this.onHistoryRelease, true) + } + this.historyHeight = 0 + this.historyScrolling.clear() + this.historyPointer = false + this.historyTouch = false this.elements?.scroller.removeEventListener('scroll', this.onScroll) this.elements?.scroller.removeEventListener('scrollend', this.onScrollEnd, true) for (const type of READING_INTENTS) this.elements?.scroller.removeEventListener(type, this.onIntent, true) @@ -772,10 +800,14 @@ export class ChatViewport { /** Retain the first eligible transcript seat in DOM order; selection reads no geometry. */ beginPaging(): void { this.stopPreserving() - const row = this.elements?.list.querySelector( + const row = this.pagingAnchor() + if (row !== null) this.retain(row) + } + + private pagingAnchor(): HTMLElement | null { + return this.elements?.list.querySelector( '[data-chat-paging-anchor]:not(:empty):not([hidden]):not([hidden] *)', - ) - if (row != null) this.retain(row) + ) ?? null } /** @@ -790,18 +822,17 @@ export class ChatViewport { this.retain(row, position) } - private retain(row: HTMLElement, position?: ChatScrollPosition, groupTop?: number): PagingPosition | null { + private measurePaging(row: HTMLElement, position?: ChatScrollPosition, previousGroup?: PagingPosition['group']): PagingPosition | null { const elements = this.elements const key = row.dataset.chatAnchorKey if (elements === null || key === undefined) return null - const previous = this.paging?.group - if (previous != null) this.observer?.unobserve(previous.content) const top = row.getBoundingClientRect().top const body = row.closest('[data-step-process-body]') const content = body?.querySelector('[data-step-process-content]') const group = body === null || content == null ? null - : { body, content, top: groupTop ?? top - body.getBoundingClientRect().top } - this.paging = { + : { body, content, top: previousGroup?.top ?? top - body.getBoundingClientRect().top, + scrollTop: previousGroup?.body === body ? previousGroup.scrollTop : body.scrollTop } + return { row, group, position: position ?? { anchorKey: key, @@ -809,8 +840,19 @@ export class ChatViewport { scrollTop: elements.scroller.scrollTop, }, } - if (group !== null) this.observer?.observe(group.content) - return this.paging + } + + private retain(row: HTMLElement, position?: ChatScrollPosition): PagingPosition | null { + const paging = this.measurePaging(row, position) + if (paging === null) return null + this.retainPaging(paging) + return paging + } + + private retainPaging(paging: PagingPosition): void { + this.stopPreserving() + this.paging = paging + if (paging.group !== null) this.observer?.observe(paging.group.content) } /** Release paging ownership and its content-size observation. */ @@ -832,28 +874,39 @@ export class ChatViewport { * @returns the actual landing, or null when no visible retained row remains. */ preserve(): ViewportLanding | null { - let paging = this.paging + const paging = this.resolvePaging(this.paging) + if (paging === null) { + this.stopPreserving() + return null + } + if (paging !== this.paging) this.retainPaging(paging) + return this.applyPaging(paging, false) + } + + private resolvePaging(paging: PagingPosition | null): PagingPosition | null { const elements = this.elements if (paging === null || elements === null) return null if (!elements.list.contains(paging.row)) { // Segmentation can remount the same semantic row under another group. const replacement = this.anchor(paging.position.anchorKey) - if (replacement === null) { - this.stopPreserving() - return null - } - paging = this.retain(replacement, paging.position, paging.group?.top) + if (replacement === null) return null + paging = this.measurePaging(replacement, paging.position, paging.group) if (paging === null) return null } + return paging.row.closest('[hidden]') !== null || paging.row.matches(':empty') ? null : paging + } + + /** Content correction includes reader movement since capture; screen preservation restores the captured position. */ + private applyPaging(paging: PagingPosition, preserveMovement: boolean): ViewportLanding | null { + const elements = this.elements + if (elements === null) return null const { row, group, position } = paging - if (row.closest('[hidden]') !== null || row.matches(':empty')) { - this.stopPreserving() - return null - } + let innerMovement = 0 if (group !== null && group.body.contains(row)) { const top = row.getBoundingClientRect().top - group.body.getBoundingClientRect().top const metrics = scrollMetrics(group.body) - const target = Math.max(0, Math.min(metrics.floor, metrics.top + top - group.top)) + if (preserveMovement) innerMovement = metrics.top - group.scrollTop + const target = Math.max(0, Math.min(metrics.floor, metrics.top + top - group.top + innerMovement)) if (metrics.top !== target) { const follow = ScrollFollow.forElement(group.body) if (follow === undefined) group.body.scrollTop = target @@ -866,7 +919,8 @@ export class ChatViewport { const metrics = this.metrics() if (metrics === null) return null const top = row.getBoundingClientRect().top - elements.scroller.getBoundingClientRect().top - const target = metrics.top + top - position.anchorTop + const movement = preserveMovement ? metrics.top - position.scrollTop + innerMovement : 0 + const target = metrics.top + top - position.anchorTop + movement return this.write(target, metrics, null, { key: position.anchorKey, top }) } @@ -919,6 +973,111 @@ export class ChatViewport { return landing } + /** + * Track the history control for request-distance checks. + * @param element - stable history control, or null when its view unmounts. + */ + readonly setHistoryEdge = (element: HTMLDivElement | null): void => { + this.historyEdge = element + } + + private pruneHistoryScrolling(): void { + for (const target of this.historyScrolling) { + if (!(target instanceof HTMLElement) || this.elements?.scroller.contains(target) !== true) { + this.historyScrolling.delete(target) + } + } + } + + /** + * Confirm the actual history distance immediately before an automatic request. + * @returns whether the control is within two viewport heights. + */ + confirmHistoryStart(): boolean { + const scroller = this.elements?.scroller + if (scroller === undefined || this.historyEdge === null || this.historyHeight <= 0) return false + const distance = scroller.getBoundingClientRect().top - this.historyEdge.getBoundingClientRect().bottom + return distance <= this.historyHeight * 2 + } + + /** + * Expose held input to history status copy. + * @returns whether a touch or pointer remains down. + */ + get historyInputHeld(): boolean { return this.historyTouch || this.historyPointer } + + /** + * Expose reader activity that prevents stationary preservation. + * @returns whether input is held or scrolling has not ended. + */ + get historyReaderActive(): boolean { + this.pruneHistoryScrolling() + return this.historyTouch || this.historyPointer || this.historyScrolling.size > 0 + } + + /** + * Select consumption behavior from current input and motion state. + * @returns the blocker or positioning path for a prepared page. + */ + historyConsumption(): 'motion' | 'touch' | 'pointer' | 'inner-scroll' | 'unavailable' | 'native' | 'preserve' | 'prepend' { + const elements = this.elements + if (elements === null) return 'unavailable' + if (this.motion.active() || this.reader?.follow.animating) return 'motion' + if (this.historyTouch) return 'touch' + if (this.historyPointer) return 'pointer' + if (this.historyScrolling.size === 0) return elements.scroller.scrollTop <= 0.5 ? 'prepend' : 'preserve' + if (this.historyScrolling.size !== 1 || !this.historyScrolling.has(elements.scroller)) return 'inner-scroll' + if (this.preserving || typeof CSS === 'undefined' || !CSS.supports('overflow-anchor', 'auto')) return 'unavailable' + return elements.scroller.scrollTop <= 0.5 ? 'prepend' : 'native' + } + + /** + * Capture one top-edge prepend anchor without acquiring continuous scroll ownership. + * @returns whether an old paging row was captured. + */ + beginHistoryPrepend(): boolean { + const row = this.pagingAnchor() + this.historyPrepend = row === null ? null : this.measurePaging(row) + return this.historyPrepend !== null + } + + /** Clear an unapplied one-shot correction when its page was not consumed. */ + cancelHistoryPrepend(): void { this.historyPrepend = null } + + /** + * Correct a committed prepend once, then release positioning ownership. + * @returns the compensated landing for reading memory, or null when no anchor remains. + */ + finishHistoryPrepend(): ViewportLanding | null { + const pending = this.historyPrepend + this.historyPrepend = null + const paging = this.resolvePaging(pending) + return paging === null ? null : this.applyPaging(paging, true) + } + + private readonly onHistoryScroll = (event: Event): void => { + if (event.target === this.elements?.scroller || event.target instanceof HTMLElement + && event.target.hasAttribute('data-step-process-body')) { + this.historyScrolling.add(event.target) + } + } + + private readonly onHistoryPointerDown = (): void => { + this.historyPointer = true + this.events?.history?.() + } + private readonly onHistoryTouch = (): void => { + this.historyTouch = true + this.events?.history?.() + } + private readonly onHistoryRelease = (event: Event): void => { + if (event.type === 'pointerup' || event.type === 'pointercancel') this.historyPointer = false + if (event.type === 'touchend' || event.type === 'touchcancel') { + this.historyTouch = typeof TouchEvent !== 'undefined' && event instanceof TouchEvent && event.touches.length > 0 + } + this.events?.history?.() + } + private readonly onScroll = (event: Event): void => { if (this.elements === null || event.target !== this.elements.scroller) return if (this.observation.landing !== null && this.elements.scroller.scrollTop === this.observation.top) return @@ -932,6 +1091,8 @@ export class ChatViewport { } private readonly onScrollEnd = (event: Event): void => { + if (event.target !== null) this.historyScrolling.delete(event.target) + this.events?.history?.() if (event.target === this.elements?.scroller) { this.events?.scrollEnd(true) this.reclaimBelow() diff --git a/packages/client/ui-chat/src/client/contract/slots.ts b/packages/client/ui-chat/src/client/contract/slots.ts index ed89452f50..676bb3fb2d 100644 --- a/packages/client/ui-chat/src/client/contract/slots.ts +++ b/packages/client/ui-chat/src/client/contract/slots.ts @@ -347,7 +347,10 @@ export interface ChatViewInjected { /** Open one HTTP(S) message link at the selected destination, using an external tab if Sidebar Browser is unavailable. */ openExternalLink: (url: string) => void openFile: (path: string, options?: OpenFileOptions) => Promise + /** Prepare one older page without publishing it. */ loadOlder: () => void + /** Consume the prepared page synchronously; false when none is ready. */ + consumeOlder: () => boolean /** Jump loader: page history back through seq; resolves when the window covers it. */ loadThrough: (seq: SessionSeq) => Promise loadImage: MessageImageLoader diff --git a/packages/client/ui-chat/src/client/locale.ts b/packages/client/ui-chat/src/client/locale.ts index 0b2bc2151d..4d22cff80e 100644 --- a/packages/client/ui-chat/src/client/locale.ts +++ b/packages/client/ui-chat/src/client/locale.ts @@ -92,6 +92,10 @@ export const zh = { 'chat.loadingHistory': '载入历史…', 'chat.loadError': '历史加载失败：{message}（{code}）', 'chat.loadOlder': '加载更早', + 'chat.loadingOlder': '正在加载', + 'chat.processingOlder': '正在处理', + 'chat.retryOlder': '重新加载', + 'chat.historyStart': '已到会话开头', 'chat.toBottom': '回到底部', 'chat.deepDiving': '深度求索中', 'chat.deepDivingFor': '深度求索中，用时 {duration} ···', @@ -302,6 +306,10 @@ export const en = { 'chat.loadingHistory': 'Loading history…', 'chat.loadError': 'Failed to load history: {message} ({code})', 'chat.loadOlder': 'Load earlier', + 'chat.loadingOlder': 'Loading earlier', + 'chat.processingOlder': 'Processing', + 'chat.retryOlder': 'Retry loading', + 'chat.historyStart': 'Start of conversation', 'chat.toBottom': 'Back to bottom', 'chat.deepDiving': 'Deep diving', 'chat.deepDivingFor': 'Deep diving for {duration} ···', @@ -509,6 +517,10 @@ export const ja = { 'chat.loadingHistory': '履歴を読み込み中…', 'chat.loadError': '履歴の読み込みに失敗しました: {message}（{code}）', 'chat.loadOlder': '以前を読み込む', + 'chat.loadingOlder': '読み込み中', + 'chat.processingOlder': '処理中', + 'chat.retryOlder': '再読み込み', + 'chat.historyStart': '会話の先頭です', 'chat.toBottom': '最下部へ戻る', 'chat.deepDiving': '深掘り中', 'chat.deepDivingFor': '{duration} の間深く分析中 ···', @@ -716,6 +728,10 @@ export const es = { 'chat.loadingHistory': 'Cargando historial…', 'chat.loadError': 'No se pudo cargar el historial: {message} ({code})', 'chat.loadOlder': 'Cargar antes', + 'chat.loadingOlder': 'Cargando anteriores', + 'chat.processingOlder': 'Procesando', + 'chat.retryOlder': 'Reintentar carga', + 'chat.historyStart': 'Inicio de la conversación', 'chat.toBottom': 'Volver al fondo', 'chat.deepDiving': 'Razonamiento profundo', 'chat.deepDivingFor': 'Razonando a fondo durante {duration} ···', @@ -923,6 +939,10 @@ export const fr = { 'chat.loadingHistory': "Chargement de l'historique…", 'chat.loadError': "Échec du chargement de l'historique : {message} ({code})", 'chat.loadOlder': 'Charger plus tôt', + 'chat.loadingOlder': 'Chargement en cours', + 'chat.processingOlder': 'Traitement en cours', + 'chat.retryOlder': 'Réessayer', + 'chat.historyStart': 'Début de la conversation', 'chat.toBottom': 'Retour en bas', 'chat.deepDiving': 'Réflexion approfondie', 'chat.deepDivingFor': 'Réflexion approfondie depuis {duration} ···', @@ -1130,6 +1150,10 @@ export const de = { 'chat.loadingHistory': 'Verlauf wird geladen…', 'chat.loadError': 'Fehler beim Laden des Verlaufs: {message} ({code})', 'chat.loadOlder': 'Früher laden', + 'chat.loadingOlder': 'Wird geladen', + 'chat.processingOlder': 'Wird verarbeitet', + 'chat.retryOlder': 'Erneut laden', + 'chat.historyStart': 'Anfang der Unterhaltung', 'chat.toBottom': 'Zurück nach unten', 'chat.deepDiving': 'Gründliches Nachdenken', 'chat.deepDivingFor': 'Gründliches Nachdenken seit {duration} ···', @@ -1337,6 +1361,10 @@ export const ru = { 'chat.loadingHistory': 'Загрузка истории…', 'chat.loadError': 'Не удалось загрузить историю: {message} ({code})', 'chat.loadOlder': 'Загрузить раньше', + 'chat.loadingOlder': 'Загрузка истории', + 'chat.processingOlder': 'Обработка', + 'chat.retryOlder': 'Повторить загрузку', + 'chat.historyStart': 'Начало разговора', 'chat.toBottom': 'Вернуться к низу', 'chat.deepDiving': 'Глубокое размышление', 'chat.deepDivingFor': 'Глубокое размышление в течение {duration} ···', @@ -1544,6 +1572,10 @@ export const pt = { 'chat.loadingHistory': 'Carregando histórico…', 'chat.loadError': 'Falha ao carregar o histórico: {message} ({code})', 'chat.loadOlder': 'Carregar mais cedo', + 'chat.loadingOlder': 'Carregando anteriores', + 'chat.processingOlder': 'Processando', + 'chat.retryOlder': 'Tentar novamente', + 'chat.historyStart': 'Início da conversa', 'chat.toBottom': 'De volta ao fundo', 'chat.deepDiving': 'Raciocínio profundo', 'chat.deepDivingFor': 'Raciocínio profundo há {duration} ···', diff --git a/packages/client/ui-chat/tests/apply-inject.client.spec.tsx b/packages/client/ui-chat/tests/apply-inject.client.spec.tsx index 8ee48b8f65..352e5d5042 100644 --- a/packages/client/ui-chat/tests/apply-inject.client.spec.tsx +++ b/packages/client/ui-chat/tests/apply-inject.client.spec.tsx @@ -253,7 +253,7 @@ describe('Chat inject API', () => { const b = await bench() const { injected } = b.chatViewApi(b.rootReference) injected.loadOlder() - expect(b.session.loadOlder).toHaveBeenCalledOnce() + expect(b.session.loadOlder).toHaveBeenCalledExactlyOnceWith({ deferConsumption: true }) void injected.loadThrough(SessionSeq(42)) expect(b.session.loadThrough).toHaveBeenCalledWith(42) diff --git a/packages/client/ui-chat/tests/chat-history-intent.client.spec.ts b/packages/client/ui-chat/tests/chat-history-intent.client.spec.ts new file mode 100644 index 0000000000..1ce525374b --- /dev/null +++ b/packages/client/ui-chat/tests/chat-history-intent.client.spec.ts @@ -0,0 +1,117 @@ +// @vitest-environment jsdom +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { SessionSeq } from '@deepseek-ai/dsh-session/types' +import { ChatNavigation, type ChatNavigationInput } from '../src/client/chat/use-chat-navigation.ts' +import { ChatReading } from '../src/client/chat/use-chat-reading.ts' +import { ChatViewport } from '../src/client/chat/use-chat-viewport.ts' +import { ScrollFollow } from '../src/client/chat/use-scroll-follow.ts' + +function fixture() { + const viewport = new ChatViewport() + vi.spyOn(viewport, 'confirmHistoryStart').mockReturnValue(true) + const mode = vi.spyOn(viewport, 'historyConsumption').mockReturnValue('native') + const reading = new ChatReading(viewport, { read: () => null, save: () => {} }, { + initialized: true, followingTail: false, activeTurn: 1, + }, () => {}, new ScrollFollow(false, 24)) + const loadOlder = vi.fn() + const consumeOlder = vi.fn(() => true) + const onIntent = vi.fn() + let input: ChatNavigationInput = { + isOpen: true, firstSeq: SessionSeq(20), hasMore: true, loadingOlder: false, + historyLoadStatus: 'idle', loadOlder, consumeOlder, loadThrough: async () => {}, + } + const navigation = new ChatNavigation(viewport, reading, input, () => {}, onIntent) + const check = () => { + navigation.requestHistoryCheck() + vi.advanceTimersByTime(20) + } + const ready = () => { + input = { ...input, historyLoadStatus: 'pendingConsumption' } + navigation.setInput(input) + } + return { viewport, navigation, reading, mode, loadOlder, consumeOlder, onIntent, check, ready } +} + +beforeEach(() => { vi.useFakeTimers() }) +afterEach(() => { vi.restoreAllMocks(); vi.useRealTimers() }) + +describe('history consumption intent', () => { + it.each(['motion', 'pointer', 'unavailable'] as const)('leaves an automatic page for a click after %s', (blocker) => { + const h = fixture() + h.check() + expect(h.loadOlder).toHaveBeenCalledOnce() + h.ready() + h.mode.mockReturnValue(blocker) + h.check() + expect(h.onIntent).toHaveBeenLastCalledWith('none') + h.mode.mockReturnValue('native') + h.ready() + h.check() + h.check() + expect(h.consumeOlder).not.toHaveBeenCalled() + expect(h.loadOlder).toHaveBeenCalledOnce() + h.navigation.loadEarlier() + h.check() + expect(h.consumeOlder).toHaveBeenCalledOnce() + h.navigation.dispose() + }) + + it.each(['touch', 'inner-scroll'] as const)('continues automatic consumption after %s ends', (blocker) => { + const h = fixture() + h.check() + h.ready() + h.mode.mockReturnValue(blocker) + h.check() + expect(h.onIntent).toHaveBeenLastCalledWith('auto') + expect(h.consumeOlder).not.toHaveBeenCalled() + h.mode.mockReturnValue('native') + h.check() + expect(h.consumeOlder).toHaveBeenCalledOnce() + expect(h.loadOlder).toHaveBeenCalledOnce() + h.navigation.dispose() + }) + + it('retains an explicit click until the fold ends without requesting the cached page again', () => { + const h = fixture() + h.ready() + h.mode.mockReturnValue('motion') + vi.spyOn(h.viewport.motion, 'foldActive').mockReturnValue(true) + let idle: (() => void) | undefined + const stop = vi.fn() + vi.spyOn(h.viewport.motion, 'onFoldIdle').mockImplementation((callback) => { idle = callback; return stop }) + h.navigation.loadEarlier() + h.check() + expect(h.onIntent).toHaveBeenLastCalledWith('manual') + expect(h.loadOlder).not.toHaveBeenCalled() + expect(h.consumeOlder).not.toHaveBeenCalled() + h.mode.mockReturnValue('native') + idle?.() + vi.advanceTimersByTime(20) + expect(h.consumeOlder).toHaveBeenCalledOnce() + h.navigation.dispose() + }) + + it('joins a pending automatic request when clicked and consumes once after it returns', () => { + const h = fixture() + h.check() + h.navigation.loadEarlier() + h.check() + expect(h.loadOlder).toHaveBeenCalledOnce() + h.ready() + h.check() + expect(h.consumeOlder).toHaveBeenCalledOnce() + expect(h.onIntent).toHaveBeenLastCalledWith('none') + h.navigation.dispose() + }) + + it('does not consume a cached page on a fresh view until clicked', () => { + const h = fixture() + h.ready() + h.check() + expect(h.consumeOlder).not.toHaveBeenCalled() + h.navigation.loadEarlier() + h.check() + expect(h.consumeOlder).toHaveBeenCalledOnce() + h.navigation.dispose() + }) +}) diff --git a/packages/client/ui-chat/tests/chat-scroll.client.spec.tsx b/packages/client/ui-chat/tests/chat-scroll.client.spec.tsx index f0555f7fff..885b16e72c 100644 --- a/packages/client/ui-chat/tests/chat-scroll.client.spec.tsx +++ b/packages/client/ui-chat/tests/chat-scroll.client.spec.tsx @@ -36,7 +36,7 @@ function ScrollHarness({ input, onMotion }: { function mountScroll(deferCompletedTurns = true, initial: Partial = {}) { let input: ChatScrollInput = { - ready: true, + isOpen: true, order: [], firstSeq: null, lastKey: null, @@ -49,7 +49,7 @@ function mountScroll(deferCompletedTurns = true, initial: Partial false), loadOlder: vi.fn(), loadThrough: vi.fn(async () => {}), chatScroll: { read: () => null, save: vi.fn() }, @@ -116,9 +116,9 @@ function mountRows(heights: readonly number[]) { } vi.stubGlobal('ResizeObserver', Observer) const input: ChatScrollInput = { - ready: true, order: [], firstSeq: null, lastKey: null, lastIsUser: false, + isOpen: true, order: [], firstSeq: null, lastKey: null, lastIsUser: false, steeringId: null, submissionId: null, submissionSteers: false, running: false, deferCompletedTurns: false, - transcriptView: 'standard', loadedTurns: [], hasMore: false, loadingOlder: false, + transcriptView: 'standard', loadedTurns: [], hasMore: false, loadingOlder: false, historyLoadStatus: 'idle', consumeOlder: vi.fn(() => false), loadOlder: vi.fn(), loadThrough: vi.fn(async () => {}), chatScroll: { read: () => null, save: vi.fn() }, } const view = render( { callbacks.motion = motion }} @@ -287,9 +287,9 @@ function mountVisibleFold(test: typeof visibleFoldCases[number], reduced: boolea let motion: FlowMotionRows | undefined const onMotion = (value: FlowMotionRows): void => { motion = value } let input: ChatScrollInput = { - ready: false, order: [], firstSeq: null, lastKey: null, lastIsUser: false, + isOpen: false, order: [], firstSeq: null, lastKey: null, lastIsUser: false, steeringId: null, submissionId: null, submissionSteers: false, running: false, deferCompletedTurns: true, - transcriptView: 'detailed', loadedTurns: [], hasMore: false, loadingOlder: false, + transcriptView: 'detailed', loadedTurns: [], hasMore: false, loadingOlder: false, historyLoadStatus: 'idle', consumeOlder: vi.fn(() => false), loadOlder: vi.fn(), loadThrough: vi.fn(async () => {}), chatScroll: { read: () => null, save: vi.fn() }, } vi.stubGlobal('matchMedia', vi.fn(() => ({ matches: reduced }))) @@ -328,7 +328,7 @@ function mountVisibleFold(test: typeof visibleFoldCases[number], reduced: boolea offsetHeight: { get: contentHeight }, getBoundingClientRect: { value: () => ({ top: -top, bottom: contentHeight() - top, height: contentHeight() }) }, }) - input = { ...input, ready: true } + input = { ...input, isOpen: true } view.rerender() top = test.top scrollTo.mockClear() @@ -495,7 +495,7 @@ it('keeps one active fold when the local echo becomes its durable input', async describe('Chat scroll collapse timing', () => { it.each(['resize', 'scroll', 'history', 'input', 'preference'] as const)( 'waits for initial history before initializing a running view after %s', (change) => { - const h = mountScroll(true, { ready: false, running: true }) + const h = mountScroll(true, { isOpen: false, running: true }) expect(h.scroller.dataset.initialized).toBe('false') if (change === 'resize') h.resize() else if (change === 'scroll') fireEvent.scroll(h.scroller) @@ -506,7 +506,7 @@ describe('Chat scroll collapse timing', () => { expect(h.scroller.scrollTop).toBe(0) expect(h.scrollTo).not.toHaveBeenCalled() - h.update({ ready: true }) + h.update({ isOpen: true }) expect(h.scroller.dataset.initialized).toBe('true') expect(h.scroller.scrollTop).toBe(600) expect(h.scrollTo).not.toHaveBeenCalled() diff --git a/packages/client/ui-chat/tests/chat-view.client.spec.tsx b/packages/client/ui-chat/tests/chat-view.client.spec.tsx index 37011ab8bb..0d66887de6 100644 --- a/packages/client/ui-chat/tests/chat-view.client.spec.tsx +++ b/packages/client/ui-chat/tests/chat-view.client.spec.tsx @@ -107,6 +107,8 @@ function sessionSnapshot(overrides: Partial = {}): TestSess openError: null, hasMore: false, loadingOlder: false, + historyLoadStatus: 'idle', + historyWasPartial: overrides.hasMore ?? false, promptError: null, blank: false, subagent: null, @@ -421,7 +423,22 @@ function makeHarness( const openFile = vi.fn<(path: string) => Promise>().mockResolvedValue(undefined) const openUserFile = vi.fn<(path: string) => void>() const renderUserContent = createUserContentFixture({ openFile: openUserFile }) - const loadOlder = vi.fn() + let preparedOlder: (() => void) | null = null + const loadOlder = vi.fn(() => { + session.set({ loadingOlder: true, historyLoadStatus: 'loading' }) + }) + const consumeOlder = vi.fn(() => { + if (preparedOlder === null) return false + const consume = preparedOlder + preparedOlder = null + consume() + session.set({ loadingOlder: false, historyLoadStatus: 'idle' }) + return true + }) + const prepareOlder = (consume: () => void) => { + preparedOlder = consume + session.set({ loadingOlder: false, historyLoadStatus: 'pendingConsumption' }) + } const loadThrough = vi.fn<(seq: number) => Promise>().mockResolvedValue(undefined) // Mutable outline holder: tests swap the value and drive a re-render via set(). let outlineValue: unknown @@ -555,6 +572,7 @@ function makeHarness( openFile, openExternalLink: vi.fn(), loadOlder, + consumeOlder, loadThrough, loadImage: vi.fn(() => Promise.reject(new Error('not used'))), chatScroll, @@ -584,7 +602,7 @@ function makeHarness( return { set, setSession: session.set, setChat: chatSource.set, ChatView, props, setConnection: (state: ConnectionState) => { connection.set(state) }, - openFile, openUserFile, loadOlder, loadThrough, openView, + openFile, openUserFile, loadOlder, consumeOlder, prepareOlder, loadThrough, openView, setOutline: (value: unknown) => { outlineValue = value }, chatScroll, forkAt, toolOwners, fenceCalls, setPerformanceUsage: (mode: 'compact' | 'detailed') => { performanceUsage.set(mode) }, @@ -643,6 +661,11 @@ function readerScroll(element: HTMLElement, top: number): void { fireEvent(element, new Event('scrollend')) } +/** Let the public history scheduler run one frame, including React's commit of its updates. */ +async function historyFrame(): Promise { + await act(async () => { await new Promise(resolve => requestAnimationFrame(() => { resolve() })) }) +} + function turnProcessControl(container: HTMLElement): HTMLButtonElement | null { return container.querySelector('[data-turn-process]') } @@ -1537,6 +1560,19 @@ describe('ChatView', () => { await act(async () => { releaseJump?.() }) }) + it('keeps the history start marker across remounts from the Session fact', () => { + const h = makeHarness({ nodes: [user(1, 'prompt'), assistant(2, 'answer')] }, { + hasMore: false, historyWasPartial: true, + }) + const first = render() + expect(first.getByRole('button', { name: '已到会话开头' }).hasAttribute('disabled')).toBe(true) + first.unmount() + const second = render() + expect(second.getByRole('button', { name: '已到会话开头' }).hasAttribute('disabled')).toBe(true) + act(() => { h.setSession({ historyWasPartial: false }) }) + expect(second.queryByRole('button', { name: '已到会话开头' })).toBeNull() + }) + it('holds a jump issued while a plain pull owns the pager and resumes it when the pull settles', async () => { const later = [userInTurn(8, 'third prompt', 3), assistant(9, 'third response', 3)] const h = makeHarness({ nodes: later }, { hasMore: true, loadingOlder: true }) @@ -1642,7 +1678,7 @@ describe('ChatView', () => { expect(h.toolOwners[0]).toMatchObject({ callId: 'w1', toolName: '' }) }) - it('prepend keeps the reader\'s latest pending-request scroll position anchored', () => { + it('prepend keeps the reader\'s latest pending-request scroll position anchored', async () => { const h = makeHarness( { nodes: [user(9, 'first visible'), user(10, 'next visible')] }, { hasMore: true }, @@ -1666,17 +1702,19 @@ describe('ChatView', () => { Object.defineProperty(scroller, 'clientHeight', { value: 200, writable: true }) readerScroll(scroller, 50) fireEvent.click(view.getByText('加载更早')) - act(() => { h.set({ loadingOlder: true }) }) + await historyFrame() // The reader moves after the request starts; this, not the click-time // row, is the intent the arriving page must preserve. firstTop = -200 nextTop = 60 readerScroll(scroller, 90) - Object.defineProperty(scroller, 'scrollHeight', { value: 1300, writable: true }) - nextTop = 560 - act(() => { - h.set({ nodes: [assistant(2, 'older'), user(9, 'first visible'), user(10, 'next visible')], loadingOlder: false }) - }) + act(() => { h.prepareOlder(() => { + Object.defineProperty(scroller, 'scrollHeight', { value: 1300, writable: true }) + nextTop = 560 + h.set({ nodes: [assistant(2, 'older'), user(9, 'first visible'), user(10, 'next visible')] }) + }) }) + await historyFrame() + expect(h.consumeOlder).toHaveBeenCalledOnce() expect(scroller.scrollTop).toBe(590) // latest 90 + the anchored row's 500px prepend shift }) @@ -1685,7 +1723,7 @@ describe('ChatView', () => { { mode: 'detailed', anchor: 'group' }, { mode: 'compact', anchor: 'node' }, { mode: 'standard', anchor: 'node' }, { mode: 'detailed', anchor: 'node' }, ] as const)( - 'keeps the first visible item in place when paging inserts a steering boundary ($mode, $anchor)', ({ mode, anchor }) => { + 'keeps the first visible item in place when paging inserts a steering boundary ($mode, $anchor)', async ({ mode, anchor }) => { const tool = (seq: number, id: string) => ({ ...toolResult(seq, id), turn: 1 }) const initial = { nodes: [tool(4, 'retained'), assistant(6, 'answer', 1, 2)], turnEnds: new Map([[1, 7]]) } let source = chatSnapshotFixture(initial) @@ -1726,14 +1764,17 @@ describe('ChatView', () => { expect(pinned.hasAttribute('data-chat-paging-anchor')).toBe(true) expect(controlSeat.hasAttribute('data-chat-paging-anchor')).toBe(false) expect(group.hasAttribute('data-chat-paging-anchor')).toBe(anchor !== 'node') - act(() => { + await historyFrame() + act(() => { h.prepareOlder(() => { paged = true source = chatSnapshotFixture({ ...initial, nodes: [ tool(1, 'earlier'), steering(2, 'historical direction', 1), tool(3, 'before-retained'), ...initial.nodes, ] }, source) h.set({ chat: installGroupedSnapshot(builder, state, groups, source) }) h.setGrouped(groups) - }) + }) }) + await historyFrame() + expect(h.consumeOlder).toHaveBeenCalledOnce() expect(pinned.getBoundingClientRect().top).toBe(expectedTop) expect(scroller.scrollTop).toBe(expectedScroll) @@ -1757,7 +1798,7 @@ describe('ChatView', () => { { mode: 'compact', hasProcess: false, hasSteering: true }, { mode: 'standard', hasProcess: false, hasSteering: true }, { mode: 'detailed', hasProcess: false, hasSteering: true }, - ] as const)('keeps the loaded message anchored when paging exposes steering inside a closed Turn ($mode, process=$hasProcess, steer=$hasSteering)', ({ mode, hasProcess, hasSteering }) => { + ] as const)('keeps the loaded message anchored when paging exposes steering inside a closed Turn ($mode, process=$hasProcess, steer=$hasSteering)', async ({ mode, hasProcess, hasSteering }) => { const tool = (seq: number, id: string) => ({ ...toolResult(seq, id), turn: 1 }) const initial = { nodes: [ @@ -1801,13 +1842,16 @@ describe('ChatView', () => { expect(control.hasAttribute('data-chat-paging-anchor')).toBe(false) fireEvent.click(view.getByText('加载更早')) - act(() => { + await historyFrame() + act(() => { h.prepareOlder(() => { source = chatSnapshotFixture({ ...initial, nodes: [ tool(1, 'earlier-work'), steering(2, 'earlier direction', 1), tool(3, 'following-work'), ...initial.nodes, ] }, source) h.set({ chat: installGroupedSnapshot(builder, state, groups, source) }) h.setGrouped(groups) - }) + }) }) + await historyFrame() + expect(h.consumeOlder).toHaveBeenCalledOnce() const steer = view.container.querySelector('[data-chat-node-key="fixture:steering:2"]')! const rows = visibleRows() @@ -1854,7 +1898,7 @@ describe('ChatView', () => { } }) - it('anchors the first transcript row for paging regardless of the viewport reading line', () => { + it('anchors the first transcript row for a prepared page consumed at the top without reading hit tests', async () => { const originalHitTest = Object.getOwnPropertyDescriptor(document, 'elementsFromPoint') const nodes = Array.from({ length: 16 }, (_, index) => user(20 + index, `row ${String(index)}`)) const h = makeHarness( @@ -1873,13 +1917,13 @@ describe('ChatView', () => { vi.spyOn(row, 'getBoundingClientRect').mockImplementation(() => { rowRectCalls += 1 const shift = prepended ? (index === 8 ? 400 : 500) : 0 - const top = 20 + (index - 8) * 60 + shift + const top = 20 + index * 60 + shift - scroller.scrollTop return { top, bottom: top + 40 } as DOMRect }) }) Object.defineProperty(scroller, 'scrollHeight', { value: 800, writable: true }) Object.defineProperty(scroller, 'clientHeight', { value: 200, writable: true }) - readerScroll(scroller, 50) + readerScroll(scroller, 0) const hitTest = vi.fn((_x: number, _y: number): Element[] => []) Object.defineProperty(document, 'elementsFromPoint', { @@ -1890,14 +1934,19 @@ describe('ChatView', () => { rowRectCalls = 0 fireEvent.click(view.getByText('加载更早')) expect(hitTest).not.toHaveBeenCalled() - expect(rowRectCalls).toBe(1) - - Object.defineProperty(scroller, 'scrollHeight', { value: 1_300, writable: true }) - prepended = true - act(() => { + expect(rowRectCalls).toBe(0) + await historyFrame() + act(() => { h.prepareOlder(() => { + expect(hitTest).not.toHaveBeenCalled() + expect(rowRectCalls).toBe(1) + Object.defineProperty(scroller, 'scrollHeight', { value: 1_300, writable: true }) + prepended = true h.setChat({ nodes: [assistant(2, 'older'), ...nodes] }) - }) - expect(scroller.scrollTop).toBe(550) + }) }) + await historyFrame() + expect(h.consumeOlder).toHaveBeenCalledOnce() + expect(scroller.scrollTop).toBe(500) + expect(rows[0]!.getBoundingClientRect().top).toBe(20) expect(h.chatScroll.read()?.anchorKey).toBe('fixture:user:20') } finally { if (originalHitTest !== undefined) { @@ -4060,7 +4109,7 @@ describe('ChatView', () => { expect(screen.queryByRole('dialog')).toBeNull() }) - it('prepend preserves a semantic row; a trailing user node force-scrolls', () => { + it('prepend preserves a semantic row; a trailing user node force-scrolls', async () => { const h = makeHarness( { nodes: [user(5, 'later'), assistant(6, 'a')] }, { hasMore: true }, @@ -4076,13 +4125,15 @@ describe('ChatView', () => { () => ({ top: anchoredTop, bottom: anchoredTop + 40 } as DOMRect), ) readerScroll(scroller, 80) - // Arm the paging anchor, then deliver an older page (head seq decreases). fireEvent.click(view.getByText('加载更早')) - Object.defineProperty(scroller, 'scrollHeight', { value: 1600, writable: true }) - anchoredTop = 700 - act(() => { + await historyFrame() + act(() => { h.prepareOlder(() => { + Object.defineProperty(scroller, 'scrollHeight', { value: 1600, writable: true }) + anchoredTop = 700 h.setChat({ nodes: [user(1, 'old'), assistant(2, 'b'), user(5, 'later'), assistant(6, 'a')] }) - }) + }) }) + await historyFrame() + expect(h.consumeOlder).toHaveBeenCalledOnce() expect(scroller.scrollTop).toBe(680) // reader offset 80 + the anchored row's 600px shift // A new trailing user bubble (own words) force-scrolls to the bottom. act(() => { @@ -4093,7 +4144,45 @@ describe('ChatView', () => { expect(scroller.scrollTop).toBe(1200) }) - it('back-to-bottom cancels an in-flight paging anchor', () => { + it('restores a one-shot prepend landing after immediate remount before scroll delivery', async () => { + const host = document.createElement('div') + host.dataset.conversationScroll = '' + document.body.append(host) + let paged = false + Object.defineProperties(host, { + clientHeight: { value: 200 }, + scrollHeight: { get: () => paged ? 1200 : 200 }, + }) + vi.spyOn(HTMLElement.prototype, 'getBoundingClientRect').mockImplementation(function (this: HTMLElement) { + if (this === host) return new DOMRect(0, 0, 500, 200) + const offset = this.dataset.chatAnchorKey === 'fixture:user:5' ? 20 + (paged ? 200 : 0) : 0 + return new DOMRect(0, offset - host.scrollTop, 500, 40) + }) + try { + const h = makeHarness({ nodes: [user(5, 'retained prompt'), assistant(6, 'answer')] }, { hasMore: true }) + const view = render(, { container: host }) + expect(host.scrollTop).toBe(0) + expect(h.chatScroll.read()).toBeNull() + fireEvent.click(view.getByRole('button', { name: '加载更早' })) + await historyFrame() + act(() => { h.prepareOlder(() => { + paged = true + h.setChat({ nodes: [user(1, 'earlier'), assistant(2, 'old answer'), user(5, 'retained prompt'), assistant(6, 'answer')] }) + }) }) + await historyFrame() + expect(host.scrollTop).toBe(200) + // A navigation can remove the view before the browser delivers the scroll caused by compensation. + view.rerender(
) + host.scrollTop = 0 + view.rerender() + expect(host.scrollTop).toBe(200) + expect(h.chatScroll.read()).toMatchObject({ anchorKey: 'fixture:user:5', anchorTop: 20, scrollTop: 200 }) + } finally { + host.remove() + } + }) + + it('back-to-bottom cancels pending history consumption without reviving it when the page arrives', async () => { const h = makeHarness({ nodes: [user(9, 'late')] }, { hasMore: true }) const view = render() const scroller = view.container.querySelector('[data-chat-flow]')!.parentElement as HTMLDivElement @@ -4101,10 +4190,16 @@ describe('ChatView', () => { Object.defineProperty(scroller, 'clientHeight', { value: 200, writable: true }) readerScroll(scroller, 50) fireEvent.click(view.getByText('加载更早')) + await historyFrame() fireEvent.click(view.getByLabelText('回到底部')) - Object.defineProperty(scroller, 'scrollHeight', { value: 1_300, writable: true }) - act(() => { h.setChat({ nodes: [assistant(2, 'older'), user(9, 'late')] }) }) - expect(scroller.scrollTop).toBe(1_100) + act(() => { h.prepareOlder(() => { + Object.defineProperty(scroller, 'scrollHeight', { value: 1_300, writable: true }) + h.setChat({ nodes: [assistant(2, 'older'), user(9, 'late')] }) + }) }) + await historyFrame() + expect(h.consumeOlder).not.toHaveBeenCalled() + expect(scroller.scrollTop).toBe(600) + expect(view.queryByText('older')).toBeNull() expect(h.chatScroll.read()).toBeNull() }) @@ -4598,13 +4693,15 @@ describe('ChatView', () => { } }) - it('paging button loads older and shows its busy label', () => { + it('paging button schedules an older request and shows its loading label', async () => { const h = makeHarness({ nodes: [user(5, 'later')] }, { hasMore: true }) const view = render() fireEvent.click(view.getByText('加载更早')) + expect(h.loadOlder).not.toHaveBeenCalled() + expect(view.getByRole('button', { name: '正在处理' }).hasAttribute('disabled')).toBe(true) + await historyFrame() expect(h.loadOlder).toHaveBeenCalledTimes(1) - act(() => { h.setSession({ loadingOlder: true }) }) - expect(view.getByText('加载中…')).toBeTruthy() + expect(view.getByRole('button', { name: '正在加载' }).hasAttribute('disabled')).toBe(true) }) it('shows open error and loading states', () => { diff --git a/packages/client/ui-chat/tests/chat-viewport.client.spec.ts b/packages/client/ui-chat/tests/chat-viewport.client.spec.ts index def772efb8..6149f9a89d 100644 --- a/packages/client/ui-chat/tests/chat-viewport.client.spec.ts +++ b/packages/client/ui-chat/tests/chat-viewport.client.spec.ts @@ -48,6 +48,100 @@ function fixture() { return { column, viewport, group, row, prepend: (height: number) => { prependedHeight += height } } } +it('updates history distance only from scrollport size observations before layout writes', () => { + let resize: ResizeObserverCallback | undefined + class Observer implements ResizeObserver { + constructor(callback: ResizeObserverCallback) { resize = callback } + observe(): void {} + unobserve(): void {} + disconnect(): void {} + } + vi.stubGlobal('ResizeObserver', Observer) + onTestFinished(() => { vi.unstubAllGlobals() }) + const h = fixture() + const edge = document.createElement('div') + h.column.prepend(edge) + const height = vi.fn(() => 300) + Object.defineProperty(h.column, 'clientHeight', { configurable: true, get: height }) + h.viewport.setHistoryEdge(edge) + expect(height).not.toHaveBeenCalled() + let afterWrite = false + vi.spyOn(h.viewport, 'reclaimBelow').mockImplementation(() => { afterWrite = true }) + height.mockImplementation(() => { + if (afterWrite) throw new Error('height read after a layout writer') + return 300 + }) + const entry: ResizeObserverEntry = { target: h.column, contentRect: new DOMRect(), + borderBoxSize: [], contentBoxSize: [], devicePixelContentBoxSize: [] } + const notify = resize + const observer = new Observer(() => {}) + notify?.([entry], observer) + height.mockClear() + notify?.([], observer) + vi.spyOn(edge, 'getBoundingClientRect').mockReturnValue(new DOMRect(0, -600, 20, 10)) + expect(h.viewport.confirmHistoryStart()).toBe(true) + vi.spyOn(edge, 'getBoundingClientRect').mockReturnValue(new DOMRect(0, -700, 20, 10)) + expect(h.viewport.confirmHistoryStart()).toBe(false) + expect(height).not.toHaveBeenCalled() +}) + +it('uses the first paging anchor for one-shot history prepending without reading-position lookup', () => { + const h = fixture() + const row = h.row(h.column, 'old', 0) + const scrollerRect = vi.spyOn(h.column, 'getBoundingClientRect') + const capture = vi.spyOn(h.viewport, 'capturePosition') + const query = vi.spyOn(h.column, 'querySelector') + expect(h.viewport.beginHistoryPrepend()).toBe(true) + expect(capture).not.toHaveBeenCalled() + expect(query).toHaveBeenCalledTimes(1) + expect(scrollerRect).toHaveBeenCalledTimes(1) + h.prepend(120) + h.viewport.finishHistoryPrepend() + expect(h.column.scrollTop).toBe(120) + expect(row.getBoundingClientRect().top).toBe(0) + h.prepend(30) + h.viewport.finishHistoryPrepend() + expect(h.column.scrollTop).toBe(120) +}) + +it('uses a collapsed group paging anchor when no ordinary content row is visible', () => { + const h = fixture() + const group = h.row(h.column, 'group', 0) + group.dataset.chatGroupKey = 'group' + const body = document.createElement('div') + body.hidden = true + group.append(body) + h.row(body, 'hidden-member', 20) + expect(h.viewport.capturePosition()).toBeNull() + expect(h.viewport.beginHistoryPrepend()).toBe(true) + h.prepend(120) + h.viewport.finishHistoryPrepend() + expect(h.column.scrollTop).toBe(120) + expect(group.getBoundingClientRect().top).toBe(0) + expect(h.viewport.preserving).toBe(false) +}) + +it('keeps reader movement during a one-shot prepend without retaining paging ownership', () => { + const h = fixture() + const row = h.row(h.column, 'old', 0) + expect(h.viewport.beginHistoryPrepend()).toBe(true) + expect(h.viewport.preserving).toBe(false) + h.column.scrollTop = 25 + h.prepend(120) + h.viewport.finishHistoryPrepend() + expect(h.column.scrollTop).toBe(145) + expect(row.getBoundingClientRect().top).toBe(-25) + expect(h.viewport.preserving).toBe(false) +}) + +it('does not prepare a correction when no paging anchor exists', () => { + const h = fixture() + expect(h.viewport.beginHistoryPrepend()).toBe(false) + h.prepend(120) + h.viewport.finishHistoryPrepend() + expect(h.column.scrollTop).toBe(0) +}) + it.each([ { count: 64, slot: false }, { count: 4096, slot: false }, { count: 64, slot: true }, { count: 4096, slot: true }, @@ -440,6 +534,42 @@ it.each([ expect(h.row.getBoundingClientRect().top).toBe(top) }) +it('applies one-shot inner compensation without subscribing or undoing reader movement', () => { + const h = nestedFixture(600, 400) + h.scroller.scrollTop = 80 + h.body.scrollTop = 40 + const screenTop = h.row.getBoundingClientRect().top + expect(h.viewport.beginHistoryPrepend()).toBe(true) + expect(h.viewport.preserving).toBe(false) + expect(h.observed.has(h.content)).toBe(false) + h.body.scrollTop = 60 + h.scroller.scrollTop = 90 + h.prepend(200) + h.viewport.finishHistoryPrepend() + expect(h.body.scrollTop).toBe(260) + expect(h.scroller.scrollTop).toBe(90) + expect(h.row.getBoundingClientRect().top).toBe(screenTop - 30) + expect(h.observed.has(h.content)).toBe(false) + h.prepend(20) + h.resize() + expect(h.body.scrollTop).toBe(260) + expect(h.scroller.scrollTop).toBe(90) +}) + +it('resolves a remounted one-shot anchor without entering retained paging', () => { + const h = fixture() + const row = h.row(h.column, 'old', 0) + expect(h.viewport.beginHistoryPrepend()).toBe(true) + row.remove() + const replacement = h.row(h.column, 'old', 0) + h.column.scrollTop = 25 + h.prepend(120) + h.viewport.finishHistoryPrepend() + expect(h.column.scrollTop).toBe(145) + expect(replacement.getBoundingClientRect().top).toBe(-25) + expect(h.viewport.preserving).toBe(false) +}) + it('gives paging compensation priority over an active inner follow animation', () => { const h = nestedFixture(600, 400) const follow = new ScrollFollow(true, 1) diff --git a/packages/client/ui-conversation/tests/conversation-registry.client.spec.ts b/packages/client/ui-conversation/tests/conversation-registry.client.spec.ts index 9a53949566..9685b65f12 100644 --- a/packages/client/ui-conversation/tests/conversation-registry.client.spec.ts +++ b/packages/client/ui-conversation/tests/conversation-registry.client.spec.ts @@ -39,7 +39,7 @@ function sessionSnapshot(): SessionSnapshot { openState: 'open', openError: null, hasMore: false, - loadingOlder: false, + loadingOlder: false, historyLoadStatus: 'idle', historyWasPartial: false, promptError: null, blank: true, lastAgentError: null, @@ -62,6 +62,7 @@ function fakeSession(): SessionFace { cancel: () => Promise.reject(new Error('unused fake Session operation')), rename: () => Promise.reject(new Error('unused fake Session operation')), loadOlder: () => Promise.reject(new Error('unused fake Session operation')), + consumeOlder: () => { throw new Error('unused fake Session operation') }, loadThrough: () => Promise.reject(new Error('unused fake Session operation')), command: () => Promise.reject(new Error('unused fake Session operation')), } diff --git a/packages/client/ui-conversation/tests/queue-dock.client.spec.tsx b/packages/client/ui-conversation/tests/queue-dock.client.spec.tsx index 8a116a9a15..713c74df88 100644 --- a/packages/client/ui-conversation/tests/queue-dock.client.spec.tsx +++ b/packages/client/ui-conversation/tests/queue-dock.client.spec.tsx @@ -56,7 +56,7 @@ interface TestSnapshot extends SessionSnapshot { function snapshotWith(queue: UserMessage[], nextStep: UserMessage[] = []): TestSnapshot { return { sessionId: SID, running: true, removed: false, openState: 'open', openError: null, - hasMore: false, loadingOlder: false, promptError: null, blank: false, subagent: null, + hasMore: false, loadingOlder: false, historyLoadStatus: 'idle', historyWasPartial: false, promptError: null, blank: false, subagent: null, pendingSubmissions: [], lastAgentError: null, promptAttempted: true, awaitingFirstTurn: false, testInbox: { 'next-turn': queue, 'next-step': nextStep }, diff --git a/packages/client/ui-schedule/tests/schedule-catalog-action.client.spec.tsx b/packages/client/ui-schedule/tests/schedule-catalog-action.client.spec.tsx index b6885853b4..8d670df9da 100644 --- a/packages/client/ui-schedule/tests/schedule-catalog-action.client.spec.tsx +++ b/packages/client/ui-schedule/tests/schedule-catalog-action.client.spec.tsx @@ -117,7 +117,7 @@ function sessionSnapshot(openState: SessionSnapshot['openState']): SessionSnapsh openState, openError: null, hasMore: false, - loadingOlder: false, + loadingOlder: false, historyLoadStatus: 'idle', historyWasPartial: false, promptError: null, blank: false, lastAgentError: null, diff --git a/packages/client/ui-session/tests/ui-session.client.spec.ts b/packages/client/ui-session/tests/ui-session.client.spec.ts index c12901a0d8..6afbe4be49 100644 --- a/packages/client/ui-session/tests/ui-session.client.spec.ts +++ b/packages/client/ui-session/tests/ui-session.client.spec.ts @@ -96,7 +96,7 @@ function createSessionsBench(ctx: Context): SessionsBench { openState: 'open', openError: null, hasMore: false, - loadingOlder: false, + loadingOlder: false, historyLoadStatus: 'idle', historyWasPartial: false, promptError: null, blank: false, lastAgentError: null, diff --git a/packages/client/ui-trajectory/tests/views.client.spec.tsx b/packages/client/ui-trajectory/tests/views.client.spec.tsx index c6dfd004d4..e0ee44c80a 100644 --- a/packages/client/ui-trajectory/tests/views.client.spec.tsx +++ b/packages/client/ui-trajectory/tests/views.client.spec.tsx @@ -126,7 +126,7 @@ function sessionSnapshot(nodes: LegacyConversationSlice['nodes']): SessionSnapsh openState: 'open', openError: null, hasMore: false, - loadingOlder: false, + loadingOlder: false, historyLoadStatus: 'idle', historyWasPartial: false, promptError: null, blank: nodes.length === 0, lastAgentError: null, diff --git a/packages/client/ui-user-questions/tests/plan-review-panel.client.spec.tsx b/packages/client/ui-user-questions/tests/plan-review-panel.client.spec.tsx index 41e0e6902f..31b9296885 100644 --- a/packages/client/ui-user-questions/tests/plan-review-panel.client.spec.tsx +++ b/packages/client/ui-user-questions/tests/plan-review-panel.client.spec.tsx @@ -43,7 +43,7 @@ const sessionState: SessionState = { openState: 'open', openError: null, hasMore: false, - loadingOlder: false, + loadingOlder: false, historyLoadStatus: 'idle', historyWasPartial: false, promptError: null, blank: false, lastAgentError: null, diff --git a/packages/client/ui-user-questions/tests/user-questions-composer.client.spec.tsx b/packages/client/ui-user-questions/tests/user-questions-composer.client.spec.tsx index b3fdffda6b..3e924b4576 100644 --- a/packages/client/ui-user-questions/tests/user-questions-composer.client.spec.tsx +++ b/packages/client/ui-user-questions/tests/user-questions-composer.client.spec.tsx @@ -49,7 +49,7 @@ const sessionState: SessionState = { openState: 'open', openError: null, hasMore: false, - loadingOlder: false, + loadingOlder: false, historyLoadStatus: 'idle', historyWasPartial: false, promptError: null, blank: false, lastAgentError: null, diff --git a/packages/extensions/cordis-client-runner/src/client/api-catalog.ts b/packages/extensions/cordis-client-runner/src/client/api-catalog.ts index 1d444f3c2f..4884a2c208 100644 --- a/packages/extensions/cordis-client-runner/src/client/api-catalog.ts +++ b/packages/extensions/cordis-client-runner/src/client/api-catalog.ts @@ -677,7 +677,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'ISession', - declaration: 'export interface ISession {\n readonly sessionId: SessionId;\n readonly projections: ProjectionsFace;\n beginSubmission(input: BeginSubmissionInput): SubmissionHandle;\n prompt(content: PromptContentPart[], mode: \'queue\' | \'steer\', signal?: AbortSignal, requestId?: SessionRequestId): Promise>;\n readAttachment(attachment: ImageAttachmentRef): Promise>;\n updateQueue(itemId: MessageId, action: QueueAction): Promise>;\n cancel(): Promise>;\n rename(title: string): Promise>;\n loadOlder(): Promise;\n loadThrough(seq: SessionSeq): Promise;\n command(line: string): Promise>;\n}', + declaration: 'export interface ISession {\n readonly sessionId: SessionId;\n readonly projections: ProjectionsFace;\n beginSubmission(input: BeginSubmissionInput): SubmissionHandle;\n prompt(content: PromptContentPart[], mode: \'queue\' | \'steer\', signal?: AbortSignal, requestId?: SessionRequestId): Promise>;\n readAttachment(attachment: ImageAttachmentRef): Promise>;\n updateQueue(itemId: MessageId, action: QueueAction): Promise>;\n cancel(): Promise>;\n rename(title: string): Promise>;\n loadOlder(options?: {\n deferConsumption?: boolean;\n }): Promise;\n consumeOlder(): boolean;\n loadThrough(seq: SessionSeq): Promise;\n command(line: string): Promise>;\n}', }, { name: 'ISessions', @@ -969,7 +969,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'SessionSnapshot', - declaration: 'export interface SessionSnapshot {\n readonly sessionId: SessionId;\n readonly pendingSubmissions: readonly PendingSubmission[];\n readonly running: boolean;\n readonly subagent: {\n readonly address: SubagentAddress;\n } | null;\n readonly removed: boolean;\n readonly openState: OpenState;\n readonly openError: RemoteFailure | null;\n readonly hasMore: boolean;\n readonly loadingOlder: boolean;\n readonly promptError: PromptError | null;\n readonly blank: boolean;\n readonly lastAgentError: string | null;\n readonly promptAttempted: boolean;\n readonly awaitingFirstTurn: boolean;\n}', + declaration: 'export interface SessionSnapshot {\n readonly sessionId: SessionId;\n readonly pendingSubmissions: readonly PendingSubmission[];\n readonly running: boolean;\n readonly subagent: {\n readonly address: SubagentAddress;\n } | null;\n readonly removed: boolean;\n readonly openState: OpenState;\n readonly openError: RemoteFailure | null;\n readonly hasMore: boolean;\n readonly historyWasPartial: boolean;\n readonly loadingOlder: boolean;\n readonly historyLoadStatus: \'idle\' | \'loading\' | \'pendingConsumption\' | \'error\';\n readonly promptError: PromptError | null;\n readonly blank: boolean;\n readonly lastAgentError: string | null;\n readonly promptAttempted: boolean;\n readonly awaitingFirstTurn: boolean;\n}', }, { name: 'SessionStandardProps', diff --git a/packages/extensions/cordis-client-runner/src/client/slot-catalog.ts b/packages/extensions/cordis-client-runner/src/client/slot-catalog.ts index 41f4010684..a882ea0ca9 100644 --- a/packages/extensions/cordis-client-runner/src/client/slot-catalog.ts +++ b/packages/extensions/cordis-client-runner/src/client/slot-catalog.ts @@ -278,7 +278,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'none', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.assistant-actions\', () => ctx.slots.register(\n { name: \'conversation.chat.assistant-actions\', id: \'my-entry\', order: 100, label: \'My entry\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:495', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:498', }, { key: 'conversation.chat.commandview', @@ -332,7 +332,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'shadows-shipped-ui', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.commandview\', () => ctx.slots.register(\n { name: \'conversation.chat.commandview\', key: \'\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:477', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:480', }, { key: 'conversation.chat.flow', @@ -379,7 +379,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'shadows-shipped-ui', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.flow\', () => ctx.slots.register(\n { name: \'conversation.chat.flow\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:433', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:436', }, { key: 'conversation.chat.markdown.fence', @@ -445,7 +445,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'wraps-owner-content', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.markdown.fence\', () => ctx.slots.register(\n { name: \'conversation.chat.markdown.fence\', id: \'my-layer\', order: 100 },\n ({ children }) => React.createElement(\'div\', null, children),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:430', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:433', }, { key: 'conversation.chat.message.images', @@ -492,7 +492,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'shadows-shipped-ui', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.message.images\', () => ctx.slots.register(\n { name: \'conversation.chat.message.images\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:469', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:472', }, { key: 'conversation.chat.node', @@ -564,7 +564,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'shadows-shipped-ui', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.node\', () => ctx.slots.register(\n { name: \'conversation.chat.node\', key: \'\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:445', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:448', }, { key: 'conversation.chat.reasoning.body', @@ -608,7 +608,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'shadows-shipped-ui', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.reasoning.body\', () => ctx.slots.register(\n { name: \'conversation.chat.reasoning.body\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:483', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:486', }, { key: 'conversation.chat.text-interaction', @@ -660,7 +660,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'shadows-shipped-ui', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.text-interaction\', () => ctx.slots.register(\n { name: \'conversation.chat.text-interaction\', key: \'\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:428', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:431', }, { key: 'conversation.chat.turnTail', @@ -727,7 +727,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'none', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.turnTail\', () => ctx.slots.register(\n { name: \'conversation.chat.turnTail\', id: \'my-entry\', order: 100, label: \'My entry\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:489', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:492', }, { key: 'conversation.chat.user.before', @@ -777,7 +777,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'none', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.user.before\', () => ctx.slots.register(\n { name: \'conversation.chat.user.before\', select: owner => null },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:459', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:462', }, { key: 'conversation.chat.user.images', @@ -824,7 +824,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'shadows-shipped-ui', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'conversation.chat.user.images\', () => ctx.slots.register(\n { name: \'conversation.chat.user.images\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:471', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:474', }, { key: 'conversation.composer', @@ -4116,7 +4116,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'none', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'shell.quota-notice\', () => ctx.slots.register(\n { name: \'shell.quota-notice\', select: owner => null },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-chat/src/client/contract/slots.ts:503', + source: 'packages/client/ui-chat/src/client/contract/slots.ts:506', }, { key: 'sidebar', diff --git a/packages/session/message-quotes/tests/quote-selection.client.spec.tsx b/packages/session/message-quotes/tests/quote-selection.client.spec.tsx index 656f8909e3..d848a09db7 100644 --- a/packages/session/message-quotes/tests/quote-selection.client.spec.tsx +++ b/packages/session/message-quotes/tests/quote-selection.client.spec.tsx @@ -14,7 +14,7 @@ function props(enabled = true, subagent: SessionSnapshot['subagent'] = null) { const value = { trigger: 'selection', dismiss: vi.fn(), readText: vi.fn((): string | null => 'Hello world'), clearSelection: vi.fn(), - inputActions, useSession: selector => selector({ sessionId: SessionId('quote-selection'), pendingSubmissions: [], running: false, removed: !enabled, openState: 'open', openError: null, hasMore: false, loadingOlder: false, promptError: null, blank: false, subagent, lastAgentError: null, promptAttempted: true, awaitingFirstTurn: false }), + inputActions, useSession: selector => selector({ sessionId: SessionId('quote-selection'), pendingSubmissions: [], running: false, removed: !enabled, openState: 'open', openError: null, hasMore: false, loadingOlder: false, historyLoadStatus: 'idle', historyWasPartial: false, promptError: null, blank: false, subagent, lastAgentError: null, promptAttempted: true, awaitingFirstTurn: false }), t: () => 'Quote text', } as Props return { value, inputActions } diff --git a/packages/test-support/client-runtime/src/fixtures.ts b/packages/test-support/client-runtime/src/fixtures.ts index 21f4abef40..c103792d55 100644 --- a/packages/test-support/client-runtime/src/fixtures.ts +++ b/packages/test-support/client-runtime/src/fixtures.ts @@ -74,7 +74,7 @@ export function sessionSnapshot(sessionId: SessionId): SessionSnapshot { openState: 'open', openError: null, hasMore: false, - loadingOlder: false, + loadingOlder: false, historyLoadStatus: 'idle', historyWasPartial: false, promptError: null, blank: false, lastAgentError: null, diff --git a/packages/test-support/client-runtime/src/sessions.ts b/packages/test-support/client-runtime/src/sessions.ts index a9fadfe91b..0d5d70d0e2 100644 --- a/packages/test-support/client-runtime/src/sessions.ts +++ b/packages/test-support/client-runtime/src/sessions.ts @@ -162,6 +162,14 @@ export class FixtureSession implements SessionFace { throw new Error(`test session "${this.sessionId}": loadOlder is not stubbed — supply it on the fixture's session face`) } + /** + * Fail-loud stub; supply `consumeOlder` on the fixture's session face to exercise it. + * @returns never — always throws. + */ + consumeOlder(): never { + throw new Error(`test session "${this.sessionId}": consumeOlder is not stubbed — supply it on the fixture's session face`) + } + /** * Fail-loud stub; supply `loadThrough` on the fixture's session face to exercise it. * @returns never — always throws. From 33f5c3ba3e109a3a5270982da205c9c3045a96eb Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 6 Nov 2026 15:52:22 +0800 Subject: [PATCH 37/44] fix(ui-chat): reuse screen preservation for history prepends --- ...chat-history-consumption-details.i18n.yaml | 16 +++--- ...-11-06-chat-history-consumption-details.md | 8 +-- ...-06-chat-history-consumption-details.zh.md | 8 +-- ...6-11-06-chat-history-consumption.i18n.yaml | 8 +-- .../2026-11-06-chat-history-consumption.md | 4 +- .../2026-11-06-chat-history-consumption.zh.md | 4 +- packages/client/ui-chat/README.i18n.yaml | 4 +- packages/client/ui-chat/README.md | 2 +- packages/client/ui-chat/README.zh.md | 2 +- .../src/client/chat/use-chat-viewport.ts | 20 +++----- .../tests/chat-viewport.client.spec.ts | 51 +++++++++++++++---- 11 files changed, 78 insertions(+), 49 deletions(-) diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml index 4d1f5caf20..c0294f6e21 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml @@ -21,14 +21,14 @@ en: 993c2985bf6f3c8f zh: 8f88fb46584fb272 /agent-note-chat-history-consumption-classes-and-interactions/proposal/shared-anchors-and-layout-reads: - en: 2439cd4cb6a81bc4 - zh: 63b8c83897a88d2c + en: 3d59702a5b7021ea + zh: 25c9762647c7c0e0 /agent-note-chat-history-consumption-classes-and-interactions/alternatives-considered: - en: 1b9b8eb12ee5dbd7 - zh: e4ecc3cdc3375438 + en: 7b3b9b0c7b8f293b + zh: e20045fc3080418f /agent-note-chat-history-consumption-classes-and-interactions/acceptance-criteria: - en: 8a51c549570a5e0d - zh: eb7b3c27b9dcd98c + en: 1b22cf3616302960 + zh: 418ead8421a91bee /agent-note-chat-history-consumption-classes-and-interactions/risks: - en: aee9f541d058818b - zh: 74e4b6098b9f4b46 + en: 29e718a7d1c07a68 + zh: a81e5bc6c84b0712 diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md index ccc137d86b..a14fa89273 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md @@ -73,7 +73,7 @@ Prefetch reuses the existing 500ms reading sample, scrollend, content commits, a At `scrollTop ≤ 0.5`, selection uses `beginPaging`'s first paging-anchor rule without a reading-position hit test. `measurePaging()` only captures a `PagingPosition`; `retainPaging()` separately owns persistent preservation and inner-size observation. Stationary preservation and one-shot correction share remount resolution in `resolvePaging()` and inner/outer calculations in `applyPaging()`. -Persistent preservation restores the captured screen position. One-shot correction retains inner and outer scrolling since capture and compensates only layout displacement; it neither enters paging state nor subscribes to inner-size changes, and releases after one application. `finishHistoryPrepend()` returns the actual landing, and commit handling immediately saves it through `reading.preservePosition()` rather than waiting for a later scroll event. The actual conversation scrollport uses `overscroll-behavior-y: none`, without creating smooth scrolling or a new momentum animation. +Persistent preservation and one-shot correction use the same `applyPaging()` formula to restore the captured screen position, compensating the inner scrollport before the outer one. They do not add changes in scrollTop since capture: native anchoring may have caused those changes. Their lifetimes differ: one-shot correction neither enters paging state nor subscribes to inner-size changes, and releases after one application. `finishHistoryPrepend()` returns the actual landing, and commit handling immediately saves it through `reading.preservePosition()` rather than waiting for a later scroll event. The actual conversation scrollport uses `overscroll-behavior-y: none`, without creating smooth scrolling or a new momentum animation. Distance confirmation still needs current geometry, and one-shot correction after prepend still needs the new position. These reads serve correctness, but placement inside rAF, ResizeObserver, or commit handling alone does not prove they add no layout. Read frequency, read/write ordering, and actual Layout/Paint costs require separate verification; position stability is not performance evidence. @@ -85,16 +85,18 @@ Distance confirmation still needs current geometry, and one-shot correction afte **Store separate coordinates and continuously track the first row for one-shot correction.** That duplicates existing paging anchors and can expand scroll ownership; share the position record and calculations while keeping lifetimes separate. +**Add captured-to-current scroll deltas through `preserveMovement`.** The deltas cannot distinguish reader input from native anchoring for the same prepend, so adding them can compensate twice. + ## Acceptance criteria - Default single-stage calls, deferred consumption, and Turn jumps coexist; an old request cannot consume or overwrite a new one. - Ordinary events do not revive automatic fallback; touch/inner-scroll continuation, manual completion, and unmount cleanup have focused verification. -- One-shot correction does not register persistent observation or undo intervening reader movement; existing persistent and inner preservation do not regress. +- One-shot correction uses the existing screen-restoration formula without persistent observation or duplicate native compensation; persistent and inner preservation do not regress. - Distance checks use cached viewport height; raw scroll listeners and consumption admission perform no layout reads. ## Risks -**Known issue: `preserveMovement` cannot distinguish native anchoring from reader input.** The shared correction adds changes in the outer and inner scroll positions since capture. A deterministic geometry model confirms duplicate compensation when the browser has already anchored the same prepend: an outer `scrollTop` of `0.5`, or outer `0` with inner `40`, enters the one-shot path and treats the native adjustment as movement to preserve. The resulting additional displacement is an algorithmic limitation, not evidence that every browser exhibits it. Code remains unchanged for this report-only finding; independent actual-browser verification is pending. +Screen restoration uses the measured residual displacement after native anchoring rather than adding scroll-position deltas. It does not preserve arbitrary input movement between capture and application or guarantee an uninterrupted momentum trajectory. Near-zero outer positions, native inner anchoring, and input arriving around the prepend require current-build browser verification. Explicit navigation and multiple views share a Session, so Chat's automatic waiting policy must not move into the global event source. Manual intent can still wait if usable positioning never becomes available; actual views need cancellation, reconnect, and anchorless-structure verification. One-shot geometry measurement, native inner anchoring, and asynchronous content growth need browser evidence from the current build. diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md index 104b65e5a6..8cebfc2fa8 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md @@ -73,7 +73,7 @@ Session.consumeOlder 顶部 `scrollTop ≤ 0.5` 使用 `beginPaging` 的首条分页锚点规则，不先执行阅读位置命中检测。`measurePaging()` 只采集 `PagingPosition`；`retainPaging()` 单独拥有持续保位和内层尺寸观察。普通静止保位与一次补偿共用 `resolvePaging()` 的重挂载查找及 `applyPaging()` 的内外层计算。 -持续保位恢复采集时的屏幕位置。一次补偿保留采集后发生的内外层滚动，只抵消布局位移；它不进入 paging 状态、不注册内层尺寸观察，应用一次后释放。`finishHistoryPrepend()` 返回实际落点，提交处理立即通过 `reading.preservePosition()` 保存阅读位置，不依赖后续 scroll 事件。会话实际滚动容器设置 `overscroll-behavior-y: none`，不创建平滑滚动或新的惯性动画。 +持续保位和一次补偿用相同的 `applyPaging()` 公式恢复采集时的屏幕位置，先补内层、再补外层。计算不叠加采集后 scrollTop 的变化，因为这些变化可能已经来自原生锚定。两者只区分生命周期：一次补偿不进入 paging 状态、不注册内层尺寸观察，应用一次后释放。`finishHistoryPrepend()` 返回实际落点，提交处理立即通过 `reading.preservePosition()` 保存阅读位置，不依赖后续 scroll 事件。会话实际滚动容器设置 `overscroll-behavior-y: none`，不创建平滑滚动或新的惯性动画。 距离检查仍需要当前几何；前插后的一次补偿仍需要读取新位置。这些读数有正确性用途，但不能仅凭位于 rAF、ResizeObserver 或提交回调中，就认定不会增加布局。读取频率、读写顺序及实际 Layout/Paint 成本分别验收；位置不漂移不能替代性能证明。 @@ -85,16 +85,18 @@ Session.consumeOlder **为一次补偿另存坐标并持续追踪首行。** 与既有分页锚点重复，并可能扩大滚动控制；共享位置记录和计算，只区分生命周期。 +**通过 `preserveMovement` 叠加采集后滚动增量。** 该增量无法区分读者输入与同一次前插的原生锚定，叠加后可能重复补偿。 + ## Acceptance criteria - 默认单阶段调用、延迟消费及 Turn 跳转共存，旧请求不能消费或覆盖新请求。 - auto 回退不会被普通事件恢复；触摸与内层滚动续接、manual 完成承诺和卸载清理都有定向验证。 -- 一次补偿不注册持续观察、不撤销期间的读者滚动；既有持续保位和内层补偿不退化。 +- 一次补偿使用既有屏幕位置恢复公式，不注册持续观察、不重复原生补偿；持续保位和内层补偿不退化。 - 距离读取使用缓存视口高度；原始滚动监听与消费准入不读取布局。 ## Risks -**已知问题：`preserveMovement` 无法区分原生锚定与读者输入。** 共享补偿计算会加上采集后内外层滚动位置的变化。确定性的几何模型已确认：浏览器已为同一次前插完成锚定时，外层 `scrollTop` 为 `0.5`，或外层为 `0`、内层为 `40`，会进入一次补偿路径，把原生调整当成需要保留的移动，从而重复补偿。产生的额外位移是算法限制，不代表所有浏览器都会复现。本问题按仅报告处理，代码不变；独立真实浏览器验证仍待完成。 +屏幕位置恢复使用原生锚定后测得的剩余位移，而不是叠加滚动位置增量。它不保留采集到应用之间任意输入造成的移动，也不保证惯性轨迹连续。外层近零位置、内层原生锚定以及前插附近到达的输入，都需要当前构建的浏览器验证。 显式导航与多个视图共享 Session，不能把 Chat 的自动等待条件放进全局事件源。手动操作若永远得不到可用定位条件仍可能等待；实际页面应验证取消、重连和无锚点结构。一次补偿的布局测量、内层浏览器锚定及异步内容增长需要当前构建的浏览器证据。 diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml index 85c5633b69..06547f7c98 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml @@ -15,8 +15,8 @@ en: 2cbfd8cc05037895 zh: 024968506e2ac994 /agent-note-chat-history-prefetch-and-safe-consumption/proposal/position-and-status-presentation: - en: 1afb940b9b2a2fcc - zh: 8201614a80598e5b + en: 28d0c3b79f07ef07 + zh: b62ac1c02f23e4fe /agent-note-chat-history-prefetch-and-safe-consumption/proposal/scope-and-retained-decisions: en: dd4eeadd2fccab4b zh: e05d53e63cc44ddf @@ -27,5 +27,5 @@ en: b3005052df98ab10 zh: c4bdf2bed8d23550 /agent-note-chat-history-prefetch-and-safe-consumption/risks: - en: 2016c51019518572 - zh: e31e4e4ac09a2c5d + en: 4f1eac5c4dbd0b5e + zh: 9475ec9fa3bd9246 diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md index bfe932f77b..778c11c43e 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md @@ -35,7 +35,7 @@ After fallback, scrolling, animation completion, and content notifications canno ### Position and status presentation -The actual conversation scrollport uses `overscroll-behavior-y: none` to suppress vertical boundary bounce and scroll chaining, without implementing a different momentum animation. A prepend at the top reuses the existing paging-position record and correction calculation, applies it once, and releases it; persistent preservation elsewhere retains its existing lifetime. Reader movement during the correction interval is preserved rather than restoring the position from request time. +The actual conversation scrollport uses `overscroll-behavior-y: none` to suppress vertical boundary bounce and scroll chaining, without implementing a different momentum animation. A prepend at the top reuses the existing paging-position record and correction calculation, applies it once, and releases it; persistent preservation elsewhere retains its existing lifetime. Both modes restore the screen position captured immediately before consumption with the same formula; they differ only in whether preservation remains active afterward. The one-shot path does not reconstruct intervening reader movement or restart momentum. The history control keeps the same height across states. A network request shows Loading earlier; an active consumption intent without a held input shows Processing; automatic fallback or a held input shows Load earlier; failure shows Retry loading; consuming the final page shows Start of conversation. The control is excluded from native anchoring. Session `historyWasPartial` records that this instance has displayed incomplete history and survives component remounts and reconnects; a short Session that opens complete has no history row. @@ -67,7 +67,7 @@ Do not add `flushSync` or change Conversation publication cadence, fold-batch ta ## Risks -**Known issue: duplicate anchoring compensation.** A deterministic geometry-model reproduction shows that one-shot correction can count native anchoring for the same page as reader movement and compensate twice: this occurs with an outer `scrollTop` of `0.5`, or with outer `0` and inner `40` when native anchoring has already moved the inner position. The algorithm is therefore not universally safe. This issue is recorded without a code fix; independent verification in actual browsers is pending. The mechanism is detailed in the [technical risks](2026-11-06-chat-history-consumption-details.md#risks). +One-shot correction restores a captured screen position rather than estimating reader velocity. Input or browser movement between capture and application can therefore affect the resulting motion; validation must cover the actual browser and input sequence. The shared formula does not add scroll-position deltas that could already include native anchoring. See the [technical risks](2026-11-06-chat-history-consumption-details.md#risks). CSS feature support alone does not establish that native anchoring can act. Automatic prefetch waits for the first size observation, and the 500ms sample can let fast scrolling reach the top first. The synchronous geometry read after insertion is a cost of one-shot correction; its call site alone does not prove that it adds no layout. A stationary manual operation that can never obtain an anchor retains its completion intent and needs coverage for that structure. diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md index 31787a81b7..d9f7c8e57a 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md @@ -35,7 +35,7 @@ Status: proposed ### 位置与状态展示 -会话实际滚动容器设置 `overscroll-behavior-y: none`，关闭纵向边界回弹和滚动传递，不实现另一套惯性动画。顶部前插复用既有分页位置记录和补偿计算，只应用一次并释放；非顶部持续保位保持原生命周期。补偿期间发生的用户滚动保留，不把读者拉回请求发出时的位置。 +会话实际滚动容器设置 `overscroll-behavior-y: none`，关闭纵向边界回弹和滚动传递，不实现另一套惯性动画。顶部前插复用既有分页位置记录和补偿计算，只应用一次并释放；非顶部持续保位保持原生命周期。两种模式用相同公式恢复消费前刚采集的屏幕位置，只区别于应用后是否继续保位。一次补偿不重建采集期间的读者移动，也不重新启动惯性。 历史控件各状态保持同高。网络请求显示“正在加载”；仍有消费意图且未按住输入时显示“正在处理”；自动回退或输入按住时显示“加载更早”；失败显示“重新加载”；最后一页消费完成后显示“已到会话开头”。控件不参与原生锚定。Session 的 `historyWasPartial` 记录本实例曾显示不完整历史，组件重挂载和重连保留此事实；首次打开即完整的短会话不添加历史行。 @@ -67,7 +67,7 @@ Status: proposed ## Risks -**已知问题：锚定重复补偿。** 确定性的几何模型复现表明，一次补偿可能把同一页已经发生的原生锚定当作读者移动，再补偿一次：外层 `scrollTop` 为 `0.5`，或外层为 `0`、内层为 `40` 且原生锚定已移动内层位置时会出现。该算法因此不能视为普遍安全。本问题仅记录，不修改代码；独立的真实浏览器验证仍待完成。机制见[技术风险](2026-11-06-chat-history-consumption-details.zh.md#risks)。 +一次补偿恢复采集的屏幕位置，不估算读者速度。采集到应用之间的输入或浏览器移动可能影响最终运动，因此需要验证实际浏览器和输入序列。共享公式不叠加可能已包含原生锚定的滚动位置增量。见[技术风险](2026-11-06-chat-history-consumption-details.zh.md#risks)。 原生锚定适用性不能只由 CSS 支持检查保证。首次尺寸观察前不会自动预取；500ms 采样可能使高速滚动先到顶部。前插后的同步几何读取是一次补偿的成本，不能仅凭调用位置保证没有额外布局。静止手动消费若始终无法获得锚点，完成意图仍会等待，需要覆盖该特殊结构。 diff --git a/packages/client/ui-chat/README.i18n.yaml b/packages/client/ui-chat/README.i18n.yaml index b17f85460e..dd8cf646da 100644 --- a/packages/client/ui-chat/README.i18n.yaml +++ b/packages/client/ui-chat/README.i18n.yaml @@ -36,8 +36,8 @@ en: e2b02942a55fb669 zh: a38bbee0cc69ae4e /deepseek-ai-dsh-client-ui-chat/scroll-ownership: - en: a8e5c51345ef61b9 - zh: 0e65169b79504da8 + en: b26ab997d694ef90 + zh: 1f8a16ba02bab301 /deepseek-ai-dsh-client-ui-chat/model-experience: en: c17b4dd43a61c569 zh: f7414309810c3a08 diff --git a/packages/client/ui-chat/README.md b/packages/client/ui-chat/README.md index 8df3db6b32..e00a884771 100644 --- a/packages/client/ui-chat/README.md +++ b/packages/client/ui-chat/README.md @@ -186,7 +186,7 @@ Within a process group, wheel, touchstart, and any pointerdown interrupt an acti `useChatReading` owns follow policy, sampled reader input, and semantic memory; `useChatNavigation` owns turn jumps and requests preservation from the viewport. Reading gestures release the paging anchor, but composer clicks, typing, and non-scrolling keys retain it. A deferred history request captures its anchor at consumption rather than at request time. `useChatScroll` coordinates their committed inputs. Explicit navigation carries its measured landing into reading policy, so it does not rediscover the known target with a hit test. -History prefetch uses the existing 500ms reading sample and content/size notifications, with one distance check before requesting within two viewport heights. At most one page waits for consumption. Touch and inner scrolling retain automatic intent until release; active motion, pointer holds, or an unavailable anchor return automatic loading to the button without retrying the cached page. Explicit clicks retain intent until consumption, exhaustion, failure, or superseding navigation. Stationary consumption uses retained positioning; ordinary outer scrolling uses browser anchoring. At scrollTop ≤ 0.5, the first paging anchor supplies one content-displacement correction that retains intervening reader movement and then releases immediately. The actual Chat scrollport disables vertical overscroll bounce, without adding a scrolling animation. The history row keeps its height through loading, processing, retry, and exhaustion; initially complete Sessions omit it. +History prefetch uses the existing 500ms reading sample and content/size notifications, with one distance check before requesting within two viewport heights. At most one page waits for consumption. Touch and inner scrolling retain automatic intent until release; active motion, pointer holds, or an unavailable anchor return automatic loading to the button without retrying the cached page. Explicit clicks retain intent until consumption, exhaustion, failure, or superseding navigation. Stationary consumption uses retained positioning; ordinary outer scrolling uses browser anchoring. At scrollTop ≤ 0.5, the first paging anchor uses the same captured-screen-position restoration as persistent preservation, once, then releases immediately; it does not add scroll-position deltas that native anchoring may already have applied. The actual Chat scrollport disables vertical overscroll bounce, without adding a scrolling animation. The history row keeps its height through loading, processing, retry, and exhaustion; initially complete Sessions omit it. Active-Turn highlighting is approximate: `readVisibleTurn` binary-searches the flow slot's direct Node/Group boxes (or the column's in standalone rendering) and retains the preceding candidate in gaps. It neither hit-tests the document nor searches Group members or all Turn markers. Empty Seats retain zero-height in-flow boxes so outer positions remain ordered without extra spacing. This lookup does not change semantic position capture or paging compensation. diff --git a/packages/client/ui-chat/README.zh.md b/packages/client/ui-chat/README.zh.md index c2144fd861..29e8a373fa 100644 --- a/packages/client/ui-chat/README.zh.md +++ b/packages/client/ui-chat/README.zh.md @@ -186,7 +186,7 @@ Chat 会在历史前插与 renderer 重新挂载时恢复语义锚点，并且 `useChatReading` 负责跟随策略、读者输入采样和语义位置记忆；`useChatNavigation` 负责轮次跳转，并向视口请求位置保持。阅读手势释放分页锚点，composer 内的点击、打字和非滚动按键则保留锚点。延迟消费的历史请求在消费时采集锚点，不在发请求时采集。`useChatScroll` 协调已提交的输入。显式导航把测得的落点交给阅读策略，因此无需通过命中测试重新寻找已知目标。 -历史预取复用已有 500ms 阅读采样及内容／尺寸通知，在请求前确认距顶部两屏以内，最多一页等待消费。触摸和内层滚动保留自动意图至释放；动画、指针按住或无法取得锚点使自动加载退回按钮，不自动重试缓存页。明确点击保留意图，直到消费、到头、失败或被新导航取代。静止消费使用保位，普通外层滚动使用浏览器锚定；scrollTop ≤ 0.5 时用首条分页锚点补偿一次内容位移，保留期间的读者滚动，随后立即释放。实际 Chat 滚动区关闭纵向越界回弹，不新增滚动动画。历史行在加载、处理、重试和到头时保持高度，首次打开就完整的 Session 不显示该行。 +历史预取复用已有 500ms 阅读采样及内容／尺寸通知，在请求前确认距顶部两屏以内，最多一页等待消费。触摸和内层滚动保留自动意图至释放；动画、指针按住或无法取得锚点使自动加载退回按钮，不自动重试缓存页。明确点击保留意图，直到消费、到头、失败或被新导航取代。静止消费使用保位，普通外层滚动使用浏览器锚定；scrollTop ≤ 0.5 时，首条分页锚点与持续保位共用采集屏幕位置的恢复公式，只应用一次便立即释放；不再叠加可能已由原生锚定产生的滚动位置增量。实际 Chat 滚动区关闭纵向越界回弹，不新增滚动动画。历史行在加载、处理、重试和到头时保持高度，首次打开就完整的 Session 不显示该行。 活跃轮次高亮采用近似定位：`readVisibleTurn` 对 flow 插槽的直属 Node/Group 布局盒二分（独立渲染时使用正文容器的直属布局盒），落在间隙时保留前方候选。它不做文档命中测试，也不查询 Group 成员或全部 Turn 标记。空 Seat 保留零高度的流内布局盒，使外层位置有序且不增加间距。这种定位不改变语义位置捕获或分页补偿。 diff --git a/packages/client/ui-chat/src/client/chat/use-chat-viewport.ts b/packages/client/ui-chat/src/client/chat/use-chat-viewport.ts index 98c6ac2186..e81ca93a6f 100644 --- a/packages/client/ui-chat/src/client/chat/use-chat-viewport.ts +++ b/packages/client/ui-chat/src/client/chat/use-chat-viewport.ts @@ -67,7 +67,7 @@ interface ClosingLayout { interface PagingPosition { readonly row: HTMLElement readonly position: ChatScrollPosition - readonly group: { readonly body: HTMLElement; readonly content: HTMLElement; readonly top: number; readonly scrollTop: number } | null + readonly group: { readonly body: HTMLElement; readonly content: HTMLElement; readonly top: number } | null } const READING_INTENTS = ['wheel', 'touchstart', 'pointerdown', 'keydown', 'beforematch'] as const @@ -830,8 +830,7 @@ export class ChatViewport { const body = row.closest('[data-step-process-body]') const content = body?.querySelector('[data-step-process-content]') const group = body === null || content == null ? null - : { body, content, top: previousGroup?.top ?? top - body.getBoundingClientRect().top, - scrollTop: previousGroup?.body === body ? previousGroup.scrollTop : body.scrollTop } + : { body, content, top: previousGroup?.top ?? top - body.getBoundingClientRect().top } return { row, group, position: position ?? { @@ -880,7 +879,7 @@ export class ChatViewport { return null } if (paging !== this.paging) this.retainPaging(paging) - return this.applyPaging(paging, false) + return this.applyPaging(paging) } private resolvePaging(paging: PagingPosition | null): PagingPosition | null { @@ -896,17 +895,15 @@ export class ChatViewport { return paging.row.closest('[hidden]') !== null || paging.row.matches(':empty') ? null : paging } - /** Content correction includes reader movement since capture; screen preservation restores the captured position. */ - private applyPaging(paging: PagingPosition, preserveMovement: boolean): ViewportLanding | null { + /** Restore the captured screen position; native anchoring already applied to this change needs no further correction. */ + private applyPaging(paging: PagingPosition): ViewportLanding | null { const elements = this.elements if (elements === null) return null const { row, group, position } = paging - let innerMovement = 0 if (group !== null && group.body.contains(row)) { const top = row.getBoundingClientRect().top - group.body.getBoundingClientRect().top const metrics = scrollMetrics(group.body) - if (preserveMovement) innerMovement = metrics.top - group.scrollTop - const target = Math.max(0, Math.min(metrics.floor, metrics.top + top - group.top + innerMovement)) + const target = Math.max(0, Math.min(metrics.floor, metrics.top + top - group.top)) if (metrics.top !== target) { const follow = ScrollFollow.forElement(group.body) if (follow === undefined) group.body.scrollTop = target @@ -919,8 +916,7 @@ export class ChatViewport { const metrics = this.metrics() if (metrics === null) return null const top = row.getBoundingClientRect().top - elements.scroller.getBoundingClientRect().top - const movement = preserveMovement ? metrics.top - position.scrollTop + innerMovement : 0 - const target = metrics.top + top - position.anchorTop + movement + const target = metrics.top + top - position.anchorTop return this.write(target, metrics, null, { key: position.anchorKey, top }) } @@ -1052,7 +1048,7 @@ export class ChatViewport { const pending = this.historyPrepend this.historyPrepend = null const paging = this.resolvePaging(pending) - return paging === null ? null : this.applyPaging(paging, true) + return paging === null ? null : this.applyPaging(paging) } private readonly onHistoryScroll = (event: Event): void => { diff --git a/packages/client/ui-chat/tests/chat-viewport.client.spec.ts b/packages/client/ui-chat/tests/chat-viewport.client.spec.ts index 6149f9a89d..e3ed867df9 100644 --- a/packages/client/ui-chat/tests/chat-viewport.client.spec.ts +++ b/packages/client/ui-chat/tests/chat-viewport.client.spec.ts @@ -121,7 +121,7 @@ it('uses a collapsed group paging anchor when no ordinary content row is visible expect(h.viewport.preserving).toBe(false) }) -it('keeps reader movement during a one-shot prepend without retaining paging ownership', () => { +it('restores the captured screen position once without retaining paging ownership', () => { const h = fixture() const row = h.row(h.column, 'old', 0) expect(h.viewport.beginHistoryPrepend()).toBe(true) @@ -129,8 +129,8 @@ it('keeps reader movement during a one-shot prepend without retaining paging own h.column.scrollTop = 25 h.prepend(120) h.viewport.finishHistoryPrepend() - expect(h.column.scrollTop).toBe(145) - expect(row.getBoundingClientRect().top).toBe(-25) + expect(h.column.scrollTop).toBe(120) + expect(row.getBoundingClientRect().top).toBe(0) expect(h.viewport.preserving).toBe(false) }) @@ -534,7 +534,7 @@ it.each([ expect(h.row.getBoundingClientRect().top).toBe(top) }) -it('applies one-shot inner compensation without subscribing or undoing reader movement', () => { +it('restores the captured inner and outer screen position once without subscribing', () => { const h = nestedFixture(600, 400) h.scroller.scrollTop = 80 h.body.scrollTop = 40 @@ -546,14 +546,14 @@ it('applies one-shot inner compensation without subscribing or undoing reader mo h.scroller.scrollTop = 90 h.prepend(200) h.viewport.finishHistoryPrepend() - expect(h.body.scrollTop).toBe(260) - expect(h.scroller.scrollTop).toBe(90) - expect(h.row.getBoundingClientRect().top).toBe(screenTop - 30) + expect(h.body.scrollTop).toBe(240) + expect(h.scroller.scrollTop).toBe(80) + expect(h.row.getBoundingClientRect().top).toBe(screenTop) expect(h.observed.has(h.content)).toBe(false) h.prepend(20) h.resize() - expect(h.body.scrollTop).toBe(260) - expect(h.scroller.scrollTop).toBe(90) + expect(h.body.scrollTop).toBe(240) + expect(h.scroller.scrollTop).toBe(80) }) it('resolves a remounted one-shot anchor without entering retained paging', () => { @@ -565,8 +565,37 @@ it('resolves a remounted one-shot anchor without entering retained paging', () = h.column.scrollTop = 25 h.prepend(120) h.viewport.finishHistoryPrepend() - expect(h.column.scrollTop).toBe(145) - expect(replacement.getBoundingClientRect().top).toBe(-25) + expect(h.column.scrollTop).toBe(120) + expect(replacement.getBoundingClientRect().top).toBe(0) + expect(h.viewport.preserving).toBe(false) +}) + +it('does not repeat native outer anchoring during a near-zero one-shot prepend', () => { + const h = fixture() + const row = h.row(h.column, 'old', 0) + h.column.scrollTop = 0.5 + const top = row.getBoundingClientRect().top + expect(h.viewport.beginHistoryPrepend()).toBe(true) + h.prepend(120) + // Model the browser's adjustment before the layout effect reads the new geometry. + h.column.scrollTop = 120.5 + h.viewport.finishHistoryPrepend() + expect(h.column.scrollTop).toBe(120.5) + expect(row.getBoundingClientRect().top).toBe(top) +}) + +it('does not repeat native inner anchoring when the outer scroller is at zero', () => { + const h = nestedFixture(600, 400) + h.body.scrollTop = 40 + const top = h.row.getBoundingClientRect().top + expect(h.viewport.beginHistoryPrepend()).toBe(true) + h.prepend(200) + // Same ordering as the Chromium regression: native inner adjustment precedes our correction. + h.body.scrollTop = 240 + h.viewport.finishHistoryPrepend() + expect(h.body.scrollTop).toBe(240) + expect(h.scroller.scrollTop).toBe(0) + expect(h.row.getBoundingClientRect().top).toBe(top) expect(h.viewport.preserving).toBe(false) }) From e48c9cc5291f376cc95b5bf26ec05463fa04dd48 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 6 Nov 2026 23:26:11 +0800 Subject: [PATCH 38/44] fix(ui-chat): keep manual history available and settle prepend anchors --- ...chat-history-consumption-details.i18n.yaml | 16 ++-- ...-11-06-chat-history-consumption-details.md | 21 ++--- ...-06-chat-history-consumption-details.zh.md | 21 ++--- ...6-11-06-chat-history-consumption.i18n.yaml | 16 ++-- .../2026-11-06-chat-history-consumption.md | 20 ++--- .../2026-11-06-chat-history-consumption.zh.md | 20 ++--- apps/web/tests/chat-scroll-contract.e2e.ts | 88 +++++++++++++------ packages/client/ui-chat/README.i18n.yaml | 8 +- packages/client/ui-chat/README.md | 4 +- packages/client/ui-chat/README.zh.md | 4 +- .../ui-chat/src/client/chat/ChatView.tsx | 2 +- .../src/client/chat/use-chat-navigation.ts | 44 +++++----- .../src/client/chat/use-chat-scroll.ts | 6 +- .../src/client/chat/use-chat-viewport.ts | 24 ++--- .../tests/chat-history-intent.client.spec.ts | 51 +++++++---- .../ui-chat/tests/chat-view.client.spec.tsx | 39 +++++++- 16 files changed, 236 insertions(+), 148 deletions(-) diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml index c0294f6e21..e921fbaeaf 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.i18n.yaml @@ -12,17 +12,17 @@ en: 5e5553981c48d0e2 zh: 1195ac8d64180451 /agent-note-chat-history-consumption-classes-and-interactions/proposal/owners-and-interfaces: - en: 58f6f76fd263cafb - zh: 13ac0a6259d30cda + en: efbe7ccc25cf1f42 + zh: 3bb8f803677824bd /agent-note-chat-history-consumption-classes-and-interactions/proposal/data-lifetimes: en: ffdbe0e796463624 zh: f10bd2ced5748017 /agent-note-chat-history-consumption-classes-and-interactions/proposal/consumption-intent-and-wakeups: - en: 993c2985bf6f3c8f - zh: 8f88fb46584fb272 + en: db8763250020e58c + zh: 4d9a17f1b209d4f2 /agent-note-chat-history-consumption-classes-and-interactions/proposal/shared-anchors-and-layout-reads: - en: 3d59702a5b7021ea - zh: 25c9762647c7c0e0 + en: 21df3db6bc83e702 + zh: 43b49ceb7407a422 /agent-note-chat-history-consumption-classes-and-interactions/alternatives-considered: en: 7b3b9b0c7b8f293b zh: e20045fc3080418f @@ -30,5 +30,5 @@ en: 1b22cf3616302960 zh: 418ead8421a91bee /agent-note-chat-history-consumption-classes-and-interactions/risks: - en: 29e718a7d1c07a68 - zh: a81e5bc6c84b0712 + en: 9a61970e311751ac + zh: b24d0eac7b12af62 diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md index a14fa89273..b01a462712 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.md @@ -18,7 +18,7 @@ Session owns a history page's network lifetime, while Chat owns whether that pag |---|---| | [Session](../../../../packages/api/session-controller/src/client/sessions/session.ts) | `loadOlder({ deferConsumption?: boolean })` prepares and optionally consumes a page; `consumeOlder()` synchronously publishes a valid staged page; `historyLoadStatus` is idle/loading/pendingConsumption/error, and `loadingOlder` derives from loading, including Turn jumps | | Session request record | Holds generation, starting cursor, page entries, hasMore, Promise, and consumption/invalidation flags; staging does not advance the visible window cursor | -| [ChatNavigation](../../../../packages/client/ui-chat/src/client/chat/use-chat-navigation.ts) | Owns none/auto/manual consumption intent and request exclusion; waits or falls back by blocker; only manual consumption registers a fold-completion callback | +| [ChatNavigation](../../../../packages/client/ui-chat/src/client/chat/use-chat-navigation.ts) | Owns none/auto/manual consumption intent and request exclusion; automatic consumption waits or falls back by blocker, while manual consumption captures an anchor and consumes without blocker-wait subscriptions | | [ChatViewport](../../../../packages/client/ui-chat/src/client/chat/use-chat-viewport.ts) | Reads internal input/motion state and scrollTop, returns a blocker or native/preserve/prepend, and performs distance measurement and existing anchor algorithms | | [useChatScroll](../../../../packages/client/ui-chat/src/client/chat/use-chat-scroll.ts) | Connects existing reading, scroll, content, and size notifications to consumption checks and settles a pending one-shot correction during commit handling | | [ChatView](../../../../packages/client/ui-chat/src/client/chat/ChatView.tsx) | Requests and consumes through injected callbacks and reads state through useSession; renders a fixed-height history control without retaining Session history data or accumulating historyWasPartial | @@ -26,13 +26,14 @@ Session owns a history page's network lifetime, while Chat owns whether that pag ```text ChatNavigation.checkHistory ├─ pendingConsumption + intent=none → retain page until clicked - ├─ pendingConsumption + intent=auto/manual → viewport.historyConsumption - │ ├─ touch/inner-scroll → retain consumption intent - │ ├─ motion/pointer/unavailable → auto falls back, manual waits + ├─ pendingConsumption + intent=manual → beginManualPaging → consumeOlder + ├─ pendingConsumption + intent=auto → viewport.historyConsumption + │ ├─ touch/inner-scroll → retain automatic intent before checking motion + │ ├─ motion/pointer/unavailable → return to button │ ├─ native → consumeOlder │ ├─ prepend → capture paging position → consumeOlder → correct once after commit │ └─ preserve → beginPreserving → consumeOlder - ├─ loading → no duplicate request + ├─ loading → no duplicate request; a click records manual intent ├─ error → wait for explicit retry └─ idle + hasMore + (manual intent or near top without following) → request a page @@ -63,17 +64,17 @@ An ordinary request captures the transport page first. It can be consumed only a ### Consumption intent and wakeups -An automatic request records auto intent; a click records manual intent. Successful consumption, failure, exhaustion, or navigation cancellation clears the intent. A cached page with none intent is neither measured nor consumed by ordinary events; its button records manual intent without requesting it again. Motion, pointer, or unavailable positioning returns auto to none; touch and inner-scroll retain auto. Manual intent waits through these blockers rather than treating deferral as success. +An automatic request records auto intent; a click records manual intent. Successful consumption, failure, exhaustion, or navigation cancellation clears the intent. A cached page with none intent is neither measured nor consumed by ordinary events; its button records manual intent without requesting it again. Touch and actual inner-scroll retain auto before motion is evaluated; after release the current state is checked again. Motion, pointer, or unavailable positioning otherwise returns auto to none. Manual intent bypasses these automatic blockers once a page is available: `beginManualPaging()` uses `beginPaging()` at `scrollTop ≤ 0.5`; elsewhere it calls `beginPreserving()` with `beginPaging()` as fallback. Consumption proceeds even if no anchor is produced. Loading and processing clicks join the current request rather than duplicating it. Prefetch reuses the existing 500ms reading sample, scrollend, content commits, and size notifications, coalescing them into one rAF check. It uses no IntersectionObserver or additional sample timer and measures nothing in the raw scroll event. Only a ResizeObserver entry for the scroller itself refreshes cached clientHeight during the callback's read phase; column growth does not reread viewport height. Once internal state permits an automatic request, it reads the history control's distance from the viewport and compares it with two cached viewport heights. -`historyConsumption()` does not query DOM nodes or measure sizes. `FlowMotion.active()` exposes existing running, batchScheduled, batch, and settling state as a read-only query; passive listeners maintain input state. Size observation removes detached scrolling targets. Finger release, inner scroll completion, transition completion/cancellation, and existing notifications can wake retained intent; fold-completion callbacks are registered only for manual intent. Without consumption intent, these notifications do not revive an abandoned automatic page. +`historyConsumption()` does not query DOM nodes or measure sizes. `FlowMotion.active()` exposes existing running, batchScheduled, batch, and settling state as a read-only query; passive listeners maintain input state. Size observation removes detached scrolling targets. Finger release, inner scroll completion, transition completion/cancellation, and existing notifications can wake retained automatic intent. Manual consumption has no fold-completion waiting subscription. Without consumption intent, these notifications do not revive an abandoned automatic page. ### Shared anchors and layout reads -At `scrollTop ≤ 0.5`, selection uses `beginPaging`'s first paging-anchor rule without a reading-position hit test. `measurePaging()` only captures a `PagingPosition`; `retainPaging()` separately owns persistent preservation and inner-size observation. Stationary preservation and one-shot correction share remount resolution in `resolvePaging()` and inner/outer calculations in `applyPaging()`. +For automatic consumption at `scrollTop ≤ 0.5`, selection uses `beginPaging`'s first paging-anchor rule without a reading-position hit test. `measurePaging()` only captures a `PagingPosition`; `retainPaging()` separately owns persistent preservation and inner-size observation. Stationary preservation and one-shot correction share remount resolution in `resolvePaging()` and inner/outer calculations in `applyPaging()`. -Persistent preservation and one-shot correction use the same `applyPaging()` formula to restore the captured screen position, compensating the inner scrollport before the outer one. They do not add changes in scrollTop since capture: native anchoring may have caused those changes. Their lifetimes differ: one-shot correction neither enters paging state nor subscribes to inner-size changes, and releases after one application. `finishHistoryPrepend()` returns the actual landing, and commit handling immediately saves it through `reading.preservePosition()` rather than waiting for a later scroll event. The actual conversation scrollport uses `overscroll-behavior-y: none`, without creating smooth scrolling or a new momentum animation. +Persistent preservation and one-shot correction use the same `applyPaging()` formula to restore the captured screen position, compensating the inner scrollport before the outer one. They do not add changes in scrollTop since capture: native anchoring may have caused those changes. Their lifetimes differ: one-shot correction neither enters paging state nor subscribes to inner-size changes, and releases after one application. `finishHistoryPrepend()` returns the actual landing, and commit handling immediately saves it through `reading.preservePosition()` rather than waiting for a later scroll event. A committed transition from `pendingConsumption` to `idle` settles the one-shot record even when the visible Node order is unchanged; a closed view or `error` discards the record without applying it. Geometry invalidation remains conditional on the existing layout-change signal, and no new page token or React publication mechanism is introduced. The actual conversation scrollport uses `overscroll-behavior-y: none`, without creating smooth scrolling or a new momentum animation. Distance confirmation still needs current geometry, and one-shot correction after prepend still needs the new position. These reads serve correctness, but placement inside rAF, ResizeObserver, or commit handling alone does not prove they add no layout. Read frequency, read/write ordering, and actual Layout/Paint costs require separate verification; position stability is not performance evidence. @@ -98,6 +99,6 @@ Distance confirmation still needs current geometry, and one-shot correction afte Screen restoration uses the measured residual displacement after native anchoring rather than adding scroll-position deltas. It does not preserve arbitrary input movement between capture and application or guarantee an uninterrupted momentum trajectory. Near-zero outer positions, native inner anchoring, and input arriving around the prepend require current-build browser verification. -Explicit navigation and multiple views share a Session, so Chat's automatic waiting policy must not move into the global event source. Manual intent can still wait if usable positioning never becomes available; actual views need cancellation, reconnect, and anchorless-structure verification. One-shot geometry measurement, native inner anchoring, and asynchronous content growth need browser evidence from the current build. +Explicit navigation and multiple views share a Session, so Chat's automatic waiting policy must not move into the global event source. Manual consumption establishes persistent preservation at consumption time; an already active touch can then enter the paging scroll branch instead of reader sampling, constraining movement until a new input releases the anchor. This [manual paging limitation](../../../../packages/client/ui-chat/README.md#known-limitations-and-deferred-work) remains unresolved. Manual consumption proceeds without preservation if no anchor can be captured, so that structure has no position-stability guarantee. Cancellation, reconnect, and removal of the selected anchor remain distinct lifecycle cases; a missing anchor cannot provide position preservation. One-shot geometry measurement, native inner anchoring, and asynchronous content growth need browser evidence from the current build. Reports must identify source/artifact versions, real or synthetic inputs, missing conditions, and cleanup results. V8 function or range hits are not TypeScript line/branch percentages; earlier iteration recordings, Chrome/Safari manual feedback, or a successful build do not establish a complete matrix pass for this proposal. diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md index 8cebfc2fa8..ce9490648d 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption-details.zh.md @@ -18,7 +18,7 @@ Status: proposed |---|---| | [Session](../../../../packages/api/session-controller/src/client/sessions/session.ts) | `loadOlder({ deferConsumption?: boolean })` 准备并按选项消费一页；`consumeOlder()` 同步发布有效暂存页；`historyLoadStatus` 表达 idle/loading/pendingConsumption/error，`loadingOlder` 从 loading 派生并包含 Turn 跳转 | | Session 请求记录 | 保存 generation、请求前游标、页内容、hasMore、Promise 与消费/失效标记；缓存页不提前改变可见窗口游标 | -| [ChatNavigation](../../../../packages/client/ui-chat/src/client/chat/use-chat-navigation.ts) | 保存 none/auto/manual 消费意图和请求互斥；按阻塞原因等待或退回按钮；仅手动消费注册折叠结束通知 | +| [ChatNavigation](../../../../packages/client/ui-chat/src/client/chat/use-chat-navigation.ts) | 保存 none/auto/manual 消费意图和请求互斥；自动按阻塞原因等待或退回按钮，手动采集锚点后消费，不注册阻塞结束等待 | | [ChatViewport](../../../../packages/client/ui-chat/src/client/chat/use-chat-viewport.ts) | 读取内部输入/动画状态与 scrollTop，报告阻塞原因或 native/preserve/prepend；执行距离测量和已有锚点算法 | | [useChatScroll](../../../../packages/client/ui-chat/src/client/chat/use-chat-scroll.ts) | 把既有阅读、滚动、内容及尺寸通知接到消费检查；在提交阶段结算待执行的一次补偿 | | [ChatView](../../../../packages/client/ui-chat/src/client/chat/ChatView.tsx) | 通过注入回调请求/消费，通过 useSession 读取状态；固定高度显示历史控件，不保存 Session 历史数据或累计 historyWasPartial | @@ -26,13 +26,14 @@ Status: proposed ```text ChatNavigation.checkHistory ├─ pendingConsumption + intent=none → retain page until clicked - ├─ pendingConsumption + intent=auto/manual → viewport.historyConsumption - │ ├─ touch/inner-scroll → retain consumption intent - │ ├─ motion/pointer/unavailable → auto falls back, manual waits + ├─ pendingConsumption + intent=manual → beginManualPaging → consumeOlder + ├─ pendingConsumption + intent=auto → viewport.historyConsumption + │ ├─ touch/inner-scroll → retain automatic intent before checking motion + │ ├─ motion/pointer/unavailable → return to button │ ├─ native → consumeOlder │ ├─ prepend → capture paging position → consumeOlder → correct once after commit │ └─ preserve → beginPreserving → consumeOlder - ├─ loading → no duplicate request + ├─ loading → no duplicate request; a click records manual intent ├─ error → wait for explicit retry └─ idle + hasMore + (manual intent or near top without following) → request a page @@ -63,17 +64,17 @@ Session.consumeOlder ### 消费意图与唤醒 -自动请求开始时登记 auto，点击登记 manual。消费成功、失败、耗尽或取消导航时清除意图。缓存存在但意图为 none 时，普通事件不测量或消费该缓存；按钮重新登记 manual，不重复取页。motion、pointer 或 unavailable 使 auto 退回 none；touch 与 inner-scroll 保留 auto。manual 在这些阻塞下继续等待，不把暂缓当成成功。 +自动请求开始时登记 auto，点击登记 manual。消费成功、失败、耗尽或取消导航时清除意图。缓存存在但意图为 none 时，普通事件不测量或消费该缓存；按钮重新登记 manual，不重复取页。touch 与实际 inner-scroll 优先于 motion 判断保留 auto，释放后重新检查当前状态；其余 motion、pointer 或 unavailable 使 auto 退回 none。manual 在页面可用时绕过这些自动阻塞：`beginManualPaging()` 在 `scrollTop ≤ 0.5` 时调用 `beginPaging()`；其他位置调用 `beginPreserving()`，并以 `beginPaging()` 兜底。没有产生锚点时仍继续消费。加载和处理状态的点击加入当前请求，不重复请求。 历史预取复用已有 500ms 阅读采样、scrollend、内容提交和尺寸通知，并合并为一次 rAF 检查。不使用 IntersectionObserver，不新增采样计时器，不在原始 scroll 事件中测量。仅 scroller 自身的 ResizeObserver 条目在回调读阶段更新 clientHeight 缓存，column 增长不重新读取视口高度。内部状态允许自动请求时，读取历史控件相对视口的位置，与缓存的两屏高度比较。 -`historyConsumption()` 不查询 DOM 或测量尺寸。`FlowMotion.active()` 从已有 running、batchScheduled、batch、settling 状态提供只读查询；输入状态来自被动事件监听。滚动目标失效的清理在尺寸观察处进行。手指释放、内层滚动结束、过渡结束/取消和既有通知可以唤醒保留的意图；折叠结束回调只为 manual 注册。没有消费意图时这些通知不恢复已经放弃的自动页。 +`historyConsumption()` 不查询 DOM 或测量尺寸。`FlowMotion.active()` 从已有 running、batchScheduled、batch、settling 状态提供只读查询；输入状态来自被动事件监听。滚动目标失效的清理在尺寸观察处进行。手指释放、内层滚动结束、过渡结束/取消和既有通知可以唤醒保留的自动意图；手动消费不注册折叠结束等待。没有消费意图时这些通知不恢复已经放弃的自动页。 ### 锚点复用与布局读取 -顶部 `scrollTop ≤ 0.5` 使用 `beginPaging` 的首条分页锚点规则，不先执行阅读位置命中检测。`measurePaging()` 只采集 `PagingPosition`；`retainPaging()` 单独拥有持续保位和内层尺寸观察。普通静止保位与一次补偿共用 `resolvePaging()` 的重挂载查找及 `applyPaging()` 的内外层计算。 +自动消费在顶部 `scrollTop ≤ 0.5` 时使用 `beginPaging` 的首条分页锚点规则，不先执行阅读位置命中检测。`measurePaging()` 只采集 `PagingPosition`；`retainPaging()` 单独拥有持续保位和内层尺寸观察。普通静止保位与一次补偿共用 `resolvePaging()` 的重挂载查找及 `applyPaging()` 的内外层计算。 -持续保位和一次补偿用相同的 `applyPaging()` 公式恢复采集时的屏幕位置，先补内层、再补外层。计算不叠加采集后 scrollTop 的变化，因为这些变化可能已经来自原生锚定。两者只区分生命周期：一次补偿不进入 paging 状态、不注册内层尺寸观察，应用一次后释放。`finishHistoryPrepend()` 返回实际落点，提交处理立即通过 `reading.preservePosition()` 保存阅读位置，不依赖后续 scroll 事件。会话实际滚动容器设置 `overscroll-behavior-y: none`，不创建平滑滚动或新的惯性动画。 +持续保位和一次补偿用相同的 `applyPaging()` 公式恢复采集时的屏幕位置，先补内层、再补外层。计算不叠加采集后 scrollTop 的变化，因为这些变化可能已经来自原生锚定。两者只区分生命周期：一次补偿不进入 paging 状态、不注册内层尺寸观察，应用一次后释放。`finishHistoryPrepend()` 返回实际落点，提交处理立即通过 `reading.preservePosition()` 保存阅读位置，不依赖后续 scroll 事件。已提交的 `pendingConsumption` 到 `idle` 状态转换，即使可见 Node 顺序未变化，也会结算一次补偿记录；视图关闭或进入 `error` 时丢弃记录，不应用补偿。几何失效仍只由已有布局变化信号触发，不增加页 token 或 React 发布机制。会话实际滚动容器设置 `overscroll-behavior-y: none`，不创建平滑滚动或新的惯性动画。 距离检查仍需要当前几何；前插后的一次补偿仍需要读取新位置。这些读数有正确性用途，但不能仅凭位于 rAF、ResizeObserver 或提交回调中，就认定不会增加布局。读取频率、读写顺序及实际 Layout/Paint 成本分别验收；位置不漂移不能替代性能证明。 @@ -98,6 +99,6 @@ Session.consumeOlder 屏幕位置恢复使用原生锚定后测得的剩余位移，而不是叠加滚动位置增量。它不保留采集到应用之间任意输入造成的移动，也不保证惯性轨迹连续。外层近零位置、内层原生锚定以及前插附近到达的输入，都需要当前构建的浏览器验证。 -显式导航与多个视图共享 Session，不能把 Chat 的自动等待条件放进全局事件源。手动操作若永远得不到可用定位条件仍可能等待；实际页面应验证取消、重连和无锚点结构。一次补偿的布局测量、内层浏览器锚定及异步内容增长需要当前构建的浏览器证据。 +显式导航与多个视图共享 Session，不能把 Chat 的自动等待条件放进全局事件源。手动消费在消费时建立持续保位；已经进行的触摸随后可能进入 paging 滚动分支而非读者采样，移动受限直到新输入释放锚点。这项[手动分页限制](../../../../packages/client/ui-chat/README.zh.md#known-limitations-and-deferred-work)仍未解决。手动消费在无法采集锚点时仍继续，因此该结构不保证位置稳定。取消、重连和所选锚点被移除属于不同生命周期情况；锚点缺失时无法提供位置保留。一次补偿的布局测量、内层浏览器锚定及异步内容增长需要当前构建的浏览器证据。 验收报告应列明源码/产物版本、真实或合成输入、未覆盖条件以及清理结果。V8 函数或区间命中不等于 TypeScript 行/分支百分比；早期迭代的录像、Chrome/Safari 手工体验和单个构建通过不能证明本提案的完整矩阵已通过。 diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml index 06547f7c98..d0ee532e45 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.i18n.yaml @@ -12,11 +12,11 @@ en: bda6c4b3dfff143e zh: 4316f6dec6adaa50 /agent-note-chat-history-prefetch-and-safe-consumption/proposal/manual-and-automatic-loading: - en: 2cbfd8cc05037895 - zh: 024968506e2ac994 + en: 9f3707f7d6c3d98e + zh: f8903e2d1d652535 /agent-note-chat-history-prefetch-and-safe-consumption/proposal/position-and-status-presentation: - en: 28d0c3b79f07ef07 - zh: b62ac1c02f23e4fe + en: 33f2466b802f3957 + zh: 095227215a22b403 /agent-note-chat-history-prefetch-and-safe-consumption/proposal/scope-and-retained-decisions: en: dd4eeadd2fccab4b zh: e05d53e63cc44ddf @@ -24,8 +24,8 @@ en: 7c6d9d470c5b5c93 zh: dc12846ee0f56aae /agent-note-chat-history-prefetch-and-safe-consumption/acceptance-criteria: - en: b3005052df98ab10 - zh: c4bdf2bed8d23550 + en: f9efd25288474b8a + zh: 9271fde45be71de6 /agent-note-chat-history-prefetch-and-safe-consumption/risks: - en: 4f1eac5c4dbd0b5e - zh: 9475ec9fa3bd9246 + en: ff3b2da49f5fb541 + zh: b11afa7556f3f313 diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md index 778c11c43e..0e511c63a0 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.md @@ -14,7 +14,7 @@ Separate requesting history from consuming it. The Client Session retains at mos ### Manual and automatic loading -A manual click retains one loading intent: consume an existing page, join an in-flight request, or request a page. A temporary blocker does not complete the click; the operation ends with effective consumption, confirmed exhaustion, or an explicit failure. A newer Turn jump, return to bottom, view closure, or other navigation or lifecycle change can cancel the earlier reading intent. +A manual click retains one loading intent: consume an existing page, join an in-flight request, or request a page. Once data is available, the click bypasses automatic motion and input blockers: it uses the first paging anchor at the top, or captures the current reading anchor elsewhere with the paging anchor as fallback, and consumes through existing preservation without waiting for an animation callback. If neither anchor is available, consumption still proceeds without a position guarantee. The operation ends with consumption, confirmed exhaustion, or an explicit failure. A newer Turn jump, return to bottom, view closure, or other navigation or lifecycle change can cancel the earlier reading intent. Automatic loading requests history after the reader leaves tail following and comes within two viewport heights of the history control. It reuses the existing 500ms reading sample, scroll completion, content commits, and size notifications, without another polling loop or IntersectionObserver. Detection can therefore wait until the next existing sample. @@ -22,12 +22,12 @@ A cached page and an intent to continue automatically are separate facts. Automa | State when data is ready | Automatic intent | Manual intent | |---|---|---| -| Ordinary outer momentum, with other conditions satisfied | Consume with native browser anchoring | Consume with native browser anchoring | +| Ordinary outer momentum, with other conditions satisfied | Consume with native browser anchoring | Consume through current-anchor preservation | | Stationary reading away from the top | Consume with existing reading preservation | Consume with existing reading preservation | -| `scrollTop ≤ 0.5`, with other conditions satisfied | Reuse the paging anchor and correct once after prepend | Reuse the paging anchor and correct once after prepend | -| Folding, revealing, or an existing follow animation | Keep the page and return to the button | Retain intent until completion | -| A finger held down or actual inner scrolling | Retain intent and continue afterward | Retain intent and continue afterward | -| A held mouse pointer, scrollbar dragging, or temporarily unavailable positioning | Keep the page and return to the button | Retain intent until usable | +| `scrollTop ≤ 0.5`, with other conditions satisfied | Reuse the paging anchor and correct once after prepend | Consume through current-anchor preservation | +| Folding, revealing, or an existing follow animation | Keep the page and return to the button | Consume the available page through existing preservation | +| A finger held down or actual inner scrolling | Retain intent before testing motion blockers; recheck afterward | Consume the available page through existing preservation | +| A held mouse pointer, scrollbar dragging, or temporarily unavailable positioning | Keep the page and return to the button | Consume the available page; no anchor means no position guarantee | | Tail following resumes after automatic prefetch | Keep the page and cancel automatic consumption | The click has paused tail following | | Streaming continues at the tail | Does not itself block consumption | Does not itself block consumption | @@ -35,9 +35,9 @@ After fallback, scrolling, animation completion, and content notifications canno ### Position and status presentation -The actual conversation scrollport uses `overscroll-behavior-y: none` to suppress vertical boundary bounce and scroll chaining, without implementing a different momentum animation. A prepend at the top reuses the existing paging-position record and correction calculation, applies it once, and releases it; persistent preservation elsewhere retains its existing lifetime. Both modes restore the screen position captured immediately before consumption with the same formula; they differ only in whether preservation remains active afterward. The one-shot path does not reconstruct intervening reader movement or restart momentum. +The actual conversation scrollport uses `overscroll-behavior-y: none` to suppress vertical boundary bounce and scroll chaining, without implementing a different momentum animation. An automatic prepend at the top reuses the existing paging-position record and correction calculation, applies it once, and releases it; manual consumption and stationary preservation retain their existing persistent lifetime. Both modes restore the screen position captured immediately before consumption with the same formula; they differ only in whether preservation remains active afterward. The one-shot path does not reconstruct intervening reader movement or restart momentum. Consumption completion also settles and clears the one-shot record when the visible Node order does not change; view closure or history failure discards it without applying it. -The history control keeps the same height across states. A network request shows Loading earlier; an active consumption intent without a held input shows Processing; automatic fallback or a held input shows Load earlier; failure shows Retry loading; consuming the final page shows Start of conversation. The control is excluded from native anchoring. Session `historyWasPartial` records that this instance has displayed incomplete history and survives component remounts and reconnects; a short Session that opens complete has no history row. +The history control keeps the same height across states. Loading and Processing remain clickable to join the current request or consume the cached page; only confirmed exhaustion disables the button. A network request shows Loading earlier; an active consumption intent without a held input shows Processing; automatic fallback or a held input shows Load earlier; failure shows Retry loading; consuming the final page shows Start of conversation. The control is excluded from native anchoring. Session `historyWasPartial` records that this instance has displayed incomplete history and survives component remounts and reconnects; a short Session that opens complete has no history row. ### Scope and retained decisions @@ -61,7 +61,7 @@ Do not add `flushSync` or change Conversation publication cadence, fold-batch ta - Requests, staging, and consumption never duplicate a page request; tail streaming continues. - Ordinary events do not revive automatic fallback; touch, inner scrolling, and manual clicks retain their specified continuation. -- Stationary and top-edge prepends preserve old content position; ordinary momentum consumption adds no outer JS scroll write. +- Stationary and top-edge prepends preserve old content position; ordinary automatic momentum consumption adds no outer JS scroll write. - Failure, no progress, exhaustion, reconnect, and explicit navigation have defined outcomes without request loops or stale-page consumption. - Control states have equal height; existing fold-driver behavior is retained, and browser evidence identifies the current build, simulated inputs, and missing cases. @@ -69,6 +69,6 @@ Do not add `flushSync` or change Conversation publication cadence, fold-batch ta One-shot correction restores a captured screen position rather than estimating reader velocity. Input or browser movement between capture and application can therefore affect the resulting motion; validation must cover the actual browser and input sequence. The shared formula does not add scroll-position deltas that could already include native anchoring. See the [technical risks](2026-11-06-chat-history-consumption-details.md#risks). -CSS feature support alone does not establish that native anchoring can act. Automatic prefetch waits for the first size observation, and the 500ms sample can let fast scrolling reach the top first. The synchronous geometry read after insertion is a cost of one-shot correction; its call site alone does not prove that it adds no layout. A stationary manual operation that can never obtain an anchor retains its completion intent and needs coverage for that structure. +CSS feature support alone does not establish that native anchoring can act. Automatic prefetch waits for the first size observation, and the 500ms sample can let fast scrolling reach the top first. The synchronous geometry read after insertion is a cost of one-shot correction; its call site alone does not prove that it adds no layout. Manual consumption without a usable anchor can move visible content. It can also establish persistent preservation during an already active touch gesture, constraining that gesture or its momentum until a new reading input releases the anchor; see the [manual paging limitation](../../../../packages/client/ui-chat/README.md#known-limitations-and-deferred-work). Audit and measurement are ongoing, so this record remains proposed. Earlier Chrome/Safari manual feedback and headless results from different iterations do not establish a complete matrix pass for the final source; actual scrollbar dragging, same-document reconnect, cross-platform input, and exceptional cleanup paths require separately attributed evidence. diff --git a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md index d9f7c8e57a..1279f75c51 100644 --- a/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md +++ b/.agents/notes/proposed/feature/2026-11-06-chat-history-consumption.zh.md @@ -14,7 +14,7 @@ Status: proposed ### 手动与自动加载 -手动点击保留一次加载意图：已有缓存就消费缓存，有在途请求就等待该请求，否则请求一页。暂时不安全不算点击完成；该操作以有效消费、确认到头或明确失败结束。显式 Turn 跳转、返回底部、会话关闭等新的导航或生命周期变化可以取消原阅读意图。 +手动点击保留一次加载意图：已有缓存就消费缓存，有在途请求就等待该请求，否则请求一页。数据可用后，点击绕过自动路径的动画和输入阻塞：在顶部使用首条分页锚点，其他位置采集当前阅读锚点并以分页锚点兜底，再通过既有保位路径消费，不等待动画回调。两个锚点都不存在时仍消费，但不保证位置稳定。该操作以消费、确认到头或明确失败结束。显式 Turn 跳转、返回底部、会话关闭等新的导航或生命周期变化可以取消原阅读意图。 自动加载在读者离开底部跟随、进入距历史控件两屏以内时请求。它复用已有 500ms 阅读采样及滚动结束、内容提交、尺寸通知，不新增轮询或 IntersectionObserver。代价是发现进入预取范围可能延后至下一次已有采样。 @@ -22,12 +22,12 @@ Status: proposed | 数据就绪时的状态 | 自动意图 | 手动意图 | |---|---|---| -| 普通外层惯性，其他条件允许 | 浏览器原生锚定，消费 | 浏览器原生锚定，消费 | +| 普通外层惯性，其他条件允许 | 浏览器原生锚定，消费 | 通过当前锚点保位消费 | | 非顶部静止阅读 | 既有阅读保位，消费 | 既有阅读保位，消费 | -| `scrollTop ≤ 0.5`，其他条件允许 | 复用分页锚点，前插后补偿一次 | 复用分页锚点，前插后补偿一次 | -| 折叠、展开或已有跟随动画 | 保留缓存，退回按钮 | 保留意图，等待结束 | -| 手指按住或实际内层滚动 | 保留意图，结束后继续 | 保留意图，结束后继续 | -| 鼠标按住、滚动条拖动或定位能力暂不可用 | 保留缓存，退回按钮 | 保留意图，等待可用 | +| `scrollTop ≤ 0.5`，其他条件允许 | 复用分页锚点，前插后补偿一次 | 通过当前锚点保位消费 | +| 折叠、展开或已有跟随动画 | 保留缓存，退回按钮 | 数据可用即通过既有保位消费 | +| 手指按住或实际内层滚动 | 优先于动画判断保留意图，结束后重查 | 数据可用即通过既有保位消费 | +| 鼠标按住、滚动条拖动或定位能力暂不可用 | 保留缓存，退回按钮 | 消费可用页；没有锚点时不保证位置稳定 | | 自动预取后恢复底部跟随 | 保留缓存，取消自动消费 | 点击已暂停底部跟随 | | 底部继续流式输出 | 本身不阻止消费 | 本身不阻止消费 | @@ -35,9 +35,9 @@ Status: proposed ### 位置与状态展示 -会话实际滚动容器设置 `overscroll-behavior-y: none`，关闭纵向边界回弹和滚动传递，不实现另一套惯性动画。顶部前插复用既有分页位置记录和补偿计算，只应用一次并释放；非顶部持续保位保持原生命周期。两种模式用相同公式恢复消费前刚采集的屏幕位置，只区别于应用后是否继续保位。一次补偿不重建采集期间的读者移动，也不重新启动惯性。 +会话实际滚动容器设置 `overscroll-behavior-y: none`，关闭纵向边界回弹和滚动传递，不实现另一套惯性动画。自动顶部前插复用既有分页位置记录和补偿计算，只应用一次并释放；手动消费和静止保位保持原有持续保位生命周期。两种模式用相同公式恢复消费前刚采集的屏幕位置，只区别于应用后是否继续保位。一次补偿不重建采集期间的读者移动，也不重新启动惯性。即使可见 Node 顺序未变化，消费完成也会结算并清除一次补偿记录；视图关闭或历史失败时直接丢弃，不再应用。 -历史控件各状态保持同高。网络请求显示“正在加载”；仍有消费意图且未按住输入时显示“正在处理”；自动回退或输入按住时显示“加载更早”；失败显示“重新加载”；最后一页消费完成后显示“已到会话开头”。控件不参与原生锚定。Session 的 `historyWasPartial` 记录本实例曾显示不完整历史，组件重挂载和重连保留此事实；首次打开即完整的短会话不添加历史行。 +历史控件各状态保持同高。“正在加载”和“正在处理”仍可点击，以加入当前请求或消费缓存；只有确认到头才禁用按钮。网络请求显示“正在加载”；仍有消费意图且未按住输入时显示“正在处理”；自动回退或输入按住时显示“加载更早”；失败显示“重新加载”；最后一页消费完成后显示“已到会话开头”。控件不参与原生锚定。Session 的 `historyWasPartial` 记录本实例曾显示不完整历史，组件重挂载和重连保留此事实；首次打开即完整的短会话不添加历史行。 ### 范围与既有决策 @@ -61,7 +61,7 @@ Status: proposed - 请求、暂存与消费期间不重复取页；尾部流式继续更新。 - 自动退回按钮后不会被普通事件恢复；触摸、内层滚动和手动点击保留约定的续接。 -- 静止和顶部前插保持旧内容位置；普通惯性消费不新增 JS 外层滚动写入。 +- 静止和顶部前插保持旧内容位置；普通自动惯性消费不新增 JS 外层滚动写入。 - 页面失败、无进展、到头、重连和显式导航均有明确结果，不循环请求或消费过期页。 - 控件各状态同高；既有折叠驱动保持行为，真实浏览器验收区分当前构建、输入模拟和未覆盖情况。 @@ -69,6 +69,6 @@ Status: proposed 一次补偿恢复采集的屏幕位置，不估算读者速度。采集到应用之间的输入或浏览器移动可能影响最终运动，因此需要验证实际浏览器和输入序列。共享公式不叠加可能已包含原生锚定的滚动位置增量。见[技术风险](2026-11-06-chat-history-consumption-details.zh.md#risks)。 -原生锚定适用性不能只由 CSS 支持检查保证。首次尺寸观察前不会自动预取；500ms 采样可能使高速滚动先到顶部。前插后的同步几何读取是一次补偿的成本，不能仅凭调用位置保证没有额外布局。静止手动消费若始终无法获得锚点，完成意图仍会等待，需要覆盖该特殊结构。 +原生锚定适用性不能只由 CSS 支持检查保证。首次尺寸观察前不会自动预取；500ms 采样可能使高速滚动先到顶部。前插后的同步几何读取是一次补偿的成本，不能仅凭调用位置保证没有额外布局。没有可用锚点的手动消费可能移动可见内容。手动消费也可能在已有触摸手势进行中建立持续保位，约束该手势或其惯性，直到新的阅读输入释放锚点；见[手动分页限制](../../../../packages/client/ui-chat/README.zh.md#known-limitations-and-deferred-work)。 审计与测量尚在进行，本记录保持 proposed。此前 Chrome/Safari 手工反馈和不同迭代的 headless 结果不等同于最终源码的全矩阵通过；真实滚动条拖动、同页面重连、多平台输入和异常清理分支应按实际证据单独报告。 diff --git a/apps/web/tests/chat-scroll-contract.e2e.ts b/apps/web/tests/chat-scroll-contract.e2e.ts index fd5f778c1e..4b76660b22 100644 --- a/apps/web/tests/chat-scroll-contract.e2e.ts +++ b/apps/web/tests/chat-scroll-contract.e2e.ts @@ -6,7 +6,7 @@ import { access, mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { fileURLToPath } from 'node:url' -import type { Browser, Page } from 'playwright' +import type { Browser, Page, Route } from 'playwright' import { chromium } from 'playwright' import { afterAll, beforeAll, describe, expect, it } from 'vitest' import type { StreamChunk } from '@deepseek-ai/dsh-llm' @@ -476,23 +476,41 @@ async function expectMarkerAboveComposer(page: Page, marker: string): Promise { - await wheelToHistoryStart(page) - const older = page.getByRole('button', { name: 'Load earlier', exact: true }) - const loading = page.getByRole('button', { name: 'Loading…', exact: true }) - await older.waitFor({ timeout: 10_000 }) - const anchor = await visibleFlowAnchor(page) - const before = await loadedFlowRows(page) - await older.click() - await expect.poll(async () => ( - await loadedFlowRows(page) > before && await loading.count() === 0 - ), { timeout: 30_000 }).toBe(true) - await nextPaint(page) - if (await page.getByRole('button', { name: 'Load earlier', exact: true }).count() === 0) { - expect(await page.locator('[data-turn-process][aria-expanded="false"]').count()).toBeGreaterThan(0) - return +async function loadEarlierWithAnchor(page: Page): Promise { + // Auto-prefetch may start while scrolling to the button. Hold its response until the reading anchor is captured. + const gate = Promise.withResolvers() + const pending = new Set>() + const hold = (route: Route): Promise => { + const work = gate.promise.then(() => route.continue()) + pending.add(work) + return work.finally(() => { pending.delete(work) }) + } + await page.route('**/api/session/page', hold) + try { + await wheelToHistoryStart(page) + const history = page.locator('[data-chat-history]') + const older = history.getByRole('button') + await older.waitFor({ timeout: 10_000 }) + if (await older.isDisabled()) return false + const anchor = await visibleFlowAnchor(page) + const before = await loadedFlowRows(page) + await older.click() + gate.resolve(undefined) + await expect.poll(async () => ( + await loadedFlowRows(page) > before && await history.getAttribute('data-chat-history') === 'idle' + ), { timeout: 30_000 }).toBe(true) + await nextPaint(page) + if (await older.isDisabled()) { + expect(await page.locator('[data-turn-process][aria-expanded="false"]').count()).toBeGreaterThan(0) + return true + } + await expectSameFlowTop(page, anchor) + return true + } finally { + gate.resolve(undefined) + await page.unroute('**/api/session/page', hold) + await Promise.all([...pending]) } - await expectSameFlowTop(page, anchor) } async function fileExists(path: string): Promise { @@ -1300,6 +1318,8 @@ describe('web e2e: long Chat scroll contract', () => { let releaseHistory = (): void => {} let held = false + let historyRequests = 0 + const heldContinued = Promise.withResolvers() let releaseGate: (() => void) | undefined const gate = new Promise((resolve) => { releaseGate = resolve }) releaseHistory = () => { releaseGate?.() } @@ -1308,12 +1328,17 @@ describe('web e2e: long Chat scroll contract', () => { method?: string payload?: { args?: { request?: { beforeSeq?: number } } } } - if (!held && request.method === 'session/page' - && request.payload?.args?.request?.beforeSeq !== undefined) { + const historyRequest = request.method === 'session/page' && request.payload?.args?.request?.beforeSeq !== undefined + if (historyRequest) historyRequests += 1 + if (!held && historyRequest) { held = true await gate - } - await route.continue() + try { + await route.continue() + } finally { + heldContinued.resolve(undefined) + } + } else await route.continue() }) const settled = world.scaffold.whenTurnSettled(60_000) @@ -1325,8 +1350,12 @@ describe('web e2e: long Chat scroll contract', () => { await world.page.getByText(LIVE_TEXT_FIRST, { exact: false }).last().waitFor({ timeout: 15_000 }) await wheelToHistoryStart(world.page) const beforeRows = await loadedFlowRows(world.page) - await world.page.getByRole('button', { name: 'Load earlier', exact: true }).click() await expect.poll(() => held, { timeout: 10_000 }).toBe(true) + const history = world.page.locator('[data-chat-history]') + await expect.poll(() => history.getAttribute('data-chat-history')).toBe('loading') + // The upward gesture already requested the page. A click joins that request even though its label is loading. + await history.getByRole('button').click() + expect(historyRequests).toBe(1) await wheelTranscript(world.page, 420) const readerAnchor = await visibleFlowAnchor(world.page) @@ -1344,6 +1373,9 @@ describe('web e2e: long Chat scroll contract', () => { } finally { releaseText() releaseHistory() + if (held) await heldContinued.promise + // Release both producers before disposing the Session even if an assertion above fails. + await settled } await settled @@ -1353,10 +1385,10 @@ describe('web e2e: long Chat scroll contract', () => { let additionalPages = 0 while (additionalPages < 8) { - await wheelToHistoryStart(world.page) - if (await world.page.getByRole('button', { name: 'Load earlier', exact: true }).count() === 0) break - await loadEarlierWithAnchor(world.page) - additionalPages += 1 + const before = await loadedFlowRows(world.page) + const consumed = await loadEarlierWithAnchor(world.page) + if (consumed || await loadedFlowRows(world.page) > before) additionalPages += 1 + if (await world.page.locator('[data-chat-history]').getByRole('button').isDisabled()) break } expect(additionalPages).toBeGreaterThan(0) // The whole log is loaded: turn 1's unique marker renders in the @@ -1364,7 +1396,9 @@ describe('web e2e: long Chat scroll contract', () => { // page remains. expect(await world.page.locator('[data-conversation-scroll]') .getByText(HISTORY_FIXTURE.markers.user(1), { exact: false }).count()).toBe(1) - expect(await world.page.getByRole('button', { name: 'Load earlier', exact: true }).count()).toBe(0) + const history = world.page.locator('[data-chat-history]') + expect(await history.getAttribute('data-chat-history')).toBe('idle') + expect(await history.getByRole('button', { name: 'Start of conversation', exact: true }).isDisabled()).toBe(true) assertClean(world) }) }, 180_000) diff --git a/packages/client/ui-chat/README.i18n.yaml b/packages/client/ui-chat/README.i18n.yaml index dd8cf646da..0ea68825c1 100644 --- a/packages/client/ui-chat/README.i18n.yaml +++ b/packages/client/ui-chat/README.i18n.yaml @@ -36,8 +36,8 @@ en: e2b02942a55fb669 zh: a38bbee0cc69ae4e /deepseek-ai-dsh-client-ui-chat/scroll-ownership: - en: b26ab997d694ef90 - zh: 1f8a16ba02bab301 + en: 1b12aa6c505dde3b + zh: 88e1654dd63816d7 /deepseek-ai-dsh-client-ui-chat/model-experience: en: c17b4dd43a61c569 zh: f7414309810c3a08 @@ -45,8 +45,8 @@ en: 15a7151ff9268f15 zh: 6eae68c718be579b /deepseek-ai-dsh-client-ui-chat/known-limitations-and-deferred-work: - en: 811787a5d2feaa90 - zh: 3f1697bbc1fc7691 + en: b3b03e8e2fe371a1 + zh: c5eb8b7205e0e880 /deepseek-ai-dsh-client-ui-chat/known-limitations-and-deferred-work/dev-note: en: e67f633dc6fe7773 zh: e88aeb617ffb74ea diff --git a/packages/client/ui-chat/README.md b/packages/client/ui-chat/README.md index e00a884771..672da9097d 100644 --- a/packages/client/ui-chat/README.md +++ b/packages/client/ui-chat/README.md @@ -186,7 +186,7 @@ Within a process group, wheel, touchstart, and any pointerdown interrupt an acti `useChatReading` owns follow policy, sampled reader input, and semantic memory; `useChatNavigation` owns turn jumps and requests preservation from the viewport. Reading gestures release the paging anchor, but composer clicks, typing, and non-scrolling keys retain it. A deferred history request captures its anchor at consumption rather than at request time. `useChatScroll` coordinates their committed inputs. Explicit navigation carries its measured landing into reading policy, so it does not rediscover the known target with a hit test. -History prefetch uses the existing 500ms reading sample and content/size notifications, with one distance check before requesting within two viewport heights. At most one page waits for consumption. Touch and inner scrolling retain automatic intent until release; active motion, pointer holds, or an unavailable anchor return automatic loading to the button without retrying the cached page. Explicit clicks retain intent until consumption, exhaustion, failure, or superseding navigation. Stationary consumption uses retained positioning; ordinary outer scrolling uses browser anchoring. At scrollTop ≤ 0.5, the first paging anchor uses the same captured-screen-position restoration as persistent preservation, once, then releases immediately; it does not add scroll-position deltas that native anchoring may already have applied. The actual Chat scrollport disables vertical overscroll bounce, without adding a scrolling animation. The history row keeps its height through loading, processing, retry, and exhaustion; initially complete Sessions omit it. +History prefetch uses the existing 500ms reading sample and content/size notifications, with one distance check before requesting within two viewport heights. At most one page waits for consumption. Touch and actual inner scrolling retain automatic intent before motion blockers are evaluated, then recheck after release; other active motion, pointer holds, or an unavailable anchor return automatic loading to the button without retrying the cached page. Loading and processing buttons remain clickable, joining the current request without duplication. Explicit clicks retain intent until consumption, exhaustion, failure, or superseding navigation; once a page is available they bypass automatic blockers, use the first paging anchor at the top or the current reading anchor elsewhere with a paging fallback, and consume through existing preservation without waiting for animation completion. If no anchor is available, manual consumption still proceeds without a position guarantee. Automatic stationary consumption uses retained positioning; ordinary outer scrolling uses browser anchoring. For automatic consumption at scrollTop ≤ 0.5, the first paging anchor uses the same captured-screen-position restoration as persistent preservation, once, then releases immediately; it does not add scroll-position deltas that native anchoring may already have applied. The one-shot record is settled on history consumption completion even when the visible Node order stays unchanged; view closure or history failure discards it without application. The actual Chat scrollport disables vertical overscroll bounce, without adding a scrolling animation. The history row keeps its height through loading, processing, retry, and exhaustion; initially complete Sessions omit it. Active-Turn highlighting is approximate: `readVisibleTurn` binary-searches the flow slot's direct Node/Group boxes (or the column's in standalone rendering) and retains the preceding candidate in gaps. It neither hit-tests the document nor searches Group members or all Turn markers. Empty Seats retain zero-height in-flow boxes so outer positions remain ordered without extra spacing. This lookup does not change semantic position capture or paging compensation. @@ -213,6 +213,8 @@ None; Chat presentation does not assemble or mutate provider requests. +- **Manual paging during an active gesture** — manual consumption captures a persistent anchor when its page arrives, even if a touch gesture is already in progress. Subsequent movement can be restored to that anchor, constraining the gesture or its momentum until a new reading input releases it. The automatic touch deferral does not protect this explicit manual path. + - **Running-icon resolution** — the animated whale uses a 28×28 image at a default 14×14 CSS size. Larger content fonts or device pixel ratios above 2 can soften the raster stroke; the static fallback remains vector-based. Browser image caching owns the animation clock, so remounting does not guarantee playback restarts at the first frame. - **Tool-change presentation** — The `developer-message` Definition shares context presentation with `input-message`. Tool-only developer messages name a single added or removed tool inline without expansion. Multiple changes show added/removed counts and expand to comma-separated tool lists, one line per change kind. Mixed content uses the generic context presentation. diff --git a/packages/client/ui-chat/README.zh.md b/packages/client/ui-chat/README.zh.md index 29e8a373fa..29a1951662 100644 --- a/packages/client/ui-chat/README.zh.md +++ b/packages/client/ui-chat/README.zh.md @@ -186,7 +186,7 @@ Chat 会在历史前插与 renderer 重新挂载时恢复语义锚点，并且 `useChatReading` 负责跟随策略、读者输入采样和语义位置记忆；`useChatNavigation` 负责轮次跳转，并向视口请求位置保持。阅读手势释放分页锚点，composer 内的点击、打字和非滚动按键则保留锚点。延迟消费的历史请求在消费时采集锚点，不在发请求时采集。`useChatScroll` 协调已提交的输入。显式导航把测得的落点交给阅读策略，因此无需通过命中测试重新寻找已知目标。 -历史预取复用已有 500ms 阅读采样及内容／尺寸通知，在请求前确认距顶部两屏以内，最多一页等待消费。触摸和内层滚动保留自动意图至释放；动画、指针按住或无法取得锚点使自动加载退回按钮，不自动重试缓存页。明确点击保留意图，直到消费、到头、失败或被新导航取代。静止消费使用保位，普通外层滚动使用浏览器锚定；scrollTop ≤ 0.5 时，首条分页锚点与持续保位共用采集屏幕位置的恢复公式，只应用一次便立即释放；不再叠加可能已由原生锚定产生的滚动位置增量。实际 Chat 滚动区关闭纵向越界回弹，不新增滚动动画。历史行在加载、处理、重试和到头时保持高度，首次打开就完整的 Session 不显示该行。 +历史预取复用已有 500ms 阅读采样及内容／尺寸通知，在请求前确认距顶部两屏以内，最多一页等待消费。触摸和实际内层滚动优先于动画阻塞判断保留自动意图，释放后重查；其余动画、指针按住或无法取得锚点使自动加载退回按钮，不自动重试缓存页。加载和处理状态的按钮仍可点击，加入当前请求而不重复请求。明确点击保留意图，直到消费、到头、失败或被新导航取代；页面可用后绕过自动阻塞，在顶部使用首条分页锚点，其他位置使用当前阅读锚点并以分页锚点兜底，通过既有保位消费，不等待动画结束。没有可用锚点时手动消费仍继续，但不保证位置稳定。自动静止消费使用保位，普通外层滚动使用浏览器锚定；自动消费在 scrollTop ≤ 0.5 时，首条分页锚点与持续保位共用采集屏幕位置的恢复公式，只应用一次便立即释放；不再叠加可能已由原生锚定产生的滚动位置增量。历史消费完成时，即使可见 Node 顺序未变化，也会结算一次补偿记录；视图关闭或历史失败时丢弃记录，不应用补偿。实际 Chat 滚动区关闭纵向越界回弹，不新增滚动动画。历史行在加载、处理、重试和到头时保持高度，首次打开就完整的 Session 不显示该行。 活跃轮次高亮采用近似定位：`readVisibleTurn` 对 flow 插槽的直属 Node/Group 布局盒二分（独立渲染时使用正文容器的直属布局盒），落在间隙时保留前方候选。它不做文档命中测试，也不查询 Group 成员或全部 Turn 标记。空 Seat 保留零高度的流内布局盒，使外层位置有序且不增加间距。这种定位不改变语义位置捕获或分页补偿。 @@ -213,6 +213,8 @@ Markdown 文件、链接、图片、引用 chip 和工具文件控件将明确 +- **进行中手势遇到手动分页**——手动消费在页面到达时采集持续保位锚点，即使触摸手势已经开始。随后移动可能被恢复到该锚点，约束手势或其惯性，直到新的阅读输入释放锚点。自动路径的触摸延迟不保护这一明确的手动路径。 + - **运行图标分辨率**——动态鲸鱼使用 28×28 图片，默认 CSS 尺寸为 14×14。较大的正文字号或大于 2 的设备像素比可能使位图描边变模糊；静态回退图标仍为矢量图。浏览器图片缓存管理动画时钟，因此重新挂载不保证从第一帧重新播放。 - **工具变更展示** — `developer-message` Definition 与 `input-message` 共用上下文展示。仅包含工具变更的 developer 消息在单个工具新增或移除时直接显示工具名，不提供展开操作。多个变更显示新增、移除数量，展开后按变更类型分别以逗号分隔显示一行工具列表。混合内容使用通用上下文展示。 diff --git a/packages/client/ui-chat/src/client/chat/ChatView.tsx b/packages/client/ui-chat/src/client/chat/ChatView.tsx index 8fa10ca0ec..c9143248a1 100644 --- a/packages/client/ui-chat/src/client/chat/ChatView.tsx +++ b/packages/client/ui-chat/src/client/chat/ChatView.tsx @@ -252,7 +252,7 @@ export function ChatView({ )} {(hasMore || historyWasPartial) && (
- + + )} +

+ )} + {result.truncated && ( +

{t('search.truncated', { count: String(result.count) })}

+ )} + {result.incomplete &&

{t('search.incomplete')}

} + + ) +} + +/** + * What the results area shows for a search the Host could not run: why, and a control that runs it again. + * @param props.failure - the failed search's Remote failure. + * @param props.onRetry - runs the search again. + * @param props.t - namespace-bound translate. + * @returns the notice. + */ +function SearchFailure({ failure, onRetry, t }: { + readonly failure: RemoteFailure + readonly onRetry: () => void + readonly t: TranslateNS<'sidebarFiles'> +}): ReactNode { + return ( +
+

{t('search.failed', { message: failureLine(t, failure) })}

+ +
+ ) +} + /** * The navigator Factory's component. * @param props - the tab it is drawn for, the pane's store records, the tree's face, and copy. * @returns the navigator, or nothing while hidden or without a workspace. */ export function FileNavigator({ - useTabInfo, page, onOpen, sessionId, useSessions, useStore, actions, useRowWidth, useCanOpen, - attach, placeBeside, hold, toggle, scrolled, bodyDrawn, currentDrawn, navigatorDrawn, t, renderFactorySlot, useContextMenu, + useTabInfo, page, onOpen, sessionId, useSessions, useStore, actions, useRowWidth, useCanOpen, useSearchPreferences, + attach, placeBeside, hold, toggle, scrolled, bodyDrawn, currentDrawn, navigatorDrawn, + setQuery, toggleSearch, retrySearch, setSearchPreferences, searchDrawn, t, renderFactorySlot, useContextMenu, }: FactoryComponentPropsOf<'sidebar.files.navigator'>): ReactNode { const { tab, panel } = useTabInfo() const { getTriggerProps } = useContextMenu() const paneId = panel.id + // A Session's working directory comes from its header and never changes once + // known, so every search result the pane's store holds is rooted at it. const cwd = useSessions(sessions => sessions.byId[sessionId]?.cwd) const state = useStore(store => store.byPane[paneId]) const rowWidth = useRowWidth(paneId) @@ -83,17 +179,38 @@ export function FileNavigator({ // Measured by the face; the viewport rule is CSS's, read again when a file opens. const mode = navigatorMode(rowWidth) const tree = state?.tree?.root === cwd ? state?.tree : undefined + // The face resolves every key to a source, so the renderer never reports one unavailable. + /* v8 ignore next -- the fallback only satisfies the keyed hook's optional snapshot type. */ + const preferences: WorkspaceSearchPreferences = useSearchPreferences(cwd ?? '') ?? defaultSearchPreferences() + const { showHidden } = preferences + const query = state?.query ?? '' + const searchBarOpen = !page || state?.searchBarOpen === true + const search = state?.search + const pending = search?.pending === true + const result = search?.result + const found = result?.kind === 'found' ? result : undefined const [dragging, setDragging] = useState(false) + const [draft, setDraft] = useState(undefined) + const settings = draft !== undefined const drag = useRef<{ readonly x: number; readonly width: number } | undefined>(undefined) + const input = useRef(null) + const trigger = useRef(null) // Stable, so each element registers once per drawing rather than on every render. const navigatorRef = useCallback( (element: HTMLElement | null) => { navigatorDrawn(paneId, element, !page) }, [navigatorDrawn, paneId, page], ) + // Also new when the container switches between the tree and the results, so + // the face restores that view's scroll offset once its rows are drawn. + const resultsShown = search !== undefined const bodyRef = useCallback( (body: HTMLDivElement | null) => { bodyDrawn(paneId, tab.contentId, body) }, - [bodyDrawn, paneId, tab.contentId], + [bodyDrawn, paneId, tab.contentId, resultsShown], ) + const inputRef = useCallback((element: HTMLInputElement | null) => { + input.current = element + searchDrawn(paneId, element) + }, [searchDrawn, paneId]) const currentRef = useCallback((row: HTMLElement | null) => { currentDrawn(paneId, row) }, [currentDrawn, paneId]) useEffect(() => attach(paneId, tab.id, tab.signal), [attach, paneId, tab.id, tab.signal]) @@ -109,13 +226,23 @@ export function FileNavigator({ const root = tree?.root const levels = tree?.levels const expanded = tree?.expanded - const entries = useMemo(() => levels === undefined ? undefined : workspaceEntries(levels), [levels]) + const entries = useMemo(() => levels === undefined ? undefined : workspaceEntries(levels, showHidden), [levels, showHidden]) const directories = useMemo( () => root === undefined || levels === undefined || expanded === undefined ? undefined - : workspaceDirectories({ root, levels, expanded }, t), - [root, levels, expanded, t], + : workspaceDirectories({ root, levels, expanded }, t, showHidden), + [root, levels, expanded, t, showHidden], ) + const resultTree = found !== undefined && found.count > 0 ? found.tree : undefined + const resultCurrent = useMemo( + () => resultTree === undefined || relativePath === undefined ? undefined : locateFile(resultTree, relativePath).current, + [resultTree, relativePath], + ) + const resultEntries = useMemo( + () => resultTree === undefined ? undefined : workspaceEntries(resultTree.levels, showHidden), + [resultTree, showHidden], + ) + const resultDirectories = useMemo(() => resultTree === undefined ? undefined : searchDirectories(resultTree), [resultTree]) // Kept between renders, so the tree redraws only when what it shows changes, // not on every measure of the row or render of the host. const view = useMemo(() => { @@ -131,12 +258,27 @@ export function FileNavigator({ : { kind, value: { sessionId, path: entry.path, paneId } }), } }, [tree, paneId, sessionId, toggle, canOpen, onOpen, getTriggerProps]) + // The results keep their own expansion; every other gesture acts as on the tree. + const resultView = useMemo(() => view === undefined || resultTree === undefined ? undefined : { + ...view, + expanded: resultTree.expanded, + onToggle: (entry: TreeEntry) => { toggleSearch(paneId, entry.path) }, + }, [view, resultTree, toggleSearch, paneId]) + const closeSettings = useCallback((rules: readonly ExclusionRule[], refocus: boolean) => { + setDraft(undefined) + if (refocus) trigger.current?.focus() + if (cwd !== undefined && rules !== preferences.rules) setSearchPreferences(cwd, { rules }) + }, [cwd, preferences.rules, setSearchPreferences]) if (cwd === undefined || !open) return null const geometry = navigatorGeometry(rowWidth, state?.width ?? DEFAULT_NAVIGATOR_WIDTH) // The page's tree always fills the page; a preview's follows its row. const shown = page ? undefined : mode const clamp = (next: number): number => Math.min(Math.max(next, geometry.minimum), geometry.maximum) + const clear = (): void => { + setQuery(paneId, cwd, '') + input.current?.focus() + } return ( ) } diff --git a/packages/client/ui-sidebar-files/src/client/FileTree.module.css b/packages/client/ui-sidebar-files/src/client/FileTree.module.css index d7bd27a608..fe7cf766fc 100644 --- a/packages/client/ui-sidebar-files/src/client/FileTree.module.css +++ b/packages/client/ui-sidebar-files/src/client/FileTree.module.css @@ -111,3 +111,9 @@ color: var(--dsw-alias-label-tertiary); font-size: 12px; } + +/* The occurrences of the highlighted text in a name. */ +.match { + color: var(--dsw-alias-state-business-primary); + background: transparent; +} diff --git a/packages/client/ui-sidebar-files/src/client/FileTree.tsx b/packages/client/ui-sidebar-files/src/client/FileTree.tsx index 7a9b129896..30516ba28e 100644 --- a/packages/client/ui-sidebar-files/src/client/FileTree.tsx +++ b/packages/client/ui-sidebar-files/src/client/FileTree.tsx @@ -5,7 +5,8 @@ * path, adds the directories the paths imply, orders every drawn level, and * draws the rows and their icons. It redraws when its * props or the occurrence's local selection change, and orders a drawn level - * again only when the names or directory kinds in that level change. + * again only when the names or directory kinds in that level change. Given a + * query to highlight, it marks each occurrence of the query inside a name. */ import { memo, useMemo } from 'react' import type { ReactNode } from 'react' @@ -14,6 +15,7 @@ import type { FactoryComponentPropsOf } from '@deepseek-ai/dsh-client-ui-slots' import type { ContextMenuTriggerProps } from '@deepseek-ai/dsh-client-ui-context-menu/client' import { FileTypeIcon, classifyFileType, IconFolderCloseRegular, IconFolderOpenRegular } from '@deepseek-ai/dsh-client-ui-primitives' import type {} from './locales.ts' +import { matchRanges } from './search-tree.ts' import css from './FileTree.module.css' /** What a row is: a directory toggles, a file opens, and anything else is listed without either. */ @@ -51,6 +53,13 @@ export interface TreeDirectoryState { readonly failure?: string /** Entries are missing from the listing. */ readonly truncated?: true + /** + * The caller left entries out, which changes what the directory says while + * none is drawn: `hidden` says that its hidden entries are not shown, and + * `unmatched`, for entries that are a selection such as search matches, says + * nothing instead of that it is empty. + */ + readonly omitted?: 'hidden' | 'unmatched' } /** The `sidebar.files.tree` Factory's input: data and callbacks only. */ @@ -70,6 +79,8 @@ export interface FileTreeInput { readonly expanded: readonly string[] /** Listing state by directory key, root included; a directory absent here is listed and complete. */ readonly directories?: Readonly> | undefined + /** Text whose occurrences inside each name are highlighted, ignoring case; nothing is highlighted without it. */ + readonly highlight?: string | undefined /** Key of the file to mark as current, when it is drawn. */ readonly current?: string | undefined /** Receives the current file's main button while drawn. @param row - the button, or `null` once it goes. */ @@ -190,12 +201,27 @@ interface TreeContext { readonly Accessory: ReturnType['useFactorySlot']> } +/** One entry's name, with the occurrences of the text to highlight marked. */ +function Name({ name, highlight }: { name: string; highlight: string | undefined }): ReactNode { + const ranges = highlight === undefined ? [] : matchRanges(name, highlight) + if (ranges.length === 0) return {name} + const parts: ReactNode[] = [] + let at = 0 + for (const [start, end] of ranges) { + if (start > at) parts.push(name.slice(at, start)) + parts.push({name.slice(start, end)}) + at = end + } + if (at < name.length) parts.push(name.slice(at)) + return {parts} +} + /** One entry's row, and its children when it is an expanded directory. */ function Entry({ node, tree }: { node: TreeNode; tree: TreeContext }): ReactNode { const { input, Accessory } = tree const { entry, kind } = node const hidden = entry.meta?.hidden === true && css.hidden - const label = {entry.name} + const label = if (kind === 'directory') { const expanded = tree.expanded.has(entry.path) return ( @@ -260,7 +286,11 @@ function Level({ path, tree }: { path: string; tree: TreeContext }): ReactNode { return ( <> {state?.failure !== undefined &&
  • {state.failure}
    • } - {level.length === 0 && state?.failure === undefined &&
  • {t('empty')}
    • } + {level.length === 0 && state?.failure === undefined && state?.omitted !== 'unmatched' && ( + state?.omitted === 'hidden' + ?
  • {t('emptyHidden')}
    • + :
  • {t('empty')}
    • + )} {order.map(index => level[index]).filter(node => node !== undefined) .map(node => )} {state?.truncated === true &&
  • {t('truncated')}
    • } diff --git a/packages/client/ui-sidebar-files/src/client/FilesBody.module.css b/packages/client/ui-sidebar-files/src/client/FilesBody.module.css index 383ecfa47d..4bddc11bb1 100644 --- a/packages/client/ui-sidebar-files/src/client/FilesBody.module.css +++ b/packages/client/ui-sidebar-files/src/client/FilesBody.module.css @@ -36,6 +36,8 @@ height: 100%; min-width: 0; min-height: 0; + /* The search bar's height, which also places the settings panel below it. */ + --files-search-bar-height: 42px; color: var(--dsw-alias-label-primary); font-size: var(--dsh-content-font-size-secondary, 13px); line-height: 1.5; @@ -198,6 +200,23 @@ font-size: 12px; } +/* "No matches" with the information button whose tooltip names the filters. */ +.emptyNote { + display: flex; + align-items: center; + gap: 2px; +} + +/* The information button keeps the note's line height. */ +.infoButton { + flex: none; + width: 20px; + height: 20px; + margin-block: -2px; + padding: 0; + color: var(--dsw-alias-label-caption); +} + .status { display: flex; flex-direction: column; @@ -252,3 +271,262 @@ .titleIcon { flex: none; } + +/* The navigator's search bar above the tree: the settings control, the + field, the hidden-entry control, and the clear control in one bordered box, + drawn like the shared `Input` atom. */ +.searchBar { + flex: 0 0 auto; + box-sizing: border-box; + height: var(--files-search-bar-height); + padding: 8px 8px 4px; +} + +.searchField { + display: flex; + gap: 2px; + align-items: center; + box-sizing: border-box; + height: 100%; + padding: 0 2px; + background: var(--dsw-alias-bg-layer-1); + border: 0.5px solid var(--dsw-alias-border-l4); + border-radius: var(--dsw-radius-md); +} + +.searchField:focus-within { + border-color: var(--dsw-alias-state-business-primary); +} + +/* The bar's icon controls: the header tool at a size that fits inside the field. */ +.searchTool { + width: 24px; + height: 24px; + padding: 4px; +} + +.searchTool svg { + width: 14px; + height: 14px; +} + +.searchInput { + flex: 1 1 auto; + min-width: 0; + padding: 0 4px; + color: var(--dsw-alias-label-primary); + font: inherit; + background: transparent; + border: none; + outline: none; +} + +.searchInput::placeholder { + color: var(--dsw-alias-label-dimmed); +} + +/* The bar's own clear control replaces the browser's. */ +.searchInput::-webkit-search-cancel-button { + display: none; +} + +/* The results status between the bar and the results: the count, and a spinner while a search is under way. */ +.searchStatus { + display: flex; + flex: 0 0 auto; + gap: 6px; + align-items: center; + min-height: 18px; + margin: 0; + padding: 2px 18px 0; + color: var(--dsw-alias-label-tertiary); + font: var(--dsw-font-xxs-12); +} + +/* One sentence of the results status: the count. */ +.searchStatusLine { + display: block; + min-width: 0; +} + +.searchSpinner { + flex: none; +} + +/* A failed search in the results' place: the reason, and the retry control below it on the note's inset. */ +.searchFailure { + display: flex; + flex-direction: column; + gap: 6px; + align-items: flex-start; + padding-bottom: 6px; +} + +.searchRetry { + margin-left: 10px; +} + +/* The `.gitignore` choice above the add field, at a rule's height. */ +.settingsOption { + border-top: none; +} + +/* The live region stays mounted while the query is blank, without taking room. */ +.searchStatusIdle { + min-height: 0; + padding: 0; +} + +/* The settings panel floats over the tree below the search bar, inside the navigator. */ +.settings { + position: absolute; + top: var(--files-search-bar-height); + right: 8px; + bottom: 8px; + left: 8px; + z-index: 3; + display: flex; + flex-direction: column; + min-height: 0; + padding: 6px 4px 6px 10px; + background: var(--dsw-alias-bg-base); + border-radius: var(--dsw-radius-lg); + outline: none; + box-shadow: var(--dsw-elevation-prominent); +} + +.settingsHeader { + display: flex; + flex: 0 0 auto; + gap: 2px; + align-items: center; +} + +.settingsTitle { + flex: 1 1 auto; + min-width: 0; + font-weight: 500; +} + +.settingsDescription { + flex: 0 0 auto; + margin: 0 0 4px; + padding-right: 6px; + color: var(--dsw-alias-label-tertiary); + font: var(--dsw-font-xxs-12); +} + +/* One rule, and the add field above them at the same height. */ +.rule { + display: flex; + flex: 0 0 auto; + gap: 6px; + align-items: center; + box-sizing: border-box; + min-height: 32px; + margin: 0; + padding: 0 0 0 2px; + border-top: 0.5px solid var(--dsw-alias-border-l3); +} + +.addRule { + padding-right: 6px; +} + +/* The add row's plus glyph rides the tertiary ink, as the tree's folder glyph does. */ +.addRule .icon { + flex: 0 0 auto; + color: var(--dsw-alias-label-tertiary); +} + +/* The add field draws no box of its own, so its row's rule marks where typing goes. */ +.addRule:focus-within { + border-top-color: var(--dsw-alias-state-business-primary); +} + +.ruleInput { + flex: 1 1 auto; + min-width: 0; + padding: 0; + color: var(--dsw-alias-label-primary); + font: inherit; + background: transparent; + border: none; + outline: none; +} + +.ruleInput::placeholder { + color: var(--dsw-alias-label-dimmed); +} + +.settingsError { + flex: 0 0 auto; + margin: 0 0 4px; + color: var(--dsw-alias-state-error-primary); + font: var(--dsw-font-xxs-12); +} + +/* Only the list scrolls; the header and the add field stay put. */ +.rules { + flex: 1 1 auto; + min-height: 0; + margin: 0; + padding: 0; + overflow-y: auto; + list-style: none; + overscroll-behavior: contain; +} + +.ruleLabel { + display: flex; + flex: 1 1 auto; + gap: 6px; + align-items: center; + min-width: 0; + cursor: pointer; +} + +.ruleCheck { + flex: 0 0 auto; + width: 14px; + height: 14px; + margin: 0 1px; + accent-color: var(--dsw-alias-state-business-primary); +} + +.ruleCheck:focus-visible { + outline: var(--dsw-focus-ring-width) solid var(--dsw-focus-ring-color, var(--dsw-alias-state-business-primary)); + outline-offset: 2px; +} + +.ruleName { + min-width: 0; + padding: 6px 0; + overflow-wrap: anywhere; +} + +/* A rule the reader turned off stays listed, in the tertiary ink. */ +.ruleOff .ruleName { + color: var(--dsw-alias-label-tertiary); +} + +@media (pointer: coarse) { + .searchTool { + width: 32px; + height: 32px; + } + + .navigator { + --files-search-bar-height: 50px; + } + + .rule { + min-height: 44px; + } + + /* Below 16px, mobile browsers zoom the page when a field takes focus. */ + .searchInput, + .ruleInput { + font-size: 16px; + } +} diff --git a/packages/client/ui-sidebar-files/src/client/FilesBody.tsx b/packages/client/ui-sidebar-files/src/client/FilesBody.tsx index b1361cf415..f7f5ae385f 100644 --- a/packages/client/ui-sidebar-files/src/client/FilesBody.tsx +++ b/packages/client/ui-sidebar-files/src/client/FilesBody.tsx @@ -1,10 +1,9 @@ /** * The `files` page: a file browser with no file chosen yet. * - * Its header spans the page: the workspace root's path, the tree's reload and - * the directory actions, in the order a preview's header ends with its own - * reload and file actions. The pane's navigator fills the page below it and - * stays shown, so the page draws no toggle; picking a file leaves this page as + * Its header spans the page: the workspace root's path, the tree's reload, + * the search-bar toggle, and the directory actions. The pane's navigator fills the page below it and + * stays shown, so the page draws no navigator toggle; picking a file leaves this page as * it is and opens the file where the right Sidebar places it by default * (`FileNavigator`). */ @@ -15,7 +14,7 @@ import type { InjectFace, PropsLocale, PropsRenderFactories, PropsRenderSlots, PropsRuntime, PropsStore, } from '@deepseek-ai/dsh-client-ui-slots' import { - IconPauseOutlineRegular, IconPlayOutlineRegular, IconRefreshOutlineRegular, PathLabel, Tooltip, + IconPauseOutlineRegular, IconPlayOutlineRegular, IconRefreshOutlineRegular, IconSearchOutlineRegular, PathLabel, Tooltip, } from '@deepseek-ai/dsh-client-ui-primitives' import type { FilesInjected } from './face.ts' import type {} from './FileNavigator.tsx' @@ -38,7 +37,10 @@ export type FilesBodyProps = * @returns the header above the pane's navigator, or the workspace line. */ export function FilesBody(props: FilesBodyProps): ReactNode { - const { useTabInfo, sessionId, useSessions, useStore, refresh, setAutoRefresh, renderSlot, renderFactorySlot, t, useContextMenu } = props + const { + useTabInfo, sessionId, useSessions, useStore, actions, refresh, focus, placeBeside, + setAutoRefresh, renderSlot, renderFactorySlot, t, useContextMenu, + } = props const { tab, panel } = useTabInfo() const { getTriggerProps } = useContextMenu() const onOpen = useCallback((address: string): void => { @@ -55,6 +57,8 @@ export function FilesBody(props: FilesBodyProps): ReactNode { const tree = store.byPane[panel.id]?.tree return tree?.root === cwd ? tree?.autoRefresh : undefined }) + const searchBarOpen = useStore(store => store.byPane[panel.id]?.searchBarOpen) === true + useEffect(() => cwd === undefined ? undefined : placeBeside(tab.id), [cwd, placeBeside, tab.id]) if (cwd === undefined) { return ( @@ -90,6 +94,23 @@ export function FilesBody(props: FilesBodyProps): ReactNode { + + + {renderSlot('sidebar.right.tab.files.actions', { absolutePath: cwd })}
    {navigator}
    diff --git a/packages/client/ui-sidebar-files/src/client/SearchSettings.tsx b/packages/client/ui-sidebar-files/src/client/SearchSettings.tsx new file mode 100644 index 0000000000..8ab9bd5ef6 --- /dev/null +++ b/packages/client/ui-sidebar-files/src/client/SearchSettings.tsx @@ -0,0 +1,155 @@ +/** + * The search settings panel edits the navigator's draft of excluded directories. + * The navigator saves it when the panel closes, including through the magnifier. + * The `.gitignore` choice above the list is saved as soon as it changes, as + * the hidden-entry control in the search bar is. + * + * It floats over the navigator below the search field. The field that adds a + * directory stays above the list, which alone scrolls; a new rule goes to the + * top. The header's restore control appears while the list differs from the + * default list in any way, order included, and replaces it with that list. The close control, Escape, and focus leaving the panel request + * closure; focus moving to the magnifier leaves closure to its click. + */ +import { useId, useRef, useState } from 'react' +import type { ReactNode, RefObject } from 'react' +import clsx from 'clsx' +import type { TranslateNS } from '@deepseek-ai/dsh-client-ui-slots' +import { + IconCloseOutlineRegular, IconPlusOutlineRegular, IconRefreshOutlineRegular, IconTrashOutlineRegular, Tooltip, +} from '@deepseek-ai/dsh-client-ui-primitives' +import type {} from './locales.ts' +import { + addRule, defaultRules, isDefaultRules, removeRule, setRuleEnabled, type ExclusionRule, type RuleInvalidReason, +} from './search-rules.ts' +import css from './FilesBody.module.css' + +/** + * The settings panel. + * @param props.rules - the navigator's current draft. + * @param props.respectGitignore - whether the workspace's search leaves out entries `.gitignore` ignores. + * @param props.onRespectGitignore - saves a new `.gitignore` choice at once. + * @param props.trigger - the control that opens and closes the panel; focus moving to it leaves the closing to its click. + * @param props.onChange - replaces the draft without saving preferences. + * @param props.onClose - saves and closes the draft, returning focus to the trigger when requested. + * @param props.t - namespace-bound translate. + * @returns the panel. + */ +export function SearchSettings({ rules, respectGitignore, onRespectGitignore, trigger, onChange, onClose, t }: { + readonly rules: readonly ExclusionRule[] + readonly respectGitignore: boolean + readonly onRespectGitignore: (respectGitignore: boolean) => void + readonly trigger: RefObject + readonly onChange: (rules: readonly ExclusionRule[]) => void + readonly onClose: (refocus: boolean) => void + readonly t: TranslateNS<'sidebarFiles'> +}): ReactNode { + const [text, setText] = useState('') + const [invalid, setInvalid] = useState(undefined) + const add = useRef(null) + const title = useId() + const error = useId() + // A control that is about to disappear hands the keyboard to the add field first, so the panel keeps it. + const keepFocus = (): void => { add.current?.focus() } + return ( +
    { + if (event.key !== 'Escape' || event.nativeEvent.isComposing) return + // Kept from the navigator's own Escape and the overlay's close on the document. + event.preventDefault() + event.stopPropagation() + onClose(true) + }} + onBlur={(event) => { + const next = event.relatedTarget + if (next instanceof Node && (event.currentTarget.contains(next) || trigger.current?.contains(next) === true)) return + onClose(false) + }} + > +
    + {t('settings.title')} + {!isDefaultRules(rules) && ( + + + + )} + + + +
    +

    {t('settings.description')}

    +
    + +
    +
    + + { + setText(event.target.value) + setInvalid(undefined) + }} + onKeyDown={(event) => { + if (event.key !== 'Enter' || event.nativeEvent.isComposing) return + event.preventDefault() + const added = addRule(rules, text) + if (added.kind === 'invalid') { + setInvalid(added.reason) + return + } + onChange(added.rules) + setText('') + }} + /> +
    + {invalid !== undefined && } +
      + {rules.map(rule => ( +
    • + + + + +
      • + ))} +
    +
    + ) +} diff --git a/packages/client/ui-sidebar-files/src/client/face.ts b/packages/client/ui-sidebar-files/src/client/face.ts index 0957339723..3bdccb9337 100644 --- a/packages/client/ui-sidebar-files/src/client/face.ts +++ b/packages/client/ui-sidebar-files/src/client/face.ts @@ -1,11 +1,17 @@ /** * The navigator's half outside React. The injected face (`filesFace`) lists * directories into the store through its own actions, reveals the tab's file, - * carries scroll and keyboard focus between drawings of a pane's tree, and - * measures the navigator's row. The component never awaits or observes - * anything: it registers what it draws and calls the face. The per-pane - * lifetimes are stated on `attach` and `hold`. Beside the face live its - * inputs: the Remote listing and watch bindings and `canOpenOf`. + * carries scroll and keyboard focus between drawings of a pane's tree, runs + * each pane's search, and measures the navigator's row. The component never + * awaits or observes anything: it registers what it draws and calls the face. + * The per-pane lifetimes are stated on `attach` and `hold`. A search asks the + * Host whenever the query or the workspace's search preferences change; each + * pane runs one search at a time, a newer one aborting the older, whose answer + * is dropped. It writes its pending state and its result to the store, and the + * results keep a scroll offset apart from the tree's. The workspace's search + * preferences come from a book shared by every Session. Beside the face live + * its inputs: the Remote listing and watch bindings, and `canOpenOf`; the + * search's binding lives in `search-workspace.ts`. */ import type { ClientRemote, RemoteResult } from '@deepseek-ai/dsh-api-remotes/client' import { createSnapshotStore, type SnapshotStore } from '@deepseek-ai/dsh-client-store' @@ -13,11 +19,15 @@ import type { PaneId, TabId } from '@deepseek-ai/dsh-client-ui-dockkit' import type { SidebarRightTabDefinition, SidebarRightTabNavigation } from '@deepseek-ai/dsh-client-ui-sidebar-right/client' import type { HostObservable } from '@deepseek-ai/dsh-client-ui-slots' import type { SessionId } from '@deepseek-ai/dsh-session/types' -import { childPath, locateFile, relativeFileOf, type DirLevel, type FilesStoreInstance } from './store.ts' +import { childPath, locateFile, relativeFileOf, type DirLevel, type FilesSearchResult, type FilesStoreInstance } from './store.ts' import type { WorkspaceFileWatchFrame } from '@deepseek-ai/dsh-api-workspace-files/types' import { RemoteError } from '@deepseek-ai/dsh-typert-protocol' import { DirectoryNode } from './directory-node.ts' import { FILL_BELOW, FULLSCREEN_QUERY } from './navigator-layout.ts' +import type { WorkspaceSearchPreferenceBook } from './search-preferences.ts' +import { enabledPaths, type WorkspaceSearchPreferences } from './search-rules.ts' +import type { SearchWorkspace } from './search-workspace.ts' +import { buildSearchTree } from './search-tree.ts' /** * The box an element is laid out in: its nearest ancestor that generates one, @@ -140,7 +150,11 @@ export interface FilesInjected { * @returns the release; the listing closes once no hold remains when the current task ends. */ readonly hold: (paneId: PaneId, root: string) => () => void - /** Refresh the open directory tree. @param paneId - owning pane. */ + /** + * Refresh the open directory tree, and run the pane's search again when it + * has a query, since its results are a snapshot of one search. + * @param paneId - owning pane. + */ readonly refresh: (paneId: PaneId) => void /** Control automatic rereads without closing watches. @param paneId - owning pane. @param enabled - automatic-refresh setting. */ readonly setAutoRefresh: (paneId: PaneId, enabled: boolean) => void @@ -153,7 +167,8 @@ export interface FilesInjected { */ readonly toggle: (paneId: PaneId, parentPath: string, path: string, expanded: readonly string[]) => void /** - * Record where a pane's tree is scrolled to, without rendering anything. + * Record where a pane's tree, or its search results while they are shown, + * is scrolled to, without rendering anything. * @param paneId - the pane being drawn. * @param scrollTop - the scroll offset in px. */ @@ -165,17 +180,61 @@ export interface FilesInjected { */ readonly hide: (paneId: PaneId) => void /** - * Show a pane's navigator and give its tree keyboard focus now, or when it - * is next drawn; a pane drawn but not yet attached shows it once attached. + * Show a pane's navigator and its search bar, and give the field keyboard focus now, + * or when it is next drawn; a pane drawn but not yet attached shows it once attached. + * Focusing the always-visible files page leaves the pane's preview-navigator show choice unchanged. * A pane with no drawing recorded is left alone. - * @param paneId - the pane whose tree takes focus. + * @param paneId - the pane whose search field takes focus. */ readonly focus: (paneId: PaneId) => void + /** + * Record an attached pane's search text. Text with more than whitespace + * searches the whole workspace with the workspace's current preferences, + * aborting the pane's running search and dropping its answer; while it holds + * such text, a change to those preferences searches again. Blank text ends + * the search, aborting a running one, and the pane draws its tree again. + * @param paneId - the pane being drawn. + * @param cwd - the Session's working directory, the search's root; fixed for the Session once known, so a + * result always shares the pane's tree root. + * @param query - the text as typed. + */ + readonly setQuery: (paneId: PaneId, cwd: string, query: string) => void + /** + * Open or collapse one directory of a pane's search results, leaving the tree's own expansion alone. + * @param paneId - the pane being drawn. + * @param path - absolute directory path. + */ + readonly toggleSearch: (paneId: PaneId, path: string) => void + /** + * Run an attached pane's search again with its current query and the + * workspace's current preferences, as after a failure; a pane without a + * query is left alone. + * @param paneId - the pane being drawn. + */ + readonly retrySearch: (paneId: PaneId) => void + /** + * Change the search preferences of a workspace for every Session and pane + * rooted there. Only the fields the patch names change; the others keep + * their value at the time of the call, not at the time the caller read them. + * @param cwd - the Session's working directory. + * @param patch - any of the exclusion rules, the hidden-entry choice, and the `.gitignore` choice. + */ + readonly setSearchPreferences: (cwd: string, patch: Partial) => void + /** + * Register the drawn search field, or with `null` as gone. Drawing takes + * keyboard focus that `focus` asked for, or that the previous drawing's + * field held when it went in the same task. + * @param paneId - the pane being drawn. + * @param input - the field, or `null` once it is removed. + */ + readonly searchDrawn: (paneId: PaneId, input: HTMLElement | null) => void /** * Register the tree's scroll container as drawn for one tab, or with `null` - * as gone. Drawing restores the pane's scroll offset, takes keyboard focus - * that the previous drawing held when it went or that `focus` asked for, and - * reveals the tab's file unless the pane revealed it last. + * as gone. Drawing restores the pane's scroll offset for what it shows — the + * tree, or the search results, each with its own offset — takes keyboard + * focus that the previous drawing held when it went, and reveals the tab's + * file in the tree unless the pane revealed it last. A drawing registers + * again when it switches between the tree and the results. * @param paneId - the pane being drawn. * @param address - the tab's content address. * @param body - the scroll container, or `null` once it is removed. @@ -216,6 +275,8 @@ export interface FilesInjected { readonly keyedHooks: { /** The measured width of the row a pane's navigator is laid out in, in CSS pixels, keyed by pane id; `undefined` until measured. */ readonly rowWidth: (paneId: string) => HostObservable + /** A workspace's search preferences, keyed by the Session's working directory; shared by every Session rooted there. */ + readonly searchPreferences: (cwd: string) => HostObservable } } @@ -264,6 +325,16 @@ interface PaneRecord { root: DirectoryNode | undefined /** Where the tree was last scrolled to, in px. */ scrollTop: number + /** Where the search results were last scrolled to, in px. */ + searchScrollTop: number + /** The trimmed query the pane's search asks for; empty while no search is shown. */ + searchQuery: string + /** The preferences the pane's search follows, while its query is not blank. */ + following: { readonly cwd: string; readonly stop: () => void } | undefined + /** The running search, whose answer the pane draws only while it is still this one. */ + searching: AbortController | undefined + /** The drawn search field. */ + input: HTMLElement | undefined /** The file whose row was revealed last, so a remount does not reveal it again. */ revealed: string | undefined /** The file whose row a reveal is walking to. */ @@ -274,8 +345,8 @@ interface PaneRecord { current: HTMLElement | undefined /** A completed reveal waits for its row to be drawn. */ scrollToCurrent: boolean - /** Who asked for keyboard focus at the tree's next drawing. */ - focus: object | undefined + /** Who asked for keyboard focus at the next drawing, and whether the tree or the search field takes it. */ + focus: { readonly target: 'tree' | 'search' } | undefined /** `focus` asked to show the navigator before the pane attached; attaching shows it. */ show: boolean /** The measured width of the row the navigator shares with the preview or the page's line. */ @@ -299,8 +370,8 @@ interface TabHome { readonly navigation: HostObservable } -/** `focus` asked for the tree's keyboard focus. */ -const REQUESTED = {} +/** `focus` asked for the search field's keyboard focus. */ +const REQUESTED = { target: 'search' } as const /** * Place a face beside a tab's content, where the workspace command finds it, @@ -312,11 +383,13 @@ const REQUESTED = {} export type PlaceFace = (tabId: TabId, face: FilesInjected) => () => void /** - * Bind the tree's face to one directory listing. + * Bind the tree's face to one directory listing and one workspace search. * @param list - the bound `workspaceFiles.list` call. * @param watch - target-scoped directory observation. * @param canOpen - whether some tab type can open an address. * @param navigationOf - the attached tab's navigation; subscriptions end with this plugin or their disposer. + * @param search - the workspace search. + * @param preferences - every workspace's search preferences, shared by every Session. * @returns the face factory: session, its store instance, and where to place the face beside tabs in, face out. */ export function filesFace( @@ -324,6 +397,8 @@ export function filesFace( watch: WatchWorkspaceDirectory, canOpen: HostObservable<(address: string) => boolean>, navigationOf: (sessionId: SessionId, tabId: TabId) => HostObservable, + search: SearchWorkspace, + preferences: WorkspaceSearchPreferenceBook, ): ( sessionId: SessionId, store: Pick, @@ -337,6 +412,7 @@ export function filesFace( if (pane === undefined) { pane = { lifetime: undefined, mounts: 0, tabs: new Set(), holds: 0, root: undefined, scrollTop: 0, + searchScrollTop: 0, searchQuery: '', following: undefined, searching: undefined, input: undefined, revealed: undefined, revealing: undefined, drawn: undefined, current: undefined, scrollToCurrent: false, focus: undefined, show: false, width: createSnapshotStore(undefined), @@ -353,6 +429,7 @@ export function filesFace( void pane.root?.close() pane.root = undefined pane.unobserve?.() + endSearch(pane) panes.delete(paneId) actions.forget(paneId) } @@ -425,8 +502,73 @@ export function filesFace( }) reveal() } + const resultsShown = (paneId: PaneId): boolean => store.getSnapshot().byPane[paneId]?.search !== undefined + const endSearch = (pane: PaneRecord): void => { + pane.searchQuery = '' + // The next search's results start at their top, even before its answer arrives. + pane.searchScrollTop = 0 + pane.following?.stop() + pane.following = undefined + pane.searching?.abort() + pane.searching = undefined + } + const runSearch = (paneId: PaneId, pane: PaneRecord, cwd: string): void => { + pane.searching?.abort() + const controller = new AbortController() + pane.searching = controller + const query = pane.searchQuery + const { rules, showHidden, respectGitignore } = preferences.source(cwd).getSnapshot() + const excludedPaths = enabledPaths(rules) + actions.searching(paneId) + const settle = (result: FilesSearchResult): void => { + // A newer search, the end of the search, or the pane's end replaced this one. + if (pane.searching !== controller) return + pane.searching = undefined + // New results start at their top; the tree drawn in their place keeps its offset. + pane.searchScrollTop = 0 + if (pane.drawn !== undefined) pane.drawn.body.scrollTop = 0 + actions.searched(paneId, result) + } + search(sessionId, { query, excludedPaths, includeHidden: showHidden, respectGitignore }, controller.signal).then((answer) => { + settle(answer.ok + ? { + kind: 'found', query, tree: buildSearchTree(cwd, answer.value.matches), count: answer.value.matches.length, + truncated: answer.value.truncated, incomplete: answer.value.incomplete, + hiddenLeftOut: !showHidden, excluded: excludedPaths.length > 0, ignored: respectGitignore, + } + : { kind: 'failed', query, failure: answer.error }) + }).catch((error: unknown) => { + // A Remote call rejects only on an assembly fault, and drawing the answer + // fails only on a defect; the pane reports either rather than wait forever. + settle({ kind: 'failed', query, failure: new RemoteError('gateway/internal', error instanceof Error ? error.message : String(error), {}) }) + }) + } + /** Run an attached pane's search again with its current query; a pane without a query is left alone. */ + const searchAgain = (paneId: PaneId): void => { + const pane = panes.get(paneId) + if (pane?.following === undefined) return + runSearch(paneId, pane, pane.following.cwd) + } + const follow = (paneId: PaneId, pane: PaneRecord, cwd: string): void => { + if (pane.following?.cwd === cwd) return + pane.following?.stop() + const source = preferences.source(cwd) + let seen = source.getSnapshot() + const stop = source.subscribe(() => { + const now = source.getSnapshot() + if (now === seen) return + seen = now + runSearch(paneId, pane, cwd) + }) + pane.following = { cwd, stop } + } + const focusInput = (pane: PaneRecord): void => { + pane.focus = undefined + // As for the tree: the dock's inert host lifts later in this commit. + queueMicrotask(() => { pane.input?.focus() }) + } const focusTree = (pane: PaneRecord): void => { - const handoff = {} + const handoff = { target: 'tree' } as const pane.focus = handoff // The dock keeps a newly shown tab's host inert until its own layout // effect, later in this drawing's commit, and the tab's body may draw the @@ -521,7 +663,8 @@ export function filesFace( pane.lifetime ??= new AbortController() if (pane.show) { pane.show = false - actions.shown(paneId, true) + if (pane.navigator === undefined || pane.closable) actions.shown(paneId, true) + else actions.searchBarShown(paneId, true) } pane.mounts += 1 let home = homes.get(tabId) @@ -571,7 +714,11 @@ export function filesFace( }) } }, - refresh: (paneId) => { void panes.get(paneId)?.root?.refreshTree() }, + refresh: (paneId) => { + const pane = panes.get(paneId) + void pane?.root?.refreshTree() + searchAgain(paneId) + }, setAutoRefresh: (paneId, enabled) => { actions.autoRefresh(paneId, enabled) panes.get(paneId)?.root?.setAutomatic(enabled) @@ -579,7 +726,9 @@ export function filesFace( toggle: (paneId, parentPath, path, expanded) => { expand(paneId, parentPath, path, expanded, true) }, scrolled: (paneId, scrollTop) => { const pane = panes.get(paneId) - if (pane?.lifetime !== undefined) pane.scrollTop = scrollTop + if (pane?.lifetime === undefined) return + if (resultsShown(paneId)) pane.searchScrollTop = scrollTop + else pane.scrollTop = scrollTop }, hide(paneId) { const pane = panes.get(paneId) @@ -600,9 +749,46 @@ export function filesFace( const pane = panes.get(paneId) if (pane === undefined) return if (pane.lifetime === undefined) pane.show = true - else actions.shown(paneId, true) - if (pane.drawn === undefined) pane.focus = REQUESTED - else focusTree(pane) + else { + if (pane.navigator === undefined || pane.closable) actions.shown(paneId, true) + else actions.searchBarShown(paneId, true) + } + if (pane.input === undefined) pane.focus = REQUESTED + else focusInput(pane) + }, + setQuery(paneId, cwd, query) { + const pane = panes.get(paneId) + if (pane?.lifetime === undefined) return + actions.queried(paneId, query) + const trimmed = query.trim() + if (trimmed === '') { + endSearch(pane) + return + } + // Whitespace around the text does not change what the search matches. + if (pane.searchQuery === trimmed && pane.following?.cwd === cwd) return + pane.searchQuery = trimmed + follow(paneId, pane, cwd) + runSearch(paneId, pane, cwd) + }, + toggleSearch: (paneId, path) => { actions.searchToggled(paneId, path) }, + retrySearch: searchAgain, + setSearchPreferences: (cwd, patch) => { preferences.update(cwd, patch) }, + searchDrawn(paneId, input) { + if (input === null) { + const pane = panes.get(paneId) + if (pane === undefined) return + const gone = pane.input + pane.input = undefined + if (gone === undefined || gone.ownerDocument.activeElement !== gone) return + const handoff = { target: 'search' } as const + pane.focus = handoff + queueMicrotask(() => { if (pane.focus === handoff) pane.focus = undefined }) + return + } + const pane = record(paneId) + pane.input = input + if (pane.focus?.target === 'search') focusInput(pane) }, bodyDrawn(paneId, address, body) { if (body === null) { @@ -616,15 +802,15 @@ export function filesFace( drop(paneId) if (gone?.contains(gone.ownerDocument.activeElement) !== true) return // Only a drawing in the same commit, the next tab's, takes the keyboard over. - const handoff = {} + const handoff = { target: 'tree' } as const pane.focus = handoff queueMicrotask(() => { if (pane.focus === handoff) pane.focus = undefined }) return } const pane = record(paneId) pane.drawn = { body, address } - body.scrollTop = pane.scrollTop - if (pane.focus !== undefined) focusTree(pane) + body.scrollTop = resultsShown(paneId) ? pane.searchScrollTop : pane.scrollTop + if (pane.focus?.target === 'tree') focusTree(pane) pane.revealing = pane.revealed === address ? undefined : address step(paneId, pane) }, @@ -762,7 +948,7 @@ export function filesFace( drop(paneId) }, hooks: { canOpen }, - keyedHooks: { rowWidth: paneId => record(paneId as PaneId).width }, + keyedHooks: { rowWidth: paneId => record(paneId as PaneId).width, searchPreferences: cwd => preferences.source(cwd) }, } return face } diff --git a/packages/client/ui-sidebar-files/src/client/index.ts b/packages/client/ui-sidebar-files/src/client/index.ts index 050f59caae..95acbdde1b 100644 --- a/packages/client/ui-sidebar-files/src/client/index.ts +++ b/packages/client/ui-sidebar-files/src/client/index.ts @@ -3,7 +3,9 @@ import type {} from '@deepseek-ai/dsh-client-ui-context-menu/client' * Browser half: register `files` as a right-Sidebar tab type, and the * workspace navigator beside every file preview. The navigator, the page body, * and the toggle share one store and one face per store instance. The package - * README lists the registrations. + * README lists the registrations. Search lives in `search-workspace.ts`, which + * binds the Host's workspace search, `search-rules.ts`, `search-tree.ts`, + * `search-preferences.ts`, and `SearchSettings.tsx`. */ import type { ShortcutCommandId } from '@deepseek-ai/dsh-client-shortcuts/client' import type { Context as ClientContext } from '@deepseek-ai/cordis' @@ -11,22 +13,27 @@ import type {} from '@deepseek-ai/dsh-api-remotes/client' import type {} from '@deepseek-ai/dsh-client-ui-renderer/client' import type {} from '@deepseek-ai/dsh-client-ui-session/client' import type {} from '@deepseek-ai/dsh-client-ui-sidebar-right/client' -import type { BoundActions } from '@deepseek-ai/dsh-client-store' +import { createSnapshotStore, type BoundActions } from '@deepseek-ai/dsh-client-store' import type { TabId } from '@deepseek-ai/dsh-client-ui-dockkit' import type { SessionId } from '@deepseek-ai/dsh-session/types' import { FILES_ID, FILES_KIND, filesDefinition } from './definition.tsx' import { canOpenOf, createList, createWatch, filesFace, type FilesInjected } from './face.ts' +import { createSearch } from './search-workspace.ts' import { FileTree } from './FileTree.tsx' import { FileNavigator } from './FileNavigator.tsx' import { FilesBody } from './FilesBody.tsx' import { FilesTitle } from './FilesTitle.tsx' import { en, ja, es, fr, de, ru, pt, zh } from './locales.ts' import { NavigatorToggle } from './NavigatorToggle.tsx' +import { createSearchPreferenceBook, SEARCH_PREFERENCES_KEY } from './search-preferences.ts' import { createFilesStore } from './store.ts' export type { SidebarFilesKey } from './locales.ts' -export type { DirLevel, FilesPaneState, FilesState, FilesTreeState, LevelState } from './store.ts' +export type { DirLevel, FilesPaneState, FilesSearchResult, FilesSearchState, FilesState, FilesTreeState, LevelState } from './store.ts' export type { FilesInjected, ListWorkspaceDirectory, WorkspaceFilesListRemote } from './face.ts' +export type { SearchWorkspace, WorkspaceFilesSearchRemote, WorkspaceSearchAnswer, WorkspaceSearchRequest } from './search-workspace.ts' +export type { SearchMatch } from './search-tree.ts' +export type { ExclusionRule, WorkspaceSearchPreferences } from './search-rules.ts' export type { FilesBodyProps } from './FilesBody.tsx' // Also keeps the Factory declaration in this entry's emitted types, which an `import type {}` would not. export type { TreeEntry, TreeEntryMeta, TreeEntryKind, TreeDirectoryState, FileTreeInput } from './FileTree.tsx' @@ -65,6 +72,8 @@ export const inject = ['slots', 'locale', 'sidebarRightTabs', 'sidebarRight', 'r export function apply(ctx: ClientContext): void { const t = ctx.locale.bind(NS) const handle = createFilesStore() + // One book for the plugin, not per Session: Sessions rooted at the same workspace share its preferences. + const preferences = createSearchPreferenceBook(createSnapshotStore({}, { persist: { name: SEARCH_PREFERENCES_KEY } })) const face = filesFace(createList(ctx.remote), createWatch(ctx.remote), canOpenOf(ctx.sidebarRightTabs), (sessionId, tabId) => { const source = ctx.sidebarRight.tabDomain.occurrence(sessionId, { id: tabId }).navigation return { @@ -78,7 +87,7 @@ export function apply(ctx: ClientContext): void { return () => { active = false; unsubscribe() } }, 'ui-sidebar-files: tab navigation'), } - }) + }, createSearch(ctx.remote), preferences) // A Factory runs its inject per occurrence, so each store instance gets its // one face when it is minted. The shortcut, which acts outside any body, finds // the face placed beside the target tab; a tab's loading and loaded drawings @@ -136,7 +145,8 @@ export function apply(ctx: ClientContext): void { if (target === undefined) return { status: 'blocked', reason: t('shortcut.noSession') } /* jscpd:ignore-end */ return { status: 'handled', run: () => { - // A docked tab with a navigator beside its content that can draw a tree keeps it: the navigator is shown and takes the keyboard. + // A docked tab with a navigator beside its content that can draw a tree keeps it: + // the navigator is shown and its search field takes the keyboard. const placed = target.tabId === undefined ? undefined : beside.get(target.sessionId)?.get(target.tabId) if (target.host === 'dock' && placed !== undefined) { if (!ctx.sidebarRight.isExpanded()) ctx.sidebarRight.toggleExpanded() diff --git a/packages/client/ui-sidebar-files/src/client/locales.ts b/packages/client/ui-sidebar-files/src/client/locales.ts index 065d408e65..968d61ead9 100644 --- a/packages/client/ui-sidebar-files/src/client/locales.ts +++ b/packages/client/ui-sidebar-files/src/client/locales.ts @@ -26,6 +26,7 @@ export const zh = { 'guide.description': '浏览会话工作区的文件', loading: '正在读取…', empty: '空目录', + emptyHidden: '隐藏项未显示', truncated: '条目太多，只显示了一部分。', noWorkspace: '这个会话没有工作区目录。', reload: '重新读取', @@ -41,6 +42,37 @@ export const zh = { 'error.outsideWorkspace': '这个目录在工作区之外，侧栏不会读取它。', 'error.notDirectory': '这不是一个目录。', 'error.unavailable': '读取失败：{message}', + 'search.placeholder': '搜索文件', + 'search.settings': '搜索设置', + 'search.clear': '清空搜索', + 'search.showHidden': '显示隐藏项', + 'search.results': '{count} 个结果', + 'search.empty': '没有匹配项', + 'search.emptyHint.hidden': '隐藏项不会显示。', + 'search.emptyHint.excluded': '排除的文件和目录不会显示。', + 'search.emptyHint.both': '隐藏项以及排除的文件和目录不会显示。', + 'search.emptyHint.ignored': '被忽略文件（如 .gitignore）忽略的文件和目录不会显示。', + 'search.emptyHint.hiddenIgnored': '隐藏项，以及被忽略文件（如 .gitignore）忽略的文件和目录不会显示。', + 'search.emptyHint.excludedIgnored': '排除的文件和目录，以及被忽略文件（如 .gitignore）忽略的文件和目录不会显示。', + 'search.emptyHint.all': '隐藏项、排除的文件和目录，以及被忽略文件（如 .gitignore）忽略的文件和目录不会显示。', + 'search.emptyHint.label': '为什么没有匹配项', + 'search.truncated': '只显示前 {count} 个结果，请缩小搜索范围。', + 'search.incomplete': '文件数超过搜索上限，或部分目录未能读取，因此结果可能不完整。', + 'search.failed': '搜索失败：{message}', + 'search.retry': '重试', + 'settings.title': '排除的文件和目录', + 'settings.description': '只影响搜索结果，不影响文件树。', + 'settings.respectGitignore': '尊重忽略文件（.gitignore 等）', + 'settings.add': '添加文件或目录', + 'settings.close': '关闭', + 'settings.restore': '还原默认列表', + 'settings.remove': '删除 {path}', + 'settings.toggle': '排除 {path}', + 'settings.invalid.empty': '请输入名称或相对路径。', + 'settings.invalid.nul': '路径不能包含 NUL 字符。', + 'settings.invalid.absolute': '请使用相对于工作区的路径。', + 'settings.invalid.parent': '路径不能包含 . 或 .. 段。', + 'settings.invalid.glob': '路径不能包含通配符 * ? [ ] { } !。', } satisfies Record /** Files dictionary key union. */ @@ -54,6 +86,7 @@ export const en = { 'guide.description': 'Browse files in this session\'s workspace', loading: 'Reading…', empty: 'Empty directory', + emptyHidden: 'Hidden entries are not shown', truncated: 'Too many entries, showing only some of them.', noWorkspace: 'This session has no workspace directory.', reload: 'Reload', @@ -69,6 +102,37 @@ export const en = { 'error.outsideWorkspace': 'That directory is outside the workspace, so the sidebar will not read it.', 'error.notDirectory': 'That is not a directory.', 'error.unavailable': 'Read failed: {message}', + 'search.placeholder': 'Search files', + 'search.settings': 'Search settings', + 'search.clear': 'Clear search', + 'search.showHidden': 'Show hidden entries', + 'search.results': 'Results: {count}', + 'search.empty': 'No matches', + 'search.emptyHint.hidden': 'Hidden entries are not shown.', + 'search.emptyHint.excluded': 'Excluded files and directories are not shown.', + 'search.emptyHint.both': 'Hidden entries and excluded files and directories are not shown.', + 'search.emptyHint.ignored': 'Files and directories ignored by ignore files such as .gitignore are not shown.', + 'search.emptyHint.hiddenIgnored': 'Hidden entries, and files and directories ignored by ignore files such as .gitignore, are not shown.', + 'search.emptyHint.excludedIgnored': 'Excluded files and directories, and those ignored by ignore files such as .gitignore, are not shown.', + 'search.emptyHint.all': 'Hidden entries, excluded files and directories, and those ignored by ignore files such as .gitignore, are not shown.', + 'search.emptyHint.label': 'Why there are no matches', + 'search.truncated': 'Showing only the first {count} results. Narrow the search to see the rest.', + 'search.incomplete': 'The search passed its file limit, or some directories could not be read, so results may be incomplete.', + 'search.failed': 'Search failed: {message}', + 'search.retry': 'Retry', + 'settings.title': 'Excluded files and directories', + 'settings.description': 'Only affects search results, not the file tree.', + 'settings.respectGitignore': 'Respect ignore files (.gitignore, etc.)', + 'settings.add': 'Add file or directory', + 'settings.close': 'Close', + 'settings.restore': 'Restore default list', + 'settings.remove': 'Remove {path}', + 'settings.toggle': 'Exclude {path}', + 'settings.invalid.empty': 'Enter a name or relative path.', + 'settings.invalid.nul': 'The path cannot contain a NUL character.', + 'settings.invalid.absolute': 'Use a path relative to the workspace.', + 'settings.invalid.parent': 'The path cannot contain . or .. segments.', + 'settings.invalid.glob': 'The path cannot contain wildcards * ? [ ] { } !.', } satisfies Record /** Japanese copy, checked against the English keys. */ @@ -79,6 +143,7 @@ export const ja = { 'guide.description': 'このセッションのワークスペース内のファイルを参照', 'loading': '読み取り中…', 'empty': '空のディレクトリ', + 'emptyHidden': '隠し項目は表示されていません', 'truncated': 'エントリが多すぎるため、一部のみ表示しています。', 'noWorkspace': 'このセッションにはワークスペースディレクトリがありません。', 'reload': '再読み込み', @@ -94,6 +159,37 @@ export const ja = { 'error.outsideWorkspace': 'そのディレクトリはワークスペース外のため、サイドバーでは読み取りません。', 'error.notDirectory': 'それはディレクトリではありません。', 'error.unavailable': '読み取りに失敗しました: {message}', + 'search.placeholder': 'ファイルを検索', + 'search.settings': '検索設定', + 'search.clear': '検索をクリア', + 'search.showHidden': '隠し項目を表示', + 'search.results': '結果: {count} 件', + 'search.empty': '一致する項目はありません', + 'search.emptyHint.hidden': '隠し項目は表示されません。', + 'search.emptyHint.excluded': '除外したファイルとディレクトリは表示されません。', + 'search.emptyHint.both': '隠し項目と、除外したファイルとディレクトリは表示されません。', + 'search.emptyHint.ignored': '.gitignore などの無視ファイルで無視されたファイルとディレクトリは表示されません。', + 'search.emptyHint.hiddenIgnored': '隠し項目と、.gitignore などの無視ファイルで無視されたファイルとディレクトリは表示されません。', + 'search.emptyHint.excludedIgnored': '除外したファイルとディレクトリ、および .gitignore などの無視ファイルで無視された項目は表示されません。', + 'search.emptyHint.all': '隠し項目、除外したファイルとディレクトリ、および .gitignore などの無視ファイルで無視された項目は表示されません。', + 'search.emptyHint.label': '一致する項目がない理由', + 'search.truncated': '最初の {count} 件のみ表示しています。検索範囲を絞り込んでください。', + 'search.incomplete': 'ファイル数が検索上限を超えたか、一部のディレクトリを読み取れなかったため、結果が不完全な可能性があります。', + 'search.failed': '検索に失敗しました: {message}', + 'search.retry': '再試行', + 'settings.title': '除外するファイルとディレクトリ', + 'settings.description': '検索結果にのみ適用され、ファイルツリーには影響しません。', + 'settings.respectGitignore': '無視ファイル（.gitignore など）に従う', + 'settings.add': 'ファイルまたはディレクトリを追加', + 'settings.close': '閉じる', + 'settings.restore': '既定のリストに戻す', + 'settings.remove': '{path} を削除', + 'settings.toggle': '{path} を除外', + 'settings.invalid.empty': '名前または相対パスを入力してください。', + 'settings.invalid.nul': 'パスに NUL 文字は使用できません。', + 'settings.invalid.absolute': 'ワークスペースからの相対パスを使用してください。', + 'settings.invalid.parent': 'パスに . や .. のセグメントは使用できません。', + 'settings.invalid.glob': 'パスにワイルドカード * ? [ ] { } ! は使用できません。', } satisfies Record /** Spanish dictionary, checked against the English keys. */ @@ -103,10 +199,11 @@ export const es = { 'guide.title': 'Archivos de espacio de trabajo', 'guide.description': 'Explorar archivos en el espacio de trabajo de esta sesión', loading: 'Leyendo…', - empty: 'directorio vacio', + empty: 'Directorio vacío', + emptyHidden: 'Las entradas ocultas no se muestran', truncated: 'Demasiadas entradas y solo se muestran algunas de ellas.', noWorkspace: 'Esta sesión no tiene ningún directorio de espacio de trabajo.', - reload: 'recargar', + reload: 'Recargar', autoRefresh: 'Actualización automática', 'autoRefresh.enable': 'Habilitar actualización automática', 'autoRefresh.disable': 'Deshabilitar la actualización automática', @@ -119,6 +216,37 @@ export const es = { 'error.outsideWorkspace': 'Ese directorio está fuera del espacio de trabajo, por lo que la barra lateral no lo leerá.', 'error.notDirectory': 'Eso no es un directorio.', 'error.unavailable': 'Error de lectura: {message}', + 'search.placeholder': 'Buscar archivos', + 'search.settings': 'Configuración de búsqueda', + 'search.clear': 'Borrar búsqueda', + 'search.showHidden': 'Mostrar entradas ocultas', + 'search.results': 'Resultados: {count}', + 'search.empty': 'Sin coincidencias', + 'search.emptyHint.hidden': 'No se muestran las entradas ocultas.', + 'search.emptyHint.excluded': 'No se muestran los archivos ni los directorios excluidos.', + 'search.emptyHint.both': 'No se muestran las entradas ocultas ni los archivos y directorios excluidos.', + 'search.emptyHint.ignored': 'No se muestran los archivos ni los directorios que ignoran los archivos de exclusión como .gitignore.', + 'search.emptyHint.hiddenIgnored': 'No se muestran las entradas ocultas ni los archivos y directorios que ignoran los archivos de exclusión como .gitignore.', + 'search.emptyHint.excludedIgnored': 'No se muestran los archivos ni los directorios excluidos o ignorados por archivos de exclusión como .gitignore.', + 'search.emptyHint.all': 'No se muestran las entradas ocultas, los archivos y directorios excluidos ni los ignorados por archivos de exclusión como .gitignore.', + 'search.emptyHint.label': 'Por qué no hay coincidencias', + 'search.truncated': 'Solo se muestran los primeros {count} resultados. Acote la búsqueda para ver el resto.', + 'search.incomplete': 'La búsqueda superó su límite de archivos o no se pudieron leer algunos directorios, por lo que los resultados pueden estar incompletos.', + 'search.failed': 'Error de búsqueda: {message}', + 'search.retry': 'Reintentar', + 'settings.title': 'Archivos y directorios excluidos', + 'settings.description': 'Solo afecta a los resultados de búsqueda, no al árbol de archivos.', + 'settings.respectGitignore': 'Respetar archivos de exclusión (.gitignore, etc.)', + 'settings.add': 'Añadir archivo o directorio', + 'settings.close': 'Cerrar', + 'settings.restore': 'Restaurar la lista predeterminada', + 'settings.remove': 'Eliminar {path}', + 'settings.toggle': 'Excluir {path}', + 'settings.invalid.empty': 'Introduzca un nombre o una ruta relativa.', + 'settings.invalid.nul': 'La ruta no puede contener el carácter NUL.', + 'settings.invalid.absolute': 'Use una ruta relativa al espacio de trabajo.', + 'settings.invalid.parent': 'La ruta no puede contener segmentos «.» ni «..».', + 'settings.invalid.glob': 'La ruta no puede contener comodines * ? [ ] { } !.', } satisfies Record /** French dictionary, checked against the English keys. */ @@ -129,6 +257,7 @@ export const fr = { 'guide.description': "Parcourir les fichiers dans l'espace de travail de cette session", loading: 'Lecture…', empty: 'Répertoire vide', + emptyHidden: 'Les entrées masquées ne sont pas affichées', truncated: "Trop d'entrées, n'en affichant que certaines.", noWorkspace: "Cette session n'a pas de répertoire d'espace de travail.", reload: 'Recharger', @@ -144,6 +273,37 @@ export const fr = { 'error.outsideWorkspace': 'Ce répertoire se trouve en dehors de l’espace de travail, donc la barre latérale ne le lira pas.', 'error.notDirectory': "Ce n'est pas un répertoire.", 'error.unavailable': 'Échec de lecture : {message}', + 'search.placeholder': 'Rechercher des fichiers', + 'search.settings': 'Paramètres de recherche', + 'search.clear': 'Effacer la recherche', + 'search.showHidden': 'Afficher les entrées masquées', + 'search.results': 'Résultats : {count}', + 'search.empty': 'Aucune correspondance', + 'search.emptyHint.hidden': 'Les entrées masquées ne sont pas affichées.', + 'search.emptyHint.excluded': 'Les fichiers et répertoires exclus ne sont pas affichés.', + 'search.emptyHint.both': 'Les entrées masquées et les fichiers et répertoires exclus ne sont pas affichés.', + 'search.emptyHint.ignored': 'Les fichiers et répertoires ignorés par des fichiers d’exclusion comme .gitignore ne sont pas affichés.', + 'search.emptyHint.hiddenIgnored': 'Les entrées masquées et les fichiers et répertoires ignorés par des fichiers d’exclusion comme .gitignore ne sont pas affichés.', + 'search.emptyHint.excludedIgnored': 'Les fichiers et répertoires exclus ou ignorés par des fichiers d’exclusion comme .gitignore ne sont pas affichés.', + 'search.emptyHint.all': 'Les entrées masquées, les fichiers et répertoires exclus et ceux ignorés par des fichiers d’exclusion comme .gitignore ne sont pas affichés.', + 'search.emptyHint.label': 'Pourquoi aucune correspondance', + 'search.truncated': 'Seuls les {count} premiers résultats sont affichés. Affinez la recherche pour voir les autres.', + 'search.incomplete': 'La recherche a dépassé sa limite de fichiers, ou certains répertoires n’ont pas pu être lus, les résultats peuvent donc être incomplets.', + 'search.failed': 'Échec de la recherche : {message}', + 'search.retry': 'Réessayer', + 'settings.title': 'Fichiers et répertoires exclus', + 'settings.description': 'Concerne uniquement les résultats de recherche, pas l’arborescence des fichiers.', + 'settings.respectGitignore': 'Respecter les fichiers d’exclusion (.gitignore, etc.)', + 'settings.add': 'Ajouter un fichier ou un répertoire', + 'settings.close': 'Fermer', + 'settings.restore': 'Restaurer la liste par défaut', + 'settings.remove': 'Supprimer {path}', + 'settings.toggle': 'Exclure {path}', + 'settings.invalid.empty': 'Saisissez un nom ou un chemin relatif.', + 'settings.invalid.nul': 'Le chemin ne peut pas contenir de caractère NUL.', + 'settings.invalid.absolute': 'Utilisez un chemin relatif à l’espace de travail.', + 'settings.invalid.parent': 'Le chemin ne peut pas contenir de segments « . » ou « .. ».', + 'settings.invalid.glob': 'Le chemin ne peut pas contenir les caractères génériques * ? [ ] { } !.', } satisfies Record /** German dictionary, checked against the English keys. */ @@ -154,6 +314,7 @@ export const de = { 'guide.description': 'Durchsuchen Sie die Dateien im Arbeitsbereich dieser Sitzung', loading: 'Lesen…', empty: 'Leeres Verzeichnis', + emptyHidden: 'Versteckte Einträge werden nicht angezeigt', truncated: 'Zu viele Einträge, es werden nur einige davon angezeigt.', noWorkspace: 'Diese Sitzung hat kein Arbeitsbereichsverzeichnis.', reload: 'Neu laden', @@ -169,6 +330,37 @@ export const de = { 'error.outsideWorkspace': 'Dieses Verzeichnis liegt außerhalb des Arbeitsbereichs und wird daher von der Seitenleiste nicht gelesen.', 'error.notDirectory': 'Das ist kein Verzeichnis.', 'error.unavailable': 'Lesen fehlgeschlagen: {message}', + 'search.placeholder': 'Dateien suchen', + 'search.settings': 'Sucheinstellungen', + 'search.clear': 'Suche löschen', + 'search.showHidden': 'Versteckte Einträge anzeigen', + 'search.results': 'Ergebnisse: {count}', + 'search.empty': 'Keine Treffer', + 'search.emptyHint.hidden': 'Versteckte Einträge werden nicht angezeigt.', + 'search.emptyHint.excluded': 'Ausgeschlossene Dateien und Verzeichnisse werden nicht angezeigt.', + 'search.emptyHint.both': 'Versteckte Einträge sowie ausgeschlossene Dateien und Verzeichnisse werden nicht angezeigt.', + 'search.emptyHint.ignored': 'Von Ignore-Dateien wie .gitignore ignorierte Dateien und Verzeichnisse werden nicht angezeigt.', + 'search.emptyHint.hiddenIgnored': 'Versteckte Einträge sowie von Ignore-Dateien wie .gitignore ignorierte Dateien und Verzeichnisse werden nicht angezeigt.', + 'search.emptyHint.excludedIgnored': 'Ausgeschlossene sowie von Ignore-Dateien wie .gitignore ignorierte Dateien und Verzeichnisse werden nicht angezeigt.', + 'search.emptyHint.all': 'Versteckte Einträge, ausgeschlossene sowie von Ignore-Dateien wie .gitignore ignorierte Dateien und Verzeichnisse werden nicht angezeigt.', + 'search.emptyHint.label': 'Warum es keine Treffer gibt', + 'search.truncated': 'Nur die ersten {count} Ergebnisse werden angezeigt. Grenzen Sie die Suche ein, um weitere zu sehen.', + 'search.incomplete': 'Die Suche hat ihr Dateilimit überschritten oder einige Verzeichnisse konnten nicht gelesen werden, daher sind die Ergebnisse möglicherweise unvollständig.', + 'search.failed': 'Suche fehlgeschlagen: {message}', + 'search.retry': 'Erneut versuchen', + 'settings.title': 'Ausgeschlossene Dateien und Verzeichnisse', + 'settings.description': 'Wirkt sich nur auf Suchergebnisse aus, nicht auf den Dateibaum.', + 'settings.respectGitignore': 'Ignore-Dateien beachten (.gitignore usw.)', + 'settings.add': 'Datei oder Verzeichnis hinzufügen', + 'settings.close': 'Schließen', + 'settings.restore': 'Standardliste wiederherstellen', + 'settings.remove': '{path} entfernen', + 'settings.toggle': '{path} ausschließen', + 'settings.invalid.empty': 'Geben Sie einen Namen oder relativen Pfad ein.', + 'settings.invalid.nul': 'Der Pfad darf kein NUL-Zeichen enthalten.', + 'settings.invalid.absolute': 'Verwenden Sie einen Pfad relativ zum Arbeitsbereich.', + 'settings.invalid.parent': 'Der Pfad darf keine Segmente . oder .. enthalten.', + 'settings.invalid.glob': 'Der Pfad darf keine Platzhalter * ? [ ] { } ! enthalten.', } satisfies Record /** Russian dictionary, checked against the English keys. */ @@ -179,6 +371,7 @@ export const ru = { 'guide.description': 'Просмотр файлов в рабочей области этого сеанса', loading: 'Чтение…', empty: 'Пустой каталог', + emptyHidden: 'Скрытые записи не показаны', truncated: 'Слишком много записей, показаны только некоторые из них.', noWorkspace: 'В этом сеансе нет каталога рабочей области.', reload: 'Перезагрузить', @@ -194,6 +387,37 @@ export const ru = { 'error.outsideWorkspace': 'Этот каталог находится за пределами рабочей области, поэтому боковая панель его не читает.', 'error.notDirectory': 'Это не каталог.', 'error.unavailable': 'Не удалось прочитать: {message}.', + 'search.placeholder': 'Поиск файлов', + 'search.settings': 'Настройки поиска', + 'search.clear': 'Очистить поиск', + 'search.showHidden': 'Показывать скрытые записи', + 'search.results': 'Результаты: {count}', + 'search.empty': 'Совпадений нет', + 'search.emptyHint.hidden': 'Скрытые записи не показываются.', + 'search.emptyHint.excluded': 'Исключённые файлы и каталоги не показываются.', + 'search.emptyHint.both': 'Скрытые записи, а также исключённые файлы и каталоги не показываются.', + 'search.emptyHint.ignored': 'Файлы и каталоги, игнорируемые файлами исключений (например, .gitignore), не показываются.', + 'search.emptyHint.hiddenIgnored': 'Скрытые записи, а также файлы и каталоги, игнорируемые файлами исключений (например, .gitignore), не показываются.', + 'search.emptyHint.excludedIgnored': 'Исключённые файлы и каталоги, а также игнорируемые файлами исключений (например, .gitignore), не показываются.', + 'search.emptyHint.all': 'Скрытые записи, исключённые файлы и каталоги, а также игнорируемые файлами исключений (например, .gitignore), не показываются.', + 'search.emptyHint.label': 'Почему нет совпадений', + 'search.truncated': 'Показаны только первые {count} результатов. Уточните запрос, чтобы увидеть остальные.', + 'search.incomplete': 'Поиск превысил лимит файлов или часть каталогов не удалось прочитать, поэтому результаты могут быть неполными.', + 'search.failed': 'Не удалось выполнить поиск: {message}', + 'search.retry': 'Повторить', + 'settings.title': 'Исключённые файлы и каталоги', + 'settings.description': 'Влияет только на результаты поиска, не на дерево файлов.', + 'settings.respectGitignore': 'Учитывать файлы исключений (.gitignore и др.)', + 'settings.add': 'Добавить файл или каталог', + 'settings.close': 'Закрыть', + 'settings.restore': 'Восстановить список по умолчанию', + 'settings.remove': 'Удалить {path}', + 'settings.toggle': 'Исключить {path}', + 'settings.invalid.empty': 'Введите имя или относительный путь.', + 'settings.invalid.nul': 'Путь не может содержать символ NUL.', + 'settings.invalid.absolute': 'Укажите путь относительно рабочей области.', + 'settings.invalid.parent': 'Путь не может содержать сегменты «.» или «..».', + 'settings.invalid.glob': 'Путь не может содержать подстановочные знаки * ? [ ] { } !.', } satisfies Record /** Portuguese dictionary, checked against the English keys. */ @@ -204,6 +428,7 @@ export const pt = { 'guide.description': 'Procure arquivos no espaço de trabalho desta sessão', loading: 'Lendo…', empty: 'Diretório vazio', + emptyHidden: 'Entradas ocultas não são mostradas', truncated: 'Muitas entradas, mostrando apenas algumas delas.', noWorkspace: 'Esta sessão não possui diretório de espaço de trabalho.', reload: 'Recarregar', @@ -219,4 +444,35 @@ export const pt = { 'error.outsideWorkspace': 'Esse diretório está fora da área de trabalho, portanto a barra lateral não o lerá.', 'error.notDirectory': 'Isso não é um diretório.', 'error.unavailable': 'Falha na leitura: {message}', + 'search.placeholder': 'Pesquisar arquivos', + 'search.settings': 'Configurações de pesquisa', + 'search.clear': 'Limpar pesquisa', + 'search.showHidden': 'Mostrar entradas ocultas', + 'search.results': 'Resultados: {count}', + 'search.empty': 'Nenhuma correspondência', + 'search.emptyHint.hidden': 'Entradas ocultas não são mostradas.', + 'search.emptyHint.excluded': 'Arquivos e diretórios excluídos não são mostrados.', + 'search.emptyHint.both': 'Entradas ocultas e arquivos e diretórios excluídos não são mostrados.', + 'search.emptyHint.ignored': 'Arquivos e diretórios ignorados por arquivos de exclusão como .gitignore não são mostrados.', + 'search.emptyHint.hiddenIgnored': 'Entradas ocultas e arquivos e diretórios ignorados por arquivos de exclusão como .gitignore não são mostrados.', + 'search.emptyHint.excludedIgnored': 'Arquivos e diretórios excluídos ou ignorados por arquivos de exclusão como .gitignore não são mostrados.', + 'search.emptyHint.all': 'Entradas ocultas, arquivos e diretórios excluídos e os ignorados por arquivos de exclusão como .gitignore não são mostrados.', + 'search.emptyHint.label': 'Por que não há correspondências', + 'search.truncated': 'Mostrando apenas os primeiros {count} resultados. Refine a pesquisa para ver o restante.', + 'search.incomplete': 'A pesquisa ultrapassou seu limite de arquivos ou alguns diretórios não puderam ser lidos, então os resultados podem estar incompletos.', + 'search.failed': 'Falha na pesquisa: {message}', + 'search.retry': 'Tentar novamente', + 'settings.title': 'Arquivos e diretórios excluídos', + 'settings.description': 'Afeta apenas os resultados da pesquisa, não a árvore de arquivos.', + 'settings.respectGitignore': 'Respeitar arquivos de exclusão (.gitignore etc.)', + 'settings.add': 'Adicionar arquivo ou diretório', + 'settings.close': 'Fechar', + 'settings.restore': 'Restaurar a lista padrão', + 'settings.remove': 'Remover {path}', + 'settings.toggle': 'Excluir {path}', + 'settings.invalid.empty': 'Digite um nome ou um caminho relativo.', + 'settings.invalid.nul': 'O caminho não pode conter o caractere NUL.', + 'settings.invalid.absolute': 'Use um caminho relativo ao espaço de trabalho.', + 'settings.invalid.parent': 'O caminho não pode conter segmentos “.” ou “..”.', + 'settings.invalid.glob': 'O caminho não pode conter curingas * ? [ ] { } !.', } satisfies Record diff --git a/packages/client/ui-sidebar-files/src/client/search-preferences.ts b/packages/client/ui-sidebar-files/src/client/search-preferences.ts new file mode 100644 index 0000000000..6afb25a0e0 --- /dev/null +++ b/packages/client/ui-sidebar-files/src/client/search-preferences.ts @@ -0,0 +1,116 @@ +/** + * The search preferences of every workspace, kept in browser storage and + * shared by every Session and pane whose workspace root is the same. + * + * The storage value maps a normalized workspace root to the fields the reader + * changed there. A field equal to its default is left out of the entry, a root + * the reader never changed has no entry, and a change back to the defaults + * removes its entry, so the defaults of a later build reach every field the + * reader left alone. Storage + * holds whatever an earlier build or the reader wrote: each entry is read + * through {@link readSearchPreferences} rather than trusted. + */ +import type { SnapshotStore } from '@deepseek-ai/dsh-client-store' +import type { HostObservable } from '@deepseek-ai/dsh-client-ui-slots' +import { + changedSearchPreferences, defaultSearchPreferences, readSearchPreferences, type WorkspaceSearchPreferences, +} from './search-rules.ts' + +/** + * The localStorage key the preferences of every workspace are stored under. + * + * Each entry holds whichever of `rules`, `showHidden`, and `respectGitignore` + * differ from their defaults. Every field is optional on read: + * {@link readSearchPreferences} gives a missing or mistyped one its current + * default and reads the others as usual. A change that an earlier build would + * misread takes a new key. + */ +export const SEARCH_PREFERENCES_KEY = 'dsh.sidebar-files.workspace-search.v1' + +/** + * The storage key of one workspace root: the root as the Host reports it, with + * trailing `/` and `\` removed except the one that makes a filesystem root + * (`/`, `C:\`, `C:/`). Case and separators are otherwise kept, so two + * spellings of one directory keep separate preferences. + * @param cwd - the Session's working directory. + * @returns the key its preferences are stored under. + */ +export function workspaceKey(cwd: string): string { + const trimmed = cwd.replace(/[/\\]+$/, '') + if (trimmed === '') return cwd.slice(0, 1) + if (/^[A-Za-z]:$/.test(trimmed) && cwd.length > trimmed.length) return cwd.slice(0, 3) + return trimmed +} + +/** + * One stored entry, without trusting the stored value's type. + * @param stored - the storage value, as rehydrated. + * @param key - a workspace key. + * @returns the entry's value, or `undefined` when the value is not an object or has no such entry. + */ +function storedEntry(stored: unknown, key: string): unknown { + if (typeof stored !== 'object' || stored === null || Array.isArray(stored) || !Object.hasOwn(stored, key)) return undefined + return Object.getOwnPropertyDescriptor(stored, key)?.value +} + +/** The search preferences of every workspace root. */ +export interface WorkspaceSearchPreferenceBook { + /** + * Observe one root's preferences. The same root, by its {@link workspaceKey}, + * gets the same source, and its snapshot keeps its reference until that + * root's preferences change. + * @param cwd - the Session's working directory. + * @returns the source. + */ + readonly source: (cwd: string) => HostObservable + /** + * Change the reader's choice for one root: the fields the patch names + * replace those of the root's current preferences, read at the call; the + * entry keeps only the fields that differ from the defaults, and is removed + * when none does. + * @param cwd - the Session's working directory. + * @param patch - any of the exclusion rules, the hidden-entry choice, and the `.gitignore` choice. + */ + readonly update: (cwd: string, patch: Partial) => void +} + +/** + * Keep the preferences of every workspace root in one persisted store. + * @param store - the persisted store; its rehydrated value may be anything JSON can hold. + * @returns the preferences by workspace root. + */ +export function createSearchPreferenceBook(store: SnapshotStore): WorkspaceSearchPreferenceBook { + const sources = new Map>() + const source = (cwd: string): HostObservable => { + const key = workspaceKey(cwd) + const known = sources.get(key) + if (known !== undefined) return known + let seen: { readonly raw: unknown; readonly preferences: WorkspaceSearchPreferences } | undefined + const created: HostObservable = { + getSnapshot: () => { + const raw = storedEntry(store.getSnapshot(), key) + if (seen === undefined || seen.raw !== raw) { + seen = { raw, preferences: raw === undefined ? defaultSearchPreferences() : readSearchPreferences(raw) } + } + return seen.preferences + }, + subscribe: listener => store.subscribe(listener), + } + sources.set(key, created) + return created + } + return { + source, + update(cwd, patch) { + const preferences = { ...source(cwd).getSnapshot(), ...patch } + const key = workspaceKey(cwd) + const stored = store.getSnapshot() + const entries = typeof stored === 'object' && stored !== null && !Array.isArray(stored) + ? Object.entries(stored).filter(([name]) => name !== key) + : [] + const changed = changedSearchPreferences(preferences) + if (changed !== undefined) entries.push([key, changed]) + store.set(Object.fromEntries(entries)) + }, + } +} diff --git a/packages/client/ui-sidebar-files/src/client/search-rules.ts b/packages/client/ui-sidebar-files/src/client/search-rules.ts new file mode 100644 index 0000000000..6751a7a3bc --- /dev/null +++ b/packages/client/ui-sidebar-files/src/client/search-rules.ts @@ -0,0 +1,233 @@ +/** + * Search exclusion rules and the search preferences that carry them. + * + * A rule names files and directories the workspace search leaves out, by name + * or by path. A path without `/` is a name and matches every file or directory + * of that name at any depth; a path with `/` is relative to the workspace root + * and matches that one file or directory. An excluded directory's contents are + * left out with it. This module keeps the editable list, checks what the + * reader types, and reads the list back from a stored value; the Host's + * workspace search applies the rules. + * + * The list records no origin: a default path the reader removed and then + * added again by hand counts as a default that is present. + */ + +/** One exclusion rule as the settings list shows and stores it. */ +export interface ExclusionRule { + /** Normalized name or `/`-separated workspace-relative path of a file or directory. */ + readonly path: string + /** Whether the search leaves this path out now. */ + readonly enabled: boolean +} + +/** Why a typed rule path was refused; each reason has its own `settings.invalid.*` line. */ +export type RuleInvalidReason = 'empty' | 'nul' | 'absolute' | 'parent' | 'glob' + +/** A typed rule path after normalization, or why it was refused. */ +export type RulePathResult = + | { readonly kind: 'ok'; readonly path: string } + | { readonly kind: 'invalid'; readonly reason: RuleInvalidReason } + +/** The search panel's persisted choices. */ +export interface WorkspaceSearchPreferences { + /** Exclusion rules, newest first. */ + readonly rules: readonly ExclusionRule[] + /** Whether results include hidden entries. */ + readonly showHidden: boolean + /** Whether the search leaves out entries the workspace's `.gitignore` files ignore. */ + readonly respectGitignore: boolean +} + +/** + * Names excluded until the reader changes the list, in the order a restore + * lists them: version-control, package-manager, and build output directories, + * then the metadata files and directories macOS and Windows write into folders + * and volumes. + */ +const DEFAULT_EXCLUDED_PATHS: readonly string[] = [ + '.git', '.hg', '.next', '.pnpm-store', '.svn', '.turbo', '.yarn', 'build', 'coverage', 'dist', 'node_modules', + '.DS_Store', '.Spotlight-V100', '.Trashes', '.fseventsd', 'Thumbs.db', 'desktop.ini', '$RECYCLE.BIN', 'System Volume Information', +] + +/** Characters that would make a literal path read as a pattern. */ +const GLOB = /[*?[\]{}!]/ + +/** + * Normalize a typed rule path: trim it, turn `\` into `/`, merge repeated + * `/`, and drop leading `./`, trailing `/`, and whitespace exposed at either + * edge. Internal whitespace stays unchanged. + * + * Refuses an empty path, one containing NUL, an absolute one (leading `/`, a + * drive letter such as `C:`, or `~`, including prefixes exposed by dropping + * leading `./` and whitespace), one with a `.` or `..` segment, and one + * containing a glob character (`* ? [ ] { } !`). Other characters, such as + * `$` and spaces, are kept literally. + * @param input - the text the reader typed. + * @returns the normalized path, unchanged by normalization again, or why it was refused. + */ +export function normalizeRulePath(input: string): RulePathResult { + const slashed = input.trim().replace(/\\/g, '/') + if (slashed === '') return { kind: 'invalid', reason: 'empty' } + if (slashed.includes('\0')) return { kind: 'invalid', reason: 'nul' } + const relative = slashed.replace(/\/{2,}/g, '/').replace(/^(?:\s|\.\/)+/, '') + if (slashed.startsWith('/') || /^(?:\/|~|[A-Za-z]:)/.test(relative)) return { kind: 'invalid', reason: 'absolute' } + // Anchor the suffix scan at the start so long internal whitespace is not scanned repeatedly. + const path = relative.replace(/^(.*[^\s/])?[\s/]*$/s, '$1') + if (GLOB.test(path)) return { kind: 'invalid', reason: 'glob' } + if (path === '') return { kind: 'invalid', reason: 'empty' } + if (path.split('/').some(segment => segment === '.' || segment === '..')) return { kind: 'invalid', reason: 'parent' } + return { kind: 'ok', path } +} + +/** + * The default rule list, every rule enabled. + * @returns a new list in default order. + */ +export function defaultRules(): ExclusionRule[] { + return DEFAULT_EXCLUDED_PATHS.map(path => ({ path, enabled: true })) +} + +/** + * The preferences before the reader changes anything. + * @returns the default rules, hidden entries not shown, and `.gitignore` respected. + */ +export function defaultSearchPreferences(): WorkspaceSearchPreferences { + return { rules: defaultRules(), showHidden: false, respectGitignore: true } +} + +/** + * Add a typed rule at the top of the list, enabled. + * + * A rule whose normalized path is already listed, enabled or not, is enabled + * and moved to the top instead of being listed twice. Paths compare exactly. + * @param rules - the current list. + * @param input - the text the reader typed. + * @returns the new list, or why the text was refused. + */ +export function addRule( + rules: readonly ExclusionRule[], + input: string, +): { readonly kind: 'ok'; readonly rules: readonly ExclusionRule[] } | { readonly kind: 'invalid'; readonly reason: RuleInvalidReason } { + const normalized = normalizeRulePath(input) + if (normalized.kind === 'invalid') return normalized + const { path } = normalized + return { kind: 'ok', rules: [{ path, enabled: true }, ...rules.filter(rule => rule.path !== path)] } +} + +/** + * Remove one rule. + * @param rules - the current list. + * @param path - the rule's normalized path. + * @returns the list without it; an unlisted path leaves the list as it was. + */ +export function removeRule(rules: readonly ExclusionRule[], path: string): readonly ExclusionRule[] { + return rules.filter(rule => rule.path !== path) +} + +/** + * Enable or disable one rule in place. + * @param rules - the current list. + * @param path - the rule's normalized path. + * @param enabled - whether the search leaves the path out. + * @returns the list with that rule changed; an unlisted path leaves the list as it was. + */ +export function setRuleEnabled(rules: readonly ExclusionRule[], path: string, enabled: boolean): readonly ExclusionRule[] { + return rules.map(rule => rule.path === path && rule.enabled !== enabled ? { path, enabled } : rule) +} + +/** + * The paths the search leaves out. + * @param rules - the current list. + * @returns the enabled rules' paths, in list order. + */ +export function enabledPaths(rules: readonly ExclusionRule[]): string[] { + return rules.filter(rule => rule.enabled).map(rule => rule.path) +} + +/** + * Whether a rule list equals the default list: the same paths in the same + * order, every rule enabled. + * @param rules - the current list. + * @returns `true` when nothing differs from {@link defaultRules}. + */ +export function isDefaultRules(rules: readonly ExclusionRule[]): boolean { + return rules.length === DEFAULT_EXCLUDED_PATHS.length + && rules.every((rule, index) => rule.path === DEFAULT_EXCLUDED_PATHS[index] && rule.enabled) +} + +/** + * Read preferences back from a value browser storage holds, which any earlier + * build or the reader may have written, after its JSON text was parsed. + * + * Each field is read on its own: a missing or mistyped `rules` array, + * `showHidden`, or `respectGitignore` takes the current default, so an entry + * that stores only the fields the reader changed keeps the defaults of the + * running build for the rest. A value that is not an object gives the + * defaults. Inside a `rules` array, a rule that is not an object with a string + * `path` and a boolean `enabled`, or whose path `normalizeRulePath` refuses, + * is dropped; a rule whose normalized path repeats an earlier one is dropped + * too. A stored empty list stays empty. + * @param value - the parsed stored value. + * @returns the preferences to use. + */ +export function readSearchPreferences(value: unknown): WorkspaceSearchPreferences { + const defaults = defaultSearchPreferences() + if (typeof value !== 'object' || value === null || Array.isArray(value)) return defaults + const field = (name: string): unknown => Object.hasOwn(value, name) ? Object.getOwnPropertyDescriptor(value, name)?.value : undefined + const rules = field('rules') + const showHidden = field('showHidden') + const respectGitignore = field('respectGitignore') + return { + rules: Array.isArray(rules) ? readRules(rules) : defaults.rules, + showHidden: typeof showHidden === 'boolean' ? showHidden : defaults.showHidden, + respectGitignore: typeof respectGitignore === 'boolean' ? respectGitignore : defaults.respectGitignore, + } +} + +/** + * The well-formed rules of a stored list, normalized, in stored order. + * @param rules - the stored list. + * @returns the rules to use. + */ +function readRules(rules: readonly unknown[]): ExclusionRule[] { + const parsed: ExclusionRule[] = [] + for (const rule of rules) { + if (typeof rule !== 'object' || rule === null || !('path' in rule) || !('enabled' in rule)) continue + const { path, enabled } = rule + if (typeof path !== 'string' || typeof enabled !== 'boolean') continue + const normalized = normalizeRulePath(path) + if (normalized.kind === 'invalid' || parsed.some(kept => kept.path === normalized.path)) continue + parsed.push({ path: normalized.path, enabled }) + } + return parsed +} + +/** + * The fields of preferences that differ from the defaults, as browser storage + * keeps them: a field equal to its default is left out, so a later build's + * default reaches it. + * @param preferences - the preferences to store. + * @returns the differing fields, or `undefined` when nothing differs from {@link defaultSearchPreferences}. + */ +export function changedSearchPreferences(preferences: WorkspaceSearchPreferences): Partial | undefined { + const defaults = defaultSearchPreferences() + const changed: { -readonly [K in keyof WorkspaceSearchPreferences]?: WorkspaceSearchPreferences[K] } = {} + if (!isDefaultRules(preferences.rules)) changed.rules = preferences.rules.map(rule => ({ path: rule.path, enabled: rule.enabled })) + if (preferences.showHidden !== defaults.showHidden) changed.showHidden = preferences.showHidden + if (preferences.respectGitignore !== defaults.respectGitignore) changed.respectGitignore = preferences.respectGitignore + return Object.keys(changed).length === 0 ? undefined : changed +} + +/** + * Whether the tree treats an entry as hidden: its name starts with `.`. The + * Host's search judges names the same way, except on a Windows Host, where + * ripgrep also treats an entry with the hidden file attribute as hidden. The + * tree lists names one level at a time under the workspace root, so a name is + * judged on its own. + * @param entry - a listed entry. + * @returns `true` for a hidden entry. + */ +export function isHiddenEntry(entry: { readonly name: string }): boolean { + return entry.name.startsWith('.') +} diff --git a/packages/client/ui-sidebar-files/src/client/search-tree.ts b/packages/client/ui-sidebar-files/src/client/search-tree.ts new file mode 100644 index 0000000000..06b51dafe2 --- /dev/null +++ b/packages/client/ui-sidebar-files/src/client/search-tree.ts @@ -0,0 +1,74 @@ +/** + * The search results drawn as a tree, and the highlights in its names. + * + * A search answers with the workspace-relative paths of its matches, and not + * necessarily with their ancestors. The tree the navigator draws + * needs every level from the root down, so this module synthesizes the levels + * the results pass through, keyed by absolute path as the listed tree keys them. + */ +import { childPath } from './store.ts' +import type { DirLevel, FilesTreeState } from './store.ts' + +/** One entry a search matched. */ +export interface SearchMatch { + /** `/`-separated path relative to the workspace root, without leading or trailing `/`. */ + readonly path: string + /** Whether the entry is a file, a directory, or something else. */ + readonly type: DirLevel['entries'][number]['type'] +} + +/** + * Draw search results as a fully expanded tree. + * + * Every directory the results pass through gets a ready level: the root, each + * result's ancestors, and each result that is itself a directory. A level lists + * the children the results put there, once each, in first-seen order; a name + * that is an ancestor anywhere is a directory. A directory that matched but + * holds no matching descendant has a level with no entries, which the + * navigator draws differently from an empty listed directory. Every level is + * expanded, no level is truncated, and the tree does not refresh itself. + * @param root - absolute path of the workspace root, as `FilesTreeState.root` holds it. + * @param entries - the search's entries. + * @returns the tree to draw. + */ +export function buildSearchTree(root: string, entries: readonly SearchMatch[]): FilesTreeState { + type Names = Map + const children = new Map([[root, new Map()]]) + const directory = (path: string): Names => { + const names = children.get(path) ?? new Map() + children.set(path, names) + return names + } + for (const entry of entries) { + const parts = entry.path.split('/') + let parent = root + for (const [index, name] of parts.entries()) { + const type = index === parts.length - 1 ? entry.type : 'directory' + const names = directory(parent) + if (type === 'directory' || !names.has(name)) names.set(name, type) + parent = childPath(parent, name) + if (type === 'directory') directory(parent) + } + } + const levels: FilesTreeState['levels'] = {} + for (const [path, names] of children) { + const level: DirLevel = { entries: [...names].map(([name, type]) => ({ name, type })), truncated: false } + levels[path] = { kind: 'ready', level } + } + return { autoRefresh: false, root, levels, expanded: [...children.keys()] } +} + +/** + * Where the query occurs in one name, for highlighting. + * + * Matching ignores case by Unicode simple case folding; the ranges index the + * name as given. An empty query has no occurrence. + * @param name - the drawn name. + * @param query - the search text. + * @returns non-overlapping `[start, end)` UTF-16 ranges, left to right. + */ +export function matchRanges(name: string, query: string): readonly (readonly [number, number])[] { + if (query === '') return [] + const pattern = new RegExp(query.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'giu') + return [...name.matchAll(pattern)].map(match => [match.index, match.index + match[0].length] as const) +} diff --git a/packages/client/ui-sidebar-files/src/client/search-workspace.ts b/packages/client/ui-sidebar-files/src/client/search-workspace.ts new file mode 100644 index 0000000000..a3f90efa93 --- /dev/null +++ b/packages/client/ui-sidebar-files/src/client/search-workspace.ts @@ -0,0 +1,67 @@ +/** + * The workspace search, bound to the Host's `workspaceFiles.search` call. + * + * The Host walks the whole workspace of the Session and applies the query, + * the exclusion rules, the hidden-entry choice, and `.gitignore`; the Client + * only draws the matches it returns. + */ +import type { ClientRemote, RemoteResult } from '@deepseek-ai/dsh-api-remotes/client' +import type { SessionId } from '@deepseek-ai/dsh-session/types' +import type { SearchMatch } from './search-tree.ts' + +/** What one search asks the Host for. */ +export interface WorkspaceSearchRequest { + /** The query without surrounding whitespace; never empty. */ + readonly query: string + /** The paths of the enabled exclusion rules, in list order, as `search-rules.ts` defines them. */ + readonly excludedPaths: readonly string[] + /** Whether hidden entries may match: names starting with `.`, and on a Windows Host also entries with the hidden file attribute. */ + readonly includeHidden: boolean + /** Whether entries the workspace's `.gitignore` files ignore are left out. */ + readonly respectGitignore: boolean +} + +/** One settled search's matches. */ +export interface WorkspaceSearchAnswer { + /** The matches, in the Host's order. */ + readonly matches: readonly SearchMatch[] + /** The Host returned only its first matches because more matched than it returns. */ + readonly truncated: boolean + /** + * Part of the workspace was not searched: the Host's search passed its file + * limit, or a directory could not be read. + */ + readonly incomplete: boolean +} + +/** + * One workspace search. The session travels with the call because the Host + * resolves the workspace root from it. A failure settles the result's error + * branch; aborting `signal` lets the caller drop the answer. + * @param sessionId - Session whose workspace is searched. + * @param request - the query and the preferences it follows. + * @param signal - the search's lifetime; the face aborts it when a newer search or the end of the search replaces it. + * @returns the matches, or why the Host could not search. + */ +export type SearchWorkspace = ( + sessionId: SessionId, + request: WorkspaceSearchRequest, + signal: AbortSignal, +) => Promise> + +/** + * The slice of the Client Remote face the search calls: the `workspaceFiles` + * namespace's `search`, exactly as the Host's generated client declares it. + */ +export type WorkspaceFilesSearchRemote = { + readonly workspaceFiles: Pick +} + +/** + * Bind the workspace search to one Remote face. + * @param remote - the Client Remote face carrying the `workspaceFiles` namespace. + * @returns the search the tree's face performs. + */ +export function createSearch(remote: WorkspaceFilesSearchRemote): SearchWorkspace { + return (sessionId, request, signal) => remote.workspaceFiles.search(sessionId, request, signal) +} diff --git a/packages/client/ui-sidebar-files/src/client/store.ts b/packages/client/ui-sidebar-files/src/client/store.ts index 182a142208..0019d90636 100644 --- a/packages/client/ui-sidebar-files/src/client/store.ts +++ b/packages/client/ui-sidebar-files/src/client/store.ts @@ -1,11 +1,12 @@ /** * The file tree's view state, one record per right-Sidebar pane: which - * directories are expanded, what each loaded level contains, and how the - * navigator is shown. A Slot-standard exclusive store (one instance per - * Session) keyed by pane, so every file tab of a pane draws the same tree; - * the face decides when a pane's record is forgotten (`FilesInjected.attach`). - * Beside it live pure reads shared by the face and the drawings: path keys, - * where a tab's file sits in the listed tree, and the navigator's geometry. + * directories are expanded, what each loaded level contains, how the + * navigator is shown, and the pane's search. A Slot-standard exclusive store + * (one instance per Session) keyed by pane, so every file tab of a pane draws + * the same tree; the face decides when a pane's record is forgotten + * (`FilesInjected.attach`). Beside it live pure reads shared by the face and + * the drawings: path keys, where a tab's file sits in the listed tree, and the + * navigator's geometry. */ import { defineStore, type EngineStoreHandle } from '@deepseek-ai/dsh-client-store' import type { RemoteFailure } from '@deepseek-ai/dsh-api-remotes/client' @@ -98,7 +99,50 @@ export interface FilesTreeState { expanded: string[] } -/** One pane's navigator: whether and how wide it is shown, and its tree once started. */ +/** + * One settled search, as the pane draws it instead of the tree: the matches, + * or why the Host could not search. + */ +export type FilesSearchResult = + | { + readonly kind: 'found' + /** The trimmed query the search answered, which the rows highlight. */ + readonly query: string + /** The matches as a tree; its `expanded` is the search's own and starts with every level. */ + readonly tree: FilesTreeState + /** How many entries the search returned. */ + readonly count: number + /** More entries matched than the Host returns, so only its first `count` are drawn. */ + readonly truncated: boolean + /** + * Part of the workspace was not searched: the Host's search passed its file + * limit, or a directory could not be read. + */ + readonly incomplete: boolean + /** Whether the search was asked to leave out hidden entries; it does not say that any were. */ + readonly hiddenLeftOut: boolean + /** Whether the search was given at least one enabled exclusion rule; it does not say that a rule matched. */ + readonly excluded: boolean + /** Whether the search was asked to respect ignore files; it does not say that one ignored anything. */ + readonly ignored: boolean + } + | { + readonly kind: 'failed' + /** The trimmed query the failed search asked for. */ + readonly query: string + /** Why the Host could not search. */ + readonly failure: RemoteFailure + } + +/** One pane's search while its query is not blank. */ +export interface FilesSearchState { + /** A search is running; the last settled result stays drawn until its answer replaces it. */ + readonly pending: boolean + /** The last settled search; `undefined` until the pane's first search settles. */ + readonly result: FilesSearchResult | undefined +} + +/** One pane's navigator: whether and how wide it is shown, its tree once started, and its search. */ export interface FilesPaneState { /** * Whether a preview's navigator is shown; `undefined`, hidden, until the @@ -109,6 +153,12 @@ export interface FilesPaneState { width: number /** The tree, listed once the navigator is first shown. */ tree: FilesTreeState | undefined + /** The search field's text, as typed; not persisted. */ + query: string + /** Whether the files page draws its search bar; hiding it retains the query and results. */ + searchBarOpen: boolean + /** The pane's search, which the navigator draws instead of the tree; `undefined` while the query is blank. */ + search: FilesSearchState | undefined } /** Every pane's navigator, keyed by pane id. */ @@ -216,7 +266,7 @@ function underRoot(path: string, root: string): string | undefined { * @returns the pane's record. */ function pane(state: FilesState, paneId: PaneId): FilesPaneState { - state.byPane[paneId] ??= { open: undefined, width: DEFAULT_NAVIGATOR_WIDTH, tree: undefined } + state.byPane[paneId] ??= { open: undefined, width: DEFAULT_NAVIGATOR_WIDTH, tree: undefined, query: '', searchBarOpen: false, search: undefined } return state.byPane[paneId] } @@ -242,8 +292,13 @@ type FilesActions = { failed: (draft: FilesState, paneId: PaneId, path: string, failure: RemoteFailure) => void toggled: (draft: FilesState, paneId: PaneId, path: string) => void shown: (draft: FilesState, paneId: PaneId, open: boolean) => void + searchBarShown: (draft: FilesState, paneId: PaneId, open: boolean) => void resized: (draft: FilesState, paneId: PaneId, width: number) => void forget: (draft: FilesState, paneId: PaneId) => void + queried: (draft: FilesState, paneId: PaneId, query: string) => void + searching: (draft: FilesState, paneId: PaneId) => void + searched: (draft: FilesState, paneId: PaneId, result: FilesSearchResult) => void + searchToggled: (draft: FilesState, paneId: PaneId, path: string) => void } /** @@ -344,6 +399,13 @@ export function createFilesStore(): EngineStoreHandle * @param open - whether the navigator is shown. */ shown: (d, paneId: PaneId, open: boolean) => { pane(d, paneId).open = open }, + /** + * Show or hide the files page's search bar without changing its search. + * @param d - draft state. + * @param paneId - the pane being drawn. + * @param open - whether the search bar is shown. + */ + searchBarShown: (d, paneId: PaneId, open: boolean) => { pane(d, paneId).searchBarOpen = open }, /** * Record one pane's navigator width preference. * @param d - draft state. @@ -359,6 +421,55 @@ export function createFilesStore(): EngineStoreHandle forget: (d, paneId: PaneId) => { d.byPane = Object.fromEntries(Object.entries(d.byPane).filter(([id]) => id !== paneId)) }, + /** + * Record the search field's text. A blank query ends the search, and the + * pane draws its tree again; otherwise the last result stays drawn until + * the next is recorded. + * @param d - draft state. + * @param paneId - the pane being drawn. + * @param query - the text as typed. + */ + queried: (d, paneId: PaneId, query: string) => { + const state = pane(d, paneId) + state.query = query + if (query.trim() === '') state.search = undefined + }, + /** + * Record that a search started, keeping the last result drawn; ignored while the query is blank. + * @param d - draft state. + * @param paneId - the pane being drawn. + */ + searching: (d, paneId: PaneId) => { + const state = d.byPane[paneId] + const result = state?.search?.result + if (state !== undefined && state.query.trim() !== '') { + state.search = { pending: true, result: result?.kind === 'failed' ? undefined : result } + } + }, + /** + * Record a search's result, ending the pending search; ignored while the query is blank. + * @param d - draft state. + * @param paneId - the pane being drawn. + * @param result - what the search found, or why it failed. + */ + searched: (d, paneId: PaneId, result: FilesSearchResult) => { + const state = d.byPane[paneId] + if (state !== undefined && state.query.trim() !== '') state.search = { pending: false, result } + }, + /** + * Open a collapsed directory of the search's tree, or collapse an open + * one; the listed tree's expansion stays as it was. + * @param d - draft state. + * @param paneId - the pane being drawn. + * @param path - absolute directory path. + */ + searchToggled: (d, paneId: PaneId, path: string) => { + const result = d.byPane[paneId]?.search?.result + if (result?.kind !== 'found') return + const at = result.tree.expanded.indexOf(path) + if (at >= 0) result.tree.expanded.splice(at, 1) + else result.tree.expanded.push(path) + }, }, }) } diff --git a/packages/client/ui-sidebar-files/src/client/workspace-tree.ts b/packages/client/ui-sidebar-files/src/client/workspace-tree.ts index 80eb21ae98..fe43fe1286 100644 --- a/packages/client/ui-sidebar-files/src/client/workspace-tree.ts +++ b/packages/client/ui-sidebar-files/src/client/workspace-tree.ts @@ -8,12 +8,17 @@ * key drops its trailing separators too, so `/` becomes the empty key and a * Windows drive root `C:\` becomes `C:`, and a top-level child's key still * ends at its parent's last `/`. + * + * Unless the reader shows them, hidden entries are left out, and a directory + * whose every entry is hidden says so. The same conversion draws a search's + * results, whose directories hold only the matches. */ import type { RemoteFailure } from '@deepseek-ai/dsh-api-remotes/client' import type { TreeDirectoryState, TreeEntry } from './FileTree.tsx' import type { TranslateNS } from '@deepseek-ai/dsh-client-ui-slots' import type {} from './locales.ts' -import { childPath, type FilesTreeState } from './store.ts' +import { isHiddenEntry } from './search-rules.ts' +import { childPath, type DirLevel, type FilesTreeState } from './store.ts' /** * Say why a directory could not be listed, in terms of the directory. @@ -52,27 +57,48 @@ export function parentOf(tree: FilesTreeState, path: string): string { return parent === treeRootOf(tree.root) ? tree.root : parent } +/** + * A listing's entries as drawn. + * @param level - one ready listing. + * @param showHidden - whether hidden entries are drawn. + * @returns the entries, without hidden ones unless they are shown. + */ +function drawnEntries(level: DirLevel, showHidden: boolean): DirLevel['entries'] { + return showHidden ? level.entries : level.entries.filter(entry => !isHiddenEntry(entry)) +} + /** * Every listed entry of a pane's tree, collapsed levels included, each named - * as its listing spells it. + * as its listing spells it. Unless hidden entries are drawn, the kept levels + * below a hidden directory are left out with it, so a directory listed while + * hidden entries were shown is not drawn again through its descendants. * @param levels - the pane's level states. + * @param showHidden - whether hidden entries are drawn. * @returns the entries keyed as the tree draws them. */ -export function workspaceEntries(levels: FilesTreeState['levels']): TreeEntry[] { - return Object.entries(levels).flatMap(([path, level]) => level.kind !== 'ready' ? [] : level.level.entries - .map(entry => ({ path: childPath(path, entry.name), name: entry.name, meta: { kind: entry.type } }))) +export function workspaceEntries(levels: FilesTreeState['levels'], showHidden: boolean): TreeEntry[] { + const ready = Object.entries(levels).flatMap(([path, level]) => level.kind === 'ready' ? [{ path, level: level.level }] : []) + const hiddenPrefixes = showHidden ? [] : ready.flatMap(({ path, level }) => level.entries + .filter(entry => entry.type === 'directory' && isHiddenEntry(entry)).map(entry => childPath(childPath(path, entry.name), ''))) + return ready + .filter(({ path }) => !hiddenPrefixes.some(prefix => `${path}/`.startsWith(prefix))) + .flatMap(({ path, level }) => drawnEntries(level, showHidden) + .map(entry => ({ path: childPath(path, entry.name), name: entry.name, meta: { kind: entry.type } }))) } /** * Each directory's listing state: loading until its first listing settles, then - * its latest failure and whether its listing is truncated. + * its latest failure, whether its listing is truncated, and whether it holds + * only hidden entries that are not drawn. * @param tree - the pane's root, level states, and expanded directories. * @param t - namespace-bound translate, for failure lines. + * @param showHidden - whether hidden entries are drawn. * @returns the states keyed as the tree draws them. */ export function workspaceDirectories( tree: Pick, t: TranslateNS<'sidebarFiles'>, + showHidden: boolean, ): Record { const { root, levels, expanded } = tree const directories: Record = {} @@ -83,11 +109,23 @@ export function workspaceDirectories( if (level.kind === 'loading') directories[key] = { loading: true } else if (level.kind === 'failed') directories[key] = { failure: failureLine(t, level.failure) } else { - const state: { failure?: string; truncated?: true } = {} + const state: { failure?: string; truncated?: true; omitted?: 'hidden' } = {} if (level.failure !== undefined) state.failure = failureLine(t, level.failure) if (level.level.truncated) state.truncated = true + if (level.level.entries.length > 0 && drawnEntries(level.level, showHidden).length === 0) state.omitted = 'hidden' directories[key] = state } } return directories } + +/** + * Each directory of a search's results, which holds only the matches below it, + * so a matched directory without matched descendants draws no empty line. + * @param tree - the results as a tree. + * @returns the states keyed as the tree draws them. + */ +export function searchDirectories(tree: Pick): Record { + return Object.fromEntries(Object.keys(tree.levels) + .map(path => [path === tree.root ? treeRootOf(path) : path, { omitted: 'unmatched' } as const])) +} diff --git a/packages/client/ui-sidebar-files/tests/face.client.spec.ts b/packages/client/ui-sidebar-files/tests/face.client.spec.ts index 8502a8d348..15542328af 100644 --- a/packages/client/ui-sidebar-files/tests/face.client.spec.ts +++ b/packages/client/ui-sidebar-files/tests/face.client.spec.ts @@ -10,6 +10,7 @@ import { childPath } from '../src/client/store.ts' import { createFilesStore } from '../src/client/store.ts' import type { DirLevel } from '../src/client/store.ts' import { DirectoryNode } from '../src/client/directory-node.ts' +import { memoryBook, scriptedSearch } from './search-fixture.client.ts' import { scriptedList } from './scripted-list.client.ts' import { sessionFileAddress } from '@deepseek-ai/dsh-util-workspace-path' import type { PaneId, TabId } from '@deepseek-ai/dsh-client-ui-dockkit' @@ -29,7 +30,7 @@ function mount() { const script = scriptedList() /** The tabs the face placed itself beside, with each placement's release. */ const placed: { readonly tabId: TabId; readonly face: FilesInjected; readonly release: Mock<() => void> }[] = [] - const face = filesFace(script.list, script.watch, CAN_OPEN, () => ({ getSnapshot: () => ({ address: '', params: undefined, revision: 0 }), subscribe: () => () => {} }))(SESSION, instance, (tabId, bound) => { + const face = filesFace(script.list, script.watch, CAN_OPEN, () => ({ getSnapshot: () => ({ address: '', params: undefined, revision: 0 }), subscribe: () => () => {} }), scriptedSearch().search, memoryBook())(SESSION, instance, (tabId, bound) => { const release = vi.fn<() => void>() placed.push({ tabId, face: bound, release }) return release @@ -228,11 +229,15 @@ describe('filesFace', () => { onTestFinished(() => { tab.abort() }) face.attach(moved, 'tab-moved' as TabId, tab.signal) expect(instance.getSnapshot().byPane[moved]?.open).toBe(true) - // The tree drawn there next takes the keyboard. + expect(instance.getSnapshot().byPane[moved]?.searchBarOpen).toBe(false) + // The search field drawn there next takes the keyboard, not the tree. const body = { scrollTop: 0, focus: vi.fn() } face.bodyDrawn(moved, `${ROOT}/a.ts`, body as Partial as HTMLElement) + const input = { focus: vi.fn() } + face.searchDrawn(moved, input as Partial as HTMLInputElement) await Promise.resolve() - expect(body.focus).toHaveBeenCalledTimes(1) + expect(body.focus).not.toHaveBeenCalled() + expect(input.focus).toHaveBeenCalledTimes(1) face.hide('pane-never' as PaneId) expect(instance.getSnapshot().byPane['pane-never' as PaneId]).toBeUndefined() }) diff --git a/packages/client/ui-sidebar-files/tests/file-tree.client.spec.tsx b/packages/client/ui-sidebar-files/tests/file-tree.client.spec.tsx index aaebe5e72b..c812f9a287 100644 --- a/packages/client/ui-sidebar-files/tests/file-tree.client.spec.tsx +++ b/packages/client/ui-sidebar-files/tests/file-tree.client.spec.tsx @@ -288,3 +288,26 @@ describe('FileTree', () => { expect(currentRef).toHaveBeenLastCalledWith(null) }) }) + +describe('caller-narrowed entries', () => { + it('highlights each occurrence of the given text in a name, ignoring case', () => { + const { view } = draw({ entries: CHANGES, expanded: ['pr/src'], highlight: 'FILE' }) + const marks = [...view.container.querySelectorAll('mark')].map(mark => mark.textContent) + expect(marks).toEqual(['file', 'file']) + expect(row(view.container, 'pr/src/file2.ts').textContent).toBe('file2.ts') + expect(row(view.container, 'pr/README.md').querySelector('mark')).toBeNull() + }) + + it('says a directory left without entries holds hidden ones, or nothing for a selection', () => { + const entries: readonly TreeEntry[] = [ + { path: 'pr/hidden', name: 'hidden', meta: { kind: 'directory' } }, + { path: 'pr/unmatched', name: 'unmatched', meta: { kind: 'directory' } }, + ] + const { view } = draw({ + entries, expanded: ['pr/hidden', 'pr/unmatched'], + directories: { 'pr/hidden': { omitted: 'hidden' }, 'pr/unmatched': { omitted: 'unmatched' } }, + }) + const notes = [...view.container.querySelectorAll('[data-files-row]')].map(note => [note.getAttribute('data-files-row'), note.textContent]) + expect(notes).toEqual([['hidden-only', en.emptyHidden]]) + }) +}) diff --git a/packages/client/ui-sidebar-files/tests/files-body.client.spec.tsx b/packages/client/ui-sidebar-files/tests/files-body.client.spec.tsx index ddb3dd2dfa..a8ef76a271 100644 --- a/packages/client/ui-sidebar-files/tests/files-body.client.spec.tsx +++ b/packages/client/ui-sidebar-files/tests/files-body.client.spec.tsx @@ -6,9 +6,11 @@ import { makeTranslate, RemoteError } from '@deepseek-ai/dsh-client-test-runtime import type { RemoteFailure } from '@deepseek-ai/dsh-api-remotes/client' import { fileAddressFor } from '@deepseek-ai/dsh-util-workspace-path' import { failureLine } from '../src/client/workspace-tree.ts' +import type { PaneId, TabId } from '@deepseek-ai/dsh-client-ui-dockkit' import type { DirLevel } from '../src/client/store.ts' import { zh } from '../src/client/locales.ts' -import { mountBody, PANE, ROOT, SESSION } from './mount.client.tsx' +import { mountBody, PANE, ROOT, SESSION, TAB } from './mount.client.tsx' +import { answer, searchesSettled } from './search-fixture.client.ts' const ROOT_LEVEL: DirLevel = { entries: [ @@ -52,7 +54,7 @@ describe('FilesBody', () => { const header = page.firstElementChild! expect(header.querySelector('[data-files-path]')?.getAttribute('title')).toBe(ROOT) expect([...header.querySelectorAll('button:not([data-files-auto-refresh])')].map(button => button.getAttribute('aria-label'))) - .toEqual([zh.reload]) + .toEqual([zh.reload, zh['search.placeholder']]) const content = page.querySelector('[data-files-content]')! expect(content.children).toHaveLength(1) const navigator = content.firstElementChild! @@ -71,7 +73,122 @@ describe('FilesBody', () => { expect(typeof bound?.refresh).toBe('function') }) - it('lists the root on mount, heads it with its path split at the last segment, and draws directories first with dotfiles kept', async () => { + it('defers a files-page focus request until its pane attaches without changing the preview-tree choice', () => { + const h = mountBody() + const paneId = 'pane-before-attach' as PaneId + const navigator = document.createElement('aside') + h.view.container.append(navigator) + const controller = new AbortController() + onTestFinished(() => { controller.abort() }) + h.face.navigatorDrawn(paneId, navigator, false) + h.face.focus(paneId) + expect(h.instance.getSnapshot().byPane[paneId]).toBeUndefined() + h.face.attach(paneId, 'tab-before-attach' as TabId, controller.signal) + expect(h.instance.getSnapshot().byPane[paneId]).toMatchObject({ open: undefined, searchBarOpen: true }) + h.face.navigatorDrawn(paneId, null, false) + navigator.remove() + }) + + it('starts with the search bar collapsed and retains the query and results through collapse and remount', async () => { + const h = mountBody() + await act(() => h.script.watches.ready(ROOT)) + await act(() => h.script.settle({ ok: true, value: ROOT_LEVEL })) + const toggle = h.view.getByRole('button', { name: zh['search.placeholder'] }) + expect(toggle.getAttribute('aria-expanded')).toBe('false') + expect(h.view.queryByRole('searchbox')).toBeNull() + expect(h.beside(TAB)).toBe(true) + await act(async () => { fireEvent.click(toggle) }) + const input = h.view.container.querySelector('[data-files-search]')! + expect(document.activeElement).toBe(input) + expect(toggle.getAttribute('aria-expanded')).toBe('true') + expect(h.instance.getSnapshot().byPane[PANE]?.open).toBeUndefined() + h.searches.answerWith(answer(['src/target.ts'])) + await act(async () => { + fireEvent.change(input, { target: { value: 'target' } }) + await searchesSettled() + }) + const search = h.instance.getSnapshot().byPane[PANE]!.search + const body = h.view.container.querySelector('[data-files-body]')! + expect(body.getAttribute('data-files-view')).toBe('search') + expect(names(h.view.container)).toContain(`${ROOT}/src/target.ts`) + fireEvent.click(toggle) + expect(document.activeElement).toBe(toggle) + expect(h.view.queryByRole('searchbox')).toBeNull() + expect(toggle.getAttribute('aria-expanded')).toBe('false') + expect(h.instance.getSnapshot().byPane[PANE]!.query).toBe('target') + expect(h.instance.getSnapshot().byPane[PANE]!.search).toBe(search) + expect(h.view.container.querySelector('[data-files-body]')).toBe(body) + h.view.unmount() + const back = h.remount() + expect(back.queryByRole('searchbox')).toBeNull() + expect(names(back.container)).toContain(`${ROOT}/src/target.ts`) + await act(async () => { fireEvent.click(back.getByRole('button', { name: zh['search.placeholder'] })) }) + expect(back.container.querySelector('[data-files-search]')?.value).toBe('target') + expect(h.search).toHaveBeenCalledTimes(1) + }) + + it('keeps focus in the tree when Escape clears results while the search bar is collapsed', async () => { + const h = mountBody() + await act(() => h.script.watches.ready(ROOT)) + await act(() => h.script.settle({ ok: true, value: ROOT_LEVEL })) + const toggle = h.view.getByRole('button', { name: zh['search.placeholder'] }) + await act(async () => { fireEvent.click(toggle) }) + h.searches.answerWith(answer(['src/target.ts'])) + await act(async () => { + fireEvent.change(h.view.getByRole('searchbox'), { target: { value: 'target' } }) + await searchesSettled() + }) + fireEvent.click(toggle) + const result = h.view.container.querySelector(`[data-files-path="${ROOT}/src/target.ts"] button`)! + result.focus() + await act(async () => { fireEvent.keyDown(result, { key: 'Escape' }) }) + const body = h.view.container.querySelector('[data-files-body]')! + expect(body.getAttribute('data-files-view')).toBe('tree') + expect(h.instance.getSnapshot().byPane[PANE]!.query).toBe('') + expect(h.view.queryByRole('searchbox')).toBeNull() + expect(document.activeElement).toBe(body) + }) + + it('keeps a search running and draws its answer while the search bar is collapsed', async () => { + const h = mountBody() + await act(() => h.script.watches.ready(ROOT)) + await act(() => h.script.settle({ ok: true, value: ROOT_LEVEL })) + const toggle = h.view.getByRole('button', { name: zh['search.placeholder'] }) + await act(async () => { fireEvent.click(toggle) }) + h.searches.answerWith(undefined) + fireEvent.change(h.view.getByRole('searchbox'), { target: { value: 'target' } }) + const running = h.searches.started[0]! + fireEvent.click(toggle) + expect(running.signal.aborted).toBe(false) + expect(h.view.container.querySelector('[data-files-body]')?.getAttribute('aria-busy')).toBe('true') + await act(async () => { + running.settle(answer(['src/target.ts'])) + await searchesSettled() + }) + expect(h.view.queryByRole('searchbox')).toBeNull() + expect(names(h.view.container)).toContain(`${ROOT}/src/target.ts`) + expect(h.view.container.querySelector('[data-files-body]')?.getAttribute('aria-busy')).toBe('false') + await act(async () => { h.face.focus(PANE) }) + expect(h.view.container.querySelector('[data-files-search]')?.value).toBe('target') + expect(document.activeElement).toBe(h.view.getByRole('searchbox')) + expect(h.search).toHaveBeenCalledTimes(1) + }) + + it('closes search settings when the toolbar collapses the search bar', async () => { + const h = mountBody() + await act(() => h.script.watches.ready(ROOT)) + await act(() => h.script.settle({ ok: true, value: ROOT_LEVEL })) + const toggle = h.view.getByRole('button', { name: zh['search.placeholder'] }) + await act(async () => { fireEvent.click(toggle) }) + fireEvent.click(h.view.getByRole('button', { name: zh['search.settings'] })) + expect(h.view.queryByRole('dialog')).not.toBeNull() + fireEvent.click(toggle) + expect(h.view.queryByRole('dialog')).toBeNull() + await act(async () => { fireEvent.click(toggle) }) + expect(h.view.queryByRole('dialog')).toBeNull() + }) + + it('lists the root on mount, heads it with its path split at the last segment, and draws directories first with dotfiles shown once the reader asks', async () => { const { view, script } = mountBody() expect(script.list).not.toHaveBeenCalled() await act(() => script.watches.ready(ROOT)) @@ -82,6 +199,9 @@ describe('FilesBody', () => { const path = view.container.querySelector('[data-files-path]') expect(path?.getAttribute('title')).toBe(ROOT) expect([...path?.firstElementChild?.children ?? []].map(span => span.textContent)).toEqual(['/work/', 'app']) + expect(names(view.container)).toEqual([`${ROOT}/src`, `${ROOT}/pipe`, `${ROOT}/README.md`]) + await act(async () => { fireEvent.click(view.getByRole('button', { name: zh['search.placeholder'] })) }) + act(() => { fireEvent.click(view.getByRole('button', { name: zh['search.showHidden'] })) }) expect(names(view.container)).toEqual([`${ROOT}/src`, `${ROOT}/.env`, `${ROOT}/pipe`, `${ROOT}/README.md`]) const envIcon = view.container.querySelector(`[data-files-path="${ROOT}/.env"] svg`)?.innerHTML const readmeIcon = view.container.querySelector(`[data-files-path="${ROOT}/README.md"] svg`)?.innerHTML @@ -94,7 +214,7 @@ describe('FilesBody', () => { await act(() => script.settle({ ok: true, value: ROOT_LEVEL })) const path = view.container.querySelector('[data-files-path]') expect([...path?.firstElementChild?.children ?? []].map(span => span.textContent)).toEqual(['/']) - expect(names(view.container)).toEqual(['/src', '/.env', '/pipe', '/README.md']) + expect(names(view.container)).toEqual(['/src', '/pipe', '/README.md']) }) it('marks the root path clipped while its text is wider than its box, re-reading on resize', async () => { diff --git a/packages/client/ui-sidebar-files/tests/files-navigator.client.spec.tsx b/packages/client/ui-sidebar-files/tests/files-navigator.client.spec.tsx index baaa8e2d53..60bb8667f7 100644 --- a/packages/client/ui-sidebar-files/tests/files-navigator.client.spec.tsx +++ b/packages/client/ui-sidebar-files/tests/files-navigator.client.spec.tsx @@ -610,6 +610,17 @@ describe('FileNavigator beside a preview', () => { expect(document.activeElement).toBe(document.body) }) + it('carries keyboard focus on the tree\'s scroll container, with no current row, to the next drawing', async () => { + const h = mountNavigator('sidebar://files') + act(() => { h.instance.actions.shown(PANE, true) }) + await act(() => h.script.watches.ready(ROOT)) + act(() => { h.view.container.querySelector('[data-files-body]')!.focus() }) + h.view.unmount() + const back = h.remount() + await endTask() + expect(document.activeElement).toBe(back.container.querySelector('[data-files-body]')) + }) + it('gives the carried focus to the drawing in place when the task ends, after the tab draws the tree again', async () => { const h = mountNavigator(A) await revealSrc(h) @@ -629,15 +640,15 @@ describe('FileNavigator beside a preview', () => { expect(document.activeElement).toBe(document.body) }) - it('takes keyboard focus when asked, at its next drawing or at once', async () => { + it('gives its search field keyboard focus when asked, at its next drawing or at once', async () => { const h = mountNavigator('sidebar://files') act(() => { h.face.focus(PANE) }) await act(() => h.script.watches.ready(ROOT)) - expect(document.activeElement?.hasAttribute('data-files-body')).toBe(true) + expect(document.activeElement).toBe(h.view.getByRole('searchbox', { name: zh['search.placeholder'] })) act(() => { (document.activeElement as HTMLElement).blur() }) act(() => { h.face.focus(PANE) }) await endTask() - expect(document.activeElement?.hasAttribute('data-files-body')).toBe(true) + expect(document.activeElement).toBe(h.view.getByRole('searchbox', { name: zh['search.placeholder'] })) }) it('keeps the pane\'s tree when a preview is replaced in place by the file it opened', async () => { diff --git a/packages/client/ui-sidebar-files/tests/mount.client.tsx b/packages/client/ui-sidebar-files/tests/mount.client.tsx index 35a660f264..32a4cc14aa 100644 --- a/packages/client/ui-sidebar-files/tests/mount.client.tsx +++ b/packages/client/ui-sidebar-files/tests/mount.client.tsx @@ -27,10 +27,12 @@ import { FilesBody } from '../src/client/FilesBody.tsx' import type { FilesBodyProps } from '../src/client/FilesBody.tsx' import { NavigatorToggle } from '../src/client/NavigatorToggle.tsx' import { zh } from '../src/client/locales.ts' +import type { WorkspaceSearchPreferenceBook } from '../src/client/search-preferences.ts' import { createFilesStore } from '../src/client/store.ts' import type { FilesStoreInstance } from '../src/client/store.ts' import { scriptedList } from './scripted-list.client.ts' import type { ScriptedList } from './scripted-list.client.ts' +import { memoryBook, scriptedSearch, type ScriptedSearch } from './search-fixture.client.ts' export const SESSION = 's-test' as SessionId export const ROOT = '/work/app' @@ -71,6 +73,12 @@ export interface Mounted { readonly view: RenderResult readonly instance: FilesStoreInstance readonly script: ScriptedList + /** The search the face calls; it matches nothing until a spec gives it matches. */ + readonly search: ScriptedSearch['search'] + /** That search's script: the searches it started, and how it answers later ones. */ + readonly searches: ScriptedSearch + /** The search preferences of every workspace, which another mount may share. */ + readonly book: WorkspaceSearchPreferenceBook readonly face: FilesInjected /** The first tab's lifetime. */ readonly controller: AbortController @@ -93,9 +101,11 @@ export interface Mounted { } /** One store instance, one face, one owner share. */ -function harness(placement: Placement, controller: AbortController) { +function harness(placement: Placement, controller: AbortController, book: WorkspaceSearchPreferenceBook) { const instance = createFilesStore().create() const script = scriptedList() + const searches = scriptedSearch() + const search = searches.search // Whether some tab type opens files: one predicate per answer, as registrations change it. const listeners = new Set<() => void>() let check = (_address: string): boolean => true @@ -115,7 +125,7 @@ function harness(placement: Placement, controller: AbortController) { // The owner's count of drawings that placed the face beside each tab. const placed = new Map() const navigation = createSnapshotStore({ address: placement.contentId, params: undefined, revision: 0 }) - const face = filesFace(script.list, script.watch, canOpen, () => navigation)(SESSION, instance, (tabId) => { + const face = filesFace(script.list, script.watch, canOpen, () => navigation, search, book)(SESSION, instance, (tabId) => { placed.set(tabId, (placed.get(tabId) ?? 0) + 1) return () => { placed.set(tabId, placed.get(tabId)! - 1) } }) @@ -164,9 +174,13 @@ function harness(placement: Placement, controller: AbortController) { }, useCanOpen: (sel: (check: (address: string) => boolean) => S) => sel(useSyncExternalStore(listener => hooks.canOpen.subscribe(listener), () => hooks.canOpen.getSnapshot())), + useSearchPreferences: (cwd: string) => { + const source = keyedHooks.searchPreferences(cwd) + return useSyncExternalStore(listener => source.subscribe(listener), () => source.getSnapshot()) + }, t, } - return { instance, script, face, controller, viewers, tabActions, onOpen, beside, shared } + return { instance, script, search, searches, book, face, controller, viewers, tabActions, onOpen, beside, shared } } /** @@ -278,16 +292,18 @@ function bodyProps(shared: object): FilesBodyProps { * @param placement - the tab's address, the pane's presentation, and the session's working directory. * @param draw - the drawing over the shared props. * @param recordObservers - capture navigator measurement notifications. + * @param book - the search preferences, shared with another mount when given. * @returns the mounted hands. */ function mount( given: Omit, draw: (shared: ReturnType['shared']) => ReactNode, recordObservers = false, + book: WorkspaceSearchPreferenceBook = memoryBook(), ): Mounted { const controller = new AbortController() const placement: Placement = { ...given, tabId: TAB, signal: controller.signal } - const { shared, ...hands } = harness(placement, controller) + const { shared, ...hands } = harness(placement, controller, book) const { row, viewport } = measureRows(recordObservers) const element = (): ReactNode =>
    {draw(shared)}
    const view = render(element()) @@ -319,9 +335,15 @@ export function mountBody(cwd: string | null = ROOT, placement: Partial = {}, cwd: string | null = ROOT): Mounted { +export function mountNavigator( + contentId: string, + placement: Partial = {}, + cwd: string | null = ROOT, + book?: WorkspaceSearchPreferenceBook, +): Mounted { return mount({ contentId, ...placement, cwd }, shared => ( <> @@ -330,5 +352,5 @@ export function mountNavigator(contentId: string, placement: Partial
    - ), true) + ), true, book) } diff --git a/packages/client/ui-sidebar-files/tests/navigation.client.spec.tsx b/packages/client/ui-sidebar-files/tests/navigation.client.spec.tsx index 7aee76c1fc..06aec36122 100644 --- a/packages/client/ui-sidebar-files/tests/navigation.client.spec.tsx +++ b/packages/client/ui-sidebar-files/tests/navigation.client.spec.tsx @@ -9,7 +9,9 @@ import type { SessionId } from '@deepseek-ai/dsh-session/types' import { sessionFileAddress } from '@deepseek-ai/dsh-util-workspace-path' import { filesFace } from '../src/client/face.ts' import type { ListWorkspaceDirectory, WatchWorkspaceDirectory } from '../src/client/face.ts' +import type { SearchWorkspace } from '../src/client/search-workspace.ts' import { createFilesStore } from '../src/client/store.ts' +import { memoryBook } from './search-fixture.client.ts' const SESSION = 'navigation-session' as SessionId const LEFT = 'pane-left' as PaneId @@ -73,7 +75,7 @@ function harness() { return () => { unsubscribe(); tab.listeners.delete(listener) } }, } - })(SESSION, instance, () => () => {}) + }, vi.fn(() => { throw new Error('Navigation must not search') }), memoryBook())(SESSION, instance, () => () => {}) const tab = (id: TabId, revision = 0): TestTab => { const address = sessionFileAddress(SESSION, `${id}.ts`) diff --git a/packages/client/ui-sidebar-files/tests/search-face.client.spec.ts b/packages/client/ui-sidebar-files/tests/search-face.client.spec.ts new file mode 100644 index 0000000000..2436eac12e --- /dev/null +++ b/packages/client/ui-sidebar-files/tests/search-face.client.spec.ts @@ -0,0 +1,224 @@ +/** The face's search: when it runs, the preferences it follows, what it aborts and drops, and the store writes it makes. */ +import { describe, expect, it, onTestFinished, vi } from 'vitest' +import { RemoteError } from '@deepseek-ai/dsh-client-test-runtime' +import type { PaneId, TabId } from '@deepseek-ai/dsh-client-ui-dockkit' +import type { SessionId } from '@deepseek-ai/dsh-session/types' +import { filesFace } from '../src/client/face.ts' +import type { WorkspaceSearchPreferenceBook } from '../src/client/search-preferences.ts' +import { defaultSearchPreferences, enabledPaths } from '../src/client/search-rules.ts' +import { createSearch } from '../src/client/search-workspace.ts' +import type { WorkspaceFilesSearchRemote } from '../src/client/search-workspace.ts' +import { createFilesStore } from '../src/client/store.ts' +import { scriptedList } from './scripted-list.client.ts' +import { answer, memoryBook, scriptedSearch, searchesSettled } from './search-fixture.client.ts' + +const ROOT = '/work/app' +const PANE = 'pane-1' as PaneId +const CAN_OPEN = { getSnapshot: () => () => true, subscribe: () => () => {} } +/** No tab navigates. */ +const NO_NAVIGATION = () => ({ getSnapshot: () => ({ address: '', params: undefined, revision: 0 }), subscribe: () => () => {} }) + +/** One Session's face over a shared book, with its pane attached. */ +function session(id: string, book: WorkspaceSearchPreferenceBook) { + const instance = createFilesStore().create() + const script = scriptedList() + const searches = scriptedSearch(['src/a.ts']) + const face = filesFace(script.list, script.watch, CAN_OPEN, NO_NAVIGATION, searches.search, book)( + id as SessionId, instance, () => () => {}, + ) + const tab = new AbortController() + const release = face.attach(PANE, 'tab-1' as TabId, tab.signal) + onTestFinished(async () => { + release() + tab.abort() + await script.dispose() + }) + return { + instance, searches, search: searches.search, face, release, tab, + state: () => instance.getSnapshot().byPane[PANE]?.search, + result: () => instance.getSnapshot().byPane[PANE]?.search?.result, + } +} + +describe('the face\'s search', () => { + it('searches nothing for a pane no navigator attached', () => { + const h = session('s-1', memoryBook()) + h.face.setQuery('pane-2' as PaneId, ROOT, 'a') + h.face.retrySearch('pane-2' as PaneId) + expect(h.search).not.toHaveBeenCalled() + expect(h.instance.getSnapshot().byPane['pane-2' as PaneId]).toBeUndefined() + // A search field that goes after its pane was forgotten leaves nothing to record. + h.face.searchDrawn('pane-2' as PaneId, null) + expect(h.instance.getSnapshot().byPane['pane-2' as PaneId]).toBeUndefined() + }) + + it('asks the Host with the trimmed query and the workspace\'s preferences, pending until the answer draws the matches as a tree', async () => { + const h = session('s-1', memoryBook()) + h.searches.answerWith(undefined) + h.face.setQuery(PANE, ROOT, ' a ') + expect(h.search).toHaveBeenCalledExactlyOnceWith('s-1', { + query: 'a', excludedPaths: enabledPaths(defaultSearchPreferences().rules), includeHidden: false, respectGitignore: true, + }, expect.any(AbortSignal)) + expect(h.state()).toEqual({ pending: true, result: undefined }) + h.searches.started[0]!.settle(answer(['src/a.ts'], { truncated: true, incomplete: true })) + await searchesSettled() + expect(h.state()?.pending).toBe(false) + expect(h.result()).toMatchObject({ + kind: 'found', query: 'a', count: 1, truncated: true, incomplete: true, hiddenLeftOut: true, excluded: true, + tree: { expanded: [ROOT, `${ROOT}/src`] }, + }) + // Whitespace around the same text searches nothing new. + h.face.setQuery(PANE, ROOT, 'a') + expect(h.search).toHaveBeenCalledOnce() + expect(h.instance.getSnapshot().byPane[PANE]?.query).toBe('a') + }) + + it('aborts the running search when a newer one starts, keeping the last result drawn and dropping the older answer', async () => { + const h = session('s-1', memoryBook()) + h.face.setQuery(PANE, ROOT, 'a') + await searchesSettled() + const first = h.result() + h.searches.answerWith(undefined) + h.face.setQuery(PANE, ROOT, 'ab') + h.face.setQuery(PANE, ROOT, 'abc') + const [, older, newer] = h.searches.started + expect(older?.signal.aborted).toBe(true) + expect(newer?.signal.aborted).toBe(false) + expect(h.state()).toEqual({ pending: true, result: first }) + // The newer answer draws; the older one, arriving later, is dropped. + newer!.settle(answer(['abc.ts'])) + await searchesSettled() + older!.settle(answer(['ab.ts'])) + await searchesSettled() + expect(h.result()).toMatchObject({ query: 'abc', count: 1 }) + expect(h.state()?.pending).toBe(false) + }) + + it('aborts a running search when the query is cleared or the pane is forgotten, and writes nothing for it', async () => { + const h = session('s-1', memoryBook()) + h.searches.answerWith(undefined) + h.face.setQuery(PANE, ROOT, 'a') + const cleared = h.searches.started[0]! + h.face.setQuery(PANE, ROOT, '') + expect(cleared.signal.aborted).toBe(true) + // Typing the same text again searches anew, and the dropped answer does not draw. + h.face.setQuery(PANE, ROOT, 'a') + cleared.settle(answer(['old.ts'])) + await searchesSettled() + expect(h.state()).toEqual({ pending: true, result: undefined }) + + const running = h.searches.started[1]! + h.release() + h.tab.abort() + await searchesSettled() + expect(running.signal.aborted).toBe(true) + running.settle(answer(['late.ts'])) + await searchesSettled() + expect(h.instance.getSnapshot().byPane[PANE]).toBeUndefined() + }) + + it('records a failed search as a failure, and searches again on retry', async () => { + const h = session('s-1', memoryBook()) + h.searches.answerWith(undefined) + h.face.setQuery(PANE, ROOT, 'a') + const failure = new RemoteError('gateway/internal', 'host is gone', {}) + h.searches.started[0]!.settle({ ok: false, error: failure }) + await searchesSettled() + expect(h.state()).toEqual({ pending: false, result: { kind: 'failed', query: 'a', failure } }) + h.face.retrySearch(PANE) + expect(h.search).toHaveBeenCalledTimes(2) + // A failure belongs to the query that failed, so a new search drops it instead of drawing it beside the spinner. + expect(h.state()).toEqual({ pending: true, result: undefined }) + // A rejected call, which only an assembly fault causes, is reported rather than left pending. + h.searches.started[1]!.reject(new Error('no search method')) + await searchesSettled() + expect(h.state()).toMatchObject({ + pending: false, result: { kind: 'failed', query: 'a', failure: { code: 'gateway/internal', message: 'no search method' } }, + }) + h.face.retrySearch(PANE) + h.searches.started[2]!.reject('refused') + await searchesSettled() + expect(h.result()).toMatchObject({ kind: 'failed', failure: { message: 'refused' } }) + // Without a query there is nothing to retry. + h.face.setQuery(PANE, ROOT, '') + h.face.retrySearch(PANE) + expect(h.search).toHaveBeenCalledTimes(3) + }) + + it('searches again on refresh while the pane has a query, and not once the query is blank', async () => { + const h = session('s-1', memoryBook()) + h.face.setQuery(PANE, ROOT, 'a') + await searchesSettled() + expect(h.search).toHaveBeenCalledTimes(1) + h.face.refresh(PANE) + expect(h.search).toHaveBeenCalledTimes(2) + expect(h.search).toHaveBeenLastCalledWith('s-1', expect.objectContaining({ query: 'a' }), expect.anything()) + h.face.setQuery(PANE, ROOT, '') + h.face.refresh(PANE) + expect(h.search).toHaveBeenCalledTimes(2) + }) + + it('follows one workspace\'s preferences, shared by every Session rooted there, until the query is blank', async () => { + const book = memoryBook() + const one = session('s-1', book) + const two = session('s-2', book) + const elsewhere = session('s-3', book) + expect(two.face.keyedHooks.searchPreferences(ROOT)).toBe(one.face.keyedHooks.searchPreferences(`${ROOT}/`)) + two.face.setQuery(PANE, ROOT, 'a') + elsewhere.face.setQuery(PANE, '/work/other', 'a') + const before = one.face.keyedHooks.searchPreferences('/work/other').getSnapshot() + one.face.setSearchPreferences(ROOT, { rules: [], showHidden: true }) + expect(two.search).toHaveBeenLastCalledWith( + 's-2', { query: 'a', excludedPaths: [], includeHidden: true, respectGitignore: true }, expect.any(AbortSignal), + ) + await searchesSettled() + expect(two.result()).toMatchObject({ hiddenLeftOut: false, excluded: false }) + expect(elsewhere.search).toHaveBeenCalledOnce() + expect(one.face.keyedHooks.searchPreferences('/work/other').getSnapshot()).toBe(before) + // The `.gitignore` choice is a preference like the others. + one.face.setSearchPreferences(ROOT, { respectGitignore: false }) + expect(two.search).toHaveBeenLastCalledWith('s-2', expect.objectContaining({ respectGitignore: false }), expect.any(AbortSignal)) + // A change to another root's entry leaves this root's search alone. + one.face.setSearchPreferences('/work/other', { ...defaultSearchPreferences(), rules: [] }) + expect(two.search).toHaveBeenCalledTimes(3) + + // A pane that moves to another root follows that root instead. + two.face.setQuery(PANE, '/work/other', 'a') + expect(two.search).toHaveBeenCalledTimes(4) + one.face.setSearchPreferences(ROOT, defaultSearchPreferences()) + expect(two.search).toHaveBeenCalledTimes(4) + + two.face.setQuery(PANE, '/work/other', ' ') + expect(two.state()).toBeUndefined() + one.face.setSearchPreferences('/work/other', { ...defaultSearchPreferences(), showHidden: true }) + expect(two.search).toHaveBeenCalledTimes(4) + }) + + it('does not search again when the listed levels change under shown results', async () => { + const h = session('s-1', memoryBook()) + h.instance.actions.start(PANE, ROOT) + h.face.setQuery(PANE, ROOT, 'a') + await searchesSettled() + const shown = h.result() + h.instance.actions.loaded(PANE, ROOT, { entries: [{ name: 'src', type: 'directory' }], truncated: false }) + await searchesSettled() + expect(h.search).toHaveBeenCalledOnce() + expect(h.result()).toBe(shown) + }) +}) + +describe('createSearch', () => { + it('passes the session, the request, and the signal through, and returns the Host\'s answer or failure as it is', async () => { + const search = vi.fn() + const value = { matches: [{ path: 'src/a.ts', type: 'file' as const }], truncated: false, incomplete: true } + search.mockResolvedValueOnce({ ok: true, value }) + const request = { query: 'a', excludedPaths: ['dist'], includeHidden: false, respectGitignore: true } + const signal = new AbortController().signal + const bound = createSearch({ workspaceFiles: { search } }) + expect(await bound('s-1' as SessionId, request, signal)).toEqual({ ok: true, value }) + expect(search).toHaveBeenCalledWith('s-1', request, signal) + const error = new RemoteError('gateway/internal', 'down', {}) + search.mockResolvedValueOnce({ ok: false, error }) + expect(await bound('s-1' as SessionId, request, signal)).toEqual({ ok: false, error }) + }) +}) diff --git a/packages/client/ui-sidebar-files/tests/search-fixture.client.ts b/packages/client/ui-sidebar-files/tests/search-fixture.client.ts new file mode 100644 index 0000000000..2b7fc0c35d --- /dev/null +++ b/packages/client/ui-sidebar-files/tests/search-fixture.client.ts @@ -0,0 +1,91 @@ +/** Scripted search answers and in-memory preferences for the search specs. */ +import { vi } from 'vitest' +import type { Mock } from 'vitest' +import type { RemoteResult } from '@deepseek-ai/dsh-api-remotes/client' +import { createSnapshotStore } from '@deepseek-ai/dsh-client-store' +import { createSearchPreferenceBook } from '../src/client/search-preferences.ts' +import type { WorkspaceSearchPreferenceBook } from '../src/client/search-preferences.ts' +import type { SearchMatch } from '../src/client/search-tree.ts' +import type { SearchWorkspace, WorkspaceSearchAnswer, WorkspaceSearchRequest } from '../src/client/search-workspace.ts' + +/** + * Matches as a search answers them. + * @param paths - matched paths, files unless they end with `/`. + * @returns the matches. + */ +export function found(paths: readonly string[]): SearchMatch[] { + return paths.map(path => path.endsWith('/') ? { path: path.slice(0, -1), type: 'directory' as const } : { path, type: 'file' as const }) +} + +/** Whether an answer is cut at the Host's match limit, or missed some directories. */ +export interface AnswerFlags { + readonly truncated?: boolean + readonly incomplete?: boolean +} + +/** + * A settled answer. + * @param paths - the matches, as {@link found} takes them. + * @param flags - whether the answer is truncated or incomplete; neither when omitted. + * @returns the answer. + */ +export function answer(paths: readonly string[], flags: AnswerFlags = {}): RemoteResult { + return { ok: true, value: { matches: found(paths), truncated: flags.truncated ?? false, incomplete: flags.incomplete ?? false } } +} + +/** One search the face started. */ +export interface StartedSearch { + /** What the face asked for. */ + readonly request: WorkspaceSearchRequest + /** The signal the face passed. */ + readonly signal: AbortSignal + /** Answer it, unless it is already answered. @param result - the answer. */ + readonly settle: (result: RemoteResult) => void + /** Reject it, as an assembly fault does. @param error - the rejection. */ + readonly reject: (error: unknown) => void +} + +/** A search the face calls: it answers at once with a fixed answer, or leaves each call open for the spec to settle. */ +export interface ScriptedSearch { + /** The mock the face receives. */ + readonly search: Mock + /** Every search started, oldest first. */ + readonly started: readonly StartedSearch[] + /** + * Answer every later search at once, or leave later searches open. + * @param result - the answer, or `undefined` to leave them open. + */ + readonly answerWith: (result: RemoteResult | undefined) => void +} + +/** + * A search that answers every request with the same matches until the spec + * changes them. + * @param paths - the matches, as {@link found} takes them. + * @returns the scripted search. + */ +export function scriptedSearch(paths: readonly string[] = []): ScriptedSearch { + const started: StartedSearch[] = [] + let fixed: RemoteResult | undefined = answer(paths) + const search = vi.fn((_sessionId, request, signal) => new Promise((resolve, reject) => { + started.push({ request, signal, settle: resolve, reject }) + if (fixed !== undefined) resolve(fixed) + })) + return { search, started, answerWith: (result) => { fixed = result } } +} + +/** + * Let settled searches reach the face: a search's answer takes two microtasks to be written. + * @returns once they have. + */ +export async function searchesSettled(): Promise { + for (let turn = 0; turn < 3; turn += 1) await Promise.resolve() +} + +/** + * A preference book over an in-memory store, as the plugin keeps one over browser storage. + * @returns the book. + */ +export function memoryBook(): WorkspaceSearchPreferenceBook { + return createSearchPreferenceBook(createSnapshotStore({})) +} diff --git a/packages/client/ui-sidebar-files/tests/search-preferences.client.spec.ts b/packages/client/ui-sidebar-files/tests/search-preferences.client.spec.ts new file mode 100644 index 0000000000..f093c16185 --- /dev/null +++ b/packages/client/ui-sidebar-files/tests/search-preferences.client.spec.ts @@ -0,0 +1,119 @@ +// @vitest-environment jsdom +/** The workspace search preferences kept in browser storage, read back without trusting what storage holds. */ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { createSnapshotStore } from '@deepseek-ai/dsh-client-store' +import { createSearchPreferenceBook, SEARCH_PREFERENCES_KEY, workspaceKey } from '../src/client/search-preferences.ts' +import { defaultSearchPreferences } from '../src/client/search-rules.ts' + +/** A book over browser storage, as the plugin's `apply` builds it. */ +function persisted() { + return createSearchPreferenceBook(createSnapshotStore({}, { persist: { name: SEARCH_PREFERENCES_KEY } })) +} +const stored = (): unknown => JSON.parse(localStorage.getItem(SEARCH_PREFERENCES_KEY) ?? 'null') + +afterEach(() => { + localStorage.clear() + vi.restoreAllMocks() +}) + +describe('workspaceKey', () => { + it.each([ + ['/work/app', '/work/app'], + ['/work/app/', '/work/app'], + ['/work/app//', '/work/app'], + ['/', '/'], + ['C:\\work\\', 'C:\\work'], + ['C:\\', 'C:\\'], + ['C:/', 'C:/'], + ['C:', 'C:'], + ])('keys %s as %s', (cwd, key) => { + expect(workspaceKey(cwd)).toBe(key) + }) +}) + +describe('createSearchPreferenceBook', () => { + it('writes only the roots the reader changed, and drops a root changed back to the defaults', () => { + const book = persisted() + const chosen = { rules: [{ path: 'out', enabled: false }], showHidden: true, respectGitignore: false } + book.update('/work/app/', chosen) + book.update('/work/lib', { ...defaultSearchPreferences(), showHidden: true }) + expect(stored()).toEqual({ '/work/app': chosen, '/work/lib': { showHidden: true } }) + expect(book.source('/work/app').getSnapshot()).toEqual(chosen) + book.update('/work/lib', defaultSearchPreferences()) + expect(stored()).toEqual({ '/work/app': chosen }) + expect(book.source('/work/never').getSnapshot()).toEqual(defaultSearchPreferences()) + }) + + it('changes only the fields a patch names, over the root\'s preferences at the time of the change', () => { + const book = persisted() + book.update('/work/app', { showHidden: true }) + book.update('/work/app', { rules: [{ path: 'out', enabled: true }] }) + expect(book.source('/work/app').getSnapshot()).toEqual({ rules: [{ path: 'out', enabled: true }], showHidden: true, respectGitignore: true }) + book.update('/work/app', { showHidden: false }) + book.update('/work/app', { respectGitignore: false }) + expect(book.source('/work/app').getSnapshot()).toEqual({ rules: [{ path: 'out', enabled: true }], showHidden: false, respectGitignore: false }) + // A `.gitignore` choice alone differs from the defaults, so only it is stored. + book.update('/work/app', { rules: defaultSearchPreferences().rules }) + expect(stored()).toEqual({ '/work/app': { respectGitignore: false } }) + }) + + it('leaves the rules out of an entry whose rules the reader did not change, so the running build\'s default list applies', () => { + persisted().update('/work/app', { showHidden: true }) + expect(stored()).toEqual({ '/work/app': { showHidden: true } }) + // An entry another build stored without rules reads the default list of this build. + localStorage.setItem(SEARCH_PREFERENCES_KEY, JSON.stringify({ '/work/app': { showHidden: true, respectGitignore: false } })) + expect(persisted().source('/work/app').getSnapshot()).toEqual({ ...defaultSearchPreferences(), showHidden: true, respectGitignore: false }) + }) + + it('keeps each root\'s snapshot until that root changes', () => { + const book = persisted() + const app = book.source('/work/app') + const first = app.getSnapshot() + expect(app.getSnapshot()).toBe(first) + const heard = vi.fn() + const stop = app.subscribe(heard) + book.update('/work/other', { rules: [], showHidden: false }) + expect(heard).toHaveBeenCalledTimes(1) + expect(app.getSnapshot()).toBe(first) + book.update('/work/app', { rules: [], showHidden: false }) + expect(app.getSnapshot()).not.toBe(first) + stop() + }) + + it('reads back what an earlier session stored, and the default for each missing or malformed field', () => { + localStorage.setItem(SEARCH_PREFERENCES_KEY, JSON.stringify({ + '/work/app': { rules: [{ path: 'out', enabled: true }, { path: '../x', enabled: true }], showHidden: true }, + '/work/lib': { rules: [], showHidden: false, respectGitignore: false }, + '/work/full': { ...defaultSearchPreferences(), showHidden: true }, + '/work/bad': { rules: 'dist', showHidden: true }, + '/work/odd': 7, + })) + const book = persisted() + expect(book.source('/work/app').getSnapshot()).toEqual({ rules: [{ path: 'out', enabled: true }], showHidden: true, respectGitignore: true }) + expect(book.source('/work/lib').getSnapshot()).toEqual({ rules: [], showHidden: false, respectGitignore: false }) + // Saving an entry that stored every field keeps only those that differ from the defaults. + book.update('/work/full', { showHidden: true }) + expect(stored()).toMatchObject({ '/work/full': { showHidden: true } }) + expect(book.source('/work/bad').getSnapshot()).toEqual({ ...defaultSearchPreferences(), showHidden: true }) + expect(book.source('/work/odd').getSnapshot()).toEqual(defaultSearchPreferences()) + }) + + it.each([ + ['a JSON array', '[]'], + ['a JSON string', '"oops"'], + ['a JSON null', 'null'], + ])('reads %s in storage as no preferences, and replaces it on the first change', (_label, raw) => { + localStorage.setItem(SEARCH_PREFERENCES_KEY, raw) + const book = persisted() + expect(book.source('/work/app').getSnapshot()).toEqual(defaultSearchPreferences()) + book.update('/work/app', { rules: [], showHidden: false }) + expect(stored()).toEqual({ '/work/app': { rules: [] } }) + }) + + it('reads storage that is not JSON as no preferences', () => { + const report = vi.spyOn(console, 'error').mockImplementation(() => {}) + localStorage.setItem(SEARCH_PREFERENCES_KEY, '{nope') + expect(persisted().source('/work/app').getSnapshot()).toEqual(defaultSearchPreferences()) + expect(report).toHaveBeenCalledTimes(1) + }) +}) diff --git a/packages/client/ui-sidebar-files/tests/search-rules.client.spec.ts b/packages/client/ui-sidebar-files/tests/search-rules.client.spec.ts new file mode 100644 index 0000000000..19addd7423 --- /dev/null +++ b/packages/client/ui-sidebar-files/tests/search-rules.client.spec.ts @@ -0,0 +1,239 @@ +/** Exclusion rule editing, default restoration, stored preferences, and hidden entries. */ +import { describe, expect, it } from 'vitest' +import { + addRule, changedSearchPreferences, defaultRules, defaultSearchPreferences, enabledPaths, isDefaultRules, isHiddenEntry, + normalizeRulePath, readSearchPreferences, removeRule, setRuleEnabled, type ExclusionRule, +} from '../src/client/search-rules.ts' + +const DEFAULTS = [ + '.git', '.hg', '.next', '.pnpm-store', '.svn', '.turbo', '.yarn', 'build', 'coverage', 'dist', 'node_modules', + '.DS_Store', '.Spotlight-V100', '.Trashes', '.fseventsd', 'Thumbs.db', 'desktop.ini', '$RECYCLE.BIN', 'System Volume Information', +] + +describe('normalizeRulePath', () => { + it.each([ + ['node_modules', 'node_modules'], + ['$RECYCLE.BIN', '$RECYCLE.BIN'], + [' System Volume Information ', 'System Volume Information'], + ['a/$b c/.DS_Store', 'a/$b c/.DS_Store'], + [' dist ', 'dist'], + ['packages\\app\\dist', 'packages/app/dist'], + ['./a/', 'a'], + ['././a', 'a'], + ['.//a', 'a'], + ['a//b///c', 'a/b/c'], + ['a/b//', 'a/b'], + ['.hidden', '.hidden'], + ['a..b', 'a..b'], + ['./././/cache///', 'cache'], + [' .\\.\\cache\\ ', 'cache'], + ['./parent/~cache', 'parent/~cache'], + ['./parent/C:/cache', 'parent/C:/cache'], + ['./ cache', 'cache'], + ['./ ./\t./\ncache', 'cache'], + ['cache /', 'cache'], + ['cache/ /\t/\n/', 'cache'], + [' ./ \t./cache / / ', 'cache'], + ['./\u00a0cache\ufeff/\u00a0/', 'cache'], + ['./ cache dir / nested folder / ', 'cache dir / nested folder'], + ['./ cache\t name/ next folder / ', 'cache\t name/ next folder'], + ])('normalizes %j to %j and keeps the result stable', (input, path) => { + expect(normalizeRulePath(input)).toEqual({ kind: 'ok', path }) + expect(normalizeRulePath(path)).toEqual({ kind: 'ok', path }) + const preferences = { rules: [{ path, enabled: true }], showHidden: false, respectGitignore: true } + expect(readSearchPreferences(preferences)).toEqual(preferences) + }) + + it('normalizes long mixed edge chains while preserving internal whitespace', () => { + const path = 'cache dir / nested folder' + const input = `${'./ \t'.repeat(10_000)}${path}${' / \t'.repeat(10_000)}` + const normalized = normalizeRulePath(input) + expect(normalized.kind === 'ok' && normalized.path.length).toBe(path.length) + expect(normalized).toEqual({ kind: 'ok', path }) + expect(normalizeRulePath(path)).toEqual({ kind: 'ok', path }) + const preferences = { rules: [{ path, enabled: false }], showHidden: true, respectGitignore: false } + expect(readSearchPreferences(preferences)).toEqual(preferences) + }) + + it.each([ + ['', 'empty'], + [' ', 'empty'], + ['./', 'empty'], + ['.//', 'empty'], + ['./ \t./ \n', 'empty'], + ['./ . / ', 'parent'], + ['a\0b', 'nul'], + ['\0', 'nul'], + ['/abs', 'absolute'], + ['/', 'absolute'], + ['///', 'absolute'], + ['\\\\server\\share', 'absolute'], + ['C:\\x', 'absolute'], + ['c:x', 'absolute'], + ['~/x', 'absolute'], + ['~', 'absolute'], + ['./~cache', 'absolute'], + ['././~cache/', 'absolute'], + ['.//./~/cache', 'absolute'], + ['./C:/cache', 'absolute'], + ['././c:cache/', 'absolute'], + ['.\\.\\C:\\cache', 'absolute'], + ['./ ~cache', 'absolute'], + ['./ C:/cache', 'absolute'], + ['./ ./\t~cache / /', 'absolute'], + ['.\\ .\\ c:cache \\ ', 'absolute'], + ['./ /cache', 'absolute'], + [' ./ ./ / ', 'absolute'], + [' \t/ ./ cache/ ', 'absolute'], + ['./ ./ /\t/', 'absolute'], + ['..', 'parent'], + ['a/../b', 'parent'], + ['a/..', 'parent'], + ['.', 'parent'], + ['a/./b', 'parent'], + ['a/.', 'parent'], + ['*.log', 'glob'], + ['a?', 'glob'], + ['[ab]', 'glob'], + ['a]', 'glob'], + ['{a,b}', 'glob'], + ['a}', 'glob'], + ['!dist', 'glob'], + ])('refuses %j as %s', (input, reason) => { + expect(normalizeRulePath(input)).toEqual({ kind: 'invalid', reason }) + }) +}) + +describe('rule list', () => { + it('starts from every default, enabled, in default order', () => { + expect(defaultRules()).toEqual(DEFAULTS.map(path => ({ path, enabled: true }))) + expect(enabledPaths(defaultRules())).toEqual(DEFAULTS) + expect(defaultSearchPreferences()).toEqual({ rules: defaultRules(), showHidden: false, respectGitignore: true }) + expect(defaultRules()).not.toBe(defaultRules()) + // Every default is a path the reader could type, unchanged by normalization. + expect(DEFAULTS.map(path => normalizeRulePath(path))).toEqual(DEFAULTS.map(path => ({ kind: 'ok', path }))) + }) + + it('adds a normalized rule at the top, enabled', () => { + const rules: ExclusionRule[] = [{ path: 'dist', enabled: true }] + expect(addRule(rules, ' ./out/ ')).toEqual({ kind: 'ok', rules: [{ path: 'out', enabled: true }, { path: 'dist', enabled: true }] }) + expect(rules).toEqual([{ path: 'dist', enabled: true }]) + }) + + it('refuses an invalid rule without changing the list', () => { + expect(addRule([], 'a/../b')).toEqual({ kind: 'invalid', reason: 'parent' }) + }) + + it('enables and moves an existing rule to the top instead of listing it twice', () => { + const rules: ExclusionRule[] = [{ path: 'a', enabled: true }, { path: 'b', enabled: true }, { path: 'dist', enabled: false }] + expect(addRule(rules, 'dist/')).toEqual({ + kind: 'ok', rules: [{ path: 'dist', enabled: true }, { path: 'a', enabled: true }, { path: 'b', enabled: true }], + }) + expect(addRule(rules, 'b')).toEqual({ + kind: 'ok', rules: [{ path: 'b', enabled: true }, { path: 'a', enabled: true }, { path: 'dist', enabled: false }], + }) + }) + + it('removes one rule and leaves others', () => { + const rules: ExclusionRule[] = [{ path: 'a', enabled: true }, { path: 'b', enabled: false }] + expect(removeRule(rules, 'b')).toEqual([{ path: 'a', enabled: true }]) + expect(removeRule(rules, 'c')).toEqual(rules) + }) + + it('switches one rule on or off in place', () => { + const rules: ExclusionRule[] = [{ path: 'a', enabled: true }, { path: 'b', enabled: true }] + const off = setRuleEnabled(rules, 'a', false) + expect(off).toEqual([{ path: 'a', enabled: false }, { path: 'b', enabled: true }]) + expect(off[1]).toBe(rules[1]) + expect(setRuleEnabled(off, 'a', true)).toEqual(rules) + expect(setRuleEnabled(rules, 'a', true)[0]).toBe(rules[0]) + expect(setRuleEnabled(rules, 'c', false)).toEqual(rules) + expect(enabledPaths(off)).toEqual(['b']) + }) + + it('treats only the default list, in default order and fully enabled, as the defaults', () => { + expect(isDefaultRules(defaultRules())).toBe(true) + expect(isDefaultRules(removeRule(defaultRules(), 'dist'))).toBe(false) + expect(isDefaultRules(setRuleEnabled(defaultRules(), '.git', false))).toBe(false) + const added = addRule(defaultRules(), 'out') + expect(added.kind === 'ok' && isDefaultRules(added.rules)).toBe(false) + // Adding a listed default again moves it to the top, which differs from the default order. + const moved = addRule(defaultRules(), 'dist') + expect(moved.kind === 'ok' && isDefaultRules(moved.rules)).toBe(false) + expect(isDefaultRules([])).toBe(false) + }) +}) + +describe('readSearchPreferences', () => { + it('reads stored preferences back', () => { + const preferences = { rules: [{ path: 'out', enabled: false }, { path: 'dist', enabled: true }], showHidden: true, respectGitignore: false } + expect(readSearchPreferences(JSON.parse(JSON.stringify(preferences)))).toEqual(preferences) + }) + + it('keeps a stored empty list', () => { + const preferences = { rules: [], showHidden: false, respectGitignore: true } + expect(readSearchPreferences(preferences)).toEqual(preferences) + }) + + it.each([ + ['no rules', { showHidden: true, respectGitignore: false }, { showHidden: true, respectGitignore: false }], + ['rules that are not an array', { rules: {}, showHidden: true }, { showHidden: true }], + ['no showHidden', { rules: [] }, { rules: [] }], + ['a showHidden that is not a boolean', { rules: [], showHidden: 'yes' }, { rules: [] }], + ['no .gitignore choice', { rules: [{ path: 'out', enabled: false }], showHidden: true }, { rules: [{ path: 'out', enabled: false }], showHidden: true }], + ['a .gitignore choice that is a string', { respectGitignore: 'no' }, {}], + ['a .gitignore choice that is a JSON null', { respectGitignore: null }, {}], + ])('gives %s its default alone, keeping the other fields', (_label, stored, kept) => { + expect(readSearchPreferences(stored)).toEqual({ ...defaultSearchPreferences(), ...kept }) + }) + + it.each([ + ['nothing stored', undefined], + ['a JSON null', null], + ['a JSON string', 'dist'], + ['an array', []], + ])('falls back to the defaults for %s', (_label, stored) => { + expect(readSearchPreferences(stored)).toEqual(defaultSearchPreferences()) + }) + + it('drops malformed, invalid, and repeated rules and normalizes the rest', () => { + const stored = { + rules: [ + null, 'dist', { path: 'a' }, { enabled: true }, { path: 1, enabled: true }, { path: 'b', enabled: 'yes' }, + { path: '../x', enabled: true }, { path: '*.log', enabled: true }, { path: '/abs', enabled: true }, { path: 'a\0b', enabled: true }, + { path: './out/', enabled: false }, { path: 'out', enabled: true }, { path: 'a\\b', enabled: true }, + ], + showHidden: false, + } + expect(readSearchPreferences(stored)).toEqual({ + rules: [{ path: 'out', enabled: false }, { path: 'a/b', enabled: true }], showHidden: false, respectGitignore: true, + }) + }) +}) + +describe('changedSearchPreferences', () => { + it('keeps only the fields that differ from the defaults, rule order and enabled state included', () => { + const defaults = defaultSearchPreferences() + expect(changedSearchPreferences(defaults)).toBeUndefined() + expect(changedSearchPreferences({ ...defaults, showHidden: true })).toEqual({ showHidden: true }) + expect(changedSearchPreferences({ ...defaults, respectGitignore: false })).toEqual({ respectGitignore: false }) + expect(changedSearchPreferences({ ...defaults, rules: defaults.rules.slice(1) })).toEqual({ rules: defaults.rules.slice(1) }) + const reversed = [...defaults.rules].reverse() + expect(changedSearchPreferences({ ...defaults, rules: reversed })).toEqual({ rules: reversed }) + const disabled = setRuleEnabled(defaults.rules, 'dist', false) + expect(changedSearchPreferences({ rules: disabled, showHidden: true, respectGitignore: false })) + .toEqual({ rules: disabled, showHidden: true, respectGitignore: false }) + }) +}) + +describe('isHiddenEntry', () => { + it.each([ + [{ name: '.env' }, true], + [{ name: '.git' }, true], + [{ name: 'desktop.ini' }, false], + [{ name: 'src' }, false], + [{ name: 'a.b' }, false], + ])('%j is hidden: %s', (entry, hidden) => { + expect(isHiddenEntry(entry)).toBe(hidden) + }) +}) diff --git a/packages/client/ui-sidebar-files/tests/search-tree.client.spec.ts b/packages/client/ui-sidebar-files/tests/search-tree.client.spec.ts new file mode 100644 index 0000000000..dfac89ae63 --- /dev/null +++ b/packages/client/ui-sidebar-files/tests/search-tree.client.spec.ts @@ -0,0 +1,86 @@ +/** Search results synthesized into a fully expanded tree, their count, and in-name highlights. */ +import { describe, expect, it } from 'vitest' +import { buildSearchTree, matchRanges } from '../src/client/search-tree.ts' +import type { SearchMatch } from '../src/client/search-tree.ts' + +const ROOT = '/work/app' + +describe('buildSearchTree', () => { + it('synthesizes every ancestor level and expands all of them', () => { + const tree = buildSearchTree(ROOT, [ + { path: 'src/search/index.ts', type: 'file' }, + { path: 'src/search.ts', type: 'file' }, + { path: 'notes/search.txt', type: 'file' }, + ]) + expect(tree).toEqual({ + autoRefresh: false, + root: ROOT, + levels: { + [ROOT]: { kind: 'ready', level: { entries: [{ name: 'src', type: 'directory' }, { name: 'notes', type: 'directory' }], truncated: false } }, + '/work/app/src': { kind: 'ready', level: { entries: [{ name: 'search', type: 'directory' }, { name: 'search.ts', type: 'file' }], truncated: false } }, + '/work/app/src/search': { kind: 'ready', level: { entries: [{ name: 'index.ts', type: 'file' }], truncated: false } }, + '/work/app/notes': { kind: 'ready', level: { entries: [{ name: 'search.txt', type: 'file' }], truncated: false } }, + }, + expanded: [ROOT, '/work/app/src', '/work/app/src/search', '/work/app/notes'], + }) + }) + + it('gives a matched directory without matched descendants an empty level', () => { + const tree = buildSearchTree(ROOT, [{ path: 'lib/search', type: 'directory' }, { path: 'odd', type: 'other' }]) + expect(tree.levels['/work/app/lib/search']).toEqual({ kind: 'ready', level: { entries: [], truncated: false } }) + expect(tree.levels[ROOT]).toEqual({ kind: 'ready', level: { entries: [{ name: 'lib', type: 'directory' }, { name: 'odd', type: 'other' }], truncated: false } }) + expect(tree.expanded).toEqual([ROOT, '/work/app/lib', '/work/app/lib/search']) + }) + + it('lists each child once and keeps a matched directory that also has matched descendants', () => { + const entries: SearchMatch[] = [ + { path: 'search', type: 'directory' }, + { path: 'search/a.ts', type: 'file' }, + { path: 'search/a.ts', type: 'file' }, + ] + const tree = buildSearchTree(ROOT, entries) + expect(tree.levels[ROOT]).toEqual({ kind: 'ready', level: { entries: [{ name: 'search', type: 'directory' }], truncated: false } }) + expect(tree.levels['/work/app/search']).toEqual({ kind: 'ready', level: { entries: [{ name: 'a.ts', type: 'file' }], truncated: false } }) + }) + + it('treats a name as a directory once any result passes through it, keeping its first position', () => { + const tree = buildSearchTree(ROOT, [{ path: 'x', type: 'file' }, { path: 'y', type: 'file' }, { path: 'x/z', type: 'file' }]) + expect(tree.levels[ROOT]).toEqual({ kind: 'ready', level: { entries: [{ name: 'x', type: 'directory' }, { name: 'y', type: 'file' }], truncated: false } }) + expect(buildSearchTree(ROOT, [{ path: 'x/z', type: 'file' }, { path: 'x', type: 'file' }]).levels[ROOT]) + .toEqual({ kind: 'ready', level: { entries: [{ name: 'x', type: 'directory' }], truncated: false } }) + }) + + it('joins paths the way the listed tree does, whatever the root spelling', () => { + const windows = buildSearchTree('C:\\work\\', [{ path: 'a/b.txt', type: 'file' }]) + expect(windows.expanded).toEqual(['C:\\work\\', 'C:\\work/a']) + expect(buildSearchTree('/', [{ path: 'etc/x', type: 'file' }]).expanded).toEqual(['/', '/etc']) + }) + + it('draws no results as an expanded empty root', () => { + expect(buildSearchTree(ROOT, [])).toEqual({ + autoRefresh: false, root: ROOT, levels: { [ROOT]: { kind: 'ready', level: { entries: [], truncated: false } } }, expanded: [ROOT], + }) + }) +}) + +describe('matchRanges', () => { + it('finds every non-overlapping occurrence regardless of case', () => { + expect(matchRanges('Search-search.SEARCH', 'search')).toEqual([[0, 6], [7, 13], [14, 20]]) + expect(matchRanges('aaaa', 'aa')).toEqual([[0, 2], [2, 4]]) + }) + + it('treats the query literally', () => { + expect(matchRanges('a.b(c)*', '.b(c)*')).toEqual([[1, 7]]) + expect(matchRanges('axb', '.')).toEqual([]) + }) + + it('has no ranges for an empty query or a miss', () => { + expect(matchRanges('name', '')).toEqual([]) + expect(matchRanges('name', 'src/name')).toEqual([]) + }) + + it('indexes the name as given around non-ASCII text', () => { + expect(matchRanges('Größe.TXT', 'größe')).toEqual([[0, 5]]) + expect(matchRanges('报告-搜索.md', '搜索')).toEqual([[3, 5]]) + }) +}) diff --git a/packages/client/ui-sidebar-files/tests/search.client.spec.tsx b/packages/client/ui-sidebar-files/tests/search.client.spec.tsx new file mode 100644 index 0000000000..3fb08ea2cf --- /dev/null +++ b/packages/client/ui-sidebar-files/tests/search.client.spec.tsx @@ -0,0 +1,666 @@ +// @vitest-environment jsdom +/** + * The navigator's search: the bar above the tree, the results drawn in the + * tree's place, the settings panel, the hidden-entry control, the keyboard, + * and the Escape order. A scripted search stands in for the Host's: it + * answers every query with fixed matches, or leaves a query open. + */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { act, cleanup, fireEvent, within } from '@testing-library/react' +import { sessionFileAddress } from '@deepseek-ai/dsh-util-workspace-path' +import { zh } from '../src/client/locales.ts' +import { defaultRules, enabledPaths } from '../src/client/search-rules.ts' +import type { WorkspaceSearchPreferences } from '../src/client/search-rules.ts' +import type { DirLevel } from '../src/client/store.ts' +import { makeTranslate } from '@deepseek-ai/dsh-client-test-runtime' +import { mountNavigator, PANE, ROOT, SESSION } from './mount.client.tsx' +import type { Mounted } from './mount.client.tsx' +import { answer, memoryBook, searchesSettled } from './search-fixture.client.ts' +import { RemoteError } from '@deepseek-ai/dsh-client-test-runtime' + +const ROOT_LEVEL: DirLevel = { + entries: [ + { name: 'src', type: 'directory' }, { name: 'node_modules', type: 'directory' }, + { name: '.env', type: 'file', size: 1 }, { name: 'README.md', type: 'file', size: 1 }, + ], + truncated: false, +} +const SRC_LEVEL: DirLevel = { entries: [{ name: 'a.ts', type: 'file', size: 1 }, { name: 'b.ts', type: 'file', size: 1 }], truncated: false } +const A = sessionFileAddress(SESSION, 'src/a.ts') +const t = makeTranslate(zh) + +beforeEach(() => { + Element.prototype.scrollIntoView = vi.fn() +}) +afterEach(() => { + cleanup() + Reflect.deleteProperty(Element.prototype, 'scrollIntoView') +}) + +/** Show the navigator on a page-less tab and answer the root listing. */ +async function shown(h: Mounted): Promise { + act(() => { h.instance.actions.shown(PANE, true) }) + await act(() => h.script.watches.ready(ROOT)) + await act(() => h.script.settle({ ok: true, value: ROOT_LEVEL })) +} + +/** Show the navigator beside `src/a.ts` and answer the root and `src` listings. */ +async function revealed(h: Mounted): Promise { + await shown(h) + await act(() => h.script.watches.ready(`${ROOT}/src`)) + await act(() => h.script.settle({ ok: true, value: SRC_LEVEL })) +} + +const field = (h: Mounted): HTMLInputElement => within(h.view.container).getByRole('searchbox', { name: zh['search.placeholder'] }) +const body = (h: Mounted): HTMLElement => h.view.container.querySelector('[data-files-body]')! + +/** + * The tooltip text of the empty result's information button, read by pinning + * the tooltip with a click and closing it again; `undefined` without the button. + */ +const emptyHint = (h: Mounted): string | undefined => { + const button = body(h).querySelector('[data-files-search-hint]') + if (button === null) return undefined + expect(button.getAttribute('aria-label')).toBe(zh['search.emptyHint.label']) + fireEvent.click(button) + const text = document.querySelector('[role="tooltip"]')?.textContent + fireEvent.click(button) + expect(document.querySelector('[role="tooltip"]')).toBeNull() + return text +} +const status = (h: Mounted): string | null | undefined => h.view.container.querySelector('[data-files-search-status]')?.textContent +const paths = (h: Mounted): string[] => [...body(h).querySelectorAll('[data-files-entry]')].map(li => li.getAttribute('data-files-path')!) +const dialog = (h: Mounted): HTMLElement | null => h.view.queryByRole('dialog') +const settingsButton = (h: Mounted): HTMLElement => h.view.getByRole('button', { name: zh['search.settings'] }) + +/** jsdom omits mousedown's focus default; honor its cancellation before clicking. */ +function pointerClick(target: HTMLElement): void { + act(() => { + fireEvent.pointerDown(target, { button: 0, pointerType: 'mouse' }) + if (fireEvent.mouseDown(target, { button: 0 })) target.focus() + fireEvent.pointerUp(target, { button: 0, pointerType: 'mouse' }) + fireEvent.mouseUp(target, { button: 0 }) + fireEvent.click(target, { button: 0 }) + }) +} + +/** Supply the focused button's native keyboard activation that jsdom omits. */ +function keyboardClick(target: HTMLElement, key: 'Enter' | ' '): void { + act(() => { + target.focus() + const allowed = fireEvent.keyDown(target, { key }) + if (allowed && key === 'Enter') target.click() + const released = fireEvent.keyUp(target, { key }) + if (allowed && released && key === ' ') target.click() + }) +} + +/** + * Type into the search field and let the search's answer draw. + * @param h - the mounted navigator. + * @param text - the field's new text. + * @param matches - what the search answers, as `answer` takes them; the previous answer when omitted. + */ +async function type(h: Mounted, text: string, matches?: readonly string[]): Promise { + if (matches !== undefined) h.searches.answerWith(answer(matches)) + await act(async () => { + fireEvent.change(field(h), { target: { value: text } }) + await searchesSettled() + }) +} +/** Run a store or face write and let a search it starts draw its answer. */ +async function settled(write: () => void): Promise { + await act(async () => { + write() + await searchesSettled() + }) +} +/** The requests the search received, oldest first. */ +const requests = (h: Mounted) => h.search.mock.calls.map(([, request]) => request) + +describe('the search bar', () => { + it('searches as the reader types and draws the matches as a highlighted tree with its count', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + expect(h.view.queryByRole('button', { name: zh['search.clear'] })).toBeNull() + await type(h, ' Ab ', ['src/tab.ts', 'lib/AB/', 'lib/AB/x.ts']) + expect(requests(h)).toEqual([{ query: 'Ab', excludedPaths: enabledPaths(defaultRules()), includeHidden: false, respectGitignore: true }]) + expect(status(h)).toBe(t('search.results', { count: '3' })) + expect(paths(h)).toEqual([`${ROOT}/lib`, `${ROOT}/lib/AB`, `${ROOT}/lib/AB/x.ts`, `${ROOT}/src`, `${ROOT}/src/tab.ts`]) + expect([...body(h).querySelectorAll('mark')].map(mark => mark.textContent)).toEqual(['AB', 'ab']) + expect(body(h).querySelector(`[data-files-path="${ROOT}/src/tab.ts"] button`)?.textContent).toBe('tab.ts') + expect(h.view.getByRole('button', { name: zh['search.clear'] })).toBeTruthy() + // Whitespace around the same text asks nothing new and keeps the count. + await type(h, 'Ab') + expect(requests(h)).toHaveLength(1) + expect(status(h)).toBe(t('search.results', { count: '3' })) + }) + + it('collapses result directories on their own, and opens a result like a tree row with the tab\'s file marked', async () => { + const h = mountNavigator(A) + await revealed(h) + await type(h, 'a', ['src/a.ts', 'empty/']) + // A matched directory without matched descendants is not called empty. + expect(body(h).querySelector('[data-files-row="empty"]')).toBeNull() + expect(body(h).querySelector('[aria-current="true"]')?.closest('[data-files-path]')?.getAttribute('data-files-path')).toBe(`${ROOT}/src/a.ts`) + const src = (): HTMLElement => body(h).querySelector(`[data-files-path="${ROOT}/src"] > div > button`)! + act(() => { fireEvent.click(src()) }) + expect(src().getAttribute('aria-expanded')).toBe('false') + expect(h.instance.getSnapshot().byPane[PANE]?.tree?.expanded).toEqual([ROOT, `${ROOT}/src`]) + act(() => { fireEvent.click(src()) }) + fireEvent.click(body(h).querySelector(`[data-files-path="${ROOT}/src/a.ts"] > div > button`)!) + expect(h.onOpen).toHaveBeenLastCalledWith(A) + }) + + it('draws the tree again as it was — expansion, scroll, and the current file — once the query is cleared', async () => { + const h = mountNavigator(A) + await revealed(h) + fireEvent.scroll(body(h), { target: { scrollTop: 64 } }) + await type(h, 'b', ['src/b.ts']) + expect(body(h).getAttribute('data-files-view')).toBe('search') + expect(body(h).scrollTop).toBe(0) + fireEvent.scroll(body(h), { target: { scrollTop: 12 } }) + act(() => { fireEvent.click(h.view.getByRole('button', { name: zh['search.clear'] })) }) + expect(document.activeElement).toBe(field(h)) + expect(status(h)).toBe('') + expect(body(h).getAttribute('data-files-view')).toBe('tree') + expect(body(h).scrollTop).toBe(64) + expect(h.instance.getSnapshot().byPane[PANE]?.tree?.expanded).toEqual([ROOT, `${ROOT}/src`]) + expect(body(h).querySelector('[aria-current="true"]')?.closest('[data-files-path]')?.getAttribute('data-files-path')).toBe(`${ROOT}/src/a.ts`) + await type(h, 'b') + expect(body(h).scrollTop).toBe(0) + // A remount during a search restores the results' own offset. + fireEvent.scroll(body(h), { target: { scrollTop: 30 } }) + h.view.unmount() + const back = h.remount() + expect(back.container.querySelector('[data-files-body]')?.scrollTop).toBe(30) + // New results start at their top. + await settled(() => { h.face.setQuery(PANE, ROOT, 'bb') }) + expect(back.container.querySelector('[data-files-body]')?.scrollTop).toBe(0) + }) + + it('says when nothing matched, with an information button whose tooltip names which of hidden entries, excluded paths, and ignored ones the search left out', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + await type(h, 'zzz') + expect(body(h).textContent).toBe(zh['search.empty']) + expect(emptyHint(h)).toBe(zh['search.emptyHint.all']) + expect(status(h)).toBe(t('search.results', { count: '0' })) + const settle = async (preferences: Partial): Promise => { + await settled(() => { h.face.setSearchPreferences(ROOT, preferences) }) + expect(body(h).textContent).toBe(zh['search.empty']) + return emptyHint(h) + } + expect(await settle({ showHidden: true })).toBe(zh['search.emptyHint.excludedIgnored']) + expect(requests(h).at(-1)).toMatchObject({ excludedPaths: enabledPaths(defaultRules()), includeHidden: true }) + expect(await settle({ rules: [] })).toBe(zh['search.emptyHint.ignored']) + expect(requests(h).at(-1)).toMatchObject({ excludedPaths: [], includeHidden: true }) + expect(await settle({ showHidden: false })).toBe(zh['search.emptyHint.hiddenIgnored']) + // Turning ignore files off leaves out only what the other two settings take. + expect(await settle({ respectGitignore: false })).toBe(zh['search.emptyHint.hidden']) + expect(requests(h).at(-1)).toMatchObject({ respectGitignore: false }) + // With nothing excluded, hidden entries shown, and ignore files not respected, the search left nothing out. + expect(await settle({ showHidden: true })).toBeUndefined() + // A rule that is listed but off excludes nothing. + expect(await settle({ rules: [{ path: 'dist', enabled: false }] })).toBeUndefined() + // Leaving out hidden entries is the only thing that remains. + expect(await settle({ showHidden: false })).toBe(zh['search.emptyHint.hidden']) + expect(await settle({ rules: defaultRules(), showHidden: true })).toBe(zh['search.emptyHint.excluded']) + expect(await settle({ showHidden: false })).toBe(zh['search.emptyHint.both']) + }) + + it('says below the results, matched or not, when only the first matches are drawn and when some directories were not searched', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + const note = (row: string): string | null | undefined => body(h).querySelector(`[data-files-row="${row}"]`)?.textContent + h.searches.answerWith(answer(['src/a.ts', 'src/ab.ts'], { truncated: true })) + await type(h, 'a') + expect(note('search-truncated')).toBe(t('search.truncated', { count: '2' })) + expect(note('search-incomplete')).toBeUndefined() + expect(note('search-empty')).toBeUndefined() + h.searches.answerWith(answer(['src/a.ts'], { incomplete: true })) + await type(h, 'ab') + expect(note('search-truncated')).toBeUndefined() + expect(note('search-incomplete')).toBe(zh['search.incomplete']) + h.searches.answerWith(answer([], { truncated: true, incomplete: true })) + await type(h, 'zzz') + expect(body(h).textContent).toBe(`${zh['search.empty']}${t('search.truncated', { count: '0' })}${zh['search.incomplete']}`) + expect(emptyHint(h)).toBe(zh['search.emptyHint.all']) + }) + + it('keeps the last results drawn with a spinner while the next search runs, and draws only a spinner before the first answer', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + const spinner = (): Element | null => h.view.container.querySelector('[data-files-search-status] [data-state="ongoing"]') + expect(spinner()).toBeNull() + h.searches.answerWith(undefined) + await type(h, 'a') + // Before the first answer the results area is empty: no tree, no note. + expect(body(h).getAttribute('data-files-view')).toBe('search') + expect(body(h).getAttribute('aria-busy')).toBe('true') + expect(body(h).textContent).toBe('') + expect(status(h)).toBe('') + expect(spinner()).not.toBeNull() + await settled(() => { h.searches.started[0]!.settle(answer(['src/a.ts'])) }) + expect(spinner()).toBeNull() + expect(body(h).getAttribute('aria-busy')).toBe('false') + expect(paths(h)).toEqual([`${ROOT}/src`, `${ROOT}/src/a.ts`]) + // A newer search keeps those results and the count until its answer replaces them. + await type(h, 'b') + expect(spinner()).not.toBeNull() + expect(paths(h)).toEqual([`${ROOT}/src`, `${ROOT}/src/a.ts`]) + expect(status(h)).toBe(t('search.results', { count: '1' })) + await settled(() => { h.searches.started[1]!.settle(answer([])) }) + expect(spinner()).toBeNull() + expect(status(h)).toBe(t('search.results', { count: '0' })) + }) + + it('says why a search failed in the results\' place, and searches again from its retry control', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + h.searches.answerWith({ ok: false, error: new RemoteError('workspace-file/not-found', 'no entry at "/work/app"', { path: '/work/app' }) }) + await type(h, 'a') + const failed = (): HTMLElement | null => body(h).querySelector('[data-files-row="search-failed"]') + // A classified failure reads as the tree's own line, not the Host's untranslated message. + expect(within(failed()!).getByRole('alert').textContent).toBe(t('search.failed', { message: t('error.notFound') })) + expect(status(h)).toBe('') + expect(paths(h)).toEqual([]) + h.searches.answerWith(answer(['src/a.ts'])) + const retry = within(failed()!).getByRole('button', { name: zh['search.retry'] }) + await settled(() => { fireEvent.click(retry) }) + expect(requests(h)).toHaveLength(2) + expect(requests(h)[1]).toEqual(requests(h)[0]) + expect(failed()).toBeNull() + expect(paths(h)).toEqual([`${ROOT}/src`, `${ROOT}/src/a.ts`]) + }) + + it('keeps the status line mounted as a live region while the query is blank, so a screen reader hears the first search', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + const live = h.view.container.querySelector('[data-files-search-status]') + expect(live?.getAttribute('aria-live')).toBe('polite') + expect(live?.textContent).toBe('') + await type(h, 'a', ['a.ts']) + expect(h.view.container.querySelector('[data-files-search-status]')).toBe(live) + expect(live?.textContent).toBe(t('search.results', { count: '1' })) + await type(h, '') + expect(h.view.container.querySelector('[data-files-search-status]')).toBe(live) + expect(live?.textContent).toBe('') + }) +}) + +describe('the hidden-entry control', () => { + it('leaves hidden entries out of the tree and the search until pressed, and shows them in both while pressed', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + const eye = (): HTMLElement => h.view.container.querySelector('[data-files-show-hidden]')! + expect(eye().getAttribute('aria-pressed')).toBe('false') + expect(eye().getAttribute('aria-label')).toBe(zh['search.showHidden']) + // The name stays fixed; only `aria-pressed` reports the state. + expect(paths(h)).not.toContain(`${ROOT}/.env`) + act(() => { fireEvent.click(eye()) }) + expect(eye().getAttribute('aria-pressed')).toBe('true') + expect(eye().getAttribute('aria-label')).toBe(zh['search.showHidden']) + expect(paths(h)).toContain(`${ROOT}/.env`) + await type(h, 'env', ['.env']) + expect(requests(h)[0]?.includeHidden).toBe(true) + expect(paths(h)).toEqual([`${ROOT}/.env`]) + // Changing it during a search searches again at once. + act(() => { fireEvent.click(eye()) }) + expect(requests(h)[1]?.includeHidden).toBe(false) + }) + + it('says a directory holding only hidden entries holds hidden entries, not that it is empty', async () => { + const h = mountNavigator('sidebar://files') + act(() => { h.instance.actions.shown(PANE, true) }) + await act(() => h.script.watches.ready(ROOT)) + await act(() => h.script.settle({ ok: true, value: { entries: [{ name: '.env', type: 'file', size: 1 }], truncated: false } })) + expect(body(h).querySelector('[data-files-row="empty"]')).toBeNull() + expect(body(h).querySelector('[data-files-row="hidden-only"]')?.textContent).toBe(zh.emptyHidden) + act(() => { fireEvent.click(h.view.container.querySelector('[data-files-show-hidden]')!) }) + expect(body(h).querySelector('[data-files-row="hidden-only"]')).toBeNull() + expect(paths(h)).toEqual([`${ROOT}/.env`]) + }) + + it('applies exclusions only to the search, so the tree still lists excluded directories', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + expect(paths(h)).toContain(`${ROOT}/node_modules`) + await type(h, 'x') + expect(requests(h)[0]?.excludedPaths).toContain('node_modules') + await type(h, '') + expect(paths(h)).toContain(`${ROOT}/node_modules`) + }) + + it('shares one workspace\'s choice across Sessions rooted there, and not with another workspace', async () => { + const book = memoryBook() + const one = mountNavigator('sidebar://files', {}, ROOT, book) + const two = mountNavigator('sidebar://files', {}, `${ROOT}/`, book) + const other = mountNavigator('sidebar://files', {}, '/work/other', book) + act(() => { + for (const h of [one, two, other]) h.instance.actions.shown(PANE, true) + }) + const eye = (h: Mounted): string | null => h.view.container.querySelector('[data-files-show-hidden]')!.getAttribute('aria-pressed') + act(() => { fireEvent.click(one.view.container.querySelector('[data-files-show-hidden]')!) }) + expect([eye(one), eye(two), eye(other)]).toEqual(['true', 'true', 'false']) + }) +}) + +describe('the settings panel', () => { + /** Open the panel from the magnifier. */ + function open(h: Mounted): HTMLElement { + pointerClick(settingsButton(h)) + expect(settingsButton(h).getAttribute('aria-expanded')).toBe('true') + return dialog(h)! + } + const stored = (h: Mounted) => h.book.source(ROOT).getSnapshot() + + it.each(['pointer', 'Enter', ' '] as const)('saves checkbox edits when the magnifier is activated with %j and searches with the saved rules', async (gesture) => { + const h = mountNavigator('sidebar://files') + await shown(h) + await type(h, 'a') + const toggle = (): void => { + if (gesture === 'pointer') pointerClick(settingsButton(h)) + else keyboardClick(settingsButton(h), gesture) + } + toggle() + const checkbox = within(dialog(h)!).getByRole('checkbox', { name: t('settings.toggle', { path: 'dist' }) }) + pointerClick(checkbox) + expect(document.activeElement).toBe(checkbox) + expect(checkbox.checked).toBe(false) + expect(stored(h).rules.find(rule => rule.path === 'dist')?.enabled).toBe(true) + expect(requests(h)).toHaveLength(1) + + toggle() + expect(dialog(h)).toBeNull() + expect(settingsButton(h).getAttribute('aria-expanded')).toBe('false') + expect(stored(h).rules.find(rule => rule.path === 'dist')?.enabled).toBe(false) + expect(document.activeElement).toBe(settingsButton(h)) + expect(requests(h)).toEqual([requests(h)[0], { + query: 'a', excludedPaths: enabledPaths(defaultRules()).filter(path => path !== 'dist'), includeHidden: false, respectGitignore: true, + }]) + + toggle() + expect(within(dialog(h)!).getByRole('checkbox', { name: t('settings.toggle', { path: 'dist' }) })).toHaveProperty('checked', false) + }) + + it('keeps added and removed paths when the magnifier closes a panel without a query', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + pointerClick(settingsButton(h)) + const panel = dialog(h)! + const add = within(panel).getByRole('textbox', { name: zh['settings.add'] }) + act(() => { + fireEvent.change(add, { target: { value: './cache/' } }) + fireEvent.keyDown(add, { key: 'Enter' }) + }) + pointerClick(within(panel).getByRole('button', { name: t('settings.remove', { path: 'build' }) })) + expect(document.activeElement).toBe(add) + expect(stored(h).rules.map(rule => rule.path)).not.toContain('cache') + expect(stored(h).rules.map(rule => rule.path)).toContain('build') + + pointerClick(settingsButton(h)) + expect(dialog(h)).toBeNull() + expect(stored(h).rules[0]).toEqual({ path: 'cache', enabled: true }) + expect(stored(h).rules.map(rule => rule.path)).not.toContain('build') + pointerClick(settingsButton(h)) + expect(within(dialog(h)!).getByRole('checkbox', { name: t('settings.toggle', { path: 'cache' }) })).toHaveProperty('checked', true) + expect(within(dialog(h)!).queryByRole('checkbox', { name: t('settings.toggle', { path: 'build' }) })).toBeNull() + expect(h.search).not.toHaveBeenCalled() + }) + + it('opens from the magnifier with the add field focused, and saves on its close control, on Escape, and when focus leaves it', async () => { + const h = mountNavigator('sidebar://files') + h.row.width = 480 + await shown(h) + // Pressing the magnifier on a closed panel takes the keyboard as any button does. + expect(fireEvent.mouseDown(settingsButton(h))).toBe(true) + let panel = open(h) + expect(within(panel).getByText(zh['settings.title'])).toBeTruthy() + expect(document.activeElement).toBe(within(panel).getByRole('textbox', { name: zh['settings.add'] })) + expect(h.view.container.querySelectorAll('[data-files-show-hidden]')).toHaveLength(1) + pointerClick(within(panel).getByRole('checkbox', { name: t('settings.toggle', { path: 'dist' }) })) + // Nothing is written while the panel is open. + expect(stored(h).rules.find(rule => rule.path === 'dist')?.enabled).toBe(true) + pointerClick(within(panel).getByRole('button', { name: zh['settings.close'] })) + expect(dialog(h)).toBeNull() + expect(document.activeElement).toBe(settingsButton(h)) + expect(stored(h).rules.find(rule => rule.path === 'dist')?.enabled).toBe(false) + + panel = open(h) + pointerClick(within(panel).getByRole('button', { name: t('settings.remove', { path: 'build' }) })) + expect(dialog(h)).not.toBeNull() + act(() => { fireEvent.keyDown(within(panel).getByRole('textbox'), { key: 'Escape' }) }) + expect(dialog(h)).toBeNull() + // The overlaying navigator stays: the panel kept the Escape. + expect(h.view.container.querySelector('[data-files-navigator]')).not.toBeNull() + expect(stored(h).rules.map(rule => rule.path)).not.toContain('build') + + panel = open(h) + pointerClick(within(panel).getByRole('checkbox', { name: t('settings.toggle', { path: 'coverage' }) })) + act(() => { body(h).focus() }) + expect(dialog(h)).toBeNull() + expect(stored(h).rules.find(rule => rule.path === 'coverage')?.enabled).toBe(false) + + // Closing without an edit writes nothing; the magnifier closes it too. + const before = stored(h) + open(h) + pointerClick(settingsButton(h)) + expect(dialog(h)).toBeNull() + expect(stored(h)).toBe(before) + }) + + it('shows the restore control whenever the list differs from the defaults, and replaces the list with the defaults', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + const panel = open(h) + const restore = (): HTMLElement | null => within(panel).queryByRole('button', { name: zh['settings.restore'] }) + const add = within(panel).getByRole('textbox', { name: zh['settings.add'] }) + const rules = (): string[] => [...panel.querySelectorAll('[data-files-rule]')].map(row => row.getAttribute('data-files-rule')!) + const defaults = rules() + const type = (value: string): void => { + act(() => { fireEvent.change(add, { target: { value } }) }) + act(() => { fireEvent.keyDown(add, { key: 'Enter' }) }) + } + expect(restore()).toBeNull() + // A custom path alone differs from the defaults. + type('out') + expect(restore()).not.toBeNull() + act(() => { fireEvent.click(within(panel).getByRole('button', { name: t('settings.remove', { path: 'out' }) })) }) + expect(document.activeElement).toBe(add) + expect(restore()).toBeNull() + // A default added again moves to the top, so the order differs. + type('dist') + expect(rules()[0]).toBe('dist') + expect(restore()).not.toBeNull() + act(() => { fireEvent.click(within(panel).getByRole('button', { name: t('settings.remove', { path: 'node_modules' }) })) }) + act(() => { fireEvent.click(within(panel).getByRole('checkbox', { name: t('settings.toggle', { path: '.git' }) })) }) + type('out') + act(() => { fireEvent.click(restore()!) }) + expect(document.activeElement).toBe(add) + expect(restore()).toBeNull() + expect(rules()).toEqual(defaults) + expect(within(panel).getByRole('checkbox', { name: t('settings.toggle', { path: '.git' }) })).toHaveProperty('checked', true) + }) + + it('refuses an invalid path where it was typed, and adds a valid one at the top', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + const panel = open(h) + const add = within(panel).getByRole('textbox', { name: zh['settings.add'] }) + act(() => { fireEvent.change(add, { target: { value: '../x' } }) }) + act(() => { fireEvent.keyDown(add, { key: 'Enter' }) }) + expect(within(panel).getByRole('alert').textContent).toBe(zh['settings.invalid.parent']) + expect(add.getAttribute('aria-invalid')).toBe('true') + act(() => { fireEvent.change(add, { target: { value: 'tmp/' } }) }) + expect(within(panel).queryByRole('alert')).toBeNull() + act(() => { fireEvent.keyDown(add, { key: 'a' }) }) + act(() => { fireEvent.keyDown(add, { key: 'Enter' }) }) + expect(add.value).toBe('') + expect(panel.querySelector('[data-files-rule]')?.getAttribute('data-files-rule')).toBe('tmp') + }) + + it('saves only the rules on close, keeping a hidden-entry choice made while the panel was open', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + const panel = open(h) + act(() => { h.face.setSearchPreferences(ROOT, { showHidden: true }) }) + pointerClick(within(panel).getByRole('checkbox', { name: t('settings.toggle', { path: 'dist' }) })) + pointerClick(within(panel).getByRole('button', { name: zh['settings.close'] })) + expect(stored(h).showHidden).toBe(true) + expect(stored(h).rules.find(rule => rule.path === 'dist')?.enabled).toBe(false) + }) + + it('saves the draft when the eye is clicked without losing the hidden-entry change', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + const panel = open(h) + pointerClick(within(panel).getByRole('checkbox', { name: t('settings.toggle', { path: 'dist' }) })) + const eye = h.view.getByRole('button', { name: zh['search.showHidden'] }) + pointerClick(eye) + expect(dialog(h)).toBeNull() + expect(document.activeElement).toBe(eye) + expect(eye.getAttribute('aria-pressed')).toBe('true') + expect(stored(h).showHidden).toBe(true) + expect(stored(h).rules.find(rule => rule.path === 'dist')?.enabled).toBe(false) + expect(paths(h)).toContain(`${ROOT}/.env`) + expect(within(open(h)).getByRole('checkbox', { name: t('settings.toggle', { path: 'dist' }) })).toHaveProperty('checked', false) + }) + + it('respects .gitignore until its checkbox is cleared, saving the choice at once and searching again with it', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + await type(h, 'a') + expect(requests(h)[0]?.respectGitignore).toBe(true) + const panel = open(h) + const gitignore = within(panel).getByRole('checkbox', { name: zh['settings.respectGitignore'] }) + expect(gitignore.checked).toBe(true) + pointerClick(within(panel).getByRole('checkbox', { name: t('settings.toggle', { path: 'dist' }) })) + pointerClick(gitignore) + // Saved while the panel is open, unlike the rules, which wait for it to close. + expect(gitignore.checked).toBe(false) + expect(stored(h).respectGitignore).toBe(false) + expect(stored(h).rules.find(rule => rule.path === 'dist')?.enabled).toBe(true) + expect(requests(h)[1]).toMatchObject({ respectGitignore: false, excludedPaths: enabledPaths(defaultRules()) }) + expect(dialog(h)).not.toBeNull() + pointerClick(within(panel).getByRole('button', { name: zh['settings.close'] })) + expect(stored(h)).toMatchObject({ respectGitignore: false }) + expect(stored(h).rules.find(rule => rule.path === 'dist')?.enabled).toBe(false) + expect(within(open(h)).getByRole('checkbox', { name: zh['settings.respectGitignore'] })).toHaveProperty('checked', false) + }) + + it('searches again with the saved rules when the panel closes during a search', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + await type(h, 'a') + const panel = open(h) + pointerClick(within(panel).getByRole('checkbox', { name: t('settings.toggle', { path: 'dist' }) })) + pointerClick(within(panel).getByRole('button', { name: zh['settings.close'] })) + expect(requests(h)[1]?.excludedPaths).not.toContain('dist') + }) +}) + +describe('the keyboard', () => { + it('walks from the field through the rows with ↓ and ↑, back to the field from the first row', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + act(() => { field(h).focus() }) + const rows = [...body(h).querySelectorAll('[data-files-path] > div > button')] + act(() => { fireEvent.keyDown(field(h), { key: 'ArrowUp' }) }) + expect(document.activeElement).toBe(field(h)) + act(() => { fireEvent.keyDown(field(h), { key: 'ArrowDown' }) }) + expect(document.activeElement).toBe(rows[0]) + act(() => { fireEvent.keyDown(rows[0]!, { key: 'ArrowDown' }) }) + expect(document.activeElement).toBe(rows[1]) + act(() => { fireEvent.keyDown(rows[1]!, { key: 'ArrowDown', shiftKey: true }) }) + expect(document.activeElement).toBe(rows[1]) + act(() => { fireEvent.keyDown(rows[1]!, { key: 'ArrowUp' }) }) + act(() => { fireEvent.keyDown(rows[0]!, { key: 'ArrowUp' }) }) + expect(document.activeElement).toBe(field(h)) + // Past the last row there is nowhere to go; the separator's arrows are its own. + const last = rows.at(-1)! + act(() => { last.focus() }) + act(() => { fireEvent.keyDown(last, { key: 'ArrowDown' }) }) + expect(document.activeElement).toBe(last) + act(() => { fireEvent.keyDown(h.view.getByRole('separator'), { key: 'ArrowDown' }) }) + // The rows are buttons, so Enter opens a file as a click does. + expect(last.tagName).toBe('BUTTON') + }) + + it('leaves ↓ to an input method composing text, and to a key someone in the navigator already handled', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + act(() => { field(h).focus() }) + act(() => { fireEvent.keyDown(field(h), { key: 'ArrowDown', isComposing: true }) }) + expect(document.activeElement).toBe(field(h)) + const keep = (event: KeyboardEvent): void => { event.preventDefault() } + field(h).addEventListener('keydown', keep) + act(() => { fireEvent.keyDown(field(h), { key: 'ArrowDown' }) }) + field(h).removeEventListener('keydown', keep) + expect(document.activeElement).toBe(field(h)) + act(() => { fireEvent.keyDown(field(h), { key: 'ArrowDown' }) }) + expect(document.activeElement).not.toBe(field(h)) + }) + + it('closes the settings on Escape first, then clears the query, and only then lets the overlay close', async () => { + const h = mountNavigator('sidebar://files') + h.row.width = 480 + await shown(h) + await type(h, 'a') + act(() => { fireEvent.click(settingsButton(h)) }) + act(() => { fireEvent.keyDown(document.activeElement!, { key: 'Escape' }) }) + expect(dialog(h)).toBeNull() + expect(field(h).value).toBe('a') + // An Escape someone in the navigator already kept does not clear it. + const keep = (event: KeyboardEvent): void => { event.preventDefault() } + field(h).addEventListener('keydown', keep) + act(() => { fireEvent.keyDown(field(h), { key: 'Escape' }) }) + field(h).removeEventListener('keydown', keep) + expect(field(h).value).toBe('a') + act(() => { fireEvent.keyDown(body(h), { key: 'Escape' }) }) + expect(field(h).value).toBe('') + expect(document.activeElement).toBe(field(h)) + expect(h.view.container.querySelector('[data-files-navigator]')).not.toBeNull() + act(() => { fireEvent.keyDown(field(h), { key: 'Escape' }) }) + expect(h.view.container.querySelector('[data-files-navigator]')).toBeNull() + }) + + it('keeps the keyboard where the reader moved it as the view switches to the results and back, each with its own scroll', async () => { + const h = mountNavigator(A) + await revealed(h) + fireEvent.scroll(body(h), { target: { scrollTop: 64 } }) + const first = body(h).querySelector('[data-files-path] > div > button')! + act(() => { first.focus() }) + act(() => { fireEvent.keyDown(first, { key: 'ArrowUp' }) }) + expect(document.activeElement).toBe(field(h)) + await type(h, 'b', ['src/b.ts']) + await act(async () => { await Promise.resolve() }) + expect(document.activeElement).toBe(field(h)) + expect(body(h).getAttribute('data-files-view')).toBe('search') + expect(body(h).scrollTop).toBe(0) + act(() => { fireEvent.keyDown(field(h), { key: 'ArrowDown' }) }) + const result = document.activeElement + expect(result?.closest('[data-files-path]')?.getAttribute('data-files-path')).toBe(`${ROOT}/src`) + // Escape on a result row clears the text and hands the keyboard to the field. + act(() => { fireEvent.keyDown(result!, { key: 'Escape' }) }) + await act(async () => { await Promise.resolve() }) + expect(document.activeElement).toBe(field(h)) + expect(body(h).getAttribute('data-files-view')).toBe('tree') + expect(body(h).scrollTop).toBe(64) + }) + + it('carries the search field\'s keyboard focus to the pane\'s next drawing in the same task only', async () => { + const h = mountNavigator('sidebar://files') + await shown(h) + act(() => { field(h).focus() }) + h.view.unmount() + const back = h.remount() + await act(async () => { await Promise.resolve() }) + expect(document.activeElement).toBe(within(back.container).getByRole('searchbox')) + back.unmount() + await act(async () => { await Promise.resolve() }) + h.remount() + await act(async () => { await Promise.resolve() }) + expect(document.activeElement).toBe(document.body) + }) +}) diff --git a/packages/client/ui-sidebar-files/tests/store.client.spec.ts b/packages/client/ui-sidebar-files/tests/store.client.spec.ts index cadf73fe6e..27df4c170a 100644 --- a/packages/client/ui-sidebar-files/tests/store.client.spec.ts +++ b/packages/client/ui-sidebar-files/tests/store.client.spec.ts @@ -3,7 +3,7 @@ import { describe, expect, it } from 'vitest' import { RemoteError } from '@deepseek-ai/dsh-client-test-runtime' import { createFilesStore } from '../src/client/store.ts' import { DEFAULT_NAVIGATOR_WIDTH } from '../src/client/store.ts' -import type { DirLevel } from '../src/client/store.ts' +import type { DirLevel, FilesSearchResult } from '../src/client/store.ts' import type { PaneId } from '@deepseek-ai/dsh-client-ui-dockkit' const ROOT = '/work/app' @@ -22,13 +22,29 @@ describe('createFilesStore', () => { expect(second.getSnapshot().byPane[PANE]).toBeUndefined() }) + it('keeps each pane\'s search-bar choice independent without changing its search', () => { + const store = createFilesStore().create() + const other = 'pane-2' as PaneId + store.actions.start(PANE, ROOT) + store.actions.start(other, ROOT) + store.actions.queried(PANE, 'target') + store.actions.searching(PANE) + const search = store.getSnapshot().byPane[PANE]!.search + store.actions.searchBarShown(PANE, true) + expect(store.getSnapshot().byPane[PANE]!.searchBarOpen).toBe(true) + expect(store.getSnapshot().byPane[other]!.searchBarOpen).toBe(false) + store.actions.searchBarShown(PANE, false) + expect(store.getSnapshot().byPane[PANE]!.query).toBe('target') + expect(store.getSnapshot().byPane[PANE]!.search).toBe(search) + }) + it('seeds a pane at its root with the root expanded and nothing loaded', () => { const store = createFilesStore().create() const { actions } = store const getSnapshot = (): ReturnType => store.getSnapshot() actions.start(PANE, ROOT) expect(getSnapshot().byPane[PANE]).toEqual({ - open: undefined, width: DEFAULT_NAVIGATOR_WIDTH, + open: undefined, width: DEFAULT_NAVIGATOR_WIDTH, query: '', searchBarOpen: false, search: undefined, tree: { root: ROOT, levels: {}, expanded: [ROOT], autoRefresh: true }, }) }) @@ -147,12 +163,12 @@ describe('createFilesStore', () => { const { actions } = store actions.shown(PANE, true) actions.resized(PANE, 320) - expect(store.getSnapshot().byPane[PANE]).toEqual({ open: true, width: 320, tree: undefined }) + expect(store.getSnapshot().byPane[PANE]).toEqual({ open: true, width: 320, tree: undefined, query: '', searchBarOpen: false, search: undefined }) actions.start(PANE, ROOT) actions.loaded(PANE, ROOT, LEVEL) actions.start(PANE, '/work/other') expect(store.getSnapshot().byPane[PANE]).toEqual({ - open: true, width: 320, + open: true, width: 320, query: '', searchBarOpen: false, search: undefined, tree: { root: '/work/other', levels: {}, expanded: ['/work/other'], autoRefresh: true }, }) }) @@ -171,4 +187,51 @@ describe('createFilesStore', () => { actions.forget(PANE) expect(Object.keys(getSnapshot().byPane)).toEqual(['pane-2']) }) + + it('keeps a pane\'s search apart from its tree, and drops a result the query already ended', () => { + const store = createFilesStore().create() + const { actions } = store + const getSnapshot = (): ReturnType => store.getSnapshot() + const search = () => getSnapshot().byPane[PANE]?.search + const found = () => { + const result = search()?.result + return result?.kind === 'found' ? result : undefined + } + const result = (query: string): FilesSearchResult => ({ + kind: 'found', query, count: 0, truncated: false, incomplete: false, hiddenLeftOut: true, excluded: true, ignored: true, + tree: { root: ROOT, levels: {}, expanded: [ROOT, `${ROOT}/src`], autoRefresh: false }, + }) + actions.searching(PANE) + actions.searched(PANE, result('a')) + actions.searchToggled(PANE, ROOT) + expect(getSnapshot().byPane).toEqual({}) + actions.queried(PANE, ' ') + actions.searching(PANE) + actions.searched(PANE, result('a')) + expect(getSnapshot().byPane[PANE]).toMatchObject({ query: ' ', search: undefined }) + actions.queried(PANE, 'a') + expect(search()).toBeUndefined() + // A search under way before its first answer has nothing to draw yet. + actions.searching(PANE) + expect(search()).toEqual({ pending: true, result: undefined }) + actions.searchToggled(PANE, ROOT) + actions.searched(PANE, result('a')) + expect(search()?.pending).toBe(false) + actions.searchToggled(PANE, ROOT) + expect(found()?.tree.expanded).toEqual([`${ROOT}/src`]) + actions.searchToggled(PANE, ROOT) + expect(found()?.tree.expanded).toEqual([`${ROOT}/src`, ROOT]) + // Other text keeps the last result, pending, until the next is recorded. + actions.queried(PANE, 'ab') + actions.searching(PANE) + expect(search()).toMatchObject({ pending: true, result: { query: 'a' } }) + const failure = new RemoteError('gateway/internal', 'down', {}) + actions.searched(PANE, { kind: 'failed', query: 'ab', failure }) + expect(search()).toEqual({ pending: false, result: { kind: 'failed', query: 'ab', failure } }) + // A failed search has no tree to toggle. + actions.searchToggled(PANE, ROOT) + expect(search()?.result).toEqual({ kind: 'failed', query: 'ab', failure }) + actions.queried(PANE, '') + expect(search()).toBeUndefined() + }) }) diff --git a/packages/client/ui-sidebar-files/tests/workspace-tree.client.spec.ts b/packages/client/ui-sidebar-files/tests/workspace-tree.client.spec.ts index 8a95bf5ff4..09fbb72caf 100644 --- a/packages/client/ui-sidebar-files/tests/workspace-tree.client.spec.ts +++ b/packages/client/ui-sidebar-files/tests/workspace-tree.client.spec.ts @@ -3,7 +3,7 @@ import { describe, expect, it } from 'vitest' import { makeTranslate, RemoteError } from '@deepseek-ai/dsh-client-test-runtime' import { zh } from '../src/client/locales.ts' import type { FilesTreeState } from '../src/client/store.ts' -import { parentOf, treeRootOf, workspaceDirectories, workspaceEntries } from '../src/client/workspace-tree.ts' +import { parentOf, searchDirectories, treeRootOf, workspaceDirectories, workspaceEntries } from '../src/client/workspace-tree.ts' const t = makeTranslate(zh) @@ -29,14 +29,15 @@ describe('workspace tree keys', () => { ['\\\\server\\share\\', '\\\\server\\share'], ])('keys root %j as %j, its entries one slash below it, and their parents back to the root as stored', (root, key) => { const tree = listed(root) - expect(workspaceEntries(tree.levels)).toEqual([ + expect(workspaceEntries(tree.levels, false)).toEqual([ { path: `${key}/src`, name: 'src', meta: { kind: 'directory' } }, { path: `${key}/pipe`, name: 'pipe', meta: { kind: 'other' } }, { path: `${key}/src/a.ts`, name: 'a.ts', meta: { kind: 'file' } }, ]) expect(parentOf(tree, `${key}/src`)).toBe(root) expect(parentOf(tree, `${key}/src/lib`)).toBe(`${key}/src`) - expect(Object.keys(workspaceDirectories(tree, t))).toEqual([key, `${key}/src`]) + expect(Object.keys(workspaceDirectories(tree, t, false))).toEqual([key, `${key}/src`]) + expect(Object.keys(searchDirectories(tree))).toEqual([key, `${key}/src`]) }) }) @@ -52,7 +53,7 @@ describe('workspaceDirectories', () => { '/r/c': { kind: 'failed', failure }, '/r/d': { kind: 'ready', level: { entries: [], truncated: false } }, }, - }, t) + }, t, false) expect(states).toEqual({ '/r': { failure: zh['error.notFound'], truncated: true }, '/r/a': { loading: true }, @@ -62,3 +63,37 @@ describe('workspaceDirectories', () => { }) }) }) + +describe('hidden entries', () => { + const tree: FilesTreeState = { + root: '/r', autoRefresh: true, expanded: ['/r', '/r/.git'], + levels: { + '/r': { kind: 'ready', level: { entries: [{ name: '.git', type: 'directory' }, { name: 'a.ts', type: 'file' }], truncated: false } }, + '/r/.git': { kind: 'ready', level: { entries: [{ name: '.keep', type: 'file' }], truncated: false } }, + }, + } + + it('leaves them out, and says a directory of only hidden entries holds some, until they are shown', () => { + expect(workspaceEntries(tree.levels, false).map(entry => entry.path)).toEqual(['/r/a.ts']) + expect(workspaceDirectories(tree, t, false)).toEqual({ '/r': {}, '/r/.git': { omitted: 'hidden' } }) + expect(workspaceEntries(tree.levels, true).map(entry => entry.path)).toEqual(['/r/.git', '/r/a.ts', '/r/.git/.keep']) + expect(workspaceDirectories(tree, t, true)).toEqual({ '/r': {}, '/r/.git': {} }) + }) + + it('leaves out the kept levels below a hidden directory once hidden entries are no longer shown', () => { + const levels: FilesTreeState['levels'] = { + '/r': { kind: 'ready', level: { entries: [{ name: '.config', type: 'directory' }, { name: 'src', type: 'directory' }], truncated: false } }, + '/r/.config': { kind: 'ready', level: { entries: [{ name: 'app', type: 'directory' }, { name: 'config.json', type: 'file' }], truncated: false } }, + '/r/.config/app': { kind: 'ready', level: { entries: [{ name: 'a.json', type: 'file' }], truncated: false } }, + '/r/.configs': { kind: 'ready', level: { entries: [{ name: 'kept.json', type: 'file' }], truncated: false } }, + '/r/src': { kind: 'ready', level: { entries: [{ name: 'b.ts', type: 'file' }], truncated: false } }, + } + // `.configs` shares the hidden directory's name as a prefix but is not below it. + expect(workspaceEntries(levels, false).map(entry => entry.path)).toEqual(['/r/src', '/r/.configs/kept.json', '/r/src/b.ts']) + expect(workspaceEntries(levels, true).map(entry => entry.path)).toContain('/r/.config/app/a.json') + }) + + it('draws a search result directory without matches below it as a selection', () => { + expect(searchDirectories(tree)).toEqual({ '/r': { omitted: 'unmatched' }, '/r/.git': { omitted: 'unmatched' } }) + }) +}) diff --git a/packages/experimental/liveshare/src/endpoint-classification.ts b/packages/experimental/liveshare/src/endpoint-classification.ts index a868012bd6..e817a48d82 100644 --- a/packages/experimental/liveshare/src/endpoint-classification.ts +++ b/packages/experimental/liveshare/src/endpoint-classification.ts @@ -139,6 +139,7 @@ export const VIEWER_ENDPOINT_CLASSES = { 'workspaceFiles/list': 'owner-only', // owner Workspace file access' 'workspaceFiles/read': 'owner-only', // absolute paths reach files outside the Workspace, so no share offers it 'workspaceFiles/readBytes': 'owner-only', // absolute paths reach files outside the Workspace, so no share offers it + 'workspaceFiles/search': 'owner-only', // owner Workspace file access 'workspaceFiles/stat': 'owner-only', // absolute paths reach files outside the Workspace, so no share offers it 'pluginManager/cancelInstall': 'owner-only', // owner plugin management 'pluginManager/inspect': 'owner-only', // owner plugin management diff --git a/packages/extensions/cordis-client-runner/src/client/slot-catalog.ts b/packages/extensions/cordis-client-runner/src/client/slot-catalog.ts index a882ea0ca9..36cbd48ee7 100644 --- a/packages/extensions/cordis-client-runner/src/client/slot-catalog.ts +++ b/packages/extensions/cordis-client-runner/src/client/slot-catalog.ts @@ -4882,7 +4882,7 @@ export const CLIENT_SLOT_API: readonly ClientSlotEntry[] = [ ], replaceRisk: 'none', example: 'return {\n inject: [\'slots\'],\n apply(ctx) {\n ctx.slots.inject(\'sidebar.right.tab.files.actions\', () => ctx.slots.register(\n { name: \'sidebar.right.tab.files.actions\', id: \'my-entry\', order: 100, label: \'My entry\' },\n () => React.createElement(\'div\', null, \'hello\'),\n ))\n },\n}', - source: 'packages/client/ui-sidebar-files/src/client/index.ts:39', + source: 'packages/client/ui-sidebar-files/src/client/index.ts:46', }, { key: 'sidebar.right.tab.guide', diff --git a/packages/extensions/tool-cordis/src/api-catalog.ts b/packages/extensions/tool-cordis/src/api-catalog.ts index 0ac2e9e122..82efc1844f 100644 --- a/packages/extensions/tool-cordis/src/api-catalog.ts +++ b/packages/extensions/tool-cordis/src/api-catalog.ts @@ -4707,6 +4707,13 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ parameters: [{ name: 'workspaceFileScope', description: 'header-derived workspace root for the Session identity on the wire.' }, { name: 'path', description: 'workspace path, absolute or relative to the workspace root.' }, { name: 'signal', description: 'caller cancellation.' }], returns: 'the directory\'s children in the backend\'s stable name order, bounded by the entry cap.', }, + { + signature: '@Remote async search( workspaceFileScope: WorkspaceFileScope, request: WorkspaceFileSearchRequest, signal: AbortSignal, ): Promise', + description: 'Search files and their inferred ancestor directories with packaged ripgrep. Native ignore rules apply unless disabled; VCS metadata, symlinks and empty directories are absent. Within the bounded set of listed files, matches sort by depth, then workspace-relative path.', + parameters: [{ name: 'workspaceFileScope', description: 'header-derived workspace root for the Session identity on the wire.' }, { name: 'request', description: 'the query, literal exclusions, and hidden/ignore switches.' }, { name: 'signal', description: 'caller cancellation, forwarded to the subprocess; rejection preserves its reason.' }], + returns: 'matches capped by search.maxResults; incomplete marks excess rg files or exit 2.', + throws: ['RemoteError `gateway/bad-request`: the query is blank.', 'RemoteError `workspace-file/not-found`: the workspace root does not exist.', 'RemoteError `workspace-file/not-directory`: the workspace root is not a directory.', 'RemoteError `workspace-file/search-failed`: subprocess is missing, rg cannot launch or execute, or the search times out.'], + }, { signature: '@Remote({ mode: \'stream\' }) changes(workspaceFileScope: WorkspaceFileScope, path: string, signal: AbortSignal): AsyncIterable', description: 'Watch one file or a directory\'s direct entries in the Session\'s filesystem. Files use the backend\'s read authority; directories remain workspace-scoped.', @@ -10486,6 +10493,18 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'WorkspaceFileScope', declaration: 'export interface WorkspaceFileScope {\n readonly sessionId: SessionId;\n readonly workspaceRoot: string;\n}', }, + { + name: 'WorkspaceFileSearchMatch', + declaration: 'export interface WorkspaceFileSearchMatch {\n readonly path: string;\n readonly type: WorkspaceDirectoryEntry[\'type\'];\n}', + }, + { + name: 'WorkspaceFileSearchRequest', + declaration: 'export interface WorkspaceFileSearchRequest {\n readonly query: string;\n readonly excludedPaths: readonly string[];\n readonly includeHidden: boolean;\n readonly respectGitignore: boolean;\n}', + }, + { + name: 'WorkspaceFileSearchResult', + declaration: 'export interface WorkspaceFileSearchResult {\n readonly matches: readonly WorkspaceFileSearchMatch[];\n readonly truncated: boolean;\n readonly incomplete: boolean;\n}', + }, { name: 'WorkspaceFileStat', declaration: 'export interface WorkspaceFileStat {\n readonly absolutePath: string;\n readonly version: string;\n readonly bytes?: number;\n}', diff --git a/packages/fs/tool-fs-search/README.i18n.yaml b/packages/fs/tool-fs-search/README.i18n.yaml index b1c75185bb..804fb56a70 100644 --- a/packages/fs/tool-fs-search/README.i18n.yaml +++ b/packages/fs/tool-fs-search/README.i18n.yaml @@ -42,8 +42,8 @@ en: 5a30f023dd86c345 zh: 558b6e80c62648ba /deepseek-ai-dsh-tool-fs-search/understand-the-implementation/how-a-search-runs: - en: 7904c9f013272819 - zh: 656763e9a7e97c1e + en: f3fc47e0bec9091f + zh: af6723f131c40f10 /deepseek-ai-dsh-tool-fs-search/understand-the-implementation/native-mode-benchmark: en: 031654f59e63d07e zh: a3b488fb1880d2f5 diff --git a/packages/fs/tool-fs-search/README.md b/packages/fs/tool-fs-search/README.md index 4c26ffa018..790f620fb1 100644 --- a/packages/fs/tool-fs-search/README.md +++ b/packages/fs/tool-fs-search/README.md @@ -107,7 +107,7 @@ Local workspace discovery is naturally a process-backed `rg` workflow, and putti Without an explicit `path`, `glob` prunes unrelated subtrees for patterns such as `packages/core/src/*.ts` when every directory component is a plain relative name outside VCS metadata. Ordered ripgrep globs exclude the tree, admit directories with the prefix's distinct names, then apply the original file pattern. Same-named directories elsewhere may also be traversed, but only the original pattern admits files. The added arguments are capped at 8 KiB; larger sets use the original traversal. The search root, path separators, symlink handling, matching, and modification-time ordering stay unchanged; other patterns retain the original traversal. -Each call resolves the packaged binary (`@vscode/ripgrep`, or the executable's `-rg` sidecar in a pkg single-file runtime), prepends `--no-config` so a host `RIPGREP_CONFIG_PATH` cannot inject a `--pre` preprocessor into the unconfined spawn, and passes every model-controlled value as a plain argv element — no shell layer exists, so no quoting applies. Collect-mode budgets bound complete stdout and a stderr tail; a lossy stdout read fails as `SEARCH_RAW_OUTPUT_OVERFLOW` rather than parsing a silently-partial stream. The tools never read a raw spill path. +Each call resolves and starts the packaged binary through the shared [ripgrep library](../../util/ripgrep/README.md), which prepends `--no-config` to prevent configuration-driven preprocessors. Model-controlled values remain plain argv elements without shell quoting. Collect-mode budgets bound complete stdout and a stderr tail; a lossy stdout read fails as `SEARCH_RAW_OUTPUT_OVERFLOW` rather than parsing a silently-partial stream. The tools never read a raw spill path. ### Native-mode benchmark diff --git a/packages/fs/tool-fs-search/README.zh.md b/packages/fs/tool-fs-search/README.zh.md index 10cae42308..f39f31c57e 100644 --- a/packages/fs/tool-fs-search/README.zh.md +++ b/packages/fs/tool-fs-search/README.zh.md @@ -107,7 +107,7 @@ Node 部署在受支持的 macOS、Linux 与 Windows 目标上获得 `@vscode/ri 未显式提供 `path` 时，如果目录部分的每一项都是普通相对名称且不属于 VCS 元数据目录，`glob` 会为 `packages/core/src/*.ts` 这样的模式跳过无关子树。按顺序添加的 ripgrep glob 先排除目录树，再按前缀中去重后的名称放行目录，最后应用原始文件模式。其他位置的同名目录也可能被遍历，但文件仍只按原模式匹配。新增参数上限为 8 KiB，超过时沿用原遍历方式。搜索根目录、路径分隔符、符号链接处理、匹配规则和修改时间排序保持不变；其他模式仍按原方式遍历。 -每次调用解析打包二进制（`@vscode/ripgrep`，或 pkg 单文件运行时中可执行程序的 `-rg` 伴随文件），前置 `--no-config`，使宿主的 `RIPGREP_CONFIG_PATH` 无法向不受约束的 spawn 注入 `--pre` 预处理器，并把每个模型控制的值作为普通 argv 元素传入——不存在 shell 层，因此不涉及 shell 引号处理。collect 模式预算限制完整 stdout 与 stderr 尾部；lossy stdout 读取以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败，而不是解析静默不完整的流。工具从不读取原始 spill 路径。 +每次调用通过共享的 [ripgrep 库](../../util/ripgrep/README.zh.md) 解析并启动打包二进制，由它前置 `--no-config` 防止配置启用预处理程序。模型控制的值作为普通 argv 元素传入，不经过 shell 引号处理。collect 模式预算限制完整 stdout 与 stderr 尾部；lossy stdout 读取以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败，而不是解析静默不完整的流。工具从不读取原始 spill 路径。 ### Native 模式 benchmark diff --git a/packages/fs/tool-fs-search/package.json b/packages/fs/tool-fs-search/package.json index 29e776f7a3..1649d99a4c 100644 --- a/packages/fs/tool-fs-search/package.json +++ b/packages/fs/tool-fs-search/package.json @@ -32,8 +32,8 @@ ], "license": "MIT", "dependencies": { - "@vscode/ripgrep": "^1.18.0", - "@deepseek-ai/schemastery": "workspace:~" + "@deepseek-ai/schemastery": "workspace:~", + "@deepseek-ai/dsh-util-ripgrep": "workspace:*" }, "peerDependencies": { "@deepseek-ai/dsh-llm": "workspace:*", @@ -63,6 +63,7 @@ "@deepseek-ai/cordis": "workspace:~", "@deepseek-ai/dsh-scope": "workspace:*", "@deepseek-ai/dsh-working-directory": "workspace:*", - "@deepseek-ai/dsh-agent-loop-testkit": "workspace:*" + "@deepseek-ai/dsh-agent-loop-testkit": "workspace:*", + "@vscode/ripgrep": "^1.18.0" } } diff --git a/packages/fs/tool-fs-search/src/search-core.ts b/packages/fs/tool-fs-search/src/search-core.ts index 794a5d43fe..970846b99a 100644 --- a/packages/fs/tool-fs-search/src/search-core.ts +++ b/packages/fs/tool-fs-search/src/search-core.ts @@ -19,9 +19,10 @@ * @module @deepseek-ai/dsh-tool-fs-search/search-core */ -import { existsSync } from 'node:fs' -import { isAbsolute, join, parse, relative, sep } from 'node:path' +import { isAbsolute, relative, sep } from 'node:path' import type { Context } from '@deepseek-ai/cordis' +import { spawnRipgrep } from '@deepseek-ai/dsh-util-ripgrep' +export { resolveRgPath } from '@deepseek-ai/dsh-util-ripgrep' import { HarnessError } from '@deepseek-ai/dsh-llm' import { ItemRetainer, TextRetainer } from '@deepseek-ai/dsh-output-retention' import type { RetainedItems } from '@deepseek-ai/dsh-output-retention' @@ -154,35 +155,6 @@ function completeStdout(toolName: string, stdout: SubprocessOutputRead, rawOutpu ) } -let rgPathPromise: Promise | undefined - -/** - * The packaged ripgrep binary path, resolved lazily once per process. - * - * A single-file runtime uses the executable's `-rg` sidecar because a native - * helper cannot be spawned from pkg's virtual filesystem. Node-mode builds - * fall back to the platform package selected by `@vscode/ripgrep`. Resolving - * at the call boundary keeps a missing or corrupt binary at the first search - * call as `SEARCH_FAILED`, rather than failing the Loader composition. - * - * @returns the packaged binary's absolute path; the memoized promise rejects - * when the platform package cannot be resolved. - */ -export function resolveRgPath(): Promise { - rgPathPromise ??= Promise.resolve().then(async () => { - const executable = parse(process.execPath) - const executableSidecar = process.platform === 'win32' - ? join(executable.dir, `${executable.name}-rg.exe`) - : `${process.execPath}-rg` - if ('pkg' in process && existsSync(executableSidecar)) return executableSidecar - const dependency = (await import('@vscode/ripgrep')).rgPath - return process.versions.electron === undefined - ? dependency - : dependency.replace(/\.asar(?=[\\/])/u, '.asar.unpacked') - }) - return rgPathPromise -} - /** * Run the packaged ripgrep binary with a plain argv vector and return its * complete raw stdout. The working directory is the calling agent's session @@ -236,8 +208,8 @@ export async function runRipgrep( : await ctx.workingDirectory.ensure(exec.agent, exec.signal) let handle: SubprocessHandle try { - handle = ctx.subprocess.spawn({ - argv: [await resolveRgPath(), '--no-config', ...argv], + handle = await spawnRipgrep(ctx.subprocess, { + argv, cwd: workdir, stdio: { stdin: 'ignore', diff --git a/packages/fs/tool-fs-search/tsconfig.json b/packages/fs/tool-fs-search/tsconfig.json index 4e6b7268ce..5971d17453 100644 --- a/packages/fs/tool-fs-search/tsconfig.json +++ b/packages/fs/tool-fs-search/tsconfig.json @@ -43,6 +43,9 @@ }, { "path": "../../session/working-directory" + }, + { + "path": "../../util/ripgrep" } ] } diff --git a/packages/util/README.i18n.yaml b/packages/util/README.i18n.yaml index f726477c4b..9a1e6e84a4 100644 --- a/packages/util/README.i18n.yaml +++ b/packages/util/README.i18n.yaml @@ -3,20 +3,20 @@ # After editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/util/README.md /: - en: 2994957e73b4e3e2 - zh: 3bf7bbfc3efeab31 + en: 4e92398c76bf480f + zh: 18840c65a44a8c38 /util-shared-utilities: en: 7283f2afb343d150 zh: 32c56bd67c0314c6 /util-shared-utilities/summary: - en: ed8863365a68c0ef - zh: 4bcff265d4d5694e + en: e1aa918f27dccf20 + zh: 7d415fdc8c0248ed /util-shared-utilities/table-of-contents: en: 1eb35b015a6c423f zh: a8a3dceee2c9e943 /util-shared-utilities/packages: - en: e345b399695a4e93 - zh: 179fee3e2c4d90cc + en: 461990a6f0e944bd + zh: 788c91b2b1cf0426 /util-shared-utilities/related-documentation: en: 432ae81110914571 zh: c5bf4efc007311f3 diff --git a/packages/util/README.md b/packages/util/README.md index ac3e59f3a1..2635eac7df 100644 --- a/packages/util/README.md +++ b/packages/util/README.md @@ -1,5 +1,5 @@ --- -description: "Package map for shared utilities: atomic file writes, branded ids, deques, JSON values, harness home paths, launch environment, native commands, output retention, time and zone vocabulary, timeouts, and official package lists." +description: "Package map for shared utilities: atomic file writes, branded ids, deques, JSON values, harness home paths, launch environment, native commands, packaged ripgrep, output retention, time and zone vocabulary, timeouts, and official package lists." kind: "package-group" --- @@ -9,7 +9,7 @@ English | [中文](README.zh.md) ## Summary -The `util/` group gives capability packages shared mechanical primitives instead of duplicate implementations. It covers atomic writes, branded ids, deques, lossless JSON values, UUIDs, Harness-home paths, launch environments, outbound proxy policy, native commands, output retention, time-zone, instant, and span vocabulary, and timeout handling. Every root entry here is a library: it registers no product service or event, and the consuming capability retains the business semantics. The exception is `official-bundles`: instead of a mechanical primitive, it holds the installation's and the plugin manager's official package lists as data their consumers apply. +The `util/` group gives capability packages shared mechanical primitives instead of duplicate implementations. It covers atomic writes, branded ids, deques, lossless JSON values, UUIDs, Harness-home paths, launch environments, outbound proxy policy, native commands, packaged ripgrep execution, output retention, time-zone, instant, and span vocabulary, and timeout handling. Every root entry here is a library: it registers no product service or event, and the consuming capability retains the business semantics. The exception is `official-bundles`: instead of a mechanical primitive, it holds the installation's and the plugin manager's official package lists as data their consumers apply. ## Table of Contents @@ -42,6 +42,7 @@ Each package provides one primitive or one data set; open a package page for how | [`atomic-write/`](atomic-write/README.md) | Atomic file replacement and cross-process writer locking | | [`lazy-require/`](lazy-require/README.md) | Loads CommonJS-compatible Host dependencies on first use with caller-relative resolution | | [`native-command/`](native-command/README.md) | Runs host-native commands directly, never through a shell string | +| [`ripgrep/`](ripgrep/README.md) | Resolves the packaged ripgrep binary, spawns it through the caller's subprocess service, and streams its output records | | [`workspace-path/`](workspace-path/README.md) | Provides browser-safe Workspace path and display helpers | | [`code-language/`](code-language/README.md) | Maps a filename suffix to the syntax-highlighting language that the Code preview, diff review, and the read card render | | [`output-retention/`](output-retention/README.md) | Bounds model-facing output and reports exact omission metadata | diff --git a/packages/util/README.zh.md b/packages/util/README.zh.md index e59f702191..0f0ddfc17d 100644 --- a/packages/util/README.zh.md +++ b/packages/util/README.zh.md @@ -1,5 +1,5 @@ --- -description: "共享工具家族的包映射：原子文件写入、品牌化 id、双端队列、JSON 值、harness 主目录路径、启动环境、原生命令、输出保留、时间与时区词汇、超时，以及官方包列表。" +description: "共享工具家族的包映射：原子文件写入、品牌化 id、双端队列、JSON 值、harness 主目录路径、启动环境、原生命令、打包 ripgrep、输出保留、时间与时区词汇、超时，以及官方包列表。" kind: "package-group" --- @@ -9,7 +9,7 @@ kind: "package-group" ## 概述 -`util/` 组为能力包提供共享的机制原语，避免重复实现。它涵盖原子写入、品牌化 id、双端队列、无损 JSON 值、UUID、Harness home 路径、启动环境、出站代理策略、原生命令、输出保留、时区、时刻与时间段词汇，以及超时处理。这里的每个根入口都是库：它不注册产品服务或事件，业务语义仍由消费它的能力负责。例外是 `official-bundles`：它不是机制原语，而是以纯数据形式保存安装方和插件管理器的官方包列表，由各自的消费方应用。 +`util/` 组为能力包提供共享的机制原语，避免重复实现。它涵盖原子写入、品牌化 id、双端队列、无损 JSON 值、UUID、Harness home 路径、启动环境、出站代理策略、原生命令、打包 ripgrep 的执行、输出保留、时区、时刻与时间段词汇，以及超时处理。这里的每个根入口都是库：它不注册产品服务或事件，业务语义仍由消费它的能力负责。例外是 `official-bundles`：它不是机制原语，而是以纯数据形式保存安装方和插件管理器的官方包列表，由各自的消费方应用。 ## 目录 @@ -42,6 +42,7 @@ kind: "package-group" | [`atomic-write/`](atomic-write/README.zh.md) | 原子文件替换与跨进程写锁 | | [`lazy-require/`](lazy-require/README.zh.md) | 在首次使用时按调用方位置加载兼容 CommonJS 的 Host 依赖 | | [`native-command/`](native-command/README.zh.md) | 直接运行宿主原生命令，绝不拼 shell 字符串 | +| [`ripgrep/`](ripgrep/README.zh.md) | 解析打包的 ripgrep 二进制，通过调用方的子进程服务启动它，并流式读取输出记录 | | [`workspace-path/`](workspace-path/README.zh.md) | 提供浏览器安全的 Workspace 路径与显示辅助函数 | | [`code-language/`](code-language/README.zh.md) | 把文件名后缀映射为 Code 预览、diff 审阅与 read 卡片所渲染的语法高亮语言 | | [`output-retention/`](output-retention/README.zh.md) | 限制面向模型的输出并报告精确的省略元数据 | diff --git a/packages/util/ripgrep/README.i18n.yaml b/packages/util/ripgrep/README.i18n.yaml new file mode 100644 index 0000000000..532bd49d2a --- /dev/null +++ b/packages/util/ripgrep/README.i18n.yaml @@ -0,0 +1,37 @@ +# Bilingual-pair consistency record for README.md (docs/i18n/README.md): per heading +# section, a hash of its English and Chinese blocks outside code blocks and generated regions. +# After editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/util/ripgrep/README.md +/: + en: bc82952963b17d8c + zh: 675b139ebabdc650 +/deepseek-ai-dsh-util-ripgrep: + en: b04d02f7de33360d + zh: 1cfe8f875568591b +/deepseek-ai-dsh-util-ripgrep/summary: + en: bb3d7c655f3adebc + zh: bffe6ea95d686080 +/deepseek-ai-dsh-util-ripgrep/table-of-contents: + en: d152484eb41ac6b4 + zh: 09388d293f9be9cb +/deepseek-ai-dsh-util-ripgrep/use-this-package: + en: 53799d1dd3fc2137 + zh: da69fc599badb99d +/deepseek-ai-dsh-util-ripgrep/understand-the-implementation: + en: b0b0e642476cd3a3 + zh: fa265461d2dabe03 +/deepseek-ai-dsh-util-ripgrep/further-exploration: + en: a95949bcc607ac71 + zh: da783fcb920b894f +/deepseek-ai-dsh-util-ripgrep/model-experience: + en: 9df66fc6f8d80d3e + zh: cf37a67237e3c082 +/deepseek-ai-dsh-util-ripgrep/model-experience/kv-cache-effect: + en: 3b134ccd04fc3436 + zh: 46c862e01cb4b9be +/deepseek-ai-dsh-util-ripgrep/known-limitations-and-deferred-work: + en: 1bfaa2f90e3c339b + zh: da09b60f225fcdf6 +/deepseek-ai-dsh-util-ripgrep/known-limitations-and-deferred-work/dev-note: + en: e67f633dc6fe7773 + zh: e88aeb617ffb74ea diff --git a/packages/util/ripgrep/README.md b/packages/util/ripgrep/README.md new file mode 100644 index 0000000000..17bd06b249 --- /dev/null +++ b/packages/util/ripgrep/README.md @@ -0,0 +1,79 @@ +--- +description: "Run the packaged ripgrep binary through the composed subprocess provider, with bounded streaming for file-search consumers." +kind: "package-library" +--- + +# @deepseek-ai/dsh-util-ripgrep + +English | [中文](README.zh.md) + +## Summary + +Use this library to start packaged ripgrep without a shell or a system `rg` installation. `tool-fs-search` uses collected output; `workspace-files` reads file paths incrementally and stops at its configured cap. Callers supply the working directory, arguments, stream modes, termination grace and cancellation signal, and own exit-code interpretation. + +## Table of Contents + +- [Use this package](#use-this-package) +- [Understand the implementation](#understand-the-implementation) +- [Further Exploration](#further-exploration) +- [Model Experience](#model-experience) +- [Known Limitations and Deferred Work](#known-limitations-and-deferred-work) +- [Dev Note](#dev-note) + +----- + + +## Use this package + +Import `spawnRipgrep` from this package and pass the composed subprocess service plus an explicit `SubprocessSpawnSpec`; its `argv` contains only rg arguments. The returned handle exposes collected output or piped streams as requested. With piped stdout, `readRipgrepLines` delivers newline- or NUL-separated UTF-8 records; returning false from the callback terminates rg and suppresses remaining records. It awaits stdout, the process outcome and managed-range quiescence before returning. Callback, stream and provider failures reject after cleanup. Callers classify exit codes and cancellation using their own signal. + +`resolveRgPath` is available independently for consumers that only need the executable path. Resolution and spawn failures reject the call; they do not prevent importing the package. This library is a dependency, not a profile plugin. + +----- + + +## Understand the implementation + +
    +Implementation internals — click to expand + +The resolver selects the executable's existing `-rg` sidecar in a single-file runtime, otherwise the `@vscode/ripgrep` platform package. Electron paths replace `app.asar` with `app.asar.unpacked`. Every spawn prepends `--no-config` so an ambient rg configuration cannot enable a preprocessor. The subprocess provider owns environment scrubbing and termination; the caller owns deadlines and result budgets. See [the implementation](src/index.ts). + +
    + +----- + + +## Further Exploration + +- [Subprocess provider contract](../../subprocess/subprocess/README.md) — explicit spawn specifications and managed process ranges. +- [Model search tools](../../fs/tool-fs-search/README.md) — collected output, tool errors and result retention. +- [Workspace file search](../../api/workspace-files/README.md#search) — native ignore rules and shallow-first file/ancestor matching. + +----- + + +## Model Experience + +Indirectly, through the search tools that consume the subprocess output. + +#### KV Cache effect + +None directly; consumers own model-visible results and request assembly. + +## Known Limitations and Deferred Work + + + +- **Co-located executable required** — resolution uses the Host's packaged binary path. A remote subprocess provider without that binary fails; there is no system-rg lookup or filesystem-traversal fallback. +- **Record buffering** — streaming retains one incomplete record plus the current stream chunk; callers whose filenames may contain newlines use NUL separators rather than newlines. Records decode as UTF-8, so a filename that is not valid UTF-8 arrives with U+FFFD replacements and cannot be opened by that string. + + +### Dev Note + +
    +Working context for maintainers — click to expand + +None. + +
    diff --git a/packages/util/ripgrep/README.zh.md b/packages/util/ripgrep/README.zh.md new file mode 100644 index 0000000000..47aba57d1f --- /dev/null +++ b/packages/util/ripgrep/README.zh.md @@ -0,0 +1,79 @@ +--- +description: "通过组合的子进程提供方运行打包的 ripgrep，为文件搜索消费方提供有界流式读取。" +kind: "package-library" +--- + +# @deepseek-ai/dsh-util-ripgrep + +[English](README.md) | 中文 + +## 概述 + +使用本库，无需 shell 或系统安装的 `rg` 即可启动打包的 ripgrep。`tool-fs-search` 使用收集后的输出；`workspace-files` 增量读取文件路径，并在配置的上限处停止。调用方显式传入工作目录、参数、流模式、终止宽限期与取消信号，并负责解释退出码。 + +## 目录 + +- [使用本包](#use-this-package) +- [理解实现](#understand-the-implementation) +- [进一步探索](#further-exploration) +- [模型体验](#model-experience) +- [已知限制与延期工作](#known-limitations-and-deferred-work) +- [开发备注](#dev-note) + +----- + + +## 使用本包 + +从本包导入 `spawnRipgrep`，传入组合的子进程服务与显式的 `SubprocessSpawnSpec`；其中 `argv` 只包含 rg 参数。返回的 handle 按请求提供已收集输出或管道流。stdout 为管道时，`readRipgrepLines` 逐条提供以换行或 NUL 分隔的 UTF-8 记录；回调返回 false 会终止 rg，并抑制剩余记录。该函数等待 stdout、进程结果与托管进程范围全部结束后返回。回调、流或提供方失败会在清理后拒绝。调用方按自身 signal 解释退出码与取消。 + +只需要可执行文件路径的消费方可以独立使用 `resolveRgPath`。解析与启动失败会拒绝调用，不阻止导入本包。本库是依赖，不是 profile 插件。 + +----- + + +## 理解实现 + +
    +实现内幕——点击展开 + +单文件运行时优先选用可执行文件旁已有的 `-rg` 附属程序，否则使用 `@vscode/ripgrep` 平台包。Electron 路径将 `app.asar` 替换为 `app.asar.unpacked`。每次启动前置 `--no-config`，防止环境中的 rg 配置启用预处理程序。子进程提供方负责环境清理与终止；调用方负责截止时间与结果预算。参见[实现](src/index.ts)。 + +
    + +----- + + +## 进一步探索 + +- [子进程提供方约定](../../subprocess/subprocess/README.zh.md)——显式启动规格与托管进程范围。 +- [模型搜索工具](../../fs/tool-fs-search/README.zh.md)——输出收集、工具错误与结果保留。 +- [工作区文件搜索](../../api/workspace-files/README.zh.md#search)——原生忽略规则与浅层优先的文件及祖先目录匹配。 + +----- + + +## 模型体验 + +间接影响，通过消费子进程输出的搜索工具。 + +#### KV Cache 影响 + +没有直接影响；模型可见结果与请求装配由消费方负责。 + +## 已知限制与延期工作 + + + +- **要求同处一个执行环境的程序**——解析使用 Host 的打包二进制路径。远端子进程提供方没有该程序时调用失败；不查找系统 rg，也不回退到文件系统遍历。 +- **记录缓冲**——流式读取保留一条尚未完整的记录与当前流块；文件名可能含换行符的调用方使用 NUL 分隔符而不是换行符。记录按 UTF-8 解码，因此不是有效 UTF-8 的文件名会带着 U+FFFD 替换字符到达，无法用该字符串打开。 + + +### 开发备注 + +
    +维护者工作上下文——点击展开 + +无。 + +
    diff --git a/packages/util/ripgrep/package.json b/packages/util/ripgrep/package.json new file mode 100644 index 0000000000..1ff4e89dd4 --- /dev/null +++ b/packages/util/ripgrep/package.json @@ -0,0 +1,43 @@ +{ + "name": "@deepseek-ai/dsh-util-ripgrep", + "description": "Packaged ripgrep resolution and subprocess streaming helpers", + "version": "0.2.2-rc.1", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/util/ripgrep" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "lib/**/*.js.map", + "src" + ], + "license": "MIT", + "dependencies": { + "@vscode/ripgrep": "^1.18.0" + }, + "peerDependencies": { + "@deepseek-ai/cordis": "workspace:~", + "@deepseek-ai/dsh-subprocess": "workspace:*" + }, + "devDependencies": { + "@deepseek-ai/cordis": "workspace:~", + "@deepseek-ai/dsh-subprocess": "workspace:*" + } +} diff --git a/packages/util/ripgrep/src/index.ts b/packages/util/ripgrep/src/index.ts new file mode 100644 index 0000000000..e9ca71e636 --- /dev/null +++ b/packages/util/ripgrep/src/index.ts @@ -0,0 +1,109 @@ +/** Packaged ripgrep resolution and subprocess execution, independent of tool or Remote consumers. */ +import { existsSync } from 'node:fs' +import { join, parse } from 'node:path' +import type { SubprocessHandle, SubprocessOutcome, SubprocessRuntime, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' + +let rgPathPromise: Promise | undefined + +/** + * Resolve the packaged executable lazily, once per process. Single-file runtimes use an existing + * `-rg` sidecar; Electron uses the unpacked ASAR path. Resolution failures reject + * this call, not package loading. No system executable or remote fallback is used. + * @returns the packaged executable's absolute path. + */ +export function resolveRgPath(): Promise { + rgPathPromise ??= Promise.resolve().then(async () => { + const executable = parse(process.execPath) + const executableSidecar = process.platform === 'win32' + ? join(executable.dir, `${executable.name}-rg.exe`) + : `${process.execPath}-rg` + if ('pkg' in process && existsSync(executableSidecar)) return executableSidecar + const dependency = (await import('@vscode/ripgrep')).rgPath + return process.versions.electron === undefined + ? dependency + : dependency.replace(/\.asar(?=[\\/])/u, '.asar.unpacked') + }) + return rgPathPromise +} + +/** + * Start packaged ripgrep with explicit subprocess settings. `spec.argv` contains + * only rg arguments, without an executable. This function prepends the resolved + * rg executable path and `--no-config`, so ambient configuration cannot enable a + * preprocessor. The caller owns exit classification and process cleanup. + * @param subprocess - the composed subprocess service. + * @param spec - cwd, rg arguments (without the executable), stdio, signal and termination grace. + * @returns the live subprocess handle; resolution and synchronous spawn failures reject. + */ +export async function spawnRipgrep(subprocess: Pick, spec: SubprocessSpawnSpec): Promise { + const executable = await resolveRgPath() + return subprocess.spawn({ ...spec, argv: [executable, '--no-config', ...spec.argv] }) +} + +/** Exit facts and whether the line consumer requested early termination. */ +export interface RipgrepLinesResult { + readonly outcome: SubprocessOutcome + readonly stopped: boolean +} + +/** + * Read UTF-8 stdout records without retaining the complete output. A false + * callback return terminates the process and suppresses remaining records. + * Completion, callback failure and stream failure all await process quiescence; + * provider failures reject. Exit codes and cancellation remain consumer-owned. + * @param handle - an owned ripgrep process spawned with piped stdout. + * Records are decoded as UTF-8, so bytes of a name that is not valid UTF-8 + * arrive replaced with U+FFFD. + * @param separator - newline for text output, or NUL for file paths that may contain newlines. + * @param onLine - synchronous record consumer; false requests early termination. + * @returns exit facts after stdout and the managed process range settle. + */ +export async function readRipgrepLines( + handle: SubprocessHandle, + separator: '\n' | '\0', + onLine: (line: string) => boolean, +): Promise { + const stdout = handle.stdout + if (stdout === undefined) { + handle.terminate() + await Promise.allSettled([handle.done, handle.waitForExit()]) + throw new Error('ripgrep requires piped stdout') + } + stdout.setEncoding('utf8') + let stopped = false + let completed = false + const reading = (async () => { + let pending = '' + for await (const chunk of stdout as AsyncIterable) { + if (stopped) continue + pending += chunk + let end: number + while ((end = pending.indexOf(separator)) !== -1) { + const line = pending.slice(0, end) + pending = pending.slice(end + 1) + if (!onLine(line)) { + stopped = true + pending = '' + handle.terminate() + break + } + } + } + if (pending.length > 0 && !onLine(pending)) { + stopped = true + handle.terminate() + } + })() + try { + const [, outcome] = await Promise.all([reading, handle.done]) + completed = true + return { outcome, stopped } + } finally { + if (!completed) { + handle.terminate() + stdout.destroy() + } + await Promise.allSettled([reading, handle.done]) + await handle.waitForExit() + } +} diff --git a/packages/fs/tool-fs-search/tests/rg-sidecar.spec.ts b/packages/util/ripgrep/tests/path.spec.ts similarity index 90% rename from packages/fs/tool-fs-search/tests/rg-sidecar.spec.ts rename to packages/util/ripgrep/tests/path.spec.ts index 9afb0cef58..37eeba956c 100644 --- a/packages/fs/tool-fs-search/tests/rg-sidecar.spec.ts +++ b/packages/util/ripgrep/tests/path.spec.ts @@ -39,7 +39,7 @@ describe('ripgrep resolution', () => { process.execPath = '/runtime/dsh' existsSync.mockReturnValue(true) const sidecar = '/runtime/dsh-rg' - const { resolveRgPath } = await import('@deepseek-ai/dsh-tool-fs-search') + const { resolveRgPath } = await import('@deepseek-ai/dsh-util-ripgrep') await expect(resolveRgPath()).resolves.toBe(sidecar) expect(existsSync).toHaveBeenCalledWith(sidecar) @@ -51,7 +51,7 @@ describe('ripgrep resolution', () => { process.execPath = 'C:\\runtime\\deepseek-harness-sdk-runtime-win-x64.exe' existsSync.mockReturnValue(true) const sidecar = 'C:\\runtime\\deepseek-harness-sdk-runtime-win-x64-rg.exe' - const { resolveRgPath } = await import('@deepseek-ai/dsh-tool-fs-search') + const { resolveRgPath } = await import('@deepseek-ai/dsh-util-ripgrep') await expect(resolveRgPath()).resolves.toBe(sidecar) expect(existsSync).toHaveBeenCalledWith(sidecar) @@ -59,7 +59,7 @@ describe('ripgrep resolution', () => { it('uses the dependency binary in an ordinary Node process', async () => { existsSync.mockReturnValue(true) - const { resolveRgPath } = await import('@deepseek-ai/dsh-tool-fs-search') + const { resolveRgPath } = await import('@deepseek-ai/dsh-util-ripgrep') await expect(resolveRgPath()).resolves.toBe(dependency.rgPath) expect(existsSync).not.toHaveBeenCalled() @@ -68,7 +68,7 @@ describe('ripgrep resolution', () => { it('uses the dependency binary when a packaged runtime has no sidecar', async () => { Reflect.defineProperty(process, 'pkg', { configurable: true, value: {} }) existsSync.mockReturnValue(false) - const { resolveRgPath } = await import('@deepseek-ai/dsh-tool-fs-search') + const { resolveRgPath } = await import('@deepseek-ai/dsh-util-ripgrep') await expect(resolveRgPath()).resolves.toBe(dependency.rgPath) const executable = parse(process.execPath) @@ -81,7 +81,7 @@ describe('ripgrep resolution', () => { it('uses the unpacked executable path for an Electron ASAR dependency', async () => { Reflect.defineProperty(process.versions, 'electron', { configurable: true, value: '44.0.0' }) dependency.rgPath = '/Applications/DeepSeek Harness.app/Contents/Resources/app.asar/dsh/node_modules/@vscode/ripgrep/bin/rg' - const { resolveRgPath } = await import('@deepseek-ai/dsh-tool-fs-search') + const { resolveRgPath } = await import('@deepseek-ai/dsh-util-ripgrep') await expect(resolveRgPath()).resolves.toBe( '/Applications/DeepSeek Harness.app/Contents/Resources/app.asar.unpacked/dsh/node_modules/@vscode/ripgrep/bin/rg', diff --git a/packages/util/ripgrep/tests/stream.spec.ts b/packages/util/ripgrep/tests/stream.spec.ts new file mode 100644 index 0000000000..3887fdf548 --- /dev/null +++ b/packages/util/ripgrep/tests/stream.spec.ts @@ -0,0 +1,99 @@ +/** Stream framing, early termination and process ownership for packaged rg. */ +import { PassThrough, Readable } from 'node:stream' +import { describe, expect, it, vi } from 'vitest' +import type { SubprocessHandle, SubprocessOutcome, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import { readRipgrepLines, resolveRgPath, spawnRipgrep } from '../src/index.ts' + +function handle( + stdout: Readable | undefined, + done: Promise = Promise.resolve({ exitCode: 0, signal: null }), +) { + return { stdout, stdin: undefined, stderr: undefined, control: undefined, collected: {}, done, + terminate: vi.fn(), waitForExit: vi.fn(async () => true) } satisfies SubprocessHandle +} + +describe('packaged rg spawning', () => { + it('forwards explicit settings and prepends --no-config', async () => { + const child = handle(undefined) + const spawn = vi.fn(() => child) + const spec: SubprocessSpawnSpec = { argv: ['--files'], cwd: '/workspace', signal: new AbortController().signal, + graceMs: 5, stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 20 } }, env: { TEST: 'value' } } + await expect(spawnRipgrep({ spawn }, spec)).resolves.toBe(child) + expect(spawn).toHaveBeenCalledWith({ ...spec, argv: [await resolveRgPath(), '--no-config', '--files'] }) + }) + + it('preserves synchronous provider failures', async () => { + const failure = new Error('cannot spawn') + await expect(spawnRipgrep({ spawn() { throw failure } }, { argv: [], cwd: '/', graceMs: 1, + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: 'pipe' } })).rejects.toBe(failure) + }) +}) + +describe('incremental stdout records', () => { + it('decodes split UTF-8 and delimiter chunks, including an unterminated final record', async () => { + const bytes = Buffer.from('雪\0two\0tail') + const child = handle(Readable.from([bytes.subarray(0, 1), bytes.subarray(1, 4), bytes.subarray(4)])) + const lines: string[] = [] + expect(await readRipgrepLines(child, '\0', (line) => { lines.push(line); return true })).toEqual({ stopped: false, outcome: { exitCode: 0, signal: null } }) + expect(lines).toEqual(['雪', 'two', 'tail']) + expect(child.terminate).not.toHaveBeenCalled() + expect(child.waitForExit).toHaveBeenCalledOnce() + }) + + it('stops callbacks at the cap while draining subsequent buffered chunks', async () => { + const child = handle(Readable.from(['one\ntwo\n', 'three\n'])) + const onLine = vi.fn(() => false) + expect((await readRipgrepLines(child, '\n', onLine)).stopped).toBe(true) + expect(onLine).toHaveBeenCalledExactlyOnceWith('one') + expect(child.terminate).toHaveBeenCalledOnce() + }) + + it('accepts an empty stream and can stop at the final unterminated record', async () => { + const empty = handle(Readable.from([])) + const onLine = vi.fn(() => true) + expect((await readRipgrepLines(empty, '\n', onLine)).stopped).toBe(false) + expect(onLine).not.toHaveBeenCalled() + const child = handle(Readable.from(['last'])) + expect((await readRipgrepLines(child, '\n', () => false)).stopped).toBe(true) + expect(child.terminate).toHaveBeenCalledOnce() + }) + + it('terminates a handle without piped stdout and waits for its outcome', async () => { + const child = handle(undefined, Promise.reject(new Error('launch failure'))) + await expect(readRipgrepLines(child, '\n', () => true)).rejects.toThrow('requires piped stdout') + expect(child.terminate).toHaveBeenCalledOnce() + expect(child.waitForExit).toHaveBeenCalledOnce() + }) + + it('propagates callback failures after terminating and awaiting cleanup', async () => { + const child = handle(Readable.from(['line\n'])) + const failure = new Error('consumer failed') + await expect(readRipgrepLines(child, '\n', () => { throw failure })).rejects.toBe(failure) + expect(child.terminate).toHaveBeenCalledOnce() + expect(child.waitForExit).toHaveBeenCalledOnce() + }) + + it('settles a provider failure even if stdout stays open', async () => { + const stdout = new PassThrough() + const failure = new Error('provider failed') + const child = handle(stdout, Promise.reject(failure)) + await expect(readRipgrepLines(child, '\n', () => true)).rejects.toBe(failure) + expect(stdout.destroyed).toBe(true) + expect(child.terminate).toHaveBeenCalledOnce() + }) + + it('waits for the managed process range after stdout and the outcome finish', async () => { + let release!: () => void + const cleanup = new Promise((resolve) => { release = resolve }) + const entered = Promise.withResolvers() + const child = { ...handle(Readable.from(['line\n'])), + waitForExit: async () => { entered.resolve(undefined); await cleanup; return true } } + let settled = false + const result = readRipgrepLines(child, '\n', () => true).then((value) => { settled = true; return value }) + await entered.promise + expect(settled).toBe(false) + release() + await result + expect(settled).toBe(true) + }) +}) diff --git a/packages/util/ripgrep/tsconfig.json b/packages/util/ripgrep/tsconfig.json new file mode 100644 index 0000000000..63eb05171f --- /dev/null +++ b/packages/util/ripgrep/tsconfig.json @@ -0,0 +1,21 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../subprocess/subprocess" + } + ] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index b48c3c87c1..d016fe8deb 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1894,6 +1894,9 @@ importers: '@deepseek-ai/dsh-typert-protocol': specifier: workspace:* version: link:../../typert/protocol + '@deepseek-ai/dsh-util-ripgrep': + specifier: workspace:* + version: link:../../util/ripgrep '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -1934,6 +1937,12 @@ importers: '@deepseek-ai/dsh-session-persistence': specifier: workspace:* version: link:../../session/session-persistence + '@deepseek-ai/dsh-subprocess': + specifier: workspace:* + version: link:../../subprocess/subprocess + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:* + version: link:../../subprocess/subprocess-local '@deepseek-ai/dsh-util-workspace-path': specifier: workspace:* version: link:../../util/workspace-path @@ -12356,12 +12365,12 @@ importers: packages/fs/tool-fs-search: dependencies: + '@deepseek-ai/dsh-util-ripgrep': + specifier: workspace:* + version: link:../../util/ripgrep '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery - '@vscode/ripgrep': - specifier: ^1.18.0 - version: 1.18.0 devDependencies: '@deepseek-ai/cordis': specifier: workspace:~ @@ -12411,6 +12420,9 @@ importers: '@deepseek-ai/dsh-working-directory': specifier: workspace:* version: link:../../session/working-directory + '@vscode/ripgrep': + specifier: ^1.18.0 + version: 1.18.0 packages/fs/tool-str-replace-editor: dependencies: @@ -17871,6 +17883,19 @@ importers: specifier: ^7.8.0 version: 7.8.0 + packages/util/ripgrep: + dependencies: + '@vscode/ripgrep': + specifier: ^1.18.0 + version: 1.18.0 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:~ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-subprocess': + specifier: workspace:* + version: link:../../subprocess/subprocess + packages/util/time: dependencies: '@js-temporal/polyfill': diff --git a/scripts/doc-standard.spec.ts b/scripts/doc-standard.spec.ts index 4ae1b2fc56..a8f2eb1095 100644 --- a/scripts/doc-standard.spec.ts +++ b/scripts/doc-standard.spec.ts @@ -107,6 +107,7 @@ const PACKAGE_LIBRARIES: Readonly> = { 'packages/util/package-manifest': 'Shared package manifest declarations with type-only exports.', 'packages/util/plugin-detail': 'Plugin detail document library.', 'packages/util/plugin-market-index': 'Plugin market index validation library.', + 'packages/util/ripgrep': 'Packaged executable resolution and caller-owned subprocess streaming without a plugin entry.', 'packages/util/time': 'Zero-dependency time-zone canonicalization utility.', 'packages/util/timeout': 'Zero-dependency timeout utility.', 'packages/util/values': 'Stateless lossless-JSON and immutable-value helpers.', diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index 9e5119387d..3f528ef5f3 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -1125,6 +1125,8 @@ export const TYPE_LINK_EXEMPTIONS: Readonly> = { WorkspaceFileRange: 'Host workspace file endpoint contract is owned by packages/api/workspace-files/README.md', WorkspaceFileStat: 'Host workspace file endpoint contract is owned by packages/api/workspace-files/README.md', WorkspaceFileText: 'Host workspace file endpoint contract is owned by packages/api/workspace-files/README.md', + WorkspaceFileSearchRequest: 'Host workspace file endpoint contract is owned by packages/api/workspace-files/README.md', + WorkspaceFileSearchResult: 'Host workspace file endpoint contract is owned by packages/api/workspace-files/README.md', TerminalShell: 'Browser terminal shell profiles are owned by packages/api/terminal-controller/README.md', TerminalEnvironment: 'Browser terminal environment fields are owned by packages/api/terminal-controller/README.md', WebTerminalInfo: 'Browser terminal metadata is owned by packages/api/terminal-controller/README.md', diff --git a/scripts/package-dependency-policy.ts b/scripts/package-dependency-policy.ts index 700ba10ca6..4ad6448e20 100644 --- a/scripts/package-dependency-policy.ts +++ b/scripts/package-dependency-policy.ts @@ -70,6 +70,8 @@ const SAFE_HOST_DEPENDENCY_EXPORTS = { '@deepseek-ai/dsh-session-format': ['sessionFormatLogFilename'], '@deepseek-ai/dsh-storage-domain': ['defineDomain'], '@deepseek-ai/dsh-timeout': ['MAX_TIMER_DELAY_MS'], + // Both operate only on the caller's subprocess service and handle; the module memoizes only the binary path. + '@deepseek-ai/dsh-util-ripgrep': ['readRipgrepLines', 'spawnRipgrep'], '@deepseek-ai/schemastery': ['default'], } as const satisfies HostDependencyExports diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index d5001cd021..6424868d38 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -113,6 +113,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/util/lazy-require': { kind: 'none', reason: 'Host module-loading primitive; registers nothing model-facing.' }, 'packages/util/plugin-detail': { kind: 'none', reason: 'Pure document selection for plugin detail pages; registers nothing model-facing.' }, 'packages/util/plugin-market-index': { kind: 'indirect', reason: 'Pure index validation; the plugin_market tool owns the entries it returns to the model.' }, + 'packages/util/ripgrep': { kind: 'indirect', reason: 'Search consumers own tool results and model context; this library only resolves and runs rg.' }, 'packages/util/time': { kind: 'indirect', reason: 'Pure zone validation; the consumer that records a canonical zone owns the model-visible line derived from it.' }, 'packages/core/agent-default-model': { kind: 'indirect', reason: 'The service supplies a ModelSelection; request assembly and adapters own the model-visible request.' }, 'packages/llm/deepseek-llm-api-extensions': { kind: 'indirect', reason: 'The registry contributes model-hidden provider fields; dsh-llm-deepseek owns their wire placement.' }, diff --git a/tsconfig.base.json b/tsconfig.base.json index 2121904985..e8ca4f8e44 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -410,6 +410,7 @@ // so these aliases stay hand-written like dsh-util-crypto above. "@deepseek-ai/dsh-util-time": ["./packages/util/time/src"], "@deepseek-ai/dsh-util-file-drag": ["./packages/util/file-drag/src"], + "@deepseek-ai/dsh-util-ripgrep": ["./packages/util/ripgrep/src"], // session-log-export lives under session-query/, so the generated aliases map only its // bare name; the browser half needs this explicit subpath for source-mode dynamic imports. "@deepseek-ai/dsh-session-log-export/client": ["./packages/session-query/session-log-export/src/client"], diff --git a/tsconfig.host.json b/tsconfig.host.json index c926055be3..9480842554 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -352,6 +352,7 @@ { "path": "./packages/util/launch-environment" }, { "path": "./packages/util/lazy-require" }, { "path": "./packages/util/native-command" }, + { "path": "./packages/util/ripgrep" }, { "path": "./packages/util/home-paths" }, { "path": "./packages/client/ui-plugin-manager/tsconfig.host.json" }, { "path": "./packages/client/ui-trajectory/tsconfig.host.json" }, From e98fcd59c25f088335c596df9c758aeb4bb38a8d Mon Sep 17 00:00:00 2001 From: Shigma Date: Tue, 3 Nov 2026 13:49:28 +0800 Subject: [PATCH 41/44] docs(vendor): drop the rc.11 plan and correct upstream pins --- vendor/README.md | 21 +++---- vendor/cordis-rc11-upgrade-plan.md | 93 ------------------------------ 2 files changed, 9 insertions(+), 105 deletions(-) delete mode 100644 vendor/cordis-rc11-upgrade-plan.md diff --git a/vendor/README.md b/vendor/README.md index 9020f5dcb6..c3a4c8e774 100644 --- a/vendor/README.md +++ b/vendor/README.md @@ -6,23 +6,19 @@ All vendored packages use the **`@deepseek-ai` scope** (`cordis` → `@deepseek- This file covers the manifest, the local-modification log, and the procedure for **updating** an existing vendored package. To **add a new** one, see the cookbook guide: [docs/cookbook/adding-a-vendored-package.md](../docs/cookbook/adding-a-vendored-package.md). -The proposed [Cordis rc.11 upgrade plan](cordis-rc11-upgrade-plan.md) records the package scope, PR dependencies and intended disposition of the local modifications. - ## Manifest -Upstream workspace: `cordis-workspace` (local checkout: `~/repos/cordis-workspace`). - | Directory | npm name | Upstream name | Version | Upstream repo | Commit | |---|---|---|---|---|---| -| `cosmokit/` | `@deepseek-ai/cosmokit` | `cosmokit` | 1.8.1 | https://github.com/deepseek-harness/cosmokit | `16f6fc058ade66e8ac5da0033d35a8d0f279f544` | -| `schemastery/` | `@deepseek-ai/schemastery` | `schemastery` | 3.18.0 | https://github.com/deepseek-harness/schemastery (`packages/core`) | `e67cee00ad725bd1534aee930a979ea3eec6f698` | +| `cosmokit/` | `@deepseek-ai/cosmokit` | `cosmokit` | 1.8.1 | https://github.com/shigma/cosmokit | `02e691c5aa7f37f6e0b1cee7ee8f4a21c2e34507` | +| `schemastery/` | `@deepseek-ai/schemastery` | `schemastery` | 3.18.0 | https://github.com/shigma/schemastery (`packages/core`) | `cf0b7e5481d0e31d10cdc30bc45c09366c332f27` | | `cordis/` | `@deepseek-ai/cordis` | `cordis` | 4.0.0-rc.11 | https://github.com/cordiverse/cordis (`packages/core`) | `0b96d9fd517b998e44e2209560c9e5f0671ce933` | | `loader/` | `@deepseek-ai/cordis-plugin-loader` | `@cordisjs/plugin-loader` | 1.0.0-rc.8 | https://github.com/cordiverse/cordis (`packages/loader`) | `0b96d9fd517b998e44e2209560c9e5f0671ce933` | | `include/` | `@deepseek-ai/cordis-plugin-include` | `@cordisjs/plugin-include` | 1.1.1 | https://github.com/cordiverse/cordis (`packages/include`) | `0b96d9fd517b998e44e2209560c9e5f0671ce933` | -| `group/` | `@deepseek-ai/cordis-plugin-group` | `@cordisjs/plugin-group` | 1.0.0 | https://github.com/deepseek-harness/cordis (`packages/group`) | `abb0a307cb1d3b0947f455d590cf5ba922d4caa4` | -| `timer/` | `@deepseek-ai/cordis-plugin-timer` | `@cordisjs/plugin-timer` | 1.1.4 | https://github.com/cordiverse/cordis (`packages/timer`) | `0b96d9fd517b998e44e2209560c9e5f0671ce933` | -| `hmr/` | `@deepseek-ai/cordis-plugin-hmr` | `@cordisjs/plugin-hmr` | 1.0.15 | https://github.com/deepseek-harness/cordis (`packages/hmr`) | `abb0a307cb1d3b0947f455d590cf5ba922d4caa4` | -| `logger-console/` | `@deepseek-ai/cordis-plugin-logger-console` | `@cordisjs/plugin-logger-console` | 1.0.0 | https://github.com/deepseek-harness/cordis (`packages/logger-console`) | `abb0a307cb1d3b0947f455d590cf5ba922d4caa4` | +| `group/` | `@deepseek-ai/cordis-plugin-group` | `@cordisjs/plugin-group` | 1.0.0 | https://github.com/cordiverse/cordis (`packages/group`) | `01b396240ce47cf2779c40af9bc445bb748fe112` | +| `timer/` | `@deepseek-ai/cordis-plugin-timer` | `@cordisjs/plugin-timer` | 1.1.4 | https://github.com/cordiverse/cordis (`packages/timer`) | `01b396240ce47cf2779c40af9bc445bb748fe112` | +| `hmr/` | `@deepseek-ai/cordis-plugin-hmr` | `@cordisjs/plugin-hmr` | 1.0.15 | https://github.com/cordiverse/cordis (`packages/hmr`) | `ed8a7755c26a27a72064a21dea036ad1d1d6833c` | +| `logger-console/` | `@deepseek-ai/cordis-plugin-logger-console` | `@cordisjs/plugin-logger-console` | 1.0.0 | https://github.com/cordiverse/cordis (`packages/logger-console`) | `01b396240ce47cf2779c40af9bc445bb748fe112` | Third-party dependencies of the vendored packages stay on npm: `@standard-schema/spec`, `js-yaml`, `chokidar`, `picomatch`, `@babel/code-frame`, `supports-color`, `node-addon-require-builtin`. @@ -31,13 +27,14 @@ Intentionally **not** vendored (verified unused by this set): `reggol`, `@cordis ## Local modifications Keep this log exhaustive — every divergence from upstream must be listed. Each entry has a stable ID: remove an entry when its divergence no longer exists, and never renumber or reuse IDs. The next new entry is V28. + - **V1 · `hmr/src/index.ts`**: removed the `./locales/en-US.yml` / `./locales/zh-CN.yml` imports, the `.i18n({...})` call on the `Config` schema, and the `src/locales/` directory. Rationale: those imports require a runtime YAML loader hook (`@cordisjs/unyaml`) that we do not vendor; the i18n texts only localize config descriptions. - **V2 · All `package.json` files**: regenerated for Harness releases with scoped names, release versions, publication metadata, precise bundled-runtime and `lib/types/**/*.d.ts` / `.d.ts.map` file entries, source exports where applicable, and declaration metadata pointing at `lib/types`. References within the vendored set use `workspace:~`. HMR declares `esbuild` as a direct dev dependency for its imported `BuildFailure` type, and Loader requires `node-addon-require-builtin@^0.1.4` to match published app runtimes. - **V3 · All `tsconfig.json` files**: regenerated to extend the repo-root `tsconfig.base.json`, emit TypeScript intermediates to `lib/types`, and declare project references. - **V4 · Vendored TypeScript source internal specifiers**: changed local relative imports/exports from upstream's specifier shape to explicit `.ts` specifiers so TypeScript rewrites emitted JS to `.js` while declarations keep explicit, NodeNext-safe `.ts` specifiers. This includes `loader/src/config/isolate.ts` using `declare module './entry.ts'`. Type-only dependencies use `import type` or inline `type` modifiers so ESM output does not retain erased interfaces. - **V5 · `schemastery/tsdown.config.ts` and `logger-console/tsdown.config.ts`**: ours, not upstream files — per-package build-shape overrides (dual ESM+CJS output; separate node/browser entries) for the repo-root tsdown build. They read the JS emitted under `lib/types` and then write the publish runtime entries under `lib/`. Like the regenerated tsconfigs, they are not part of the upstream sync surface. - **V6 · `cordis/src/fiber.ts` lifecycle hardening**: locally closes three reentrant disposal gaps. An effect's owner-list wrapper is registered before its setup body runs, so an unload begun from inside setup awaits setup and every collected cleanup; synchronous setup failure removes the wrapper and rolls back collected cleanup. Async cleanup stays owner-visible until quiescence, and Cordis's internal effect composition joins an already-running cleanup while repeated public disposer calls retain their upstream single-shot result. Effect creation is rejected while the owner is `UNLOADING` (while `PENDING` and `LOADING` remain legal), preventing cleanup-time registrations from escaping the unload snapshot. Child fibers register and receive their parent-owned disposer before `internal/plugin` publication, resolve dependency declarations added by that notification before activation, drain effects attached while pending, and contain teardown-notification failures per observer so one callback cannot starve peers or interrupt ownership cleanup. -- **V7 · Cordis, Loader and Timer JSDoc enrichment**: added `@param`/`@returns` tags and contract documentation (disposal semantics, waterfall veto, bail conditions, error cases) across the public plugin-author surface — `Context` (class, statics, and the `Context` interface properties incl. `root`), module re-exports, shared utility helpers, `EventsService`, `Fiber`, `RegistryService`, `ReflectService`, `Service`, `LoggerService` and their `declare module './context.ts'` overloads. Loader documentation covers `loader/src/{index,internal,config/entry,config/group,config/isolate,config/tree,config/utils}.ts`, including entry ownership, tree mutation methods, and the `!!js` discriminator. The upstream `EventsService.dispatch()` deprecation annotation is omitted: DSH enumerates callbacks to contain each observer failure independently, which the typed dispatch methods do not provide. Timer retains its local class and helper summaries; its README is local documentation because the upstream package has none. Comment-only; no code changes. Motivation: the website API-reference generator renders these docs and hard-errors on undocumented members. Retire this entry when the enrichment is upstreamed to the fork. +- **V7 · JSDoc enrichment**: added `@param`/`@returns` tags and contract documentation (disposal semantics, waterfall veto, bail conditions, error cases) across the public plugin-author surface — `Context` (class, statics, and the `Context` interface properties incl. `root`), module re-exports, shared utility helpers, `EventsService`, `Fiber`, `RegistryService`, `ReflectService`, `Service`, `LoggerService` and their `declare module './context.ts'` overloads. Loader documentation covers `loader/src/{index,internal,config/entry,config/group,config/isolate,config/tree,config/utils}.ts`, including entry ownership, tree mutation methods, and the `!!js` discriminator. The upstream `EventsService.dispatch()` deprecation annotation is omitted: DSH enumerates callbacks to contain each observer failure independently, which the typed dispatch methods do not provide. Timer retains its local class and helper summaries; its README is local documentation because the upstream package has none. Cosmokit and Schemastery add summaries to their exported helpers, types and namespaces, logger-console to its exporters and config namespace, and HMR to its build-failure logger in `hmr/src/error.ts`. Comment-only; no code changes. Motivation: the website API-reference generator renders these docs and hard-errors on undocumented members. Retire this entry when the enrichment is upstreamed. - **V9 · `hmr/src/index.ts` module-watcher readiness and native paths**: realpath the existing watch base, classify framework modules and attach listeners before reporting readiness, and compare config paths using both canonical and configured spellings. This preserves Node module-cache identity across Windows short-name paths and filesystem aliases. The main watcher uses `ignoreInitial: true` so startup reads do not trigger another refresh; exact profile patch watching is owned by `packages/boot/hmr/src/watch-config.ts`. The Include change listener logs a rejected `Include.refresh()` instead of leaving the rejection unhandled; the rejection reports unsaved runtime changes while Include keeps its running tree. Covered by `packages/boot/hmr/tests/watch-config.spec.ts` and `scripts/cordis-vendor-hmr-include.spec.ts`. - **V10 · Vendored Node-compatible TypeScript**: marked erased imports explicitly across `cordis`, `loader`, `include`, `hmr`, and `schemastery` so Node's native TypeScript transform does not request types as runtime exports. Schemastery's source uses an ESM default export and its package declares `type: module`. Conditional exports select `.mjs` for import and `.cjs` for require; without them, concurrent imports under module-hook hosts reach the CJS entry and can race its require of ESM Cosmokit (`ERR_REQUIRE_ESM_RACE_CONDITION`). - **V11 · Include patch and YAML exports**: `include/src/index.ts` exports upstream `applyPatches` as `applyEntryPatches` and the `!!js` YAML dialect as `entryListSchema`, so `dsh --dump-config` composes entries with the same algorithm that mounting uses. Covered by `packages/boot/app-boot/tests/config-reload.spec.ts` and `scripts/cordis-entry-commit.spec.ts`. @@ -63,7 +60,7 @@ Relative to the resolver source in Cordis `v4.0.0-rc.11`: To update a vendored package from upstream: -1. In the upstream workspace, note `git rev-parse HEAD` of the relevant submodule. +1. In the upstream repository listed in the manifest, check out the release tag or commit to sync and note its full commit SHA. 2. Copy the package's `src/` (and `bin.js`, `README.md`, `LICENSE` if changed) over the vendored directory. 3. Re-apply the local modifications listed above (or drop them if upstream made them unnecessary — update the log either way). 4. Update the version and commit hash in the manifest table. diff --git a/vendor/cordis-rc11-upgrade-plan.md b/vendor/cordis-rc11-upgrade-plan.md deleted file mode 100644 index 7833cf3514..0000000000 --- a/vendor/cordis-rc11-upgrade-plan.md +++ /dev/null @@ -1,93 +0,0 @@ -# Cordis rc.11 vendor upgrade plan - -Status: proposed - -## Problem - -The upgrade starts from the [vendored](README.md) baselines Cordis core 4.0.0-rc.7, Loader 1.0.0-rc.5, Include 1.0.4, HMR 1.0.15 and Timer 1.1.2, with 24 local-modification entries. [Cordis v4.0.0-rc.11](https://github.com/cordiverse/cordis/releases/tag/v4.0.0-rc.11) incorporates some fixes and replaces other mechanisms. Updating every package and retiring every patch in one change would combine configuration, persistence, lifecycle and module-replacement decisions. - -DSH also has consumers that upstream Cordis does not own. Volatile configuration fields are stable references updated without restarting plugins; the settings UI reads their live values. A profile selects ordered plugin bundles and configuration overrides. [DSH HMR](../packages/boot/hmr/README.md) coordinates module replacement and profile changes through `ctx.hmr`. An entry's module namespace is its raw import result, retained before selecting `default` or `apply` so HMR can identify the module already loaded. These consumers must remain usable at every merge point. - -## Proposal - -Keep source vendoring and stage the upgrade as independently reviewable PRs, each covering a complete concern with its producers and consumers. Upgrade Core with the minimum cross-package compatibility changes, then complete Loader, Include and vendor HMR separately. Retire an existing patch when its replacement is adopted and verified; retain independently applicable patches until their behavior has a replacement. PR 0 records this plan only. It does not approve the open runtime decisions or implement the upgrade. - -### Package scope - -| Package | Target from the rc.11 tag | Treatment | -|---|---|---| -| `vendor/cordis` | 4.0.0-rc.11 | Upgrade with the Core compatibility slice. | -| `vendor/loader` | 1.0.0-rc.8 | Import required slices first; complete the package after resolution and change reporting land. | -| `vendor/include` | 1.1.1 | Separate patch composition from journal adoption. | -| `vendor/hmr` | 1.1.1 | Upgrade independently of the Include journal. | -| `vendor/timer` | 1.1.4 | Upgrade independently of Core. | -| `vendor/group` | Keep the existing pin | Its entry source matches the tag after rescoping. The Group implementation lives in Loader and still requires migration. | -| `vendor/logger-console` | Keep the existing pin | All three source files match the tag after normalizing scope, relative import extensions and comments. Core's logger changes remain part of the Core upgrade. | -| `vendor/cosmokit`, `vendor/schemastery` | Keep the existing pins | Retain the volatile extensions; their upstream upgrades are outside this plan. | - -The comparison above checks the current vendored source against the release tag; it does not assume equal version numbers imply equal code. DSH package versions, exports and build settings remain governed by the repository. A partially synchronized Loader, Include or HMR keeps its old manifest baseline and records the imported upstream slice in the local-modification log. Advance the package baseline only when the complete package has been reconciled, with remaining differences recorded explicitly. - -### PR dependencies - -The identifiers below are plan labels, not GitHub issue numbers. Preconditions are merge requirements. Independent PRs can be developed and reviewed concurrently; a dependent PR can start against the proposed changes before its predecessors merge, but must validate the actual combined result before merging. - -`Vn` below is the stable ID of an entry in the [local-modification log](README.md#local-modifications), not a PR; a retired ID no longer appears in the log. **Retire** means the upstream implementation replaces the local difference; **partial** means some behavior still needs a local adaptation; **retain** means the requirement remains; **prepare** does not retire a patch. These are proposed outcomes, confirmed only by the owning PR's tests. - -| PR | Scope | Prerequisite PRs | Existing vendor modifications | Required evidence | -|---|---|---|---|---| -| 0 | This plan, package comparison and open decisions. | None | None retired; records the proposed disposition. | Documentation checks; no runtime changes. | -| 1 | Define access to raw config, candidate resolution and the active instance's effective config, and migrate settings, Config Editor, volatile updates and HMR readers together. Initially delegate to the old framework fields. | 0 | Prepare V15 (lazy/raw config) and V22 (volatile); retire neither yet. | Preserve existing objects, validation, settings and volatile behavior. Candidate validation does not replace the active instance's references; introduce no dependency from vendor code to DSH packages. | -| 2 | Add upstream-compatible `hmr.watch()` to both HMR providers with compatible typing, migrate the existing Include to explicit registration, and remove the equivalent Include scans. Use DSH HMR's existing coordination queue. | 0 | Adapt V9's config-path matching to explicit watches; retain its readiness and native-path requirements. | Callback ordering, disposal, explicit paths and `root: []` under the old Core; one Include refresh per change under either provider. Preserve profile watching and both module-replacement algorithms. | -| 3 | Upgrade Timer to 1.1.4, retaining DSH locale resources and build metadata. | 0 | Retain V24 (display metadata); no logged runtime patch retired. | Concurrent interval reads, throttle cancellation and void return types; check callers using the old Core. | -| 4a | Upgrade Core to rc.11 with the minimum Loader, Include and HMR compatibility slice described below. | 1 | Retire V12 (activation observer), V21 (logger disposer) and, after canonical-fiber audits, V20 (Entry identity workaround). Partial V6 (lifecycle) and V15 (config model). Retain/adapt V22 and V23 (module namespace). | Config resolution, volatile identity, service isolation, dependent-before-provider teardown, retained effect regressions and application startup. | -| 4b | Import Loader's project-relative bare-specifier resolver, runtime API detection and fallback when Node internals are absent. | 0 | Retire V19: upstream now detects the internal loader by its available APIs. | Resolve in the config project's package scope, retain subpaths, handle read-only helper creation and test Node 24's older v1 API. | -| 4c | Complete Loader's `EntryChange` / `commit(change)` reporting and remaining changes, and upgrade Include to its 1.1.1 journal implementation in the same PR. Both HMR providers keep finding Include through their existing scans; keep `applyEntryPatches` and `entryListSchema` exports and in-memory no-op commits. | 4a, 4b | Retire V18 (`disabled` expressions), V8 (old refresh implementation) and V13 (`writeTask` type). Partial V11: retire the duplicate patch algorithm and insert-index fix; retain public exports. Partial V14: replace the write queue; retain rejection while the journal contains unsaved changes. Retain/adapt V22, including the schema receiver fix, V23 and the V15 child-expression marker. | Create, move, remove and self-update reports; no false self-disposal writes; later patches address inserted rows; stable ids for anonymous entries; file/runtime conflict handling, patch-owned writes, shutdown results and nested expressions; cloning and config dumps agree with mounting. | -| 4e | Complete vendor HMR's upgrade to 1.1.1, preserving required local path, readiness and explicit-watch behavior. | 2, 3, 4a | Partial V9: adopt upstream ordering and path handling where equivalent; retain explicit readiness and realpath gaps. Retain V1 (no YAML locale imports). | Three-stage module replacement, nested entries, disabled/removed entries, explicit watches and the existing local path cases. | -| 4f | Register the journal Include through the explicit watch adopted in PR 2 and remove the remaining Include scans. | 2, 4c | Adapt V9's config-path matching to the journal Include. | One coordinated refresh per change under either HMR provider; profile watching unchanged. | - -Shared integration differences remain in every affected sync: V2 (manifests), V3 (TypeScript projects), V4/V10 (imports and Node-compatible modules), V5 (build overrides), V7 (API documentation), V16 (published Core source) and V17 (scope). Untouched packages retain them without edits. V6 only loses the stale-epoch guard already supplied upstream; its remaining ownership and cleanup guarantees stay. V15 replaces the old raw/effective-config and event implementation, but the Include child-expression behavior remains local. V22 is not upstreamed by this release, including its schema-bound `simplify` call. - -After PR 0, PRs 1, 2, 3 and 4b have no implementation dependency on one another. Core depends on configuration preparation; it does not depend on Timer. Loader completion and vendor HMR can proceed independently after Core. Include journal adoption is part of Loader completion; PR 4f only connects that Include to PR 2's explicit watches and does not depend on the new vendor HMR replacement algorithm, so PRs 4e and 4f can merge in either order. No single GitHub stack should impose a dependency between unrelated branches. Shared files such as the vendor manifest may still require merge-conflict resolution. - -### Minimum Core compatibility slice - -PR 4a must migrate every caller of the changed Core APIs in the same PR, including callers outside vendor. This is a complete Core upgrade with local patches, paired with selected Loader changes; it is not a claim that Loader already equals 1.0.0-rc.8. - -- Adopt upstream raw `Fiber.config`, `internal/config(fiber, next)`, `internal/commit` and the new `internal/update` arguments and awaited result. Update the config accessors, provider hooks, boot observers and old Include's update listener together. The temporary commit listener writes through the old tree writer until PR 4c. -- Carry the required Loader entry/context awaiting chain, activation-failure observer and isolation changes with Core's `Impl.ctx`, `Impl.isolate`, `committed` and dependent tracking. Update direct implementation readers such as the preset registry. Include Group behavior owned by `vendor/loader/src/config/group.ts`; do not confuse it with the unchanged `vendor/group` entry package. Before retiring V20, ensure volatile updates and every other direct lifecycle-field access use the canonical `fiber.ctx.fiber`. -- Keep Include's nested-entry expressions literal until the owning child resolves them. Keep the namespace field needed by DSH HMR and preserve active volatile references through the agreed config mechanism. Update raw-config access in both HMR implementations without replacing the DSH module-replacement policy. -- Reapply the still-needed effect ownership, reentrant disposal and observer-containment guarantees to the new Core lifecycle. Remove patches already supplied by the imported Core source. Verify the combination with provider/dependent teardown before treating the compatibility slice as complete. - -Module resolution, structured tree-change reporting, journal reconciliation and the new vendor HMR algorithm are not prerequisites for these Core interfaces. Their PRs remain separate unless executable compatibility tests expose an additional dependency; amend this plan before broadening a PR silently. - -### Temporary compatibility code - -PR 4c changes tree mutation producers to `commit(change)` and adopts the journal Include in the same PR: upstream `patch.ts` clones file and inserted entries, while the old Include relied on sharing entry objects with the running tree for its cached data and generated ids. In-memory trees, including `PresetTree`, implement a no-op commit. Until PR 4f, both HMR providers find the journal Include through their existing scans and call its `refresh()`. - -The public patch export may remain an alias; it must not retain a second algorithm. Every temporary divergence has an owning introduction PR and a named removal PR. No runtime version detection or second Core event protocol is planned. - -PR 2 moves Include refresh ownership in one step: both HMR providers expose `watch()`, the existing Include registers its callback, and both providers stop their equivalent Include scans. The upstream HMR source no longer performs that scan, so importing it before migrating the Include consumer would lose config refresh. Moving the complete watch concern first avoids a temporary scan adapter and makes the later module-replacement and persistence decisions independent. - -### Decisions owned by individual PRs - -| Decision | Owning PR | Proposal and unresolved choice | -|---|---|---| -| Active effective config | 1, 4a | Store or expose the object actually supplied to the current instance. Resolving a candidate through `internal/config` must not overwrite it. Decide the publication point and invalidation rules before switching Core; a WeakMap alone does not establish those rules. | -| Persistence outcome | 4c | Use the journal's retry and reconciliation mechanism. Explicit `flush()` and `stop()` reject while changes remain unsaved; later successful retries clear previous errors. Keep this local adaptation minimal. | -| Nested config interpolation | 4a, 4c | Preserve child-owned expression evaluation. Keep the existing Include marker initially; any refinement that permits interpolation of Include's own fields while excluding child lists is a separate behavior change. | -| HMR replacement failure | 4e | Adopt upstream behavior for vendor HMR. Preserve DSH HMR's current source-reload rollback policy in this series; replacing that policy requires its own proposal and tests. | -| Effect cleanup guarantees | 4a | Retain guarantees not supplied by rc.11. Further upstreaming or retirement is separate work, not a prerequisite for reaching the release baseline. | - -## Acceptance criteria - -- Every implementation PR leaves supported applications usable and passes checks appropriate to its changes. A later PR may not be required to repair an earlier PR's startup, types, persistence or teardown. -- Each source-vendor update follows the [sync procedure](README.md#sync-procedure), including `pnpm run test` and `pnpm run build`, and updates its own manifest, local-modification log, affected documentation and generated references. The [pre-push workflow](../.agents/skills/dsh-pre-push-checks/SKILL.md) selects additional focused checks. Documentation freshness is not deferred to a final cleanup PR. -- Every changed behavior has an explicit decision and focused regression coverage. Tests for an intentionally superseded behavior change with its implementation and explain the new expectation. -- Once all implementation PRs are complete, Core, Loader, Include, vendor HMR and Timer use their selected rc.11-tag sources with enumerated remaining patches. Group, logger-console, Cosmokit and Schemastery keep their existing pins. All temporary writer and config adapters have been removed or reduced to the agreed permanent accessors. -- Final integration verification exercises settings and volatile updates, profile recomposition, both HMR providers, Include persistence and representative built application launches. This is an acceptance checkpoint, not an otherwise empty PR. - -## Risks - -PR 4a remains the largest necessary compatibility change: Core's service lifetime and configuration model cross repository package boundaries. The minimum slice is based on source inspection, not a completed migration experiment. Focused tests may identify additional inseparable changes. - -Intermediate Loader, Include and HMR copies intentionally combine an old baseline with documented upstream slices. Missing slice records would make a later sync unsafe. Deferred config, candidate validation, journal writes and asynchronous disposal must be tested in combination rather than inferred from independently passing unit cases. A new Include's type dependency on HMR must not force both concrete providers into the same incompatible `Context.hmr` declaration. From e89d2b031fe23b8bfe1386bc49db7516b1995cc5 Mon Sep 17 00:00:00 2001 From: Dudu-0223 Date: Sat, 7 Nov 2026 01:03:59 +0800 Subject: [PATCH 42/44] fix(ui): show subagent descriptions above names without arrows (#7730) * fix(ui): show subagent descriptions above names without disclosure arrows * fix(ui): narrow subagent result cards * Revert "fix(ui): narrow subagent result cards" This reverts commit 23a5a353ac9daf0e322f28e19c9d2c2cfbb15acb. * test(web): sample agent row geometry in one browser task --- apps/web/tests/tool-details.e2e.ts | 10 ++++++ packages/client/locale/src/locales/de.ts | 2 -- packages/client/locale/src/locales/en.ts | 2 -- packages/client/locale/src/locales/es.ts | 2 -- packages/client/locale/src/locales/fr.ts | 2 -- packages/client/locale/src/locales/ja.ts | 2 -- packages/client/locale/src/locales/pt.ts | 2 -- packages/client/locale/src/locales/ru.ts | 2 -- packages/client/locale/src/locales/zh.ts | 2 -- packages/client/ui-tool/README.i18n.yaml | 4 +-- packages/client/ui-tool/README.md | 2 +- packages/client/ui-tool/README.zh.md | 2 +- .../tool/components/ToolDetails.module.css | 36 ------------------- .../client/tool/components/ToolDetails.tsx | 22 +++--------- .../tool/models/control-details-model.ts | 11 ++++-- .../tests/details-models.client.spec.ts | 10 +++--- .../ui-tool/tests/details-row.client.spec.tsx | 21 ++++------- snapshots/web/tool-details/ui.expected.md | 18 +++------- 18 files changed, 43 insertions(+), 109 deletions(-) diff --git a/apps/web/tests/tool-details.e2e.ts b/apps/web/tests/tool-details.e2e.ts index 8a51e66d44..93963c070e 100644 --- a/apps/web/tests/tool-details.e2e.ts +++ b/apps/web/tests/tool-details.e2e.ts @@ -62,6 +62,16 @@ describe.skipIf(MODE === 'record')('web e2e: compact Tool details', () => { expect(await row.locator('[class*="ioCard"]').count()).toBe(0) expect(await row.getByRole('button', { name: 'Inspect', exact: true }).count()).toBe(1) } + const agentRow = page.locator('[data-tool="list_agents"]').getByRole('listitem').first() + expect(await agentRow.getByRole('button').count()).toBe(0) + expect(await agentRow.locator('svg').count()).toBe(0) + // Read both positions in one browser task so scrolling cannot move the viewport between measurements. + const agentLayout = await agentRow.evaluate((element) => { + const description = element.querySelector('[class*="heading"] > span')!.getBoundingClientRect() + const name = element.querySelector('[class*="subtitle"]')!.getBoundingClientRect() + return { descriptionBottom: description.bottom, nameTop: name.top } + }) + expect(agentLayout.descriptionBottom).toBeLessThanOrEqual(agentLayout.nameTop) const changedTodos = page.locator('[data-tool="todo_write"]').nth(1) await changedTodos.getByText('1 added · 1 updated · 1 removed', { exact: true }).waitFor() for (const summary of await page.locator('[data-tool] details > summary').all()) await summary.click() diff --git a/packages/client/locale/src/locales/de.ts b/packages/client/locale/src/locales/de.ts index cb0bb45579..dd88cf6fc4 100644 --- a/packages/client/locale/src/locales/de.ts +++ b/packages/client/locale/src/locales/de.ts @@ -33,8 +33,6 @@ export const de = { 'more': 'Mehr', 'collapse': 'Einklappen', 'expand': 'Ausklappen', - 'subtitle.expand': 'Beschreibung für {name} ausklappen', - 'subtitle.collapse': 'Beschreibung für {name} einklappen', 'back': 'Zurück', 'brand.localBuild': 'Lokaler DSH-Build', 'workspace.defaultName': 'Standardarbeitsbereich', diff --git a/packages/client/locale/src/locales/en.ts b/packages/client/locale/src/locales/en.ts index 47a03d69ef..b649bbc3dc 100644 --- a/packages/client/locale/src/locales/en.ts +++ b/packages/client/locale/src/locales/en.ts @@ -32,8 +32,6 @@ export const en = { 'more': 'More', 'collapse': 'Collapse', 'expand': 'Expand', - 'subtitle.expand': 'Expand description for {name}', - 'subtitle.collapse': 'Collapse description for {name}', 'back': 'Back', 'brand.localBuild': 'DSH Local Build', 'workspace.defaultName': 'Default workspace', diff --git a/packages/client/locale/src/locales/es.ts b/packages/client/locale/src/locales/es.ts index 4fba984885..7eb95374e1 100644 --- a/packages/client/locale/src/locales/es.ts +++ b/packages/client/locale/src/locales/es.ts @@ -33,8 +33,6 @@ export const es = { 'more': 'Más', 'collapse': 'Contraer', 'expand': 'Expandir', - 'subtitle.expand': 'Expandir la descripción de {name}', - 'subtitle.collapse': 'Contraer la descripción de {name}', 'back': 'Atrás', 'brand.localBuild': 'Compilación local de DSH', 'workspace.defaultName': 'Espacio de trabajo predeterminado', diff --git a/packages/client/locale/src/locales/fr.ts b/packages/client/locale/src/locales/fr.ts index f0e5030f41..c57f1a3b11 100644 --- a/packages/client/locale/src/locales/fr.ts +++ b/packages/client/locale/src/locales/fr.ts @@ -33,8 +33,6 @@ export const fr = { 'more': 'Plus', 'collapse': 'Réduire', 'expand': 'Développer', - 'subtitle.expand': 'Développer la description de {name}', - 'subtitle.collapse': 'Réduire la description de {name}', 'back': 'Retour', 'brand.localBuild': 'Version locale de DSH', 'workspace.defaultName': 'Espace de travail par défaut', diff --git a/packages/client/locale/src/locales/ja.ts b/packages/client/locale/src/locales/ja.ts index 2b70cbb156..1186838c31 100644 --- a/packages/client/locale/src/locales/ja.ts +++ b/packages/client/locale/src/locales/ja.ts @@ -33,8 +33,6 @@ export const ja = { 'more': 'その他', 'collapse': '折りたたむ', 'expand': '展開', - 'subtitle.expand': '{name} の説明を展開', - 'subtitle.collapse': '{name} の説明を折りたたむ', 'back': '戻る', 'brand.localBuild': 'DSHローカルビルド', 'workspace.defaultName': 'デフォルトワークスペース', diff --git a/packages/client/locale/src/locales/pt.ts b/packages/client/locale/src/locales/pt.ts index adf711152d..75cdb19c29 100644 --- a/packages/client/locale/src/locales/pt.ts +++ b/packages/client/locale/src/locales/pt.ts @@ -33,8 +33,6 @@ export const pt = { 'more': 'Mais', 'collapse': 'Recolher', 'expand': 'Expandir', - 'subtitle.expand': 'Expandir descrição de {name}', - 'subtitle.collapse': 'Recolher descrição de {name}', 'back': 'Voltar', 'brand.localBuild': 'Versão local do DSH', 'workspace.defaultName': 'Espaço de trabalho padrão', diff --git a/packages/client/locale/src/locales/ru.ts b/packages/client/locale/src/locales/ru.ts index e75620fe21..a8a6d829da 100644 --- a/packages/client/locale/src/locales/ru.ts +++ b/packages/client/locale/src/locales/ru.ts @@ -33,8 +33,6 @@ export const ru = { 'more': 'Ещё', 'collapse': 'Свернуть', 'expand': 'Развернуть', - 'subtitle.expand': 'Развернуть описание {name}', - 'subtitle.collapse': 'Свернуть описание {name}', 'back': 'Назад', 'brand.localBuild': 'Локальная сборка DSH', 'workspace.defaultName': 'Рабочая область по умолчанию', diff --git a/packages/client/locale/src/locales/zh.ts b/packages/client/locale/src/locales/zh.ts index 6350caae3c..11ff789b8a 100644 --- a/packages/client/locale/src/locales/zh.ts +++ b/packages/client/locale/src/locales/zh.ts @@ -30,8 +30,6 @@ export const zh = { 'more': '更多', 'collapse': '收起', 'expand': '展开', - 'subtitle.expand': '展开{name}的描述', - 'subtitle.collapse': '收起{name}的描述', 'back': '返回', 'brand.localBuild': 'DSH 本地构建', 'workspace.defaultName': '默认工作区', diff --git a/packages/client/ui-tool/README.i18n.yaml b/packages/client/ui-tool/README.i18n.yaml index 4182c93cfe..59dfd47110 100644 --- a/packages/client/ui-tool/README.i18n.yaml +++ b/packages/client/ui-tool/README.i18n.yaml @@ -15,8 +15,8 @@ en: d152484eb41ac6b4 zh: 09388d293f9be9cb /deepseek-ai-dsh-client-ui-tool/use-this-package: - en: 09231e7cc6515a01 - zh: c306bb141cf9bf9d + en: 4d8cb38024095705 + zh: 1baf6fa42f3fb80f /deepseek-ai-dsh-client-ui-tool/use-this-package/registering-a-business-tool-view: en: e458aff61e634190 zh: 650e417f450b3edf diff --git a/packages/client/ui-tool/README.md b/packages/client/ui-tool/README.md index b460da0f8f..448a4e197b 100644 --- a/packages/client/ui-tool/README.md +++ b/packages/client/ui-tool/README.md @@ -25,7 +25,7 @@ English | [中文](README.zh.md) ## Use this package -Tool calls appear in the conversation as cards: a root call tree with its nested subcalls, each atomic call rendered by its owning view. Every lifecycle state retains the tool's ordinary business glyph; failure and interruption remain explicit through the frozen call/result state, accessible status text, and failure summary. Users can open files or inspect calls through the Host callbacks. A collapsed `web_fetch` row links its http(s) URL, which opens in a new browser tab. Control receipts display the agent name supplied by the recorded call, or the agent ID in historical calls. Text agent lists show each agent name, or the historical ID, above a single-line description, with compact spacing and recorded status. Click the description to expand or collapse its full text; its accessible name follows the selected locale. +Tool calls appear in the conversation as cards: a root call tree with its nested subcalls, each atomic call rendered by its owning view. Every lifecycle state retains the tool's ordinary business glyph; failure and interruption remain explicit through the frozen call/result state, accessible status text, and failure summary. Users can open files or inspect calls through the Host callbacks. A collapsed `web_fetch` row links its http(s) URL, which opens in a new browser tab. Control receipts display the agent name supplied by the recorded call, or the agent ID in historical calls. Agent lists show the full description above the name, or the historical ID, with compact spacing and recorded status. Both lines wrap as needed and have no disclosure arrows; rows without a description show only the name. Settled foreground commands with a nonzero exit code, a terminating signal, or an explicit missing exit code keep their description in the ordinary gray text and show a neutral capsule with a red dot immediately after it. The capsule displays only the recorded code, signal, or localized missing-code status and stays on the same line; the description truncates with an ellipsis to reserve its space. Running, successful, and background calls omit it. Generic execution errors and stopped summaries retain their error and warning colors, including on hover. diff --git a/packages/client/ui-tool/README.zh.md b/packages/client/ui-tool/README.zh.md index 3b7811a823..37757c47a7 100644 --- a/packages/client/ui-tool/README.zh.md +++ b/packages/client/ui-tool/README.zh.md @@ -25,7 +25,7 @@ kind: "package-reference" ## 使用本包 -工具调用在对话中显示为卡片：一个根调用树带其嵌套子调用，每个原子调用由所属视图渲染。所有生命周期状态都保留工具的普通业务图标；失败与中断仍通过冻结调用／结果状态、无障碍状态文本和失败摘要明确表达。用户可通过宿主回调打开文件或检查调用。折叠的 `web_fetch` 行把其 http(s) URL 显示为链接，在新浏览器标签页中打开。控制工具回执显示记录的调用所提供的智能体名称，历史调用则显示智能体 ID。文本形式的智能体列表以紧凑间距展示智能体名称（历史记录为 ID）、下方的单行描述及记录中的状态。点击描述可展开或收起全文；其无障碍名称使用当前语言。 +工具调用在对话中显示为卡片：一个根调用树带其嵌套子调用，每个原子调用由所属视图渲染。所有生命周期状态都保留工具的普通业务图标；失败与中断仍通过冻结调用／结果状态、无障碍状态文本和失败摘要明确表达。用户可通过宿主回调打开文件或检查调用。折叠的 `web_fetch` 行把其 http(s) URL 显示为链接，在新浏览器标签页中打开。控制工具回执显示记录的调用所提供的智能体名称，历史调用则显示智能体 ID。智能体列表以紧凑间距在上方展示完整描述，下方展示名称（历史记录为 ID），并保留记录中的状态。两行均按需换行，不带展开箭头；没有描述时只显示名称。 已结束的前台命令若返回非零退出码、终止信号，或明确记录为无退出码，描述保持普通灰色，后面紧接带红点的中性胶囊。胶囊仅显示记录的退出码、信号或本地化的“未正常退出”状态，并保持在同一行；描述超出可用宽度时显示省略号，为胶囊预留空间。运行中、成功和后台调用不显示胶囊。通用执行错误与停止摘要仍保留错误色和警告色，悬停时也不改变。 diff --git a/packages/client/ui-tool/src/client/tool/components/ToolDetails.module.css b/packages/client/ui-tool/src/client/tool/components/ToolDetails.module.css index 0180f9fa0d..44d9f2ef8d 100644 --- a/packages/client/ui-tool/src/client/tool/components/ToolDetails.module.css +++ b/packages/client/ui-tool/src/client/tool/components/ToolDetails.module.css @@ -188,42 +188,6 @@ padding: 6px 10px 4px; } -.subtitleToggle { - display: flex; - align-items: flex-start; - gap: 4px; - width: 100%; - padding: 0; - border: 0; - background: transparent; - color: var(--dsw-alias-label-tertiary); - font: inherit; - font-size: 12px; - text-align: left; - cursor: pointer; -} - -.subtitleToggle > span { - min-width: 0; - overflow: hidden; - white-space: nowrap; - text-overflow: ellipsis; -} - -.subtitleToggle > svg { - flex: none; - margin-top: 3px; -} - -.subtitleToggle[aria-expanded='true'] > span { - white-space: pre-wrap; - overflow-wrap: anywhere; -} - -.subtitleToggle[aria-expanded='true'] > svg { - transform: rotate(90deg); -} - .description { margin: 6px 0; color: var(--dsw-alias-label-caption); diff --git a/packages/client/ui-tool/src/client/tool/components/ToolDetails.tsx b/packages/client/ui-tool/src/client/tool/components/ToolDetails.tsx index 84ee30b93b..65e4db7c84 100644 --- a/packages/client/ui-tool/src/client/tool/components/ToolDetails.tsx +++ b/packages/client/ui-tool/src/client/tool/components/ToolDetails.tsx @@ -1,5 +1,4 @@ /** Compact, read-only fields and lists for recorded Tool results. */ -import { useState } from 'react' import { CodeBlock, MarkdownText, IconCheckOutlineRegular, IconChevronRightOutlineRegular, IconPlayOutlineRegular, } from '@deepseek-ai/dsh-client-ui-primitives' @@ -28,7 +27,7 @@ export interface ToolDetailItem { /** Localized display data shared by the built-in detail cards. */ export interface ToolDetailsModel { items: readonly ToolDetailItem[] - /** Tight rows with single-line subtitles that can expand in place. */ + /** Tight spacing for recorded entity rows. */ compact?: boolean summary?: string /** Header text while expanded, omitting a receipt status shown in the body. */ @@ -43,9 +42,8 @@ interface DetailContentProps { onOpenFile?: ((path: string, options?: OpenFileOptions) => void) | undefined } -function DetailItem({ item, compact, t, onOpenFile }: { item: ToolDetailItem; compact?: boolean | undefined } & DetailContentProps) { +function DetailItem({ item, t, onOpenFile }: { item: ToolDetailItem } & DetailContentProps) { const status = item.status - const [expanded, setExpanded] = useState(false) return (
  • {item.title !== undefined && ( @@ -77,19 +75,7 @@ function DetailItem({ item, compact, t, onOpenFile }: { item: ToolDetailItem; co )} )} - {item.subtitle !== undefined && (compact ? ( - - ) :
    {item.subtitle}
    )} + {item.subtitle !== undefined &&
    {item.subtitle}
    } {item.description !== undefined &&

    {item.description}

    } {item.fields.length > 0 && (
    @@ -144,7 +130,7 @@ export function ToolDetails({ model, hasInspect = false, t, onOpenFile }: { {model.caption !== undefined &&
    {model.caption}
    } {model.items.length === 0 ?

    {model.empty}

    : (
      - {model.items.map((item, index) => )} + {model.items.map((item, index) => )}
    )} {model.unchanged !== undefined && ( diff --git a/packages/client/ui-tool/src/client/tool/models/control-details-model.ts b/packages/client/ui-tool/src/client/tool/models/control-details-model.ts index ef6fb9a8a5..c7f73005b0 100644 --- a/packages/client/ui-tool/src/client/tool/models/control-details-model.ts +++ b/packages/client/ui-tool/src/client/tool/models/control-details-model.ts @@ -29,7 +29,14 @@ function teamTaskItems(value: unknown, t: DetailTranslate): DetailItem[] { } function agentList(text: string, json: unknown, t: DetailTranslate): ToolDetailsModel | null { - if (Array.isArray(json)) return detailList(inspectionItems(json, t), t('detail.agents.count', { count: json.length }), t) + if (Array.isArray(json)) { + const items = inspectionItems(json, t).map((item) => { + if (item.title === undefined || item.description === undefined) return item + const { title, description, ...rest } = item + return { ...rest, title: description, subtitle: title } + }) + return { ...detailList(items, t('detail.agents.count', { count: json.length }), t), compact: true } + } if (text === '(no subagents)') return detailList([], t('detail.agents.count', { count: 0 }), t) const items: DetailItem[] = [] for (const line of text.split('\n')) { @@ -39,7 +46,7 @@ function agentList(text: string, json: unknown, t: DetailTranslate): ToolDetails const [, agent, state, title] = match if (agent === undefined || state === undefined) return null items.push({ - title: agent, ...(title === undefined ? {} : { subtitle: title }), + title: title || agent, ...(title ? { subtitle: agent } : {}), badge: detailBadge(state, t), fields: [], }) diff --git a/packages/client/ui-tool/tests/details-models.client.spec.ts b/packages/client/ui-tool/tests/details-models.client.spec.ts index 7147aff0f7..bbf9f5f2e8 100644 --- a/packages/client/ui-tool/tests/details-models.client.spec.ts +++ b/packages/client/ui-tool/tests/details-models.client.spec.ts @@ -93,20 +93,20 @@ describe('control detail adapters', () => { const agents = details('list_agents', 'lead [running] — root agent\nreviewer/checker [inactive] — Check tests\nold-id [unavailable]') expect(agents?.summary).toBe('3 agents') expect(agents?.items).toEqual([ - { title: 'lead', subtitle: 'root agent', badge: { label: 'Running', tone: 'info' }, fields: [] }, - { title: 'reviewer/checker', subtitle: 'Check tests', badge: { label: 'Inactive', tone: 'warning' }, fields: [] }, + { title: 'root agent', subtitle: 'lead', badge: { label: 'Running', tone: 'info' }, fields: [] }, + { title: 'Check tests', subtitle: 'reviewer/checker', badge: { label: 'Inactive', tone: 'warning' }, fields: [] }, { title: 'old-id', badge: { label: 'Unavailable', tone: 'neutral' }, fields: [] }, ]) const historical = details('list_agents', 'a1 [running] — Build\na2 [ready] parent=a1 depth=1 — Review\na3 [diagnostic: corrupt] parent=a1 depth=1', { scope: 'descendants' }) expect(historical?.summary).toBe('3 agents') expect(historical?.items).toEqual([ - { title: 'a1', subtitle: 'Build', badge: { label: 'Running', tone: 'info' }, fields: [] }, - { title: 'a2', subtitle: 'Review', badge: { label: 'Ready', tone: 'neutral' }, fields: [] }, + { title: 'Build', subtitle: 'a1', badge: { label: 'Running', tone: 'info' }, fields: [] }, + { title: 'Review', subtitle: 'a2', badge: { label: 'Ready', tone: 'neutral' }, fields: [] }, { title: 'a3', badge: { label: 'diagnostic: corrupt', tone: 'neutral' }, fields: [] }, ]) const structured = details('list_agents', JSON.stringify([{ name: 'lead', status: 'running', description: 'root agent' }, { name: 'reviewer', status: 'inactive' }])) expect(structured?.items).toEqual([ - { title: 'lead', description: 'root agent', badge: { label: 'Running', tone: 'info' }, fields: [] }, + { title: 'root agent', subtitle: 'lead', badge: { label: 'Running', tone: 'info' }, fields: [] }, { title: 'reviewer', badge: { label: 'Inactive', tone: 'warning' }, fields: [] }, ]) expect(details('list_agents', 'reviewer running')).toBeNull() diff --git a/packages/client/ui-tool/tests/details-row.client.spec.tsx b/packages/client/ui-tool/tests/details-row.client.spec.tsx index b5dde4a667..e21206109f 100644 --- a/packages/client/ui-tool/tests/details-row.client.spec.tsx +++ b/packages/client/ui-tool/tests/details-row.client.spec.tsx @@ -153,26 +153,17 @@ describe('todosDetail', () => { }) describe('DetailsRow', () => { - it('puts agent names first and lets a compact description expand and collapse', () => { + it('shows the full agent description above its name without a disclosure', () => { const description = 'Verify every construction bound. '.repeat(20) const block = { ...result('list_agents', null), content: [{ type: 'text' as const, text: `reviewer/checker [running] — ${description}` }] } const model = detailsCardModel(block, t, 'en')! render() const name = screen.getByText('reviewer/checker') - const toggle = screen.getByRole('button', { name: 'Expand description for reviewer/checker' }) - expect(name.compareDocumentPosition(toggle) & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy() - expect(toggle.textContent).toBe(description) - fireEvent.click(toggle) - expect(screen.getByRole('button', { name: 'Collapse description for reviewer/checker' }).getAttribute('aria-expanded')).toBe('true') - fireEvent.click(toggle) - expect(toggle.getAttribute('aria-expanded')).toBe('false') - }) - - it('uses an unqualified accessible name when a compact description has no title', () => { - render() - const toggle = screen.getByRole('button', { name: 'Expand' }) - fireEvent.click(toggle) - expect(screen.getByRole('button', { name: 'Collapse' })).toBe(toggle) + const title = screen.getByText(description.trim()) + expect(title.compareDocumentPosition(name) & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy() + expect(screen.getByRole('listitem').textContent).toBe(`${description}Runningreviewer/checker`) + expect(screen.queryByRole('button')).toBeNull() + expect(screen.getByRole('listitem').querySelector('svg')).toBeNull() }) it('expands fields with the keyboard and keeps Inspect available', () => { diff --git a/snapshots/web/tool-details/ui.expected.md b/snapshots/web/tool-details/ui.expected.md index a069e23631..e55e355f3f 100644 --- a/snapshots/web/tool-details/ui.expected.md +++ b/snapshots/web/tool-details/ui.expected.md @@ -477,15 +477,9 @@ - button "List subagents 3 agents" [expanded] - text: Recorded result - list: - - listitem: - - text: agent-demo-review Running - - button "Expand description for agent-demo-review": 检查工具卡片布局 - - listitem: - - text: agent-demo-docs Idle - - button "Expand description for agent-demo-docs": 整理工具展示清单 - - listitem: - - text: agent-demo-icons Ready - - button "Expand description for agent-demo-icons": 检查状态图标 + - listitem: 检查工具卡片布局 Running agent-demo-review + - listitem: 整理工具展示清单 Idle agent-demo-docs + - listitem: 检查状态图标 Ready agent-demo-icons - button "Inspect" - button "Send message agent-demo-review" [expanded] - text: Recorded result @@ -658,8 +652,7 @@ - term: Role - definition: lead - listitem: - - text: ui-review Running - - paragraph: 检查工具卡片的可读性 + - text: 检查工具卡片的可读性 Running ui-review - term: ID - definition: agent-demo-review - term: Role @@ -669,8 +662,7 @@ - term: Context - definition: fresh - listitem: - - text: docs Inactive - - paragraph: 整理工具清单 + - text: 整理工具清单 Inactive docs - term: ID - definition: agent-demo-docs - term: Role From 42abd55ef224fa03d52f561d9567935eb1428154 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Sat, 7 Nov 2026 01:31:34 +0800 Subject: [PATCH 43/44] test(web): stabilize history paging e2e against auto-prefetch --- apps/web/tests/chat-scroll-contract.e2e.ts | 87 +++++++++++++++++----- apps/web/tests/preview-boot.e2e.ts | 9 ++- 2 files changed, 76 insertions(+), 20 deletions(-) diff --git a/apps/web/tests/chat-scroll-contract.e2e.ts b/apps/web/tests/chat-scroll-contract.e2e.ts index 1a0c751c8d..bccac447e1 100644 --- a/apps/web/tests/chat-scroll-contract.e2e.ts +++ b/apps/web/tests/chat-scroll-contract.e2e.ts @@ -367,13 +367,71 @@ async function flingTranscript(page: Page, deltaY: number): Promise { await nextPaint(page) } -async function wheelToHistoryStart(page: Page): Promise { - for (let attempt = 0; attempt < 12; attempt += 1) { - if ((await scrollGeometry(page)).scrollTop <= 1) break - await wheelTranscript(page, -2_400) +/** + * Hold paging responses so auto-prefetch cannot publish a page mid-gesture. + * @param page - Chat page under test. + * @returns an idempotent release that resumes held responses and awaits them. + */ +async function holdHistoryPages(page: Page): Promise<() => Promise> { + const gate = Promise.withResolvers() + const pending = new Set>() + const hold = (route: Route): Promise => { + const work = gate.promise.then(() => route.continue()) + pending.add(work) + return work.finally(() => { pending.delete(work) }) } - await expect.poll(async () => (await scrollGeometry(page)).scrollTop, { timeout: 10_000 }) - .toBeLessThanOrEqual(1) + await page.route('**/api/session/page', hold) + let released = false + return async () => { + if (released) return + released = true + gate.resolve(undefined) + // Resume held requests before removing the handler: unregistering with a + // hold still pending lets Playwright handle that route itself, after which + // the delayed continue() fails with "Route is already handled". + while (pending.size > 0) await Promise.all([...pending]) + await page.unroute('**/api/session/page', hold) + } +} + +/** + * Wait until a released page has landed and the history control stopped changing. + * @param page - Chat page under test. + * @returns the settled rendered row count. + */ +async function settleHistoryPaging(page: Page): Promise { + let rows = -1 + await expect.poll(async () => { + if (await page.locator('[data-chat-history]').getAttribute('data-chat-history') === 'loading') return false + const current = await loadedFlowRows(page) + const stable = current === rows + rows = current + return stable + }, { timeout: 30_000 }).toBe(true) + await nextPaint(page) + return rows +} + +/** + * Wheel back to the start of the loaded window. + * @param page - Chat page under test. + * @param options - holdPaging keeps auto-prefetch from publishing a page that re-anchors the viewport mid-scroll. + */ +async function wheelToHistoryStart(page: Page, options: { holdPaging?: boolean } = {}): Promise { + const release = options.holdPaging === true ? await holdHistoryPages(page) : undefined + try { + for (let attempt = 0; attempt < 12; attempt += 1) { + if ((await scrollGeometry(page)).scrollTop <= 1) break + await wheelTranscript(page, -2_400) + } + await expect.poll(async () => (await scrollGeometry(page)).scrollTop, { timeout: 10_000 }) + .toBeLessThanOrEqual(1) + } finally { + await release?.() + } + // A held page is released at the history edge, where the client publishes it + // as soon as it arrives; settle that prepend before the caller measures. + if (options.holdPaging === true) await settleHistoryPaging(page) } async function wheelUntilMounted(page: Page, selector: string, deltaY: number): Promise { @@ -478,14 +536,7 @@ async function expectMarkerAboveComposer(page: Page, marker: string): Promise { // Auto-prefetch may start while scrolling to the button. Hold its response until the reading anchor is captured. - const gate = Promise.withResolvers() - const pending = new Set>() - const hold = (route: Route): Promise => { - const work = gate.promise.then(() => route.continue()) - pending.add(work) - return work.finally(() => { pending.delete(work) }) - } - await page.route('**/api/session/page', hold) + const release = await holdHistoryPages(page) try { await wheelToHistoryStart(page) const history = page.locator('[data-chat-history]') @@ -495,7 +546,7 @@ async function loadEarlierWithAnchor(page: Page): Promise { const anchor = await visibleFlowAnchor(page) const before = await loadedFlowRows(page) await older.click() - gate.resolve(undefined) + await release() await expect.poll(async () => ( await loadedFlowRows(page) > before && await history.getAttribute('data-chat-history') === 'idle' ), { timeout: 30_000 }).toBe(true) @@ -507,9 +558,7 @@ async function loadEarlierWithAnchor(page: Page): Promise { await expectSameFlowTop(page, anchor) return true } finally { - gate.resolve(undefined) - await page.unroute('**/api/session/page', hold) - await Promise.all([...pending]) + await release() } } @@ -1754,7 +1803,7 @@ describe('web e2e: long Chat scroll contract', () => { ) await loadEarlierWithAnchor(world.page) await loadEarlierWithAnchor(world.page) - await wheelToHistoryStart(world.page) + await wheelToHistoryStart(world.page, { holdPaging: true }) await wheelTranscript(world.page, 1_300) const sessionAnchor = await visibleFlowAnchor(world.page) diff --git a/apps/web/tests/preview-boot.e2e.ts b/apps/web/tests/preview-boot.e2e.ts index f1ea40b25c..72ce0e6872 100644 --- a/apps/web/tests/preview-boot.e2e.ts +++ b/apps/web/tests/preview-boot.e2e.ts @@ -551,7 +551,14 @@ async function bootPreview(site: Site, manifest: ImageManifest, browser: Browser await catalog.getByRole('treeitem', { name: /Continue preview verification/ }).waitFor() await catalog.press('Escape') - await page.getByRole('button', { name: 'Load earlier', exact: true }).click() + // Auto-prefetch may already have published the page, and a pointer click can + // land on whatever moved under it; activate the control only while it still + // offers a page, then wait for the oldest seeded row. + const historyEdge = page.locator('[data-chat-history]') + await historyEdge.getByRole('button').evaluate((button) => { + if (!(button instanceof HTMLButtonElement)) throw new Error('history control is not a button') + if (!button.disabled) button.click() + }) await page.getByText(SHOWCASE_OLDEST, { exact: true }).waitFor({ timeout: 15_000 }) await requestLazyChunk(page, site, manifest) From f8237c051ec95acf055777d482e8e44fd5d5e9d4 Mon Sep 17 00:00:00 2001 From: creatixchu Date: Sat, 7 Nov 2026 11:34:09 +0800 Subject: [PATCH 44/44] Describe mouseMove as a request to move to the target --- packages/experimental/cua-macos/README.md | 2 +- packages/experimental/cua-macos/README.zh.md | 2 +- packages/experimental/cua/src/guidance.ts | 2 +- packages/experimental/cua/src/types.ts | 7 ++++--- snapshots/session/cua-native/system-prompt.expected.md | 2 +- snapshots/session/cua-ptc/system-prompt.expected.md | 2 +- .../session/ptc-script-retained/system-prompt.expected.md | 2 +- 7 files changed, 10 insertions(+), 9 deletions(-) diff --git a/packages/experimental/cua-macos/README.md b/packages/experimental/cua-macos/README.md index 9a388e5d0c..46b035112e 100644 --- a/packages/experimental/cua-macos/README.md +++ b/packages/experimental/cua-macos/README.md @@ -210,7 +210,7 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request a mouse move to the target without pressing a button; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. - click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua-macos/README.zh.md b/packages/experimental/cua-macos/README.zh.md index 1ae598bdc9..341796b8c4 100644 --- a/packages/experimental/cua-macos/README.zh.md +++ b/packages/experimental/cua-macos/README.zh.md @@ -210,7 +210,7 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request a mouse move to the target without pressing a button; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. - click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua/src/guidance.ts b/packages/experimental/cua/src/guidance.ts index 0ce5da8944..745ccfe9be 100644 --- a/packages/experimental/cua/src/guidance.ts +++ b/packages/experimental/cua/src/guidance.ts @@ -32,7 +32,7 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request a mouse move to the target without pressing a button; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. - click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/packages/experimental/cua/src/types.ts b/packages/experimental/cua/src/types.ts index 9f13f5b3c4..b84663f9e2 100644 --- a/packages/experimental/cua/src/types.ts +++ b/packages/experimental/cua/src/types.ts @@ -191,12 +191,13 @@ export interface CuaWindow extends CuaSurface { */ getScreenshot(options?: CuaObservationOptions): Promise /** - * Hover over an element or a point without pressing a button. The provider reports the route and whether the system pointer moved. + * Request a mouse move to an element or a point without pressing a button. + * The provider reports the route and whether the system pointer moved. * Returning and subsequent observations do not restore the previous pointer position. - * Other input and application changes can interrupt hover; observe before continuing. + * Other input and application changes can interrupt the interaction; observe before continuing. * @param target - current element id or screenshot-local [x,y]. * @param options - coordinate mode; applies only to [x,y]. - * @returns dispatch evidence and the selected route; observe the resulting UI after its hover delay. + * @returns dispatch evidence and the selected route; observe the resulting UI after any UI delay. */ mouseMove(target: CuaElementId | readonly [number, number], options?: CuaCoordinateOptions): Promise /** diff --git a/snapshots/session/cua-native/system-prompt.expected.md b/snapshots/session/cua-native/system-prompt.expected.md index a67bc45042..0b9c1b29f6 100644 --- a/snapshots/session/cua-native/system-prompt.expected.md +++ b/snapshots/session/cua-native/system-prompt.expected.md @@ -67,7 +67,7 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request a mouse move to the target without pressing a button; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. - click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/snapshots/session/cua-ptc/system-prompt.expected.md b/snapshots/session/cua-ptc/system-prompt.expected.md index 000bbe3322..587436227a 100644 --- a/snapshots/session/cua-ptc/system-prompt.expected.md +++ b/snapshots/session/cua-ptc/system-prompt.expected.md @@ -69,7 +69,7 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request a mouse move to the target without pressing a button; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. - click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window. diff --git a/snapshots/session/ptc-script-retained/system-prompt.expected.md b/snapshots/session/ptc-script-retained/system-prompt.expected.md index 09031991fc..dd88fa8d9e 100644 --- a/snapshots/session/ptc-script-retained/system-prompt.expected.md +++ b/snapshots/session/ptc-script-retained/system-prompt.expected.md @@ -69,7 +69,7 @@ Element ids are opaque strings from the latest successful tree; the object facad ## Simulated mouse Use an explicit window for screenshots and pointer input. - getScreenshot(options?: { emit?: boolean }) -> { bytes: Uint8Array, mimeType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif', pixelWidth: number, pixelHeight: number, windowId: number } — capture the selected window as one encoded image; bytes is a JavaScript byte array, and mimeType identifies its encoding, so do not assume PNG. -- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — hover over the target without pressing; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. +- mouseMove(target: string | [number, number], options?: { relative?: boolean }) -> { dispatched: boolean, foregroundChanged?: boolean | null, hoverRoute: 'accessibility' | 'system-pointer', systemPointerMoved: boolean } — request a mouse move to the target without pressing a button; inspect hoverRoute and systemPointerMoved, and expect no automatic pointer restoration. Default hover requires both AXShowAlternateUI and AXShowDefaultUI on an enabled element or ancestor. Without an AX route it returns HOVER_UNAVAILABLE before sending input. Prefer click on a visible control or performAction with an advertised action. For web content without an AX hover action, use browser tools. Only systemPointerHover: true permits moving the visible cursor. If this returns FOREGROUND_REQUIRED, activate the app and check that the point is unobscured. - click(target: string | [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', clickCount?: 1 | 2, modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], holdMs?: number, relative?: boolean }) -> ActionResult — click an element id or point, or perform a left-button double-click; holdMs controls the press duration of each press. click is a Window method: select a window with const win = await app.getWindow(windowId), then call win.click(target). For a background control’s default action, use app.invoke(id). For a mouse click on a background app, call app.activate() first and re-read the target. Both target forms send mouse-down and mouse-up events. click(id) uses the center of the full element bounds. If that center is outside the window, take a fresh win.getScreenshot() and use click([x, y]) on a visible part of the control in screenshot pixels. A background app can ignore the events even when dispatched is true. Verify the expected effect, such as a changed selection, before typing or retrying. - scroll(options: { x: number, y: number, deltaLinesX: number, deltaLinesY: number, relative?: boolean }) -> ActionResult — scroll at a point; deltas are integers from -10 to 10 with at least one nonzero, and positive values reveal content above or left. - drag(from: [number, number], to: [number, number], options?: { mouseButton?: 'left' | 'right' | 'middle', modifiers?: ('command' | 'control' | 'option' | 'shift' | 'fn')[], relative?: boolean }) -> ActionResult — drag between two points of one window.