feat(cua): support optional mouse hold duration

packages/experimental/cua-macos/README.md

@@ -175,7 +175,7 @@ App and Window methods:
 - getScreenshot(options?: { emit?: boolean }) -> CuaScreenshot
 - getAXStateAndScreenshot(options?) -> { state: string, screenshot: CuaScreenshot }
 - hover(target: number | [x, y], options?: { relative?: boolean }) -> { dispatched, verified, hoverRoute, systemPointerMoved }
-- click(target: number | [x, y], options?: { mouseButton?: "left" | "right" | "middle", clickCount?: 1 | 2, modifiers?: Array<"command" | "control" | "option" | "shift" | "fn">, relative?: boolean }) -> action result
+- click(target: number | [x, y], options?: { mouseButton?: "left" | "right" | "middle", clickCount?: 1 | 2, holdMs?: number, modifiers?: Array<"command" | "control" | "option" | "shift" | "fn">, relative?: boolean }) -> action result
 - setValue(elementIndex: number, value: string) -> action result
 - selectText(elementIndex: number, text: string, options?: { prefix?: string, suffix?: string, selectionType?: "text" | "cursor_before" | "cursor_after" }) -> action result
 - performSecondaryAction(elementIndex: number, action: string) -> action result
@@ -188,7 +188,7 @@ Await operations sequentially. After input, read fresh state to verify the effec
 
 Hover first uses advertised AX alternate UI without moving the system cursor, preserving revealed descendants until the pointer target leaves their region. When unavailable, it moves the visible system cursor without pressing a button; that route requires the target point to be unobscured in the foreground app. FOREGROUND_REQUIRED requires explicit activation or raising and a fresh observation before retrying. Returning, AX reads and screenshots do not restore the previous point. Wait for the application's hover delay, observe, then hover into the revealed controls before clicking. Use successive hover calls for intermediate points. The result reports hoverRoute and systemPointerMoved. Other input or application changes can interrupt the interaction; app objects do not reserve applications between calls.
 
-Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. Foreground requirements vary by application; for example, ChatGPT/Codex was observed to accept background Paste without reading clipboard data. PASTE_NOT_CONSUMED means the command was dispatched but no clipboard read was observed before the configured deadline. Inspect the target contents, explicitly activate the app and focus the intended input, then observe again before deciding whether to retry; a delayed paste may still run. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
+Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Optional holdMs is an integer from 1 to 2147483647 milliseconds for each press, including each press of a double-click. It uses pointer input even for AX targets; omission preserves ordinary clicks. The operation deadline still applies, and cancellation or timeout releases the held button. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. Foreground requirements vary by application; for example, ChatGPT/Codex was observed to accept background Paste without reading clipboard data. PASTE_NOT_CONSUMED means the command was dispatched but no clipboard read was observed before the configured deadline. Inspect the target contents, explicitly activate the app and focus the intended input, then observe again before deciding whether to retry; a delayed paste may still run. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
 ```
 
 #### Token effect

packages/experimental/cua-macos/README.zh.md

@@ -175,7 +175,7 @@ App and Window methods:
 - getScreenshot(options?: { emit?: boolean }) -> CuaScreenshot
 - getAXStateAndScreenshot(options?) -> { state: string, screenshot: CuaScreenshot }
 - hover(target: number | [x, y], options?: { relative?: boolean }) -> { dispatched, verified, hoverRoute, systemPointerMoved }
-- click(target: number | [x, y], options?: { mouseButton?: "left" | "right" | "middle", clickCount?: 1 | 2, modifiers?: Array<"command" | "control" | "option" | "shift" | "fn">, relative?: boolean }) -> action result
+- click(target: number | [x, y], options?: { mouseButton?: "left" | "right" | "middle", clickCount?: 1 | 2, holdMs?: number, modifiers?: Array<"command" | "control" | "option" | "shift" | "fn">, relative?: boolean }) -> action result
 - setValue(elementIndex: number, value: string) -> action result
 - selectText(elementIndex: number, text: string, options?: { prefix?: string, suffix?: string, selectionType?: "text" | "cursor_before" | "cursor_after" }) -> action result
 - performSecondaryAction(elementIndex: number, action: string) -> action result
@@ -188,7 +188,7 @@ Await operations sequentially. After input, read fresh state to verify the effec
 
 Hover first uses advertised AX alternate UI without moving the system cursor, preserving revealed descendants until the pointer target leaves their region. When unavailable, it moves the visible system cursor without pressing a button; that route requires the target point to be unobscured in the foreground app. FOREGROUND_REQUIRED requires explicit activation or raising and a fresh observation before retrying. Returning, AX reads and screenshots do not restore the previous point. Wait for the application's hover delay, observe, then hover into the revealed controls before clicking. Use successive hover calls for intermediate points. The result reports hoverRoute and systemPointerMoved. Other input or application changes can interrupt the interaction; app objects do not reserve applications between calls.
 
-Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. Foreground requirements vary by application; for example, ChatGPT/Codex was observed to accept background Paste without reading clipboard data. PASTE_NOT_CONSUMED means the command was dispatched but no clipboard read was observed before the configured deadline. Inspect the target contents, explicitly activate the app and focus the intended input, then observe again before deciding whether to retry; a delayed paste may still run. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
+Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Optional holdMs is an integer from 1 to 2147483647 milliseconds for each press, including each press of a double-click. It uses pointer input even for AX targets; omission preserves ordinary clicks. The operation deadline still applies, and cancellation or timeout releases the held button. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. Foreground requirements vary by application; for example, ChatGPT/Codex was observed to accept background Paste without reading clipboard data. PASTE_NOT_CONSUMED means the command was dispatched but no clipboard read was observed before the configured deadline. Inspect the target contents, explicitly activate the app and focus the intended input, then observe again before deciding whether to retry; a delayed paste may still run. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
 ```
 
 #### Token 影响

packages/experimental/cua-macos/native/README.i18n.yaml

@@ -21,8 +21,8 @@
   en: 3202c567a6e1a654
   zh: 143eebd0096a0487
 /dsh-cua-native-helper/observation-and-input:
-  en: f2db99ae46a21c61
-  zh: 48c6752bd71dbf27
+  en: 917bfd96c4c59afc
+  zh: d65ad2a957b62e54
 /dsh-cua-native-helper/verification:
   en: 2afe5ceb0a3c4965
   zh: 86bf7187ddae489f

packages/experimental/cua-macos/native/README.md

@@ -69,7 +69,9 @@ Protocol declarations are generated from [the shared protocol](../../cua/src/pro
 
 `getState` optionally scopes AX traversal to an owned window or nested sheet `windowId`. A new or failed refresh invalidates the previous snapshot for that app/window scope, while other window scopes remain usable. When an owned CG window has no matching AX surface, getState returns a new snapshot with empty tree/elements and no truncation; its coordinates and screenshots remain available. It rechecks process lifetime, window ownership, the unlocked session and Accessibility permission after traversal. Window-discovery APIDisabled and communication errors remain failures; unsupported or absent optional AX attributes can produce an empty observation. Missing or foreign-owned windows never produce an empty success. An AX element invalidated during traversal rejects the whole read with `STALE_SNAPSHOT`; no partial snapshot is published. Callers may read again after the application changes, without repeating input. Element actions require the exact snapshot ID. Native references never cross JSON. An optional snapshot ID on coordinate actions must belong to that window or an app-wide observation. Window-list refresh releases snapshots for closed windows. Native process identity, snapshot generation, and window ownership checks do not lock the desktop against simultaneous user or application changes.
 
-Element clicks prefer advertised AX actions and otherwise use pointer delivery. Coordinate click, scroll and drag validate window ownership and bounds, then use background PID/window-directed primitives. Each event uses SkyLight when available, otherwise the public PID-targeted API. A posted event is never repeated through the other API. Coordinates are macOS points from the top-left of the entire window, including its title bar. Positive `deltaLinesY` reveals content above; negative reveals below. Positive `deltaLinesX` reveals content left; negative reveals right. Each axis accepts an integer from -10 to 10, and at least one must be nonzero. The helper sends line-unit scroll events directly. Click and drag accept left/right/middle buttons and command/control/option/shift/fn modifiers. Only left clicks support double-click. Modified element clicks use pointer delivery. Drag uses one button press/move/release in one window; callers explicitly supply durationMs (1..5000) and steps (1..240). The movement delay budget is divided with fractional precision across the segments; fixed press/release settling and scheduler latency are additional. No implicit fallback sends global desktop input.
+Element clicks without holdMs prefer advertised AX actions and otherwise use pointer delivery. Coordinate click, scroll and drag validate window ownership and bounds, then use background PID/window-directed primitives. Each event uses SkyLight when available, otherwise the public PID-targeted API. A posted event is never repeated through the other API. Coordinates are macOS points from the top-left of the entire window, including its title bar. Positive `deltaLinesY` reveals content above; negative reveals below. Positive `deltaLinesX` reveals content left; negative reveals right. Each axis accepts an integer from -10 to 10, and at least one must be nonzero. The helper sends line-unit scroll events directly. Click and drag accept left/right/middle buttons and command/control/option/shift/fn modifiers. Only left clicks support double-click. Modified element clicks use pointer delivery. Drag uses one button press/move/release in one window; callers explicitly supply durationMs (1..5000) and steps (1..240). The movement delay budget is divided with fractional precision across the segments; fixed press/release settling and scheduler latency are additional. No implicit fallback sends global desktop input.
+
+`click.holdMs` is an optional integer from 1 to 2147483647 milliseconds per press. It bypasses AX actions and uses the element center or supplied coordinates. Omission retains ordinary click timing. The complete gesture holds the request scheduler slot; cancellation, timeout and disconnect interrupt the hold and post the pending mouse-up before releasing that slot. Received events remain effective, and forced process termination cannot guarantee cleanup.
 
 Capture and pointer delivery read CG bounds before and after an exact-window AX frame query. Movement during the query returns WINDOW_CHANGED. Valid AX/CG frames must have equal outward-rounded integral point rectangles; a known mismatch returns WINDOW_CHANGED before mouse movement, priming or button delivery. The helper preserves both coordinate sources rather than substituting AX geometry for CG geometry. Missing or unusable AX geometry remains unknown, while AX communication, permission and invalid-element errors propagate. Capture without Accessibility permission retains its CG/SCK checks. Advertised AX actions bypass this pointer preflight; an AX action error does not trigger pointer input. These reads do not lock the window against later movement, and no geometry check activates or raises it.
 
@@ -79,7 +81,7 @@ The responsible host/helper needs macOS Accessibility permission for AX state an
 
 Observation reads each non-layout node’s descriptive properties and children through `AXUIElementCopyMultipleAttributeValues`. Per-attribute errors retain the individual-read rules; unsupported batch APIs use individual reads. Values are retained only during that observation, and input reads live attributes. Application state can change between nodes; a batch does not make the whole tree atomic.
 
-System file-panel keyboard routing resolves the remote recipient through the selected app’s focused AX sheet. Only Apple’s Open and Save Panel Service is accepted, and its focused ancestry must still contain the identified remote window. Element pointer fallbacks resolve an app-owned window or sheet through AX parents when AXWindow is unavailable. AX actions retain priority over pointer input; visual click markers need no system-cursor movement.
+System file-panel keyboard routing resolves the remote recipient through the selected app’s focused AX sheet. Only Apple’s Open and Save Panel Service is accepted, and its focused ancestry must still contain the identified remote window. Element pointer fallbacks resolve an app-owned window or sheet through AX parents when AXWindow is unavailable. Ordinary clicks retain AX action priority over pointer input; visual click markers need no system-cursor movement.
 
 Request modifier arrays accept command/control/option/shift/fn. The object API normalizes key-chord aliases before dispatch.
 

packages/experimental/cua-macos/native/README.zh.md

@@ -69,7 +69,9 @@ AppKit 在主事件循环上运行。阻塞式 AX 和输入使用每条连接的
 
 `getState` 可将 AX 遍历限定到应用所属窗口或嵌套 sheet 的 `windowId`。新的刷新或失败的刷新使同一应用/窗口范围的旧快照失效，其他窗口范围仍可使用。应用所属 CG 窗口没有匹配 AX 界面时，getState 返回空 tree/elements、truncated 为 false 的新快照，仍可截图和坐标输入。遍历后重新检查进程生命期、窗口归属、会话未锁定及辅助功能权限。窗口发现中的 APIDisabled 和通信错误仍会失败；不支持或缺失的可选 AX 属性可以产生空观察。不存在或属于其他应用的窗口不会返回空成功结果。遍历中 AX 元素失效会以 `STALE_SNAPSHOT` 拒绝整次读取，不发布部分树。应用变化后可以重新观察，但不重复输入。元素动作要求精确的快照 ID，原生引用不跨 JSON 传递。坐标动作的可选快照 ID 必须属于该窗口或应用级观察。刷新窗口列表会释放已关闭窗口的快照。进程身份、快照代次和窗口归属检查不会锁住桌面，用户或应用仍可同时改变状态。
 
-元素点击优先使用声明的 AX 动作，否则通过鼠标事件发送。坐标点击、滚动和拖动核实窗口归属和边界后，使用后台 PID/窗口定向输入。事件优先使用 SkyLight，不可用时使用公开的 PID 定向 API；已发送的事件不通过另一 API 重发。坐标是整个窗口左上角起算的 macOS point，包含标题栏。`deltaLinesY` 为正时显示上方内容，为负时显示下方内容；`deltaLinesX` 为正时显示左侧内容，为负时显示右侧内容。每个轴接受 -10 到 10 的整数，至少一个轴非零。helper 直接发送按行计量的滚轮事件。点击和拖动支持左、右、中键及 command/control/option/shift/fn 修饰键，只有左键支持双击。带修饰键的元素点击使用鼠标输入。拖动在一个窗口中完成一次按下、移动、抬起，调用方显式提供 durationMs（1..5000）和 steps（1..240）。移动延迟按小数精度分配到各段，按下和抬起的固定等待及调度延迟另计。不存在隐式改用全局桌面输入的后备路径。
+未指定 holdMs 的元素点击优先使用声明的 AX 动作，否则通过鼠标事件发送。坐标点击、滚动和拖动核实窗口归属和边界后，使用后台 PID/窗口定向输入。事件优先使用 SkyLight，不可用时使用公开的 PID 定向 API；已发送的事件不通过另一 API 重发。坐标是整个窗口左上角起算的 macOS point，包含标题栏。`deltaLinesY` 为正时显示上方内容，为负时显示下方内容；`deltaLinesX` 为正时显示左侧内容，为负时显示右侧内容。每个轴接受 -10 到 10 的整数，至少一个轴非零。helper 直接发送按行计量的滚轮事件。点击和拖动支持左、右、中键及 command/control/option/shift/fn 修饰键，只有左键支持双击。带修饰键的元素点击使用鼠标输入。拖动在一个窗口中完成一次按下、移动、抬起，调用方显式提供 durationMs（1..5000）和 steps（1..240）。移动延迟按小数精度分配到各段，按下和抬起的固定等待及调度延迟另计。不存在隐式改用全局桌面输入的后备路径。
+
+`click.holdMs` 是可选的每次按住时长，单位为毫秒，接受 1 至 2147483647 的整数。指定后使用元素中心或给定坐标发送鼠标事件，跳过 AX 动作。省略时保留普通点击时长。整个手势占用请求调度槽位；取消、超时和断开连接会中断等待，在释放槽位前发送对应的松开事件。已接收的事件仍然生效，进程被强制终止时无法保证清理。
 
 截图和鼠标输入在精确窗口的 AX 边界查询前后读取 CG 边界，查询期间移动会返回 WINDOW_CHANGED。有效的 AX/CG 边界向外取整后必须一致；已知不一致会在移动、预备事件或按键发送前返回 WINDOW_CHANGED。helper 保留两个坐标来源，不用 AX 几何替换 CG 几何。缺失或无效的 AX 几何仍表示未知，AX 通信、权限和无效元素错误继续传递。无辅助功能权限的截图保留 CG/SCK 检查。声明的 AX 动作不经过鼠标预检，AX 动作失败也不改用鼠标输入。这些读取不会锁住窗口以阻止后续移动，几何检查不激活或抬起窗口。
 
@@ -79,7 +81,7 @@ AppKit 在主事件循环上运行。阻塞式 AX 和输入使用每条连接的
 
 观察通过 `AXUIElementCopyMultipleAttributeValues` 读取每个非布局节点的描述属性和子节点。逐属性错误保留单独读取的规则；不支持批量 API 时使用单独读取。值只在本次观察期间保留，输入读取即时属性。应用状态可能在节点之间变化，批量读取不使整棵树具备原子性。
 
-系统文件面板的键盘路由通过所选应用的 AX 焦点 sheet 解析远端接收方。只接受 Apple 的 Open and Save Panel Service，且其焦点祖先链仍须包含已识别的远端窗口。AXWindow 不可用时，元素鼠标后备路径沿 AX 父节点解析应用所属窗口或 sheet。AX 动作保持优先，可视点击标记不需要系统鼠标移动。
+系统文件面板的键盘路由通过所选应用的 AX 焦点 sheet 解析远端接收方。只接受 Apple 的 Open and Save Panel Service，且其焦点祖先链仍须包含已识别的远端窗口。AXWindow 不可用时，元素鼠标后备路径沿 AX 父节点解析应用所属窗口或 sheet。普通点击保持 AX 动作优先，可视点击标记不需要系统鼠标移动。
 
 请求中的修饰键数组接受 command/control/option/shift/fn。对象 API 在分派前统一组合键别名。
 

packages/experimental/cua-macos/native/Sources/CuaNative/AXWorker.swift

@@ -199,7 +199,7 @@ actor AXWorker {
     }
 
     func execute(_ method: String, params: RequestParameters, catalog: ApplicationCatalogSnapshot, installed: InstalledApplication? = nil,
-                 trace: ScreenshotTrace? = nil) throws -> DriverOutcome {
+                 trace: ScreenshotTrace? = nil) async throws -> DriverOutcome {
         try cancellation.check()
         guard !closed else { throw NativeFailure(code: "NATIVE_CLOSED", message: "Helper is closing") }
         let locked = observeSession()
@@ -343,6 +343,7 @@ actor AXWorker {
             }
             try ax().perform(selected, action)
         case "click", "hover":
+            let holdMs = params.contains("holdMs") ? try params.integer("holdMs", in: 1...2147483647) : nil
             let button = try params.optionalString("mouseButton") ?? "left"
             let count = params.contains("clickCount") ? try params.integer("clickCount", in: 1...2) : 1
             guard button == "left" || (["right", "middle"].contains(button) && count == 1) else { throw RequestParameters.invalid("Unsupported mouse button or click count") }
@@ -356,7 +357,7 @@ actor AXWorker {
                 hoverElement = selected
                 if method == "click" {
                     let names = try ax().actions(selected)
-                    let action: String? = !modifiers.isEmpty ? nil : button == "right" && names.contains("AXShowMenu") ? "AXShowMenu"
+                    let action: String? = holdMs != nil || !modifiers.isEmpty ? nil : button == "right" && names.contains("AXShowMenu") ? "AXShowMenu"
                         : (button == "left" && count == 1 ? (names.contains("AXPress") ? "AXPress" : names.contains("AXPick") ? "AXPick" : nil) : nil)
                     if let action {
                         try ax().perform(selected, action)
@@ -387,7 +388,7 @@ actor AXWorker {
                 }
                 return .hover(route, point)
             }
-            try inputs().1.click(in: route, at: point, button: button, count: count, modifiers: modifiers)
+            try await inputs().1.click(in: route, at: point, button: button, count: count, modifiers: modifiers, holdMs: holdMs)
         case "scroll", "drag":
             let id = try params.windowID()
             try coordinateSnapshot(handle, params, id)

packages/experimental/cua-macos/native/Sources/CuaNative/NativeProtocol.generated.swift

@@ -3,7 +3,7 @@ enum NativeProtocol {
     static let modelMethods: [String] = ["activate", "click", "drag", "findApps", "getApp", "getCapabilities", "getState", "hover", "listApps", "listWindows", "paste", "performSecondaryAction", "pressKey", "screenshot", "scroll", "selectText", "setValue", "show", "typeText"]
     static let fields: [String: Set<String>] = [
         "activate": ["appHandle"],
-        "click": ["appHandle", "clickCount", "elementIndex", "modifiers", "mouseButton", "snapshotId", "windowId", "x", "y"],
+        "click": ["appHandle", "clickCount", "elementIndex", "holdMs", "modifiers", "mouseButton", "snapshotId", "windowId", "x", "y"],
         "drag": ["appHandle", "durationMs", "fromX", "fromY", "modifiers", "mouseButton", "snapshotId", "steps", "toX", "toY", "windowId"],
         "findApps": ["maxAppMatches", "query"],
         "getApp": ["selector"],

packages/experimental/cua-macos/native/Sources/CuaNative/WindowPointerInput.swift

@@ -57,8 +57,9 @@ final class WindowPointerInput {
         }
     }
 
-    /// Post one or two left clicks, or one right/middle click, with a shared window-routing group.
-    func click(in route: WindowRoute, at point: WindowPoint, button: String, count: Int, modifiers: [String] = []) throws {
+    /// Post balanced mouse presses; an explicit hold bypasses ordinary pacing and remains cancellable.
+    func click(in route: WindowRoute, at point: WindowPoint, button: String, count: Int, modifiers: [String] = [],
+               holdMs: Int? = nil, isolation: isolated (any Actor)? = #isolation) async throws {
         try checkCancellation()
         try Self.validate(point, in: route)
         guard (button == "left" && (count == 1 || count == 2)) || (["right", "middle"].contains(button) && count == 1) else {
@@ -84,10 +85,10 @@ final class WindowPointerInput {
             }
             try checkCancellation()
             poster.post(move, to: route.owner.pid)
-            Thread.sleep(forTimeInterval: 0.015)
+            try await Task.sleep(for: .milliseconds(15))
             for (index, pair) in pairs.enumerated() {
-                try postPair(pair, to: route.owner.pid, downInterval: 0.001)
-                if index + 1 < pairs.count { Thread.sleep(forTimeInterval: 0.080) }
+                try await postPair(pair, to: route.owner.pid, downInterval: .milliseconds(holdMs ?? 1))
+                if index + 1 < pairs.count { try await Task.sleep(for: .milliseconds(80)) }
             }
         } else {
             let pair = try mousePair(
@@ -96,8 +97,8 @@ final class WindowPointerInput {
             )
             try checkCancellation()
             poster.post(move, to: route.owner.pid)
-            Thread.sleep(forTimeInterval: 0.012)
-            try postPair(pair, to: route.owner.pid, downInterval: 0.028)
+            try await Task.sleep(for: .milliseconds(12))
+            try await postPair(pair, to: route.owner.pid, downInterval: .milliseconds(holdMs ?? 28))
         }
         try checkCancellation()
     }
@@ -206,7 +207,8 @@ final class WindowPointerInput {
         try checkCancellation()
     }
 
-    private func postPair(_ pair: MousePair, to pid: Int32, downInterval: TimeInterval) throws {
+    private func postPair(_ pair: MousePair, to pid: Int32, downInterval: Duration,
+                          isolation: isolated (any Actor)? = #isolation) async throws {
         var held = false
         defer {
             if held { poster.post(pair.up, to: pid) }
@@ -214,7 +216,7 @@ final class WindowPointerInput {
         try checkCancellation()
         poster.post(pair.down, to: pid)
         held = true
-        Thread.sleep(forTimeInterval: downInterval)
+        try await Task.sleep(for: downInterval)
         try checkCancellation()
         poster.post(pair.up, to: pid)
         held = false

packages/experimental/cua-macos/native/protocol.schema.json

@@ -412,6 +412,12 @@
                       "const": 2
                     }
                   ]
+                },
+                "holdMs": {
+                  "description": "Milliseconds to hold each mouse press before release. Uses pointer input even for AX targets; omission preserves ordinary click behavior. The operation deadline still applies.",
+                  "type": "integer",
+                  "minimum": 1,
+                  "maximum": 2147483647
                 }
               },
               "required": [
@@ -477,6 +483,12 @@
                       "const": 2
                     }
                   ]
+                },
+                "holdMs": {
+                  "description": "Milliseconds to hold each mouse press before release. Uses pointer input even for AX targets; omission preserves ordinary click behavior. The operation deadline still applies.",
+                  "type": "integer",
+                  "minimum": 1,
+                  "maximum": 2147483647
                 }
               },
               "required": [

packages/experimental/cua-macos/native/tests/CapabilitiesFixture.swift

@@ -3,6 +3,10 @@ import AppKit
 
 final class CapabilityPad: NSView {
     var received: ((NSEvent) -> Void)?
+    var pressed: (() -> Void)?
+    override func isAccessibilityElement() -> Bool { true }
+    override func accessibilityRole() -> NSAccessibility.Role? { .button }
+    override func accessibilityPerformPress() -> Bool { pressed?(); return true }
     override var isFlipped: Bool { true }
     override var acceptsFirstResponder: Bool { true }
     override func acceptsFirstMouse(for event: NSEvent?) -> Bool { true }
@@ -41,6 +45,7 @@ final class CapabilitiesFixture: NSObject, NSApplicationDelegate, NSWindowDelega
     var pasteAttempts: [Bool] = []
     var pad: CapabilityPad!
     var events: [[String: Any]] = []
+    var axPresses = 0
     init(statePath: String) { self.statePath = statePath }
     func applicationDidFinishLaunching(_ notification: Notification) {
         let bar = NSMenu()
@@ -70,10 +75,11 @@ final class CapabilitiesFixture: NSObject, NSApplicationDelegate, NSWindowDelega
         text.attemptedPaste = { [weak self] owned in self?.pasteAttempts.append(owned); self?.persist() }
         window.contentView!.addSubview(text)
         pad = CapabilityPad(frame: NSRect(x: 20, y: 25, width: 600, height: 170))
-        pad.setAccessibilityLabel("Input receiver")
+        pad.setAccessibilityTitle("Input receiver")
+        pad.pressed = { [weak self] in self?.axPresses += 1; self?.persist() }
         pad.received = { [weak self] event in
             guard let self else { return }
-            var value: [String: Any] = ["type": event.type.rawValue, "flags": event.modifierFlags.rawValue,
+            var value: [String: Any] = ["type": event.type.rawValue, "timestamp": event.timestamp, "flags": event.modifierFlags.rawValue,
                                        "shift": event.modifierFlags.contains(.shift), "command": event.modifierFlags.contains(.command)]
             if [.keyDown, .keyUp].contains(event.type) { value["keyCode"] = event.keyCode; value["characters"] = event.characters ?? "" }
             else { value["button"] = event.buttonNumber }
@@ -109,7 +115,7 @@ final class CapabilitiesFixture: NSObject, NSApplicationDelegate, NSWindowDelega
         let result: [String: Any] = ["pid": ProcessInfo.processInfo.processIdentifier, "windowId": window.windowNumber,
             "active": NSApplication.shared.isActive, "keyWindow": window.isKeyWindow,
             "textFocused": window.firstResponder === text, "pasteAttempts": pasteAttempts,
-            "text": text.string, "selection": [selection.location, selection.length], "runs": runs, "events": events,
+            "text": text.string, "selection": [selection.location, selection.length], "runs": runs, "events": events, "axPresses": axPresses,
             "pad": [frame.minX, window.frame.height - frame.maxY, frame.width, frame.height]]
         try! JSONSerialization.data(withJSONObject: result).write(to: URL(fileURLWithPath: statePath), options: .atomic)
     }

packages/experimental/cua-macos/native/tests/capabilities_e2e.py

@@ -11,7 +11,7 @@ import tempfile
 import time
 import uuid
 
-from native_test_support import ROOT, NativeHelper, run_cleanup
+from native_test_support import ROOT, NativeHelper, line, run_cleanup
 
 
 def wait_for_state(path, predicate):
@@ -44,6 +44,47 @@ def stop_application(pid):
     raise AssertionError("fixture did not exit after SIGTERM")
 
 
+def check_click_holds(helper, owner, window, point, state_file):
+    observed = helper.call("getState", {**owner, "maxElements": 2000, "maxDepth": 25})
+    pad = next(item for item in observed["elements"] if item["role"] == "AXButton" and item["title"] == "Input receiver")
+    target = {**owner, "snapshotId": observed["snapshotId"], "elementIndex": pad["index"]}
+    state = json.loads(state_file.read_text())
+    presses = state["axPresses"]
+    helper.call("click", target)
+    state = wait_for_state(state_file, lambda value: value["axPresses"] == presses + 1)
+    count = len(state["events"])
+    for invalid in [0, -1, 1.5, 2147483648, "800", None]:
+        helper.call("click", {**target, "holdMs": invalid}, "INVALID_ARGUMENT")
+    assert json.loads(state_file.read_text())["axPresses"] == presses + 1
+    for params, types in [({**target, "holdMs": 150}, [1, 2]),
+                          ({**window, **point, "holdMs": 150, "mouseButton": "right"}, [3, 4]),
+                          ({**window, **point, "holdMs": 150, "mouseButton": "middle"}, [25, 26]),
+                          ({**window, **point, "holdMs": 150, "clickCount": 2}, [1, 2, 1, 2])]:
+        helper.call("click", params)
+        state = wait_for_state(state_file, lambda value: len(value["events"]) >= count + len(types))
+        events = state["events"][count:]
+        assert [event["type"] for event in events] == types, events
+        for index in range(0, len(events), 2):
+            assert events[index + 1]["timestamp"] - events[index]["timestamp"] >= 0.150, events
+        assert state["axPresses"] == presses + 1, state
+        count += len(types)
+    # Observe the received down event before cancelling; the requested minute cannot expire during this check.
+    request = {"id": "cancel-held-click", "method": "click", "params": {**window, **point, "holdMs": 60000}}
+    helper.process.stdin.write(json.dumps(request) + "\n")
+    helper.process.stdin.flush()
+    wait_for_state(state_file, lambda value: len(value["events"]) == count + 1)
+    helper.process.stdin.write(json.dumps({"type": "cancel", "id": request["id"]}) + "\n")
+    helper.process.stdin.flush()
+    response = json.loads(line(helper.process))
+    helper.transcript.append({"request": request, "response": response})
+    assert response["id"] == request["id"] and response["error"]["code"] == "NATIVE_CANCELLED", response
+    state = wait_for_state(state_file, lambda value: len(value["events"]) >= count + 2)
+    assert [event["type"] for event in state["events"][count:]] == [1, 2], state
+    helper.call("click", {**window, **point})
+    state = wait_for_state(state_file, lambda value: len(value["events"]) >= count + 4)
+    assert [event["type"] for event in state["events"][count:]] == [1, 2, 1, 2], state
+
+
 def main():
     parser = argparse.ArgumentParser()
     parser.add_argument("--helper", type=Path, required=True)
@@ -112,12 +153,23 @@ def main():
             call("pressKey", {**owner, "key": "KP_0"})
             call("pressKey", {**owner, "key": "KP_Enter"})
             state = wait_for_state(state_file, lambda value: {82, 76}.issubset({event.get("keyCode") for event in value["events"]}))
+            check_click_holds(helper, owner, window, point, state_file)
+            timed = NativeHelper(transcript, args.helper, ["--timeout-ms", "5000"])
+            try:
+                timed_app = timed.call("getApp", {"selector": str(application)})
+                count = len(json.loads(state_file.read_text())["events"])
+                timed.call("click", {"appHandle": timed_app["appHandle"], "windowId": state["windowId"], **point,
+                                     "holdMs": 60000}, "NATIVE_TIMEOUT")
+                state = wait_for_state(state_file, lambda value: len(value["events"]) >= count + 2)
+                assert [event["type"] for event in state["events"][count:]] == [1, 2], state
+            finally:
+                timed.stop()
             if args.artifacts:
                 args.artifacts.mkdir(parents=True, exist_ok=True)
                 shot = call("screenshot", window)
                 (args.artifacts / "capabilities.png").write_bytes(base64.b64decode(shot["data"]))
                 (args.artifacts / "state.json").write_text(json.dumps(state, ensure_ascii=False, indent=2))
-            print("PASS: stopped-app discovery and launch, repeated Unicode selection, plain/HTML paste, middle/modifier drag and keypad identity")
+            print("PASS: stopped-app discovery and launch, repeated Unicode selection, plain/HTML paste, middle/modifier drag, keypad identity, click holds and cancellation/timeout release")
         finally:
             if pid is None and state_file.exists():
                 pid = json.loads(state_file.read_text())["pid"]

packages/experimental/cua-macos/native/tests/native_test_support.py

@@ -73,9 +73,9 @@ def stop_group(process):
 
 class NativeHelper:
     """One serial helper whose request transcript remains available on failure."""
-    def __init__(self, transcript, helper_path=None):
+    def __init__(self, transcript, helper_path=None, arguments=()):
         binary = Path(helper_path).resolve() if helper_path is not None else BINARY
-        self.process = subprocess.Popen([str(binary)], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+        self.process = subprocess.Popen([str(binary), *arguments], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                                         text=True, start_new_session=True)
         self.sequence = 0
         self.transcript = transcript

packages/experimental/cua-macos/src/guidance.ts

@@ -28,7 +28,7 @@ App and Window methods:
 - getScreenshot(options?: { emit?: boolean }) -> CuaScreenshot
 - getAXStateAndScreenshot(options?) -> { state: string, screenshot: CuaScreenshot }
 - hover(target: number | [x, y], options?: { relative?: boolean }) -> { dispatched, verified, hoverRoute, systemPointerMoved }
-- click(target: number | [x, y], options?: { mouseButton?: "left" | "right" | "middle", clickCount?: 1 | 2, modifiers?: Array<"command" | "control" | "option" | "shift" | "fn">, relative?: boolean }) -> action result
+- click(target: number | [x, y], options?: { mouseButton?: "left" | "right" | "middle", clickCount?: 1 | 2, holdMs?: number, modifiers?: Array<"command" | "control" | "option" | "shift" | "fn">, relative?: boolean }) -> action result
 - setValue(elementIndex: number, value: string) -> action result
 - selectText(elementIndex: number, text: string, options?: { prefix?: string, suffix?: string, selectionType?: "text" | "cursor_before" | "cursor_after" }) -> action result
 - performSecondaryAction(elementIndex: number, action: string) -> action result
@@ -41,4 +41,4 @@ Await operations sequentially. After input, read fresh state to verify the effec
 
 Hover first uses advertised AX alternate UI without moving the system cursor, preserving revealed descendants until the pointer target leaves their region. When unavailable, it moves the visible system cursor without pressing a button; that route requires the target point to be unobscured in the foreground app. FOREGROUND_REQUIRED requires explicit activation or raising and a fresh observation before retrying. Returning, AX reads and screenshots do not restore the previous point. Wait for the application's hover delay, observe, then hover into the revealed controls before clicking. Use successive hover calls for intermediate points. The result reports hoverRoute and systemPointerMoved. Other input or application changes can interrupt the interaction; app objects do not reserve applications between calls.
 
-Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. Foreground requirements vary by application; for example, ChatGPT/Codex was observed to accept background Paste without reading clipboard data. PASTE_NOT_CONSUMED means the command was dispatched but no clipboard read was observed before the configured deadline. Inspect the target contents, explicitly activate the app and focus the intended input, then observe again before deciding whether to retry; a delayed paste may still run. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.`
+Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Optional holdMs is an integer from 1 to 2147483647 milliseconds for each press, including each press of a double-click. It uses pointer input even for AX targets; omission preserves ordinary clicks. The operation deadline still applies, and cancellation or timeout releases the held button. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. Foreground requirements vary by application; for example, ChatGPT/Codex was observed to accept background Paste without reading clipboard data. PASTE_NOT_CONSUMED means the command was dispatched but no clipboard read was observed before the configured deadline. Inspect the target contents, explicitly activate the app and focus the intended input, then observe again before deciding whether to retry; a delayed paste may still run. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.`

packages/experimental/cua-macos/tests/loader-composition.spec.ts

@@ -28,7 +28,7 @@ const calls = [
   { method: 'getCapabilities', params: {} },
   { method: 'getApp', params: { selector: 'test.fixture' } },
   { method: 'getState', params: { appHandle: 'fixture-app', windowId: 7 } },
-  { method: 'click', params: { appHandle: 'fixture-app', snapshotId: 'fixture-snapshot-1', elementIndex: 0 } },
+  { method: 'click', params: { appHandle: 'fixture-app', snapshotId: 'fixture-snapshot-1', elementIndex: 0, holdMs: 800 } },
   { method: 'getState', params: { appHandle: 'fixture-app', windowId: 7 } },
   { method: 'screenshot', params: { appHandle: 'fixture-app', windowId: 7 } },
 ]
@@ -128,6 +128,8 @@ it.skipIf(process.platform === 'win32')('shares one application between retained
   expect(events.filter(event => event.event === 'call').map(event => event.requestId)).toEqual(toolCalls.map(event => event.data.callId))
   expect(events.find(event => event.event === 'state')?.count).toBe(1)
   expect(events.find(event => event.method === 'getState')?.params).toMatchObject({ maxElements: 2000, maxDepth: 25 })
+  expect(events.find(event => event.method === 'click')?.params).toMatchObject({ holdMs: 800 })
+  expect(toolCalls[3]?.data.arguments).toMatchObject({ method: 'click', params: { holdMs: 800 } })
 
   const direct = await context.tools.execute({ agent, signal: new AbortController().signal,
     callId: ToolCallId('programmatic-window'), name: 'cua', arguments: { method: 'screenshot', params: { appHandle: 'fixture-app', windowId: 7 } },

packages/experimental/cua/README.i18n.yaml

@@ -15,8 +15,8 @@
   en: d152484eb41ac6b4
   zh: 1db86285abf6c499
 /deepseek-ai-dsh-cua/use-this-package:
-  en: 52cacbe2d2aeb2bc
-  zh: b3c65814d9a17052
+  en: e32415b37444ed06
+  zh: d5a8e94d6e28a9d7
 /deepseek-ai-dsh-cua/understand-the-implementation:
   en: effcee57d8854c32
   zh: d02c6b76e7249051

packages/experimental/cua/README.md

@@ -37,6 +37,8 @@ Application consent precedes launch, observation, capture, and input. Task acces
 
 Actions return `dispatched:true, verified:false`; read fresh state to verify the result. Coordinate actions default to pixels in the latest returned screenshot; `relative: true` accepts integers from 0 to 1000 after a successful window observation. The Host resolves attachment normalization and the selected model route’s image projection before returning the screenshot, then converts input to native window points. Resizing invalidates the screenshot mapping; `SCREENSHOT_REQUIRED` requires a new screenshot for absolute coordinates. Resizing rejects old coordinates with `WINDOW_CHANGED`; movement alone preserves window-local coordinates.
 
+`click(target, { holdMs: 800 })` holds the mouse button for 800 milliseconds before release. `holdMs` accepts integers from 1 to 2147483647 and applies to each press of a double-click. Explicit holds use pointer input for both AX indices and coordinates; omission preserves ordinary click behavior. The operation deadline includes the hold, and cancellation or timeout releases the button. Received input cannot be undone; forced helper termination cannot guarantee release.
+
 -----
 
 <a id="understand-the-implementation"></a>

packages/experimental/cua/README.zh.md

@@ -37,6 +37,8 @@ kind: "package-library"
 
 动作返回 `dispatched:true, verified:false`；请读取新状态以验证结果。坐标操作默认使用最近返回截图中的像素；成功观察窗口后，传入 `relative: true` 可使用 0～1000 的整数比例。Host 在返回截图前解析附件归一化和当前模型的图片缩放规则，再将输入换算为原生窗口 point。窗口尺寸变化会使截图映射失效，绝对坐标遇到 `SCREENSHOT_REQUIRED` 时需要重新截图。窗口缩放后以 `WINDOW_CHANGED` 拒绝旧坐标；单纯移动窗口不改变窗口内坐标。
 
+`click(target, { holdMs: 800 })` 按住鼠标按钮 800 毫秒后松开。`holdMs` 接受 1 至 2147483647 的整数，双击时应用于每次按下。指定时长后，AX 索引和坐标目标都使用鼠标事件；省略时保留普通点击行为。按住时间计入操作超时，取消或超时会松开按钮。已接收的输入无法撤销；辅助进程被强制终止时无法保证松开。
+
 -----
 
 <a id="understand-the-implementation"></a>

packages/experimental/cua/src/api.ts

@@ -127,14 +127,15 @@ export function createCuaApi(dispatch: CuaDispatch, sink?: CuaObservationSink):
         const clickCount = options?.clickCount ?? 1
         if (mouseButton !== 'left' && clickCount !== 1) throw new Error('Only left double-click is supported')
         const modifiers = options?.modifiers === undefined ? {} : { modifiers: [...options.modifiers] }
+        const hold = options?.holdMs === undefined ? {} : { holdMs: options.holdMs }
         const relative = options?.relative === undefined ? {} : { relative: options.relative }
         if (typeof target === 'number') {
-          return action('click', { ...snapshotParams(), ...modifiers, ...relative, elementIndex: target,
+          return action('click', { ...snapshotParams(), ...modifiers, ...hold, ...relative, elementIndex: target,
             ...mouseButton === 'left' ? {} : { mouseButton }, ...clickCount === 1 ? {} : { clickCount },
           })
         }
         const [x, y] = target
-        return action('click', { ...snapshotParams(), windowId: windowId(), x, y, mouseButton, clickCount, ...modifiers, ...relative })
+        return action('click', { ...snapshotParams(), windowId: windowId(), x, y, mouseButton, clickCount, ...modifiers, ...hold, ...relative })
       },
       async setValue(index, value) { return action('setValue', { ...snapshotParams(), elementIndex: index, value }) },
       async selectText(index, text, options) {

packages/experimental/cua/src/protocol.ts

@@ -10,6 +10,9 @@ const relativePoint = z.number().int().min(0).max(1000)
 const app = { appHandle: z.string().min(1) }
 const pointer = { mouseButton: z.enum(['left', 'right', 'middle']).optional(),
   modifiers: z.array(z.enum(['command', 'control', 'option', 'shift', 'fn'])).optional() }
+const click = { ...pointer, clickCount: z.union([z.literal(1), z.literal(2)]).optional(),
+  holdMs: z.number().int().min(1).max(2 ** 31 - 1).optional()
+    .describe('Milliseconds to hold each mouse press before release. Uses pointer input even for AX targets; omission preserves ordinary click behavior. The operation deadline still applies.') }
 const window = { ...app, windowId: id }
 const snapshot = { ...app, snapshotId: z.string().min(1), elementIndex: z.number().int().nonnegative() }
 const coordinate = { ...window, snapshotId: z.string().min(1).optional(), x: point, y: point }
@@ -43,9 +46,9 @@ export const Request = z.discriminatedUnion('method', [
     z.object(snapshot).strict(), z.object(absoluteCoordinate).strict(), z.object(relativeCoordinate).strict(),
   ]) }).strict(),
   z.object({ method: z.literal('click'), params: z.union([
-    z.object({ ...snapshot, ...pointer, clickCount: z.union([z.literal(1), z.literal(2)]).optional() }).strict(),
-    z.object({ ...absoluteCoordinate, ...pointer, clickCount: z.union([z.literal(1), z.literal(2)]).optional() }).strict(),
-    z.object({ ...relativeCoordinate, ...pointer, clickCount: z.union([z.literal(1), z.literal(2)]).optional() }).strict(),
+    z.object({ ...snapshot, ...click }).strict(),
+    z.object({ ...absoluteCoordinate, ...click }).strict(),
+    z.object({ ...relativeCoordinate, ...click }).strict(),
   ]) }).strict(),
   operation('setValue', { ...snapshot, value: z.string() }),
   operation('selectText', { ...snapshot, text: z.string().min(1), prefix: z.string().optional(), suffix: z.string().optional(),

packages/experimental/cua/src/types.ts

@@ -66,8 +66,11 @@ export type CuaShowResult = z.output<typeof ShowResult>
 export type CuaPointerModifier = NonNullable<CuaParameters<'click'>['modifiers']>[number]
 /** Buttons supported by window-directed clicks and drags. */
 export type CuaMouseButton = NonNullable<CuaParameters<'click'>['mouseButton']>
-/** Left clicks support one or two presses; right and middle clicks support one. */
-export type CuaClickOptions = Pick<CuaParameters<'click'>, 'mouseButton' | 'clickCount' | 'modifiers'> & CuaCoordinateOptions
+/**
+ * Left clicks support one or two presses; right and middle clicks support one.
+ * holdMs holds each press for 1–2147483647 milliseconds through pointer input; omission preserves ordinary clicks.
+ */
+export type CuaClickOptions = Pick<CuaParameters<'click'>, 'mouseButton' | 'clickCount' | 'modifiers' | 'holdMs'> & CuaCoordinateOptions
 /** One same-window drag; omitted button is left and omitted modifiers are empty. */
 export type CuaDragOptions = Pick<CuaParameters<'drag'>, 'mouseButton' | 'modifiers'> & CuaCoordinateOptions
 /** Clipboard representation delivered through the application's Paste command. */
@@ -120,8 +123,9 @@ export interface CuaSurface {
   hover(target: number | readonly [number, number], options?: CuaCoordinateOptions): Promise<CuaHoverResult>
   /**
    * Click a current AX index or screenshot-local point.
+   * Cancellation or timeout releases the held button; received input cannot be undone. The operation deadline includes holdMs.
    * @param target - index or [x,y].
-   * @param options - button, click count, modifiers and coordinate mode.
+   * @param options - button, click count, modifiers, optional hold duration and coordinate mode.
    * @returns dispatch evidence.
    */
   click(target: number | readonly [number, number], options?: CuaClickOptions): Promise<CuaActionResult>

packages/experimental/cua/tests/api.spec.ts

@@ -171,6 +171,21 @@ describe('hover across observations', () => {
 })
 
 describe('pointer buttons and modifiers', () => {
+  it('preserves hold duration for app and window clicks at elements and coordinates', async () => {
+    const f = fixture()
+    const app = await f.api.getApp('Editor')
+    const window = await app.getWindow(23)
+    for (const surface of [app, window]) {
+      for (const target of [0, [10, 20] as const]) {
+        await surface.click(target, { holdMs: 800, mouseButton: 'right' })
+        expect(f.dispatch).toHaveBeenLastCalledWith({ method: 'click', params: expect.objectContaining({ holdMs: 800, mouseButton: 'right' }) })
+        f.dispatch.mockClear()
+        await expect(surface.click(target, { holdMs: 0 })).rejects.toThrow('holdMs')
+        expect(f.dispatch).not.toHaveBeenCalled()
+      }
+    }
+  })
+
   it('retains middle-button and modifier intent through click and drag dispatch', async () => {
     const f = fixture()
     const app = await f.api.getApp('Editor')

packages/experimental/cua/tests/protocol.spec.ts

@@ -18,6 +18,20 @@ it.each(rejectedRequests)('rejects $reason at operation admission', ({ request }
   expect(Request.safeParse(request).success).toBe(false)
 })
 
+it.each([
+  { appHandle: 'app', snapshotId: 'snapshot', elementIndex: 0 },
+  { appHandle: 'app', windowId: 23, x: 1, y: 2 },
+  { appHandle: 'app', windowId: 23, x: 1, y: 2, relative: true },
+])('validates optional per-press hold duration for %j', (params) => {
+  for (const holdMs of [1, 800, 2 ** 31 - 1]) {
+    const request = { method: 'click', params: { ...params, holdMs } }
+    expect(Request.parse(request)).toEqual(request)
+  }
+  for (const holdMs of [0, -1, 1.5, 2 ** 31, Infinity, NaN, '800', null]) {
+    expect(Request.safeParse({ method: 'click', params: { ...params, holdMs } }).success).toBe(false)
+  }
+})
+
 it.each([
   '', 'not base64!', 'AQ', 'AQ===', 'AQ==\n', 'A Q==', 'AQ-_', 'AB==', 'AAF=',
 ])('rejects noncanonical screenshot base64 %j', (data) => {

snapshots/session/cua-native/replay.override.json

@@ -113,7 +113,7 @@
           "type": "tool-call",
           "id": "cua-native-4",
           "name": "cua",
-          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementIndex\":0}}"
+          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-1\",\"elementIndex\":0,\"holdMs\":800}}"
         }
       },
       {
@@ -311,7 +311,7 @@
           "type": "tool-call",
           "id": "cua-native-10",
           "name": "cua",
-          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-3\",\"x\":0,\"y\":0}}"
+          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-3\",\"x\":0,\"y\":0,\"holdMs\":800}}"
         }
       },
       {
@@ -542,7 +542,7 @@
           "type": "tool-call",
           "id": "cua-native-stale-index",
           "name": "cua",
-          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-2\",\"elementIndex\":0}}"
+          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-2\",\"elementIndex\":0,\"holdMs\":800}}"
         }
       },
       {
@@ -575,7 +575,7 @@
           "type": "tool-call",
           "id": "cua-native-other-window",
           "name": "cua",
-          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-4\",\"x\":0,\"y\":0}}"
+          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":8,\"snapshotId\":\"fixture-snapshot-4\",\"x\":0,\"y\":0,\"holdMs\":800}}"
         }
       },
       {
@@ -641,7 +641,7 @@
           "type": "tool-call",
           "id": "cua-native-fresh-click",
           "name": "cua",
-          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-5\",\"elementIndex\":0}}"
+          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-5\",\"elementIndex\":0,\"holdMs\":800}}"
         }
       },
       {
@@ -773,7 +773,7 @@
           "type": "tool-call",
           "id": "cua-native-geometry-click",
           "name": "cua",
-          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"snapshotId\":\"fixture-snapshot-7\",\"x\":20,\"y\":20,\"relative\":true}}"
+          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":9,\"snapshotId\":\"fixture-snapshot-7\",\"x\":20,\"y\":20,\"relative\":true,\"holdMs\":800}}"
         }
       },
       {
@@ -872,7 +872,7 @@
           "type": "tool-call",
           "id": "cua-native-geometry-ax",
           "name": "cua",
-          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-7\",\"elementIndex\":0}}"
+          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"snapshotId\":\"fixture-snapshot-7\",\"elementIndex\":0,\"holdMs\":800}}"
         }
       },
       {
@@ -1235,7 +1235,7 @@
           "type": "tool-call",
           "id": "cua-hover-click",
           "name": "cua",
-          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-10\",\"x\":100,\"y\":200,\"relative\":true}}"
+          "arguments": "{\"method\":\"click\",\"params\":{\"appHandle\":\"fixture-app\",\"windowId\":7,\"snapshotId\":\"fixture-snapshot-10\",\"x\":100,\"y\":200,\"relative\":true,\"holdMs\":800}}"
         }
       },
       {