docs(cua): explain unread paste recovery and foreground evidence

.agents/notes/implemented/architecture/2026-09-17-experimental-cua-object-api.i18n.yaml

@@ -9,11 +9,11 @@
   en: a22ddcfb339e503d
   zh: a1f4fadb46fce1e7
 /agent-note-native-cua-objects-independent-of-evaluator-ownership/decision:
-  en: 9d8853d57a29d000
-  zh: 1740cdee995ee286
+  en: 2780d1b0f5ce9ea8
+  zh: ff986f8c1c90300f
 /agent-note-native-cua-objects-independent-of-evaluator-ownership/alternatives-considered:
-  en: 27fb4e2da4bfab2e
-  zh: ad4db8fd242e4625
+  en: 3d0bc77e3c432cb9
+  zh: 50a20eda1811f16c
 /agent-note-native-cua-objects-independent-of-evaluator-ownership/consequences:
   en: 528c644d5810e60b
   zh: 9649dfc3b1172a6d

.agents/notes/implemented/architecture/2026-09-17-experimental-cua-object-api.md

@@ -20,7 +20,7 @@ Native process ownership follows the [shared helper decision](2026-10-09-cua-sha
 
 The operation schema owns shared data declarations. Host-only limits and launch continuations remain separate from model parameters, and generated native field/enum tables preserve independent native admission without hand-maintained copies. Native code continues to own system calls, permissions and cancellation. AX formatting consumes immutable observations; the walker retains sole ownership of indices and AX references.
 
-Clipboard paste is distinct from Unicode key input: Return and Tab key events can trigger actions instead of inserting clipboard content. Pasteboard access runs on the AppKit main actor and retains promised data until a reader requests it. AX command completion alone does not establish that the application has fetched the data. Restoration checks ownership before replacing contents; the caller still verifies the application result.
+Clipboard paste is distinct from Unicode key input: Return and Tab key events can trigger actions instead of inserting clipboard content. Pasteboard access runs on the AppKit main actor and retains promised data until a reader requests it. AX command completion alone does not establish that the application has fetched the data. Restoration checks ownership before replacing contents; the caller still verifies the application result. After successful command dispatch, `pasteReadTimeoutMs` bounds waiting for a data request, with a default of 1000 ms. Expiry returns `PASTE_NOT_CONSUMED`, closes the data provider against late delivery, and restores the clipboard only while still owned. The failure reports no read within that interval; it does not prove the application will never process its queued Paste command. A delayed command may read restored clipboard contents, so callers inspect the target and never retry automatically. Foreground-dependent consumption was observed in the installed ChatGPT/Codex application (`com.openai.codex`); other applications need their own validation. Guidance presents this as an example and applies recovery instructions to any `PASTE_NOT_CONSUMED` result.
 
 CUA registers its own Harness tool and returns operation values directly. Screenshot capture requires a resolved image-capable model and the shared attachment service. The tool admits each screenshot once and renders its durable image reference; the object API, combined observation and image sink retain the same `CuaScreenshot` containing encoded `bytes`, `mimeType`, `pixelWidth`, `pixelHeight` and `windowId` from the validated screenshot result. The bridge preserves this metadata without another upload or image emission. Callers use the reported format and dimensions when saving or declaring images.
 
@@ -34,6 +34,8 @@ Reusing the MCP result adapter would impose MCP envelopes and wire validation on
 
 **Fixing the screenshot format** would require otherwise unnecessary re-encoding and discard the attachment pipeline’s pass-through optimization. A format convention alone still forces callers to guess, and cannot tell them when encoding policy changes. Returning the already computed format and dimensions preserves pass-through and makes each result describe its actual bytes.
 
+**Waiting until the full native-operation timeout** hides successful command dispatch behind a generic timeout. A separate configurable read budget reports the missing acknowledgement without treating slow applications as unsupported. Requiring every application to be foreground would claim a prerequisite that the available observations do not establish.
+
 ## Consequences
 
 The package remains opt-in and macOS-only. Observations do not lock the desktop, and input success records dispatch rather than task completion. Callers must verify effects and must not retry uncertain input automatically. Native fixture tests verify actual window state separately from protocol and model-transcript tests. Persistent execution, binary distribution, permission onboarding and real-model task success remain separately owned work.

.agents/notes/implemented/architecture/2026-09-17-experimental-cua-object-api.zh.md

@@ -20,7 +20,7 @@ Computer-use 程序需要一致的应用和窗口操作。将这些操作与 CUA
 
 操作 schema 负责共享数据声明。Host 专属限制和启动续接与模型参数分开，生成的原生字段、枚举表保留独立原生校验，同时避免手工维护副本。系统调用、权限和取消仍由原生代码负责。AX 格式化只消费不可变观察数据，遍历器独自负责索引与 AX 引用。
 
-剪贴板粘贴与 Unicode 按键输入分别实现，因为 Return 和 Tab 按键可能触发动作，而不是插入剪贴板内容。剪贴板访问使用 AppKit 主线程，保留延迟提供的数据直到读取方请求。AX 命令完成本身不能证明应用已取得数据。恢复前检查剪贴板归属；调用方仍需验证应用中的结果。
+剪贴板粘贴与 Unicode 按键输入分别实现，因为 Return 和 Tab 按键可能触发动作，而不是插入剪贴板内容。剪贴板访问使用 AppKit 主线程，保留延迟提供的数据直到读取方请求。AX 命令完成本身不能证明应用已取得数据。恢复前检查剪贴板归属；调用方仍需验证应用中的结果。 命令成功分派后，`pasteReadTimeoutMs` 限制等待数据请求的时间，默认 1000 毫秒。到期返回 `PASTE_NOT_CONSUMED`，关闭数据提供方以阻止迟到的数据交付，并仅在仍持有剪贴板时恢复原内容。该失败只说明这段时间内没有读取，不证明应用以后不会处理已排队的 Paste 命令。迟到的命令可能读取恢复后的剪贴板内容，因此调用方必须检查目标，不能自动重试。已安装的 ChatGPT/Codex 应用（`com.openai.codex`）已观察到依赖前台才能读取的行为，其他应用需要分别验证。指引将其写为例子，并对任何 `PASTE_NOT_CONSUMED` 结果提供恢复步骤。
 
 CUA 注册自己的 Harness 工具，直接返回操作结果。截图要求解析到支持图片输入的模型，并提供共享附件服务。工具只接入一次截图，再渲染持久图片引用；对象 API、组合观察和图片输出回调保留同一个 `CuaScreenshot`，其中的编码 `bytes`、`mimeType`、`pixelWidth`、`pixelHeight` 和 `windowId` 来自已校验的截图结果。桥接保留这些元数据，不再次上传或输出图片。调用方落盘或声明图片时使用返回的格式和尺寸。
 
@@ -34,6 +34,8 @@ CUA 专用持久执行器会重复实现执行生命周期规则，使 PTC 无
 
 **固定截图格式**会要求原本不需要的重新编码，并失去附件处理的原样直通优化。仅约定格式仍要求调用方猜测，也无法告知编码策略的变化。返回已经算出的格式和尺寸，既保留直通优化，也让每个结果说明其实际字节。
 
+**等到原生操作的总超时**会用一条通用超时掩盖命令已经成功分派的事实。独立、可配置的读取等待上限能报告缺失的读取确认，而不将响应慢的应用认定为不支持粘贴。要求所有应用都必须位于前台，则超出了现有观察能支持的结论。
+
 ## 后果
 
 本包需要显式启用，且只提供 macOS 实现。观察不会锁定桌面，输入成功只记录分派完成，不能证明任务完成。调用方必须验证效果，不能自动重试结果不确定的输入。原生测试通过实际窗口状态验证效果，与协议和模型消息测试分别进行。持久执行、二进制分发、权限引导及真实模型任务成功率是分别负责的工作。

packages/experimental/cua-macos/README.md

@@ -188,7 +188,7 @@ Await operations sequentially. After input, read fresh state to verify the effec
 
 Hover first uses advertised AX alternate UI without moving the system cursor, preserving revealed descendants until the pointer target leaves their region. When unavailable, it moves the visible system cursor without pressing a button; that route requires the target point to be unobscured in the foreground app. FOREGROUND_REQUIRED requires explicit activation or raising and a fresh observation before retrying. Returning, AX reads and screenshots do not restore the previous point. Wait for the application's hover delay, observe, then hover into the revealed controls before clicking. Use successive hover calls for intermediate points. The result reports hoverRoute and systemPointerMoved. Other input or application changes can interrupt the interaction; app objects do not reserve applications between calls.
 
-Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. In ChatGPT/Codex, activate the app and focus that input before calling paste; a background AXPress can succeed without the app reading clipboard data. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
+Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. Foreground requirements vary by application; for example, ChatGPT/Codex was observed to accept background Paste without reading clipboard data. PASTE_NOT_CONSUMED means the command was dispatched but no clipboard read was observed before the configured deadline. Inspect the target contents, explicitly activate the app and focus the intended input, then observe again before deciding whether to retry; a delayed paste may still run. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
 ```
 
 #### Token effect

packages/experimental/cua-macos/README.zh.md

@@ -188,7 +188,7 @@ Await operations sequentially. After input, read fresh state to verify the effec
 
 Hover first uses advertised AX alternate UI without moving the system cursor, preserving revealed descendants until the pointer target leaves their region. When unavailable, it moves the visible system cursor without pressing a button; that route requires the target point to be unobscured in the foreground app. FOREGROUND_REQUIRED requires explicit activation or raising and a fresh observation before retrying. Returning, AX reads and screenshots do not restore the previous point. Wait for the application's hover delay, observe, then hover into the revealed controls before clicking. Use successive hover calls for intermediate points. The result reports hoverRoute and systemPointerMoved. Other input or application changes can interrupt the interaction; app objects do not reserve applications between calls.
 
-Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. In ChatGPT/Codex, activate the app and focus that input before calling paste; a background AXPress can succeed without the app reading clipboard data. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
+Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. Foreground requirements vary by application; for example, ChatGPT/Codex was observed to accept background Paste without reading clipboard data. PASTE_NOT_CONSUMED means the command was dispatched but no clipboard read was observed before the configured deadline. Inspect the target contents, explicitly activate the app and focus the intended input, then observe again before deciding whether to retry; a delayed paste may still run. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.
 ```
 
 #### Token 影响

packages/experimental/cua-macos/src/guidance.ts

@@ -41,4 +41,4 @@ Await operations sequentially. After input, read fresh state to verify the effec
 
 Hover first uses advertised AX alternate UI without moving the system cursor, preserving revealed descendants until the pointer target leaves their region. When unavailable, it moves the visible system cursor without pressing a button; that route requires the target point to be unobscured in the foreground app. FOREGROUND_REQUIRED requires explicit activation or raising and a fresh observation before retrying. Returning, AX reads and screenshots do not restore the previous point. Wait for the application's hover delay, observe, then hover into the revealed controls before clicking. Use successive hover calls for intermediate points. The result reports hoverRoute and systemPointerMoved. Other input or application changes can interrupt the interaction; app objects do not reserve applications between calls.
 
-Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. In ChatGPT/Codex, activate the app and focus that input before calling paste; a background AXPress can succeed without the app reading clipboard data. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.`
+Use selectText only on controls advertising selectionWritable. It selects a unique literal occurrence, or places the cursor before/after it; prefix and suffix must be immediately adjacent. AMBIGUOUS_TEXT requires more context; TEXT_NOT_FOUND or SELECTION_CHANGED requires a fresh observation before deciding on input. Left clicks support single/double clicks; right and middle clicks support single clicks. Modified clicks use pointer input. Keypad names such as KP_0 and KP_Enter retain their physical key identity. Window objects expose observation and pointer/AX actions; keyboard input remains app-scoped and follows current focus. Scroll deltas are integer lines from -10 to 10; positive values reveal content above or left. Paste delivers plain text or HTML through the application's enabled standard Paste menu command. Paste follows the target app's focused text input. Foreground requirements vary by application; for example, ChatGPT/Codex was observed to accept background Paste without reading clipboard data. PASTE_NOT_CONSUMED means the command was dispatched but no clipboard read was observed before the configured deadline. Inspect the target contents, explicitly activate the app and focus the intended input, then observe again before deciding whether to retry; a delayed paste may still run. Original clipboard formats are restored if the clipboard still belongs to this paste; a later copy is preserved. Unsupported Paste commands fail without typing fallback. CLIPBOARD_CHANGED means another writer intervened; observe before deciding on further input. Browser applications can be controlled with getApp like other native apps; browser tabs are unavailable. Installed-app search covers standard macOS application directories; use an absolute application path for another location. Launch does not restore a window or document, and cancellation may leave an application already started. The execution host must install the object API and own variable persistence; this plugin does not preload a cua global. In persistent PTC, use the host-provided initialization snippet below once. Its installation-owned file URL works independently of the project directory. Retain app objects between calls. Observation methods emit by default and also return their value. Use void await app.getAXState() to avoid a duplicate REPL completion, or pass emit:false and print the returned text once. End screenshot calls with void await app.getScreenshot() or void await app.getAXStateAndScreenshot() so image bytes are not serialized as the program completion value.`