test(cua): refresh relative-coordinate prompt snapshots
This commit is contained in:
1 parent
7e8bf2808d
commit
a46f095f01
3 files changed
+15
-15
No files matched your search
@@ -29,7 +29,7 @@ Use the workflow tool ONLY when the user explicitly asks for a workflow or for l
|
||||
|
||||
Start independent delegations with `subagent` or `subagent_fork` together in one assistant message and continue useful work while they run.
|
||||
|
||||
Use cua with method getCapabilities and params {} to inspect permissions and sessionLocked. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI attempts and ask the user to unlock macOS. A null sessionLocked means no usable console session was verified. After unlocking, read fresh state; screenshots from a locked desktop are not current observations. Use listApps or cua.getState to discover running applications, and findApps(query) to search installed applications by name, bundle ID or absolute path. getApp requests user consent before launching a stopped application in the background, or before observations and input on a running one. If access is denied or revoked, stop using that application until the user explicitly permits another authorization request. Never switch tools to bypass a refusal. Keep the returned appHandle; every app operation requires it. listWindows returns window IDs and point bounds. Raw getState returns AX text and snapshotId. Menu-bar entries are shown by default; includeMenus=true expands their contents. details=true adds identifiers, global-point bounds and complete action names. Both options also apply to getAXState. Each read replaces the indices for that app/window scope. Use its element indices only with that snapshotId. Raw screenshot and coordinate operations require a windowId. Coordinates are pixels in the latest returned screenshot, measured from its top-left. Set relative:true for integer thousandths from 0 to 1000 instead. The Host converts both modes to native window points; AX detail bounds remain global macOS points. A verified window can expose no AX elements. Its empty snapshot permits screenshots and coordinate input; element-index actions require an element from a fresh AX observation.
|
||||
Use cua with method getCapabilities and params {} to inspect permissions and sessionLocked. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI attempts and ask the user to unlock macOS. A null sessionLocked means no usable console session was verified. After unlocking, read fresh state; screenshots from a locked desktop are not current observations. Use listApps or cua.getState to discover running applications, and findApps(query) to search installed applications by name, bundle ID or absolute path. getApp requests user consent before launching a stopped application in the background, or before observations and input on a running one. If access is denied or revoked, stop using that application until the user explicitly permits another authorization request. Never switch tools to bypass a refusal. Keep the returned appHandle; every app operation requires it. listWindows returns window IDs and point bounds. Raw getState returns AX text and snapshotId. Menu-bar entries are shown by default; includeMenus=true expands their contents. details=true adds identifiers, global-point bounds and complete action names. Both options also apply to getAXState. Each read replaces the indices for that app/window scope. Use its element indices only with that snapshotId. Raw screenshot and coordinate operations require a windowId. Coordinates are pixels in the latest returned screenshot, measured from its top-left. Set relative:true for integer coordinates from 0 to 999, normalized by 999 to the window width and height. The Host converts both modes to native window points; AX detail bounds remain global macOS points. A verified window can expose no AX elements. Its empty snapshot permits screenshots and coordinate input; element-index actions require an element from a fresh AX observation.
|
||||
|
||||
Initialize the object API once in persistent PTC. Reuse its variables in later calls; put temporary declarations inside { ... } blocks to avoid redeclaration errors. All object methods are async. getAXState returns text, not a structured state object. Its text separates values and placeholders, and reports subroles, help, URLs, disabled/focused/selected/expanded state and UTF-16 selection ranges when available. Writability is true, false or unknown on relevant controls; omitted state fields do not assert false. selection=unavailable means the reported range exceeds the observed text. Compact output omits ordinary Press/Pick (click), ShowMenu (right click) and ScrollToVisible actions; details=true lists every available action. Use the displayed action name with performSecondaryAction; ambiguous aliases require the complete name from details=true. Truncation reports the reached provider limit. Narrow the window or increase that provider limit before reading again.
|
||||
|
||||
|
||||
@@ -380,17 +380,17 @@
|
||||
"x": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
},
|
||||
"y": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
},
|
||||
"relative": {
|
||||
"type": "boolean",
|
||||
"const": true,
|
||||
"description": "Use integer thousandths of the window width and height, from 0 to 1000."
|
||||
"description": "Use integer coordinates from 0 to 999, normalized by 999 to the window width and height."
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -579,17 +579,17 @@
|
||||
"x": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
},
|
||||
"y": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
},
|
||||
"relative": {
|
||||
"type": "boolean",
|
||||
"const": true,
|
||||
"description": "Use integer thousandths of the window width and height, from 0 to 1000."
|
||||
"description": "Use integer coordinates from 0 to 999, normalized by 999 to the window width and height."
|
||||
},
|
||||
"mouseButton": {
|
||||
"type": "string",
|
||||
@@ -983,17 +983,17 @@
|
||||
"x": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
},
|
||||
"y": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
},
|
||||
"relative": {
|
||||
"type": "boolean",
|
||||
"const": true,
|
||||
"description": "Use integer thousandths of the window width and height, from 0 to 1000."
|
||||
"description": "Use integer coordinates from 0 to 999, normalized by 999 to the window width and height."
|
||||
},
|
||||
"deltaLinesX": {
|
||||
"type": "integer",
|
||||
@@ -1147,22 +1147,22 @@
|
||||
"fromX": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
},
|
||||
"fromY": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
},
|
||||
"toX": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
},
|
||||
"toY": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 1000
|
||||
"maximum": 999
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
|
||||
@@ -31,7 +31,7 @@ Use the workflow tool ONLY when the user explicitly asks for a workflow or for l
|
||||
|
||||
Start independent delegations with `subagent` or `subagent_fork` together in one assistant message and continue useful work while they run.
|
||||
|
||||
Use cua with method getCapabilities and params {} to inspect permissions and sessionLocked. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI attempts and ask the user to unlock macOS. A null sessionLocked means no usable console session was verified. After unlocking, read fresh state; screenshots from a locked desktop are not current observations. Use listApps or cua.getState to discover running applications, and findApps(query) to search installed applications by name, bundle ID or absolute path. getApp requests user consent before launching a stopped application in the background, or before observations and input on a running one. If access is denied or revoked, stop using that application until the user explicitly permits another authorization request. Never switch tools to bypass a refusal. Keep the returned appHandle; every app operation requires it. listWindows returns window IDs and point bounds. Raw getState returns AX text and snapshotId. Menu-bar entries are shown by default; includeMenus=true expands their contents. details=true adds identifiers, global-point bounds and complete action names. Both options also apply to getAXState. Each read replaces the indices for that app/window scope. Use its element indices only with that snapshotId. Raw screenshot and coordinate operations require a windowId. Coordinates are pixels in the latest returned screenshot, measured from its top-left. Set relative:true for integer thousandths from 0 to 1000 instead. The Host converts both modes to native window points; AX detail bounds remain global macOS points. A verified window can expose no AX elements. Its empty snapshot permits screenshots and coordinate input; element-index actions require an element from a fresh AX observation.
|
||||
Use cua with method getCapabilities and params {} to inspect permissions and sessionLocked. If sessionLocked is true or an operation reports SESSION_LOCKED, stop UI attempts and ask the user to unlock macOS. A null sessionLocked means no usable console session was verified. After unlocking, read fresh state; screenshots from a locked desktop are not current observations. Use listApps or cua.getState to discover running applications, and findApps(query) to search installed applications by name, bundle ID or absolute path. getApp requests user consent before launching a stopped application in the background, or before observations and input on a running one. If access is denied or revoked, stop using that application until the user explicitly permits another authorization request. Never switch tools to bypass a refusal. Keep the returned appHandle; every app operation requires it. listWindows returns window IDs and point bounds. Raw getState returns AX text and snapshotId. Menu-bar entries are shown by default; includeMenus=true expands their contents. details=true adds identifiers, global-point bounds and complete action names. Both options also apply to getAXState. Each read replaces the indices for that app/window scope. Use its element indices only with that snapshotId. Raw screenshot and coordinate operations require a windowId. Coordinates are pixels in the latest returned screenshot, measured from its top-left. Set relative:true for integer coordinates from 0 to 999, normalized by 999 to the window width and height. The Host converts both modes to native window points; AX detail bounds remain global macOS points. A verified window can expose no AX elements. Its empty snapshot permits screenshots and coordinate input; element-index actions require an element from a fresh AX observation.
|
||||
|
||||
Initialize the object API once in persistent PTC. Reuse its variables in later calls; put temporary declarations inside { ... } blocks to avoid redeclaration errors. All object methods are async. getAXState returns text, not a structured state object. Its text separates values and placeholders, and reports subroles, help, URLs, disabled/focused/selected/expanded state and UTF-16 selection ranges when available. Writability is true, false or unknown on relevant controls; omitted state fields do not assert false. selection=unavailable means the reported range exceeds the observed text. Compact output omits ordinary Press/Pick (click), ShowMenu (right click) and ScrollToVisible actions; details=true lists every available action. Use the displayed action name with performSecondaryAction; ambiguous aliases require the complete name from details=true. Truncation reports the reached provider limit. Narrow the window or increase that provider limit before reading again.
|
||||
|
||||
|
||||
Reference in new issue
Block a user