Merge pull request #4125 from deepseek-harness/worktree/benchmark-trajectory-observer-20260913

fix(benchmarks): sample Trajectory readiness on animation frames

.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.i18n.yaml

@@ -20,9 +20,12 @@
 /agent-note-frontend-large-session-performance-budgets/calibration/standard-hosted-expectations-and-input-scheduling:
   en: 346cd1b3d5acf0ea
   zh: 42e08cd387f982c8
+/agent-note-frontend-large-session-performance-budgets/calibration/trajectory-readiness-polling:
+  en: bdcfefd7358e335b
+  zh: f718f66ca5944203
 /agent-note-frontend-large-session-performance-budgets/alternatives-considered:
-  en: 1472bd425ef0daec
-  zh: a89811758d70331e
+  en: 4d2a4a64ccd14507
+  zh: 8a4aaf4d4abea13d
 /agent-note-frontend-large-session-performance-budgets/consequences:
   en: 3f4f510612875d54
   zh: 3a4af464f159bfe8

.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.md

@@ -72,6 +72,14 @@ A controlled mouse-refocus delay waits for the real DONE marker without pausing
 
 The current Trajectory acceptance limit is 650 ms, represented by a 520 ms target with the shared 1.25× headroom. Run 35095609422 records medians of 666.651723 ms and 630.843184 ms on its original attempt and retry. This explicit 4% relaxation of the former 625 ms limit accepts the retry median but still rejects the original median; it is a budget decision, not a measured product speedup. Recorded retry samples exercise the accepted range, and the same verdict assertion rejects 651 ms. Other endpoint budgets and workload parameters remain unchanged.
 
+### Trajectory readiness polling
+
+Trajectory activation samples its visible searchbox and a visible record inside `[data-trajectory-scroll]` on animation frames. Playwright's selector polling uses increasing delays up to 500 ms, so DOM readiness can precede the observer by hundreds of milliseconds. The trusted tab click, its actionability checks and the final two-frame wait remain timed; workload and the current 650 ms budget are unchanged.
+
+A controlled macOS arm64 diagnostic on Node 24.19.0, Chromium 149.0.7827.55 and Playwright 1.61.1 makes the searchbox and record ready after 300 or 350 ms. Selector waits take 793.082/791.053 ms and observe readiness 493.5/439.2 ms late; frame sampling takes 307.324/357.423 ms and observes it 8.1/7.9 ms late. This isolates conditional observer delay, not product latency or the cause of the first #4119 CI miss.
+
+Complete local 240-turn runs produce Trajectory samples of 157.795/146.737/148.571 ms before the observer change and 130.184/191.247/157.543 ms afterward, with medians 148.571 and 157.543 ms. Both runs pass every fixed endpoint budget; no local speedup or hosted calibration is claimed. The readiness regression rejects a hidden searchbox or record despite a visible unrelated table row; restoring the original selector waits makes that test fail.
+
 ## Alternatives considered
 
 **Use the Node fold as paint evidence.** Rejected because it never performs DOM mutation, layout, or browser scheduling. The focused reconnect case likewise makes no GUI speed claim.
@@ -82,6 +90,8 @@ The current Trajectory acceptance limit is 650 ms, represented by a 520 ms targe
 
 **Search the entire loaded history for every stream marker.** Rejected because Playwright injects text and accessibility scans into the same renderer whose CPU the benchmark measures. Scoping queries to the composer and latest Assistant preserves visible completion checks without making observer cost proportional to loaded history.
 
+**Raise the Trajectory budget or retry failed samples.** This retains the measured selector backoff and can accept slower product behavior. Frame-based readiness removes that observer delay without changing the budget.
+
 **Measure stream CPU alone.** Rejected because transport stalls and final-settlement delays can leave main-thread CPU low. The independent input, first-reply, and complete-wall budgets cover those waits.
 
 ## Consequences

.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.zh.md

@@ -72,6 +72,14 @@ Node 对话折叠很快，并不能证明浏览器能绘制长对话或在流式
 
 当前 Trajectory 验收上限为 650 ms，由 520 ms 目标与共享的 1.25× 余量表示。运行 35095609422 的首次执行与重试分别记录中位数 666.651723 ms 和 630.843184 ms。相较于原先的 625 ms 上限，这次明确的 4% 放宽接受重试中位数，但仍拒绝首次中位数；这是预算决策，不代表测得产品提速。记录的重试样本覆盖新增的接受区间，同一个判定断言拒绝 651 ms。其他终点预算与工作负载参数保持不变。
 
+### Trajectory 就绪轮询
+
+Trajectory 激活在动画帧上采样可见搜索框与 `[data-trajectory-scroll]` 内的可见记录。Playwright 选择器轮询使用逐步增加、最高 500 ms 的延迟，因此 DOM 可能比观察器早数百毫秒就绪。真实 tab 点击、其可交互性检查与最后两帧等待仍计时；工作负载与当前 650 ms 预算不变。
+
+在 macOS arm64、Node 24.19.0、Chromium 149.0.7827.55 和 Playwright 1.61.1 上，受控诊断使搜索框与记录在 300 或 350 ms 后就绪。选择器等待耗时 793.082/791.053 ms，晚 493.5/439.2 ms 观察到就绪；动画帧采样耗时 307.324/357.423 ms，晚 8.1/7.9 ms 观察到就绪。这隔离了有条件发生的观察器延迟，不代表产品延迟，也不能证明 #4119 首次 CI 超限的原因。
+
+完整本地 240 轮运行在修改观察器前得到 Trajectory 样本 157.795/146.737/148.571 ms，修改后为 130.184/191.247/157.543 ms，中位数分别为 148.571 和 157.543 ms。两次运行均通过所有固定终点预算；不声称本地提速或托管环境校准。即使存在可见的无关表格行，就绪回归测试仍拒绝隐藏的搜索框或记录；恢复原选择器等待会使该测试失败。
+
 ## 考虑过的替代方案
 
 **用 Node 折叠作为绘制证据。** 拒绝，因为它不执行 DOM 修改、布局或浏览器调度。聚焦重连用例同样不声称 GUI 提速。
@@ -82,6 +90,8 @@ Node 对话折叠很快，并不能证明浏览器能绘制长对话或在流式
 
 **每次流式标记查找都搜索全部已加载历史。** 拒绝，因为 Playwright 把文本与无障碍扫描注入同一个被测 CPU 的渲染进程。将查询限定在输入框和最新 Assistant 中，可保留可见完成检查，同时避免观察器成本随已加载历史增长。
 
+**提高 Trajectory 预算或重试失败样本。** 这会保留已测得的选择器退避，并可能接受更慢的产品行为。基于动画帧的就绪观察移除该观察器延迟，不改变预算。
+
 **只测量流式 CPU。** 拒绝，因为传输停顿和最终结算延迟可能不增加主线程 CPU。独立的输入、首段回复和完整壁钟预算覆盖这些等待。
 
 ## 影响

benchmarks/long-session-browser/README.i18n.yaml

@@ -9,5 +9,5 @@
   en: e52e52132fec9fbd
   zh: fd2265d32153358a
 /long-session-browser-benchmark/measurements:
-  en: 5182df5387e843a9
-  zh: 3e67d96252c27400
+  en: dd010ca8b05cf27a
+  zh: 7921924e92ac145c

benchmarks/long-session-browser/README.md

@@ -12,6 +12,8 @@ The required Chromium workflow in [long-session.bench.ts](long-session.bench.ts)
 
 Three fresh browser processes and scaffold worlds produce raw samples and median verdicts. Open and paging end after the expected transcript state and two animation frames; this includes a rendering opportunity, not a hardware presentation timestamp. Paging reports every page and gates the median of each sample’s slowest page. Stream reports first visible reply, trusted draft typing, complete reply wall time, and Chromium main-thread task duration. Enter submits from the focused composer; draft typing retains that focus without a mouse click. Reply-marker lookups and the input-event text witness read only the latest Assistant step, avoiding repeated whole-history text and accessibility scans. The input witness is installed before submission, and draft typing starts as soon as the first marker is visible, without an extra pre-input animation-frame wait. Reply markers are sampled on animation frames, with visible text required inside the latest step. The first observation captures marker state and focus in the browser; its diagnostics are retrieved after typing so they add no pre-input round trip. Diagnostics also include browser-clock timestamps and focus at the first input event. They do not pause replay; a delayed first observation or input can still fail overlap. The actual first input event must observe an unfinished reply; completion waits for the new rendered turn-tail after Host settlement. After measurement, a trusted keystroke after DONE must fail the same overlap assertion. Open, the slowest older page, and first Trajectory use standard-hosted expectations of 900/700/520 ms. Shared 1.25× headroom gives limits of 1125/875/650 ms respectively; stream endpoint overhead budgets are unchanged. Heap after forced GC and DOM counts are diagnostics, not leak budgets.
 
+Trajectory readiness is sampled on animation frames from the visible searchbox and a visible record inside its own table. The trusted tab click and final two-frame wait remain inside the measured interval; hidden records and unrelated table rows cannot satisfy readiness.
+
 The fixture reserves an empty system head before the first user message, with each user message inside its step. It contains mixed-language prompts, prose, reasoning, 20 code fences, and 40 synthetic tool results. Every historical Assistant includes a compact stream built by the production accumulator from matching reasoning, text, tool arguments, usage, and finish chunks. No model, tool, external network, recorded Session, or private Harness home supplies its content. Streaming uses 120 text deltas at 16 ms replay pacing through the real composer, agent loop, transport, and persistence.
 
 The [decision record](../../.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.md) owns calibration, exclusions, and alternatives. The larger [manual diagnostic](../../apps/web/tests/complex-history.perf.ts) remains separate.

benchmarks/long-session-browser/README.zh.md

@@ -12,6 +12,8 @@
 
 三个全新浏览器进程与 scaffold 环境产生原始样本及中位数判定。打开和分页在预期 transcript（文本记录）状态出现且经过两次动画帧后结束；这包含一次渲染机会，而非硬件显示时间戳。分页报告每一页，并对各样本最慢分页时间的中位数执行预算检查。流式报告首次可见回复、受信任的草稿键入、完整回复壁钟时间和 Chromium 主线程任务时间。Enter 从已聚焦的输入框提交；草稿键入保留该焦点，不执行鼠标点击。回复标记查找与输入事件文本观察器仅读取最新 Assistant 步骤，避免重复进行全量历史文本扫描和无障碍扫描。输入事件文本观察器在提交前安装，首个标记可见后立即开始草稿键入，不额外等待输入前动画帧。回复标记在动画帧上采样，要求文本在最新步骤内可见。首次观察在浏览器内记录标记状态与焦点；诊断在键入后取回，不增加输入前的通信往返。诊断还包含首个输入事件的浏览器时钟时间戳与焦点。诊断不会暂停回放；首次观察或输入延迟仍可能导致重叠失败。实际首个输入事件必须观察到未完成的回复；完整回复的测量会在 Host 结算后等待新 turn-tail 渲染完成。测量后，在 DONE 之后发送的受信任按键必须无法通过同一个重叠断言。打开、最慢的较早页面和首次 Trajectory 使用标准托管预期 900/700/520 ms。共享的 1.25× 余量分别产生 1125/875/650 ms 上限；流式终点的额外开销预算不变。强制 GC 后的 heap 与 DOM 数量仅供诊断，不作为泄漏预算。
 
+Trajectory 就绪状态在动画帧上采样，要求搜索框可见且其自身表格中存在可见记录。真实 tab 点击和最后的两帧等待仍计入测量区间；隐藏记录与无关表格行不能满足就绪条件。
+
 fixture（测试前置数据）在首条用户消息前保留空 system 头节点，每条用户消息都位于其步骤内。它包含混合语言提示词、正文、推理（reasoning）、20 个围栏代码块和 40 个合成工具结果。每条历史 Assistant 都含紧凑流，由生产 accumulator 从匹配的推理、文本、工具参数、usage 和 finish 分片构建。其内容不来自模型、工具、外部网络、录制会话或私有 Harness 主目录。流式回复以 16 ms 回放间隔发送 120 个文本 delta，经过真实输入框、agent loop（智能体循环）、传输与持久化。
 
 [决策记录](../../.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.zh.md)拥有校准、排除项与替代方案。更大规模的[手动诊断](../../apps/web/tests/complex-history.perf.ts)保持独立。

benchmarks/long-session-browser/long-session.bench.ts

@@ -68,6 +68,17 @@ async function waitForReplyMarker(page: Page, marker: string, timeout = 30000) {
   }, { marker, first: FIRST, done: DONE }, { polling: 'raf', timeout })
 }
 
+async function waitForTrajectory(page: Page, timeout = 30000): Promise<void> {
+  // Selector retry backoff can delay observation after the view is already ready.
+  const ready = await page.waitForFunction(() => {
+    const search = document.querySelector('input[type="search"][aria-label="Search trajectory"]')
+    if (!search?.checkVisibility({ checkVisibilityCSS: true })) return false
+    return Array.from(document.querySelectorAll('[data-trajectory-scroll] [data-trajectory-row-key]'))
+      .some(row => row.checkVisibility({ checkVisibilityCSS: true }))
+  }, undefined, { polling: 'raf', timeout })
+  await ready.dispose()
+}
+
 async function watchInputOverlap(composer: Locator): Promise<void> {
   await composer.evaluate((element, markers) => {
     element.removeAttribute('data-benchmark-input-witness')
@@ -128,6 +139,22 @@ it('waits for visible marker text in the latest Assistant step', async () => {
   }
 })
 
+it('waits for visible Trajectory controls and records rather than unrelated table rows', async () => {
+  const browser = await chromium.launch({ headless: true })
+  try {
+    const page = await browser.newPage()
+    await page.setContent('<input type="search" aria-label="Search trajectory" style="visibility:hidden"><table><tbody><tr><td>unrelated</td></tr></tbody></table><div data-trajectory-scroll><table><tbody><tr data-trajectory-row-key="record"><td>record</td></tr></tbody></table></div>')
+    await expect(waitForTrajectory(page, 100)).rejects.toThrow('Timeout')
+    await page.locator('[data-trajectory-row-key]').evaluate(element => { element.style.display = 'none' })
+    await page.locator('input').evaluate(element => { element.style.visibility = 'visible' })
+    await expect(waitForTrajectory(page, 100)).rejects.toThrow('Timeout')
+    await page.locator('[data-trajectory-row-key]').evaluate(element => { element.style.display = 'table-row' })
+    await waitForTrajectory(page)
+  } finally {
+    await browser.close()
+  }
+})
+
 it('opens, pages, navigates and streams into a 240-turn browser history', async () => {
   if (webSnapshotMode() !== 'replay') throw new Error('browser benchmarks require keyless replay mode')
   const samples: { open: number; page: number; trajectory: number; first: number; streamTask: number; streamWall: number; input: number; inputOverlapped: boolean; heapMb: number; nodes: number }[] = []
@@ -173,8 +200,7 @@ it('opens, pages, navigates and streams into a 240-turn browser history', async
           }
           const trajectory = await measure(page, async () => {
             await page.getByRole('tab', { name: 'Trajectory', exact: true }).click()
-            await page.getByRole('searchbox', { name: 'Search trajectory', exact: true }).waitFor()
-            await page.getByRole('row').last().waitFor()
+            await waitForTrajectory(page)
           })
           await page.getByRole('tab', { name: 'Chat', exact: true }).click()
           await page.waitForFunction(({ selector, expected }) => document.querySelectorAll(selector).length === expected, { selector: TAIL, expected: HISTORY_TURNS })