From 5d86fb0e79de3d1d32568224a0bafd684b8eedd9 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Wed, 30 Sep 2026 09:14:17 +0800 Subject: [PATCH] docs(benchmarks): record reconnect CI calibration --- .../testing/2026-09-06-frontend-performance-budgets.i18n.yaml | 4 ++-- .../testing/2026-09-06-frontend-performance-budgets.md | 4 +++- .../testing/2026-09-06-frontend-performance-budgets.zh.md | 4 +++- benchmarks/active-stream-reconnect/README.i18n.yaml | 4 ++-- benchmarks/active-stream-reconnect/README.md | 2 +- benchmarks/active-stream-reconnect/README.zh.md | 2 +- 6 files changed, 12 insertions(+), 8 deletions(-) diff --git a/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.i18n.yaml b/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.i18n.yaml index 891ae724e9..4dc12c0baf 100644 --- a/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.i18n.yaml +++ b/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.i18n.yaml @@ -18,8 +18,8 @@ en: d4d6aebafcdb8c12 zh: 112d3ca1a4ea8ae1 /agent-note-frontend-large-session-performance-budgets/calibration/standard-hosted-expectations-and-input-scheduling: - en: 8cb6299dbb42a9fe - zh: 51ef49d4fcaeb92f + en: 346cd1b3d5acf0ea + zh: 42e08cd387f982c8 /agent-note-frontend-large-session-performance-budgets/alternatives-considered: en: 1472bd425ef0daec zh: a89811758d70331e diff --git a/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.md b/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.md index 7a24a649db..36cd7458b3 100644 --- a/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.md +++ b/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.md @@ -58,7 +58,9 @@ All six browser samples report `inputOverlapped: true` and finish after the 241s ### Standard hosted expectations and input scheduling -Run 34033336246, job 101487216170 on standard hosted Ubuntu with two CPUs records reconnect replacements of 46.574411, 46.067910, and 44.193704 ms, with 23.028 MiB retained heap. The endpoint-specific expectation is 50 ms; the existing 1.25× headroom gives a 63 ms integer ceiling. The 30 MiB memory budget and shared machine factor remain unchanged. Browser open records 681.276514 and 541.051233 ms before the third sample fails input overlap; both exceed the historical 500 ms limit. Repeated hosted open measurements below set its expectation and ceiling. Deterministic controls pass these recorded values and reject values above the new ceilings through the same assertions as the measured verdicts. Complete repeated hosted verdicts remain required; the two open values are not a three-sample median. +Reconnect uses a 50 ms reference-machine allowance through the shared 2× CI time scale and 1.25× headroom, giving a 125 ms ceiling. Two isolated three-sample runs on Linux x64 EPYC 7763 / Node 24.18.0 measure 47.13 and 43.08 ms medians. Hosted Node 24.21.0 runs 36619857675 and 36621172285 measure 55.47 ms on Xeon 8573C and 65.69 ms on EPYC 9V74 with two available CPUs; their measured production path is unchanged. The latter run is 1.39–1.53× the local reconnect medians; its preparing-write and preparing-bash medians are 1.54–1.55× and 1.34–1.42× their local counterparts. The shared 2× factor covers these observed ratios; the raw samples do not isolate hardware from Node-version or scheduling effects. Both local and hosted reconnect runs retain 23.03 MiB, so the 30 MiB memory ceiling is unchanged. The same time assertion accepts recorded hosted medians, rejects the 65.69 ms median at the former 63 ms ceiling, and rejects synthetic 126 and 150 ms values at 125 ms. This is budget calibration, not a product speedup or a passing future CI verdict. + +Run 34033336246, job 101487216170 on standard hosted Ubuntu with two CPUs records reconnect replacements of 46.574411, 46.067910, and 44.193704 ms, with 23.028 MiB retained heap. Browser open records 681.276514 and 541.051233 ms before the third sample fails input overlap; both exceed the historical 500 ms limit. Repeated hosted open measurements below set its expectation and ceiling. Deterministic controls pass these recorded values and reject values above the new ceilings through the same assertions as the measured verdicts. Complete repeated hosted verdicts remain required; the two open values are not a three-sample median. A local diagnostic with temporary 3× Chromium CPU throttling reproduces the overlap failure: the first marker becomes visible at 1321 ms, two animation frames finish at 1370 ms, and the composer click finishes at 1660 ms; the actual input is trusted but already sees DONE. Removing the frame wait and installing the witness before Send still leaves a run with first visibility at 1415 ms and click completion at 1726 ms, after the original 992 ms scripted stream. The fixed 16 ms cadence keeps the same 120 deltas and payload, providing 1984 ms of scripted pacing for this workload. Only that pacing term changes in the complete-wall allowance (4484 ms); input, first-reply, and main-thread overhead allowances remain unchanged. With the same diagnostic slowdown, three 16 ms samples reach first visibility at 1307/1479/1599 ms and accept trusted input before DONE; their post-DONE controls reject it. The diagnostic is not a CPU-ratio calibration. Each measured sample still requires trusted input while FIRST is present and DONE absent; a post-measurement trusted key after DONE must fail that same assertion. Host settlement and the 241st rendered turn-tail remain completion witnesses. diff --git a/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.zh.md b/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.zh.md index 7afd4c1c95..8a6bbaff8e 100644 --- a/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.zh.md +++ b/.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.zh.md @@ -58,7 +58,9 @@ Node 对话折叠很快，并不能证明浏览器能绘制长对话或在流式 ### 标准托管预期与输入调度 -运行 34033336246，job 101487216170 在双 CPU 标准托管 Ubuntu 上记录重连替换时间 46.574411、46.067910 和 44.193704 ms，保留 heap 为 23.028 MiB。该终点的预期为 50 ms；现有 1.25× 余量产生向上取整后的 63 ms 上限。30 MiB 内存预算及共享机器倍率不变。浏览器打开记录 681.276514 和 541.051233 ms，第三个样本因输入重叠失败而中止；两个值均超过历史 500 ms 上限。下文的托管打开重复测量决定其预期与上限。确定性对照通过这些记录值，并使用与测量判定相同的断言拒绝超过新上限的值。仍需完整的托管重复运行判定；这两个打开值不是三样本中位数。 +重连使用 50 ms 参考机器额度，通过共享的 2× CI 时间倍率与 1.25× 余量得到 125 ms 上限。Linux x64 EPYC 7763 / Node 24.18.0 上两轮独立三样本运行的中位数为 47.13 和 43.08 ms。托管 Node 24.21.0 运行 36619857675 与 36621172285 均有两个可用 CPU，分别在 Xeon 8573C 和 EPYC 9V74 上测得 55.47 与 65.69 ms；两次被测的生产路径没有变化。后一轮为本地重连中位数的 1.39–1.53×，其中 preparing-write 与 preparing-bash 中位数分别为本地对应值的 1.54–1.55× 和 1.34–1.42×。共享 2× 系数覆盖这些实测比例；原始样本未区分硬件、Node 版本或调度各自的影响。本地和托管重连都保留 23.03 MiB，因此 30 MiB 内存上限不变。同一个时间断言接受托管记录中位数，在旧 63 ms 上限下拒绝 65.69 ms 中位数，并在 125 ms 上限下拒绝合成的 126 与 150 ms 值。这是预算校准，不代表产品提速，也不代表后续 CI 已通过。 + +运行 34033336246，job 101487216170 在双 CPU 标准托管 Ubuntu 上记录重连替换时间 46.574411、46.067910 和 44.193704 ms，保留 heap 为 23.028 MiB。浏览器打开记录 681.276514 和 541.051233 ms，第三个样本因输入重叠失败而中止；两个值均超过历史 500 ms 上限。下文的托管打开重复测量决定其预期与上限。确定性对照通过这些记录值，并使用与测量判定相同的断言拒绝超过新上限的值。仍需完整的托管重复运行判定；这两个打开值不是三样本中位数。 临时使用 3× Chromium CPU 降速的本地诊断复现重叠失败：首个标记在 1321 ms 可见，两次动画帧在 1370 ms 结束，输入框点击在 1660 ms 完成；实际输入是真实事件，但已看到 DONE。移除帧等待并在发送前安装观察器后，一次运行仍在 1415 ms 才看到首个标记，点击在 1726 ms 完成，晚于原先 992 ms 的脚本流。固定 16 ms 节奏保留相同的 120 个 delta 和负载，为该工作负载提供 1984 ms 脚本节奏。完整壁钟额度仅改变该节奏项（4484 ms）；输入、首段回复及主线程额外开销额度不变。在相同诊断降速下，三个 16 ms 样本在 1307/1479/1599 ms 达到首段可见状态，并接受 DONE 之前的真实输入；其 DONE 之后的对照拒绝该输入。该诊断不是 CPU 比率校准。每个测量样本仍要求真实输入发生时 FIRST 存在且 DONE 不存在；测量后在 DONE 之后发送的真实按键必须无法通过同一个断言。Host 结算和第 241 个已渲染 turn-tail 仍是完成证据。 diff --git a/benchmarks/active-stream-reconnect/README.i18n.yaml b/benchmarks/active-stream-reconnect/README.i18n.yaml index 25d326e31e..06b98b7145 100644 --- a/benchmarks/active-stream-reconnect/README.i18n.yaml +++ b/benchmarks/active-stream-reconnect/README.i18n.yaml @@ -3,5 +3,5 @@ # After editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write benchmarks/active-stream-reconnect/README.md /active-assistant-reconnect-benchmark: - en: 74be7a07bbc676b8 - zh: e6aeac1f27343c24 + en: 3fe52f5c243a6fde + zh: 89c0c51aa79f0604 diff --git a/benchmarks/active-stream-reconnect/README.md b/benchmarks/active-stream-reconnect/README.md index e75a41eba3..b9c7a64249 100644 --- a/benchmarks/active-stream-reconnect/README.md +++ b/benchmarks/active-stream-reconnect/README.md @@ -2,6 +2,6 @@ English | [中文](README.zh.md) -[reconnect.bench.client.ts](reconnect.bench.client.ts) measures the production Client fold when a reconnect carries an unfinished 100,000-delta reasoning prefix. A compiled private adapter reaches `ClientAssistantStream.replace()` without adding product exports. Three fresh plain-Node workers synthesize the compact baseline before timing; replacement time and retained heap after forced GC have separate median budgets. The next dense live frame must still be accepted. Standard hosted CI uses a 50 ms replacement expectation with the shared 1.25× headroom (63 ms ceiling); the retained-heap budget remains 30 MiB. Recorded-sample and synthetic-regression controls exercise the same time assertion as the worker verdict. +[reconnect.bench.client.ts](reconnect.bench.client.ts) measures the production Client fold when a reconnect carries an unfinished 100,000-delta reasoning prefix. A compiled private adapter reaches `ClientAssistantStream.replace()` without adding product exports. Three fresh plain-Node workers synthesize the compact baseline before timing; replacement time and retained heap after forced GC have separate median budgets. The next dense live frame must still be accepted. The 50 ms reference-machine expectation uses the shared 2× CI time scale and 1.25× headroom, giving a 125 ms ceiling; the retained-heap budget remains 30 MiB without time scaling. Recorded-sample and synthetic-regression controls exercise the same time assertion as the worker verdict. Build with `pnpm run build:bench`, then select `benchmarks/active-stream-reconnect` in `vitest.bench.config.ts`. This focused Node workload neither builds nor measures browser rendering. [Frontend performance budgets](../../.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.md) records calibration and exclusions. diff --git a/benchmarks/active-stream-reconnect/README.zh.md b/benchmarks/active-stream-reconnect/README.zh.md index 3d83120814..8e2f330e83 100644 --- a/benchmarks/active-stream-reconnect/README.zh.md +++ b/benchmarks/active-stream-reconnect/README.zh.md @@ -2,6 +2,6 @@ [English](README.md) | 中文 -[reconnect.bench.client.ts](reconnect.bench.client.ts) 测量重连携带一个包含 100,000 个 delta、尚未完成的 reasoning 前缀时，生产 Client 的折叠成本。编译后的私有适配器调用 `ClientAssistantStream.replace()`，不增加产品导出。三个全新的纯 Node worker 在计时前合成紧凑基线；替换时间与强制 GC 后的保留堆分别执行中位数预算检查。下一个稠密序号的实时 frame 仍须被接受。标准托管 CI 使用 50 ms 替换预期及共享的 1.25× 余量（上限为 63 ms）；保留 heap 预算仍为 30 MiB。实测样本和合成回归对照使用与 worker 判定相同的时间断言。 +[reconnect.bench.client.ts](reconnect.bench.client.ts) 测量重连携带一个包含 100,000 个 delta、尚未完成的 reasoning 前缀时，生产 Client 的折叠成本。编译后的私有适配器调用 `ClientAssistantStream.replace()`，不增加产品导出。三个全新的纯 Node worker 在计时前合成紧凑基线；替换时间与强制 GC 后的保留堆分别执行中位数预算检查。下一个稠密序号的实时 frame 仍须被接受。50 ms 参考机器预期使用共享的 2× CI 时间倍率与 1.25× 余量，得到 125 ms 上限；保留 heap 预算仍为 30 MiB，不应用时间倍率。实测样本和合成回归对照使用与 worker 判定相同的时间断言。 通过 `pnpm run build:bench` 构建，再在 `vitest.bench.config.ts` 中选择 `benchmarks/active-stream-reconnect`。这项针对 Node 的工作负载既不构建也不测量浏览器渲染。[前端性能预算](../../.agents/notes/implemented/testing/2026-09-06-frontend-performance-budgets.zh.md)记录校准与排除项。