fix(snapshot): bind fixture servers to OS-assigned ports

This commit is contained in:
Yichen Jiang committed 2026-08-28 21:37:38 +08:00
1 parent 84df0f11ae
commit 9055a8af3a
8 files changed
+72 -17

No files matched your search

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-08-24-session-log-snapshot-corpus.md
2026-08-24-session-log-snapshot-corpus.md: 328f0346554158d531dbda4b31a28277e37cc6dc
2026-08-24-session-log-snapshot-corpus.zh.md: 374ea2a1939e6e063f348e21fb74642371c345ac
2026-08-24-session-log-snapshot-corpus.md: 8b2f98e0a691e3085ff2286af3183048209f7ab8
2026-08-24-session-log-snapshot-corpus.zh.md: 19952f05c4803d50a6e3c7c987cadd9482d5e87b
@@ -18,6 +18,8 @@ This decision supersedes the ACP-specific placement and controller ownership in
The recorded session remains the primary input and expected output. Human-originated messages drive the selected public interface, recorded assistant chunks drive deterministic model replay, and the normalized persisted result must equal the fixture. Parent and child sessions share one typed redaction map. Committed fixtures contain relationship-preserving identity tokens and replace request system prompts and tool schemas with tokens; each distinct header class retains one explicit sidecar owner.
Scenario-owned HTTP fixtures separate the stable authority recorded in the session from their transport listener. Each fixture binds loopback port `0`, lets the operating system allocate and bind the port atomically, and maps the recorded URL or endpoint through the real provider to that listener. Any process-global transport interception matches only the recorded endpoint, is owned by the fixture fiber, and is restored before the listener closes.
Every existing ACP scenario receives a behavior-preserving destination. Ordinary one-shot behavior uses the headless profile, persistent machine control uses the SDK profile, and only ACP protocol behavior remains ACP-owned. Web scenarios driven by a recorded session join the corpus and retain their ARIA or geometry expected output as secondary evidence. Web and package tests without a recorded-session source keep owner-local expected output and stop using snapshot paths or filenames.
Workspace inputs remain scenario-local. A mutating scenario compares a complete expected final workspace that record and refresh never rewrite, so a model or tool self-report cannot satisfy the test. Existing intentional session reuse remains an explicit acyclic owner reference; the corpus adds no workspace inheritance or general fixture-merging mechanism.
@@ -34,6 +36,10 @@ Workspace inputs remain scenario-local. A mutating scenario compares a complete
**Deduplicate workspaces and recorded sessions automatically.** The current workspace duplication is small and intentional locality is easier to review. Only existing semantic session reuse justifies an explicit reference.
**Bind the recorded URL's numeric port.** A stable listener port keeps transport and transcript values identical, but concurrent snapshot jobs on one host share the network namespace and race for that port.
**Probe an unused port before launching the scenario.** Releasing a probed port before the child binds it creates a time-of-check/time-of-use race. Binding port `0` inside the owning process keeps allocation and ownership atomic.
## Invariants
- Every existing recorded-session scenario has one passing replacement before its old owner is removed.
@@ -43,11 +49,12 @@ Workspace inputs remain scenario-local. A mutating scenario compares a complete
- Mutating scenarios verify their final workspace externally.
- Owner-local process expectations use `*.expected.e2e.ts` and a separate built-output gate.
- Source and built adapters install replay-only packages in isolated profile fallbacks; distinct prompt-section orders keep their request headers byte-identical.
- Scenario HTTP fixtures bind OS-assigned loopback ports while preserving their recorded model-visible authorities.
- Source and built launch modes, browser replay, SDK projections, packaged Python runtime cases, documentation gates, and repository hygiene pass.
## Consequences
The corpus makes controller ownership visible: ordinary Agent behavior no longer inherits ACP protocol output, SDK and Web projections retain their interface-specific evidence, and only ACP cancellation and permission exchanges remain ACP-owned. Contributors review one normalized session diff plus the sidecars or UI expectations that add independent evidence. Adding a composition requires a manifest class pin; adding a volatile identity requires a typed relationship-preserving redaction rule rather than a broader text scrubber.
The corpus makes controller ownership visible: ordinary Agent behavior no longer inherits ACP protocol output, SDK and Web projections retain their interface-specific evidence, and only ACP cancellation and permission exchanges remain ACP-owned. Contributors review one normalized session diff plus the sidecars or UI expectations that add independent evidence. Adding a composition requires a manifest class pin; adding a volatile identity requires a typed relationship-preserving redaction rule rather than a broader text scrubber. Concurrent jobs can replay network-backed fixtures without reserving repository-wide ports, at the cost of a fixture-local mapping between the recorded authority and its transport listener.
## Risks
@@ -18,6 +18,8 @@ Status: implemented
录制会话仍是主要输入和预期输出。来自用户的消息驱动所选公开接口,录制的 assistant chunk 驱动确定性模型回放,规范化后的持久化结果必须等于 fixture。父会话和子会话共享同一类型化脱敏映射。提交的 fixture 使用保留关系的身份 token,并将请求 system prompt 和工具 schema 替换为 token;每个不同 header 类仍保留一个显式 sidecar 所有者。
场景拥有的 HTTP fixture 将会话中录制的稳定 authority 与传输 listener 分离。每个 fixture 在回环地址上绑定端口 `0`,由操作系统以一次原子操作分配并绑定端口,再将录制的 URL 或 endpoint 通过真实 provider 映射到该 listener。任何进程全局传输拦截只匹配录制 endpoint,由 fixture fiber 拥有,并在关闭 listener 前恢复。
每个现有 ACP 场景都获得一个保留行为的目标。普通单次行为使用 headless profile,需要持久机器控制的行为使用 SDK profile,只有 ACP 协议行为继续归 ACP 所有。由录制会话驱动的 Web 场景加入该语料,并保留其 ARIA 或几何预期输出作为辅助证据。没有录制会话来源的 Web 和包级测试保留归属方本地的预期输出,并停止使用快照路径或文件名。
Workspace 输入继续归各场景本地所有。变更文件的场景比较完整的预期最终 workspace,record 与 refresh 绝不改写该预期,因此模型或工具的自报结果无法满足测试。现有的有意会话复用继续使用显式、无环的所有者引用;语料不增加 workspace 继承或通用 fixture 合并机制。
@@ -34,6 +36,10 @@ Workspace 输入继续归各场景本地所有。变更文件的场景比较完
**自动去重 workspace 和录制会话。** 当前 workspace 重复很少,有意保持本地性更易审查。只有现有的语义会话复用值得显式引用。
**直接绑定录制 URL 的数值端口。** 稳定 listener 端口使传输值与 transcript 值一致,但同一主机上的并发快照 job 共享网络命名空间,会争用该端口。
**在启动场景前探测未使用端口。** 子进程绑定前释放已探测端口会产生检查时间与使用时间竞态。在拥有该端口的进程内绑定端口 `0`,可使分配与所有权保持原子性。
## Invariants
- 每个现有录制会话场景都在移除旧所有者之前拥有一个通过的替代场景。
@@ -43,11 +49,12 @@ Workspace 输入继续归各场景本地所有。变更文件的场景比较完
- 变更内容的场景从外部验证最终 workspace。
- 所属位置的进程预期使用 `*.expected.e2e.ts`,并由单独的构建产物门禁运行。
- 源码与构建适配器在隔离的 profile fallback 中安装仅回放包;不同的提示词 section 顺序值使两种模式的请求 header 保持字节一致。
- 场景 HTTP fixture 绑定由操作系统分配的回环端口,同时保留录制的模型可见 authority。
- 源码和构建启动模式、浏览器回放、SDK 投影、打包 Python 运行时场景、文档门禁和仓库卫生检查通过。
## Consequences
该语料让控制器所有权可见:普通 Agent 行为不再继承 ACP 协议输出,SDK 和 Web 投影保留各自接口专有的证据,只有 ACP 取消与权限交换仍归 ACP 所有。贡献者审查一份规范化会话差异,以及提供独立证据的 sidecar 或 UI 预期。新增组合必须提供 manifest 类别 pin;新增易变身份必须添加保留关系的带类型脱敏规则,而不是扩大文本清洗范围。
该语料让控制器所有权可见:普通 Agent 行为不再继承 ACP 协议输出,SDK 和 Web 投影保留各自接口专有的证据,只有 ACP 取消与权限交换仍归 ACP 所有。贡献者审查一份规范化会话差异,以及提供独立证据的 sidecar 或 UI 预期。新增组合必须提供 manifest 类别 pin;新增易变身份必须添加保留关系的带类型脱敏规则,而不是扩大文本清洗范围。并发 job 可以回放依赖网络的 fixture,而无需预留仓库级端口,代价是 fixture 内需要维护录制 authority 与传输 listener 的映射。
## Risks
+13
View File
@@ -507,6 +507,19 @@ describe('headless recorded-session snapshots', () => {
}
})
it('binds scenario HTTP fixtures only to OS-assigned ports', async () => {
for (const scenario of scenarios) {
const fixtureNames = (await readdir(scenario.dir)).filter(name => name.endsWith('.mjs'))
for (const fixtureName of fixtureNames) {
const source = await readFile(join(scenario.dir, fixtureName), 'utf8')
const boundPorts = [...source.matchAll(/\.listen\(\s*([^,\s)]+)/gu)].map(match => match[1])
for (const port of boundPorts) {
expect(port, `${scenario.name}/${fixtureName}: listener port`).toBe('0')
}
}
}
})
it('stores session-owned inputs with typed redaction and no ACP transcript', async () => {
for (const scenario of scenarios) {
const fixtures = await fixtureSessions(scenario)
@@ -1,15 +1,15 @@
/**
* Deterministic HTTP provider for the web-fetch snapshot scenario: a small
* HTML page (headings, named entities, a GFM table, nested formatting) on a
* fixed loopback port behind the real address-pinned transport. Recording and
* replay therefore exercise fetch and markdown rendering without
* external network. The port is fixed because the fetched URL is recorded.
* OS-assigned loopback port behind the real address-pinned transport. Recording
* and replay therefore exercise fetch and markdown rendering without external
* network while retaining the recorded request URL.
*/
import { createServer } from 'node:http'
import { HttpFetchProvider } from '@deepseek-ai/dsh-web-fetch-http'
/** Fixed loopback port the scenario prompt points `web_fetch` at. */
const PORT = 43117
/** Model-visible URL retained by the recorded session. */
const RECORDED_URL = 'http://public.test:43117/menu.html'
const PAGE = `<!doctype html>
<html><head><title>Menu</title><style>.x{color:red}</style><script>ignored()</script></head>
@@ -40,7 +40,7 @@ const LIMITS = {
* Register the deterministic provider and start its loopback server.
* @param ctx - Cordis context; the effect disposes the server with the fiber.
*/
export function apply(ctx) {
export async function apply(ctx) {
const server = createServer((req, res) => {
if (req.url === '/menu.html') {
res.writeHead(200, { 'content-type': 'text/html; charset=utf-8' })
@@ -52,14 +52,19 @@ export function apply(ctx) {
})
const listening = new Promise((resolve, reject) => {
server.once('error', reject)
server.listen(PORT, '127.0.0.1', () => resolve(undefined))
server.listen(0, '127.0.0.1', () => resolve(undefined))
})
void listening.catch(() => undefined)
await listening
const address = server.address()
if (address === null || typeof address === 'string') {
throw new Error('web-fetch-fixture-server: loopback listener has no TCP address')
}
const transportUrl = new URL(RECORDED_URL)
transportUrl.port = String(address.port)
// The fixture must never hold the process open past protocol shutdown.
server.unref()
const resolveAddresses = async (hostname) => {
await listening
if (hostname !== 'public.test') throw new Error(`unexpected snapshot hostname: ${hostname}`)
return [{ address: '127.0.0.1', family: 4 }]
}
@@ -71,5 +76,14 @@ export function apply(ctx) {
server.closeAllConnections()
})
}, 'web-fetch-fixture-server')
ctx.web.registerFetchProvider(new HttpFetchProvider(LIMITS, resolveAddresses))
const provider = new HttpFetchProvider(LIMITS, resolveAddresses)
ctx.web.registerFetchProvider({
id: provider.id,
available: () => provider.available(),
fetch: async (request, signal) => {
if (request.url !== RECORDED_URL) throw new Error(`unexpected snapshot URL: ${request.url}`)
const result = await provider.fetch({ url: transportUrl.toString() }, signal)
return { ...result, url: RECORDED_URL }
},
})
}
@@ -29,4 +29,5 @@
name: '@deepseek-ai/dsh-web-search-deepseek'
config:
apiKey: snapshot-key
# The fixture maps this recorded authority to its OS-assigned listener.
baseURL: http://127.0.0.1:43118/anthropic/v1
@@ -7,4 +7,5 @@
name: '@deepseek-ai/dsh-web-search-deepseek'
config:
apiKey: snapshot-key
# The fixture maps this recorded authority to its OS-assigned listener.
baseURL: http://127.0.0.1:43118/anthropic/v1
@@ -1,8 +1,8 @@
/** Deterministic authentication failure for the search endpoint guidance snapshot. */
import { createServer } from 'node:http'
/** Fixed loopback port recorded in the provider diagnostic. */
const PORT = 43118
/** Model-visible endpoint retained by the recorded session. */
const RECORDED_ENDPOINT = 'http://127.0.0.1:43118/anthropic/v1/messages'
/** Cordis plugin name. */
export const name = 'web-search-error-fixture'
@@ -20,10 +20,22 @@ export async function apply(ctx) {
})
await new Promise((resolve, reject) => {
server.once('error', reject)
server.listen(PORT, '127.0.0.1', () => resolve(undefined))
server.listen(0, '127.0.0.1', () => resolve(undefined))
})
const address = server.address()
if (address === null || typeof address === 'string') {
throw new Error('web-search-error-fixture: loopback listener has no TCP address')
}
const transportEndpoint = `http://127.0.0.1:${String(address.port)}/anthropic/v1/messages`
const originalFetch = globalThis.fetch
const fixtureFetch = (input, init) => originalFetch(
typeof input === 'string' && input === RECORDED_ENDPOINT ? transportEndpoint : input,
init,
)
globalThis.fetch = fixtureFetch
server.unref()
ctx.effect(() => async () => {
globalThis.fetch = originalFetch
await new Promise((resolve, reject) => {
server.close(error => error ? reject(error) : resolve(undefined))
server.closeAllConnections()