Merge commit '060de4c547f020e1953c88b98b7ab749fd9ef20f' into worktree/session-format-06-v2-snapshot-rollout
This commit is contained in:
commit
7299d5e093
20 files changed
+214
-68
No files matched your search
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-01-v2-embedded-assistant-streams.md
|
||||
2026-09-01-v2-embedded-assistant-streams.md: e50af4848523c92daaac1bd8b1120c26cdf62bdc
|
||||
2026-09-01-v2-embedded-assistant-streams.zh.md: 34924a43150c34571a6c5f2e7837d1b662d2ec46
|
||||
2026-09-01-v2-embedded-assistant-streams.md: 805e52579be250ed372585c48fad434f7e15aadc
|
||||
2026-09-01-v2-embedded-assistant-streams.zh.md: 05e1a76f149a92c35af1ede28babfd2513773352
|
||||
@@ -33,7 +33,7 @@ The Web follow adapter opts into these process-local frames and adds the last du
|
||||
|
||||
The adjacent migration validates the complete frozen v1 artifact, groups chunks by turn, step, terminal boundary, and exact message provenance, and then substitutes one settlement per attempt. A successful group's chunks move into its message. An unclaimed group becomes `assistant/attempt` at the last consumed chunk's position. Unrelated interleaved events retain their relative order, and survivors receive dense v2 sequence numbers.
|
||||
|
||||
The edge remaps the finite declared reference inventory: envelope provenance, surface replacement endpoints, command source events, compaction ranges and shadowed lists, and title message lists. A reference to a consumed chunk refuses migration; it is never redirected to a settlement with different meaning. The edge also refuses an inherited cut that splits an attempt.
|
||||
The edge remaps the finite declared reference inventory: envelope provenance, surface replacement endpoints, command source events, compaction ranges and shadowed lists, and title message lists. The model-visible text of a validated `session/title-llm-request` remains byte-identical in the source sequence namespace while its `messageSeqs` field moves to the v2 namespace; target validation therefore does not reconstruct that text from remapped sequences. A reference to a consumed chunk refuses migration; it is never redirected to a settlement with different meaning. The edge also refuses an inherited cut that splits an attempt.
|
||||
|
||||
The v2 physical header requires `isSeeded` and stores no numeric cut. A seeded artifact marks its exact cut with `session/end-seed { inherited: true }`; decoding derives the cut from the last tagged marker. The v2 codec writes one durable event per physical row, range-encodes only `sourceEventSeqs`, and validates physical envelopes without freezing ordinary event vocabulary or payload additions. The v1-to-v2 target validator separately freezes the released-v2 inventory, while current restoration uses the installed Session vocabulary. Frozen v0 and v1 codecs retain packed-row decoding for their immutable historical generations.
|
||||
|
||||
@@ -45,7 +45,7 @@ Generation selection and publication follow the [released Session migration deci
|
||||
|
||||
## Verification
|
||||
|
||||
The compact-stream tests pin exact accumulation and expansion for text, reasoning, tool arguments, raw chunks, timestamp gaps, malformed records, and detached snapshots. The v1-to-v2 tests cover successful and failed attempts, interleaving, dense sequence and reference remapping, seed-cut insertion and split refusal, strict source and target validation, one-row v2 encoding, provenance ranges, raw and Zstandard publication, and no-write current reads.
|
||||
The compact-stream tests pin exact accumulation and expansion for text, reasoning, tool arguments, raw chunks, timestamp gaps, malformed records, and detached snapshots. The v1-to-v2 tests cover successful and failed attempts, interleaving, dense sequence and reference remapping, source-sequence title framing, seed-cut insertion and split refusal, strict source and target validation, one-row v2 encoding, backend-compatible provenance ranges, raw and Zstandard publication, and no-write current reads.
|
||||
|
||||
The manual performance acceptance measures static catalog-routing overhead against direct released-v2 restoration of the same already parsed physical rows across three runs, 100 warmup pairs, and 600 measured pairs. It does not compare v1 with v2 or time backend I/O. Every pooled median and p95 regression must remain within 5%; the accepted run's worst p95 regression was 3.150%. `--smoke` reports a non-gating diagnostic sample.
|
||||
|
||||
|
||||
+2
-2
@@ -33,7 +33,7 @@ Web follow adapter 显式选择接收这些进程本地 frame,并为每个 sta
|
||||
|
||||
相邻迁移会校验完整的冻结 v1 产物,按 turn、step、terminal boundary 与精确 message provenance 对 chunk 分组,再为每个 attempt 替换一个 settlement。成功分组的 chunk 移入其 message。未被认领的分组会在最后一个被消费 chunk 的位置变成 `assistant/attempt`。无关的交错事件保持相对顺序,存活事件获得密集 v2 序号。
|
||||
|
||||
该迁移边会重映射有限的已声明引用清单:信封 provenance、surface replacement 端点、command source event、compaction range 与 shadowed list,以及 title message list。指向被消费 chunk 的引用会使迁移失败;它绝不会被重定向到含义不同的 settlement。该迁移边也会拒绝切开 attempt 的继承切点。
|
||||
该迁移边会重映射有限的已声明引用清单:信封 provenance、surface replacement 端点、command source event、compaction range 与 shadowed list,以及 title message list。经过校验的 `session/title-llm-request` 模型可见文本会在源序号命名空间中保持逐字节不变,而它的 `messageSeqs` 字段会迁移到 v2 命名空间;因此目标校验不会根据重映射后的序号重建该文本。指向被消费 chunk 的引用会使迁移失败;它绝不会被重定向到含义不同的 settlement。该迁移边也会拒绝切开 attempt 的继承切点。
|
||||
|
||||
v2 物理 header 要求 `isSeeded`,且不存储数值切点。带 seed 的产物用 `session/end-seed { inherited: true }` 标记其精确切点;解码从最后一个 tagged marker 推导切点。v2 编解码器为每个持久事件写一条物理行,只对 `sourceEventSeqs` 做范围编码,并在不冻结普通事件词汇或 payload 新增项的前提下校验物理 envelope。v1-to-v2 target validator 会另行冻结 released-v2 清单,current restoration 则使用 installed Session 词汇。冻结的 v0 与 v1 编解码器继续为不可变历史 generation 解码 packed row。
|
||||
|
||||
@@ -45,7 +45,7 @@ Generation 选择与发布遵循[已发布 Session 迁移决策](2026-08-31-rele
|
||||
|
||||
## 验证
|
||||
|
||||
紧凑 stream 测试固定 text、reasoning、tool argument、raw chunk、时间戳间隔、格式错误 record 与分离 snapshot 的精确累积和展开。v1 到 v2 测试覆盖成功与失败 attempt、交错、密集序号与引用重映射、seed 切点插入与切分拒绝、严格源与目标校验、每行一个事件的 v2 编码、provenance range、原始与 Zstandard 发布,以及无写入的当前读取。
|
||||
紧凑 stream 测试固定 text、reasoning、tool argument、raw chunk、时间戳间隔、格式错误 record 与分离 snapshot 的精确累积和展开。v1 到 v2 测试覆盖成功与失败 attempt、交错、密集序号与引用重映射、源序号 title framing、seed 切点插入与切分拒绝、严格源与目标校验、每行一个事件的 v2 编码、与 backend 兼容的 provenance range、原始与 Zstandard 发布,以及无写入的当前读取。
|
||||
|
||||
手工 performance acceptance 会在三轮、100 组 warmup pair 与 600 组 measured pair 下,针对同一批已经解析的物理 row,把静态 catalog routing 与直接 released-v2 restoration 比较;它不比较 v1 与 v2,也不计入 backend I/O。每个 pooled median 与 p95 regression 都必须保持在 5% 以内;已接受运行的最差 p95 regression 为 3.150%。`--smoke` 报告不参与 gate 的诊断 sample。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-format-v0-to-v1/README.md
|
||||
README.md: 2ae4e4a20d07009766df917aca9cd46070fc3639
|
||||
README.zh.md: 64137fccae4ee144b6a0a22f95ae4aed18044643
|
||||
README.md: 1685831b49814b80979a0e54cb1a6a6d6b618a1d
|
||||
README.zh.md: 4a511920f4b364d730e9db2bb59e7b43ac47c63a
|
||||
@@ -38,7 +38,7 @@ const migratedV1 = sessionFormatV0ToV1.migrate(decodedV0)
|
||||
|
||||
`releasedV0SessionFormatCodec` reads the exact v0 header and physical rows, including packed assistant deltas and range-encoded provenance. `sessionFormatV0ToV1` normalizes and strictly validates a complete detached artifact. `releasedV1SessionFormatCodec` preserves the v1 physical layout without freezing the ordinary event vocabulary; the catalog restores current events against the installed Session package.
|
||||
|
||||
The alpha edge refuses every event type outside its frozen inventory, including an unknown event marked `ignorable: true`. It also refuses unexpected payload members. `tool/result.meta` and nested PTC `arguments` remain explicit opaque JSON fields and are preserved without Session-sequence interpretation. The disposition inventory separately names merge-extensible nested discriminants: unknown content-block `type`, message-source `kind`, assistant finish-reason `kind`, and `turn/end` reason `kind` arms remain owner-opaque JSON while their known arms receive structural validation.
|
||||
The alpha edge refuses every event type outside its frozen inventory, including an unknown event marked `ignorable: true`. It also refuses unexpected payload members. `tool/result.meta` and nested PTC `arguments` remain explicit opaque JSON fields and are preserved without Session-sequence interpretation. Unknown content-block `type`, message-source `kind`, assistant finish-reason `kind`, and `turn/end` reason `kind` arms remain owner-opaque JSON while their known arms receive structural validation.
|
||||
|
||||
The bounded historical normalizers convert `steering/message` to `user/message`, remove `turn/start.trigger`, convert retired `turn/end` reasons, add the current message wrappers and deterministic legacy message ids, and remove the obsolete `request/header.header.messagePrefix` duplicate. Retired `request/header-delta`, `mode/set`, and the `request/header` fallback reason refuse migration. No other event, reference, source, or payload fact may change.
|
||||
|
||||
|
||||
@@ -38,7 +38,7 @@ const migratedV1 = sessionFormatV0ToV1.migrate(decodedV0)
|
||||
|
||||
`releasedV0SessionFormatCodec` 读取精确的 v0 标头与物理行,包括打包的 Assistant 增量和范围编码的来源序号。`sessionFormatV0ToV1` 规范化并严格校验一个完整且分离的产物。`releasedV1SessionFormatCodec` 在不冻结普通事件词表的前提下保留 v1 物理布局;目录会根据已安装的 Session 包还原当前事件。
|
||||
|
||||
Alpha 迁移边会拒绝冻结清单之外的所有事件类型,包括带有 `ignorable: true` 标记的未知事件。它也会拒绝意外的 payload 成员。`tool/result.meta` 与嵌套 PTC `arguments` 是显式的不透明 JSON 字段;迁移会原样保留它们,不把其中的数字解释为 Session 序号。处置清单会单独指明可合并扩展的嵌套判别字段:未知 content-block `type`、message-source `kind`、assistant finish-reason `kind` 与 `turn/end` reason `kind` 分支保持 owner-opaque JSON,已知分支则接受结构校验。
|
||||
Alpha 迁移边会拒绝冻结清单之外的所有事件类型,包括带有 `ignorable: true` 标记的未知事件。它也会拒绝意外的 payload 成员。`tool/result.meta` 与嵌套 PTC `arguments` 是显式的不透明 JSON 字段;迁移会原样保留它们,不把其中的数字解释为 Session 序号。未知 content-block `type`、message-source `kind`、assistant finish-reason `kind` 与 `turn/end` reason `kind` 分支保持 owner-opaque JSON,已知分支则接受结构校验。
|
||||
|
||||
有限的历史规范化会把 `steering/message` 转换为 `user/message`、移除 `turn/start.trigger`、转换已停用的 `turn/end` reason、添加当前消息包装层与确定性的旧消息 id,并移除已停用且重复的 `request/header.header.messagePrefix`。已停用的 `request/header-delta`、`mode/set` 和 `request/header` fallback reason 会使迁移失败。除此之外,任何事件、引用、来源或 payload 事实都不得改变。
|
||||
|
||||
|
||||
@@ -4,8 +4,6 @@ export interface ReleasedV0PayloadDisposition {
|
||||
readonly optional: readonly string[]
|
||||
/** JSON members whose nested representation is intentionally owner-opaque. */
|
||||
readonly opaque: readonly string[]
|
||||
/** Nested discriminant paths whose unknown arms remain owner-opaque JSON. */
|
||||
readonly extensionArms: readonly string[]
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -20,13 +18,11 @@ export function defineReleasedPayloadDisposition(
|
||||
required: readonly string[],
|
||||
optional: readonly string[] = [],
|
||||
opaque: readonly string[] = [],
|
||||
extensionArms: readonly string[] = [],
|
||||
): ReleasedV0PayloadDisposition {
|
||||
return Object.freeze({
|
||||
required: Object.freeze([...required]),
|
||||
optional: Object.freeze([...optional]),
|
||||
opaque: Object.freeze([...opaque]),
|
||||
extensionArms: Object.freeze([...extensionArms]),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -35,32 +31,23 @@ const disposition = defineReleasedPayloadDisposition
|
||||
/**
|
||||
* Frozen released-v0 event and payload-member inventory.
|
||||
* Every listed member is preserved by the identity edge. Members in `opaque`
|
||||
* remain lossless JSON without nested Session-sequence interpretation. Paths
|
||||
* in `extensionArms` validate known variants and preserve unknown variants as
|
||||
* owner-opaque JSON, matching the merge-extensible runtime vocabulary.
|
||||
* remain lossless JSON without nested Session-sequence interpretation. Nested
|
||||
* merge-extensible discriminants validate known variants and preserve
|
||||
* unknown variants as owner-opaque JSON.
|
||||
*/
|
||||
export const RELEASED_V0_EVENT_DISPOSITIONS: Readonly<Record<string, ReleasedV0PayloadDisposition>> = Object.freeze({
|
||||
'agent-preset/selected': disposition(['agentPreset']),
|
||||
'agent/inbox/spliced': disposition(
|
||||
['target', 'start', 'inserted'],
|
||||
['removedCount', 'outcome'],
|
||||
[],
|
||||
['inserted[].content[].type', 'inserted[].source.kind'],
|
||||
),
|
||||
'approval/asked': disposition(['id', 'toolName'], ['callId', 'reason']),
|
||||
'approval/decided': disposition(['id', 'outcome']),
|
||||
'approval/policy': disposition(['policy'], ['source']),
|
||||
'assistant/chunk': disposition(
|
||||
['turn', 'step', 'chunk'],
|
||||
[],
|
||||
[],
|
||||
['chunk.blockType', 'chunk.block.type', 'chunk.reason.kind'],
|
||||
),
|
||||
'assistant/chunk': disposition(['turn', 'step', 'chunk']),
|
||||
'assistant/message': disposition(
|
||||
['turn', 'step', 'message'],
|
||||
['usage', 'interrupted'],
|
||||
[],
|
||||
['message.content[].type'],
|
||||
),
|
||||
'command/done': disposition(['commandId', 'kind'], ['text', 'sourceEventSeq']),
|
||||
'command/run': disposition(['commandId', 'name', 'source'], ['args']),
|
||||
@@ -70,8 +57,6 @@ export const RELEASED_V0_EVENT_DISPOSITIONS: Readonly<Record<string, ReleasedV0P
|
||||
'compaction/summary': disposition(
|
||||
['compactionId', 'summary', 'shadowedRange', 'shadowedSeqs', 'shadowedTokenCount', 'provider', 'model'],
|
||||
['sourceCommandId', 'maxTokens', 'usage', 'rawOutput', 'llmStreamCall'],
|
||||
[],
|
||||
['summary[].type', 'rawOutput[].type'],
|
||||
),
|
||||
'feedback/record': disposition(['text']),
|
||||
'goal/change': disposition(
|
||||
@@ -102,9 +87,6 @@ export const RELEASED_V0_EVENT_DISPOSITIONS: Readonly<Record<string, ReleasedV0P
|
||||
'session/title': disposition(['title', 'messageSeqs', 'source']),
|
||||
'session/title-llm-request': disposition(
|
||||
['titleProvider', 'messageSeqs', 'route', 'system', 'messages', 'maxTokens'],
|
||||
[],
|
||||
[],
|
||||
['messages[].content[].type', 'messages[].source.kind'],
|
||||
),
|
||||
'step/end': disposition(['turn', 'step']),
|
||||
'step/start': disposition(['turn', 'step']),
|
||||
@@ -115,12 +97,7 @@ export const RELEASED_V0_EVENT_DISPOSITIONS: Readonly<Record<string, ReleasedV0P
|
||||
'subagent/model-selection-policy': disposition(['allowedModels']),
|
||||
'team/member': disposition(['version', 'teamId', 'member']),
|
||||
'team/message/delivered': disposition(['version', 'teamId', 'messageId', 'targetId']),
|
||||
'team/message/queued': disposition(
|
||||
['version', 'teamId', 'message'],
|
||||
[],
|
||||
[],
|
||||
['message.content[].type'],
|
||||
),
|
||||
'team/message/queued': disposition(['version', 'teamId', 'message']),
|
||||
'team/task': disposition(['version', 'teamId', 'task']),
|
||||
'todo/write': disposition(['todos']),
|
||||
'tool-workflow/agent-end': disposition(['runId', 'seq', 'outcome']),
|
||||
@@ -132,7 +109,6 @@ export const RELEASED_V0_EVENT_DISPOSITIONS: Readonly<Record<string, ReleasedV0P
|
||||
['rootCallId', 'parentCallId', 'subCallId', 'name', 'arguments', 'isError', 'content'],
|
||||
[],
|
||||
['arguments'],
|
||||
['content[].type'],
|
||||
),
|
||||
'tool/code-dispatch-start': disposition(
|
||||
['rootCallId', 'parentCallId', 'subCallId', 'name', 'arguments'],
|
||||
@@ -143,16 +119,10 @@ export const RELEASED_V0_EVENT_DISPOSITIONS: Readonly<Record<string, ReleasedV0P
|
||||
['turn', 'step', 'message'],
|
||||
['error', 'meta'],
|
||||
['meta'],
|
||||
['message.content[].content[].type'],
|
||||
),
|
||||
'turn/end': disposition(['turn', 'reason'], [], [], ['reason.kind']),
|
||||
'turn/end': disposition(['turn', 'reason']),
|
||||
'turn/start': disposition(['turn']),
|
||||
'user/message': disposition(
|
||||
['role', 'id', 'content', 'source'],
|
||||
[],
|
||||
[],
|
||||
['content[].type', 'source.kind'],
|
||||
),
|
||||
'user/message': disposition(['role', 'id', 'content', 'source']),
|
||||
'web/deepseek-search-llm-request': disposition(['endpoint', 'apiVersion', 'body']),
|
||||
})
|
||||
|
||||
|
||||
@@ -128,7 +128,9 @@ export function assertReleasedPayloadSemantics(event: SessionFormatEvent, versio
|
||||
} else if (data['maxRetries'] !== undefined) {
|
||||
throw new SessionFormatError(`${label} always mode must omit maxRetries`)
|
||||
}
|
||||
if (countValue(data['delayMs'], `${label} delayMs`) > 2_147_483_647) {
|
||||
const delayMs = finiteNumberValue(data['delayMs'], `${label} delayMs`)
|
||||
if (delayMs < 0) throw new SessionFormatError(`${label} delayMs must be non-negative`)
|
||||
if (delayMs > 2_147_483_647) {
|
||||
throw new SessionFormatError(`${label} delayMs exceeds the timer range`)
|
||||
}
|
||||
llmFailureValue(data['failure'], `${label} failure`)
|
||||
|
||||
@@ -31,6 +31,8 @@ interface ToolLifecycle {
|
||||
export interface ReleasedRelationshipExtensions {
|
||||
/** Event types that must occur inside the current open step. */
|
||||
readonly stepEvents?: ReadonlySet<string>
|
||||
/** Title-request model input was source-validated and preserved across sequence remapping. */
|
||||
readonly preservedSourceTitleRequestText?: true
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -221,7 +223,12 @@ export function assertReleasedArtifactRelationships(
|
||||
}
|
||||
case 'session/title':
|
||||
case 'session/title-llm-request':
|
||||
assertTitleSources(artifact.events, event, data)
|
||||
assertTitleSources(
|
||||
artifact.events,
|
||||
event,
|
||||
data,
|
||||
extensions.preservedSourceTitleRequestText !== true,
|
||||
)
|
||||
break
|
||||
case 'command/run': {
|
||||
const id = data['commandId'] as string
|
||||
@@ -408,6 +415,7 @@ function assertTitleSources(
|
||||
events: readonly SessionFormatEvent[],
|
||||
event: SessionFormatEvent,
|
||||
data: Record<string, SessionFormatJsonValue>,
|
||||
validateFramedText: boolean,
|
||||
): void {
|
||||
const seqs = data['messageSeqs'] as readonly number[]
|
||||
if (event.type === 'session/title') {
|
||||
@@ -444,7 +452,8 @@ function assertTitleSources(
|
||||
throw new SessionFormatError('session/title-llm-request messages do not represent messageSeqs')
|
||||
}
|
||||
const framed = content[0]
|
||||
if (framed === undefined || framed['type'] !== 'text' || framed['text'] !== expected) {
|
||||
if (framed === undefined || framed['type'] !== 'text'
|
||||
|| validateFramedText && framed['text'] !== expected) {
|
||||
throw new SessionFormatError('session/title-llm-request messages do not represent messageSeqs')
|
||||
}
|
||||
}
|
||||
|
||||
@@ -270,8 +270,10 @@ describe('released v1 whole-artifact relationships', () => {
|
||||
expect(() => decode([...prefix, retry({ retry: 2, maxRetries: 1 })])).toThrow()
|
||||
expect(() => decode([...prefix, retry({ provider: 'q' })])).toThrow(/provider/)
|
||||
expect(() => decode([...prefix, retry({ failure: { message: 'x', code: 'X', status: 99 } })])).toThrow(/status/)
|
||||
expect(() => decode([...prefix, retry({ delayMs: -0.5 })])).toThrow(/non-negative/)
|
||||
expect(() => decode([...prefix, retry({ delayMs: 2_147_483_648 })])).toThrow(/timer/)
|
||||
expect(() => decode([...prefix, retry({ failure: { message: 'x', code: 'X', providerRetryAfterMs: 0 } })])).toThrow(/positive/)
|
||||
expect(decode([...prefix, retry({ delayMs: 1.5 })]).events).toHaveLength(4)
|
||||
expect(decode([...prefix, retry({ failure: { message: 'x', code: 'X', providerRetryAfterMs: 1.5 } })]).events)
|
||||
.toHaveLength(4)
|
||||
})
|
||||
|
||||
@@ -290,13 +290,24 @@ describe('released event and payload inventory', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('publishes every merge-extensible nested arm as owner-opaque policy', () => {
|
||||
expect(RELEASED_V0_EVENT_DISPOSITIONS['user/message']?.extensionArms)
|
||||
.toEqual(['content[].type', 'source.kind'])
|
||||
expect(RELEASED_V0_EVENT_DISPOSITIONS['assistant/chunk']?.extensionArms)
|
||||
.toContain('chunk.reason.kind')
|
||||
expect(RELEASED_V0_EVENT_DISPOSITIONS['turn/end']?.extensionArms)
|
||||
.toEqual(['reason.kind'])
|
||||
it.each([
|
||||
['user/message content block', 'user/message', {
|
||||
...userMessage,
|
||||
content: [{ type: 'future-block', private: { preserved: true } }],
|
||||
}],
|
||||
['user/message source', 'user/message', {
|
||||
...userMessage,
|
||||
source: { kind: 'future-source', private: { preserved: true } },
|
||||
}],
|
||||
['assistant finish reason', 'assistant/chunk', {
|
||||
turn: 1, step: 0,
|
||||
chunk: { type: 'finish', reason: { kind: 'future-reason', private: { preserved: true } } },
|
||||
}],
|
||||
['turn/end reason', 'turn/end', {
|
||||
turn: 1, reason: { kind: 'future-reason', private: { preserved: true } },
|
||||
}],
|
||||
] as const)('preserves an unknown merge-extensible %s arm', (_name, type, data) => {
|
||||
expect(() => { assertPayload(type, data) }).not.toThrow()
|
||||
})
|
||||
|
||||
it('refuses unknown v0 events even when the envelope marks them ignorable', () => {
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-format-v1-to-v2/README.md
|
||||
README.md: 5ead8532b2cce52a0cbf4c39bfe9e54ef14002f1
|
||||
README.zh.md: 1f41d24007f95f799a9667910acb93d494a2aa2e
|
||||
README.md: 0e2661259b89cdbf4d5e138fe6d7a8fdfeaaea56
|
||||
README.zh.md: c066b994a7fe63fa2628fe0d5b4ae950f67da72b
|
||||
@@ -40,7 +40,7 @@ const migratedV2 = sessionFormatV1ToV2.migrate(decodedV1)
|
||||
|
||||
A successful v1 `assistant/message` must cite its complete ordered attempt. The migration removes the cited top-level chunks and obsolete message provenance, compacts the chunks without joining token boundaries, and stores the stream on that message. An unclaimed attempt becomes one log-only `assistant/attempt` at its final chunk position. Unrelated interleaved events keep their relative order.
|
||||
|
||||
The migration refuses a reference to a consumed chunk instead of redirecting it to a different semantic event. It remaps declared event provenance, surface replacements, command source events, compaction ranges and lists, and title message lists. A seeded source also refuses an inherited cut that splits an Assistant attempt; the target marks the exact cut with `session/end-seed { inherited: true }`.
|
||||
The migration refuses a reference to a consumed chunk instead of redirecting it to a different semantic event. It remaps declared event provenance, surface replacements, command source events, compaction ranges and lists, and title message lists. The already model-visible `session/title-llm-request.messages` text remains byte-identical after source validation, so target validation does not reinterpret the old sequence numbers embedded in that prompt. A seeded source also refuses an inherited cut that splits an Assistant attempt; the target marks the exact cut with `session/end-seed { inherited: true }`.
|
||||
|
||||
The v2 physical header requires `isSeeded` and does not store a numeric cut. The codec derives the cut from the last inherited end-seed marker, writes one event per row, range-encodes only `sourceEventSeqs`, and remains neutral to ordinary event vocabulary and payload growth. Strict migration-target validation freezes the released-v2 inventory and rejects unknown types or members. Current restoration instead admits event types known to the installed Session package plus unknown events carrying `ignorable: true`, then delegates payload and stream semantics to the installed current restorer. All paths retain strict header, event-envelope, sequence, and inherited-cut validation.
|
||||
|
||||
|
||||
@@ -40,7 +40,7 @@ const migratedV2 = sessionFormatV1ToV2.migrate(decodedV1)
|
||||
|
||||
成功的 v1 `assistant/message` 必须引用其完整有序 attempt。迁移会移除这些顶层 chunk 和已停用的 message provenance,在不合并 token 边界的前提下压缩 chunk,并把 stream 存到该 message 上。未被 message 认领的 attempt 会在其最后一个 chunk 的位置变成一个仅日志可见的 `assistant/attempt`。无关的交错事件保持相对顺序。
|
||||
|
||||
如果引用指向被消费的 chunk,迁移会失败,而不会把它重定向到语义不同的事件。它会重映射已声明的事件 provenance、surface replacement、command source event、compaction range 与 list,以及 title message list。带 seed 的源若让继承切点切开一个 Assistant attempt,也会迁移失败;目标会用 `session/end-seed { inherited: true }` 标出精确切点。
|
||||
如果引用指向被消费的 chunk,迁移会失败,而不会把它重定向到语义不同的事件。它会重映射已声明的事件 provenance、surface replacement、command source event、compaction range 与 list,以及 title message list。已经对模型可见的 `session/title-llm-request.messages` 文本会在源校验后保持逐字节不变,因此目标校验不会重新解释该 prompt 中嵌入的旧序号。带 seed 的源若让继承切点切开一个 Assistant attempt,也会迁移失败;目标会用 `session/end-seed { inherited: true }` 标出精确切点。
|
||||
|
||||
v2 物理 header 要求 `isSeeded`,且不存储数值切点。编解码器从最后一个 inherited end-seed marker 推导切点,每行写入一个事件,只对 `sourceEventSeqs` 做范围编码,并对普通事件词汇与 payload 扩展保持中立。严格的迁移目标校验会冻结 released-v2 清单并拒绝未知 type 或 member。当前恢复则准入 installed Session package 已知的事件 type,以及携带 `ignorable: true` 的未知事件,再把 payload 与 stream 语义交给 installed current restorer。所有路径仍严格校验 header、event envelope、sequence 与 inherited cut。
|
||||
|
||||
|
||||
@@ -168,6 +168,7 @@ function encodeProvenance(event: SessionFormatEvent): SessionFormatJsonObject {
|
||||
function decodeSeqRanges(value: SessionFormatJsonValue, maxEntries: number): readonly number[] {
|
||||
if (!Array.isArray(value)) throw new SessionFormatError('sourceEventSeqs must be an array')
|
||||
const output: number[] = []
|
||||
let hasRange = false
|
||||
for (const entry of value) {
|
||||
if (!Array.isArray(entry)) {
|
||||
output.push(sessionFormatCount(entry, 'sourceEventSeqs member'))
|
||||
@@ -180,6 +181,7 @@ function decodeSeqRanges(value: SessionFormatJsonValue, maxEntries: number): rea
|
||||
throw new SessionFormatError('sourceEventSeqs range exceeds its event seq')
|
||||
}
|
||||
for (let current = start; current <= end; current += 1) output.push(current)
|
||||
hasRange = true
|
||||
}
|
||||
const seen = new Set<number>()
|
||||
for (const source of output) {
|
||||
@@ -188,10 +190,14 @@ function decodeSeqRanges(value: SessionFormatJsonValue, maxEntries: number): rea
|
||||
}
|
||||
seen.add(source)
|
||||
}
|
||||
if (hasRange && output.some((source, index) => index > 0 && source <= (output[index - 1] as number))) {
|
||||
throw new SessionFormatError('sourceEventSeqs ranges must be strictly increasing')
|
||||
}
|
||||
return output
|
||||
}
|
||||
|
||||
function encodeSeqRanges(values: readonly number[]): readonly SessionFormatJsonValue[] {
|
||||
if (values.some((value, index) => index > 0 && value <= (values[index - 1] as number))) return [...values]
|
||||
const output: SessionFormatJsonValue[] = []
|
||||
for (let index = 0; index < values.length;) {
|
||||
const start = values[index] as number
|
||||
|
||||
@@ -31,6 +31,7 @@ const LOG_OPTIONAL = ['ignorable'] as const
|
||||
const RELEASED_V2_EVENT_TYPE_SET = new Set(RELEASED_V2_EVENT_TYPES)
|
||||
const RELEASED_V2_RELATIONSHIP_EXTENSIONS = {
|
||||
stepEvents: new Set(['assistant/attempt']),
|
||||
preservedSourceTitleRequestText: true,
|
||||
} as const
|
||||
|
||||
/**
|
||||
|
||||
@@ -4,6 +4,7 @@ import type {
|
||||
SessionFormatEvent,
|
||||
SessionFormatJsonObject,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import { decodeSeqRanges as decodeCurrentSeqRanges } from '@deepseek-ai/dsh-session'
|
||||
import { releasedV2SessionFormatCodec } from '@deepseek-ai/dsh-session-format-v1-to-v2'
|
||||
|
||||
const minimalPhysicalHeader = {
|
||||
@@ -126,6 +127,21 @@ describe('releasedV2SessionFormatCodec rows', () => {
|
||||
expect(releasedV2SessionFormatCodec.decodeArtifact(encoded.header, encoded.rows)).toStrictEqual(source)
|
||||
})
|
||||
|
||||
it('keeps non-monotonic provenance scalar-only for the current backend reader', () => {
|
||||
const sourceEventSeqs = [4, 5, 1, 2, 3]
|
||||
const source = artifact([
|
||||
feedback(0), feedback(1), feedback(2), feedback(3), feedback(4), feedback(5),
|
||||
userMessage(6, sourceEventSeqs),
|
||||
])
|
||||
|
||||
const encoded = releasedV2SessionFormatCodec.encodeArtifact(source)
|
||||
const stored = encoded.rows[6]?.['sourceEventSeqs']
|
||||
|
||||
expect(stored).toStrictEqual(sourceEventSeqs)
|
||||
expect(decodeCurrentSeqRanges(stored, 6)).toStrictEqual(sourceEventSeqs)
|
||||
expect(releasedV2SessionFormatCodec.decodeArtifact(encoded.header, encoded.rows)).toStrictEqual(source)
|
||||
})
|
||||
|
||||
it('keeps the v2 physical codec vocabulary-neutral for current growth and a future source freeze', () => {
|
||||
const source = artifact([
|
||||
{ type: 'external/required', seq: 0, time: 1, data: { retained: true } },
|
||||
@@ -164,6 +180,7 @@ describe('releasedV2SessionFormatCodec rows', () => {
|
||||
['scalar at the event', [4], /unique earlier/],
|
||||
['duplicate scalars', [0, 0], /unique earlier/],
|
||||
['overlapping range and scalar', [[0, 1], 1], /unique earlier/],
|
||||
['non-monotonic range', [3, [0, 2]], /strictly increasing/],
|
||||
])('refuses malformed stored provenance: %s', (_name, sourceEventSeqs, message) => {
|
||||
const rows = [feedback(0), feedback(1), feedback(2), feedback(3), {
|
||||
...userMessage(4), sourceEventSeqs,
|
||||
|
||||
@@ -650,6 +650,52 @@ describe('sessionFormatV1ToV2', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('preserves source-sequence text in a title request while remapping its references', () => {
|
||||
const framed = 'Generate the session title from this JSON array of human messages:\n'
|
||||
+ JSON.stringify([{ seq: 6, text: 'question' }])
|
||||
const source: SessionFormatArtifact = {
|
||||
header: {
|
||||
version: 1, id: 'v1-title-source-seq', createdAt: 1,
|
||||
isSeeded: false, delegationDepth: 0,
|
||||
},
|
||||
inheritedEventCount: 0,
|
||||
events: [
|
||||
event('turn/start', 0, 1, { turn: 1 }),
|
||||
event('step/start', 1, 2, { turn: 1, step: 1 }),
|
||||
event('assistant/chunk', 2, 3, {
|
||||
turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'hello' },
|
||||
}),
|
||||
event('assistant/chunk', 3, 4, {
|
||||
turn: 1, step: 1, chunk: { type: 'finish', reason: { kind: 'stop' } },
|
||||
}),
|
||||
{
|
||||
...event('assistant/message', 4, 5, { turn: 1, step: 1, message }),
|
||||
sourceEventSeqs: [2, 3], surfaceOp: 'append',
|
||||
},
|
||||
event('step/end', 5, 6, { turn: 1, step: 1 }),
|
||||
{ ...event('user/message', 6, 7, userMessage), surfaceOp: 'append' },
|
||||
event('session/title-llm-request', 7, 8, {
|
||||
titleProvider: 'title-1', messageSeqs: [6], route: { provider: 'mock', model: 'mock' },
|
||||
system: 'title',
|
||||
messages: [{
|
||||
id: 'title-request', role: 'user', content: [{ type: 'text', text: framed }],
|
||||
source: { kind: 'plugin', plugin: 'dsh-session-title-llm' },
|
||||
}],
|
||||
maxTokens: 20,
|
||||
}),
|
||||
event('turn/end', 8, 9, { turn: 1, reason: { kind: 'completed' } }),
|
||||
],
|
||||
}
|
||||
|
||||
const migrated = sessionFormatV1ToV2.migrate(source)
|
||||
const titleRequest = migrated.events.find(event => event.type === 'session/title-llm-request')
|
||||
|
||||
expect(titleRequest?.data).toMatchObject({
|
||||
messageSeqs: [4],
|
||||
messages: [{ content: [{ text: framed }] }],
|
||||
})
|
||||
})
|
||||
|
||||
it('remaps compaction summaries and closes prior-turn groups independently', () => {
|
||||
const source: SessionFormatArtifact = {
|
||||
header: {
|
||||
|
||||
@@ -239,6 +239,11 @@ function isEEXIST(error: unknown): boolean {
|
||||
return (error as NodeJS.ErrnoException | null)?.code === 'EEXIST'
|
||||
}
|
||||
|
||||
/** Whether a filesystem-owned failure should retain its original errno and path. */
|
||||
function isErrnoException(error: unknown): error is NodeJS.ErrnoException {
|
||||
return typeof (error as NodeJS.ErrnoException | null)?.code === 'string'
|
||||
}
|
||||
|
||||
function identity(value: JsonlPhysicalIdentity): string {
|
||||
return [value.dev, value.ino, value.size, value.mtimeNs, value.ctimeNs].join(':')
|
||||
}
|
||||
@@ -247,15 +252,19 @@ function fingerprint(value: JsonlPhysicalIdentity, bytes: Buffer): string {
|
||||
return `${identity(value)}:${createHash('sha256').update(bytes).digest('hex')}`
|
||||
}
|
||||
|
||||
/** Read exact physical bytes only when the stat identity brackets one stable read. */
|
||||
/**
|
||||
* Read one stable revision with a single retry. If an append overlaps both
|
||||
* reads, return the second read's committed pre-read prefix instead of
|
||||
* starving behind a continuous writer.
|
||||
*/
|
||||
async function readStableSnapshot(
|
||||
path: string,
|
||||
signal: AbortSignal | undefined,
|
||||
fs: GenerationFileSystem,
|
||||
): Promise<StablePhysicalFile> {
|
||||
for (;;) {
|
||||
signal?.throwIfAborted()
|
||||
const before = await fs.stat(path)
|
||||
signal?.throwIfAborted()
|
||||
let before = await fs.stat(path)
|
||||
for (let attempt = 0; ; attempt += 1) {
|
||||
const bytes = await fs.readFile(path, signal)
|
||||
signal?.throwIfAborted()
|
||||
const after = await fs.stat(path)
|
||||
@@ -263,6 +272,10 @@ async function readStableSnapshot(
|
||||
signal?.throwIfAborted()
|
||||
return { bytes, identity: after }
|
||||
}
|
||||
if (attempt === 1) {
|
||||
return { bytes: bytes.subarray(0, Number(before.size)), identity: before }
|
||||
}
|
||||
before = after
|
||||
}
|
||||
}
|
||||
|
||||
@@ -647,10 +660,14 @@ async function reopenExpectedCurrent(
|
||||
throw new Error(`target is a ${kind}`)
|
||||
}
|
||||
const snapshot = await validatePhysicalCurrent(currentPath, compression, format, signal, internals)
|
||||
if (!snapshot.bytes.equals(expectedBytes)) throw new Error('target bytes differ from the migrated generation')
|
||||
if (snapshot.bytes.length < expectedBytes.length
|
||||
|| !snapshot.bytes.subarray(0, expectedBytes.length).equals(expectedBytes)) {
|
||||
throw new Error('target bytes do not begin with the migrated generation')
|
||||
}
|
||||
return snapshot
|
||||
} catch (error: unknown) {
|
||||
if (signal?.aborted) signal.throwIfAborted()
|
||||
if (isErrnoException(error)) throw error
|
||||
throw new JsonlGenerationTargetConflictError(currentPath, asError(error))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -169,6 +169,28 @@ describe('JSONL immutable generation publication', () => {
|
||||
expect(await readFile(request.sourcePath, 'utf8')).toBe(contents)
|
||||
})
|
||||
|
||||
it('bounds current snapshot retries under continuous revision churn', async () => {
|
||||
const root = await tempRoot()
|
||||
const request = options(root, 'none', adapter(), 1)
|
||||
const contents = line(header(1)) + line(event0)
|
||||
await writeFile(request.sourcePath, contents)
|
||||
let revision = 0n
|
||||
const statFile = vi.fn(async (path: string) => {
|
||||
const value = await stat(path, { bigint: true })
|
||||
revision += 1n
|
||||
return { ...value, mtimeNs: value.mtimeNs + revision }
|
||||
})
|
||||
const readChangingFile = vi.fn(async () => Buffer.from(contents + line(event1)))
|
||||
|
||||
const result = await __jsonlGenerationTest.ensure(request, {
|
||||
fs: { stat: statFile, readFile: readChangingFile },
|
||||
})
|
||||
|
||||
expect(result.snapshot.bytes.toString('utf8')).toBe(contents)
|
||||
expect(readChangingFile).toHaveBeenCalledTimes(2)
|
||||
expect(statFile).toHaveBeenCalledTimes(3)
|
||||
})
|
||||
|
||||
it('returns a disposable Zstandard body owner on the current fast path', async () => {
|
||||
const root = await tempRoot()
|
||||
const request = options(root, 'zstd', adapter(), 1)
|
||||
@@ -603,6 +625,32 @@ describe('JSONL immutable generation publication', () => {
|
||||
expect((await readdir(root)).sort()).toEqual(['session.jsonl', 'session.v1.jsonl'])
|
||||
})
|
||||
|
||||
it.each(['none', 'zstd'] as const)(
|
||||
'accepts a valid append on a %s target created by another migration',
|
||||
async (compression) => {
|
||||
const root = await tempRoot()
|
||||
const request = options(root, compression)
|
||||
const source = compression === 'zstd'
|
||||
? await encodeZstd(0, [event0])
|
||||
: Buffer.from(line(header(0)) + line(event0))
|
||||
const expected = compression === 'zstd'
|
||||
? await encodeZstd(1, [event0])
|
||||
: Buffer.from(line(header(1)) + line(event0))
|
||||
const appended = compression === 'zstd'
|
||||
? await compressZstdFrame(line(event1))
|
||||
: Buffer.from(line(event1))
|
||||
const winner = Buffer.concat([expected, appended])
|
||||
await writeFile(request.sourcePath, source)
|
||||
await writeFile(request.currentPath, winner)
|
||||
|
||||
const result = await ensureJsonlGenerationCurrent(request)
|
||||
|
||||
expect(result).toMatchObject({ status: 'migrated', path: request.currentPath })
|
||||
expect(result.snapshot.bytes).toEqual(winner)
|
||||
expect(await readFile(request.currentPath)).toEqual(winner)
|
||||
},
|
||||
)
|
||||
|
||||
it('accepts an identical regular hardlink target', async () => {
|
||||
const root = await tempRoot()
|
||||
const request = options(root)
|
||||
@@ -755,7 +803,7 @@ describe('JSONL immutable generation publication', () => {
|
||||
expect((await readdir(root)).every(name => !name.includes('.tmp'))).toBe(true)
|
||||
})
|
||||
|
||||
it('reports an absent exclusive-publication winner as a target conflict', async () => {
|
||||
it('preserves ENOENT when an exclusive-publication winner disappears', async () => {
|
||||
const root = await tempRoot()
|
||||
const request = options(root)
|
||||
await writeFile(request.sourcePath, line(header(0)) + line(event0))
|
||||
@@ -763,7 +811,24 @@ describe('JSONL immutable generation publication', () => {
|
||||
await expect(__jsonlGenerationTest.ensure(request, {
|
||||
platform: 'darwin',
|
||||
fs: posixSimulationFs({ link: async () => { throw fsError('EEXIST') } }),
|
||||
})).rejects.toBeInstanceOf(JsonlGenerationTargetConflictError)
|
||||
})).rejects.toMatchObject({ code: 'ENOENT', path: request.currentPath })
|
||||
})
|
||||
|
||||
it('preserves a filesystem error while reopening a committed target', async () => {
|
||||
const root = await tempRoot()
|
||||
const request = options(root)
|
||||
const failure = fsError('EACCES', 'current target is unreadable')
|
||||
failure.path = request.currentPath
|
||||
await writeFile(request.sourcePath, line(header(0)) + line(event0))
|
||||
|
||||
await expect(__jsonlGenerationTest.ensure(request, {
|
||||
fs: {
|
||||
readFile: async (path, signal) => {
|
||||
if (path === request.currentPath) throw failure
|
||||
return readFile(path, signal === undefined ? undefined : { signal })
|
||||
},
|
||||
},
|
||||
})).rejects.toBe(failure)
|
||||
})
|
||||
|
||||
it('rethrows the exact abort reason during committed reopen and leaves the target', async () => {
|
||||
|
||||
Reference in new issue
Block a user