fix(session): refuse retired tool-result content during V4 admission

Released V3 accepts tool-result wrappers in ordinary user content. Leaving
those wrappers in a V4 artifact violates its declared content types and
reaches provider serializers that cannot represent them. Converting such
rows would require a new event, role, or source ownership decision.

Reject retired result tags only in known interpreted content slots and
raw Assistant stream block starts/ends. Reuse the native retired-syntax
admission path before recoverable suffix suppression and target artifact
restoration before successor publication. Preserve the existing system
and tool-result validators and the canonical V3 tool/result wrapper lift.

Known ignorable developer rows receive the same check during encoding;
unaware physical readers still defer their opaque developer payloads.
Tool arguments, replay state, schema parameters, nested metadata, and
unknown ignorable event payloads are not recursively scanned.

Real JSONL regressions cover V3 migration and native V4 read/write in
plain and Zstandard storage, including a corrupt native prefix. Refusals
preserve source bytes and the sole generation; opaque payloads survive.
No declared schema, frozen V0-V3 codec, or snapshot generation changes.
This commit is contained in:
Tianyi Cui committed 2026-09-18 23:49:20 +08:00
1 parent 0112eaf3a8
commit cbb4df8750
7 files changed
+239 -11

No files matched your search

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-17-native-v4-read-validation.md
2026-09-17-native-v4-read-validation.md: e98cfff679254557ebd5be35c23810cbd2b920f9
2026-09-17-native-v4-read-validation.zh.md: 2b586bc3eb45a20875b69ce789c995d0ab208f7d
2026-09-17-native-v4-read-validation.md: b111e9f1dd4aae6557b7893a9f334445ef0c85bc
2026-09-17-native-v4-read-validation.zh.md: 20f1dc53b6eda42834e169c18f7b269e2933d27c
@@ -18,6 +18,8 @@ An unfinished tail may retain an open turn, step, tool call, or compaction. A cl
Installed Session adoption continues to own event envelopes, message roles and metadata, canonical request headers, and surface replacement reference coverage. Optional runtime invariants provide diagnostics; their installation is not required for durable admission. Native system-message admission also validates coordinates, identity, content blocks, and known image/tool-call fields before physical recovery can discard rows. Additional JSON fields and unknown nonempty content tags remain intact; retired tool-result wrappers are refused. Retired `request/header.header.system` and required `tool/code-dispatch*` tags remain hard refusals before recovery; obsolete ignorable PTC records remain opaque. Physical framing alone does not establish semantic validity.
A retired tool-result wrapper outside its owning `tool/result` row would require changing the event, role, or source to represent it in V4. The adjacent migration lifts only the canonical owned wrapper; target restoration refuses wrappers remaining in interpreted content. Native row admission applies the same refusal before recoverable tail suppression. The check follows declared message, content, and Assistant-stream slots instead of recursively inspecting JSON, so tool arguments, replay state, schema parameters, nested extensions, and unknown ignorable payloads retain their recorded values.
`EpochHeader.system` explicitly reserves the retired key as `system?: never` with `@persistenceReserved`. The catalog retains its empty value set so permitting a value is a change to a forbidden field, rather than an ordinary optional-field addition. The native reader and Session adoption continue to reject that key; system prompts remain `system/message` events.
## Alternatives considered
@@ -28,4 +30,4 @@ Installed Session adoption continues to own event envelopes, message roles and m
## Consequences
The native restorer maintains one bounded state machine per read and rejects contradictions before successor publication. Frozen V0–V3 validators and committed Session generations remain unchanged. Public catalog regressions cover invalid relationships alongside partial tails, inherited cuts, immutable unknown attribution, and complete tool exchanges; these checks define finalized V4 admission. Breaking changes require a successor; compatible additions use new acknowledgements, and implementation fixes preserve the declared meaning.
The native restorer maintains one bounded state machine per read and rejects contradictions before successor publication. Frozen V0–V3 validators and committed Session generations remain unchanged. Public catalog regressions cover invalid relationships alongside partial tails, inherited cuts, immutable unknown attribution, and complete tool exchanges; these checks enforce the declared V4 representation without changing its schemas. Breaking changes require a successor; compatible additions use new acknowledgements, and implementation fixes preserve the declared meaning.
@@ -18,6 +18,8 @@ V4 格式恢复器与当前 JSONL 扫描器共用同一强制跨事件关系校
已安装的 Session 接纳器继续负责事件封装、消息角色与元数据、规范请求头,以及 surface 替换的引用覆盖。可选运行时不变式提供诊断;持久化准入不要求安装它们。原生 system 消息准入还会在物理恢复可能丢弃行之前,校验坐标、身份、内容块及已知 image/tool-call 字段。额外 JSON 字段和非空的未知内容标签保持不变,退役的 tool-result 包装会被拒绝。退役的 `request/header.header.system` 和必需的 `tool/code-dispatch*` 标签仍在恢复前强制拒绝;标记为可忽略的过时 PTC 记录保持不透明。仅通过物理分帧不能证明语义有效。
退役的 tool-result 包装若不属于对应的 `tool/result` 行,就需要改变事件、角色或来源才能在 V4 中表示。相邻迁移只提升规范的自有包装;目标恢复器拒绝解释内容中剩余的包装。原生行准入在可恢复尾部被忽略之前执行同样的拒绝。检查沿声明的消息、内容和 Assistant stream 字段进行,不递归扫描 JSON,因此工具参数、replay state、schema 参数、嵌套扩展和未知可忽略载荷保留其记录值。
`EpochHeader.system` 通过带有 `@persistenceReserved` 的 `system?: never` 显式保留退役键。目录保留其空值集合,使允许某个值被识别为对禁止字段的修改,而非普通可选字段添加。原生读取方和 Session 接纳继续拒绝该键;系统提示词仍由 `system/message` 事件承载。
## Alternatives considered
@@ -28,4 +30,4 @@ V4 格式恢复器与当前 JSONL 扫描器共用同一强制跨事件关系校
## Consequences
原生恢复器为每次读取维护独立状态机,在发布后继文件之前拒绝矛盾。冻结的 V0–V3 校验器及已提交的 Session 代际保持不变。公开目录回归覆盖无效关系、部分尾部、继承切分点、未知归属的不可变保存,以及完整工具交互;这些检查定义已定稿 V4 的接纳规则。破坏性变更要求后继版本;兼容新增使用新的确认记录,实现修复须保留声明的含义。
原生恢复器为每次读取维护独立状态机,在发布后继文件之前拒绝矛盾。冻结的 V0–V3 校验器及已提交的 Session 代际保持不变。公开目录回归覆盖无效关系、部分尾部、继承切分点、未知归属的不可变保存,以及完整工具交互;这些检查落实已声明的 V4 表示,不改变其 schema。破坏性变更要求后继版本;兼容新增使用新的确认记录,实现修复须保留声明的含义。
@@ -44,7 +44,10 @@ export const releasedV4SessionFormatCodec = Object.freeze({
return { ...releasedV2SessionFormatCodec.encodeHeader({ ...header, version: 2 }, inheritedEventCount), version: 4 }
},
encodeEvent(event: SessionFormatEvent) {
if (event.type === 'developer/message' && event['ignorable'] === true) assertV4DeveloperData(event)
if (event.type === 'developer/message' && event['ignorable'] === true) {
assertV4DeveloperData(event)
assertV4RetiredSyntax(event)
}
assertV4RowAdmission(event)
return releasedV2SessionFormatCodec.encodeEvent(event)
},
@@ -2,8 +2,23 @@
import { SessionFormatError, SessionFormatUnsupportedMigrationError, isSessionFormatJsonObject } from '@deepseek-ai/dsh-session-format'
function assertBlock(block: unknown, subject: string): void {
if (isSessionFormatJsonObject(block) && block['type'] === 'tool-result') {
throw new SessionFormatError(`${subject} must not contain a released tool-result wrapper`)
}
}
function assertContent(content: unknown, subject: string): void {
if (Array.isArray(content)) content.forEach((block) => { assertBlock(block, subject) })
}
function assertMessageContent(message: unknown, subject: string): void {
if (isSessionFormatJsonObject(message)) assertContent(message['content'], subject)
}
/**
* Refuse retired request-header system text and predecessor PTC tags without interpreting extension payloads.
* Refuse retired headers, PTC tags, and tool-result blocks in interpreted content slots.
* Tool arguments, replay state, schema parameters, and nested extension values remain opaque.
* @param row - parsed physical row or complete native logical event.
*/
export function assertV4RetiredSyntax(row: unknown): void {
@@ -11,12 +26,50 @@ export function assertV4RetiredSyntax(row: unknown): void {
if ((row['type'] === 'tool/code-dispatch-start' || row['type'] === 'tool/code-dispatch') && row['ignorable'] !== true) {
throw new SessionFormatUnsupportedMigrationError(`format v4 rejects retired event type ${row['type']}`)
}
if (row['type'] !== 'request/header') return
const data = row['data']
if (!isSessionFormatJsonObject(data) || !isSessionFormatJsonObject(data['header'])) {
throw new SessionFormatError('format v4 request/header requires data and header objects')
if (row['type'] === 'request/header') {
if (!isSessionFormatJsonObject(data) || !isSessionFormatJsonObject(data['header'])) {
throw new SessionFormatError('format v4 request/header requires data and header objects')
}
if (Object.hasOwn(data['header'], 'system')) {
throw new SessionFormatError('format v4 request/header rejects retired header.system')
}
return
}
if (Object.hasOwn(data['header'], 'system')) {
throw new SessionFormatError('format v4 request/header rejects retired header.system')
if (!isSessionFormatJsonObject(data)) return
const subject = `format v4 ${row['type']} at seq ${row['seq']} content`
switch (row['type']) {
case 'user/message':
assertContent(data['content'], subject)
break
case 'developer/message':
case 'assistant/message':
case 'team/message/queued':
assertMessageContent(data['message'], subject)
break
case 'agent/inbox/spliced':
case 'session/title-llm-request': {
const messages = data[row['type'] === 'agent/inbox/spliced' ? 'inserted' : 'messages']
if (Array.isArray(messages)) messages.forEach((message) => { assertMessageContent(message, subject) })
break
}
case 'compaction/summary':
assertContent(data['summary'], subject)
assertContent(data['rawOutput'], subject)
break
case 'tool/ptc-dispatch':
assertContent(data['content'], subject)
break
}
if ((row['type'] === 'assistant/message' || row['type'] === 'assistant/attempt') && Array.isArray(data['stream'])) {
for (const entry of data['stream']) {
if (!isSessionFormatJsonObject(entry) || entry['type'] !== 'chunk') continue
const chunk = entry['chunk']
if (!isSessionFormatJsonObject(chunk)) continue
if (chunk['type'] === 'block-end') assertBlock(chunk['block'], subject)
if (chunk['type'] === 'block-start' && chunk['blockType'] === 'tool-result') {
throw new SessionFormatError(`${subject} must not contain a released tool-result wrapper`)
}
}
}
}
@@ -0,0 +1,112 @@
import { describe, expect, it } from 'vitest'
import { SessionFormatEventCollector } from '@deepseek-ai/dsh-session-format'
import type { SessionFormatEvent, SessionFormatJsonObject } from '@deepseek-ai/dsh-session-format'
import { createSessionFormatCatalogWithChildren, historicalSessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog'
import { assertV4RowAdmission, releasedV4SessionFormatCodec, restoreReleasedV4Artifact } from '../src/index.ts'
const header = { version: 4, id: 'retired-content', createdAt: 1, isSeeded: false, delegationDepth: 0 }
const wrapper: SessionFormatJsonObject = {
type: 'tool-result', toolCallId: 'retired-call', content: [{ type: 'text', text: 'saved result' }],
}
const content = [wrapper]
const user = { id: 'user', role: 'user', source: { kind: 'user' }, content }
const message = (role: string): SessionFormatJsonObject => ({
id: role, role, content,
source: role === 'assistant' ? { kind: 'model', provider: 'test', model: 'test' }
: role === 'tool' ? { kind: 'tool', callId: 'call' } : { kind: `${role}-prompt` },
...(role === 'tool' ? { toolCallId: 'call' } : {}),
})
const cases: readonly { name: string; type: string; data: SessionFormatJsonObject; error?: RegExp }[] = [
{ name: 'user message', type: 'user/message', data: user },
...['system', 'developer', 'assistant'].map(role => ({
name: `${role} message`, type: `${role}/message`, data: { turn: 1, step: 1, message: message(role) },
...(role === 'system' ? { error: /rejects retired tool-result/ } : {}),
})),
{ name: 'native tool message', type: 'tool/result', data: { turn: 1, step: 1, message: message('tool') } },
{ name: 'queued input', type: 'agent/inbox/spliced', data: { target: 'next-turn', start: 0, inserted: [user] } },
{ name: 'title request', type: 'session/title-llm-request', data: { messages: [user] } },
{ name: 'compaction summary', type: 'compaction/summary', data: { summary: content } },
{ name: 'compaction raw output', type: 'compaction/summary', data: { summary: [], rawOutput: content } },
{ name: 'PTC result', type: 'tool/ptc-dispatch', data: { content } },
{ name: 'team message', type: 'team/message/queued', data: { message: user } },
...['assistant/message', 'assistant/attempt'].flatMap(type => ['block-start', 'block-end'].map(kind => ({
name: `${type} stream ${kind}`, type, data: {
turn: 1, step: 1, message: { ...message('assistant'), content: [] },
stream: [{ type: 'chunk', elapsedMs: 0, chunk: kind === 'block-start'
? { type: kind, index: 0, blockType: 'tool-result' }
: { type: kind, index: 0, block: wrapper } }],
},
}))),
]
function row(type: string, data: SessionFormatJsonObject): SessionFormatEvent {
return { type, data, seq: 0, time: 1 }
}
describe('retired tool-result content admission', () => {
it.each(cases)('refuses $name in native encoding, restoration and recoverable suffixes', ({ type, data, error = /released tool-result wrapper/ }) => {
const event = row(type, data)
const before = structuredClone(event)
expect(() => releasedV4SessionFormatCodec.encodeEvent(event)).toThrow(error)
expect(() => restoreReleasedV4Artifact({ header, inheritedEventCount: 0, events: [event] }, new Set([type])))
.toThrow(error)
for (const recovery of ['strict', 'recoverable'] as const) {
const decoder = releasedV4SessionFormatCodec.createDecoder({ type: 'session', ...header }, recovery)
const collector = new SessionFormatEventCollector()
if (recovery === 'recoverable') decoder.decodeRow(null, collector)
expect(() => { decoder.decodeRow(event, collector) }).toThrow(error)
}
expect(event).toEqual(before)
})
it('refuses a V3 user wrapper at the V4 target while leaving the released reader and source intact', () => {
const event = { ...row('user/message', user), surfaceOp: 'append' } as SessionFormatEvent
const before = structuredClone(event)
const physical = { type: 'session', ...header, version: 3 }
const historical = historicalSessionFormatCatalog.createRestore(physical, { recovery: 'strict', validation: 'current' })
historical.decodeRow(event)
expect(historical.finish().events).toEqual([event])
const current = createSessionFormatCatalogWithChildren([]).createRestore(physical, { recovery: 'strict', validation: 'current' })
expect(() => { current.decodeRow(event); current.finish() }).toThrow(/released tool-result wrapper/)
expect(event).toEqual(before)
})
it('interprets ignorable developer content only for a reader that knows the event, or for encoding', () => {
const event = { ...row('developer/message', { turn: 1, step: 1, message: message('developer') }), ignorable: true }
const collector = new SessionFormatEventCollector()
const decoder = releasedV4SessionFormatCodec.createDecoder({ type: 'session', ...header }, 'strict')
decoder.decodeRow(event, collector)
expect(collector.values).toEqual([event])
expect(() => { assertV4RowAdmission(event, new Set()) }).not.toThrow()
expect(() => { assertV4RowAdmission(event, new Set(['developer/message'])) }).toThrow(/released tool-result wrapper/)
expect(() => releasedV4SessionFormatCodec.encodeEvent(event)).toThrow(/released tool-result wrapper/)
const artifact = { header, inheritedEventCount: 0, events: [event] }
expect(restoreReleasedV4Artifact(artifact, new Set())).toBe(artifact)
})
it('keeps tool arguments, replay state, schemas, nested metadata, and unknown ignorable payloads opaque', () => {
const rows: SessionFormatEvent[] = [
row('user/message', { ...user, content: [{ type: 'text', text: 'text', metadata: wrapper }], replayState: { content } }),
row('tool/call', { arguments: JSON.stringify(wrapper), metadata: { content } }),
row('tool/ptc-dispatch', { content: [{ type: 'text', text: 'result' }], arguments: wrapper }),
row('request/header', { header: { config: { provider: 'test', model: 'test' }, tools: [{
name: 'example', description: '', parameters: { type: 'object', examples: [wrapper] },
}] } }),
{ ...row('future/opaque', { content, message: user, stream: [{ type: 'chunk', chunk: { type: 'block-end', block: wrapper } }] }), ignorable: true },
]
for (const event of rows) {
const before = structuredClone(event)
const encoded = releasedV4SessionFormatCodec.encodeEvent(event)
const collector = new SessionFormatEventCollector()
releasedV4SessionFormatCodec.createDecoder({ type: 'session', ...header }, 'strict').decodeRow(encoded, collector)
expect(collector.values).toEqual([before])
expect(event).toEqual(before)
}
const opaque = rows.at(-1)!
const migrated = createSessionFormatCatalogWithChildren([]).createRestore({ type: 'session', ...header, version: 3 }, {
recovery: 'strict', validation: 'current',
})
migrated.decodeRow(opaque)
expect(migrated.finish().events).toEqual([opaque])
})
})
@@ -0,0 +1,56 @@
/** Interpreted retired content must fail before migration publication or recoverable native-tail suppression. */
import { Context } from '@deepseek-ai/cordis'
import { SessionId } from '@deepseek-ai/dsh-session'
import { SessionFormatUnsupportedError, SessionPersistenceCorruptionError } from '@deepseek-ai/dsh-session-persistence'
import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl'
import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { basename, dirname, join } from 'node:path'
import { describe, expect, it, onTestFinished } from 'vitest'
import { generationLogPath } from '../src/format.ts'
import { compressZstdFrame } from '../src/zstd.ts'
const id = SessionId('retired-content-admission')
const wrapper = { type: 'tool-result', toolCallId: 'retired-call', content: [{ type: 'text', text: 'result' }] }
const modes = (['none', 'zstd'] as const).flatMap(compression =>
(['read', 'write'] as const).map(access => ({ compression, access })))
describe.each(modes)('retired content in JSONL ($compression, $access)', ({ compression, access }) => {
async function stored(version: 3 | 4, opaque: boolean, corruptPrefix: boolean) {
const root = await mkdtemp(join(tmpdir(), 'dsh-retired-content-'))
const ctx = new Context()
onTestFinished(async () => {
try { await ctx.fiber.dispose() } finally { await rm(root, { recursive: true, force: true }) }
})
const event = { type: opaque ? 'future/opaque' : 'user/message', seq: 0, time: 1,
...(opaque ? { ignorable: true } : { surfaceOp: 'append' }),
data: { id: 'user', role: 'user', source: { kind: 'user' }, content: [wrapper] },
}
const header = { type: 'session', version, id, createdAt: 1, delegationDepth: 0, isSeeded: false }
const lines = [JSON.stringify(header) + '\n', ...(corruptPrefix ? ['{invalid json}\n'] : []), JSON.stringify(event) + '\n']
const bytes = compression === 'none' ? Buffer.from(lines.join(''))
: Buffer.concat(await Promise.all(lines.map(compressZstdFrame)))
const path = generationLogPath(root, undefined, id, version, compression)
await mkdir(dirname(path), { recursive: true })
await writeFile(path, bytes)
await ctx.plugin(JsonlSessionPersistence, { root, compression })
return { ctx, path, bytes, event }
}
it.each([{ version: 3, corruptPrefix: false }, { version: 4, corruptPrefix: false }, { version: 4, corruptPrefix: true }] as const)
('refuses V$version content with corruptPrefix=$corruptPrefix and preserves the only generation', async ({ version, corruptPrefix }) => {
const { ctx, path, bytes } = await stored(version, false, corruptPrefix)
const opened = ctx.sessionPersistence.open(id, access).then(async (handle) => { await handle.close() })
await expect(opened).rejects.toBeInstanceOf(version === 3 ? SessionFormatUnsupportedError : SessionPersistenceCorruptionError)
await expect(opened).rejects.toThrow(/released tool-result wrapper/)
expect(await readFile(path)).toEqual(bytes)
expect((await readdir(dirname(path))).filter(name => name !== 'session.lock')).toEqual([basename(path)])
})
it.each([3, 4] as const)('preserves an unknown ignorable V%s payload without interpreting its content', async (version) => {
const { ctx, path, bytes, event } = await stored(version, true, false)
const handle = await ctx.sessionPersistence.open(id, access)
try { expect((await handle.read()).events).toEqual([event]) } finally { await handle.close() }
expect(await readFile(path)).toEqual(bytes)
})
})