From cc970738d8129ee2ad1797a69bd081d37fc73cf2 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 19 Sep 2026 13:43:08 +0800 Subject: [PATCH] docs(session): scope checkpoint JSON preservation to domain reads The cache reader preserves opaque JSON keys, but each projection still parses the returned checkpoint through its own stateSchema before hydration. State that ownership explicitly so documentation does not imply this reader fix also changes projection-defined transformations or cloning behavior. Distinguish the synthetic V7 StorageDomain regression fixture from released cache archives. Describe its read, put and reopen path directly, and group its JSON validation checks separately from archived-version recovery. Translate the Chinese Agent Note headings and keep the paired docs aligned. The note retains the borrowed-value tradeoff and links the independent predecessor-format and lifecycle-identity decision. Production expressions and all test assertions remain unchanged; only contracts, fixture descriptions, test labels and grouping are clarified. The fix does not add another persistence format or broaden migration scope. --- ...ssless-projection-checkpoint-json.i18n.yaml | 4 ++-- ...9-19-lossless-projection-checkpoint-json.md | 4 +++- ...9-lossless-projection-checkpoint-json.zh.md | 12 +++++++----- .../session-projection-cache/README.i18n.yaml | 4 ++-- .../session/session-projection-cache/README.md | 2 +- .../session-projection-cache/README.zh.md | 2 +- .../session-projection-cache/src/spec.ts | 4 ++-- .../tests/fixtures.spec.ts | 18 ++++++++++-------- 8 files changed, 28 insertions(+), 22 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.i18n.yaml b/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.i18n.yaml index c05ca67333..a66f2afe77 100644 --- a/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.md -2026-09-19-lossless-projection-checkpoint-json.md: 9ee1b428b894469a195d8c7a361868591749e201 -2026-09-19-lossless-projection-checkpoint-json.zh.md: 4eb10c80be6bb99bcd284a1a731da1128179ed82 +2026-09-19-lossless-projection-checkpoint-json.md: 9c7d15dbd2d570eaa544f168d2c2ef882558d5ca +2026-09-19-lossless-projection-checkpoint-json.zh.md: 8c2b720358351af33a6d07888002c65d0cc30363 diff --git a/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.md b/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.md index 9ee1b428b8..9c7d15dbd2 100644 --- a/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.md +++ b/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.md @@ -19,4 +19,6 @@ The checkpoint value schema uses the existing `isJsonValue` predicate from `dsh- ## Consequences -Every valid own key survives domain reopen, including nested `__proto__` and `constructor` properties. The domain version remains unchanged because the stored JSON representation is unchanged; this corrects its reader. Regression coverage opens a fixture produced by the real writer, then writes, closes, and reopens it through `StorageDomain`. Session format versions and historical generations do not change. +Domain reads preserve every valid own key in checkpoint values, including nested `__proto__` and `constructor` properties. A projection's `stateSchema` still owns its hydration value; opaque fields must use validation that preserves their keys. The borrowed-value validator does not create an independent copy or project to JSON Schema. + +The domain version remains unchanged because the stored JSON representation is unchanged; this corrects its reader. Regression coverage opens a synthetic fixture produced by the real writer, then writes, closes, and reopens it through `StorageDomain`. Session format versions and historical generations do not change. [Predecessor recovery and Session-format binding](2026-09-02-projcache-cross-version-read-compat.md) continue to own cache-version and identity compatibility. diff --git a/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.zh.md b/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.zh.md index 4eb10c80be..8c2b720358 100644 --- a/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.zh.md +++ b/.agents/notes/implemented/architecture/2026-09-19-lossless-projection-checkpoint-json.zh.md @@ -4,19 +4,21 @@ Status: implemented [English](2026-09-19-lossless-projection-checkpoint-json.md) | 中文 -## Problem +## 问题 投影检查点可以包含不透明扩展数据和消息元数据。名为 `__proto__` 的 JSON 键是普通记录数据。Zod JSON 解析器在重建对象时丢弃该自有键，因此重新打开有效检查点可能得到与回放 Session 日志不同的投影状态。 -## Decision +## 决策 检查点值 schema 使用 `dsh-util-values` 中既有的 `isJsonValue` 谓词。它执行与检查点写入器 `snapshotJsonValue` 相同的无损 JSON 规则，不重建有效对象。校验仍拒绝非 JSON 和有损值。Storage-domain 表值是不可变的借用记录；校验器不提供防御性复制保证。 -## Alternatives considered +## 备选方案 - **保留 `z.json()`。** 其对象重建会移除有效自有键，因此校验成功仍可能改变检查点值。 - **用 `snapshotJsonValue` 校验并复制。** 这能保留键，但会复制已经不可变的存储值。只读谓词符合 storage-domain 的借用值约定。 -## Consequences +## 影响 -每个有效自有键都在 domain 重新打开后保留，包括嵌套的 `__proto__` 和 `constructor` 属性。存储 JSON 表示未变，因此 domain 版本不变；本修改修正其读取器。回归测试打开真实写入器生成的 fixture，再通过 `StorageDomain` 写入、关闭并重新打开。Session 格式版本与历史代际不变。 +Domain 读取保留检查点值中的每个有效自有键，包括嵌套的 `__proto__` 和 `constructor` 属性。Projection 的 `stateSchema` 仍拥有其 hydration 值；不透明字段必须使用保留其键的校验器。借用值校验器不生成独立副本，也不投影为 JSON Schema。 + +存储 JSON 表示未变，因此 domain 版本不变；本修改修正其读取器。回归测试打开真实写入器生成的合成 fixture，再通过 `StorageDomain` 写入、关闭并重新打开。Session 格式版本与历史代际不变。[前代恢复与 Session 格式绑定](2026-09-02-projcache-cross-version-read-compat.zh.md) 继续负责缓存版本与身份兼容性。 diff --git a/packages/session/session-projection-cache/README.i18n.yaml b/packages/session/session-projection-cache/README.i18n.yaml index 630e0b2a4a..9945e2d6dd 100644 --- a/packages/session/session-projection-cache/README.i18n.yaml +++ b/packages/session/session-projection-cache/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-projection-cache/README.md -README.md: ab2336a7d13dada307322a96fffb9d6900bd5411 -README.zh.md: 20275354ff8c7c5cb6c058899873e1dc5199d37a +README.md: 0417601210b697983993f770688a4e944d518790 +README.zh.md: 32dcbd93c8f18e504ca3a350586860b2e5cbd5b6 diff --git a/packages/session/session-projection-cache/README.md b/packages/session/session-projection-cache/README.md index ab2336a7d1..0417601210 100644 --- a/packages/session/session-projection-cache/README.md +++ b/packages/session/session-projection-cache/README.md @@ -62,7 +62,7 @@ Three mandatory points always write: session creation persists the seed-derived ### What the cache guarantees -The log leads and the cache follows: a live checkpoint flushes the session's buffered events durably before the cache row lands, so a crash can leave the cache behind the log but never ahead of it. Reads and writes share the storage domain's coherent in-memory state; the per-unit write chain mutates memory only after durability. Each version-stamped record must match the live unit schema and complete lifecycle identity (`formatVersion`, `createdAt`, `cwd`, `isSeeded`, and `inheritedEventCount`), so a row folded from another Session format generation or fork cut cannot seed the caller. The JSON backend stores each record at `/session_projcache/sessions/.json` in an owner-only directory tree. State validation preserves every own JSON key, including `__proto__` and `constructor` inside opaque metadata. It rejects values that cannot survive a lossless JSON round trip and uses the same rules as checkpoint writes. +The log leads and the cache follows: a live checkpoint flushes the session's buffered events durably before the cache row lands, so a crash can leave the cache behind the log but never ahead of it. Reads and writes share the storage domain's coherent in-memory state; the per-unit write chain mutates memory only after durability. Each version-stamped record must match the live unit schema and complete lifecycle identity (`formatVersion`, `createdAt`, `cwd`, `isSeeded`, and `inheritedEventCount`), so a row folded from another Session format generation or fork cut cannot seed the caller. The JSON backend stores each record at `/session_projcache/sessions/.json` in an owner-only directory tree. Domain read validation preserves every own JSON key in checkpoint values, including `__proto__` and `constructor` inside opaque metadata. It rejects values that cannot survive a lossless JSON round trip, using the same rules as checkpoint writes. Each projection then validates its hydration state with its own `stateSchema`; fields carrying opaque JSON need a validator that preserves their keys. Upgrades never block startup or expose an unproven fold. Records stamped with a version in the spec's `compatibleVersions` remain structurally readable for a current checkpoint rewrite, but a missing or older `formatVersion` never matches a current Session and therefore cannot seed hydration. A lifecycle-matching predecessor title remains available only through the listing hint above because title text is invariant across the adjacent Session-format edges and its row still passes the current projection `stateVersion` and schema. Once the format matches, absent lineage fields decode as the unseeded lineage — exact for unseeded sessions, while a seeded caller fails the identity match and refolds cold. A stored record that still fails schema validation is moved aside as `.json.bak.` under the domain's `invalidRecords: 'backup-and-skip'` policy, logged with its cause, and rebuilt by the next checkpoint. diff --git a/packages/session/session-projection-cache/README.zh.md b/packages/session/session-projection-cache/README.zh.md index 20275354ff..32dcbd93c8 100644 --- a/packages/session/session-projection-cache/README.zh.md +++ b/packages/session/session-projection-cache/README.zh.md @@ -62,7 +62,7 @@ kind: "package-reference" ### 缓存保证什么 -日志领先，缓存跟随：活会话检查点先把会话的缓冲事件持久化，然后才保存缓存记录。因此崩溃可能让缓存落后于日志，但绝不会让缓存领先。读取和写入共享存储域内一致的内存状态；逐单元写入链只在持久化成功后修改内存。每个带版本戳的记录必须匹配当前运行单元的 schema 与完整生命周期身份（`formatVersion`、`createdAt`、`cwd`、`isSeeded` 和 `inheritedEventCount`），因此从另一会话格式代或 fork 切点折叠出的行不能播种调用方。JSON 后端把每条记录存于仅所有者可访问的 `/session_projcache/sessions/.json` 目录树中。 状态校验保留所有自有 JSON 键，包括不透明元数据中的 `__proto__` 与 `constructor`。它拒绝无法无损完成 JSON 往返的值，并与检查点写入使用相同规则。 +日志领先，缓存跟随：活会话检查点先把会话的缓冲事件持久化，然后才保存缓存记录。因此崩溃可能让缓存落后于日志，但绝不会让缓存领先。读取和写入共享存储域内一致的内存状态；逐单元写入链只在持久化成功后修改内存。每个带版本戳的记录必须匹配当前运行单元的 schema 与完整生命周期身份（`formatVersion`、`createdAt`、`cwd`、`isSeeded` 和 `inheritedEventCount`），因此从另一会话格式代或 fork 切点折叠出的行不能播种调用方。JSON 后端把每条记录存于仅所有者可访问的 `/session_projcache/sessions/.json` 目录树中。Domain 读取校验保留检查点值中的所有自有 JSON 键，包括不透明元数据中的 `__proto__` 与 `constructor`。它拒绝无法无损完成 JSON 往返的值，并与检查点写入使用相同规则。随后，每个 projection 用自己的 `stateSchema` 校验 hydration 状态；承载不透明 JSON 的字段需要使用保留其键的校验器。 升级绝不拖垮启动，也不会暴露未经证明的折叠结果。版本戳落在 spec `compatibleVersions` 集合内的记录仍可被结构化读取并等待当前检查点重写，但缺失或更旧的 `formatVersion` 绝不匹配当前 Session，因此不能作为 hydrate seed。生命周期匹配的 predecessor title 只能通过上述列表 hint 读取，因为 title 文本在相邻 Session format edge 之间保持不变，并且该 row 仍须通过当前 projection `stateVersion` 与 schema。格式匹配后，缺失的 lineage 字段解码为 unseeded lineage——对非 fork 会话精确无误，seeded 调用方则通不过身份比对、回落冷折叠。仍然通不过 schema 校验的存量记录会按域的 `invalidRecords: 'backup-and-skip'` 策略移出为 `.json.bak.<时间戳>`、连同原因写入日志，并由下一次检查点重建。 diff --git a/packages/session/session-projection-cache/src/spec.ts b/packages/session/session-projection-cache/src/spec.ts index 12bd558616..c8bcda38b8 100644 --- a/packages/session/session-projection-cache/src/spec.ts +++ b/packages/session/session-projection-cache/src/spec.ts @@ -22,8 +22,8 @@ import { defineDomain, domainTable } from '@deepseek-ai/dsh-storage-domain' * minus the two record keys). `val` is the unit's internal state — plain * JSON by the unit contract. Validation uses the same lossless JSON rules as * writes and preserves every state key without cloning. A row is never wrong, - * only possibly stale: `seq` says exactly - * how stale, and a `ver` mismatch against the live unit's `stateVersion` + * only possibly stale: `seq` says exactly how stale, and a `ver` mismatch + * against the live unit's `stateVersion` * discards it at read time (never a migration). */ export const checkpointRow = z.object({ diff --git a/packages/session/session-projection-cache/tests/fixtures.spec.ts b/packages/session/session-projection-cache/tests/fixtures.spec.ts index a537e3b108..3b3de927d6 100644 --- a/packages/session/session-projection-cache/tests/fixtures.spec.ts +++ b/packages/session/session-projection-cache/tests/fixtures.spec.ts @@ -1,16 +1,16 @@ /** - * Cross-version recovery over archived on-disk artifacts. `fixtures/` holds - * real `session_projcache` media, each produced by driving the named release - * through its own web app (session created over RPC, real model turns, a - * rename): the v3 whole-unit file (published 0.1.1-rc.2), a v4 per-record + * Cross-version recovery and lossless checkpoint JSON validation. The released + * `session_projcache` fixtures were captured through their own web app + * (session created over RPC, real model turns, a rename): the v3 whole-unit + * file (published 0.1.1-rc.2), a v4 per-record * document (published 0.1.2-alpha.3), a published v5 document, and the * v5-stamped lineage-less document reproducing byte-for-byte what the * formerly unguarded legacy bootstrap wrote over v3 records. Each must open * through the real storage stack without becoming a fold shortcut for the * current Session format, then accept a current checkpoint rewrite. A record * that fails schema validation is backed up and skipped instead of failing the - * boot. The V7 opaque-state fixture was captured through the real storage - * domain before replacing the checkpoint state's cloning JSON validator. + * boot. The synthetic V7 fixture comes from the real StorageDomain per-record + * writer and contains opaque keys and arrays, independent of Session messages. */ import { afterEach, describe, expect, it, vi } from 'vitest' @@ -150,8 +150,8 @@ afterEach(async () => { await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 }))) }) -describe('archived version recovery', () => { - it('preserves every opaque key through an archived V7 read, current rewrite, and StorageDomain reopen', async () => { +describe('checkpoint JSON preservation', () => { + it('preserves opaque keys through StorageDomain read, put, and reopen', async () => { const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-opaque-')) const id = SessionId('opaque-fixture') const archived = await placeDoc(root, id, 'v7-opaque-session-doc.json') @@ -192,7 +192,9 @@ describe('archived version recovery', () => { const val = JSON.parse('{"__proto__":{"saved":true},"constructor":{"saved":false},"nested":{"__proto__":[null,true,4,"text"]}}') as Record expect(checkpointRow.parse({ ver: 1, seq: 0, val }).val).toBe(val) }) +}) +describe('archived version recovery', () => { it('recovers the v3 whole-unit archive through the legacy bootstrap', async () => { const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-fx-')) await cp(join(FIXTURES, 'v3-single-unit.json'), join(root, `${projectionCacheDomainSpec.name}.json`))