Merge master into session format migration
# Conflicts: # packages/session/session-projection-cache/tests/cache.spec.ts
This commit is contained in:
commit
cf9948b147
43 files changed
+1625
-148
No files matched your search
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-19-projection-cache-per-session-files.md
|
||||
2026-08-19-projection-cache-per-session-files.md: 0fd171649c1d8c7c3be7a8089287d43782b84714
|
||||
2026-08-19-projection-cache-per-session-files.zh.md: 47decb598cc70233c287feab2dddff46bd7bcbc9
|
||||
2026-08-19-projection-cache-per-session-files.md: 0792ed6cac23d1ae09767f608c74f82430db68e9
|
||||
2026-08-19-projection-cache-per-session-files.zh.md: 201146ce8568b66eebf6793fcdc489929a3e4cdb
|
||||
+3
-3
@@ -20,8 +20,8 @@ Reads and writes share ONE coherent state: every read (`cachedSnapshot`) is a sy
|
||||
- Listing is a synchronous in-memory read; a session without a record document simply lacks the projection column.
|
||||
- ACP, headless, SDK, and Web sessions publish cache rows for later consumers. The log-leading durability barrier may flush a covered prefix at the cache cadence and split otherwise coalesced physical JSONL runs; recorded profile snapshots re-pack the logical event stream so cache timing does not define fixture layout.
|
||||
- The per-record contract scopes failure: a malformed or stale-version document reads as an absent record at open, so one bad file never bricks the cache, and a checkpoint schema bump discards stale sessions per record instead of rejecting the whole domain.
|
||||
- The json backend bootstraps the per-record tree from the legacy whole-unit cache only when enumeration finds no new-layout document path and the legacy unit name and version match the requested descriptor. A different version remains untouched and the new domain opens empty; storage never relabels its values as the current version. Any new document path, including an unreadable or stale file, suppresses the bootstrap for the whole unit; missing session rows refold from the log.
|
||||
- The `session_projcache` domain uses version 6. Every version-5 record reads as absent, including a healthy one, so a poisoned version-5 record cannot fail domain validation. Session headers and event logs remain in session persistence; an exact read refolds them and writes a version-6 cache record, while zero-I/O listings lack that projection until the cache returns.
|
||||
- The json backend bootstraps the per-record tree from the legacy whole-unit cache only when enumeration finds no new-layout document path, the legacy unit name matches, and its version is current or declared compatible. A version outside that accepted set remains untouched and the new domain opens empty; storage never relabels a version the domain owner did not approve. Any new document path, including an unreadable or stale file, suppresses the bootstrap for the whole unit; missing session rows refold from the log. The [cross-version read-compatibility decision](2026-09-02-projcache-cross-version-read-compat.md) owns the version policy.
|
||||
- The `session_projcache` domain uses version 7 and declares versions 3 through 6 structurally compatible. Predecessor identities lack the Session `formatVersion`, so they cannot seed current projections and refold cold; a current checkpoint rewrites them with the complete identity. Once the format matches, absent lineage fields normalize to an unseeded identity, and a seeded caller rejects that identity and refolds cold. A record that still fails schema validation is backed up and skipped; every subsequent write stamps version 7.
|
||||
- The cache record is bound to the same log lifecycle as before: the stored `{createdAt, cwd, isSeeded, inheritedEventCount}` identity guards against a recreated id or a mismatched inherited prefix.
|
||||
|
||||
## Alternatives considered
|
||||
@@ -30,4 +30,4 @@ Reads and writes share ONE coherent state: every read (`cachedSnapshot`) is a sy
|
||||
- **Cache-owned per-session files** (`<root>/<session-id>/projection_cache.json`, the first revision of this change). Tried and reverted in review: the cache hand-rolled the medium — paths, per-path write chains, in-flight tracking, owner-only file modes, and a sqlite no-path special case — and its listing read hit the disk directly on every call while writes were throttled, so reads and writes were never consistent.
|
||||
- **Resolve the path through `sessionPersistence.locate(meta)`** (the file beside the session log). Rejected: the cache would have to guess "beside the log" from a log artifact path (`dirname` + fixed filename), coupling the cache to the persistence service and to a backend's layout.
|
||||
- **Make `per-record` a mode of the existing unit instead of a separate unit class.** Rejected: the two layouts have genuinely different state models — `single` is memory-authoritative with whole-file publish, `per-record` is stateless (the directory is the state; `loadAll` re-reads the tree) — so they are separate small classes behind one backend, with record keys validated path-safe instead of encoded.
|
||||
- **Copy legacy values across unit versions.** Rejected: the json backend does not know a domain's record schema and cannot derive session-lineage fields. Copying raw values under the requested version relabels data without migrating it. A domain that requires compatibility owns an explicit migration; the projection cache instead discards old records and rebuilds them from session logs.
|
||||
- **Copy legacy values across undeclared unit versions.** Rejected: the json backend does not know a domain's record schema and cannot derive session-lineage fields. It copies an older record only when the domain explicitly lists that version in `compatibleVersions` and its current schema accepts the value; otherwise the record stays untouched and reads as absent.
|
||||
+3
-3
@@ -20,8 +20,8 @@ Status: implemented
|
||||
- 列表读取是同步内存读;没有记录文档的会话只是缺少投影列。
|
||||
- ACP、headless、SDK 与 Web 会话都会发布缓存行,供后续消费方使用。确保日志领先的持久性屏障可能按缓存节奏 flush 已覆盖的前缀,并拆分原本会合并的物理 JSONL 行;各 profile 的录制快照会重新 pack 逻辑事件流,因此缓存时序不会决定 fixture 布局。
|
||||
- per-record 契约把故障范围缩小到单记录:畸形或过期版本的文档在打开时读作"无此记录",单个坏文件不会拖垮整个缓存;检查点 schema 升级按会话丢弃过期行,而不是拒绝整个域。
|
||||
- json 后端仅在枚举时没有发现任何新布局文档路径,且旧单元名称和版本与请求的 descriptor 相同时,才从旧整单元缓存引导 per-record 目录树。版本不同时,旧文件保持不变,新域为空;存储不会把旧值改标为当前版本。只要存在任意新文档路径,即使文件不可读或版本陈旧,也会对整个单元禁用引导;缺失的会话行从日志重折叠。
|
||||
- `session_projcache` 域使用版本 6。所有版本 5 记录都读作缺失,包括健康记录,因此被污染的版本 5 记录不能再使域校验失败。会话 header 和事件日志仍保存在会话持久化中;精确读取会重折叠这些数据并写入版本 6 缓存,而零 I/O 列表在缓存恢复前缺少对应投影。
|
||||
- json 后端仅在枚举时没有发现任何新布局文档路径、旧单元名称匹配,且其版本为当前版本或已声明兼容版本时,才从旧整单元缓存引导 per-record 目录树。接受集合之外的版本保持不变,新域为空;存储绝不把域 owner 未批准的版本改标为当前版本。只要存在任意新文档路径,即使文件不可读或版本陈旧,也会对整个单元禁用引导;缺失的会话行从日志重折叠。[跨版本读兼容决策](2026-09-02-projcache-cross-version-read-compat.zh.md)是版本策略的权威说明。
|
||||
- `session_projcache` 域使用版本 7,并声明版本 3 至 6 在结构上兼容。前代 identity 缺少 Session `formatVersion`,因此不能播种当前投影并会回落冷折叠;当前检查点会用完整 identity 重写它们。格式匹配后,缺失的 lineage 字段归一化为 unseeded 身份,seeded 调用方会拒绝该身份并回落冷折叠。仍然通不过 schema 校验的记录会被备份并跳过;后续每次写入都使用版本 7。
|
||||
- 缓存记录仍绑定同一日志生命周期:存储的 `{createdAt, cwd, isSeeded, inheritedEventCount}` 身份防止被重建的 id 或不匹配的继承前缀误导。
|
||||
|
||||
## Alternatives considered
|
||||
@@ -30,4 +30,4 @@ Status: implemented
|
||||
- **缓存自持的每会话文件**(`<root>/<session-id>/projection_cache.json`,本改动的第一版)。试过并在评审中回退:缓存手搓了介质——路径、按路径的写链、在途跟踪、仅属主文件权限,以及 sqlite 无路径特判——而且它的列表读每次调用都直读磁盘、写却在节流,读写永不一致。
|
||||
- **经 `sessionPersistence.locate(meta)` 解析路径**(文件放在会话日志旁)。未采用:缓存得从日志 artifact 路径"猜"日志旁边(`dirname` + 固定文件名),把缓存耦合到持久化服务与后端的布局。
|
||||
- **把 `per-record` 做成既有单元的一种模式而非独立单元类。** 未采用:两种布局的状态模型本质不同——`single` 内存权威、整文件发布;`per-record` 无状态(目录即状态,`loadAll` 重扫目录树)——所以它们是同一后端下的两个小型独立类,记录键做路径安全校验而非编码。
|
||||
- **跨单元版本复制旧值。** 未采用:json 后端不知道域的记录 schema,也无法推导会话谱系字段。按请求版本复制原始值只会修改数据标签,不会迁移数据。需要兼容性的域负责显式迁移;投影缓存改为丢弃旧记录,并从会话日志重建。
|
||||
- **跨未声明单元版本复制旧值。** 未采用:json 后端不知道域的记录 schema,也无法推导会话 lineage 字段。只有当域在 `compatibleVersions` 中明确列出旧版本,且当前 schema 接受该值时,后端才复制旧记录;否则记录保持不变并读作不存在。
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md
|
||||
2026-08-31-released-session-format-migrations.md: 65d89b680e827b29a3a0e5d485a53f7a6eb49f63
|
||||
2026-08-31-released-session-format-migrations.zh.md: 6089f6dc4f231aaeec31b89600485ef2a06fb9b0
|
||||
2026-08-31-released-session-format-migrations.md: 5f6a8145dcf67872461c17408d18f5a22d820ba6
|
||||
2026-08-31-released-session-format-migrations.zh.md: 0aa872877d82d919f43f4a16f5570f0c655e34df
|
||||
+2
@@ -28,6 +28,8 @@ The current-format fast path classifies the header from one stable source snapsh
|
||||
|
||||
The first edge, `@deepseek-ai/dsh-session-format-v0-to-v1`, is intentionally identity-shaped: aside from the version and bounded historical normalizations already accepted by v0, it preserves logical headers, events, sequence numbers, references, timestamps, payloads, and the configured compression choice. The exact `session.jsonl[.zstd]` source remains byte- and inode-identical, while the current writer encodes the new `session.v1.jsonl[.zstd]` successor. This exercises the complete publication lifecycle before a cardinality-changing format needs it.
|
||||
|
||||
Projection-cache records bind their fold to the Session header's `formatVersion`. The `session_projcache` v7 reader may load predecessor domain records structurally, but a record without the format generation cannot seed a current Session; the authoritative log refolds it and the next checkpoint writes the complete current identity. This prevents a cache row produced before a bounded normalizer or cardinality-changing edge from bypassing that migration.
|
||||
|
||||
## Consequences
|
||||
|
||||
Reading event bodies with a newer build may durably add a higher generation. The exact old generation remains available, but the runtime thereafter selects the highest canonical filename; retention does not promise that an older build can safely downgrade or that the newer build will fall back when the successor is corrupt. A read-only filesystem reports an actionable migration failure instead of returning an in-memory current view that differs from disk.
|
||||
|
||||
+2
@@ -28,6 +28,8 @@ JSONL provider 在 `open` 为已存储 Session 返回句柄前完成 ensure-curr
|
||||
|
||||
第一条迁移边 `@deepseek-ai/dsh-session-format-v0-to-v1` 有意保持恒等形态:除版本和 v0 已接纳的有限历史归一化外,它保留逻辑 header、事件、序号、引用、时间戳、payload 与已配置的压缩选择。精确的 `session.jsonl[.zstd]` 源保持字节与 inode 相同,当前 writer 则编码新的 `session.v1.jsonl[.zstd]` 后继。这样可在出现改变基数的格式前先验证完整发布生命周期。
|
||||
|
||||
投影缓存记录把自己的折叠结果绑定到 Session header 的 `formatVersion`。`session_projcache` v7 reader 可以在结构上载入前代 domain 记录,但缺少格式代的记录不能播种当前 Session;权威日志会重新折叠它,下一次检查点写入完整的当前 identity。这样,任何在有界规范化或基数变化边之前产生的缓存行都不能绕过该迁移。
|
||||
|
||||
## 后果
|
||||
|
||||
较新 build 读取事件正文时可能持久增加一个更高 generation。精确旧 generation 仍然可用,但 runtime 此后选择最高规范文件名;保留不承诺旧 build 能安全 downgrade,也不保证新 build 在后继损坏时 fallback。只读文件系统会报告可操作的迁移失败,而不会返回与磁盘不一致的内存当前视图。
|
||||
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-02-projcache-cross-version-read-compat.md
|
||||
2026-09-02-projcache-cross-version-read-compat.md: 7463871852234d5df232f6c426dfbec0543777d5
|
||||
2026-09-02-projcache-cross-version-read-compat.zh.md: 663a645cd55a16a26add6edba4e414023f8c622c
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
# Agent Note: Projection-cache predecessor recovery and Session-format binding (session_projcache v3-v6 → v7)
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-09-02-projcache-cross-version-read-compat.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The `session_projcache` storage domain evolved through several on-disk generations. An upgraded DSH_HOME exposed three risks:
|
||||
|
||||
- **A v3 single-file home bricked startup after the upgrade**: the per-record layout's legacy bootstrap migrated the old whole-unit file without checking its `unit.version`, stamping the old records with the current version into the new tree; the domain layer's per-record zod validation at open then hit the missing now-required fields → `invalid-record` → the whole domain refused to open → the plugin tree failed to load. And because the bootstrap writes before validation runs, **the first boot permanently wrote the bad documents into the new tree** ("poisoning") — every later boot saw a non-empty tree, never took the legacy path again, and the home stayed unusable.
|
||||
- **A v4 per-record home lost its listing titles after the upgrade**: v4 documents were silently discarded by the version-stamp check (the per-record contract), and SessionList is a zero-I/O cache-only read, so a miss served the row without projections; titles only returned as each session was individually reopened.
|
||||
- **A Session-format bump could reuse a fold produced under older event semantics**: versions 3 through 6 did not record the Session format generation. Treating an absent generation as current would let a cache row bypass bounded historical normalization or a cardinality-changing migration.
|
||||
|
||||
The cache domain's own contract is "a stale or unreadable cache costs a longer tail replay, never a wrong value, never a refused load" — the hard failure and the wholesale discard each broke the first half of that contract or the product expectation.
|
||||
|
||||
## The on-disk generations
|
||||
|
||||
| domain version | shipped in | layout | on-disk form | identity fields | row fields |
|
||||
|---|---|---|---|---|---|
|
||||
| 3 | 0.1.1-rc.2 | single | one file `storages/session_projcache.json` (`{unit:{name,version}, global, tables}`) | `createdAt`, `cwd?` | `ver`, `seq`, `val` |
|
||||
| 4 | 0.1.2-alpha.3 | per-record | one file per session `storages/session_projcache/sessions/<sessionId>.json` (`{version, record}`) | `createdAt`, `cwd?` | same |
|
||||
| 5 | 0.1.2-alpha.4 | per-record | same as v4 | + `isSeeded` (shipped required; now optional), `inheritedEventCount` (same) | same (`seq` numbers mean the same as v4; only type brands were added) |
|
||||
| 6 | pre-v1 mainline | per-record | same as v5 | same as v5 | same |
|
||||
| 7 | current | per-record | same as v5 | + `formatVersion`; current writes also require both lineage fields | same |
|
||||
|
||||
The only substantive v4→v5 difference is the two lineage identity fields; v6 changed only the write stamp. The `ver/seq/val` row representation is identical across these predecessor generations, and `seq` numbering did not change ([the 2026-08-31 seq/offset brands note](2026-08-31-session-sequence-and-log-offset-brands.md) pins the on-disk numbers as unchanged). v3→v4 was a layout migration with identical record content. Version 7 adds the Session format generation to the cache identity because row semantics cannot be inferred from the domain stamp.
|
||||
|
||||
One derived shape also exists: a v3 home that ran the v5 build once (the poisoned state) — its new tree holds documents **stamped 5 whose content is a v3 record** (no lineage fields).
|
||||
|
||||
## Decision
|
||||
|
||||
Declared read compatibility — reads tolerate vouched-for older versions, writes always stamp the current one:
|
||||
|
||||
1. **`DomainSpec.compatibleVersions` (new, optional)**: the domain owner declares "records stored under these older versions are also readable under the current record schemas" (typically by declaring the fields old records lack as optional). `defineDomain` validates each entry as a non-negative integer below the current version; `descriptorOf` projects the set onto the backend `KvUnitDescriptor`.
|
||||
2. **json backend per-record reads** accept version stamps in "current ∪ compatibleVersions"; anything outside the set is still discarded as foreign. **The write path always stamps the current version** (the first checkpoint after reading an old record naturally advances it). The `single` layout stays exact-version.
|
||||
3. **Legacy-bootstrap version gate (the actual bug fix)**: the old whole-unit file's `unit.version` must fall inside the accepted set to be migrated; otherwise the file is left alone and the unit reads empty — stamping records the owner never vouched for turns a discardable stale cache into hard schema failures at the domain layer.
|
||||
4. **The projcache domain declares `version: 7, compatibleVersions: [3, 4, 5, 6]`**, and the format and lineage identity fields are optional in the stored schema so vouched-for predecessor records can open. Current writes always include all three fields.
|
||||
5. **Identity matching is stricter than structural admission**: an absent `formatVersion` never matches a current Session, so predecessor rows cannot seed a projection and refold from the authoritative log. Once the format matches, `identityMatches` normalizes absent lineage to unseeded (`?? false` / `?? 0`): exact for an unseeded session, while a seeded expectation fails the match. Poisoned v5 homes therefore boot safely, but their unbound rows are not exposed as current values.
|
||||
6. **Schema-validation backstop: `invalidRecords: 'backup-and-skip'` (declared by this domain only)**. A stored record that still fails to parse beyond read compatibility no longer refuses the whole domain: the domain layer calls the backend's `KvUnit.backupRecord` (json per-record implementation = rename the document to `<key>.json.bak.<YYYYMMDDHHmm>`, bytes kept, never read again), prints the concrete failure with `logger.error` (domain, table, key, destination, zod cause), and continues the open with the record absent; the next cold read rebuilds and rewrites that session's cache. **The policy is an explicit per-domain declaration and the default stays fail-loud** — other domains still refuse the whole load on invalid stored data, and a backend without `backupRecord` (single layout, row stores) also falls back to fail-loud. Naming history: quarantine → backup-and-skip (user ruling: the word must carry both "back up" and "skip", sharing its root with the `.bak` suffix; skip-backup was rejected because the CLI `--skip-X` convention reads it as "do not back up"). For this domain it supersedes the reset/destroy recovery path of the [2026-07-28 storage recovery proposal](../../proposed/architecture/2026-07-28-storage-root-and-derived-medium-recovery.md), which stays live for authoritative and whole-medium damage.
|
||||
|
||||
### v3-v6 → v7 disposition
|
||||
|
||||
Versions 3 through 6 remain structurally readable because their record and row representations are valid inputs to the current schema. Their identities lack `formatVersion`, so they are deliberately unusable as current fold shortcuts. A cold read or live checkpoint rebuilds the values from the migrated Session log and writes a v7 record with the complete format and lineage identity. No eager value migration runs at startup; a schema-invalid accepted record follows `backup-and-skip`.
|
||||
|
||||
### Upgrade matrix
|
||||
|
||||
| home shape | behavior after the fix |
|
||||
|---|---|
|
||||
| v3 single-file (not poisoned) | bootstrap migrates (3 ∈ accepted set) → boot succeeds; unbound fold waits for cold rebuild |
|
||||
| v3 + poisoned new tree | new-tree documents parse under optional fields → boot restored; unbound fold waits for cold rebuild |
|
||||
| v4/v5/v6 per-record | documents read structurally → missing format generation rejects the shortcut; current checkpoint rewrites v7 |
|
||||
| v7 current with matching identity | cached values serve normally |
|
||||
| matching-format record without lineage | unseeded caller may use it; seeded caller rejects it and refolds cold |
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Reject predecessor stamps in the storage layer**: safe for projections, but prevents the guarded legacy bootstrap and loses the ability to retain a structurally sound record until an authoritative refold replaces it. Structural admission plus semantic identity rejection keeps the boot recoverable without serving an unproven value.
|
||||
- **Treat a missing format generation as current**: preserves cached titles, but lets a pre-migration fold bypass the Session-format edge. Rejected because v0→v1 includes bounded historical normalizers and later edges may change event cardinality.
|
||||
- **Schema `.default()` fills**: behaviorally equivalent to optional + reader normalization, but bakes the "absent = unseeded" interpretation into the durable schema's output type; ruled for optional — the schema honestly describes every accepted on-disk shape and the interpretation lives at the consumer (user ruling, 2026-09-02).
|
||||
- **Roll the domain version back to 4**: a small diff, but breaks version monotonicity, depends on the "bootstrap skips no versions" bug itself, and drops every poisoned and healthy v5 home's cache.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A deployment routing this domain to the sqlite backend gets none of the tolerance: sqlite implements neither `compatibleVersions` nor `backupRecord`, so behavior degrades to the old strict-version semantics (a whole-unit version mismatch still refuses with `version-mismatch`; nothing loosens, nothing serves wrong values). Shipped compositions route this domain to json, so this stays a deployment-configuration risk only.
|
||||
- The optional format field lets predecessor records pass structural validation, but absence always fails current identity matching. Optional lineage is normalized only after the format matches; a seeded caller still refuses a lineage-less record. The per-row `ver` guard continues to screen every served value.
|
||||
- `backupRecord` overwrites a same-minute backup of the same key (the newer bytes win); distinct minutes and distinct keys never collide.
|
||||
|
||||
## Testing
|
||||
|
||||
- `storage-json` unit tests: compat-stamped reads / out-of-set discards / writes stamping current; legacy bootstrap migrating only accepted versions (including the migrated-documents-stamp-current assertion); `backupRecord` move / absent read / rewrite / closed guard.
|
||||
- `storage-domain` unit tests: `compatibleVersions` / `invalidRecords` declaration validation; backup-and-skip falling back to fail-loud when the backend has no `backupRecord`.
|
||||
- `session-projection-cache` unit tests: a matching format with absent lineage serves only an unseeded Session; a predecessor record without a format generation never serves.
|
||||
- **Archived-fixture recovery tests** (`tests/fixtures.spec.ts` + `tests/fixtures/`): four media archives produced by the real released builds — `v3-single-unit.json` (the 0.1.1-rc.2 whole-unit file), `v4-session-doc.json` (0.1.2-alpha.3), `v5-session-doc.json` (0.1.2-alpha.4), `v5-lineageless-doc.json` (the unguarded bootstrap's poisoned form, synthesized from the v3 record) — each opens through the real storage stack without serving its unbound fold, then a live write replaces it with a v7 record carrying the complete identity and fresh value. The same suite proves backup-and-skip for a schema-failing record: boot survives, `.bak` lands, diagnostics name the failure, and a neighboring predecessor record remains rewritable.
|
||||
|
||||
Future bump procedure: add an older domain version to `compatibleVersions` only when the current stored schema can parse it, and let the owning reader decide whether its semantic identity is sufficient. A Session-format change never inherits an absent format generation. The package README requires every bump to land with archived fixtures and tests proving structural admission, semantic use or rejection, and current rewrite.
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
# Agent Note: 投影缓存前代恢复与 Session 格式绑定(session_projcache v3-v6 → v7)
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-09-02-projcache-cross-version-read-compat.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`session_projcache` 存储域演进过多代磁盘结构。升级后的 DSH_HOME 暴露了三类风险:
|
||||
|
||||
- **v3 单文件 home 升级后启动硬失败**:per-record 布局的 legacy bootstrap 迁移旧单文件时不检查其 `unit.version`,把旧记录原样打上当前版本戳写入新树;domain 层开域时逐条 zod 校验,旧记录缺新增必填字段 → `invalid-record` → 整个域拒开 → 插件树加载失败。且 bootstrap 先写盘后校验,**首次启动即把坏文档永久写入新树**("投毒")——此后每次启动新树非空、连 legacy 路径都不再走,home 持续不可用。
|
||||
- **v4 per-record home 升级后列表丢标题**:v4 文档被版本戳检查静默丢弃(per-record 契约),SessionList 是零 I/O 纯缓存读,miss 后整行不带投影;标题要等每个会话被逐个重新打开后才恢复。
|
||||
- **Session 格式递增可能复用按旧事件语义折叠的结果**:版本 3 至 6 不记录 Session 格式代。若把缺失代视为当前代,缓存行就能绕过有界历史规范化或改变基数的迁移。
|
||||
|
||||
缓存域自身的契约是"过期或不可读的缓存只付出更长的尾部重放,绝不给出错值、绝不拒载"——硬失败与整体丢弃都违背该契约的前半句或产品预期。
|
||||
|
||||
## 磁盘结构代际
|
||||
|
||||
| domain version | 携带发布 | 布局 | 磁盘形态 | identity 字段 | 行字段 |
|
||||
|---|---|---|---|---|---|
|
||||
| 3 | 0.1.1-rc.2 | single | 单文件 `storages/session_projcache.json`(`{unit:{name,version}, global, tables}`) | `createdAt`, `cwd?` | `ver`, `seq`, `val` |
|
||||
| 4 | 0.1.2-alpha.3 | per-record | 每会话一份 `storages/session_projcache/sessions/<sessionId>.json`(`{version, record}`) | `createdAt`, `cwd?` | 同上 |
|
||||
| 5 | 0.1.2-alpha.4 | per-record | 同 v4 | + `isSeeded`(v5 首发必填;现为 optional)、`inheritedEventCount`(同前) | 同上(`seq` 数值语义与 v4 相同,仅类型加 brand) |
|
||||
| 6 | v1 之前的 mainline | per-record | 同 v5 | 同 v5 | 同上 |
|
||||
| 7 | 当前版本 | per-record | 同 v5 | + `formatVersion`;当前写入也要求两个 lineage 字段 | 同上 |
|
||||
|
||||
v4→v5 的唯一实质差异是 identity 新增两个 lineage 字段;v6 只改变写入版本戳。这些前代的行内 `ver/seq/val` 表示一致,`seq` 的数值含义未变([2026-08-31 seq/offset brands note](2026-08-31-session-sequence-and-log-offset-brands.zh.md) 明确 on-disk 数值不变)。v3→v4 是布局迁移,记录内容结构一致。v7 把 Session 格式代加入缓存 identity,因为无法从 domain 版本戳推导行语义。
|
||||
|
||||
另有一种衍生形态:跑过一次 v5 版本的 v3 home(投毒态)——新树里存在**版本戳为 5 但内容是 v3 记录**(缺 lineage 字段)的文档。
|
||||
|
||||
## 决策
|
||||
|
||||
声明式读兼容——读容忍 owner 背书过的旧版本,写恒戳当前版本:
|
||||
|
||||
1. **`DomainSpec.compatibleVersions`(新增,可选)**:域 owner 声明"这些旧版本的存量记录在当前记录 schema 下也可读"(典型手段:新增字段标 optional)。`defineDomain` 校验各项为小于当前 version 的非负整数;`descriptorOf` 透传到后端 `KvUnitDescriptor`。
|
||||
2. **json 后端 per-record 读**:接受"当前版本 ∪ compatibleVersions"内的版本戳,集合外照旧视为 foreign 丢弃;**写路径永远戳当前版本**(读到旧记录后的下一次 checkpoint 自然把它推进到当前版本)。single 布局维持 exact-version 不变。
|
||||
3. **legacy bootstrap 版本把关(bug 修复本体)**:旧单文件的 `unit.version` 必须落在接受集合内才迁移,否则视为空 unit 留在原地——为 owner 未背书的记录打当前版本戳,会把"可丢弃的过期缓存"变成 domain 层的 schema 硬失败。
|
||||
4. **projcache 域声明 `version: 7, compatibleVersions: [3, 4, 5, 6]`**;存储 schema 中的格式与 lineage identity 字段均为 optional,使 owner 背书的前代记录可以打开。当前写入始终包含这三个字段。
|
||||
5. **identity 匹配比结构准入更严格**:缺失 `formatVersion` 的记录绝不匹配当前 Session,因此前代行不能播种投影,而会从权威日志重新折叠。格式匹配后,`identityMatches` 才把缺失 lineage 归一化为 unseeded(`?? false` / `?? 0`):对 unseeded 会话精确,对 seeded 期望则匹配失败。v5 投毒 home 因而可以安全启动,但其未绑定行不会作为当前值暴露。
|
||||
6. **schema 校验兜底:`invalidRecords: 'backup-and-skip'`(仅本域声明)**。读兼容之外仍然解析失败的存量记录不再让整个域拒开:domain 层调用后端的 `KvUnit.backupRecord`(json per-record 实现=把文档改名为 `<key>.json.bak.<YYYYMMDDHHmm>`,字节留档、不再被读取),用 `logger.error` 打印具体失败信息(域名、表、键、移动去向、zod 失败原因),随后当该记录不存在继续启动;下一次冷读会重建并重写该会话的缓存。**该策略是域级显式声明,缺省仍为 fail-loud**——其他业务域的存量数据校验失败照旧整域拒载;后端没有 `backupRecord` 能力(single 布局、行存储)时也回退 fail-loud。命名沿革:quarantine → backup-and-skip(用户裁决:词要同时含"备份"与"跳过"两义,且与 `.bak` 后缀同源;skip-backup 因 CLI `--skip-X` 惯例存在"不备份"反读而弃用)。对本域而言,该策略取代了 [2026-07-28 存储恢复提案](../../proposed/architecture/2026-07-28-storage-root-and-derived-medium-recovery.zh.md)中 reset/destroy 的恢复途径;该提案对权威介质与整介质损坏仍然有效。
|
||||
|
||||
### v3-v6 → v7 处置
|
||||
|
||||
版本 3 至 6 仍可结构化读取,因为它们的记录与行表示是当前 schema 的有效输入。其 identity 缺少 `formatVersion`,因此有意不能作为当前折叠捷径。冷读或实时检查点会从迁移后的 Session 日志重建值,并写入带完整格式与 lineage identity 的 v7 记录。启动时不运行 eager 值迁移;schema 校验失败的已接受记录执行 `backup-and-skip`。
|
||||
|
||||
### 升级矩阵
|
||||
|
||||
| home 形态 | 修复后行为 |
|
||||
|---|---|
|
||||
| v3 单文件(未投毒) | bootstrap 迁移(3 ∈ 接受集)→ 启动成功;未绑定折叠等待冷重建 |
|
||||
| v3 + 投毒新树 | optional 字段让新树文档可解析 → 启动恢复;未绑定折叠等待冷重建 |
|
||||
| v4/v5/v6 per-record | 文档结构化读入 → 缺格式代而拒绝捷径;当前检查点重写 v7 |
|
||||
| identity 匹配的 v7 当前记录 | 正常服务缓存值 |
|
||||
| 格式匹配但缺 lineage 的记录 | unseeded 调用方可以使用;seeded 调用方拒绝并回落冷折叠 |
|
||||
|
||||
## 备选方案
|
||||
|
||||
- **在存储层拒绝前代版本戳**:对投影安全,但会阻止受保护的 legacy bootstrap,也无法保留结构完好的记录直到权威重折叠替换它。结构准入加语义 identity 拒绝既让启动可恢复,也不服务未经证明的值。
|
||||
- **把缺失格式代视为当前代**:可以保留缓存标题,但让迁移前折叠绕过 Session 格式边。不采用,因为 v0→v1 含有界历史规范化,后续边还可能改变事件基数。
|
||||
- **schema `.default()` 填缺省**:行为与 optional+读点归一化等价,但把"缺失=unseeded"的解释固化进 durable schema 的输出类型;拍板为 optional——schema 如实描述介质上所有被接受的形态,解释权在消费点(2026-09-02 用户裁决)。
|
||||
- **域版本回退到 4**:改动很小,但破坏版本单调性、依赖"bootstrap 不查版本"这个 bug 本身、且投毒态与正常 v5 home 的缓存全被丢弃。
|
||||
|
||||
## 影响
|
||||
|
||||
- 部署方若把本域路由到 sqlite 后端,得不到任何容忍能力:sqlite 既未实现 `compatibleVersions` 也没有 `backupRecord`,行为退化为原有的严格版本语义(整 unit 版本不匹配仍 `version-mismatch` 拒开;不放松、不出错值)。shipped 组合固定路由 json,此风险仅存在于部署配置层面。
|
||||
- optional 格式字段允许前代记录通过结构校验,但缺失格式始终无法通过当前 identity 匹配。只有格式匹配后才归一化 optional lineage;seeded 调用方仍拒绝缺 lineage 的记录。逐行 `ver` 守卫继续筛查每个实际服务的值。
|
||||
- `backupRecord` 对同一键的同一分钟内重复备份会覆盖前一份(新字节胜出);不同分钟、不同键永不冲突。
|
||||
|
||||
## 测试
|
||||
|
||||
- `storage-json` 单测:compat 版本戳读入/集合外丢弃/写恒当前版本;legacy bootstrap 仅在版本被接受时迁移(含迁移后文档戳当前版本断言);`backupRecord` 移档/读缺席/重写/封闭守卫。
|
||||
- `storage-domain` 单测:`compatibleVersions`/`invalidRecords` 声明校验;后端无 `backupRecord` 时 backup-and-skip 回退 fail-loud。
|
||||
- `session-projection-cache` 单测:格式匹配且缺 lineage 的记录只服务 unseeded Session;缺格式代的前代记录永不服务。
|
||||
- **归档 fixtures 独立恢复测试**(`tests/fixtures.spec.ts` + `tests/fixtures/`):真实发布物产出的四份介质存档——`v3-single-unit.json`(0.1.1-rc.2 整域单文件)、`v4-session-doc.json`(0.1.2-alpha.3)、`v5-session-doc.json`(0.1.2-alpha.4)、`v5-lineageless-doc.json`(无守卫 bootstrap 的投毒形态,由 v3 记录合成)——逐一走真实存储栈开域且不服务其未绑定折叠,随后由实时写入替换成带完整 identity 与新值的 v7 记录。同一套件还证明 schema 失败记录的 backup-and-skip:启动不失败、`.bak` 落盘、诊断点名失败,且邻近前代记录仍可重写。
|
||||
|
||||
未来 bump 流程:只有当前存储 schema 能解析旧 domain 版本时,才把它加入 `compatibleVersions`,再由 owner reader 判断其语义 identity 是否充分。Session 格式变化绝不继承缺失的格式代。包 README 要求每次 bump 都随附归档 fixture 与测试,证明结构准入、语义使用或拒绝,以及当前重写。
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/proposed/architecture/2026-07-28-storage-root-and-derived-medium-recovery.md
|
||||
2026-07-28-storage-root-and-derived-medium-recovery.md: 1505be1c58d5cf829327b2919113bb2e42798ce7
|
||||
2026-07-28-storage-root-and-derived-medium-recovery.zh.md: 1bab5ab663df1419cc826c3d6acb59bd8bff7de0
|
||||
2026-07-28-storage-root-and-derived-medium-recovery.md: 68a6792c38a8fe097558de3d5a857ab4ea0d3533
|
||||
2026-07-28-storage-root-and-derived-medium-recovery.zh.md: 0c712b3b928f4b3df44e706b154c3a90a784403c
|
||||
+1
-1
@@ -10,7 +10,7 @@ The persisted projection cache ([note](2026-07-27-session-projection-and-command
|
||||
|
||||
**Where the files actually live (root mismatch closed; resolve-once residual still open).** The shared base defaults the session store to the global harness home (`$DSH_HOME/sessions`, default `~/.dsh/sessions`), while the shipped Web overlay used to give the json backend the relative root `./.storages`: `workspace.json` and `session_projcache.json` landed under `<launch dir>/.storages/` — two launches from different directories shared their sessions yet saw different workspace registries and different projection caches, and the cache exists precisely to serve the cross-session cold listing, which missed for every session last cached under another launch directory. That mismatch is now closed: the overlay anchors `storage-json.root` to `$DSH_HOME/storages` with the same `!!js` expression the session root uses (`apps/cli/config/web.cordis.yml`). The residual hazard: `JsonStorageBackend` still never resolves its root — each unit open joins the path against whatever `process.cwd()` is at that moment (packages/storage/storage-json/src/index.ts); the shipped overlay root is already absolute and unaffected, but any relative root (bare Loader boots, tests) still splits on a later cwd change — the exact hazard the JSONL session backend resolves-once to prevent ("later process.cwd() changes cannot split one backend across roots", packages/session/session-persistence-jsonl/src/index.ts).
|
||||
|
||||
**Recovery behavior.** Inside a healthy medium the cache is fully self-healing by design: a `stateVersion`-mismatched row is discarded and refolded, a log shrunk below a row's watermark is detected by the anchored restore floor and answered with one full re-read, and every background write is fail-soft. But at the *medium* level there is no recovery at all: a truncated, hand-edited, or version-bumped `session_projcache.json` fails `openJsonUnit` with `malformed-medium`/`version-mismatch` (packages/storage/storage-json/src/format.ts), a schema-drifted record fails domain open with `invalid-record` (packages/storage/storage-domain/src/index.ts), the rejection propagates through `SessionProjectionCache[Service.init]`, and under the CLI's fail-loud boot the assembly refuses to start. A file whose entire content is rebuildable from session logs can brick boot. This contradicts the cache package's own stated stance ("a stale or unreadable cache costs a longer tail replay, never a wrong value") and the cache domain spec's JSDoc ("version bumps discard the whole medium"), which describes an aspiration, not the implementation. Partially superseded for the projection cache: [the per-session cache files note](../../implemented/architecture/2026-08-19-projection-cache-per-session-files.md) removed the global `session_projcache` domain, so the cache half of this proposal (recovery on that domain) no longer applies; the `workspace.json` half remains current. The same fail-loud path is *correct* for `workspace.json` — workspace records are authoritative, not derivable — so the missing concept is a per-domain declaration of authority, not a global behavior change.
|
||||
**Recovery behavior.** Inside a healthy medium the cache is fully self-healing by design: a `stateVersion`-mismatched row is discarded and refolded, a log shrunk below a row's watermark is detected by the anchored restore floor and answered with one full re-read, and every background write is fail-soft. But at the *medium* level there is no recovery at all: a truncated, hand-edited, or version-bumped `session_projcache.json` fails `openJsonUnit` with `malformed-medium`/`version-mismatch` (packages/storage/storage-json/src/format.ts), a schema-drifted record fails domain open with `invalid-record` (packages/storage/storage-domain/src/index.ts), the rejection propagates through `SessionProjectionCache[Service.init]`, and under the CLI's fail-loud boot the assembly refuses to start. A file whose entire content is rebuildable from session logs can brick boot. This contradicts the cache package's own stated stance ("a stale or unreadable cache costs a longer tail replay, never a wrong value") and the cache domain spec's JSDoc ("version bumps discard the whole medium"), which describes an aspiration, not the implementation. Partially superseded for the projection cache: [the per-session cache files note](../../implemented/architecture/2026-08-19-projection-cache-per-session-files.md) removed the global `session_projcache` domain, so the cache half of this proposal (recovery on that domain) no longer applies; the `workspace.json` half remains current. The `invalid-record` class for the per-record projection cache is now also superseded: the shipped domain declares `invalidRecords: 'backup-and-skip'` ([cross-version read-compat note](../../implemented/architecture/2026-09-02-projcache-cross-version-read-compat.md)), which backs the failing record up and skips it at open, so the reset/destroy proposal below stays relevant only for whole-medium damage on authoritative or single-document media. The same fail-loud path is *correct* for `workspace.json` — workspace records are authoritative, not derivable — so the missing concept is a per-domain declaration of authority, not a global behavior change.
|
||||
|
||||
## Proposal
|
||||
|
||||
|
||||
+1
-1
@@ -10,7 +10,7 @@ Status: proposed
|
||||
|
||||
**文件到底存在哪(根错位已收口,resolve-once 残余仍开放)。** 共享 base 将会话存储默认为全局 harness home(`$DSH_HOME/sessions`,默认 `~/.dsh/sessions`),而出厂 Web overlay 曾给 json 后端相对根 `./.storages`:`workspace.json` 和 `session_projcache.json` 落在 `<启动目录>/.storages/` 下——从两个不同目录启动,会话相同,工作区注册表和投影缓存却各是一份,而缓存存在的意义恰恰是跨会话冷列表,凡上次在别的启动目录下缓存过的会话全部 miss。这一错位已消除:overlay 现以与会话根同一段 `!!js` 表达式把 `storage-json.root` 锚定到 `$DSH_HOME/storages`(`apps/cli/config/web.cordis.yml`)。残余隐患:`JsonStorageBackend` 仍从不 resolve 根——每次打开 unit 都把路径 join 到当时的 `process.cwd()` 上(packages/storage/storage-json/src/index.ts);出厂 overlay 的根已是绝对路径不受影响,但任何相对根(裸 Loader 启动、测试)仍会被后续 cwd 变化劈开,JSONL 会话后端用「构造时 resolve 一次」防住的正是它("later process.cwd() changes cannot split one backend across roots",packages/session/session-persistence-jsonl/src/index.ts)。
|
||||
|
||||
**恢复行为。** 在健康介质内部,缓存按设计完全自愈:`stateVersion` 不匹配的行被丢弃重折,日志缩短到行水位以下由带锚的 restore floor 检出并以一次全量重读回答,每次后台写都是 fail-soft。但在*介质*层面完全没有恢复:被截断、被手改或版本被 bump 的 `session_projcache.json` 会让 `openJsonUnit` 以 `malformed-medium`/`version-mismatch` 失败(packages/storage/storage-json/src/format.ts),schema 漂移的记录让域 open 以 `invalid-record` 失败(packages/storage/storage-domain/src/index.ts),拒绝一路穿过 `SessionProjectionCache[Service.init]`,在 CLI 的 fail-loud 启动下整个组装拒绝启动。一个内容完全可从会话日志重建的文件能把启动搞死。这与缓存包自己声明的立场("a stale or unreadable cache costs a longer tail replay, never a wrong value")和缓存域 spec 的 JSDoc("version bumps discard the whole medium")相矛盾——后者描述的是愿望而非实现。投影缓存半边已被[每会话缓存文件 note](../../implemented/architecture/2026-08-19-projection-cache-per-session-files.zh.md) 部分取代:全局 `session_projcache` domain 已移除,本提案的缓存恢复半边不再适用;`workspace.json` 半边仍然有效。同一条 fail-loud 路径对 `workspace.json` 却是*正确*的——工作区记录是权威数据,不可派生——所以缺的概念是按域声明权威性,而不是全局改行为。
|
||||
**恢复行为。** 在健康介质内部,缓存按设计完全自愈:`stateVersion` 不匹配的行被丢弃重折,日志缩短到行水位以下由带锚的 restore floor 检出并以一次全量重读回答,每次后台写都是 fail-soft。但在*介质*层面完全没有恢复:被截断、被手改或版本被 bump 的 `session_projcache.json` 会让 `openJsonUnit` 以 `malformed-medium`/`version-mismatch` 失败(packages/storage/storage-json/src/format.ts),schema 漂移的记录让域 open 以 `invalid-record` 失败(packages/storage/storage-domain/src/index.ts),拒绝一路穿过 `SessionProjectionCache[Service.init]`,在 CLI 的 fail-loud 启动下整个组装拒绝启动。一个内容完全可从会话日志重建的文件能把启动搞死。这与缓存包自己声明的立场("a stale or unreadable cache costs a longer tail replay, never a wrong value")和缓存域 spec 的 JSDoc("version bumps discard the whole medium")相矛盾——后者描述的是愿望而非实现。投影缓存半边已被[每会话缓存文件 note](../../implemented/architecture/2026-08-19-projection-cache-per-session-files.zh.md) 部分取代:全局 `session_projcache` domain 已移除,本提案的缓存恢复半边不再适用;`workspace.json` 半边仍然有效。per-record 投影缓存的 `invalid-record` 一类如今也已被取代:已发布的域声明了 `invalidRecords: 'backup-and-skip'`([跨版本读兼容 note](../../implemented/architecture/2026-09-02-projcache-cross-version-read-compat.zh.md)),open 时把失败记录备份后跳过,因此下文 reset/destroy 提案仅对权威介质或单文档介质的整介质损坏仍然相关。同一条 fail-loud 路径对 `workspace.json` 却是*正确*的——工作区记录是权威数据,不可派生——所以缺的概念是按域声明权威性,而不是全局改行为。
|
||||
|
||||
## 提案
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
"projectTitle": "DSH Issue Management",
|
||||
"lifecycleActor": "dsh-issue-management",
|
||||
"priorityField": "Priority",
|
||||
"startDateField": "Start date",
|
||||
"startDateField": "Start Date",
|
||||
"projectTimeZone": "Asia/Shanghai",
|
||||
"allowUnassignedOwner": true,
|
||||
"statuses": [
|
||||
|
||||
@@ -484,13 +484,7 @@ async function projectContext(number, includeStatusActor = false, includeStartDa
|
||||
title
|
||||
fields(first: 50) {
|
||||
nodes {
|
||||
... on ProjectV2Field {
|
||||
id
|
||||
name
|
||||
dataType
|
||||
isIssueField
|
||||
issueField { ... on IssueFieldDate { id } }
|
||||
}
|
||||
... on ProjectV2Field { id name dataType isIssueField }
|
||||
... on ProjectV2SingleSelectField { id name dataType options { id name } }
|
||||
}
|
||||
}
|
||||
@@ -518,9 +512,7 @@ async function projectContext(number, includeStatusActor = false, includeStartDa
|
||||
}
|
||||
startDateValue: fieldValueByName(name: $startDateField)
|
||||
@include(if: $includeStartDate) {
|
||||
... on ProjectV2ItemIssueFieldValue {
|
||||
issueFieldValue { ... on IssueFieldDateValue { value } }
|
||||
}
|
||||
... on ProjectV2ItemFieldDateValue { date }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -552,8 +544,8 @@ async function projectContext(number, includeStatusActor = false, includeStartDa
|
||||
if (startDateField && startDateField.dataType !== 'DATE') {
|
||||
throw new Error(`Project ${config.startDateField} 字段必须为 Date`)
|
||||
}
|
||||
if (startDateField && (!startDateField.isIssueField || !startDateField.issueField?.id)) {
|
||||
throw new Error(`Project ${config.startDateField} 字段必须为 Issue Date 字段`)
|
||||
if (startDateField?.isIssueField) {
|
||||
throw new Error(`Project ${config.startDateField} 字段必须为 Project Date 字段`)
|
||||
}
|
||||
const item = issue.projectItems.nodes.find((candidate) => candidate.project.id === project.id)
|
||||
const latestStatusEvent = issue.timelineItems?.nodes
|
||||
@@ -593,24 +585,27 @@ async function ensureProjectItem(number, includeStartDate = false) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize one Issue's organization Start date when it is empty.
|
||||
* Initialize one Issue's Project Start Date when it is empty.
|
||||
* @param {number} number Same-repository Issue number.
|
||||
* @param {string} date Date in YYYY-MM-DD form.
|
||||
* @returns {Promise<void>} Resolves after the conditional Issue-field update.
|
||||
* @returns {Promise<void>} Resolves after the conditional Project update.
|
||||
*/
|
||||
export async function initializeIssueStartDate(number, date) {
|
||||
const context = await ensureProjectItem(number, true)
|
||||
if (context.item.startDateValue?.issueFieldValue?.value) return
|
||||
if (context.item.startDateValue?.date) return
|
||||
await graphql(
|
||||
`mutation($issueId: ID!, $fieldId: ID!, $date: String!) {
|
||||
updateIssueFieldValue(input: {
|
||||
issueId: $issueId,
|
||||
issueField: {fieldId: $fieldId, dateValue: $date}
|
||||
}) { issue { id } }
|
||||
`mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $date: Date!) {
|
||||
updateProjectV2ItemFieldValue(input: {
|
||||
projectId: $projectId,
|
||||
itemId: $itemId,
|
||||
fieldId: $fieldId,
|
||||
value: {date: $date}
|
||||
}) { projectV2Item { id } }
|
||||
}`,
|
||||
{
|
||||
issueId: context.issue.id,
|
||||
fieldId: context.startDateField.issueField.id,
|
||||
projectId: context.project.id,
|
||||
itemId: context.item.id,
|
||||
fieldId: context.startDateField.id,
|
||||
date,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -21,8 +21,7 @@ const projectGraphqlData = ({
|
||||
startDate = null,
|
||||
startDateField = true,
|
||||
startDateType = 'DATE',
|
||||
startDateIsIssueField = true,
|
||||
startDateIssueField = true,
|
||||
startDateIsIssueField = false,
|
||||
} = {}) => ({
|
||||
organization: {
|
||||
projectV2: {
|
||||
@@ -34,11 +33,10 @@ const projectGraphqlData = ({
|
||||
...(startDateField
|
||||
? [
|
||||
{
|
||||
id: 'start-date-project-field-id',
|
||||
name: 'Start date',
|
||||
id: 'start-date-field-id',
|
||||
name: 'Start Date',
|
||||
dataType: startDateType,
|
||||
isIssueField: startDateIsIssueField,
|
||||
issueField: startDateIssueField ? { id: 'start-date-issue-field-id' } : null,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
@@ -56,8 +54,7 @@ const projectGraphqlData = ({
|
||||
id: 'item-id',
|
||||
project: { id: 'project-id' },
|
||||
fieldValueByName: { name: 'Inbox', optionId: 'inbox-option-id' },
|
||||
startDateValue:
|
||||
startDate === null ? null : { issueFieldValue: { value: startDate } },
|
||||
startDateValue: startDate === null ? null : { date: startDate },
|
||||
},
|
||||
]
|
||||
: [],
|
||||
@@ -269,27 +266,29 @@ test('initializes every referenced Issue only for a PR opened event', async () =
|
||||
assert.equal(writes.length, 3)
|
||||
})
|
||||
|
||||
test('writes an empty Issue Start date with the configured field', async (t) => {
|
||||
test('writes an empty Project Start Date with the configured field', async (t) => {
|
||||
const requests = mockGraphql(t, (request) => {
|
||||
if (request.query.includes('query(')) return projectGraphqlData()
|
||||
return { updateIssueFieldValue: { issue: { id: 'issue-id' } } }
|
||||
return { updateProjectV2ItemFieldValue: { projectV2Item: { id: 'item-id' } } }
|
||||
})
|
||||
|
||||
await initializeIssueStartDate(42, '2026-08-28')
|
||||
|
||||
assert.equal(requests.length, 2)
|
||||
assert.match(requests[0].query, /isIssueField/)
|
||||
assert.match(requests[0].query, /ProjectV2ItemIssueFieldValue/)
|
||||
assert.match(requests[1].query, /updateIssueFieldValue/)
|
||||
assert.match(requests[1].query, /issueField: \{fieldId: \$fieldId, dateValue: \$date\}/)
|
||||
assert.doesNotMatch(requests[0].query, /issueField\s*\{/)
|
||||
assert.match(requests[0].query, /ProjectV2ItemFieldDateValue/)
|
||||
assert.match(requests[1].query, /updateProjectV2ItemFieldValue/)
|
||||
assert.match(requests[1].query, /value: \{date: \$date\}/)
|
||||
assert.deepEqual(requests[1].variables, {
|
||||
issueId: 'issue-id',
|
||||
fieldId: 'start-date-issue-field-id',
|
||||
projectId: 'project-id',
|
||||
itemId: 'item-id',
|
||||
fieldId: 'start-date-field-id',
|
||||
date: '2026-08-28',
|
||||
})
|
||||
})
|
||||
|
||||
test('preserves an existing Issue Start date', async (t) => {
|
||||
test('preserves an existing Project Start Date', async (t) => {
|
||||
const requests = mockGraphql(t, () => projectGraphqlData({ startDate: '2026-08-01' }))
|
||||
|
||||
await initializeIssueStartDate(42, '2026-08-28')
|
||||
@@ -297,13 +296,13 @@ test('preserves an existing Issue Start date', async (t) => {
|
||||
assert.equal(requests.length, 1)
|
||||
})
|
||||
|
||||
test('adds a referenced Issue to the Project before setting Start date', async (t) => {
|
||||
test('adds a referenced Issue to the Project before setting Start Date', async (t) => {
|
||||
const requests = mockGraphql(t, (request) => {
|
||||
if (request.query.includes('query(')) return projectGraphqlData({ projectItem: false })
|
||||
if (request.query.includes('addProjectV2ItemById')) {
|
||||
return { addProjectV2ItemById: { item: { id: 'new-item-id' } } }
|
||||
}
|
||||
return { updateIssueFieldValue: { issue: { id: 'issue-id' } } }
|
||||
return { updateProjectV2ItemFieldValue: { projectV2Item: { id: 'new-item-id' } } }
|
||||
})
|
||||
|
||||
await initializeIssueStartDate(42, '2026-08-28')
|
||||
@@ -311,30 +310,26 @@ test('adds a referenced Issue to the Project before setting Start date', async (
|
||||
assert.equal(requests.length, 3)
|
||||
assert.deepEqual(requests[1].variables, { projectId: 'project-id', contentId: 'issue-id' })
|
||||
assert.deepEqual(requests[2].variables, {
|
||||
issueId: 'issue-id',
|
||||
fieldId: 'start-date-issue-field-id',
|
||||
projectId: 'project-id',
|
||||
itemId: 'new-item-id',
|
||||
fieldId: 'start-date-field-id',
|
||||
date: '2026-08-28',
|
||||
})
|
||||
})
|
||||
|
||||
test('rejects a missing, non-Date, or Project-local Start date field', async (t) => {
|
||||
test('rejects a missing, non-Date, or Issue-level Start Date field', async (t) => {
|
||||
let response = projectGraphqlData({ startDateField: false })
|
||||
const requests = mockGraphql(t, () => response)
|
||||
|
||||
await assert.rejects(initializeIssueStartDate(42, '2026-08-28'), /Project 缺少 Start date 字段/)
|
||||
await assert.rejects(initializeIssueStartDate(42, '2026-08-28'), /Project 缺少 Start Date 字段/)
|
||||
response = projectGraphqlData({ startDateType: 'TEXT' })
|
||||
await assert.rejects(initializeIssueStartDate(42, '2026-08-28'), /Start date 字段必须为 Date/)
|
||||
response = projectGraphqlData({ startDateIsIssueField: false })
|
||||
await assert.rejects(initializeIssueStartDate(42, '2026-08-28'), /Start Date 字段必须为 Date/)
|
||||
response = projectGraphqlData({ startDateIsIssueField: true })
|
||||
await assert.rejects(
|
||||
initializeIssueStartDate(42, '2026-08-28'),
|
||||
/Start date 字段必须为 Issue Date 字段/,
|
||||
/Start Date 字段必须为 Project Date 字段/,
|
||||
)
|
||||
response = projectGraphqlData({ startDateIssueField: false })
|
||||
await assert.rejects(
|
||||
initializeIssueStartDate(42, '2026-08-28'),
|
||||
/Start date 字段必须为 Issue Date 字段/,
|
||||
)
|
||||
assert.equal(requests.length, 4)
|
||||
assert.equal(requests.length, 3)
|
||||
})
|
||||
|
||||
test('does not treat pull request references as Issue associations', () => {
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/config-catalog.md
|
||||
config-catalog.md: 6404a091001bf052a4cd33a2a069f2e347197692
|
||||
config-catalog.zh.md: b29eb49b114b8b534dd3beb8debc53493eb23254
|
||||
config-catalog.md: cedb4508fc474b4985be64765655708179f5336e
|
||||
config-catalog.zh.md: 8dc1c3befd39449fc7ddf8c767475d0db7ff52fb
|
||||
@@ -1878,7 +1878,7 @@ export interface Config {
|
||||
}
|
||||
```
|
||||
|
||||
Source: [`packages/session/session-projection-cache/src/index.ts:54`](../packages/session/session-projection-cache/src/index.ts)
|
||||
Source: [`packages/session/session-projection-cache/src/index.ts:61`](../packages/session/session-projection-cache/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-session-query-sqlite"></a>
|
||||
|
||||
|
||||
@@ -1880,7 +1880,7 @@ export interface Config {
|
||||
}
|
||||
```
|
||||
|
||||
来源:[`packages/session/session-projection-cache/src/index.ts:54`](../packages/session/session-projection-cache/src/index.ts)
|
||||
来源:[`packages/session/session-projection-cache/src/index.ts:61`](../packages/session/session-projection-cache/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-session-query-sqlite"></a>
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/subsystems/storage.md
|
||||
storage.md: 1e4141e6ef1c6f8e1c2593e21e788b626d6b1ed7
|
||||
storage.zh.md: f0433c600674741c3de0ce3e99430297839ce124
|
||||
storage.md: 03b0fa1c674064994039d37c4fec906dc45794b3
|
||||
storage.zh.md: ded8783fbfe7f0d7d8a846c7a66d780b48de1445
|
||||
@@ -44,7 +44,7 @@ interface StorageBackend {
|
||||
}
|
||||
```
|
||||
|
||||
A backend owns one medium (a file-tree root, a database file) and exposes optional operation groups; `kv` is the only shipped group. `KvFacet.open(descriptor)` opens one named unit — `KvUnitDescriptor` carries the name, format version, table names, and whether a global singleton slot exists — and returns a `KvUnit` with `loadAll`, `putRecord`, `deleteRecord`, `setGlobal`, and `close`. Unit and table names must match `UNIT_NAME_RE` (safe as a file name and as a SQL identifier segment); record keys are arbitrary strings that never reach file paths. A unit does not serialize concurrent writes — ordering belongs to the caller — but each single call is atomic on the medium and durable once resolved. A medium stamped with a different version rejects `version-mismatch`; one that cannot be parsed as the unit rejects `malformed-medium` (no migration, pre-release stance). [`backend.ts`](../../packages/storage/storage/src/backend.ts) is the normative clause-by-clause contract, and the shared conformance suite in [`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) checks every clause against each backend. The [json backend](../../packages/storage/storage-json/README.md) republishes one whole human-readable file per unit atomically; the [sqlite backend](../../packages/storage/storage-sqlite/README.md) stores one document per row in one database for frequently updated data.
|
||||
A backend owns one medium (a file-tree root, a database file) and exposes optional operation groups; `kv` is the only shipped group. `KvFacet.open(descriptor)` opens one named unit — `KvUnitDescriptor` carries the name, current format version, optional compatible record versions, table names, and whether a global singleton slot exists — and returns a `KvUnit` with `loadAll`, `putRecord`, `deleteRecord`, `setGlobal`, and `close`. Unit and table names must match `UNIT_NAME_RE` (safe as a file name and as a SQL identifier segment); record keys are arbitrary strings that never reach file paths. A unit does not serialize concurrent writes — ordering belongs to the caller — but each single call is atomic on the medium and durable once resolved. A `single` medium stamped with a different version rejects `version-mismatch`; a `per-record` document stamped outside the accepted set reads as absent. A medium that cannot be parsed as the unit rejects `malformed-medium`. [`backend.ts`](../../packages/storage/storage/src/backend.ts) is the normative clause-by-clause contract, and the shared conformance suite in [`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) checks every clause against each backend. The [json backend](../../packages/storage/storage-json/README.md) republishes one whole human-readable file per unit atomically; the [sqlite backend](../../packages/storage/storage-sqlite/README.md) stores one document per row in one database for frequently updated data.
|
||||
|
||||
## Declaring a domain
|
||||
|
||||
@@ -55,16 +55,36 @@ A domain is declared once by its owning package as a spec object — the single
|
||||
interface DomainSpec {
|
||||
/** Domain name; must match `UNIT_NAME_RE` (doubles as the backend unit name). */
|
||||
readonly name: string
|
||||
/** Domain format version; a medium stamped with a different version rejects at open. */
|
||||
/** Current domain format version; reads enforce it according to the selected layout. */
|
||||
readonly version: number
|
||||
/**
|
||||
* Medium layout for the backend unit: `single` (the default) stores the
|
||||
* whole unit as one document; `per-record` stores each record as its own
|
||||
* document, for units whose records are large, sparse, or individually
|
||||
* disposable — the projection cache — and scopes version bumps per record
|
||||
* (a stale record document is discarded, never migrated).
|
||||
* disposable — the projection cache — and scopes version checks per record
|
||||
* (an unaccepted record document is discarded, never migrated).
|
||||
*/
|
||||
readonly layout?: 'single' | 'per-record'
|
||||
/**
|
||||
* Older domain versions whose stored records the current record schemas
|
||||
* also accept (the declaring owner vouches for that, typically by
|
||||
* declaring the fields older records lack as optional). `per-record` backends
|
||||
* read documents stamped with a listed version instead of discarding them,
|
||||
* and accept a legacy whole-unit file so stamped for the one-time
|
||||
* bootstrap; writes always stamp {@link version}.
|
||||
*/
|
||||
readonly compatibleVersions?: readonly number[]
|
||||
/**
|
||||
* What `open` does with a stored table record that fails its zod schema.
|
||||
* Absent (the default), the whole open rejects with `invalid-record` —
|
||||
* right for authoritative data. `'backup-and-skip'` is for domains whose
|
||||
* records are disposable derived data: the backend moves the record's
|
||||
* document aside (`KvUnit.backupRecord`), the failure is logged with
|
||||
* its cause, and the open continues with the record absent. A backend
|
||||
* without `backupRecord` (no per-record document to move) falls back
|
||||
* to the rejecting default. The global slot always rejects.
|
||||
*/
|
||||
readonly invalidRecords?: 'backup-and-skip'
|
||||
/** Optional global singleton slot. */
|
||||
readonly global?: DomainGlobalSpec<unknown>
|
||||
/** Table declarations keyed by table name; each name must match `UNIT_NAME_RE`. */
|
||||
@@ -180,7 +200,10 @@ The mounted domain facility. Opens declared domains over routed backends; one fa
|
||||
* (`facet-unsupported`); open the unit projected from the spec (backend
|
||||
* `version-mismatch`/`malformed-medium` pass through); load and validate
|
||||
* every stored record against the spec's zod schemas (`invalid-record`
|
||||
* with the offending table and key); construct the domain.
|
||||
* with the offending table and key — unless the spec declares
|
||||
* `invalidRecords: 'backup-and-skip'` and the unit can move documents aside, in
|
||||
* which case the failing record is backed up, logged, and skipped);
|
||||
* construct the domain.
|
||||
*
|
||||
* Lifecycle: the CALLER owns the returned handle and closes it via
|
||||
* `Domain.close()` (typically as its own `ctx.effect` disposer) — the
|
||||
|
||||
@@ -44,7 +44,7 @@ interface StorageBackend {
|
||||
}
|
||||
```
|
||||
|
||||
一个后端拥有一个介质(一棵文件树的根目录、一个数据库文件),并提供可选的操作组;`kv` 是唯一已交付的操作组。`KvFacet.open(descriptor)` 打开一个具名 unit——`KvUnitDescriptor` 携带名称、格式版本、表名清单,以及是否存在全局单例 slot——并返回提供 `loadAll`、`putRecord`、`deleteRecord`、`setGlobal` 和 `close` 的 `KvUnit`。unit 名与表名必须匹配 `UNIT_NAME_RE`(既可安全用作文件名,也可安全用作 SQL 标识符片段);记录键是任意字符串,绝不进入文件路径。unit 不对并发写入做串行化——顺序由调用方负责——但每次单独调用在介质上都是原子的,且 resolve 后即已持久。介质上记录的版本与之不同时拒绝 `version-mismatch`;无法按该 unit 解析的介质拒绝 `malformed-medium`(不做迁移:预发布立场)。[`backend.ts`](../../packages/storage/storage/src/backend.ts) 是逐条款的规范性约定,[`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) 中的共享一致性套件会针对每个后端检查每项条款。[json 后端](../../packages/storage/storage-json/README.zh.md)以原子方式为每个 unit 整文件重新发布一份人类可读文件;[sqlite 后端](../../packages/storage/storage-sqlite/README.zh.md)在单个数据库中每行存储一份文档,用于频繁更新的数据。
|
||||
一个后端拥有一个介质(一棵文件树的根目录、一个数据库文件),并提供可选的操作组;`kv` 是唯一已交付的操作组。`KvFacet.open(descriptor)` 打开一个具名 unit——`KvUnitDescriptor` 携带名称、当前格式版本、可选的兼容记录版本、表名清单,以及是否存在全局单例 slot——并返回提供 `loadAll`、`putRecord`、`deleteRecord`、`setGlobal` 和 `close` 的 `KvUnit`。unit 名与表名必须匹配 `UNIT_NAME_RE`(既可安全用作文件名,也可安全用作 SQL 标识符片段);记录键是任意字符串,绝不进入文件路径。unit 不对并发写入做串行化——顺序由调用方负责——但每次单独调用在介质上都是原子的,且 resolve 后即已持久。`single` 介质上记录的版本不同时拒绝 `version-mismatch`;`per-record` 文档的版本在接受集合之外时读作不存在。无法按该 unit 解析的介质拒绝 `malformed-medium`。[`backend.ts`](../../packages/storage/storage/src/backend.ts) 是逐条款的规范性约定,[`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) 中的共享一致性套件会针对每个后端检查每项条款。[json 后端](../../packages/storage/storage-json/README.zh.md)以原子方式为每个 unit 整文件重新发布一份人类可读文件;[sqlite 后端](../../packages/storage/storage-sqlite/README.zh.md)在单个数据库中每行存储一份文档,用于频繁更新的数据。
|
||||
|
||||
## 声明领域
|
||||
|
||||
@@ -55,16 +55,36 @@ interface StorageBackend {
|
||||
interface DomainSpec {
|
||||
/** Domain name; must match `UNIT_NAME_RE` (doubles as the backend unit name). */
|
||||
readonly name: string
|
||||
/** Domain format version; a medium stamped with a different version rejects at open. */
|
||||
/** Current domain format version; reads enforce it according to the selected layout. */
|
||||
readonly version: number
|
||||
/**
|
||||
* Medium layout for the backend unit: `single` (the default) stores the
|
||||
* whole unit as one document; `per-record` stores each record as its own
|
||||
* document, for units whose records are large, sparse, or individually
|
||||
* disposable — the projection cache — and scopes version bumps per record
|
||||
* (a stale record document is discarded, never migrated).
|
||||
* disposable — the projection cache — and scopes version checks per record
|
||||
* (an unaccepted record document is discarded, never migrated).
|
||||
*/
|
||||
readonly layout?: 'single' | 'per-record'
|
||||
/**
|
||||
* Older domain versions whose stored records the current record schemas
|
||||
* also accept (the declaring owner vouches for that, typically by
|
||||
* declaring the fields older records lack as optional). `per-record` backends
|
||||
* read documents stamped with a listed version instead of discarding them,
|
||||
* and accept a legacy whole-unit file so stamped for the one-time
|
||||
* bootstrap; writes always stamp {@link version}.
|
||||
*/
|
||||
readonly compatibleVersions?: readonly number[]
|
||||
/**
|
||||
* What `open` does with a stored table record that fails its zod schema.
|
||||
* Absent (the default), the whole open rejects with `invalid-record` —
|
||||
* right for authoritative data. `'backup-and-skip'` is for domains whose
|
||||
* records are disposable derived data: the backend moves the record's
|
||||
* document aside (`KvUnit.backupRecord`), the failure is logged with
|
||||
* its cause, and the open continues with the record absent. A backend
|
||||
* without `backupRecord` (no per-record document to move) falls back
|
||||
* to the rejecting default. The global slot always rejects.
|
||||
*/
|
||||
readonly invalidRecords?: 'backup-and-skip'
|
||||
/** Optional global singleton slot. */
|
||||
readonly global?: DomainGlobalSpec<unknown>
|
||||
/** Table declarations keyed by table name; each name must match `UNIT_NAME_RE`. */
|
||||
@@ -180,7 +200,10 @@ The mounted domain facility. Opens declared domains over routed backends; one fa
|
||||
* (`facet-unsupported`); open the unit projected from the spec (backend
|
||||
* `version-mismatch`/`malformed-medium` pass through); load and validate
|
||||
* every stored record against the spec's zod schemas (`invalid-record`
|
||||
* with the offending table and key); construct the domain.
|
||||
* with the offending table and key — unless the spec declares
|
||||
* `invalidRecords: 'backup-and-skip'` and the unit can move documents aside, in
|
||||
* which case the failing record is backed up, logged, and skipped);
|
||||
* construct the domain.
|
||||
*
|
||||
* Lifecycle: the CALLER owns the returned handle and closes it via
|
||||
* `Domain.close()` (typically as its own `ctx.effect` disposer) — the
|
||||
|
||||
@@ -2109,7 +2109,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
methods: [
|
||||
{
|
||||
signature: 'async open<S extends DomainSpec>(spec: S): Promise<Domain<S>>',
|
||||
description: 'Open one declared domain. Steps, each failing the whole call: reject a name that is already open (`already-open`); resolve the backend route (`backend-not-found` passes through from the hub); require its `kv` facet (`facet-unsupported`); open the unit projected from the spec (backend `version-mismatch`/`malformed-medium` pass through); load and validate every stored record against the spec\'s zod schemas (`invalid-record` with the offending table and key); construct the domain.\n\nLifecycle: the CALLER owns the returned handle and closes it via `Domain.close()` (typically as its own `ctx.effect` disposer) — the facility does not tie the domain to any consumer fiber. Domains still open when the facility unmounts are closed by the plugin disposer.',
|
||||
description: 'Open one declared domain. Steps, each failing the whole call: reject a name that is already open (`already-open`); resolve the backend route (`backend-not-found` passes through from the hub); require its `kv` facet (`facet-unsupported`); open the unit projected from the spec (backend `version-mismatch`/`malformed-medium` pass through); load and validate every stored record against the spec\'s zod schemas (`invalid-record` with the offending table and key — unless the spec declares `invalidRecords: \'backup-and-skip\'` and the unit can move documents aside, in which case the failing record is backed up, logged, and skipped); construct the domain.\n\nLifecycle: the CALLER owns the returned handle and closes it via `Domain.close()` (typically as its own `ctx.effect` disposer) — the facility does not tie the domain to any consumer fiber. Domains still open when the facility unmounts are closed by the plugin disposer.',
|
||||
parameters: [{ name: 'spec', description: 'The domain declaration, typically from `defineDomain`.' }],
|
||||
returns: 'the opened domain handle, typed by the spec.',
|
||||
},
|
||||
@@ -3910,7 +3910,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'DomainSpec',
|
||||
declaration: 'export interface DomainSpec {\n readonly name: string;\n readonly version: number;\n readonly layout?: \'single\' | \'per-record\';\n readonly global?: DomainGlobalSpec<unknown>;\n readonly tables: Record<string, DomainTableSpec>;\n}',
|
||||
declaration: 'export interface DomainSpec {\n readonly name: string;\n readonly version: number;\n readonly layout?: \'single\' | \'per-record\';\n readonly compatibleVersions?: readonly number[];\n readonly invalidRecords?: \'backup-and-skip\';\n readonly global?: DomainGlobalSpec<unknown>;\n readonly tables: Record<string, DomainTableSpec>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'DomainTableSpec',
|
||||
@@ -4218,11 +4218,11 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'KvUnit',
|
||||
declaration: 'export interface KvUnit {\n loadAll(): Promise<{\n tables: Record<string, Record<string, unknown>>;\n global: unknown;\n }>;\n putRecord(table: string, key: string, value: unknown): Promise<void>;\n deleteRecord(table: string, key: string): Promise<void>;\n setGlobal(value: unknown): Promise<void>;\n close(): Promise<void>;\n}',
|
||||
declaration: 'export interface KvUnit {\n loadAll(): Promise<{\n tables: Record<string, Record<string, unknown>>;\n global: unknown;\n }>;\n putRecord(table: string, key: string, value: unknown): Promise<void>;\n deleteRecord(table: string, key: string): Promise<void>;\n backupRecord?(table: string, key: string): Promise<string>;\n setGlobal(value: unknown): Promise<void>;\n close(): Promise<void>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'KvUnitDescriptor',
|
||||
declaration: 'export interface KvUnitDescriptor {\n readonly name: string;\n readonly version: number;\n readonly tables: readonly string[];\n readonly hasGlobal: boolean;\n readonly layout?: \'single\' | \'per-record\';\n}',
|
||||
declaration: 'export interface KvUnitDescriptor {\n readonly name: string;\n readonly version: number;\n readonly tables: readonly string[];\n readonly hasGlobal: boolean;\n readonly layout?: \'single\' | \'per-record\';\n readonly compatibleVersions?: readonly number[];\n}',
|
||||
},
|
||||
{
|
||||
name: 'LlmAdapter',
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-projection-cache/README.md
|
||||
README.md: 8b519a811834aa245fdafacd4d93a1b951efdcd7
|
||||
README.zh.md: ed53309b320b43cf050ae54cd687b7cc7435bd9a
|
||||
README.md: 4b454e277d2a9341baaae333f6aa5fa02a86f42c
|
||||
README.zh.md: c166592d071669e2075040111fd61c1b747c0264
|
||||
@@ -64,6 +64,8 @@ Three mandatory points always write: session creation persists the seed-derived
|
||||
|
||||
The log leads and the cache follows: a live checkpoint flushes the session's buffered events durably before the cache row lands, so a crash can leave the cache behind the log but never ahead of it. Reads and writes share the storage domain's coherent in-memory state; the per-unit write chain mutates memory only after durability. Each version-stamped record must match the live unit schema and complete lifecycle identity (`formatVersion`, `createdAt`, `cwd`, `isSeeded`, and `inheritedEventCount`), so a row folded from another Session format generation or fork cut cannot seed the caller. The JSON backend stores each record at `<root>/session_projcache/sessions/<id>.json` in an owner-only directory tree.
|
||||
|
||||
Upgrades never cost the boot or expose an unproven fold. Records stamped with a version in the spec's `compatibleVersions` remain structurally readable for a current checkpoint rewrite, but a missing `formatVersion` never matches a current Session and therefore refolds cold. Once the format matches, absent lineage fields decode as the unseeded lineage — exact for unseeded sessions, while a seeded caller fails the identity match and refolds cold. A stored record that still fails schema validation is moved aside as `<id>.json.bak.<stamp>` under the domain's `invalidRecords: 'backup-and-skip'` policy, logged with its cause, and rebuilt by the next checkpoint.
|
||||
|
||||
-----
|
||||
|
||||
<a id="understand-the-implementation"></a>
|
||||
@@ -126,6 +128,7 @@ These limits define where the cache needs operational care. They are current pac
|
||||
- **No eviction or retention surface** — records accumulate per session; pruning stored checkpoints is out-of-band maintenance, same stance as session persistence itself.
|
||||
- **Interval throttle is per-session coarse** — the timer arms at the first dirty event after a clean write; a steady sub-threshold trickle writes once per interval, not a sliding window.
|
||||
- **No cache-side cold refold** — the cache serves and refreshes its rows but never reads the session log (it does not depend on the persistence layer); a consumer that needs a guaranteed cold snapshot refolds from the log itself.
|
||||
- **Every schema or domain-version change must prove its upgrade story** — a change to the stored record schema or the domain version lands in the same PR with an archived fixture of the previously shipped on-disk format under `tests/fixtures/` and test cases in `tests/fixtures.spec.ts` proving the chosen disposition: read-compat recovery (`compatibleVersions`), current-version rewrite, or backup-and-skip salvage. A bump whose old records are simply discarded still proves that the discard neither fails the boot nor poisons the tree.
|
||||
|
||||
<a id="dev-note"></a>
|
||||
### Dev Note
|
||||
|
||||
@@ -64,6 +64,8 @@ kind: "package-reference"
|
||||
|
||||
日志领先,缓存跟随:实时检查点先把会话的缓冲事件持久化,然后才保存缓存记录。因此崩溃可能让缓存落后于日志,但绝不会让缓存领先。读取和写入共享存储域内一致的内存状态;逐单元写入链只在持久化成功后修改内存。每个带版本戳的记录必须匹配实时单元 schema 与完整生命周期身份(`formatVersion`、`createdAt`、`cwd`、`isSeeded` 和 `inheritedEventCount`),因此从另一会话格式代或 fork 切点折叠出的行不能播种调用方。JSON 后端把每条记录存于仅所有者可访问的 `<root>/session_projcache/sessions/<id>.json` 目录树中。
|
||||
|
||||
升级绝不拖垮启动,也不会暴露未经证明的折叠结果。版本戳落在 spec `compatibleVersions` 集合内的记录仍可被结构化读取并等待当前检查点重写,但缺失 `formatVersion` 的记录绝不匹配当前 Session,因此会回落冷折叠。格式匹配后,缺失的 lineage 字段解码为 unseeded lineage——对非 fork 会话精确无误,seeded 调用方则通不过身份比对、回落冷折叠。仍然通不过 schema 校验的存量记录会按域的 `invalidRecords: 'backup-and-skip'` 策略移出为 `<id>.json.bak.<时间戳>`、连同原因写入日志,并由下一次检查点重建。
|
||||
|
||||
-----
|
||||
|
||||
<a id="understand-the-implementation"></a>
|
||||
@@ -126,6 +128,7 @@ kind: "package-reference"
|
||||
- **无淘汰或保留接口**——记录按会话持续累积;清理已存储检查点属于带外维护,与会话持久化采用相同策略。
|
||||
- **间隔节流采用按会话的粗粒度控制**——一次无脏数据的写入完成后,计时器在首个脏事件到达时启动;持续但低于条数阈值的事件流每间隔写入一次,而非滑动窗口。
|
||||
- **缓存侧不做冷重折叠**——缓存只服务并刷新自己的记录,从不读取会话日志,因为它不依赖持久化层;需要保证冷快照的消费方自行从日志重新折叠。
|
||||
- **每次 schema 或域版本变更都必须论证升级路径**——改动存储记录 schema 或域版本时,同一 PR 必须在 `tests/fixtures/` 下归档此前已发布的磁盘格式样本,并在 `tests/fixtures.spec.ts` 中用测试论证所选的处置方式:读兼容恢复(`compatibleVersions`)、当前版本重写,或 backup-and-skip 抢救。即便选择直接丢弃旧记录的 bump,也要证明丢弃既不炸启动、也不污染缓存树。
|
||||
|
||||
<a id="dev-note"></a>
|
||||
### 开发备注
|
||||
|
||||
@@ -35,6 +35,13 @@ import type { KvTable } from '@deepseek-ai/dsh-storage-domain'
|
||||
import { projectionCacheDomainSpec } from './spec.ts'
|
||||
import type { CheckpointIdentity, CheckpointRecord } from './spec.ts'
|
||||
|
||||
/** Complete identity written by the current cache generation. */
|
||||
type CurrentCheckpointIdentity = CheckpointIdentity & {
|
||||
formatVersion: number
|
||||
isSeeded: boolean
|
||||
inheritedEventCount: SessionLogOffset
|
||||
}
|
||||
|
||||
export { checkpointIdentity, checkpointRecord, checkpointRow, projectionCacheDomainSpec } from './spec.ts'
|
||||
export type { CheckpointIdentity, CheckpointRecord } from './spec.ts'
|
||||
|
||||
@@ -112,7 +119,7 @@ export class SessionProjectionCache extends Service {
|
||||
* @param expected - the log identity the caller holds (live or stored header).
|
||||
* @returns the identity-matching record, or `undefined` (absent or unrelated).
|
||||
*/
|
||||
private recordFor(id: SessionId, expected: CheckpointIdentity): CheckpointRecord | undefined {
|
||||
private recordFor(id: SessionId, expected: CurrentCheckpointIdentity): CheckpointRecord | undefined {
|
||||
const record = this.requireTable().get(id)
|
||||
if (record === undefined) return undefined
|
||||
return identityMatches(record.identity, expected) ? record : undefined
|
||||
@@ -350,7 +357,7 @@ export class SessionProjectionCache extends Service {
|
||||
function identityOf(
|
||||
header: SessionHeader,
|
||||
inheritedEventCount: SessionLogOffset,
|
||||
): CheckpointIdentity {
|
||||
): CurrentCheckpointIdentity {
|
||||
const cut = SessionLogOffset(inheritedEventCount)
|
||||
if (!header.isSeeded && cut !== 0) {
|
||||
throw new Error('unseeded projection-cache identity inherited event count must be 0')
|
||||
@@ -364,13 +371,19 @@ function identityOf(
|
||||
}
|
||||
}
|
||||
|
||||
/** Whether a stored record's bound identity names the caller's lifecycle. */
|
||||
function identityMatches(stored: CheckpointIdentity, expected: CheckpointIdentity): boolean {
|
||||
/**
|
||||
* Whether a stored record's bound identity names the caller's lifecycle.
|
||||
* An absent format generation cannot prove the fold semantics and never
|
||||
* matches. Once the format matches, absent lineage fields (records admitted
|
||||
* via `compatibleVersions` predate them) read as the unseeded lineage: exact
|
||||
* for an unseeded caller, while a seeded caller fails the match.
|
||||
*/
|
||||
function identityMatches(stored: CheckpointIdentity, expected: CurrentCheckpointIdentity): boolean {
|
||||
return stored.formatVersion === expected.formatVersion
|
||||
&& stored.createdAt === expected.createdAt
|
||||
&& stored.cwd === expected.cwd
|
||||
&& stored.isSeeded === expected.isSeeded
|
||||
&& stored.inheritedEventCount === expected.inheritedEventCount
|
||||
&& (stored.isSeeded ?? false) === expected.isSeeded
|
||||
&& (stored.inheritedEventCount ?? 0) === expected.inheritedEventCount
|
||||
}
|
||||
|
||||
export default SessionProjectionCache
|
||||
@@ -38,13 +38,20 @@ export const checkpointRow = z.object({
|
||||
* old record pass every watermark check and seed state folded from an
|
||||
* unrelated log. Reads validate this against the live header (listing) or
|
||||
* the stored header (cold read) before accepting any record.
|
||||
*
|
||||
* The format and lineage fields are optional because records admitted through
|
||||
* `compatibleVersions` predate them. The reader (`identityMatches`) refuses an
|
||||
* absent format generation because no current Session log can prove that
|
||||
* record's fold semantics. It interprets absent lineage as unseeded only after
|
||||
* the format generation matches. Current-version writes always store all three
|
||||
* fields.
|
||||
*/
|
||||
export const checkpointIdentity = z.object({
|
||||
formatVersion: z.number().int().nonnegative(),
|
||||
formatVersion: z.number().int().nonnegative().optional(),
|
||||
createdAt: z.number().int().nonnegative(),
|
||||
cwd: z.string().optional(),
|
||||
isSeeded: z.boolean(),
|
||||
inheritedEventCount: z.number().int().nonnegative().transform(SessionLogOffset),
|
||||
isSeeded: z.boolean().optional(),
|
||||
inheritedEventCount: z.number().int().nonnegative().transform(SessionLogOffset).optional(),
|
||||
})
|
||||
|
||||
/** The identity fields a record is bound to, inferred from {@link checkpointIdentity}. */
|
||||
@@ -69,11 +76,26 @@ export type CheckpointRecord = z.infer<typeof checkpointRecord>
|
||||
* bumps per session: after a bump, a stale session document is discarded on
|
||||
* open (cache semantics — a stale or unreadable cache costs a longer tail
|
||||
* replay, never a wrong value) while the rest of the domain stays usable,
|
||||
* instead of rejecting the whole medium.
|
||||
* instead of rejecting the whole medium. The `compatibleVersions` entries
|
||||
* keep structurally valid predecessor records available for a later current
|
||||
* checkpoint rewrite. Records without `formatVersion` remain unusable as fold
|
||||
* shortcuts because they cannot prove which Session event semantics produced
|
||||
* their rows; the per-record version map and disposition live in the read-compat Agent Note
|
||||
* (.agents/notes/implemented/architecture/2026-09-02-projcache-cross-version-read-compat.md).
|
||||
* The per-row `ver` guard and the identity match still discard anything the
|
||||
* current fold semantics cannot vouch for.
|
||||
*
|
||||
* `invalidRecords: 'backup-and-skip'`: a stored record that fails the schema
|
||||
* anyway is disposable derived data, so it must never cost the boot — the
|
||||
* domain layer moves the document aside as `<key>.json.bak.<stamp>`, logs
|
||||
* the concrete validation failure, and serves the session as uncached (a
|
||||
* cold read rebuilds and rewrites it).
|
||||
*/
|
||||
export const projectionCacheDomainSpec = defineDomain({
|
||||
name: 'session_projcache',
|
||||
version: 6,
|
||||
version: 7,
|
||||
compatibleVersions: [3, 4, 5, 6],
|
||||
invalidRecords: 'backup-and-skip',
|
||||
layout: 'per-record',
|
||||
tables: { sessions: domainTable<SessionId, CheckpointRecord>(checkpointRecord) },
|
||||
})
|
||||
@@ -44,10 +44,12 @@ declare module '@deepseek-ai/dsh-session-projection/types' {
|
||||
'cache-test/marks2': Map<string, string>
|
||||
'cache-test/count': number
|
||||
'cache-test/secret': string
|
||||
'cache-test/marks3': MarksState
|
||||
}
|
||||
interface SessionProjectionMap {
|
||||
'cache-test/marks': { marks: string[] }
|
||||
'cache-test/secondary-marks': { marks: string[] }
|
||||
'cache-test/marks3': { marks: string[] }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -74,6 +76,18 @@ const marksUnit = (stateVersion = 1) => ({
|
||||
stateVersion,
|
||||
}) satisfies ProjectionDefinition<'cache-test/marks', MarksState>
|
||||
|
||||
const marks3Unit = {
|
||||
key: 'cache-test/marks3',
|
||||
stateSchema: z.object({ marks: z.array(z.string()) }).nullable(),
|
||||
init: () => null,
|
||||
apply: state => state,
|
||||
wire: {
|
||||
viewSchema: z.object({ marks: z.array(z.string()) }),
|
||||
view: state => state ?? { marks: [] },
|
||||
},
|
||||
stateVersion: 1,
|
||||
} satisfies ProjectionDefinition<'cache-test/marks3', MarksState>
|
||||
|
||||
const secretUnit = {
|
||||
key: 'cache-test/secret',
|
||||
stateSchema: z.string(),
|
||||
@@ -443,14 +457,35 @@ describe('SessionProjectionCache listing read', () => {
|
||||
.toBeUndefined()
|
||||
})
|
||||
|
||||
it('returns undefined when the stored record is version-mismatched', async () => {
|
||||
it('carries ONE cut across multiple served rows: the lowest watermark wins', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-'))
|
||||
roots.push(root)
|
||||
// A stale version-stamped document is discarded at open: absent record.
|
||||
// Equal watermarks: whichever row is visited second cannot lower the cut,
|
||||
// so the one-cut fold sees both a lowering and a non-lowering row in
|
||||
// every iteration order.
|
||||
await seedRecord(root, 'multi-row', {
|
||||
'cache-test/marks': { ver: 1, seq: SessionSeq(4), val: { marks: ['a'] } },
|
||||
'cache-test/marks3': { ver: 1, seq: SessionSeq(4), val: { marks: ['b'] } },
|
||||
})
|
||||
const { ctx, cache } = await harness({ root })
|
||||
ctx.sessionProjections.register(marks3Unit)
|
||||
const block = cache.cachedSnapshot(headerOf(SessionId('multi-row')), SessionLogOffset(0))
|
||||
expect(block?.values).toEqual({
|
||||
'cache-test/marks': { marks: ['a'] },
|
||||
'cache-test/marks3': { marks: ['b'] },
|
||||
})
|
||||
expect(block?.asOfSeq).toBe(4)
|
||||
})
|
||||
|
||||
it('returns undefined when the stored record version is not accepted', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-'))
|
||||
roots.push(root)
|
||||
// Version 2 is neither current nor declared compatible, so the document
|
||||
// is discarded at open and the record reads as absent.
|
||||
const path = recordPath(root, SessionId('all-stale'))
|
||||
await mkdir(dirname(path), { recursive: true })
|
||||
await writeFile(path, JSON.stringify({
|
||||
version: projectionCacheDomainSpec.version - 1,
|
||||
version: 2,
|
||||
record: {
|
||||
identity: {
|
||||
formatVersion: SESSION_FORMAT_VERSION,
|
||||
@@ -466,6 +501,48 @@ describe('SessionProjectionCache listing read', () => {
|
||||
.toBeUndefined()
|
||||
})
|
||||
|
||||
it('serves a pre-lineage record (accepted old version) to an unseeded caller only', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-'))
|
||||
roots.push(root)
|
||||
// A document stamped with an accepted older version whose identity
|
||||
// predates the lineage fields: absent lineage reads as unseeded.
|
||||
const path = recordPath(root, SessionId('pre-lineage'))
|
||||
await mkdir(dirname(path), { recursive: true })
|
||||
await writeFile(path, JSON.stringify({
|
||||
version: 4,
|
||||
record: {
|
||||
identity: { formatVersion: SESSION_FORMAT_VERSION, createdAt: 0 },
|
||||
rows: { 'cache-test/marks': { ver: 1, seq: 4, val: { marks: ['kept'] } } },
|
||||
},
|
||||
}))
|
||||
const { cache } = await harness({ root })
|
||||
const id = SessionId('pre-lineage')
|
||||
// Unseeded caller: the absent lineage is exactly its identity — served.
|
||||
expect(cache.cachedSnapshot(headerOf(id), SessionLogOffset(0)))
|
||||
.toEqual({ asOfSeq: 4, values: { 'cache-test/marks': { marks: ['kept'] } } })
|
||||
// Seeded caller: the lineage-less record cannot vouch for the cut — refused.
|
||||
expect(cache.cachedSnapshot({ ...headerOf(id), isSeeded: true }, SessionLogOffset(2)))
|
||||
.toBeUndefined()
|
||||
})
|
||||
|
||||
it('refuses an accepted predecessor record without a Session format generation', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-'))
|
||||
roots.push(root)
|
||||
const id = SessionId('pre-format-identity')
|
||||
const path = recordPath(root, id)
|
||||
await mkdir(dirname(path), { recursive: true })
|
||||
await writeFile(path, JSON.stringify({
|
||||
version: 6,
|
||||
record: {
|
||||
identity: { createdAt: 0, isSeeded: false, inheritedEventCount: 0 },
|
||||
rows: { 'cache-test/marks': { ver: 1, seq: 4, val: { marks: ['unbound'] } } },
|
||||
},
|
||||
}))
|
||||
const { cache } = await harness({ root })
|
||||
|
||||
expect(cache.cachedSnapshot(headerOf(id), SessionLogOffset(0))).toBeUndefined()
|
||||
})
|
||||
|
||||
it('returns undefined when every stored row is version-mismatched', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-'))
|
||||
roots.push(root)
|
||||
|
||||
@@ -0,0 +1,255 @@
|
||||
/**
|
||||
* Cross-version recovery over archived on-disk artifacts. `fixtures/` holds
|
||||
* real `session_projcache` media, each produced by driving the named release
|
||||
* through its own web app (session created over RPC, real model turns, a
|
||||
* rename): the v3 whole-unit file (published 0.1.1-rc.2), a v4 per-record
|
||||
* document (published 0.1.2-alpha.3), a published v5 document, and the
|
||||
* v5-stamped lineage-less document reproducing byte-for-byte what the
|
||||
* formerly unguarded legacy bootstrap wrote over v3 records. Each must open
|
||||
* through the real storage stack without becoming a fold shortcut for the
|
||||
* current Session format, then accept a current checkpoint rewrite. A record
|
||||
* that fails schema validation is backed up and skipped instead of failing the
|
||||
* boot.
|
||||
*/
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { cp, mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { dirname, join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import { z } from 'zod'
|
||||
import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection'
|
||||
import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection'
|
||||
import Storage from '@deepseek-ai/dsh-storage'
|
||||
import {
|
||||
apply as storageJsonApply, Config as storageJsonConfig, inject as storageJsonInject, name as storageJsonName,
|
||||
} from '@deepseek-ai/dsh-storage-json'
|
||||
import {
|
||||
apply as storageDomainApply, Config as storageDomainConfig, inject as storageDomainInject, name as storageDomainName,
|
||||
} from '@deepseek-ai/dsh-storage-domain'
|
||||
import SessionProjectionCache from '../src/index.ts'
|
||||
import { projectionCacheDomainSpec } from '../src/spec.ts'
|
||||
|
||||
// Declarations must match the shipped title unit's exactly (the repo-wide
|
||||
// compile face sees both).
|
||||
declare module '@deepseek-ai/dsh-session-projection/types' {
|
||||
interface SessionProjectionStateMap {
|
||||
title: string | null
|
||||
}
|
||||
interface SessionProjectionMap {
|
||||
title: string | null
|
||||
}
|
||||
}
|
||||
|
||||
declare module '@deepseek-ai/dsh-session/types' {
|
||||
interface SessionEventMap {
|
||||
'fixtures-test/set-title': { title: string }
|
||||
}
|
||||
|
||||
interface OutOfBandSessionEventMap {
|
||||
'fixtures-test/set-title': true
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors the shipped title unit's storage face: stateVersion 1, bare-string
|
||||
// state (the fixture rows carry exactly this shape in every archived
|
||||
// version), folding a test event so the rewrite path has fresh data.
|
||||
const titleUnit = {
|
||||
key: 'title',
|
||||
stateSchema: z.string().nullable(),
|
||||
init: () => null,
|
||||
apply: (state, event) => (event.type === 'fixtures-test/set-title' ? event.data.title : state),
|
||||
wire: { viewSchema: z.string().nullable(), view: state => state },
|
||||
stateVersion: 1,
|
||||
} satisfies ProjectionDefinition<'title', string | null>
|
||||
|
||||
const FIXTURES = fileURLToPath(new URL('./fixtures/', import.meta.url))
|
||||
|
||||
/** One archived per-record document (`{version, record}`). */
|
||||
interface FixtureDoc {
|
||||
version: number
|
||||
record: {
|
||||
identity: { createdAt: number; cwd?: string }
|
||||
rows: Record<string, { ver: number; seq: number; val: unknown }>
|
||||
}
|
||||
}
|
||||
|
||||
async function fixtureJson<T>(name: string): Promise<T> {
|
||||
return JSON.parse(await readFile(join(FIXTURES, name), 'utf8')) as T
|
||||
}
|
||||
|
||||
/** Header for the session a fixture record is bound to (identity witness). */
|
||||
function headerFor(id: SessionId, identity: FixtureDoc['record']['identity']): SessionHeader {
|
||||
return {
|
||||
version: SESSION_FORMAT_VERSION,
|
||||
id,
|
||||
createdAt: identity.createdAt,
|
||||
isSeeded: false,
|
||||
...identity.cwd === undefined ? {} : { cwd: identity.cwd },
|
||||
}
|
||||
}
|
||||
|
||||
const contexts: Context[] = []
|
||||
const roots: string[] = []
|
||||
|
||||
async function harness(root: string) {
|
||||
roots.push(root)
|
||||
const ctx = new Context()
|
||||
contexts.push(ctx)
|
||||
await ctx.plugin(Storage)
|
||||
await ctx.plugin({ name: storageJsonName, inject: storageJsonInject, apply: storageJsonApply, Config: storageJsonConfig }, { root })
|
||||
await ctx.plugin({ name: storageDomainName, inject: storageDomainInject, apply: storageDomainApply, Config: storageDomainConfig }, { backend: 'json' })
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionProjectionRegistry)
|
||||
ctx.sessionProjections.register(titleUnit)
|
||||
await ctx.plugin(SessionProjectionCache, { writeEveryEvents: 100, writeIntervalMs: 60_000 })
|
||||
return { ctx, cache: ctx.sessionProjectionCache }
|
||||
}
|
||||
|
||||
/** Lay one per-record fixture document into a fresh backend root. */
|
||||
async function placeDoc(root: string, id: string, name: string): Promise<FixtureDoc> {
|
||||
const path = join(root, projectionCacheDomainSpec.name, 'sessions', `${id}.json`)
|
||||
await mkdir(dirname(path), { recursive: true })
|
||||
await cp(join(FIXTURES, name), path)
|
||||
return fixtureJson<FixtureDoc>(name)
|
||||
}
|
||||
|
||||
/**
|
||||
* Drive a live write over a recovered session id and assert the archived
|
||||
* document is replaced by a current-version one: current domain and Session
|
||||
* format stamps, lineage, and the freshly folded title.
|
||||
*/
|
||||
async function assertRewrite(ctx: Context, root: string, id: SessionId): Promise<void> {
|
||||
const session = ctx.sessions.create(id)
|
||||
session.append('fixtures-test/set-title', { title: '重写标题' })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
const path = join(root, projectionCacheDomainSpec.name, 'sessions', `${id}.json`)
|
||||
await vi.waitFor(async () => {
|
||||
const doc = JSON.parse(await readFile(path, 'utf8')) as FixtureDoc
|
||||
expect(doc.version).toBe(projectionCacheDomainSpec.version)
|
||||
expect(doc.record.identity).toMatchObject({
|
||||
formatVersion: SESSION_FORMAT_VERSION,
|
||||
isSeeded: false,
|
||||
inheritedEventCount: 0,
|
||||
})
|
||||
expect(doc.record.rows['title']?.val).toBe('重写标题')
|
||||
}, { timeout: 5_000 })
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose()))
|
||||
await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 })))
|
||||
})
|
||||
|
||||
describe('archived version recovery', () => {
|
||||
it('recovers the v3 whole-unit archive through the legacy bootstrap', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-fx-'))
|
||||
await cp(join(FIXTURES, 'v3-single-unit.json'), join(root, `${projectionCacheDomainSpec.name}.json`))
|
||||
type SingleUnit = {
|
||||
unit: { version: number }
|
||||
tables: { sessions: Record<string, FixtureDoc['record']> }
|
||||
}
|
||||
const archive = await fixtureJson<SingleUnit>('v3-single-unit.json')
|
||||
expect(archive.unit.version).toBe(3) // the fixture IS the old format
|
||||
const [sid, record] = Object.entries(archive.tables.sessions)[0]!
|
||||
|
||||
const { ctx, cache } = await harness(root)
|
||||
expect(cache.cachedSnapshot(
|
||||
headerFor(SessionId(sid), record.identity),
|
||||
SessionLogOffset(0),
|
||||
['title'],
|
||||
)).toBeUndefined()
|
||||
|
||||
// The one-time bootstrap materialized a current-version document.
|
||||
const migrated = JSON.parse(
|
||||
await readFile(join(root, projectionCacheDomainSpec.name, 'sessions', `${sid}.json`), 'utf8'),
|
||||
) as { version: number }
|
||||
expect(migrated.version).toBe(projectionCacheDomainSpec.version)
|
||||
|
||||
await assertRewrite(ctx, root, SessionId(sid))
|
||||
})
|
||||
|
||||
for (const [fixture, storedVersion] of [
|
||||
['v4-session-doc.json', 4],
|
||||
['v5-session-doc.json', 5],
|
||||
['v5-lineageless-doc.json', 5],
|
||||
] as const) {
|
||||
it(`opens ${fixture} without serving its unbound fold, then rewrites it current`, async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-fx-'))
|
||||
const id = SessionId('fixture-session')
|
||||
const doc = await placeDoc(root, id, fixture)
|
||||
expect(doc.version).toBe(storedVersion)
|
||||
|
||||
const { ctx, cache } = await harness(root)
|
||||
expect(cache.cachedSnapshot(
|
||||
headerFor(id, doc.record.identity),
|
||||
SessionLogOffset(0),
|
||||
['title'],
|
||||
)).toBeUndefined()
|
||||
|
||||
await assertRewrite(ctx, root, id)
|
||||
})
|
||||
}
|
||||
|
||||
it('refuses a lineage-less archive for a seeded caller (identity mismatch, cold rebuild)', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-fx-'))
|
||||
const id = SessionId('fixture-seeded')
|
||||
const doc = await placeDoc(root, id, 'v5-lineageless-doc.json')
|
||||
|
||||
const { cache } = await harness(root)
|
||||
const seeded = { ...headerFor(id, doc.record.identity), isSeeded: true }
|
||||
expect(cache.cachedSnapshot(seeded, SessionLogOffset(2), ['title'])).toBeUndefined()
|
||||
})
|
||||
|
||||
it('backs up and skips a record that fails schema validation instead of failing the boot', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-fx-'))
|
||||
roots.push(root)
|
||||
const sessionsDir = join(root, projectionCacheDomainSpec.name, 'sessions')
|
||||
await mkdir(sessionsDir, { recursive: true })
|
||||
// Current-version stamp, hopeless record content: no compat rung can save it.
|
||||
await writeFile(join(sessionsDir, 'broken.json'), JSON.stringify({
|
||||
version: projectionCacheDomainSpec.version,
|
||||
record: { identity: { createdAt: 'not-a-number' }, rows: 'not-an-object' },
|
||||
}))
|
||||
const good = await placeDoc(root, SessionId('survivor'), 'v5-session-doc.json')
|
||||
|
||||
const ctx = new Context()
|
||||
contexts.push(ctx)
|
||||
await ctx.plugin(Storage)
|
||||
await ctx.plugin({ name: storageJsonName, inject: storageJsonInject, apply: storageJsonApply, Config: storageJsonConfig }, { root })
|
||||
await ctx.plugin({ name: storageDomainName, inject: storageDomainInject, apply: storageDomainApply, Config: storageDomainConfig }, { backend: 'json' })
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionProjectionRegistry)
|
||||
ctx.sessionProjections.register(titleUnit)
|
||||
const error = vi.spyOn(ctx.logger, 'error').mockImplementation(() => {})
|
||||
// The boot survives the broken record — this line rejecting IS the fixed bug.
|
||||
await ctx.plugin(SessionProjectionCache, { writeEveryEvents: 100, writeIntervalMs: 60_000 })
|
||||
|
||||
// Concrete console diagnostics: which record, where it went, and why.
|
||||
expect(error).toHaveBeenCalledWith(expect.stringContaining("record 'broken'"))
|
||||
expect(error).toHaveBeenCalledWith(expect.stringContaining('.json.bak.'))
|
||||
|
||||
// The document was moved aside as <key>.json.bak.<YYYYMMDDHHmm>, bytes intact.
|
||||
const entries = await readdir(sessionsDir)
|
||||
expect(entries).not.toContain('broken.json')
|
||||
const backup = entries.find(name => /^broken\.json\.bak\.\d{12}$/.test(name))
|
||||
expect(backup).toBeDefined()
|
||||
expect(JSON.parse(await readFile(join(sessionsDir, backup!), 'utf8')))
|
||||
.toMatchObject({ record: { rows: 'not-an-object' } })
|
||||
|
||||
// The broken record reads as absent; its predecessor-stamped neighbor
|
||||
// remains available for a safe current rewrite.
|
||||
const cache = ctx.sessionProjectionCache
|
||||
expect(cache.cachedSnapshot(headerFor(SessionId('broken'), { createdAt: 0 }), SessionLogOffset(0)))
|
||||
.toBeUndefined()
|
||||
expect(cache.cachedSnapshot(
|
||||
headerFor(SessionId('survivor'), good.record.identity),
|
||||
SessionLogOffset(0),
|
||||
['title'],
|
||||
)).toBeUndefined()
|
||||
await assertRewrite(ctx, root, SessionId('survivor'))
|
||||
})
|
||||
})
|
||||
+136
@@ -0,0 +1,136 @@
|
||||
{
|
||||
"unit": {
|
||||
"name": "session_projcache",
|
||||
"version": 3
|
||||
},
|
||||
"global": null,
|
||||
"tables": {
|
||||
"sessions": {
|
||||
"session-1374fa81-15da-44ca-be12-b4c6fe8076a3": {
|
||||
"identity": {
|
||||
"createdAt": 1788286864454,
|
||||
"cwd": "/tmp"
|
||||
},
|
||||
"rows": {
|
||||
"sessionStats": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"turns": 2,
|
||||
"steps": 2,
|
||||
"llmMs": 2169,
|
||||
"toolMs": 0,
|
||||
"ttftMs": 1872,
|
||||
"ttftSteps": 2,
|
||||
"decodeMs": 297,
|
||||
"decodeTokens": 20,
|
||||
"lastTurn": 2,
|
||||
"openStep": null,
|
||||
"pendingCalls": {}
|
||||
}
|
||||
},
|
||||
"title": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": "验收标题-rc2"
|
||||
},
|
||||
"goal": {
|
||||
"ver": 4,
|
||||
"seq": 54,
|
||||
"val": null
|
||||
},
|
||||
"tokenUsage": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"totals": {
|
||||
"uncachedInputTokens": 6083,
|
||||
"outputTokens": 20,
|
||||
"cacheReadTokens": 9728,
|
||||
"cacheWriteTokens": 0
|
||||
},
|
||||
"last": {
|
||||
"turn": 2,
|
||||
"step": 1,
|
||||
"buckets": {
|
||||
"uncachedInputTokens": 112,
|
||||
"outputTokens": 4,
|
||||
"cacheReadTokens": 7808,
|
||||
"cacheWriteTokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"contextPressure": {
|
||||
"ver": 4,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"surfaceTokens": 178,
|
||||
"contextWindow": 1000000,
|
||||
"pressureTokens": 7920,
|
||||
"sampledSurfaceTokens": 168
|
||||
}
|
||||
},
|
||||
"contextBreakdown": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"systemTokens": 1620,
|
||||
"toolsTokens": 6475,
|
||||
"messageTokens": 178
|
||||
}
|
||||
},
|
||||
"subagentTiming": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"descriptorSeen": false,
|
||||
"settledMs": 0
|
||||
}
|
||||
},
|
||||
"subagent": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": {}
|
||||
},
|
||||
"permissions": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"preset": "workspace-write",
|
||||
"sandbox": "workspace-write",
|
||||
"approval": "ask"
|
||||
}
|
||||
},
|
||||
"sessionListMetadata": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"blank": false,
|
||||
"lastPromptAt": 1788286867410
|
||||
}
|
||||
},
|
||||
"imageLimits": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": null
|
||||
},
|
||||
"todos": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": null
|
||||
},
|
||||
"plan": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"active": false,
|
||||
"wanted": null,
|
||||
"running": null
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+212
@@ -0,0 +1,212 @@
|
||||
{
|
||||
"version": 4,
|
||||
"record": {
|
||||
"identity": {
|
||||
"createdAt": 1788286912530,
|
||||
"cwd": "/tmp"
|
||||
},
|
||||
"rows": {
|
||||
"title": {
|
||||
"ver": 1,
|
||||
"seq": 68,
|
||||
"val": "验收标题-alpha3"
|
||||
},
|
||||
"titleInput": {
|
||||
"ver": 3,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"first": {
|
||||
"seq": 7,
|
||||
"text": "请只回复一个词:pong"
|
||||
},
|
||||
"count": 2,
|
||||
"lastSeq": 58
|
||||
}
|
||||
},
|
||||
"llmRetry": {
|
||||
"ver": 1,
|
||||
"seq": 68,
|
||||
"val": {}
|
||||
},
|
||||
"sandboxMode": {
|
||||
"ver": 1,
|
||||
"seq": 68,
|
||||
"val": "workspace-write"
|
||||
},
|
||||
"goal": {
|
||||
"ver": 6,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"current": null,
|
||||
"seenGoalIds": [],
|
||||
"failure": null
|
||||
}
|
||||
},
|
||||
"tokenUsage": {
|
||||
"ver": 2,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"totals": {
|
||||
"uncachedInputTokens": 8192,
|
||||
"outputTokens": 35,
|
||||
"cacheReadTokens": 8064,
|
||||
"cacheWriteTokens": 0
|
||||
},
|
||||
"last": {
|
||||
"turn": 2,
|
||||
"step": 1,
|
||||
"buckets": {
|
||||
"uncachedInputTokens": 86,
|
||||
"outputTokens": 4,
|
||||
"cacheReadTokens": 8064,
|
||||
"cacheWriteTokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"contextPressure": {
|
||||
"ver": 4,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"surfaceTokens": 193,
|
||||
"contextWindow": 1000000,
|
||||
"pressureTokens": 8150,
|
||||
"sampledSurfaceTokens": 183
|
||||
}
|
||||
},
|
||||
"contextBreakdown": {
|
||||
"ver": 2,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"systemTokens": 1760,
|
||||
"toolsTokens": 6579,
|
||||
"messageTokens": 193
|
||||
}
|
||||
},
|
||||
"turnBoundary": {
|
||||
"ver": 2,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"openTurnStartSeq": null,
|
||||
"lastStepStartSeq": 57,
|
||||
"lastStepBoundary": {
|
||||
"kind": "end",
|
||||
"seq": 67
|
||||
},
|
||||
"lastTurn": 2
|
||||
}
|
||||
},
|
||||
"sessionStats": {
|
||||
"ver": 1,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"turns": 2,
|
||||
"steps": 2,
|
||||
"llmMs": 3486,
|
||||
"toolMs": 0,
|
||||
"ttftMs": 3199,
|
||||
"ttftSteps": 2,
|
||||
"decodeMs": 287,
|
||||
"decodeTokens": 35,
|
||||
"lastTurn": 2,
|
||||
"openStep": null,
|
||||
"pendingCalls": {}
|
||||
}
|
||||
},
|
||||
"turnOutline": {
|
||||
"ver": 2,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"turns": [
|
||||
{
|
||||
"turn": 1,
|
||||
"seq": 4,
|
||||
"prompt": "请只回复一个词:pong",
|
||||
"response": "pong"
|
||||
},
|
||||
{
|
||||
"turn": 2,
|
||||
"seq": 55,
|
||||
"prompt": "请只回复一个词:pong2",
|
||||
"response": "pong2"
|
||||
}
|
||||
],
|
||||
"draft": ""
|
||||
}
|
||||
},
|
||||
"agentPreset": {
|
||||
"ver": 1,
|
||||
"seq": 68,
|
||||
"val": "standard"
|
||||
},
|
||||
"subagentTiming": {
|
||||
"ver": 2,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"descriptorSeen": false,
|
||||
"settledMs": 0
|
||||
}
|
||||
},
|
||||
"subagent": {
|
||||
"ver": 2,
|
||||
"seq": 68,
|
||||
"val": {}
|
||||
},
|
||||
"permissions": {
|
||||
"ver": 2,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"preset": "workspace-write",
|
||||
"sandbox": "workspace-write",
|
||||
"approval": "ask",
|
||||
"seeded": false
|
||||
}
|
||||
},
|
||||
"modelSelection": {
|
||||
"ver": 2,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"lastUsed": {
|
||||
"provider": "deepseek-official",
|
||||
"model": "deepseek-v4-flash",
|
||||
"reasoningEffort": "high"
|
||||
},
|
||||
"pending": null
|
||||
}
|
||||
},
|
||||
"sessionListMetadata": {
|
||||
"ver": 1,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"blank": false,
|
||||
"lastPromptAt": 1788286917834
|
||||
}
|
||||
},
|
||||
"imageLimits": {
|
||||
"ver": 1,
|
||||
"seq": 68,
|
||||
"val": null
|
||||
},
|
||||
"todos": {
|
||||
"ver": 2,
|
||||
"seq": 68,
|
||||
"val": null
|
||||
},
|
||||
"plan": {
|
||||
"ver": 3,
|
||||
"seq": 68,
|
||||
"val": {
|
||||
"active": false,
|
||||
"wanted": null,
|
||||
"running": null,
|
||||
"activeAtLastHeader": false
|
||||
}
|
||||
},
|
||||
"subagentModelSelectionPolicy": {
|
||||
"ver": 1,
|
||||
"seq": 68,
|
||||
"val": null
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+128
@@ -0,0 +1,128 @@
|
||||
{
|
||||
"version": 5,
|
||||
"record": {
|
||||
"identity": {
|
||||
"createdAt": 1788286864454,
|
||||
"cwd": "/tmp"
|
||||
},
|
||||
"rows": {
|
||||
"sessionStats": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"turns": 2,
|
||||
"steps": 2,
|
||||
"llmMs": 2169,
|
||||
"toolMs": 0,
|
||||
"ttftMs": 1872,
|
||||
"ttftSteps": 2,
|
||||
"decodeMs": 297,
|
||||
"decodeTokens": 20,
|
||||
"lastTurn": 2,
|
||||
"openStep": null,
|
||||
"pendingCalls": {}
|
||||
}
|
||||
},
|
||||
"title": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": "\u9a8c\u6536\u6807\u9898-rc2"
|
||||
},
|
||||
"goal": {
|
||||
"ver": 4,
|
||||
"seq": 54,
|
||||
"val": null
|
||||
},
|
||||
"tokenUsage": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"totals": {
|
||||
"uncachedInputTokens": 6083,
|
||||
"outputTokens": 20,
|
||||
"cacheReadTokens": 9728,
|
||||
"cacheWriteTokens": 0
|
||||
},
|
||||
"last": {
|
||||
"turn": 2,
|
||||
"step": 1,
|
||||
"buckets": {
|
||||
"uncachedInputTokens": 112,
|
||||
"outputTokens": 4,
|
||||
"cacheReadTokens": 7808,
|
||||
"cacheWriteTokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"contextPressure": {
|
||||
"ver": 4,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"surfaceTokens": 178,
|
||||
"contextWindow": 1000000,
|
||||
"pressureTokens": 7920,
|
||||
"sampledSurfaceTokens": 168
|
||||
}
|
||||
},
|
||||
"contextBreakdown": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"systemTokens": 1620,
|
||||
"toolsTokens": 6475,
|
||||
"messageTokens": 178
|
||||
}
|
||||
},
|
||||
"subagentTiming": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"descriptorSeen": false,
|
||||
"settledMs": 0
|
||||
}
|
||||
},
|
||||
"subagent": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": {}
|
||||
},
|
||||
"permissions": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"preset": "workspace-write",
|
||||
"sandbox": "workspace-write",
|
||||
"approval": "ask"
|
||||
}
|
||||
},
|
||||
"sessionListMetadata": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"blank": false,
|
||||
"lastPromptAt": 1788286867410
|
||||
}
|
||||
},
|
||||
"imageLimits": {
|
||||
"ver": 1,
|
||||
"seq": 54,
|
||||
"val": null
|
||||
},
|
||||
"todos": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": null
|
||||
},
|
||||
"plan": {
|
||||
"ver": 2,
|
||||
"seq": 54,
|
||||
"val": {
|
||||
"active": false,
|
||||
"wanted": null,
|
||||
"running": null
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+220
@@ -0,0 +1,220 @@
|
||||
{
|
||||
"version": 5,
|
||||
"record": {
|
||||
"identity": {
|
||||
"createdAt": 1788286864454,
|
||||
"cwd": "/tmp",
|
||||
"isSeeded": false,
|
||||
"inheritedEventCount": 0
|
||||
},
|
||||
"rows": {
|
||||
"title": {
|
||||
"ver": 1,
|
||||
"seq": 71,
|
||||
"val": "验收标题-rc2"
|
||||
},
|
||||
"titleInput": {
|
||||
"ver": 3,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"first": {
|
||||
"seq": 7,
|
||||
"text": "请只回复一个词:pong"
|
||||
},
|
||||
"count": 3,
|
||||
"lastSeq": 60
|
||||
}
|
||||
},
|
||||
"llmRetry": {
|
||||
"ver": 1,
|
||||
"seq": 71,
|
||||
"val": {}
|
||||
},
|
||||
"sandboxMode": {
|
||||
"ver": 1,
|
||||
"seq": 71,
|
||||
"val": "workspace-write"
|
||||
},
|
||||
"goal": {
|
||||
"ver": 6,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"current": null,
|
||||
"seenGoalIds": [],
|
||||
"failure": null
|
||||
}
|
||||
},
|
||||
"tokenUsage": {
|
||||
"ver": 2,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"totals": {
|
||||
"uncachedInputTokens": 14219,
|
||||
"outputTokens": 24,
|
||||
"cacheReadTokens": 9728,
|
||||
"cacheWriteTokens": 0
|
||||
},
|
||||
"last": {
|
||||
"turn": 3,
|
||||
"step": 1,
|
||||
"buckets": {
|
||||
"uncachedInputTokens": 8136,
|
||||
"outputTokens": 4,
|
||||
"cacheReadTokens": 0,
|
||||
"cacheWriteTokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"contextPressure": {
|
||||
"ver": 4,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"surfaceTokens": 200,
|
||||
"contextWindow": 1000000,
|
||||
"pressureTokens": 8136,
|
||||
"sampledSurfaceTokens": 190
|
||||
}
|
||||
},
|
||||
"contextBreakdown": {
|
||||
"ver": 2,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"systemTokens": 1729,
|
||||
"toolsTokens": 6611,
|
||||
"messageTokens": 200
|
||||
}
|
||||
},
|
||||
"sessionStats": {
|
||||
"ver": 1,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"turns": 3,
|
||||
"steps": 3,
|
||||
"llmMs": 4368,
|
||||
"toolMs": 0,
|
||||
"ttftMs": 4030,
|
||||
"ttftSteps": 3,
|
||||
"decodeMs": 338,
|
||||
"decodeTokens": 24,
|
||||
"lastTurn": 3,
|
||||
"openStep": null,
|
||||
"pendingCalls": {}
|
||||
}
|
||||
},
|
||||
"agentPreset": {
|
||||
"ver": 1,
|
||||
"seq": 71,
|
||||
"val": "standard"
|
||||
},
|
||||
"subagentTiming": {
|
||||
"ver": 2,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"descriptorSeen": false,
|
||||
"settledMs": 0
|
||||
}
|
||||
},
|
||||
"subagent": {
|
||||
"ver": 2,
|
||||
"seq": 71,
|
||||
"val": {}
|
||||
},
|
||||
"turnBoundary": {
|
||||
"ver": 2,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"openTurnStartSeq": null,
|
||||
"lastStepStartSeq": 59,
|
||||
"lastStepBoundary": {
|
||||
"kind": "end",
|
||||
"seq": 70
|
||||
},
|
||||
"lastTurn": 3
|
||||
}
|
||||
},
|
||||
"turnOutline": {
|
||||
"ver": 2,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"turns": [
|
||||
{
|
||||
"turn": 1,
|
||||
"seq": 4,
|
||||
"prompt": "请只回复一个词:pong",
|
||||
"response": "pong"
|
||||
},
|
||||
{
|
||||
"turn": 2,
|
||||
"seq": 40,
|
||||
"prompt": "请只回复一个词:pong2",
|
||||
"response": "pong2"
|
||||
},
|
||||
{
|
||||
"turn": 3,
|
||||
"seq": 57,
|
||||
"prompt": "请只回复一个词:pong3",
|
||||
"response": "pong3"
|
||||
}
|
||||
],
|
||||
"draft": ""
|
||||
}
|
||||
},
|
||||
"permissions": {
|
||||
"ver": 2,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"preset": "workspace-write",
|
||||
"sandbox": "workspace-write",
|
||||
"approval": "ask",
|
||||
"seeded": true
|
||||
}
|
||||
},
|
||||
"modelSelection": {
|
||||
"ver": 2,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"lastUsed": {
|
||||
"provider": "deepseek-official",
|
||||
"model": "deepseek-v4-flash",
|
||||
"reasoningEffort": "high"
|
||||
},
|
||||
"pending": null
|
||||
}
|
||||
},
|
||||
"sessionListMetadata": {
|
||||
"ver": 1,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"blank": false,
|
||||
"lastPromptAt": 1788287660106
|
||||
}
|
||||
},
|
||||
"imageLimits": {
|
||||
"ver": 1,
|
||||
"seq": 71,
|
||||
"val": null
|
||||
},
|
||||
"todos": {
|
||||
"ver": 2,
|
||||
"seq": 71,
|
||||
"val": null
|
||||
},
|
||||
"plan": {
|
||||
"ver": 3,
|
||||
"seq": 71,
|
||||
"val": {
|
||||
"active": false,
|
||||
"wanted": null,
|
||||
"running": null,
|
||||
"activeAtLastHeader": false
|
||||
}
|
||||
},
|
||||
"subagentModelSelectionPolicy": {
|
||||
"ver": 1,
|
||||
"seq": 71,
|
||||
"val": null
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -88,7 +88,10 @@ export class DomainFacility {
|
||||
* (`facet-unsupported`); open the unit projected from the spec (backend
|
||||
* `version-mismatch`/`malformed-medium` pass through); load and validate
|
||||
* every stored record against the spec's zod schemas (`invalid-record`
|
||||
* with the offending table and key); construct the domain.
|
||||
* with the offending table and key — unless the spec declares
|
||||
* `invalidRecords: 'backup-and-skip'` and the unit can move documents aside, in
|
||||
* which case the failing record is backed up, logged, and skipped);
|
||||
* construct the domain.
|
||||
*
|
||||
* Lifecycle: the CALLER owns the returned handle and closes it via
|
||||
* `Domain.close()` (typically as its own `ctx.effect` disposer) — the
|
||||
@@ -118,7 +121,23 @@ export class DomainFacility {
|
||||
for (const [table, tableSpec] of Object.entries(spec.tables)) {
|
||||
const records = new Map<string, unknown>()
|
||||
for (const [key, raw] of Object.entries(snapshot.tables[table] ?? {})) {
|
||||
records.set(key, parseRecord(spec.name, table, key, () => tableSpec.valueSchema.parse(raw)))
|
||||
let parsed: unknown
|
||||
try {
|
||||
parsed = parseRecord(spec.name, table, key, () => tableSpec.valueSchema.parse(raw))
|
||||
} catch (error) {
|
||||
// Backup-and-skip policy (disposable derived data): move the record's
|
||||
// document aside, log the concrete failure, and open without the
|
||||
// record. Backends that cannot move a document keep the loud path.
|
||||
if (spec.invalidRecords !== 'backup-and-skip' || unit.backupRecord === undefined) throw error
|
||||
const moved = await unit.backupRecord(table, key)
|
||||
// parseRecord always wraps the zod failure as the cause.
|
||||
this.ctx.logger.error(
|
||||
`domain '${spec.name}': stored record '${key}' in table '${table}' failed schema validation; `
|
||||
+ `moved to '${moved}' and treated as absent. Cause: ${String((error as DomainError).cause)}`,
|
||||
)
|
||||
continue
|
||||
}
|
||||
records.set(key, parsed)
|
||||
}
|
||||
tables.set(table, records)
|
||||
}
|
||||
|
||||
@@ -35,16 +35,36 @@ export interface DomainTableSpec<K extends string = string, V = unknown> {
|
||||
export interface DomainSpec {
|
||||
/** Domain name; must match `UNIT_NAME_RE` (doubles as the backend unit name). */
|
||||
readonly name: string
|
||||
/** Domain format version; a medium stamped with a different version rejects at open. */
|
||||
/** Current domain format version; reads enforce it according to the selected layout. */
|
||||
readonly version: number
|
||||
/**
|
||||
* Medium layout for the backend unit: `single` (the default) stores the
|
||||
* whole unit as one document; `per-record` stores each record as its own
|
||||
* document, for units whose records are large, sparse, or individually
|
||||
* disposable — the projection cache — and scopes version bumps per record
|
||||
* (a stale record document is discarded, never migrated).
|
||||
* disposable — the projection cache — and scopes version checks per record
|
||||
* (an unaccepted record document is discarded, never migrated).
|
||||
*/
|
||||
readonly layout?: 'single' | 'per-record'
|
||||
/**
|
||||
* Older domain versions whose stored records the current record schemas
|
||||
* also accept (the declaring owner vouches for that, typically by
|
||||
* declaring the fields older records lack as optional). `per-record` backends
|
||||
* read documents stamped with a listed version instead of discarding them,
|
||||
* and accept a legacy whole-unit file so stamped for the one-time
|
||||
* bootstrap; writes always stamp {@link version}.
|
||||
*/
|
||||
readonly compatibleVersions?: readonly number[]
|
||||
/**
|
||||
* What `open` does with a stored table record that fails its zod schema.
|
||||
* Absent (the default), the whole open rejects with `invalid-record` —
|
||||
* right for authoritative data. `'backup-and-skip'` is for domains whose
|
||||
* records are disposable derived data: the backend moves the record's
|
||||
* document aside (`KvUnit.backupRecord`), the failure is logged with
|
||||
* its cause, and the open continues with the record absent. A backend
|
||||
* without `backupRecord` (no per-record document to move) falls back
|
||||
* to the rejecting default. The global slot always rejects.
|
||||
*/
|
||||
readonly invalidRecords?: 'backup-and-skip'
|
||||
/** Optional global singleton slot. */
|
||||
readonly global?: DomainGlobalSpec<unknown>
|
||||
/** Table declarations keyed by table name; each name must match `UNIT_NAME_RE`. */
|
||||
@@ -91,6 +111,13 @@ export function defineDomain<S extends DomainSpec>(spec: S): S {
|
||||
if (!Number.isInteger(spec.version) || spec.version < 0) {
|
||||
throw new Error(`domain '${spec.name}' version must be a non-negative integer, got ${spec.version}`)
|
||||
}
|
||||
for (const compat of spec.compatibleVersions ?? []) {
|
||||
if (!Number.isInteger(compat) || compat < 0 || compat >= spec.version) {
|
||||
throw new Error(
|
||||
`domain '${spec.name}' compatibleVersions entries must be non-negative integers below version ${spec.version}, got ${compat}`,
|
||||
)
|
||||
}
|
||||
}
|
||||
if (spec.layout !== undefined) {
|
||||
// Runtime boundary: the union type is compile-time only — a spec built
|
||||
// from config could carry any value, and a bad one must fail loud here.
|
||||
@@ -99,6 +126,12 @@ export function defineDomain<S extends DomainSpec>(spec: S): S {
|
||||
throw new Error(`domain '${spec.name}' layout must be 'single' or 'per-record', got ${layout}`)
|
||||
}
|
||||
}
|
||||
if (spec.invalidRecords !== undefined) {
|
||||
const policy: string = spec.invalidRecords
|
||||
if (policy !== 'backup-and-skip') {
|
||||
throw new Error(`domain '${spec.name}' invalidRecords must be 'backup-and-skip' when present, got ${policy}`)
|
||||
}
|
||||
}
|
||||
for (const table of Object.keys(spec.tables)) {
|
||||
if (!UNIT_NAME_RE.test(table)) {
|
||||
throw new Error(`domain '${spec.name}' table name '${table}' must match ${UNIT_NAME_RE}`)
|
||||
@@ -125,5 +158,6 @@ export function descriptorOf(spec: DomainSpec): KvUnitDescriptor {
|
||||
tables: Object.keys(spec.tables),
|
||||
hasGlobal: spec.global !== undefined,
|
||||
...spec.layout === undefined ? {} : { layout: spec.layout },
|
||||
...spec.compatibleVersions === undefined ? {} : { compatibleVersions: spec.compatibleVersions },
|
||||
}
|
||||
}
|
||||
@@ -58,6 +58,25 @@ describe('defineDomain', () => {
|
||||
})).toThrow(/must not accept null/)
|
||||
})
|
||||
|
||||
it('validates compatibleVersions entries and projects them onto the descriptor', () => {
|
||||
expect(() => defineDomain({ name: 'ok', version: 2, compatibleVersions: [1.5], tables: {} }))
|
||||
.toThrow(/compatibleVersions/)
|
||||
expect(() => defineDomain({ name: 'ok', version: 2, compatibleVersions: [2], tables: {} }))
|
||||
.toThrow(/below version/)
|
||||
expect(() => defineDomain({ name: 'ok', version: 2, compatibleVersions: [-1], tables: {} }))
|
||||
.toThrow(/compatibleVersions/)
|
||||
expect(descriptorOf(defineDomain({ name: 'ok', version: 2, compatibleVersions: [0, 1], tables: {} })))
|
||||
.toMatchObject({ compatibleVersions: [0, 1] })
|
||||
// An undeclared set is absent from the descriptor.
|
||||
expect(descriptorOf(spec)).not.toHaveProperty('compatibleVersions')
|
||||
})
|
||||
|
||||
it('rejects an unknown invalidRecords policy', () => {
|
||||
expect(() => defineDomain({
|
||||
name: 'ok', version: 1, invalidRecords: 'zap' as 'backup-and-skip', tables: {},
|
||||
})).toThrow(/invalidRecords/)
|
||||
})
|
||||
|
||||
it('rejects an invalid layout and projects the declared one onto the descriptor', () => {
|
||||
// A spec built from config can carry any value; the union type is
|
||||
// compile-time only, so the runtime boundary check must reject it.
|
||||
@@ -139,6 +158,28 @@ describe('DomainFacility.open', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps the rejecting default under backup-and-skip when the backend cannot move documents', async () => {
|
||||
// The memory backend has no backupRecord, so the declared policy cannot
|
||||
// apply and the open falls back to failing loud.
|
||||
const salvageSpec = defineDomain({
|
||||
name: 'salvage',
|
||||
version: 1,
|
||||
invalidRecords: 'backup-and-skip',
|
||||
tables: { items: domainTable<string, Item>(itemSchema) },
|
||||
})
|
||||
const pool = new MemoryMediaPool()
|
||||
{
|
||||
const { facility } = await harness({ pool })
|
||||
await (await facility.open(salvageSpec)).table('items').put('bad', { label: 'x', count: 2 })
|
||||
}
|
||||
pool.media.get('salvage')!.tables.get('items')!.set('bad', { label: 'x', count: 'NaN' })
|
||||
const { facility } = await harness({ pool })
|
||||
await expect(facility.open(salvageSpec)).rejects.toMatchObject({
|
||||
code: 'invalid-record',
|
||||
detail: { table: 'items', key: 'bad' },
|
||||
})
|
||||
})
|
||||
|
||||
it('rejects a stored global that fails its schema with the global marker', async () => {
|
||||
const pool = new MemoryMediaPool()
|
||||
pool.versions.set('demo', 1)
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/storage/storage-json/README.md
|
||||
README.md: 2972a617248c73e6105e8f19635e2325337405b6
|
||||
README.zh.md: 5c9f52d2a12981f2bff36e4df198270d09f2a8e5
|
||||
README.md: d36c6333392c3efe4462aebd82cf3ff9e76f66ae
|
||||
README.zh.md: 9c266537945941d4bc779a2cb1a349e2409f21dc
|
||||
@@ -53,9 +53,9 @@ The generated [configuration catalog](../../../docs/config-catalog.md#deepseek-a
|
||||
|
||||
### Observable behavior
|
||||
|
||||
A missing `single` file or `per-record` directory opens as an empty unit and materializes on the first write. In `single`, malformed content rejects with `malformed-medium`, and a different stored version rejects with `version-mismatch`. In `per-record`, each malformed, unreadable, or differently versioned document reads as an absent record, so one bad document does not reject the unit. Record keys must match `[a-zA-Z0-9_-]+`; an unsafe key rejects before any file operation. Every resolved write is durable, and operations after close reject with `closed`.
|
||||
A missing `single` file or `per-record` directory opens as an empty unit and materializes on the first write. In `single`, malformed content rejects with `malformed-medium`, and a different stored version rejects with `version-mismatch`. In `per-record`, each malformed or unreadable document, and each document whose version is outside the descriptor's current and compatible versions, reads as an absent record, so one bad document does not reject the unit. Record keys must match `[a-zA-Z0-9_-]+`; an unsafe key rejects before any file operation. Every resolved write is durable, and operations after close reject with `closed`.
|
||||
|
||||
An empty `per-record` tree can initialize its declared tables from a valid `<root>/<unit>.json` whole-unit document only when the source unit name and version match the current descriptor. The backend leaves that source file unchanged. A different source version leaves the new tree empty. Any document path in a declared table, or a declared `global.json`, suppresses this initialization for the complete unit, even if that document is unreadable or stale.
|
||||
An empty `per-record` tree can initialize its declared tables from a valid `<root>/<unit>.json` whole-unit document only when the source unit name matches and its version is current or declared compatible. The backend leaves that source file unchanged and stamps migrated records with the current version. A source version outside the accepted set leaves the new tree empty. Any document path in a declared table, or a declared `global.json`, suppresses this initialization for the complete unit, even if that document is unreadable or stale.
|
||||
|
||||
-----
|
||||
|
||||
|
||||
@@ -53,9 +53,9 @@ kind: "package-reference"
|
||||
|
||||
### 可观察行为
|
||||
|
||||
缺失的 `single` 文件或 `per-record` 目录会作为空单元打开,并在第一次写入时物化。在 `single` 中,畸形内容以 `malformed-medium` 拒绝,不同的已存版本以 `version-mismatch` 拒绝。在 `per-record` 中,每份畸形、不可读或版本不同的文档都读作记录不存在,因此单个坏文档不会使单元被拒绝。记录键必须匹配 `[a-zA-Z0-9_-]+`;不安全的键在任何文件操作前被拒绝。每次已完成的写入都已持久化,关闭后的操作以 `closed` 拒绝。
|
||||
缺失的 `single` 文件或 `per-record` 目录会作为空单元打开,并在第一次写入时物化。在 `single` 中,畸形内容以 `malformed-medium` 拒绝,不同的已存版本以 `version-mismatch` 拒绝。在 `per-record` 中,每份畸形或不可读的文档,以及版本不在 descriptor 当前版本和兼容版本内的文档,都读作记录不存在,因此单个坏文档不会使单元被拒绝。记录键必须匹配 `[a-zA-Z0-9_-]+`;不安全的键在任何文件操作前被拒绝。每次已完成的写入都已持久化,关闭后的操作以 `closed` 拒绝。
|
||||
|
||||
只有当源单元名称和版本与当前描述符相同时,空的 `per-record` 目录树才可以从有效的 `<root>/<unit>.json` 整单元文档初始化其已声明表。后端保持该源文件不变。源版本不同会使新目录树保持为空。已声明表中只要存在任意文档路径,或存在已声明的 `global.json`,就会对整个单元禁止该初始化,即使该文档不可读或版本陈旧。
|
||||
只有当源单元名称匹配,且源版本为当前版本或已声明的兼容版本时,空的 `per-record` 目录树才可以从有效的 `<root>/<unit>.json` 整单元文档初始化其已声明表。后端保持该源文件不变,并为迁移的记录写入当前版本戳。接受集合之外的源版本会使新目录树保持为空。已声明表中只要存在任意文档路径,或存在已声明的 `global.json`,就会对整个单元禁止该初始化,即使该文档不可读或版本陈旧。
|
||||
|
||||
-----
|
||||
|
||||
|
||||
@@ -100,16 +100,18 @@ export function serializeRecord(version: number, value: unknown): string {
|
||||
|
||||
/**
|
||||
* Parse one per-record document, validating its version stamp. A document
|
||||
* that is malformed or stamped with a different version is FOREIGN and reads
|
||||
* as absent — the per-record contract: one bad or stale record file must not
|
||||
* brick the whole unit, and a version bump discards stale records instead of
|
||||
* migrating them (the whole-unit format rejects instead, because there is
|
||||
* exactly one document).
|
||||
* that is malformed or stamped with an unaccepted version is FOREIGN and
|
||||
* reads as absent — the per-record contract: one bad or stale record file
|
||||
* must not brick the whole unit, and an unaccepted version stamp discards the
|
||||
* record instead of migrating it (the whole-unit format rejects instead,
|
||||
* because there is exactly one document).
|
||||
* @param text - Raw per-record document content.
|
||||
* @param version - Expected unit version; a mismatch discards the document.
|
||||
* @param versions - Accepted unit versions (the current one plus the
|
||||
* descriptor's compatibleVersions); any other stamp discards the
|
||||
* document.
|
||||
* @returns the record value, or `undefined` for a foreign document.
|
||||
*/
|
||||
export function parseRecord(text: string, version: number): unknown {
|
||||
export function parseRecord(text: string, versions: readonly number[]): unknown {
|
||||
let document: unknown
|
||||
try {
|
||||
document = JSON.parse(text)
|
||||
@@ -118,6 +120,6 @@ export function parseRecord(text: string, version: number): unknown {
|
||||
}
|
||||
if (typeof document !== 'object' || document === null) return undefined
|
||||
const { version: stamped, record } = document as Record<string, unknown>
|
||||
if (stamped !== version) return undefined
|
||||
if (typeof stamped !== 'number' || !versions.includes(stamped)) return undefined
|
||||
return record
|
||||
}
|
||||
@@ -10,21 +10,23 @@
|
||||
* memory unchanged.
|
||||
*
|
||||
* Per-record contract: a record document that is malformed or stamped with a
|
||||
* different version reads as an absent record — one bad or stale file never
|
||||
* bricks the whole unit, and a version bump discards stale records instead
|
||||
* of migrating them. Record keys become path segments, so they must be
|
||||
* path-safe (`[a-zA-Z0-9_-]+`); an unsafe key rejects at write.
|
||||
* version outside the accepted set (the descriptor's current version plus
|
||||
* its `compatibleVersions`) reads as an absent record — one bad or stale
|
||||
* file never bricks the whole unit, and an unaccepted version stamp discards
|
||||
* the record instead of migrating it. Record keys become path segments, so
|
||||
* they must be path-safe (`[a-zA-Z0-9_-]+`); an unsafe key rejects at write.
|
||||
*
|
||||
* Legacy bootstrap: when the new tree has no document path, a legacy
|
||||
* whole-unit file `<root>/<name>.json` (the pre-per-record layout) seeds
|
||||
* per-record documents only when its unit name and version match the current
|
||||
* descriptor. Any new document path, including one whose contents are
|
||||
* per-record documents, provided its stored unit version is in the accepted
|
||||
* set — a legacy file stamped with any other version is left alone and reads
|
||||
* as the empty unit. Any new document path, including one whose contents are
|
||||
* unreadable or stale, suppresses the bootstrap for the whole unit. The
|
||||
* legacy file is never changed or deleted.
|
||||
* @module @deepseek-ai/dsh-storage-json/src/per-record-unit
|
||||
*/
|
||||
|
||||
import { mkdir, readFile, readdir, rm } from 'node:fs/promises'
|
||||
import { mkdir, readFile, readdir, rename, rm } from 'node:fs/promises'
|
||||
import { dirname, join } from 'node:path'
|
||||
import type { Dirent } from 'node:fs'
|
||||
import { StorageError } from '@deepseek-ai/dsh-storage'
|
||||
@@ -58,13 +60,14 @@ export async function openPerRecordUnit(
|
||||
* Read every record document under the unit directory: each declared table's
|
||||
* `<key>.json` files plus `global.json`. A missing directory is the empty
|
||||
* unit (materialization defers to the first write); a foreign document
|
||||
* (missing, malformed, or stamped with another version) reads as an absent
|
||||
* (missing, malformed, or stamped with an unaccepted version) reads as an absent
|
||||
* record, per the per-record contract.
|
||||
* @param descriptor - Static identity and shape of the unit.
|
||||
* @param dir - Absolute unit directory path.
|
||||
* @returns the authoritative state reconstructed from the tree.
|
||||
*/
|
||||
async function loadPerRecordState(descriptor: KvUnitDescriptor, dir: string): Promise<UnitState> {
|
||||
const versions = acceptedStamps(descriptor)
|
||||
const state: UnitState = {
|
||||
version: descriptor.version,
|
||||
global: null,
|
||||
@@ -84,11 +87,11 @@ async function loadPerRecordState(descriptor: KvUnitDescriptor, dir: string): Pr
|
||||
if (entry.isDirectory()) {
|
||||
const records = state.tables.get(entry.name)
|
||||
if (records !== undefined) {
|
||||
return loadTableRecords(records, descriptor.version, join(dir, entry.name))
|
||||
return loadTableRecords(records, versions, join(dir, entry.name))
|
||||
}
|
||||
}
|
||||
if (entry.name === 'global.json' && descriptor.hasGlobal) {
|
||||
const global = await readRecord(join(dir, entry.name), descriptor.version)
|
||||
const global = await readRecord(join(dir, entry.name), versions)
|
||||
if (global !== undefined) state.global = global
|
||||
return true
|
||||
}
|
||||
@@ -98,13 +101,21 @@ async function loadPerRecordState(descriptor: KvUnitDescriptor, dir: string): Pr
|
||||
return state
|
||||
}
|
||||
|
||||
/** The version stamps this unit reads as its own: current plus declared compatible versions. */
|
||||
function acceptedStamps(descriptor: KvUnitDescriptor): readonly number[] {
|
||||
return [descriptor.version, ...descriptor.compatibleVersions ?? []]
|
||||
}
|
||||
|
||||
/**
|
||||
* Bootstrap an empty per-record tree from a legacy whole-unit file
|
||||
* (`<root>/<name>.json`, the pre-per-record layout). Every declared-table
|
||||
* record is copied into a same-version document, while the legacy file is
|
||||
* retained unchanged. A missing, foreign (another unit name or version),
|
||||
* malformed, or non-unit legacy file is left alone; other read failures
|
||||
* propagate.
|
||||
* record is copied into a current-version document, while the legacy file is
|
||||
* retained unchanged. A missing, foreign (another unit's name), malformed,
|
||||
* or non-unit legacy file is left alone, and so is one whose stored unit
|
||||
* version is outside the accepted set — migrating records the owner never
|
||||
* vouched for would stamp them with the current version and turn a
|
||||
* discardable stale cache into schema failures at the domain layer. Other
|
||||
* read failures propagate.
|
||||
* @param descriptor - Static identity and shape of the unit.
|
||||
* @param dir - The per-record unit directory (`<root>/<name>`).
|
||||
* @param state - The empty tree state; bootstrapped records are added.
|
||||
@@ -118,15 +129,18 @@ async function bootstrapLegacyUnit(descriptor: KvUnitDescriptor, dir: string, st
|
||||
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error
|
||||
return
|
||||
}
|
||||
// The legacy document is runtime data: its unit identity and the tables map
|
||||
// shape are checked here; the domain layer's schemas judge record values.
|
||||
// The legacy document is runtime data: only `unit.name`, `unit.version`,
|
||||
// and the tables map shape are checked here — the record values are
|
||||
// migrated as-is and the domain layer's schemas judge them.
|
||||
let document: { unit?: { name?: unknown; version?: unknown }; tables?: unknown }
|
||||
try {
|
||||
document = JSON.parse(text) as { unit?: { name?: unknown; version?: unknown }; tables?: unknown }
|
||||
} catch {
|
||||
return // Malformed legacy file: not ours to interpret or delete.
|
||||
}
|
||||
if (document.unit?.name !== descriptor.name || document.unit.version !== descriptor.version) return
|
||||
if (document.unit?.name !== descriptor.name) return
|
||||
const stamped = document.unit.version
|
||||
if (typeof stamped !== 'number' || !acceptedStamps(descriptor).includes(stamped)) return
|
||||
const tables = document.tables
|
||||
if (typeof tables !== 'object' || tables === null) return
|
||||
const recordsByTable = tables as Record<string, Record<string, unknown>>
|
||||
@@ -147,14 +161,14 @@ async function bootstrapLegacyUnit(descriptor: KvUnitDescriptor, dir: string, st
|
||||
* @returns whether the directory contains any `.json` document path,
|
||||
* independently of key safety, readability, or stored version.
|
||||
*/
|
||||
async function loadTableRecords(records: Map<string, unknown>, version: number, dir: string): Promise<boolean> {
|
||||
async function loadTableRecords(records: Map<string, unknown>, versions: readonly number[], dir: string): Promise<boolean> {
|
||||
const files = await readdir(dir, { withFileTypes: true })
|
||||
const hasDocuments = files.some(file => file.name.endsWith('.json'))
|
||||
const loaded = await Promise.all(files.map(async (file) => {
|
||||
if (!file.name.endsWith('.json')) return
|
||||
const key = file.name.slice(0, -'.json'.length)
|
||||
if (!SAFE_KEY_RE.test(key)) return
|
||||
const record = await readRecord(join(dir, file.name), version)
|
||||
const record = await readRecord(join(dir, file.name), versions)
|
||||
if (record !== undefined) return [key, record] as const
|
||||
}))
|
||||
for (const record of loaded) {
|
||||
@@ -164,9 +178,9 @@ async function loadTableRecords(records: Map<string, unknown>, version: number,
|
||||
}
|
||||
|
||||
/** Read one record document; a foreign (unreadable or stale) one reads as absent. */
|
||||
async function readRecord(path: string, version: number): Promise<unknown> {
|
||||
async function readRecord(path: string, versions: readonly number[]): Promise<unknown> {
|
||||
try {
|
||||
return parseRecord(await readFile(path, 'utf8'), version)
|
||||
return parseRecord(await readFile(path, 'utf8'), versions)
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
@@ -214,6 +228,22 @@ export class PerRecordJsonUnit implements KvUnit {
|
||||
await this.tracked(rm(join(this.tableDir(table), `${key}.json`), { force: true }))
|
||||
}
|
||||
|
||||
/**
|
||||
* Move one record's document aside as `<key>.json.bak.<YYYYMMDDHHmm>`. The
|
||||
* moved file no longer ends in `.json`, so every later read ignores it; the
|
||||
* bytes stay on disk for inspection. A same-minute backup of the same
|
||||
* key overwrites the previous backup (the newer bytes are the ones worth
|
||||
* keeping).
|
||||
*/
|
||||
async backupRecord(table: string, key: string): Promise<string> {
|
||||
this.assertOpen()
|
||||
assertSafeKey(this.descriptor.name, key)
|
||||
const path = join(this.tableDir(table), `${key}.json`)
|
||||
const moved = `${path}.bak.${backupStamp(new Date())}`
|
||||
await this.tracked(rename(path, moved))
|
||||
return moved
|
||||
}
|
||||
|
||||
/** Durably replace the global singleton. Only valid when declared. */
|
||||
async setGlobal(value: unknown): Promise<void> {
|
||||
this.assertOpen()
|
||||
@@ -268,6 +298,12 @@ export class PerRecordJsonUnit implements KvUnit {
|
||||
}
|
||||
}
|
||||
|
||||
/** Local-time `YYYYMMDDHHmm` suffix for backed-up documents. */
|
||||
function backupStamp(now: Date): string {
|
||||
const pad = (value: number): string => String(value).padStart(2, '0')
|
||||
return `${String(now.getFullYear())}${pad(now.getMonth() + 1)}${pad(now.getDate())}${pad(now.getHours())}${pad(now.getMinutes())}`
|
||||
}
|
||||
|
||||
/** Reject a record key that would be unsafe as a path segment. */
|
||||
function assertSafeKey(unit: string, key: string): void {
|
||||
if (!SAFE_KEY_RE.test(key)) {
|
||||
|
||||
@@ -347,10 +347,14 @@ describe('per-record layout', () => {
|
||||
await backend.close()
|
||||
})
|
||||
|
||||
it('leaves an older-version legacy whole-unit file unconverted', async () => {
|
||||
it('bootstraps from a legacy file only when its stored version is accepted', async () => {
|
||||
// Version 3 is neither current (2) nor declared compat: the legacy file
|
||||
// is left alone and the unit reads empty — migrating unvouched records
|
||||
// would stamp them current and surface as schema failures at the domain
|
||||
// layer instead of a discardable stale cache.
|
||||
const root = await freshRoot()
|
||||
const legacy = JSON.stringify({
|
||||
unit: { name: 'recs', version: descriptor.version - 1 },
|
||||
unit: { name: 'recs', version: 3 },
|
||||
global: null,
|
||||
tables: { t: { old: { v: 1 } } },
|
||||
})
|
||||
@@ -360,6 +364,56 @@ describe('per-record layout', () => {
|
||||
expect(await unit.loadAll()).toEqual({ tables: { t: {} }, global: null })
|
||||
await expect(readFile(recordPath(root, 'old'), 'utf8')).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
await expect(readFile(join(root, 'recs.json'), 'utf8')).resolves.toBe(legacy)
|
||||
await unit.close()
|
||||
await backend.close()
|
||||
|
||||
// The same file bootstraps once version 3 is declared read-compatible…
|
||||
const root2 = await freshRoot()
|
||||
await writeFile(join(root2, 'recs.json'), legacy, 'utf8')
|
||||
const backend2 = new JsonStorageBackend(root2)
|
||||
const compat = { ...descriptor, version: 4, compatibleVersions: [3] }
|
||||
const unit2 = await backend2.kv.open(compat)
|
||||
expect(await unit2.loadAll()).toEqual({ tables: { t: { old: { v: 1 } } }, global: null })
|
||||
// …and the migrated documents are stamped with the CURRENT version.
|
||||
expect(JSON.parse(await readFile(join(root2, 'recs', 't', 'old.json'), 'utf8')))
|
||||
.toEqual({ version: 4, record: { v: 1 } })
|
||||
await unit2.close()
|
||||
await backend2.close()
|
||||
})
|
||||
|
||||
it('backupRecord moves the document aside; reads see it absent and a write recreates it', async () => {
|
||||
const root = await freshRoot()
|
||||
const backend = new JsonStorageBackend(root)
|
||||
const unit = await backend.kv.open(descriptor)
|
||||
await unit.putRecord('t', 'k', { v: 1 })
|
||||
const moved = await unit.backupRecord!('t', 'k')
|
||||
expect(moved).toMatch(/k\.json\.bak\.\d{12}$/)
|
||||
expect(JSON.parse(await readFile(moved, 'utf8'))).toEqual({ version: 2, record: { v: 1 } })
|
||||
await expect(readFile(recordPath(root, 'k'), 'utf8')).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
// The moved file no longer ends in .json, so it reads as absent…
|
||||
expect(await unit.loadAll()).toEqual({ tables: { t: {} }, global: null })
|
||||
// …and the key is free for a fresh write.
|
||||
await unit.putRecord('t', 'k', { v: 2 })
|
||||
expect(await unit.loadAll()).toEqual({ tables: { t: { k: { v: 2 } } }, global: null })
|
||||
await expect(unit.backupRecord!('t', 'a/b')).rejects.toThrow(/not path-safe/)
|
||||
await unit.close()
|
||||
await expect(unit.backupRecord!('t', 'k')).rejects.toMatchObject({ code: 'closed' })
|
||||
await backend.close()
|
||||
})
|
||||
|
||||
it('reads per-record documents stamped with a declared compat version and stamps writes current', async () => {
|
||||
const root = await freshRoot()
|
||||
const backend = new JsonStorageBackend(root)
|
||||
const compat = { ...descriptor, compatibleVersions: [1] }
|
||||
await mkdir(join(root, 'recs', 't'), { recursive: true })
|
||||
await writeFile(recordPath(root, 'oldrec'), JSON.stringify({ version: 1, record: { v: 'old' } }), 'utf8')
|
||||
await writeFile(recordPath(root, 'ancient'), JSON.stringify({ version: 0, record: { v: 'no' } }), 'utf8')
|
||||
const unit = await backend.kv.open(compat)
|
||||
// Version 1 is declared compat and served; version 0 is not and discards.
|
||||
expect(await unit.loadAll()).toEqual({ tables: { t: { oldrec: { v: 'old' } } }, global: null })
|
||||
await unit.putRecord('t', 'oldrec', { v: 'new' })
|
||||
expect(JSON.parse(await readFile(recordPath(root, 'oldrec'), 'utf8')))
|
||||
.toEqual({ version: 2, record: { v: 'new' } })
|
||||
await backend.close()
|
||||
})
|
||||
|
||||
@@ -416,6 +470,7 @@ describe('per-record layout', () => {
|
||||
await backend4.close()
|
||||
|
||||
const root5 = await freshRoot()
|
||||
// A current-version stamp so the shapeless `tables` is what stops the bootstrap.
|
||||
await writeFile(
|
||||
join(root5, 'recs.json'),
|
||||
JSON.stringify({ unit: { name: 'recs', version: descriptor.version }, tables: 'not an object' }),
|
||||
|
||||
@@ -56,11 +56,21 @@ export interface KvUnitDescriptor {
|
||||
* Medium layout. `single` (the default) keeps the whole unit in one
|
||||
* document; `per-record` keeps each record in its own document, so a unit
|
||||
* whose records are large or sparse never rewrites the rest on one write,
|
||||
* and a version bump discards stale records instead of rejecting the whole
|
||||
* unit. Backends that only serve one layout accept the other's units as
|
||||
* foreign documents.
|
||||
* and an unaccepted version stamp discards only that record instead of
|
||||
* rejecting the whole unit. Backends that only serve one layout accept the
|
||||
* other's units as foreign documents.
|
||||
*/
|
||||
readonly layout?: 'single' | 'per-record'
|
||||
/**
|
||||
* Older unit versions whose stored records are also readable under the
|
||||
* declaring owner's current record schemas (the owner vouches for that —
|
||||
* typically by declaring the fields old records lack as optional). Reads of
|
||||
* a `per-record` unit accept documents stamped with any listed version, and
|
||||
* the legacy whole-unit bootstrap accepts a legacy file stamped with one;
|
||||
* writes always stamp {@link version}. `single`-layout reads stay
|
||||
* exact-version.
|
||||
*/
|
||||
readonly compatibleVersions?: readonly number[]
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -99,6 +109,19 @@ export interface KvUnit {
|
||||
*/
|
||||
deleteRecord(table: string, key: string): Promise<void>
|
||||
|
||||
/**
|
||||
* Move one record's stored document out of the unit's readable set,
|
||||
* preserving its bytes for inspection instead of deleting them. Backends
|
||||
* whose medium has no per-record document to move (the `single` layout, a
|
||||
* row store) omit this member, and the caller falls back to its
|
||||
* reject-loud path. Absent after the move: a later {@link loadAll} reads
|
||||
* the key as missing and a later {@link putRecord} recreates it fresh.
|
||||
* @param table - Declared table name.
|
||||
* @param key - Record key.
|
||||
* @returns the medium location the document was moved to (diagnostics).
|
||||
*/
|
||||
backupRecord?(table: string, key: string): Promise<string>
|
||||
|
||||
/**
|
||||
* Write the global singleton durably. Only valid when the descriptor
|
||||
* declared `hasGlobal`.
|
||||
|
||||
Reference in new issue
Block a user