Merge pull request #3585 from deepseek-harness/session-migration-streaming
perf(session): stream released format migrations
This commit is contained in:
imccyu authored and GitHub committed
commit
80306f51a6
94 files changed
+6053
-2526
No files matched your search
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md
|
||||
2026-08-31-released-session-format-migrations.md: 2c75d0b57a0b513c218b6a67b8c2b31c7cae4d0f
|
||||
2026-08-31-released-session-format-migrations.zh.md: d88c643cbfaf7f3d4f52ca6e5fa244a26917eb99
|
||||
2026-08-31-released-session-format-migrations.md: 3c4626c8426526a474cfba76ac820905da05beaf
|
||||
2026-08-31-released-session-format-migrations.zh.md: 806e63f2c8689476e4ca215cc6732ee835393006
|
||||
+160
-25
@@ -1,4 +1,4 @@
|
||||
# Agent Note: Released Session formats migrate on body read through adjacent pure edges
|
||||
# Agent Note: Released Session formats migrate through stateful streaming stages
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -6,52 +6,187 @@ English | [中文](2026-08-31-released-session-format-migrations.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Session format v0 shipped in an alpha release, so a structural writer change can no longer treat existing JSONL as disposable pre-release state. Stored event bodies reach consumers through read or write `SessionHandle` instances used by resume, query, export, fork, and continuation paths. Migrating only one consumer would let callers observe different logical generations or fail only when a later writer reaches the old file.
|
||||
Session format v0 shipped in an alpha release, so a structural writer change can no longer treat existing JSONL as disposable pre-release state. The first whole-artifact migration implementation made those logs convertible, but its data model turned a 116 MB real Session into an operation that exhausted a 16 GB Node process before returning a handle.
|
||||
|
||||
Migration must retain the exact source path, bytes, and inode, including a torn physical tail, while giving every published format one unambiguous canonical filename. Plain JSONL and Zstandard are encoding choices for the same logical format and must not create parallel migration implementations.
|
||||
### Whole-artifact performance failure
|
||||
|
||||
- Zstandard input was split into 317,540 frames and each frame used a separate asynchronous decompression call. The implementation retained every plaintext frame and then concatenated them before JSON parsing, creating the same number of Promise, thread-pool, and native decode transitions.
|
||||
- Physical Decode materialized a complete plaintext Buffer, one complete string, every JSONL row, expanded source events, migrated target events, encoded target rows, a joined target string, and target physical Buffers at overlapping points in the same request.
|
||||
- Every codec and migration edge called `snapshotSessionFormatJson()` or `snapshotSessionFormatArtifact()`. These operations detached, recursively copied, and deeply froze whole headers, rows, payloads, and event arrays before and after adjacent migrations.
|
||||
- Released packed Assistant chunks expanded into about 9.14 million logical v0/v1 events before v1-to-v2 folded them into 72,784 current events. The whole-artifact API required both representations and the old-to-new sequence map to coexist.
|
||||
- Encoding built the complete JSONL and compressed output in memory. The successful path then decoded the staged target, decoded the committed target, and decoded it again in persistence to construct the business object; it also reread the source for a full fingerprint comparison.
|
||||
- Per-frame `await` calls did not provide useful bounded scheduling. The pre-migration reader instead reused one synchronous decoder and yielded from the outer loop about every 500 ms, avoiding hundreds of thousands of asynchronous transitions.
|
||||
|
||||
### The interfaces prevented local fixes from composing
|
||||
|
||||
`SessionFormatCodec` decoded and encoded complete arrays, each adjacent `SessionFormatMigration` accepted and returned a complete `SessionFormatArtifact`, and the compiled chain could only hand one materialized artifact to the next edge. A faster physical decoder therefore still encountered source-row arrays, expanded-event arrays, per-edge snapshots, and target-row arrays downstream.
|
||||
|
||||
The migrations are stateful even though the API presented them as one-shot functions. v0-to-v1 tracks message and retry identity. v1-to-v2 buffers one unsettled Assistant attempt, tracks events blocked behind it, and maintains old-to-new sequence references. Wrapping that state in closures or push/finish helper objects made the runtime structure different from the static declarations and made production, Worker verification, fixtures, and replay use different entry paths.
|
||||
|
||||
## Decision
|
||||
|
||||
`SESSION_FORMAT_VERSION` is a monotonic current-writer integer. One profile-independent pure package owns each adjacent `vN -> vN+1` conversion. `@deepseek-ai/dsh-session-format` supplies only lossless snapshots, unique gap-free planning, header-only conversion, and whole-artifact composition; `@deepseek-ai/dsh-session-format-catalog` statically imports the complete chain independently of mounted Cordis plugins. Historical codecs and normalizers live in the named edge package, while current Session and persistence code accept only the latest logical types.
|
||||
The Session format packages use a stateful synchronous Stage API. Static migration declarations describe one adjacent version edge and create a new stage for each restored artifact. A stage owns that artifact's mutable state; no stage instance is shared across Sessions.
|
||||
|
||||
Each edge freezes strict source and target semantics, while its target physical codec remains vocabulary-neutral so ordinary event growth can stay within one format version. The catalog restores the final generation through the installed peer `@deepseek-ai/dsh-session` and its current `KNOWN_SESSION_EVENT_TYPES`, preventing a frozen historical edge from becoming the current vocabulary owner.
|
||||
### Stage and Context protocol
|
||||
|
||||
The JSONL provider completes ensure-current work before `open` returns a handle for a stored Session. It selects the highest canonical generation, migrates a supported historical body, and decodes the current result from one physical snapshot; the public `SessionPersistence` and `SessionHandle` interfaces contain no migration operations. Header-only `stat` and `list` rescan Session directories, translate supported historical headers in memory, and never publish a successor. `create` checks canonical filenames independently of header readability, so every existing generation reserves its Session id.
|
||||
```text
|
||||
interface SessionFormatMigrationContext {
|
||||
emitEvent(event: SessionFormatEvent): void
|
||||
emitRun(run: SessionFormatEventRun): void
|
||||
}
|
||||
|
||||
Cancellation belongs to the `open`, `stat`, or `list` call that supplied it. Discovery, stable reads, decoding, and pre-publication checks observe that signal; once an immutable successor is published and its directory entry is synced, later cancellation does not delete the committed generation.
|
||||
interface SessionFormatMigrationStage {
|
||||
readonly headerInheritedEventCount?: number
|
||||
transformEvent(
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void
|
||||
transformRun(
|
||||
run: SessionFormatEventRun,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void
|
||||
finish(context: SessionFormatMigrationContext): number
|
||||
}
|
||||
```
|
||||
|
||||
The configured JSONL encoding owns one full suffix, `.jsonl` or `.jsonl.zstd`. Migration reads a stable exact source, decodes the recoverable logical prefix, composes every required edge in memory, validates and syncs a same-directory temporary stage for only the final target, rechecks the source fingerprint, publishes that previously absent target without overwrite, syncs the namespace, and reopens it through current validation before returning a handle. The source never moves or changes; only disposable temporary stages may be moved, linked, or removed. Migration does not synthesize interrupted-turn events: agent-loop appends those repairs through the write handle, while read-only query paths balance them in memory.
|
||||
`SessionFormatMigrationContext.emitEvent()` and `emitRun()` are synchronous. The producer declares whether it emits a scalar event or a compact run, so the hot path never infers the category from properties on a parsed file object. The caller owns scheduling and supplies the context to each operation instead of injecting a callback into the stage constructor. One input may emit zero, one, or many outputs without allocating a temporary return array or retaining an internal output queue.
|
||||
|
||||
Canonical filenames encode the physical format generation: v0 is `session.jsonl` or `session.jsonl.zstd`; every positive generation is lowercase `session.vN.jsonl` or `session.vN.jsonl.zstd`. `dsh-session-format` owns the raw basename rule (`sessionFormatLogFilename`, `parseSessionFormatLogFilename`); the JSONL provider, the session-log export archive, and recorded-session fixtures append only the compression suffix. Publication never renames, replaces, or deletes a committed generation path. If the target already exists, it is accepted only as a regular current-format file with exactly the expected bytes; any other target refuses. Lower generations remain for operator inspection or explicit copying, but normal runtime operations select the numerically highest canonical name and never use retained predecessors as automatic fallback, restore, or downgrade support.
|
||||
`SessionFormatMigration` remains an immutable declaration: version numbers, header migration, target-header validation, and `createStage()`. `CompiledSessionFormatChain` validates a unique gap-free edge sequence once, creates per-artifact stages in source-to-target order, and connects them with context objects in reverse order. `finish()` settles stages in source-to-target order so each stage can emit its tail before the downstream stage closes.
|
||||
|
||||
The current-format fast path classifies the header from one stable source snapshot, invokes no historical converter or generation write, and passes that snapshot to current decoding without another file read. The decoded log enters the existing bounded revision-keyed memo for an immediate observe-to-resume handoff, while `stat` and `list` deliberately rescan. Multiple edges leave the original generation unchanged and publish only the final target; intermediate versions exist only in memory. A source fingerprint recheck restarts migration when content changes, and exclusive target publication accepts a racing winner only when its bytes match exactly. Cross-process append fencing remains outside this guarantee.
|
||||
```text
|
||||
JSONL record
|
||||
→ released physical row decoder
|
||||
→ v0-to-v1 stage
|
||||
→ v1-to-v2 stage
|
||||
→ current event collector
|
||||
```
|
||||
|
||||
The first edge, `@deepseek-ai/dsh-session-format-v0-to-v1`, is intentionally identity-shaped: aside from the version and bounded historical normalizations already accepted by v0, it preserves logical headers, events, sequence numbers, references, timestamps, payloads, and the configured compression choice. The exact `session.jsonl[.zstd]` source remains byte- and inode-identical, while the current writer encodes the new `session.v1.jsonl[.zstd]` successor. This exercises the complete publication lifecycle before a cardinality-changing format needs it.
|
||||
The chain contains no `flatMap`, spread expansion, intermediate event array, or scheduler. The final event collector expands a compact run only after every migration stage has had the opportunity to consume it directly.
|
||||
|
||||
Projection-cache records bind their fold to the Session header's `formatVersion`. The `session_projcache` v7 reader may load predecessor domain records structurally, but a record without the format generation cannot seed a current Session; the authoritative log refolds it and the next checkpoint writes the complete current identity. This prevents a cache row produced before a bounded normalizer or cardinality-changing edge from bypassing that migration.
|
||||
### Physical codecs and packed runs
|
||||
|
||||
## Consequences
|
||||
Each released codec creates a row decoder with explicit `strict` or `recoverable` recovery. The decoder validates and emits one event or one codec-owned `SessionFormatEventRun` at a time through separate context methods. v0-to-v1 and v1-to-v2 implement both `transformEvent()` and `transformRun()`, so packed Assistant chunks can reach the folding edge without first becoming millions of ordinary events.
|
||||
|
||||
Reading event bodies with a newer build may durably add a higher generation. The exact old generation remains available, but the runtime thereafter selects the highest canonical filename; retention does not promise that an older build can safely downgrade or that the newer build will fall back when the successor is corrupt. A read-only filesystem reports an actionable migration failure instead of returning an in-memory current view that differs from disk.
|
||||
The v0-to-v1 edge preserves logical headers, sequence numbers, references, timestamps, and payloads except for bounded released-v0 normalizations. It translates the retired `steering/message` and `compact/*` event names, accepts a released `llm/retry` after its matching `step/end`, deterministically supplies a missing `llm/retry.retryId` per turn/step/provider/policy chain, and supplies one deterministic `compactionId` across a legacy compaction group that omitted it. The v1-to-v2 edge owns attempt folding and reference remapping, and emits only settled current events. It splits a legacy goal-sourced user message into `goal/change` plus the original model-visible message. It also inserts an interrupted `turn/end` for the bounded released restart in which an open turn with no open step is followed by a non-empty `next-turn` inbox splice and the next numbered `turn/start`.
|
||||
|
||||
JSONL publication uses POSIX hard-link creation plus directory sync, and Windows uses no-overwrite `MoveFileExW` with write-through. A competing writer that wins target creation is accepted only when the committed bytes exactly match. One process-local writer per Session is the supported concurrency model. A future per-Session cross-process lock can close the remaining source-check-to-publication race without changing the format edge interface.
|
||||
The catalog exposes one `createRestore()` operation for production, Worker, fixture, and replay callers. Recovery policy and final validation policy are chosen once at restore creation. Historical production uses recoverable source parsing with transformed-current validation; this validates the released current result after migration, while input that is already current receives only codec validation. Worker and fixture verification use strict parsing with full installed current restoration. A migration-stage or transformed-current validation refusal remains `SessionFormatUnsupportedMigrationError`; physical decoding failures remain corruption. Test support keeps only fixture-specific token and envelope materialization.
|
||||
|
||||
Retained generations are not a live-stream write-ahead log. A future optional WAL sidecar may preserve unfinished assistant streams across a hard crash. Explicit generation inspection or copying, retention tooling, compression conversion, and streamed whole-artifact transformation are separate features; automatic fallback and downgrade compatibility are not implied future work.
|
||||
### JSONL integration
|
||||
|
||||
This note supersedes the continue-only persistence rule and the deferred-chain status in [Session log versioning](2026-08-10-session-log-version-mechanism.md). That note remains the authority for when to bump the version and for ordinary equal-version `ignorable` event behavior.
|
||||
The JSONL provider scans frame boundaries once, reuses one Zstandard decoder, parses complete JSONL records incrementally, and feeds rows directly into the catalog restore. The outer loop yields at a bounded cadence; there is no per-frame `await` and no complete plaintext or source-row array.
|
||||
|
||||
Current encoding is record based. The provider serializes about 1 MiB of plaintext per main-thread slice, streams it through one Zstandard context with source-error propagation, writes compressed output in 4 MiB batches to an exclusively created same-directory temporary file, and syncs it before publication. A process-wide scheduler admits at most two full verification Workers and hands a released permit directly to the oldest waiter.
|
||||
|
||||
Cancellation is observed at the existing approximately 500 ms Decode yield boundary and the approximately 1 MiB encode yield boundary. A queued verifier removes its waiter when cancelled; an active verifier terminates its Worker and awaits exit before releasing the permit. This does not make the underlying file writes newly interruptible, and cancellation never rolls back a generation that has already been published.
|
||||
|
||||
This decision deliberately preserves the existing serial persistence lifecycle:
|
||||
|
||||
```text
|
||||
read/write open
|
||||
→ decode and migrate historical source
|
||||
→ encode and sync temporary current generation
|
||||
→ Worker verify
|
||||
→ recheck source
|
||||
→ publish without overwrite
|
||||
→ verify/reopen committed generation
|
||||
→ return handle
|
||||
```
|
||||
|
||||
Read-only preparation and write publication are not separated here. Both handle kinds wait for the current generation. That scheduling problem remains independently changeable without restoring the whole-artifact format API.
|
||||
|
||||
### Durable format and publication rules
|
||||
|
||||
Canonical filenames encode physical format generation: v0 is `session.jsonl[.zstd]` and positive generations use `session.vN.jsonl[.zstd]`. Migration never moves, replaces, or deletes a committed generation and writes only the final current target; intermediate versions exist only as stage state.
|
||||
|
||||
POSIX publication uses hard-link creation plus directory sync. Windows uses no-overwrite, write-through `MoveFileExW`. An existing target is accepted only when its verified migration prefix equals the staged bytes; any append tail belongs to current-generation reading rather than migration winner verification.
|
||||
|
||||
Existing write handles retain the process-local claim and kernel-backed cross-process `SessionWriteLease`. Header-only `stat` and `list` translate supported historical headers without opening the body or publishing a generation. Projection-cache records bind their fold to the Session header's format version so a cache row cannot bypass a cardinality-changing migration.
|
||||
|
||||
## Problem-to-solution mapping
|
||||
|
||||
| Whole-artifact problem | Implemented mechanism | Result |
|
||||
|---|---|---|
|
||||
| One asynchronous decode call per Zstandard frame | One reusable decoder; outer 500 ms scheduling cadence | Removes 317,540 async transitions |
|
||||
| Complete plaintext, string, and row arrays | Incremental JSONL parser and row decoder | Retains only one cross-chunk record fragment |
|
||||
| Complete event array between every edge | Context-connected stateful stages | No intermediate version event arrays |
|
||||
| Packed chunks expand before folding | `SessionFormatEventRun` plus `transformRun()` | 9.14 million source events need not materialize |
|
||||
| Whole-artifact snapshot and deep freeze at every edge | Stage-owned exclusive values and final validation | Removes repeated recursive copy/freeze |
|
||||
| One-shot migration functions hide state | Per-artifact stage classes from immutable declarations | State ownership and concurrency are explicit |
|
||||
| Bulk current encode builds whole strings and Buffers | Record encoder, 1 MiB input slices, 4 MiB write batches | Bounds allocation and main-thread slices |
|
||||
| Verification repeats on the main thread | At most two complete-generation Workers | Keeps verification CPU off the main thread |
|
||||
| Production and fixture migration use different APIs | Catalog `createRestore()` with explicit policies | One decoder/chain implementation |
|
||||
|
||||
## Verification
|
||||
|
||||
Release verification runs the committed Session-format corpus gate over every versioned persisted-or-projected `session*.jsonl` fixture under `snapshots/`, `packages/`, and `scripts/snapshots/python-sdk-single-exe/`. Fixture-only omitted envelopes and request-header tokens are materialized before the real static catalog; every fixture reaches the current v1 view through current restoration or historical migration. Released-v0 replay inputs remain suffixless, while fresh v1 writer outputs use `session.v1.jsonl` for a parent and `session.<ordinal>.v1.jsonl` for children. Record and refresh preserve every completed generation, including generations of a child role absent from a later run. Malformed historical fixtures are repaired at their source rather than admitted through path-dependent replay policy. The continuing gate discovers the corpus dynamically and fails every restoration refusal; separate assembled JSONL tests own exact physical-byte migration.
|
||||
### Benchmark input and meanings
|
||||
|
||||
Handle-integration verification runs the pure format, catalog, persistence-seam, and JSONL provider suites together: 420 tests cover both encodings, immutable publication races, header-only observation, read and write handles, migration refusal, append after migration, cancellation, and crash-tail behavior with per-file 100% statement, branch, function, and line coverage. Repository typecheck and lint, 113 keyless recorded-session replays with two declared skips, and 28 owner-local expected-output cases also pass on the merged master checkpoint.
|
||||
The benchmark uses Node v24.18.0 and one 116,228,655-byte v0 Zstandard log containing 317,540 frames and 454,151 physical rows. The old reader restores 9,143,111 expanded v0 events. Migration produces 72,784 current v2 events with artifact SHA-256 `fa16ff9472ca350595a3112c20a3db79655bc2673973469987ecaf2a57ebd17c`.
|
||||
|
||||
The assembled headless profile test stages `session.jsonl`, resumes it through the shipped composition, observes v1 before Session construction, verifies that the exact v0 bytes and inode remain while `session.v1.jsonl` appears, and proves the next append targets v1. JSONL contract tests exercise raw and Zstandard exclusive publication, torn-tail preservation, source changes, target collisions, future-highest refusal, revision-keyed parsed-log reuse, listing rescans, temporary cleanup, committed reopen, and current-format bypass.
|
||||
Runs use built artifacts under plain Node, one process per sample, and a 16 GB V8 heap limit. “Retained heap” is measured after forced GC while the restored Session remains live. Values below are three-run medians except the whole-artifact failure, which consistently cannot reach a handle.
|
||||
|
||||
### Physical Decode
|
||||
|
||||
| Data path | Decode time | Peak RSS | Scheduling |
|
||||
|---|---:|---:|---|
|
||||
| Pre-migration optimized reader | 1.553s | 916MB | One decoder; 2–3 outer yields |
|
||||
| Whole-artifact migration | 7.527s | 7,219MB | 317,540 async decoder calls |
|
||||
| Streaming Stage path | 1.467s | 908MB | One decoder; 2 outer yields |
|
||||
|
||||
### Historical-file cold open
|
||||
|
||||
| Version | Time to restored Session | CPU time | Peak RSS | Retained heap | Restored events | Outcome |
|
||||
|---|---:|---:|---:|---:|---:|---|
|
||||
| Pre-migration high-performance v0 reader | 4.594s | 6.048s | 2.720GB | 2.016GB | 9,143,111 | Reads v0; does not migrate |
|
||||
| Whole-artifact migration | >72.8s | — | ≥7.219GB during Decode | — | — | OOM before a handle |
|
||||
| Streaming Stage migration with serial publication | 6.241s | 8.493s | 2.107GB | 477MB | 72,784 | Publishes and opens v2 |
|
||||
|
||||
The old reader has lower one-time wall time because it performs no format conversion or durable publication. It also keeps the 9.14-million-event representation live. The Stage path pays encode and verification once, then retains the folded v2 state.
|
||||
|
||||
### Current-format cold open
|
||||
|
||||
| Version reading its current format | Time to restored Session | Peak RSS | Retained heap |
|
||||
|---|---:|---:|---:|
|
||||
| Old reader on v0 | 4.594s | 2.720GB | 2.016GB |
|
||||
| Whole-artifact-era reader on v2 | 1.273s | 1.107GB | 476MB |
|
||||
| Streaming Stage reader on v2 | 1.284s | 1.109GB | 476MB |
|
||||
|
||||
The current-v2 fast path remains performance-equivalent. The architectural change does not route current data through historical stages.
|
||||
|
||||
### Streaming serial migration breakdown
|
||||
|
||||
| Phase | Median |
|
||||
|---|---:|
|
||||
| Source Decode and migration | 2.784s |
|
||||
| Encode, write, and sync | 0.956s |
|
||||
| Full staged-file Worker verification | 1.415s |
|
||||
| Source recheck and no-overwrite publication | 0.106s |
|
||||
| Committed-prefix verification and header reopen | 0.046s |
|
||||
| Generation ensure-current total | 5.318s |
|
||||
| Final current decode observed by persistence | 0.620s |
|
||||
| Session restoration | 0.594s |
|
||||
| End-to-end restored Session | 6.241s |
|
||||
|
||||
The generation breakdown and end-to-end table come from separate instrumented runs, so rounded rows are not expected to sum exactly.
|
||||
|
||||
Format, catalog, edge, JSONL, fixture, replay, and built-Worker tests cover both encodings, packed runs, header-only classification, torn tails, migration refusal, deterministic legacy normalization, source changes, target collisions, write leases, and Worker failure.
|
||||
|
||||
## Consequences
|
||||
|
||||
At least one final current-event array remains necessary because Session restoration and Agent execution retain complete history. The Stage architecture removes full source and intermediate target arrays; it does not promise memory proportional to a page window.
|
||||
|
||||
Decoded scalar `assistant/chunk` rows receive envelope validation and final target validation, but their complete frozen-v1 source payload-member validation is deferred because that per-event check materially affects Decode and migration time on released logs. Packed Assistant runs remain strictly decoded. The scalar check must be restored only with performance evidence that preserves this migration path's measured behavior.
|
||||
|
||||
The serial persistence lifecycle still makes a read open wait for encode, verification, and publication. Separating logical readability from durable write readiness is a follow-up scheduling decision, not another format-pipeline rewrite.
|
||||
|
||||
Lower generations remain for operator inspection. Retention does not promise downgrade compatibility, automatic fallback, or that an older runtime can safely interpret a newer generation.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Migrate only on continuation** — leaves query, export, fork, and suffix consumers on old generations and duplicates restoration policy.
|
||||
- **Return a migrated in-memory view without persisting** — lets one process observe state that does not match the highest committed generation and postpones failure until a later writer.
|
||||
- **Persist every intermediate version** — consumes space and creates recovery states with no runtime consumer; only the source and final generation are durable.
|
||||
- **Let mounted event-owner plugins register migrations** — makes historical readability deployment-dependent; the static catalog must work before feature plugins mount.
|
||||
- **Reuse one filename for every current format and relocate its predecessor** — rejected because migration would move or overwrite committed evidence, require collision and retention rules, and make the filename disagree with the stored format. Canonical immutable generation names let discovery select the highest version directly.
|
||||
- **Optimize only Zstandard Decode** — restores physical Decode speed but leaves source rows, expanded events, snapshots, intermediate artifacts, and bulk encode in memory.
|
||||
- **Synchronous Generator stages** — retain execution frames and batches at each yield. Real-log measurements increased migration time and migrate-complete RSS from about 1.0 GB to about 1.2 GB.
|
||||
- **Return arrays from each stage** — preserves the old allocation, traversal, and flattening costs under a new name.
|
||||
- **Give each stage an internal output queue** — adds drain, EOF, and error ownership while still retaining intermediate values.
|
||||
- **Inject an emit callback through constructors** — forces reverse construction or a partially connected lifecycle. Passing a context to operations keeps stage construction independent of downstream wiring.
|
||||
- **Share stateful codec instances globally** — would mix pending attempts, mappings, and counters across concurrent Session restores.
|
||||
- **Persist every intermediate format version** — creates durable states with no runtime consumer; only the exact source and final current generation are needed.
|
||||
- **Let mounted plugins register migrations** — makes historical readability deployment dependent. The static catalog must restore released formats before feature plugins mount.
|
||||
+160
-25
@@ -1,4 +1,4 @@
|
||||
# Agent Note: 已发布 Session 格式在读取正文时通过相邻纯迁移边升级
|
||||
# Agent Note: 已发布 Session 格式通过有状态流式 Stage 迁移
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -6,52 +6,187 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
Session 格式 v0 已随 alpha 版本发布，因此结构化 writer 变更不能再把已有 JSONL 当作可丢弃的预发布状态。已存储事件正文通过读或写 `SessionHandle` 到达恢复、查询、导出、分叉与继续路径。只迁移一个消费方会让调用方看到不同的逻辑 generation，或只在后续 writer 到达旧文件时失败。
|
||||
Session 格式 v0 已随 alpha 版本发布，因此结构化 writer 变更不能再把已有 JSONL 当作可丢弃的预发布状态。第一版 whole-artifact migration 让这些日志在语义上可迁移，但它的数据模型会让一份 116 MB 真实 Session 在返回 handle 前耗尽 16 GB Node 进程。
|
||||
|
||||
迁移必须保留精确源路径、字节与 inode，包括撕裂的物理尾部，同时为每个已发布格式提供一个无歧义的规范文件名。普通 JSONL 与 Zstandard 是同一逻辑格式的编码选择，不能产生两套并行迁移实现。
|
||||
### Whole-artifact 性能问题
|
||||
|
||||
- Zstandard 输入包含 317,540 个 frame，每个 frame 都单独执行一次异步解压。实现先保留全部 plaintext frame，再在 JSON 解析前统一拼接，因此创建了同等数量的 Promise、线程池与 native Decode 调度。
|
||||
- Physical Decode 会在同一请求的重叠阶段物化完整 plaintext Buffer、完整字符串、全部 JSONL row、展开后的 source events、迁移后的 target events、编码后的 target rows、拼接后的目标字符串与目标 physical Buffer。
|
||||
- 每个 codec 与 migration edge 都会调用 `snapshotSessionFormatJson()` 或 `snapshotSessionFormatArtifact()`，在相邻迁移前后递归复制并 deep freeze 完整 header、row、payload 与 event array。
|
||||
- 已发布的 packed Assistant chunk 会先展开成约 914 万个 v0/v1 逻辑事件，再由 v1-to-v2 折叠成 72,784 个 current events。Whole-artifact API 要求两种表示和 old-to-new seq map 同时存活。
|
||||
- Encode 会在内存中构造完整 JSONL 与压缩输出。成功路径随后 Decode staged target、Decode committed target，并由 persistence 再 Decode 一次以创建业务对象；它还会完整重读 source 以比较 fingerprint。
|
||||
- 逐 frame `await` 没有形成有意义的有界调度。迁移前的高性能 reader 会复用一个同步 decoder，只由外层循环约每 500 ms yield 一次，从而避免数十万次异步切换。
|
||||
|
||||
### 既有接口使单点优化无法组合
|
||||
|
||||
`SessionFormatCodec` 以完整数组 Decode 与 Encode；每条相邻 `SessionFormatMigration` 接收并返回完整 `SessionFormatArtifact`；compiled chain 只能把已经物化的 artifact 交给下一条 edge。因此即使 physical decoder 单点变快，下游仍会重新创建 source-row array、expanded-event array、逐 edge snapshot 与 target-row array。
|
||||
|
||||
Migration 实际有状态，但 API 把它们表现为一次性函数。v0-to-v1 需要跟踪 message 与 retry identity；v1-to-v2 需要暂存一个尚未结算的 Assistant attempt、被它阻塞的后续事件，并维护 old-to-new seq 引用。把这些状态隐藏在 closure 或 push/finish helper object 中，会让运行结构与静态声明分离，也让 production、Worker verify、fixture 与 replay 使用不同入口。
|
||||
|
||||
## 决策
|
||||
|
||||
`SESSION_FORMAT_VERSION` 是单调递增的当前 writer 整数。每个相邻 `vN -> vN+1` 转换由一个与 profile 无关的纯包负责。`@deepseek-ai/dsh-session-format` 只提供无损快照、唯一且无缺口的规划、仅 header 转换与整产物组合；`@deepseek-ai/dsh-session-format-catalog` 静态导入完整链，不依赖已挂载的 Cordis 插件。历史 codec 和归一化器位于具名迁移边包中，而当前 Session 与持久化代码只接纳最新逻辑类型。
|
||||
Session format 包采用有状态同步 Stage API。静态 migration declaration 描述一条相邻版本边，并为每次 artifact restore 创建新的 stage。Stage 拥有该 artifact 的可变状态；不同 Session 之间绝不共享 stage instance。
|
||||
|
||||
每条迁移边都会冻结严格的源与目标语义，其目标物理 codec 则保持词汇中立，使普通事件增长可以留在同一格式版本内。目录通过已安装的 peer `@deepseek-ai/dsh-session` 及其当前 `KNOWN_SESSION_EVENT_TYPES` 还原最终代，避免冻结的历史迁移边反过来成为当前词汇 owner。
|
||||
### Stage 与 Context 协议
|
||||
|
||||
JSONL provider 在 `open` 为已存储 Session 返回句柄前完成 ensure-current 工作。它选择最高规范 generation、迁移受支持的历史正文，并从同一物理快照解码当前结果；公开 `SessionPersistence` 与 `SessionHandle` 接口不包含迁移操作。仅 header 的 `stat` 与 `list` 会重新扫描 Session 目录，在内存中转换受支持的历史 header，且绝不发布后继。`create` 独立于 header 可读性检查规范文件名，因此每个现有 generation 都会占用其 Session id。
|
||||
```text
|
||||
interface SessionFormatMigrationContext {
|
||||
emitEvent(event: SessionFormatEvent): void
|
||||
emitRun(run: SessionFormatEventRun): void
|
||||
}
|
||||
|
||||
取消属于提供信号的 `open`、`stat` 或 `list` 调用。发现、稳定读取、解码与发布前检查都会观察该信号；不可变后继一旦发布且其目录项已经同步，后续取消不会删除已提交 generation。
|
||||
interface SessionFormatMigrationStage {
|
||||
readonly headerInheritedEventCount?: number
|
||||
transformEvent(
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void
|
||||
transformRun(
|
||||
run: SessionFormatEventRun,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void
|
||||
finish(context: SessionFormatMigrationContext): number
|
||||
}
|
||||
```
|
||||
|
||||
配置的 JSONL 编码拥有一个完整后缀：`.jsonl` 或 `.jsonl.zstd`。迁移读取稳定的精确源，解码可恢复逻辑前缀，在内存中组合全部必需迁移边，只为最终目标校验并同步同目录临时 stage，重新检查源 fingerprint，以不覆盖方式发布此前不存在的目标，同步 namespace，并在返回句柄前通过当前格式校验重新打开。源永不移动或改变；只有可丢弃临时 stage 可以被移动、链接或移除。迁移不会合成中断轮次事件：agent-loop 通过写句柄追加这些修复，而只读查询路径在内存中补齐它们。
|
||||
`SessionFormatMigrationContext.emitEvent()` 与 `emitRun()` 都是同步操作。Producer 会声明其发出单个事件还是紧凑 run，因此热路径不会根据已解析文件对象的属性推断类别。调度归 caller 所有，context 在每次调用时传入，而不是把 callback 注入 stage constructor。一个输入可以输出零个、一个或多个值，不需要分配临时返回数组，也不需要 stage 内部保留输出队列。
|
||||
|
||||
规范文件名编码物理格式 generation：v0 是 `session.jsonl` 或 `session.jsonl.zstd`；每个正 generation 都是小写 `session.vN.jsonl` 或 `session.vN.jsonl.zstd`。`dsh-session-format` 拥有原始 basename 规则（`sessionFormatLogFilename`、`parseSessionFormatLogFilename`）；JSONL provider、session-log 导出归档与 recorded-session fixture 只追加压缩后缀。发布绝不重命名、替换或删除已提交 generation 路径。目标已经存在时，只有它是普通当前格式文件且字节与预期完全相同时才接受；其他目标都会拒绝。低 generation 为 operator 检查或显式复制而保留，但普通 runtime 操作选择数值最高的规范名称，绝不把保留的前任当作自动 fallback、restore 或 downgrade 支持。
|
||||
`SessionFormatMigration` 继续作为 immutable declaration，声明版本号、header migration、target-header validation 与 `createStage()`。`CompiledSessionFormatChain` 只校验一次唯一、无缺口的 edge 序列，按 source-to-target 顺序创建每次 artifact 独占的 stage，再按反方向用 context 连接它们。`finish()` 按 source-to-target 顺序关闭 stage，使每一级都能在下游关闭前发出尾部数据。
|
||||
|
||||
当前格式快速路径从一个稳定源快照分类 header，不调用历史 converter，不写 generation，并把该快照交给当前格式解码，而不再次读取文件。解码日志进入现有按 revision 为键的有界 memo，供紧接的观察到恢复交接复用，而 `stat` 与 `list` 会有意重新扫描。多条迁移边保持原 generation 不变，并只发布最终目标；中间版本只存在于内存。源 fingerprint 重新检查会在内容变化时重启迁移，排他目标发布只在竞争胜者字节完全相同时接受它。跨进程 append 隔离不在此保证内。
|
||||
```text
|
||||
JSONL record
|
||||
→ released physical row decoder
|
||||
→ v0-to-v1 stage
|
||||
→ v1-to-v2 stage
|
||||
→ current event collector
|
||||
```
|
||||
|
||||
第一条迁移边 `@deepseek-ai/dsh-session-format-v0-to-v1` 有意保持恒等形态：除版本和 v0 已接纳的有限历史归一化外，它保留逻辑 header、事件、序号、引用、时间戳、payload 与已配置的压缩选择。精确的 `session.jsonl[.zstd]` 源保持字节与 inode 相同，当前 writer 则编码新的 `session.v1.jsonl[.zstd]` 后继。这样可在出现改变基数的格式前先验证完整发布生命周期。
|
||||
Chain 中不存在 `flatMap`、spread expansion、中间 event array 或 scheduler。只有在每个 migration stage 都已获得直接消费 compact run 的机会后，最终 event collector 才会展开它。
|
||||
|
||||
投影缓存记录把自己的折叠结果绑定到 Session header 的 `formatVersion`。`session_projcache` v7 reader 可以在结构上载入前代 domain 记录，但缺少格式代的记录不能播种当前 Session；权威日志会重新折叠它，下一次检查点写入完整的当前 identity。这样，任何在有界规范化或基数变化边之前产生的缓存行都不能绕过该迁移。
|
||||
### Physical codec 与 packed run
|
||||
|
||||
## 后果
|
||||
每个 released codec 会用显式 `strict` 或 `recoverable` 策略创建 row decoder。Decoder 每次通过不同的 context 方法校验并 emit 一个 event 或 codec-owned `SessionFormatEventRun`。v0-to-v1 与 v1-to-v2 都实现 `transformEvent()` 和 `transformRun()`，因此 packed Assistant chunk 可以直接到达 folding edge，无需先变成数百万个普通事件。
|
||||
|
||||
较新 build 读取事件正文时可能持久增加一个更高 generation。精确旧 generation 仍然可用，但 runtime 此后选择最高规范文件名；保留不承诺旧 build 能安全 downgrade，也不保证新 build 在后继损坏时 fallback。只读文件系统会报告可操作的迁移失败，而不会返回与磁盘不一致的内存当前视图。
|
||||
v0-to-v1 除了有限的 released-v0 归一化外，会保留逻辑 header、seq、引用、时间戳与 payload。它转换已移除的 `steering/message` 与 `compact/*` 事件名称，接受出现在对应 `step/end` 之后的已发布 `llm/retry`，按 turn／step／provider／policy chain 为缺失的 `llm/retry.retryId` 确定性补值，并为省略 id 的旧 compaction group 确定性补充同一个 `compactionId`。v1-to-v2 负责 attempt folding 与引用重写，并且只 emit 已结算的 current event。它会把旧的 goal 来源 user message 拆成 `goal/change` 与原本的模型可见 message。它还会为一种有限的已发布 restart 插入 interrupted `turn/end`：一个没有 open step 的 open turn 后出现非空 `next-turn` inbox splice，随后直接开始编号连续的下一轮。
|
||||
|
||||
JSONL 发布在 POSIX 上使用硬链接创建与目录同步，在 Windows 上使用 write-through 且不覆盖的 `MoveFileExW`。竞争 writer 已先创建目标时，只有已提交字节完全匹配才接受。每个 Session 只支持一个进程内 writer。未来逐 Session 跨进程锁可以关闭剩余的源检查到发布竞态，而无需改变格式迁移边接口。
|
||||
Catalog 为 production、Worker、fixture 与 replay 暴露同一个 `createRestore()`。Recovery policy 与最终 validation policy 在 restore 创建时一次确定。Historical production 使用 recoverable source parsing 与 transformed-current validation；这种策略会在迁移后校验已发布 current 结果，而已经是 current 的输入只接受 codec 校验。Worker 与 fixture verification 使用 strict parsing 与已安装 current 格式的完整 restoration。Migration stage 或 transformed-current validation 的拒绝会保持为 `SessionFormatUnsupportedMigrationError`；物理解码失败仍是 corruption。Test support 只保留 fixture 自身需要的 token 和 envelope materialization。
|
||||
|
||||
保留的 generation 不是实时流 WAL。未来可选 WAL sidecar 可以在硬崩溃间保留未完成 assistant 流。显式 generation 检查或复制、保留策略工具、压缩转换与流式整产物转换都是独立功能；自动 fallback 与 downgrade compatibility 并非隐含 future work。
|
||||
### JSONL 串联
|
||||
|
||||
本记录取代 [Session 日志版本机制](2026-08-10-session-log-version-mechanism.zh.md) 中仅在继续时持久化和迁移链仍推迟的规则。原记录继续负责何时递增版本，以及普通同版本 `ignorable` 事件行为。
|
||||
JSONL provider 只扫描一次 frame boundary，复用一个 Zstandard decoder，增量解析完整 JSONL record，并把 row 直接送入 catalog restore。外层循环按有界 cadence yield；不存在逐 frame `await`、完整 plaintext 或 source-row array。
|
||||
|
||||
Current encode 以单条 record 为单位。Provider 在主线程每个 slice 序列化约 1 MiB plaintext，通过一个会传播 source error 的 Zstandard context 流式压缩，以 4 MiB batch 写入同目录排他创建的临时文件，并在 publication 前 sync。进程级 scheduler 最多允许两个完整 verification Worker 并行，并把释放的 permit 直接交给最早的 waiter。
|
||||
|
||||
Cancellation 会在现有的约 500 ms Decode yield 边界和约 1 MiB encode yield 边界被观察到。排队 verifier 在取消时会移除自己的 waiter；活动 verifier 会终止 Worker，并等待其退出后再释放 permit。该行为不会让底层文件写入新增可中断能力，取消也绝不会回滚已经发布的 generation。
|
||||
|
||||
本决策有意保持既有串行 persistence lifecycle：
|
||||
|
||||
```text
|
||||
read/write open
|
||||
→ decode and migrate historical source
|
||||
→ encode and sync temporary current generation
|
||||
→ Worker verify
|
||||
→ recheck source
|
||||
→ publish without overwrite
|
||||
→ verify/reopen committed generation
|
||||
→ return handle
|
||||
```
|
||||
|
||||
这里不拆分 read-only preparation 与 write publication。两种 handle 都会等待 current generation 完成。该调度问题可以独立调整，不需要恢复 whole-artifact format API。
|
||||
|
||||
### Durable format 与 publication 规则
|
||||
|
||||
规范文件名编码 physical format generation：v0 使用 `session.jsonl[.zstd]`，正 generation 使用 `session.vN.jsonl[.zstd]`。Migration 不会移动、覆盖或删除任何 committed generation，并且只写最终 current target；中间版本只存在于 stage state。
|
||||
|
||||
POSIX publication 使用 hard-link creation 加目录 sync；Windows 使用 no-overwrite、write-through 的 `MoveFileExW`。已有 target 只有在其已校验 migration prefix 等于 staged bytes 时才会被接受；任何 append tail 都属于 current-generation reader，而不是 migration winner verification。
|
||||
|
||||
既有 write handle 继续使用进程内 claim 与内核支持的跨进程 `SessionWriteLease`。仅 header 的 `stat` 与 `list` 可以转换受支持的历史 header，但不打开 body，也不发布 generation。Projection-cache record 会把 fold 绑定到 Session header 的 format version，使 cache row 不能绕过改变 event 基数的 migration。
|
||||
|
||||
## 问题与方案对照
|
||||
|
||||
| Whole-artifact 问题 | 实现机制 | 结果 |
|
||||
|---|---|---|
|
||||
| 每个 Zstandard frame 单独异步 Decode | 一个可复用 decoder；外层 500 ms 调度 cadence | 删除 317,540 次异步切换 |
|
||||
| 完整 plaintext、string 与 row array | 增量 JSONL parser 与 row decoder | 只保留一条跨 chunk 残行 |
|
||||
| 每条 edge 之间都形成完整 event array | Context 直连的有状态 stage | 不保留中间版本 event array |
|
||||
| Packed chunk 在 folding 前完整展开 | `SessionFormatEventRun` 与 `transformRun()` | 无需物化 914 万 source events |
|
||||
| 每条 edge 都 whole-artifact snapshot/deep freeze | Stage-owned 独占值与最终 validation | 删除重复递归复制与冻结 |
|
||||
| One-shot migration function 隐藏状态 | Immutable declaration 创建每次 artifact 独占的 stage class | 状态 ownership 与并发关系显式化 |
|
||||
| Bulk current encode 构造完整 string/Buffer | 单条 record encoder、1 MiB input slice、4 MiB write batch | 限制分配与主线程 slice |
|
||||
| 主线程重复执行完整 verification | 最多两个 complete-generation Worker | Verification CPU 不占用主线程 |
|
||||
| Production 与 fixture 使用不同 migration API | Catalog `createRestore()` 加显式 policy | 只保留一套 decoder/chain 实现 |
|
||||
|
||||
## 验证
|
||||
|
||||
发布验证针对 `snapshots/`、`packages/` 与 `scripts/snapshots/python-sdk-single-exe/` 下每个带版本、来自持久化或投影的 `session*.jsonl` fixture 运行已提交 Session 格式语料门禁。fixture 专用的缺失信封与 request-header token 会先被实体化，再进入真实静态 catalog；每个 fixture 都会通过当前格式 restore 或历史迁移得到当前 v1 视图。Released-v0 replay 输入保持无后缀，而新鲜 v1 writer 输出对 parent 使用 `session.v1.jsonl`、对 child 使用 `session.<ordinal>.v1.jsonl`。Record 与 refresh 会保留每个已完成 generation，包括后续运行不再产生的 child role generation。Malformed 历史 fixture 在来源处修复，不通过依赖路径的 replay 策略准入。持续运行的门禁会动态发现语料，并拒绝每个 restore failure；独立组装式 JSONL 测试负责精确物理字节迁移。
|
||||
### Benchmark 输入与口径
|
||||
|
||||
句柄集成验证会一起运行纯格式、catalog、持久化 seam 与 JSONL provider 测试套件：420 个测试覆盖两种编码、不可变发布竞态、仅 header 观察、读写句柄、迁移拒绝、迁移后 append、取消与崩溃尾部行为，并达到逐文件 100% statement、branch、function 与 line coverage。仓库 typecheck 与 lint、含两个已声明 skip 的 113 个无密钥 recorded-session replay，以及 28 个 owner-local expected-output case 也都在合并 master 的 checkpoint 上通过。
|
||||
Benchmark 使用 Node v24.18.0 和一份 116,228,655-byte 的 v0 Zstandard 日志，其中包含 317,540 个 frame 与 454,151 个 physical row。老 reader 会恢复 9,143,111 个展开后的 v0 event；migration 会生成 72,784 个 current v2 event，artifact SHA-256 为 `fa16ff9472ca350595a3112c20a3db79655bc2673973469987ecaf2a57ebd17c`。
|
||||
|
||||
组装后的 headless profile 测试会暂存 `session.jsonl`，通过随附组合恢复它，在构造 Session 前观察到 v1，验证精确 v0 字节与 inode 保持不变而 `session.v1.jsonl` 出现，并证明下一次 append 以 v1 为目标。JSONL 约定测试覆盖 raw 与 Zstandard 排他发布、撕裂尾部保留、源变化、目标冲突、最高未来版本拒绝、按 revision 复用已解析日志、列表重新扫描、临时文件清理、已提交重开与当前格式直通。
|
||||
所有样本均通过 plain Node 运行 build artifact，每个样本使用独立进程，V8 heap limit 为 16 GB。“Retained heap”表示 restored Session 仍存活时强制 GC 后的 heap。除无法得到 handle 的 whole-artifact 失败外，下表使用三次运行中位数。
|
||||
|
||||
### Physical Decode
|
||||
|
||||
| 数据路径 | Decode 耗时 | 峰值 RSS | 调度 |
|
||||
|---|---:|---:|---|
|
||||
| Migration 前的高性能 reader | 1.553s | 916MB | 一个 decoder；外层 yield 2–3 次 |
|
||||
| Whole-artifact migration | 7.527s | 7,219MB | 317,540 次 async decoder 调用 |
|
||||
| Streaming Stage 路径 | 1.467s | 908MB | 一个 decoder；外层 yield 2 次 |
|
||||
|
||||
### 历史文件首次冷打开
|
||||
|
||||
| 版本 | Session restore 完成 | CPU 时间 | 峰值 RSS | Retained heap | Restore event 数 | 结果 |
|
||||
|---|---:|---:|---:|---:|---:|---|
|
||||
| Migration 前的高性能 v0 reader | 4.594s | 6.048s | 2.720GB | 2.016GB | 9,143,111 | 读取 v0，不迁移 |
|
||||
| Whole-artifact migration | >72.8s | — | Decode 阶段已 ≥7.219GB | — | — | 返回 handle 前 OOM |
|
||||
| Streaming Stage migration + 串行 publication | 6.241s | 8.493s | 2.107GB | 477MB | 72,784 | 发布并打开 v2 |
|
||||
|
||||
老 reader 的一次性 wall time 更低，因为它不做格式转换和 durable publication；同时它会常驻 914 万 event 的表示。Stage 路径只多支付一次 encode 与 verification，随后保留折叠后的 v2 state。
|
||||
|
||||
### Current-format 冷打开
|
||||
|
||||
| 版本读取自己的 current format | Session restore 完成 | 峰值 RSS | Retained heap |
|
||||
|---|---:|---:|---:|
|
||||
| 老 reader 读取 v0 | 4.594s | 2.720GB | 2.016GB |
|
||||
| Whole-artifact 时代 reader 读取 v2 | 1.273s | 1.107GB | 476MB |
|
||||
| Streaming Stage reader 读取 v2 | 1.284s | 1.109GB | 476MB |
|
||||
|
||||
Current-v2 快路径保持性能等价。架构改造不会让 current data 进入 historical stage。
|
||||
|
||||
### Streaming 串行 migration 分段
|
||||
|
||||
| 阶段 | 中位耗时 |
|
||||
|---|---:|
|
||||
| Source Decode + migration | 2.784s |
|
||||
| Encode + write + sync | 0.956s |
|
||||
| staged 文件完整 Worker verify | 1.415s |
|
||||
| Source recheck + no-overwrite publication | 0.106s |
|
||||
| committed-prefix verify + header reopen | 0.046s |
|
||||
| Generation ensure-current 总计 | 5.318s |
|
||||
| Persistence 观察到的最终 current Decode | 0.620s |
|
||||
| Session restore | 0.594s |
|
||||
| 端到端 Session restore 完成 | 6.241s |
|
||||
|
||||
Generation 分段与端到端数据来自不同 instrumented run，因此四舍五入后的各行不要求精确相加。
|
||||
|
||||
Format、catalog、edge、JSONL、fixture、replay 与 built-Worker 测试覆盖两种编码、packed run、仅 header 分类、torn tail、migration refusal、确定性 legacy normalization、source change、target collision、write lease 与 Worker failure。
|
||||
|
||||
## 后果
|
||||
|
||||
最终 current-event array 仍然不可消除，因为 Session restore 与 Agent 执行需要完整历史。Stage 架构删除完整 source 与中间 target array，但不承诺内存与分页窗口大小成正比。
|
||||
|
||||
解码后的单条 `assistant/chunk` 会接受 envelope 校验与最终 target 校验，但其完整冻结 v1 source payload 成员校验仍处于延期状态，因为这项逐事件检查会显著影响已发布日志的 Decode 与 migration 耗时。Packed Assistant run 仍接受严格解码。只有性能证据表明不会破坏该迁移路径的已测表现时，才能恢复单条 chunk 校验。
|
||||
|
||||
串行 persistence lifecycle 仍会让 read open 等待 encode、verification 与 publication。把逻辑 readable 与 durable writable 分开属于后续调度决策，不需要再次改写 format pipeline。
|
||||
|
||||
低 generation 为 operator 检查而保留。Retention 不承诺 downgrade compatibility、automatic fallback，也不保证旧 runtime 能安全理解新 generation。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **只在继续时迁移**——让查询、导出、分叉与后缀消费者停留在旧代际，并重复恢复策略。
|
||||
- **返回迁移后的内存视图但不持久化**——让进程观察到与最高已提交 generation 不一致的状态，并把失败推迟到后续 writer。
|
||||
- **持久化每个中间版本**——消耗空间并产生没有 runtime 消费者的恢复状态；只有源与最终代际应持久。
|
||||
- **让已挂载事件 owner 插件注册迁移**——使历史可读性依赖部署；静态 catalog 必须在功能插件挂载前工作。
|
||||
- **让每个当前格式复用同一个文件名并迁走前任**——不予采用，因为迁移会移动或覆盖已提交证据，需要冲突与保留规则，并让文件名与存储格式不一致。规范不可变 generation 名让发现流程直接选择最高版本。
|
||||
- **只优化 Zstandard Decode**——可以恢复 physical Decode 速度，但 source rows、expanded events、snapshot、intermediate artifact 与 bulk encode 仍会留在内存中。
|
||||
- **同步 Generator stage**——每个 yield 都会保留执行帧与 batch。真实日志测量使 migration 更慢，并让 migrate-complete RSS 从约 1.0 GB 增长到约 1.2 GB。
|
||||
- **每个 stage 返回数组**——只是给旧的 allocation、遍历与 flattening cost 换了名字。
|
||||
- **Stage 内部输出队列**——增加 drain、EOF 与 error ownership，同时仍会保留中间值。
|
||||
- **通过 constructor 注入 emit callback**——迫使 chain 反向构建或引入 partially connected lifecycle。操作时传 context 可以让 stage construction 不依赖下游 wiring。
|
||||
- **全局复用有状态 codec instance**——会让不同 Session 的 pending attempt、mapping 与 counter 相互污染。
|
||||
- **持久化每个中间格式版本**——产生没有 runtime consumer 的 durable state；只需要精确 source 与最终 current generation。
|
||||
- **让 mounted plugin 注册 migration**——使历史可读性依赖部署。Static catalog 必须在 feature plugin 挂载前恢复已发布格式。
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-01-v2-embedded-assistant-streams.md
|
||||
2026-09-01-v2-embedded-assistant-streams.md: c9d7a66481d4258de8f9f7abdaa12d25f5217510
|
||||
2026-09-01-v2-embedded-assistant-streams.zh.md: ff4caceb68c34e470d2dd81fc9cf214555ad676e
|
||||
2026-09-01-v2-embedded-assistant-streams.md: a2ed4e49e5ea19f13cba00cfd85f8d8d73dc375c
|
||||
2026-09-01-v2-embedded-assistant-streams.zh.md: b6ae4a29ff794bc6bbbd9fe1fe7c7752f16f469f
|
||||
@@ -27,7 +27,9 @@ The current v2 validator requires the embedded stream to reproduce a non-empty `
|
||||
|
||||
`agent/assistant-stream` publishes process-local start, transient chunk, and end frames. The loop appends the complete `assistant/message` or `assistant/attempt` before a committed end frame names its type and sequence. An abandoned end has no settlement.
|
||||
|
||||
The Web follow adapter opts into these process-local frames and adds the last durable sequence observed at each start. It presents chunks as Client-only `assistant/live-chunk` updates between durable cursors, stages only a later matching settlement until the committed end, and reopens follow on a revision gap. A committed end publishes a named settlement delta that removes the attempt's transient matches, adds the durable entry, and replays only affected Conversation Contexts; an abandoned end publishes the same delta without an entry. A reconnect baseline carries the active attempt's durable start cursor and compact prefix. Paged history, replay, telemetry, token accounting, and cold UI assembly read the durable embedded stream rather than the live frames.
|
||||
The Web follow adapter opts into these process-local frames and adds the last durable sequence observed at each start. It presents chunks as Client-only `assistant/live-chunk` updates between durable cursors, stages only a later matching settlement until the committed end, and reopens follow on a revision gap. A committed end publishes a named settlement delta that removes the attempt's transient matches, adds the durable entry, and replays only affected Conversation Contexts; an abandoned end publishes the same delta without an entry. A reconnect baseline carries the active attempt's durable start cursor and compact prefix.
|
||||
|
||||
The Client event source passes durable settlements through unchanged. The Chat and Trajectory Assistant nodes fold `assistant/live-chunk` while an attempt is active, build settled output directly from `assistant/message`, and do not replay an `assistant/attempt` stream for presentation. Cold settled presentation therefore does not reconstruct per-token timing; other consumers may expand the durable stream when they require its exact evidence.
|
||||
|
||||
### Released v1 to v2 migration
|
||||
|
||||
@@ -49,7 +51,7 @@ The compact-stream tests pin exact accumulation and expansion for text, reasonin
|
||||
|
||||
The pre-merge performance acceptance measured static catalog-routing overhead against direct released-v2 restoration of the same already parsed physical rows across three runs, 100 warmup pairs, and 600 measured pairs; it did not compare v1 with v2 or time backend I/O. Every pooled median and p95 regression stayed within the 5% budget, with a worst p95 regression of 3.150%.
|
||||
|
||||
Agent-loop tests pin durable-before-end ordering, interrupted visible prefixes, failed and retry attempts, abandonment, usage, and replay metadata. Session Controller and Conversation tests pin live transient display, reconnect baselines, committed settlement release, history replay, Chat and Trajectory parity, while TypeScript and Python SDK snapshots pin the external event representation.
|
||||
Agent-loop tests pin durable-before-end ordering, interrupted visible prefixes, failed and retry attempts, abandonment, usage, and replay metadata. Session Controller and Conversation tests pin live transient display, reconnect baselines, committed settlement release, and history replay. Chat and Trajectory tests pin live partial presentation and direct final-message projection, while TypeScript and Python SDK snapshots pin the external event representation.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -59,13 +61,15 @@ Agent-loop tests pin durable-before-end ordering, interrupted visible prefixes,
|
||||
|
||||
**Carry packed chunk rows through the history API.** This reduces wire and Client work for v1 but gives the Client a second event vocabulary and keeps transport coupled to token-row cardinality. The current API carries scalar durable settlements plus a separate live transient stream.
|
||||
|
||||
**Strip embedded streams in Session Controller.** This reduces retained Client memory but creates a second durable event type and makes a transport-facing owner decide which evidence presentation consumers need. The measured bottleneck is repeated expansion, so each UI consumer decides whether to inspect the unchanged settlement.
|
||||
|
||||
**Store the stream in a sidecar or replay-only fixture.** This splits one attempt's message and evidence across durability owners and cannot give ordinary resumed sessions the same failed-output and timing facts. The settlement is the atomic owner.
|
||||
|
||||
**Redirect references from consumed chunks to their settlement.** A chunk and an attempt settlement are not interchangeable facts. Refusal prevents a migration from silently changing the meaning of plugin-owned references.
|
||||
|
||||
## Consequences
|
||||
|
||||
Current logs, telemetry, history pages, and cold Client assembly scale by model attempts rather than token chunks while retaining exact stream evidence inside each settlement. Live presentation remains incremental and intentionally process-local.
|
||||
Current logs, telemetry, and history pages scale by model attempts rather than token chunks while retaining exact stream evidence inside each settlement. The Client event window retains that compact evidence, but the Chat and Trajectory Assistant nodes do not expand settled streams into per-delta objects. Live presentation remains incremental and intentionally process-local.
|
||||
|
||||
Unlike v1 top-level chunks, which the buffered persistence writer could flush before an attempt ended, v2 has no durable attempt evidence until settlement. A hard process or host loss before settlement discards the complete in-flight stream; `agent/assistant-stream` is not a write-ahead log. This tradeoff avoids a second durability owner for live output.
|
||||
|
||||
|
||||
+7
-3
@@ -27,7 +27,9 @@ Session format v2 没有顶层 `assistant/chunk` 事件。每个模型 attempt
|
||||
|
||||
`agent/assistant-stream` 发布进程本地 start、瞬态 chunk 与 end frame。loop 会在 committed end frame 命名其类型和序号前追加完整的 `assistant/message` 或 `assistant/attempt`。abandoned end 没有 settlement。
|
||||
|
||||
Web follow adapter 显式选择接收这些进程本地 frame，并为每个 start 补充当时观察到的最后一个持久序号。它把 chunk 呈现为持久 cursor 之间的 Client-only `assistant/live-chunk` update，只暂存 start 之后匹配的 settlement，并在 revision 缺口时重新打开 follow。committed end 会发布具名 settlement delta，删除该 attempt 的 transient match、加入持久 entry，并只重放受影响的 Conversation Context；abandoned end 会发布不含 entry 的同类 delta。重连 baseline 携带活跃 attempt 的持久起始 cursor 与紧凑前缀。分页历史、replay、遥测、token 记账与冷 UI 组装读取持久嵌入式 stream，而不是 live frame。
|
||||
Web follow adapter 显式选择接收这些进程本地 frame，并为每个 start 补充当时观察到的最后一个持久序号。它把 chunk 呈现为持久 cursor 之间的 Client-only `assistant/live-chunk` update，只暂存 start 之后匹配的 settlement，并在 revision 缺口时重新打开 follow。committed end 会发布具名 settlement delta，删除该 attempt 的 transient match、加入持久 entry，并只重放受影响的 Conversation Context；abandoned end 会发布不含 entry 的同类 delta。重连 baseline 携带活跃 attempt 的持久起始 cursor 与紧凑前缀。
|
||||
|
||||
Client event source 原样传递持久 settlement。Chat 与 Trajectory 的 Assistant node 在 attempt 活跃期间折叠 `assistant/live-chunk`，直接从 `assistant/message` 构建 settled output，并且不为展示重放 `assistant/attempt` stream。因此冷恢复的 settled presentation 不会重建逐 token timing；其他消费方需要精确证据时仍可展开持久 stream。
|
||||
|
||||
### 已发布 v1 到 v2 迁移
|
||||
|
||||
@@ -49,7 +51,7 @@ Generation 选择与发布遵循[已发布 Session 迁移决策](2026-08-31-rele
|
||||
|
||||
合并前的 performance acceptance 在三轮、100 组 warmup pair 与 600 组 measured pair 下，针对同一批已经解析的物理 row，把静态 catalog routing 与直接 released-v2 restoration 比较；它不比较 v1 与 v2，也不计入 backend I/O。每个 pooled median 与 p95 regression 都保持在 5% 预算以内，最差 p95 regression 为 3.150%。
|
||||
|
||||
Agent-loop 测试固定先持久后 end 的顺序、中断的可见前缀、失败与重试 attempt、abandonment、usage 与 replay metadata。Session Controller 与 Conversation 测试固定实时瞬态显示、重连 baseline、committed settlement 发布、历史回放以及 Chat 与 Trajectory 一致性；TypeScript 与 Python SDK snapshot 固定外部事件表示。
|
||||
Agent-loop 测试固定先持久后 end 的顺序、中断的可见前缀、失败与重试 attempt、abandonment、usage 与 replay metadata。Session Controller 与 Conversation 测试固定实时瞬态显示、重连 baseline、committed settlement 发布与历史回放。Chat 与 Trajectory 测试固定实时 partial 展示和最终 message 的直接投影；TypeScript 与 Python SDK snapshot 固定外部事件表示。
|
||||
|
||||
## 备选方案
|
||||
|
||||
@@ -59,13 +61,15 @@ Agent-loop 测试固定先持久后 end 的顺序、中断的可见前缀、失
|
||||
|
||||
**通过历史 API 传递 packed chunk row。** 这会减少 v1 的 wire 与 Client 工作，却让 Client 拥有第二套事件词汇，并让传输继续与 token-row 基数耦合。当前 API 携带标量持久 settlement，并使用独立的实时瞬态 stream。
|
||||
|
||||
**在 Session Controller 中删除嵌入式 stream。** 这会减少 Client 保留的内存，却会引入第二种持久事件类型，并让面向传输的 owner 决定展示消费方需要哪些证据。实测瓶颈来自重复展开，因此由各 UI 消费方决定是否检查原样传递的 settlement。
|
||||
|
||||
**把 stream 存在 sidecar 或 replay-only fixture 中。** 这会把一个 attempt 的 message 与证据拆给不同持久性 owner，也无法让普通恢复 Session 获得相同的失败输出与时间事实。settlement 是原子 owner。
|
||||
|
||||
**把被消费 chunk 的引用重定向到其 settlement。** Chunk 与 attempt settlement 不是可互换事实。拒绝可以防止迁移悄然改变插件自有引用的含义。
|
||||
|
||||
## 后果
|
||||
|
||||
当前日志、遥测、历史页与冷 Client 组装按模型 attempt 而非 token chunk 扩展，同时在每个 settlement 内保留精确 stream 证据。实时呈现保持增量，并且有意仅存在于进程内。
|
||||
当前日志、遥测与历史页按模型 attempt 而非 token chunk 扩展，同时在每个 settlement 内保留精确 stream 证据。Client event window 保留这份紧凑证据，但 Chat 与 Trajectory 的 Assistant node 不会把 settled stream 展开成逐 delta 对象。实时呈现保持增量，并且有意仅存在于进程内。
|
||||
|
||||
v1 的顶层 chunk 可能在 attempt 结束前由带缓冲的持久化 writer 刷盘；与之不同，v2 在 settlement 之前没有持久 attempt 证据。如果进程或主机在 settlement 前硬中断，完整的 in-flight stream 都会丢失；`agent/assistant-stream` 不是 write-ahead log。这项取舍避免为实时输出增加第二个持久性 owner。
|
||||
|
||||
|
||||
@@ -242,6 +242,7 @@ function galleryFixture(imageUrl: string): string {
|
||||
...(call.meta === undefined ? {} : { meta: call.meta }),
|
||||
}, { surfaceOp: 'append', sourceEventSeqs: [source.seq] })
|
||||
}
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('step/start', { turn: 1, step: 2 })
|
||||
session.append('assistant/message', {
|
||||
stream: [],
|
||||
|
||||
@@ -201,4 +201,4 @@
|
||||
- text: DeepSeek-V4-Flash
|
||||
- img
|
||||
- button "Send message" [disabled]
|
||||
- text: 1 turns · 1 steps LLM {{duration}} · Tool call {{duration}}
|
||||
- text: 1 turns · 2 steps LLM {{duration}} · Tool call {{duration}}
|
||||
@@ -264,17 +264,7 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
await page.evaluate(() => { document.body.removeAttribute('data-ds-dark-theme') })
|
||||
await page.getByRole('tab', { name: 'Result' }).click()
|
||||
await expect.poll(() => page.getByText('NAVIGATION_OK', { exact: false }).count(), { timeout: 10_000 }).toBeGreaterThanOrEqual(1)
|
||||
const assistantSpan = page.locator('[data-timeline-span="message"][data-assistant-timing="true"]').first()
|
||||
await assistantSpan.hover()
|
||||
const timingTooltip = page.getByRole('tooltip')
|
||||
await timingTooltip.waitFor({ timeout: 5_000 })
|
||||
await expect.poll(() => timingTooltip.textContent(), { timeout: 5_000 }).toMatch(/TTFT .* Decoding/)
|
||||
const assistantTimingStyle = await assistantSpan.evaluate(node => ({
|
||||
background: getComputedStyle(node).backgroundImage,
|
||||
ttft: getComputedStyle(node).getPropertyValue('--trajectory-assistant-ttft'),
|
||||
}))
|
||||
expect(assistantTimingStyle.background).toContain('linear-gradient')
|
||||
expect(assistantTimingStyle.ttft).toMatch(/%$/)
|
||||
expect(await page.locator('[data-timeline-span="message"][data-assistant-timing="true"]').count()).toBe(0)
|
||||
const snapshot = (await captureStableAria(page, '[class*="viewArea"]', scaffold.workspaceCwd))
|
||||
.split(SEED_ID).join('{{seededId}}')
|
||||
await compareOrRefreshGolden(TRAJECTORY_EXPECTED, snapshot, MODE)
|
||||
|
||||
@@ -869,19 +869,15 @@ export async function launchWebScaffold(options: LaunchOptions = {}): Promise<We
|
||||
* in-memory record-mode harvest, so the on-disk zstd default never matters.
|
||||
*/
|
||||
function rawSessionLog(session: Session): string {
|
||||
const encoded = sessionFormatCatalog.encodeCurrent({
|
||||
header: {
|
||||
...session.header,
|
||||
delegationDepth: session.header.delegationDepth ?? 0,
|
||||
},
|
||||
inheritedEventCount: session.inheritedEventCount,
|
||||
// Session validates durable payloads as JSON; its closed event unions do
|
||||
// not carry the index signature used by the format package's JSON types.
|
||||
events: session.snapshotEvents() as unknown as readonly SessionFormatEvent[],
|
||||
})
|
||||
const encodedEvents = (session.snapshotEvents() as unknown as readonly SessionFormatEvent[])
|
||||
.map(event => sessionFormatCatalog.encodeCurrentEvent(event))
|
||||
const header = sessionFormatCatalog.encodeCurrentHeader({
|
||||
...session.header,
|
||||
delegationDepth: session.header.delegationDepth ?? 0,
|
||||
}, session.inheritedEventCount)
|
||||
return [
|
||||
JSON.stringify(encoded.header),
|
||||
...encoded.rows.map(record => JSON.stringify(record)),
|
||||
JSON.stringify(header),
|
||||
...encodedEvents.map(record => JSON.stringify(record)),
|
||||
'',
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
@@ -168,7 +168,7 @@ describe('web e2e: assistant IconActions wait for the turn to end', () => {
|
||||
}, 120_000)
|
||||
|
||||
it.skipIf(MODE === 'record')('shows exact completed-Turn usage and expands its available facts', async () => {
|
||||
await launch()
|
||||
await launch(undefined, 5)
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-turn-usage-expanded'))
|
||||
const { settled } = await sendPrompt(120_000)
|
||||
await settled
|
||||
@@ -200,8 +200,8 @@ describe('web e2e: assistant IconActions wait for the turn to end', () => {
|
||||
await timeTrigger.click()
|
||||
const timeDialog = page.getByRole('dialog', { name: 'Turn time and speed' })
|
||||
expect(await timeDialog.count()).toBe(1)
|
||||
expect(await timeDialog.getByText(/tok\/s/).count()).toBe(1)
|
||||
expect(await timeDialog.getByText('Time to first token (TTFT)', { exact: true }).count()).toBe(1)
|
||||
expect(await timeDialog.getByText(/tok\/s/).count()).toBe(0)
|
||||
expect(await timeDialog.getByText('Time to first token (TTFT)', { exact: true }).count()).toBe(0)
|
||||
await page.keyboard.press('Escape')
|
||||
await trigger.click()
|
||||
|
||||
@@ -212,7 +212,7 @@ describe('web e2e: assistant IconActions wait for the turn to end', () => {
|
||||
}, 120_000)
|
||||
|
||||
it.skipIf(MODE === 'record')('folds the Turn process after the completed reply becomes the answer', async () => {
|
||||
await launch()
|
||||
await launch(undefined, 5)
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-turn-tail-actions-completed'))
|
||||
const { settled } = await sendPrompt()
|
||||
await settled
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/architecture.md
|
||||
architecture.md: 959ea9edf8f337b1ebe20df8b998f11fe35beb2e
|
||||
architecture.zh.md: 9666f3589ce3710f84306d685ec1c5e7a49d1b78
|
||||
architecture.md: 76dda23c8a2200892587687417e326dc7cd15d80
|
||||
architecture.zh.md: 4e81625a0bba2698fc286d1ab0520fe4ec56a469
|
||||
@@ -106,7 +106,7 @@ Details: the [sequence diagram](agent-lifecycle.md), the [tool pipeline](tool-ex
|
||||
|
||||
The session log is the source of the context the model sees. `deriveMessages()` projects model history from it. Each `assistant/message` embeds the exact compact timed stream that produced its assembled content; `assistant/attempt` retains settled failed, retried, cancelled, and stream-error attempts without adding model history. Fork, resume, transcripts, telemetry, and persistence all derive from these durable settlements, while live UI incrementality comes from `agent/assistant-stream`; a hard process loss before settlement leaves no durable attempt stream ([decision](../.agents/notes/implemented/architecture/2026-09-01-v2-embedded-assistant-streams.md)).
|
||||
|
||||
Session consumers know only the current logical format. Header-only `stat` and `list` rescan each Session directory, select its numerically highest canonical generation, and translate a supported historical header without loading events or publishing a successor. A stored-session `open` selects that same generation, refuses a future version, or composes the static adjacent migration chain in memory, validates the final result, and exclusively publishes only that version-named successor beside the unchanged source before returning a handle; semantic interrupted-turn repair remains a handle consumer responsibility. JSONL v0 uses `session.jsonl[.zstd]`, v1 and later use lowercase `session.vN.jsonl[.zstd]`, and committed generation paths are never renamed, replaced, or deleted. The JSONL provider owns physical framing, compression, generation selection, and exclusive publication, while each adjacent migration package owns exactly one `vN -> vN+1` step ([decision](../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md)).
|
||||
Session consumers know only the current logical format. Header-only `stat` and `list` rescan each Session directory, select its numerically highest canonical generation, and translate a supported historical header without loading events or publishing a successor. A stored-session `open` selects that same generation, refuses a future version, or composes the static adjacent migration chain in memory, validates the final result, and exclusively publishes only that version-named successor beside the unchanged source before returning a handle. Ordinary repair of an unsealed interrupted tail remains a handle consumer responsibility; migration inserts a missing interrupted `turn/end` only for the bounded released restart already sealed by a later `turn/start`. JSONL v0 uses `session.jsonl[.zstd]`, v1 and later use lowercase `session.vN.jsonl[.zstd]`, and committed generation paths are never renamed, replaced, or deleted. The JSONL provider owns physical framing, compression, generation selection, and exclusive publication, while each adjacent migration package owns exactly one `vN -> vN+1` step ([decision](../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md)).
|
||||
|
||||
**Model-visible means logged.** Anything that reaches a model request must be reconstructable from the log, and a runtime invariant asserts it. This is why a new model-visible input requires a new session event: extend `SessionEventMap` and render from the log.
|
||||
|
||||
|
||||
@@ -110,7 +110,7 @@ turn/end
|
||||
|
||||
会话日志是模型所见上下文的来源。`deriveMessages()` 从中投影出模型历史。每个 `assistant/message` 都嵌入产生其组装内容的精确紧凑带时间 stream；`assistant/attempt` 保留已到达 settlement 的失败、重试、取消与 stream error attempt，且不添加模型历史。fork、恢复、transcript（文本记录）、遥测与持久化都从这些持久 settlement 派生，实时 UI 增量则来自 `agent/assistant-stream`；如果进程在 settlement 前硬中断，则不会留下持久 attempt stream（见[决策](../.agents/notes/implemented/architecture/2026-09-01-v2-embedded-assistant-streams.zh.md)）。
|
||||
|
||||
Session 消费方只了解当前逻辑格式。仅 header 的 `stat` 与 `list` 会重新扫描每个 Session 目录，选择数值最高的规范 generation，并在不加载事件或发布后继的情况下转换受支持的历史 header。已存储 Session 的 `open` 选择同一 generation，拒绝未来版本，或在内存中组合静态相邻迁移链、校验最终结果，并在返回句柄前以不覆盖方式只发布该版本命名的后继文件且保持源文件不变；语义层的中断轮次修复仍由句柄消费方负责。JSONL v0 使用 `session.jsonl[.zstd]`，v1 及后续版本使用小写 `session.vN.jsonl[.zstd]`；已提交 generation 路径绝不重命名、替换或删除。JSONL provider 负责物理 framing、压缩、generation 选择与排他发布，每个相邻迁移包只负责一个 `vN -> vN+1` 步骤（[决策](../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md)）。
|
||||
Session 消费方只了解当前逻辑格式。仅 header 的 `stat` 与 `list` 会重新扫描每个 Session 目录，选择数值最高的规范 generation，并在不加载事件或发布后继的情况下转换受支持的历史 header。已存储 Session 的 `open` 选择同一 generation，拒绝未来版本，或在内存中组合静态相邻迁移链、校验最终结果，并在返回句柄前以不覆盖方式只发布该版本命名的后继文件且保持源文件不变。未被后续事件封住的普通中断尾部仍由句柄消费方修复；只有在后续 `turn/start` 已经封住一种有限的已发布 restart 时，migration 才会插入缺失的 interrupted `turn/end`。JSONL v0 使用 `session.jsonl[.zstd]`，v1 及后续版本使用小写 `session.vN.jsonl[.zstd]`；已提交 generation 路径绝不重命名、替换或删除。JSONL provider 负责物理 framing、压缩、generation 选择与排他发布，每个相邻迁移包只负责一个 `vN -> vN+1` 步骤（[决策](../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md)）。
|
||||
|
||||
**模型可见即已记录。** 抵达模型请求的一切都必须能从日志重建，并由一项运行时不变量断言这一点。因此，新增一项模型可见输入就需要新增一个会话事件：扩展 `SessionEventMap` 并从日志渲染。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/config-catalog.md
|
||||
config-catalog.md: 8cca25f5feb63ad3157253e65cf10e78d1cd4707
|
||||
config-catalog.md: df4b4e4c5fde61ca3f2a97182463e196aa9e40b0
|
||||
config-catalog.zh.md: f7c4adf580cf11f05fbe209cf0ffeb4dfb50b45a
|
||||
@@ -1381,7 +1381,7 @@ export interface ReplayModelConfig {
|
||||
|
||||
Depends on: [`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
|
||||
|
||||
Source: [`packages/test-support/llm-replay/src/index.ts:1294`](../packages/test-support/llm-replay/src/index.ts)
|
||||
Source: [`packages/test-support/llm-replay/src/index.ts:1278`](../packages/test-support/llm-replay/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-llm-retry"></a>
|
||||
|
||||
@@ -1870,7 +1870,7 @@ export interface Config {
|
||||
export type JsonlCompression = 'zstd' | 'none'
|
||||
```
|
||||
|
||||
Source: [`packages/session/session-persistence-jsonl/src/index.ts:86`](../packages/session/session-persistence-jsonl/src/index.ts)
|
||||
Source: [`packages/session/session-persistence-jsonl/src/index.ts:87`](../packages/session/session-persistence-jsonl/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-session-projection-cache"></a>
|
||||
|
||||
|
||||
@@ -4,9 +4,8 @@ import type {
|
||||
ConversationNodeContext, ConversationNodeDefinition,
|
||||
} from '@deepseek-ai/dsh-client-ui-conversation/client'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { expandAssistantStream } from '@deepseek-ai/dsh-llm/assistant-stream'
|
||||
import type {} from '@deepseek-ai/dsh-llm-retry/types'
|
||||
import { isAppendSurfaceEvent } from '@deepseek-ai/dsh-session/surface'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session/types'
|
||||
import type { AssistantChatData } from '../contract/chat-nodes.ts'
|
||||
import { CHAT_SYNTHETIC_SEQ_OFFSETS, chatNode } from './common.ts'
|
||||
import {
|
||||
@@ -162,15 +161,20 @@ function updateChunk(
|
||||
}
|
||||
}
|
||||
|
||||
function updateEmbedded(
|
||||
function settleMessage(
|
||||
state: AssistantState,
|
||||
event: Extract<ConversationMatch['event'], { type: 'assistant/message' | 'assistant/attempt' }>,
|
||||
match: ConversationMatch,
|
||||
event: SessionEvent<'assistant/message'>,
|
||||
): AssistantState {
|
||||
let next = state
|
||||
for (const member of expandAssistantStream(event.data.stream)) {
|
||||
next = updateChunk(next, member.chunk, event.seq, member.time)
|
||||
const blocks = toAssistantBlocks(event.data.message.content)
|
||||
return {
|
||||
...state,
|
||||
blocks,
|
||||
visibleBlocks: countVisibleBlocks(blocks),
|
||||
hidden: false,
|
||||
final: match,
|
||||
usage: event.data.usage,
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function closedBoundary(location: ConversationLocation): { seq: number; time: number } | undefined {
|
||||
@@ -232,21 +236,9 @@ function fallbackState(context: ConversationNodeContext<AssistantState>): Assist
|
||||
state = updateChunk(state, match.event.data.chunk, match.event.seq, match.event.time)
|
||||
continue
|
||||
}
|
||||
if (match.event.type === 'assistant/message' || match.event.type === 'assistant/attempt') {
|
||||
state ??= initialState(match.event.data.turn, match.event.data.step)
|
||||
state = updateEmbedded(state, match.event)
|
||||
}
|
||||
if (match.event.type === 'assistant/message') {
|
||||
state ??= initialState(match.event.data.turn, match.event.data.step)
|
||||
const blocks = toAssistantBlocks(match.event.data.message.content)
|
||||
state = {
|
||||
...state,
|
||||
blocks,
|
||||
visibleBlocks: countVisibleBlocks(blocks),
|
||||
hidden: false,
|
||||
final: match,
|
||||
usage: match.event.data.usage,
|
||||
}
|
||||
state = settleMessage(state, match, match.event)
|
||||
continue
|
||||
}
|
||||
if (match.event.type === 'llm/retry' && state !== undefined) {
|
||||
@@ -304,8 +296,7 @@ export const assistantDefinition: ConversationNodeDefinition<AssistantState> = {
|
||||
match: (event) => {
|
||||
if (event.type === 'step/start') return { id: `${event.data.turn}:${event.data.step}`, role: 'start' }
|
||||
if (event.type === 'assistant/live-chunk'
|
||||
|| event.type === 'assistant/attempt'
|
||||
|| (event.type === 'assistant/message' && isAppendSurfaceEvent(event))) {
|
||||
|| (event.type === 'assistant/message' && event.surfaceOp === 'append')) {
|
||||
return { id: `${event.data.turn}:${event.data.step}`, role: 'update' }
|
||||
}
|
||||
if (event.type === 'llm/retry') {
|
||||
@@ -321,19 +312,7 @@ export const assistantDefinition: ConversationNodeDefinition<AssistantState> = {
|
||||
if (match.event.type === 'assistant/live-chunk') {
|
||||
return updateChunk(context.state, match.event.data.chunk, match.event.seq, match.event.time)
|
||||
}
|
||||
if (match.event.type === 'assistant/attempt') return updateEmbedded(context.state, match.event)
|
||||
if (match.event.type === 'assistant/message') {
|
||||
const streamed = updateEmbedded(context.state, match.event)
|
||||
const blocks = toAssistantBlocks(match.event.data.message.content)
|
||||
return {
|
||||
...streamed,
|
||||
blocks,
|
||||
visibleBlocks: countVisibleBlocks(blocks),
|
||||
hidden: false,
|
||||
final: match,
|
||||
usage: match.event.data.usage,
|
||||
}
|
||||
}
|
||||
if (match.event.type === 'assistant/message') return settleMessage(context.state, match, match.event)
|
||||
if (match.event.type === 'llm/retry') {
|
||||
return resetForRetry(context.state)
|
||||
}
|
||||
|
||||
@@ -4,8 +4,6 @@ import type {
|
||||
} from '@deepseek-ai/dsh-client-ui-conversation/client'
|
||||
import type {} from '@deepseek-ai/dsh-llm-retry/types'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { expandAssistantStream } from '@deepseek-ai/dsh-llm/assistant-stream'
|
||||
import { isAppendSurfaceEvent } from '@deepseek-ai/dsh-session/surface'
|
||||
import type {} from '@deepseek-ai/dsh-tools/types'
|
||||
import { hasAssistantReplyContent } from '../contract/assistant-content.ts'
|
||||
import type { AssistantChatData, ChatNode, FinalAssistantChatData } from '../contract/chat-nodes.ts'
|
||||
@@ -63,11 +61,9 @@ function visibleChunk(chunk: StreamChunk): boolean {
|
||||
|
||||
function visibleAssistantEvent(event: ConversationEvent): boolean {
|
||||
if (event.type === 'assistant/live-chunk') return visibleChunk(event.data.chunk)
|
||||
if (event.type === 'assistant/attempt') {
|
||||
return expandAssistantStream(event.data.stream).some(member => visibleChunk(member.chunk))
|
||||
}
|
||||
if (event.type === 'assistant/attempt') return false
|
||||
return event.type === 'assistant/message'
|
||||
&& isAppendSurfaceEvent(event)
|
||||
&& event.surfaceOp === 'append'
|
||||
&& toAssistantBlocks(event.data.message.content).some((block) => {
|
||||
if (block.kind === 'tool-call') return false
|
||||
if (block.kind === 'text' || block.kind === 'reasoning') return block.text.trim() !== ''
|
||||
@@ -87,7 +83,7 @@ function processEvidence(event: ConversationEvent): ProcessEvidence | undefined
|
||||
return { kind: 'assistant', seq: event.seq, step: event.data.step }
|
||||
}
|
||||
if (event.type === 'tool/call'
|
||||
|| (event.type === 'tool/result' && isAppendSurfaceEvent(event))
|
||||
|| (event.type === 'tool/result' && event.surfaceOp === 'append')
|
||||
|| event.type === 'llm/retry') return { kind: 'other', seq: event.seq }
|
||||
return undefined
|
||||
}
|
||||
@@ -174,7 +170,7 @@ function processSpec(state: TurnProcessState, turn: TurnLocation): TurnProcessSp
|
||||
function updateProcessState(state: TurnProcessState, event: ConversationEvent): TurnProcessState {
|
||||
let current = state
|
||||
if (event.type === 'assistant/message'
|
||||
&& isAppendSurfaceEvent(event)
|
||||
&& event.surfaceOp === 'append'
|
||||
&& hasAssistantReplyContent(toAssistantBlocks(event.data.message.content))) {
|
||||
const messageCountByStep = new Map(current.messageCountByStep)
|
||||
messageCountByStep.set(event.data.step, (messageCountByStep.get(event.data.step) ?? 0) + 1)
|
||||
@@ -219,7 +215,6 @@ export const turnProcessDefinition: ConversationNodeDefinition<TurnProcessState>
|
||||
if (turn === undefined) return null
|
||||
if (event.type === 'assistant/live-chunk'
|
||||
|| event.type === 'assistant/message'
|
||||
|| event.type === 'assistant/attempt'
|
||||
|| event.type === 'tool/call'
|
||||
|| event.type === 'tool/result'
|
||||
|| event.type === 'llm/retry'
|
||||
|
||||
@@ -4,8 +4,6 @@ import type {
|
||||
} from '@deepseek-ai/dsh-client-ui-conversation/client'
|
||||
import type {} from '@deepseek-ai/dsh-llm-retry/types'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { expandAssistantStream } from '@deepseek-ai/dsh-llm/assistant-stream'
|
||||
import { isAppendSurfaceEvent } from '@deepseek-ai/dsh-session/surface'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session/types'
|
||||
import { deriveTurnTokenUsage } from '@deepseek-ai/dsh-token-meter/client'
|
||||
import type {
|
||||
@@ -45,7 +43,7 @@ function isSessionEvent(event: ConversationMatch['event']): event is SessionEven
|
||||
|
||||
function hasTextAssistant(event: Parameters<ConversationNodeDefinition['match']>[0]): boolean {
|
||||
return event.type === 'assistant/message'
|
||||
&& isAppendSurfaceEvent(event)
|
||||
&& event.surfaceOp === 'append'
|
||||
&& toAssistantBlocks(event.data.message.content)
|
||||
.some(block => block.kind === 'text' && block.text.trim() !== '')
|
||||
}
|
||||
@@ -57,12 +55,6 @@ function chunkHasText(chunk: StreamChunk): boolean {
|
||||
&& chunk.block.text.trim() !== ''
|
||||
}
|
||||
|
||||
function eventStreamHasText(event: Parameters<ConversationNodeDefinition['match']>[0]): boolean {
|
||||
if (event.type === 'assistant/live-chunk') return chunkHasText(event.data.chunk)
|
||||
if (event.type !== 'assistant/attempt') return false
|
||||
return expandAssistantStream(event.data.stream).some(member => chunkHasText(member.chunk))
|
||||
}
|
||||
|
||||
function turnCoordinates(event: Parameters<ConversationNodeDefinition['match']>[0]): {
|
||||
readonly turn: number
|
||||
readonly step?: number
|
||||
@@ -92,10 +84,10 @@ function closingAnchor(context: ConversationNodeContext<TurnTailState>): number
|
||||
const coordinates = turnCoordinates(event)
|
||||
if (coordinates?.step === undefined) continue
|
||||
const previous = steps.get(coordinates.step) ?? { streamedText: false, finalized: false }
|
||||
if (event.type === 'assistant/live-chunk' || event.type === 'assistant/attempt') {
|
||||
if (event.type === 'assistant/live-chunk') {
|
||||
steps.set(coordinates.step, {
|
||||
...previous,
|
||||
streamedText: previous.streamedText || eventStreamHasText(event),
|
||||
streamedText: previous.streamedText || chunkHasText(event.data.chunk),
|
||||
})
|
||||
continue
|
||||
}
|
||||
@@ -145,7 +137,7 @@ function tailData(context: ConversationNodeContext<TurnTailState>): TurnTailChat
|
||||
for (const match of context.matches) {
|
||||
const event = match.event
|
||||
const candidate = event.type === 'tool/call'
|
||||
|| (event.type === 'tool/result' && isAppendSurfaceEvent(event))
|
||||
|| (event.type === 'tool/result' && event.surfaceOp === 'append')
|
||||
|| (event.type === 'turn/end' && event.data.reason.kind === 'error')
|
||||
|| event.type === 'llm/retry'
|
||||
? event.seq
|
||||
|
||||
@@ -163,55 +163,6 @@ function node(value: ChatSnapshot, kind: string): ChatConversationViewNode | und
|
||||
return value.nodes.values().find(candidate => candidate.kind === kind)
|
||||
}
|
||||
|
||||
function comparableSnapshot(value: ChatSnapshot) {
|
||||
const nodes = value.nodes.values()
|
||||
return {
|
||||
order: value.order,
|
||||
nodes: nodes.map(candidate => ({
|
||||
...candidate,
|
||||
location: candidate.location.kind === 'step'
|
||||
? {
|
||||
kind: 'step',
|
||||
turn: candidate.location.turn.turn,
|
||||
turnStatus: candidate.location.turn.status,
|
||||
step: candidate.location.step.step,
|
||||
stepStatus: candidate.location.step.status,
|
||||
}
|
||||
: candidate.location.kind === 'turn'
|
||||
? {
|
||||
kind: 'turn',
|
||||
turn: candidate.location.turn.turn,
|
||||
turnStatus: candidate.location.turn.status,
|
||||
}
|
||||
: { kind: candidate.location.kind },
|
||||
})),
|
||||
processes: nodes.map(candidate => [
|
||||
candidate.key,
|
||||
value.nodes.processSource(candidate.key).getSnapshot(),
|
||||
]),
|
||||
navigation: value.navigation.items(),
|
||||
legacy: value.legacy,
|
||||
}
|
||||
}
|
||||
|
||||
function withoutEmbeddedSequenceAnchors(value: unknown): unknown {
|
||||
if (Array.isArray(value)) return value.map(withoutEmbeddedSequenceAnchors)
|
||||
if (value instanceof Map) {
|
||||
return new Map([...value].map(([key, entry]) => [key, withoutEmbeddedSequenceAnchors(entry)]))
|
||||
}
|
||||
if (typeof value !== 'object' || value === null) return value
|
||||
return Object.fromEntries(Object.entries(value).map(([key, entry]) => [
|
||||
key,
|
||||
key === 'anchorSeq' || key === 'controlAnchorSeq' || key === 'processStartSeq'
|
||||
? '<representation-owned-seq>'
|
||||
: withoutEmbeddedSequenceAnchors(entry),
|
||||
]))
|
||||
}
|
||||
|
||||
function comparableEmbeddedSnapshot(value: ChatSnapshot): unknown {
|
||||
return withoutEmbeddedSequenceAnchors(comparableSnapshot(value))
|
||||
}
|
||||
|
||||
function textMessage(id: string, text: string) {
|
||||
return {
|
||||
id,
|
||||
@@ -954,7 +905,7 @@ describe('built-in conversation node Definitions', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('folds packed Assistant runs to the same Chat content and Turn Tail state as scalar deltas', () => {
|
||||
it('uses live Assistant deltas without replaying settled embedded streams', () => {
|
||||
const runningHistory = [
|
||||
at(1, 'turn/start', { turn: 1 }),
|
||||
at(2, 'step/start', { turn: 1, step: 1 }),
|
||||
@@ -1001,9 +952,7 @@ describe('built-in conversation node Definitions', () => {
|
||||
expect(runningAttempt.data.stream.length).toBeGreaterThan(0)
|
||||
const packed = assembler(packedHistory)
|
||||
|
||||
expect(comparableEmbeddedSnapshot(snapshot(packed))).toEqual(comparableEmbeddedSnapshot(snapshot(scalar)))
|
||||
const running = node(snapshot(packed), 'assistant-step')
|
||||
expect(running).toMatchObject({ anchorSeq: 12 })
|
||||
const running = node(snapshot(scalar), 'assistant-step')
|
||||
expect(running?.data).toMatchObject({
|
||||
time: 1_004,
|
||||
blocks: [
|
||||
@@ -1012,25 +961,25 @@ describe('built-in conversation node Definitions', () => {
|
||||
{ kind: 'tool-call', callId: 'call-1', name: '', argsRaw: '{"x":1}' },
|
||||
],
|
||||
})
|
||||
expect(snapshot(packed).legacy.partial).toBeNull()
|
||||
expect(node(snapshot(packed), 'assistant-step')).toBeUndefined()
|
||||
|
||||
for (const value of [scalar, packed]) {
|
||||
value.append(at(13, 'step/end', { turn: 1, step: 1 }))
|
||||
value.append(at(14, 'turn/end', { turn: 1, reason: { kind: 'completed' } }))
|
||||
value.flush()
|
||||
}
|
||||
expect(comparableEmbeddedSnapshot(snapshot(packed))).toEqual(comparableEmbeddedSnapshot(snapshot(scalar)))
|
||||
expect(node(snapshot(packed), 'turn-tail')?.anchorSeq).toBe(12.2)
|
||||
expect(node(snapshot(scalar), 'assistant-step')?.data).toMatchObject({ status: 'interrupted' })
|
||||
expect(node(snapshot(packed), 'assistant-step')).toBeUndefined()
|
||||
|
||||
const partialHistory = [
|
||||
...runningHistory.slice(2),
|
||||
at(13, 'step/end', { turn: 1, step: 1 }),
|
||||
at(14, 'turn/end', { turn: 1, reason: { kind: 'completed' } }),
|
||||
]
|
||||
const partialScalar = snapshot(assembler(partialHistory, true))
|
||||
const partialPacked = snapshot(assembler(packedInputs(partialHistory), true))
|
||||
expect(comparableEmbeddedSnapshot(partialPacked)).toEqual(comparableEmbeddedSnapshot(partialScalar))
|
||||
expect(node(partialPacked, 'assistant-step')?.data).toMatchObject({ status: 'interrupted' })
|
||||
expect(node(partialPacked, 'turn-tail')?.anchorSeq).toBe(12.2)
|
||||
expect(partialPacked.legacy.partial).toBeNull()
|
||||
expect(node(partialPacked, 'assistant-step')).toBeUndefined()
|
||||
|
||||
const finalizedHistory = [
|
||||
at(20, 'turn/start', { turn: 2 }),
|
||||
@@ -1062,16 +1011,17 @@ describe('built-in conversation node Definitions', () => {
|
||||
turn: 2, step: 1, message: assistantMessage('packed-final', 'done'),
|
||||
}, { surfaceOp: 'append' }),
|
||||
]
|
||||
const finalizedScalar = snapshot(assembler(finalizedHistory))
|
||||
const finalizedInputs = packedInputs(finalizedHistory)
|
||||
expect(finalizedInputs.filter(input => input.event.type === 'assistant/attempt')).toHaveLength(1)
|
||||
const finalizedMessage = finalizedInputs.find(input => input.event.type === 'assistant/message')?.event
|
||||
if (finalizedMessage?.type !== 'assistant/message') throw new Error('expected packed final message')
|
||||
expect(finalizedMessage.data.stream.length).toBeGreaterThan(0)
|
||||
const finalizedPacked = snapshot(assembler(finalizedInputs))
|
||||
expect(comparableEmbeddedSnapshot(finalizedPacked)).toEqual(comparableEmbeddedSnapshot(finalizedScalar))
|
||||
const finalNode = (node(finalizedPacked, 'assistant-step')?.data as AssistantChatData).finalNode
|
||||
expect(finalNode?.timing?.firstTokenTime).toBe(1_999)
|
||||
expect(finalNode).toMatchObject({
|
||||
blocks: [{ kind: 'text', text: 'done' }],
|
||||
timing: { firstTokenTime: null },
|
||||
})
|
||||
|
||||
const namedToolHistory = [
|
||||
at(40, 'turn/start', { turn: 3 }),
|
||||
@@ -1089,15 +1039,16 @@ describe('built-in conversation node Definitions', () => {
|
||||
},
|
||||
}, { surfaceOp: 'append' }),
|
||||
]
|
||||
const namedToolScalar = snapshot(assembler(namedToolHistory))
|
||||
const namedToolInputs = packedInputs(namedToolHistory)
|
||||
const namedToolMessage = namedToolInputs.find(input => input.event.type === 'assistant/message')?.event
|
||||
if (namedToolMessage?.type !== 'assistant/message') throw new Error('expected packed named-tool message')
|
||||
expect(namedToolMessage.data.stream.length).toBeGreaterThan(0)
|
||||
const namedToolPacked = snapshot(assembler(namedToolInputs))
|
||||
expect(comparableEmbeddedSnapshot(namedToolPacked)).toEqual(comparableEmbeddedSnapshot(namedToolScalar))
|
||||
const namedTool = (node(namedToolPacked, 'assistant-step')?.data as AssistantChatData).finalNode
|
||||
expect(namedTool?.timing?.firstTokenTime).toBe(4_000)
|
||||
expect(namedTool).toMatchObject({
|
||||
blocks: [{ kind: 'tool-call', callId: 'call-2', name: 'read', argsRaw: '' }],
|
||||
timing: { firstTokenTime: null },
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps one keyed Tool node from running through settlement and replays nested dispatch after prepend', () => {
|
||||
|
||||
@@ -5,7 +5,7 @@ import type {
|
||||
PartialAssistant, RequestView,
|
||||
} from '@deepseek-ai/dsh-client-ui-conversation/client'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { expandAssistantStream } from '@deepseek-ai/dsh-llm/assistant-stream'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session/types'
|
||||
import { trajectoryNode } from './trajectory-definition-common.ts'
|
||||
import {
|
||||
displayFailure, emptyAssistantBlock, isTokenDelta, toAssistantBlock, toAssistantBlocks,
|
||||
@@ -191,15 +191,20 @@ function updateChunk(
|
||||
}
|
||||
}
|
||||
|
||||
function updateEmbedded(
|
||||
function settleMessage(
|
||||
state: AssistantState,
|
||||
event: Extract<ConversationMatch['event'], { type: 'assistant/message' | 'assistant/attempt' }>,
|
||||
match: ConversationMatch,
|
||||
event: SessionEvent<'assistant/message'>,
|
||||
): AssistantState {
|
||||
let next = state
|
||||
for (const member of expandAssistantStream(event.data.stream)) {
|
||||
next = updateChunk(next, member.chunk, event.seq, member.time)
|
||||
const blocks = toAssistantBlocks(event.data.message.content)
|
||||
return {
|
||||
...state,
|
||||
sawChunk: false,
|
||||
blocks,
|
||||
visibleBlocks: countVisibleBlocks(blocks),
|
||||
final: match,
|
||||
usage: event.data.usage,
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function closedBoundary(
|
||||
@@ -221,18 +226,9 @@ function fallbackState(context: ConversationNodeContext<AssistantState>): Assist
|
||||
if (event.type === 'assistant/live-chunk') {
|
||||
state ??= initialState(event.data.turn, event.data.step, event.seq, event.time, false)
|
||||
state = updateChunk(state, event.data.chunk, event.seq, event.time)
|
||||
} else if (event.type === 'assistant/message' || event.type === 'assistant/attempt') {
|
||||
} else if (event.type === 'assistant/message') {
|
||||
state ??= initialState(event.data.turn, event.data.step, event.seq, event.time, false)
|
||||
state = updateEmbedded(state, event)
|
||||
if (event.type === 'assistant/attempt') continue
|
||||
const blocks = toAssistantBlocks(event.data.message.content)
|
||||
state = {
|
||||
...state,
|
||||
blocks,
|
||||
visibleBlocks: countVisibleBlocks(blocks),
|
||||
final: match,
|
||||
usage: state.usage ?? event.data.usage,
|
||||
}
|
||||
state = settleMessage(state, match, event)
|
||||
} else if (event.type === 'step/end' && state !== undefined) {
|
||||
state = { ...state, stepEnd: match }
|
||||
}
|
||||
@@ -329,7 +325,6 @@ const trajectoryAssistantDefinition: ConversationNodeDefinition<AssistantState>
|
||||
}
|
||||
if (event.type === 'assistant/live-chunk'
|
||||
|| event.type === 'assistant/message'
|
||||
|| event.type === 'assistant/attempt'
|
||||
|| event.type === 'llm/retry'
|
||||
|| event.type === 'step/end') {
|
||||
return { id: `${event.data.turn}:${event.data.step}`, role: 'update' }
|
||||
@@ -352,18 +347,7 @@ const trajectoryAssistantDefinition: ConversationNodeDefinition<AssistantState>
|
||||
if (match.event.type === 'assistant/live-chunk') {
|
||||
return updateChunk(context.state, match.event.data.chunk, match.event.seq, match.event.time)
|
||||
}
|
||||
if (match.event.type === 'assistant/attempt') return updateEmbedded(context.state, match.event)
|
||||
if (match.event.type === 'assistant/message') {
|
||||
const streamed = updateEmbedded(context.state, match.event)
|
||||
const blocks = toAssistantBlocks(match.event.data.message.content)
|
||||
return {
|
||||
...streamed,
|
||||
blocks,
|
||||
visibleBlocks: countVisibleBlocks(blocks),
|
||||
final: match,
|
||||
usage: streamed.usage ?? match.event.data.usage,
|
||||
}
|
||||
}
|
||||
if (match.event.type === 'assistant/message') return settleMessage(context.state, match, match.event)
|
||||
if (match.event.type === 'step/end') return { ...context.state, stepEnd: match }
|
||||
if (match.event.type !== 'llm/retry') return context.state
|
||||
const data = match.event.data
|
||||
|
||||
@@ -224,7 +224,7 @@ describe('Trajectory conversation Definitions', () => {
|
||||
}])
|
||||
})
|
||||
|
||||
it('folds packed Assistant runs to the same Trajectory state as scalar deltas', () => {
|
||||
it('uses live Assistant deltas without replaying settled embedded streams', () => {
|
||||
const runningHistory = [
|
||||
at(1, 'turn/start', { turn: 1 }),
|
||||
at(2, 'step/start', { turn: 1, step: 1 }),
|
||||
@@ -267,29 +267,20 @@ describe('Trajectory conversation Definitions', () => {
|
||||
if (runningAttempt?.type !== 'assistant/attempt') throw new Error('expected packed running attempt')
|
||||
expect(runningAttempt.data.stream.length).toBeGreaterThan(0)
|
||||
const runningPacked = snapshot(assembler(packedHistory))
|
||||
expect(runningPacked).toEqual(runningScalar)
|
||||
expect(runningPacked.partial?.blocks).toEqual([
|
||||
expect(runningScalar.partial?.blocks).toEqual([
|
||||
{ kind: 'text', text: ' answer' },
|
||||
{ kind: 'reasoning', text: 'thinking' },
|
||||
{ kind: 'tool-call', callId: 'call-1', name: '', argsRaw: '{"x":1}' },
|
||||
])
|
||||
expect(runningPacked.partial).toBeNull()
|
||||
|
||||
const partialHistory = [
|
||||
...runningHistory.slice(2),
|
||||
at(12, 'step/end', { turn: 1, step: 1 }),
|
||||
]
|
||||
const partialScalar = snapshot(assembler(partialHistory))
|
||||
const partialPacked = snapshot(assembler(packedInputs(partialHistory)))
|
||||
expect(partialPacked).toEqual(partialScalar)
|
||||
expect(partialPacked.eventNodes).toMatchObject([{
|
||||
kind: 'assistant',
|
||||
interrupted: true,
|
||||
blocks: [
|
||||
{ kind: 'text', text: ' answer' },
|
||||
{ kind: 'reasoning', text: 'thinking' },
|
||||
{ kind: 'tool-call', callId: 'call-1', name: '', argsRaw: '{"x":1}' },
|
||||
],
|
||||
}])
|
||||
expect(partialPacked.partial).toBeNull()
|
||||
expect(partialPacked.eventNodes).toEqual([])
|
||||
|
||||
const finalizedHistory = [
|
||||
at(20, 'turn/start', { turn: 2 }),
|
||||
@@ -325,20 +316,18 @@ describe('Trajectory conversation Definitions', () => {
|
||||
}),
|
||||
at(31, 'step/end', { turn: 2, step: 1 }),
|
||||
]
|
||||
const finalizedScalar = snapshot(assembler(finalizedHistory))
|
||||
const finalizedInputs = packedInputs(finalizedHistory)
|
||||
expect(finalizedInputs.filter(input => input.event.type === 'assistant/attempt')).toHaveLength(1)
|
||||
const finalizedMessage = finalizedInputs.find(input => input.event.type === 'assistant/message')?.event
|
||||
if (finalizedMessage?.type !== 'assistant/message') throw new Error('expected packed final message')
|
||||
expect(finalizedMessage.data.stream.length).toBeGreaterThan(0)
|
||||
const finalizedPacked = snapshot(assembler(finalizedInputs))
|
||||
expect(finalizedPacked).toEqual(finalizedScalar)
|
||||
expect(finalizedPacked.eventNodes.find(node => node.kind === 'assistant')).toMatchObject({
|
||||
timing: { firstTokenTime: 3_000 },
|
||||
blocks: [{ kind: 'text', text: 'done' }],
|
||||
timing: { firstTokenTime: null },
|
||||
})
|
||||
expect(finalizedPacked.requests).toMatchObject([{
|
||||
purpose: 'assistant',
|
||||
usage: { inputTokens: 10, outputTokens: 3 },
|
||||
retry: 1,
|
||||
}])
|
||||
|
||||
@@ -358,15 +347,14 @@ describe('Trajectory conversation Definitions', () => {
|
||||
},
|
||||
}),
|
||||
]
|
||||
const namedToolScalar = snapshot(assembler(namedToolHistory))
|
||||
const namedToolInputs = packedInputs(namedToolHistory)
|
||||
const namedToolMessage = namedToolInputs.find(input => input.event.type === 'assistant/message')?.event
|
||||
if (namedToolMessage?.type !== 'assistant/message') throw new Error('expected packed named-tool message')
|
||||
expect(namedToolMessage.data.stream.length).toBeGreaterThan(0)
|
||||
const namedToolPacked = snapshot(assembler(namedToolInputs))
|
||||
expect(namedToolPacked).toEqual(namedToolScalar)
|
||||
expect(namedToolPacked.eventNodes.find(node => node.kind === 'assistant')).toMatchObject({
|
||||
timing: { firstTokenTime: 4_000 },
|
||||
blocks: [{ kind: 'tool-call', callId: 'call-2', name: 'read', argsRaw: '' }],
|
||||
timing: { firstTokenTime: null },
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-format-catalog/README.md
|
||||
README.md: 120a770ba5624d5ecde040d94399d21dcf915bd9
|
||||
README.zh.md: 5c0a7a816d637ea08e84a82f129c1387762d6299
|
||||
README.md: 28dc1ac5b7c4711e8bba964f611fd48f7cfed298
|
||||
README.zh.md: fe48e3466adaab61772ef51ddb14a6d3be7bddae
|
||||
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
## Summary
|
||||
|
||||
`dsh-session-format-catalog` gives persistence one deterministic Session format reader without consulting mounted plugins. It assembles the frozen v0, v1, and v2 codecs with the adjacent v0-to-v1 and v1-to-v2 edges, checks the complete gap-free chain at module initialization, and exposes physical dispatch, header-only classification, migration, and current encoding through `sessionFormatCatalog`.
|
||||
`dsh-session-format-catalog` gives persistence one deterministic Session format reader without consulting mounted plugins. It assembles the frozen v0, v1, and v2 codecs with the adjacent v0-to-v1 and v1-to-v2 edges, checks the complete gap-free chain at module initialization, and exposes physical dispatch, header-only classification, single-pass row restoration, and current record encoding through `sessionFormatCatalog`.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -27,16 +27,22 @@ English | [中文](README.zh.md)
|
||||
|
||||
### When to use it
|
||||
|
||||
Import this library from persistence and test-support readers that need the complete first-party released-format inventory before any feature plugin mounts. Feature compositions do not register or reorder its entries. No runtime invariant companion is published because construction rejects an invalid static inventory and each read validates its complete result; the catalog retains no independently mutable runtime relationship.
|
||||
Import this library from persistence and test-support readers that need the complete first-party released-format inventory before any feature plugin mounts. Feature compositions do not register or reorder its entries. No runtime invariant companion is published because construction rejects an invalid static inventory and each completed restore validates its result; mutable row-decoder state belongs to one caller-owned streaming restore.
|
||||
|
||||
### Entry point
|
||||
|
||||
```text
|
||||
const descriptor = sessionFormatCatalog.readHeader(physicalHeader)
|
||||
const current = sessionFormatCatalog.migrate(sessionFormatCatalog.decodeArtifact(physicalHeader, rows))
|
||||
const restore = sessionFormatCatalog.createRestore(physicalHeader, { recovery: 'recoverable', validation: 'transformed' })
|
||||
for (const row of physicalRows) restore.decodeRow(row)
|
||||
const current = restore.finish()
|
||||
const headerRecord = sessionFormatCatalog.encodeCurrentHeader(current.header, current.inheritedEventCount)
|
||||
const eventRecords = current.events.map(sessionFormatCatalog.encodeCurrentEvent)
|
||||
```
|
||||
|
||||
Import `sessionFormatCatalog` from the package root. JSONL readers pass parsed header and row JSON values to `decodeArtifact()` or `decodeRecoverableArtifact()`, migrate the logical result with `migrate()`, and serialize only the validated current artifact with `encodeCurrent()`. Listing calls `readHeader()` and never opens event bodies. Header reads validate every adjacent target and then restore the final header through the installed current Session package.
|
||||
Import `sessionFormatCatalog` from the package root. JSONL and fixture readers create one restore, push each parsed physical row through `decodeRow()`, and call `finish()` once. Writers serialize the returned current artifact through `encodeCurrentHeader()` and `encodeCurrentEvent()`. Listing calls `readHeader()` and never opens event bodies.
|
||||
|
||||
Production historical reads select `{ recovery: 'recoverable', validation: 'transformed' }`. Worker and fixture verification select `{ recovery: 'strict', validation: 'current' }`. Transformed validation runs the released-current rules after migration but deliberately skips installed semantic validation for input that is already current.
|
||||
|
||||
The catalog contains all supported historical readers directly. A profile cannot add, remove, or reorder an edge by mounting a feature plugin. Its peer dependency on `dsh-session` supplies the installed current event vocabulary and current restoration rules, while historical edge validators remain frozen.
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ kind: "package-library"
|
||||
|
||||
## 概述
|
||||
|
||||
`dsh-session-format-catalog` 为持久化提供一个确定性的 Session 格式读取器，且无需查询已挂载插件。它把冻结的 v0、v1 与 v2 编解码器和相邻的 v0 到 v1、v1 到 v2 迁移边装配起来，在模块初始化时校验完整且无缺口的迁移链，并通过 `sessionFormatCatalog` 暴露物理分派、仅标头分类、迁移和当前格式编码。
|
||||
`dsh-session-format-catalog` 为持久化提供一个确定性的 Session 格式读取器，且无需查询已挂载插件。它把冻结的 v0、v1 与 v2 编解码器和相邻的 v0 到 v1、v1 到 v2 迁移边装配起来，在模块初始化时校验完整且无缺口的迁移链，并通过 `sessionFormatCatalog` 暴露物理分派、仅 header 分类、单遍行还原和当前格式逐记录编码。
|
||||
|
||||
## 目录
|
||||
|
||||
@@ -27,16 +27,22 @@ kind: "package-library"
|
||||
|
||||
### 何时使用
|
||||
|
||||
当持久化与测试支持读取方需要在任何功能插件挂载前取得完整第一方已发布格式清单时，导入本库。功能组合不会注册或重排其条目。它不发布运行时不变式伴生入口，因为构造过程会拒绝无效静态清单，每次读取也会校验完整结果；目录不保留可独立分叉的运行时可变关系。
|
||||
当持久化与测试支持读取方需要在任何功能插件挂载前取得完整第一方已发布格式清单时，导入本库。功能组合不会注册或重排其条目。它不发布运行时不变式伴生入口，因为构造过程会拒绝无效静态清单，每次完成的还原也会校验结果；可变行 decoder 状态只属于一次由调用方持有的流式还原。
|
||||
|
||||
### 入口
|
||||
|
||||
```text
|
||||
const descriptor = sessionFormatCatalog.readHeader(physicalHeader)
|
||||
const current = sessionFormatCatalog.migrate(sessionFormatCatalog.decodeArtifact(physicalHeader, rows))
|
||||
const restore = sessionFormatCatalog.createRestore(physicalHeader, { recovery: 'recoverable', validation: 'transformed' })
|
||||
for (const row of physicalRows) restore.decodeRow(row)
|
||||
const current = restore.finish()
|
||||
const headerRecord = sessionFormatCatalog.encodeCurrentHeader(current.header, current.inheritedEventCount)
|
||||
const eventRecords = current.events.map(sessionFormatCatalog.encodeCurrentEvent)
|
||||
```
|
||||
|
||||
从包根导入 `sessionFormatCatalog`。JSONL 读取方把解析后的标头与行 JSON 值传给 `decodeArtifact()` 或 `decodeRecoverableArtifact()`，使用 `migrate()` 迁移逻辑结果，并且只使用 `encodeCurrent()` 序列化经过校验的当前产物。列表读取调用 `readHeader()`，绝不打开事件正文。标头读取会校验每个相邻目标，然后通过已安装的当前 Session 包还原最终标头。
|
||||
从包根导入 `sessionFormatCatalog`。JSONL 与 fixture 读取方创建一次 restore，把每个已解析物理行传给 `decodeRow()`，再调用一次 `finish()`。Writer 通过 `encodeCurrentHeader()` 与 `encodeCurrentEvent()` 序列化返回的当前 artifact。列表读取调用 `readHeader()`，绝不打开事件正文。
|
||||
|
||||
Production 历史读取使用 `{ recovery: 'recoverable', validation: 'transformed' }`。Worker 与 fixture 校验使用 `{ recovery: 'strict', validation: 'current' }`。Transformed validation 会在迁移后执行已发布 current 规则，但对已经是 current 的输入有意跳过已安装语义校验。
|
||||
|
||||
该目录直接包含所有受支持的历史读取器。Profile 无法通过挂载功能插件来添加、移除或重新排列迁移边。它通过对 `dsh-session` 的 peer 依赖获得已安装的当前事件词表与当前还原规则，而历史迁移边校验器保持冻结。
|
||||
|
||||
|
||||
@@ -13,13 +13,16 @@ import { assertReleasedV2Header, releasedV2SessionFormatCodec, restoreReleasedV2
|
||||
export const sessionFormatCatalog = createSessionFormatCatalog({
|
||||
currentVersion: 2,
|
||||
codecs: [releasedV0SessionFormatCodec, releasedV1SessionFormatCodec, releasedV2SessionFormatCodec],
|
||||
encodeCurrentArtifact: artifact => releasedV2SessionFormatCodec.encodeArtifact(artifact),
|
||||
currentEncoder: releasedV2SessionFormatCodec,
|
||||
migrations: [sessionFormatV0ToV1, sessionFormatV1ToV2],
|
||||
restoreCurrent(artifact) {
|
||||
const restored = restoreReleasedV2Artifact(artifact, KNOWN_SESSION_EVENT_TYPES)
|
||||
validateInstalledCurrentSessionArtifact(restored)
|
||||
return restored
|
||||
},
|
||||
restoreTransformedCurrent(artifact) {
|
||||
return restoreReleasedV2Artifact(artifact, KNOWN_SESSION_EVENT_TYPES)
|
||||
},
|
||||
restoreCurrentHeader(header) {
|
||||
assertReleasedV2Header(header)
|
||||
validateInstalledCurrentSessionHeader(header)
|
||||
|
||||
@@ -27,10 +27,11 @@ describe('first-party Session format catalog', () => {
|
||||
})
|
||||
|
||||
const v1Header = { ...header, version: 1 }
|
||||
const current = sessionFormatCatalog.decodeArtifact(v1Header, [
|
||||
{ type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } },
|
||||
])
|
||||
expect(sessionFormatCatalog.migrate(current)).toMatchObject({
|
||||
const restore = sessionFormatCatalog.createRestore(v1Header, {
|
||||
recovery: 'strict', validation: 'current',
|
||||
})
|
||||
restore.decodeRow({ type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } })
|
||||
expect(restore.finish()).toMatchObject({
|
||||
header: { version: 2, id: 'catalog' },
|
||||
})
|
||||
})
|
||||
@@ -39,21 +40,38 @@ describe('first-party Session format catalog', () => {
|
||||
const header = {
|
||||
type: 'session', version: 2, id: 'current-growth', createdAt: 1, isSeeded: false, delegationDepth: 0,
|
||||
}
|
||||
const extended = sessionFormatCatalog.decodeArtifact(header, [{
|
||||
const restore = (rows: readonly unknown[]) => {
|
||||
const current = sessionFormatCatalog.createRestore(header, {
|
||||
recovery: 'strict', validation: 'current',
|
||||
})
|
||||
for (const row of rows) current.decodeRow(row)
|
||||
return current.finish()
|
||||
}
|
||||
const extended = restore([{
|
||||
type: 'turn/start', seq: 0, time: 1, data: { turn: 1, postReleaseMember: true },
|
||||
}])
|
||||
expect(sessionFormatCatalog.migrate(extended).events).toEqual(extended.events)
|
||||
|
||||
const unknownRequired = sessionFormatCatalog.decodeArtifact(header, [{
|
||||
type: 'ordinary/not-installed', seq: 0, time: 1, data: 'future',
|
||||
expect(extended.events).toEqual([{
|
||||
type: 'turn/start', seq: 0, time: 1, data: { turn: 1, postReleaseMember: true },
|
||||
}])
|
||||
expect(() => sessionFormatCatalog.migrate(unknownRequired)).toThrow(/unknown event type/)
|
||||
|
||||
const extension = sessionFormatCatalog.decodeArtifact(header, [{
|
||||
expect(() => restore([{
|
||||
type: 'ordinary/not-installed', seq: 0, time: 1, data: 'future',
|
||||
}])).toThrow(/unknown event type/)
|
||||
|
||||
const extension = restore([{
|
||||
type: 'ordinary/external', seq: 0, time: 1, data: null, ignorable: true,
|
||||
}])
|
||||
expect(sessionFormatCatalog.migrate(extension).events).toEqual([{
|
||||
expect(extension.events).toEqual([{
|
||||
type: 'ordinary/external', seq: 0, time: 1, data: null, ignorable: true,
|
||||
}])
|
||||
})
|
||||
|
||||
it('validates complete relationships after streaming migration', () => {
|
||||
const stream = sessionFormatCatalog.createRestore({
|
||||
type: 'session', version: 1, id: 'invalid-stream', createdAt: 1, delegationDepth: 0,
|
||||
}, { recovery: 'strict', validation: 'current' })
|
||||
stream.decodeRow({ type: 'step/start', seq: 0, time: 2, data: { turn: 1, step: 1 } })
|
||||
|
||||
expect(() => stream.finish()).toThrow(/open turn/)
|
||||
})
|
||||
})
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-format-v0-to-v1/README.md
|
||||
README.md: 829bca816c7770aecfc8cf98fbec34f3ee3a9741
|
||||
README.zh.md: 4c5566d28687bb9528d2e6070c7a03bdc52ff611
|
||||
README.md: 308e61fdd7fc97d4ab90bc965bdc7d9ac7e39b57
|
||||
README.zh.md: 6002e753ef5ac2924cfef4a28554487983145a9e
|
||||
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
## Summary
|
||||
|
||||
`dsh-session-format-v0-to-v1` decodes the complete released-v0 JSONL record language and converts it into the shared-layout v1 format. The edge preserves validated header and event facts except for `version: 0` becoming `version: 1`; it also applies the finite legacy normalizers that v0 persistence accepted. The package freezes the v0 reader, the strict v1 migration target validator, and a vocabulary-neutral v1 physical codec that a later edge can reuse without importing the latest Session representation. Most of its source is the frozen released v0/v1 event vocabulary rather than the identity conversion: `payload-validation.ts` and `relationships.ts` pin the payload members and lifecycle pairings of every first-party event type, so a malformed historical log is refused as an unsupported migration with its source retained before the installed current restorer runs, and a later edge that restructures released events can trust their shapes without importing the current Session package.
|
||||
`dsh-session-format-v0-to-v1` decodes the released-v0 JSONL record language one physical row at a time and converts it into the shared-layout v1 format. The edge preserves validated header and event facts except for `version: 0` becoming `version: 1`; it also applies the finite legacy normalizers that v0 persistence accepted. The package freezes the v0 reader, the strict v1 migration target validator, and a vocabulary-neutral v1 physical codec that a later edge can reuse without importing the latest Session representation. Most of its source is the frozen released v0/v1 event vocabulary rather than the identity conversion: `payload-validation.ts` and `relationships.ts` pin the payload members and lifecycle pairings of every first-party event type, so a malformed historical log is refused as an unsupported migration with its source retained before the installed current restorer runs, and a later edge that restructures released events can trust their fields without importing the current Session package.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -27,20 +27,24 @@ English | [中文](README.zh.md)
|
||||
|
||||
### When to use it
|
||||
|
||||
Persistence obtains this edge through `dsh-session-format-catalog`; feature compositions do not mount it. Import it directly only when assembling or testing the static released-format catalog. No runtime invariant companion is published because every codec and migration call validates its complete source or target artifact and retains no runtime state.
|
||||
Persistence obtains this edge through `dsh-session-format-catalog`; feature compositions do not mount it. Import it directly only when assembling or testing the static released-format catalog. No runtime invariant companion is published because the package has no independently observable runtime registrations whose state can diverge; decoder and migration-stage state belongs to one restore.
|
||||
|
||||
### Entry point
|
||||
|
||||
```text
|
||||
const decodedV0 = releasedV0SessionFormatCodec.decodeArtifact(header, rows)
|
||||
const migratedV1 = sessionFormatV0ToV1.migrate(decodedV0)
|
||||
const decoder = releasedV0SessionFormatCodec.createDecoder(physicalHeader, 'recoverable')
|
||||
for (const row of physicalRows) decoder.decodeRow(row, migrationContext)
|
||||
const inheritedEventCount = decoder.finish(migrationContext)
|
||||
const stage = sessionFormatV0ToV1.createStage(stageInput)
|
||||
stage.transformEvent(event, migrationContext)
|
||||
const targetInheritedEventCount = stage.finish(migrationContext)
|
||||
```
|
||||
|
||||
`releasedV0SessionFormatCodec` reads the exact v0 header and physical rows, including packed assistant deltas and range-encoded provenance. `sessionFormatV0ToV1` normalizes and strictly validates a complete detached artifact. `releasedV1SessionFormatCodec` preserves the v1 physical layout without freezing the ordinary event vocabulary; the catalog restores current events against the installed Session package.
|
||||
`releasedV0SessionFormatCodec` reads the exact v0 header and physical rows, including packed Assistant deltas and range-encoded provenance. Its decoder emits either a scalar event or a codec-owned compact run through `emitEvent()` and `emitRun()`. `sessionFormatV0ToV1` creates one stateful stage per restore; the static catalog connects that decoder and stage so migration does not retain a physical-row array. `releasedV1SessionFormatCodec` exposes the same row-at-a-time decoder for the v1 physical layout without freezing the ordinary event vocabulary.
|
||||
|
||||
The alpha edge refuses every event type outside its frozen inventory, including an unknown event marked `ignorable: true`. It also refuses unexpected payload members. `tool/result.meta` and nested PTC `arguments` remain explicit opaque JSON fields and are preserved without Session-sequence interpretation. Unknown content-block `type`, message-source `kind`, assistant finish-reason `kind`, and `turn/end` reason `kind` arms remain owner-opaque JSON while their known arms receive structural validation.
|
||||
|
||||
The bounded historical normalizers convert `steering/message` to `user/message`, remove `turn/start.trigger`, convert retired `turn/end` reasons, add the current message wrappers and deterministic legacy message ids, and remove the obsolete `request/header.header.messagePrefix` duplicate. Retired `request/header-delta`, `mode/set`, and the `request/header` fallback reason refuse migration. No other event, reference, source, or payload fact may change.
|
||||
The bounded historical normalizers convert `steering/message` to `user/message`, rename `compact/*` events to `compaction/*`, remove `turn/start.trigger`, convert retired `turn/end` reasons, add current message wrappers and deterministic ids for legacy messages, retry chains, and compaction groups, and remove the obsolete `request/header.header.messagePrefix` duplicate. Retired `request/header-delta`, `mode/set`, and the `request/header` fallback reason refuse migration. No other event, reference, source, or payload fact may change.
|
||||
|
||||
-----
|
||||
|
||||
@@ -50,7 +54,7 @@ The bounded historical normalizers convert `steering/message` to `user/message`,
|
||||
<details>
|
||||
<summary>Implementation internals — click to expand</summary>
|
||||
|
||||
The physical codec expands each packed row atomically and never mutates parsed input. Recoverable decoding rolls back a complete faulty row and keeps the preceding prefix unless a later decoded `turn/end` proves that the faulty region was committed. The migration validates the frozen payload disposition before changing the header version and validates the exact v1 target again.
|
||||
The physical codec validates each packed row atomically, emits it as a compact run, and never mutates parsed input. Recoverable decoding drops a complete faulty row and keeps the preceding prefix unless a later decoded `turn/end` proves that the faulty region was committed. The incremental normalizer retains only message, retry, and open-compaction identities; the catalog performs complete relationship validation on the final current artifact.
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
|
||||
@@ -9,7 +9,7 @@ kind: "package-library"
|
||||
|
||||
## 概述
|
||||
|
||||
`dsh-session-format-v0-to-v1` 解码完整的已发布 v0 JSONL 记录语言，并把它转换为共享布局的 v1 格式。除把 `version: 0` 改为 `version: 1` 外，该迁移边会保留经过校验的标头与事件事实；它也会应用 v0 持久化曾接受的有限旧格式规范化。该包冻结 v0 读取器、严格的 v1 迁移目标校验器，以及不冻结事件词表的 v1 物理编解码器，使后续迁移边无需导入最新 Session 表示即可复用它。它的大部分源码是冻结的已发布 v0/v1 事件词表而不是恒等转换本身：`payload-validation.ts` 与 `relationships.ts` 钉住每种第一方事件类型的 payload 成员与生命周期配对，使畸形历史日志在已安装的 current 恢复器运行之前就以「不支持的迁移」被拒绝并保留源文件，也使后续重构已发布事件的迁移边无需导入当前 Session 包即可信任其形状。
|
||||
`dsh-session-format-v0-to-v1` 逐个物理行解码已发布 v0 JSONL 记录语言，并把它转换为共享布局的 v1 格式。除把 `version: 0` 改为 `version: 1` 外，该迁移边会保留经过校验的 header 与事件事实；它也会应用 v0 持久化曾接受的有限旧格式规范化。该包冻结 v0 reader、严格的 v1 迁移目标校验器，以及词汇中立的 v1 物理 codec，使后续迁移边无需导入最新 Session 表示即可复用它。它的大部分源码是冻结的已发布 v0/v1 事件词表而不是恒等转换本身：`payload-validation.ts` 与 `relationships.ts` 钉住每种第一方事件类型的 payload 成员与生命周期配对，使畸形历史日志在已安装的 current restorer 运行之前就以「不支持的迁移」被拒绝并保留源文件，也使后续重构已发布事件的迁移边无需导入当前 Session 包即可信任其字段。
|
||||
|
||||
## 目录
|
||||
|
||||
@@ -27,20 +27,24 @@ kind: "package-library"
|
||||
|
||||
### 何时使用
|
||||
|
||||
持久化通过 `dsh-session-format-catalog` 获取该迁移边；功能组合不会挂载它。只有在装配或测试静态已发布格式目录时，才直接导入本包。它不发布运行时不变式伴生入口，因为每次 codec 与迁移调用都会校验完整的源或目标 artifact，且不保留运行时状态。
|
||||
持久化通过 `dsh-session-format-catalog` 获取该迁移边；功能组合不会挂载它。只有在装配或测试静态已发布格式目录时，才直接导入本包。它不发布运行时不变式伴生入口，因为本包没有状态可能彼此分歧的、可独立观测的运行时注册项；decoder 与 migration stage 的状态只属于一次还原。
|
||||
|
||||
### 入口
|
||||
|
||||
```text
|
||||
const decodedV0 = releasedV0SessionFormatCodec.decodeArtifact(header, rows)
|
||||
const migratedV1 = sessionFormatV0ToV1.migrate(decodedV0)
|
||||
const decoder = releasedV0SessionFormatCodec.createDecoder(physicalHeader, 'recoverable')
|
||||
for (const row of physicalRows) decoder.decodeRow(row, migrationContext)
|
||||
const inheritedEventCount = decoder.finish(migrationContext)
|
||||
const stage = sessionFormatV0ToV1.createStage(stageInput)
|
||||
stage.transformEvent(event, migrationContext)
|
||||
const targetInheritedEventCount = stage.finish(migrationContext)
|
||||
```
|
||||
|
||||
`releasedV0SessionFormatCodec` 读取精确的 v0 标头与物理行，包括打包的 Assistant 增量和范围编码的来源序号。`sessionFormatV0ToV1` 规范化并严格校验一个完整且分离的产物。`releasedV1SessionFormatCodec` 在不冻结普通事件词表的前提下保留 v1 物理布局；目录会根据已安装的 Session 包还原当前事件。
|
||||
`releasedV0SessionFormatCodec` 读取精确的 v0 header 与物理行，包括打包的 Assistant 增量和范围编码的来源序号。它的 decoder 通过 `emitEvent()` 与 `emitRun()` 发出单个事件或 codec 自有的紧凑 run。`sessionFormatV0ToV1` 为每次还原创建一个有状态 Stage；静态 catalog 连接该 decoder 与 Stage，使迁移无需保留物理行数组。`releasedV1SessionFormatCodec` 为 v1 物理布局暴露相同的逐行 decoder，同时不冻结普通事件词表。
|
||||
|
||||
Alpha 迁移边会拒绝冻结清单之外的所有事件类型，包括带有 `ignorable: true` 标记的未知事件。它也会拒绝意外的 payload 成员。`tool/result.meta` 与嵌套 PTC `arguments` 是显式的不透明 JSON 字段；迁移会原样保留它们，不把其中的数字解释为 Session 序号。未知 content-block `type`、message-source `kind`、assistant finish-reason `kind` 与 `turn/end` reason `kind` 分支保持 owner-opaque JSON，已知分支则接受结构校验。
|
||||
|
||||
有限的历史规范化会把 `steering/message` 转换为 `user/message`、移除 `turn/start.trigger`、转换已停用的 `turn/end` reason、添加当前消息包装层与确定性的旧消息 id，并移除已停用且重复的 `request/header.header.messagePrefix`。已停用的 `request/header-delta`、`mode/set` 和 `request/header` fallback reason 会使迁移失败。除此之外，任何事件、引用、来源或 payload 事实都不得改变。
|
||||
有限的历史规范化会把 `steering/message` 转换为 `user/message`、把 `compact/*` 事件重命名为 `compaction/*`、移除 `turn/start.trigger`、转换已停用的 `turn/end` reason、添加当前消息包装层，并为旧 message、retry chain 与 compaction group 补充确定性 id，同时移除已停用且重复的 `request/header.header.messagePrefix`。已停用的 `request/header-delta`、`mode/set` 和 `request/header` fallback reason 会使迁移失败。除此之外，任何事件、引用、来源或 payload 事实都不得改变。
|
||||
|
||||
-----
|
||||
|
||||
@@ -50,7 +54,7 @@ Alpha 迁移边会拒绝冻结清单之外的所有事件类型，包括带有 `
|
||||
<details>
|
||||
<summary>实现细节——点击展开</summary>
|
||||
|
||||
物理编解码器会以行为原子单位展开每个打包行，且绝不修改已解析输入。可恢复解码会回滚完整的故障行并保留此前前缀，除非后续成功解码的 `turn/end` 证明故障区域已经提交。迁移会先校验冻结的 payload 处置，再更改标头版本，并再次校验精确的 v1 目标。
|
||||
物理 codec 会以行为原子单位校验每个打包行，以紧凑 run 发出它，且绝不修改已解析输入。可恢复解码会丢弃完整的故障行并保留此前前缀，除非后续成功解码的 `turn/end` 证明故障区域已经提交。增量 normalizer 只保留 message、retry 与未结束 compaction 的 identity；catalog 会在最终当前 artifact 上执行完整关系校验。
|
||||
|
||||
| 文件 | 职责 |
|
||||
|---|---|
|
||||
|
||||
@@ -1,32 +1,46 @@
|
||||
import {
|
||||
SessionFormatError,
|
||||
isSessionFormatJsonObject,
|
||||
sessionFormatCount,
|
||||
sessionFormatSafeInteger,
|
||||
snapshotSessionFormatArtifact,
|
||||
snapshotSessionFormatJson,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
EncodedSessionFormatArtifact,
|
||||
SessionFormatArtifact,
|
||||
SessionFormatArtifactDecoder,
|
||||
SessionFormatCodec,
|
||||
SessionFormatEncodeOptions,
|
||||
SessionFormatEvent,
|
||||
SessionFormatEventRun,
|
||||
SessionFormatHeader,
|
||||
SessionFormatJsonObject,
|
||||
SessionFormatJsonValue,
|
||||
SessionFormatMigrationContext,
|
||||
SessionFormatRecovery,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
assertReleasedSessionFormatHeader,
|
||||
assertReleasedV0SourceArtifact,
|
||||
assertReleasedV1PhysicalArtifact,
|
||||
} from './validation.ts'
|
||||
import { assertReleasedSessionFormatHeader } from './validation.ts'
|
||||
import { assertReleasedV0Keys, releasedV0Record } from './validation-helpers.ts'
|
||||
|
||||
const PHYSICAL_HEADER_REQUIRED = ['type', 'version', 'id', 'createdAt', 'delegationDepth'] as const
|
||||
const PHYSICAL_HEADER_OPTIONAL = ['cwd', 'parentSession', 'seedLength', 'origin', 'agentPreset'] as const
|
||||
const PACKED_TAGS = new Set(['text-chunks', 'reasoning-chunks', 'tool-call-chunks'])
|
||||
|
||||
/** A released packed Assistant row retained until v1-to-v2 embeds its compact stream. */
|
||||
export interface ReleasedAssistantChunkRun extends SessionFormatEventRun {
|
||||
readonly runType: 'released-assistant-chunks'
|
||||
readonly turn: number
|
||||
readonly step: number
|
||||
readonly lastSeq: number
|
||||
readonly lastTime: number
|
||||
readonly stream: SessionFormatJsonObject
|
||||
}
|
||||
|
||||
/**
|
||||
* Test whether a compact migration item is a released packed Assistant row.
|
||||
* @param run - compact migration item to classify.
|
||||
* @returns whether the item carries the released Assistant chunk representation.
|
||||
*/
|
||||
export function isReleasedAssistantChunkRun(run: SessionFormatEventRun): run is ReleasedAssistantChunkRun {
|
||||
return run.runType === 'released-assistant-chunks'
|
||||
}
|
||||
|
||||
/** Frozen physical JSON codec for the released v0 layout. */
|
||||
export const releasedV0SessionFormatCodec = createReleasedCodec(0)
|
||||
|
||||
@@ -37,40 +51,88 @@ function createReleasedCodec(version: 0 | 1) {
|
||||
return Object.freeze({
|
||||
version,
|
||||
decodeHeader: (value: unknown) => decodeHeader(value, version),
|
||||
decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]) {
|
||||
createDecoder(headerValue: unknown, recovery: SessionFormatRecovery) {
|
||||
const physical = decodePhysicalHeader(headerValue, version)
|
||||
const artifact = snapshotSessionFormatArtifact({
|
||||
const scanner = scanRows(recovery === 'recoverable')
|
||||
return {
|
||||
header: physical.header,
|
||||
inheritedEventCount: physical.inheritedEventCount,
|
||||
events: scanRows(rowValues, false).events,
|
||||
}, `released v${version} artifact`)
|
||||
if (version === 0) assertReleasedV0SourceArtifact(artifact)
|
||||
else assertReleasedV1PhysicalArtifact(artifact)
|
||||
return artifact
|
||||
headerInheritedEventCount: physical.inheritedEventCount,
|
||||
decodeRow: (rowValue, context) => { scanner.decodeRow(rowValue, context) },
|
||||
finish(_context) {
|
||||
scanner.finish(physical.inheritedEventCount)
|
||||
return physical.inheritedEventCount
|
||||
},
|
||||
} satisfies SessionFormatArtifactDecoder
|
||||
},
|
||||
decodeRecoverableArtifact(headerValue: unknown, rowValues: readonly unknown[]) {
|
||||
const physical = decodePhysicalHeader(headerValue, version)
|
||||
const recovered = scanRows(rowValues, true)
|
||||
const artifact = snapshotSessionFormatArtifact({
|
||||
header: physical.header,
|
||||
inheritedEventCount: physical.inheritedEventCount,
|
||||
events: recovered.events,
|
||||
}, `released v${version} recoverable artifact`)
|
||||
if (version === 0) assertReleasedV0SourceArtifact(artifact)
|
||||
else assertReleasedV1PhysicalArtifact(artifact)
|
||||
return artifact
|
||||
} satisfies SessionFormatCodec)
|
||||
}
|
||||
|
||||
function scanRows(
|
||||
recoverable: boolean,
|
||||
): {
|
||||
decodeRow(
|
||||
rowValue: unknown,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void
|
||||
finish(inheritedEventCount: number): void
|
||||
} {
|
||||
let rowIndex = 0
|
||||
let eventCount = 0
|
||||
let issue: SessionFormatError | undefined
|
||||
return {
|
||||
decodeRow(rowValue, context) {
|
||||
const currentRow = rowIndex
|
||||
rowIndex += 1
|
||||
let packed = false
|
||||
let decoded: SessionFormatEvent | ReleasedAssistantChunkRun
|
||||
try {
|
||||
const record = releasedV0Record(rowValue, `released Session row ${currentRow}`)
|
||||
const type = record['type']
|
||||
if (typeof type === 'string' && PACKED_TAGS.has(type)) {
|
||||
packed = true
|
||||
decoded = decodePackedRun(record, type, currentRow)
|
||||
} else {
|
||||
decoded = decodeEvent(record, currentRow)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
const current = error instanceof SessionFormatError
|
||||
? error
|
||||
: new SessionFormatError(`released Session row ${currentRow} is malformed`, { cause: error })
|
||||
if (!recoverable) throw current
|
||||
issue ??= current
|
||||
return
|
||||
}
|
||||
if (issue !== undefined) {
|
||||
if (!packed && (decoded as SessionFormatEvent).type === 'turn/end') throw issue
|
||||
return
|
||||
}
|
||||
const seq = packed
|
||||
? (decoded as ReleasedAssistantChunkRun).firstSeq
|
||||
: (decoded as SessionFormatEvent).seq
|
||||
if (seq !== eventCount) {
|
||||
const gap = new SessionFormatError(
|
||||
`released Session row ${currentRow} has seq gap (expected ${eventCount}, got ${seq})`,
|
||||
)
|
||||
if (!recoverable) throw gap
|
||||
issue = gap
|
||||
if (!packed && (decoded as SessionFormatEvent).type === 'turn/end') throw gap
|
||||
return
|
||||
}
|
||||
if (packed) {
|
||||
const run = decoded as ReleasedAssistantChunkRun
|
||||
eventCount += run.eventCount
|
||||
context.emitRun(run)
|
||||
} else {
|
||||
eventCount += 1
|
||||
context.emitEvent(decoded as SessionFormatEvent)
|
||||
}
|
||||
},
|
||||
encodeArtifact(artifact: SessionFormatArtifact, options: SessionFormatEncodeOptions) {
|
||||
if (version === 0) assertReleasedV0SourceArtifact(artifact)
|
||||
else assertReleasedV1PhysicalArtifact(artifact)
|
||||
return encodeArtifact(artifact, options, version)
|
||||
finish(inheritedEventCount) {
|
||||
if (inheritedEventCount > eventCount) {
|
||||
throw new SessionFormatError('Session inheritedEventCount exceeds its event count')
|
||||
}
|
||||
},
|
||||
} satisfies SessionFormatCodec & {
|
||||
encodeArtifact(
|
||||
artifact: SessionFormatArtifact,
|
||||
options: SessionFormatEncodeOptions,
|
||||
): EncodedSessionFormatArtifact
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
function decodeHeader(value: unknown, version: 0 | 1): SessionFormatHeader {
|
||||
@@ -109,7 +171,7 @@ function decodePhysicalHeader(
|
||||
if (record['origin'] !== undefined && record['origin'] !== 'subagent') {
|
||||
throw new SessionFormatError(`released v${version} header origin must be "subagent"`)
|
||||
}
|
||||
const header = snapshotSessionFormatJson({
|
||||
const header = {
|
||||
version,
|
||||
id: record['id'],
|
||||
createdAt,
|
||||
@@ -119,101 +181,34 @@ function decodePhysicalHeader(
|
||||
...(record['origin'] === undefined ? {} : { origin: record['origin'] }),
|
||||
delegationDepth,
|
||||
...(record['agentPreset'] === undefined ? {} : { agentPreset: record['agentPreset'] }),
|
||||
}, `released v${version} logical header`) as SessionFormatHeader
|
||||
} as SessionFormatHeader
|
||||
assertReleasedSessionFormatHeader(header, version)
|
||||
return { header, inheritedEventCount: seedLength }
|
||||
}
|
||||
|
||||
function encodeArtifact(
|
||||
artifact: SessionFormatArtifact,
|
||||
options: SessionFormatEncodeOptions,
|
||||
version: 0 | 1,
|
||||
): EncodedSessionFormatArtifact {
|
||||
const header = artifact.header
|
||||
const physicalHeader = snapshotSessionFormatJson({
|
||||
type: 'session',
|
||||
version,
|
||||
id: header.id,
|
||||
createdAt: header.createdAt,
|
||||
...(header.cwd === undefined ? {} : { cwd: header.cwd }),
|
||||
...(header.parentSession === undefined ? {} : { parentSession: header.parentSession }),
|
||||
...(header.isSeeded ? { seedLength: artifact.inheritedEventCount } : {}),
|
||||
...(header.origin === undefined ? {} : { origin: header.origin }),
|
||||
delegationDepth: header.delegationDepth,
|
||||
...(header.agentPreset === undefined ? {} : { agentPreset: header.agentPreset }),
|
||||
}, `released v${version} encoded header`) as SessionFormatJsonObject
|
||||
const records = options.packChunks ? packChunkRuns(artifact.events) : [...artifact.events]
|
||||
const rows = Object.freeze(records.map(record => encodeProvenance(record)))
|
||||
return Object.freeze({ header: physicalHeader, rows })
|
||||
}
|
||||
|
||||
function scanRows(
|
||||
rowValues: readonly unknown[],
|
||||
recoverable: boolean,
|
||||
): { readonly events: readonly SessionFormatEvent[] } {
|
||||
const events: SessionFormatEvent[] = []
|
||||
let issue: SessionFormatError | undefined
|
||||
for (const [rowIndex, value] of rowValues.entries()) {
|
||||
let decoded: readonly SessionFormatEvent[]
|
||||
try {
|
||||
const row = snapshotSessionFormatJson(value, `released Session row ${rowIndex}`)
|
||||
decoded = decodeRow(row, rowIndex)
|
||||
} catch (error: unknown) {
|
||||
const current = error instanceof SessionFormatError
|
||||
? error
|
||||
: new SessionFormatError(`released Session row ${rowIndex} is malformed`, { cause: error })
|
||||
if (!recoverable) throw current
|
||||
issue ??= current
|
||||
continue
|
||||
}
|
||||
if (issue !== undefined) {
|
||||
if (decoded.some(event => event.type === 'turn/end')) throw issue
|
||||
continue
|
||||
}
|
||||
const rowStart = events.length
|
||||
for (const event of decoded) {
|
||||
if (event.seq !== events.length) {
|
||||
const gap = new SessionFormatError(
|
||||
`released Session row ${rowIndex} has seq gap (expected ${events.length}, got ${event.seq})`,
|
||||
)
|
||||
events.length = rowStart
|
||||
if (!recoverable) throw gap
|
||||
issue = gap
|
||||
break
|
||||
}
|
||||
events.push(event)
|
||||
}
|
||||
if (issue !== undefined) {
|
||||
if (decoded.some(event => event.type === 'turn/end')) throw issue
|
||||
continue
|
||||
}
|
||||
}
|
||||
return Object.freeze({ events: Object.freeze(events) })
|
||||
}
|
||||
|
||||
function decodeRow(value: SessionFormatJsonValue, rowIndex: number): readonly SessionFormatEvent[] {
|
||||
const record = releasedV0Record(value, `released Session row ${rowIndex}`)
|
||||
const type = record['type']
|
||||
if (typeof type === 'string' && PACKED_TAGS.has(type)) return expandPackedRow(record, type, rowIndex)
|
||||
function decodeEvent(
|
||||
record: Record<string, SessionFormatJsonValue>,
|
||||
rowIndex: number,
|
||||
): SessionFormatEvent {
|
||||
if (record['sourceEventSeqs'] !== undefined) {
|
||||
const seq = sessionFormatCount(record['seq'], `released Session row ${rowIndex} seq`)
|
||||
return Object.freeze([{
|
||||
return {
|
||||
...record,
|
||||
sourceEventSeqs: decodeSeqRanges(record['sourceEventSeqs'], seq),
|
||||
} as unknown as SessionFormatEvent])
|
||||
} as unknown as SessionFormatEvent
|
||||
}
|
||||
return Object.freeze([record as unknown as SessionFormatEvent])
|
||||
return record as unknown as SessionFormatEvent
|
||||
}
|
||||
|
||||
function expandPackedRow(
|
||||
function decodePackedRun(
|
||||
row: Record<string, SessionFormatJsonValue>,
|
||||
type: string,
|
||||
rowIndex: number,
|
||||
): readonly SessionFormatEvent[] {
|
||||
): ReleasedAssistantChunkRun {
|
||||
const label = `released ${type} row ${rowIndex}`
|
||||
assertReleasedV0Keys(row, ['type', 'seq0', 'time0', 'data'], [], label)
|
||||
const seq0 = sessionFormatCount(row['seq0'], `${label} seq0`)
|
||||
let time = sessionFormatSafeInteger(row['time0'], `${label} time0`)
|
||||
const time0 = sessionFormatSafeInteger(row['time0'], `${label} time0`)
|
||||
const data = releasedV0Record(row['data'], `${label} data`)
|
||||
const isTool = type === 'tool-call-chunks'
|
||||
assertReleasedV0Keys(
|
||||
@@ -230,37 +225,73 @@ function expandPackedRow(
|
||||
if (!Array.isArray(gaps) || gaps.length !== payload.length - 1) {
|
||||
throw new SessionFormatError(`${label} dt length must match its payload`)
|
||||
}
|
||||
for (const gap of gaps) sessionFormatSafeInteger(gap, `${label} dt member`)
|
||||
if (typeof data['turn'] !== 'number' || typeof data['step'] !== 'number' || typeof data['index'] !== 'number') {
|
||||
throw new SessionFormatError(`${label} turn, step, and index must be numbers`)
|
||||
let lastTime = time0
|
||||
for (const gap of gaps) {
|
||||
const validGap = sessionFormatSafeInteger(gap, `${label} dt member`)
|
||||
lastTime = sessionFormatSafeInteger(lastTime + validGap, `${label} member time`)
|
||||
}
|
||||
const turn = sessionFormatCount(data['turn'], `${label} turn`)
|
||||
const step = sessionFormatCount(data['step'], `${label} step`)
|
||||
const chunkIndex = sessionFormatCount(data['index'], `${label} index`)
|
||||
if (isTool && (typeof data['id'] !== 'string'
|
||||
|| data['id'].length === 0
|
||||
|| (data['name'] !== undefined && typeof data['name'] !== 'string'))) {
|
||||
throw new SessionFormatError(`${label} id and optional name must be strings`)
|
||||
}
|
||||
const output: SessionFormatEvent[] = []
|
||||
for (let index = 0; index < payload.length; index += 1) {
|
||||
if (index > 0) time = sessionFormatSafeInteger(time + (gaps[index - 1] as number), `${label} member time`)
|
||||
const member = payload[index] as string
|
||||
const chunk = type === 'text-chunks'
|
||||
? { type: 'text-delta', index: data['index'], text: member }
|
||||
: type === 'reasoning-chunks'
|
||||
? { type: 'reasoning-delta', index: data['index'], text: member }
|
||||
const lastSeq = sessionFormatCount(seq0 + payload.length - 1, `${label} final seq`)
|
||||
const stream = (type === 'tool-call-chunks'
|
||||
? {
|
||||
type,
|
||||
time0,
|
||||
index: chunkIndex,
|
||||
dt: gaps,
|
||||
id: data['id'],
|
||||
...(data['name'] === undefined ? {} : { name: data['name'] }),
|
||||
args: payload,
|
||||
}
|
||||
: { type, time0, index: chunkIndex, dt: gaps, texts: payload }) as SessionFormatJsonObject
|
||||
const run: ReleasedAssistantChunkRun = {
|
||||
runType: 'released-assistant-chunks',
|
||||
firstSeq: seq0,
|
||||
eventCount: payload.length,
|
||||
turn,
|
||||
step,
|
||||
lastSeq,
|
||||
lastTime,
|
||||
stream,
|
||||
expand: () => expandAssistantChunkRun(run),
|
||||
}
|
||||
return run
|
||||
}
|
||||
|
||||
function* expandAssistantChunkRun(run: ReleasedAssistantChunkRun): Iterable<SessionFormatEvent> {
|
||||
const stream = run.stream
|
||||
const gaps = stream['dt'] as readonly number[]
|
||||
const members = stream['type'] === 'tool-call-chunks'
|
||||
? stream['args'] as readonly string[]
|
||||
: stream['texts'] as readonly string[]
|
||||
let time = run.stream['time0'] as number
|
||||
for (let index = 0; index < members.length; index += 1) {
|
||||
if (index > 0) time += gaps[index - 1] as number
|
||||
const member = members[index] as string
|
||||
const chunk = stream['type'] === 'text-chunks'
|
||||
? { type: 'text-delta', index: stream['index'], text: member }
|
||||
: stream['type'] === 'reasoning-chunks'
|
||||
? { type: 'reasoning-delta', index: stream['index'], text: member }
|
||||
: {
|
||||
type: 'tool-call-delta',
|
||||
index: data['index'],
|
||||
id: data['id'],
|
||||
...(data['name'] === undefined ? {} : { name: data['name'] }),
|
||||
index: stream['index'],
|
||||
id: stream['id'],
|
||||
...(stream['name'] === undefined ? {} : { name: stream['name'] }),
|
||||
argumentsDelta: member,
|
||||
}
|
||||
output.push(snapshotSessionFormatJson({
|
||||
yield {
|
||||
type: 'assistant/chunk',
|
||||
seq: sessionFormatCount(seq0 + index, `${label} member seq`),
|
||||
seq: run.firstSeq + index,
|
||||
time,
|
||||
data: { turn: data['turn'], step: data['step'], chunk },
|
||||
}, `${label} member`) as SessionFormatEvent)
|
||||
data: { turn: run.turn, step: run.step, chunk },
|
||||
} as SessionFormatEvent
|
||||
}
|
||||
return Object.freeze(output)
|
||||
}
|
||||
|
||||
function decodeSeqRanges(value: SessionFormatJsonValue, maxEntries: number): readonly SessionFormatJsonValue[] {
|
||||
@@ -287,130 +318,5 @@ function decodeSeqRanges(value: SessionFormatJsonValue, maxEntries: number): rea
|
||||
if (hasRange && output.some((member, index) => index > 0 && member <= (output[index - 1] as number))) {
|
||||
throw new SessionFormatError('sourceEventSeqs ranges must be strictly increasing')
|
||||
}
|
||||
return Object.freeze(output)
|
||||
}
|
||||
|
||||
function encodeProvenance(record: SessionFormatEvent | SessionFormatJsonObject): SessionFormatJsonObject {
|
||||
if (!Object.hasOwn(record, 'sourceEventSeqs')) return record
|
||||
const sourceEventSeqs = record['sourceEventSeqs'] as readonly SessionFormatJsonValue[]
|
||||
const values = sourceEventSeqs.map(value => sessionFormatCount(value, 'sourceEventSeqs member'))
|
||||
return snapshotSessionFormatJson({ ...record, sourceEventSeqs: encodeSeqRanges(values) }) as SessionFormatJsonObject
|
||||
}
|
||||
|
||||
function encodeSeqRanges(values: readonly number[]): readonly SessionFormatJsonValue[] {
|
||||
if (values.some((value, index) => index > 0 && value <= (values[index - 1] as number))) return Object.freeze([...values])
|
||||
const output: SessionFormatJsonValue[] = []
|
||||
for (let start = 0; start < values.length;) {
|
||||
let end = start
|
||||
while (end + 1 < values.length && values[end + 1] === (values[end] as number) + 1) end += 1
|
||||
if (end - start >= 2) output.push(Object.freeze([values[start] as number, values[end] as number]))
|
||||
else for (let index = start; index <= end; index += 1) output.push(values[index] as number)
|
||||
start = end + 1
|
||||
}
|
||||
return Object.freeze(output)
|
||||
}
|
||||
|
||||
type ChunkKind = 'text-delta' | 'reasoning-delta' | 'tool-call-delta'
|
||||
|
||||
function packChunkRuns(events: readonly SessionFormatEvent[]): readonly (SessionFormatEvent | SessionFormatJsonObject)[] {
|
||||
const output: Array<SessionFormatEvent | SessionFormatJsonObject> = []
|
||||
let kind: ChunkKind | undefined
|
||||
let run: SessionFormatEvent[] = []
|
||||
const flush = (): void => {
|
||||
if (kind !== undefined && run.length >= 3) output.push(buildPackedRow(kind, run))
|
||||
else output.push(...run)
|
||||
kind = undefined
|
||||
run = []
|
||||
}
|
||||
for (const event of events) {
|
||||
const candidate = classifyChunk(event)
|
||||
const previous = run.at(-1)
|
||||
if (candidate !== undefined && candidate === kind && previous !== undefined && continuesChunk(previous, event, candidate)) {
|
||||
run.push(event)
|
||||
continue
|
||||
}
|
||||
flush()
|
||||
if (candidate === undefined) output.push(event)
|
||||
else {
|
||||
kind = candidate
|
||||
run = [event]
|
||||
}
|
||||
}
|
||||
flush()
|
||||
return Object.freeze(output)
|
||||
}
|
||||
|
||||
function classifyChunk(event: SessionFormatEvent): ChunkKind | undefined {
|
||||
if (event.type !== 'assistant/chunk' || !hasExactKeys(event, ['type', 'seq', 'time', 'data'])) return undefined
|
||||
const data = event.data
|
||||
if (!isSessionFormatJsonObject(data) || !hasExactKeys(data, ['turn', 'step', 'chunk'])) return undefined
|
||||
const chunk = data['chunk']
|
||||
if (!isSessionFormatJsonObject(chunk)
|
||||
|| typeof chunk['index'] !== 'number'
|
||||
|| typeof chunk['type'] !== 'string') return undefined
|
||||
if (chunk['type'] === 'text-delta' || chunk['type'] === 'reasoning-delta') {
|
||||
return hasExactKeys(chunk, ['type', 'index', 'text']) && typeof chunk['text'] === 'string'
|
||||
? chunk['type']
|
||||
: undefined
|
||||
}
|
||||
if (chunk['type'] !== 'tool-call-delta') return undefined
|
||||
const exact = hasExactKeys(chunk, ['type', 'index', 'id', 'argumentsDelta'])
|
||||
|| hasExactKeys(chunk, ['type', 'index', 'id', 'name', 'argumentsDelta'])
|
||||
return exact && typeof chunk['id'] === 'string'
|
||||
&& typeof chunk['argumentsDelta'] === 'string'
|
||||
&& (chunk['name'] === undefined || typeof chunk['name'] === 'string')
|
||||
? 'tool-call-delta'
|
||||
: undefined
|
||||
}
|
||||
|
||||
function continuesChunk(previous: SessionFormatEvent, next: SessionFormatEvent, kind: ChunkKind): boolean {
|
||||
const previousData = previous.data as SessionFormatJsonObject
|
||||
const nextData = next.data as SessionFormatJsonObject
|
||||
const previousChunk = previousData['chunk'] as SessionFormatJsonObject
|
||||
const nextChunk = nextData['chunk'] as SessionFormatJsonObject
|
||||
if (!Number.isSafeInteger(next.time - previous.time)) return false
|
||||
if (nextData['turn'] !== previousData['turn'] || nextData['step'] !== previousData['step']) return false
|
||||
if (nextChunk['index'] !== previousChunk['index']) return false
|
||||
if (kind !== 'tool-call-delta') return true
|
||||
return nextChunk['id'] === previousChunk['id']
|
||||
&& Object.hasOwn(nextChunk, 'name') === Object.hasOwn(previousChunk, 'name')
|
||||
&& nextChunk['name'] === previousChunk['name']
|
||||
}
|
||||
|
||||
function buildPackedRow(kind: ChunkKind, run: readonly SessionFormatEvent[]): SessionFormatJsonObject {
|
||||
const first = run[0] as SessionFormatEvent
|
||||
const firstData = first.data as SessionFormatJsonObject
|
||||
const firstChunk = firstData['chunk'] as SessionFormatJsonObject
|
||||
const base = {
|
||||
turn: firstData['turn'],
|
||||
step: firstData['step'],
|
||||
index: firstChunk['index'],
|
||||
dt: run.slice(1).map((event, index) => event.time - (run[index] as SessionFormatEvent).time),
|
||||
}
|
||||
if (kind === 'tool-call-delta') {
|
||||
return snapshotSessionFormatJson({
|
||||
type: 'tool-call-chunks',
|
||||
seq0: first.seq,
|
||||
time0: first.time,
|
||||
data: {
|
||||
...base,
|
||||
id: firstChunk['id'],
|
||||
...(firstChunk['name'] === undefined ? {} : { name: firstChunk['name'] }),
|
||||
args: run.map(event => ((event.data as SessionFormatJsonObject)['chunk'] as SessionFormatJsonObject)['argumentsDelta']),
|
||||
},
|
||||
}) as SessionFormatJsonObject
|
||||
}
|
||||
return snapshotSessionFormatJson({
|
||||
type: kind === 'text-delta' ? 'text-chunks' : 'reasoning-chunks',
|
||||
seq0: first.seq,
|
||||
time0: first.time,
|
||||
data: {
|
||||
...base,
|
||||
texts: run.map(event => ((event.data as SessionFormatJsonObject)['chunk'] as SessionFormatJsonObject)['text']),
|
||||
},
|
||||
}) as SessionFormatJsonObject
|
||||
}
|
||||
|
||||
function hasExactKeys(record: Readonly<Record<string, unknown>>, keys: readonly string[]): boolean {
|
||||
return Object.keys(record).length === keys.length && keys.every(key => Object.hasOwn(record, key))
|
||||
return output
|
||||
}
|
||||
@@ -7,7 +7,7 @@ export { assertReleasedPayloadSemantics } from './payload-validation.ts'
|
||||
export { assertReleasedArtifactRelationships } from './relationships.ts'
|
||||
export {
|
||||
assertReleasedSurfaceMetadata,
|
||||
assertReleasedV1Artifact,
|
||||
assertReleasedEventPayload,
|
||||
assertReleasedV1Header,
|
||||
restoreReleasedV1Artifact,
|
||||
} from './validation.ts'
|
||||
@@ -3,19 +3,20 @@ import {
|
||||
SessionFormatUnsupportedMigrationError,
|
||||
defineSessionFormatMigration,
|
||||
sessionFormatCount,
|
||||
snapshotSessionFormatArtifact,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
SessionFormatEvent,
|
||||
SessionFormatEventRun,
|
||||
SessionFormatHeader,
|
||||
SessionFormatJsonObject,
|
||||
SessionFormatJsonValue,
|
||||
SessionFormatMigrationContext,
|
||||
SessionFormatMigrationStage,
|
||||
SessionFormatMigrationStageInput,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import { isReleasedAssistantChunkRun } from './codec.ts'
|
||||
import {
|
||||
assertReleasedEventPayload,
|
||||
assertNormalizedReleasedV0Artifact,
|
||||
assertReleasedV0SourceArtifact,
|
||||
assertReleasedV1Artifact,
|
||||
assertReleasedV1Header,
|
||||
} from './validation.ts'
|
||||
import { assertReleasedV0Keys, releasedV0Record } from './validation-helpers.ts'
|
||||
@@ -29,45 +30,179 @@ export const sessionFormatV0ToV1 = defineSessionFormatMigration({
|
||||
assertHeaderVersion(header, 0)
|
||||
return { ...header, version: 1 }
|
||||
},
|
||||
migrate(source) {
|
||||
assertReleasedV0SourceArtifact(source)
|
||||
const events = normalizeReleasedV0Events(source.events, source.header.id)
|
||||
assertNormalizedReleasedV0Artifact({ ...source, events })
|
||||
const target = snapshotSessionFormatArtifact({
|
||||
header: { ...source.header, version: 1 },
|
||||
inheritedEventCount: source.inheritedEventCount,
|
||||
events,
|
||||
}, 'released v0-to-v1 target')
|
||||
assertReleasedV1Artifact(target)
|
||||
return target
|
||||
createStage(input) {
|
||||
return new ReleasedV0ToV1Stage(input)
|
||||
},
|
||||
validateTarget: assertReleasedV1Artifact,
|
||||
validateTargetHeader: assertReleasedV1Header,
|
||||
})
|
||||
|
||||
class ReleasedV0ToV1Stage implements SessionFormatMigrationStage {
|
||||
readonly headerInheritedEventCount: number
|
||||
private readonly state: LegacyNormalizationState = { messageIds: new Map(), retryIds: new Map() }
|
||||
|
||||
constructor(private readonly input: SessionFormatMigrationStageInput) {
|
||||
assertHeaderVersion(input.sourceHeader, 0)
|
||||
this.headerInheritedEventCount = input.sourceInheritedEventCount
|
||||
}
|
||||
|
||||
transformEvent(
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
const normalized = normalizeReleasedV0Event(event, this.input.sourceHeader.id, this.state)
|
||||
assertSourceDeliveryMarker(normalized, this.input)
|
||||
context.emitEvent(normalized)
|
||||
}
|
||||
|
||||
transformRun(
|
||||
run: SessionFormatEventRun,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
if (isReleasedAssistantChunkRun(run)) {
|
||||
context.emitRun(run)
|
||||
return
|
||||
}
|
||||
for (const event of run.expand()) this.transformEvent(event, context)
|
||||
}
|
||||
|
||||
finish(_context: SessionFormatMigrationContext): number {
|
||||
return this.input.sourceInheritedEventCount
|
||||
}
|
||||
}
|
||||
|
||||
function assertHeaderVersion(header: SessionFormatHeader, version: 0 | 1): void {
|
||||
if (header.version !== version) throw new SessionFormatError(`expected format v${version} header`)
|
||||
}
|
||||
|
||||
function normalizeReleasedV0Events(
|
||||
events: readonly SessionFormatEvent[],
|
||||
interface LegacyNormalizationState {
|
||||
readonly messageIds: Map<number, string>
|
||||
readonly retryIds: Map<string, string>
|
||||
compactionId?: string
|
||||
}
|
||||
|
||||
function normalizeReleasedV0Event(
|
||||
event: SessionFormatEvent,
|
||||
sessionId: string,
|
||||
): readonly SessionFormatEvent[] {
|
||||
const messageIds = new Map<number, string>()
|
||||
const output: SessionFormatEvent[] = []
|
||||
for (const event of events) {
|
||||
assertSupportedLegacyType(event, sessionId)
|
||||
const start = normalizeLegacyTurnStart(event, sessionId)
|
||||
const end = normalizeLegacyTurnEnd(start, sessionId)
|
||||
const header = normalizeLegacyRequestHeader(end, sessionId)
|
||||
const steering = normalizeLegacySteering(header, sessionId)
|
||||
const message = normalizeLegacyMessage(steering, sessionId, messageIds)
|
||||
assertReleasedEventPayload(message, 0)
|
||||
output.push(message)
|
||||
const messageId = eventMessageId(message)
|
||||
if (messageId !== undefined) messageIds.set(message.seq, messageId)
|
||||
state: LegacyNormalizationState,
|
||||
): SessionFormatEvent {
|
||||
const named = normalizeLegacyCompactionType(event)
|
||||
assertSupportedLegacyType(named, sessionId)
|
||||
const start = normalizeLegacyTurnStart(named, sessionId)
|
||||
const end = normalizeLegacyTurnEnd(start, sessionId)
|
||||
const header = normalizeLegacyRequestHeader(end, sessionId)
|
||||
const steering = normalizeLegacySteering(header, sessionId)
|
||||
const retry = normalizeLegacyRetry(steering, sessionId, state.retryIds)
|
||||
const compaction = normalizeLegacyCompaction(retry, sessionId, state)
|
||||
const message = normalizeLegacyMessage(compaction, sessionId, state.messageIds)
|
||||
if (message.type !== 'assistant/chunk') assertReleasedEventPayload(message, 0)
|
||||
const messageId = eventMessageId(message)
|
||||
if (messageId !== undefined) state.messageIds.set(message.seq, messageId)
|
||||
return message
|
||||
}
|
||||
|
||||
function normalizeLegacyCompactionType(event: SessionFormatEvent): SessionFormatEvent {
|
||||
const type: string = event.type
|
||||
switch (type) {
|
||||
case 'compact/start':
|
||||
return { ...event, type: 'compaction/start' }
|
||||
case 'compact/summary':
|
||||
return { ...event, type: 'compaction/summary' }
|
||||
case 'compact/end':
|
||||
return { ...event, type: 'compaction/end' }
|
||||
case 'compact/prune':
|
||||
return { ...event, type: 'compaction/prune' }
|
||||
default:
|
||||
return event
|
||||
}
|
||||
return Object.freeze(output)
|
||||
}
|
||||
|
||||
function assertSourceDeliveryMarker(
|
||||
event: SessionFormatEvent,
|
||||
input: SessionFormatMigrationStageInput,
|
||||
): void {
|
||||
if (event.type !== 'session-log-deepseek/delivery-accepted') return
|
||||
const data = releasedV0Record(event.data, `${event.type} ${event.seq} data`)
|
||||
const acceptedVersion = data['sessionFormatVersion'] ?? 0
|
||||
const inherited = input.sourceHeader.parentSession !== undefined
|
||||
&& event.seq < input.sourceInheritedEventCount
|
||||
if (acceptedVersion === 0 && !inherited && data['sessionId'] !== input.sourceHeader.id) {
|
||||
throw new SessionFormatError('current-generation delivery marker names the wrong Session')
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeLegacyRetry(
|
||||
event: SessionFormatEvent,
|
||||
sessionId: string,
|
||||
retryIds: Map<string, string>,
|
||||
): SessionFormatEvent {
|
||||
if (event.type !== 'llm/retry') return event
|
||||
const data = releasedV0Record(event.data, `llm/retry ${event.seq} data`)
|
||||
const chain = [data['turn'], data['step'], data['provider'], data['policyKey']]
|
||||
.map(value => JSON.stringify(value))
|
||||
.join('\0')
|
||||
const retryId = data['retryId']
|
||||
if (typeof retryId === 'string' && retryId.length > 0) {
|
||||
retryIds.set(chain, retryId)
|
||||
return event
|
||||
}
|
||||
if (Object.hasOwn(data, 'retryId')) return event
|
||||
const migratedRetryId = retryIds.get(chain) ?? `legacy-retry:${sessionId}:${event.seq}`
|
||||
retryIds.set(chain, migratedRetryId)
|
||||
return { ...event, data: { ...data, retryId: migratedRetryId } }
|
||||
}
|
||||
|
||||
function normalizeLegacyCompaction(
|
||||
event: SessionFormatEvent,
|
||||
sessionId: string,
|
||||
state: LegacyNormalizationState,
|
||||
): SessionFormatEvent {
|
||||
if (event.type === 'session/end-seed') {
|
||||
delete state.compactionId
|
||||
return event
|
||||
}
|
||||
if (event.type === 'compaction/start') {
|
||||
const data = releasedV0Record(event.data, `compaction/start ${event.seq} data`)
|
||||
const existing = data['compactionId']
|
||||
if (typeof existing === 'string' && existing.length > 0) {
|
||||
state.compactionId = existing
|
||||
return event
|
||||
}
|
||||
if (Object.hasOwn(data, 'compactionId')) return event
|
||||
const id = `legacy-compaction:${sessionId}:${event.seq}`
|
||||
state.compactionId = id
|
||||
return { ...event, data: { ...data, compactionId: id } }
|
||||
}
|
||||
const compactionId = state.compactionId
|
||||
if (compactionId === undefined) return event
|
||||
if (event.type === 'compaction/summary' || event.type === 'compaction/end') {
|
||||
const normalized = addLegacyCompactionId(event, compactionId)
|
||||
if (event.type === 'compaction/end') delete state.compactionId
|
||||
return normalized
|
||||
}
|
||||
if (event.type !== 'user/message') return event
|
||||
const data = releasedV0Record(event.data, `user/message ${event.seq} data`)
|
||||
const source = data['source']
|
||||
if (!releasedIsRecord(source) || source['kind'] !== 'plugin' || source['plugin'] !== 'compact'
|
||||
|| Object.hasOwn(source, 'compactionId')) return event
|
||||
return {
|
||||
...event,
|
||||
data: {
|
||||
...data,
|
||||
source: {
|
||||
...source,
|
||||
compactionId,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function addLegacyCompactionId(
|
||||
event: SessionFormatEvent,
|
||||
compactionId: string,
|
||||
): SessionFormatEvent {
|
||||
const data = releasedV0Record(event.data, `${event.type} ${event.seq} data`)
|
||||
if (Object.hasOwn(data, 'compactionId')) return event
|
||||
return { ...event, data: { ...data, compactionId } }
|
||||
}
|
||||
|
||||
function normalizeLegacyRequestHeader(event: SessionFormatEvent, sessionId: string): SessionFormatEvent {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { SessionFormatError, sessionFormatCount, sessionFormatSafeInteger } from '@deepseek-ai/dsh-session-format'
|
||||
import { deepEqualJson } from '@deepseek-ai/dsh-util-values'
|
||||
import type {
|
||||
SessionFormatEvent,
|
||||
SessionFormatJsonValue,
|
||||
@@ -489,7 +490,12 @@ function messageValue(
|
||||
if (role === undefined) literalValue(message['role'], ['system', 'user', 'assistant'], `${label} role`)
|
||||
else literalValue(message['role'], [role], `${label} role`)
|
||||
contentBlocksValue(message['content'], `${label} content`, version)
|
||||
messageSourceValue(message['source'], `${label} source`, version, expected)
|
||||
const source = releasedV0Record(message['source'], `${label} source`)
|
||||
if (version < 2 && expected === 'user' && source['kind'] === 'goal' && source['change'] !== undefined) {
|
||||
legacyGoalMessageValue(message, source, label)
|
||||
} else {
|
||||
messageSourceValue(source, `${label} source`, version, expected)
|
||||
}
|
||||
if (expected === 'tool') {
|
||||
const content = message['content']
|
||||
const block = Array.isArray(content) && content.length === 1
|
||||
@@ -502,6 +508,34 @@ function messageValue(
|
||||
}
|
||||
}
|
||||
|
||||
function legacyGoalMessageValue(message: JsonRecord, source: JsonRecord, label: string): void {
|
||||
assertReleasedV0Keys(source, ['kind', 'goalId', 'revision', 'round', 'change'], [], `${label} source`)
|
||||
nonEmptyString(source['goalId'], `${label} source goalId`)
|
||||
positiveIntegerValue(source['revision'], `${label} source revision`)
|
||||
if (source['round'] !== 0) throw new SessionFormatError(`${label} legacy goal source round must be 0`)
|
||||
const change = releasedV0Record(source['change'], `${label} source change`)
|
||||
goalChangeValue(change, `${label} source change`)
|
||||
const ref = releasedV0Record(
|
||||
change['operation'] === 'clear' ? change['cleared'] : change['goal'],
|
||||
`${label} source change ref`,
|
||||
)
|
||||
if (source['goalId'] !== ref['id'] || source['revision'] !== ref['revision']) {
|
||||
throw new SessionFormatError(`${label} legacy goal source does not match its change`)
|
||||
}
|
||||
const payload = change['operation'] === 'clear'
|
||||
? { cleared: change['cleared'], clearedAt: change['clearedAt'] }
|
||||
: {
|
||||
goal: change['goal'],
|
||||
roundsStarted: change['roundsStarted'],
|
||||
createdAt: change['createdAt'],
|
||||
updatedAt: change['updatedAt'],
|
||||
}
|
||||
const expected = [{ type: 'text', text: `<goal_state>${JSON.stringify(payload)}</goal_state>` }]
|
||||
if (!deepEqualJson(message['content'], expected)) {
|
||||
throw new SessionFormatError(`${label} legacy goal content does not match its change`)
|
||||
}
|
||||
}
|
||||
|
||||
function messageSourceValue(
|
||||
value: SessionFormatJsonValue | undefined,
|
||||
label: string,
|
||||
|
||||
@@ -34,6 +34,8 @@ export interface ReleasedRelationshipExtensions {
|
||||
readonly stepEvents?: ReadonlySet<string>
|
||||
/** Title-request model input was source-validated and preserved across sequence remapping. */
|
||||
readonly preservedSourceTitleRequestText?: true
|
||||
/** Admit the released resume pattern whose next-turn inbox insert omitted the prior turn/end. */
|
||||
readonly legacyInterruptedTurnRestart?: true
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -75,7 +77,22 @@ export function assertReleasedArtifactRelationships(
|
||||
}
|
||||
|
||||
switch (event.type) {
|
||||
case 'turn/start':
|
||||
case 'turn/start': {
|
||||
const previous = artifact.events[event.seq - 1]
|
||||
if (extensions.legacyInterruptedTurnRestart === true
|
||||
&& openTurn !== null
|
||||
&& openStep === null
|
||||
&& data['turn'] === openTurn + 1
|
||||
&& nextTurn === openTurn
|
||||
&& previous?.type === 'agent/inbox/spliced') {
|
||||
const splice = releasedV0Record(previous.data, `agent/inbox/spliced ${previous.seq} data`)
|
||||
if (splice['target'] === 'next-turn'
|
||||
&& Array.isArray(splice['inserted'])
|
||||
&& splice['inserted'].length > 0) {
|
||||
openTurn = null
|
||||
nextTurn += 1
|
||||
}
|
||||
}
|
||||
if (openTurn !== null || data['turn'] !== nextTurn) {
|
||||
throw new SessionFormatError(`turn/start ${JSON.stringify(data['turn'])} does not open expected turn ${nextTurn}`)
|
||||
}
|
||||
@@ -84,6 +101,7 @@ export function assertReleasedArtifactRelationships(
|
||||
toolLifecycles.clear()
|
||||
nextStep = 1
|
||||
break
|
||||
}
|
||||
case 'turn/end':
|
||||
if (openTurn !== data['turn']) {
|
||||
throw new SessionFormatError(`turn/end ${JSON.stringify(data['turn'])} has no matching open turn`)
|
||||
@@ -200,7 +218,11 @@ export function assertReleasedArtifactRelationships(
|
||||
break
|
||||
}
|
||||
case 'llm/retry':
|
||||
requireOpenStep(event, data, openTurn, openStep)
|
||||
if (openTurn !== data['turn']
|
||||
|| data['step'] !== (openStep ?? nextStep - 1)
|
||||
|| openTurn === null) {
|
||||
throw new SessionFormatError('llm/retry does not match the current turn and step')
|
||||
}
|
||||
if (data['provider'] !== openStepProvider) {
|
||||
throw new SessionFormatError('llm/retry provider does not match the open request/header')
|
||||
}
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
import { createSessionFormatCatalog } from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatCurrentEncoder,
|
||||
SessionFormatRecovery,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import { releasedV0SessionFormatCodec, releasedV1SessionFormatCodec } from '../codec.ts'
|
||||
import { sessionFormatV0ToV1 } from '../migration.ts'
|
||||
import { assertReleasedV1Header } from '../validation.ts'
|
||||
import {
|
||||
assertReleasedV1Artifact,
|
||||
assertReleasedV1MigrationSource,
|
||||
} from './validation.ts'
|
||||
|
||||
const unusedEncoder: SessionFormatCurrentEncoder = {
|
||||
/* v8 ignore next -- this restore-only test facade never encodes a header. */
|
||||
encodeHeader: header => header,
|
||||
/* v8 ignore next -- this restore-only test facade never encodes an event. */
|
||||
encodeEvent: event => event,
|
||||
}
|
||||
|
||||
const catalog = createSessionFormatCatalog({
|
||||
currentVersion: 1,
|
||||
codecs: [releasedV0SessionFormatCodec, releasedV1SessionFormatCodec],
|
||||
currentEncoder: unusedEncoder,
|
||||
migrations: [sessionFormatV0ToV1],
|
||||
restoreCurrent(artifact) {
|
||||
assertReleasedV1Artifact(artifact)
|
||||
return artifact
|
||||
},
|
||||
restoreTransformedCurrent(artifact) {
|
||||
assertReleasedV1MigrationSource(artifact)
|
||||
return artifact
|
||||
},
|
||||
/* v8 ignore next 3 -- this restore-only test facade never performs a header-only read. */
|
||||
restoreCurrentHeader(header) {
|
||||
assertReleasedV1Header(header)
|
||||
return header
|
||||
},
|
||||
})
|
||||
|
||||
function restore(
|
||||
header: unknown,
|
||||
rows: readonly unknown[],
|
||||
recovery: SessionFormatRecovery,
|
||||
validation: 'transformed' | 'current',
|
||||
): SessionFormatArtifact {
|
||||
const current = catalog.createRestore(header, { recovery, validation })
|
||||
for (const row of rows) current.decodeRow(row)
|
||||
return current.finish()
|
||||
}
|
||||
|
||||
/**
|
||||
* Restore released v0 rows through the production decoder and v0-to-v1 stage.
|
||||
* @param header - released v0 physical header.
|
||||
* @param rows - released v0 physical rows.
|
||||
* @param recovery - strict or recoverable row policy.
|
||||
* @returns the transformed released v1 artifact.
|
||||
*/
|
||||
export function restoreV0ToV1(
|
||||
header: unknown,
|
||||
rows: readonly unknown[],
|
||||
recovery: SessionFormatRecovery = 'strict',
|
||||
): SessionFormatArtifact {
|
||||
return restore(header, rows, recovery, 'transformed')
|
||||
}
|
||||
|
||||
/**
|
||||
* Restore released v1 rows through the production decoder and current collector.
|
||||
* @param header - released v1 physical header.
|
||||
* @param rows - released v1 physical rows.
|
||||
* @param recovery - strict or recoverable row policy.
|
||||
* @returns the restored released v1 artifact.
|
||||
*/
|
||||
export function restoreV1(
|
||||
header: unknown,
|
||||
rows: readonly unknown[],
|
||||
recovery: SessionFormatRecovery = 'strict',
|
||||
): SessionFormatArtifact {
|
||||
return restore(header, rows, recovery, 'current')
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
import type { SessionFormatArtifact } from '@deepseek-ai/dsh-session-format'
|
||||
import { RELEASED_V0_EVENT_DISPOSITIONS } from '../dispositions.ts'
|
||||
import { assertReleasedArtifactRelationships } from '../relationships.ts'
|
||||
import {
|
||||
assertReleasedArtifactCoordinates,
|
||||
assertReleasedEventPayload,
|
||||
assertReleasedSessionFormatHeader,
|
||||
} from '../validation.ts'
|
||||
|
||||
const RELEASED_V0_EVENT_TYPE_SET: ReadonlySet<string> = new Set(Object.keys(RELEASED_V0_EVENT_DISPOSITIONS))
|
||||
|
||||
/**
|
||||
* Validate one decoded released-v0 source artifact in tests.
|
||||
* @param artifact - released-v0 source artifact.
|
||||
*/
|
||||
export function assertReleasedV0SourceArtifact(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedSessionFormatHeader(artifact.header, 0)
|
||||
assertReleasedArtifactCoordinates(artifact, true, RELEASED_V0_EVENT_TYPE_SET, false, true)
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate normalized v0 output before the identity header bump in tests.
|
||||
* @param artifact - normalized v0 artifact.
|
||||
*/
|
||||
export function assertNormalizedReleasedV0Artifact(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedSessionFormatHeader(artifact.header, 0)
|
||||
assertReleasedArtifactCoordinates(artifact, false, RELEASED_V0_EVENT_TYPE_SET, false, true)
|
||||
for (const event of artifact.events) assertReleasedEventPayload(event, 0)
|
||||
assertReleasedArtifactRelationships(artifact, { legacyInterruptedTurnRestart: true })
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate v1 input accepted by the v1-to-v2 migration in tests.
|
||||
* @param artifact - v1 migration source artifact.
|
||||
*/
|
||||
export function assertReleasedV1MigrationSource(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedSessionFormatHeader(artifact.header, 1)
|
||||
assertReleasedArtifactCoordinates(artifact, false, RELEASED_V0_EVENT_TYPE_SET, false, true)
|
||||
for (const event of artifact.events) {
|
||||
if (RELEASED_V0_EVENT_DISPOSITIONS[event.type] !== undefined) assertReleasedEventPayload(event, 1)
|
||||
}
|
||||
assertReleasedArtifactRelationships(artifact, { legacyInterruptedTurnRestart: true })
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the exact released-v1 logical artifact in tests.
|
||||
* @param artifact - released-v1 logical artifact.
|
||||
*/
|
||||
export function assertReleasedV1Artifact(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedSessionFormatHeader(artifact.header, 1)
|
||||
assertReleasedArtifactCoordinates(artifact, false, RELEASED_V0_EVENT_TYPE_SET, false, true)
|
||||
for (const event of artifact.events) {
|
||||
if (RELEASED_V0_EVENT_DISPOSITIONS[event.type] !== undefined) assertReleasedEventPayload(event, 1)
|
||||
}
|
||||
assertReleasedArtifactRelationships(artifact)
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate released-v1 physical decoding without interpreting event vocabulary in tests.
|
||||
* @param artifact - released-v1 physical artifact.
|
||||
*/
|
||||
export function assertReleasedV1PhysicalArtifact(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedSessionFormatHeader(artifact.header, 1)
|
||||
assertReleasedArtifactCoordinates(artifact, false, undefined, true)
|
||||
}
|
||||
@@ -4,8 +4,8 @@ import {
|
||||
SessionFormatUnsupportedMigrationError,
|
||||
sessionFormatCount,
|
||||
sessionFormatSafeInteger,
|
||||
snapshotSessionFormatJson,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import { isJsonValue } from '@deepseek-ai/dsh-util-values'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatEvent,
|
||||
@@ -14,7 +14,6 @@ import type {
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import { RELEASED_V0_EVENT_DISPOSITIONS } from './dispositions.ts'
|
||||
import { assertReleasedPayloadSemantics } from './payload-validation.ts'
|
||||
import { assertReleasedArtifactRelationships } from './relationships.ts'
|
||||
import { assertReleasedV0Keys, releasedV0Record } from './validation-helpers.ts'
|
||||
|
||||
const HEADER_REQUIRED = ['version', 'id', 'createdAt', 'isSeeded', 'delegationDepth'] as const
|
||||
@@ -23,8 +22,15 @@ const EVENT_REQUIRED = ['type', 'seq', 'time', 'data'] as const
|
||||
const SURFACE_EVENT_TYPES = new Set(['user/message', 'assistant/message', 'tool/result'])
|
||||
const SURFACE_OPTIONAL = ['ignorable', 'sourceEventSeqs', 'surfaceOp'] as const
|
||||
const LOG_OPTIONAL = ['ignorable'] as const
|
||||
const LEGACY_SOURCE_TYPES = new Set(['steering/message', 'request/header-delta', 'mode/set'])
|
||||
const RELEASED_V0_EVENT_TYPE_SET: ReadonlySet<string> = new Set(Object.keys(RELEASED_V0_EVENT_DISPOSITIONS))
|
||||
const LEGACY_SOURCE_TYPES = new Set([
|
||||
'steering/message',
|
||||
'request/header-delta',
|
||||
'mode/set',
|
||||
'compact/start',
|
||||
'compact/summary',
|
||||
'compact/end',
|
||||
'compact/prune',
|
||||
])
|
||||
|
||||
/**
|
||||
* Validate the logical header shared by released v0 and v1.
|
||||
@@ -62,39 +68,6 @@ export function assertReleasedV1Header(header: SessionFormatHeader): void {
|
||||
assertReleasedSessionFormatHeader(header, 1)
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate v0 before historical normalizers run.
|
||||
* @param artifact - decoded released-v0 source.
|
||||
*/
|
||||
export function assertReleasedV0SourceArtifact(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedSessionFormatHeader(artifact.header, 0)
|
||||
assertArtifactCoordinates(artifact, true, RELEASED_V0_EVENT_TYPE_SET)
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate normalized v0 events before the identity header version changes.
|
||||
* @param artifact - normalized released-v0 artifact.
|
||||
*/
|
||||
export function assertNormalizedReleasedV0Artifact(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedSessionFormatHeader(artifact.header, 0)
|
||||
assertArtifactCoordinates(artifact, false, RELEASED_V0_EVENT_TYPE_SET)
|
||||
for (const event of artifact.events) assertReleasedEventPayload(event, 0)
|
||||
assertReleasedArtifactRelationships(artifact)
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the exact logical image emitted by the released v1 writer.
|
||||
* @param artifact - decoded or migration-produced v1 artifact.
|
||||
*/
|
||||
export function assertReleasedV1Artifact(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedV1Header(artifact.header)
|
||||
assertArtifactCoordinates(artifact, false, RELEASED_V0_EVENT_TYPE_SET)
|
||||
for (const event of artifact.events) {
|
||||
if (RELEASED_V0_EVENT_DISPOSITIONS[event.type] !== undefined) assertReleasedEventPayload(event, 1)
|
||||
}
|
||||
assertReleasedArtifactRelationships(artifact)
|
||||
}
|
||||
|
||||
/**
|
||||
* Restore v1 against the installed build's ordinary event vocabulary without freezing payload additions.
|
||||
* @param artifact - vocabulary-neutral released-v1 physical decode.
|
||||
@@ -106,24 +79,24 @@ export function restoreReleasedV1Artifact(
|
||||
knownEventTypes: ReadonlySet<string>,
|
||||
): SessionFormatArtifact {
|
||||
assertReleasedV1Header(artifact.header)
|
||||
assertArtifactCoordinates(artifact, false, knownEventTypes)
|
||||
assertReleasedArtifactCoordinates(artifact, false, knownEventTypes)
|
||||
return artifact
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate released-v1 physical layout without interpreting event vocabulary.
|
||||
* @param artifact - physical-codec output.
|
||||
* Validate released-v0/v1 artifact coordinates under an explicit vocabulary policy.
|
||||
* @param artifact - logical artifact to validate.
|
||||
* @param allowLegacySteering - whether to accept the retired steering event name.
|
||||
* @param knownEventTypes - installed event vocabulary, when vocabulary-aware.
|
||||
* @param vocabularyNeutral - whether unknown event types remain opaque.
|
||||
* @param frozenEnvelope - whether event envelopes must already be frozen.
|
||||
*/
|
||||
export function assertReleasedV1PhysicalArtifact(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedV1Header(artifact.header)
|
||||
assertArtifactCoordinates(artifact, false, undefined, true)
|
||||
}
|
||||
|
||||
function assertArtifactCoordinates(
|
||||
export function assertReleasedArtifactCoordinates(
|
||||
artifact: SessionFormatArtifact,
|
||||
allowLegacySteering: boolean,
|
||||
knownEventTypes?: ReadonlySet<string>,
|
||||
vocabularyNeutral = false,
|
||||
frozenEnvelope = false,
|
||||
): void {
|
||||
const inheritedEventCount = sessionFormatCount(artifact.inheritedEventCount, 'Session inheritedEventCount')
|
||||
if (inheritedEventCount > artifact.events.length) {
|
||||
@@ -151,7 +124,6 @@ function assertArtifactCoordinates(
|
||||
`format v1 contains unknown required event type ${JSON.stringify(type)} at seq ${index}`,
|
||||
)
|
||||
}
|
||||
const frozenEnvelope = !vocabularyNeutral && knownEventTypes === RELEASED_V0_EVENT_TYPE_SET
|
||||
const surface = disposition !== undefined ? SURFACE_EVENT_TYPES.has(type) : type === 'steering/message'
|
||||
const optional = frozenEnvelope
|
||||
? surface ? SURFACE_OPTIONAL : LOG_OPTIONAL
|
||||
@@ -195,8 +167,7 @@ export function assertReleasedSurfaceMetadata(
|
||||
}
|
||||
seen.add(current)
|
||||
}
|
||||
if (sources.length === 0
|
||||
&& (type !== 'assistant/message' || assistantSources === 'forbid-assistant')) {
|
||||
if (sources.length === 0 && type !== 'assistant/message') {
|
||||
throw new SessionFormatError(`${type} ${seq} sourceEventSeqs must be non-empty`)
|
||||
}
|
||||
}
|
||||
@@ -220,7 +191,7 @@ export function assertReleasedEventPayload(event: SessionFormatEvent, version: 0
|
||||
/* v8 ignore next -- artifact coordinate validation admits only the frozen inventory before payload validation. */
|
||||
if (disposition === undefined) {
|
||||
throw new SessionFormatUnsupportedMigrationError(
|
||||
`format v0 contains unknown event type ${JSON.stringify(event.type)} at seq ${event.seq}`,
|
||||
`format v0 contains unknown historical event type ${JSON.stringify(event.type)} at seq ${event.seq}; migration refuses unknown historical events even when ignorable`,
|
||||
)
|
||||
}
|
||||
const data = releasedV0Record(event.data, `${event.type} ${event.seq} data`)
|
||||
@@ -238,7 +209,9 @@ export function assertReleasedEventPayload(event: SessionFormatEvent, version: 0
|
||||
: disposition.optional
|
||||
assertReleasedV0Keys(data, disposition.required, versionOptional, `${event.type} ${event.seq} data`)
|
||||
for (const key of disposition.opaque) {
|
||||
if (Object.hasOwn(data, key)) snapshotSessionFormatJson(data[key], `${event.type} ${event.seq} opaque ${key}`)
|
||||
if (Object.hasOwn(data, key) && !isJsonValue(data[key])) {
|
||||
throw new SessionFormatError(`${event.type} ${event.seq} opaque ${key} is not lossless JSON`)
|
||||
}
|
||||
}
|
||||
assertReleasedPayloadSemantics(event, version)
|
||||
}
|
||||
@@ -1,5 +1,14 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import type { SessionFormatArtifact, SessionFormatEvent } from '@deepseek-ai/dsh-session-format'
|
||||
import { SessionFormatEventCollector } from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatArtifactDecoder,
|
||||
SessionFormatEvent,
|
||||
SessionFormatEventRun,
|
||||
SessionFormatJsonValue,
|
||||
SessionFormatMigrationContext,
|
||||
SessionFormatRecovery,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
releasedV0SessionFormatCodec,
|
||||
releasedV1SessionFormatCodec,
|
||||
@@ -11,33 +20,42 @@ const fullHeader = {
|
||||
} as const
|
||||
const textBlock = { type: 'text', text: 'text' } as const
|
||||
|
||||
function chunk(
|
||||
seq: number,
|
||||
type: 'text-delta' | 'reasoning-delta' | 'tool-call-delta',
|
||||
value: string,
|
||||
options: { turn?: number; step?: number; index?: number; time?: number; name?: string } = {},
|
||||
): SessionFormatEvent {
|
||||
const stream = type === 'tool-call-delta'
|
||||
? { type, index: options.index ?? 0, id: 'call', ...(options.name === undefined ? {} : { name: options.name }), argumentsDelta: value }
|
||||
: { type, index: options.index ?? 0, text: value }
|
||||
return {
|
||||
type: 'assistant/chunk', seq, time: options.time ?? seq + 1,
|
||||
data: { turn: options.turn ?? 1, step: options.step ?? 0, chunk: stream },
|
||||
class DecodedItemCollector implements SessionFormatMigrationContext {
|
||||
readonly values: Array<SessionFormatEvent | SessionFormatEventRun> = []
|
||||
|
||||
emitEvent(event: SessionFormatEvent): void {
|
||||
this.values.push(event)
|
||||
}
|
||||
|
||||
emitRun(run: SessionFormatEventRun): void {
|
||||
this.values.push(run)
|
||||
}
|
||||
}
|
||||
|
||||
function artifact(events: readonly SessionFormatEvent[], overrides: Partial<SessionFormatArtifact> = {}): SessionFormatArtifact {
|
||||
return {
|
||||
header: { version: 1, id: 'codec', createdAt: 1, isSeeded: false, delegationDepth: 0 },
|
||||
inheritedEventCount: 0,
|
||||
events,
|
||||
...overrides,
|
||||
}
|
||||
function decodeRow(
|
||||
decoder: SessionFormatArtifactDecoder,
|
||||
row: SessionFormatJsonValue,
|
||||
): Array<SessionFormatEvent | SessionFormatEventRun> {
|
||||
const output = new DecodedItemCollector()
|
||||
decoder.decodeRow(row, output)
|
||||
return output.values
|
||||
}
|
||||
|
||||
function decodeArtifact(
|
||||
codec: typeof releasedV0SessionFormatCodec,
|
||||
header: unknown,
|
||||
rows: readonly unknown[],
|
||||
recovery: SessionFormatRecovery = 'strict',
|
||||
): SessionFormatArtifact {
|
||||
const decoder = codec.createDecoder(header, recovery)
|
||||
const context = new SessionFormatEventCollector()
|
||||
for (const row of rows) decoder.decodeRow(row, context)
|
||||
return { header: decoder.header, inheritedEventCount: decoder.finish(context), events: context.values }
|
||||
}
|
||||
|
||||
describe('released v0/v1 physical codecs', () => {
|
||||
it('round-trips every physical header field and seeded zero cut', () => {
|
||||
const decoded = releasedV1SessionFormatCodec.decodeArtifact(fullHeader, [])
|
||||
it('decodes every physical header field and seeded zero cut', () => {
|
||||
const decoded = decodeArtifact(releasedV1SessionFormatCodec, fullHeader, [])
|
||||
expect(decoded).toEqual({
|
||||
header: {
|
||||
version: 1, id: 'codec', createdAt: 1, cwd: '/work', parentSession: 'parent',
|
||||
@@ -46,11 +64,8 @@ describe('released v0/v1 physical codecs', () => {
|
||||
inheritedEventCount: 0,
|
||||
events: [],
|
||||
})
|
||||
expect(releasedV1SessionFormatCodec.encodeArtifact(decoded, { packChunks: false }).header).toEqual(fullHeader)
|
||||
expect(releasedV0SessionFormatCodec.encodeArtifact({
|
||||
...decoded,
|
||||
header: { ...decoded.header, version: 0 },
|
||||
}, { packChunks: false }).header).toEqual({ ...fullHeader, version: 0 })
|
||||
expect(releasedV0SessionFormatCodec.decodeHeader({ ...fullHeader, version: 0 }))
|
||||
.toEqual({ ...decoded.header, version: 0 })
|
||||
})
|
||||
|
||||
it.each([
|
||||
@@ -69,43 +84,60 @@ describe('released v0/v1 physical codecs', () => {
|
||||
expect(() => releasedV1SessionFormatCodec.decodeHeader(header)).toThrow()
|
||||
})
|
||||
|
||||
it('packs and expands text, reasoning, and named tool-call runs exactly', () => {
|
||||
const events = [
|
||||
chunk(0, 'text-delta', 'a'), chunk(1, 'text-delta', 'b'), chunk(2, 'text-delta', 'c'),
|
||||
chunk(3, 'reasoning-delta', 'd'), chunk(4, 'reasoning-delta', 'e'), chunk(5, 'reasoning-delta', 'f'),
|
||||
chunk(6, 'tool-call-delta', '{', { name: 'read' }),
|
||||
chunk(7, 'tool-call-delta', '}', { name: 'read' }),
|
||||
chunk(8, 'tool-call-delta', '', { name: 'read' }),
|
||||
]
|
||||
const v0 = { ...artifact(events), header: { ...artifact(events).header, version: 0 } }
|
||||
const encoded = releasedV0SessionFormatCodec.encodeArtifact(v0, { packChunks: true })
|
||||
expect(encoded.rows.map(row => row['type'])).toEqual(['text-chunks', 'reasoning-chunks', 'tool-call-chunks'])
|
||||
expect(releasedV0SessionFormatCodec.decodeArtifact(encoded.header, encoded.rows).events).toEqual(events)
|
||||
it('expands valid text, reasoning, and named or unnamed tool-call rows exactly', () => {
|
||||
const cases = [
|
||||
[{
|
||||
type: 'text-chunks', seq0: 0, time0: 1,
|
||||
data: { turn: 1, step: 2, index: 3, dt: [2], texts: ['a', 'b'] },
|
||||
}, [
|
||||
{ type: 'assistant/chunk', seq: 0, time: 1, data: {
|
||||
turn: 1, step: 2, chunk: { type: 'text-delta', index: 3, text: 'a' },
|
||||
} },
|
||||
{ type: 'assistant/chunk', seq: 1, time: 3, data: {
|
||||
turn: 1, step: 2, chunk: { type: 'text-delta', index: 3, text: 'b' },
|
||||
} },
|
||||
]],
|
||||
[{
|
||||
type: 'reasoning-chunks', seq0: 0, time0: 4,
|
||||
data: { turn: 2, step: 3, index: 1, dt: [1], texts: ['c', 'd'] },
|
||||
}, [
|
||||
{ type: 'assistant/chunk', seq: 0, time: 4, data: {
|
||||
turn: 2, step: 3, chunk: { type: 'reasoning-delta', index: 1, text: 'c' },
|
||||
} },
|
||||
{ type: 'assistant/chunk', seq: 1, time: 5, data: {
|
||||
turn: 2, step: 3, chunk: { type: 'reasoning-delta', index: 1, text: 'd' },
|
||||
} },
|
||||
]],
|
||||
[{
|
||||
type: 'tool-call-chunks', seq0: 0, time0: 6,
|
||||
data: { turn: 3, step: 4, index: 2, id: 'call', name: 'read', dt: [1], args: ['{', '}'] },
|
||||
}, [
|
||||
{ type: 'assistant/chunk', seq: 0, time: 6, data: {
|
||||
turn: 3, step: 4,
|
||||
chunk: { type: 'tool-call-delta', index: 2, id: 'call', name: 'read', argumentsDelta: '{' },
|
||||
} },
|
||||
{ type: 'assistant/chunk', seq: 1, time: 7, data: {
|
||||
turn: 3, step: 4,
|
||||
chunk: { type: 'tool-call-delta', index: 2, id: 'call', name: 'read', argumentsDelta: '}' },
|
||||
} },
|
||||
]],
|
||||
[{
|
||||
type: 'tool-call-chunks', seq0: 0, time0: 8,
|
||||
data: { turn: 4, step: 5, index: 0, id: 'call', dt: [], args: ['x'] },
|
||||
}, [
|
||||
{ type: 'assistant/chunk', seq: 0, time: 8, data: {
|
||||
turn: 4, step: 5,
|
||||
chunk: { type: 'tool-call-delta', index: 0, id: 'call', argumentsDelta: 'x' },
|
||||
} },
|
||||
]],
|
||||
] as const
|
||||
|
||||
const unnamed = [
|
||||
chunk(0, 'tool-call-delta', 'a'),
|
||||
chunk(1, 'tool-call-delta', 'b'),
|
||||
chunk(2, 'tool-call-delta', 'c'),
|
||||
]
|
||||
const unnamedV0 = { ...artifact(unnamed), header: { ...artifact(unnamed).header, version: 0 } }
|
||||
const unnamedEncoded = releasedV0SessionFormatCodec.encodeArtifact(unnamedV0, { packChunks: true })
|
||||
expect(unnamedEncoded.rows[0]?.['data']).not.toHaveProperty('name')
|
||||
expect(releasedV0SessionFormatCodec.decodeArtifact(unnamedEncoded.header, unnamedEncoded.rows).events).toEqual(unnamed)
|
||||
})
|
||||
|
||||
it('keeps short or non-continuing chunk runs unpacked', () => {
|
||||
const events = [
|
||||
chunk(0, 'text-delta', 'a'),
|
||||
chunk(1, 'text-delta', 'b', { index: 1 }),
|
||||
chunk(2, 'text-delta', 'c', { turn: 2 }),
|
||||
chunk(3, 'text-delta', 'd', { step: 1 }),
|
||||
chunk(4, 'tool-call-delta', 'a'),
|
||||
chunk(5, 'tool-call-delta', 'b', { name: 'read' }),
|
||||
chunk(6, 'tool-call-delta', 'c', { name: 'read' }),
|
||||
]
|
||||
const v0 = { ...artifact(events), header: { ...artifact(events).header, version: 0 } }
|
||||
const encoded = releasedV0SessionFormatCodec.encodeArtifact(v0, { packChunks: true })
|
||||
expect(encoded.rows).toEqual(events)
|
||||
for (const [row, expected] of cases) {
|
||||
const decoder = releasedV1SessionFormatCodec.createDecoder(fullHeader, 'strict')
|
||||
const item = decodeRow(decoder, row)[0]
|
||||
if (item === undefined || !('runType' in item)) throw new Error('expected a packed Assistant run')
|
||||
expect([...(item as SessionFormatEventRun).expand()]).toEqual(expected)
|
||||
}
|
||||
})
|
||||
|
||||
it.each([
|
||||
@@ -118,7 +150,7 @@ describe('released v0/v1 physical codecs', () => {
|
||||
['tool name', { type: 'tool-call-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 0, index: 0, id: 'id', name: 1, dt: [], args: ['a'] } }],
|
||||
['unsafe time sum', { type: 'text-chunks', seq0: 0, time0: Number.MAX_SAFE_INTEGER, data: { turn: 1, step: 0, index: 0, dt: [1], texts: ['a', 'b'] } }],
|
||||
])('refuses malformed packed row: %s', (_name, row) => {
|
||||
expect(() => releasedV1SessionFormatCodec.decodeArtifact(fullHeader, [row])).toThrow()
|
||||
expect(() => decodeArtifact(releasedV1SessionFormatCodec, fullHeader, [row])).toThrow()
|
||||
})
|
||||
|
||||
it.each([
|
||||
@@ -135,39 +167,83 @@ describe('released v0/v1 physical codecs', () => {
|
||||
id: 'u', role: 'user', content: [{ type: 'text', text: 'x' }], source: { kind: 'user' },
|
||||
}, sourceEventSeqs, surfaceOp: 'append' },
|
||||
]
|
||||
expect(() => releasedV1SessionFormatCodec.decodeArtifact(
|
||||
expect(() => decodeArtifact(releasedV1SessionFormatCodec,
|
||||
{ type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 },
|
||||
rows,
|
||||
)).toThrow()
|
||||
})
|
||||
|
||||
it('contains non-SessionFormatError row failures during recoverable scans', () => {
|
||||
const bad = new Proxy({}, { ownKeys: () => { throw new Error('proxy failure') } })
|
||||
const recovered = releasedV1SessionFormatCodec.decodeRecoverableArtifact(
|
||||
const bad = new Proxy({}, { get: () => { throw new Error('proxy failure') } })
|
||||
const decoder = releasedV1SessionFormatCodec.createDecoder(
|
||||
{ type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 },
|
||||
[bad],
|
||||
'recoverable',
|
||||
)
|
||||
expect(recovered).toMatchObject({ events: [] })
|
||||
expect(decodeRow(decoder, bad)).toEqual([])
|
||||
expect(() => decodeRow(decoder, {
|
||||
type: 'turn/end', seq: 0, time: 2, data: { turn: 1, reason: { kind: 'completed' } },
|
||||
})).toThrow('released Session row 0 is malformed')
|
||||
})
|
||||
|
||||
it('streams the same row-atomic recoverable prefix', () => {
|
||||
const create = (header: unknown) => releasedV1SessionFormatCodec.createDecoder(header, 'recoverable')
|
||||
const header = { type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 }
|
||||
|
||||
const malformed = create(header)
|
||||
expect(decodeRow(malformed, null)).toEqual([])
|
||||
const malformedProxy = create(header)
|
||||
const proxy = new Proxy({}, { get: () => { throw new Error('proxy failure') } })
|
||||
expect(decodeRow(malformedProxy, proxy)).toEqual([])
|
||||
expect(decodeRow(malformedProxy, { type: 'step/start', seq: 0, time: 1, data: { turn: 1, step: 1 } })).toEqual([])
|
||||
expect(() => decodeRow(malformedProxy, {
|
||||
type: 'turn/end', seq: 0, time: 2, data: { turn: 1, reason: { kind: 'completed' } },
|
||||
})).toThrow('released Session row 0 is malformed')
|
||||
|
||||
const gap = create(header)
|
||||
expect(decodeRow(gap, { type: 'step/start', seq: 1, time: 1, data: { turn: 1, step: 1 } })).toEqual([])
|
||||
const terminalGap = create(header)
|
||||
expect(() => decodeRow(terminalGap, {
|
||||
type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } },
|
||||
})).toThrow(/seq gap/)
|
||||
|
||||
const seeded = create({ ...header, seedLength: 1 })
|
||||
expect(() => seeded.finish(new DecodedItemCollector())).toThrow(/inheritedEventCount exceeds/)
|
||||
|
||||
const packed = create(header)
|
||||
const runs = decodeRow(packed, {
|
||||
type: 'text-chunks', seq0: 0, time0: 1,
|
||||
data: { turn: 1, step: 1, index: 0, dt: [1], texts: ['a', 'b'] },
|
||||
})
|
||||
expect(runs).toHaveLength(1)
|
||||
expect([...((runs[0] as SessionFormatEventRun).expand())]).toHaveLength(2)
|
||||
const provenance = create(header)
|
||||
decodeRow(provenance, { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } })
|
||||
const item = decodeRow(provenance, {
|
||||
type: 'user/message', seq: 1, time: 2,
|
||||
data: { id: 'm', role: 'user', content: [], source: { kind: 'user' } },
|
||||
sourceEventSeqs: [0], surfaceOp: 'append',
|
||||
})[0]
|
||||
expect(item !== undefined && !('runType' in item) ? item.sourceEventSeqs : undefined).toEqual([0])
|
||||
})
|
||||
|
||||
it('ignores decodable non-terminal rows after the first recoverable issue', () => {
|
||||
const header = { type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 }
|
||||
const recovered = releasedV1SessionFormatCodec.decodeRecoverableArtifact(header, [
|
||||
const recovered = decodeArtifact(releasedV1SessionFormatCodec, header, [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
{ type: 'turn/start', seq: 4, time: 2, data: { turn: 2 } },
|
||||
{ type: 'step/start', seq: 1, time: 3, data: { turn: 1, step: 0 } },
|
||||
])
|
||||
], 'recoverable')
|
||||
expect(recovered).toMatchObject({ events: [{ seq: 0 }] })
|
||||
})
|
||||
|
||||
it('rejects strict gaps and a recoverable gap row that itself closes a turn', () => {
|
||||
const currentHeader = { type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 }
|
||||
expect(() => releasedV1SessionFormatCodec.decodeArtifact(currentHeader, [
|
||||
expect(() => decodeArtifact(releasedV1SessionFormatCodec, currentHeader, [
|
||||
{ type: 'turn/start', seq: 1, time: 1, data: { turn: 1 } },
|
||||
])).toThrow(/seq gap/)
|
||||
expect(() => releasedV1SessionFormatCodec.decodeRecoverableArtifact(currentHeader, [
|
||||
expect(() => decodeArtifact(releasedV1SessionFormatCodec, currentHeader, [
|
||||
{ type: 'turn/end', seq: 1, time: 1, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
])).toThrow(/seq gap/)
|
||||
], 'recoverable')).toThrow(/seq gap/)
|
||||
})
|
||||
|
||||
it('refuses overlapping ranges after a valid first range', () => {
|
||||
@@ -182,74 +258,6 @@ describe('released v0/v1 physical codecs', () => {
|
||||
data: { id: 'u', role: 'user', content: [textBlock], source: { kind: 'user' } },
|
||||
},
|
||||
]
|
||||
expect(() => releasedV1SessionFormatCodec.decodeArtifact(header, rows)).toThrow(/strictly increasing/)
|
||||
})
|
||||
|
||||
it('keeps non-consecutive provenance scalar and leaves invalid v0 chunks unpacked', () => {
|
||||
const events = [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
{ type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } },
|
||||
{ type: 'step/end', seq: 2, time: 3, data: { turn: 1, step: 1 } },
|
||||
{
|
||||
type: 'user/message', seq: 3, time: 4, surfaceOp: 'append', sourceEventSeqs: [0, 2],
|
||||
data: { id: 'u', role: 'user', content: [textBlock], source: { kind: 'user' } },
|
||||
},
|
||||
] as SessionFormatEvent[]
|
||||
expect(releasedV1SessionFormatCodec.encodeArtifact(artifact(events), { packChunks: false }).rows[3])
|
||||
.toEqual(events[3])
|
||||
|
||||
const invalidChunk = { type: 'assistant/chunk', seq: 0, time: 1, data: { turn: 1, step: 0, chunk: null } }
|
||||
const v0 = {
|
||||
header: { version: 0, id: 'codec', createdAt: 1, isSeeded: false, delegationDepth: 0 },
|
||||
inheritedEventCount: 0,
|
||||
events: [invalidChunk],
|
||||
} as unknown as SessionFormatArtifact
|
||||
expect(releasedV0SessionFormatCodec.encodeArtifact(v0, { packChunks: true }).rows).toEqual([invalidChunk])
|
||||
|
||||
const causal = {
|
||||
...v0,
|
||||
events: [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
{ type: 'user/message', seq: 1, time: 2, surfaceOp: 'append', data: {
|
||||
id: 'one', role: 'user', content: [textBlock], source: { kind: 'user' },
|
||||
} },
|
||||
{ type: 'user/message', seq: 2, time: 3, surfaceOp: 'append', data: {
|
||||
id: 'two', role: 'user', content: [textBlock], source: { kind: 'user' },
|
||||
} },
|
||||
{ type: 'user/message', seq: 3, time: 4, surfaceOp: 'append', sourceEventSeqs: [2, 0], data: {
|
||||
id: 'three', role: 'user', content: [textBlock], source: { kind: 'user' },
|
||||
} },
|
||||
],
|
||||
} as unknown as SessionFormatArtifact
|
||||
expect(releasedV0SessionFormatCodec.encodeArtifact(causal, { packChunks: false }).rows[3]?.['sourceEventSeqs'])
|
||||
.toEqual([2, 0])
|
||||
|
||||
for (const badData of [
|
||||
null,
|
||||
{ turn: 1, step: 1, chunk: null },
|
||||
{ turn: 1, step: 1, chunk: { type: 'other', index: 0 } },
|
||||
{ turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 1 } },
|
||||
{ turn: 1, step: 1, chunk: { type: 'tool-call-delta', index: 0, id: 1, argumentsDelta: 'x' } },
|
||||
]) {
|
||||
const bad = { ...v0, events: [{ type: 'assistant/chunk', seq: 0, time: 1, data: badData }] } as unknown as SessionFormatArtifact
|
||||
expect(releasedV0SessionFormatCodec.encodeArtifact(bad, { packChunks: true }).rows).toHaveLength(1)
|
||||
}
|
||||
|
||||
const farTimes = [
|
||||
chunk(0, 'text-delta', 'a', { time: Number.MIN_SAFE_INTEGER }),
|
||||
chunk(1, 'text-delta', 'b', { time: Number.MAX_SAFE_INTEGER }),
|
||||
chunk(2, 'text-delta', 'c', { time: Number.MAX_SAFE_INTEGER }),
|
||||
]
|
||||
const far = { ...v0, events: farTimes } as unknown as SessionFormatArtifact
|
||||
expect(releasedV0SessionFormatCodec.encodeArtifact(far, { packChunks: true }).rows).toHaveLength(3)
|
||||
|
||||
const ignorable = [
|
||||
{ ...chunk(0, 'text-delta', 'a'), ignorable: true },
|
||||
{ ...chunk(1, 'text-delta', 'b'), ignorable: true },
|
||||
{ ...chunk(2, 'text-delta', 'c'), ignorable: true },
|
||||
] as SessionFormatEvent[]
|
||||
const ignorableV0 = { ...v0, events: ignorable } as unknown as SessionFormatArtifact
|
||||
expect(releasedV0SessionFormatCodec.encodeArtifact(ignorableV0, { packChunks: true }).rows)
|
||||
.toEqual(ignorable)
|
||||
expect(() => decodeArtifact(releasedV1SessionFormatCodec, header, rows)).toThrow(/strictly increasing/)
|
||||
})
|
||||
})
|
||||
@@ -1,9 +1,9 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SessionFormatUnsupportedMigrationError } from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
releasedV0SessionFormatCodec,
|
||||
sessionFormatV0ToV1,
|
||||
} from '../src/index.ts'
|
||||
import { restoreV0ToV1 } from '../src/testing/restore.ts'
|
||||
|
||||
const header = {
|
||||
type: 'session',
|
||||
@@ -14,7 +14,7 @@ const header = {
|
||||
} as const
|
||||
|
||||
function migrate(rows: readonly unknown[]) {
|
||||
return sessionFormatV0ToV1.migrate(releasedV0SessionFormatCodec.decodeArtifact(header, rows))
|
||||
return restoreV0ToV1(header, rows)
|
||||
}
|
||||
|
||||
describe('released v0 legacy normalization', () => {
|
||||
|
||||
@@ -1,11 +1,33 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SessionFormatEventCollector } from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
SessionFormatEvent,
|
||||
SessionFormatEventRun,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
RELEASED_V0_EVENT_TYPES,
|
||||
releasedV0SessionFormatCodec,
|
||||
releasedV1SessionFormatCodec,
|
||||
restoreReleasedV1Artifact,
|
||||
sessionFormatV0ToV1,
|
||||
} from '../src/index.ts'
|
||||
import { restoreV0ToV1, restoreV1 } from '../src/testing/restore.ts'
|
||||
|
||||
function createMigrationStage(id: string) {
|
||||
const sourceHeader = { version: 0, id, createdAt: 1, isSeeded: false, delegationDepth: 0 }
|
||||
const stage = sessionFormatV0ToV1.createStage({
|
||||
sourceHeader,
|
||||
targetHeader: sessionFormatV0ToV1.migrateHeader(sourceHeader),
|
||||
sourceInheritedEventCount: 0,
|
||||
sourceKind: 'decoded',
|
||||
})
|
||||
return {
|
||||
transform(event: SessionFormatEvent): SessionFormatEvent[] {
|
||||
const context = new SessionFormatEventCollector()
|
||||
stage.transformEvent(event, context)
|
||||
return context.values
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
describe('released Session format v0 to v1', () => {
|
||||
it('changes only the version of a canonical decoded artifact', () => {
|
||||
@@ -27,19 +49,71 @@ describe('released Session format v0 to v1', () => {
|
||||
data: { turn: 1, step: 1, index: 0, dt: [1, 1], texts: ['a', 'b', 'c'] },
|
||||
},
|
||||
]
|
||||
const source = releasedV0SessionFormatCodec.decodeArtifact(header, rows)
|
||||
|
||||
const migrated = sessionFormatV0ToV1.migrate(source)
|
||||
const migrated = restoreV0ToV1(header, rows)
|
||||
|
||||
expect(migrated).toEqual({
|
||||
...source,
|
||||
header: { ...source.header, version: 1 },
|
||||
header: {
|
||||
version: 1, id: 'identity', createdAt: 1, cwd: '/work', isSeeded: false, delegationDepth: 0,
|
||||
},
|
||||
inheritedEventCount: 0,
|
||||
events: [
|
||||
rows[0],
|
||||
rows[1],
|
||||
{ type: 'assistant/chunk', seq: 2, time: 4, data: {
|
||||
turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'a' },
|
||||
} },
|
||||
{ type: 'assistant/chunk', seq: 3, time: 5, data: {
|
||||
turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'b' },
|
||||
} },
|
||||
{ type: 'assistant/chunk', seq: 4, time: 6, data: {
|
||||
turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'c' },
|
||||
} },
|
||||
],
|
||||
})
|
||||
sessionFormatV0ToV1.validateTarget(migrated)
|
||||
expect(releasedV1SessionFormatCodec.encodeArtifact(migrated, { packChunks: true })).toEqual({
|
||||
header: { ...header, version: 1 },
|
||||
rows,
|
||||
})
|
||||
|
||||
it('expands an unrecognized compact run through the identity stage', () => {
|
||||
const sourceHeader = {
|
||||
version: 0, id: 'generic-run', createdAt: 1, isSeeded: false, delegationDepth: 0,
|
||||
} as const
|
||||
const stage = sessionFormatV0ToV1.createStage({
|
||||
sourceHeader,
|
||||
targetHeader: sessionFormatV0ToV1.migrateHeader(sourceHeader),
|
||||
sourceInheritedEventCount: 0,
|
||||
sourceKind: 'transformed',
|
||||
})
|
||||
const source = {
|
||||
type: 'feedback/record', seq: 0, time: 2, data: { text: 'retained' },
|
||||
} as const
|
||||
const run: SessionFormatEventRun = {
|
||||
runType: 'test-run', firstSeq: 0, eventCount: 1, *expand() { yield source },
|
||||
}
|
||||
const output = new SessionFormatEventCollector()
|
||||
|
||||
stage.transformRun(run, output)
|
||||
|
||||
expect(output.values).toEqual([source])
|
||||
})
|
||||
|
||||
it('accepts an inherited delivery marker for its source generation', () => {
|
||||
const sourceHeader = {
|
||||
version: 0, id: 'child', createdAt: 1, parentSession: 'parent', isSeeded: true, delegationDepth: 0,
|
||||
} as const
|
||||
const stage = sessionFormatV0ToV1.createStage({
|
||||
sourceHeader,
|
||||
targetHeader: sessionFormatV0ToV1.migrateHeader(sourceHeader),
|
||||
sourceInheritedEventCount: 2,
|
||||
sourceKind: 'decoded',
|
||||
})
|
||||
const marker = {
|
||||
type: 'session-log-deepseek/delivery-accepted', seq: 1, time: 2,
|
||||
data: { sessionId: 'parent', throughSeq: 0 },
|
||||
} as const
|
||||
const output = new SessionFormatEventCollector()
|
||||
|
||||
stage.transformEvent(marker, output)
|
||||
|
||||
expect(output.values).toEqual([marker])
|
||||
})
|
||||
|
||||
it('recovers only the complete row prefix and refuses a later committing turn end', () => {
|
||||
@@ -61,13 +135,13 @@ describe('released Session format v0 to v1', () => {
|
||||
data: { turn: 2, step: 0, index: 0, dt: [1], texts: ['x', 'y'] },
|
||||
}
|
||||
|
||||
expect(releasedV0SessionFormatCodec.decodeRecoverableArtifact(header, [...prefix, badRow]))
|
||||
.toEqual(releasedV0SessionFormatCodec.decodeArtifact(header, prefix))
|
||||
expect(() => releasedV0SessionFormatCodec.decodeRecoverableArtifact(header, [
|
||||
expect(restoreV0ToV1(header, [...prefix, badRow], 'recoverable'))
|
||||
.toEqual(restoreV0ToV1(header, prefix))
|
||||
expect(() => restoreV0ToV1(header, [
|
||||
...prefix,
|
||||
badRow,
|
||||
{ type: 'turn/end', seq: 2, time: 6, data: { turn: 2, reason: { kind: 'interrupted' } } },
|
||||
])).toThrow(/seq gap/)
|
||||
], 'recoverable')).toThrow(/seq gap/)
|
||||
})
|
||||
|
||||
it('requires canonical delegation depth and decodes provenance without mutating source rows', () => {
|
||||
@@ -100,15 +174,15 @@ describe('released Session format v0 to v1', () => {
|
||||
provenanceRow,
|
||||
]
|
||||
|
||||
const decoded = releasedV0SessionFormatCodec.decodeArtifact(header, rows)
|
||||
const migrated = restoreV0ToV1(header, rows)
|
||||
|
||||
expect(() => releasedV0SessionFormatCodec.decodeArtifact(incompleteHeader, rows)).toThrow(/delegationDepth/)
|
||||
expect(decoded.header.delegationDepth).toBe(0)
|
||||
expect(decoded.events[3]?.sourceEventSeqs).toEqual([0, 1, 2])
|
||||
expect(() => restoreV0ToV1(incompleteHeader, rows)).toThrow(/delegationDepth/)
|
||||
expect(migrated.header.delegationDepth).toBe(0)
|
||||
expect(migrated.events[3]?.sourceEventSeqs).toEqual([0, 1, 2])
|
||||
expect(provenanceRow.sourceEventSeqs).toEqual([[0, 2]])
|
||||
const migrated = sessionFormatV0ToV1.migrate(decoded)
|
||||
expect(releasedV1SessionFormatCodec.encodeArtifact(migrated, { packChunks: false }).header)
|
||||
.toEqual({ ...header, version: 1 })
|
||||
expect(migrated.header).toEqual({
|
||||
version: 1, id: 'old', createdAt: 1, isSeeded: false, delegationDepth: 0,
|
||||
})
|
||||
})
|
||||
|
||||
it('refuses v1-only generation fields in v0 and accepts them in v1', () => {
|
||||
@@ -122,10 +196,91 @@ describe('released Session format v0 to v1', () => {
|
||||
const v0 = { type: 'session', version: 0, id: 'delivery', createdAt: 1, delegationDepth: 0 }
|
||||
const v1 = { ...v0, version: 1 }
|
||||
|
||||
expect(() => sessionFormatV0ToV1.migrate(
|
||||
releasedV0SessionFormatCodec.decodeArtifact(v0, [prefix, event]),
|
||||
)).toThrow(/unexpected member "sessionFormatVersion"/)
|
||||
expect(releasedV1SessionFormatCodec.decodeArtifact(v1, [prefix, event]).events).toEqual([prefix, event])
|
||||
expect(() => restoreV0ToV1(v0, [prefix, event])).toThrow(/unexpected member "sessionFormatVersion"/)
|
||||
expect(restoreV1(v1, [prefix, event]).events).toEqual([prefix, event])
|
||||
})
|
||||
|
||||
it('does not synthesize over an explicitly invalid retry id', () => {
|
||||
const header = { type: 'session', version: 0, id: 'retry', createdAt: 1, delegationDepth: 0 }
|
||||
const rows = [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
{ type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } },
|
||||
{ type: 'request/header', seq: 2, time: 3, data: {
|
||||
header: { config: { provider: 'p', model: 'm' } }, reason: 'initial',
|
||||
} },
|
||||
{ type: 'llm/retry', seq: 3, time: 4, data: {
|
||||
retryId: null, turn: 1, step: 1, provider: 'p', mode: 'normal', policyKey: 'k',
|
||||
retry: 1, maxRetries: 1, delayMs: 0, failure: { message: 'retry', code: 'SERVER' },
|
||||
} },
|
||||
]
|
||||
expect(() => restoreV0ToV1(header, rows)).toThrow(/retryId/)
|
||||
|
||||
const transformer = createMigrationStage('retry')
|
||||
const retry = (seq: number, attempt: number) => ({
|
||||
type: 'llm/retry', seq, time: seq + 1,
|
||||
data: {
|
||||
turn: 1, step: 1, provider: 'p', mode: 'normal', policyKey: 'k', retry: attempt,
|
||||
maxRetries: 2, delayMs: 0, failure: { message: 'retry', code: 'SERVER' },
|
||||
},
|
||||
})
|
||||
expect(transformer.transform(retry(0, 1))[0]?.data).toMatchObject({ retryId: 'legacy-retry:retry:0' })
|
||||
expect(transformer.transform(retry(1, 2))[0]?.data).toMatchObject({ retryId: 'legacy-retry:retry:0' })
|
||||
})
|
||||
|
||||
it('normalizes legacy compaction names and assigns one deterministic id to the group', () => {
|
||||
const transformer = createMigrationStage('compact')
|
||||
const rows = [
|
||||
{ type: 'compaction/start', seq: 0, time: 1, data: { turn: null } },
|
||||
{
|
||||
type: 'compaction/summary', seq: 1, time: 2,
|
||||
data: {
|
||||
summary: [{ type: 'text', text: 'summary' }], shadowedRange: { start: 0, end: 0 },
|
||||
shadowedSeqs: [0], shadowedTokenCount: 0, provider: 'p', model: 'm',
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'compaction/prune', seq: 2, time: 3,
|
||||
data: { shadowedRange: { start: 0, end: 0 }, shadowedSeqs: [0], shadowedTokenCount: 1 },
|
||||
},
|
||||
{
|
||||
type: 'user/message', seq: 3, time: 4,
|
||||
data: {
|
||||
id: 'checkpoint', role: 'user', content: [{ type: 'text', text: 'summary' }],
|
||||
source: { kind: 'plugin', plugin: 'compact' },
|
||||
},
|
||||
sourceEventSeqs: [0, 1], surfaceOp: { op: 'replace', start: 0, end: 0 },
|
||||
},
|
||||
{ type: 'compaction/end', seq: 4, time: 5, data: { turn: null } },
|
||||
]
|
||||
|
||||
const migrated = rows.map(row => transformer.transform(row)[0])
|
||||
expect(migrated.map(event => event?.type === 'user/message'
|
||||
? (event.data as { source: { compactionId: string } }).source.compactionId
|
||||
: (event?.data as { compactionId: string }).compactionId))
|
||||
.toEqual([
|
||||
'legacy-compaction:compact:0',
|
||||
'legacy-compaction:compact:0',
|
||||
undefined,
|
||||
'legacy-compaction:compact:0',
|
||||
'legacy-compaction:compact:0',
|
||||
])
|
||||
|
||||
const historical = rows.map(row => ({
|
||||
...row,
|
||||
type: row.type.replace(/^compaction\//, 'compact/'),
|
||||
}))
|
||||
const historicalTransformer = createMigrationStage('compact')
|
||||
expect(historical.map(row => historicalTransformer.transform(row as SessionFormatEvent)[0])).toEqual(migrated)
|
||||
|
||||
const reset = createMigrationStage('compact')
|
||||
expect(() => reset.transform({
|
||||
...(rows[0] as SessionFormatEvent), data: { turn: null, compactionId: null },
|
||||
})).toThrow(/compactionId/)
|
||||
reset.transform(rows[0] as never)
|
||||
expect(reset.transform({ type: 'feedback/record', seq: 1, time: 2, data: { text: 'kept' } })[0]?.type)
|
||||
.toBe('feedback/record')
|
||||
reset.transform({ type: 'session/end-seed', seq: 2, time: 3, data: {} })
|
||||
expect(() => reset.transform({ ...(rows[1] as SessionFormatEvent), seq: 3 })).toThrow(/compactionId/)
|
||||
})
|
||||
|
||||
it('preserves a complete canonical multi-owner log except for header.version', () => {
|
||||
@@ -209,9 +364,14 @@ describe('released Session format v0 to v1', () => {
|
||||
},
|
||||
{ type: 'turn/end', seq: 19, time: 20, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
const source = releasedV0SessionFormatCodec.decodeArtifact(physicalHeader, rows)
|
||||
const migrated = sessionFormatV0ToV1.migrate(source)
|
||||
expect(migrated).toEqual({ ...source, header: { ...source.header, version: 1 } })
|
||||
const migrated = restoreV0ToV1(physicalHeader, rows)
|
||||
expect(migrated).toEqual({
|
||||
header: {
|
||||
version: 1, id: 'full-identity', createdAt: 1, cwd: '/work', isSeeded: false, delegationDepth: 0,
|
||||
},
|
||||
inheritedEventCount: 0,
|
||||
events: rows,
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps the v1 physical codec vocabulary-neutral for current growth and a future source freeze', () => {
|
||||
@@ -219,10 +379,17 @@ describe('released Session format v0 to v1', () => {
|
||||
type: 'session', version: 1, id: 'ordinary-growth', createdAt: 1, delegationDepth: 0,
|
||||
}
|
||||
const ordinary = { type: 'ordinary/post-v1', seq: 0, time: 1, data: { required: true } }
|
||||
const decoded = releasedV1SessionFormatCodec.decodeArtifact(physicalHeader, [ordinary])
|
||||
const decoder = releasedV1SessionFormatCodec.createDecoder(physicalHeader, 'strict')
|
||||
const events = new SessionFormatEventCollector()
|
||||
decoder.decodeRow(ordinary, events)
|
||||
const decoded = {
|
||||
header: decoder.header,
|
||||
inheritedEventCount: decoder.finish(events),
|
||||
events: events.values,
|
||||
}
|
||||
|
||||
expect(decoded.events).toEqual([ordinary])
|
||||
expect(() => { sessionFormatV0ToV1.validateTarget(decoded) }).toThrow(/unknown required event/)
|
||||
expect(RELEASED_V0_EVENT_TYPES).not.toContain(ordinary.type)
|
||||
|
||||
const generatedCurrentTypes = new Set([...RELEASED_V0_EVENT_TYPES, ordinary.type])
|
||||
expect(() => restoreReleasedV1Artifact(decoded, generatedCurrentTypes)).not.toThrow()
|
||||
@@ -230,10 +397,19 @@ describe('released Session format v0 to v1', () => {
|
||||
const frozenFutureV1SourceTypes = new Set(generatedCurrentTypes)
|
||||
expect(() => restoreReleasedV1Artifact(decoded, frozenFutureV1SourceTypes)).not.toThrow()
|
||||
|
||||
const extendedKnownPayload = releasedV1SessionFormatCodec.decodeArtifact(physicalHeader, [{
|
||||
const extendedDecoder = releasedV1SessionFormatCodec.createDecoder(physicalHeader, 'strict')
|
||||
const extendedEvents = new SessionFormatEventCollector()
|
||||
extendedDecoder.decodeRow({
|
||||
type: 'turn/start', seq: 0, time: 1, data: { turn: 1, postReleaseMember: true },
|
||||
}, extendedEvents)
|
||||
const extendedKnownPayload = {
|
||||
header: extendedDecoder.header,
|
||||
inheritedEventCount: extendedDecoder.finish(extendedEvents),
|
||||
events: extendedEvents.values,
|
||||
}
|
||||
expect(extendedKnownPayload.events).toEqual([{
|
||||
type: 'turn/start', seq: 0, time: 1, data: { turn: 1, postReleaseMember: true },
|
||||
}])
|
||||
expect(() => { sessionFormatV0ToV1.validateTarget(extendedKnownPayload) }).toThrow(/unexpected member/)
|
||||
expect(() => restoreReleasedV1Artifact(extendedKnownPayload, generatedCurrentTypes)).not.toThrow()
|
||||
})
|
||||
})
|
||||
@@ -1,11 +1,9 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SessionFormatUnsupportedMigrationError } from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
releasedV0SessionFormatCodec,
|
||||
releasedV1SessionFormatCodec,
|
||||
assertReleasedV1Artifact,
|
||||
sessionFormatV0ToV1,
|
||||
assertReleasedArtifactRelationships,
|
||||
} from '../src/index.ts'
|
||||
import { restoreV0ToV1, restoreV1 } from '../src/testing/restore.ts'
|
||||
|
||||
const header = {
|
||||
type: 'session', version: 1, id: 'relationships', createdAt: 1, delegationDepth: 0,
|
||||
@@ -15,9 +13,7 @@ const user = (id: string, source: object = { kind: 'user' }) => ({
|
||||
})
|
||||
|
||||
function decode(rows: readonly unknown[], physicalHeader: unknown = header) {
|
||||
const artifact = releasedV1SessionFormatCodec.decodeArtifact(physicalHeader, rows)
|
||||
assertReleasedV1Artifact(artifact)
|
||||
return artifact
|
||||
return restoreV1(physicalHeader, rows)
|
||||
}
|
||||
|
||||
describe('released v1 whole-artifact relationships', () => {
|
||||
@@ -299,6 +295,7 @@ describe('released v1 whole-artifact relationships', () => {
|
||||
})
|
||||
expect(() => decode([...prefix, retry({ maxRetries: undefined })])).toThrow()
|
||||
expect(() => decode([...prefix, retry({ retry: 2, maxRetries: 1 })])).toThrow()
|
||||
expect(() => decode([...prefix, retry({ turn: 2 })])).toThrow(/current turn/)
|
||||
expect(() => decode([...prefix, retry({ provider: 'q' })])).toThrow(/provider/)
|
||||
expect(() => decode([...prefix, retry({ failure: { message: 'x', code: 'X', status: 99 } })])).toThrow(/status/)
|
||||
expect(() => decode([...prefix, retry({ delayMs: -0.5 })])).toThrow(/non-negative/)
|
||||
@@ -309,6 +306,25 @@ describe('released v1 whole-artifact relationships', () => {
|
||||
.toHaveLength(4)
|
||||
})
|
||||
|
||||
it('accepts a legacy retry scheduled immediately after its step closes', () => {
|
||||
const rows = [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
{ type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } },
|
||||
{ type: 'request/header', seq: 2, time: 3, data: { header: { config: { provider: 'p', model: 'm' } }, reason: 'initial' } },
|
||||
{ type: 'step/end', seq: 3, time: 4, data: { turn: 1, step: 1 } },
|
||||
{
|
||||
type: 'llm/retry', seq: 4, time: 5,
|
||||
data: {
|
||||
retryId: 'r', turn: 1, step: 1, provider: 'p', mode: 'normal', policyKey: 'k', retry: 1,
|
||||
maxRetries: 2, delayMs: 1, failure: { message: 'retry', code: 'SERVER' },
|
||||
},
|
||||
},
|
||||
{ type: 'turn/end', seq: 5, time: 6, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
|
||||
expect(decode(rows).events).toHaveLength(rows.length)
|
||||
})
|
||||
|
||||
it('enforces command pairing and authoritative source-event rules', () => {
|
||||
expect(() => decode([{
|
||||
type: 'command/done', seq: 0, time: 1, data: { commandId: 'c', kind: 'success' },
|
||||
@@ -366,7 +382,9 @@ describe('released v1 whole-artifact relationships', () => {
|
||||
}
|
||||
const decoded = decode([...prefix, inertV0])
|
||||
expect(decoded.events[1]).toEqual(inertV0)
|
||||
expect(() => { sessionFormatV0ToV1.validateTarget(decoded) }).not.toThrow()
|
||||
expect(() => {
|
||||
assertReleasedArtifactRelationships(decoded, { legacyInterruptedTurnRestart: true })
|
||||
}).not.toThrow()
|
||||
})
|
||||
|
||||
it('validates versioned subagent descriptors by source/current policy', () => {
|
||||
@@ -376,7 +394,7 @@ describe('released v1 whole-artifact relationships', () => {
|
||||
}
|
||||
expect(decode([future]).events).toEqual([future])
|
||||
const v0Header = { ...header, version: 0 }
|
||||
expect(() => sessionFormatV0ToV1.migrate(releasedV0SessionFormatCodec.decodeArtifact(v0Header, [future])))
|
||||
expect(() => restoreV0ToV1(v0Header, [future]))
|
||||
.toThrow(SessionFormatUnsupportedMigrationError)
|
||||
})
|
||||
|
||||
@@ -465,10 +483,30 @@ describe('released v1 whole-artifact relationships', () => {
|
||||
data: { sessionId: 'wrong', throughSeq: 0 },
|
||||
},
|
||||
]
|
||||
expect(() => sessionFormatV0ToV1.migrate(releasedV0SessionFormatCodec.decodeArtifact(v0, rows)))
|
||||
expect(() => restoreV0ToV1(v0, rows))
|
||||
.toThrow(/wrong Session/)
|
||||
})
|
||||
|
||||
it('accepts the released interrupted-turn restart sequence during v0 migration', () => {
|
||||
const v0 = { ...header, version: 0 }
|
||||
const rows = [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
{ type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } },
|
||||
{ type: 'step/end', seq: 2, time: 3, data: { turn: 1, step: 1 } },
|
||||
{
|
||||
type: 'agent/inbox/spliced', seq: 3, time: 4,
|
||||
data: { target: 'next-turn', start: 0, inserted: [user('queued')] },
|
||||
},
|
||||
{ type: 'turn/start', seq: 4, time: 5, data: { turn: 2 } },
|
||||
{ type: 'turn/end', seq: 5, time: 6, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
]
|
||||
|
||||
expect(restoreV0ToV1(v0, rows).events).toEqual(rows)
|
||||
expect(() => restoreV0ToV1(v0, rows.map(candidate => candidate.seq === 3
|
||||
? { ...candidate, data: { ...candidate.data, inserted: [] } }
|
||||
: candidate))).toThrow(/does not open expected turn/)
|
||||
})
|
||||
|
||||
it('accepts recoverable open tails for later interrupted-turn repair', () => {
|
||||
const v0 = { ...header, version: 0 }
|
||||
const rows = [
|
||||
@@ -500,8 +538,7 @@ describe('released v1 whole-artifact relationships', () => {
|
||||
},
|
||||
{ type: 'compaction/start', seq: 7, time: 8, data: { compactionId: 'c', turn: 1 } },
|
||||
]
|
||||
const recovered = releasedV0SessionFormatCodec.decodeRecoverableArtifact(v0, rows)
|
||||
expect(sessionFormatV0ToV1.migrate(recovered).events).toHaveLength(rows.length)
|
||||
expect(restoreV0ToV1(v0, rows, 'recoverable').events).toHaveLength(rows.length)
|
||||
})
|
||||
|
||||
it('refuses invalid core openings, request placement, and replacement placement', () => {
|
||||
|
||||
@@ -4,12 +4,16 @@ import { KNOWN_SESSION_EVENT_TYPES } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
RELEASED_V0_EVENT_TYPES,
|
||||
RELEASED_V0_EVENT_DISPOSITIONS,
|
||||
assertReleasedV1Artifact,
|
||||
releasedV0SessionFormatCodec,
|
||||
releasedV1SessionFormatCodec,
|
||||
sessionFormatV0ToV1,
|
||||
} from '../src/index.ts'
|
||||
import { assertReleasedEventPayload } from '../src/validation.ts'
|
||||
import { assertReleasedEventPayload, assertReleasedSurfaceMetadata } from '../src/validation.ts'
|
||||
import { restoreV0ToV1, restoreV1 } from '../src/testing/restore.ts'
|
||||
import {
|
||||
assertNormalizedReleasedV0Artifact,
|
||||
assertReleasedV0SourceArtifact,
|
||||
assertReleasedV1Artifact,
|
||||
assertReleasedV1MigrationSource,
|
||||
assertReleasedV1PhysicalArtifact,
|
||||
} from '../src/testing/validation.ts'
|
||||
|
||||
const v0Header = {
|
||||
type: 'session', version: 0, id: 'validation', createdAt: 1, delegationDepth: 0,
|
||||
@@ -290,6 +294,15 @@ describe('released event and payload inventory', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('refuses opaque members that are not lossless JSON without snapshotting them', () => {
|
||||
const data = {
|
||||
...(validPayloads['tool/result'] as Record<string, unknown>),
|
||||
meta: { value: undefined },
|
||||
}
|
||||
expect(() => { assertPayload('tool/result', data as unknown as SessionFormatJsonValue) })
|
||||
.toThrow(/opaque meta is not lossless JSON/)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['user/message content block', 'user/message', {
|
||||
...userMessage,
|
||||
@@ -312,10 +325,58 @@ describe('released event and payload inventory', () => {
|
||||
|
||||
it('refuses unknown v0 events even when the envelope marks them ignorable', () => {
|
||||
const row = { type: 'plugin/unknown', seq: 0, time: 1, data: {}, ignorable: true }
|
||||
expect(() => releasedV0SessionFormatCodec.decodeArtifact(v0Header, [row]))
|
||||
expect(() => restoreV0ToV1(v0Header, [row]))
|
||||
.toThrow(/unknown historical event.*refuses.*ignorable/)
|
||||
})
|
||||
|
||||
it('exercises each released test validation policy', () => {
|
||||
const v0 = {
|
||||
header: { version: 0, id: 'validation', createdAt: 1, isSeeded: false, delegationDepth: 0 },
|
||||
inheritedEventCount: 0,
|
||||
events: [],
|
||||
} as const
|
||||
const v1 = { ...v0, header: { ...v0.header, version: 1 } } as const
|
||||
const forwardCompatible = {
|
||||
...v1,
|
||||
events: [{ type: 'plugin/future', seq: 0, time: 1, data: {}, ignorable: true }],
|
||||
} as const
|
||||
|
||||
expect(() => { assertReleasedV0SourceArtifact(v0) }).not.toThrow()
|
||||
expect(() => { assertNormalizedReleasedV0Artifact({
|
||||
...v0,
|
||||
events: [{ type: 'feedback/record', seq: 0, time: 1, data: { text: 'retained' } }],
|
||||
}) }).not.toThrow()
|
||||
expect(() => { assertReleasedV1MigrationSource(v1) }).not.toThrow()
|
||||
expect(() => { assertReleasedV1MigrationSource(forwardCompatible) }).not.toThrow()
|
||||
expect(() => { assertReleasedV1PhysicalArtifact(v1) }).not.toThrow()
|
||||
expect(() => { assertReleasedV0SourceArtifact({
|
||||
...v0,
|
||||
events: [{
|
||||
type: 'steering/message', seq: 0, time: 1, surfaceOp: 'append',
|
||||
data: { turn: 1, content: [], source: { kind: 'user' } },
|
||||
}],
|
||||
}) }).not.toThrow()
|
||||
expect(() => { assertReleasedV0SourceArtifact({
|
||||
...v0,
|
||||
events: [{ type: 'compact/start', seq: 0, time: 1, data: { turn: null } }],
|
||||
}) }).not.toThrow()
|
||||
expect(() => { assertReleasedV0SourceArtifact({
|
||||
...v0,
|
||||
events: [{ type: 'plugin/unknown', seq: 0, time: 1, data: {}, ignorable: true }],
|
||||
}) }).toThrow(/unknown historical event/)
|
||||
})
|
||||
|
||||
it('permits empty Assistant provenance only under the released-v1 policy', () => {
|
||||
const assistant = {
|
||||
type: 'assistant/message', seq: 1, time: 2, data: {},
|
||||
sourceEventSeqs: [], surfaceOp: 'append',
|
||||
}
|
||||
expect(() => { assertReleasedSurfaceMetadata(assistant, 1, assistant.type, 'allow-empty-assistant') })
|
||||
.not.toThrow()
|
||||
expect(() => { assertReleasedSurfaceMetadata(assistant, 1, assistant.type, 'forbid-assistant') })
|
||||
.toThrow(/obsolete chunk provenance/)
|
||||
})
|
||||
|
||||
it('keeps capturedFormatVersion v1-only inside session-reference sources', () => {
|
||||
const data = {
|
||||
id: 'reference', role: 'user', content: [textBlock],
|
||||
@@ -329,9 +390,9 @@ describe('released event and payload inventory', () => {
|
||||
},
|
||||
}
|
||||
const event = { type: 'user/message', seq: 0, time: 1, data, surfaceOp: 'append' }
|
||||
expect(() => sessionFormatV0ToV1.migrate(releasedV0SessionFormatCodec.decodeArtifact(v0Header, [event])))
|
||||
expect(() => restoreV0ToV1(v0Header, [event]))
|
||||
.toThrow(/capturedFormatVersion/)
|
||||
expect(releasedV1SessionFormatCodec.decodeArtifact(v1Header, [event]).events).toEqual([event])
|
||||
expect(restoreV1(v1Header, [event]).events).toEqual([event])
|
||||
})
|
||||
|
||||
it('accepts every released nested union variant and optional member', () => {
|
||||
@@ -489,6 +550,30 @@ describe('released event and payload inventory', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('validates legacy round-zero goal mutation messages', () => {
|
||||
const change = {
|
||||
kind: 'goal/change', version: 1, operation: 'clear',
|
||||
cleared: { id: 'goal', revision: 2 }, clearedAt: 3,
|
||||
}
|
||||
const content = [{
|
||||
type: 'text',
|
||||
text: `<goal_state>${JSON.stringify({ cleared: change.cleared, clearedAt: change.clearedAt })}</goal_state>`,
|
||||
}]
|
||||
const message = {
|
||||
id: 'legacy-goal', role: 'user', content,
|
||||
source: { kind: 'goal', goalId: 'goal', revision: 2, round: 0, change },
|
||||
}
|
||||
expect(() => { assertPayload('user/message', message) }).not.toThrow()
|
||||
expect(() => { assertPayload('user/message', {
|
||||
...message, source: { ...message.source, round: 1 },
|
||||
}) }).toThrow(/round must be 0/)
|
||||
expect(() => { assertPayload('user/message', {
|
||||
...message, source: { ...message.source, goalId: 'other' },
|
||||
}) }).toThrow(/does not match/)
|
||||
expect(() => { assertPayload('user/message', { ...message, content: [textBlock] }) })
|
||||
.toThrow(/content does not match/)
|
||||
})
|
||||
|
||||
it('refuses malformed logical headers, cuts, event envelopes, and surface metadata', () => {
|
||||
const base = v1Artifact('turn/start', { turn: 1 })
|
||||
const invalidHeaders = [
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-format-v1-to-v2/README.md
|
||||
README.md: 1a61e99d8955ab07cca20bd67cc35c8a48f7a918
|
||||
README.zh.md: 3072807faf72f7f62b2342151914c76f62ab247e
|
||||
README.md: 74a780b41735bb67676c79d44bef89dc0eb08d6e
|
||||
README.zh.md: 6007833173f4a98cb40fea68a2bd6f423b3e802f
|
||||
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
## Summary
|
||||
|
||||
`dsh-session-format-v1-to-v2` converts a complete released-v1 Session into the released-v2 event model. It consumes top-level `assistant/chunk` events, embeds their exact timed stream in the matching `assistant/message`, and records an `assistant/attempt` when a failed, retried, cancelled, or stream-error attempt reached settlement without a surface message. The edge densely remaps surviving events and every declared same-Session sequence reference, while the v2 codec stores one event per row and derives the inherited cut from a tagged `session/end-seed` marker.
|
||||
`dsh-session-format-v1-to-v2` converts a released-v1 Session into the released-v2 event model through one stateful event stage. It consumes top-level `assistant/chunk` events, embeds their exact timed stream in the matching `assistant/message`, and records an `assistant/attempt` when a failed, retried, cancelled, or stream-error attempt reached settlement without a surface message. The edge densely remaps surviving events and every declared same-Session sequence reference, while the v2 codec stores one event per row and derives the inherited cut from a tagged `session/end-seed` marker.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -27,22 +27,29 @@ English | [中文](README.zh.md)
|
||||
|
||||
### When to use it
|
||||
|
||||
Persistence obtains this edge through `dsh-session-format-catalog`; feature compositions do not mount it. Import it directly only when assembling or testing the static released-format catalog or inspecting the exact v1-to-v2 transformation. No runtime invariant companion is published because every codec and migration call validates its complete source or target artifact and retains no runtime state.
|
||||
Persistence obtains this edge through `dsh-session-format-catalog`; feature compositions do not mount it. Import it directly only when assembling or testing the static released-format catalog or inspecting the exact v1-to-v2 transformation. No runtime invariant companion is published because the package has no independently observable runtime registrations whose state can diverge; decoder and transformer state belongs to one restore.
|
||||
|
||||
### Entry point
|
||||
|
||||
```text
|
||||
const decodedV1 = releasedV1SessionFormatCodec.decodeArtifact(header, rows)
|
||||
const migratedV2 = sessionFormatV1ToV2.migrate(decodedV1)
|
||||
const decoder = releasedV1SessionFormatCodec.createDecoder(physicalHeader, 'strict')
|
||||
for (const row of physicalRows) decoder.decodeRow(row, migrationContext)
|
||||
const stage = sessionFormatV1ToV2.createStage(stageInput)
|
||||
stage.transformEvent(event, migrationContext)
|
||||
const targetInheritedEventCount = stage.finish(migrationContext)
|
||||
const headerRecord = releasedV2SessionFormatCodec.encodeHeader(currentHeader, targetInheritedEventCount)
|
||||
const eventRecord = releasedV2SessionFormatCodec.encodeEvent(currentEvent)
|
||||
```
|
||||
|
||||
`releasedV1SessionFormatCodec` reads the frozen v1 physical language. `sessionFormatV1ToV2` validates that complete source, performs the cardinality-changing transformation, remaps declared references, and validates the exact v2 result. `releasedV2SessionFormatCodec` then encodes or decodes the current physical representation.
|
||||
`releasedV1SessionFormatCodec` reads the frozen v1 physical language one row at a time. `sessionFormatV1ToV2` creates the cardinality-changing Stage that the static catalog connects to that decoder without retaining a v1 event array. The catalog remaps declared references and validates the released-v2 envelope, inherited cut, event admission, and relationships. Persistence applies full installed-current validation in its Worker before publication. `releasedV2SessionFormatCodec` creates a current row decoder and encodes current headers and events one record at a time.
|
||||
|
||||
A successful v1 `assistant/message` must cite its complete ordered attempt. The migration removes the cited top-level chunks and obsolete message provenance, compacts the chunks without joining token boundaries, and stores the stream on that message. An unclaimed attempt becomes one log-only `assistant/attempt` at its final chunk position. Unrelated interleaved events keep their relative order.
|
||||
|
||||
The edge also closes the bounded legacy restart pattern in which a non-empty `next-turn` inbox insertion is followed by the next `turn/start` without the prior `turn/end`. It records that prior turn as interrupted. A legacy round-zero goal mutation becomes a `goal/change` followed by the original model-visible message with ordinary plugin attribution, so both durable goal state and historical model input survive.
|
||||
|
||||
The migration refuses a reference to a consumed chunk instead of redirecting it to a different semantic event. It remaps declared event provenance, surface replacements, command source events, compaction ranges and lists, and title message lists. The already model-visible `session/title-llm-request.messages` text remains byte-identical after source validation, so target validation does not reinterpret the old sequence numbers embedded in that prompt. A seeded source also refuses an inherited cut that splits an Assistant attempt; the target marks the exact cut with `session/end-seed { inherited: true }`.
|
||||
|
||||
The v2 physical header requires `isSeeded` and does not store a numeric cut. The codec derives the cut from the last inherited end-seed marker, writes one event per row, range-encodes only `sourceEventSeqs`, and remains neutral to ordinary event vocabulary and payload growth. Strict migration-target validation freezes the released-v2 inventory and rejects unknown types or members. Current restoration instead admits event types known to the installed Session package plus unknown events carrying `ignorable: true`, then delegates payload and stream semantics to the installed current restorer. All paths retain strict header, event-envelope, sequence, and inherited-cut validation.
|
||||
The v2 physical header requires `isSeeded` and does not store a numeric cut. The codec derives the cut from the last inherited end-seed marker, writes one event per row, range-encodes only `sourceEventSeqs`, and remains neutral to ordinary event vocabulary and payload growth. Released-current restoration admits event types known to the installed Session package plus unknown events carrying `ignorable: true`, and validates event members and relationships. Full current restoration additionally delegates payload and embedded-stream semantics to the installed Session package. The frozen exact writer-image validator lives under `src/testing` for edge fixtures.
|
||||
|
||||
-----
|
||||
|
||||
@@ -52,13 +59,13 @@ The v2 physical header requires `isSeeded` and does not store a numeric cut. The
|
||||
<details>
|
||||
<summary>Implementation internals — click to expand</summary>
|
||||
|
||||
The edge first groups v1 chunks by turn, step, terminal finish, and explicit message provenance. It stages survivors in source order, substitutes one settlement for each group, computes a dense old-to-new sequence map, and rewrites only the reference fields declared by the frozen event inventory. Source and target validators bracket the transformation so a partially understood artifact is never admitted.
|
||||
The incremental edge retains one unsettled Assistant attempt, events whose output position depends on that attempt, and the dense old-to-new sequence map. It emits settled survivors in source order and rewrites only reference fields declared by the frozen event inventory. Released-current validation rejects any relationship the transformation cannot preserve.
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| [`src/migration.ts`](src/migration.ts) | Attempt grouping, settlement substitution, dense sequence mapping, and reference rewriting |
|
||||
| [`src/codec.ts`](src/codec.ts) | Released-v2 header, one-event-per-row encoding, provenance ranges, and recoverable prefix decoding |
|
||||
| [`src/validation.ts`](src/validation.ts) | Physical v2 envelope/cut validation, exact migration-target policy, and vocabulary-neutral current restoration |
|
||||
| [`src/validation.ts`](src/validation.ts) | Physical v2 envelope/cut validation and released-current event admission and relationships |
|
||||
| [`src/dispositions.ts`](src/dispositions.ts) | Frozen released-v2 event and payload-member inventory |
|
||||
|
||||
</details>
|
||||
@@ -97,7 +104,7 @@ The restored model-message sequence stays unchanged, so the migration alone does
|
||||
<a id="known-limitations-and-deferred-work"></a>
|
||||
|
||||
- **Closed first-party source inventory** — an unknown v1 event refuses migration, including an event marked `ignorable: true`.
|
||||
- **Whole-artifact transformation** — the edge materializes the source, target, and sequence map in memory; it does not stream the rewrite.
|
||||
- **Linear remap state** — streaming retains no complete v1 event array, but the final v2 event array and old-to-new sequence map remain O(event count).
|
||||
- **No publication or compatibility fallback** — persistence owns exclusive successor publication, and retained v1 generations are not automatic downgrade or restore inputs.
|
||||
|
||||
<a id="dev-note"></a>
|
||||
|
||||
@@ -9,7 +9,7 @@ kind: "package-reference"
|
||||
|
||||
## 概述
|
||||
|
||||
`dsh-session-format-v1-to-v2` 把完整的已发布 v1 Session 转换为已发布 v2 事件模型。它会消费顶层 `assistant/chunk` 事件，把精确的带时间流嵌入匹配的 `assistant/message`，并在失败、重试、取消或 stream error attempt 已到达 settlement、但没有产生 surface message 时记录 `assistant/attempt`。该迁移边会密集重映射存活事件和每个已声明的同 Session 序号引用；v2 编解码器则让每行只存一个事件，并从带标记的 `session/end-seed` 事件推导继承切点。
|
||||
`dsh-session-format-v1-to-v2` 通过一个有状态事件 Stage，把已发布 v1 Session 转换为已发布 v2 事件模型。它会消费顶层 `assistant/chunk` 事件，把精确的带时间流嵌入匹配的 `assistant/message`，并在失败、重试、取消或 stream error attempt 已到达 settlement、但没有产生 surface message 时记录 `assistant/attempt`。该迁移边会密集重映射存活事件和每个已声明的同 Session 序号引用；v2 codec 则让每行只存一个事件，并从带标记的 `session/end-seed` 事件推导继承切点。
|
||||
|
||||
## 目录
|
||||
|
||||
@@ -27,22 +27,29 @@ kind: "package-reference"
|
||||
|
||||
### 何时使用
|
||||
|
||||
持久化通过 `dsh-session-format-catalog` 获取该迁移边；功能组合不会挂载它。只有在装配或测试静态已发布格式目录，或检查精确的 v1 到 v2 转换时，才直接导入本包。它不发布运行时不变式伴生入口，因为每次 codec 与迁移调用都会校验完整的源或目标 artifact，且不保留运行时状态。
|
||||
持久化通过 `dsh-session-format-catalog` 获取该迁移边；功能组合不会挂载它。只有在装配或测试静态已发布格式目录，或检查精确的 v1 到 v2 转换时，才直接导入本包。它不发布运行时不变式伴生入口，因为本包没有状态可能彼此分歧的、可独立观测的运行时注册项；decoder 与 transformer 状态只属于一次还原。
|
||||
|
||||
### 入口
|
||||
|
||||
```text
|
||||
const decodedV1 = releasedV1SessionFormatCodec.decodeArtifact(header, rows)
|
||||
const migratedV2 = sessionFormatV1ToV2.migrate(decodedV1)
|
||||
const decoder = releasedV1SessionFormatCodec.createDecoder(physicalHeader, 'strict')
|
||||
for (const row of physicalRows) decoder.decodeRow(row, migrationContext)
|
||||
const stage = sessionFormatV1ToV2.createStage(stageInput)
|
||||
stage.transformEvent(event, migrationContext)
|
||||
const targetInheritedEventCount = stage.finish(migrationContext)
|
||||
const headerRecord = releasedV2SessionFormatCodec.encodeHeader(currentHeader, targetInheritedEventCount)
|
||||
const eventRecord = releasedV2SessionFormatCodec.encodeEvent(currentEvent)
|
||||
```
|
||||
|
||||
`releasedV1SessionFormatCodec` 读取冻结的 v1 物理语言。`sessionFormatV1ToV2` 校验完整源产物、执行基数变化转换、重映射已声明引用，并校验精确的 v2 结果。`releasedV2SessionFormatCodec` 随后编码或解码当前物理表示。
|
||||
`releasedV1SessionFormatCodec` 逐行读取冻结的 v1 物理语言。`sessionFormatV1ToV2` 创建改变事件基数的 Stage，静态 catalog 把它连接到 decoder，且不保留 v1 事件数组。Catalog 会重映射已声明引用，并校验 released-v2 envelope、inherited cut、事件准入与关系。持久化在发布前通过 Worker 执行完整 installed-current 校验。`releasedV2SessionFormatCodec` 创建当前格式的逐行 decoder，并逐条编码当前 header 与事件。
|
||||
|
||||
成功的 v1 `assistant/message` 必须引用其完整有序 attempt。迁移会移除这些顶层 chunk 和已停用的 message provenance，在不合并 token 边界的前提下压缩 chunk，并把 stream 存到该 message 上。未被 message 认领的 attempt 会在其最后一个 chunk 的位置变成一个仅日志可见的 `assistant/attempt`。无关的交错事件保持相对顺序。
|
||||
|
||||
该 edge 还会闭合一种有限的旧版恢复模式：非空的 `next-turn` inbox 插入后直接出现下一个 `turn/start`，但缺少前一轮的 `turn/end`；迁移将前一轮记录为 interrupted。旧版 round-zero goal mutation 会变成一个 `goal/change`，随后保留原本模型可见的 message 并改用普通 plugin attribution，因此持久 goal 状态与历史模型输入都会保留。
|
||||
|
||||
如果引用指向被消费的 chunk，迁移会失败，而不会把它重定向到语义不同的事件。它会重映射已声明的事件 provenance、surface replacement、command source event、compaction range 与 list，以及 title message list。已经对模型可见的 `session/title-llm-request.messages` 文本会在源校验后保持逐字节不变，因此目标校验不会重新解释该 prompt 中嵌入的旧序号。带 seed 的源若让继承切点切开一个 Assistant attempt，也会迁移失败；目标会用 `session/end-seed { inherited: true }` 标出精确切点。
|
||||
|
||||
v2 物理 header 要求 `isSeeded`，且不存储数值切点。编解码器从最后一个 inherited end-seed marker 推导切点，每行写入一个事件，只对 `sourceEventSeqs` 做范围编码，并对普通事件词汇与 payload 扩展保持中立。严格的迁移目标校验会冻结 released-v2 清单并拒绝未知 type 或 member。当前恢复则准入 installed Session package 已知的事件 type，以及携带 `ignorable: true` 的未知事件，再把 payload 与 stream 语义交给 installed current restorer。所有路径仍严格校验 header、event envelope、sequence 与 inherited cut。
|
||||
v2 物理 header 要求 `isSeeded`，且不存储数值切点。编解码器从最后一个 inherited end-seed marker 推导切点，每行写入一个事件，只对 `sourceEventSeqs` 做范围编码，并对普通事件词汇与 payload 扩展保持中立。Released-current restoration 准入 installed Session package 已知的事件 type，以及携带 `ignorable: true` 的未知事件，并校验事件 member 与关系。完整 current restoration 还会把 payload 与嵌入 stream 语义交给 installed Session package。冻结的精确 writer-image 校验器位于 `src/testing`，供 edge fixture 使用。
|
||||
|
||||
-----
|
||||
|
||||
@@ -52,13 +59,13 @@ v2 物理 header 要求 `isSeeded`，且不存储数值切点。编解码器从
|
||||
<details>
|
||||
<summary>实现细节——点击展开</summary>
|
||||
|
||||
该迁移边先按 turn、step、terminal finish 和显式 message provenance 对 v1 chunk 分组。它按源顺序暂存存活事件，为每组替换一个 settlement，计算密集的旧序号到新序号映射，并且只改写冻结事件清单声明的引用字段。源与目标校验器包围整个转换，因此部分理解的产物绝不会被接纳。
|
||||
增量迁移边会保留一个尚未结算的 Assistant attempt、输出位置取决于该 attempt 的事件，以及密集的旧序号到新序号映射。它按源顺序发出已结算的存活事件，并且只重写冻结事件清单声明的引用字段。Released-current 校验会拒绝转换无法保留的任何关系。
|
||||
|
||||
| 文件 | 职责 |
|
||||
|---|---|
|
||||
| [`src/migration.ts`](src/migration.ts) | Attempt 分组、settlement 替换、密集序号映射与引用重写 |
|
||||
| [`src/codec.ts`](src/codec.ts) | 已发布 v2 header、每行一个事件的编码、provenance 范围与可恢复前缀解码 |
|
||||
| [`src/validation.ts`](src/validation.ts) | v2 物理 envelope／cut 校验、精确 migration-target 策略与 vocabulary-neutral current restoration |
|
||||
| [`src/validation.ts`](src/validation.ts) | v2 物理 envelope／cut 校验，以及 released-current 事件准入与关系校验 |
|
||||
| [`src/dispositions.ts`](src/dispositions.ts) | 冻结的已发布 v2 事件与 payload 成员清单 |
|
||||
|
||||
</details>
|
||||
@@ -97,7 +104,7 @@ v2 物理 header 要求 `isSeeded`，且不存储数值切点。编解码器从
|
||||
<a id="known-limitations-and-deferred-work"></a>
|
||||
|
||||
- **封闭的第一方源清单**——未知 v1 事件会使迁移失败，包括带有 `ignorable: true` 的事件。
|
||||
- **全产物转换**——该迁移边会在内存中物化源、目标和序号映射；它不会流式改写。
|
||||
- **线性重映射状态**——流式处理不保留完整 v1 事件数组，但最终 v2 事件数组和旧到新序号映射仍为 O(事件数)。
|
||||
- **不负责发布或兼容回退**——持久化拥有排他 successor 发布，保留的 v1 generation 不是自动 downgrade 或 restore 输入。
|
||||
|
||||
<a id="dev-note"></a>
|
||||
|
||||
@@ -1,40 +1,42 @@
|
||||
import {
|
||||
SessionFormatError,
|
||||
sessionFormatCount,
|
||||
snapshotSessionFormatArtifact,
|
||||
sessionFormatSafeInteger,
|
||||
snapshotSessionFormatJson,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
EncodedSessionFormatArtifact,
|
||||
SessionFormatArtifact,
|
||||
SessionFormatArtifactDecoder,
|
||||
SessionFormatCodec,
|
||||
SessionFormatCurrentEncoder,
|
||||
SessionFormatEvent,
|
||||
SessionFormatHeader,
|
||||
SessionFormatJsonObject,
|
||||
SessionFormatJsonValue,
|
||||
SessionFormatRecovery,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import { assertReleasedV2Header, assertReleasedV2PhysicalArtifact } from './validation.ts'
|
||||
import { assertReleasedV2Header } from './validation.ts'
|
||||
|
||||
const HEADER_REQUIRED = ['type', 'version', 'id', 'createdAt', 'isSeeded', 'delegationDepth'] as const
|
||||
const HEADER_OPTIONAL = ['cwd', 'parentSession', 'origin', 'agentPreset'] as const
|
||||
const EVENT_REQUIRED = ['type', 'seq', 'time', 'data'] as const
|
||||
const EVENT_OPTIONAL = ['ignorable', 'sourceEventSeqs', 'surfaceOp'] as const
|
||||
const EVENT_KEYS: ReadonlySet<string> = new Set([...EVENT_REQUIRED, ...EVENT_OPTIONAL])
|
||||
/** Frozen physical JSON codec for released v2. */
|
||||
export const releasedV2SessionFormatCodec = Object.freeze({
|
||||
version: 2,
|
||||
decodeHeader(value: unknown) {
|
||||
return decodePhysicalHeader(value)
|
||||
},
|
||||
decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]) {
|
||||
return decodeArtifact(headerValue, rowValues, false)
|
||||
createDecoder(headerValue: unknown, recovery: SessionFormatRecovery) {
|
||||
return createDecoder(headerValue, recovery)
|
||||
},
|
||||
decodeRecoverableArtifact(headerValue: unknown, rowValues: readonly unknown[]) {
|
||||
return decodeArtifact(headerValue, rowValues, true)
|
||||
encodeHeader(header: SessionFormatHeader, inheritedEventCount: number) {
|
||||
return encodeHeader(header, inheritedEventCount)
|
||||
},
|
||||
encodeArtifact(artifact: SessionFormatArtifact) {
|
||||
return encodeArtifact(artifact)
|
||||
encodeEvent(event: SessionFormatEvent) {
|
||||
return encodeProvenance(event)
|
||||
},
|
||||
} satisfies SessionFormatCodec & {
|
||||
encodeArtifact(artifact: SessionFormatArtifact): EncodedSessionFormatArtifact
|
||||
})
|
||||
} satisfies SessionFormatCodec & SessionFormatCurrentEncoder)
|
||||
|
||||
function decodePhysicalHeader(value: unknown): SessionFormatHeader {
|
||||
const snapshot = snapshotSessionFormatJson(value, 'released v2 physical header')
|
||||
@@ -70,78 +72,96 @@ function decodePhysicalHeader(value: unknown): SessionFormatHeader {
|
||||
return header
|
||||
}
|
||||
|
||||
function decodeArtifact(
|
||||
function createDecoder(
|
||||
headerValue: unknown,
|
||||
rowValues: readonly unknown[],
|
||||
recoverable: boolean,
|
||||
): SessionFormatArtifact {
|
||||
recovery: SessionFormatRecovery,
|
||||
): SessionFormatArtifactDecoder {
|
||||
const header = decodePhysicalHeader(headerValue)
|
||||
const events: SessionFormatEvent[] = []
|
||||
let rowIndex = 0
|
||||
let eventCount = 0
|
||||
let inheritedEventCount: number | undefined
|
||||
let issue: SessionFormatError | undefined
|
||||
for (const [rowIndex, value] of rowValues.entries()) {
|
||||
let event: SessionFormatEvent
|
||||
try {
|
||||
event = decodeEvent(value, rowIndex)
|
||||
} catch (error: unknown) {
|
||||
const current = error instanceof SessionFormatError
|
||||
? error
|
||||
: new SessionFormatError(`released v2 row ${rowIndex} is malformed`, { cause: error })
|
||||
if (!recoverable) throw current
|
||||
issue ??= current
|
||||
continue
|
||||
}
|
||||
if (issue !== undefined) {
|
||||
if (event.type === 'turn/end') throw issue
|
||||
continue
|
||||
}
|
||||
if (event.seq !== events.length) {
|
||||
const gap = new SessionFormatError(
|
||||
`released v2 row ${rowIndex} has seq gap (expected ${events.length}, got ${event.seq})`,
|
||||
)
|
||||
if (!recoverable) throw gap
|
||||
issue = gap
|
||||
if (event.type === 'turn/end') throw issue
|
||||
continue
|
||||
}
|
||||
events.push(event)
|
||||
return {
|
||||
header,
|
||||
decodeRow(value, context) {
|
||||
const currentRow = rowIndex
|
||||
rowIndex += 1
|
||||
let event: SessionFormatEvent
|
||||
try {
|
||||
event = decodeEvent(value, currentRow)
|
||||
} catch (error: unknown) {
|
||||
const current = error instanceof SessionFormatError
|
||||
? error
|
||||
: new SessionFormatError(`released v2 row ${currentRow} is malformed`, { cause: error })
|
||||
if (recovery === 'strict') throw current
|
||||
issue ??= current
|
||||
return
|
||||
}
|
||||
if (issue !== undefined) {
|
||||
if (event.type === 'turn/end') throw issue
|
||||
return
|
||||
}
|
||||
if (event.seq !== eventCount) {
|
||||
const gap = new SessionFormatError(
|
||||
`released v2 row ${currentRow} has seq gap (expected ${eventCount}, got ${event.seq})`,
|
||||
)
|
||||
if (recovery === 'strict') throw gap
|
||||
issue = gap
|
||||
if (event.type === 'turn/end') throw issue
|
||||
return
|
||||
}
|
||||
eventCount += 1
|
||||
if (event.type === 'session/end-seed') {
|
||||
const data = jsonRecord(event.data, `session/end-seed ${event.seq} data`)
|
||||
if (data['inherited'] === true) inheritedEventCount = event.seq
|
||||
}
|
||||
context.emitEvent(event)
|
||||
},
|
||||
finish(_context) {
|
||||
if (header.isSeeded && inheritedEventCount === undefined) {
|
||||
throw new SessionFormatError('released v2 seeded Session lacks an inherited end-seed marker')
|
||||
}
|
||||
if (!header.isSeeded && inheritedEventCount !== undefined) {
|
||||
throw new SessionFormatError('released v2 unseeded Session contains an inherited end-seed marker')
|
||||
}
|
||||
return inheritedEventCount ?? 0
|
||||
},
|
||||
}
|
||||
const inheritedEventCount = deriveInheritedEventCount(header, events)
|
||||
const artifact = snapshotSessionFormatArtifact({ header, inheritedEventCount, events }, 'released v2 artifact')
|
||||
assertReleasedV2PhysicalArtifact(artifact)
|
||||
return artifact
|
||||
}
|
||||
|
||||
function decodeEvent(value: unknown, rowIndex: number): SessionFormatEvent {
|
||||
const snapshot = snapshotSessionFormatJson(value, `released v2 row ${rowIndex}`)
|
||||
const record = jsonRecord(snapshot, `released v2 row ${rowIndex}`)
|
||||
const record = jsonRecord(value as SessionFormatJsonValue, `released v2 row ${rowIndex}`)
|
||||
const missing = EVENT_REQUIRED.find(key => !Object.hasOwn(record, key))
|
||||
if (missing !== undefined) throw new SessionFormatError(`released v2 row ${rowIndex} lacks required field ${missing}`)
|
||||
const unexpected = Object.keys(record).find(key => !EVENT_KEYS.has(key))
|
||||
if (unexpected !== undefined) {
|
||||
throw new SessionFormatError(`released v2 row ${rowIndex} has unexpected field ${unexpected}`)
|
||||
}
|
||||
if (typeof record['type'] !== 'string') {
|
||||
throw new SessionFormatError(`released v2 row ${rowIndex} type must be a string`)
|
||||
}
|
||||
sessionFormatSafeInteger(record['time'], `released v2 row ${rowIndex} time`)
|
||||
if (record['ignorable'] !== undefined && record['ignorable'] !== true) {
|
||||
throw new SessionFormatError(`released v2 row ${rowIndex} ignorable must be true when present`)
|
||||
}
|
||||
if (record['sourceEventSeqs'] === undefined) return record as unknown as SessionFormatEvent
|
||||
const seq = sessionFormatCount(record['seq'], `released v2 row ${rowIndex} seq`)
|
||||
return snapshotSessionFormatJson({
|
||||
return {
|
||||
...record,
|
||||
sourceEventSeqs: decodeSeqRanges(record['sourceEventSeqs'], seq),
|
||||
}, `released v2 row ${rowIndex} provenance`) as SessionFormatEvent
|
||||
} as unknown as SessionFormatEvent
|
||||
}
|
||||
|
||||
function deriveInheritedEventCount(header: SessionFormatHeader, events: readonly SessionFormatEvent[]): number {
|
||||
let cut: number | undefined
|
||||
for (const event of events) {
|
||||
if (event.type !== 'session/end-seed') continue
|
||||
const data = jsonRecord(event.data, `session/end-seed ${event.seq} data`)
|
||||
if (data['inherited'] === true) cut = event.seq
|
||||
function encodeHeader(
|
||||
header: SessionFormatHeader,
|
||||
inheritedEventCount: number,
|
||||
): SessionFormatJsonObject {
|
||||
assertReleasedV2Header(header)
|
||||
const cut = sessionFormatCount(inheritedEventCount, 'format v2 inherited event count')
|
||||
if (!header.isSeeded && cut !== 0) {
|
||||
throw new SessionFormatError('unseeded format v2 Session has inherited events')
|
||||
}
|
||||
if (header.isSeeded && cut === undefined) {
|
||||
throw new SessionFormatError('released v2 seeded Session lacks an inherited end-seed marker')
|
||||
}
|
||||
if (!header.isSeeded && cut !== undefined) {
|
||||
throw new SessionFormatError('released v2 unseeded Session contains an inherited end-seed marker')
|
||||
}
|
||||
return cut ?? 0
|
||||
}
|
||||
|
||||
function encodeArtifact(artifact: SessionFormatArtifact): EncodedSessionFormatArtifact {
|
||||
assertReleasedV2PhysicalArtifact(artifact)
|
||||
const header = artifact.header
|
||||
const physicalHeader = snapshotSessionFormatJson({
|
||||
return {
|
||||
type: 'session',
|
||||
version: 2,
|
||||
id: header.id,
|
||||
@@ -152,17 +172,15 @@ function encodeArtifact(artifact: SessionFormatArtifact): EncodedSessionFormatAr
|
||||
...(header.origin === undefined ? {} : { origin: header.origin }),
|
||||
delegationDepth: header.delegationDepth,
|
||||
...(header.agentPreset === undefined ? {} : { agentPreset: header.agentPreset }),
|
||||
}, 'released v2 encoded header') as SessionFormatJsonObject
|
||||
const rows = Object.freeze(artifact.events.map(event => encodeProvenance(event)))
|
||||
return Object.freeze({ header: physicalHeader, rows })
|
||||
}
|
||||
}
|
||||
|
||||
function encodeProvenance(event: SessionFormatEvent): SessionFormatJsonObject {
|
||||
if (event.sourceEventSeqs === undefined) return event
|
||||
return snapshotSessionFormatJson({
|
||||
return {
|
||||
...event,
|
||||
sourceEventSeqs: encodeSeqRanges(event.sourceEventSeqs as readonly number[]),
|
||||
}, `released v2 event ${event.seq} provenance`) as SessionFormatJsonObject
|
||||
}
|
||||
}
|
||||
|
||||
function decodeSeqRanges(value: SessionFormatJsonValue, maxEntries: number): readonly number[] {
|
||||
|
||||
@@ -4,4 +4,4 @@ export { releasedV1SessionFormatCodec } from '@deepseek-ai/dsh-session-format-v0
|
||||
export * from './codec.ts'
|
||||
export * from './dispositions.ts'
|
||||
export * from './migration.ts'
|
||||
export * from './validation.ts'
|
||||
export { assertReleasedV2Header, restoreReleasedV2Artifact } from './validation.ts'
|
||||
@@ -1,33 +1,41 @@
|
||||
import { AssistantStreamAccumulator } from '@deepseek-ai/dsh-llm'
|
||||
import type { AssistantStreamRecord } from '@deepseek-ai/dsh-llm'
|
||||
import {
|
||||
SessionFormatUnsupportedMigrationError,
|
||||
defineSessionFormatMigration,
|
||||
snapshotSessionFormatArtifact,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatEvent,
|
||||
SessionFormatEventRun,
|
||||
SessionFormatJsonObject,
|
||||
SessionFormatJsonValue,
|
||||
SessionFormatHeader,
|
||||
SessionFormatMigrationContext,
|
||||
SessionFormatMigrationStage,
|
||||
SessionFormatMigrationStageInput,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
RELEASED_V0_EVENT_DISPOSITIONS,
|
||||
assertReleasedV1Artifact,
|
||||
assertReleasedEventPayload,
|
||||
assertReleasedV1Header,
|
||||
isReleasedAssistantChunkRun,
|
||||
} from '@deepseek-ai/dsh-session-format-v0-to-v1'
|
||||
import { assertReleasedV2Artifact, assertReleasedV2Header } from './validation.ts'
|
||||
import { assertReleasedV2Header } from './validation.ts'
|
||||
|
||||
const CHUNK_EVENT_REQUIRED = ['type', 'seq', 'time', 'data'] as const
|
||||
const CHUNK_EVENT_OPTIONAL = ['ignorable', 'sourceEventSeqs', 'surfaceOp'] as const
|
||||
const CHUNK_EVENT_KEYS: ReadonlySet<string> = new Set([...CHUNK_EVENT_REQUIRED, ...CHUNK_EVENT_OPTIONAL])
|
||||
|
||||
interface AttemptGroup {
|
||||
readonly turn: number
|
||||
readonly step: number
|
||||
readonly chunks: SessionFormatEvent[]
|
||||
readonly spans: Array<{ firstSeq: number; eventCount: number }>
|
||||
readonly stream: Array<{ record: AssistantStreamRecord; lastTime: number }>
|
||||
accumulator?: AssistantStreamAccumulator
|
||||
chunkCount: number
|
||||
lastChunkSeq?: number
|
||||
lastChunkTime?: number
|
||||
terminal: boolean
|
||||
messageSeq?: number
|
||||
}
|
||||
|
||||
interface StagedEvent {
|
||||
readonly origin: number
|
||||
readonly event: SessionFormatEvent
|
||||
}
|
||||
|
||||
/** Adjacent migration that embeds released-v1 top-level Assistant chunks into v2 attempt events. */
|
||||
@@ -39,164 +47,501 @@ export const sessionFormatV1ToV2 = defineSessionFormatMigration({
|
||||
assertReleasedV1Header(header)
|
||||
return { ...header, version: 2 }
|
||||
},
|
||||
migrate(source) {
|
||||
assertReleasedV1Artifact(source)
|
||||
const unknown = source.events.find(event => RELEASED_V0_EVENT_DISPOSITIONS[event.type] === undefined)
|
||||
if (unknown !== undefined) {
|
||||
throw refusal(`format v1 contains unknown event type ${JSON.stringify(unknown.type)} at seq ${unknown.seq}`)
|
||||
}
|
||||
const groups = collectAttemptGroups(source.events)
|
||||
const groupByChunk = new Map<number, AttemptGroup>()
|
||||
const groupByMessage = new Map<number, AttemptGroup>()
|
||||
for (const group of groups) {
|
||||
for (const chunk of group.chunks) groupByChunk.set(chunk.seq, group)
|
||||
if (group.messageSeq !== undefined) groupByMessage.set(group.messageSeq, group)
|
||||
}
|
||||
|
||||
const staged: StagedEvent[] = []
|
||||
const oldToNew = new Map<number, number>()
|
||||
for (const sourceEvent of source.events) {
|
||||
const group = groupByChunk.get(sourceEvent.seq)
|
||||
if (group !== undefined) {
|
||||
if (group.messageSeq === undefined && sourceEvent.seq === group.chunks.at(-1)?.seq) {
|
||||
stage(staged, oldToNew, sourceEvent.seq, attemptEvent(group))
|
||||
}
|
||||
continue
|
||||
}
|
||||
const messageGroup = groupByMessage.get(sourceEvent.seq)
|
||||
if (messageGroup !== undefined) {
|
||||
stage(staged, oldToNew, sourceEvent.seq, messageEvent(sourceEvent, messageGroup))
|
||||
continue
|
||||
}
|
||||
const event = source.header.isSeeded
|
||||
&& sourceEvent.seq === source.inheritedEventCount
|
||||
&& sourceEvent.type === 'session/end-seed'
|
||||
? { ...sourceEvent, data: { inherited: true } }
|
||||
: sourceEvent
|
||||
stage(staged, oldToNew, sourceEvent.seq, event)
|
||||
}
|
||||
|
||||
const inheritedEventCount = remapInheritedCut(source, groups, staged)
|
||||
if (source.header.isSeeded && source.events[source.inheritedEventCount]?.type !== 'session/end-seed') {
|
||||
const next = source.events[source.inheritedEventCount]
|
||||
const previous = source.events[source.inheritedEventCount - 1]
|
||||
staged.splice(inheritedEventCount, 0, {
|
||||
origin: -1,
|
||||
event: {
|
||||
type: 'session/end-seed',
|
||||
seq: inheritedEventCount,
|
||||
time: next?.time ?? previous?.time ?? source.header.createdAt,
|
||||
data: { inherited: true },
|
||||
},
|
||||
})
|
||||
oldToNew.clear()
|
||||
for (const [seq, candidate] of staged.entries()) {
|
||||
if (candidate.origin >= 0) oldToNew.set(candidate.origin, seq)
|
||||
}
|
||||
}
|
||||
for (const group of groups) {
|
||||
for (const chunk of group.chunks) oldToNew.delete(chunk.seq)
|
||||
}
|
||||
|
||||
const target = snapshotSessionFormatArtifact({
|
||||
header: { ...source.header, version: 2 },
|
||||
inheritedEventCount,
|
||||
events: staged.map(({ event }, seq) => remapReferences(event, seq, oldToNew)),
|
||||
}, 'released v1-to-v2 target')
|
||||
assertReleasedV2Artifact(target)
|
||||
return target
|
||||
createStage(input) {
|
||||
return input.sourceKind === 'decoded'
|
||||
? new DecodedReleasedV1ToV2Stage(input)
|
||||
: new TransformedReleasedV1ToV2Stage(input)
|
||||
},
|
||||
validateTarget: assertReleasedV2Artifact,
|
||||
validateTargetHeader: assertReleasedV2Header,
|
||||
})
|
||||
|
||||
function collectAttemptGroups(events: readonly SessionFormatEvent[]): readonly AttemptGroup[] {
|
||||
const groups: AttemptGroup[] = []
|
||||
const current = new Map<string, AttemptGroup>()
|
||||
for (const event of events) {
|
||||
if (event.type === 'assistant/chunk') {
|
||||
const data = record(event.data)
|
||||
const turn = coordinate(data['turn'])
|
||||
const step = coordinate(data['step'])
|
||||
const key = `${turn}:${step}`
|
||||
let group = current.get(key)
|
||||
if (group === undefined || group.terminal) {
|
||||
group = { turn, step, chunks: [], terminal: false }
|
||||
groups.push(group)
|
||||
current.set(key, group)
|
||||
}
|
||||
group.chunks.push(event)
|
||||
const chunk = record(data['chunk'])
|
||||
if (chunk['type'] === 'finish') group.terminal = true
|
||||
continue
|
||||
class TransformedReleasedV1ToV2Stage implements SessionFormatMigrationStage {
|
||||
private readonly state: ReleasedV1ToV2State
|
||||
|
||||
constructor(input: SessionFormatMigrationStageInput) {
|
||||
assertReleasedV1Header(input.sourceHeader)
|
||||
this.state = {
|
||||
sourceHeader: input.sourceHeader,
|
||||
sourceCut: input.sourceInheritedEventCount,
|
||||
mapping: new Map(),
|
||||
legacyTurns: legacyTurnState(),
|
||||
pending: undefined,
|
||||
targetSeq: 0,
|
||||
targetCut: input.sourceHeader.isSeeded ? undefined : 0,
|
||||
lastTime: input.sourceHeader.createdAt,
|
||||
}
|
||||
if (event.type !== 'assistant/message') {
|
||||
closeAttemptAtBoundary(event, current)
|
||||
continue
|
||||
}
|
||||
const data = record(event.data)
|
||||
const turn = coordinate(data['turn'])
|
||||
const step = coordinate(data['step'])
|
||||
const sources = event.sourceEventSeqs
|
||||
if (!Array.isArray(sources)) {
|
||||
const unclaimed = groups.some(candidate => candidate.messageSeq === undefined
|
||||
&& candidate.turn === turn
|
||||
&& candidate.step === step)
|
||||
if (unclaimed) throw refusal(`assistant/message ${event.seq} does not cite its complete v1 chunk attempt`)
|
||||
groups.push({ turn, step, chunks: [], terminal: true, messageSeq: event.seq })
|
||||
continue
|
||||
}
|
||||
if (sources.length === 0) {
|
||||
// Released v1 uses an explicit empty list to state that this message
|
||||
// owns no preceding chunks; an absent list cannot make that claim.
|
||||
groups.push({ turn, step, chunks: [], terminal: true, messageSeq: event.seq })
|
||||
continue
|
||||
}
|
||||
const group = groups.find(candidate => candidate.messageSeq === undefined
|
||||
&& candidate.turn === turn
|
||||
&& candidate.step === step
|
||||
&& sameNumbers(candidate.chunks.map(chunk => chunk.seq), sources))
|
||||
if (group === undefined) {
|
||||
throw refusal(`assistant/message ${event.seq} chunk provenance is not one complete ordered attempt`)
|
||||
}
|
||||
group.messageSeq = event.seq
|
||||
group.terminal = true
|
||||
}
|
||||
return groups
|
||||
|
||||
transformEvent(
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
transformReleasedEvent(this.state, event, context)
|
||||
}
|
||||
|
||||
transformRun(
|
||||
run: SessionFormatEventRun,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
transformReleasedRun(this.state, run, context)
|
||||
}
|
||||
|
||||
finish(context: SessionFormatMigrationContext): number {
|
||||
return finishMigration(this.state, context)
|
||||
}
|
||||
}
|
||||
|
||||
function closeAttemptAtBoundary(
|
||||
event: SessionFormatEvent,
|
||||
current: ReadonlyMap<string, AttemptGroup>,
|
||||
): void {
|
||||
if (event.type === 'turn/end') {
|
||||
const data = record(event.data)
|
||||
const turn = coordinate(data['turn'])
|
||||
for (const group of current.values()) {
|
||||
if (group.turn === turn) group.terminal = true
|
||||
class DecodedReleasedV1ToV2Stage extends TransformedReleasedV1ToV2Stage {
|
||||
override transformEvent(
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
if (event.type !== 'assistant/chunk'
|
||||
&& RELEASED_V0_EVENT_DISPOSITIONS[event.type] !== undefined) {
|
||||
assertReleasedEventPayload(event, 1)
|
||||
}
|
||||
super.transformEvent(event, context)
|
||||
}
|
||||
}
|
||||
|
||||
interface StreamingAttempt {
|
||||
readonly group: AttemptGroup
|
||||
readonly afterLastChunk: SessionFormatEvent[]
|
||||
}
|
||||
|
||||
interface ReleasedV1ToV2State {
|
||||
readonly sourceHeader: SessionFormatHeader
|
||||
readonly sourceCut: number
|
||||
readonly mapping: Map<number, number>
|
||||
readonly legacyTurns: LegacyTurnState
|
||||
pending: StreamingAttempt | undefined
|
||||
targetSeq: number
|
||||
targetCut: number | undefined
|
||||
lastTime: number
|
||||
}
|
||||
|
||||
function transformReleasedEvent(
|
||||
state: ReleasedV1ToV2State,
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
if (event.type === 'assistant/chunk') assertChunkEnvelope(event)
|
||||
if (RELEASED_V0_EVENT_DISPOSITIONS[event.type] === undefined) {
|
||||
throw refusal(`format v1 contains unknown event type ${JSON.stringify(event.type)} at seq ${event.seq}`)
|
||||
}
|
||||
const interrupted = legacyInterruptedTurn(state.legacyTurns, event)
|
||||
if (event.type === 'turn/start' && state.legacyTurns.openTurn !== null && interrupted === undefined) {
|
||||
throw refusal(`turn/start ${JSON.stringify(record(event.data)['turn'])} does not close the prior turn`)
|
||||
}
|
||||
assertSourceDeliveryMarker(state, event)
|
||||
observeLegacyTurn(state.legacyTurns, event)
|
||||
state.lastTime = event.time
|
||||
if (interrupted !== undefined) {
|
||||
finishAttempt(state, context)
|
||||
emitGenerated(state, event.seq, interrupted, context)
|
||||
}
|
||||
const legacyGoal = splitLegacyGoalChange(event)
|
||||
if (legacyGoal !== undefined) {
|
||||
emitGenerated(state, event.seq, legacyGoal.change, context)
|
||||
emitSource(state, legacyGoal.message, context)
|
||||
return
|
||||
}
|
||||
if (event.type !== 'step/end'
|
||||
&& event.type !== 'llm/retry'
|
||||
&& event.type !== 'llm/retry-started') return
|
||||
if (event.type === 'assistant/chunk') {
|
||||
transformChunk(state, event, context)
|
||||
return
|
||||
}
|
||||
if (event.type === 'assistant/message') {
|
||||
transformMessage(state, event, context)
|
||||
return
|
||||
}
|
||||
if (closesAttempt(event)) {
|
||||
finishAttempt(state, context)
|
||||
emitSource(state, event, context)
|
||||
return
|
||||
}
|
||||
if (state.pending !== undefined) {
|
||||
state.pending.afterLastChunk.push(event)
|
||||
return
|
||||
}
|
||||
emitSource(state, event, context)
|
||||
}
|
||||
|
||||
function assertChunkEnvelope(event: SessionFormatEvent): void {
|
||||
const unexpected = Object.keys(event).find(key => !CHUNK_EVENT_KEYS.has(key))
|
||||
if (unexpected !== undefined) throw refusal(`assistant/chunk ${event.seq} has unexpected member ${unexpected}`)
|
||||
const missing = CHUNK_EVENT_REQUIRED.find(key => !Object.hasOwn(event, key))
|
||||
if (missing !== undefined) throw refusal(`assistant/chunk ${event.seq} lacks required member ${missing}`)
|
||||
if (event.ignorable !== undefined && event.ignorable !== true) {
|
||||
throw refusal(`assistant/chunk ${event.seq} ignorable must be true when present`)
|
||||
}
|
||||
}
|
||||
|
||||
function assertSourceDeliveryMarker(state: ReleasedV1ToV2State, event: SessionFormatEvent): void {
|
||||
if (event.type !== 'session-log-deepseek/delivery-accepted') return
|
||||
const data = record(event.data)
|
||||
const inherited = state.sourceHeader.parentSession !== undefined && event.seq < state.sourceCut
|
||||
if (data['sessionFormatVersion'] === 1 && !inherited && data['sessionId'] !== state.sourceHeader.id) {
|
||||
throw refusal('current-generation delivery marker names the wrong Session')
|
||||
}
|
||||
}
|
||||
|
||||
function transformReleasedRun(
|
||||
state: ReleasedV1ToV2State,
|
||||
run: SessionFormatEventRun,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
if (!isReleasedAssistantChunkRun(run)) {
|
||||
for (const event of run.expand()) transformReleasedEvent(state, event, context)
|
||||
return
|
||||
}
|
||||
state.legacyTurns.previous = undefined
|
||||
state.lastTime = run.lastTime
|
||||
if (state.pending !== undefined
|
||||
&& (state.pending.group.terminal
|
||||
|| state.pending.group.turn !== run.turn
|
||||
|| state.pending.group.step !== run.step)) {
|
||||
finishAttempt(state, context)
|
||||
} else if (state.pending !== undefined) {
|
||||
flushBuffered(state, state.pending, context)
|
||||
}
|
||||
state.pending ??= { group: attemptGroup(run.turn, run.step), afterLastChunk: [] }
|
||||
assertAttemptRange(state, run.firstSeq, run.lastSeq)
|
||||
flushAccumulator(state.pending.group)
|
||||
appendStreamRecord(
|
||||
state.pending.group,
|
||||
run.stream as unknown as AssistantStreamRecord,
|
||||
run.lastTime,
|
||||
)
|
||||
recordChunkSpan(state.pending.group, run.firstSeq, run.eventCount, run.lastTime)
|
||||
}
|
||||
|
||||
function finishMigration(
|
||||
state: ReleasedV1ToV2State,
|
||||
context: SessionFormatMigrationContext,
|
||||
): number {
|
||||
finishAttempt(state, context)
|
||||
if (state.sourceHeader.isSeeded && state.targetCut === undefined) {
|
||||
state.targetCut = state.targetSeq
|
||||
context.emitEvent({
|
||||
type: 'session/end-seed',
|
||||
seq: state.targetSeq,
|
||||
time: state.lastTime,
|
||||
data: { inherited: true },
|
||||
})
|
||||
state.targetSeq += 1
|
||||
}
|
||||
return state.targetCut as number
|
||||
}
|
||||
|
||||
function transformChunk(
|
||||
state: ReleasedV1ToV2State,
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
const data = record(event.data)
|
||||
const turn = coordinate(data['turn'])
|
||||
const step = coordinate(data['step'])
|
||||
const group = current.get(`${turn}:${step}`)
|
||||
if (group !== undefined) group.terminal = true
|
||||
const chunk = record(data['chunk'])
|
||||
if (state.pending !== undefined
|
||||
&& (state.pending.group.terminal
|
||||
|| state.pending.group.turn !== turn
|
||||
|| state.pending.group.step !== step)) {
|
||||
finishAttempt(state, context)
|
||||
} else if (state.pending !== undefined) {
|
||||
flushBuffered(state, state.pending, context)
|
||||
}
|
||||
state.pending ??= { group: attemptGroup(turn, step), afterLastChunk: [] }
|
||||
assertAttemptCut(state, state.pending.group, event.seq)
|
||||
state.pending.group.accumulator ??= new AssistantStreamAccumulator()
|
||||
state.pending.group.accumulator.push({
|
||||
time: event.time,
|
||||
chunk: data['chunk'] as Parameters<AssistantStreamAccumulator['push']>[0]['chunk'],
|
||||
})
|
||||
recordChunkSpan(state.pending.group, event.seq, 1, event.time)
|
||||
if (chunk['type'] === 'finish') state.pending.group.terminal = true
|
||||
}
|
||||
|
||||
function transformMessage(
|
||||
state: ReleasedV1ToV2State,
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
const data = record(event.data)
|
||||
const turn = coordinate(data['turn'])
|
||||
const step = coordinate(data['step'])
|
||||
const sources = event.sourceEventSeqs
|
||||
const pending = state.pending
|
||||
if (pending !== undefined && (pending.group.turn !== turn || pending.group.step !== step)) {
|
||||
finishAttempt(state, context)
|
||||
emitSource(state, messageEvent(event, attemptGroup(turn, step)), context)
|
||||
return
|
||||
}
|
||||
if (!Array.isArray(sources)) {
|
||||
if (pending !== undefined) {
|
||||
throw refusal(`assistant/message ${event.seq} does not cite its complete v1 chunk attempt`)
|
||||
}
|
||||
emitSource(state, messageEvent(event, attemptGroup(turn, step)), context)
|
||||
return
|
||||
}
|
||||
if (sources.length === 0) {
|
||||
finishAttempt(state, context)
|
||||
emitSource(state, messageEvent(event, attemptGroup(turn, step)), context)
|
||||
return
|
||||
}
|
||||
if (pending === undefined
|
||||
|| !matchesChunkSources(pending.group, sources)) {
|
||||
throw refusal(`assistant/message ${event.seq} chunk provenance is not one complete ordered attempt`)
|
||||
}
|
||||
assertAttemptCut(state, pending.group, event.seq)
|
||||
pending.group.terminal = true
|
||||
flushBuffered(state, pending, context)
|
||||
emitSource(state, messageEvent(event, pending.group), context)
|
||||
state.pending = undefined
|
||||
}
|
||||
|
||||
function finishAttempt(
|
||||
state: ReleasedV1ToV2State,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
const pending = state.pending
|
||||
if (pending === undefined) return
|
||||
emitGenerated(state, pending.group.lastChunkSeq as number, attemptEvent(pending.group), context)
|
||||
flushBuffered(state, pending, context)
|
||||
state.pending = undefined
|
||||
}
|
||||
|
||||
function flushBuffered(
|
||||
state: ReleasedV1ToV2State,
|
||||
pending: StreamingAttempt,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
for (const event of pending.afterLastChunk) emitSource(state, event, context)
|
||||
pending.afterLastChunk.length = 0
|
||||
}
|
||||
|
||||
function emitSource(
|
||||
state: ReleasedV1ToV2State,
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
let source = event
|
||||
if (state.sourceHeader.isSeeded
|
||||
&& event.seq === state.sourceCut
|
||||
&& event.type === 'session/end-seed') {
|
||||
source = { ...event, data: { inherited: true } }
|
||||
}
|
||||
ensureTargetCut(state, event.seq, event.time, source.type, context)
|
||||
state.mapping.set(event.seq, state.targetSeq)
|
||||
context.emitEvent(remapReferences(source, state.targetSeq, state.mapping))
|
||||
state.targetSeq += 1
|
||||
}
|
||||
|
||||
function emitGenerated(
|
||||
state: ReleasedV1ToV2State,
|
||||
origin: number,
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
ensureTargetCut(state, origin, event.time, event.type, context)
|
||||
context.emitEvent(remapReferences(event, state.targetSeq, state.mapping))
|
||||
state.targetSeq += 1
|
||||
}
|
||||
|
||||
function ensureTargetCut(
|
||||
state: ReleasedV1ToV2State,
|
||||
origin: number,
|
||||
time: number,
|
||||
type: string,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void {
|
||||
if (!state.sourceHeader.isSeeded || state.targetCut !== undefined || origin < state.sourceCut) return
|
||||
state.targetCut = state.targetSeq
|
||||
if (origin === state.sourceCut && type === 'session/end-seed') return
|
||||
context.emitEvent({
|
||||
type: 'session/end-seed',
|
||||
seq: state.targetSeq,
|
||||
time,
|
||||
data: { inherited: true },
|
||||
})
|
||||
state.targetSeq += 1
|
||||
}
|
||||
|
||||
function assertAttemptCut(state: ReleasedV1ToV2State, group: AttemptGroup, member: number): void {
|
||||
const first = group.spans[0]?.firstSeq ?? member
|
||||
if ((first < state.sourceCut) !== (member < state.sourceCut)) {
|
||||
throw refusal(`inherited Session cut ${state.sourceCut} splits one Assistant attempt`)
|
||||
}
|
||||
}
|
||||
|
||||
function assertAttemptRange(state: ReleasedV1ToV2State, first: number, last: number): void {
|
||||
if ((first < state.sourceCut) !== (last < state.sourceCut)) {
|
||||
throw refusal(`inherited Session cut ${state.sourceCut} splits one Assistant attempt`)
|
||||
}
|
||||
}
|
||||
|
||||
interface LegacyTurnState {
|
||||
openTurn: number | null
|
||||
openStep: number | null
|
||||
previous: SessionFormatEvent | undefined
|
||||
}
|
||||
|
||||
function legacyTurnState(): LegacyTurnState {
|
||||
return { openTurn: null, openStep: null, previous: undefined }
|
||||
}
|
||||
|
||||
function legacyInterruptedTurn(
|
||||
state: LegacyTurnState,
|
||||
event: SessionFormatEvent,
|
||||
): SessionFormatEvent | undefined {
|
||||
if (event.type !== 'turn/start' || state.openTurn === null || state.openStep !== null
|
||||
|| coordinate(record(event.data)['turn']) !== state.openTurn + 1
|
||||
|| state.previous?.type !== 'agent/inbox/spliced') return undefined
|
||||
const splice = record(state.previous.data)
|
||||
if (splice['target'] !== 'next-turn' || !Array.isArray(splice['inserted']) || splice['inserted'].length === 0) {
|
||||
return undefined
|
||||
}
|
||||
return {
|
||||
type: 'turn/end',
|
||||
seq: event.seq,
|
||||
time: event.time,
|
||||
data: { turn: state.openTurn, reason: { kind: 'interrupted' } },
|
||||
}
|
||||
}
|
||||
|
||||
function observeLegacyTurn(state: LegacyTurnState, event: SessionFormatEvent): void {
|
||||
const data = record(event.data)
|
||||
if (event.type === 'turn/start') {
|
||||
state.openTurn = coordinate(data['turn'])
|
||||
state.openStep = null
|
||||
} else if (event.type === 'turn/end') {
|
||||
state.openTurn = null
|
||||
state.openStep = null
|
||||
} else if (event.type === 'step/start') {
|
||||
state.openStep = coordinate(data['step'])
|
||||
} else if (event.type === 'step/end') {
|
||||
state.openStep = null
|
||||
}
|
||||
state.previous = event
|
||||
}
|
||||
|
||||
function splitLegacyGoalChange(event: SessionFormatEvent): {
|
||||
readonly change: SessionFormatEvent
|
||||
readonly message: SessionFormatEvent
|
||||
} | undefined {
|
||||
if (event.type !== 'user/message') return undefined
|
||||
const data = record(event.data)
|
||||
const source = record(data['source'])
|
||||
if (source['kind'] !== 'goal' || source['change'] === undefined) return undefined
|
||||
return {
|
||||
change: {
|
||||
type: 'goal/change',
|
||||
seq: event.seq,
|
||||
time: event.time,
|
||||
data: source['change'],
|
||||
},
|
||||
message: {
|
||||
...event,
|
||||
data: { ...data, source: { kind: 'plugin', plugin: 'goal' } },
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function closesAttempt(event: SessionFormatEvent): boolean {
|
||||
return event.type === 'turn/end'
|
||||
|| event.type === 'step/end'
|
||||
|| event.type === 'llm/retry'
|
||||
|| event.type === 'llm/retry-started'
|
||||
}
|
||||
|
||||
function attemptGroup(turn: number, step: number): AttemptGroup {
|
||||
return { turn, step, spans: [], stream: [], chunkCount: 0, terminal: false }
|
||||
}
|
||||
|
||||
function recordChunkSpan(group: AttemptGroup, firstSeq: number, eventCount: number, lastTime: number): void {
|
||||
const previous = group.spans.at(-1)
|
||||
if (previous !== undefined && previous.firstSeq + previous.eventCount === firstSeq) {
|
||||
previous.eventCount += eventCount
|
||||
} else {
|
||||
group.spans.push({ firstSeq, eventCount })
|
||||
}
|
||||
group.chunkCount += eventCount
|
||||
group.lastChunkSeq = firstSeq + eventCount - 1
|
||||
group.lastChunkTime = lastTime
|
||||
}
|
||||
|
||||
function matchesChunkSources(group: AttemptGroup, sources: readonly unknown[]): boolean {
|
||||
if (sources.length !== group.chunkCount) return false
|
||||
let index = 0
|
||||
for (const span of group.spans) {
|
||||
for (let offset = 0; offset < span.eventCount; offset += 1) {
|
||||
if (sources[index] !== span.firstSeq + offset) return false
|
||||
index += 1
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
function recordLastTime(record: AssistantStreamRecord): number {
|
||||
if (record.type === 'chunk') return record.time
|
||||
return record.dt.reduce((time, gap) => time + gap, record.time0)
|
||||
}
|
||||
|
||||
function mutableRecord(record: AssistantStreamRecord): AssistantStreamRecord {
|
||||
if (record.type === 'chunk') return record
|
||||
if (record.type === 'tool-call-chunks') {
|
||||
return { ...record, dt: [...record.dt], args: [...record.args] }
|
||||
}
|
||||
return { ...record, dt: [...record.dt], texts: [...record.texts] }
|
||||
}
|
||||
|
||||
function appendStreamRecord(
|
||||
group: AttemptGroup,
|
||||
source: AssistantStreamRecord,
|
||||
lastTime: number,
|
||||
owned = true,
|
||||
): void {
|
||||
const previous = group.stream.at(-1)
|
||||
if (previous === undefined || source.type === 'chunk' || previous.record.type !== source.type) {
|
||||
group.stream.push({ record: owned ? source : mutableRecord(source), lastTime })
|
||||
return
|
||||
}
|
||||
const gap = source.time0 - previous.lastTime
|
||||
if (previous.record.index !== source.index || !Number.isSafeInteger(gap)) {
|
||||
group.stream.push({ record: owned ? source : mutableRecord(source), lastTime })
|
||||
return
|
||||
}
|
||||
if (source.type === 'tool-call-chunks') {
|
||||
const target = previous.record as Extract<AssistantStreamRecord, { readonly type: 'tool-call-chunks' }>
|
||||
if (target.id !== source.id || target.name !== source.name) {
|
||||
group.stream.push({ record: owned ? source : mutableRecord(source), lastTime })
|
||||
return
|
||||
}
|
||||
;(target.dt as number[]).push(gap)
|
||||
for (const value of source.dt) (target.dt as number[]).push(value)
|
||||
for (const value of source.args) (target.args as string[]).push(value)
|
||||
} else {
|
||||
const target = previous.record as Extract<AssistantStreamRecord, {
|
||||
readonly type: 'text-chunks' | 'reasoning-chunks'
|
||||
}>
|
||||
;(target.dt as number[]).push(gap)
|
||||
for (const value of source.dt) (target.dt as number[]).push(value)
|
||||
for (const value of source.texts) (target.texts as string[]).push(value)
|
||||
}
|
||||
previous.lastTime = lastTime
|
||||
}
|
||||
|
||||
function flushAccumulator(group: AttemptGroup): void {
|
||||
const accumulator = group.accumulator
|
||||
if (accumulator === undefined) return
|
||||
for (const record of accumulator.snapshot()) {
|
||||
appendStreamRecord(group, record, recordLastTime(record), false)
|
||||
}
|
||||
delete group.accumulator
|
||||
}
|
||||
|
||||
function streamOf(group: AttemptGroup) {
|
||||
const accumulator = new AssistantStreamAccumulator()
|
||||
for (const event of group.chunks) {
|
||||
const data = record(event.data)
|
||||
accumulator.push({
|
||||
time: event.time,
|
||||
chunk: data['chunk'] as Parameters<AssistantStreamAccumulator['push']>[0]['chunk'],
|
||||
})
|
||||
}
|
||||
return accumulator.snapshot() as unknown as SessionFormatJsonValue
|
||||
flushAccumulator(group)
|
||||
return group.stream.map(({ record }) => record) as unknown as SessionFormatJsonValue
|
||||
}
|
||||
|
||||
function messageEvent(source: SessionFormatEvent, group: AttemptGroup): SessionFormatEvent {
|
||||
@@ -209,42 +554,14 @@ function messageEvent(source: SessionFormatEvent, group: AttemptGroup): SessionF
|
||||
}
|
||||
|
||||
function attemptEvent(group: AttemptGroup): SessionFormatEvent {
|
||||
const last = group.chunks.at(-1) as SessionFormatEvent
|
||||
return {
|
||||
type: 'assistant/attempt',
|
||||
seq: last.seq,
|
||||
time: last.time,
|
||||
seq: group.lastChunkSeq as number,
|
||||
time: group.lastChunkTime as number,
|
||||
data: { turn: group.turn, step: group.step, stream: streamOf(group) },
|
||||
}
|
||||
}
|
||||
|
||||
function stage(
|
||||
staged: StagedEvent[],
|
||||
oldToNew: Map<number, number>,
|
||||
origin: number,
|
||||
event: SessionFormatEvent,
|
||||
): void {
|
||||
oldToNew.set(origin, staged.length)
|
||||
staged.push({ origin, event })
|
||||
}
|
||||
|
||||
function remapInheritedCut(
|
||||
source: SessionFormatArtifact,
|
||||
groups: readonly AttemptGroup[],
|
||||
staged: readonly StagedEvent[],
|
||||
): number {
|
||||
const cut = source.inheritedEventCount
|
||||
for (const group of groups) {
|
||||
const members = group.messageSeq === undefined
|
||||
? group.chunks.map(chunk => chunk.seq)
|
||||
: [...group.chunks.map(chunk => chunk.seq), group.messageSeq]
|
||||
const before = members.some(seq => seq < cut)
|
||||
const after = members.some(seq => seq >= cut)
|
||||
if (before && after) throw refusal(`inherited Session cut ${cut} splits one Assistant attempt`)
|
||||
}
|
||||
return staged.filter(candidate => candidate.origin < cut).length
|
||||
}
|
||||
|
||||
function remapReferences(
|
||||
source: SessionFormatEvent,
|
||||
targetSeq: number,
|
||||
@@ -360,7 +677,6 @@ function mapOne(
|
||||
return mapped
|
||||
}
|
||||
|
||||
/* assertReleasedV1Artifact validates these payload coordinates before migration. */
|
||||
function record(value: SessionFormatJsonValue | undefined): SessionFormatJsonObject {
|
||||
return value as SessionFormatJsonObject
|
||||
}
|
||||
@@ -373,10 +689,6 @@ function coordinate(value: SessionFormatJsonValue | undefined): number {
|
||||
return value as number
|
||||
}
|
||||
|
||||
function sameNumbers(left: readonly number[], right: readonly number[]): boolean {
|
||||
return left.length === right.length && left.every((value, index) => value === right[index])
|
||||
}
|
||||
|
||||
function refusal(message: string): SessionFormatUnsupportedMigrationError {
|
||||
return new SessionFormatUnsupportedMigrationError(message)
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
import { BlockAssembler, expandAssistantStream } from '@deepseek-ai/dsh-llm'
|
||||
import { deepEqualJson } from '@deepseek-ai/dsh-util-values'
|
||||
import {
|
||||
SessionFormatError,
|
||||
SessionFormatUnsupportedMigrationError,
|
||||
sessionFormatCount,
|
||||
snapshotSessionFormatJson,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatEvent,
|
||||
SessionFormatJsonValue,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
assertReleasedPayloadSemantics,
|
||||
assertReleasedSurfaceMetadata,
|
||||
} from '@deepseek-ai/dsh-session-format-v0-to-v1'
|
||||
import { RELEASED_V2_EVENT_DISPOSITIONS, RELEASED_V2_EVENT_TYPES } from '../dispositions.ts'
|
||||
import {
|
||||
assertReleasedV2Keys,
|
||||
assertReleasedV2PhysicalArtifact,
|
||||
releasedV2Record,
|
||||
restoreReleasedV2Artifact,
|
||||
} from '../validation.ts'
|
||||
|
||||
const RELEASED_V2_EVENT_TYPE_SET = new Set(RELEASED_V2_EVENT_TYPES)
|
||||
const SURFACE_TYPES = new Set(['user/message', 'assistant/message', 'tool/result'])
|
||||
|
||||
/**
|
||||
* Validate the frozen released-v2 writer image in tests.
|
||||
* @param artifact - released-v2 logical artifact.
|
||||
*/
|
||||
export function assertReleasedV2Artifact(artifact: SessionFormatArtifact): void {
|
||||
assertReleasedV2PhysicalArtifact(artifact)
|
||||
for (const [index, event] of artifact.events.entries()) {
|
||||
const disposition = RELEASED_V2_EVENT_DISPOSITIONS[event.type]
|
||||
if (disposition === undefined) {
|
||||
throw new SessionFormatUnsupportedMigrationError(
|
||||
`format v2 contains unknown event type ${JSON.stringify(event.type)} at seq ${index}`,
|
||||
)
|
||||
}
|
||||
if (SURFACE_TYPES.has(event.type)) {
|
||||
assertReleasedSurfaceMetadata(event, index, event.type, 'forbid-assistant')
|
||||
}
|
||||
assertPayload(event, disposition)
|
||||
}
|
||||
restoreReleasedV2Artifact(artifact, RELEASED_V2_EVENT_TYPE_SET)
|
||||
}
|
||||
|
||||
export { assertReleasedV2PhysicalArtifact } from '../validation.ts'
|
||||
|
||||
function assertPayload(
|
||||
event: SessionFormatEvent,
|
||||
disposition: (typeof RELEASED_V2_EVENT_DISPOSITIONS)[string],
|
||||
): void {
|
||||
const data = releasedV2Record(event.data, `${event.type} ${event.seq} data`)
|
||||
assertReleasedV2Keys(data, disposition.required, disposition.optional, `${event.type} ${event.seq} data`)
|
||||
for (const key of disposition.opaque) {
|
||||
if (Object.hasOwn(data, key)) snapshotSessionFormatJson(data[key], `${event.type} ${event.seq} opaque ${key}`)
|
||||
}
|
||||
if (event.type === 'assistant/attempt' || event.type === 'assistant/message') {
|
||||
const turn = sessionFormatCount(data['turn'], `${event.type} ${event.seq} turn`)
|
||||
const step = sessionFormatCount(data['step'], `${event.type} ${event.seq} step`)
|
||||
const assembler = new BlockAssembler()
|
||||
let timed: ReturnType<typeof expandAssistantStream>
|
||||
try {
|
||||
timed = expandAssistantStream(data['stream'] as never)
|
||||
for (const member of timed) {
|
||||
assertReleasedPayloadSemantics({
|
||||
type: 'assistant/chunk',
|
||||
seq: event.seq,
|
||||
time: member.time,
|
||||
data: { turn, step, chunk: member.chunk } as unknown as SessionFormatJsonValue,
|
||||
}, 2)
|
||||
assembler.push(member.chunk)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
throw new SessionFormatError(`${event.type} ${event.seq} has an invalid embedded stream`, { cause: error })
|
||||
}
|
||||
if (event.type === 'assistant/attempt') return
|
||||
assertReleasedPayloadSemantics(event, 2)
|
||||
if (timed.length > 0) {
|
||||
const message = releasedV2Record(data['message'], `assistant/message ${event.seq} message`)
|
||||
const content = data['interrupted'] === true ? assembler.interruptedBlocks() : assembler.blocks()
|
||||
if (!deepEqualJson(message['content'], content)) {
|
||||
throw new SessionFormatError(`assistant/message ${event.seq} message content disagrees with its embedded stream`)
|
||||
}
|
||||
if (!deepEqualJson(data['usage'], assembler.usage)) {
|
||||
throw new SessionFormatError(`assistant/message ${event.seq} usage disagrees with its embedded stream`)
|
||||
}
|
||||
const source = releasedV2Record(message['source'], `assistant/message ${event.seq} source`)
|
||||
if (!deepEqualJson(source['replayState'], assembler.replayState)) {
|
||||
throw new SessionFormatError(`assistant/message ${event.seq} replay state disagrees with its embedded stream`)
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
if (event.type === 'session/end-seed') {
|
||||
if (data['inherited'] !== undefined && data['inherited'] !== true) {
|
||||
throw new SessionFormatError(`session/end-seed ${event.seq} inherited must be true when present`)
|
||||
}
|
||||
return
|
||||
}
|
||||
assertReleasedPayloadSemantics(event, 2)
|
||||
}
|
||||
@@ -1,26 +1,18 @@
|
||||
import { isAbsolute } from 'node:path'
|
||||
import { BlockAssembler, expandAssistantStream } from '@deepseek-ai/dsh-llm'
|
||||
import { deepEqualJson } from '@deepseek-ai/dsh-util-values'
|
||||
import {
|
||||
SessionFormatError,
|
||||
SessionFormatUnsupportedMigrationError,
|
||||
sessionFormatCount,
|
||||
sessionFormatSafeInteger,
|
||||
snapshotSessionFormatJson,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatEvent,
|
||||
SessionFormatHeader,
|
||||
SessionFormatJsonObject,
|
||||
SessionFormatJsonValue,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
assertReleasedArtifactRelationships,
|
||||
assertReleasedPayloadSemantics,
|
||||
assertReleasedSurfaceMetadata,
|
||||
} from '@deepseek-ai/dsh-session-format-v0-to-v1'
|
||||
import { RELEASED_V2_EVENT_DISPOSITIONS, RELEASED_V2_EVENT_TYPES } from './dispositions.ts'
|
||||
import { assertReleasedArtifactRelationships } from '@deepseek-ai/dsh-session-format-v0-to-v1'
|
||||
import { RELEASED_V2_EVENT_DISPOSITIONS } from './dispositions.ts'
|
||||
|
||||
const HEADER_REQUIRED = ['version', 'id', 'createdAt', 'isSeeded', 'delegationDepth'] as const
|
||||
const HEADER_OPTIONAL = ['cwd', 'parentSession', 'origin', 'agentPreset'] as const
|
||||
@@ -28,7 +20,6 @@ const EVENT_REQUIRED = ['type', 'seq', 'time', 'data'] as const
|
||||
const SURFACE_TYPES = new Set(['user/message', 'assistant/message', 'tool/result'])
|
||||
const SURFACE_OPTIONAL = ['ignorable', 'sourceEventSeqs', 'surfaceOp'] as const
|
||||
const LOG_OPTIONAL = ['ignorable'] as const
|
||||
const RELEASED_V2_EVENT_TYPE_SET = new Set(RELEASED_V2_EVENT_TYPES)
|
||||
const RELEASED_V2_RELATIONSHIP_EXTENSIONS = {
|
||||
stepEvents: new Set(['assistant/attempt']),
|
||||
preservedSourceTitleRequestText: true,
|
||||
@@ -40,8 +31,8 @@ const RELEASED_V2_RELATIONSHIP_EXTENSIONS = {
|
||||
* @throws {SessionFormatError} when the header is not an exact released-v2 value.
|
||||
*/
|
||||
export function assertReleasedV2Header(header: SessionFormatHeader): void {
|
||||
const record = jsonRecord(header, 'format v2 header')
|
||||
exactKeys(record, HEADER_REQUIRED, HEADER_OPTIONAL, 'format v2 header')
|
||||
const record = releasedV2Record(header, 'format v2 header')
|
||||
assertReleasedV2Keys(record, HEADER_REQUIRED, HEADER_OPTIONAL, 'format v2 header')
|
||||
if (record['version'] !== 2) throw new SessionFormatError('expected format v2 header')
|
||||
if (typeof record['id'] !== 'string') throw new SessionFormatError('format v2 header id must be a string')
|
||||
sessionFormatCount(record['createdAt'], 'format v2 header createdAt')
|
||||
@@ -60,28 +51,9 @@ export function assertReleasedV2Header(header: SessionFormatHeader): void {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the exact logical image emitted by the released v2 writer.
|
||||
* @param artifact - complete decoded released-v2 Session artifact.
|
||||
* @throws {SessionFormatError} when an envelope, payload, relationship, or inherited cut is invalid.
|
||||
* @throws {SessionFormatUnsupportedMigrationError} when the artifact contains an unknown event type.
|
||||
*/
|
||||
export function assertReleasedV2Artifact(artifact: SessionFormatArtifact): void {
|
||||
validateReleasedV2Artifact(artifact, 'target', RELEASED_V2_EVENT_TYPE_SET)
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate only the released-v2 physical header, event envelopes, and inherited cut.
|
||||
* Event vocabulary and payload semantics belong to target or installed-current restoration.
|
||||
* @param artifact - complete physical-codec output.
|
||||
*/
|
||||
export function assertReleasedV2PhysicalArtifact(artifact: SessionFormatArtifact): void {
|
||||
validateReleasedV2Artifact(artifact, 'physical')
|
||||
}
|
||||
|
||||
function validateReleasedV2Artifact(
|
||||
artifact: SessionFormatArtifact,
|
||||
mode: 'target' | 'current' | 'physical',
|
||||
mode: 'current' | 'physical',
|
||||
knownEventTypes?: ReadonlySet<string>,
|
||||
): void {
|
||||
assertReleasedV2Header(artifact.header)
|
||||
@@ -90,15 +62,13 @@ function validateReleasedV2Artifact(
|
||||
if (!artifact.header.isSeeded && cut !== 0) throw new SessionFormatError('unseeded format v2 Session has inherited events')
|
||||
let lastInheritedMarker: number | undefined
|
||||
for (const [index, event] of artifact.events.entries()) {
|
||||
const record = jsonRecord(event, `format v2 event ${index}`)
|
||||
const record = releasedV2Record(event, `format v2 event ${index}`)
|
||||
const type = record['type']
|
||||
if (typeof type !== 'string') throw new SessionFormatError(`format v2 event ${index} type must be a string`)
|
||||
const disposition = RELEASED_V2_EVENT_DISPOSITIONS[type]
|
||||
const installed = knownEventTypes?.has(type) === true
|
||||
const ignorableUnknown = disposition === undefined
|
||||
&& mode === 'current'
|
||||
&& record['ignorable'] === true
|
||||
if (mode !== 'physical' && disposition === undefined && !installed && !ignorableUnknown) {
|
||||
const ignorableUnknown = disposition === undefined && record['ignorable'] === true
|
||||
if (mode === 'current' && disposition === undefined && !installed && !ignorableUnknown) {
|
||||
throw new SessionFormatUnsupportedMigrationError(
|
||||
`format v2 contains unknown event type ${JSON.stringify(type)} at seq ${index}`,
|
||||
)
|
||||
@@ -107,16 +77,14 @@ function validateReleasedV2Artifact(
|
||||
const optional = mode === 'physical' || disposition === undefined
|
||||
? SURFACE_OPTIONAL
|
||||
: surface ? SURFACE_OPTIONAL : LOG_OPTIONAL
|
||||
exactKeys(record, EVENT_REQUIRED, optional, `format v2 event ${index}`)
|
||||
assertReleasedV2Keys(record, EVENT_REQUIRED, optional, `format v2 event ${index}`)
|
||||
if (record['seq'] !== index) throw new SessionFormatError(`format v2 event ${index} is not dense`)
|
||||
sessionFormatSafeInteger(record['time'], `format v2 event ${index} time`)
|
||||
if (record['ignorable'] !== undefined && record['ignorable'] !== true) {
|
||||
throw new SessionFormatError(`format v2 event ${index} ignorable must be true when present`)
|
||||
}
|
||||
if (mode === 'target' && surface) assertReleasedSurfaceMetadata(record, index, type, 'forbid-assistant')
|
||||
if (mode === 'target' && disposition !== undefined) assertPayload(event, disposition)
|
||||
if (type === 'session/end-seed') {
|
||||
const data = jsonRecord(event.data, `session/end-seed ${index} data`)
|
||||
const data = releasedV2Record(event.data, `session/end-seed ${index} data`)
|
||||
if (data['inherited'] === true) lastInheritedMarker = index
|
||||
}
|
||||
}
|
||||
@@ -126,74 +94,35 @@ function validateReleasedV2Artifact(
|
||||
if (!artifact.header.isSeeded && lastInheritedMarker !== undefined) {
|
||||
throw new SessionFormatError('format v2 unseeded Session contains an inherited end-seed marker')
|
||||
}
|
||||
if (mode === 'target') {
|
||||
if (mode === 'current') {
|
||||
assertReleasedArtifactRelationships(artifact, RELEASED_V2_RELATIONSHIP_EXTENSIONS)
|
||||
}
|
||||
}
|
||||
|
||||
function assertPayload(
|
||||
event: SessionFormatEvent,
|
||||
disposition: (typeof RELEASED_V2_EVENT_DISPOSITIONS)[string],
|
||||
): void {
|
||||
const data = jsonRecord(event.data, `${event.type} ${event.seq} data`)
|
||||
exactKeys(data, disposition.required, disposition.optional, `${event.type} ${event.seq} data`)
|
||||
for (const key of disposition.opaque) {
|
||||
if (Object.hasOwn(data, key)) snapshotSessionFormatJson(data[key], `${event.type} ${event.seq} opaque ${key}`)
|
||||
}
|
||||
if (event.type === 'assistant/attempt' || event.type === 'assistant/message') {
|
||||
const turn = sessionFormatCount(data['turn'], `${event.type} ${event.seq} turn`)
|
||||
const step = sessionFormatCount(data['step'], `${event.type} ${event.seq} step`)
|
||||
const assembler = new BlockAssembler()
|
||||
let timed: ReturnType<typeof expandAssistantStream>
|
||||
try {
|
||||
timed = expandAssistantStream(data['stream'] as never)
|
||||
for (const member of timed) {
|
||||
assertReleasedPayloadSemantics({
|
||||
type: 'assistant/chunk',
|
||||
seq: event.seq,
|
||||
time: member.time,
|
||||
data: { turn, step, chunk: member.chunk } as unknown as SessionFormatJsonValue,
|
||||
}, 2)
|
||||
assembler.push(member.chunk)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
throw new SessionFormatError(`${event.type} ${event.seq} has an invalid embedded stream`, { cause: error })
|
||||
}
|
||||
if (event.type === 'assistant/attempt') return
|
||||
assertReleasedPayloadSemantics(event, 2)
|
||||
if (timed.length > 0) {
|
||||
const message = jsonRecord(data['message'], `assistant/message ${event.seq} message`)
|
||||
const content = data['interrupted'] === true ? assembler.interruptedBlocks() : assembler.blocks()
|
||||
if (!deepEqualJson(message['content'], content)) {
|
||||
throw new SessionFormatError(`assistant/message ${event.seq} message content disagrees with its embedded stream`)
|
||||
}
|
||||
if (!deepEqualJson(data['usage'], assembler.usage)) {
|
||||
throw new SessionFormatError(`assistant/message ${event.seq} usage disagrees with its embedded stream`)
|
||||
}
|
||||
const source = jsonRecord(message['source'], `assistant/message ${event.seq} source`)
|
||||
if (!deepEqualJson(source['replayState'], assembler.replayState)) {
|
||||
throw new SessionFormatError(`assistant/message ${event.seq} replay state disagrees with its embedded stream`)
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
if (event.type === 'session/end-seed') {
|
||||
if (data['inherited'] !== undefined && data['inherited'] !== true) {
|
||||
throw new SessionFormatError(`session/end-seed ${event.seq} inherited must be true when present`)
|
||||
}
|
||||
return
|
||||
}
|
||||
assertReleasedPayloadSemantics(event, 2)
|
||||
}
|
||||
|
||||
function jsonRecord(value: SessionFormatJsonValue | undefined, label: string): SessionFormatJsonObject {
|
||||
/**
|
||||
* Require one released-v2 value to be a JSON object.
|
||||
* @param value - value to narrow.
|
||||
* @param label - diagnostic subject.
|
||||
* @returns the narrowed object.
|
||||
*/
|
||||
export function releasedV2Record(
|
||||
value: SessionFormatJsonValue | undefined,
|
||||
label: string,
|
||||
): SessionFormatJsonObject {
|
||||
if (typeof value !== 'object' || value === null || Array.isArray(value)) {
|
||||
throw new SessionFormatError(`${label} must be an object`)
|
||||
}
|
||||
return value as SessionFormatJsonObject
|
||||
}
|
||||
|
||||
function exactKeys(
|
||||
/**
|
||||
* Require one released-v2 object to contain exactly the admitted keys.
|
||||
* @param value - object to inspect.
|
||||
* @param required - keys that must be present.
|
||||
* @param optional - additional keys that may be present.
|
||||
* @param label - diagnostic subject.
|
||||
*/
|
||||
export function assertReleasedV2Keys(
|
||||
value: SessionFormatJsonObject,
|
||||
required: readonly string[],
|
||||
optional: readonly string[],
|
||||
@@ -219,3 +148,11 @@ export function restoreReleasedV2Artifact(
|
||||
validateReleasedV2Artifact(artifact, 'current', knownEventTypes)
|
||||
return artifact
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the released-v2 physical envelope without interpreting event vocabulary.
|
||||
* @param artifact - released-v2 physical artifact.
|
||||
*/
|
||||
export function assertReleasedV2PhysicalArtifact(artifact: SessionFormatArtifact): void {
|
||||
validateReleasedV2Artifact(artifact, 'physical')
|
||||
}
|
||||
@@ -1,8 +1,11 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SessionFormatEventCollector } from '@deepseek-ai/dsh-session-format'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatArtifactDecoder,
|
||||
SessionFormatEvent,
|
||||
SessionFormatJsonObject,
|
||||
SessionFormatRecovery,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import { decodeSeqRanges as decodeCurrentSeqRanges } from '@deepseek-ai/dsh-session'
|
||||
import { releasedV2SessionFormatCodec } from '@deepseek-ai/dsh-session-format-v1-to-v2'
|
||||
@@ -21,6 +24,10 @@ const fullPhysicalHeader = {
|
||||
|
||||
const textBlock = { type: 'text', text: 'text' } as const
|
||||
|
||||
function throwUnknown(value: unknown): never {
|
||||
throw value
|
||||
}
|
||||
|
||||
function feedback(seq: number): SessionFormatEvent {
|
||||
return { type: 'feedback/record', seq, time: seq + 1, data: { text: `feedback-${seq}` } }
|
||||
}
|
||||
@@ -50,6 +57,27 @@ function artifact(
|
||||
}
|
||||
}
|
||||
|
||||
function decodeV2(
|
||||
header: unknown,
|
||||
rows: readonly unknown[],
|
||||
recovery: SessionFormatRecovery = 'strict',
|
||||
): SessionFormatArtifact {
|
||||
const decoder: SessionFormatArtifactDecoder = releasedV2SessionFormatCodec.createDecoder(header, recovery)
|
||||
const context = new SessionFormatEventCollector()
|
||||
for (const row of rows) decoder.decodeRow(row, context)
|
||||
return { header: decoder.header, inheritedEventCount: decoder.finish(context), events: context.values }
|
||||
}
|
||||
|
||||
function encodeV2(source: SessionFormatArtifact): {
|
||||
readonly header: SessionFormatJsonObject
|
||||
readonly rows: readonly SessionFormatJsonObject[]
|
||||
} {
|
||||
return {
|
||||
header: releasedV2SessionFormatCodec.encodeHeader(source.header, source.inheritedEventCount),
|
||||
rows: source.events.map(event => releasedV2SessionFormatCodec.encodeEvent(event)),
|
||||
}
|
||||
}
|
||||
|
||||
describe('releasedV2SessionFormatCodec headers', () => {
|
||||
it('round-trips the minimal and complete optional header images', () => {
|
||||
expect(releasedV2SessionFormatCodec.version).toBe(2)
|
||||
@@ -68,7 +96,7 @@ describe('releasedV2SessionFormatCodec headers', () => {
|
||||
agentPreset: 'default',
|
||||
})
|
||||
|
||||
const encodedMinimal = releasedV2SessionFormatCodec.encodeArtifact(artifact([]))
|
||||
const encodedMinimal = encodeV2(artifact([]))
|
||||
expect(encodedMinimal).toStrictEqual({ header: minimalPhysicalHeader, rows: [] })
|
||||
const complete = artifact([], {
|
||||
header: {
|
||||
@@ -83,8 +111,12 @@ describe('releasedV2SessionFormatCodec headers', () => {
|
||||
agentPreset: 'default',
|
||||
},
|
||||
})
|
||||
expect(releasedV2SessionFormatCodec.encodeArtifact(complete).header)
|
||||
expect(encodeV2(complete).header)
|
||||
.toStrictEqual(fullPhysicalHeader)
|
||||
expect(() => releasedV2SessionFormatCodec.encodeHeader(
|
||||
artifact([]).header,
|
||||
1,
|
||||
)).toThrow(/unseeded.*inherited events/)
|
||||
})
|
||||
|
||||
it.each([
|
||||
@@ -117,14 +149,14 @@ describe('releasedV2SessionFormatCodec rows', () => {
|
||||
userMessage(2, [0, 1]),
|
||||
userMessage(3, [0, 1, 2]),
|
||||
])
|
||||
const encoded = releasedV2SessionFormatCodec.encodeArtifact(source)
|
||||
const encoded = encodeV2(source)
|
||||
expect(encoded.rows.map(row => row['sourceEventSeqs'])).toStrictEqual([
|
||||
undefined,
|
||||
[0],
|
||||
[0, 1],
|
||||
[[0, 2]],
|
||||
])
|
||||
expect(releasedV2SessionFormatCodec.decodeArtifact(encoded.header, encoded.rows)).toStrictEqual(source)
|
||||
expect(decodeV2(encoded.header, encoded.rows)).toStrictEqual(source)
|
||||
})
|
||||
|
||||
it('keeps non-monotonic provenance scalar-only for the current backend reader', () => {
|
||||
@@ -134,12 +166,12 @@ describe('releasedV2SessionFormatCodec rows', () => {
|
||||
userMessage(6, sourceEventSeqs),
|
||||
])
|
||||
|
||||
const encoded = releasedV2SessionFormatCodec.encodeArtifact(source)
|
||||
const encoded = encodeV2(source)
|
||||
const stored = encoded.rows[6]?.['sourceEventSeqs']
|
||||
|
||||
expect(stored).toStrictEqual(sourceEventSeqs)
|
||||
expect(decodeCurrentSeqRanges(stored, 6)).toStrictEqual(sourceEventSeqs)
|
||||
expect(releasedV2SessionFormatCodec.decodeArtifact(encoded.header, encoded.rows)).toStrictEqual(source)
|
||||
expect(decodeV2(encoded.header, encoded.rows)).toStrictEqual(source)
|
||||
})
|
||||
|
||||
it('keeps the v2 physical codec vocabulary-neutral for current growth and a future source freeze', () => {
|
||||
@@ -149,9 +181,9 @@ describe('releasedV2SessionFormatCodec rows', () => {
|
||||
{ type: 'turn/start', seq: 2, time: 3, data: { turn: 1, postReleaseMember: true } },
|
||||
])
|
||||
|
||||
const encoded = releasedV2SessionFormatCodec.encodeArtifact(source)
|
||||
const encoded = encodeV2(source)
|
||||
|
||||
expect(releasedV2SessionFormatCodec.decodeArtifact(encoded.header, encoded.rows)).toStrictEqual(source)
|
||||
expect(decodeV2(encoded.header, encoded.rows)).toStrictEqual(source)
|
||||
})
|
||||
|
||||
it('expands mixed stored ranges and preserves non-provenance rows', () => {
|
||||
@@ -159,10 +191,10 @@ describe('releasedV2SessionFormatCodec rows', () => {
|
||||
...userMessage(5),
|
||||
sourceEventSeqs: [[0, 2], 4],
|
||||
}]
|
||||
const decoded = releasedV2SessionFormatCodec.decodeArtifact(minimalPhysicalHeader, rows)
|
||||
const decoded = decodeV2(minimalPhysicalHeader, rows)
|
||||
expect(decoded.events[0]).toStrictEqual(feedback(0))
|
||||
expect(decoded.events[5]?.sourceEventSeqs).toStrictEqual([0, 1, 2, 4])
|
||||
const descending = releasedV2SessionFormatCodec.decodeArtifact(minimalPhysicalHeader, [
|
||||
const descending = decodeV2(minimalPhysicalHeader, [
|
||||
feedback(0), feedback(1), { ...userMessage(2), sourceEventSeqs: [1, 0] },
|
||||
])
|
||||
expect(descending.events[2]?.sourceEventSeqs).toStrictEqual([1, 0])
|
||||
@@ -185,42 +217,61 @@ describe('releasedV2SessionFormatCodec rows', () => {
|
||||
const rows = [feedback(0), feedback(1), feedback(2), feedback(3), {
|
||||
...userMessage(4), sourceEventSeqs,
|
||||
}]
|
||||
expect(() => releasedV2SessionFormatCodec.decodeArtifact(minimalPhysicalHeader, rows)).toThrow(message)
|
||||
expect(() => decodeV2(minimalPhysicalHeader, rows)).toThrow(message)
|
||||
})
|
||||
|
||||
it('contains ordinary and non-SessionFormatError row failures in a recoverable tail', () => {
|
||||
const explosive = new Proxy({}, { ownKeys: () => { throw new Error('proxy failure') } })
|
||||
expect(releasedV2SessionFormatCodec.decodeRecoverableArtifact(
|
||||
const explosive = new Proxy(feedback(1), { ownKeys: () => { throw new Error('proxy failure') } })
|
||||
const primitiveFailure = new Proxy(feedback(1), {
|
||||
ownKeys: () => throwUnknown('primitive proxy failure'),
|
||||
})
|
||||
expect(decodeV2(
|
||||
minimalPhysicalHeader,
|
||||
[feedback(0), explosive],
|
||||
'recoverable',
|
||||
).events).toStrictEqual([feedback(0)])
|
||||
expect(releasedV2SessionFormatCodec.decodeRecoverableArtifact(
|
||||
expect(decodeV2(
|
||||
minimalPhysicalHeader,
|
||||
[feedback(0), null],
|
||||
'recoverable',
|
||||
).events).toStrictEqual([feedback(0)])
|
||||
expect(releasedV2SessionFormatCodec.decodeRecoverableArtifact(
|
||||
expect(decodeV2(
|
||||
minimalPhysicalHeader,
|
||||
[feedback(0), primitiveFailure],
|
||||
'recoverable',
|
||||
).events).toStrictEqual([feedback(0)])
|
||||
expect(decodeV2(
|
||||
minimalPhysicalHeader,
|
||||
[null, [], feedback(0)],
|
||||
'recoverable',
|
||||
).events).toStrictEqual([])
|
||||
})
|
||||
|
||||
it('refuses malformed strict rows, strict gaps, and terminal recoverable tails', () => {
|
||||
expect(() => releasedV2SessionFormatCodec.decodeArtifact(minimalPhysicalHeader, [null]))
|
||||
expect(() => decodeV2(minimalPhysicalHeader, [null]))
|
||||
.toThrow(/row 0/)
|
||||
expect(() => releasedV2SessionFormatCodec.decodeArtifact(minimalPhysicalHeader, [feedback(1)]))
|
||||
expect(() => decodeV2(minimalPhysicalHeader, [feedback(1)]))
|
||||
.toThrow(/seq gap/)
|
||||
expect(() => releasedV2SessionFormatCodec.decodeRecoverableArtifact(minimalPhysicalHeader, [{
|
||||
expect(() => decodeV2(minimalPhysicalHeader, [{
|
||||
type: 'turn/end', seq: 1, time: 1, data: { turn: 1, reason: { kind: 'completed' } },
|
||||
}])).toThrow(/seq gap/)
|
||||
expect(() => releasedV2SessionFormatCodec.decodeRecoverableArtifact(minimalPhysicalHeader, [null, {
|
||||
}], 'recoverable')).toThrow(/seq gap/)
|
||||
expect(() => decodeV2(minimalPhysicalHeader, [null, {
|
||||
type: 'turn/end', seq: 0, time: 1, data: { turn: 1, reason: { kind: 'completed' } },
|
||||
}])).toThrow(/row 0/)
|
||||
}], 'recoverable')).toThrow(/row 0/)
|
||||
expect(() => decodeV2(minimalPhysicalHeader, [{ ...feedback(0), extra: true }]))
|
||||
.toThrow(/unexpected field extra/)
|
||||
expect(() => decodeV2(minimalPhysicalHeader, [{ type: 'feedback/record', seq: 0, time: 1 }]))
|
||||
.toThrow(/lacks required field data/)
|
||||
expect(() => decodeV2(minimalPhysicalHeader, [{ ...feedback(0), type: 1 }]))
|
||||
.toThrow(/type must be a string/)
|
||||
expect(() => decodeV2(minimalPhysicalHeader, [{ ...feedback(0), ignorable: false }]))
|
||||
.toThrow(/ignorable must be true/)
|
||||
})
|
||||
|
||||
it('keeps the prefix before a recoverable non-terminal seq gap', () => {
|
||||
expect(releasedV2SessionFormatCodec.decodeRecoverableArtifact(minimalPhysicalHeader, [
|
||||
expect(decodeV2(minimalPhysicalHeader, [
|
||||
feedback(0), feedback(3), feedback(1),
|
||||
]).events).toStrictEqual([feedback(0)])
|
||||
], 'recoverable').events).toStrictEqual([feedback(0)])
|
||||
})
|
||||
|
||||
it('derives the last tagged seed marker and rejects lineage disagreements', () => {
|
||||
@@ -230,13 +281,13 @@ describe('releasedV2SessionFormatCodec rows', () => {
|
||||
{ type: 'session/end-seed', seq: 1, time: 2, data: {} },
|
||||
{ type: 'session/end-seed', seq: 2, time: 3, data: { inherited: true } },
|
||||
]
|
||||
expect(releasedV2SessionFormatCodec.decodeArtifact(seededHeader, markers).inheritedEventCount).toBe(2)
|
||||
expect(releasedV2SessionFormatCodec.decodeArtifact(seededHeader, [markers[0]]).inheritedEventCount).toBe(0)
|
||||
expect(() => releasedV2SessionFormatCodec.decodeArtifact(seededHeader, []))
|
||||
expect(decodeV2(seededHeader, markers).inheritedEventCount).toBe(2)
|
||||
expect(decodeV2(seededHeader, [markers[0]]).inheritedEventCount).toBe(0)
|
||||
expect(() => decodeV2(seededHeader, []))
|
||||
.toThrow(/lacks an inherited end-seed marker/)
|
||||
expect(() => releasedV2SessionFormatCodec.decodeArtifact(minimalPhysicalHeader, [markers[0]]))
|
||||
expect(() => decodeV2(minimalPhysicalHeader, [markers[0]]))
|
||||
.toThrow(/unseeded Session contains an inherited end-seed marker/)
|
||||
expect(() => releasedV2SessionFormatCodec.decodeArtifact(seededHeader, [{
|
||||
expect(() => decodeV2(seededHeader, [{
|
||||
type: 'session/end-seed', seq: 0, time: 1, data: null,
|
||||
}])).toThrow(/must be an object/)
|
||||
})
|
||||
|
||||
@@ -1,6 +1,23 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { sessionFormatV1ToV2 } from '@deepseek-ai/dsh-session-format-v1-to-v2'
|
||||
import type { SessionFormatArtifact, SessionFormatEvent } from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
assertReleasedV2Header,
|
||||
releasedV2SessionFormatCodec,
|
||||
sessionFormatV1ToV2,
|
||||
} from '@deepseek-ai/dsh-session-format-v1-to-v2'
|
||||
import { assertReleasedV2Artifact } from '../src/testing/validation.ts'
|
||||
import {
|
||||
releasedV0SessionFormatCodec,
|
||||
releasedV1SessionFormatCodec,
|
||||
sessionFormatV0ToV1,
|
||||
} from '@deepseek-ai/dsh-session-format-v0-to-v1'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatEvent,
|
||||
SessionFormatEventRun,
|
||||
SessionFormatHeader,
|
||||
SessionFormatJsonObject,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import { createSessionFormatCatalog, SessionFormatEventCollector } from '@deepseek-ai/dsh-session-format'
|
||||
|
||||
const message = {
|
||||
id: 'assistant-1',
|
||||
@@ -20,6 +37,94 @@ function event(type: string, seq: number, time: number, data: SessionFormatEvent
|
||||
return { type, seq, time, data }
|
||||
}
|
||||
|
||||
const catalog = createSessionFormatCatalog({
|
||||
currentVersion: 2,
|
||||
codecs: [releasedV0SessionFormatCodec, releasedV1SessionFormatCodec, releasedV2SessionFormatCodec],
|
||||
currentEncoder: releasedV2SessionFormatCodec,
|
||||
migrations: [sessionFormatV0ToV1, sessionFormatV1ToV2],
|
||||
restoreCurrent(artifact) {
|
||||
assertReleasedV2Artifact(artifact)
|
||||
return artifact
|
||||
},
|
||||
restoreTransformedCurrent(artifact) {
|
||||
assertReleasedV2Artifact(artifact)
|
||||
return artifact
|
||||
},
|
||||
restoreCurrentHeader(header) {
|
||||
assertReleasedV2Header(header)
|
||||
return header
|
||||
},
|
||||
})
|
||||
|
||||
function physicalV1Header(header: SessionFormatHeader, inheritedEventCount: number) {
|
||||
return {
|
||||
type: 'session',
|
||||
version: 1,
|
||||
id: header.id,
|
||||
createdAt: header.createdAt,
|
||||
...(header.cwd === undefined ? {} : { cwd: header.cwd }),
|
||||
...(header.parentSession === undefined ? {} : { parentSession: header.parentSession }),
|
||||
...(header.isSeeded ? { seedLength: inheritedEventCount } : {}),
|
||||
...(header.origin === undefined ? {} : { origin: header.origin }),
|
||||
delegationDepth: header.delegationDepth,
|
||||
...(header.agentPreset === undefined ? {} : { agentPreset: header.agentPreset }),
|
||||
}
|
||||
}
|
||||
|
||||
function migrateV1ToV2(source: SessionFormatArtifact): SessionFormatArtifact {
|
||||
const restore = catalog.createRestore(
|
||||
physicalV1Header(source.header, source.inheritedEventCount),
|
||||
{ recovery: 'strict', validation: 'current' },
|
||||
)
|
||||
for (const row of source.events) restore.decodeRow(row)
|
||||
return restore.finish()
|
||||
}
|
||||
|
||||
function stageHarness(options: {
|
||||
readonly id?: string
|
||||
readonly sourceCut?: number
|
||||
readonly seeded?: boolean
|
||||
readonly sourceKind?: 'decoded' | 'transformed'
|
||||
} = {}) {
|
||||
const sourceHeader: SessionFormatHeader = {
|
||||
version: 1,
|
||||
id: options.id ?? 'stage',
|
||||
createdAt: 1,
|
||||
...(options.seeded === true ? { parentSession: 'parent' } : {}),
|
||||
isSeeded: options.seeded === true,
|
||||
delegationDepth: 0,
|
||||
}
|
||||
const stage = sessionFormatV1ToV2.createStage({
|
||||
sourceHeader,
|
||||
targetHeader: sessionFormatV1ToV2.migrateHeader(sourceHeader),
|
||||
sourceInheritedEventCount: options.sourceCut ?? 0,
|
||||
sourceKind: options.sourceKind ?? 'decoded',
|
||||
})
|
||||
return { stage, output: new SessionFormatEventCollector() }
|
||||
}
|
||||
|
||||
function packedRun(options: {
|
||||
readonly firstSeq: number
|
||||
readonly eventCount?: number
|
||||
readonly turn?: number
|
||||
readonly step?: number
|
||||
readonly lastTime: number
|
||||
readonly stream: SessionFormatJsonObject
|
||||
}): SessionFormatEventRun {
|
||||
const eventCount = options.eventCount ?? 1
|
||||
return {
|
||||
runType: 'released-assistant-chunks',
|
||||
firstSeq: options.firstSeq,
|
||||
eventCount,
|
||||
turn: options.turn ?? 1,
|
||||
step: options.step ?? 1,
|
||||
lastSeq: options.firstSeq + eventCount - 1,
|
||||
lastTime: options.lastTime,
|
||||
stream: options.stream,
|
||||
*expand() {},
|
||||
} as SessionFormatEventRun
|
||||
}
|
||||
|
||||
describe('sessionFormatV1ToV2', () => {
|
||||
it('migrates one exact released-v1 header without reading events', () => {
|
||||
const header = {
|
||||
@@ -33,6 +138,346 @@ describe('sessionFormatV1ToV2', () => {
|
||||
expect(() => sessionFormatV1ToV2.migrateHeader({ ...header, version: 0 })).toThrow(/v1 header/)
|
||||
})
|
||||
|
||||
it('validates a directly decoded v1 source before transforming it', () => {
|
||||
const header = {
|
||||
version: 1, id: 'source-validation', createdAt: 1, isSeeded: false, delegationDepth: 0,
|
||||
}
|
||||
expect(() => migrateV1ToV2({
|
||||
header,
|
||||
inheritedEventCount: 0,
|
||||
events: [event('external/post-v1', 0, 1, null)],
|
||||
})).toThrow(/unknown event type/)
|
||||
expect(() => migrateV1ToV2({
|
||||
header,
|
||||
inheritedEventCount: 0,
|
||||
events: [{
|
||||
type: 'turn/start', seq: 0, time: 1, data: { turn: 1, postReleaseMember: true },
|
||||
}],
|
||||
})).toThrow(/unexpected member/)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['unexpected member', { ...event('assistant/chunk', 0, 1, {}), extra: true }, /unexpected member extra/],
|
||||
['missing member', { type: 'assistant/chunk', seq: 0, time: 1 }, /lacks required member data/],
|
||||
['invalid ignorable marker', { ...event('assistant/chunk', 0, 1, {}), ignorable: false }, /ignorable must be true/],
|
||||
])('refuses an Assistant chunk envelope with an %s', (_name, candidate, expected) => {
|
||||
const { stage, output } = stageHarness({ sourceKind: 'transformed' })
|
||||
expect(() => { stage.transformEvent(candidate as SessionFormatEvent, output) }).toThrow(expected)
|
||||
})
|
||||
|
||||
it('checks own-generation delivery markers but accepts inherited markers', () => {
|
||||
const marker = (sessionId: string): SessionFormatEvent => event(
|
||||
'session-log-deepseek/delivery-accepted',
|
||||
1,
|
||||
2,
|
||||
{ sessionId, throughSeq: 0, sessionFormatVersion: 1 },
|
||||
)
|
||||
const own = stageHarness({ id: 'child' })
|
||||
expect(() => { own.stage.transformEvent(marker('other'), own.output) }).toThrow(/wrong Session/)
|
||||
|
||||
const inherited = stageHarness({ id: 'child', seeded: true, sourceCut: 2 })
|
||||
inherited.stage.transformEvent(marker('parent'), inherited.output)
|
||||
expect(inherited.output.values).toEqual([{ ...marker('parent'), seq: 0 }])
|
||||
})
|
||||
|
||||
it('expands generic runs and refuses a packed run split by the inherited cut', () => {
|
||||
const generic = stageHarness({ sourceKind: 'transformed' })
|
||||
const retained = event('feedback/record', 0, 1, { text: 'retained' })
|
||||
generic.stage.transformRun({
|
||||
runType: 'test-run', firstSeq: 0, eventCount: 1, *expand() { yield retained },
|
||||
}, generic.output)
|
||||
expect(generic.output.values).toEqual([retained])
|
||||
|
||||
const split = stageHarness({ seeded: true, sourceCut: 1 })
|
||||
expect(() => { split.stage.transformRun(packedRun({
|
||||
firstSeq: 0,
|
||||
eventCount: 2,
|
||||
lastTime: 2,
|
||||
stream: { type: 'text-chunks', time0: 1, index: 0, dt: [1], texts: ['a', 'b'] },
|
||||
}), split.output) }).toThrow(/splits one Assistant attempt/)
|
||||
})
|
||||
|
||||
it('coalesces adjacent packed text and tool-call runs without expanding them', () => {
|
||||
const text = stageHarness()
|
||||
text.stage.transformRun(packedRun({
|
||||
firstSeq: 0, lastTime: 1,
|
||||
stream: { type: 'text-chunks', time0: 1, index: 0, dt: [], texts: ['a'] },
|
||||
}), text.output)
|
||||
text.stage.transformRun(packedRun({
|
||||
firstSeq: 1, eventCount: 2, lastTime: 4,
|
||||
stream: { type: 'text-chunks', time0: 3, index: 0, dt: [1], texts: ['b', 'c'] },
|
||||
}), text.output)
|
||||
text.stage.finish(text.output)
|
||||
expect(text.output.values[0]?.data).toMatchObject({
|
||||
stream: [{ type: 'text-chunks', time0: 1, index: 0, dt: [2, 1], texts: ['a', 'b', 'c'] }],
|
||||
})
|
||||
|
||||
const tool = stageHarness()
|
||||
tool.stage.transformEvent(event('assistant/chunk', 0, 1, {
|
||||
turn: 1, step: 1,
|
||||
chunk: { type: 'tool-call-delta', index: 0, id: 'call', name: 'read', argumentsDelta: '{' },
|
||||
}), tool.output)
|
||||
tool.stage.transformRun(packedRun({
|
||||
firstSeq: 1, eventCount: 2, lastTime: 3,
|
||||
stream: {
|
||||
type: 'tool-call-chunks', time0: 2, index: 0,
|
||||
id: 'call', name: 'read', dt: [1], args: ['}', '!'],
|
||||
},
|
||||
}), tool.output)
|
||||
tool.stage.finish(tool.output)
|
||||
expect(tool.output.values[0]?.data).toMatchObject({
|
||||
stream: [{
|
||||
type: 'tool-call-chunks', time0: 1, index: 0,
|
||||
id: 'call', name: 'read', dt: [1, 1], args: ['{', '}', '!'],
|
||||
}],
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps incompatible packed records and attempt groups separate', () => {
|
||||
const mismatched = [
|
||||
{ index: 1, id: 'call', name: 'read', time0: 2 },
|
||||
{ index: 0, id: 'other', name: 'read', time0: 2 },
|
||||
{ index: 0, id: 'call', name: 'write', time0: 2 },
|
||||
{ index: 0, id: 'call', name: 'read', time0: Number.MAX_SAFE_INTEGER },
|
||||
] as const
|
||||
for (const [index, next] of mismatched.entries()) {
|
||||
const current = stageHarness()
|
||||
current.stage.transformRun(packedRun({
|
||||
firstSeq: 0,
|
||||
lastTime: index === mismatched.length - 1 ? Number.MIN_SAFE_INTEGER : 1,
|
||||
stream: {
|
||||
type: 'tool-call-chunks', time0: index === mismatched.length - 1 ? Number.MIN_SAFE_INTEGER : 1,
|
||||
index: 0, id: 'call', name: 'read', dt: [], args: ['a'],
|
||||
},
|
||||
}), current.output)
|
||||
current.stage.transformRun(packedRun({
|
||||
firstSeq: 1,
|
||||
lastTime: next.time0,
|
||||
stream: {
|
||||
type: 'tool-call-chunks', time0: next.time0,
|
||||
index: next.index, id: next.id, name: next.name, dt: [], args: ['b'],
|
||||
},
|
||||
}), current.output)
|
||||
current.stage.finish(current.output)
|
||||
expect((current.output.values[0]?.data as { stream: unknown[] }).stream).toHaveLength(2)
|
||||
}
|
||||
|
||||
const groups = stageHarness()
|
||||
groups.stage.transformRun(packedRun({
|
||||
firstSeq: 0, lastTime: 1,
|
||||
stream: { type: 'text-chunks', time0: 1, index: 0, dt: [], texts: ['a'] },
|
||||
}), groups.output)
|
||||
groups.stage.transformRun(packedRun({
|
||||
firstSeq: 1, step: 2, lastTime: 2,
|
||||
stream: { type: 'text-chunks', time0: 2, index: 0, dt: [], texts: ['b'] },
|
||||
}), groups.output)
|
||||
groups.stage.finish(groups.output)
|
||||
expect(groups.output.values).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('copies incompatible accumulator records behind an owned packed prefix', () => {
|
||||
const text = stageHarness()
|
||||
text.stage.transformRun(packedRun({
|
||||
firstSeq: 0, lastTime: 1,
|
||||
stream: { type: 'text-chunks', time0: 1, index: 0, dt: [], texts: ['packed'] },
|
||||
}), text.output)
|
||||
text.stage.transformEvent(event('assistant/chunk', 1, 2, {
|
||||
turn: 1, step: 1, chunk: { type: 'text-delta', index: 1, text: 'a' },
|
||||
}), text.output)
|
||||
text.stage.transformEvent(event('assistant/chunk', 2, 3, {
|
||||
turn: 1, step: 1, chunk: { type: 'text-delta', index: 1, text: 'b' },
|
||||
}), text.output)
|
||||
text.stage.transformRun(packedRun({
|
||||
firstSeq: 3, lastTime: 4,
|
||||
stream: { type: 'reasoning-chunks', time0: 4, index: 0, dt: [], texts: ['flush'] },
|
||||
}), text.output)
|
||||
text.stage.finish(text.output)
|
||||
expect((text.output.values[0]?.data as { stream: unknown[] }).stream).toHaveLength(3)
|
||||
|
||||
const tool = stageHarness()
|
||||
tool.stage.transformRun(packedRun({
|
||||
firstSeq: 0, lastTime: 1,
|
||||
stream: { type: 'tool-call-chunks', time0: 1, index: 0, id: 'call', dt: [], args: ['a'] },
|
||||
}), tool.output)
|
||||
tool.stage.transformEvent(event('assistant/chunk', 1, 2, {
|
||||
turn: 1, step: 1,
|
||||
chunk: { type: 'tool-call-delta', index: 0, id: 'other', argumentsDelta: 'b' },
|
||||
}), tool.output)
|
||||
tool.stage.transformRun(packedRun({
|
||||
firstSeq: 2, lastTime: 3,
|
||||
stream: { type: 'text-chunks', time0: 3, index: 0, dt: [], texts: ['flush'] },
|
||||
}), tool.output)
|
||||
tool.stage.finish(tool.output)
|
||||
expect((tool.output.values[0]?.data as { stream: unknown[] }).stream).toHaveLength(3)
|
||||
})
|
||||
|
||||
it('separates a terminal packed successor and a message from another attempt', () => {
|
||||
const terminal = stageHarness()
|
||||
terminal.stage.transformEvent(event('assistant/chunk', 0, 1, {
|
||||
turn: 1, step: 1, chunk: { type: 'finish', reason: { kind: 'stop' } },
|
||||
}), terminal.output)
|
||||
terminal.stage.transformRun(packedRun({
|
||||
firstSeq: 1, lastTime: 2,
|
||||
stream: { type: 'text-chunks', time0: 2, index: 0, dt: [], texts: ['next'] },
|
||||
}), terminal.output)
|
||||
terminal.stage.finish(terminal.output)
|
||||
expect(terminal.output.values).toHaveLength(2)
|
||||
|
||||
const messageMismatch = stageHarness()
|
||||
messageMismatch.stage.transformEvent(event('assistant/chunk', 0, 1, {
|
||||
turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'partial' },
|
||||
}), messageMismatch.output)
|
||||
messageMismatch.stage.transformEvent({
|
||||
...event('assistant/message', 1, 2, { turn: 2, step: 1, message }),
|
||||
surfaceOp: 'append',
|
||||
}, messageMismatch.output)
|
||||
expect(messageMismatch.output.values).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('settles incremental attempts and rejects ambiguous streamed input', () => {
|
||||
const header = {
|
||||
version: 1, id: 'streaming', createdAt: 1, isSeeded: false, delegationDepth: 0,
|
||||
}
|
||||
const chunk = (seq: number, value: string | undefined, turn = 1): SessionFormatEvent => event(
|
||||
'assistant/chunk',
|
||||
seq,
|
||||
seq + 1,
|
||||
{
|
||||
turn,
|
||||
step: 1,
|
||||
chunk: value === undefined
|
||||
? { type: 'finish', reason: { kind: 'stop' } }
|
||||
: { type: 'text-delta', index: 0, text: value },
|
||||
},
|
||||
)
|
||||
const assistant = (
|
||||
seq: number,
|
||||
turn: number,
|
||||
sourceEventSeqs?: readonly number[],
|
||||
): SessionFormatEvent => ({
|
||||
...event('assistant/message', seq, seq + 1, { turn, step: 1, message }),
|
||||
...(sourceEventSeqs === undefined ? {} : { sourceEventSeqs }),
|
||||
surfaceOp: 'append',
|
||||
})
|
||||
|
||||
const restarted = migrateV1ToV2({
|
||||
header,
|
||||
inheritedEventCount: 0,
|
||||
events: [
|
||||
event('turn/start', 0, 1, { turn: 1 }),
|
||||
event('step/start', 1, 2, { turn: 1, step: 1 }),
|
||||
chunk(2, 'first'),
|
||||
chunk(3, undefined),
|
||||
chunk(4, 'second'),
|
||||
event('step/end', 5, 6, { turn: 1, step: 1 }),
|
||||
event('turn/end', 6, 7, { turn: 1, reason: { kind: 'completed' } }),
|
||||
],
|
||||
})
|
||||
expect(restarted.events.filter(candidate => candidate.type === 'assistant/attempt')).toHaveLength(2)
|
||||
|
||||
expect(() => migrateV1ToV2({
|
||||
header,
|
||||
inheritedEventCount: 0,
|
||||
events: [chunk(0, 'partial'), assistant(1, 1)],
|
||||
})).toThrow(/does not cite/)
|
||||
|
||||
expect(() => migrateV1ToV2({
|
||||
header,
|
||||
inheritedEventCount: 0,
|
||||
events: [chunk(0, 'partial'), assistant(1, 1, [1])],
|
||||
})).toThrow(/complete ordered attempt/)
|
||||
expect(() => migrateV1ToV2({
|
||||
header,
|
||||
inheritedEventCount: 0,
|
||||
events: [event('external/unknown', 0, 1, null)],
|
||||
})).toThrow(/unknown event type/)
|
||||
|
||||
const seededHeader = { ...header, parentSession: 'parent', isSeeded: true }
|
||||
expect(migrateV1ToV2({ header: seededHeader, inheritedEventCount: 0, events: [] }).events).toEqual([{
|
||||
type: 'session/end-seed', seq: 0, time: 1, data: { inherited: true },
|
||||
}])
|
||||
expect(migrateV1ToV2({
|
||||
header: seededHeader,
|
||||
inheritedEventCount: 0,
|
||||
events: [event('turn/start', 0, 2, { turn: 1 })],
|
||||
}).events[0]).toEqual({
|
||||
type: 'session/end-seed', seq: 0, time: 2, data: { inherited: true },
|
||||
})
|
||||
|
||||
expect(() => migrateV1ToV2({
|
||||
header: seededHeader,
|
||||
inheritedEventCount: 1,
|
||||
events: [chunk(0, 'partial'), assistant(1, 1, [0])],
|
||||
})).toThrow(/splits one Assistant attempt/)
|
||||
})
|
||||
|
||||
it('splits a legacy goal mutation from its preserved model-visible message', () => {
|
||||
const change = {
|
||||
kind: 'goal/change', version: 1, operation: 'create',
|
||||
goal: { id: 'goal-1', revision: 1, objective: 'finish', phase: 'active', maxGoalRounds: 2 },
|
||||
roundsStarted: 0, createdAt: 1, updatedAt: 1,
|
||||
}
|
||||
const payload = {
|
||||
goal: change.goal,
|
||||
roundsStarted: change.roundsStarted,
|
||||
createdAt: change.createdAt,
|
||||
updatedAt: change.updatedAt,
|
||||
}
|
||||
const source: SessionFormatArtifact = {
|
||||
header: { version: 1, id: 'legacy-goal', createdAt: 1, isSeeded: false, delegationDepth: 0 },
|
||||
inheritedEventCount: 0,
|
||||
events: [
|
||||
{
|
||||
...event('user/message', 0, 1, {
|
||||
id: 'goal-message', role: 'user',
|
||||
content: [{ type: 'text', text: `<goal_state>${JSON.stringify(payload)}</goal_state>` }],
|
||||
source: { kind: 'goal', goalId: 'goal-1', revision: 1, round: 0, change },
|
||||
}),
|
||||
surfaceOp: 'append',
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
expect(migrateV1ToV2(source).events).toMatchObject([
|
||||
{ type: 'goal/change', seq: 0, data: change },
|
||||
{ type: 'user/message', seq: 1, data: { source: { kind: 'plugin', plugin: 'goal' } } },
|
||||
])
|
||||
})
|
||||
|
||||
it('closes the prior turn for the bounded legacy next-turn resume pattern', () => {
|
||||
const source: SessionFormatArtifact = {
|
||||
header: { version: 1, id: 'legacy-turn', createdAt: 1, isSeeded: false, delegationDepth: 0 },
|
||||
inheritedEventCount: 0,
|
||||
events: [
|
||||
event('turn/start', 0, 1, { turn: 1 }),
|
||||
event('step/start', 1, 2, { turn: 1, step: 1 }),
|
||||
event('step/end', 2, 3, { turn: 1, step: 1 }),
|
||||
event('agent/inbox/spliced', 3, 4, {
|
||||
target: 'next-turn', start: 0, inserted: [userMessage],
|
||||
}),
|
||||
event('turn/start', 4, 5, { turn: 2 }),
|
||||
event('turn/end', 5, 6, { turn: 2, reason: { kind: 'completed' } }),
|
||||
],
|
||||
}
|
||||
|
||||
const migrated = migrateV1ToV2(source)
|
||||
expect(migrated.events.map(candidate => candidate.type)).toEqual([
|
||||
'turn/start', 'step/start', 'step/end', 'agent/inbox/spliced',
|
||||
'turn/end', 'turn/start', 'turn/end',
|
||||
])
|
||||
expect(migrated.events[4]).toMatchObject({
|
||||
data: { turn: 1, reason: { kind: 'interrupted' } },
|
||||
})
|
||||
|
||||
const invalid = {
|
||||
...source,
|
||||
events: source.events.map(candidate => candidate.seq === 3
|
||||
? event('agent/inbox/spliced', 3, 4, { target: 'next-turn', start: 0, inserted: [] })
|
||||
: candidate),
|
||||
}
|
||||
expect(() => migrateV1ToV2(invalid)).toThrow(/turn\/start 2/)
|
||||
})
|
||||
|
||||
it('embeds an interleaved successful stream and densely remaps survivors', () => {
|
||||
const source: SessionFormatArtifact = {
|
||||
header: {
|
||||
@@ -73,7 +518,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
],
|
||||
}
|
||||
|
||||
expect(sessionFormatV1ToV2.migrate(source)).toStrictEqual({
|
||||
expect(migrateV1ToV2(source)).toStrictEqual({
|
||||
header: { ...source.header, version: 2 },
|
||||
inheritedEventCount: 0,
|
||||
events: [
|
||||
@@ -133,7 +578,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
],
|
||||
}
|
||||
|
||||
expect(sessionFormatV1ToV2.migrate(source)).toStrictEqual({
|
||||
expect(migrateV1ToV2(source)).toStrictEqual({
|
||||
header: { ...source.header, version: 2 },
|
||||
inheritedEventCount: 0,
|
||||
events: [
|
||||
@@ -203,7 +648,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
],
|
||||
}
|
||||
|
||||
const migrated = sessionFormatV1ToV2.migrate(source)
|
||||
const migrated = migrateV1ToV2(source)
|
||||
expect(migrated.events.filter(event => event.type.startsWith('assistant/'))).toStrictEqual([
|
||||
event('assistant/attempt', 3, 110, {
|
||||
turn: 1,
|
||||
@@ -260,7 +705,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
],
|
||||
}
|
||||
|
||||
const migrated = sessionFormatV1ToV2.migrate(source)
|
||||
const migrated = migrateV1ToV2(source)
|
||||
expect(migrated.inheritedEventCount).toBe(5)
|
||||
expect(migrated.events[5]).toStrictEqual({
|
||||
type: 'session/end-seed',
|
||||
@@ -284,7 +729,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
events: [],
|
||||
}
|
||||
|
||||
expect(sessionFormatV1ToV2.migrate(source)).toStrictEqual({
|
||||
expect(migrateV1ToV2(source)).toStrictEqual({
|
||||
header: { ...source.header, version: 2 },
|
||||
inheritedEventCount: 0,
|
||||
events: [{
|
||||
@@ -305,7 +750,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
inheritedEventCount: 1,
|
||||
events: [event('feedback/record', 0, 9, { text: 'inherited' })],
|
||||
}
|
||||
expect(sessionFormatV1ToV2.migrate(source)).toMatchObject({
|
||||
expect(migrateV1ToV2(source)).toMatchObject({
|
||||
inheritedEventCount: 1,
|
||||
events: [
|
||||
{ type: 'feedback/record', seq: 0 },
|
||||
@@ -351,7 +796,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
],
|
||||
}
|
||||
|
||||
expect(() => sessionFormatV1ToV2.migrate(source)).toThrow(/message content disagrees with its embedded stream/)
|
||||
expect(() => migrateV1ToV2(source)).toThrow(/message content disagrees with its embedded stream/)
|
||||
})
|
||||
|
||||
it('refuses an undeclared reference into the last consumed chunk of a failed attempt', () => {
|
||||
@@ -391,7 +836,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
],
|
||||
}
|
||||
|
||||
expect(() => sessionFormatV1ToV2.migrate(source)).toThrow(
|
||||
expect(() => migrateV1ToV2(source)).toThrow(
|
||||
/command\/done 7 sourceEventSeq targets consumed assistant\/chunk 3/,
|
||||
)
|
||||
})
|
||||
@@ -412,7 +857,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
}],
|
||||
}
|
||||
|
||||
expect(() => sessionFormatV1ToV2.migrate(source)).toThrow(
|
||||
expect(() => migrateV1ToV2(source)).toThrow(
|
||||
/format v1 contains unknown event type "external\/info" at seq 0/,
|
||||
)
|
||||
})
|
||||
@@ -438,7 +883,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
event('turn/end', 4, 5, { turn: 1, reason: { kind: 'completed' } }),
|
||||
],
|
||||
}
|
||||
const migrated = sessionFormatV1ToV2.migrate(source)
|
||||
const migrated = migrateV1ToV2(source)
|
||||
expect(migrated.events[2]).toMatchObject({
|
||||
type: 'assistant/message', data: { stream: [] }, surfaceOp: 'append',
|
||||
})
|
||||
@@ -470,9 +915,51 @@ describe('sessionFormatV1ToV2', () => {
|
||||
event('turn/end', 6, 7, { turn: 1, reason: { kind: 'completed' } }),
|
||||
],
|
||||
})
|
||||
expect(() => sessionFormatV1ToV2.migrate(build(undefined))).toThrow(/does not cite/)
|
||||
expect(() => sessionFormatV1ToV2.migrate(build([2]))).toThrow(/complete ordered attempt/)
|
||||
expect(() => sessionFormatV1ToV2.migrate(build([3, 2]))).toThrow(/complete ordered attempt/)
|
||||
expect(() => migrateV1ToV2(build(undefined))).toThrow(/does not cite/)
|
||||
expect(() => migrateV1ToV2(build([2]))).toThrow(/complete ordered attempt/)
|
||||
expect(() => migrateV1ToV2(build([3, 2]))).toThrow(/complete ordered attempt/)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['a message after its attempt step has closed', [
|
||||
event('turn/start', 0, 1, { turn: 1 }),
|
||||
event('step/start', 1, 2, { turn: 1, step: 1 }),
|
||||
event('assistant/chunk', 2, 3, {
|
||||
turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'hello' },
|
||||
}),
|
||||
event('assistant/chunk', 3, 4, {
|
||||
turn: 1, step: 1, chunk: { type: 'finish', reason: { kind: 'stop' } },
|
||||
}),
|
||||
event('step/end', 4, 5, { turn: 1, step: 1 }),
|
||||
{ ...event('assistant/message', 5, 6, { turn: 1, step: 1, message }), surfaceOp: 'append' },
|
||||
event('turn/end', 6, 7, { turn: 1, reason: { kind: 'completed' } }),
|
||||
]],
|
||||
['a message that cites a different pending attempt', [
|
||||
event('turn/start', 0, 1, { turn: 1 }),
|
||||
event('step/start', 1, 2, { turn: 1, step: 1 }),
|
||||
event('assistant/chunk', 2, 3, {
|
||||
turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'hello' },
|
||||
}),
|
||||
event('assistant/chunk', 3, 4, {
|
||||
turn: 1, step: 1, chunk: { type: 'finish', reason: { kind: 'stop' } },
|
||||
}),
|
||||
{
|
||||
...event('assistant/message', 4, 5, { turn: 1, step: 2, message }),
|
||||
sourceEventSeqs: [2, 3],
|
||||
surfaceOp: 'append',
|
||||
},
|
||||
event('step/end', 5, 6, { turn: 1, step: 1 }),
|
||||
event('turn/end', 6, 7, { turn: 1, reason: { kind: 'completed' } }),
|
||||
]],
|
||||
] as const)('rejects %s during complete target validation', (_name, events) => {
|
||||
expect(() => migrateV1ToV2({
|
||||
header: {
|
||||
version: 1, id: 'v1-final-validation', createdAt: 1,
|
||||
isSeeded: false, delegationDepth: 0,
|
||||
},
|
||||
inheritedEventCount: 0,
|
||||
events,
|
||||
})).toThrow(/does not match an open turn and step/)
|
||||
})
|
||||
|
||||
it('refuses a lineage cut between members of one interleaved stream', () => {
|
||||
@@ -501,7 +988,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
event('turn/end', 7, 8, { turn: 1, reason: { kind: 'completed' } }),
|
||||
],
|
||||
}
|
||||
expect(() => sessionFormatV1ToV2.migrate(source)).toThrow(/cut 3 splits one Assistant attempt/)
|
||||
expect(() => migrateV1ToV2(source)).toThrow(/cut 3 splits one Assistant attempt/)
|
||||
})
|
||||
|
||||
it('refuses a lineage cut between a complete stream and its committed message', () => {
|
||||
@@ -530,7 +1017,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
event('turn/end', 7, 8, { turn: 1, reason: { kind: 'completed' } }),
|
||||
],
|
||||
}
|
||||
expect(() => sessionFormatV1ToV2.migrate(source)).toThrow(/cut 4 splits one Assistant attempt/)
|
||||
expect(() => migrateV1ToV2(source)).toThrow(/cut 4 splits one Assistant attempt/)
|
||||
})
|
||||
|
||||
it('keeps referenced-session generation provenance frozen', () => {
|
||||
@@ -565,7 +1052,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
events: [{ ...event('user/message', 0, 1, reference), surfaceOp: 'append' }],
|
||||
}
|
||||
|
||||
const migrated = sessionFormatV1ToV2.migrate(source)
|
||||
const migrated = migrateV1ToV2(source)
|
||||
expect(migrated.events[0]?.data).toStrictEqual(reference)
|
||||
})
|
||||
|
||||
@@ -634,7 +1121,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
event('turn/end', 15, 16, { turn: 1, reason: { kind: 'completed' } }),
|
||||
],
|
||||
}
|
||||
const migrated = sessionFormatV1ToV2.migrate(source)
|
||||
const migrated = migrateV1ToV2(source)
|
||||
expect(migrated.events.find(event => event.type === 'compaction/prune')?.data).toMatchObject({
|
||||
shadowedRange: { start: 2, end: 2 }, shadowedSeqs: [2],
|
||||
})
|
||||
@@ -687,7 +1174,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
],
|
||||
}
|
||||
|
||||
const migrated = sessionFormatV1ToV2.migrate(source)
|
||||
const migrated = migrateV1ToV2(source)
|
||||
const titleRequest = migrated.events.find(event => event.type === 'session/title-llm-request')
|
||||
|
||||
expect(titleRequest?.data).toMatchObject({
|
||||
@@ -728,7 +1215,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
event('turn/end', 10, 11, { turn: 2, reason: { kind: 'completed' } }),
|
||||
],
|
||||
}
|
||||
expect(sessionFormatV1ToV2.migrate(source).events[3]?.data).toMatchObject({
|
||||
expect(migrateV1ToV2(source).events[3]?.data).toMatchObject({
|
||||
shadowedRange: { start: 1, end: 1 }, shadowedSeqs: [1],
|
||||
})
|
||||
})
|
||||
@@ -758,7 +1245,7 @@ describe('sessionFormatV1ToV2', () => {
|
||||
event('turn/end', 9, 10, { turn: 2, reason: { kind: 'error', error: failure } }),
|
||||
],
|
||||
}
|
||||
expect(sessionFormatV1ToV2.migrate(source).events.filter(event => event.type === 'assistant/attempt'))
|
||||
expect(migrateV1ToV2(source).events.filter(event => event.type === 'assistant/attempt'))
|
||||
.toHaveLength(2)
|
||||
})
|
||||
})
|
||||
@@ -8,10 +8,10 @@ import type {
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import {
|
||||
RELEASED_V2_EVENT_TYPES,
|
||||
assertReleasedV2Artifact,
|
||||
assertReleasedV2Header,
|
||||
restoreReleasedV2Artifact,
|
||||
} from '@deepseek-ai/dsh-session-format-v1-to-v2'
|
||||
import { assertReleasedV2Artifact } from '../src/testing/validation.ts'
|
||||
|
||||
const textBlock = { type: 'text', text: 'hello' } as const
|
||||
const usage = { inputTokens: 3, outputTokens: 2 } as const
|
||||
@@ -208,6 +208,22 @@ describe('released v2 event envelopes and payloads', () => {
|
||||
expect(() => { assertReleasedV2Artifact(value) }).toThrow(message)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['inherited count beyond events', artifact([], { inheritedEventCount: 1 }), /exceeds its events/],
|
||||
['unseeded inherited count', artifact([
|
||||
event('feedback/record', 0, { text: 'x' }),
|
||||
], { inheritedEventCount: 1 }), /unseeded.*inherited events/],
|
||||
['non-string type', artifact([
|
||||
{ type: 1, seq: 0, time: 1, data: {} } as unknown as SessionFormatEvent,
|
||||
]), /type must be a string/],
|
||||
['non-dense seq', artifact([event('feedback/record', 1, { text: 'x' })]), /not dense/],
|
||||
['false ignorable marker', artifact([
|
||||
event('feedback/record', 0, { text: 'x' }, { ignorable: false }),
|
||||
]), /ignorable must be true/],
|
||||
])('rejects production artifact drift before payload restoration: %s', (_name, value, message) => {
|
||||
expect(() => { restoreReleasedV2Artifact(value, new Set(RELEASED_V2_EVENT_TYPES)) }).toThrow(message)
|
||||
})
|
||||
|
||||
it('validates present and absent opaque payload members before relationship checks', () => {
|
||||
for (const includeMeta of [false, true]) {
|
||||
const value = artifact([
|
||||
@@ -298,6 +314,16 @@ describe('released v2 seed and surface relationships', () => {
|
||||
expect(() => { assertReleasedV2Artifact(value) }).toThrow(message)
|
||||
})
|
||||
|
||||
it('rejects production seed lineage disagreement', () => {
|
||||
const known = new Set(RELEASED_V2_EVENT_TYPES)
|
||||
expect(() => { restoreReleasedV2Artifact(artifact([], {
|
||||
header: { version: 2, id: 'seeded', createdAt: 1, isSeeded: true, delegationDepth: 0 },
|
||||
}), known) }).toThrow(/seeded header disagrees/)
|
||||
expect(() => { restoreReleasedV2Artifact(artifact([
|
||||
event('session/end-seed', 0, { inherited: true }),
|
||||
]), known) }).toThrow(/unseeded.*inherited end-seed/)
|
||||
})
|
||||
|
||||
it('accepts append and exact replacement surface operations', () => {
|
||||
expect(() => { assertReleasedV2Artifact(artifact([
|
||||
event('user/message', 0, userData('one'), { surfaceOp: 'append' }),
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-format/README.md
|
||||
README.md: ffaa1f7a610fb915227c260d4fe2d4d1b60c04e9
|
||||
README.zh.md: ab9f0dd8d66d4f92bcdf2a136f76b9b7587037f8
|
||||
README.md: d44a7091bc5840afcb5d1e73f594284c3d5c8206
|
||||
README.zh.md: f4b6acd8e0ba3c28a1b5088fc577e1b4289ca9a0
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
description: "Pure adjacent Session format planning, lossless JSON snapshots, header-only migration, and physical codec dispatch."
|
||||
description: "Pure adjacent Session format planning, lossless JSON value checks, header-only migration, and physical codec dispatch."
|
||||
kind: "package-library"
|
||||
---
|
||||
|
||||
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
## Summary
|
||||
|
||||
`dsh-session-format` lets persistence code restore a current Session directly or compose a unique sequence of adjacent whole-artifact migrations. It snapshots every durable input and output as detached lossless JSON, validates exact version progress, and keeps header-only listing separate from body reads. Physical framing, compression, immutable generation naming, exclusive publication, and Cordis lifecycle behavior remain outside this pure library.
|
||||
`dsh-session-format` lets persistence code restore a current Session directly or compose a unique sequence of adjacent migrations while consuming physical rows once. A restore transfers caller-owned parsed values through stateful stages without intermediate artifact copies or freezing. Physical framing, compression, immutable generation naming, exclusive publication, and Cordis lifecycle behavior remain outside this library.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -27,16 +27,23 @@ English | [中文](README.zh.md)
|
||||
|
||||
### When to use it
|
||||
|
||||
Use this library from persistence or format-catalog code that must classify a physical Session header, restore current logical values, or compose released adjacent migrations. It is not a Cordis plugin and has no profile mount row. No runtime invariant companion is published because every operation validates its borrowed artifact before returning and retains no cross-call mutable state.
|
||||
Use this library from persistence or format-catalog code that must classify a physical Session header, restore current logical values, or compose released adjacent migrations. It is not a Cordis plugin and has no profile mount row. No runtime invariant companion is published because each completed operation validates its result; decoder and transformer state belongs to one unfinished streaming restore and is never shared across restores.
|
||||
|
||||
### Entry point
|
||||
|
||||
```text
|
||||
const catalog = createSessionFormatCatalog({ currentVersion, codecs, encodeCurrentArtifact, migrations, restoreCurrent, restoreCurrentHeader })
|
||||
const catalog = createSessionFormatCatalog({ currentVersion, codecs, currentEncoder, migrations, restoreCurrent, restoreTransformedCurrent, restoreCurrentHeader })
|
||||
const descriptor = catalog.readHeader(physicalHeader)
|
||||
const restore = catalog.createRestore(physicalHeader, { recovery: 'recoverable', validation: 'transformed' })
|
||||
for (const row of physicalRows) restore.decodeRow(row)
|
||||
const current = restore.finish()
|
||||
const headerRecord = catalog.encodeCurrentHeader(current.header, current.inheritedEventCount)
|
||||
const eventRecords = current.events.map(catalog.encodeCurrentEvent)
|
||||
```
|
||||
|
||||
`createSessionFormatCatalog()` accepts one frozen decoder per supported version, the current format's encoder, one migration per adjacent version pair, and current artifact and header restorers. `readHeader()` returns a `current`, `migration-required`, `unsupported`, or `malformed` descriptor without reading events. Each edge validates its target header before the final current-header restorer runs. Body readers call `decodeArtifact()` or `decodeRecoverableArtifact()`, then `migrate()`; writers call `encodeCurrent()` only with a validated current artifact. Frozen v0/v1 codec exports retain their format-specific `packChunks` option without adding that historical control to the current writer or common decoder interface.
|
||||
`createSessionFormatCatalog()` accepts one frozen codec per supported version, the current record encoder, one migration per adjacent version pair, and current artifact and header restorers. `readHeader()` returns a `current`, `migration-required`, `unsupported`, or `malformed` descriptor without reading events. Body readers create one restore, push each parsed physical row through `decodeRow()`, and call `finish()` once for a current artifact. Writers encode its header and events record by record.
|
||||
|
||||
The `recovery` option selects strict row failure or recoverable suffix handling. `validation: 'current'` applies all installed current-format validation. `validation: 'transformed'` applies released current-format validation after historical migration, while already-current input receives only its codec's physical validation.
|
||||
|
||||
The recoverable decoder returns the accepted logical prefix. A codec may drop one malformed or sequence-gapped row and its uncommitted suffix, but a later decoded `turn/end` makes the original issue fatal.
|
||||
|
||||
@@ -48,7 +55,7 @@ The recoverable decoder returns the accepted logical prefix. A codec may drop on
|
||||
<details>
|
||||
<summary>Implementation internals — click to expand</summary>
|
||||
|
||||
The chain validates unique gap-free ordering at construction. A current artifact bypasses every migration callback and passes through only the current restorer. An old artifact runs each adjacent whole-document function in memory; only the caller decides whether and how to publish the final result.
|
||||
The chain validates unique gap-free ordering at construction. The catalog composes one row decoder with stateful adjacent event transformers, retains only their bounded state and the final current events, and performs target validation at `finish()`; only the caller decides whether and how to publish that result.
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
@@ -91,7 +98,7 @@ No direct effect. A migration that changes current history can change the cache
|
||||
|
||||
<a id="known-limitations-and-deferred-work"></a>
|
||||
|
||||
- **Whole-artifact memory use** — supported migrations materialize the complete logical Session; streamed transformation is deferred until measured artifacts require it.
|
||||
- **Final current history remains resident** — streaming retains only bounded intermediate state, but the returned current event array and any required sequence-remap table remain O(event count).
|
||||
- **Adjacent integer versions only** — the library does not expose spans, stable event identities, or a general reference-rewrite algebra.
|
||||
|
||||
<a id="dev-note"></a>
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
description: "纯函数式相邻 Session 格式规划、无损 JSON 快照、仅标头迁移与物理编解码分派。"
|
||||
description: "纯函数式相邻 Session 格式规划、无损 JSON 值检查、仅标头迁移与物理编解码分派。"
|
||||
kind: "package-library"
|
||||
---
|
||||
|
||||
@@ -9,7 +9,7 @@ kind: "package-library"
|
||||
|
||||
## 概述
|
||||
|
||||
`dsh-session-format` 让持久化代码可以直接还原当前 Session，或组合唯一的相邻全产物迁移序列。它会把每个持久化输入和输出快照为分离的无损 JSON，校验精确的版本推进，并把仅标头的列表读取与正文读取分开。物理分帧、压缩、不可变 generation 命名、排他发布和 Cordis 生命周期行为不属于这个纯函数库。
|
||||
`dsh-session-format` 让持久化代码可以直接还原当前 Session，或在只消费一次物理行的同时组合唯一的相邻迁移序列。一次还原会让调用方拥有的已解析值流经有状态 Stage，不复制或冻结中间 artifact。物理分帧、压缩、不可变 generation 命名、排他发布和 Cordis 生命周期行为不属于本库。
|
||||
|
||||
## 目录
|
||||
|
||||
@@ -27,16 +27,23 @@ kind: "package-library"
|
||||
|
||||
### 何时使用
|
||||
|
||||
当持久化或格式目录代码需要分类物理 Session header、还原当前逻辑值或组合已发布相邻迁移时，使用本库。它不是 Cordis 插件，也没有 profile 挂载行。它不发布运行时不变式伴生入口，因为每个操作都会在返回前校验借入的完整 artifact，且不保留跨调用的可变状态。
|
||||
当持久化或格式目录代码需要分类物理 Session header、还原当前逻辑值或组合已发布相邻迁移时，使用本库。它不是 Cordis 插件，也没有 profile 挂载行。它不发布运行时不变式伴生入口，因为每个已完成操作都会校验结果；decoder 与 transformer 状态只属于一次尚未完成的流式还原，绝不在多次还原间共享。
|
||||
|
||||
### 入口
|
||||
|
||||
```text
|
||||
const catalog = createSessionFormatCatalog({ currentVersion, codecs, encodeCurrentArtifact, migrations, restoreCurrent, restoreCurrentHeader })
|
||||
const catalog = createSessionFormatCatalog({ currentVersion, codecs, currentEncoder, migrations, restoreCurrent, restoreTransformedCurrent, restoreCurrentHeader })
|
||||
const descriptor = catalog.readHeader(physicalHeader)
|
||||
const restore = catalog.createRestore(physicalHeader, { recovery: 'recoverable', validation: 'transformed' })
|
||||
for (const row of physicalRows) restore.decodeRow(row)
|
||||
const current = restore.finish()
|
||||
const headerRecord = catalog.encodeCurrentHeader(current.header, current.inheritedEventCount)
|
||||
const eventRecords = current.events.map(catalog.encodeCurrentEvent)
|
||||
```
|
||||
|
||||
`createSessionFormatCatalog()` 接收每个受支持版本的一个冻结解码器、当前格式的编码器、每组相邻版本的一个迁移，以及当前产物与标头还原器。`readHeader()` 在不读取事件的情况下返回 `current`、`migration-required`、`unsupported` 或 `malformed` 描述符。每个迁移边会先校验自己的目标标头，然后再运行最终的当前标头还原器。正文读取方调用 `decodeArtifact()` 或 `decodeRecoverableArtifact()`，然后调用 `migrate()`；写入方只使用经过校验的当前产物调用 `encodeCurrent()`。冻结的 v0/v1 编解码器导出会保留其格式专用的 `packChunks` 选项，但不会把这项历史控制加入当前 writer 或通用解码器接口。
|
||||
`createSessionFormatCatalog()` 接收每个受支持版本的一个冻结 codec、当前格式的逐记录 encoder、每组相邻版本的一个迁移，以及当前 artifact 与 header 还原器。`readHeader()` 在不读取事件的情况下返回 `current`、`migration-required`、`unsupported` 或 `malformed` 描述符。正文读取方创建一次 restore，把每个已解析物理行传给 `decodeRow()`，再调用一次 `finish()` 获得当前 artifact。写入方逐条编码其 header 与事件。
|
||||
|
||||
`recovery` 选项决定严格拒绝故障行，还是执行可恢复后缀处理。`validation: 'current'` 会执行已安装 current 格式的全部校验。`validation: 'transformed'` 会在历史迁移后执行已发布 current 格式校验；已经是 current 的输入则只接受其 codec 的物理校验。
|
||||
|
||||
可恢复解码器返回已接受的逻辑前缀。编解码器可以丢弃一个格式错误或序号不连续的行及其未提交后缀，但后续成功解码的 `turn/end` 会使原始问题成为致命错误。
|
||||
|
||||
@@ -48,7 +55,7 @@ const descriptor = catalog.readHeader(physicalHeader)
|
||||
<details>
|
||||
<summary>实现细节——点击展开</summary>
|
||||
|
||||
迁移链在构造时校验唯一且无缺口的顺序。当前产物绕过所有迁移回调，只经过当前格式还原器。旧产物在内存中依次运行每个相邻的全产物函数；只有调用方决定是否发布最终结果以及如何发布。
|
||||
迁移链在构造时校验唯一且无缺口的顺序。Catalog 把一个行 decoder 与有状态的相邻事件 transformer 组合起来，只保留其有界状态与最终当前事件，并在 `finish()` 时执行目标校验；只有调用方决定是否发布该结果以及如何发布。
|
||||
|
||||
| 文件 | 职责 |
|
||||
|---|---|
|
||||
@@ -91,7 +98,7 @@ const descriptor = catalog.readHeader(physicalHeader)
|
||||
|
||||
<a id="known-limitations-and-deferred-work"></a>
|
||||
|
||||
- **全产物内存占用**——受支持的迁移会物化完整逻辑 Session；只有实测产物规模提出要求时，才会引入流式转换。
|
||||
- **最终当前历史仍常驻内存**——流式处理只保留有界中间状态，但返回的当前事件数组和必需的序号重映射表仍为 O(事件数)。
|
||||
- **仅支持相邻整数版本**——本库不暴露 span、稳定事件身份或通用引用重写代数。
|
||||
|
||||
<a id="dev-note"></a>
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-session-format",
|
||||
"description": "Pure adjacent whole-artifact Session format migration machinery",
|
||||
"description": "Streaming adjacent Session format migration machinery",
|
||||
"version": "0.1.3-alpha.1",
|
||||
"publishConfig": {
|
||||
"access": "public"
|
||||
|
||||
@@ -1,19 +1,24 @@
|
||||
import { createSessionFormatChain } from './chain.ts'
|
||||
import { SessionFormatEventCollector } from './context.ts'
|
||||
import { SessionFormatError, SessionFormatUnsupportedMigrationError } from './error.ts'
|
||||
import {
|
||||
inspectSessionFormatVersion,
|
||||
snapshotSessionFormatArtifact,
|
||||
snapshotSessionFormatHeader,
|
||||
snapshotSessionFormatJson,
|
||||
sessionFormatVersion,
|
||||
} from './json.ts'
|
||||
import type {
|
||||
EncodedSessionFormatArtifact,
|
||||
SessionFormatArtifact,
|
||||
SessionFormatArtifactDecoder,
|
||||
SessionFormatCatalog,
|
||||
SessionFormatCatalogOptions,
|
||||
SessionFormatCodec,
|
||||
SessionFormatEvent,
|
||||
SessionFormatEventRun,
|
||||
SessionFormatHeaderReadResult,
|
||||
SessionFormatJsonObject,
|
||||
SessionFormatMigrationContext,
|
||||
SessionFormatMigrationStream,
|
||||
SessionFormatRestore,
|
||||
SessionFormatRestoreOptions,
|
||||
} from './types.ts'
|
||||
|
||||
/**
|
||||
@@ -102,48 +107,185 @@ export function createSessionFormatCatalog(options: SessionFormatCatalogOptions)
|
||||
return { storedVersion, codec }
|
||||
}
|
||||
|
||||
function decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]) {
|
||||
const { storedVersion, codec } = artifactCodec(headerValue)
|
||||
return snapshotSessionFormatArtifact(
|
||||
codec.decodeArtifact(headerValue, rowValues),
|
||||
`format v${storedVersion} decoded artifact`,
|
||||
)
|
||||
}
|
||||
|
||||
function decodeRecoverableArtifact(headerValue: unknown, rowValues: readonly unknown[]) {
|
||||
const { storedVersion, codec } = artifactCodec(headerValue)
|
||||
return snapshotSessionFormatArtifact(
|
||||
codec.decodeRecoverableArtifact(headerValue, rowValues),
|
||||
`format v${storedVersion} recoverable artifact`,
|
||||
)
|
||||
}
|
||||
|
||||
function encodeCurrent(
|
||||
artifact: Parameters<SessionFormatCatalog['encodeCurrent']>[0],
|
||||
): EncodedSessionFormatArtifact {
|
||||
if (inspectSessionFormatVersion(artifact.header) !== chain.currentVersion) {
|
||||
function encodeCurrentHeader(
|
||||
header: Parameters<SessionFormatCatalog['encodeCurrentHeader']>[0],
|
||||
inheritedEventCount: number,
|
||||
) {
|
||||
if (inspectSessionFormatVersion(header) !== chain.currentVersion) {
|
||||
throw new SessionFormatError(`encodeCurrent requires Session format v${chain.currentVersion}`)
|
||||
}
|
||||
const encoded = options.encodeCurrentArtifact(artifact)
|
||||
const header = snapshotSessionFormatJson(encoded.header, 'encoded current Session header') as SessionFormatJsonObject
|
||||
const rows = Object.freeze(encoded.rows.map((row, index) =>
|
||||
snapshotSessionFormatJson(row, `encoded current Session row ${index}`) as SessionFormatJsonObject))
|
||||
if (inspectSessionFormatVersion(header) !== chain.currentVersion) {
|
||||
const encoded = options.currentEncoder.encodeHeader(header, inheritedEventCount)
|
||||
if (inspectSessionFormatVersion(encoded) !== chain.currentVersion) {
|
||||
throw new SessionFormatError('current Session codec returned a non-current header')
|
||||
}
|
||||
return Object.freeze({ header, rows })
|
||||
return encoded
|
||||
}
|
||||
|
||||
function createRestore(
|
||||
headerValue: unknown,
|
||||
restoreOptions: SessionFormatRestoreOptions,
|
||||
): SessionFormatRestore {
|
||||
const { storedVersion, codec } = artifactCodec(headerValue)
|
||||
const decoder = codec.createDecoder(headerValue, restoreOptions.recovery)
|
||||
const sourceCut = decoder.headerInheritedEventCount
|
||||
if (storedVersion === chain.currentVersion) {
|
||||
return new CurrentSessionFormatRestore(
|
||||
decoder,
|
||||
sourceCut,
|
||||
restoreOptions.validation === 'current' ? options.restoreCurrent : identityArtifact,
|
||||
chain.currentVersion,
|
||||
)
|
||||
}
|
||||
const collector = new SessionFormatEventCollector()
|
||||
const migration = chain.createStream(
|
||||
decoder.header,
|
||||
requiredHistoricalCut(storedVersion, sourceCut),
|
||||
collector,
|
||||
)
|
||||
return new MigratingSessionFormatRestore(
|
||||
decoder,
|
||||
sourceCut,
|
||||
migration,
|
||||
collector,
|
||||
restoreOptions.validation === 'current'
|
||||
? options.restoreCurrent
|
||||
: options.restoreTransformedCurrent,
|
||||
restoreOptions.validation,
|
||||
storedVersion,
|
||||
chain.currentVersion,
|
||||
)
|
||||
}
|
||||
|
||||
return Object.freeze({
|
||||
currentVersion: chain.currentVersion,
|
||||
readHeader,
|
||||
decodeArtifact,
|
||||
decodeRecoverableArtifact,
|
||||
migrate: chain.migrate.bind(chain),
|
||||
encodeCurrent,
|
||||
createRestore,
|
||||
encodeCurrentHeader,
|
||||
encodeCurrentEvent: options.currentEncoder.encodeEvent.bind(options.currentEncoder),
|
||||
})
|
||||
}
|
||||
|
||||
type SessionFormatArtifactRestorer = (artifact: SessionFormatArtifact) => SessionFormatArtifact
|
||||
|
||||
class CurrentSessionFormatRestore implements SessionFormatRestore {
|
||||
readonly header: SessionFormatArtifact['header']
|
||||
private readonly collector = new SessionFormatEventCollector()
|
||||
|
||||
constructor(
|
||||
private readonly decoder: SessionFormatArtifactDecoder,
|
||||
private readonly sourceInheritedEventCount: number | undefined,
|
||||
private readonly restoreArtifact: SessionFormatArtifactRestorer,
|
||||
private readonly currentVersion: number,
|
||||
) {
|
||||
this.header = decoder.header
|
||||
}
|
||||
|
||||
decodeRow(rowValue: unknown): void {
|
||||
this.decoder.decodeRow(rowValue, this.collector)
|
||||
}
|
||||
|
||||
finish(): SessionFormatArtifact {
|
||||
const inheritedEventCount = finishDecoder(
|
||||
this.decoder,
|
||||
this.collector,
|
||||
this.sourceInheritedEventCount,
|
||||
)
|
||||
return restoreCurrentVersion(this.restoreArtifact({
|
||||
header: this.header,
|
||||
inheritedEventCount,
|
||||
events: this.collector.values,
|
||||
}), this.currentVersion)
|
||||
}
|
||||
}
|
||||
|
||||
class MigratingSessionFormatRestore implements
|
||||
SessionFormatRestore,
|
||||
SessionFormatMigrationContext {
|
||||
readonly header: SessionFormatArtifact['header']
|
||||
|
||||
constructor(
|
||||
private readonly decoder: SessionFormatArtifactDecoder,
|
||||
private readonly sourceInheritedEventCount: number | undefined,
|
||||
private readonly migration: SessionFormatMigrationStream,
|
||||
private readonly collector: SessionFormatEventCollector,
|
||||
private readonly restoreArtifact: SessionFormatArtifactRestorer,
|
||||
private readonly validation: SessionFormatRestoreOptions['validation'],
|
||||
private readonly sourceVersion: number,
|
||||
private readonly currentVersion: number,
|
||||
) {
|
||||
this.header = migration.header
|
||||
}
|
||||
|
||||
decodeRow(rowValue: unknown): void {
|
||||
this.decoder.decodeRow(rowValue, this)
|
||||
}
|
||||
|
||||
emitEvent(event: SessionFormatEvent): void {
|
||||
this.migration.emitEvent(event)
|
||||
}
|
||||
|
||||
emitRun(run: SessionFormatEventRun): void {
|
||||
this.migration.emitRun(run)
|
||||
}
|
||||
|
||||
finish(): SessionFormatArtifact {
|
||||
finishDecoder(this.decoder, this, this.sourceInheritedEventCount)
|
||||
const artifact = {
|
||||
header: this.header,
|
||||
inheritedEventCount: this.migration.finish(),
|
||||
events: this.collector.values,
|
||||
}
|
||||
let restored: SessionFormatArtifact
|
||||
try {
|
||||
restored = this.restoreArtifact(artifact)
|
||||
} catch (error: unknown) {
|
||||
if (this.validation === 'current'
|
||||
|| error instanceof SessionFormatUnsupportedMigrationError) throw error
|
||||
const detail = error instanceof Error ? error.message : String(error)
|
||||
throw new SessionFormatUnsupportedMigrationError(
|
||||
`Session migration from v${this.sourceVersion} to v${this.currentVersion} refuses the transformed artifact: ${detail}`,
|
||||
{ cause: error },
|
||||
)
|
||||
}
|
||||
return restoreCurrentVersion(restored, this.currentVersion)
|
||||
}
|
||||
}
|
||||
|
||||
function finishDecoder(
|
||||
decoder: SessionFormatArtifactDecoder,
|
||||
context: SessionFormatMigrationContext,
|
||||
sourceInheritedEventCount: number | undefined,
|
||||
): number {
|
||||
const inheritedEventCount = decoder.finish(context)
|
||||
if (sourceInheritedEventCount !== undefined && inheritedEventCount !== sourceInheritedEventCount) {
|
||||
throw new SessionFormatError('streaming decoder changed its predeclared inherited cut')
|
||||
}
|
||||
return inheritedEventCount
|
||||
}
|
||||
|
||||
function restoreCurrentVersion(
|
||||
artifact: SessionFormatArtifact,
|
||||
currentVersion: number,
|
||||
): SessionFormatArtifact {
|
||||
if (artifact.header.version !== currentVersion) {
|
||||
throw new SessionFormatError(
|
||||
`current Session restorer returned v${artifact.header.version}; expected v${currentVersion}`,
|
||||
)
|
||||
}
|
||||
return artifact
|
||||
}
|
||||
|
||||
function identityArtifact(artifact: SessionFormatArtifact): SessionFormatArtifact {
|
||||
return artifact
|
||||
}
|
||||
|
||||
function requiredHistoricalCut(version: number, cut: number | undefined): number {
|
||||
if (cut === undefined) {
|
||||
throw new SessionFormatError(`format v${version} decoder must expose its inherited cut before migration`)
|
||||
}
|
||||
return cut
|
||||
}
|
||||
|
||||
function malformed(targetVersion: number, error: unknown, storedVersion?: number): SessionFormatHeaderReadResult {
|
||||
return Object.freeze({
|
||||
status: 'malformed',
|
||||
|
||||
@@ -1,16 +1,19 @@
|
||||
import { SessionFormatError, SessionFormatUnsupportedMigrationError } from './error.ts'
|
||||
import {
|
||||
inspectSessionFormatVersion,
|
||||
snapshotSessionFormatArtifact,
|
||||
snapshotSessionFormatHeader,
|
||||
sessionFormatCount,
|
||||
sessionFormatVersion,
|
||||
} from './json.ts'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatChain,
|
||||
SessionFormatChainOptions,
|
||||
SessionFormatEvent,
|
||||
SessionFormatEventRun,
|
||||
SessionFormatHeader,
|
||||
SessionFormatMigration,
|
||||
SessionFormatMigrationContext,
|
||||
SessionFormatMigrationStage,
|
||||
SessionFormatMigrationStream,
|
||||
} from './types.ts'
|
||||
|
||||
/**
|
||||
@@ -33,7 +36,7 @@ export function defineSessionFormatMigration(migration: SessionFormatMigration):
|
||||
/**
|
||||
* Compile a unique, complete adjacent migration chain.
|
||||
* @param options - current version, adjacent declarations, and current restorer.
|
||||
* @returns immutable planner and whole-artifact runner.
|
||||
* @returns immutable planner and streaming migration compiler.
|
||||
*/
|
||||
export function createSessionFormatChain(options: SessionFormatChainOptions): SessionFormatChain {
|
||||
return new CompiledSessionFormatChain(options)
|
||||
@@ -42,12 +45,10 @@ export function createSessionFormatChain(options: SessionFormatChainOptions): Se
|
||||
class CompiledSessionFormatChain implements SessionFormatChain {
|
||||
readonly currentVersion: number
|
||||
private readonly migrations: readonly SessionFormatMigration[]
|
||||
private readonly restoreCurrent: SessionFormatChainOptions['restoreCurrent']
|
||||
private readonly restoreCurrentHeader: SessionFormatChainOptions['restoreCurrentHeader']
|
||||
|
||||
constructor(options: SessionFormatChainOptions) {
|
||||
this.currentVersion = sessionFormatVersion(options.currentVersion, 'current Session format version')
|
||||
this.restoreCurrent = options.restoreCurrent
|
||||
this.restoreCurrentHeader = options.restoreCurrentHeader
|
||||
const byFrom = new Map<number, SessionFormatMigration>()
|
||||
const names = new Set<string>()
|
||||
@@ -75,7 +76,7 @@ class CompiledSessionFormatChain implements SessionFormatChain {
|
||||
this.migrations = Object.freeze(ordered)
|
||||
}
|
||||
|
||||
plan(fromVersion: number): readonly SessionFormatMigration[] {
|
||||
private plan(fromVersion: number): readonly SessionFormatMigration[] {
|
||||
const from = sessionFormatVersion(fromVersion, 'stored Session format version')
|
||||
if (from > this.currentVersion) {
|
||||
throw new SessionFormatUnsupportedMigrationError(
|
||||
@@ -85,54 +86,54 @@ class CompiledSessionFormatChain implements SessionFormatChain {
|
||||
return Object.freeze(this.migrations.slice(from))
|
||||
}
|
||||
|
||||
migrate(source: SessionFormatArtifact): SessionFormatArtifact {
|
||||
const storedVersion = inspectSessionFormatVersion(source.header)
|
||||
let current = snapshotSessionFormatArtifact(source, `format v${storedVersion} source`)
|
||||
if (storedVersion === this.currentVersion) {
|
||||
current = snapshotSessionFormatArtifact(this.restoreCurrent(current), 'current Session restoration')
|
||||
this.assertCurrent(current)
|
||||
return current
|
||||
}
|
||||
for (const migration of this.plan(storedVersion)) {
|
||||
let migrated: SessionFormatArtifact
|
||||
createStream(
|
||||
sourceHeader: SessionFormatHeader,
|
||||
sourceCut: number,
|
||||
output: SessionFormatMigrationContext,
|
||||
): SessionFormatMigrationStream {
|
||||
let header = sourceHeader
|
||||
const validatedSourceCut = sessionFormatCount(sourceCut, 'Session inherited event count')
|
||||
let inheritedEventCount = validatedSourceCut
|
||||
const stages: Array<{
|
||||
readonly migration: SessionFormatMigration
|
||||
readonly stage: SessionFormatMigrationStage
|
||||
}> = []
|
||||
const plan = this.plan(header.version)
|
||||
for (const [index, migration] of plan.entries()) {
|
||||
const targetHeader = this.advanceHeader(migration, header)
|
||||
let stage: SessionFormatMigrationStage
|
||||
try {
|
||||
migrated = migration.migrate(snapshotSessionFormatArtifact(current, `${migration.name} input`))
|
||||
stage = migration.createStage({
|
||||
sourceHeader: header,
|
||||
targetHeader,
|
||||
sourceInheritedEventCount: inheritedEventCount,
|
||||
sourceKind: index === 0 ? 'decoded' : 'transformed',
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
throwUnsupportedRefusal(migration, error)
|
||||
}
|
||||
current = snapshotSessionFormatArtifact(migrated, `${migration.name} output`)
|
||||
if (current.header.version !== migration.toVersion) {
|
||||
throw new SessionFormatError(`${migration.name} returned v${current.header.version}; expected v${migration.toVersion}`)
|
||||
}
|
||||
try {
|
||||
migration.validateTarget(current)
|
||||
} catch (error: unknown) {
|
||||
throwUnsupportedRefusal(migration, error)
|
||||
header = targetHeader
|
||||
stages.push({ migration, stage })
|
||||
if (index + 1 < plan.length) {
|
||||
const targetCut = stage.headerInheritedEventCount
|
||||
if (targetCut === undefined) {
|
||||
throw new SessionFormatError(`${migration.name} must expose its inherited cut before the next migration`)
|
||||
}
|
||||
inheritedEventCount = targetCut
|
||||
}
|
||||
}
|
||||
current = snapshotSessionFormatArtifact(this.restoreCurrent(current), 'current Session restoration')
|
||||
this.assertCurrent(current)
|
||||
return current
|
||||
return new CompiledSessionFormatMigrationStream(
|
||||
header,
|
||||
validatedSourceCut,
|
||||
stages,
|
||||
output,
|
||||
)
|
||||
}
|
||||
|
||||
migrateHeader(source: SessionFormatHeader): SessionFormatHeader {
|
||||
let current = snapshotSessionFormatHeader(source, 'stored Session header')
|
||||
for (const migration of this.plan(current.version)) {
|
||||
let migrated: SessionFormatHeader
|
||||
try {
|
||||
migrated = migration.migrateHeader(snapshotSessionFormatHeader(current, `${migration.name} header input`))
|
||||
} catch (error: unknown) {
|
||||
throwUnsupportedRefusal(migration, error, 'Session header')
|
||||
}
|
||||
current = snapshotSessionFormatHeader(migrated, `${migration.name} header output`)
|
||||
if (current.version !== migration.toVersion) {
|
||||
throw new SessionFormatError(`${migration.name} header returned v${current.version}; expected v${migration.toVersion}`)
|
||||
}
|
||||
try {
|
||||
migration.validateTargetHeader(current)
|
||||
} catch (error: unknown) {
|
||||
throwUnsupportedRefusal(migration, error, 'Session header')
|
||||
}
|
||||
current = this.advanceHeader(migration, current)
|
||||
}
|
||||
current = snapshotSessionFormatHeader(this.restoreCurrentHeader(current), 'current Session header restoration')
|
||||
if (current.version !== this.currentVersion) {
|
||||
@@ -143,12 +144,104 @@ class CompiledSessionFormatChain implements SessionFormatChain {
|
||||
return current
|
||||
}
|
||||
|
||||
private assertCurrent(artifact: SessionFormatArtifact): void {
|
||||
if (artifact.header.version !== this.currentVersion) {
|
||||
throw new SessionFormatError(
|
||||
`current Session restorer returned v${artifact.header.version}; expected v${this.currentVersion}`,
|
||||
)
|
||||
private advanceHeader(
|
||||
migration: SessionFormatMigration,
|
||||
source: SessionFormatHeader,
|
||||
): SessionFormatHeader {
|
||||
let target: SessionFormatHeader
|
||||
try {
|
||||
target = migration.migrateHeader(snapshotSessionFormatHeader(source, `${migration.name} header input`))
|
||||
} catch (error: unknown) {
|
||||
throwUnsupportedRefusal(migration, error, 'Session header')
|
||||
}
|
||||
const current = snapshotSessionFormatHeader(target, `${migration.name} header output`)
|
||||
if (current.version !== migration.toVersion) {
|
||||
throw new SessionFormatError(`${migration.name} header returned v${current.version}; expected v${migration.toVersion}`)
|
||||
}
|
||||
try {
|
||||
migration.validateTargetHeader(current)
|
||||
} catch (error: unknown) {
|
||||
throwUnsupportedRefusal(migration, error, 'Session header')
|
||||
}
|
||||
return current
|
||||
}
|
||||
}
|
||||
|
||||
interface CompiledMigrationStage {
|
||||
readonly migration: SessionFormatMigration
|
||||
readonly stage: SessionFormatMigrationStage
|
||||
}
|
||||
|
||||
class ChainedMigrationContext implements SessionFormatMigrationContext {
|
||||
constructor(
|
||||
private readonly entry: CompiledMigrationStage,
|
||||
private readonly output: SessionFormatMigrationContext,
|
||||
) {}
|
||||
|
||||
emitEvent(event: SessionFormatEvent): void {
|
||||
try {
|
||||
this.entry.stage.transformEvent(event, this.output)
|
||||
} catch (error: unknown) {
|
||||
throwUnsupportedRefusal(this.entry.migration, error)
|
||||
}
|
||||
}
|
||||
|
||||
emitRun(run: SessionFormatEventRun): void {
|
||||
try {
|
||||
this.entry.stage.transformRun(run, this.output)
|
||||
} catch (error: unknown) {
|
||||
throwUnsupportedRefusal(this.entry.migration, error)
|
||||
}
|
||||
}
|
||||
|
||||
finish(): number {
|
||||
let targetCut: number
|
||||
try {
|
||||
targetCut = this.entry.stage.finish(this.output)
|
||||
} catch (error: unknown) {
|
||||
throwUnsupportedRefusal(this.entry.migration, error)
|
||||
}
|
||||
if (this.entry.stage.headerInheritedEventCount !== undefined
|
||||
&& this.entry.stage.headerInheritedEventCount !== targetCut) {
|
||||
throw new SessionFormatError(`${this.entry.migration.name} changed its predeclared inherited cut`)
|
||||
}
|
||||
return targetCut
|
||||
}
|
||||
}
|
||||
|
||||
class CompiledSessionFormatMigrationStream implements SessionFormatMigrationStream {
|
||||
private readonly input: SessionFormatMigrationContext
|
||||
private readonly stages: readonly ChainedMigrationContext[]
|
||||
|
||||
constructor(
|
||||
readonly header: SessionFormatHeader,
|
||||
private readonly sourceInheritedEventCount: number,
|
||||
entries: readonly CompiledMigrationStage[],
|
||||
output: SessionFormatMigrationContext,
|
||||
) {
|
||||
const stages = new Array<ChainedMigrationContext>(entries.length)
|
||||
let downstream = output
|
||||
for (const [offset, entry] of entries.toReversed().entries()) {
|
||||
const context = new ChainedMigrationContext(entry, downstream)
|
||||
stages[entries.length - offset - 1] = context
|
||||
downstream = context
|
||||
}
|
||||
this.input = downstream
|
||||
this.stages = stages
|
||||
}
|
||||
|
||||
emitEvent(event: SessionFormatEvent): void {
|
||||
this.input.emitEvent(event)
|
||||
}
|
||||
|
||||
emitRun(run: SessionFormatEventRun): void {
|
||||
this.input.emitRun(run)
|
||||
}
|
||||
|
||||
finish(): number {
|
||||
let inheritedEventCount = this.sourceInheritedEventCount
|
||||
for (const stage of this.stages) inheritedEventCount = stage.finish()
|
||||
return inheritedEventCount
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
import type {
|
||||
SessionFormatEvent,
|
||||
SessionFormatEventRun,
|
||||
SessionFormatMigrationContext,
|
||||
} from './types.ts'
|
||||
|
||||
/** Migration output context that expands compact runs into retained events. */
|
||||
export class SessionFormatEventCollector implements SessionFormatMigrationContext {
|
||||
/** Events retained by this collector in source order. */
|
||||
readonly values: SessionFormatEvent[] = []
|
||||
|
||||
/**
|
||||
* Retain one settled event.
|
||||
* @param event - settled event emitted by the upstream stage.
|
||||
*/
|
||||
emitEvent(event: SessionFormatEvent): void {
|
||||
this.values.push(event)
|
||||
}
|
||||
|
||||
/**
|
||||
* Expand one compact run directly into retained events.
|
||||
* @param run - compact event run emitted by the upstream stage.
|
||||
*/
|
||||
emitRun(run: SessionFormatEventRun): void {
|
||||
for (const event of run.expand()) this.values.push(event)
|
||||
}
|
||||
}
|
||||
@@ -1,7 +1,8 @@
|
||||
/** Pure adjacent whole-artifact Session format migration machinery. */
|
||||
/** Pure adjacent streaming Session format migration machinery. */
|
||||
|
||||
export * from './chain.ts'
|
||||
export * from './catalog.ts'
|
||||
export * from './context.ts'
|
||||
export * from './error.ts'
|
||||
export * from './filename.ts'
|
||||
export * from './json.ts'
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
import { deepFreeze, snapshotJsonValue } from '@deepseek-ai/dsh-util-values'
|
||||
import { SessionFormatError } from './error.ts'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatHeader,
|
||||
SessionFormatJsonObject,
|
||||
SessionFormatJsonValue,
|
||||
} from './types.ts'
|
||||
|
||||
@@ -78,42 +76,6 @@ export function snapshotSessionFormatJson(value: unknown, label = 'Session value
|
||||
return deepFreeze(snapshot) as SessionFormatJsonValue
|
||||
}
|
||||
|
||||
/**
|
||||
* Snapshot one complete artifact and validate its shared coordinates.
|
||||
* @param artifact - borrowed logical artifact.
|
||||
* @param label - diagnostic subject.
|
||||
* @returns immutable detached artifact.
|
||||
*/
|
||||
export function snapshotSessionFormatArtifact(
|
||||
artifact: SessionFormatArtifact,
|
||||
label = 'Session artifact',
|
||||
): SessionFormatArtifact {
|
||||
const snapshot = snapshotSessionFormatJson(artifact, label) as SessionFormatJsonObject
|
||||
const header = snapshot['header']
|
||||
const inheritedEventCount = snapshot['inheritedEventCount']
|
||||
const events = snapshot['events']
|
||||
if (!isSessionFormatJsonObject(header)) throw new SessionFormatError(`${label} header must be a JSON object`)
|
||||
inspectSessionFormatVersion(header)
|
||||
sessionFormatCount(inheritedEventCount, `${label} inheritedEventCount`)
|
||||
if (!Array.isArray(events)) throw new SessionFormatError(`${label} events must be an array`)
|
||||
for (let index = 0; index < events.length; index += 1) {
|
||||
const event: unknown = events[index]
|
||||
if (!isSessionFormatJsonObject(event)) throw new SessionFormatError(`${label} event ${index} must be a JSON object`)
|
||||
if (event['seq'] !== index) {
|
||||
throw new SessionFormatError(`${label} event ${index} has non-dense seq ${String(event['seq'])}`)
|
||||
}
|
||||
if (typeof event['type'] !== 'string' || event['type'].length === 0) {
|
||||
throw new SessionFormatError(`${label} event ${index} type must be a non-empty string`)
|
||||
}
|
||||
sessionFormatSafeInteger(event['time'], `${label} event ${index} time`)
|
||||
if (!Object.hasOwn(event, 'data')) throw new SessionFormatError(`${label} event ${index} lacks data`)
|
||||
}
|
||||
if (inheritedEventCount as number > events.length) {
|
||||
throw new SessionFormatError(`${label} inheritedEventCount exceeds its event count`)
|
||||
}
|
||||
return snapshot as unknown as SessionFormatArtifact
|
||||
}
|
||||
|
||||
/**
|
||||
* Snapshot one logical header without inspecting an event body.
|
||||
* @param header - borrowed logical header.
|
||||
|
||||
@@ -41,65 +41,126 @@ export interface SessionFormatArtifact {
|
||||
readonly events: readonly SessionFormatEvent[]
|
||||
}
|
||||
|
||||
/** One independently maintained adjacent whole-artifact migration. */
|
||||
/** One independently maintained adjacent streaming migration. */
|
||||
export interface SessionFormatMigration {
|
||||
readonly name: string
|
||||
readonly fromVersion: number
|
||||
readonly toVersion: number
|
||||
/** Convert one header without reading event bodies. */
|
||||
migrateHeader(header: SessionFormatHeader): SessionFormatHeader
|
||||
/** Convert one detached complete artifact to exactly {@link toVersion}. */
|
||||
migrate(artifact: SessionFormatArtifact): SessionFormatArtifact
|
||||
/** Refuse any artifact that the adjacent target writer cannot emit. */
|
||||
validateTarget(artifact: SessionFormatArtifact): void
|
||||
/** Create the stateful body stage for one source artifact. */
|
||||
createStage(input: SessionFormatMigrationStageInput): SessionFormatMigrationStage
|
||||
/** Refuse any header that the adjacent target writer cannot emit. */
|
||||
validateTargetHeader(header: SessionFormatHeader): void
|
||||
}
|
||||
|
||||
/** Headers and inherited cut supplied when one adjacent body stage is created. */
|
||||
export interface SessionFormatMigrationStageInput {
|
||||
/** Validated source metadata for this adjacent edge. */
|
||||
readonly sourceHeader: SessionFormatHeader
|
||||
/** Validated target metadata produced by this edge's header migration. */
|
||||
readonly targetHeader: SessionFormatHeader
|
||||
/** Exact inherited prefix length in source coordinates. */
|
||||
readonly sourceInheritedEventCount: number
|
||||
/** Whether this edge consumes physical decode output or a prior migration's validated output. */
|
||||
readonly sourceKind: 'decoded' | 'transformed'
|
||||
}
|
||||
|
||||
/** Inputs that compile the unique complete migration chain. */
|
||||
export interface SessionFormatChainOptions {
|
||||
readonly currentVersion: number
|
||||
readonly migrations: readonly SessionFormatMigration[]
|
||||
/** Restore and validate a detached current artifact through the current parser. */
|
||||
readonly restoreCurrent: (artifact: SessionFormatArtifact) => SessionFormatArtifact
|
||||
/** Restore and validate a detached current header without reading event bodies. */
|
||||
readonly restoreCurrentHeader: (header: SessionFormatHeader) => SessionFormatHeader
|
||||
}
|
||||
|
||||
/** Pure adjacent planner and whole-artifact migration runner. */
|
||||
/** Pure adjacent planner and streaming migration compiler. */
|
||||
export interface SessionFormatChain {
|
||||
readonly currentVersion: number
|
||||
/** Return the complete ordered plan from one supported stored version. */
|
||||
plan(fromVersion: number): readonly SessionFormatMigration[]
|
||||
/** Restore current input directly or migrate old input entirely in memory. */
|
||||
migrate(artifact: SessionFormatArtifact): SessionFormatArtifact
|
||||
/** Compile the complete migration stage chain for one decoded source artifact. */
|
||||
createStream(
|
||||
header: SessionFormatHeader,
|
||||
inheritedEventCount: number,
|
||||
context: SessionFormatMigrationContext,
|
||||
): SessionFormatMigrationStream
|
||||
/** Convert only a supported header to the current logical representation. */
|
||||
migrateHeader(header: SessionFormatHeader): SessionFormatHeader
|
||||
}
|
||||
|
||||
/** Physical JSON records emitted by one format-specific codec. */
|
||||
export interface EncodedSessionFormatArtifact {
|
||||
readonly header: SessionFormatJsonObject
|
||||
readonly rows: readonly SessionFormatJsonObject[]
|
||||
}
|
||||
|
||||
/** Options that affect only physical row layout, never logical contents. */
|
||||
export interface SessionFormatEncodeOptions {
|
||||
readonly packChunks: boolean
|
||||
}
|
||||
/** Physical-row failure policy selected once for one restore. */
|
||||
export type SessionFormatRecovery = 'strict' | 'recoverable'
|
||||
|
||||
/** Pure physical JSON codec frozen with one released Session format. */
|
||||
export interface SessionFormatCodec {
|
||||
readonly version: number
|
||||
/** Decode one physical header into body-independent logical metadata. */
|
||||
decodeHeader(value: unknown): SessionFormatHeader
|
||||
/** Decode one complete physical header and row sequence into logical events. */
|
||||
decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]): SessionFormatArtifact
|
||||
/** Decode the row-atomic recoverable prefix used by crash-tail repair. */
|
||||
decodeRecoverableArtifact(
|
||||
headerValue: unknown,
|
||||
rowValues: readonly unknown[],
|
||||
): SessionFormatArtifact
|
||||
/** Create one row-at-a-time decoder with an explicit failure policy. */
|
||||
createDecoder(headerValue: unknown, recovery: SessionFormatRecovery): SessionFormatArtifactDecoder
|
||||
}
|
||||
|
||||
/** Stateful physical-row decoder used by streaming persistence restores. */
|
||||
export interface SessionFormatArtifactDecoder {
|
||||
readonly header: SessionFormatHeader
|
||||
/** Inherited cut known before body decoding; current formats may derive it at EOF. */
|
||||
readonly headerInheritedEventCount?: number
|
||||
/** Decode one physical row and synchronously emit its events or compact run. */
|
||||
decodeRow(
|
||||
rowValue: unknown,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void
|
||||
/** Finish row validation and return the exact inherited cut. */
|
||||
finish(context: SessionFormatMigrationContext): number
|
||||
}
|
||||
|
||||
/** Stateless physical record encoder for the installed current format. */
|
||||
export interface SessionFormatCurrentEncoder {
|
||||
/** Encode the physical header record for one current artifact. */
|
||||
encodeHeader(header: SessionFormatHeader, inheritedEventCount: number): SessionFormatJsonObject
|
||||
/** Encode one current logical event as one physical record. */
|
||||
encodeEvent(event: SessionFormatEvent): SessionFormatJsonObject
|
||||
}
|
||||
|
||||
/** A codec-owned compact run that adjacent migrations may consume without expanding. */
|
||||
export interface SessionFormatEventRun {
|
||||
readonly runType: string
|
||||
readonly firstSeq: number
|
||||
readonly eventCount: number
|
||||
/** Expand the run for a migration that has no direct handler. */
|
||||
expand(): Iterable<SessionFormatEvent>
|
||||
}
|
||||
|
||||
/** Synchronous output channel owned by a compiled migration stream. */
|
||||
export interface SessionFormatMigrationContext {
|
||||
/** Deliver one settled event to the next stage before returning. */
|
||||
emitEvent(event: SessionFormatEvent): void
|
||||
/** Deliver one compact event run to the next stage before returning. */
|
||||
emitRun(run: SessionFormatEventRun): void
|
||||
}
|
||||
|
||||
/** Stateful adjacent migration stage used by streaming persistence restores. */
|
||||
export interface SessionFormatMigrationStage {
|
||||
/** Target inherited cut when it is unchanged and known before EOF. */
|
||||
readonly headerInheritedEventCount?: number
|
||||
/** Transform one source event and synchronously emit every settled target item. */
|
||||
transformEvent(
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void
|
||||
/** Transform one compact source run without requiring an intermediate expansion array. */
|
||||
transformRun(
|
||||
run: SessionFormatEventRun,
|
||||
context: SessionFormatMigrationContext,
|
||||
): void
|
||||
/** Emit trailing target items and return the exact target inherited cut. */
|
||||
finish(context: SessionFormatMigrationContext): number
|
||||
}
|
||||
|
||||
/** One composed migration chain that emits settled current events to its owner. */
|
||||
export interface SessionFormatMigrationStream extends SessionFormatMigrationContext {
|
||||
readonly header: SessionFormatHeader
|
||||
/** Settle all migration stages and return the exact current inherited cut. */
|
||||
finish(): number
|
||||
}
|
||||
|
||||
/** Header-only classification that never inspects event rows. */
|
||||
@@ -127,8 +188,22 @@ export type SessionFormatHeaderReadResult =
|
||||
/** Inputs for a build-static physical codec and migration catalog. */
|
||||
export interface SessionFormatCatalogOptions extends SessionFormatChainOptions {
|
||||
readonly codecs: readonly SessionFormatCodec[]
|
||||
/** Encode one already-restored current artifact through its format-specific writer. */
|
||||
readonly encodeCurrentArtifact: (artifact: SessionFormatArtifact) => EncodedSessionFormatArtifact
|
||||
/** Restore and validate a complete current artifact. */
|
||||
readonly restoreCurrent: (artifact: SessionFormatArtifact) => SessionFormatArtifact
|
||||
/** Encode current records without materializing an artifact-sized row array. */
|
||||
readonly currentEncoder: SessionFormatCurrentEncoder
|
||||
/** Validate an exclusively owned transformed artifact without copying or freezing it. */
|
||||
readonly restoreTransformedCurrent: (artifact: SessionFormatArtifact) => SessionFormatArtifact
|
||||
}
|
||||
|
||||
/** Policies applied by one physical-row restore. */
|
||||
export interface SessionFormatRestoreOptions {
|
||||
readonly recovery: SessionFormatRecovery
|
||||
/**
|
||||
* `current` applies all installed current-format validation. `transformed` applies
|
||||
* released current-format validation only after migration; current input receives only codec validation.
|
||||
*/
|
||||
readonly validation: 'transformed' | 'current'
|
||||
}
|
||||
|
||||
/** Build-static physical dispatch and adjacent migration catalog. */
|
||||
@@ -136,15 +211,20 @@ export interface SessionFormatCatalog {
|
||||
readonly currentVersion: number
|
||||
/** Classify and translate one header without reading event rows. */
|
||||
readHeader(headerValue: unknown): SessionFormatHeaderReadResult
|
||||
/** Dispatch a complete physical JSON artifact through its frozen version codec. */
|
||||
decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]): SessionFormatArtifact
|
||||
/** Dispatch a physical artifact through its released row-prefix recovery rules. */
|
||||
decodeRecoverableArtifact(
|
||||
headerValue: unknown,
|
||||
rowValues: readonly unknown[],
|
||||
): SessionFormatArtifact
|
||||
/** Restore current input directly or run all required adjacent migrations in memory. */
|
||||
migrate(artifact: SessionFormatArtifact): SessionFormatArtifact
|
||||
/** Encode one current artifact that `migrate` returned or a live Session produced; it is not re-validated here. */
|
||||
encodeCurrent(artifact: SessionFormatArtifact): EncodedSessionFormatArtifact
|
||||
/** Create one single-pass physical-row restore into current logical events. */
|
||||
createRestore(headerValue: unknown, options: SessionFormatRestoreOptions): SessionFormatRestore
|
||||
/** Encode one current physical header record. */
|
||||
encodeCurrentHeader(header: SessionFormatHeader, inheritedEventCount: number): SessionFormatJsonObject
|
||||
/** Encode one current physical event record. */
|
||||
encodeCurrentEvent(event: SessionFormatEvent): SessionFormatJsonObject
|
||||
}
|
||||
|
||||
/** One caller-owned physical-row restore whose final value is a current logical artifact. */
|
||||
export interface SessionFormatRestore {
|
||||
/** Current logical header available before body decoding. */
|
||||
readonly header: SessionFormatHeader
|
||||
/** Decode one physical row in file order. */
|
||||
decodeRow(rowValue: unknown): void
|
||||
/** Finish every decoder and migration stage and return the current artifact. */
|
||||
finish(): SessionFormatArtifact
|
||||
}
|
||||
@@ -1,132 +1,228 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import {
|
||||
createSessionFormatCatalog,
|
||||
defineSessionFormatMigration,
|
||||
SessionFormatUnsupportedMigrationError,
|
||||
type SessionFormatArtifact,
|
||||
type SessionFormatCatalogOptions,
|
||||
type SessionFormatCodec,
|
||||
type SessionFormatCurrentEncoder,
|
||||
type SessionFormatEvent,
|
||||
type SessionFormatEventRun,
|
||||
type SessionFormatMigration,
|
||||
type SessionFormatMigrationContext,
|
||||
type SessionFormatMigrationStageInput,
|
||||
} from '../src/index.ts'
|
||||
|
||||
function codec(version: number) {
|
||||
function codec(version: number, inheritedEventCount = 0): SessionFormatCodec & SessionFormatCurrentEncoder {
|
||||
return {
|
||||
version,
|
||||
decodeHeader(value: unknown) {
|
||||
return value as never
|
||||
return value as SessionFormatArtifact['header']
|
||||
},
|
||||
decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]) {
|
||||
const header = headerValue as SessionFormatArtifact['header']
|
||||
createDecoder(headerValue: unknown) {
|
||||
return {
|
||||
header,
|
||||
inheritedEventCount: 0,
|
||||
events: rowValues as SessionFormatArtifact['events'],
|
||||
header: headerValue as SessionFormatArtifact['header'],
|
||||
headerInheritedEventCount: inheritedEventCount,
|
||||
decodeRow(rowValue: unknown, context: SessionFormatMigrationContext) {
|
||||
context.emitEvent(rowValue as SessionFormatEvent)
|
||||
},
|
||||
finish: () => inheritedEventCount,
|
||||
}
|
||||
},
|
||||
decodeRecoverableArtifact(headerValue: unknown, rowValues: readonly unknown[]) {
|
||||
return this.decodeArtifact(headerValue, rowValues)
|
||||
encodeHeader(header) {
|
||||
return header
|
||||
},
|
||||
encodeArtifact(artifact: SessionFormatArtifact) {
|
||||
return { header: artifact.header, rows: artifact.events }
|
||||
encodeEvent(event) {
|
||||
return event
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
describe('Session format catalog', () => {
|
||||
it('classifies headers without reading bodies and dispatches physical values by version', () => {
|
||||
const migrate = vi.fn((artifact: SessionFormatArtifact): SessionFormatArtifact => ({
|
||||
...artifact,
|
||||
header: { ...artifact.header, version: 1 },
|
||||
}))
|
||||
const catalog = createSessionFormatCatalog({
|
||||
currentVersion: 1,
|
||||
codecs: [codec(0), codec(1)],
|
||||
encodeCurrentArtifact: (artifact: SessionFormatArtifact) => codec(1).encodeArtifact(artifact),
|
||||
migrations: [defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1',
|
||||
fromVersion: 0,
|
||||
toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate,
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader: () => {},
|
||||
})],
|
||||
restoreCurrent: artifact => artifact,
|
||||
restoreCurrentHeader: header => header,
|
||||
})
|
||||
const oldHeader = {
|
||||
version: 0,
|
||||
id: 'old',
|
||||
createdAt: 1,
|
||||
isSeeded: true,
|
||||
delegationDepth: 0,
|
||||
} as const
|
||||
function identityMigrationStage(inheritedEventCount: number) {
|
||||
return {
|
||||
headerInheritedEventCount: inheritedEventCount,
|
||||
transformEvent(
|
||||
event: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
) {
|
||||
context.emitEvent(event)
|
||||
},
|
||||
transformRun(
|
||||
run: SessionFormatEventRun,
|
||||
context: SessionFormatMigrationContext,
|
||||
) {
|
||||
context.emitRun(run)
|
||||
},
|
||||
finish: () => inheritedEventCount,
|
||||
}
|
||||
}
|
||||
|
||||
expect(catalog.readHeader(oldHeader)).toEqual({
|
||||
function edge(overrides: Partial<SessionFormatMigration> = {}): SessionFormatMigration {
|
||||
return {
|
||||
name: '@test/v0-to-v1',
|
||||
fromVersion: 0,
|
||||
toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
createStage: ({ sourceInheritedEventCount }) => identityMigrationStage(sourceInheritedEventCount),
|
||||
validateTargetHeader: () => {},
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
function catalog(
|
||||
migration: SessionFormatMigration = edge(),
|
||||
restoreVersion = 1,
|
||||
sourceCodec: SessionFormatCodec = codec(0),
|
||||
currentEncoder: SessionFormatCurrentEncoder = codec(1),
|
||||
restoreCurrentHeader: SessionFormatCatalogOptions['restoreCurrentHeader'] = header => header,
|
||||
) {
|
||||
const currentCodec = codec(1)
|
||||
return createSessionFormatCatalog({
|
||||
currentVersion: 1,
|
||||
codecs: [sourceCodec, currentCodec],
|
||||
currentEncoder,
|
||||
migrations: [migration],
|
||||
restoreCurrent: artifact => ({
|
||||
...artifact,
|
||||
header: { ...artifact.header, version: restoreVersion },
|
||||
}),
|
||||
restoreTransformedCurrent: artifact => ({
|
||||
...artifact,
|
||||
header: { ...artifact.header, version: restoreVersion },
|
||||
}),
|
||||
restoreCurrentHeader,
|
||||
})
|
||||
}
|
||||
|
||||
const oldHeader = {
|
||||
version: 0,
|
||||
id: 'old',
|
||||
createdAt: 1,
|
||||
isSeeded: false,
|
||||
delegationDepth: 0,
|
||||
} as const
|
||||
|
||||
const event = { type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } } as const
|
||||
|
||||
function throwUnknown(value: unknown): never {
|
||||
throw value
|
||||
}
|
||||
|
||||
describe('Session format catalog', () => {
|
||||
it('classifies headers and restores physical rows through the compiled chain', () => {
|
||||
const createStage = vi.fn(({ sourceInheritedEventCount }: SessionFormatMigrationStageInput) =>
|
||||
identityMigrationStage(sourceInheritedEventCount))
|
||||
const current = catalog(edge({ createStage }))
|
||||
|
||||
expect(current.readHeader(oldHeader)).toEqual({
|
||||
status: 'migration-required',
|
||||
storedVersion: 0,
|
||||
targetVersion: 1,
|
||||
header: { ...oldHeader, version: 1 },
|
||||
})
|
||||
expect(catalog.readHeader({ version: 2 })).toMatchObject({
|
||||
status: 'unsupported',
|
||||
storedVersion: 2,
|
||||
targetVersion: 1,
|
||||
expect(current.readHeader({ version: 2 })).toMatchObject({
|
||||
status: 'unsupported', storedVersion: 2, targetVersion: 1,
|
||||
})
|
||||
expect(catalog.readHeader({ version: 'broken' })).toMatchObject({ status: 'malformed', targetVersion: 1 })
|
||||
expect(current.readHeader({ ...oldHeader, version: 1 })).toMatchObject({
|
||||
status: 'current', storedVersion: 1, targetVersion: 1,
|
||||
})
|
||||
expect(current.readHeader({ version: 'broken' })).toMatchObject({ status: 'malformed', targetVersion: 1 })
|
||||
|
||||
const decoded = catalog.decodeArtifact(oldHeader, [
|
||||
{ type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } },
|
||||
])
|
||||
expect(decoded.header.version).toBe(0)
|
||||
expect(catalog.migrate(decoded).header.version).toBe(1)
|
||||
expect(migrate).toHaveBeenCalledOnce()
|
||||
const restore = current.createRestore(oldHeader, { recovery: 'strict', validation: 'current' })
|
||||
restore.decodeRow(event)
|
||||
expect(restore.finish()).toMatchObject({ header: { version: 1 }, events: [event] })
|
||||
expect(createStage).toHaveBeenCalledOnce()
|
||||
})
|
||||
|
||||
it('uses the current codec directly for recovery and encoding', () => {
|
||||
const currentCodec = codec(1)
|
||||
const catalog = createSessionFormatCatalog({
|
||||
it('restores current rows without a migration and encodes one record at a time', () => {
|
||||
const current = catalog()
|
||||
const header = { ...oldHeader, version: 1 }
|
||||
const restore = current.createRestore(header, { recovery: 'strict', validation: 'current' })
|
||||
restore.decodeRow(event)
|
||||
|
||||
expect(restore.finish()).toEqual({ header, inheritedEventCount: 0, events: [event] })
|
||||
expect(current.encodeCurrentHeader(header, 0)).toEqual(header)
|
||||
expect(current.encodeCurrentEvent(event)).toEqual(event)
|
||||
expect(() => current.encodeCurrentHeader(oldHeader, 0)).toThrow(/requires Session format v1/)
|
||||
|
||||
const physicalOnly = current.createRestore(header, { recovery: 'strict', validation: 'transformed' })
|
||||
physicalOnly.decodeRow(event)
|
||||
expect(physicalOnly.finish()).toEqual({ header, inheritedEventCount: 0, events: [event] })
|
||||
|
||||
const transformed = current.createRestore(oldHeader, { recovery: 'strict', validation: 'transformed' })
|
||||
transformed.decodeRow(event)
|
||||
expect(transformed.finish()).toMatchObject({ header: { version: 1 }, events: [event] })
|
||||
})
|
||||
|
||||
it.each([
|
||||
new Error('target artifact is incompatible'),
|
||||
'non-Error target artifact refusal',
|
||||
])('classifies migrated target validation failures as unsupported', (targetFailure) => {
|
||||
const current = createSessionFormatCatalog({
|
||||
currentVersion: 1,
|
||||
codecs: [codec(0), currentCodec],
|
||||
encodeCurrentArtifact: artifact => currentCodec.encodeArtifact(artifact),
|
||||
migrations: [defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader: () => {},
|
||||
})],
|
||||
codecs: [codec(0), codec(1)],
|
||||
currentEncoder: codec(1),
|
||||
migrations: [edge()],
|
||||
restoreCurrent: artifact => artifact,
|
||||
restoreTransformedCurrent: () => { throw targetFailure },
|
||||
restoreCurrentHeader: header => header,
|
||||
})
|
||||
const current = {
|
||||
header: {
|
||||
version: 1, id: 'current', createdAt: 1, isSeeded: false, delegationDepth: 0,
|
||||
},
|
||||
inheritedEventCount: 0,
|
||||
events: [{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }],
|
||||
} satisfies SessionFormatArtifact
|
||||
const restore = current.createRestore(oldHeader, { recovery: 'strict', validation: 'transformed' })
|
||||
restore.decodeRow(event)
|
||||
|
||||
expect(catalog.readHeader(current.header)).toMatchObject({ status: 'current', header: current.header })
|
||||
expect(catalog.decodeRecoverableArtifact(current.header, current.events)).toEqual(current)
|
||||
expect(catalog.encodeCurrent(current)).toEqual({
|
||||
header: current.header,
|
||||
rows: current.events,
|
||||
})
|
||||
expect(() => catalog.encodeCurrent({ ...current, header: { ...current.header, version: 0 } }))
|
||||
.toThrow(/requires Session format v1/)
|
||||
try {
|
||||
restore.finish()
|
||||
throw new Error('expected transformed target validation to fail')
|
||||
} catch (error: unknown) {
|
||||
expect(error).toBeInstanceOf(SessionFormatUnsupportedMigrationError)
|
||||
expect((error as Error).cause).toBe(targetFailure)
|
||||
}
|
||||
})
|
||||
|
||||
it('preserves current-validation failures and explicit migration refusals', () => {
|
||||
const currentFailure = new Error('installed current validation failed')
|
||||
const current = createSessionFormatCatalog({
|
||||
currentVersion: 1,
|
||||
codecs: [codec(0), codec(1)],
|
||||
currentEncoder: codec(1),
|
||||
migrations: [edge()],
|
||||
restoreCurrent: () => { throw currentFailure },
|
||||
restoreTransformedCurrent: artifact => artifact,
|
||||
restoreCurrentHeader: header => header,
|
||||
}).createRestore(oldHeader, { recovery: 'strict', validation: 'current' })
|
||||
expect(() => current.finish()).toThrow(currentFailure)
|
||||
|
||||
const explicit = new SessionFormatUnsupportedMigrationError('explicit target refusal')
|
||||
const transformed = createSessionFormatCatalog({
|
||||
currentVersion: 1,
|
||||
codecs: [codec(0), codec(1)],
|
||||
currentEncoder: codec(1),
|
||||
migrations: [edge()],
|
||||
restoreCurrent: artifact => artifact,
|
||||
restoreTransformedCurrent: () => { throw explicit },
|
||||
restoreCurrentHeader: header => header,
|
||||
}).createRestore(oldHeader, { recovery: 'strict', validation: 'transformed' })
|
||||
expect(() => transformed.finish()).toThrow(explicit)
|
||||
})
|
||||
|
||||
it('rejects a current encoder that returns a non-current header', () => {
|
||||
const badEncoder: SessionFormatCurrentEncoder = {
|
||||
...codec(1),
|
||||
encodeHeader: header => ({ ...header, version: 0 }),
|
||||
}
|
||||
const current = catalog(edge(), 1, codec(0), badEncoder)
|
||||
|
||||
expect(() => current.encodeCurrentHeader({ ...oldHeader, version: 1 }, 0))
|
||||
.toThrow(/non-current header/)
|
||||
})
|
||||
|
||||
it('rejects duplicate, missing, and future codec declarations', () => {
|
||||
const edge = defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader: () => {},
|
||||
})
|
||||
const options = {
|
||||
currentVersion: 1,
|
||||
migrations: [edge],
|
||||
encodeCurrentArtifact: (artifact: SessionFormatArtifact) => codec(1).encodeArtifact(artifact),
|
||||
migrations: [edge()],
|
||||
currentEncoder: codec(1),
|
||||
restoreCurrent: (value: SessionFormatArtifact) => value,
|
||||
restoreTransformedCurrent: (value: SessionFormatArtifact) => value,
|
||||
restoreCurrentHeader: (value: SessionFormatArtifact['header']) => value,
|
||||
}
|
||||
expect(() => createSessionFormatCatalog({ ...options, codecs: [codec(0), codec(0), codec(1)] }))
|
||||
@@ -136,93 +232,137 @@ describe('Session format catalog', () => {
|
||||
.toThrow(/codec v2 is newer/)
|
||||
})
|
||||
|
||||
it('returns malformed descriptors for supported headers that their codec refuses', () => {
|
||||
const refusing: SessionFormatCodec = {
|
||||
it('preserves unsupported header and migration failures', () => {
|
||||
const refusingCodec: SessionFormatCodec = {
|
||||
...codec(0),
|
||||
decodeHeader: () => { throw 'bad header' },
|
||||
decodeHeader: () => { throw new Error('bad header') },
|
||||
createDecoder: () => { throw new Error('bad body') },
|
||||
}
|
||||
const catalog = createSessionFormatCatalog({
|
||||
currentVersion: 1,
|
||||
codecs: [refusing, codec(1)],
|
||||
encodeCurrentArtifact: artifact => codec(1).encodeArtifact(artifact),
|
||||
migrations: [defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader: () => {},
|
||||
})],
|
||||
restoreCurrent: artifact => artifact,
|
||||
restoreCurrentHeader: header => header,
|
||||
const current = catalog(edge(), 1, refusingCodec)
|
||||
expect(current.readHeader(oldHeader)).toMatchObject({ status: 'malformed', reason: 'bad header' })
|
||||
expect(() => current.createRestore(oldHeader, { recovery: 'strict', validation: 'current' }))
|
||||
.toThrow('bad body')
|
||||
expect(() => current.createRestore({ version: 2 }, { recovery: 'strict', validation: 'current' }))
|
||||
.toThrow(/newer/)
|
||||
|
||||
const nonError = catalog({
|
||||
...edge(),
|
||||
}, 1, {
|
||||
...codec(0),
|
||||
decodeHeader: () => throwUnknown('non-Error header failure'),
|
||||
})
|
||||
expect(catalog.readHeader({ version: 0 })).toEqual({
|
||||
status: 'malformed', storedVersion: 0, targetVersion: 1, reason: 'bad header',
|
||||
expect(nonError.readHeader(oldHeader)).toMatchObject({
|
||||
status: 'malformed', reason: 'non-Error header failure',
|
||||
})
|
||||
expect(() => catalog.decodeArtifact({ version: 2 }, [])).toThrow(/newer/)
|
||||
expect(() => catalog.decodeRecoverableArtifact({ version: 2 }, [])).toThrow(/newer/)
|
||||
})
|
||||
|
||||
it('rejects a current encoder that returns a non-current header', () => {
|
||||
const bad = {
|
||||
...codec(1),
|
||||
encodeArtifact: (artifact: SessionFormatArtifact) => ({
|
||||
header: { ...artifact.header, version: 0 }, rows: artifact.events,
|
||||
it('classifies migrated header refusals as unsupported and current corruption as malformed', () => {
|
||||
const unsupported = catalog(edge({
|
||||
validateTargetHeader: () => { throw new Error('target header lacks marker') },
|
||||
}))
|
||||
expect(unsupported.readHeader(oldHeader)).toMatchObject({
|
||||
status: 'unsupported',
|
||||
reason: expect.stringContaining('target header lacks marker') as string,
|
||||
})
|
||||
|
||||
const malformed = catalog(
|
||||
edge(),
|
||||
1,
|
||||
codec(0),
|
||||
codec(1),
|
||||
() => { throw new Error('current header is corrupt') },
|
||||
)
|
||||
expect(malformed.readHeader({ ...oldHeader, version: 1 })).toMatchObject({
|
||||
status: 'malformed',
|
||||
reason: 'current header is corrupt',
|
||||
})
|
||||
})
|
||||
|
||||
it('enforces decoder, stage, and restorer lifecycle results', () => {
|
||||
const changedCut = edge({
|
||||
createStage: () => ({
|
||||
headerInheritedEventCount: 1,
|
||||
transformEvent: (candidate, context) => { context.emitEvent(candidate) },
|
||||
transformRun: (candidate, context) => { context.emitRun(candidate) },
|
||||
finish: () => 0,
|
||||
}),
|
||||
}
|
||||
const catalog = createSessionFormatCatalog({
|
||||
currentVersion: 1,
|
||||
codecs: [codec(0), bad],
|
||||
encodeCurrentArtifact: bad.encodeArtifact,
|
||||
migrations: [defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader: () => {},
|
||||
})],
|
||||
restoreCurrent: artifact => artifact,
|
||||
restoreCurrentHeader: header => header,
|
||||
})
|
||||
const current = {
|
||||
header: { version: 1, id: 'bad', createdAt: 1, isSeeded: false, delegationDepth: 0 },
|
||||
inheritedEventCount: 0,
|
||||
events: [],
|
||||
const changed = catalog(changedCut, 1, codec(0, 1)).createRestore(
|
||||
{ ...oldHeader, isSeeded: true },
|
||||
{ recovery: 'strict', validation: 'current' },
|
||||
)
|
||||
expect(() => changed.finish()).toThrow(/changed its predeclared inherited cut/)
|
||||
|
||||
const trailing = catalog(edge({
|
||||
createStage: () => ({
|
||||
headerInheritedEventCount: 0,
|
||||
transformEvent: () => {},
|
||||
transformRun: () => {},
|
||||
finish: (context) => { context.emitEvent(event); return 0 },
|
||||
}),
|
||||
})).createRestore(oldHeader, { recovery: 'strict', validation: 'current' })
|
||||
expect(trailing.finish().events).toEqual([event])
|
||||
|
||||
const wrongVersion = catalog(edge(), 0).createRestore(
|
||||
oldHeader,
|
||||
{ recovery: 'strict', validation: 'current' },
|
||||
)
|
||||
expect(() => wrongVersion.finish()).toThrow(/returned v0/)
|
||||
|
||||
const changedDecoderCut: SessionFormatCodec = {
|
||||
...codec(0),
|
||||
createDecoder(headerValue) {
|
||||
return {
|
||||
header: headerValue as SessionFormatArtifact['header'],
|
||||
headerInheritedEventCount: 0,
|
||||
decodeRow: () => {},
|
||||
finish: () => 1,
|
||||
}
|
||||
},
|
||||
}
|
||||
expect(() => catalog.encodeCurrent(current)).toThrow(/non-current header/)
|
||||
const changedDecoder = catalog(edge(), 1, changedDecoderCut).createRestore(
|
||||
oldHeader,
|
||||
{ recovery: 'strict', validation: 'current' },
|
||||
)
|
||||
expect(() => changedDecoder.finish()).toThrow(/decoder changed its predeclared inherited cut/)
|
||||
|
||||
const deferredCut: SessionFormatCodec = {
|
||||
...codec(0),
|
||||
createDecoder(headerValue) {
|
||||
return {
|
||||
header: headerValue as SessionFormatArtifact['header'],
|
||||
decodeRow: () => {},
|
||||
finish: () => 0,
|
||||
}
|
||||
},
|
||||
}
|
||||
expect(() => catalog(edge(), 1, deferredCut).createRestore(
|
||||
oldHeader,
|
||||
{ recovery: 'strict', validation: 'current' },
|
||||
)).toThrow(/must expose its inherited cut/)
|
||||
})
|
||||
|
||||
it('classifies malformed migrated and direct-current logical headers', () => {
|
||||
const validateTargetHeader = (header: SessionFormatArtifact['header']): void => {
|
||||
if (header['targetMarker'] !== true) throw new Error('latest header lacks marker')
|
||||
it('expands an unhandled compact run without an intermediate array', () => {
|
||||
const run: SessionFormatEventRun = {
|
||||
runType: 'test-run', firstSeq: 0, eventCount: 1, expand: function* () { yield event },
|
||||
}
|
||||
const migration = defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader,
|
||||
})
|
||||
const catalog = createSessionFormatCatalog({
|
||||
currentVersion: 1,
|
||||
codecs: [codec(0), codec(1)],
|
||||
encodeCurrentArtifact: artifact => codec(1).encodeArtifact(artifact),
|
||||
migrations: [migration],
|
||||
restoreCurrent: artifact => artifact,
|
||||
restoreCurrentHeader: (header: SessionFormatArtifact['header']) => {
|
||||
if (typeof header.id !== 'string') throw new Error('latest header lacks id')
|
||||
return header
|
||||
const source: SessionFormatCodec = {
|
||||
...codec(0),
|
||||
createDecoder(headerValue) {
|
||||
return {
|
||||
header: headerValue as SessionFormatArtifact['header'],
|
||||
headerInheritedEventCount: 0,
|
||||
decodeRow(_rowValue, context) { context.emitRun(run) },
|
||||
finish: () => 0,
|
||||
}
|
||||
},
|
||||
})
|
||||
}
|
||||
const restore = catalog(edge(), 1, source).createRestore(
|
||||
oldHeader,
|
||||
{ recovery: 'strict', validation: 'current' },
|
||||
)
|
||||
restore.decodeRow({})
|
||||
|
||||
const migrated = catalog.readHeader({
|
||||
version: 0, id: 'old', createdAt: 1, isSeeded: false, delegationDepth: 0,
|
||||
})
|
||||
expect(migrated.status).toBe('unsupported')
|
||||
if (migrated.status !== 'unsupported') throw new Error('expected unsupported header')
|
||||
expect(migrated.reason).toContain('latest header lacks marker')
|
||||
const current = catalog.readHeader({ version: 1 })
|
||||
expect(current.status).toBe('malformed')
|
||||
if (current.status !== 'malformed') throw new Error('expected malformed current header')
|
||||
expect(current.reason).toMatch(/id/)
|
||||
expect(restore.finish().events).toEqual([event])
|
||||
})
|
||||
})
|
||||
@@ -2,20 +2,74 @@ import { describe, expect, it, vi } from 'vitest'
|
||||
import {
|
||||
createSessionFormatChain,
|
||||
defineSessionFormatMigration,
|
||||
SessionFormatEventCollector,
|
||||
SessionFormatUnsupportedMigrationError,
|
||||
type SessionFormatArtifact,
|
||||
type SessionFormatEvent,
|
||||
type SessionFormatEventRun,
|
||||
type SessionFormatHeader,
|
||||
type SessionFormatMigration,
|
||||
type SessionFormatMigrationContext,
|
||||
type SessionFormatMigrationStageInput,
|
||||
} from '../src/index.ts'
|
||||
|
||||
const currentArtifact: SessionFormatArtifact = {
|
||||
header: {
|
||||
version: 1,
|
||||
id: 'session-1',
|
||||
createdAt: 1,
|
||||
isSeeded: false,
|
||||
delegationDepth: 0,
|
||||
},
|
||||
inheritedEventCount: 0,
|
||||
events: [{ type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } }],
|
||||
const currentHeader: SessionFormatHeader = {
|
||||
version: 1,
|
||||
id: 'session-1',
|
||||
createdAt: 1,
|
||||
isSeeded: false,
|
||||
delegationDepth: 0,
|
||||
}
|
||||
|
||||
const event: SessionFormatEvent = {
|
||||
type: 'turn/start', seq: 0, time: 2, data: { turn: 1 },
|
||||
}
|
||||
|
||||
const discard: SessionFormatMigrationContext = {
|
||||
emitEvent() {},
|
||||
emitRun() {},
|
||||
}
|
||||
|
||||
function throwUnknown(value: unknown): never {
|
||||
throw value
|
||||
}
|
||||
|
||||
function identityStage(inheritedEventCount: number) {
|
||||
return {
|
||||
headerInheritedEventCount: inheritedEventCount,
|
||||
transformEvent(
|
||||
value: SessionFormatEvent,
|
||||
context: SessionFormatMigrationContext,
|
||||
) {
|
||||
context.emitEvent(value)
|
||||
},
|
||||
transformRun(
|
||||
run: SessionFormatEventRun,
|
||||
context: SessionFormatMigrationContext,
|
||||
) {
|
||||
context.emitRun(run)
|
||||
},
|
||||
finish: () => inheritedEventCount,
|
||||
}
|
||||
}
|
||||
|
||||
function migration(overrides: Partial<SessionFormatMigration> = {}): SessionFormatMigration {
|
||||
return {
|
||||
name: '@test/v0-to-v1',
|
||||
fromVersion: 0,
|
||||
toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
createStage: ({ sourceInheritedEventCount }) => identityStage(sourceInheritedEventCount),
|
||||
validateTargetHeader: () => {},
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
function chain(edge: SessionFormatMigration = migration()) {
|
||||
return createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [edge],
|
||||
restoreCurrentHeader: header => header,
|
||||
})
|
||||
}
|
||||
|
||||
function captureError(run: () => unknown): Error {
|
||||
@@ -29,223 +83,211 @@ function captureError(run: () => unknown): Error {
|
||||
}
|
||||
|
||||
describe('Session format chain', () => {
|
||||
it('restores current input without invoking an adjacent migration', () => {
|
||||
const migrate = vi.fn()
|
||||
const chain = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1',
|
||||
fromVersion: 0,
|
||||
toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate,
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader: () => {},
|
||||
})],
|
||||
restoreCurrent: artifact => artifact,
|
||||
restoreCurrentHeader: header => header,
|
||||
})
|
||||
it('bypasses adjacent stages for current input', () => {
|
||||
const createStage = vi.fn(({ sourceInheritedEventCount }: SessionFormatMigrationStageInput) =>
|
||||
identityStage(sourceInheritedEventCount))
|
||||
const current = chain(migration({ createStage }))
|
||||
const events = new SessionFormatEventCollector()
|
||||
const stream = current.createStream(currentHeader, 0, events)
|
||||
|
||||
const result = chain.migrate(currentArtifact)
|
||||
stream.emitEvent(event)
|
||||
|
||||
expect(result).toEqual(currentArtifact)
|
||||
expect(result).not.toBe(currentArtifact)
|
||||
expect(Object.isFrozen(result)).toBe(true)
|
||||
expect(migrate).not.toHaveBeenCalled()
|
||||
expect(stream.header).toEqual(currentHeader)
|
||||
expect(stream.finish()).toBe(0)
|
||||
expect(events.values).toEqual([event])
|
||||
expect(createStage).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('runs one adjacent whole-artifact edge and its header converter', () => {
|
||||
const validateTarget = vi.fn()
|
||||
const edge = defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1',
|
||||
fromVersion: 0,
|
||||
toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
validateTarget,
|
||||
validateTargetHeader: () => {},
|
||||
})
|
||||
const chain = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [edge],
|
||||
restoreCurrent: value => value,
|
||||
restoreCurrentHeader: value => value,
|
||||
})
|
||||
const source = { ...currentArtifact, header: { ...currentArtifact.header, version: 0 } }
|
||||
it('composes one adjacent stage and its header conversion', () => {
|
||||
const current = chain()
|
||||
const sourceHeader = { ...currentHeader, version: 0 }
|
||||
const events = new SessionFormatEventCollector()
|
||||
const stream = current.createStream(sourceHeader, 0, events)
|
||||
|
||||
expect(chain.plan(0)).toEqual([edge])
|
||||
expect(chain.migrate(source)).toMatchObject({ header: { version: 1 } })
|
||||
expect(validateTarget).toHaveBeenCalledOnce()
|
||||
expect(chain.migrateHeader(source.header).version).toBe(1)
|
||||
stream.emitEvent(event)
|
||||
|
||||
expect(stream.header.version).toBe(1)
|
||||
expect(stream.finish()).toBe(0)
|
||||
expect(events.values).toEqual([event])
|
||||
expect(current.migrateHeader(sourceHeader).version).toBe(1)
|
||||
})
|
||||
|
||||
it('rejects invalid declarations and incomplete chain construction', () => {
|
||||
const base = {
|
||||
name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1,
|
||||
migrateHeader: (header: SessionFormatArtifact['header']) => ({ ...header, version: 1 }),
|
||||
migrate: (artifact: SessionFormatArtifact) => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader: () => {},
|
||||
}
|
||||
const base = migration()
|
||||
expect(() => defineSessionFormatMigration({ ...base, name: '' })).toThrow(/name/)
|
||||
expect(() => defineSessionFormatMigration({ ...base, toVersion: 2 })).toThrow(/adjacent/)
|
||||
expect(() => createSessionFormatChain({
|
||||
currentVersion: 1, migrations: [], restoreCurrent: value => value, restoreCurrentHeader: value => value,
|
||||
}))
|
||||
.toThrow(/missing/)
|
||||
currentVersion: 1, migrations: [], restoreCurrentHeader: value => value,
|
||||
})).toThrow(/missing/)
|
||||
expect(() => createSessionFormatChain({
|
||||
currentVersion: 1, migrations: [base, base], restoreCurrent: value => value, restoreCurrentHeader: value => value,
|
||||
}))
|
||||
.toThrow(/duplicated/)
|
||||
currentVersion: 1, migrations: [base, base], restoreCurrentHeader: value => value,
|
||||
})).toThrow(/duplicated/)
|
||||
expect(() => createSessionFormatChain({
|
||||
currentVersion: 2,
|
||||
migrations: [base, { ...base, name: base.name, fromVersion: 1, toVersion: 2 }],
|
||||
restoreCurrent: value => value,
|
||||
restoreCurrentHeader: value => value,
|
||||
})).toThrow(/name .* duplicated/)
|
||||
expect(() => createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [base, { ...base, name: '@test/v1-to-v2', fromVersion: 1, toVersion: 2 }],
|
||||
restoreCurrent: value => value,
|
||||
restoreCurrentHeader: value => value,
|
||||
})).toThrow(/does not lead/)
|
||||
})
|
||||
|
||||
it('rejects newer inputs and callbacks that return the wrong version', () => {
|
||||
const edge = defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1,
|
||||
migrateHeader: header => header,
|
||||
migrate: artifact => artifact,
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader: () => {},
|
||||
it('requires every intermediate stage to expose its inherited cut before the next edge', () => {
|
||||
const first = migration({
|
||||
createStage: () => ({
|
||||
transformEvent: (value, context) => { context.emitEvent(value) },
|
||||
transformRun: (value, context) => { context.emitRun(value) },
|
||||
finish: () => 0,
|
||||
}),
|
||||
})
|
||||
const chain = createSessionFormatChain({
|
||||
currentVersion: 1, migrations: [edge], restoreCurrent: value => value, restoreCurrentHeader: value => value,
|
||||
const second: SessionFormatMigration = {
|
||||
...migration(),
|
||||
name: '@test/v1-to-v2',
|
||||
fromVersion: 1,
|
||||
toVersion: 2,
|
||||
migrateHeader: header => ({ ...header, version: 2 }),
|
||||
}
|
||||
const current = createSessionFormatChain({
|
||||
currentVersion: 2,
|
||||
migrations: [first, second],
|
||||
restoreCurrentHeader: header => header,
|
||||
})
|
||||
const source = { ...currentArtifact, header: { ...currentArtifact.header, version: 0 } }
|
||||
expect(() => chain.plan(2)).toThrow(/newer/)
|
||||
expect(() => chain.plan(-1)).toThrow(/non-negative/)
|
||||
expect(() => chain.migrate(source)).toThrow(/returned v0/)
|
||||
expect(() => chain.migrateHeader(source.header)).toThrow(/header returned v0/)
|
||||
|
||||
const badRestore = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [defineSessionFormatMigration({
|
||||
...edge,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
})],
|
||||
restoreCurrent: value => ({ ...value, header: { ...value.header, version: 0 } }),
|
||||
restoreCurrentHeader: value => value,
|
||||
expect(() => current.createStream(
|
||||
{ ...currentHeader, version: 0 },
|
||||
0,
|
||||
discard,
|
||||
)).toThrow(/expose its inherited cut/)
|
||||
|
||||
const createSecondStage = vi.fn(({ sourceKind }: SessionFormatMigrationStageInput) => {
|
||||
expect(sourceKind).toBe('transformed')
|
||||
return identityStage(0)
|
||||
})
|
||||
expect(() => badRestore.migrate(currentArtifact)).toThrow(/current Session restorer returned v0/)
|
||||
const complete = createSessionFormatChain({
|
||||
currentVersion: 2,
|
||||
migrations: [migration(), { ...second, createStage: createSecondStage }],
|
||||
restoreCurrentHeader: header => header,
|
||||
})
|
||||
const stream = complete.createStream(
|
||||
{ ...currentHeader, version: 0 },
|
||||
0,
|
||||
discard,
|
||||
)
|
||||
expect(stream.finish()).toBe(0)
|
||||
expect(createSecondStage).toHaveBeenCalledOnce()
|
||||
})
|
||||
|
||||
it('classifies adjacent target-policy refusal as unsupported but current restoration failure as corruption', () => {
|
||||
const policyFailure = new Error('target relationship is invalid')
|
||||
const edge = defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
validateTarget: () => { throw policyFailure },
|
||||
validateTargetHeader: () => {},
|
||||
})
|
||||
const chain = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [edge],
|
||||
restoreCurrent: value => value,
|
||||
restoreCurrentHeader: value => value,
|
||||
})
|
||||
const source = { ...currentArtifact, header: { ...currentArtifact.header, version: 0 } }
|
||||
it('rejects newer inputs and wrong header versions', () => {
|
||||
const current = chain(migration({ migrateHeader: header => header }))
|
||||
const source = { ...currentHeader, version: 0 }
|
||||
|
||||
const refusal = captureError(() => chain.migrate(source))
|
||||
expect(() => current.createStream({ ...currentHeader, version: 2 }, 0, discard)).toThrow(/newer/)
|
||||
expect(() => current.createStream({ ...currentHeader, version: -1 }, 0, discard)).toThrow(/non-negative/)
|
||||
expect(() => current.createStream(source, 0, discard)).toThrow(/header returned v0/)
|
||||
expect(() => current.migrateHeader(source)).toThrow(/header returned v0/)
|
||||
})
|
||||
|
||||
it('classifies stage failures as unsupported and preserves explicit refusals', () => {
|
||||
const policyFailure = new Error('target relationship is invalid')
|
||||
const source = { ...currentHeader, version: 0 }
|
||||
const failed = chain(migration({ createStage: () => { throw policyFailure } }))
|
||||
const refusal = captureError(() => failed.createStream(source, 0, discard))
|
||||
expect(refusal).toBeInstanceOf(SessionFormatUnsupportedMigrationError)
|
||||
expect(refusal.message).toContain('target relationship is invalid')
|
||||
expect(refusal.cause).toBe(policyFailure)
|
||||
|
||||
const brokenCurrent = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [edge],
|
||||
restoreCurrent: () => { throw new Error('current corruption') },
|
||||
restoreCurrentHeader: value => value,
|
||||
})
|
||||
expect(() => brokenCurrent.migrate(currentArtifact)).toThrow('current corruption')
|
||||
|
||||
const migrationFailure = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [{
|
||||
...edge,
|
||||
migrate: () => { throw 'source policy token' },
|
||||
validateTarget: () => {},
|
||||
}],
|
||||
restoreCurrent: value => value,
|
||||
restoreCurrentHeader: value => value,
|
||||
})
|
||||
expect(() => migrationFailure.migrate(source)).toThrow(/source policy token/)
|
||||
|
||||
const alreadyUnsupported = new SessionFormatUnsupportedMigrationError('explicit edge refusal')
|
||||
const preserved = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [{ ...edge, validateTarget: () => { throw alreadyUnsupported } }],
|
||||
restoreCurrent: value => value,
|
||||
restoreCurrentHeader: value => value,
|
||||
const preserved = chain(migration({ createStage: () => { throw alreadyUnsupported } }))
|
||||
expect(() => preserved.createStream(source, 0, discard)).toThrow(alreadyUnsupported)
|
||||
|
||||
const finishFailure = new Error('finish relationship is invalid')
|
||||
const finishing = chain(migration({
|
||||
createStage: () => ({
|
||||
...identityStage(0),
|
||||
finish: () => { throw finishFailure },
|
||||
}),
|
||||
})).createStream(source, 0, discard)
|
||||
const finishRefusal = captureError(() => finishing.finish())
|
||||
expect(finishRefusal).toBeInstanceOf(SessionFormatUnsupportedMigrationError)
|
||||
expect(finishRefusal.cause).toBe(finishFailure)
|
||||
|
||||
const runFailure = new Error('run relationship is invalid')
|
||||
const runFailing = chain(migration({
|
||||
createStage: () => ({
|
||||
...identityStage(0),
|
||||
transformRun: () => { throw runFailure },
|
||||
}),
|
||||
})).createStream(source, 0, discard)
|
||||
const runRefusal = captureError(() => {
|
||||
runFailing.emitRun({
|
||||
runType: 'test-run',
|
||||
firstSeq: 0,
|
||||
eventCount: 1,
|
||||
*expand() { yield event },
|
||||
})
|
||||
})
|
||||
expect(() => preserved.migrate(source)).toThrow(alreadyUnsupported)
|
||||
expect(runRefusal).toBeInstanceOf(SessionFormatUnsupportedMigrationError)
|
||||
expect(runRefusal.cause).toBe(runFailure)
|
||||
|
||||
const nonError = chain(migration({
|
||||
createStage: () => throwUnknown('non-Error stage refusal'),
|
||||
}))
|
||||
expect(() => nonError.createStream(source, 0, discard))
|
||||
.toThrow(/non-Error stage refusal/)
|
||||
})
|
||||
|
||||
it('collects compact runs as expanded events', () => {
|
||||
const collector = new SessionFormatEventCollector()
|
||||
collector.emitRun({
|
||||
runType: 'test-run',
|
||||
firstSeq: 0,
|
||||
eventCount: 1,
|
||||
*expand() { yield event },
|
||||
})
|
||||
expect(collector.values).toEqual([event])
|
||||
})
|
||||
|
||||
it('validates every adjacent target header and the final current header', () => {
|
||||
const validateTargetHeader = vi.fn((header: SessionFormatArtifact['header']) => {
|
||||
const validateTargetHeader = vi.fn((header: SessionFormatHeader) => {
|
||||
if (header['targetMarker'] !== true) throw new Error('target header lacks marker')
|
||||
})
|
||||
const restoreCurrentHeader = vi.fn((header: SessionFormatArtifact['header']) => {
|
||||
const restoreCurrentHeader = vi.fn((header: SessionFormatHeader) => {
|
||||
if (typeof header.id !== 'string') throw new Error('current header lacks id')
|
||||
return header
|
||||
})
|
||||
const edge = defineSessionFormatMigration({
|
||||
name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1,
|
||||
migrateHeader: header => ({ ...header, version: 1 }),
|
||||
migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }),
|
||||
validateTarget: () => {},
|
||||
validateTargetHeader,
|
||||
})
|
||||
const chain = createSessionFormatChain({
|
||||
const current = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [edge],
|
||||
restoreCurrent: value => value,
|
||||
migrations: [migration({ validateTargetHeader })],
|
||||
restoreCurrentHeader,
|
||||
})
|
||||
const source = { ...currentArtifact.header, version: 0 }
|
||||
const source = { ...currentHeader, version: 0 }
|
||||
|
||||
const refusal = captureError(() => chain.migrateHeader(source))
|
||||
const refusal = captureError(() => current.migrateHeader(source))
|
||||
expect(refusal).toBeInstanceOf(SessionFormatUnsupportedMigrationError)
|
||||
expect(refusal.message).toContain('target header lacks marker')
|
||||
expect(validateTargetHeader).toHaveBeenCalledOnce()
|
||||
|
||||
const rejectingHeader = createSessionFormatChain({
|
||||
const rejecting = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [{
|
||||
...edge,
|
||||
migrateHeader: () => { throw new Error('historical header policy') },
|
||||
}],
|
||||
restoreCurrent: value => value,
|
||||
migrations: [migration({ migrateHeader: () => { throw new Error('historical header policy') } })],
|
||||
restoreCurrentHeader,
|
||||
})
|
||||
expect(() => rejectingHeader.migrateHeader(source)).toThrow(/historical header policy/)
|
||||
expect(() => rejecting.migrateHeader(source)).toThrow(/historical header policy/)
|
||||
|
||||
const badCurrent = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [edge],
|
||||
restoreCurrent: value => value,
|
||||
migrations: [migration()],
|
||||
restoreCurrentHeader: () => ({ version: 1 } as never),
|
||||
})
|
||||
expect(() => badCurrent.migrateHeader(currentArtifact.header)).toThrow(/current Session header restoration id/)
|
||||
expect(() => badCurrent.migrateHeader(currentHeader)).toThrow(/current Session header restoration id/)
|
||||
|
||||
const wrongCurrentVersion = createSessionFormatChain({
|
||||
currentVersion: 1,
|
||||
migrations: [edge],
|
||||
restoreCurrent: value => value,
|
||||
migrations: [migration()],
|
||||
restoreCurrentHeader: header => ({ ...header, version: 0 }),
|
||||
})
|
||||
expect(() => wrongCurrentVersion.migrateHeader(currentArtifact.header)).toThrow(/header restorer returned v0/)
|
||||
expect(() => wrongCurrentVersion.migrateHeader(currentHeader)).toThrow(/header restorer returned v0/)
|
||||
})
|
||||
})
|
||||
@@ -3,7 +3,6 @@ import {
|
||||
inspectSessionFormatVersion,
|
||||
sessionFormatCount,
|
||||
sessionFormatSafeInteger,
|
||||
snapshotSessionFormatArtifact,
|
||||
snapshotSessionFormatHeader,
|
||||
snapshotSessionFormatJson,
|
||||
} from '../src/index.ts'
|
||||
@@ -49,19 +48,6 @@ describe('lossless Session format JSON snapshots', () => {
|
||||
expect(() => snapshotSessionFormatJson(new ArrayValue(1))).toThrow(/not lossless JSON/)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['non-object header', { header: null, inheritedEventCount: 0, events: [] }],
|
||||
['non-array events', { header: { version: 1 }, inheritedEventCount: 0, events: null }],
|
||||
['non-object event', { header: { version: 1 }, inheritedEventCount: 0, events: [null] }],
|
||||
['non-dense seq', { header: { version: 1 }, inheritedEventCount: 0, events: [{ type: 'x', seq: 1, time: 1, data: {} }] }],
|
||||
['empty type', { header: { version: 1 }, inheritedEventCount: 0, events: [{ type: '', seq: 0, time: 1, data: {} }] }],
|
||||
['invalid time', { header: { version: 1 }, inheritedEventCount: 0, events: [{ type: 'x', seq: 0, time: 1.5, data: {} }] }],
|
||||
['missing data', { header: { version: 1 }, inheritedEventCount: 0, events: [{ type: 'x', seq: 0, time: 1 }] }],
|
||||
['oversized cut', { header: { version: 1 }, inheritedEventCount: 1, events: [] }],
|
||||
])('refuses an artifact with %s', (_name, artifact) => {
|
||||
expect(() => snapshotSessionFormatArtifact(artifact as never)).toThrow()
|
||||
})
|
||||
|
||||
it('refuses a non-object header snapshot', () => {
|
||||
expect(() => snapshotSessionFormatHeader(null as never)).toThrow(/header|object/)
|
||||
expect(() => snapshotSessionFormatHeader({
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-persistence-jsonl/README.md
|
||||
README.md: 1632b971c1577c7c406fdbd18b732add1c97bfee
|
||||
README.zh.md: 289e3f3aefff5b98c4053d7682c2fb50f8692c71
|
||||
README.md: c4ed0769621a51223af616746ef877824abf48c9
|
||||
README.zh.md: 0b7ef652ef1a43b811dd0e77000a84217ffb2d40
|
||||
@@ -71,11 +71,11 @@ Session ids are injectively escaped to one safe path segment before use (no trav
|
||||
|
||||
### Durability and crash semantics
|
||||
|
||||
A session is materialized lazily: `create(header)` writes nothing and returns the owned write handle, and the handle's first `append` writes and `fsync`s the encoded header and first batch through a no-overwrite publish — so a created-but-never-appended session leaves nothing on disk unless its owner calls `handle.flush()`, which publishes one header frame without an event. Each subsequent batch appends lines or one compressed frame and `fsync`s before the append resolves; a caught write or sync failure rolls the file back to its prior length. Committed events are never rewritten. After a crash, the stored log keeps its interrupted final turn — every record in the committed prefix survives, and the resuming reader appends synthetic closers through its write handle. A torn tail — an incomplete final line, or a torn final frame — is never returned to a reader and is discarded whole, truncated durably before the write handle's first new append, because its own append never resolved and nothing in it was acknowledged durable; checksum, decompression, or structural failure in the committed prefix rejects as corruption.
|
||||
A session is materialized lazily: `create(header)` writes nothing and returns the owned write handle, and the handle's first `append` writes and `fsync`s the encoded header and first batch through a no-overwrite publish — so a created-but-never-appended session leaves nothing on disk unless its owner calls `handle.flush()`, which publishes one header frame without an event. Each subsequent batch appends lines or one compressed frame and `fsync`s before the append resolves; a caught write or sync failure rolls the file back to its prior length. Committed events are never rewritten. After a crash, the stored log keeps its interrupted final turn — every record in the committed prefix survives, and the resuming reader appends synthetic closers through its write handle. An incomplete final raw line is discarded. A torn final Zstandard frame contributes only its complete decoded JSONL records; a write handle truncates the torn bytes and durably rewrites those recovered records before its first new batch. Checksum, decompression, or structural failure in a complete committed frame rejects as corruption.
|
||||
|
||||
### Reading the logs
|
||||
|
||||
`open(id, 'read'|'write')` selects the highest canonical generation and publishes a current successor beside a supported historical source before returning the handle; the source remains byte-identical. The handle's `read(offset?, length?)` then serves validated contiguous slices, never a torn tail. A torn final Zstandard frame is partially decoded: complete JSONL records already flushed into it are recovered into the logical log, and the write handle's first mutation truncates current-generation torn bytes and durably rewrites the recovered records ahead of its own batch. A write open primes the handle with the validated stored prefix, and a bounded revision-keyed memo lets an immediate observe-to-resume handoff reuse that parse. `stat(id)` and `list()` select and translate only the highest generation header without reading event rows or publishing migration output; snapshots carry `sizeBytes` and a best-effort stat-derived revision for the selected file. With `compression: 'none'`, the log is newline-delimited text an external reader can consume directly; the compressed default must be read through the backend.
|
||||
`open(id, 'read'|'write')` selects the highest canonical generation. Current input follows the ordinary fast path. Before either kind of handle returns for historical input, the backend decodes and migrates the source once, encodes a same-directory temporary file in bounded chunks, verifies it in a Worker Thread, rechecks the source revision, publishes the current successor without overwrite, and verifies and reopens the committed generation. The source remains byte-identical. The handle's `read(offset?, length?)` serves validated contiguous slices under the durability rules above. A write open primes the handle with the validated stored prefix, and a bounded revision-keyed memo lets an immediate observe-to-resume handoff reuse that parse. `stat(id)` and `list()` select and translate only the highest generation header without reading event rows or starting migration; snapshots carry `sizeBytes` and a best-effort stat-derived revision for the selected file. With `compression: 'none'`, the log is newline-delimited text an external reader can consume directly; the compressed default must be read through the backend.
|
||||
|
||||
-----
|
||||
|
||||
@@ -89,11 +89,11 @@ This section explains the physical encoding and write path; the observable contr
|
||||
|
||||
### Design concept
|
||||
|
||||
The backend owns its complete storage runtime (`src/storage.ts`): `JsonlSessionHandle` carries the per-handle mutation chain, the routed live-event buffer with its fixed batching window and single-flight drain, monotonic reads, and idempotent close; a tracker holds the in-process single-writer claims, the open-handle set teardown sweeps, and the created-but-unmaterialized pending sessions the backend's own session listeners route into. The package deliberately exposes only its default plugin export plus configuration types — the concrete class is not a named export, so consumers couple to `ctx.sessionPersistence`, and the shared seam suites (`runPersistenceContract`/`runLiveWritePathContract`) pin its observable behavior. Its change token is a best-effort file revision: device, inode, size, and nanosecond timestamps identify one log for `stat`/`list` and for the stable-read loop that retries a read torn by a concurrent append.
|
||||
The backend owns its complete storage runtime (`src/storage.ts`): `JsonlSessionHandle` carries the per-handle mutation chain, the routed live-event buffer with its fixed batching window and single-flight drain, monotonic reads, and idempotent close; a tracker holds the in-process single-writer claims, the open-handle set teardown sweeps, and the created-but-unmaterialized pending sessions the backend's own session listeners route into. Historical body reads run the same serial ensure-current operation before constructing a handle. The package deliberately exposes only its default plugin export plus configuration types — the concrete class is not a named export, so consumers couple to `ctx.sessionPersistence`, and the shared seam suites (`runPersistenceContract`/`runLiveWritePathContract`) pin its observable behavior. Its change token is a best-effort file revision: device, inode, size, and nanosecond timestamps identify one log for `stat`/`list`, for the stable-read loop that retries a read torn by a concurrent append, and for the pre-publication source check.
|
||||
|
||||
### Physical encoding
|
||||
|
||||
The default artifact is a standard concatenation of independent [Zstandard frames](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md): one checksummed frame containing only the header line, then one checksummed frame per durable append batch, using Node's built-in Zstandard API at its default compression level (no level knob). Current v2 writes one event per row; `sourceEventSeqs` uses a lossless storage representation in which consecutive runs of at least three sequence numbers become `[start, end]` pairs, any other list stays verbatim, and reading expands the exact in-memory array. Listing reads and validates only the header frame. `compression: 'none'` keeps the same storage-form logical lines without frame compression. A root belongs to one encoding: startup discovery and targeted lookup reject generations with the other suffix; format migration preserves the configured encoding, while compression conversion, mixed-root fallback, and dual write remain unsupported. Frozen v0 and v1 codecs retain their packed-row decoders solely for historical generations.
|
||||
The default artifact is a standard concatenation of independent [Zstandard frames](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md): one checksummed frame containing only the header line, then one checksummed frame per durable append batch, using Node's built-in Zstandard API at its default compression level (no level knob). Current v2 writes one event per row; `sourceEventSeqs` uses a lossless storage representation in which consecutive runs of at least three sequence numbers become `[start, end]` pairs, any other list stays verbatim, and reading expands the exact in-memory array. Historical migration reuses one Zstandard decoder, passes parsed rows through stateful format stages, and streams current records through one compression context in about 1 MiB main-thread slices while retaining only final current events, bounded decoder state, and the required sequence-remap table. Listing reads and validates only the header frame. `compression: 'none'` keeps the same storage-form logical lines without frame compression. A root belongs to one encoding: startup discovery and targeted lookup reject generations with the other suffix; format migration preserves the configured encoding, while compression conversion, mixed-root fallback, and dual write remain unsupported. Frozen v0 and v1 codecs retain their packed-row decoders solely for historical generations.
|
||||
|
||||
### Source map
|
||||
|
||||
@@ -102,7 +102,8 @@ The default artifact is a standard concatenation of independent [Zstandard frame
|
||||
| [`src/index.ts`](src/index.ts) | Plugin entry: `Config` schema, the backend service class, and file storage primitives |
|
||||
| [`src/storage.ts`](src/storage.ts) | The JSONL handle, routed live-event buffer, in-process writer bookkeeping, listeners, teardown |
|
||||
| [`src/format.ts`](src/format.ts) | Log path derivation, header encoding, and current record scanning |
|
||||
| [`src/generation.ts`](src/generation.ts) | Stable generation reads, format-adapter invocation, exclusive successor publication, committed reopen |
|
||||
| [`src/generation.ts`](src/generation.ts) | Single-pass historical restore, bounded stage encoding, source revision check, and exclusive successor publication |
|
||||
| [`src/migration-verifier.ts`](src/migration-verifier.ts) | Worker lifecycle for staged and competing-generation verification |
|
||||
| [`src/zstd.ts`](src/zstd.ts) | Zstandard frame compression, decoding, and frame scanning |
|
||||
| [`src/win32.ts`](src/win32.ts) | Windows write-through publish and directory creation |
|
||||
| — | No runtime invariant companion is published; persistence correctness requires backend round-trip and crash-tail tests; this package exposes no continuously observable in-process relation. |
|
||||
|
||||
@@ -71,11 +71,11 @@ kind: "package-reference"
|
||||
|
||||
### 持久性与崩溃语义
|
||||
|
||||
会话延迟实体化：`create(header)` 不写入任何内容并返回持有的写句柄，句柄的第一次 `append` 通过无覆盖发布写入并 `fsync` 编码后的 header 与第一批——因此已创建但从未 append 的会话不留下任何磁盘内容，除非其所有者调用 `handle.flush()`，以无事件的单个 header 帧发布它。后续每个批次追加行或一个压缩帧，并在 append 完成前 `fsync`；捕获到写入或同步失败时把文件回滚到之前的字节长度。已提交事件绝不重写。崩溃后，已存储日志保留被中断的最终轮次——已提交前缀中的每条记录都保留下来，由执行恢复的读方通过其写句柄追加合成 closer。撕裂尾部——不完整的最后一行，或撕裂的最后一帧——绝不返回给读取方并被整体丢弃，在写句柄的第一次新 append 之前被持久截断，因为其自身的 append 从未成功返回，其中没有任何内容被确认为已持久；已提交前缀中的校验和、解压或结构失败以损坏拒绝。
|
||||
会话延迟实体化：`create(header)` 不写入任何内容并返回持有的写句柄，句柄的第一次 `append` 通过无覆盖发布写入并 `fsync` 编码后的 header 与第一批——因此已创建但从未 append 的会话不留下任何磁盘内容，除非其所有者调用 `handle.flush()`，以无事件的单个 header 帧发布它。后续每个批次追加行或一个压缩帧，并在 append 完成前 `fsync`；捕获到写入或同步失败时把文件回滚到之前的字节长度。已提交事件绝不重写。崩溃后，已存储日志保留被中断的最终轮次——已提交前缀中的每条记录都保留下来，由执行恢复的读方通过其写句柄追加合成 closer。不完整的最终原始行会被丢弃。撕裂的最终 Zstandard 帧只贡献其中完整解码出的 JSONL 记录；写句柄会截掉撕裂字节，并在第一次新批次之前持久重写这些恢复出的记录。完整已提交帧中的校验和、解压或结构失败以损坏拒绝。
|
||||
|
||||
### 读取日志
|
||||
|
||||
`open(id, 'read'|'write')` 选择最高规范 generation，并在返回句柄前为受支持的历史源发布一个并列的当前后继；源保持逐字节不变。句柄的 `read(offset?, length?)` 随后提供经过验证的连续切片，绝不包含撕裂尾部。撕裂的最终 Zstandard 帧会被部分解码：其中已刷入的完整 JSONL 记录被恢复进逻辑日志，写句柄的第一次修改会截掉当前 generation 的撕裂字节并在自己的批次之前持久重写这些恢复的记录。写 open 会用已验证的存储前缀预热句柄，一个按 revision 为键的有界 memo 让紧接的观察到恢复交接复用该解析。`stat(id)` 与 `list()` 只选择并转换最高 generation 的 header，不读取事件行，也不发布迁移输出；快照携带所选文件的 `sizeBytes` 与尽力而为的 stat 派生修订号。选择 `compression: 'none'` 后，日志是外部读取方可直接消费的换行分隔文本；压缩默认值必须经后端读取。
|
||||
`open(id, 'read'|'write')` 选择最高规范 generation。当前格式输入走普通快速路径。对于历史输入，两种句柄都会在返回前等待后端单遍解码并迁移源、按有界分片编码同目录临时文件、在 Worker Thread 中校验、复查源修订、以不覆盖方式发布当前后继，并校验和重新打开已提交 generation。源保持逐字节不变。句柄的 `read(offset?, length?)` 按上述持久性规则提供经过验证的连续切片。写 open 会用已验证的存储前缀预热句柄，一个按 revision 为键的有界 memo 让紧接的观察到恢复交接复用该解析。`stat(id)` 与 `list()` 只选择并转换最高 generation 的 header，不读取事件行，也不启动迁移；快照携带所选文件的 `sizeBytes` 与尽力而为的 stat 派生修订号。选择 `compression: 'none'` 后，日志是外部读取方可直接消费的换行分隔文本；压缩默认值必须经后端读取。
|
||||
|
||||
-----
|
||||
|
||||
@@ -89,11 +89,11 @@ kind: "package-reference"
|
||||
|
||||
### 设计理念
|
||||
|
||||
该后端拥有自己完整的存储运行时（`src/storage.ts`）：`JsonlSessionHandle` 承载逐句柄修改链、带固定批处理窗口与 single-flight 排空的已路由实时事件缓冲、单调读取与幂等 close；一个 tracker 持有进程内单写者认领、teardown 清扫所遍历的打开句柄集合，以及后端自己的会话监听器所路由进的已创建但未实体化待定会话。本包有意只暴露默认插件导出与配置类型——具体类不是具名导出，因此消费方只耦合 `ctx.sessionPersistence`，其可观察行为由共享 seam 测试套件（`runPersistenceContract`/`runLiveWritePathContract`）钉住。其变更令牌是尽力而为的文件修订值：device、inode、size 与纳秒时间戳标识一份日志，供 `stat`/`list` 以及在并发 append 撕裂读取时重试的稳定读取循环使用。
|
||||
该后端拥有自己完整的存储运行时（`src/storage.ts`）：`JsonlSessionHandle` 承载逐句柄修改链、带固定批处理窗口与 single-flight 排空的已路由实时事件缓冲、单调读取与幂等 close；一个 tracker 持有进程内单写者认领、teardown 清扫所遍历的打开句柄集合，以及后端自己的会话监听器所路由进的已创建但未实体化待定会话。历史正文读取会在构造句柄前执行同一个串行 ensure-current 操作。本包有意只暴露默认插件导出与配置类型——具体类不是具名导出，因此消费方只耦合 `ctx.sessionPersistence`，其可观察行为由共享 seam 测试套件（`runPersistenceContract`/`runLiveWritePathContract`）钉住。其变更令牌是尽力而为的文件修订值：device、inode、size 与纳秒时间戳标识一份日志，供 `stat`/`list`、在并发 append 撕裂读取时重试的稳定读取循环，以及发布前源检查使用。
|
||||
|
||||
### 物理编码
|
||||
|
||||
默认产物是独立 [Zstandard 帧](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md) 的标准拼接：一个仅包含 header 行的带校验和帧，后跟每个持久 append 批次一个带校验和帧，使用 Node 内置 Zstandard API 的默认压缩级别（无级别开关）。当前 v2 为每个事件写一行；`sourceEventSeqs` 使用无损存储形式：至少包含三个序列号的连续段会变成 `[start, end]` 区间对，其他列表原样保留；读取时会展开回精确的内存数组。列表只读取并验证 header 帧。`compression: 'none'` 保留相同的存储形式逻辑行，但不使用帧压缩。一个根只属于一种编码：启动发现与定向查找会拒绝使用另一后缀的 generation；格式迁移保留已配置编码，而压缩转换、混合根回退与双写仍不受支持。冻结的 v0 与 v1 codec 仅为历史 generation 保留 packed-row decoder。
|
||||
默认产物是独立 [Zstandard 帧](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md) 的标准拼接：一个仅包含 header 行的带校验和帧，后跟每个持久 append 批次一个带校验和帧，使用 Node 内置 Zstandard API 的默认压缩级别（无级别开关）。当前 v2 为每个事件写一行；`sourceEventSeqs` 使用无损存储形式：至少包含三个序列号的连续段会变成 `[start, end]` 区间对，其他列表原样保留；读取时会展开回精确的内存数组。历史迁移会复用一个 Zstandard decoder，让已解析行流经有状态格式 Stage，并通过一个压缩 context 以约 1 MiB 主线程分片流式写入当前记录，同时只保留最终当前事件、有界 decoder 状态与必需的序号重映射表。列表只读取并验证 header 帧。`compression: 'none'` 保留相同的存储形式逻辑行，但不使用帧压缩。一个根只属于一种编码：启动发现与定向查找会拒绝使用另一后缀的 generation；格式迁移保留已配置编码，而压缩转换、混合根回退与双写仍不受支持。冻结的 v0 与 v1 codec 仅为历史 generation 保留 packed-row decoder。
|
||||
|
||||
### 源码地图
|
||||
|
||||
@@ -102,7 +102,8 @@ kind: "package-reference"
|
||||
| [`src/index.ts`](src/index.ts) | 插件入口：`Config` schema、后端服务类与文件存储原语 |
|
||||
| [`src/storage.ts`](src/storage.ts) | JSONL 句柄、已路由实时事件缓冲、进程内写入者记账、监听器、teardown |
|
||||
| [`src/format.ts`](src/format.ts) | 日志路径派生、header 编码与当前记录扫描 |
|
||||
| [`src/generation.ts`](src/generation.ts) | 稳定 generation 读取、格式 adapter 调用、排他后继发布与已提交 reopen |
|
||||
| [`src/generation.ts`](src/generation.ts) | 单遍历史还原、有界 stage 编码、源 revision 检查与排他后继发布 |
|
||||
| [`src/migration-verifier.ts`](src/migration-verifier.ts) | stage 与竞争 generation 校验的 Worker 生命周期 |
|
||||
| [`src/zstd.ts`](src/zstd.ts) | Zstandard 帧压缩、解码与帧扫描 |
|
||||
| [`src/win32.ts`](src/win32.ts) | Windows write-through 发布与目录创建 |
|
||||
| — | 不发布运行时不变式伴生入口；身份在存储层强制。 |
|
||||
|
||||
@@ -23,6 +23,7 @@
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/worker.cjs",
|
||||
"lib/types/**/*.d.ts"
|
||||
],
|
||||
"license": "MIT",
|
||||
@@ -32,6 +33,7 @@
|
||||
"@deepseek-ai/cordis": "workspace:^"
|
||||
},
|
||||
"dependencies": {
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-format": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-format-catalog": "workspace:^",
|
||||
"fs-ext": "2.1.1",
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
import { isAbsolute, join } from 'node:path'
|
||||
import {
|
||||
decodeSeqRanges, encodeSeqRanges, SESSION_FORMAT_VERSION,
|
||||
SESSION_FORMAT_VERSION,
|
||||
SessionLogOffset,
|
||||
} from '@deepseek-ai/dsh-session'
|
||||
import type {
|
||||
@@ -20,6 +20,9 @@ import type {
|
||||
SessionLogOffset as SessionLogOffsetType,
|
||||
} from '@deepseek-ai/dsh-session'
|
||||
import { parseSessionFormatLogFilename, sessionFormatLogFilename } from '@deepseek-ai/dsh-session-format'
|
||||
import type { SessionFormatEvent } from '@deepseek-ai/dsh-session-format'
|
||||
import type { SessionFormatRecovery, SessionFormatRestore } from '@deepseek-ai/dsh-session-format'
|
||||
import { sessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog'
|
||||
import {
|
||||
SessionFormatUnsupportedError,
|
||||
sessionFormatVersionRefusal,
|
||||
@@ -122,18 +125,10 @@ export function toHeaderLine(
|
||||
if (!header.isSeeded && cut !== 0) {
|
||||
throw new Error('unseeded session header inherited event count must be 0')
|
||||
}
|
||||
return {
|
||||
type: 'session',
|
||||
version: header.version,
|
||||
id: header.id,
|
||||
createdAt: header.createdAt,
|
||||
...header.cwd !== undefined ? { cwd: header.cwd } : {},
|
||||
...header.parentSession !== undefined ? { parentSession: header.parentSession } : {},
|
||||
isSeeded: header.isSeeded,
|
||||
...header.origin !== undefined ? { origin: header.origin } : {},
|
||||
return sessionFormatCatalog.encodeCurrentHeader({
|
||||
...header,
|
||||
delegationDepth: header.delegationDepth ?? 0,
|
||||
...header.agentPreset !== undefined ? { agentPreset: header.agentPreset } : {},
|
||||
}
|
||||
}, cut) as unknown as HeaderLine
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -314,38 +309,16 @@ export function logPath(
|
||||
* @returns the batch's JSONL text; the writer adds the final newline.
|
||||
*/
|
||||
export function eventLines(events: readonly SessionEvent[]): string {
|
||||
return events.map(record => JSON.stringify(encodeProvenanceForStorage(record))).join('\n')
|
||||
return events.map(eventLine).join('\n')
|
||||
}
|
||||
|
||||
/**
|
||||
* Losslessly shrink a record's `sourceEventSeqs` for the log: consecutive
|
||||
* runs of at least three seqs become `[start, end]` pairs, and any other list
|
||||
* stays verbatim.
|
||||
* @param record - one stored record (event or packed row).
|
||||
* @returns the record with its provenance in storage form (widened from the
|
||||
* in-memory `SessionSeq[]`; {@link expandProvenanceFromStorage} restores it).
|
||||
* Serialize one v2 event as one JSONL record without its trailing newline.
|
||||
* @param event - current event to encode.
|
||||
* @returns one physical JSON record.
|
||||
*/
|
||||
function encodeProvenanceForStorage(record: SessionEvent): unknown {
|
||||
if (!('sourceEventSeqs' in record)) return record
|
||||
return { ...record, sourceEventSeqs: encodeSeqRanges(record.sourceEventSeqs) }
|
||||
}
|
||||
|
||||
/**
|
||||
* Expand a parsed line's storage-form provenance back to `SessionSeq[]`.
|
||||
* @param parsed - the JSON-parsed value of one stored line.
|
||||
* @returns the value with provenance expanded.
|
||||
* @throws when the record or its storage-form provenance is malformed.
|
||||
*/
|
||||
function expandProvenanceFromStorage(parsed: unknown): unknown {
|
||||
if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) {
|
||||
throw new TypeError('stored session records must be objects')
|
||||
}
|
||||
const record = parsed as { seq?: unknown; sourceEventSeqs?: unknown }
|
||||
if (record.sourceEventSeqs === undefined) return parsed
|
||||
if (!Number.isSafeInteger(record.seq) || (record.seq as number) < 0) {
|
||||
throw new TypeError('stored session event seq must be a non-negative safe integer')
|
||||
}
|
||||
return { ...record, sourceEventSeqs: decodeSeqRanges(record.sourceEventSeqs, record.seq as number) }
|
||||
export function eventLine(event: SessionEvent): string {
|
||||
return JSON.stringify(sessionFormatCatalog.encodeCurrentEvent(event as unknown as SessionFormatEvent))
|
||||
}
|
||||
|
||||
interface SessionLogScan {
|
||||
@@ -355,21 +328,6 @@ interface SessionLogScan {
|
||||
committedBytes: number
|
||||
}
|
||||
|
||||
/** Derive the v2 fork cut from the last lineage-tagged seed marker. */
|
||||
function inheritedCut(meta: SessionHeader, events: readonly SessionEvent[]): SessionLogOffsetType {
|
||||
let cut: SessionLogOffsetType | undefined
|
||||
for (const event of events) {
|
||||
if (event.type === 'session/end-seed' && event.data.inherited === true) cut = SessionLogOffset(event.seq)
|
||||
}
|
||||
if (meta.isSeeded && cut === undefined) {
|
||||
throw new Error('corrupt session log: seeded v2 header lacks an inherited end-seed marker')
|
||||
}
|
||||
if (!meta.isSeeded && cut !== undefined) {
|
||||
throw new Error('corrupt session log: unseeded v2 header contains an inherited end-seed marker')
|
||||
}
|
||||
return cut ?? SessionLogOffset(0)
|
||||
}
|
||||
|
||||
/**
|
||||
* Refuse a header carrying a format version this build does not read BEFORE
|
||||
* validating the current header shape or decoding any event row: a future
|
||||
@@ -377,8 +335,7 @@ function inheritedCut(meta: SessionHeader, events: readonly SessionEvent[]): Ses
|
||||
* must see "upgrade the harness", never "corrupt session log".
|
||||
* @param parsed - the JSON-parsed first line of a session artifact.
|
||||
*/
|
||||
function refuseForeignFormatVersion(parsed: unknown): void {
|
||||
if (typeof parsed !== 'object' || parsed === null) return
|
||||
function refuseForeignFormatVersion(parsed: object): void {
|
||||
const { version, id } = parsed as { version?: unknown; id?: unknown }
|
||||
if (typeof version !== 'number' || version === SESSION_FORMAT_VERSION) return
|
||||
throw new SessionFormatUnsupportedError(
|
||||
@@ -387,7 +344,7 @@ function refuseForeignFormatVersion(parsed: unknown): void {
|
||||
}
|
||||
|
||||
/** Parse one complete header record supplied independently from event rows. */
|
||||
function parseHeaderRecord(record: Buffer): ReturnType<typeof fromHeaderLine> {
|
||||
function parseHeaderRecord(record: Buffer): { readonly meta: SessionHeader; readonly restore: SessionFormatRestore } {
|
||||
if (record.length === 0 || record.at(-1) !== 0x0A || record.indexOf(0x0A) !== record.length - 1) {
|
||||
throw new Error('empty or header-less session log')
|
||||
}
|
||||
@@ -397,12 +354,25 @@ function parseHeaderRecord(record: Buffer): ReturnType<typeof fromHeaderLine> {
|
||||
} catch {
|
||||
throw new Error('corrupt session log: header line is not valid JSON')
|
||||
}
|
||||
if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) {
|
||||
throw new Error('corrupt session log: first line is not a JSON object')
|
||||
}
|
||||
refuseForeignFormatVersion(parsed)
|
||||
assertNoRetiredHeaderFields(parsed)
|
||||
if (!isHeaderLine(parsed)) {
|
||||
throw new Error('corrupt session log: first line is not a session header')
|
||||
}
|
||||
return fromHeaderLine(parsed)
|
||||
let restore: SessionFormatRestore
|
||||
try {
|
||||
restore = sessionFormatCatalog.createRestore(parsed, {
|
||||
recovery: 'strict',
|
||||
validation: 'transformed',
|
||||
})
|
||||
} catch {
|
||||
/* v8 ignore next -- isHeaderLine matches the current codec; this preserves classification if it tightens. */
|
||||
throw new Error('corrupt session log: first line is not a session header')
|
||||
}
|
||||
return { meta: fromHeaderLine(parsed).meta, restore }
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -413,7 +383,8 @@ function parseHeaderRecord(record: Buffer): ReturnType<typeof fromHeaderLine> {
|
||||
*/
|
||||
export class SessionLogScanner {
|
||||
private readonly meta: SessionHeader
|
||||
private readonly events: SessionEvent[] = []
|
||||
private readonly restore: SessionFormatRestore
|
||||
private eventCount = 0
|
||||
private fragments: Buffer[] = []
|
||||
private fragmentBytes = 0
|
||||
private inputBytes: number
|
||||
@@ -426,9 +397,13 @@ export class SessionLogScanner {
|
||||
* Create an event scanner from exactly one newline-terminated header record.
|
||||
* @param headerRecord - the complete first JSONL record, including its newline.
|
||||
*/
|
||||
constructor(headerRecord: Buffer) {
|
||||
constructor(
|
||||
headerRecord: Buffer,
|
||||
private readonly recovery: SessionFormatRecovery = 'recoverable',
|
||||
) {
|
||||
const parsed = parseHeaderRecord(headerRecord)
|
||||
this.meta = parsed.meta
|
||||
this.restore = parsed.restore
|
||||
this.inputBytes = headerRecord.length
|
||||
this.committedBytes = headerRecord.length
|
||||
}
|
||||
@@ -477,7 +452,7 @@ export class SessionLogScanner {
|
||||
return {
|
||||
inputBytes: this.inputBytes,
|
||||
committedBytes: this.committedBytes,
|
||||
eventCount: SessionLogOffset(this.events.length),
|
||||
eventCount: SessionLogOffset(this.eventCount),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -487,10 +462,11 @@ export class SessionLogScanner {
|
||||
*/
|
||||
finish(): SessionLogScan {
|
||||
this.finished = true
|
||||
const artifact = this.restore.finish()
|
||||
return {
|
||||
meta: this.meta,
|
||||
inheritedEventCount: inheritedCut(this.meta, this.events),
|
||||
events: this.events,
|
||||
inheritedEventCount: SessionLogOffset(artifact.inheritedEventCount),
|
||||
events: artifact.events as unknown as SessionEvent[],
|
||||
committedBytes: this.committedBytes,
|
||||
}
|
||||
}
|
||||
@@ -498,33 +474,36 @@ export class SessionLogScanner {
|
||||
/** Decode one complete event row and update the contiguous prefix. */
|
||||
private consumeEventLine(line: Buffer, endByte: number): void {
|
||||
this.eventLine += 1
|
||||
let decoded: SessionEvent[]
|
||||
let decoded: unknown
|
||||
try {
|
||||
decoded = [expandProvenanceFromStorage(JSON.parse(line.toString('utf8'))) as SessionEvent]
|
||||
decoded = JSON.parse(line.toString('utf8')) as unknown
|
||||
} catch {
|
||||
this.issue ??= new Error(`corrupt session log: unparsable committed event at line ${this.eventLine}`)
|
||||
const issue = new Error(`corrupt session log: unparsable committed event at line ${this.eventLine}`)
|
||||
if (this.recovery === 'strict') throw issue
|
||||
this.issue ??= issue
|
||||
return
|
||||
}
|
||||
|
||||
if (this.issue !== undefined) {
|
||||
if (decoded.some(event => event.type === 'turn/end')) throw this.issue
|
||||
if (typeof decoded === 'object' && decoded !== null
|
||||
&& (decoded as { type?: unknown }).type === 'turn/end') throw this.issue
|
||||
return
|
||||
}
|
||||
|
||||
const rowStart = this.events.length
|
||||
for (const event of decoded) {
|
||||
if (event.seq !== this.events.length) {
|
||||
const expected = this.events.length
|
||||
this.events.length = rowStart
|
||||
this.issue = new Error(
|
||||
`corrupt session log: seq gap in committed region at line ${this.eventLine} `
|
||||
+ `(expected ${expected}, got ${event.seq})`,
|
||||
)
|
||||
if (decoded.some(candidate => candidate.type === 'turn/end')) throw this.issue
|
||||
return
|
||||
}
|
||||
this.events.push(event)
|
||||
try {
|
||||
this.restore.decodeRow(decoded)
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- every production Session format decoder rejects with Error. */
|
||||
const detail = error instanceof Error ? error.message : String(error)
|
||||
const issue = new Error(`corrupt session log: invalid committed event at line ${this.eventLine}: ${detail}`, {
|
||||
cause: error,
|
||||
})
|
||||
if (this.recovery === 'strict') throw issue
|
||||
this.issue = issue
|
||||
if (typeof decoded === 'object' && decoded !== null
|
||||
&& (decoded as { type?: unknown }).type === 'turn/end') throw issue
|
||||
return
|
||||
}
|
||||
this.eventCount += 1
|
||||
this.committedBytes = endByte
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,33 +19,45 @@ import {
|
||||
type FileHandle,
|
||||
} from 'node:fs/promises'
|
||||
import { basename, dirname, join } from 'node:path'
|
||||
import { performance } from 'node:perf_hooks'
|
||||
import { pipeline, Readable } from 'node:stream'
|
||||
import { scheduler } from 'node:timers/promises'
|
||||
import { constants, createZstdCompress } from 'node:zlib'
|
||||
import { Session } from '@deepseek-ai/dsh-session'
|
||||
import type {
|
||||
SessionFormatArtifact,
|
||||
SessionFormatJsonValue,
|
||||
SessionFormatRestore,
|
||||
} from '@deepseek-ai/dsh-session-format'
|
||||
import { validateStoredEvents } from '@deepseek-ai/dsh-session-persistence'
|
||||
import type { JsonlCompression } from './format.ts'
|
||||
import { generationLogFilename, logSuffix } from './format.ts'
|
||||
import { generationLogFilename, logSuffix, SessionLogScanner } from './format.ts'
|
||||
import { publishNewFileWin32 } from './win32.ts'
|
||||
import {
|
||||
compressZstdFrame,
|
||||
createZstdFrameDecoder,
|
||||
decompressZstdFrame,
|
||||
decompressZstdPrefix,
|
||||
scanZstdFrames,
|
||||
} from './zstd.ts'
|
||||
|
||||
/** Parsed JSONL values supplied to the format catalog. */
|
||||
export interface JsonlDecodedGeneration {
|
||||
readonly header: Record<string, unknown>
|
||||
readonly rows: readonly unknown[]
|
||||
/** Internal scheduling bounds: preserve old decode cadence and cap each synchronous encode slice. */
|
||||
const MIGRATION_DECODE_YIELD_INTERVAL_MS = 500
|
||||
const MIGRATION_WORK_CHUNK_BYTES = 1024 * 1024
|
||||
const MIGRATION_WRITE_CHUNK_BYTES = 4 * 1024 * 1024
|
||||
const ZSTD_CHECKSUM_OPTIONS = {
|
||||
chunkSize: MIGRATION_WORK_CHUNK_BYTES,
|
||||
params: { [constants.ZSTD_c_checksumFlag]: 1 },
|
||||
}
|
||||
|
||||
/** Current JSONL values returned by the format catalog for physical encoding. */
|
||||
export interface JsonlCurrentGeneration extends JsonlDecodedGeneration {}
|
||||
|
||||
/** Pure adapter between backend-owned JSONL framing and the format catalog. */
|
||||
export interface JsonlGenerationFormatAdapter {
|
||||
readonly currentVersion: number
|
||||
/** Convert one detached historical generation to exact current JSON values. */
|
||||
migrate(source: JsonlDecodedGeneration): JsonlCurrentGeneration
|
||||
/** Validate one decoded current generation, including after committed reopen. */
|
||||
validateCurrent(candidate: JsonlCurrentGeneration): void
|
||||
/** Create the single-pass codec and migration state for a historical header. */
|
||||
createRestore(header: Record<string, unknown>): SessionFormatRestore
|
||||
/** Encode one current header record without materializing body rows. */
|
||||
encodeHeader(header: SessionFormatArtifact['header'], inheritedEventCount: number): SessionFormatJsonValue
|
||||
/** Encode one current event record. */
|
||||
encodeEvent(event: SessionFormatArtifact['events'][number]): SessionFormatJsonValue
|
||||
/** Classify a supported-version artifact that policy refuses to migrate. */
|
||||
isUnsupportedMigrationError?(error: unknown): error is Error
|
||||
}
|
||||
@@ -64,9 +76,31 @@ export interface EnsureJsonlGenerationOptions {
|
||||
readonly validateHistoricalHeader?: (
|
||||
header: Readonly<Record<string, unknown>>,
|
||||
) => void | Promise<void>
|
||||
/** Validate the staged file in an isolated worker before publication. */
|
||||
readonly verifyCurrentFile: (
|
||||
path: string,
|
||||
compression: JsonlCompression,
|
||||
expectedId: string,
|
||||
expectedEventCount: number,
|
||||
expectedPrefix?: JsonlExpectedPrefix,
|
||||
signal?: AbortSignal,
|
||||
) => Promise<JsonlVerifiedGeneration>
|
||||
readonly signal?: AbortSignal
|
||||
}
|
||||
|
||||
/** Small physical identity returned by an isolated generation verifier. */
|
||||
export interface JsonlVerifiedGeneration {
|
||||
readonly identity: JsonlPhysicalIdentity
|
||||
readonly bytes: number
|
||||
readonly digest: string
|
||||
}
|
||||
|
||||
/** Physical byte prefix already proven to be a valid complete generation. */
|
||||
export interface JsonlExpectedPrefix {
|
||||
readonly bytes: number
|
||||
readonly digest: string
|
||||
}
|
||||
|
||||
/** Result of current classification or exclusive publication. */
|
||||
export type EnsureJsonlGenerationResult =
|
||||
| {
|
||||
@@ -88,11 +122,6 @@ export type EnsureJsonlGenerationResult =
|
||||
export class JsonlGenerationNewerVersionError extends Error {
|
||||
override readonly name = 'JsonlGenerationNewerVersionError'
|
||||
|
||||
/**
|
||||
* @param storedVersion - version read from the highest stored generation.
|
||||
* @param currentVersion - version this build writes.
|
||||
* @param storedId - minimally decoded identity used in the refusal diagnostic.
|
||||
*/
|
||||
constructor(
|
||||
readonly storedVersion: number,
|
||||
readonly currentVersion: number,
|
||||
@@ -143,10 +172,8 @@ export interface JsonlPhysicalIdentity {
|
||||
readonly ctimeNs: bigint
|
||||
}
|
||||
|
||||
/** One revision-stable physical artifact read reusable by the immediate backend hook. */
|
||||
export interface JsonlPhysicalSnapshot {
|
||||
readonly bytes: Buffer
|
||||
readonly identity: JsonlPhysicalIdentity
|
||||
/** One revision-stable physical artifact returned to the immediate backend decoder. */
|
||||
export interface JsonlPhysicalSnapshot extends StablePhysicalFile {
|
||||
readonly headerValue: Record<string, unknown>
|
||||
readonly headerRecord: Buffer
|
||||
}
|
||||
@@ -162,11 +189,6 @@ interface JsonlPhysicalHeader {
|
||||
readonly record: Buffer
|
||||
}
|
||||
|
||||
interface DecodedPhysicalJsonl {
|
||||
readonly bytes: Buffer
|
||||
readonly torn: boolean
|
||||
}
|
||||
|
||||
interface GenerationFileSystem {
|
||||
open(path: string, flags: string, mode?: number): Promise<FileHandle>
|
||||
readFile(path: string, signal?: AbortSignal): Promise<Buffer>
|
||||
@@ -189,10 +211,24 @@ interface JsonlGenerationInternals {
|
||||
readonly barrier: (phase: GenerationBarrierPhase, attempt: number) => void | Promise<void>
|
||||
}
|
||||
|
||||
type JsonlGenerationTestOverrides = Partial<Omit<JsonlGenerationInternals, 'fs'>> & {
|
||||
/** Dependency overrides for an isolated generation runtime. */
|
||||
export type JsonlGenerationRuntimeOverrides = Partial<Omit<JsonlGenerationInternals, 'fs'>> & {
|
||||
readonly fs?: Partial<GenerationFileSystem>
|
||||
}
|
||||
|
||||
/** Bound generation operations used by production defaults and deterministic tests. */
|
||||
export interface JsonlGenerationRuntime {
|
||||
readStable(path: string, signal?: AbortSignal): Promise<StablePhysicalFile>
|
||||
ensure(options: EnsureJsonlGenerationOptions): Promise<EnsureJsonlGenerationResult>
|
||||
verify(
|
||||
path: string,
|
||||
compression: JsonlCompression,
|
||||
expectedId: string,
|
||||
expectedEventCount: number,
|
||||
expectedPrefix?: JsonlExpectedPrefix,
|
||||
): Promise<JsonlVerifiedGeneration>
|
||||
}
|
||||
|
||||
const defaultFileSystem: GenerationFileSystem = {
|
||||
open: (path, flags, mode) => fsOpen(path, flags, mode),
|
||||
readFile: (path, signal) => fsReadFile(path, signal === undefined ? undefined : { signal }),
|
||||
@@ -240,7 +276,7 @@ export async function readStableJsonlFile(
|
||||
path: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<StablePhysicalFile> {
|
||||
return readStableSnapshot(path, signal, defaultFileSystem)
|
||||
return defaultGenerationRuntime.readStable(path, signal)
|
||||
}
|
||||
|
||||
async function readStableSnapshot(
|
||||
@@ -289,33 +325,282 @@ function parseJson(text: string, subject: string): unknown {
|
||||
}
|
||||
}
|
||||
|
||||
function parseGeneration(bytes: Buffer, recoverSuffix = false): JsonlDecodedGeneration {
|
||||
/* v8 ignore next -- decodePhysicalJsonl supplies a non-empty newline-terminated prefix. */
|
||||
if (bytes.length === 0 || bytes.at(-1) !== 0x0A) {
|
||||
throw new Error('empty or header-less session log')
|
||||
/** Incremental JSONL parser that retains only one cross-frame record fragment. */
|
||||
class MigratingJsonlRows {
|
||||
private fragments: Buffer[] = []
|
||||
private fragmentBytes = 0
|
||||
private rowIndex = 0
|
||||
private issue: Error | undefined
|
||||
|
||||
constructor(private readonly restore: SessionFormatRestore) {}
|
||||
|
||||
/** Consume plaintext bytes following the independently decoded header. */
|
||||
write(chunk: Buffer): void {
|
||||
/* jscpd:ignore-start -- migration parsing and readable-log scanning own different recovery and byte-accounting state. */
|
||||
let lineStart = 0
|
||||
for (
|
||||
let newline = chunk.indexOf(0x0A);
|
||||
newline !== -1;
|
||||
newline = chunk.indexOf(0x0A, lineStart)
|
||||
) {
|
||||
const fragment = chunk.subarray(lineStart, newline)
|
||||
let line = fragment
|
||||
if (this.fragments.length > 0) {
|
||||
if (fragment.length > 0) this.fragments.push(fragment)
|
||||
line = Buffer.concat(this.fragments, this.fragmentBytes + fragment.length)
|
||||
this.fragments = []
|
||||
this.fragmentBytes = 0
|
||||
}
|
||||
this.consume(line)
|
||||
lineStart = newline + 1
|
||||
}
|
||||
if (lineStart < chunk.length) {
|
||||
const fragment = Buffer.from(chunk.subarray(lineStart))
|
||||
this.fragments.push(fragment)
|
||||
this.fragmentBytes += fragment.length
|
||||
}
|
||||
/* jscpd:ignore-end */
|
||||
}
|
||||
const records = bytes.toString('utf8').slice(0, -1).split('\n')
|
||||
const parsedHeader = parseJson(records[0] as string, 'header line')
|
||||
storedVersion(parsedHeader)
|
||||
const rows: unknown[] = []
|
||||
let issue: Error | undefined
|
||||
for (const [index, record] of records.slice(1).entries()) {
|
||||
|
||||
/** Refuse a record fragment left by structurally complete Zstandard frames. */
|
||||
assertCompleteFramesEndOnRecord(): void {
|
||||
if (this.fragments.length > 0) {
|
||||
throw new Error('corrupt Zstandard session log: complete frame contains a torn JSONL record')
|
||||
}
|
||||
}
|
||||
|
||||
finish(): SessionFormatArtifact {
|
||||
return this.restore.finish()
|
||||
}
|
||||
|
||||
private consume(line: Buffer): void {
|
||||
const index = this.rowIndex
|
||||
this.rowIndex += 1
|
||||
let row: unknown
|
||||
try {
|
||||
row = parseJson(record, `row ${index + 1}`)
|
||||
} catch (error) {
|
||||
if (!recoverSuffix) throw error
|
||||
issue ??= error as Error
|
||||
continue
|
||||
row = parseJson(line.toString('utf8'), `row ${index + 1}`)
|
||||
} catch (error: unknown) {
|
||||
this.issue ??= asError(error)
|
||||
return
|
||||
}
|
||||
if (issue !== undefined) {
|
||||
if (this.issue !== undefined) {
|
||||
if (typeof row === 'object' && row !== null
|
||||
&& (row as { type?: unknown }).type === 'turn/end') throw issue
|
||||
continue
|
||||
&& (row as { type?: unknown }).type === 'turn/end') throw this.issue
|
||||
return
|
||||
}
|
||||
rows.push(row)
|
||||
this.restore.decodeRow(row)
|
||||
}
|
||||
return { header: parsedHeader as Record<string, unknown>, rows }
|
||||
}
|
||||
|
||||
interface StartedMigrationStream {
|
||||
readonly parser: MigratingJsonlRows
|
||||
}
|
||||
|
||||
async function startMigrationStream(
|
||||
headerRecord: Buffer,
|
||||
format: JsonlGenerationFormatAdapter,
|
||||
validateHistoricalHeader?: EnsureJsonlGenerationOptions['validateHistoricalHeader'],
|
||||
): Promise<StartedMigrationStream> {
|
||||
const value = parseJson(headerRecord.subarray(0, -1).toString('utf8'), 'header line')
|
||||
const header = value as Record<string, unknown>
|
||||
const validation = validateHistoricalHeader?.(header)
|
||||
if (validation !== undefined) await validation
|
||||
const stream = format.createRestore(header)
|
||||
return { parser: new MigratingJsonlRows(stream) }
|
||||
}
|
||||
|
||||
async function consumeMigrationBytes(
|
||||
rows: MigratingJsonlRows,
|
||||
chunks: Iterable<Buffer>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
signal?.throwIfAborted()
|
||||
let yieldDeadline = performance.now() + MIGRATION_DECODE_YIELD_INTERVAL_MS
|
||||
for (const bytes of chunks) {
|
||||
for (let offset = 0; offset < bytes.length; offset += MIGRATION_WORK_CHUNK_BYTES) {
|
||||
rows.write(bytes.subarray(offset, offset + MIGRATION_WORK_CHUNK_BYTES))
|
||||
if (performance.now() < yieldDeadline) continue
|
||||
await scheduler.yield()
|
||||
signal?.throwIfAborted()
|
||||
yieldDeadline = performance.now() + MIGRATION_DECODE_YIELD_INTERVAL_MS
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function decodeStreamingMigration(
|
||||
bytes: Buffer,
|
||||
compression: JsonlCompression,
|
||||
format: JsonlGenerationFormatAdapter,
|
||||
validateHistoricalHeader: EnsureJsonlGenerationOptions['validateHistoricalHeader'],
|
||||
signal?: AbortSignal,
|
||||
): Promise<SessionFormatArtifact> {
|
||||
signal?.throwIfAborted()
|
||||
if (compression === 'none') {
|
||||
const headerEnd = bytes.indexOf(0x0A)
|
||||
/* v8 ignore next -- ensureCurrent's physical-header preflight already requires this newline. */
|
||||
if (headerEnd === -1) throw new Error('empty or header-less session log')
|
||||
const stream = await startMigrationStream(
|
||||
bytes.subarray(0, headerEnd + 1),
|
||||
format,
|
||||
validateHistoricalHeader,
|
||||
)
|
||||
signal?.throwIfAborted()
|
||||
const bodyEnd = bytes.lastIndexOf(0x0A)
|
||||
if (bodyEnd > headerEnd) {
|
||||
await consumeMigrationBytes(
|
||||
stream.parser,
|
||||
[bytes.subarray(headerEnd + 1, bodyEnd + 1)],
|
||||
signal,
|
||||
)
|
||||
}
|
||||
return stream.parser.finish()
|
||||
}
|
||||
|
||||
const { frames, tornStart } = scanZstdFrames(bytes)
|
||||
/* v8 ignore next -- ensureCurrent's physical-header preflight already requires a complete header frame. */
|
||||
if (frames.length === 0) throw new Error('empty or header-less Zstandard session log')
|
||||
const decoder = createZstdFrameDecoder()
|
||||
try {
|
||||
const decoded = decoder.decode(bytes, frames)
|
||||
const first = decoded.next()
|
||||
/* v8 ignore next -- a non-empty structural frame list yields once or throws. */
|
||||
if (first.done) throw new Error('empty or header-less Zstandard session log')
|
||||
assertIndependentHeaderFrame(first.value)
|
||||
const stream = await startMigrationStream(
|
||||
first.value,
|
||||
format,
|
||||
validateHistoricalHeader,
|
||||
)
|
||||
signal?.throwIfAborted()
|
||||
await consumeMigrationBytes(stream.parser, decoded, signal)
|
||||
stream.parser.assertCompleteFramesEndOnRecord()
|
||||
if (tornStart !== undefined) {
|
||||
let recovered: Buffer = Buffer.alloc(0)
|
||||
try {
|
||||
recovered = await decompressZstdPrefix(bytes.subarray(tornStart))
|
||||
} catch {
|
||||
/* v8 ignore next -- decoder failure plus concurrent abort is timing-dependent. */
|
||||
if (signal?.aborted) signal.throwIfAborted()
|
||||
}
|
||||
signal?.throwIfAborted()
|
||||
const newline = recovered.lastIndexOf(0x0A)
|
||||
if (newline !== -1) {
|
||||
await consumeMigrationBytes(
|
||||
stream.parser,
|
||||
[recovered.subarray(0, newline + 1)],
|
||||
signal,
|
||||
)
|
||||
}
|
||||
}
|
||||
return stream.parser.finish()
|
||||
} finally {
|
||||
decoder.close()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read and validate one complete current generation for an isolated verifier.
|
||||
* @param path - staged or competing current-generation path.
|
||||
* @param compression - configured physical encoding.
|
||||
* @param expectedId - Session identity expected in the header.
|
||||
* @param expectedEventCount - exact logical event count expected after decoding.
|
||||
* @param expectedPrefix - verified migration prefix; an append tail may be present and is not validated.
|
||||
* @returns stable physical identity and digest for publication comparison.
|
||||
*/
|
||||
export async function verifyJsonlCurrentGeneration(
|
||||
path: string,
|
||||
compression: JsonlCompression,
|
||||
expectedId: string,
|
||||
expectedEventCount: number,
|
||||
expectedPrefix?: JsonlExpectedPrefix,
|
||||
): Promise<JsonlVerifiedGeneration> {
|
||||
return defaultGenerationRuntime.verify(path, compression, expectedId, expectedEventCount, expectedPrefix)
|
||||
}
|
||||
|
||||
async function verifyCurrentGeneration(
|
||||
path: string,
|
||||
compression: JsonlCompression,
|
||||
expectedId: string,
|
||||
expectedEventCount: number,
|
||||
fs: GenerationFileSystem,
|
||||
expectedPrefix?: JsonlExpectedPrefix,
|
||||
): Promise<JsonlVerifiedGeneration> {
|
||||
const before = await fs.stat(path)
|
||||
const bytes = await fs.readFile(path)
|
||||
const after = await fs.stat(path)
|
||||
if (expectedPrefix !== undefined) {
|
||||
if (bytes.length < expectedPrefix.bytes) {
|
||||
throw new Error('target bytes are shorter than the migrated generation')
|
||||
}
|
||||
const digest = createHash('sha256').update(bytes.subarray(0, expectedPrefix.bytes)).digest('hex')
|
||||
if (digest !== expectedPrefix.digest) {
|
||||
throw new Error('target bytes do not begin with the migrated generation')
|
||||
}
|
||||
return { identity: after, bytes: expectedPrefix.bytes, digest }
|
||||
}
|
||||
if (identity(before) !== identity(after)) {
|
||||
throw new Error('current session generation changed during verification')
|
||||
}
|
||||
const snapshot = { bytes, identity: after }
|
||||
const generation = decodeCurrentGeneration(snapshot.bytes, compression)
|
||||
validateStoredEvents(generation.meta, generation.events, { kind: 'jsonl', path })
|
||||
if (generation.meta.id !== expectedId) {
|
||||
throw new Error(`current session generation contains id "${generation.meta.id}", expected "${expectedId}"`)
|
||||
}
|
||||
if (generation.events.length !== expectedEventCount) {
|
||||
throw new Error(
|
||||
`current session generation contains ${generation.events.length} events, expected ${expectedEventCount}`,
|
||||
)
|
||||
}
|
||||
Session.fromRestore(
|
||||
generation.meta.id,
|
||||
generation.events,
|
||||
generation.meta,
|
||||
generation.inheritedEventCount,
|
||||
)
|
||||
return {
|
||||
identity: snapshot.identity,
|
||||
bytes: snapshot.bytes.length,
|
||||
digest: createHash('sha256').update(snapshot.bytes).digest('hex'),
|
||||
}
|
||||
}
|
||||
|
||||
function decodeCurrentGeneration(
|
||||
bytes: Buffer,
|
||||
compression: JsonlCompression,
|
||||
): ReturnType<SessionLogScanner['finish']> {
|
||||
if (compression === 'none') {
|
||||
const headerEnd = bytes.indexOf(0x0A)
|
||||
if (headerEnd === -1) throw new Error('empty or header-less session log')
|
||||
const scanner = new SessionLogScanner(bytes.subarray(0, headerEnd + 1), 'strict')
|
||||
scanner.write(bytes.subarray(headerEnd + 1))
|
||||
return finishCurrentGenerationScan(scanner)
|
||||
}
|
||||
const { frames, tornStart } = scanZstdFrames(bytes)
|
||||
if (frames.length === 0) throw new Error('empty or header-less Zstandard session log')
|
||||
if (tornStart !== undefined) throw new Error('current session generation has a torn physical tail')
|
||||
const decoder = createZstdFrameDecoder()
|
||||
try {
|
||||
const plaintext = decoder.decode(bytes, frames)
|
||||
const header = plaintext.next()
|
||||
/* v8 ignore next -- a non-empty structural frame list yields once or throws. */
|
||||
if (header.done) throw new Error('empty or header-less Zstandard session log')
|
||||
assertIndependentHeaderFrame(header.value)
|
||||
const scanner = new SessionLogScanner(header.value, 'strict')
|
||||
for (const chunk of plaintext) scanner.write(chunk)
|
||||
return finishCurrentGenerationScan(scanner)
|
||||
} finally {
|
||||
decoder.close()
|
||||
}
|
||||
}
|
||||
|
||||
function finishCurrentGenerationScan(
|
||||
scanner: SessionLogScanner,
|
||||
): ReturnType<SessionLogScanner['finish']> {
|
||||
const inputBytes = scanner.checkpoint().inputBytes
|
||||
const decoded = scanner.finish()
|
||||
if (decoded.committedBytes !== inputBytes) throw new Error('current session generation has a torn physical tail')
|
||||
return decoded
|
||||
}
|
||||
|
||||
function stringifyJson(value: unknown, subject: string): string {
|
||||
@@ -329,82 +614,12 @@ function stringifyJson(value: unknown, subject: string): string {
|
||||
return text
|
||||
}
|
||||
|
||||
function encodeLogicalJsonl(generation: JsonlCurrentGeneration): Buffer {
|
||||
const records = [
|
||||
stringifyJson(generation.header, 'migrated session header'),
|
||||
...generation.rows.map((row, index) => stringifyJson(row, `migrated session row ${index + 1}`)),
|
||||
]
|
||||
return Buffer.from(`${records.join('\n')}\n`)
|
||||
}
|
||||
|
||||
function assertIndependentHeaderFrame(plaintext: Buffer): void {
|
||||
if (plaintext.length === 0 || plaintext.indexOf(0x0A) !== plaintext.length - 1) {
|
||||
throw new Error('corrupt Zstandard session log: first frame is not exactly one header line')
|
||||
}
|
||||
}
|
||||
|
||||
async function decodeZstdJsonl(bytes: Buffer, signal?: AbortSignal): Promise<DecodedPhysicalJsonl> {
|
||||
signal?.throwIfAborted()
|
||||
const { frames, tornStart } = scanZstdFrames(bytes)
|
||||
/* v8 ignore next -- the independent header probe already established the first frame. */
|
||||
if (frames.length === 0) throw new Error('empty or header-less Zstandard session log')
|
||||
const complete: Buffer[] = []
|
||||
for (const [index, frame] of frames.entries()) {
|
||||
signal?.throwIfAborted()
|
||||
const plaintext = await decompressZstdFrame(bytes.subarray(frame.start, frame.end))
|
||||
if (index === 0) assertIndependentHeaderFrame(plaintext)
|
||||
complete.push(plaintext)
|
||||
}
|
||||
const completeBytes = Buffer.concat(complete)
|
||||
if (completeBytes.at(-1) !== 0x0A) {
|
||||
throw new Error('corrupt Zstandard session log: complete frame contains a torn JSONL record')
|
||||
}
|
||||
if (tornStart === undefined) return { bytes: completeBytes, torn: false }
|
||||
|
||||
let recovered = Buffer.alloc(0)
|
||||
try {
|
||||
recovered = Buffer.from(await decompressZstdPrefix(bytes.subarray(tornStart)))
|
||||
} catch {
|
||||
/* v8 ignore next -- an abort racing decoder failure is timing-dependent */
|
||||
if (signal?.aborted) signal.throwIfAborted()
|
||||
// A structurally torn frame may produce no plaintext; prior frames remain valid.
|
||||
}
|
||||
signal?.throwIfAborted()
|
||||
const newline = recovered.lastIndexOf(0x0A)
|
||||
return {
|
||||
bytes: newline === -1
|
||||
? completeBytes
|
||||
: Buffer.concat([completeBytes, recovered.subarray(0, newline + 1)]),
|
||||
torn: true,
|
||||
}
|
||||
}
|
||||
|
||||
async function decodePhysicalJsonl(
|
||||
bytes: Buffer,
|
||||
compression: JsonlCompression,
|
||||
signal?: AbortSignal,
|
||||
): Promise<DecodedPhysicalJsonl> {
|
||||
if (compression === 'zstd') return decodeZstdJsonl(bytes, signal)
|
||||
signal?.throwIfAborted()
|
||||
const newline = bytes.lastIndexOf(0x0A)
|
||||
/* v8 ignore next -- physical header classification already found a newline in the same stable bytes. */
|
||||
if (newline === -1) throw new Error('empty or header-less session log')
|
||||
return { bytes: bytes.subarray(0, newline + 1), torn: newline + 1 !== bytes.length }
|
||||
}
|
||||
|
||||
async function encodePhysicalJsonl(
|
||||
logical: Buffer,
|
||||
generation: JsonlCurrentGeneration,
|
||||
compression: JsonlCompression,
|
||||
): Promise<Buffer> {
|
||||
if (compression === 'none') return logical
|
||||
const header = Buffer.from(`${stringifyJson(generation.header, 'migrated session header')}\n`)
|
||||
const headerFrame = await compressZstdFrame(header)
|
||||
if (generation.rows.length === 0) return headerFrame
|
||||
const body = logical.subarray(header.length)
|
||||
return Buffer.concat([headerFrame, await compressZstdFrame(body)])
|
||||
}
|
||||
|
||||
function readRawHeader(bytes: Buffer): JsonlPhysicalHeader {
|
||||
const newline = bytes.indexOf(0x0A)
|
||||
if (newline === -1) throw new Error('empty or header-less session log')
|
||||
@@ -441,8 +656,7 @@ function readPhysicalHeader(
|
||||
compression: JsonlCompression,
|
||||
signal: AbortSignal | undefined,
|
||||
): JsonlPhysicalHeader {
|
||||
if (compression === 'zstd') return readZstdHeader(bytes, signal)
|
||||
return readRawHeader(bytes)
|
||||
return compression === 'zstd' ? readZstdHeader(bytes, signal) : readRawHeader(bytes)
|
||||
}
|
||||
|
||||
function assertGenerationPaths(
|
||||
@@ -477,48 +691,127 @@ async function syncDirectory(path: string, internals: JsonlGenerationInternals):
|
||||
}
|
||||
}
|
||||
|
||||
interface StreamedMigrationStage {
|
||||
readonly path: string
|
||||
readonly bytes: number
|
||||
readonly digest: string
|
||||
}
|
||||
|
||||
/** Produce bounded JSONL chunks while yielding between main-thread encoding slices. */
|
||||
async function* encodeMigrationRows(
|
||||
artifact: SessionFormatArtifact,
|
||||
format: JsonlGenerationFormatAdapter,
|
||||
signal?: AbortSignal,
|
||||
): AsyncGenerator<Buffer, void, void> {
|
||||
signal?.throwIfAborted()
|
||||
let lines: string[] = []
|
||||
let bytes = 0
|
||||
for (const value of artifact.events) {
|
||||
const line = `${stringifyJson(format.encodeEvent(value), `migrated Session event ${value.seq}`)}\n`
|
||||
const lineBytes = Buffer.byteLength(line)
|
||||
if (bytes > 0 && bytes + lineBytes > MIGRATION_WORK_CHUNK_BYTES) {
|
||||
yield Buffer.from(lines.join(''))
|
||||
await scheduler.yield()
|
||||
signal?.throwIfAborted()
|
||||
lines = []
|
||||
bytes = 0
|
||||
}
|
||||
lines.push(line)
|
||||
bytes += lineBytes
|
||||
}
|
||||
yield Buffer.from(lines.join(''))
|
||||
}
|
||||
|
||||
async function writeMigrationChunks(
|
||||
chunks: AsyncIterable<Buffer>,
|
||||
write: (chunk: Buffer) => Promise<void>,
|
||||
): Promise<void> {
|
||||
let pending: Buffer[] = []
|
||||
let bytes = 0
|
||||
for await (const chunk of chunks) {
|
||||
pending.push(chunk)
|
||||
bytes += chunk.length
|
||||
if (bytes < MIGRATION_WRITE_CHUNK_BYTES) continue
|
||||
await write(pending.length === 1 ? pending[0] as Buffer : Buffer.concat(pending, bytes))
|
||||
pending = []
|
||||
bytes = 0
|
||||
}
|
||||
if (bytes > 0) await write(pending.length === 1 ? pending[0] as Buffer : Buffer.concat(pending, bytes))
|
||||
}
|
||||
|
||||
/** Encode directly into one synced stage without a whole-artifact row or byte buffer. */
|
||||
async function writeSyncedTemp(
|
||||
currentPath: string,
|
||||
suffix: string,
|
||||
bytes: Buffer,
|
||||
compression: JsonlCompression,
|
||||
artifact: SessionFormatArtifact,
|
||||
format: JsonlGenerationFormatAdapter,
|
||||
signal: AbortSignal | undefined,
|
||||
internals: JsonlGenerationInternals,
|
||||
): Promise<string> {
|
||||
): Promise<StreamedMigrationStage> {
|
||||
signal?.throwIfAborted()
|
||||
let path: string
|
||||
let handle: FileHandle
|
||||
for (;;) {
|
||||
const path = join(dirname(currentPath), `session.migration.${internals.randomToken()}${suffix}.tmp`)
|
||||
let handle: FileHandle
|
||||
path = join(dirname(currentPath), `session.migration.${internals.randomToken()}${suffix}.tmp`)
|
||||
try {
|
||||
handle = await internals.fs.open(path, 'wx', 0o600)
|
||||
break
|
||||
} catch (error) {
|
||||
if (isEEXIST(error)) continue
|
||||
throw error
|
||||
}
|
||||
let failure: unknown
|
||||
try {
|
||||
await handle.writeFile(bytes)
|
||||
await handle.sync()
|
||||
} catch (error: unknown) {
|
||||
failure = error
|
||||
}
|
||||
try {
|
||||
await handle.close()
|
||||
} catch (error: unknown) {
|
||||
failure = failure === undefined
|
||||
? error
|
||||
: new AggregateError([failure, error], `failed to write and close migration stage "${path}"`)
|
||||
}
|
||||
if (failure !== undefined) {
|
||||
const writeError = failure instanceof Error
|
||||
? failure
|
||||
: new Error('migration stage write failed with a non-Error rejection', { cause: failure })
|
||||
try {
|
||||
await internals.fs.rm(path)
|
||||
} catch (cleanupError: unknown) {
|
||||
throw new AggregateError([writeError, cleanupError], `failed to clean migration stage "${path}"`)
|
||||
}
|
||||
throw writeError
|
||||
}
|
||||
return path
|
||||
}
|
||||
const hash = createHash('sha256')
|
||||
let bytes = 0
|
||||
const write = async (chunk: Buffer): Promise<void> => {
|
||||
await handle.writeFile(chunk)
|
||||
hash.update(chunk)
|
||||
bytes += chunk.length
|
||||
}
|
||||
let failure: unknown
|
||||
try {
|
||||
const headerValue = format.encodeHeader(artifact.header, artifact.inheritedEventCount)
|
||||
const header = Buffer.from(`${stringifyJson(headerValue, 'migrated session header')}\n`)
|
||||
await write(compression === 'zstd' ? await compressZstdFrame(header) : header)
|
||||
if (artifact.events.length > 0) {
|
||||
const rows = encodeMigrationRows(artifact, format, signal)
|
||||
if (compression === 'none') {
|
||||
await writeMigrationChunks(rows, write)
|
||||
} else {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
pipeline(
|
||||
Readable.from(rows, { objectMode: false, highWaterMark: MIGRATION_WORK_CHUNK_BYTES }),
|
||||
createZstdCompress(ZSTD_CHECKSUM_OPTIONS),
|
||||
async (source) => { await writeMigrationChunks(source as AsyncIterable<Buffer>, write) },
|
||||
(error: Error | null | undefined) => {
|
||||
if (error instanceof Error) reject(error)
|
||||
else resolve()
|
||||
},
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
signal?.throwIfAborted()
|
||||
await handle.sync()
|
||||
} catch (error: unknown) {
|
||||
failure = error
|
||||
}
|
||||
try {
|
||||
await handle.close()
|
||||
} catch (error: unknown) {
|
||||
failure = failure === undefined
|
||||
? error
|
||||
: new AggregateError([failure, error], `failed to write and close migration stage "${path}"`)
|
||||
}
|
||||
if (failure !== undefined) {
|
||||
const writeError = failure instanceof Error
|
||||
? failure
|
||||
: new Error('migration stage write failed with a non-Error rejection', { cause: failure })
|
||||
await removeTemporary(path, writeError, internals)
|
||||
throw writeError
|
||||
}
|
||||
return { path, bytes, digest: hash.digest('hex') }
|
||||
}
|
||||
|
||||
/** Remove one temporary file without hiding the operation failure that made it disposable. */
|
||||
@@ -550,31 +843,6 @@ async function removeCommittedTemporary(
|
||||
}
|
||||
}
|
||||
|
||||
async function validatePhysicalCurrent(
|
||||
path: string,
|
||||
compression: JsonlCompression,
|
||||
format: JsonlGenerationFormatAdapter,
|
||||
signal: AbortSignal | undefined,
|
||||
internals: JsonlGenerationInternals,
|
||||
): Promise<JsonlPhysicalSnapshot> {
|
||||
const snapshot = await readStableSnapshot(path, signal, internals.fs)
|
||||
const decoded = await decodePhysicalJsonl(snapshot.bytes, compression, signal)
|
||||
if (decoded.torn) throw new Error('staged current session generation has a torn physical tail')
|
||||
const generation = parseGeneration(decoded.bytes)
|
||||
if (storedVersion(generation.header) !== format.currentVersion) {
|
||||
throw new Error(`staged session generation is not current v${format.currentVersion}`)
|
||||
}
|
||||
format.validateCurrent(generation)
|
||||
const headerEnd = decoded.bytes.indexOf(0x0A)
|
||||
/* v8 ignore next -- parseGeneration already required the header newline. */
|
||||
if (headerEnd === -1) throw new Error('empty or header-less session log')
|
||||
return {
|
||||
...snapshot,
|
||||
headerValue: generation.header,
|
||||
headerRecord: Buffer.from(decoded.bytes.subarray(0, headerEnd + 1)),
|
||||
}
|
||||
}
|
||||
|
||||
async function publishCurrentExclusive(
|
||||
staged: string,
|
||||
currentPath: string,
|
||||
@@ -609,15 +877,12 @@ function asError(error: unknown): Error {
|
||||
})
|
||||
}
|
||||
|
||||
async function reopenExpectedCurrent(
|
||||
async function inspectExpectedCurrent<T>(
|
||||
currentPath: string,
|
||||
expectedBytes: Buffer,
|
||||
compression: JsonlCompression,
|
||||
format: JsonlGenerationFormatAdapter,
|
||||
signal: AbortSignal | undefined,
|
||||
checkCanonicalTargetName: boolean,
|
||||
internals: JsonlGenerationInternals,
|
||||
): Promise<JsonlPhysicalSnapshot> {
|
||||
inspect: () => Promise<T>,
|
||||
): Promise<T> {
|
||||
try {
|
||||
if (checkCanonicalTargetName) {
|
||||
const expectedName = basename(currentPath)
|
||||
@@ -631,23 +896,16 @@ async function reopenExpectedCurrent(
|
||||
}
|
||||
const info = await internals.fs.lstat(currentPath)
|
||||
if (info.isSymbolicLink() || !info.isFile()) {
|
||||
const kind = info.isSymbolicLink() ? 'symbolic link' : 'non-regular file'
|
||||
throw new Error(`target is a ${kind}`)
|
||||
throw new Error(`target is a ${info.isSymbolicLink() ? 'symbolic link' : 'non-regular file'}`)
|
||||
}
|
||||
const snapshot = await validatePhysicalCurrent(currentPath, compression, format, signal, internals)
|
||||
if (snapshot.bytes.length < expectedBytes.length
|
||||
|| !snapshot.bytes.subarray(0, expectedBytes.length).equals(expectedBytes)) {
|
||||
throw new Error('target bytes do not begin with the migrated generation')
|
||||
}
|
||||
return snapshot
|
||||
return await inspect()
|
||||
} catch (error: unknown) {
|
||||
if (signal?.aborted) signal.throwIfAborted()
|
||||
if (isErrnoException(error)) throw error
|
||||
throw new JsonlGenerationTargetConflictError(currentPath, asError(error))
|
||||
}
|
||||
}
|
||||
|
||||
function withOverrides(overrides: JsonlGenerationTestOverrides): JsonlGenerationInternals {
|
||||
function withOverrides(overrides: JsonlGenerationRuntimeOverrides): JsonlGenerationInternals {
|
||||
return {
|
||||
...defaultInternals,
|
||||
...overrides,
|
||||
@@ -655,6 +913,35 @@ function withOverrides(overrides: JsonlGenerationTestOverrides): JsonlGeneration
|
||||
}
|
||||
}
|
||||
|
||||
async function reopenExpectedCurrent(
|
||||
currentPath: string,
|
||||
staged: StreamedMigrationStage,
|
||||
compression: JsonlCompression,
|
||||
expectedId: string,
|
||||
expectedEventCount: number,
|
||||
verifyCurrentFile: EnsureJsonlGenerationOptions['verifyCurrentFile'],
|
||||
signal: AbortSignal | undefined,
|
||||
checkCanonicalTargetName: boolean,
|
||||
internals: JsonlGenerationInternals,
|
||||
): Promise<JsonlPhysicalSnapshot> {
|
||||
return inspectExpectedCurrent(currentPath, checkCanonicalTargetName, internals, async () => {
|
||||
const verified = await verifyCurrentFile(
|
||||
currentPath,
|
||||
compression,
|
||||
expectedId,
|
||||
expectedEventCount,
|
||||
staged,
|
||||
signal,
|
||||
)
|
||||
if (verified.bytes !== staged.bytes || verified.digest !== staged.digest) {
|
||||
throw new Error('target bytes differ from the migrated generation')
|
||||
}
|
||||
const snapshot = await readStableSnapshot(currentPath, signal, internals.fs)
|
||||
const header = readPhysicalHeader(snapshot.bytes, compression, signal)
|
||||
return { ...snapshot, headerValue: header.value, headerRecord: header.record }
|
||||
})
|
||||
}
|
||||
|
||||
async function ensureCurrent(
|
||||
options: EnsureJsonlGenerationOptions,
|
||||
internals: JsonlGenerationInternals,
|
||||
@@ -681,68 +968,87 @@ async function ensureCurrent(
|
||||
)
|
||||
}
|
||||
if (sourceVersion > format.currentVersion) {
|
||||
throw new JsonlGenerationNewerVersionError(sourceVersion, format.currentVersion, storedId(quickHeader.value))
|
||||
throw new JsonlGenerationNewerVersionError(
|
||||
sourceVersion,
|
||||
format.currentVersion,
|
||||
storedId(quickHeader.value),
|
||||
)
|
||||
}
|
||||
if (sourceVersion === format.currentVersion) {
|
||||
return {
|
||||
status: 'current',
|
||||
version: quickVersion,
|
||||
path: sourcePath,
|
||||
snapshot: { ...source, headerValue: quickHeader.value, headerRecord: quickHeader.record },
|
||||
snapshot: {
|
||||
...source,
|
||||
headerValue: quickHeader.value,
|
||||
headerRecord: quickHeader.record,
|
||||
},
|
||||
}
|
||||
}
|
||||
const validation = options.validateHistoricalHeader?.(quickHeader.value)
|
||||
if (validation !== undefined) await validation
|
||||
|
||||
const decodedSource = await decodePhysicalJsonl(source.bytes, compression, signal)
|
||||
const parsedSource = parseGeneration(decodedSource.bytes, true)
|
||||
const fromVersion = storedVersion(parsedSource.header)
|
||||
/* v8 ignore next -- both headers come from the same stable physical snapshot. */
|
||||
if (fromVersion !== quickVersion) throw new Error('session format changed within one stable physical snapshot')
|
||||
const sourceFingerprint = fingerprint(source.identity, source.bytes)
|
||||
|
||||
let migrated: JsonlCurrentGeneration
|
||||
let artifact: SessionFormatArtifact
|
||||
try {
|
||||
migrated = format.migrate(parsedSource)
|
||||
artifact = await decodeStreamingMigration(
|
||||
source.bytes,
|
||||
compression,
|
||||
format,
|
||||
options.validateHistoricalHeader,
|
||||
signal,
|
||||
)
|
||||
} catch (error: unknown) {
|
||||
if (format.isUnsupportedMigrationError?.(error) === true) {
|
||||
throw new JsonlGenerationUnsupportedMigrationError(fromVersion, error)
|
||||
throw new JsonlGenerationUnsupportedMigrationError(sourceVersion, error)
|
||||
}
|
||||
throw error
|
||||
}
|
||||
if (storedVersion(migrated.header) !== format.currentVersion) {
|
||||
throw new Error(`format migration returned v${storedVersion(migrated.header)}, expected v${format.currentVersion}`)
|
||||
if (artifact.header.version !== format.currentVersion) {
|
||||
throw new Error(`format migration returned v${artifact.header.version}, expected v${format.currentVersion}`)
|
||||
}
|
||||
const logical = encodeLogicalJsonl(migrated)
|
||||
const physical = await encodePhysicalJsonl(logical, migrated, compression)
|
||||
let staged = await writeSyncedTemp(currentPath, suffix, physical, internals)
|
||||
|
||||
await scheduler.yield()
|
||||
signal?.throwIfAborted()
|
||||
const sourceFingerprint = fingerprint(source.identity, source.bytes)
|
||||
const eventCount = artifact.events.length
|
||||
let staged = await writeSyncedTemp(currentPath, suffix, compression, artifact, format, signal, internals)
|
||||
let failure: unknown
|
||||
try {
|
||||
await validatePhysicalCurrent(staged, compression, format, signal, internals)
|
||||
const verifiedStage = await options.verifyCurrentFile(
|
||||
staged.path,
|
||||
compression,
|
||||
artifact.header.id,
|
||||
eventCount,
|
||||
undefined,
|
||||
signal,
|
||||
)
|
||||
if (verifiedStage.bytes !== staged.bytes || verifiedStage.digest !== staged.digest) {
|
||||
throw new Error('staged session generation changed during verification')
|
||||
}
|
||||
await internals.barrier('before-source-check', attempt)
|
||||
const beforePublish = await readStableSnapshot(sourcePath, signal, internals.fs)
|
||||
if (fingerprint(beforePublish.identity, beforePublish.bytes) !== sourceFingerprint) continue
|
||||
|
||||
const published = await publishCurrentExclusive(staged, currentPath, internals)
|
||||
if (published && internals.platform === 'win32') staged = ''
|
||||
const published = await publishCurrentExclusive(staged.path, currentPath, internals)
|
||||
if (published && internals.platform === 'win32') staged = { ...staged, path: '' }
|
||||
await internals.barrier('after-publication', attempt)
|
||||
signal?.throwIfAborted()
|
||||
const committed = await reopenExpectedCurrent(
|
||||
currentPath,
|
||||
physical,
|
||||
staged,
|
||||
compression,
|
||||
format,
|
||||
artifact.header.id,
|
||||
eventCount,
|
||||
options.verifyCurrentFile,
|
||||
signal,
|
||||
!published,
|
||||
internals,
|
||||
)
|
||||
if (staged !== '') {
|
||||
await removeCommittedTemporary(staged, internals)
|
||||
staged = ''
|
||||
if (staged.path !== '') {
|
||||
await removeCommittedTemporary(staged.path, internals)
|
||||
staged = { ...staged, path: '' }
|
||||
}
|
||||
return {
|
||||
status: 'migrated',
|
||||
fromVersion,
|
||||
fromVersion: sourceVersion,
|
||||
toVersion: format.currentVersion,
|
||||
path: currentPath,
|
||||
sourcePath,
|
||||
@@ -752,32 +1058,38 @@ async function ensureCurrent(
|
||||
failure = error
|
||||
throw error
|
||||
} finally {
|
||||
if (staged !== '') await removeTemporary(staged, failure, internals)
|
||||
if (staged.path !== '') await removeTemporary(staged.path, failure, internals)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure one resolved generation has a current-format successor. Current input reads one
|
||||
* coherent physical snapshot, inspects only its independently readable header,
|
||||
* invokes no body decoder or migration callback, and returns that snapshot for
|
||||
* the immediate body-reading backend hook. Historical input remains unchanged;
|
||||
* only a previously absent current filename can be published.
|
||||
* @param options - resolved source and target, configured encoding, format adapter, and cancellation.
|
||||
* @returns whether the source was already current or which immutable successor was published.
|
||||
* Ensure one resolved generation has a current-format successor before returning.
|
||||
* @param options - resolved source, current target, format adapter, verification, and cancellation.
|
||||
* @returns the current source or the verified and reopened migrated successor.
|
||||
*/
|
||||
export function ensureJsonlGenerationCurrent(
|
||||
options: EnsureJsonlGenerationOptions,
|
||||
): Promise<EnsureJsonlGenerationResult> {
|
||||
return ensureCurrent(options, defaultInternals)
|
||||
return defaultGenerationRuntime.ensure(options)
|
||||
}
|
||||
|
||||
/** Private deterministic filesystem, platform, and race seams for package tests. */
|
||||
export const __jsonlGenerationTest = {
|
||||
ensure(
|
||||
options: EnsureJsonlGenerationOptions,
|
||||
overrides: JsonlGenerationTestOverrides,
|
||||
): Promise<EnsureJsonlGenerationResult> {
|
||||
return ensureCurrent(options, withOverrides(overrides))
|
||||
},
|
||||
/**
|
||||
* Create one generation runtime with fixed filesystem and publication dependencies.
|
||||
* @param overrides - deterministic filesystem, platform, and race dependencies.
|
||||
* @returns bound generation operations.
|
||||
*/
|
||||
export function createJsonlGenerationRuntime(
|
||||
overrides: JsonlGenerationRuntimeOverrides = {},
|
||||
): JsonlGenerationRuntime {
|
||||
const internals = withOverrides(overrides)
|
||||
return {
|
||||
readStable: (path, signal) => readStableSnapshot(path, signal, internals.fs),
|
||||
ensure: options => ensureCurrent(options, internals),
|
||||
verify: (path, compression, expectedId, expectedEventCount, expectedPrefix) => verifyCurrentGeneration(
|
||||
path, compression, expectedId, expectedEventCount, internals.fs, expectedPrefix,
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
const defaultGenerationRuntime = createJsonlGenerationRuntime()
|
||||
@@ -42,6 +42,7 @@ import {
|
||||
compressZstdFrame, createZstdFrameDecoder, decompressZstdFrame, decompressZstdPrefix, scanZstdFrames,
|
||||
} from './zstd.ts'
|
||||
import { ensureDurableDirectoryWin32, publishNewFileWin32 } from './win32.ts'
|
||||
import { verifyCurrentGenerationInWorker } from './migration-verifier.ts'
|
||||
import {
|
||||
ensureJsonlGenerationCurrent,
|
||||
JsonlGenerationNewerVersionError,
|
||||
@@ -180,15 +181,13 @@ class JsonlSessionPersistence extends SessionPersistence {
|
||||
this.compression = config.compression ?? DEFAULT_COMPRESSION
|
||||
this.generationFormat = {
|
||||
currentVersion: sessionFormatCatalog.currentVersion,
|
||||
migrate: (source) => {
|
||||
const decoded = sessionFormatCatalog.decodeRecoverableArtifact(source.header, source.rows)
|
||||
const current = sessionFormatCatalog.migrate(decoded)
|
||||
return sessionFormatCatalog.encodeCurrent(current)
|
||||
},
|
||||
validateCurrent: (candidate) => {
|
||||
const decoded = sessionFormatCatalog.decodeArtifact(candidate.header, candidate.rows)
|
||||
sessionFormatCatalog.migrate(decoded)
|
||||
},
|
||||
createRestore: header => sessionFormatCatalog.createRestore(header, {
|
||||
recovery: 'recoverable',
|
||||
validation: 'transformed',
|
||||
}),
|
||||
encodeHeader: (header, inheritedEventCount) =>
|
||||
sessionFormatCatalog.encodeCurrentHeader(header, inheritedEventCount),
|
||||
encodeEvent: event => sessionFormatCatalog.encodeCurrentEvent(event),
|
||||
isUnsupportedMigrationError: (error): error is SessionFormatUnsupportedMigrationError =>
|
||||
error instanceof SessionFormatUnsupportedMigrationError,
|
||||
}
|
||||
@@ -397,8 +396,6 @@ class JsonlSessionPersistence extends SessionPersistence {
|
||||
}
|
||||
}
|
||||
const current = await this.ensureCurrentLog(id, signal, selected)
|
||||
/* v8 ignore next -- supplying a resolved generation makes absence unreachable. */
|
||||
if (current === undefined) throw new SessionPersistenceNotFoundError(id)
|
||||
return this.decodeStoredLog(
|
||||
current.path,
|
||||
id,
|
||||
@@ -411,12 +408,10 @@ class JsonlSessionPersistence extends SessionPersistence {
|
||||
/** Select and, when required, publish one immutable current generation. */
|
||||
private async ensureCurrentLog(
|
||||
id: SessionId,
|
||||
signal?: AbortSignal,
|
||||
resolved?: ResolvedJsonlGeneration,
|
||||
): Promise<EnsureJsonlGenerationResult | undefined> {
|
||||
signal: AbortSignal | undefined,
|
||||
selected: ResolvedJsonlGeneration,
|
||||
): Promise<EnsureJsonlGenerationResult> {
|
||||
signal?.throwIfAborted()
|
||||
const selected = resolved ?? await this.findLog(id, signal)
|
||||
if (selected === undefined) return undefined
|
||||
try {
|
||||
return await ensureJsonlGenerationCurrent({
|
||||
sourcePath: selected.sourcePath,
|
||||
@@ -424,6 +419,7 @@ class JsonlSessionPersistence extends SessionPersistence {
|
||||
currentPath: selected.currentPath,
|
||||
compression: this.compression,
|
||||
format: this.generationFormat,
|
||||
verifyCurrentFile: verifyCurrentGenerationInWorker,
|
||||
validateHistoricalHeader: headerValue => this.validateSourceIdentity(
|
||||
selected,
|
||||
headerValue,
|
||||
@@ -549,8 +545,8 @@ class JsonlSessionPersistence extends SessionPersistence {
|
||||
const selected = await this.findLog(id, signal)
|
||||
if (selected === undefined) return undefined
|
||||
if (selected.sourceVersion === SESSION_FORMAT_VERSION) return selected.sourcePath
|
||||
const current = await this.ensureCurrentLog(id, signal)
|
||||
return current?.path
|
||||
const current = await this.ensureCurrentLog(id, signal, selected)
|
||||
return current.path
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -794,8 +790,14 @@ class JsonlSessionPersistence extends SessionPersistence {
|
||||
)
|
||||
}
|
||||
if (result.status === 'unsupported') {
|
||||
const physicalId = String((value as { id?: unknown }).id)
|
||||
let reason = result.reason
|
||||
/* v8 ignore else -- released historical header migrations cannot refuse after physical decoding. */
|
||||
if (result.storedVersion > SESSION_FORMAT_VERSION) {
|
||||
reason = sessionFormatVersionRefusal(physicalId, result.storedVersion)
|
||||
}
|
||||
throw new SessionFormatUnsupportedError(
|
||||
`${result.reason} (raw log: ${selected.sourcePath})`,
|
||||
`${reason} (raw log: ${selected.sourcePath})`,
|
||||
{ kind: 'jsonl', path: selected.sourcePath },
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,193 @@
|
||||
/** Isolated verification for a staged or competing current JSONL generation. */
|
||||
|
||||
import { Worker } from 'node:worker_threads'
|
||||
import type { WorkerOptions } from 'node:worker_threads'
|
||||
import type { JsonlCompression } from './format.ts'
|
||||
import type { JsonlExpectedPrefix, JsonlVerifiedGeneration } from './generation.ts'
|
||||
|
||||
interface VerificationRequest {
|
||||
readonly path: string
|
||||
readonly compression: JsonlCompression
|
||||
readonly expectedId: string
|
||||
readonly expectedEventCount: number
|
||||
readonly expectedPrefix?: JsonlExpectedPrefix
|
||||
}
|
||||
|
||||
type VerificationResponse =
|
||||
| { readonly ok: true; readonly result: JsonlVerifiedGeneration }
|
||||
| { readonly ok: false; readonly message: string; readonly stack?: string }
|
||||
|
||||
/** Process-wide memory bound for full-generation verification isolates. */
|
||||
const MAX_CONCURRENT_VERIFIERS = 2
|
||||
|
||||
class VerificationScheduler {
|
||||
private active = 0
|
||||
private readonly waiting: Array<{ grant(): void }> = []
|
||||
|
||||
async run<T>(operation: () => Promise<T>, signal?: AbortSignal): Promise<T> {
|
||||
const permit = this.acquire(signal)
|
||||
if (permit !== undefined) await permit
|
||||
try {
|
||||
signal?.throwIfAborted()
|
||||
return await operation()
|
||||
} finally {
|
||||
this.release()
|
||||
}
|
||||
}
|
||||
|
||||
private acquire(signal?: AbortSignal): Promise<void> | undefined {
|
||||
signal?.throwIfAborted()
|
||||
if (this.active < MAX_CONCURRENT_VERIFIERS) {
|
||||
this.active += 1
|
||||
return
|
||||
}
|
||||
return new Promise<void>((resolve, reject) => {
|
||||
const waiter = {
|
||||
grant: (): void => {
|
||||
signal?.removeEventListener('abort', abort)
|
||||
resolve()
|
||||
},
|
||||
}
|
||||
const abort = (): void => {
|
||||
const index = this.waiting.indexOf(waiter)
|
||||
this.waiting.splice(index, 1)
|
||||
reject(verifierAbortError(signal))
|
||||
}
|
||||
this.waiting.push(waiter)
|
||||
signal?.addEventListener('abort', abort, { once: true })
|
||||
})
|
||||
}
|
||||
|
||||
private release(): void {
|
||||
const next = this.waiting.shift()
|
||||
if (next === undefined) {
|
||||
this.active -= 1
|
||||
return
|
||||
}
|
||||
next.grant()
|
||||
}
|
||||
}
|
||||
|
||||
const verificationScheduler = new VerificationScheduler()
|
||||
|
||||
function workerSpawn(request: VerificationRequest): { readonly entry: string | URL; readonly options: WorkerOptions } {
|
||||
/* v8 ignore next 3 -- built-worker coverage owns the bundled path. */
|
||||
if (!import.meta.url.endsWith('.ts')) {
|
||||
return {
|
||||
entry: new URL('./worker.cjs', import.meta.url),
|
||||
options: { workerData: request, execArgv: [] },
|
||||
}
|
||||
}
|
||||
const workerEntry = new URL('./worker.ts', import.meta.url)
|
||||
const bootstrap = [
|
||||
`import { register as registerEsm } from ${JSON.stringify(import.meta.resolve('tsx/esm/api'))}`,
|
||||
`import { register as registerCjs } from ${JSON.stringify(import.meta.resolve('tsx/cjs/api'))}`,
|
||||
'registerCjs()',
|
||||
'registerEsm()',
|
||||
`await import(${JSON.stringify(workerEntry.href)})`,
|
||||
].join('\n')
|
||||
return {
|
||||
entry: new URL(`data:text/javascript,${encodeURIComponent(bootstrap)}`),
|
||||
options: {
|
||||
workerData: request,
|
||||
execArgv: [],
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify one current generation in a fresh Worker Thread.
|
||||
* @param path - staged or competing current-generation path.
|
||||
* @param compression - configured physical encoding.
|
||||
* @param expectedId - Session id expected in the decoded header.
|
||||
* @param expectedEventCount - exact logical event count expected after decoding.
|
||||
* @param expectedPrefix - verified physical prefix; an append tail may be present and is not validated.
|
||||
* @param signal - optional cancellation for scheduler wait and Worker execution.
|
||||
* @returns stable physical identity and digest observed by the worker.
|
||||
*/
|
||||
export function verifyCurrentGenerationInWorker(
|
||||
path: string,
|
||||
compression: JsonlCompression,
|
||||
expectedId: string,
|
||||
expectedEventCount: number,
|
||||
expectedPrefix?: JsonlExpectedPrefix,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JsonlVerifiedGeneration> {
|
||||
return verificationScheduler.run(() => runVerificationWorker(
|
||||
path,
|
||||
compression,
|
||||
expectedId,
|
||||
expectedEventCount,
|
||||
expectedPrefix,
|
||||
signal,
|
||||
), signal)
|
||||
}
|
||||
|
||||
function runVerificationWorker(
|
||||
path: string,
|
||||
compression: JsonlCompression,
|
||||
expectedId: string,
|
||||
expectedEventCount: number,
|
||||
expectedPrefix?: JsonlExpectedPrefix,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JsonlVerifiedGeneration> {
|
||||
signal?.throwIfAborted()
|
||||
const request: VerificationRequest = {
|
||||
path, compression, expectedId, expectedEventCount,
|
||||
...(expectedPrefix === undefined ? {} : { expectedPrefix }),
|
||||
}
|
||||
const { entry, options } = workerSpawn(request)
|
||||
const worker = new Worker(entry, options)
|
||||
return new Promise((resolve, reject) => {
|
||||
let settled = false
|
||||
const cleanup = (): void => {
|
||||
signal?.removeEventListener('abort', abort)
|
||||
}
|
||||
const fail = (error: Error): void => {
|
||||
/* v8 ignore next -- a late error/exit races only after another terminal callback settled. */
|
||||
if (settled) return
|
||||
settled = true
|
||||
cleanup()
|
||||
void worker.terminate().then(
|
||||
() => { reject(error) },
|
||||
(cleanup: unknown) => {
|
||||
reject(new AggregateError([error, cleanup], 'migration verifier termination failed'))
|
||||
},
|
||||
)
|
||||
}
|
||||
worker.once('message', (value: unknown) => {
|
||||
/* v8 ignore next -- a duplicate message races only after another terminal callback settled. */
|
||||
if (settled) return
|
||||
if (typeof value !== 'object' || value === null || typeof (value as { ok?: unknown }).ok !== 'boolean') {
|
||||
fail(new Error('migration verifier returned an invalid response'))
|
||||
return
|
||||
}
|
||||
const response = value as VerificationResponse
|
||||
if (!response.ok) {
|
||||
const error = new Error(response.message)
|
||||
if (response.stack !== undefined) error.stack = response.stack
|
||||
fail(error)
|
||||
return
|
||||
}
|
||||
settled = true
|
||||
cleanup()
|
||||
void worker.terminate().then(
|
||||
() => { resolve(response.result) },
|
||||
(error: unknown) => { reject(error instanceof Error ? error : new Error(String(error))) },
|
||||
)
|
||||
})
|
||||
worker.once('error', fail)
|
||||
worker.once('exit', (code) => {
|
||||
if (!settled) fail(new Error(`migration verifier exited before reporting a result (code ${code})`))
|
||||
})
|
||||
const abort = (): void => { fail(verifierAbortError(signal)) }
|
||||
signal?.addEventListener('abort', abort, { once: true })
|
||||
})
|
||||
}
|
||||
|
||||
function verifierAbortError(signal?: AbortSignal): Error {
|
||||
const reason: unknown = signal?.reason
|
||||
return reason instanceof Error
|
||||
? reason
|
||||
: new Error('migration verifier aborted', { cause: reason })
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
import {
|
||||
createJsonlGenerationRuntime,
|
||||
type JsonlGenerationRuntime,
|
||||
type JsonlGenerationRuntimeOverrides,
|
||||
} from '../generation.ts'
|
||||
|
||||
/**
|
||||
* Create generation operations with deterministic I/O and race seams for tests.
|
||||
* @param overrides - deterministic filesystem, platform, and race dependencies.
|
||||
* @returns bound generation operations.
|
||||
*/
|
||||
export function createJsonlGenerationTestRuntime(
|
||||
overrides: JsonlGenerationRuntimeOverrides = {},
|
||||
): JsonlGenerationRuntime {
|
||||
return createJsonlGenerationRuntime(overrides)
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
/** Worker entry for current-generation physical and logical verification. */
|
||||
|
||||
import { parentPort, workerData } from 'node:worker_threads'
|
||||
import { verifyJsonlCurrentGeneration } from './generation.ts'
|
||||
import type { JsonlExpectedPrefix } from './generation.ts'
|
||||
import type { JsonlCompression } from './format.ts'
|
||||
|
||||
interface VerificationRequest {
|
||||
readonly path: string
|
||||
readonly compression: JsonlCompression
|
||||
readonly expectedId: string
|
||||
readonly expectedEventCount: number
|
||||
readonly expectedPrefix?: JsonlExpectedPrefix
|
||||
}
|
||||
|
||||
function parseRequest(value: unknown): VerificationRequest {
|
||||
if (typeof value !== 'object' || value === null) throw new Error('migration verifier request must be an object')
|
||||
const request = value as Partial<VerificationRequest>
|
||||
if (typeof request.path !== 'string'
|
||||
|| request.compression !== 'none' && request.compression !== 'zstd'
|
||||
|| typeof request.expectedId !== 'string'
|
||||
|| !Number.isSafeInteger(request.expectedEventCount)
|
||||
|| (request.expectedEventCount as number) < 0
|
||||
|| request.expectedPrefix !== undefined
|
||||
&& (!Number.isSafeInteger(request.expectedPrefix.bytes)
|
||||
|| request.expectedPrefix.bytes < 0
|
||||
|| !/^[0-9a-f]{64}$/.test(request.expectedPrefix.digest))) {
|
||||
throw new Error('migration verifier request is malformed')
|
||||
}
|
||||
return request as VerificationRequest
|
||||
}
|
||||
|
||||
if (parentPort === null) throw new Error('migration verifier requires a parent port')
|
||||
const port = parentPort
|
||||
|
||||
const request = parseRequest(workerData)
|
||||
|
||||
async function verify(): Promise<void> {
|
||||
try {
|
||||
const result = await verifyJsonlCurrentGeneration(
|
||||
request.path,
|
||||
request.compression,
|
||||
request.expectedId,
|
||||
request.expectedEventCount,
|
||||
request.expectedPrefix,
|
||||
)
|
||||
port.postMessage({ ok: true, result })
|
||||
} catch (error: unknown) {
|
||||
const failure = error instanceof Error ? error : new Error(String(error))
|
||||
port.postMessage({ ok: false, message: failure.message, stack: failure.stack })
|
||||
} finally {
|
||||
port.close()
|
||||
}
|
||||
}
|
||||
|
||||
void verify()
|
||||
@@ -0,0 +1,49 @@
|
||||
import { existsSync } from 'node:fs'
|
||||
import { join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { execa } from 'execa'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
const packageRoot = fileURLToPath(new URL('..', import.meta.url))
|
||||
const built = ['lib/index.js', 'lib/worker.cjs']
|
||||
.every(path => existsSync(join(packageRoot, path)))
|
||||
|
||||
describe.skipIf(!built)('built migration verifier (plain node)', () => {
|
||||
it('publishes a historical generation through the bundled worker', async () => {
|
||||
const script = `
|
||||
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-built-migration-'))
|
||||
const id = 'built-migration-worker'
|
||||
const directory = join(root, '_no-cwd', id)
|
||||
const ctx = new Context()
|
||||
try {
|
||||
await mkdir(directory, { recursive: true })
|
||||
await writeFile(join(directory, 'session.jsonl'), JSON.stringify({
|
||||
type: 'session', version: 0, id, createdAt: 1, delegationDepth: 0,
|
||||
}) + '\\n')
|
||||
await ctx.plugin(JsonlSessionPersistence, { root, compression: 'none' })
|
||||
const handle = await ctx.sessionPersistence.open(id, 'read')
|
||||
await handle.close()
|
||||
await ctx.sessionPersistence.flush()
|
||||
const header = JSON.parse((await readFile(join(directory, 'session.v2.jsonl'), 'utf8')).trim())
|
||||
console.log(JSON.stringify({ id: header.id, version: header.version }))
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await rm(root, { recursive: true, force: true })
|
||||
}
|
||||
`
|
||||
const { exitCode, stdout, stderr } = await execa(
|
||||
process.execPath,
|
||||
['--input-type=module', '-e', script],
|
||||
{ cwd: packageRoot, stdin: 'ignore', timeout: 30_000, killSignal: 'SIGKILL', reject: false },
|
||||
)
|
||||
|
||||
expect(exitCode, `stderr:\n${stderr}`).toBe(0)
|
||||
expect(JSON.parse(stdout.trim())).toEqual({ id: 'built-migration-worker', version: 2 })
|
||||
})
|
||||
})
|
||||
File diff suppressed because it is too large.
Load diff
@@ -19,7 +19,6 @@ import {
|
||||
import { runLiveWritePathContract } from '../../session-persistence/tests/live-write-contract.ts'
|
||||
import { LIVE_WRITE_BATCH_MAX_DELAY_MS, type JsonlSessionHandle } from '../src/storage.ts'
|
||||
import SessionStore from '@deepseek-ai/dsh-session'
|
||||
import { releasedV1SessionFormatCodec } from '@deepseek-ai/dsh-session-format-v0-to-v1'
|
||||
|
||||
const statRace = vi.hoisted(() => ({
|
||||
path: undefined as string | undefined,
|
||||
@@ -154,15 +153,15 @@ function releasedV1PackedPhysicalLog(header: SessionHeader): string {
|
||||
: {}),
|
||||
})),
|
||||
]
|
||||
const encoded = releasedV1SessionFormatCodec.encodeArtifact({
|
||||
header: { ...header, version: 1, delegationDepth: header.delegationDepth ?? 0 },
|
||||
inheritedEventCount: 0,
|
||||
events,
|
||||
} as never, { packChunks: true })
|
||||
if (!encoded.rows.some(row => row['type'] === 'text-chunks')) {
|
||||
throw new Error('released v1 test fixture did not produce a packed text row')
|
||||
const packed = {
|
||||
type: 'text-chunks',
|
||||
seq0: 4,
|
||||
time0: 3,
|
||||
data: { turn: 1, step: 1, index: 0, dt: [0, 0], texts: ['hello', '', ''] },
|
||||
}
|
||||
return [encoded.header, ...encoded.rows].map(row => JSON.stringify(row)).join('\n') + '\n'
|
||||
const rows = [...events.slice(0, 4), packed, ...events.slice(7)]
|
||||
return [{ ...releasedV0Header(header), version: 1 }, ...rows]
|
||||
.map(row => JSON.stringify(row)).join('\n') + '\n'
|
||||
}
|
||||
|
||||
/** Create + append + close: persist one whole log through the write handle. */
|
||||
@@ -273,8 +272,8 @@ describe('JsonlSessionPersistence: format helpers', () => {
|
||||
it('ignores retired-header checks for non-object values', () => {
|
||||
expect(() => { assertNoRetiredHeaderFields(null) }).not.toThrow()
|
||||
expect(() => { assertNoRetiredHeaderFields('header') }).not.toThrow()
|
||||
expect(() => scanLog(Buffer.from('42\n'))).toThrow(/session header/)
|
||||
expect(() => scanLog(Buffer.from('null\n'))).toThrow(/session header/)
|
||||
expect(() => scanLog(Buffer.from('42\n'))).toThrow(/first line is not a JSON object/)
|
||||
expect(() => scanLog(Buffer.from('null\n'))).toThrow(/first line is not a JSON object/)
|
||||
expect(() => scanLog(Buffer.from(
|
||||
`${JSON.stringify({ type: 'session', version: 2, id: 123 })}\n`,
|
||||
))).toThrow(/first line is not a session header/)
|
||||
@@ -522,7 +521,7 @@ describe('JsonlSessionPersistence: stored-format refusals', () => {
|
||||
// Simulate the race: the header-only stat sees nothing although the full
|
||||
// log is present and readable.
|
||||
vi.spyOn(ctx.sessionPersistence, 'stat').mockResolvedValue(undefined)
|
||||
const handle = await ctx.sessionPersistence.open(m.id, 'read')
|
||||
const handle = await ctx.sessionPersistence.open(m.id, 'read', { signal: new AbortController().signal })
|
||||
try {
|
||||
expect(handle.header).toMatchObject({ id: m.id, cwd: '/work' })
|
||||
expect(await handle.read()).toEqual(oneTurnLog())
|
||||
@@ -612,7 +611,6 @@ describe('JsonlSessionPersistence: immutable format generations', () => {
|
||||
meta: { ...header, delegationDepth: 0 },
|
||||
events: oneTurnLog(),
|
||||
})
|
||||
|
||||
expect(await readFile(sourcePath)).toEqual(source)
|
||||
const current = (await readFile(currentPath, 'utf8')).trimEnd().split('\n')
|
||||
expect(JSON.parse(current[0] as string)).toMatchObject({
|
||||
@@ -720,14 +718,6 @@ describe('JsonlSessionPersistence: immutable format generations', () => {
|
||||
expect(await readFile(sourcePath, 'utf8')).toBe(`${JSON.stringify(releasedV0Header(header))}\n`)
|
||||
})
|
||||
|
||||
it('reports no current generation when ensure-current finds no stored id', async () => {
|
||||
const storage = ctx.sessionPersistence as unknown as {
|
||||
ensureCurrentLog(id: SessionId, signal?: AbortSignal): Promise<unknown>
|
||||
}
|
||||
|
||||
await expect(storage.ensureCurrentLog(SessionId('missing-generation'))).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('opens the migrated successor for append while retaining the historical source', async () => {
|
||||
const header = meta('released-v0-write', '/work')
|
||||
const sourcePath = historicalLogPath(root, header.cwd, header.id)
|
||||
@@ -771,6 +761,7 @@ describe('JsonlSessionPersistence: immutable format generations', () => {
|
||||
const handle = await ctx.sessionPersistence.create(header, {
|
||||
inheritedEventCount: SessionLogOffset(0),
|
||||
})
|
||||
expect(handle.inheritedEventCount).toBe(SessionLogOffset(0))
|
||||
await handle.append([{
|
||||
type: 'session/end-seed', seq: SessionSeq(0), time: 1, data: { inherited: true },
|
||||
}])
|
||||
@@ -1002,16 +993,16 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => {
|
||||
const m = meta('opening-claim', '/work')
|
||||
await writeLog(ctx.sessionPersistence, m, oneTurnLog())
|
||||
const service = ctx.sessionPersistence as unknown as {
|
||||
ensureCurrentLog: (
|
||||
requireStoredLog: (
|
||||
id: SessionId,
|
||||
signal?: AbortSignal,
|
||||
resolved?: unknown,
|
||||
) => Promise<unknown>
|
||||
}
|
||||
const original = service.ensureCurrentLog.bind(service)
|
||||
const original = service.requireStoredLog.bind(service)
|
||||
const gate = Promise.withResolvers<undefined>()
|
||||
const entered = Promise.withResolvers<undefined>()
|
||||
vi.spyOn(service, 'ensureCurrentLog').mockImplementationOnce(async (...args) => {
|
||||
vi.spyOn(service, 'requireStoredLog').mockImplementationOnce(async (...args) => {
|
||||
entered.resolve(undefined)
|
||||
await gate.promise
|
||||
return original(...args)
|
||||
@@ -1637,18 +1628,34 @@ describe('JsonlSessionPersistence: scanLog unit', () => {
|
||||
expect(() => new SessionLogScanner(Buffer.from(`${header}\n${header}\n`))).toThrow(/header-less/)
|
||||
})
|
||||
|
||||
it('fails immediately on an invalid row in strict scanner mode', () => {
|
||||
const header = Buffer.from(`${JSON.stringify(toHeaderLine(meta('scanner-strict')))}\n`)
|
||||
const scanner = new SessionLogScanner(header, 'strict')
|
||||
expect(() => { scanner.write(Buffer.from('null\n')) }).toThrow(/invalid committed event/)
|
||||
})
|
||||
|
||||
it('expands valid stored provenance ranges', () => {
|
||||
const log = [
|
||||
JSON.stringify(toHeaderLine(meta('scanner-provenance'))),
|
||||
JSON.stringify({ type: 'external', seq: 0, time: 2, data: null, sourceEventSeqs: [] }),
|
||||
'',
|
||||
].join('\n')
|
||||
const restored = scanLog(Buffer.from(log)).events[0]
|
||||
expect(restored !== undefined && 'sourceEventSeqs' in restored ? restored.sourceEventSeqs : undefined).toEqual([])
|
||||
})
|
||||
|
||||
it('requires the tagged inherited cut to agree with the v2 header lineage', () => {
|
||||
const seeded = { ...meta('scanner-seeded-cut'), isSeeded: true }
|
||||
const seededHeader = JSON.stringify(toHeaderLine(seeded, SessionLogOffset(0)))
|
||||
expect(() => scanLog(Buffer.from(`${seededHeader}\n`)))
|
||||
.toThrow(/seeded v2 header lacks an inherited end-seed marker/)
|
||||
.toThrow(/seeded Session lacks an inherited end-seed marker/)
|
||||
|
||||
const unseededHeader = JSON.stringify(toHeaderLine(meta('scanner-unseeded-cut')))
|
||||
const inheritedMarker = JSON.stringify({
|
||||
type: 'session/end-seed', seq: 0, time: 1, data: { inherited: true },
|
||||
})
|
||||
expect(() => scanLog(Buffer.from(`${unseededHeader}\n${inheritedMarker}\n`)))
|
||||
.toThrow(/unseeded v2 header contains an inherited end-seed marker/)
|
||||
.toThrow(/unseeded Session contains an inherited end-seed marker/)
|
||||
})
|
||||
|
||||
it('handles empty writes, boundary newlines, torn fragments, and scanner completion', () => {
|
||||
@@ -1681,7 +1688,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => {
|
||||
expect(() => { committed.write(Buffer.from([
|
||||
JSON.stringify({ type: 'turn/end', seq: SessionSeq(1), time: 2, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
'',
|
||||
].join('\n'))) }).toThrow(/seq gap in committed region/)
|
||||
].join('\n'))) }).toThrow(/seq gap/)
|
||||
})
|
||||
|
||||
it('incrementally scans records split across reusable decoder chunks', () => {
|
||||
@@ -1806,22 +1813,22 @@ describe('JsonlSessionPersistence: scanLog unit', () => {
|
||||
].join('\n') + '\n'
|
||||
// A turn/end exists, so the prefix up to it is committed — but it has a hole.
|
||||
// Truncating it would silently drop committed data → unloadable.
|
||||
expect(() => scanLog(Buffer.from(log))).toThrow(/seq gap in committed region/)
|
||||
expect(() => scanLog(Buffer.from(log))).toThrow(/seq gap/)
|
||||
})
|
||||
|
||||
it('rejects malformed records before a later committed turn/end', () => {
|
||||
const corruptRecords = [
|
||||
'{not json',
|
||||
'null',
|
||||
JSON.stringify({ type: 'assistant/message', sourceEventSeqs: [0], data: {} }),
|
||||
]
|
||||
for (const record of corruptRecords) {
|
||||
['{not json', /unparsable committed event/],
|
||||
['null', /invalid committed event/],
|
||||
[JSON.stringify({ type: 'assistant/message', sourceEventSeqs: [0], data: {} }), /invalid committed event/],
|
||||
] as const
|
||||
for (const [record, message] of corruptRecords) {
|
||||
const log = [
|
||||
JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: 'c', createdAt: 1, isSeeded: false, delegationDepth: 0 }),
|
||||
record,
|
||||
JSON.stringify({ type: 'turn/end', seq: SessionSeq(1), time: 2, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
].join('\n') + '\n'
|
||||
expect(() => scanLog(Buffer.from(log))).toThrow(/unparsable committed event/)
|
||||
expect(() => scanLog(Buffer.from(log))).toThrow(message)
|
||||
}
|
||||
})
|
||||
|
||||
@@ -1953,7 +1960,7 @@ describe('JsonlSessionPersistence: nested v2 Assistant streams', () => {
|
||||
JSON.stringify({ type: 'text-chunks', seq0: 1, time0: 2, data: { turn: 1, step: 1, index: 0, dt: [1, 1], texts: ['a', 'b', 'c'] } }),
|
||||
JSON.stringify({ type: 'turn/end', seq: SessionSeq(4), time: 5, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
].join('\n') + '\n'
|
||||
expect(() => scanLog(Buffer.from(logText))).toThrow(/seq gap in committed region/)
|
||||
expect(() => scanLog(Buffer.from(logText))).toThrow(/lacks .*seq/)
|
||||
})
|
||||
|
||||
it('scanLog treats a malformed removed packed row as a committed seq hole', () => {
|
||||
@@ -1966,7 +1973,7 @@ describe('JsonlSessionPersistence: nested v2 Assistant streams', () => {
|
||||
JSON.stringify({ type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [], texts: ['a', 'b'] } }),
|
||||
JSON.stringify({ type: 'turn/end', seq: SessionSeq(2), time: 3, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
].join('\n') + '\n'
|
||||
expect(() => scanLog(Buffer.from(logText))).toThrow(/seq gap in committed region/)
|
||||
expect(() => scanLog(Buffer.from(logText))).toThrow(/lacks .*seq/)
|
||||
})
|
||||
|
||||
it('scanLog: a packed row with a mid-run seq gap after the last turn/end drops the whole row', () => {
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { verifyCurrentGenerationInWorker } from '../src/migration-verifier.ts'
|
||||
|
||||
const state = vi.hoisted(() => ({ workers: [] as unknown[] }))
|
||||
|
||||
vi.mock('node:worker_threads', () => ({
|
||||
Worker: class {
|
||||
readonly listeners = new Map<string, (value: never) => void>()
|
||||
readonly terminate = vi.fn<() => Promise<number>>(() => Promise.resolve(0))
|
||||
|
||||
constructor(readonly entry: string | URL, readonly options: unknown) {
|
||||
state.workers.push(this)
|
||||
}
|
||||
|
||||
once(event: string, listener: (value: never) => void): this {
|
||||
this.listeners.set(event, listener)
|
||||
return this
|
||||
}
|
||||
|
||||
emit(event: string, value: unknown): void {
|
||||
this.listeners.get(event)?.(value as never)
|
||||
}
|
||||
},
|
||||
}))
|
||||
|
||||
interface FakeWorker {
|
||||
readonly entry: string | URL
|
||||
readonly options: { readonly workerData: unknown }
|
||||
readonly terminate: ReturnType<typeof vi.fn<() => Promise<number>>>
|
||||
emit(event: string, value: unknown): void
|
||||
}
|
||||
|
||||
function worker(index = 0): FakeWorker {
|
||||
const candidate = state.workers[index]
|
||||
if (candidate === undefined) throw new Error('verification did not create a Worker')
|
||||
return candidate as FakeWorker
|
||||
}
|
||||
|
||||
const result = {
|
||||
identity: { dev: 1n, ino: 2n, size: 3n, mtimeNs: 4n, ctimeNs: 5n },
|
||||
bytes: 3,
|
||||
digest: 'digest',
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
state.workers.length = 0
|
||||
})
|
||||
|
||||
describe('migration verifier Worker lifecycle', () => {
|
||||
it('resolves only after terminating a successful Worker', async () => {
|
||||
const verification = verifyCurrentGenerationInWorker('/stage', 'none', 'session', 2)
|
||||
const instance = worker()
|
||||
expect(instance.options.workerData).toEqual({
|
||||
path: '/stage', compression: 'none', expectedId: 'session', expectedEventCount: 2,
|
||||
})
|
||||
instance.emit('message', { ok: true, result })
|
||||
|
||||
await expect(verification).resolves.toEqual(result)
|
||||
expect(instance.terminate).toHaveBeenCalledOnce()
|
||||
})
|
||||
|
||||
it('reconstructs a Worker-reported error', async () => {
|
||||
const verification = verifyCurrentGenerationInWorker('/stage', 'zstd', 'session', 0)
|
||||
worker().emit('message', { ok: false, message: 'invalid stage', stack: 'worker stack' })
|
||||
|
||||
await expect(verification).rejects.toMatchObject({ message: 'invalid stage', stack: 'worker stack' })
|
||||
})
|
||||
|
||||
it('accepts an error response without a stack', async () => {
|
||||
const verification = verifyCurrentGenerationInWorker('/stage', 'none', 'session', 0)
|
||||
worker().emit('message', { ok: false, message: 'invalid stage' })
|
||||
await expect(verification).rejects.toThrow('invalid stage')
|
||||
})
|
||||
|
||||
it.each([
|
||||
['invalid response', 'message', null, /invalid response/],
|
||||
['non-object response', 'message', 'invalid', /invalid response/],
|
||||
['missing discriminator', 'message', {}, /invalid response/],
|
||||
['invalid discriminator', 'message', { ok: 'yes' }, /invalid response/],
|
||||
['worker error', 'error', new Error('worker failed'), /worker failed/],
|
||||
['early exit', 'exit', 7, /code 7/],
|
||||
])('rejects an %s', async (_name, event, value, expected) => {
|
||||
const verification = verifyCurrentGenerationInWorker('/stage', 'none', 'session', 0)
|
||||
worker().emit(event, value)
|
||||
await expect(verification).rejects.toThrow(expected)
|
||||
})
|
||||
|
||||
it('aggregates termination failure after a Worker failure', async () => {
|
||||
const verification = verifyCurrentGenerationInWorker('/stage', 'none', 'session', 0)
|
||||
const instance = worker()
|
||||
instance.terminate.mockRejectedValueOnce(new Error('terminate failed'))
|
||||
instance.emit('error', new Error('worker failed'))
|
||||
|
||||
await expect(verification).rejects.toBeInstanceOf(AggregateError)
|
||||
})
|
||||
|
||||
it('rejects a successful result when termination fails', async () => {
|
||||
const verification = verifyCurrentGenerationInWorker('/stage', 'none', 'session', 0)
|
||||
const instance = worker()
|
||||
instance.terminate.mockRejectedValueOnce('terminate failed')
|
||||
instance.emit('message', { ok: true, result })
|
||||
|
||||
await expect(verification).rejects.toThrow('terminate failed')
|
||||
})
|
||||
|
||||
it('preserves an Error from successful-result termination', async () => {
|
||||
const verification = verifyCurrentGenerationInWorker('/stage', 'none', 'session', 0)
|
||||
const instance = worker()
|
||||
instance.terminate.mockRejectedValueOnce(new Error('terminate failed'))
|
||||
instance.emit('message', { ok: true, result })
|
||||
|
||||
await expect(verification).rejects.toThrow('terminate failed')
|
||||
})
|
||||
|
||||
it('ignores terminal signals after a result settles', async () => {
|
||||
const verification = verifyCurrentGenerationInWorker('/stage', 'none', 'session', 0)
|
||||
const instance = worker()
|
||||
instance.emit('message', { ok: true, result })
|
||||
instance.emit('error', new Error('late error'))
|
||||
instance.emit('exit', 1)
|
||||
instance.emit('message', null)
|
||||
|
||||
await expect(verification).resolves.toEqual(result)
|
||||
expect(instance.terminate).toHaveBeenCalledOnce()
|
||||
})
|
||||
|
||||
it('starts at most two verification Workers concurrently', async () => {
|
||||
const first = verifyCurrentGenerationInWorker('/first', 'none', 'session', 0)
|
||||
const second = verifyCurrentGenerationInWorker('/second', 'none', 'session', 0)
|
||||
const third = verifyCurrentGenerationInWorker('/third', 'none', 'session', 0)
|
||||
expect(state.workers).toHaveLength(2)
|
||||
|
||||
worker(0).emit('message', { ok: true, result })
|
||||
await first
|
||||
await vi.waitFor(() => { expect(state.workers).toHaveLength(3) })
|
||||
|
||||
worker(1).emit('message', { ok: true, result })
|
||||
worker(2).emit('message', { ok: true, result })
|
||||
await expect(Promise.all([second, third])).resolves.toEqual([result, result])
|
||||
})
|
||||
|
||||
it('hands a released permit directly to the oldest waiter', async () => {
|
||||
const first = verifyCurrentGenerationInWorker('/first', 'none', 'session', 0)
|
||||
const second = verifyCurrentGenerationInWorker('/second', 'none', 'session', 0)
|
||||
const third = verifyCurrentGenerationInWorker('/third', 'none', 'session', 0)
|
||||
let fourth: Promise<typeof result> | undefined
|
||||
worker(0).terminate.mockReturnValueOnce({
|
||||
then(onFulfilled: (value: number) => unknown) {
|
||||
onFulfilled(0)
|
||||
queueMicrotask(() => {
|
||||
fourth = verifyCurrentGenerationInWorker('/fourth', 'none', 'session', 0)
|
||||
})
|
||||
return Promise.resolve()
|
||||
},
|
||||
} as unknown as Promise<number>)
|
||||
|
||||
worker(0).emit('message', { ok: true, result })
|
||||
await first
|
||||
await vi.waitFor(() => { expect(state.workers).toHaveLength(3) })
|
||||
expect(worker(2).options.workerData).toMatchObject({ path: '/third' })
|
||||
|
||||
worker(1).emit('message', { ok: true, result })
|
||||
await second
|
||||
await vi.waitFor(() => { expect(state.workers).toHaveLength(4) })
|
||||
expect(worker(3).options.workerData).toMatchObject({ path: '/fourth' })
|
||||
if (fourth === undefined) throw new Error('fourth verification was not scheduled')
|
||||
|
||||
worker(2).emit('message', { ok: true, result })
|
||||
worker(3).emit('message', { ok: true, result })
|
||||
await expect(Promise.all([third, fourth])).resolves.toEqual([result, result])
|
||||
})
|
||||
|
||||
it('removes an aborted waiter without starting another Worker', async () => {
|
||||
const first = verifyCurrentGenerationInWorker('/first', 'none', 'session', 0)
|
||||
const second = verifyCurrentGenerationInWorker('/second', 'none', 'session', 0)
|
||||
const controller = new AbortController()
|
||||
const reason = new Error('queued verification cancelled')
|
||||
const queued = verifyCurrentGenerationInWorker(
|
||||
'/queued', 'none', 'session', 0, undefined, controller.signal,
|
||||
)
|
||||
|
||||
controller.abort(reason)
|
||||
await expect(queued).rejects.toBe(reason)
|
||||
expect(state.workers).toHaveLength(2)
|
||||
|
||||
worker(0).emit('message', { ok: true, result })
|
||||
worker(1).emit('message', { ok: true, result })
|
||||
await expect(Promise.all([first, second])).resolves.toEqual([result, result])
|
||||
expect(state.workers).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('terminates an active Worker before rejecting cancellation', async () => {
|
||||
const controller = new AbortController()
|
||||
const reason = new Error('active verification cancelled')
|
||||
const verification = verifyCurrentGenerationInWorker(
|
||||
'/stage', 'none', 'session', 0, undefined, controller.signal,
|
||||
)
|
||||
const instance = worker()
|
||||
let finishTermination: ((value: number) => void) | undefined
|
||||
instance.terminate.mockReturnValueOnce(new Promise((resolve) => {
|
||||
finishTermination = resolve
|
||||
}))
|
||||
let settled = false
|
||||
void verification.then(
|
||||
() => { settled = true },
|
||||
() => { settled = true },
|
||||
)
|
||||
|
||||
controller.abort(reason)
|
||||
expect(instance.terminate).toHaveBeenCalledOnce()
|
||||
await Promise.resolve()
|
||||
expect(settled).toBe(false)
|
||||
|
||||
finishTermination?.(0)
|
||||
await expect(verification).rejects.toBe(reason)
|
||||
})
|
||||
|
||||
it('wraps a non-Error active cancellation reason', async () => {
|
||||
const controller = new AbortController()
|
||||
const verification = verifyCurrentGenerationInWorker(
|
||||
'/stage', 'none', 'session', 0, undefined, controller.signal,
|
||||
)
|
||||
|
||||
controller.abort('cancelled')
|
||||
await expect(verification).rejects.toMatchObject({
|
||||
message: 'migration verifier aborted',
|
||||
cause: 'cancelled',
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -428,7 +428,6 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => {
|
||||
meta: { ...header, delegationDepth: 0 },
|
||||
events: oneTurnLog(),
|
||||
})
|
||||
|
||||
expect(await readFile(sourcePath)).toEqual(source)
|
||||
const current = (await decodeCompleteFrames(await readFile(currentPath))).toString().split('\n')
|
||||
expect(JSON.parse(current[0] as string)).toMatchObject({
|
||||
|
||||
@@ -17,6 +17,9 @@
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
import { defineConfig } from 'tsdown'
|
||||
|
||||
/** Build the backend and its path-loaded verifier as separate bundles. */
|
||||
export default defineConfig([
|
||||
{
|
||||
entry: ['lib/types/index.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
{
|
||||
entry: ['lib/types/worker.js'],
|
||||
outDir: 'lib',
|
||||
format: ['cjs'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
])
|
||||
@@ -48,15 +48,10 @@ interface ParsedSessionFixture {
|
||||
readonly createdAt: number
|
||||
readonly inheritedEventCount: SessionLogOffsetType
|
||||
readonly events: SessionEvent[]
|
||||
readonly artifact: ReturnType<typeof sessionFormatCatalog.migrate>
|
||||
readonly artifact: ReturnType<ReturnType<typeof sessionFormatCatalog.createRestore>['finish']>
|
||||
readonly sourceHeader: Readonly<Record<string, unknown>>
|
||||
}
|
||||
|
||||
interface FixtureJsonLine {
|
||||
readonly lineNumber: number
|
||||
readonly value: Record<string, unknown>
|
||||
}
|
||||
|
||||
/**
|
||||
* One recorded model call. `throw` may replay prefix chunks before failing;
|
||||
* `hang` models cancellation. Derived chunk entries come from ordinary model
|
||||
@@ -210,7 +205,13 @@ export function parseSessionLog(text: string): SessionEvent[] {
|
||||
|
||||
/** Parse, complete, decode, and migrate one projected snapshot artifact without writing its source. */
|
||||
function parseSessionFixture(text: string): ParsedSessionFixture {
|
||||
const parsed: FixtureJsonLine[] = []
|
||||
let headerLineNumber: number | undefined
|
||||
let sourceHeader: Record<string, unknown> | undefined
|
||||
let restore: ReturnType<typeof sessionFormatCatalog.createRestore> | undefined
|
||||
const rowLines: number[] = []
|
||||
const eventLines: number[] = []
|
||||
let bodyKind: 'complete' | 'projected' | undefined
|
||||
let nextSeq = 0
|
||||
for (const [index, line] of text.split(/\r?\n/).entries()) {
|
||||
if (line.trim().length === 0) continue
|
||||
let value: unknown
|
||||
@@ -222,19 +223,22 @@ function parseSessionFixture(text: string): ParsedSessionFixture {
|
||||
if (value === null || typeof value !== 'object' || Array.isArray(value)) {
|
||||
throw new Error(`session snapshot line ${index + 1} must be a JSON object`)
|
||||
}
|
||||
parsed.push({ lineNumber: index + 1, value: value as Record<string, unknown> })
|
||||
}
|
||||
const headerLine = parsed[0]
|
||||
if (headerLine === undefined) throw new Error('session snapshot must start with a session header')
|
||||
|
||||
const header = normalizeProjectedHeader(headerLine.value)
|
||||
const rows: Record<string, unknown>[] = []
|
||||
const rowLines: number[] = []
|
||||
const eventLines: number[] = []
|
||||
let bodyKind: 'complete' | 'projected' | undefined
|
||||
let nextSeq = 0
|
||||
for (const source of parsed.slice(1)) {
|
||||
const record = normalizeProjectedRow(source.value)
|
||||
const lineNumber = index + 1
|
||||
const recordValue = value as Record<string, unknown>
|
||||
if (restore === undefined) {
|
||||
headerLineNumber = lineNumber
|
||||
sourceHeader = recordValue
|
||||
try {
|
||||
restore = sessionFormatCatalog.createRestore(normalizeProjectedHeader(recordValue), {
|
||||
recovery: 'strict',
|
||||
validation: 'current',
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
throw fixtureFormatError(error, lineNumber, [], [])
|
||||
}
|
||||
continue
|
||||
}
|
||||
const record = normalizeProjectedRow(recordValue)
|
||||
const packed = PACKED_CHUNK_ROW_TYPES.has(record.type as string)
|
||||
const seqKey = packed ? 'seq0' : 'seq'
|
||||
const timeKey = packed ? 'time0' : 'time'
|
||||
@@ -242,12 +246,12 @@ function parseSessionFixture(text: string): ParsedSessionFixture {
|
||||
const hasTime = Object.hasOwn(record, timeKey)
|
||||
if (hasSeq !== hasTime) {
|
||||
throw new Error(
|
||||
`session snapshot line ${source.lineNumber} must contain both ${seqKey} and ${timeKey}, or neither`,
|
||||
`session snapshot line ${lineNumber} must contain both ${seqKey} and ${timeKey}, or neither`,
|
||||
)
|
||||
}
|
||||
const currentKind = hasSeq ? 'complete' : 'projected'
|
||||
if (bodyKind !== undefined && currentKind !== bodyKind) {
|
||||
throw new Error(`session snapshot line ${source.lineNumber} cannot mix projected and complete body rows`)
|
||||
throw new Error(`session snapshot line ${lineNumber} cannot mix projected and complete body rows`)
|
||||
}
|
||||
bodyKind = currentKind
|
||||
if (currentKind === 'projected') {
|
||||
@@ -255,36 +259,28 @@ function parseSessionFixture(text: string): ParsedSessionFixture {
|
||||
record[timeKey] = 0
|
||||
}
|
||||
const cardinality = physicalRowCardinality(record)
|
||||
rows.push(record)
|
||||
rowLines.push(source.lineNumber)
|
||||
eventLines.push(...Array.from({ length: cardinality }, () => source.lineNumber))
|
||||
rowLines.push(lineNumber)
|
||||
eventLines.push(...Array.from({ length: cardinality }, () => lineNumber))
|
||||
nextSeq += cardinality
|
||||
try {
|
||||
restore.decodeRow(record)
|
||||
} catch (error: unknown) {
|
||||
throw fixtureFormatError(error, headerLineNumber as number, rowLines, eventLines, rowLines.length - 1)
|
||||
}
|
||||
}
|
||||
|
||||
let decoded: ReturnType<typeof sessionFormatCatalog.decodeArtifact>
|
||||
try {
|
||||
decoded = sessionFormatCatalog.decodeArtifact(header, rows)
|
||||
} catch (error: unknown) {
|
||||
const physicalRow = locateUnlabelledPhysicalFailure(error, header, rows)
|
||||
throw fixtureFormatError(
|
||||
error,
|
||||
headerLine.lineNumber,
|
||||
rowLines,
|
||||
eventLines,
|
||||
physicalRow,
|
||||
)
|
||||
if (restore === undefined || sourceHeader === undefined || headerLineNumber === undefined) {
|
||||
throw new Error('session snapshot must start with a session header')
|
||||
}
|
||||
try {
|
||||
const current = sessionFormatCatalog.migrate(decoded)
|
||||
return parsedSessionFixture(current, headerLine.value)
|
||||
return parsedSessionFixture(restore.finish(), sourceHeader)
|
||||
} catch (error: unknown) {
|
||||
throw fixtureFormatError(error, headerLine.lineNumber, rowLines, eventLines)
|
||||
throw fixtureFormatError(error, headerLineNumber, rowLines, eventLines)
|
||||
}
|
||||
}
|
||||
|
||||
/** Materialize the common replay view from a migrated artifact. */
|
||||
function parsedSessionFixture(
|
||||
artifact: ReturnType<typeof sessionFormatCatalog.decodeArtifact>,
|
||||
artifact: ParsedSessionFixture['artifact'],
|
||||
sourceHeader: Readonly<Record<string, unknown>>,
|
||||
): ParsedSessionFixture {
|
||||
return {
|
||||
@@ -410,17 +406,16 @@ function migrateWrappedEventGroup(
|
||||
time: 0,
|
||||
data: { text: `comparison prefix ${String(seq)}` },
|
||||
}))
|
||||
const migrated = sessionFormatCatalog.migrate({
|
||||
header: {
|
||||
version: 1,
|
||||
id: first.sessionId,
|
||||
createdAt: 0,
|
||||
isSeeded: false,
|
||||
delegationDepth: 0,
|
||||
},
|
||||
inheritedEventCount: 0,
|
||||
events: [...prefix, ...entries.map(entry => normalizeProjectedRow(entry.event))] as never,
|
||||
}).events.slice(prefix.length) as readonly Readonly<Record<string, unknown>>[]
|
||||
const restore = sessionFormatCatalog.createRestore({
|
||||
type: 'session',
|
||||
version: 1,
|
||||
id: first.sessionId,
|
||||
createdAt: 0,
|
||||
delegationDepth: 0,
|
||||
}, { recovery: 'strict', validation: 'current' })
|
||||
for (const event of prefix) restore.decodeRow(event)
|
||||
for (const entry of entries) restore.decodeRow(normalizeProjectedRow(entry.event))
|
||||
const migrated = restore.finish().events.slice(prefix.length) as readonly Readonly<Record<string, unknown>>[]
|
||||
|
||||
const assigned = new Map<number, Readonly<Record<string, unknown>>[]>()
|
||||
let migratedIndex = 0
|
||||
@@ -498,11 +493,18 @@ export function prepareSessionEventNotificationsForComparison(text: string): str
|
||||
|
||||
/** Encode one migrated fixture while retaining a projected cwd token. */
|
||||
function encodeCurrentSessionSnapshotFixture(text: string, parsed: ParsedSessionFixture): string {
|
||||
const encoded = sessionFormatCatalog.encodeCurrent(parsed.artifact)
|
||||
const header = { ...encoded.header }
|
||||
const header = {
|
||||
...sessionFormatCatalog.encodeCurrentHeader(
|
||||
parsed.artifact.header,
|
||||
parsed.artifact.inheritedEventCount,
|
||||
),
|
||||
}
|
||||
const sourceCwd = parsed.sourceHeader['cwd']
|
||||
if (typeof sourceCwd === 'string' && /^\{\{cwd\}\}(?:\/|$)/.test(sourceCwd)) header['cwd'] = sourceCwd
|
||||
const output = [header, ...encoded.rows].map(record => JSON.stringify(record)).join('\n')
|
||||
const output = [
|
||||
JSON.stringify(header),
|
||||
...parsed.artifact.events.map(event => JSON.stringify(sessionFormatCatalog.encodeCurrentEvent(event))),
|
||||
].join('\n')
|
||||
return text.endsWith('\n') ? `${output}\n` : output
|
||||
}
|
||||
|
||||
@@ -564,13 +566,15 @@ function fixtureFormatError(
|
||||
const locationDetail = error instanceof Error && error.cause instanceof Error
|
||||
? error.cause.message
|
||||
: detail
|
||||
const row = /released (?:Session|(?:text|reasoning|tool-call)-chunks) row (\d+)/.exec(locationDetail)
|
||||
const storedRow = /^released Session row (\d+)/.exec(locationDetail)
|
||||
const event = /Session event (\d+)/.exec(locationDetail)
|
||||
?? / at seq (\d+)/.exec(locationDetail)
|
||||
?? /^[^ ]+ (\d+) /.exec(locationDetail)
|
||||
const line = physicalRow === undefined && row === null
|
||||
? event === null ? headerLine : eventLines[Number(event[1])] ?? headerLine
|
||||
: rowLines[physicalRow ?? Number(row?.[1])] ?? headerLine
|
||||
?? /inherited Session cut (\d+)/.exec(locationDetail)
|
||||
let line: number
|
||||
if (physicalRow !== undefined) line = rowLines[physicalRow] as number
|
||||
else if (storedRow !== null) line = rowLines[Number(storedRow[1])] ?? headerLine
|
||||
else if (event === null) line = headerLine
|
||||
else line = eventLines[Number(event[1])] ?? headerLine
|
||||
const message = `session snapshot line ${line}: ${detail}`
|
||||
if (error instanceof SessionFormatUnsupportedMigrationError) {
|
||||
return new SessionFormatUnsupportedMigrationError(message, { cause: error })
|
||||
@@ -578,26 +582,6 @@ function fixtureFormatError(
|
||||
return new Error(message, { cause: error })
|
||||
}
|
||||
|
||||
/** Locate range-decoder failures whose frozen diagnostic predates physical-row labels. */
|
||||
function locateUnlabelledPhysicalFailure(
|
||||
error: unknown,
|
||||
header: Readonly<Record<string, unknown>>,
|
||||
rows: readonly Readonly<Record<string, unknown>>[],
|
||||
): number | undefined {
|
||||
const detail = error instanceof Error ? error.message : String(error)
|
||||
if (!detail.startsWith('sourceEventSeqs ')) return undefined
|
||||
const diagnosticHeader = Object.hasOwn(header, 'seedLength') ? { ...header, seedLength: 0 } : header
|
||||
for (let index = 0; index < rows.length; index += 1) {
|
||||
try {
|
||||
sessionFormatCatalog.decodeArtifact(diagnosticHeader, rows.slice(0, index + 1))
|
||||
} catch (candidate: unknown) {
|
||||
const candidateDetail = candidate instanceof Error ? candidate.message : String(candidate)
|
||||
if (candidateDetail === detail) return index
|
||||
}
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Read replay identity, ordering, and fork-seed facts from the JSONL header.
|
||||
*
|
||||
|
||||
@@ -235,7 +235,7 @@ describe('Session format package parity', () => {
|
||||
})
|
||||
|
||||
describe('fixture format diagnostics', () => {
|
||||
it('attaches the header line to a non-Error catalog failure', async () => {
|
||||
it('attaches the header line to a restore-construction failure', async () => {
|
||||
vi.resetModules()
|
||||
vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('@deepseek-ai/dsh-session-format-catalog')>()
|
||||
@@ -243,7 +243,7 @@ describe('fixture format diagnostics', () => {
|
||||
...actual,
|
||||
sessionFormatCatalog: {
|
||||
...actual.sessionFormatCatalog,
|
||||
decodeArtifact(): never {
|
||||
createRestore(): never {
|
||||
const failure: unknown = 'decoder exploded'
|
||||
throw failure
|
||||
},
|
||||
@@ -261,6 +261,37 @@ describe('fixture format diagnostics', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('attaches the header line to a restore-finalization failure', async () => {
|
||||
vi.resetModules()
|
||||
vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('@deepseek-ai/dsh-session-format-catalog')>()
|
||||
return {
|
||||
...actual,
|
||||
sessionFormatCatalog: {
|
||||
...actual.sessionFormatCatalog,
|
||||
createRestore() {
|
||||
return {
|
||||
header: { version: 2, id: 'fixture', createdAt: 0, isSeeded: false, delegationDepth: 0 },
|
||||
decodeRow() {},
|
||||
finish(): never {
|
||||
throw new Error('Session event 99 restore finalization failed')
|
||||
},
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
})
|
||||
try {
|
||||
const replay = await import('../src/index.ts')
|
||||
|
||||
expect(() => replay.parseSessionLog(sessionJsonl([])))
|
||||
.toThrow('session snapshot line 1: Session event 99 restore finalization failed')
|
||||
} finally {
|
||||
vi.doUnmock('@deepseek-ai/dsh-session-format-catalog')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('falls back to the header when a source-range diagnostic has no matching physical prefix', async () => {
|
||||
vi.resetModules()
|
||||
vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => {
|
||||
@@ -269,8 +300,14 @@ describe('fixture format diagnostics', () => {
|
||||
...actual,
|
||||
sessionFormatCatalog: {
|
||||
...actual.sessionFormatCatalog,
|
||||
decodeArtifact(): never {
|
||||
throw new Error('sourceEventSeqs synthetic unmatched failure')
|
||||
createRestore() {
|
||||
return {
|
||||
header: { version: 2, id: 'fixture', createdAt: 0, isSeeded: false, delegationDepth: 0 },
|
||||
decodeRow() {},
|
||||
finish(): never {
|
||||
throw new Error('sourceEventSeqs synthetic unmatched failure')
|
||||
},
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -286,22 +323,24 @@ describe('fixture format diagnostics', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('maps a matching non-Error source-range prefix failure to its physical row', async () => {
|
||||
it('maps a non-Error row failure to its physical row', async () => {
|
||||
vi.resetModules()
|
||||
vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('@deepseek-ai/dsh-session-format-catalog')>()
|
||||
let callCount = 0
|
||||
let row = 0
|
||||
return {
|
||||
...actual,
|
||||
sessionFormatCatalog: {
|
||||
...actual.sessionFormatCatalog,
|
||||
decodeArtifact(): never {
|
||||
callCount += 1
|
||||
if (callCount === 1) throw new Error('sourceEventSeqs synthetic prefix failure')
|
||||
const failure: unknown = callCount === 2
|
||||
? 'sourceEventSeqs different prefix failure'
|
||||
: 'sourceEventSeqs synthetic prefix failure'
|
||||
throw failure
|
||||
createRestore() {
|
||||
return {
|
||||
header: { version: 2, id: 'fixture', createdAt: 0, isSeeded: false, delegationDepth: 0 },
|
||||
decodeRow(): void {
|
||||
row += 1
|
||||
if (row === 2) throw 'row decoder exploded'
|
||||
},
|
||||
finish(): never { throw new Error('unexpected finish') },
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -314,14 +353,21 @@ describe('fixture format diagnostics', () => {
|
||||
]
|
||||
|
||||
expect(() => replay.parseSessionLog(sessionJsonl(events)))
|
||||
.toThrow('session snapshot line 3: sourceEventSeqs synthetic prefix failure')
|
||||
.toThrow('session snapshot line 3: row decoder exploded')
|
||||
} finally {
|
||||
vi.doUnmock('@deepseek-ai/dsh-session-format-catalog')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('falls back to the header for an out-of-range physical-row diagnostic', async () => {
|
||||
it.each([
|
||||
['physical row', 'released Session row 0 is malformed', 2],
|
||||
['logical event', 'Session event 0 is malformed', 2],
|
||||
['event seq', 'format v1 contains an invalid event at seq 0', 2],
|
||||
['inherited cut', 'inherited Session cut 0 splits one Assistant attempt', 2],
|
||||
['out-of-range physical row', 'released Session row 99 is malformed', 1],
|
||||
['out-of-range logical event', 'Session event 99 is malformed', 1],
|
||||
])('maps a %s finalization diagnostic to its source line', async (_label, message, line) => {
|
||||
vi.resetModules()
|
||||
vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('@deepseek-ai/dsh-session-format-catalog')>()
|
||||
@@ -329,8 +375,12 @@ describe('fixture format diagnostics', () => {
|
||||
...actual,
|
||||
sessionFormatCatalog: {
|
||||
...actual.sessionFormatCatalog,
|
||||
decodeArtifact(): never {
|
||||
throw new Error('released Session row 99 is malformed')
|
||||
createRestore() {
|
||||
return {
|
||||
header: { version: 2, id: 'fixture', createdAt: 0, isSeeded: false, delegationDepth: 0 },
|
||||
decodeRow() {},
|
||||
finish(): never { throw new Error(message) },
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -340,33 +390,7 @@ describe('fixture format diagnostics', () => {
|
||||
const event: SessionEvent = { type: 'turn/start', seq: SessionSeq(0), time: 0, data: { turn: 1 } }
|
||||
|
||||
expect(() => replay.parseSessionLog(sessionJsonl([event])))
|
||||
.toThrow('session snapshot line 1: released Session row 99 is malformed')
|
||||
} finally {
|
||||
vi.doUnmock('@deepseek-ai/dsh-session-format-catalog')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('falls back to the header for an out-of-range logical-event diagnostic', async () => {
|
||||
vi.resetModules()
|
||||
vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('@deepseek-ai/dsh-session-format-catalog')>()
|
||||
return {
|
||||
...actual,
|
||||
sessionFormatCatalog: {
|
||||
...actual.sessionFormatCatalog,
|
||||
migrate(): never {
|
||||
throw new Error('Session event 99 is malformed')
|
||||
},
|
||||
},
|
||||
}
|
||||
})
|
||||
try {
|
||||
const replay = await import('../src/index.ts')
|
||||
const event: SessionEvent = { type: 'turn/start', seq: SessionSeq(0), time: 0, data: { turn: 1 } }
|
||||
|
||||
expect(() => replay.parseSessionLog(sessionJsonl([event])))
|
||||
.toThrow('session snapshot line 1: Session event 99 is malformed')
|
||||
.toThrow(`session snapshot line ${line}: ${message}`)
|
||||
} finally {
|
||||
vi.doUnmock('@deepseek-ai/dsh-session-format-catalog')
|
||||
vi.resetModules()
|
||||
|
||||
Generated
+3
@@ -7370,6 +7370,9 @@ importers:
|
||||
|
||||
packages/session/session-persistence-jsonl:
|
||||
dependencies:
|
||||
'@deepseek-ai/dsh-llm':
|
||||
specifier: workspace:^
|
||||
version: link:../../llm/llm
|
||||
'@deepseek-ai/dsh-session-format':
|
||||
specifier: workspace:^
|
||||
version: link:../session-format
|
||||
|
||||
@@ -156,6 +156,9 @@ const packageFileExtras: Readonly<Record<string, readonly string[]>> = {
|
||||
// The Web Host mounts the default-off settings owner independently of each
|
||||
// Agent-scoped delegation-tool instance.
|
||||
'@deepseek-ai/dsh-tool-subagent': ['lib/model-selection-settings.js'],
|
||||
// The JSONL backend resolves its private verification Worker relative to
|
||||
// import.meta.url; it is shipped without a public package subpath.
|
||||
'@deepseek-ai/dsh-session-persistence-jsonl': ['lib/worker.cjs'],
|
||||
// The argv-prefix runner entry ships beside the lib as its own bundle;
|
||||
// sandbox-local resolves it through the package's ./runner export. tsdown
|
||||
// also shares its generated FFI code through a hashed runtime chunk.
|
||||
|
||||
@@ -103,8 +103,9 @@ describe('session format catalog generator', () => {
|
||||
expect(declarations.map(item => [item.from, item.to])).toEqual([[0, 1], [1, 2]])
|
||||
expect(output).toContain("from '@deepseek-ai/dsh-session-format-v0-to-v1'")
|
||||
expect(output).toContain('currentVersion: 2')
|
||||
expect(output).toContain('encodeCurrentArtifact: artifact => releasedV2SessionFormatCodec.encodeArtifact(artifact)')
|
||||
expect(output).toContain('currentEncoder: releasedV2SessionFormatCodec')
|
||||
expect(output).toContain('restoreReleasedV2Artifact(artifact, KNOWN_SESSION_EVENT_TYPES)')
|
||||
expect(output).toContain('restoreTransformedCurrent(artifact)')
|
||||
expect(output).toContain('assertReleasedV2Header(header)')
|
||||
expect(output).toContain('validateInstalledCurrentSessionHeader(header)')
|
||||
expect(output).toContain("from '@deepseek-ai/dsh-session'")
|
||||
|
||||
@@ -191,13 +191,16 @@ export function renderSessionFormatCatalog(
|
||||
'export const sessionFormatCatalog = createSessionFormatCatalog({',
|
||||
` currentVersion: ${currentVersion},`,
|
||||
` codecs: [${codecs.join(', ')}],`,
|
||||
` encodeCurrentArtifact: artifact => ${currentCodec}.encodeArtifact(artifact),`,
|
||||
` currentEncoder: ${currentCodec},`,
|
||||
` migrations: [${declarations.map(item => item.migration).join(', ')}],`,
|
||||
' restoreCurrent(artifact) {',
|
||||
` const restored = ${restorer}(artifact, KNOWN_SESSION_EVENT_TYPES)`,
|
||||
' validateInstalledCurrentSessionArtifact(restored)',
|
||||
' return restored',
|
||||
' },',
|
||||
' restoreTransformedCurrent(artifact) {',
|
||||
` return ${restorer}(artifact, KNOWN_SESSION_EVENT_TYPES)`,
|
||||
' },',
|
||||
' restoreCurrentHeader(header) {',
|
||||
` ${headerValidator}(header)`,
|
||||
' validateInstalledCurrentSessionHeader(header)',
|
||||
|
||||
@@ -791,6 +791,7 @@ function builtBinSmokeGate(needs: string[] = ['build']): Gate {
|
||||
// unbuilt, so these files self-skip there.
|
||||
'packages/workflow/workflow-worker-thread/tests/built-worker.e2e.ts',
|
||||
'packages/code-runtime/code-runtime-worker-thread/tests/built-lib.e2e.ts',
|
||||
'packages/session/session-persistence-jsonl/tests/built-migration-worker.e2e.ts',
|
||||
'packages/lsp/lsp-stdio/tests/built-lib.e2e.ts',
|
||||
], {
|
||||
label: 'built-bin smoke',
|
||||
|
||||
@@ -58,6 +58,51 @@ function validationHeader(value: unknown): unknown {
|
||||
return header
|
||||
}
|
||||
|
||||
function validationRow(source: Readonly<Record<string, unknown>>): Record<string, unknown> {
|
||||
if (source.type !== 'request/header') return { ...source }
|
||||
const data = source.data
|
||||
if (data === null || typeof data !== 'object' || Array.isArray(data)) return { ...source }
|
||||
const header = (data as Record<string, unknown>).header
|
||||
if (header === null || typeof header !== 'object' || Array.isArray(header)) return { ...source }
|
||||
if ((header as Record<string, unknown>).tools !== '{{tools}}') return { ...source }
|
||||
return {
|
||||
...source,
|
||||
data: {
|
||||
...data,
|
||||
header: { ...header, tools: [] },
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function restoreRequestHeaderTokens(
|
||||
events: readonly SessionEvent[],
|
||||
rows: readonly Readonly<Record<string, unknown>>[],
|
||||
): SessionEvent[] {
|
||||
const sources = rows.filter(row => row.type === 'request/header')
|
||||
let sourceIndex = 0
|
||||
return events.map((event) => {
|
||||
if (event.type !== 'request/header') return event
|
||||
const source = sources[sourceIndex]
|
||||
sourceIndex += 1
|
||||
const sourceData = source?.data
|
||||
const sourceHeader = sourceData !== null && typeof sourceData === 'object' && !Array.isArray(sourceData)
|
||||
? (sourceData as Record<string, unknown>).header
|
||||
: undefined
|
||||
if (sourceHeader === null || typeof sourceHeader !== 'object' || Array.isArray(sourceHeader)
|
||||
|| (sourceHeader as Record<string, unknown>).tools !== '{{tools}}') return event
|
||||
return {
|
||||
...event,
|
||||
data: {
|
||||
...event.data,
|
||||
header: {
|
||||
...(event.data as unknown as { header: Record<string, unknown> }).header,
|
||||
tools: '{{tools}}',
|
||||
},
|
||||
},
|
||||
} as SessionEvent
|
||||
})
|
||||
}
|
||||
|
||||
function renderFixture(headerLine: string, events: readonly SessionEvent[]): string {
|
||||
return [
|
||||
headerLine,
|
||||
@@ -95,6 +140,7 @@ function parseFixtureObjectLine(line: string, lineNumber: number): Record<string
|
||||
function parseFixtureRows(content: string, headerValue: unknown): SessionEvent[] {
|
||||
const rows: Record<string, unknown>[] = []
|
||||
const rowLines: number[] = []
|
||||
const eventLines: number[] = []
|
||||
let nextSeq: SessionLogOffsetType = SessionLogOffset(0)
|
||||
let headerSkipped = false
|
||||
for (const [index, line] of content.split(/\r?\n/).entries()) {
|
||||
@@ -113,7 +159,9 @@ function parseFixtureRows(content: string, headerValue: unknown): SessionEvent[]
|
||||
if (!Object.hasOwn(record, timeKey)) record[timeKey] = 0
|
||||
rows.push(record)
|
||||
rowLines.push(index + 1)
|
||||
nextSeq = SessionLogOffset(nextSeq + projectedRowCardinality(record))
|
||||
const cardinality = projectedRowCardinality(record)
|
||||
for (let offset = 0; offset < cardinality; offset += 1) eventLines.push(index + 1)
|
||||
nextSeq = SessionLogOffset(nextSeq + cardinality)
|
||||
}
|
||||
// Versionless protocol fixtures and current projected snapshots use scalar
|
||||
// event rows. Current snapshots may contain owner-restored scrub tokens such
|
||||
@@ -143,18 +191,52 @@ function parseFixtureRows(content: string, headerValue: unknown): SessionEvent[]
|
||||
}
|
||||
})
|
||||
}
|
||||
let restore: ReturnType<typeof sessionFormatCatalog.createRestore>
|
||||
try {
|
||||
return [
|
||||
...sessionFormatCatalog.decodeArtifact(validationHeader(headerValue), rows).events,
|
||||
] as unknown as SessionEvent[]
|
||||
restore = sessionFormatCatalog.createRestore(validationHeader(headerValue), {
|
||||
recovery: 'strict',
|
||||
validation: 'current',
|
||||
})
|
||||
} catch (error) {
|
||||
const detail = error instanceof Error ? error.message : String(error)
|
||||
const storedRow = /\brow (\d+)\b/.exec(detail)
|
||||
const line = storedRow === null ? 1 : rowLines[Number(storedRow[1])] ?? 1
|
||||
throw new Error(`session snapshot line 1: ${detail}`, { cause: error })
|
||||
}
|
||||
for (const [index, row] of rows.entries()) {
|
||||
try {
|
||||
restore.decodeRow(validationRow(row))
|
||||
} catch (error) {
|
||||
const detail = error instanceof Error ? error.message : String(error)
|
||||
throw new Error(`session snapshot line ${rowLines[index] ?? 1}: ${detail}`, { cause: error })
|
||||
}
|
||||
}
|
||||
try {
|
||||
return restoreRequestHeaderTokens(
|
||||
[...restore.finish().events] as unknown as SessionEvent[],
|
||||
rows,
|
||||
)
|
||||
} catch (error) {
|
||||
const detail = error instanceof Error ? error.message : String(error)
|
||||
const line = fixtureDiagnosticLine(error, rowLines, eventLines)
|
||||
throw new Error(`session snapshot line ${line}: ${detail}`, { cause: error })
|
||||
}
|
||||
}
|
||||
|
||||
function fixtureDiagnosticLine(
|
||||
error: unknown,
|
||||
rowLines: readonly number[],
|
||||
eventLines: readonly number[],
|
||||
): number {
|
||||
const detail = error instanceof Error && error.cause instanceof Error
|
||||
? error.cause.message
|
||||
: error instanceof Error ? error.message : String(error)
|
||||
const physicalRow = /^released Session row (\d+)/.exec(detail)
|
||||
if (physicalRow !== null) return rowLines[Number(physicalRow[1])] ?? 1
|
||||
const event = /Session event (\d+)/.exec(detail)
|
||||
?? / at seq (\d+)/.exec(detail)
|
||||
?? /inherited Session cut (\d+)/.exec(detail)
|
||||
return event === null ? 1 : eventLines[Number(event[1])] ?? 1
|
||||
}
|
||||
|
||||
function withoutEnvelope(events: readonly SessionEvent[]): Array<Omit<SessionEvent, 'seq' | 'time'>> {
|
||||
return events.map((event) => {
|
||||
const { seq: _seq, time: _time, ...projected } = event
|
||||
|
||||
@@ -46,11 +46,7 @@
|
||||
- img
|
||||
- text: Read
|
||||
- button "b.txt"
|
||||
- button "Think This path was interrupted.":
|
||||
- img
|
||||
- img
|
||||
- text: Think This path was interrupted.
|
||||
- text: Stopped Now give the final answer. 7/25 {{clock}}
|
||||
- text: Now give the final answer. 7/25 {{clock}}
|
||||
- button "Copy":
|
||||
- img
|
||||
- paragraph: DONE
|
||||
|
||||
@@ -4,8 +4,7 @@
|
||||
- button "Collapse calls": Calls
|
||||
- img
|
||||
- searchbox "Search trajectory"
|
||||
- region "Trajectory timeline":
|
||||
- tooltip "ASSISTANT {{clock}} → {{clock}} Total {{duration}} · TTFT {{duration}} · Decoding {{duration}}"
|
||||
- region "Trajectory timeline"
|
||||
- table:
|
||||
- rowgroup:
|
||||
- row "SYSTEM, Initial System Prompt":
|
||||
|
||||
Reference in new issue
Block a user