Files
deepseek-harness/docs/cookbook/adding-a-session-format-version.md
T
Tianyi Cui 5152c097bb docs(session): catalog every V3-to-V4 transformation and refusal
The V3-to-V4 README described the main representation changes but did not
enumerate the complete admission and preservation rules. That left reviewers
without a package-owned reference for supported histories, child evidence,
native validation, or cases that must fail without publishing a successor.

Document historical conversion separately from native V4 admission, including
source renames and collisions, first-class tool results, child catalogs,
delivery metadata, retained identities and references, lifecycle checks,
developer-message constraints, fork behavior and recovery entry points.
Explicitly identify the retired-wrapper refusal and the limited scope of
general content validation rather than implying a complete payload validator.

Require this transformation/admission catalog in the version-addition
cookbook so future adjacent migrations document each supported mapping,
unchanged representation, refusal and preservation obligation. Preserve the
accepted-baseline policy: compatible V4 additions use new acknowledgements;
only breaking changes require a new writer version.
2026-09-18 23:49:21 +08:00

15 KiB
Raw Blame History

Cookbook: adding a Session log format version

English | 中文

Summary

Use this tutorial to introduce the next structural Session log version without rewriting released data. Read the version and release-status authority to identify the checkout writer and the latest released format. Let N denote that verified released format and N+1 the target; substitute numeric values for these placeholders in names and metadata. Start with a working contributor checkout and read the package checklist, format library, and released-format decision.

Table of Contents

1. Choose the version and release base

Bump the format for a structural change to headers, event envelopes, core event semantics, or surface reconstruction. Backward-compatible changes can remain at the current version through new acknowledgements; follow the versioning rule and type-change rules. Distinguish the Session format integer from package release versions, SQLite schema versions, projection-unit versions, and protocol-wrapper versions.

Let N be the accepted or published version in version/status when planning the next breaking change. A breaking change after accepted V4 therefore needs V5 even before V4 publication is recorded; compatible additions do not require that step.

Use a shared release/* integration base for N+1. The base change adds the writer, codec, catalog wiring, identity migration, and verification. Create each independent child branch from that base and target its PR at the release branch, not another independent child’s branch. Each child adds its structural transformation, validators, consumers, and tests to the same adjacent migration package. Do not allocate extra versions just to represent review order. Merge reviewed children into the release branch through PRs, then validate the combined result before release. Honor release-branch force-push and deletion protections; do not force-sync it.

Released codecs and accepted migration meanings remain stable. An unaccepted N→N+1 transition may combine coordinated breaking changes. Once accepted, preserve that transition: later compatible evolution uses new same-version records, while new breaking changes require the next edge and their own header increase.

Use disposable, isolated Harness homes while N+1 remains open for integration. An interim N+1 file already has the target writer version, so a later edit to N→N+1 will not migrate that file again. Re-run from unchanged historical input in a fresh test home; never repair this by rewriting a committed generation or reusing a real user's home.

Before changing the writer, use the archive command to preserve its complete persistence schema, then add the bilingual format reference under docs/persistence-changes/historical-formats/vN.*. Preserve all earlier records. The format coverage check requires every integer below the new writer to have its own document; the current generated catalog covers only the new writer.

2. Add an identity edge

Follow the package checklist to create a library for N→N+1, not a mounted plugin. An identity body conversion is only an initial wiring scaffold. The V2-to-V3 specification is a fixed example of explicit transformations and preservation rules, not an edge to extend or treat as an identity conversion.

Declare dsh.sessionFormatMigration with numeric from: N and to: N+1, an export path, and the exported migration, source codec, target codec, target-header validator, and target restorer. Reuse the source codec exported by the preceding edge package and depend on that package; do not copy or redefine a released codec. Export the target codec and validators from the new package. Add the edge as a direct dependency of the catalog and add the workspace’s TypeScript paths and project references.

Set SESSION_FORMAT_VERSION in core Session types to N+1 alongside the new edge declarations, then generate the catalog. The command below generates only the declared chain; it does not implement a new version:

pnpm run gen-session-format-catalog

The generator requires exactly one adjacent package for every step from zero to the writer version, matching directory/package names, matching adjacent codec exports, and declared dependencies. It rejects gaps, duplicate or extra edges, unknown metadata members, and a catalog that does not share Session through peer plus development dependencies. Fix the declarations rather than hand-editing generated.ts. The catalog is build-static; plugin mounting must not determine historical readability.

3. Implement per-artifact stages and validation

Use the Stage interfaces, not a whole-artifact array-to-array migrator. An immutable SessionFormatMigration declaration supplies migrateHeader, validateTargetHeader, and createStage. Every call to createStage creates independent state for one source artifact. Keep counters, pending events, and reference maps there; never share a mutable stage across Sessions.

Implement transformEvent(event, context), transformRun(run, context), and finish(context). Emit synchronously through context.emitEvent or context.emitRun; a call can produce zero, one, or many outputs. Let a stage consume codec-owned compact runs directly, or iterate run.expand() without materializing an intermediate array. The caller owns scheduling, and the chain finishes upstream stages before downstream stages.

Treat the inherited cut as a logical event count, not a physical row count. Expose headerInheritedEventCount only when it is known before EOF; finish returns the exact target cut. A preceding cardinality-changing edge can make that count unavailable at construction. Derive it from validated seed markers when required, and test seeded multi-hop restoration from each supported historical generation through N+1, not just direct N input. Never substitute zero for an unknown cut.

Define the new edge's event admission and transformation rules explicitly. The V2-to-V3 source audit and alpha V0→V1 rule own the policies of those released edges, not the new edge. Do not generalize either to every edge. A change to structure or event positions requires classifying source events, payload members, and references, and explicitly deciding whether opaque data can remain valid. Equal-version retention alone does not prove a structural transformation safe. Validate target semantics and give each newly accepted case a rejecting counterexample; never widen older edges to hide an unsupported transformation.

Maintain the new edge's README as its single complete transformation and admission catalog, following the V2-to-V3 specification. Enumerate every converted header, event, message slot, and payload field; exact rename or collision rules; preserved ids, coordinates, references, and inherited cuts; external prerequisites; refusal and opaque-data policies; and unsupported or deferred behavior. Separate historical source conversion from native target admission, and state which codec, restorer, installed validator, and recovery policy performs each check. Record any validation that is deliberately narrower than the declared schema. Update this catalog in the same change as its code; a generated schema inventory or PR description does not replace it.

Prove strict restoration through sessionFormatCatalog.createRestore(header, { recovery: 'strict', validation: 'current' }), feeding rows in order and calling finish(). This exercises physical decoding, the complete chain, and installed current Session validation. Production's recoverable/transformed policy is not a replacement for strict fixture and publication verification. Preserve documented historical validation exceptions rather than claiming stricter source validation than the edge actually performs.

4. Update current-version consumers

Trace each current-version consumer, including Session creation/restoration, JSONL filename selection and publication, the catalog's current encoder/restorer, projection-cache generation identity, replay and snapshot normalization, physical fixture-layout validation, and TypeScript/Python SDK recordings. Use the writer constant where a value means current; keep literal historical versions in released codecs and historical fixtures. Update current documentation and generated references through their owners.

Do not bump unrelated versions automatically. A request wrapper's sessionFormatVersion identifies its embedded Session generation; its outer schema version has its own meaning. Projection-unit state versions likewise do not replace the cache's Session-generation identity.

Verify both read and write paths. Header-only listing must not read bodies or publish. Historical read open may return the migrated in-memory artifact without writing; write open must verify and publish only the final current successor before append. The source path, bytes, and inode stay unchanged. A newer or invalid selected generation must not cause fallback to a predecessor. The preparation decision owns publication timing.

5. Create snapshot successors

Read snapshot ownership and the snapshot library. The corpus policy keeps its declared baseline as replay input so upgrades exercise the adjacent chain; a writer bump does not require refreshing that corpus. Select the owning scenario when a current-generation successor is needed, keep each historical file, and use the target version’s canonical parent and child filenames. Never rename a predecessor to the target filename or change only its header.

For unchanged replay input, use keyless refresh on the owner, then replay without write-back. These SDK commands use text-turn and the checkout's writer version. Implement and wire N+1 before using them to generate that version, and select the actual affected owner for a feature:

pnpm run test:snapshot:refresh snapshots/sdk/sdk.snapshot.ts -t text-turn
pnpm run test:snapshot snapshots/sdk/sdk.snapshot.ts -t text-turn

Review the new generation, request sidecars, and protocol output together. Verify every predecessor remains byte-identical and that parent/child roles remain contiguous. Selection uses the numerically highest generation, so update shared references to the owner's selected parent. Do not use the packed-layout migrator as a version upgrader. If the model transcript must change, the scenario owner uses live recording under the testing policy, with its required provider key.

Keep cases older than the retained baseline explicit through snapshot.yml's sessionFormat.version and supported coverage names; record and refresh leave pinned Session fixtures untouched. Update the corpus policy for the current generation while retaining focused direct-edge, multi-hop, packed-row, retry/failure, and shipped-profile coverage. Check the corpus and both SDK projections; do not mass-refresh unrelated scenarios merely to silence a validation failure.

6. Validate the integrated result

Run from the repository root. These commands check catalog declarations, Stage composition, the released V2→V3 edge, and generation selection. They are a baseline; add focused coverage for the new edge:

pnpm run verify-session-format-catalog
pnpm exec vitest run scripts/gen-session-format-catalog.spec.ts packages/session/session-format/tests packages/session/session-format-v2-to-v3/tests packages/session/session-format-catalog/tests
pnpm run test:snapshot scripts/session-snapshot-corpus.corpus.ts

After implementing the new edge, add its actual test path to the focused Vitest run. Add the changed JSONL, replay, projection, and SDK tests selected by the actual diff, plus the built publication-Worker smoke when that path changes. Require successful strict migration, identity preservation for the skeleton, malformed and unknown-required-event refusal, deterministic repeated restores, independent concurrent stage state, seeded multi-hop cuts, unchanged predecessors, and no fallback. Report exact commands and failures, not an inferred full-suite result.

Update the owning Agent Note rather than adding a redundant decision record. Keep the release record unchanged until publication; after publication, update it with verified release evidence. Audit related active notes for supersession; retain independent rationale and leave archived notes frozen. Update bilingual prose together, re-record each changed pair with the repository tool, then run documentation checks:

pnpm run verify-translation-pairing --write docs/cookbook/adding-a-session-format-version.md
pnpm run test:docs
pnpm run doc-sync
pnpm run lint
git diff --check

Dev Note

None.