Keep the synthetic compressed historical fixture step-first without changing frozen generation data or shared fixtures. Pin its migrated empty system head and unchanged source bytes. Exercise malformed native system rows before and after recoverable damage through scanner and read/write handles. Remove the unreachable duplicate unsupported-format catch after current-row admission; preserve its owned rejection and generic error path.
Validation: original zstd regression red; admission-bypass negative control 3/3 red; five focused specs 314/314 pass with format.ts 100% statements/branches/functions/lines. Final admission cleanup 15/15 pass. Full build-backed lint found one matcher typing error; exact identifier oracle replaces it and focused type-aware lint passes.
Compose the shared release branch PTC rename with the structural system-head migration in the existing V2-to-V3 stage. Audit source payloads and remap local coordinates before renaming dispatch vocabulary and owned plugin attribution; retain original message identities and generation-qualified references. Native V3 validation combines the same system and PTC lifecycle semantics through a private released-validation view.
Reuse the V3 row-admission owner in current JSONL scanning before recoverable-tail suppression, so malformed rows cannot hide retired required PTC tags or header.system. The new corruption-prefix regressions and all173 existing JSONL scanner tests pass. Combined migration/catalog/persistence suite:300 tests; executable V3 migration source has100% coverage. No released generation, prior migration implementation, shared release branch, benchmark or CI workflow is modified by this integration.
Treat Vitest asymmetric matchers as unknown expected values and spell void assertion callbacks explicitly. This preserves the migration identity and refusal checks under the full type-aware lint configuration without relaxing validation or suppressing a rule.
Require a string call identity before constructing the private repair-validation prefix; do not stringify arbitrary durable JSON. Remove a redundant narrowed version comparison and use explicit void test callbacks.
Verified full type-aware package Oxlint (90 rules), all 71 focused cases with 100% exact V3 statements/branches/functions/lines, and host TypeScript build. No coverage ignores or broad lint exceptions.
Exercise malformed source envelopes, feedback metadata, file attachments, native system ownership, protected head replacement, started versus repaired tools, dangling coordinate mappings and successful compaction ranges. The focused package suite now has 71 cases with exact V3 source coverage at 100% statements, branches, functions and lines.
Remove unreachable rechecks after owned payload validation instead of adding artificial tests or coverage ignores. Unseeded inherited markers already fail at arrival; message arrays and IDs are validated before observation. Bind the unchanged frozen decoder run callback without an unreachable wrapper.
Construct supported historical inputs with the first surface inside step/start instead of weakening publication, singleflight, packed-row, and read-handoff assertions into refusal cases. Account for the inserted empty system head and shifted event sequence when V2 rows become V3; preserve exact native V3 expectations and original source bytes.
Current V3 promotion must preserve chronology, so historical user surfaces before the first step are unsupported. Assert refusal for read and write opens, preserve source bytes and identity, and forbid intermediate or current publication. Frozen codec validation remains in its existing owner tests; V2 no-downgrade coverage lives in v2-system-migration.spec.ts.
V2 assistant attempts have no V0 semantic dispatch case: validate their exact V2 members and positive step coordinates locally while preserving the embedded stream. Explicitly classify agent-message relay attribution and file attachment metadata; both preserve foreign identities and non-Session byte counts, and reject unknown members.
Native V3 request headers retain current extension fields while rejecting retired header.system, independently of the closed V2 migration inventory. Add direct and multihop attempt regressions, exact source/file negative cases, and native header roundtrip coverage. All 41 focused tests and host tsc pass; scoped lint and bilingual pairing pass.
Stream an empty protected head after the first step and replace it before changed or cleared request headers, without moving source events. Remap only audited local sequence references, derive inherited cuts after upstream cardinality changes, and preserve generation-qualified captures and message identities.
Refuse pre-step surfaces and out-of-step prompt changes rather than invent lifecycle events. Reject unclassified migration payloads and detect deterministic generated-ID collisions in either source order. Native V3 accepts empty and in-history system messages, protects only the head, and reuses frozen relationships through a private nonescaping projection; historical repair IDs remain opaque.
Keep released codecs untouched and distinguish migration admission from native extension admission. V3 structural payload preflight cannot disappear behind recoverable row corruption. Validation: 35 focused tests, host TypeScript build, focused Oxlint, documentation quick gates and paired README recording. Parent owns installed-core/catalog/persistence integration and lockfile propagation.
Consolidate the representation-change PR and its rebase reconciliations into one baseline. Preserve the exact tree and keep the in-history feature in the dependent PR. Follow-up fixes remain separate.
Session format v2 embeds each attempt's compact stream in
assistant/message and assistant/attempt, but Host and client consumers
still expanded it into per-member TimedStreamChunk arrays and did
per-member work; expandAssistantStream materializes the full array before
find/toReversed/break can answer. Session Stats (the projection phase of
every Session open), the token meter's usage and provider-assembly folds,
the subagent output fold, and the Session Controller image lookup still
paid O(members) allocation and time per settlement.
The Chat and Trajectory definitions were already settled from
message.content on master; the remaining per-member folds stay.
dsh-llm now exports record-level readers (first token, visible content,
visible text, last raw chunk of a type, raw chunks of a type, joined
text, run-aware assembly, per-run first-token/first-visible times) that
scan the compact records once with early exit. Session Stats reads
assistantStreamFirstTokenTime, the token meter reads
lastAssistantStreamChunk(stream, 'usage') and assembles through
assembleAssistantStream, the subagent output fold appends
joinAssistantStreamText, and the Session Controller scans
assistantStreamChunks(stream, 'block-end').
expandAssistantStream is deliberately not memoized: retaining expansions
costs roughly ten times the compact stream for the Session's lifetime.
It remains the validating path at durable boundaries.
Synthetic 200-turn v0 migration benchmark, median of five: first-open
projection 28.0 ms -> 5.4 ms, first-open total 76.9 -> 50.0 ms, peak RSS
137.2 -> 94.9 MB; reopen projection 17.8 -> 5.6 ms; all phase budgets and
the 128 MB heap constraint keep passing.
The migrating first open measured 1,963 ms on the CI runner under the
128 MB heap limit; 4,000 ms keeps a 2x margin while the repeated-snapshot
implementation still fails by heap exhaustion and would need ~10 s.
The migrating first open of the synthetic log costs about 1 s on the
reference machine under the 128 MB heap limit and about twice that on the
CI runner, so 3,000 ms keeps headroom while staying far below the ~5 s
(~10 s on CI) of the repeated-snapshot implementation.
Add a benchmark lane (`vitest.bench.config.ts`, `pnpm run test:bench`,
gate mode `ci-bench`) and a required `node 24 / benchmarks` CI job that
runs it alone. Benchmarks synthesize their input in-process from fixed
parameters and fail on documented budgets:
- `open-generation.bench.ts`: a 200-turn released-v0 log with 500 text
and 125 reasoning deltas per reply (127,400 events, ~2.8 MB) encoded
through the frozen v0 codec; the migrating first `open()` must finish
within 2,000 ms in a child process capped at 128 MB of old space, and a
fresh process must open the published current generation within 500 ms.
- `conversation-fold.bench.client.ts`: 200 replies whose compact streams
hold 2,000 text + 500 reasoning deltas each, folded through every Chat
Definition by the real assembler; the fold must finish within 150 ms
and stay within 3x the fold of the same window with 100 deltas per
reply.
On this commit both gates fail: the migration exhausts the 128 MB heap
(4.8 s and 696 MB peak RSS without the cap; the pre-stack decode of the
same bytes took 34 ms and 168 MB) and the fold scales 11x with the delta
count. The stacked fixes bring both paths to O(records).
Write handles now hold a durable lease (session.lock.json beside the
log): a random owner token, diagnostic pid, and an expiry. Acquisition
wins by exclusive create; a second process's create or write open rejects
while the record is unrenewed for less than leaseTtlMs (default 5 min),
and takes over after that — a crashed holder is waited out, never
reclaimed by pid. The holder renews every leaseRenewIntervalMs (default
4 min); a renewal that finds a foreign, vanished, or expired record — or
fails outright — marks the lease lost permanently, so every later
append/flush rejects with SessionOwnershipLostError while reads continue.
Close releases the record; read handles never touch it. Takeover of an
expired record is eventually exclusive: a replaced holder stops within
one renewal interval.
Refs #3245
`SESSION_FORMAT_VERSION` is the only value that names the Session format
generation. Two other version fields had moved to 2 alongside it:
- The `dsh_session_log` request extension's outer schema `version` returns
to 1 and its Session header projection keeps `seedLength`, derived from
the logical inherited cut, so the external wire is unchanged in this PR;
`sessionFormatVersion` alone identifies the embedded generation.
- The `sessionStats` projection unit's `stateVersion` returns to 1: the
projection cache binds every checkpoint to the format generation, so a
generation change discards old rows without a unit version bump.
Test doubles and the standalone Client fixture that build a current
header now spell `SESSION_FORMAT_VERSION` instead of the literal.
The README file table listed four of the six source modules and omitted
the two largest, so the package read as an oversized identity conversion.
The summary now states that most of the source is the frozen released
v0/v1 event vocabulary, why the edge refuses a malformed historical log
before the installed current restorer runs, and that later edges reuse
those shapes; the table gains `payload-validation.ts` and
`relationships.ts` rows.
`session[.vN].jsonl` was assembled independently by the JSONL provider
(twice), the session-log export archive, and the recorded-session fixture
helpers. `dsh-session-format` now owns `sessionFormatLogFilename` and
`parseSessionFormatLogFilename`; the three consumers append only their
compression suffix, and the migration note names the owner.
`snapshotSessionFormatJson` re-implemented the lossless JSON walk that
`dsh-util-values` already publishes as `snapshotJsonValue` + `deepFreeze`,
and the frozen v0 relationship validator carried a third structural JSON
comparison next to `deepEqualJson`. Both now delegate; the format package
reports one `is not lossless JSON` diagnostic per labelled subject instead
of ten member-specific ones, and enumerable accessors whose values survive
a JSON round trip are accepted like `JSON.stringify` accepts them.