Expose the pure Host formatSessionMarkdown(events, sourcePath?) converter at the package entry. Persistence stat/list now report the selected materialized JSONL artifact path, and the Markdown copy route includes that exact optional path after the title. Frontend progressive display and Session format remain unchanged.
list(), the root encoding check, and historical corpus fingerprints read
project directories, Session directories, headers, and stats one at a time.
On a network filesystem every listing paid the summed round trips. These
reads now run through one p-limit limiter per backend (listConcurrency,
default 3), shared by concurrent listings; results and the first reported
failure keep the sequential order.
A successful verifier message was treated as terminal even though the
Worker Thread was still alive. The parent immediately terminated the
worker and detached cancellation, so an error, nonzero exit, or caller
cancellation after that message could be ignored and verification could
resolve successfully.
Keep the first successful response pending until an observed natural exit
with code 0. A missing response or nonzero exit rejects after the worker
has already exited. Error and abort callbacks retain the existing
terminate-and-join path, including its first-error and AggregateError
behavior. The scheduler stays unchanged: its permit remains held until
the verification promise settles, now after successful worker quiescence.
The worker already closes its parent port in finally, so success needs
neither a forced stop nor another protocol message.
Replace success-termination expectations with deterministic message/exit
barriers and cover post-response errors, nonzero exit, cancellation, and
permit handoff. Remove the obsolete hand-written termination thenable and
its retired unknown-assertion baseline entry. Add real source workers for
plain and Zstandard input, and require the built plain-Node migration smoke
to complete with zero termination calls. Document the settlement condition.
Native Windows CI recorded process access violation 0xC0000005 while a
migration fixture was active. Removing unnecessary successful-worker
termination is consistent with that evidence, but this commit does not
claim that the native faulting module or mechanism has been established.
The deterministic regressions prove the lifecycle defect independently;
the next native CI run must assess whether the observed crash recurs.
Validation:
- Original source failed all new message-to-exit regression scenarios;
the updated focused lifecycle file passes 19 tests with 100% coverage.
- Node 24 owning package: 19 files, 741 passed, 1 skipped; every included
package source file reaches 100% statements/branches/functions/lines.
- Real source workers passed for none/zstd; built plain-Node smoke passed
with zero terminate calls and no timeout or process signal.
- Exact Node 22.19 source-mode none/zstd workers naturally exited 0 with
zero terminate calls, exercising the async Module.register fallback.
- Host TypeScript build, focused lint, no-new-unknown-assertion gate,
README translation pairing, and all 42 doc-sync gates passed.
Related to #4387 and #4394.
Cold current-generation reopen freezes the complete decoded event graph
before publishing shared-frozen history. Compact assistant streams carry
125,000 text values and 124,600 time deltas in the required benchmark;
for-in traversal enumerates their array indexes as string property keys.
An open-only profile attributes about 4.2 ms to complete event freezing.
Use numeric indexing for arrays produced by JSON decoding and the fixed
migration catalog. Those arrays contain indexed JSON values only. Keep
every Object.freeze call and every nested-object visit, preserve the
object-key path including own __proto__ and constructor data, and never
infer deep immutability from a shallow Object.isFrozen result. No decoding,
admission, relationship validation, or freezing moves past open().
The unchanged five-sample V4 reopen median falls from 22.6 to 17.7 ms on
M4 Pro / Node 24.19. Restoring and rebuilding the exact old traversal gives
21.7 ms. First-history and resume also improve; Agent retained heap stays
4.5 MB and every 128 MB case completes. Keep the existing 63 ms gate,
fixture dimensions, cold-process setup, and all other budgets unchanged.
Local measurements establish removed work; hosted CI remains the required
runner verdict.
Cover nested opaque arrays and objects, special JSON keys, rejected
mutations, and shared event identity through historical V3 and native V4
reads. Record the before/candidate/restored samples and the local-versus-CI
limit in the existing performance note.
Session V4 reserves developer-role messages for incremental model-visible
session changes. The role, tool addition/removal blocks, and deferred-loading
marker need a durable representation before independent provider and UI
integrations can emit or consume them. Known developer events must match an
open turn and step; unsupported providers and UI consumers fail explicitly.
Request headers already capture full tool definitions. Persisting another
ToolSchema in each addition would duplicate Session information and create
two definitions that could drift. Additions therefore store toolName, and
the containing developer event uses headerSeq to select one earlier known
request/header for all its additions. A same-name replacement keeps its
historical definition identity through restart, fork and surface changes.
Removals require no definition copy, and ordinary developer messages omit
headerSeq. Generic sourceEventSeqs remains independent derivation metadata.
The rejected inline tool field is optional never with @persistenceReserved.
The schema extractor retains that uninhabited slot, so permitting a future
value requires a format bump instead of being mistaken for compatible
optional metadata. Unrelated optional fields remain additive and readable.
Native and Session admission reject missing, forward, non-header, unknown,
ambiguous and incomplete referenced definitions, and reject the retired
inline tool field. Validate complete schema fields only for referenced
definitions, preserving unrelated header behavior and extra JSON metadata.
Unknown ignorable records remain opaque. Empty developer nodes retain their
surface position without producing a message or shadowing the system head.
No automatic emitter, provider activation, UI implementation, or speculative
public resolver is added.
Keep the core-owned developer source binding beside the introduced role,
including the request-only type rejection and attribution round trips. Update
consumer guards, SDK/snapshot handling, bilingual owner documentation, and
the same unreleased V4 acknowledgement. Keep earlier tool/source migration
work and its documentation in the parent. Preserve the existing recorded
Cordis inspection generation from the original developer change.
Integration keeps the final source layer as the sole parent and preserves
master's Messages input filtering, empty-user omission, empty tool results,
and system-update placement. Developer/deferred guards still run before
conversion. Native V4 admission, header references, reservation declarations,
and generic tooling match the validated marker candidate. Parent-owned
fixture and generator fixes remain in the parent.
Earlier feature validation passed full build, full lint, 41 doc-sync gates,
and 16 hygiene gates. The unchanged native implementation retains that evidence.
The focused Session/V4/request-input run passes 884 tests across 38 files with
100% statement, branch, function and line coverage for the V3-to-V4 package
and Session surface. Consumer tests pass 482 with one existing skip; tool,
message and optional-field tests pass 146. Cordis inspection and compaction
replays pass in both source and built modes (two cases per mode). Ten invalid
referenced-definition/unknown-header controls failed before their scoped
guards and pass after them.
Reservation validation: the actual generated developer root records tool as
optional primitive never. Widening it requires a bump, while traceId remains
additive and preserved; all 85 focused reader/classifier tests pass.
Final-parent validation: 205 tests across eight provider, type, catalog,
Session and native-format files pass. The preserved parent fixture verifies
Messages input-history replay; runtime/declaration and parent-owned-file
comparisons show no loss during the final rebase.
Co-authored-by: Tianyi Cui <[email protected]>
Represent tool results as tool-role messages and migrate released wrappers
without losing outer own JSON fields, call identity, error state, or writer
field order. Refuse nested or wrapper-only information that has no lossless
current representation before successor publication.
Validate current event relationships directly instead of projecting V4 data
back into V3. Keep lifecycle, PTC, title/command, compaction, system-field and
retired-syntax checks on catalog restoration and ordinary JSONL opens. The
Worker JSON equality helper must inspect own keys so it cannot admit PTC
arguments that Node rejects.
Keep the source role change, required native admission and consumer fixture
updates in one complete review layer. Explain shared-type changes in the
persistence record and the native reader rationale in its existing note.
Use checked branded fixture identities and discriminant narrowing instead
of unknown double casts; fixture values and assertions remain intact.
This recomposes the original tool-role work by _Kerman with the independently
validated migration and admission repairs by Tianyi Cui. Frozen generations
and snapshot input version selection are retained.
Co-authored-by: Tianyi Cui <[email protected]>
Merge 3ea52fc3e475a3e7c430812bde4209258416e57b into a0a61a8237f61344a01448a97954df603d8fff73. Retain the V2-to-V3 migration dependency alongside native system flock support, removing the replaced fs-ext dependency. Union the lockfile workspace importer entries; frozen installation validates the result without regeneration.
Preserve release publint async lane-signal cancellation, close ownership, and spawn-error regressions, satisfying the incoming CI budget intent. Keep canonical, prompt, preset, and latest migration regression behavior. Rename an arbitrary preset fixture value to custom-agent to avoid the inherited vendor-rescope false positive without changing test semantics.
Validation: frozen install; full build including Darwin addon; focused V3/persistence/prompt/publint and runtime tests; built migration, two-process lease, and loader smokes; native flock and package-matrix tests; touched dependency and CI owner tests; all hygiene leaves (vendor residue repaired and rechecked); doc-sync 34/34.
Keep the synthetic compressed historical fixture step-first without changing frozen generation data or shared fixtures. Pin its migrated empty system head and unchanged source bytes. Exercise malformed native system rows before and after recoverable damage through scanner and read/write handles. Remove the unreachable duplicate unsupported-format catch after current-row admission; preserve its owned rejection and generic error path.
Validation: original zstd regression red; admission-bypass negative control 3/3 red; five focused specs 314/314 pass with format.ts 100% statements/branches/functions/lines. Final admission cleanup 15/15 pass. Full build-backed lint found one matcher typing error; exact identifier oracle replaces it and focused type-aware lint passes.
Compose the shared release branch PTC rename with the structural system-head migration in the existing V2-to-V3 stage. Audit source payloads and remap local coordinates before renaming dispatch vocabulary and owned plugin attribution; retain original message identities and generation-qualified references. Native V3 validation combines the same system and PTC lifecycle semantics through a private released-validation view.
Reuse the V3 row-admission owner in current JSONL scanning before recoverable-tail suppression, so malformed rows cannot hide retired required PTC tags or header.system. The new corruption-prefix regressions and all173 existing JSONL scanner tests pass. Combined migration/catalog/persistence suite:300 tests; executable V3 migration source has100% coverage. No released generation, prior migration implementation, shared release branch, benchmark or CI workflow is modified by this integration.
Write handles now hold a durable lease (session.lock.json beside the
log): a random owner token, diagnostic pid, and an expiry. Acquisition
wins by exclusive create; a second process's create or write open rejects
while the record is unrenewed for less than leaseTtlMs (default 5 min),
and takes over after that — a crashed holder is waited out, never
reclaimed by pid. The holder renews every leaseRenewIntervalMs (default
4 min); a renewal that finds a foreign, vanished, or expired record — or
fails outright — marks the lease lost permanently, so every later
append/flush rejects with SessionOwnershipLostError while reads continue.
Close releases the record; read handles never touch it. Takeover of an
expired record is eventually exclusive: a replaced holder stops within
one renewal interval.
Refs #3245
`session[.vN].jsonl` was assembled independently by the JSONL provider
(twice), the session-log export archive, and the recorded-session fixture
helpers. `dsh-session-format` now owns `sessionFormatLogFilename` and
`parseSessionFormatLogFilename`; the three consumers append only their
compression suffix, and the migration note names the owner.
`snapshotSessionFormatJson` re-implemented the lossless JSON walk that
`dsh-util-values` already publishes as `snapshotJsonValue` + `deepFreeze`,
and the frozen v0 relationship validator carried a third structural JSON
comparison next to `deepEqualJson`. Both now delegate; the format package
reports one `is not lossless JSON` diagnostic per labelled subject instead
of ten member-specific ones, and enumerable accessors whose values survive
a JSON round trip are accepted like `JSON.stringify` accepts them.
`ensureJsonlGenerationCurrent` returned a live Zstandard body iterator that
every production caller disposed unread before decoding the same bytes again;
the header read now decodes only the first frame and owns no decoder past its
return. The JSONL backend reuses the generation module's stable-revision read
instead of keeping a second copy, `encodeCurrent` encodes the validated
current artifact it is documented to receive instead of re-running the
migration chain on it, and the test-only `inspectVersion` catalog method and
the `migrateSessionSnapshotFixture` duplicate of
`prepareSessionSnapshotFixtureForComparison` are removed.
The persistence seam is now create/open/stat/list returning per-session
SessionHandles (read/append/flush/close); every log read and write flows
through the owning handle. The seam package exports only the service and
handle contracts, consumer-visible errors, and pure durable-data
validation helpers; each backend owns its complete storage runtime, and
the shared contract suites pin equivalent observable behavior. The
backend routes published sessions' live events by id into the active
write handle; agent-loop only acquires, seeds, and closes the handle.
Resume appends interruptedTurnClosers through its write handle;
session-query owns the revision-keyed cold cache. Legacy-only surfaces
are removed in the same swap: locate/readRaw/supportsRawArtifacts, the
legacy event-shape read migration, zstd torn-frame salvage,
DSH_SESSION_JSONL, and hook transcript_path population; a torn final
zstd frame is discarded whole; the session-list cold blank probe returns
on stat metadata (eventCount derived from the last physical row,
sizeBytes). The WebUI ZIP export serializes the logical log from a read
handle, so both backends export identically.
Refs #3245
`JsonlSessionPersistence.exists` treats ENOENT as absence and stats the
path's parent first, so a session directory blocked by a regular file
stays a storage fault. Only Windows needs that: it reports ENOENT rather
than ENOTDIR for `regular-file/child`, while POSIX open reports ENOTDIR
before the branch is reached.
The stat ran on every platform and every absent probe. findLog issues
four probes per project directory and the coordinator resolves an id
twice per inspect, so nearly every probe paid it. Counting fs calls
against a five-project store, loading an existing session drops from
40 open + 41 stat to 40 open + 3 stat.
Tianyi Cui