Commit Graph
397 Commits
Author SHA1 Message Date
Tianyi Cui 95400aa52c Merge V3 preset-reference migration into system-prompt representation
Compose the root’s exact legacy code-to-ptc preset conversion with the existing structural system-head migration. Keep dense source events, audited payload admission, local reference mapping, and native V3 extension behavior unchanged. Preserve the historical preset migration fixture byte-for-byte and test unsupported sparse/opaque sources explicitly.
2026-09-07 14:51:05 +08:00
Yichen Jiang ff33f79eb1 fix(session): migrate legacy code preset references to ptc 2026-09-07 13:52:22 +08:00
Tianyi Cui 78f03ffef9 fix(session): cover V3 admission and keep compressed fixture migratable
Keep the synthetic compressed historical fixture step-first without changing frozen generation data or shared fixtures. Pin its migrated empty system head and unchanged source bytes. Exercise malformed native system rows before and after recoverable damage through scanner and read/write handles. Remove the unreachable duplicate unsupported-format catch after current-row admission; preserve its owned rejection and generic error path.

Validation: original zstd regression red; admission-bypass negative control 3/3 red; five focused specs 314/314 pass with format.ts 100% statements/branches/functions/lines. Final admission cleanup 15/15 pass. Full build-backed lint found one matcher typing error; exact identifier oracle replaces it and focused type-aware lint passes.
2026-09-07 13:14:18 +08:00
Tianyi Cui 71b0d83822 Merge current V3 PTC vocabulary into system-prompt representation
Compose the shared release branch PTC rename with the structural system-head migration in the existing V2-to-V3 stage. Audit source payloads and remap local coordinates before renaming dispatch vocabulary and owned plugin attribution; retain original message identities and generation-qualified references. Native V3 validation combines the same system and PTC lifecycle semantics through a private released-validation view.

Reuse the V3 row-admission owner in current JSONL scanning before recoverable-tail suppression, so malformed rows cannot hide retired required PTC tags or header.system. The new corruption-prefix regressions and all173 existing JSONL scanner tests pass. Combined migration/catalog/persistence suite:300 tests; executable V3 migration source has100% coverage. No released generation, prior migration implementation, shared release branch, benchmark or CI workflow is modified by this integration.
2026-09-06 21:33:35 +08:00
Tianyi Cui b47bfa711b test(session): pin structural v3 chain expectations 2026-09-06 21:12:50 +08:00
Tianyi Cui cfc10f7e1f test(session): keep migration assertions type-aware
Treat Vitest asymmetric matchers as unknown expected values and spell void assertion callbacks explicitly. This preserves the migration identity and refusal checks under the full type-aware lint configuration without relaxing validation or suppressing a rule.
2026-09-06 21:00:21 +08:00
Tianyi Cui 49e1050092 fix(session-format): narrow repair identities for native validation
Require a string call identity before constructing the private repair-validation prefix; do not stringify arbitrary durable JSON. Remove a redundant narrowed version comparison and use explicit void test callbacks.

Verified full type-aware package Oxlint (90 rules), all 71 focused cases with 100% exact V3 statements/branches/functions/lines, and host TypeScript build. No coverage ignores or broad lint exceptions.
2026-09-06 20:58:57 +08:00
Tianyi Cui 9e5f8a1e44 test(session-format): cover V3 durable admission branches
Exercise malformed source envelopes, feedback metadata, file attachments, native system ownership, protected head replacement, started versus repaired tools, dangling coordinate mappings and successful compaction ranges. The focused package suite now has 71 cases with exact V3 source coverage at 100% statements, branches, functions and lines.

Remove unreachable rechecks after owned payload validation instead of adding artificial tests or coverage ignores. Unseeded inherited markers already fail at arrival; message arrays and IDs are validated before observation. Bind the unchanged frozen decoder run callback without an unreachable wrapper.
2026-09-06 20:58:57 +08:00
Tianyi Cui 5c48f8101a test(session): retain positive predecessor migration lifecycle coverage
Construct supported historical inputs with the first surface inside step/start instead of weakening publication, singleflight, packed-row, and read-handoff assertions into refusal cases. Account for the inserted empty system head and shifted event sequence when V2 rows become V3; preserve exact native V3 expectations and original source bytes.
2026-09-06 20:52:37 +08:00
Tianyi Cui ab7f19cd3d test(session): refuse frozen pre-step fixture at current open
Current V3 promotion must preserve chronology, so historical user surfaces before the first step are unsupported. Assert refusal for read and write opens, preserve source bytes and identity, and forbid intermediate or current publication. Frozen codec validation remains in its existing owner tests; V2 no-downgrade coverage lives in v2-system-migration.spec.ts.
2026-09-06 20:45:00 +08:00
Tianyi Cui 3d898cb954 test(session): cover V2 system prompt persistence migration 2026-09-06 20:45:00 +08:00
Tianyi Cui 6f5fc05ea3 fix(session-format): admit audited V2 corpus payloads
V2 assistant attempts have no V0 semantic dispatch case: validate their exact V2 members and positive step coordinates locally while preserving the embedded stream. Explicitly classify agent-message relay attribution and file attachment metadata; both preserve foreign identities and non-Session byte counts, and reject unknown members.

Native V3 request headers retain current extension fields while rejecting retired header.system, independently of the closed V2 migration inventory. Add direct and multihop attempt regressions, exact source/file negative cases, and native header roundtrip coverage. All 41 focused tests and host tsc pass; scoped lint and bilingual pairing pass.
2026-09-06 20:45:00 +08:00
Tianyi Cui e04cfc4c87 feat(session-format): migrate V2 prompts into V3 system surfaces
Stream an empty protected head after the first step and replace it before changed or cleared request headers, without moving source events. Remap only audited local sequence references, derive inherited cuts after upstream cardinality changes, and preserve generation-qualified captures and message identities.

Refuse pre-step surfaces and out-of-step prompt changes rather than invent lifecycle events. Reject unclassified migration payloads and detect deterministic generated-ID collisions in either source order. Native V3 accepts empty and in-history system messages, protects only the head, and reuses frozen relationships through a private nonescaping projection; historical repair IDs remain opaque.

Keep released codecs untouched and distinguish migration admission from native extension admission. V3 structural payload preflight cannot disappear behind recoverable row corruption. Validation: 35 focused tests, host TypeScript build, focused Oxlint, documentation quick gates and paired README recording. Parent owns installed-core/catalog/persistence integration and lockfile propagation.
2026-09-06 20:45:00 +08:00
Tianyi Cui ee956c720d refactor(session): represent the system prompt as surface node zero
Consolidate the representation-change PR and its rebase reconciliations into one baseline. Preserve the exact tree and keep the in-history feature in the dependent PR. Follow-up fixes remain separate.
2026-09-06 20:42:43 +08:00
Tianyi Cui 368ec64489 Merge corrected Session V3 base into PTC vocabulary 2026-09-06 18:29:14 +08:00
Tianyi Cui e53df8b64a fix(session): guard V3 delivery activation and smoke writer generation 2026-09-06 17:39:15 +08:00
Tianyi Cui 5895f46b34 fix(session): preserve unsupported V3 events during tail recovery 2026-09-06 17:29:21 +08:00
Tianyi Cui eaae3502ae test(session): synchronize shared migration cancellation waiters 2026-09-06 16:17:42 +08:00
Tianyi Cui d8d86b7eb8 Cover PTC dispatches through explicit TypeScript SDK profile patch 2026-09-06 15:37:15 +08:00
Tianyi Cui eb4259e405 test(session): distinguish frozen PTC tags from current inventory 2026-09-06 15:31:46 +08:00
Tianyi Cui bad4254d71 Rename durable PTC vocabulary with identity-preserving v2 migration 2026-09-06 14:53:16 +08:00
Tianyi Cui d56f2d2cb4 test(session): use explicit void callbacks in migration assertions 2026-09-06 14:11:49 +08:00
Tianyi Cui 705f84f952 docs(session): document format upgrades and advance current V3 fixtures 2026-09-06 14:01:01 +08:00
Tianyi Cui f7a6221158 feat(session): add identity V2-to-V3 migration and writer skeleton 2026-09-06 13:52:21 +08:00
Tianyi Cui 165cc31eb8 perf(llm,host): read embedded Assistant streams per compact record
Session format v2 embeds each attempt's compact stream in
assistant/message and assistant/attempt, but Host and client consumers
still expanded it into per-member TimedStreamChunk arrays and did
per-member work; expandAssistantStream materializes the full array before
find/toReversed/break can answer. Session Stats (the projection phase of
every Session open), the token meter's usage and provider-assembly folds,
the subagent output fold, and the Session Controller image lookup still
paid O(members) allocation and time per settlement.

The Chat and Trajectory definitions were already settled from
message.content on master; the remaining per-member folds stay.

dsh-llm now exports record-level readers (first token, visible content,
visible text, last raw chunk of a type, raw chunks of a type, joined
text, run-aware assembly, per-run first-token/first-visible times) that
scan the compact records once with early exit. Session Stats reads
assistantStreamFirstTokenTime, the token meter reads
lastAssistantStreamChunk(stream, 'usage') and assembles through
assembleAssistantStream, the subagent output fold appends
joinAssistantStreamText, and the Session Controller scans
assistantStreamChunks(stream, 'block-end').

expandAssistantStream is deliberately not memoized: retaining expansions
costs roughly ten times the compact stream for the Session's lifetime.
It remains the validating path at durable boundaries.

Synthetic 200-turn v0 migration benchmark, median of five: first-open
projection 28.0 ms -> 5.4 ms, first-open total 76.9 -> 50.0 ms, peak RSS
137.2 -> 94.9 MB; reopen projection 17.8 -> 5.6 ms; all phase budgets and
the 128 MB heap constraint keep passing.
2026-09-06 12:41:25 +08:00
imccyu 591d12ce86 docs(session): document preparation-first restoration 2026-09-06 02:10:28 +08:00
imccyu 9b78f99dec perf(session): transfer frozen persistence reads into restore 2026-09-06 02:10:23 +08:00
imccyu a3e5edbdbf perf(session-persistence): prepare historical reads before publication 2026-09-06 02:08:48 +08:00
imccyu 98d2baf2e5 docs(session): document streaming format migration 2026-09-06 02:06:22 +08:00
imccyu ec2f63dbdb perf(session-persistence): stream migration publication and verification 2026-09-06 02:05:40 +08:00
imccyu 46196d6f95 perf(session-format): stream released v0-to-v2 migrations 2026-09-06 02:05:27 +08:00
imccyu a84a8da9e1 refactor(session-format): introduce streaming migration stages 2026-09-06 02:05:09 +08:00
imccyu 47a6fa0f28 test(perf): centralize lifecycle benchmarks 2026-09-06 01:01:03 +08:00
Tianyi Cui c53b0bb5ed test(perf): double the CI cost in the migration budget
The migrating first open measured 1,963 ms on the CI runner under the
128 MB heap limit; 4,000 ms keeps a 2x margin while the repeated-snapshot
implementation still fails by heap exhaustion and would need ~10 s.
2026-09-06 01:01:03 +08:00
Tianyi Cui 9048e8ee11 test(perf): size the migration budget for the CI runner
The migrating first open of the synthetic log costs about 1 s on the
reference machine under the 128 MB heap limit and about twice that on the
CI runner, so 3,000 ms keeps headroom while staying far below the ~5 s
(~10 s on CI) of the repeated-snapshot implementation.
2026-09-06 01:01:03 +08:00
Tianyi Cui 2395f112d8 test(perf): gate opening large Sessions with synthetic CI benchmarks
Add a benchmark lane (`vitest.bench.config.ts`, `pnpm run test:bench`,
gate mode `ci-bench`) and a required `node 24 / benchmarks` CI job that
runs it alone. Benchmarks synthesize their input in-process from fixed
parameters and fail on documented budgets:

- `open-generation.bench.ts`: a 200-turn released-v0 log with 500 text
  and 125 reasoning deltas per reply (127,400 events, ~2.8 MB) encoded
  through the frozen v0 codec; the migrating first `open()` must finish
  within 2,000 ms in a child process capped at 128 MB of old space, and a
  fresh process must open the published current generation within 500 ms.
- `conversation-fold.bench.client.ts`: 200 replies whose compact streams
  hold 2,000 text + 500 reasoning deltas each, folded through every Chat
  Definition by the real assembler; the fold must finish within 150 ms
  and stay within 3x the fold of the same window with 100 deltas per
  reply.

On this commit both gates fail: the migration exhausts the 128 MB heap
(4.8 s and 696 MB peak RSS without the cap; the pre-stack decode of the
same bytes took 34 ms and 168 MB) and the fold scales 11x with the delta
count. The stacked fixes bring both paths to O(records).
2026-09-06 01:01:03 +08:00
Tianyi Cui 9ffe85a512 OTel uploads explicit feedback for all users by default 2026-09-05 23:21:23 +08:00
ChinesezjcandTianyi Cui 02a029e679 Feedback is recorded in the session log and delivered through DeepSeek
Co-authored-by: Tianyi Cui <[email protected]>
2026-09-05 23:21:22 +08:00
Yichen Jiang ed67c22496 release(dsh): 0.1.3-alpha.1 2026-09-04 15:38:55 +08:00
Turtle c58097a826 feat(session-persistence-jsonl): cross-process write-ownership lease
Write handles now hold a durable lease (session.lock.json beside the
log): a random owner token, diagnostic pid, and an expiry. Acquisition
wins by exclusive create; a second process's create or write open rejects
while the record is unrenewed for less than leaseTtlMs (default 5 min),
and takes over after that — a crashed holder is waited out, never
reclaimed by pid. The holder renews every leaseRenewIntervalMs (default
4 min); a renewal that finds a foreign, vanished, or expired record — or
fails outright — marks the lease lost permanently, so every later
append/flush rejects with SessionOwnershipLostError while reads continue.
Close releases the record; read handles never touch it. Takeover of an
expired record is eventually exclusive: a replaced holder stops within
one renewal interval.

Refs #3245
2026-09-04 14:12:14 +08:00
Tianyi Cui 8cb1d6a7eb Merge commit '85c23a2df3' into worktree/session-format-06-v2-snapshot-rollout 2026-09-03 22:37:26 +08:00
Tianyi Cui 86241a442d fix(session): keep independent version fields off the format generation
`SESSION_FORMAT_VERSION` is the only value that names the Session format
generation. Two other version fields had moved to 2 alongside it:

- The `dsh_session_log` request extension's outer schema `version` returns
  to 1 and its Session header projection keeps `seedLength`, derived from
  the logical inherited cut, so the external wire is unchanged in this PR;
  `sessionFormatVersion` alone identifies the embedded generation.
- The `sessionStats` projection unit's `stateVersion` returns to 1: the
  projection cache binds every checkpoint to the format generation, so a
  generation change discards old rows without a unit version bump.

Test doubles and the standalone Client fixture that build a current
header now spell `SESSION_FORMAT_VERSION` instead of the literal.
2026-09-03 22:36:51 +08:00
Tianyi Cui 6c7603aacd Merge commit '51f9d5f6c1' into worktree/session-format-06-v2-snapshot-rollout 2026-09-03 22:24:50 +08:00
Tianyi Cui 9fb0678f0b Merge commit '22472e4092' into worktree/session-format-05-v1-v2-chunk-migration 2026-09-03 22:24:36 +08:00
Tianyi Cui 6c419795f6 docs(session): name the frozen v0/v1 vocabulary in the v0-to-v1 README
The README file table listed four of the six source modules and omitted
the two largest, so the package read as an oversized identity conversion.
The summary now states that most of the source is the frozen released
v0/v1 event vocabulary, why the edge refuses a malformed historical log
before the installed current restorer runs, and that later edges reuse
those shapes; the table gains `payload-validation.ts` and
`relationships.ts` rows.
2026-09-03 22:23:53 +08:00
Tianyi Cui 17fb719138 Merge commit 'cf8f75faaf' into worktree/session-format-06-v2-snapshot-rollout 2026-09-03 22:10:02 +08:00
Tianyi Cui d58964a010 Merge commit 'be5db297c7' into worktree/session-format-05-v1-v2-chunk-migration
# Conflicts:
#	.agents/notes/implemented/architecture/2026-08-31-live-assistant-stream-frames.i18n.yaml
#	.agents/notes/implemented/architecture/2026-08-31-live-assistant-stream-frames.md
#	.agents/notes/implemented/architecture/2026-08-31-live-assistant-stream-frames.zh.md
#	docs/event-producer-consumer.i18n.yaml
#	docs/event-producer-consumer.md
#	docs/event-producer-consumer.zh.md
#	packages/api/session-controller/README.i18n.yaml
#	packages/api/session-controller/README.md
#	packages/api/session-controller/README.zh.md
#	packages/api/session-controller/src/assistant-stream.ts
#	packages/api/session-controller/src/client/sessions/assistant-stream.ts
#	packages/api/session-controller/src/types.ts
#	packages/api/session-controller/tests/assistant-stream.client.spec.ts
#	packages/api/session-controller/tests/session-history-journal.host.spec.ts
#	packages/api/session-controller/tests/sessions-service.client.spec.ts
#	packages/api/session-controller/tests/transport.client.spec.ts
#	packages/core/agent/README.i18n.yaml
#	packages/core/agent/README.md
#	packages/core/agent/README.zh.md
#	packages/extensions/tool-cordis/src/api-catalog.ts
#	scripts/package-dependency-policy.ts
2026-09-03 22:09:31 +08:00
Tianyi Cui c97df989ff refactor(session): share the canonical log basename and JSON snapshots
`session[.vN].jsonl` was assembled independently by the JSONL provider
(twice), the session-log export archive, and the recorded-session fixture
helpers. `dsh-session-format` now owns `sessionFormatLogFilename` and
`parseSessionFormatLogFilename`; the three consumers append only their
compression suffix, and the migration note names the owner.

`snapshotSessionFormatJson` re-implemented the lossless JSON walk that
`dsh-util-values` already publishes as `snapshotJsonValue` + `deepFreeze`,
and the frozen v0 relationship validator carried a third structural JSON
comparison next to `deepEqualJson`. Both now delegate; the format package
reports one `is not lossless JSON` diagnostic per labelled subject instead
of ten member-specific ones, and enumerable accessors whose values survive
a JSON round trip are accepted like `JSON.stringify` accepts them.
2026-09-03 21:48:09 +08:00
Tianyi Cui 71089f8df7 Merge commit '8f7250f904' into worktree/session-format-06-v2-snapshot-rollout
# Conflicts:
#	apps/cli/tests/profiles/headless/tests/expected/subagent-settlement/child.expected.jsonl
2026-09-03 19:41:58 +08:00
Tianyi Cui d5044d5675 Merge commit 'ee4bad5cde' into worktree/session-format-05-v1-v2-chunk-migration 2026-09-03 19:25:29 +08:00