Give durable user context its producer-owned source kind instead of a generic plugin wrapper. The V3-to-V4 conversion preserves the frozen rename and collision rules, all own JSON metadata, event coordinates, and historical request-only kinds. Native catalog and JSONL admission retain source validation so unknown attribution can survive without its producer while malformed owned sources are refused.
Use an explicit user-only RequestUserInput for auxiliary LLM prompts that do not own durable message identity. Auto Review and compaction remove their request-only source registrations while retaining prompt content and freezing; provider serializers and projections keep the same wire data, assistant replay isolation, image ordering, and tool correlation. Session writes, Agent delivery, and recorded title requests continue to require identified durable messages.
Bind the saved compatibility policy only to user sources and qualify tmux location attribution without weakening fixed system, model, or tool semantics. Rebuild the current inventory and unpublished V4 acknowledgement on the actual tool-role parent, preserving its rationale and all accepted historical records. The catalog format version records tooling metadata; Session remains V4. Developer-role policy and activation belong to their own feature.
The latest-master Messages adapter, serialization, request-input typing, EpochHeader reservation, and attribution checks pass 103 tests across five files. The retained V3 Messages input-history snapshot passes through the source profile; Host/Client typechecks and all 41 documentation gates pass. The generated comparison records four changed roots and 447 unchanged type fingerprints. Latest-master ARIA changes are retained, with producer source labels applied independently; new Messages fixtures retain their producer names as declared source kinds and use the fixed system-message helper; recorded Session generations are unchanged by this regrouping.
Co-authored-by: Tianyi Cui <[email protected]>
Represent tool results as tool-role messages and migrate released wrappers
without losing outer own JSON fields, call identity, error state, or writer
field order. Refuse nested or wrapper-only information that has no lossless
current representation before successor publication.
Validate current event relationships directly instead of projecting V4 data
back into V3. Keep lifecycle, PTC, title/command, compaction, system-field and
retired-syntax checks on catalog restoration and ordinary JSONL opens. The
Worker JSON equality helper must inspect own keys so it cannot admit PTC
arguments that Node rejects.
Keep the source role change, required native admission and consumer fixture
updates in one complete review layer. Explain shared-type changes in the
persistence record and the native reader rationale in its existing note.
Use checked branded fixture identities and discriminant narrowing instead
of unknown double casts; fixture values and assertions remain intact.
This recomposes the original tool-role work by _Kerman with the independently
validated migration and admission repairs by Tianyi Cui. Frozen generations
and snapshot input version selection are retained.
Co-authored-by: Tianyi Cui <[email protected]>
Drop the image/offload event and every session change with it. An adapter's
IMAGE_OFFLOAD_REQUIRED failure now makes dsh-llm-retry replace the surface
nodes carrying the named oldest images with copies whose blocks are marked
offloaded, through the same surfaceOp: replace compaction uses, then retry the
step. ImageBlock.offloaded becomes a durable field the routes project and the
token meter prices; the session core, its event map, derivation, and
validation return to master.
The parent representation fix now correctly invalidates header-based caches with scalar state version3. The dependent in-history projection has a different compact live-surface schema and must not reuse that version. Advance it to4 and prove that a lower-layer version3 cache is discarded and refolded, alongside the existing version2 regression. No historical Session migration or compatibility mechanism is added.
Addresses ds-review-bot cache-version thread3921300994 across the stack. Validation:62 focused projection, adapter and admission tests passed; context.ts and breakdown-projection.ts each have100% exact-source coverage; paired README record updated.
The breakdown refactor correctly stopped consuming scalar shadow claims, but its old negative test was the only coverage of the pressure fold rejection path. Exercise the remaining real pressure consumer with mismatched start and end claims, preserving the runtime invariant rather than deleting or ignoring its guard.
Validation: Linux and Windows CI reported the two uncovered locations in surface-projection.ts. All118 token-meter tests pass and the exact pressure fold file now has100% statement, branch, function and line coverage.
The session projection API can return undefined when a unit is not registered. Assert the registered test fixture produced state before inspecting compact entries, rather than assuming a value that the current master type does not guarantee. This preserves the retained-state regression and adds no production fallback.
Validation: the aggregate build identified three TS2532 errors in this test; all context-breakdown projection cases pass after narrowing. Full build is rerun separately.
Scalar shadow prices cannot restore an earlier effective prompt when compaction removes the newest system node: subtracting that price from messages can permanently make the published bucket negative. Reuse the positional measurement planner over compact current surface entries and choose the last nonempty system by surface order, not event sequence or provenance citations.
Explicitly trade the scalar checkpoint for O(current retained surface) state, dropping shadowed entries and message bodies. Version 3 invalidates old scalar checkpoints; pure transitions preserve wire identity for same-price rewrites. Cover newest/middle removal, reversed sequence ranges, extra sources, empty fallback nodes, replay and late registration, cache version rejection, and route-independent heuristic totals. Document the bounded-state exception and rejected prompt-only ancestry approach in both languages.
Consolidate the existing in-history capability, loop, UI and artifacts into one baseline. Preserve the reviewed tree so each independently verified correction has a subsequent rationale-rich commit.
Address ds-review-bot thread 3921300994 (PRRT_kwDOS3Pfcs6eyLsu). contextBreakdown now folds system/message instead of request/header.system, so version-2 checkpoints have different semantics even though their numeric fields still pass the current schema. Bump the lower projection to stateVersion 3 so cache-only reads omit old values and restore replays the full log; derived caches are not covered by the Session-log persistence compatibility waiver. No migration or fallback is added.
Regression seeds a schema-valid v2 row at the current watermark with stale system/message prices. Before the bump, cache views returned it, restoreFloor selected seq 2 rather than 0, and restore retained system=0/message=17 instead of system=8/message=9. After the bump, the 13-test owning suite passes and exact src/breakdown-projection.ts coverage is 100% statements, branches, functions, and lines; refreshed rows equal a fresh fold at version 3. Existing token-meter README prose documents the current system/message fold without a version literal, so it needs no edit.
Propagation requirement: the upper layer already uses version 3 for a different compact cache representation. Advance that layer to version 4 when propagating this fix to avoid assigning one version to two schemas.
The loop appends step/start before system/message and the entered user
messages. Capturing nodes at step/start therefore omits inputs already
included in the provider's successful usage, then adds those inputs back
as a positive surface delta. Prompt replacement can also incorrectly add
or subtract the difference from the prior prompt on a completed call.
Snapshot the current priced surface immediately before assistant/message
commits. Keep provider output separate from the durable assistant node so
listener rewrites retain their signed delta. The invariant is zero delta
immediately after an unchanged successful output: provider usage already
includes every admitted prompt input. Later appends/replacements still
produce signed deltas, and low or absent usage keeps heuristic fallback.
Delete stepStart.nodes rather than adding prompt-specific corrections or
another request snapshot: the existing transactional surface fold already
contains the successful request inputs, including replacements made during
same-step retry recovery. Keep turn/step state and all overlap, mismatch,
and late-assistant lifecycle validation. Retry attempts are log-only and
request middleware changes configuration; injected messages remain queued
until admission. No loop, event format, projection, or retry policy changes.
Exercise the real loop with reported usage and initial, growing, shrinking,
and empty prompts; same-step failed attempt plus retry prompt replacement;
request middleware; eager observation and fresh seeded replay. The two
regressions fail before the fix with spurious deltas of +48 and +18 tokens.
Retain existing durable-output rewrite, route repricing, missing/low usage,
transactional failure, and lifecycle tests. Update README EN/ZH and the
existing system-prompt surface-node Agent Note, including pairing records.
Validation (dedicated worktree, no full unit suite):
- pnpm exec vitest run packages/llm/token-meter/tests packages/compaction/compaction-basic/tests/compaction-loop-repro.spec.ts --coverage --coverage.include='packages/llm/token-meter/src/index.ts'
118 passed; exact changed runtime file 100% statements/branches/functions/lines.
- pnpm exec vitest run packages/core/agent-loop/tests/request-reconstruction.spec.ts packages/compaction/compaction-basic/tests/compaction-basic.spec.ts packages/compaction/compaction-basic/tests/loader-composition.spec.ts
118 passed, including retry reconstruction and real Loader composition.
- pnpm run doc-sync: 33 gates passed.
- pnpm run test:docs: 15 gates passed.
- pnpm run lint: passed, 0 warnings/errors.
- git diff --cached --check: passed.
Baseline normalize.ts comment conflict marker is intentionally untouched.
Consolidate the representation-change PR and its rebase reconciliations into one baseline. Preserve the exact tree and keep the in-history feature in the dependent PR. Follow-up fixes remain separate.
Session format v2 embeds each attempt's compact stream in
assistant/message and assistant/attempt, but Host and client consumers
still expanded it into per-member TimedStreamChunk arrays and did
per-member work; expandAssistantStream materializes the full array before
find/toReversed/break can answer. Session Stats (the projection phase of
every Session open), the token meter's usage and provider-assembly folds,
the subagent output fold, and the Session Controller image lookup still
paid O(members) allocation and time per settlement.
The Chat and Trajectory definitions were already settled from
message.content on master; the remaining per-member folds stay.
dsh-llm now exports record-level readers (first token, visible content,
visible text, last raw chunk of a type, raw chunks of a type, joined
text, run-aware assembly, per-run first-token/first-visible times) that
scan the compact records once with early exit. Session Stats reads
assistantStreamFirstTokenTime, the token meter reads
lastAssistantStreamChunk(stream, 'usage') and assembles through
assembleAssistantStream, the subagent output fold appends
joinAssistantStreamText, and the Session Controller scans
assistantStreamChunks(stream, 'block-end').
expandAssistantStream is deliberately not memoized: retaining expansions
costs roughly ten times the compact stream for the Session's lifetime.
It remains the validating path at durable boundaries.
Synthetic 200-turn v0 migration benchmark, median of five: first-open
projection 28.0 ms -> 5.4 ms, first-open total 76.9 -> 50.0 ms, peak RSS
137.2 -> 94.9 MB; reopen projection 17.8 -> 5.6 ms; all phase budgets and
the 128 MB heap constraint keep passing.
Record the image offload point as the core image/offload session event so the
derived surface, every adapter, and the token meter share one offloaded set.
The agent loop plans the advance from the route's declared request-image
budget before dispatch; an adapter whose exact accounting still overflows
fails with IMAGE_OFFLOAD_REQUIRED and the loop advances by the named count.
Tianyi Cui