The loop appends step/start before system/message and the entered user
messages. Capturing nodes at step/start therefore omits inputs already
included in the provider's successful usage, then adds those inputs back
as a positive surface delta. Prompt replacement can also incorrectly add
or subtract the difference from the prior prompt on a completed call.
Snapshot the current priced surface immediately before assistant/message
commits. Keep provider output separate from the durable assistant node so
listener rewrites retain their signed delta. The invariant is zero delta
immediately after an unchanged successful output: provider usage already
includes every admitted prompt input. Later appends/replacements still
produce signed deltas, and low or absent usage keeps heuristic fallback.
Delete stepStart.nodes rather than adding prompt-specific corrections or
another request snapshot: the existing transactional surface fold already
contains the successful request inputs, including replacements made during
same-step retry recovery. Keep turn/step state and all overlap, mismatch,
and late-assistant lifecycle validation. Retry attempts are log-only and
request middleware changes configuration; injected messages remain queued
until admission. No loop, event format, projection, or retry policy changes.
Exercise the real loop with reported usage and initial, growing, shrinking,
and empty prompts; same-step failed attempt plus retry prompt replacement;
request middleware; eager observation and fresh seeded replay. The two
regressions fail before the fix with spurious deltas of +48 and +18 tokens.
Retain existing durable-output rewrite, route repricing, missing/low usage,
transactional failure, and lifecycle tests. Update README EN/ZH and the
existing system-prompt surface-node Agent Note, including pairing records.
Validation (dedicated worktree, no full unit suite):
- pnpm exec vitest run packages/llm/token-meter/tests packages/compaction/compaction-basic/tests/compaction-loop-repro.spec.ts --coverage --coverage.include='packages/llm/token-meter/src/index.ts'
118 passed; exact changed runtime file 100% statements/branches/functions/lines.
- pnpm exec vitest run packages/core/agent-loop/tests/request-reconstruction.spec.ts packages/compaction/compaction-basic/tests/compaction-basic.spec.ts packages/compaction/compaction-basic/tests/loader-composition.spec.ts
118 passed, including retry reconstruction and real Loader composition.
- pnpm run doc-sync: 33 gates passed.
- pnpm run test:docs: 15 gates passed.
- pnpm run lint: passed, 0 warnings/errors.
- git diff --cached --check: passed.
Baseline normalize.ts comment conflict marker is intentionally untouched.
Consolidate the representation-change PR and its rebase reconciliations into one baseline. Preserve the exact tree and keep the in-history feature in the dependent PR. Follow-up fixes remain separate.
Session format v2 embeds each attempt's compact stream in
assistant/message and assistant/attempt, but Host and client consumers
still expanded it into per-member TimedStreamChunk arrays and did
per-member work; expandAssistantStream materializes the full array before
find/toReversed/break can answer. Session Stats (the projection phase of
every Session open), the token meter's usage and provider-assembly folds,
the subagent output fold, and the Session Controller image lookup still
paid O(members) allocation and time per settlement.
The Chat and Trajectory definitions were already settled from
message.content on master; the remaining per-member folds stay.
dsh-llm now exports record-level readers (first token, visible content,
visible text, last raw chunk of a type, raw chunks of a type, joined
text, run-aware assembly, per-run first-token/first-visible times) that
scan the compact records once with early exit. Session Stats reads
assistantStreamFirstTokenTime, the token meter reads
lastAssistantStreamChunk(stream, 'usage') and assembles through
assembleAssistantStream, the subagent output fold appends
joinAssistantStreamText, and the Session Controller scans
assistantStreamChunks(stream, 'block-end').
expandAssistantStream is deliberately not memoized: retaining expansions
costs roughly ten times the compact stream for the Session's lifetime.
It remains the validating path at durable boundaries.
Synthetic 200-turn v0 migration benchmark, median of five: first-open
projection 28.0 ms -> 5.4 ms, first-open total 76.9 -> 50.0 ms, peak RSS
137.2 -> 94.9 MB; reopen projection 17.8 -> 5.6 ms; all phase budgets and
the 128 MB heap constraint keep passing.
- permission-presets: register the permissions unit synchronously before the
existing-session sweep, so a remount reads folded knob state instead of
treating every session as fresh; add a regression test for that path
- sandbox-policy/terminal-bash: mount the projection registry in the 5 pwsh
terminal tests, the sdk-minimal bundle, and the e2b fixture composition;
declare the new package dependency in both manifests
- plan-mode: restore .strict() on the plan unit state schema and drop the
deleted foldPlanMode from the bilingual READMEs
- session-title/session-projection: sync bilingual READMEs to the mandatory
projection seam and re-record translation pairing
- docs: turnBoundary reader contract, subagent schema comment, token-meter
import comment, sandbox-policy module docstring
foldSurfaceTokens rebuilt the priced surface on every surface event: an
append allocated [...nodes, node] and a replacement copied the whole
array before splicing, charging every well-formed event O(surface) for
an atomicity property only malformed events need. Benchmarks put the
copy at ~99.9% of an append's cost (100µs at a 50k-node surface vs
0.1µs for pricing) with O(S²) accumulation over a session, inside the
synchronous session/event publication path.
Split the fold into the session core's planSurfaceEvent/applySurfacePlan
shape: planSurfaceTokens performs every fallible step against the
read-only surface, commitSurfaceTokens applies the plan in place and is
infallible by construction. _foldEvent plans first, runs the remaining
fallible anchor validation, and only then commits, so retry identity is
preserved by ordering instead of by allocation. Appends drop to
amortized O(1) (100.3µs -> 1.9µs at 50k nodes); replacements keep their
O(surface) findIndex but stop paying the extra full copy (21µs -> 4.2µs).
A new regression test pins the one hazard this introduces: an event
whose surface plan is valid but whose later anchor validation throws
must leave the priced surface and running total uncommitted across
repeated failures.
Machine-produced by `pnpm run rescope-vendor --apply` plus the regeneration it
prints: `pnpm install` for the lockfile, `pnpm run gen-third-party-notices`,
`verify-translation-pairing --write` for the touched bilingual pairs,
`gen-doc-graphs`, and one typert snapshot whose ids embed character offsets.
`pnpm run rescope-vendor --check` verifies the result.
Renames nine vendored packages (cordis, cosmokit, schemastery and the six
@cordisjs plugins) and every reference that resolves them: manifest names and
dependency keys, module specifiers including declare-module merges, cordis.yml
plugin names, tsconfig paths, every Markdown fence, and `docs/` prose.
Directory names, upstream versions, and dependency ranges are unchanged, so
vendor/README.md still reads as an upstream snapshot; its manifest table gains
an upstream-name column so THIRD_PARTY_NOTICES keeps MIT attribution pointed
at each fork's origin.
The tutorial tier follows the rename end to end: its yaml fences named plugins
the Loader can no longer resolve, its `ts ignore-check` fences disagreed with
the compiled fences beside them, and its prose quoted both. The contracts that
told readers to keep upstream names — the root convention and the vendoring
cookbook's tree comment and manifest invariant — now say to rescope instead.
Two rules read `@deepseek-ai/` as "another workspace plugin": the client bundle
purity gate now names the vendored libraries a browser bundle inlines, and the
files where a bare `cordis` is an agent-preset id keep that product data.
`contextBreakdown.messageTokens` and `measure().surfaceTokens` answer the same
question in the same heuristic vocabulary, and the panel's composition rows are
only honest while they agree. Each owner carried its own copy of the positional
fold — same pricing, same `{seq, tokens}` node list, same replace-range lookup
and guard, differing only in mutable versus immutable application — so an edit
to either one would have moved the panel away from `measure()` with both sides
still green. The duplication gate caught the shared 62 tokens.
`src/surface-fold.ts` now owns `foldSurfaceTokens`: total, allocation-fresh,
returning the event's price, the next surface, and the signed total delta. The
service assigns that result where it used to prepare a commit closure, which
keeps its validate-before-mutate replay transaction intact — the fold throws
before any state is touched, so a malformed event still fails identically on
every retry. `_prepareSurfaceMutation` and `_estimateSurfaceEvent` go away with
it, and the projection's apply drops to one call.
Covers the identity with a session that appends and then compacts, asserting
the projection figure equals the service surface at each boundary; the test
fails when either side of the fold is perturbed.
Assistant footers and the stats line gain TTFT/tok-per-second readings
folded from step timings; context occupancy moves off the stats line onto
a composer ring whose panel shows a heuristic system/tools/messages
breakdown from the new token-meter contextBreakdown session projection.
Conflict resolution notes:
- StatsLine: master redesigned the row into pipe-separated groups with LLM and
tool wall times. Kept that design and swapped only the token accounting
source, so counts and durations stay window-scoped while billing and context
occupancy read the durable projections.
- Generated artifacts (cordis catalog, module graph, event producer/consumer,
i18n pairing hashes) and web snapshots took master's side; they are
regenerated and re-recorded after this merge.
- Web e2e goldens and details-panel/timeline assertions took master's side:
that evolution is unrelated to this branch.
- ui-conversation package.json: kept master's devDependency ordering, re-adding
only the token-meter entry this branch needs.
Replace the transient `session/model-request` mux frame with ordinary durable
session state. Occupancy now rides two last-wins projection fields instead of a
non-replayable frame that needed removal tombstones and cross-stream fencing.
The frame was the only non-replayable class on the mux stream. Because host and
mux are independent SSE streams with no cross-stream order, a request emitted
before a removal could arrive after `host/session-removed`, and a legitimate
request for a new lifecycle reusing the same id could be fenced by a late
removal. Fixing that needed a lifecycle generation on every frame; the frame
itself was the problem.
Removed: the `session/model-request` frame and schema, the `agent/model-request`
core event, the ApiProxy measurement point, the client-side telemetry map and
removal tombstone, and the synthetic `cancelled` open error used to signal
reconnect through the error channel.
Added: `request/context`, a log-only session event recording the
registration-bound capacity of the route a request resolved to, appended beside
`request/header` from the lookup that already prepared the call and skipped when
the route is unchanged. Capacity stays out of `EpochHeader` because it is
adapter metadata about a route, not an input the request was built from, so it
must not join request reconstruction or header equality.
The `contextPressure` projection pairs the newest provider-reported prompt size
with the newest recorded capacity. The two are deliberately not one atomic
request observation: switching models can pair a fresh capacity with the prior
route's pressure until the next request reports usage. The figure is a
user-facing reference, and this matches how the TUI status line has always
computed occupancy.