diff --git a/packages/compaction/compaction-basic/README.i18n.yaml b/packages/compaction/compaction-basic/README.i18n.yaml index 9e67ab2df6..9e3c8f8fcf 100644 --- a/packages/compaction/compaction-basic/README.i18n.yaml +++ b/packages/compaction/compaction-basic/README.i18n.yaml @@ -27,8 +27,8 @@ en: 3446c54c062d7e34 zh: fb502c941e508ae0 /deepseek-ai-dsh-compaction-basic/use-this-package/what-happens-when-condensation-runs: - en: c872a8ef069596f0 - zh: b3a48e309ae830d4 + en: 399bce86e63b8e19 + zh: 727c023210085f5c /deepseek-ai-dsh-compaction-basic/use-this-package/switching-models: en: 48978abc46da1afa zh: 11931af05db7b41f @@ -51,8 +51,8 @@ en: 967531b71453eb88 zh: c7919b46e36763e6 /deepseek-ai-dsh-compaction-basic/understand-the-implementation/the-region-transaction: - en: f4330f6bff22d400 - zh: 46e16c51bb66a0f7 + en: 0492e288c55028cb + zh: 88a85c06c0cc78d2 /deepseek-ai-dsh-compaction-basic/understand-the-implementation/config-resolution: en: 313e019b1a9f8aa8 zh: 3c75d11b02adb693 @@ -90,8 +90,8 @@ en: 39a362ee4890dbdb zh: 3b98699120b1548e /deepseek-ai-dsh-compaction-basic/model-experience/auxiliary-summarizer-request/token-effect: - en: 5e0679cb7cb4291a - zh: c9df8ebc526b05d9 + en: ae2e2b748de72d77 + zh: ba11a49cbc263dca /deepseek-ai-dsh-compaction-basic/model-experience/auxiliary-summarizer-request/kv-cache-effect: en: d31d3e3d46542c91 zh: 5a42f873993e0c79 diff --git a/packages/compaction/compaction-basic/README.md b/packages/compaction/compaction-basic/README.md index 3956c8a57d..6a4485e503 100644 --- a/packages/compaction/compaction-basic/README.md +++ b/packages/compaction/compaction-basic/README.md @@ -83,6 +83,8 @@ Misconfiguration fails fast: unknown settings, duplicate per-model overrides, in The oldest balanced span is replaced by one summary message and the recent tail stays verbatim; the conversation continues from the summary. The operation reports how many history items were condensed and the estimated tokens freed. If nothing can be condensed safely — for example the whole conversation is one indivisible unit — nothing changes and nothing is written to the session log. If no model is available to write the summary (no configured target and no routed request yet), condensation fails with a clear error telling you to configure the summarization provider and model or route one request. +If an automatically selected or `/compact` summary receives `CONTEXT_WINDOW_EXCEEDED`, condensation retries the first half of the selected complete history units, keeping each tool call with its results. Each further overflow halves the remaining units; a single unit that still overflows fails. A successful checkpoint replaces only the prefix actually summarized, leaving the remaining messages and images intact. Explicit `compactRegion(start, end)` calls keep their exact selection and propagate summary overflow. + ### Switching models Before a changed provider/model request is committed, automatic compaction measures the existing history against the destination model's capacity, output reservation, and pressure policy. When condensation is needed, it uses the previous routed model and its summary settings; an explicitly configured summary target still takes precedence. The destination request proceeds only after history falls below its threshold. Failure reports `MODEL_SWITCH_COMPACTION_FAILED` and leaves the previous request header intact; select a larger model or retry after correcting the reported cause. Completed pruning or condensation remains in the session even when a later attempt fails. The check runs when the next request is prepared, and `auto: false` disables it. @@ -130,7 +132,7 @@ A direct `ctx.llm.stream()` call uses the configured provider/model pair and cap ### The region transaction -Failed summary requests dispatch synchronous `compaction/summary-error` after checking cancellation and selection stability. A recovery listener must record a durable input change before requesting retry. The backend re-derives the selected messages and refreshes their token prices and shrink baseline. The image-offload plugin owns image selection; its recorded omissions remain effective if the summary later fails or is cancelled. +Failed summary requests dispatch synchronous `compaction/summary-error` after checking cancellation and selection stability. A recovery listener must record a durable input change before requesting retry. The backend re-derives the selected messages and refreshes their token prices and shrink baseline. The image-offload plugin owns image selection; its recorded omissions remain effective if the summary later fails or is cancelled. When no listener handles a canonical context overflow, self-selected ranges may shrink within the same transaction; only the successful prefix appears in the checkpoint's cited range. The transaction validates the surface span and the durable lock, appends `compaction/start`, summarizes through the hook, revalidates stability (whole-surface for automatic calls, selected-span for manual calls), rejects a summary that does not shrink its source, appends `compaction/summary` plus the replacement `user/message`, and makes exactly one `compaction/end` attempt. A live unmatched start is the durable lock: an unmatched marker before a newer `session/end-seed` is stale evidence from a prior lifecycle and does not block; one after that boundary reports `busy`. A failed close deliberately leaves a blocking orphan. Cancellation remains authoritative after cleanup and durability. @@ -236,7 +238,7 @@ Rules: #### Token effect -This is a separate model call: the replayed conversation prefix plus the fixed instruction as input, with `maxTokens`-capped output. Convergence retries can pay this cost more than once. +Each summary attempt is a separate model call: the replayed conversation prefix plus the fixed instruction as input, with `maxTokens`-capped output. Convergence retries and smaller-prefix retries can pay this cost more than once. #### KV Cache effect diff --git a/packages/compaction/compaction-basic/README.zh.md b/packages/compaction/compaction-basic/README.zh.md index 283a4d19eb..6cebca3e80 100644 --- a/packages/compaction/compaction-basic/README.zh.md +++ b/packages/compaction/compaction-basic/README.zh.md @@ -83,6 +83,8 @@ kind: "package-reference" 最旧的平衡范围会被替换为一条摘要消息,近期尾部保持逐字不变;对话从摘要继续。操作会报告压缩了多少历史项以及估算释放的 token 数。如果没有任何内容可以安全压缩——例如整个对话就是一个不可分单元——则不会有任何改变,也不会向会话日志写入任何内容。如果没有模型可以撰写摘要(既未配置目标,也还没有已路由请求),压缩会失败并给出清晰错误,提示你配置摘要提供方与模型,或先路由一次请求。 +自动选区或 `/compact` 的摘要收到 `CONTEXT_WINDOW_EXCEEDED` 时,压缩会重试所选完整历史单元的前一半,每个工具调用及其结果保持在一起。每次再次超限都会将剩余单元折半;单个单元仍超限时失败。成功检查点只替换实际完成摘要的前缀,其余消息和图片保持不变。显式 `compactRegion(start, end)` 调用保持精确选区,并向调用方报告摘要超限。 + ### 切换模型 在提供方或模型发生变化的请求写入日志前,自动压缩会按目标模型的容量、输出预留和压力策略测量现有历史。需要压缩时,使用原路由模型及其摘要设置;显式配置的摘要目标仍然优先。只有历史降到目标阈值以下,才会继续目标请求。失败会报告 `MODEL_SWITCH_COMPACTION_FAILED`,并保留原请求头;可选择更大窗口的模型,或修正报告的原因后重试。即使后续尝试失败,已经完成的裁剪或压缩仍保留在会话中。检查在下次请求准备时执行,`auto: false` 会禁用它。 @@ -130,7 +132,7 @@ kind: "package-reference" ### 区域事务 -摘要请求失败后,后端先检查取消和选区稳定性,再触发同步的 `compaction/summary-error`。恢复监听器必须先记录持久的输入变更,才能请求重试。后端重新派生所选消息,更新 token 价格及判断摘要是否缩短输入的基准。图片省略插件负责选图;后续摘要失败或取消后,已经记录的省略仍然有效。 +摘要请求失败后,后端先检查取消和选区稳定性,再触发同步的 `compaction/summary-error`。恢复监听器必须先记录持久的输入变更,才能请求重试。后端重新派生所选消息,更新 token 价格及判断摘要是否缩短输入的基准。图片省略插件负责选图;后续摘要失败或取消后,已经记录的省略仍然有效。没有监听器处理规范上下文超限时,自选范围可以在同一事务中缩小;检查点只引用成功摘要的前缀范围。 事务验证表层范围与持久锁,追加 `compaction/start`,通过钩子生成摘要,重新验证稳定性(自动调用要求整个表层、手动调用只要求所选范围),拒绝不缩小源内容的摘要,追加 `compaction/summary` 与替换 `user/message`,并恰好进行一次 `compaction/end` 尝试。活动的未匹配 start 是持久锁:位于较新 `session/end-seed` 之前的未匹配标记是先前生命周期留下的陈旧证据,不会阻塞;位于该边界之后的标记报告 `busy`。闭合失败会有意留下阻塞性的未匹配标记。完成清理与持久化后,取消仍具有最终决定权。 @@ -236,7 +238,7 @@ Rules: #### Token 影响 -这是一次独立模型调用:输入是已回放会话前缀加固定指令,输出受 `maxTokens` 限制。收敛重试可能多次支付这项成本。 +每次摘要尝试都是一次独立模型调用:输入是已回放会话前缀加固定指令,输出受 `maxTokens` 限制。收敛重试和缩小前缀后的重试可能多次支付这项成本。 #### KV Cache 影响 diff --git a/packages/compaction/compaction-basic/src/index.ts b/packages/compaction/compaction-basic/src/index.ts index 096fa72fe4..8b45fb878e 100644 --- a/packages/compaction/compaction-basic/src/index.ts +++ b/packages/compaction/compaction-basic/src/index.ts @@ -304,7 +304,7 @@ export class BasicCompactionEngine extends CompactionEngine { const measurement = this.ctx.tokenMeter.measure(agent.session) const range = selectCompactableRange(agent.session, measurement, 0) if (range === null) return null - return this.compactRegion(range.start, range.end, agent, signal) + return this.compactAutomaticRegion(range.start, range.end, agent, signal) } /* v8 ignore next -- closed-union exhaustiveness guard */ default: @@ -359,7 +359,7 @@ export class BasicCompactionEngine extends CompactionEngine { + `(${measurement.totalTokens} estimated tokens >= threshold ${spec.thresholdTokens})`, ) } - result = await this.compactRegion(range.start, range.end, agent, signal) + result = await this.compactAutomaticRegion(range.start, range.end, agent, signal) measurement = meter.measure(agent.session, header) if (measurement.totalTokens < spec.thresholdTokens) return result } @@ -370,8 +370,26 @@ export class BasicCompactionEngine extends CompactionEngine { ) } + /** Compact an automatically selected prefix, allowing smaller balanced prefixes on summary overflow. */ + private compactAutomaticRegion( + start: SessionSeq, + end: SessionSeq, + agent: Agent, + signal: AbortSignal, + ): Promise { + return compactSurfaceRegion( + this.regionDependencies(), + agent.session, + start, + end, + agent, + { owner: 'current-turn', stability: 'whole-surface', allowSmallerPrefix: true }, + signal, + ) + } + /** - * Compact one inclusive positional range from the agent-owned surface using + * Compact one exact inclusive positional range from the agent-owned surface using * the effective token meter for all retention and shrink pricing. * @param start - inclusive first surface-node seq. * @param end - inclusive last surface-node seq. @@ -430,6 +448,7 @@ export class BasicCompactionEngine extends CompactionEngine { { owner: null, stability: 'selected-span', + allowSmallerPrefix: true, ...sourceCommandId === undefined ? {} : { sourceCommandId }, flush: async () => { await this.ctx.sessions.flush(agent.session) diff --git a/packages/compaction/compaction-basic/src/region.ts b/packages/compaction/compaction-basic/src/region.ts index 7b3d77d299..eb4c7174c5 100644 --- a/packages/compaction/compaction-basic/src/region.ts +++ b/packages/compaction/compaction-basic/src/region.ts @@ -16,7 +16,7 @@ import { } from '@deepseek-ai/dsh-compaction' import type { CompactionResult } from '@deepseek-ai/dsh-compaction' import type { CommandId } from '@deepseek-ai/dsh-commands/brand' -import { createUserMessage, errorChain } from '@deepseek-ai/dsh-llm' +import { CONTEXT_WINDOW_EXCEEDED_CODE, createUserMessage, errorChain, LlmError } from '@deepseek-ai/dsh-llm' import type { Message, UserMessage } from '@deepseek-ai/dsh-llm' import type { TokenMeasurement, TokenMeter } from '@deepseek-ai/dsh-token-meter' import { SessionSeq, type Session, type SessionEvent } from '@deepseek-ai/dsh-session' @@ -61,6 +61,8 @@ interface CompactionTransactionOptions { readonly flush?: () => Promise /** Manual command that initiated this transaction, when present. */ readonly sourceCommandId?: CommandId + /** Allow a self-selected range to shrink after summary overflow; absent keeps the exact range. */ + readonly allowSmallerPrefix?: true } interface CompactionEntryState { @@ -227,6 +229,7 @@ export async function compactSurfaceRegion( compactionId, options.sourceCommandId, assertStable, + options.allowSmallerPrefix === true, signal, ) if (options.owner === null) signal?.throwIfAborted() @@ -391,6 +394,7 @@ async function summarizeCompaction( compactionId: CompactionResult['compactionId'], sourceCommandId: CommandId | undefined, assertStable: StabilityCheck, + allowSmallerPrefix: boolean, signal?: AbortSignal, ): Promise { let summaryResult: SummaryResult @@ -402,9 +406,16 @@ async function summarizeCompaction( } catch (error: unknown) { if (signal?.aborted === true) throw error assertStable(dependencies, agent.session, prepared) - if (!dependencies.recover(error, agent, prepared.shadowedSeqs, signal)) throw error - prepared = prepareCompaction(dependencies, agent.session, - validateSurfaceRegion(agent.session, prepared.start, prepared.end)) + let selection: SurfaceSelection + if (dependencies.recover(error, agent, prepared.shadowedSeqs, signal)) { + selection = validateSurfaceRegion(agent.session, prepared.start, prepared.end) + } else { + if (!allowSmallerPrefix || !(error instanceof LlmError) || error.code !== CONTEXT_WINDOW_EXCEEDED_CODE) throw error + const smaller = smallerBalancedPrefix(agent.session, prepared) + if (smaller === null) throw error + selection = smaller + } + prepared = prepareCompaction(dependencies, agent.session, selection) } } const checkpointMessage = createUserMessage({ @@ -427,6 +438,16 @@ async function summarizeCompaction( } } +/** Halve complete tool-balanced units; one remaining unit cannot be shortened safely. */ +function smallerBalancedPrefix(session: Session, selection: SurfaceSelection): SurfaceSelection | null { + const ends = selection.shadowedSeqs.filter(seq => toolPairingBalancedAfter(session, seq)) + if (ends.length < 2) return null + // A validated span ends balanced; two or more units leave a nonempty proper prefix. + // oxlint-disable-next-line typescript/no-non-null-assertion + const end = ends[Math.ceil(ends.length / 2) - 1]! + return validateSurfaceRegion(session, selection.start, end) +} + /** Reject a summary prepared against any earlier surface generation. */ function assertWholeSurfaceUnchanged( dependencies: RegionDependencies, diff --git a/packages/compaction/compaction-basic/tests/compaction-basic.spec.ts b/packages/compaction/compaction-basic/tests/compaction-basic.spec.ts index cf6780933d..789dd32fdd 100644 --- a/packages/compaction/compaction-basic/tests/compaction-basic.spec.ts +++ b/packages/compaction/compaction-basic/tests/compaction-basic.spec.ts @@ -850,15 +850,19 @@ describe('pressure measurement and retention', () => { expect(await compactIfNeeded(compact, retained)).toBeNull() }) - it('uses one unified measurement for each pressure-and-retention decision', async () => { + it('preserves history when a pressure summary fails', async () => { const ctx = createContext() const compact = service(compactConfig, ctx) - const measure = vi.spyOn(ctx.tokenMeter, 'measure') - const stop = new Error('stop after first decision') - vi.spyOn(compact, 'compactRegion').mockRejectedValueOnce(stop) + const session = conversation(4) + const originalNodes = [...session.surface.nodes] + const stop = new Error('summary failed') + compact.error = stop - await expect(compactIfNeeded(compact, conversation(4))).rejects.toBe(stop) - expect(measure).toHaveBeenCalledTimes(1) + await expect(compactIfNeeded(compact, session)).rejects.toBe(stop) + expect(compact.calls).toHaveLength(1) + expect(session.surface.nodes).toEqual(originalNodes) + expect(session.snapshotEvents().filter(event => event.type === 'compaction/summary')).toEqual([]) + expect(session.snapshotEvents().at(-1)).toMatchObject({ type: 'compaction/end', data: { error: 'summary failed' } }) }) it('bounds retries when a shrinking checkpoint remains above threshold', async () => { diff --git a/packages/compaction/compaction-basic/tests/summary-overflow-loop.spec.ts b/packages/compaction/compaction-basic/tests/summary-overflow-loop.spec.ts new file mode 100644 index 0000000000..91f747fc1e --- /dev/null +++ b/packages/compaction/compaction-basic/tests/summary-overflow-loop.spec.ts @@ -0,0 +1,116 @@ +/** Summary overflow recovery through the real agent loop and image projections. */ + +import { describe, expect, it } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import AgentLoop from '@deepseek-ai/dsh-agent-loop' +import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-testkit' +import { defineContentToolFixture } from '@deepseek-ai/dsh-tool-testkit' +import { + CONTEXT_WINDOW_EXCEEDED_CODE, LlmAdapter, LlmError, ToolCallId, createUserMessage, +} from '@deepseek-ai/dsh-llm' +import type { ContentBlock, GenerateOptions, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm' +import { SessionId } from '@deepseek-ai/dsh-session' +import TokenMeter from '@deepseek-ai/dsh-token-meter' +import * as imageOffload from '@deepseek-ai/dsh-compaction-image-offload' +import BasicCompactionEngine from '../src/index.ts' + +type ImageBlock = Extract + +function image(name: string): ImageBlock { + return { + type: 'image', + attachment: { + attachmentId: `sha256:${'a'.repeat(64)}` as ImageBlock['attachment']['attachmentId'], + name, mediaType: 'image/png', bytes: 1024, width: 1920, height: 1080, + }, + } +} + +function imageNames(options: GenerateOptions): string[] { + return options.messages.flatMap(message => message.content.flatMap(block => + block.type === 'image' && block.offloaded !== true ? [block.attachment.name ?? ''] : [])) +} + +/** Capacity is computed from valid image occurrences; successful calls report exact fixture usage. */ +class ImageCapacityAdapter extends LlmAdapter { + readonly requests: Array<{ purpose: string; images: string[]; overflow: boolean }> = [] + private toolIssued = false + + override resolveModel(provider: string, model: string): Promise { + return Promise.resolve({ provider, id: model, name: model, inputModalities: ['text', 'image'], context: { contextWindow: 10_000 } }) + } + + override async * stream(options: GenerateOptions): AsyncIterable { + const images = imageNames(options) + const textTokens = options.messages.reduce((total, message) => total + message.content.reduce((sum, block) => + sum + (block.type === 'text' ? Math.ceil(block.text.length / 4) : 0), 4), 0) + const inputTokens = images.length * 2000 + textTokens + const overflow = inputTokens + (options.maxTokens ?? 0) > 10_000 + this.requests.push({ purpose: options.purpose ?? 'conversation', images, overflow }) + if (overflow) throw new LlmError('fixture image context exceeds the model window', CONTEXT_WINDOW_EXCEEDED_CODE) + + const lastText = options.messages.at(-1)?.content.flatMap(block => block.type === 'text' ? [block.text] : []).join('') + const callsTool = options.purpose !== 'compaction' && lastText === 'Collect two more screenshots.' && !this.toolIssued + if (callsTool) { + this.toolIssued = true + yield { type: 'block-start', index: 0, blockType: 'tool-call' } + yield { type: 'block-end', index: 0, block: { type: 'tool-call', id: ToolCallId('capture'), name: 'capture_images', arguments: '{}' } } + } else { + yield { type: 'block-start', index: 0, blockType: 'text' } + yield { type: 'block-end', index: 0, block: { type: 'text', text: options.purpose === 'compaction' ? 'Earlier screenshots were reviewed.' : 'Received.' } } + } + yield { type: 'usage', usage: { inputTokens, outputTokens: 10 } } + yield { type: 'finish', reason: { kind: callsTool ? 'tool-calls' : 'stop' } } + } +} + +describe('summary overflow through the agent loop', () => { + it('summarizes a smaller prefix and retries with the untouched image suffix', async () => { + const ctx = new Context() + const adapter = new ImageCapacityAdapter() + try { + await mountAgentLoopTestDependencies(ctx) + await ctx.plugin(imageOffload) + await ctx.plugin(AgentLoop, { agents: [] }) + await ctx.plugin(TokenMeter) + ctx.effect(() => ctx.llm.registerAdapter(['mock'], adapter)) + ctx.effect(() => ctx.tools.register(defineContentToolFixture({ + name: 'capture_images', description: 'Return two screenshot references', parameters: {}, + async execute(_args, exec) { + if (exec.agent === undefined) throw new Error('image fixture requires an agent') + // Steering makes the complete tool exchange eligible for prefix compaction. + exec.agent.steer(createUserMessage({ content: [{ type: 'text', text: 'Continue inspecting the screenshots.' }], source: { kind: 'user' } })) + return [image('recent-a'), image('recent-b')] + }, + }))) + await ctx.plugin(BasicCompactionEngine, { headroomTokens: 0, thresholdRatio: 1, retainTokens: 100, maxTokens: 256 }) + const agent = await ctx.agentLoop.create(SessionId('image-summary-overflow'), { provider: 'mock', model: 'vision', maxTokens: 256 }) + for (const label of ['old', 'middle']) { + agent.followup(createUserMessage({ + content: [{ type: 'text', text: `${label}: ${'dashboard details '.repeat(40)}` }, image(`${label}-a`), image(`${label}-b`)], + source: { kind: 'user' }, + })) + await agent.whenIdle() + expect(agent.session.snapshotEvents().at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) + } + agent.followup(createUserMessage({ content: [{ type: 'text', text: 'Collect two more screenshots.' }], source: { kind: 'user' } })) + await agent.whenIdle() + + const summaries = adapter.requests.filter(request => request.purpose === 'compaction') + expect(summaries).toHaveLength(2) + expect(summaries[0]).toMatchObject({ overflow: true, images: ['old-a', 'old-b', 'middle-a', 'middle-b', 'recent-a', 'recent-b'] }) + expect(summaries[1]).toMatchObject({ overflow: false, images: ['old-a', 'old-b', 'middle-a', 'middle-b'] }) + expect(adapter.requests.at(-1)).toMatchObject({ purpose: 'conversation', overflow: false, images: ['recent-a', 'recent-b'] }) + + const events = agent.session.snapshotEvents() + expect(events.filter(event => event.type === 'compaction/start')).toHaveLength(1) + expect(events.filter(event => event.type === 'compaction/end')).toHaveLength(1) + expect(events.filter(event => event.type === 'compaction/summary')).toHaveLength(1) + expect(events.filter(event => event.type === 'image/offload')).toEqual([]) + expect(events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) + expect(agent.session.surface.replaceGeneration).toBe(1) + } finally { + await ctx.fiber.dispose() + } + }) +}) diff --git a/packages/compaction/compaction-basic/tests/summary-overflow.spec.ts b/packages/compaction/compaction-basic/tests/summary-overflow.spec.ts new file mode 100644 index 0000000000..da34b56156 --- /dev/null +++ b/packages/compaction/compaction-basic/tests/summary-overflow.spec.ts @@ -0,0 +1,250 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import type { Agent } from '@deepseek-ai/dsh-agent' +import { CONTEXT_WINDOW_EXCEEDED_CODE, createAssistantMessage, createSystemMessage, createToolResultMessage, createUserMessage, LlmError, ToolCallId } from '@deepseek-ai/dsh-llm' +import { Session, SessionId } from '@deepseek-ai/dsh-session' +import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' +import TokenMeter from '@deepseek-ai/dsh-token-meter' +import { compactSurfaceRegion } from '../src/region.ts' +import type { SummarizationInput, SummaryResult } from '../src/summarizer.ts' + +const contexts: Context[] = [] +afterEach(async () => { await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose())) }) + +const historyText = 'A retained source fact with enough detail for a smaller checkpoint. '.repeat(80) + +function overflow(): LlmError { + return new LlmError('summary input exceeds the model context', CONTEXT_WINDOW_EXCEEDED_CODE) +} + +function summary(text = 'The older task details are summarized.'): SummaryResult { + return { summary: [{ type: 'text', text }], provider: 'test', model: 'summary' } +} + +/** Each tool unit contains two calls and both results; user units contain one message. */ +function harness(units: readonly ('user' | 'tools')[]) { + const ctx = new Context() + contexts.push(ctx) + new SessionProjectionRegistry(ctx) + const meter = new TokenMeter(ctx) + const session = Session.create(SessionId('summary-overflow')) + session.append('turn/start', { turn: 1 }) + const head = session.append('system/message', { + turn: 1, step: 1, message: createSystemMessage(historyText), + }, { surfaceOp: 'append' }) + for (const [index, unit] of units.entries()) { + if (unit === 'user') { + session.append('user/message', createUserMessage({ + content: [{ type: 'text', text: `${index}: ${historyText}` }], source: { kind: 'user' }, + }), { surfaceOp: 'append' }) + continue + } + const step = index + 1 + const callIds = [ToolCallId(`call-${index}-a`), ToolCallId(`call-${index}-b`)] + session.append('step/start', { turn: 1, step }) + session.append('assistant/message', { + turn: 1, step, stream: [], + message: createAssistantMessage({ + source: { provider: 'test', model: 'test' }, + content: callIds.map(id => ({ type: 'tool-call', id, name: 'read', arguments: '{}' })), + }), + }, { surfaceOp: 'append' }) + for (const callId of callIds) { + session.append('tool/call', { turn: 1, step, callId, name: 'read', arguments: '{}' }) + session.append('tool/result', { + turn: 1, step, + message: createToolResultMessage({ callId, content: [{ type: 'text', text: historyText }], isError: false }), + }, { surfaceOp: 'append' }) + } + session.append('step/end', { turn: 1, step }) + } + const selected = session.surface.nodes.slice(1) + const start = selected[0] + const end = selected.at(-1) + if (start === undefined || end === undefined) throw new Error('fixture needs a nonempty selection') + const tail = session.append('user/message', createUserMessage({ + content: [{ type: 'text', text: 'Keep the newest request verbatim.' }], source: { kind: 'user' }, + }), { surfaceOp: 'append' }) + const agent = { session, options: { provider: 'test', model: 'test' } } as Agent + const summarize = vi.fn<(input: SummarizationInput) => Promise>() + const recover = vi.fn[0]['recover']>(() => false) + const requests: SummarizationInput[] = [] + const run = (allowSmallerPrefix = true, signal?: AbortSignal, owner: 'current-turn' | null = 'current-turn') => compactSurfaceRegion({ + meter, + summarize(input) { requests.push(input); return summarize(input) }, + recover, + }, session, start, end, agent, { + owner, stability: owner === null ? 'selected-span' : 'whole-surface', + ...allowSmallerPrefix ? { allowSmallerPrefix: true as const } : {}, + }, signal) + return { session, head, tail, selected, requests, summarize, recover, run } +} + +function sourceCount(input: SummarizationInput): number { + return input.messages.filter(message => message.role !== 'system').length +} + +function compactionTypes(session: Session): string[] { + return session.snapshotEvents().filter(event => event.type.startsWith('compaction/')).map(event => event.type) +} + +describe('summary context overflow', () => { + it.each([8, 9])('halves complete prefixes until one fits and preserves the excluded suffix: %s units', async (count) => { + const h = harness(Array.from({ length: count }, () => 'user')) + h.summarize.mockImplementation(async (input) => { + if (sourceCount(input) > 2) throw overflow() + return summary() + }) + const result = await h.run() + expect(h.requests.map(sourceCount)).toEqual(count === 8 ? [8, 4, 2] : [9, 5, 3, 2]) + expect(result.shadowedSeqs).toEqual(h.selected.slice(0, 2)) + expect(h.session.surface.nodes.slice(2)).toEqual([...h.selected.slice(2), h.tail.seq]) + expect(h.session.surface.nodes[0]).toBe(h.head.seq) + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/summary', 'compaction/end']) + }) + + it('counts a tool-call message and all its results as one indivisible unit', async () => { + const h = harness(['tools', 'user', 'user', 'user']) + h.summarize.mockRejectedValueOnce(overflow()).mockResolvedValue(summary()) + const result = await h.run() + expect(h.requests.map(sourceCount)).toEqual([6, 4]) + expect(h.requests[1]?.messages.map(message => message.role)).toEqual(['system', 'assistant', 'tool', 'tool', 'user']) + expect(result.shadowedSeqs).toEqual(h.selected.slice(0, 4)) + expect(h.session.surface.nodes.slice(2)).toEqual([...h.selected.slice(4), h.tail.seq]) + }) + + it.each(['user', 'tools'] as const)('stops at one indivisible %s unit without committing a checkpoint', async (unit) => { + const h = harness([unit]) + const error = overflow() + h.summarize.mockRejectedValue(error) + const original = [...h.session.surface.nodes] + await expect(h.run()).rejects.toBe(error) + expect(h.summarize).toHaveBeenCalledTimes(1) + expect(h.session.surface.nodes).toEqual(original) + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/end']) + }) + + it('exhausts progressively smaller prefixes when even the first unit overflows', async () => { + const h = harness(Array.from({ length: 8 }, () => 'user')) + const error = overflow() + h.summarize.mockRejectedValue(error) + const original = [...h.session.surface.nodes] + await expect(h.run()).rejects.toBe(error) + expect(h.requests.map(sourceCount)).toEqual([8, 4, 2, 1]) + expect(h.session.surface.nodes).toEqual(original) + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/end']) + }) + + it('keeps an explicitly requested range exact', async () => { + const h = harness(['user', 'user', 'user', 'user']) + const error = overflow() + h.summarize.mockRejectedValue(error) + await expect(h.run(false)).rejects.toBe(error) + expect(h.summarize).toHaveBeenCalledTimes(1) + }) + + it.each([ + new LlmError('credentials expired', 'AUTH'), + new LlmError('provider unavailable', 'SERVER'), + new LlmError('summary truncated', 'MAX_TOKENS'), + Object.assign(new Error('unclassified failure'), { code: CONTEXT_WINDOW_EXCEEDED_CODE }), + ])('does not shorten for other failures: %s', async (error) => { + const h = harness(['user', 'user', 'user', 'user']) + h.summarize.mockRejectedValue(error) + await expect(h.run()).rejects.toBe(error) + expect(h.summarize).toHaveBeenCalledTimes(1) + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/end']) + }) + + it('does not retry or invoke recovery after cancellation', async () => { + const h = harness(['user', 'user', 'user', 'user']) + const controller = new AbortController() + const error = overflow() + h.summarize.mockImplementation(async () => { controller.abort('cancelled'); throw error }) + await expect(h.run(true, controller.signal)).rejects.toBe(error) + expect(h.summarize).toHaveBeenCalledTimes(1) + expect(h.recover).not.toHaveBeenCalled() + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/end']) + }) + + it('rejects a changed surface before considering a smaller selection', async () => { + const h = harness(['user', 'user', 'user', 'user']) + h.summarize.mockImplementation(async () => { + h.session.append('user/message', createUserMessage({ + content: [{ type: 'text', text: 'concurrent input' }], source: { kind: 'user' }, + }), { surfaceOp: 'append' }) + throw overflow() + }) + await expect(h.run()).rejects.toThrow('session surface changed during summarization') + expect(h.summarize).toHaveBeenCalledTimes(1) + expect(h.recover).not.toHaveBeenCalled() + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/end']) + }) + + it.each(['manual', 'automatic'] as const)('retains %s stability rules when the excluded suffix changes after shortening', async (mode) => { + const h = harness(Array.from({ length: 8 }, () => 'user')) + if (mode === 'manual') h.session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + const replaced = h.selected[6] + if (replaced === undefined) throw new Error('fixture needs a suffix node') + let replacement = replaced + h.summarize.mockRejectedValueOnce(overflow()).mockImplementation(async () => { + replacement = h.session.append('user/message', createUserMessage({ + content: [{ type: 'text', text: 'Updated excluded suffix' }], source: { kind: 'user' }, + }), { + surfaceOp: { op: 'replace', startSeq: replaced, endSeq: replaced }, sourceEventSeqs: [replaced], + }).seq + return summary() + }) + const operation = h.run(true, undefined, mode === 'manual' ? null : 'current-turn') + if (mode === 'manual') { + const result = await operation + expect(result.shadowedSeqs).toEqual(h.selected.slice(0, 4)) + expect(h.session.surface.nodes.slice(2)).toEqual([ + ...h.selected.slice(4, 6), replacement, ...h.selected.slice(7), h.tail.seq, + ]) + expect(h.session.snapshotEvents().find(event => event.type === 'compaction/start')?.data).toMatchObject({ turn: null }) + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/summary', 'compaction/end']) + } else { + await expect(operation).rejects.toThrow('session surface changed during summarization') + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/end']) + } + expect(h.requests.map(sourceCount)).toEqual([8, 4]) + }) + + it('allows a registered durable repair to retain the entire selection', async () => { + const h = harness(['user', 'user', 'user', 'user']) + h.summarize.mockRejectedValueOnce(overflow()).mockResolvedValue(summary()) + h.recover.mockImplementationOnce(() => { + h.session.append('system/message', { + turn: 1, step: 1, message: createSystemMessage('Reduced system input'), + }, { surfaceOp: { op: 'replace', startSeq: h.head.seq, endSeq: h.head.seq }, sourceEventSeqs: [h.head.seq] }) + return true + }) + const result = await h.run() + expect(h.requests.map(sourceCount)).toEqual([4, 4]) + expect(h.requests[1]?.messages[0]?.content).toEqual([{ type: 'text', text: 'Reduced system input' }]) + expect(result.shadowedSeqs).toEqual(h.selected) + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/summary', 'compaction/end']) + }) + + it('preserves the original surface if a smaller summary fails for another reason', async () => { + const h = harness(['user', 'user', 'user', 'user']) + const error = new LlmError('provider unavailable', 'SERVER') + h.summarize.mockRejectedValueOnce(overflow()).mockRejectedValue(error) + const original = [...h.session.surface.nodes] + await expect(h.run()).rejects.toBe(error) + expect(h.requests.map(sourceCount)).toEqual([4, 2]) + expect(h.session.surface.nodes).toEqual(original) + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/end']) + }) + + it('compares the checkpoint with the smaller source span before committing', async () => { + const h = harness(['user', 'user', 'user', 'user']) + h.summarize.mockRejectedValueOnce(overflow()).mockResolvedValue(summary(historyText.repeat(3))) + const original = [...h.session.surface.nodes] + await expect(h.run()).rejects.toThrow('summary is not smaller than the shadowed content') + expect(h.requests.map(sourceCount)).toEqual([4, 2]) + expect(h.session.surface.nodes).toEqual(original) + expect(compactionTypes(h.session)).toEqual(['compaction/start', 'compaction/end']) + }) +}) diff --git a/snapshots/acp/acp.snapshot.ts b/snapshots/acp/acp.snapshot.ts index 1cde2b7a68..a7900854f4 100644 --- a/snapshots/acp/acp.snapshot.ts +++ b/snapshots/acp/acp.snapshot.ts @@ -41,6 +41,11 @@ const controllerCases: readonly { hasModelTurn: true, configPath: join(corpusDir, 'image-compaction', 'cordis.yml'), }, + { + name: 'summary-overflow', + hasModelTurn: true, + configPath: join(corpusDir, 'summary-overflow', 'cordis.yml'), + }, ] as const function localScenarioSource(source: string | undefined): string | undefined { diff --git a/snapshots/acp/summary-overflow/cordis.snapshot.yml b/snapshots/acp/summary-overflow/cordis.snapshot.yml new file mode 100644 index 0000000000..a9e6f599da --- /dev/null +++ b/snapshots/acp/summary-overflow/cordis.snapshot.yml @@ -0,0 +1,58 @@ +- id: llm-deepseek + name: '@deepseek-ai/dsh-llm-deepseek-api-key' + disabled: true + +- id: acp + name: '@deepseek-ai/dsh-acp' + config: + provider: deepseek-official + model: deepseek-v4-flash-vision-exp + +- id: session-persistence-jsonl + name: '@deepseek-ai/dsh-session-persistence-jsonl' + config: + root: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions' + compression: none + +- id: agent-instructions + name: '@deepseek-ai/dsh-agent-instructions' + config: + maxBytes: 65536 + +- id: system-prompt + name: '@deepseek-ai/dsh-system-prompt' + config: + personaPrefix: | + You are a coding assistant powered by the {{model}} model. Your bash tool runs under a file sandbox — a `[sandbox: file access denied …]` result is policy, not a command bug. + + Verify your work by running the code or tests. Keep answers brief and factual. + +- insert: + - id: llm-replay + name: '@deepseek-ai/dsh-llm-replay' + config: + providers: + - id: deepseek-official + name: DeepSeek + models: + - id: deepseek-v4-flash + inputModalities: [text] + - id: deepseek-v4-pro + inputModalities: [text] + - id: deepseek-v4-flash-vision-exp + inputModalities: [text, image] + contextWindow: 1000000 + imageRequestTokens: 384 + +- id: attachment-local + name: '@deepseek-ai/dsh-attachment-local' + +- id: compaction-basic + name: '@deepseek-ai/dsh-compaction-basic' + config: + headroomTokens: 0 + maxTokens: 8192 + retainTokens: 100 + +- insert: + - name: './summary-window.mjs' diff --git a/snapshots/acp/summary-overflow/cordis.yml b/snapshots/acp/summary-overflow/cordis.yml new file mode 100644 index 0000000000..eef7dbd2bf --- /dev/null +++ b/snapshots/acp/summary-overflow/cordis.yml @@ -0,0 +1,37 @@ +- id: acp + name: '@deepseek-ai/dsh-acp' + config: + provider: deepseek-official + model: deepseek-v4-flash-vision-exp + +- id: session-persistence-jsonl + name: '@deepseek-ai/dsh-session-persistence-jsonl' + config: + root: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions' + compression: none + +- id: agent-instructions + name: '@deepseek-ai/dsh-agent-instructions' + config: + maxBytes: 65536 + +- id: system-prompt + name: '@deepseek-ai/dsh-system-prompt' + config: + personaPrefix: | + You are a coding assistant powered by the {{model}} model. Your bash tool runs under a file sandbox — a `[sandbox: file access denied …]` result is policy, not a command bug. + + Verify your work by running the code or tests. Keep answers brief and factual. + +- id: attachment-local + name: '@deepseek-ai/dsh-attachment-local' + +- id: compaction-basic + name: '@deepseek-ai/dsh-compaction-basic' + config: + headroomTokens: 0 + maxTokens: 8192 + retainTokens: 100 + +- insert: + - name: './summary-window.mjs' diff --git a/snapshots/acp/summary-overflow/input.json b/snapshots/acp/summary-overflow/input.json new file mode 100644 index 0000000000..e5115716cf --- /dev/null +++ b/snapshots/acp/summary-overflow/input.json @@ -0,0 +1,67 @@ +{ + "steps": [ + { + "op": "initialize" + }, + { + "op": "newSession" + }, + { + "op": "promptContent", + "content": [ + { + "type": "text", + "text": "First group: inspect these three reference screenshots and acknowledge receipt." + }, + { + "type": "image", + "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC", + "mimeType": "image/png" + }, + { + "type": "image", + "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC", + "mimeType": "image/png" + }, + { + "type": "image", + "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC", + "mimeType": "image/png" + } + ] + }, + { + "op": "promptContent", + "content": [ + { + "type": "text", + "text": "Second group: inspect these three reference screenshots and acknowledge receipt." + }, + { + "type": "image", + "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC", + "mimeType": "image/png" + }, + { + "type": "image", + "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC", + "mimeType": "image/png" + }, + { + "type": "image", + "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC", + "mimeType": "image/png" + } + ] + }, + { + "op": "promptContent", + "content": [ + { + "type": "text", + "text": "Now reply with exactly the single word DONE." + } + ] + } + ] +} diff --git a/snapshots/acp/summary-overflow/session.v4.jsonl b/snapshots/acp/summary-overflow/session.v4.jsonl new file mode 100644 index 0000000000..f4e078caf3 --- /dev/null +++ b/snapshots/acp/summary-overflow/session.v4.jsonl @@ -0,0 +1,40 @@ +{"type":"session","version":4,"id":"{{session:1}}","createdAt":1783952000000,"cwd":"{{cwd}}","isSeeded":false,"delegationDepth":0} +{"type":"permission/preset","data":{"preset":"danger-full-access"}} +{"type":"sandbox/mode","data":{"mode":"danger-full-access"}} +{"type":"approval/policy","data":{"policy":"never"}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"First group: inspect these three reference screenshots and acknowledge receipt."},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}}],"source":{"kind":"user"},"role":"user","id":"{{message:1}}"}]}} +{"type":"turn/start","data":{"turn":1}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","data":{"turn":1,"step":1}} +{"type":"system/message","data":{"turn":1,"step":1,"message":{"role":"system","content":[{"type":"text","text":"{{system}}"}],"source":{"kind":"system-prompt"},"id":"{{message:2}}"}},"surfaceOp":"append"} +{"type":"user/message","data":{"content":[{"type":"text","text":"First group: inspect these three reference screenshots and acknowledge receipt."},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}}],"source":{"kind":"user"},"role":"user","id":"{{message:1}}"},"surfaceOp":"append"} +{"type":"user/message","data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent working directory: \"{{cwd}}\".\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}],"source":{"kind":"runtime-context","form":"snapshot","sections":[{"name":"working-directory:current","text":"Current working directory: \"{{cwd}}\"."},{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}]},"role":"user","id":"{{message:3}}"},"surfaceOp":"append"} +{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/context","data":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp","contextWindow":1000000}} +{"type":"session/title","data":{"title":"First group: inspect these three","messageSeqs":[8],"source":{"kind":"fallback"}}} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"Received the first three frames."}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:4}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1792679887018,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"chunk","time":1792679887019,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"Received the first three frames."}}},{"type":"chunk","time":1792679887019,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1792679887019,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":1}} +{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Second group: inspect these three reference screenshots and acknowledge receipt."},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}}],"source":{"kind":"user"},"role":"user","id":"{{message:5}}"}]}} +{"type":"turn/start","data":{"turn":2}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","data":{"turn":2,"step":1}} +{"type":"user/message","data":{"content":[{"type":"text","text":"Second group: inspect these three reference screenshots and acknowledge receipt."},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}}],"source":{"kind":"user"},"role":"user","id":"{{message:5}}"},"surfaceOp":"append"} +{"type":"assistant/message","data":{"turn":2,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"Received the next three frames."}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:6}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1792679887081,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"chunk","time":1792679887081,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"Received the next three frames."}}},{"type":"chunk","time":1792679887081,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1792679887081,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"} +{"type":"step/end","data":{"turn":2,"step":1}} +{"type":"turn/end","data":{"turn":2,"reason":{"kind":"completed"}}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Now reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"{{message:7}}"}]}} +{"type":"turn/start","data":{"turn":3}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","data":{"turn":3,"step":1}} +{"type":"user/message","data":{"content":[{"type":"text","text":"Now reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"{{message:7}}"},"surfaceOp":"append"} +{"type":"assistant/attempt","data":{"turn":3,"step":1,"stream":[{"type":"chunk","time":1792679887096,"chunk":{"type":"finish","reason":{"kind":"error","failure":{"code":"CONTEXT_WINDOW_EXCEEDED","message":"screenshot history exceeds the authored context window"}}}}]}} +{"type":"compaction/start","data":{"compactionId":"{{id:1}}","turn":3}} +{"type":"compaction/summary","data":{"compactionId":"{{id:1}}","summary":[{"type":"text","text":"The first group of three dashboard screenshots was inspected and acknowledged."}],"rawOutput":[{"type":"text","text":"The first group of three dashboard screenshots was inspected and acknowledged."}],"llmStreamCall":true,"shadowedRange":{"start":8,"end":13},"shadowedSeqs":[8,9,13],"shadowedTokenCount":319,"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp","maxTokens":8192,"usage":{"inputTokens":20,"outputTokens":16}}} +{"type":"user/message","data":{"content":[{"type":"text","text":"This is an automatically generated checkpoint condensing an earlier span of the conversation to free up context. Treat the captured context as established background and build on it without restating it. Continue the task directly from the messages that follow, without acknowledging this checkpoint.\n\n"},{"type":"text","text":"The first group of three dashboard screenshots was inspected and acknowledged."},{"type":"text","text":""}],"source":{"kind":"compact-checkpoint","compactionId":"{{id:1}}"},"role":"user","id":"{{message:8}}"},"sourceEventSeqs":[30,31,8,9,13],"surfaceOp":{"op":"replace","startSeq":8,"endSeq":13}} +{"type":"compaction/end","data":{"compactionId":"{{id:1}}","turn":3}} +{"type":"user/message","data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent working directory: \"{{cwd}}\".\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}],"source":{"kind":"runtime-context","form":"snapshot","sections":[{"name":"working-directory:current","text":"Current working directory: \"{{cwd}}\"."},{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}]},"role":"user","id":"{{message:9}}"},"surfaceOp":"append"} +{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"tools":"{{tools}}"},"reason":"series"}} +{"type":"assistant/message","data":{"turn":3,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"DONE"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:10}}"},"usage":{"inputTokens":3,"outputTokens":3},"stream":[{"type":"chunk","time":1792679887109,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"chunk","time":1792679887109,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DONE"}}},{"type":"chunk","time":1792679887109,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}},{"type":"chunk","time":1792679887109,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"} +{"type":"step/end","data":{"turn":3,"step":1}} +{"type":"turn/end","data":{"turn":3,"reason":{"kind":"completed"}}} diff --git a/snapshots/acp/summary-overflow/snapshot.yml b/snapshots/acp/summary-overflow/snapshot.yml new file mode 100644 index 0000000000..13495f3a7d --- /dev/null +++ b/snapshots/acp/summary-overflow/snapshot.yml @@ -0,0 +1,16 @@ +version: 1 +scenario: summary-overflow +profile: acp +composition: summary-overflow +recording: authored +header: + class: summary-overflow + pin: true + systemPromptSource: session/read-image + toolSchemasSource: escalation-approved +permission: danger-full-access +input: + attachments: + - id: sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640 + mediaType: image/png + data: iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElFTkSuQmCC diff --git a/snapshots/acp/summary-overflow/stdout.expected.jsonl b/snapshots/acp/summary-overflow/stdout.expected.jsonl new file mode 100644 index 0000000000..6b812a367a --- /dev/null +++ b/snapshots/acp/summary-overflow/stdout.expected.jsonl @@ -0,0 +1,11 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"mcpCapabilities":{"http":true},"promptCapabilities":{"image":true,"audio":false,"embeddedContext":false},"sessionCapabilities":{"close":{},"list":{},"resume":{}}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}","configOptions":[{"id":"model","name":"Model","category":"model","type":"select","currentValue":"[\"deepseek-official\",\"deepseek-v4-flash-vision-exp\"]","options":[{"group":"deepseek-official","name":"DeepSeek","options":[{"value":"[\"deepseek-official\",\"deepseek-v4-flash\"]","name":"deepseek-v4-flash"},{"value":"[\"deepseek-official\",\"deepseek-v4-pro\"]","name":"deepseek-v4-pro"},{"value":"[\"deepseek-official\",\"deepseek-v4-flash-vision-exp\"]","name":"deepseek-v4-flash-vision-exp"}]}]}]}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","messageId":"{{messageId}}","content":{"type":"text","text":"Received the first three frames."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"usage_update","used":"{{usedTokens}}","size":1000000}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","messageId":"{{messageId}}","content":{"type":"text","text":"Received the next three frames."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"usage_update","used":"{{usedTokens}}","size":1000000}}} +{"jsonrpc":"2.0","id":4,"result":{"stopReason":"end_turn"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","messageId":"{{messageId}}","content":{"type":"text","text":"DONE"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"usage_update","used":"{{usedTokens}}","size":1000000}}} +{"jsonrpc":"2.0","id":5,"result":{"stopReason":"end_turn"}} diff --git a/snapshots/acp/summary-overflow/summary-window.mjs b/snapshots/acp/summary-overflow/summary-window.mjs new file mode 100644 index 0000000000..07913f6f56 --- /dev/null +++ b/snapshots/acp/summary-overflow/summary-window.mjs @@ -0,0 +1,26 @@ +/** Reject summaries containing both screenshot groups without consuming a replay entry. */ +export const name = 'summary-window' +export const inject = ['llm'] + +/** @param {import('@deepseek-ai/cordis').Context} ctx - Scenario-local context. */ +export function apply(ctx) { + ctx.on('llm/stream', async function* (options, next) { + const images = options.messages.flatMap(message => message.content) + .filter(block => block.type === 'image') + if (options.purpose === 'compaction' && images.length > 3) { + yield { + type: 'finish', + reason: { kind: 'error', failure: { + code: 'CONTEXT_WINDOW_EXCEEDED', message: 'summary screenshots exceed the authored context window', + } }, + } + return + } + const hasCheckpoint = options.messages.some(message => message.source?.kind === 'compact-checkpoint') + if (options.purpose !== 'compaction' && hasCheckpoint + && (images.length !== 3 || images.some(image => image.offloaded === true))) { + throw new Error('summary recovery must retain the second screenshot group') + } + yield* next() + }) +} diff --git a/snapshots/acp/summary-overflow/system-prompt.expected.md b/snapshots/acp/summary-overflow/system-prompt.expected.md new file mode 120000 index 0000000000..7d5e489c58 --- /dev/null +++ b/snapshots/acp/summary-overflow/system-prompt.expected.md @@ -0,0 +1 @@ +../../session/read-image/system-prompt.expected.md \ No newline at end of file