Merge pull request #7583 from deepseek-harness/turtle/compact-opus-hang-2d6253c47af9

fix(llm-pi-ai): omit Anthropic thinking whose request prefix changed
This commit is contained in:
Turtle authored and GitHub committed 2026-11-05 15:06:51 +08:00
commit 1e0ca162d9
11 files changed
+424 -30

No files matched your search

+10 -10
View File
@@ -51,8 +51,8 @@
en: 615ca9834bf6544f
zh: 2d70b548f99000d1
/deepseek-ai-dsh-llm-pi-ai/understand-the-implementation/source-map:
en: 64912b8a3a336419
zh: 33a245969b4194f6
en: 01520c709ee2202c
zh: de919d3f2ff61046
/deepseek-ai-dsh-llm-pi-ai/understand-the-implementation/registration-and-directory:
en: 0c6df3e8b6a75372
zh: 476666a6f209542c
@@ -60,8 +60,8 @@
en: 477d277e9ac35c8f
zh: 96870ca333fb6510
/deepseek-ai-dsh-llm-pi-ai/understand-the-implementation/replay-and-vocabulary:
en: e64173763fdbcfae
zh: 73ed2c2af54b62f6
en: e68f92d78150d341
zh: 45850c739a7329d3
/deepseek-ai-dsh-llm-pi-ai/further-exploration:
en: a2a2ccdeae59d408
zh: c47ee091aa90ad91
@@ -72,14 +72,14 @@
en: e7348db3f7115960
zh: 5fd5fb09d7d09982
/deepseek-ai-dsh-llm-pi-ai/model-experience/provider-request-through-pi-ai/what-the-model-sees:
en: 06f40c26b79df3cb
zh: 78676d41999b1ef2
en: f618a730aaa904ce
zh: 0b77a5b799b049e3
/deepseek-ai-dsh-llm-pi-ai/model-experience/provider-request-through-pi-ai/token-effect:
en: 820221e8ae37aa61
zh: 7e6ff4db10067fc0
/deepseek-ai-dsh-llm-pi-ai/model-experience/provider-request-through-pi-ai/kv-cache-effect:
en: cc7d4ec548afa30c
zh: ff3b01a65512f4ac
en: 6613d52e459d86da
zh: 7d0627e7ff8299e8
/deepseek-ai-dsh-llm-pi-ai/model-experience/provider-response:
en: 802d7ff60a1d03ff
zh: c41c56e28f3e8680
@@ -93,8 +93,8 @@
en: b6bd570c0159ba08
zh: 7d2d91eea6a8d830
/deepseek-ai-dsh-llm-pi-ai/known-limitations-and-deferred-work:
en: b8117f745275a043
zh: 29946f7b53ec1681
en: 07e32eef563bb747
zh: 225bde55120955f9
/deepseek-ai-dsh-llm-pi-ai/known-limitations-and-deferred-work/dev-note:
en: 3678dd0f6642c2a5
zh: 75cbdd064cc64282
+7 -2
View File
@@ -166,6 +166,7 @@ This event covers inference calls with or without a Session id, including auxili
| [`src/context.ts`](src/context.ts) | Harness-to-pi-ai context conversion, image handling, replay restore |
| [`src/stream.ts`](src/stream.ts) | pi-ai event conversion into harness `StreamChunk` values |
| [`src/replay.ts`](src/replay.ts) | Versioned `ReplayEnvelope` storage and validation |
| [`src/prefix-binding.ts`](src/prefix-binding.ts) | Anthropic thinking omission after a request-prefix change |
| [`src/discovery.ts`](src/discovery.ts) | Endpoint interrogation for configuration surfaces |
### Registration and directory
@@ -195,6 +196,8 @@ The `pre-commit` hook runs `node_modules/.bin/tsx packages/llm/llm-pi-ai/scripts
Successful assistant responses store a versioned, lossless-JSON replay state beside the provider and model that produced them — response-level facts plus one per-block entry per streamed block. At request time, `LlmRuntime` passes replay state only when the same adapter instance owns both routes; the adapter validates it and restores native response ids, provider signatures, and optional `providerThinkingLevel` effort metadata, keeping absent effort metadata absent. Replay validates the requested model identity against the assistant source and uses it to retain same-model signatures; provider-reported response models remain diagnostic metadata. Absent or unusable replay state degrades to provider-neutral content while preserving the assistant source’s required provider and model. pi-ai tool-call arguments are parsed objects, so the adapter parses input and re-stringifies output to the harness raw-JSON convention; pi-ai in-stream error events map to terminal `finish` chunks. Protocol conversion operates on request-local copies. Switching from A to B and back to A preserves the stored content, including empty text blocks, tool ids, original block order, and native replay metadata; requests to a different provider, API, or model omit unsupported signatures and reasoning blocks. Degraded replay without native metadata also omits reasoning. Valid native replay retains the stored reasoning when the original model resumes.
Anthropic can bind each signed thinking block to the request prefix it was generated under: system prompt, tools, and earlier messages. Accounts and requests that enforce this check reject a request that replays a stale block. Anthropic Messages responses therefore record a SHA-256 digest of their request context as `prefixDigest`. A later Anthropic Messages request recomputes the digest of the reconstructed prefix before each replayed response and omits that response's signed and redacted thinking when the digests differ, as after compaction, tool-result pruning, image offload, or a system-prompt or tool change. Replay state without `prefixDigest`, written before this check existed, is treated the same way, which repairs conversations that already fail. Text, tool calls, and unsigned thinking remain; a response that held only signed thinking is no longer sent. Omitted thinking stays omitted on later requests. Manual thinking requires the final assistant turn of a tool round to start with a thinking block, so when a request enables thinking and the model's compat does not set `forceAdaptiveThinking`, the assistant message whose tool round the request continues with tool results keeps its thinking.
</details>
-----
@@ -220,7 +223,7 @@ Read these pages when the package-level contract is not enough. They move from t
#### What the model sees
The selected catalog model receives system-message text, conversation history, tools, and supported sampling fields. The adapter preserves later system messages in history order when the catalog model declares `supportsMidConvoSystemMessages`. Otherwise, it converts each non-empty later system message to a user message wrapped in `<system-reminder>` before passing it to pi-ai. The wrapper does not grant system-role priority. One-shot `GenerateOptions.system` precedes the initial history prompt. Repeated system messages remain repeated; empty system messages are skipped without discarding earlier text. Tool-update capabilities select the shared historical declaration projection; pi-ai receives initially active tools and grouped developer updates carrying additions and supported removals. When every projected declaration of an Anthropic request with tool updates is deferred, the initially active tools are one placeholder, `DeferredToolPlaceholder`, whose description tells the model never to call it; a call reaches the ordinary unknown-tool result. Anthropic receives later tool definitions inline in the update message. Replay preserves tool-call namespaces for dynamically loaded Responses tools. Each retained image is preceded by text naming its complete attachment id and actual request dimensions. When the current execution filesystem maps the attachment provider's host object, the text also carries a read-only normalized-object path and warns that normalization or request projection may have resized or re-encoded the upload. Each occurrence selected by a logged image-offload decision keeps its own identity and currently resolved access in replacement text, and its normalized attachment is not read or transformed. When the retained occurrences still exceed the route's `maxRequestImages`, or their exact base64 payload exceeds its `maxRequestImageBytes`, the call fails with `IMAGE_OFFLOAD_REQUIRED` so `dsh-compaction-image-offload` records the selected occurrences in an `image/offload` event and retries the step. Provider-native replay metadata is restored only when the adapter validates it for the historical content. `imageRequestTarget(provider, model, image)` exposes the same request-image dimensions and byte target to screenshot consumers; adapters without image resizing preserve the normalized source.
The selected catalog model receives system-message text, conversation history, tools, and supported sampling fields. The adapter preserves later system messages in history order when the catalog model declares `supportsMidConvoSystemMessages`. Otherwise, it converts each non-empty later system message to a user message wrapped in `<system-reminder>` before passing it to pi-ai. The wrapper does not grant system-role priority. One-shot `GenerateOptions.system` precedes the initial history prompt. Repeated system messages remain repeated; empty system messages are skipped without discarding earlier text. Tool-update capabilities select the shared historical declaration projection; pi-ai receives initially active tools and grouped developer updates carrying additions and supported removals. When every projected declaration of an Anthropic request with tool updates is deferred, the initially active tools are one placeholder, `DeferredToolPlaceholder`, whose description tells the model never to call it; a call reaches the ordinary unknown-tool result. Anthropic receives later tool definitions inline in the update message. Replay preserves tool-call namespaces for dynamically loaded Responses tools. Each retained image is preceded by text naming its complete attachment id and actual request dimensions. When the current execution filesystem maps the attachment provider's host object, the text also carries a read-only normalized-object path and warns that normalization or request projection may have resized or re-encoded the upload. Each occurrence selected by a logged image-offload decision keeps its own identity and currently resolved access in replacement text, and its normalized attachment is not read or transformed. When the retained occurrences still exceed the route's `maxRequestImages`, or their exact base64 payload exceeds its `maxRequestImageBytes`, the call fails with `IMAGE_OFFLOAD_REQUIRED` so `dsh-compaction-image-offload` records the selected occurrences in an `image/offload` event and retries the step. Provider-native replay metadata is restored only when the adapter validates it for the historical content. On Anthropic Messages routes, a replayed response whose request prefix changed since generation, for example through compaction, loses its thinking blocks; its text and tool calls remain. `imageRequestTarget(provider, model, image)` exposes the same request-image dimensions and byte target to screenshot consumers; adapters without image resizing preserve the normalized source.
#### Token effect
@@ -228,7 +231,7 @@ Provider tokenization governs exact input. Retained images add the stable attach
#### KV Cache effect
Later system messages supplied to the adapter stay out of the initial prompt, preserving the preceding request prefix. This does not change the agent loop’s prompt admission, which consolidates updates on routes without native system-update support. Conversion preserves logical request order, while image handles and offload placeholders add model-visible text. A changed execution-world path rewrites a historical handle and can prevent reuse from that image even when attachment identity and request bytes stay stable. Changing adapter instance, provider, model, or another upstream token has the same suffix effect. An offload decision turns an earlier image into placeholder text, so reuse ends at that message; the omission never reverts, so the prefix stays stable until the next decision. A count overflow offloads `imageOffloadCountQuantum` images at once, so a saturated route ends reuse at the oldest retained image once per that many new images.
Later system messages supplied to the adapter stay out of the initial prompt, preserving the preceding request prefix. This does not change the agent loop’s prompt admission, which consolidates updates on routes without native system-update support. Conversion preserves logical request order, while image handles and offload placeholders add model-visible text. A changed execution-world path rewrites a historical handle and can prevent reuse from that image even when attachment identity and request bytes stay stable. Changing adapter instance, provider, model, or another upstream token has the same suffix effect. Omitting stale Anthropic thinking follows a prefix change that already ended reuse at the changed message. The first request that omits thinking recorded without a digest ends reuse at the first such message once. An offload decision turns an earlier image into placeholder text, so reuse ends at that message; the omission never reverts, so the prefix stays stable until the next decision. A count overflow offloads `imageOffloadCountQuantum` images at once, so a saturated route ends reuse at the oldest retained image once per that many new images.
### Provider response
@@ -270,6 +273,8 @@ These limits define where the adapter stops and future work begins. They are cur
- **Provider HTTP status is unavailable** — pi-ai error events do not expose a stable HTTP status across providers.
- **Retry policy is provider-owned, not an SDK retry** — pi-ai SDK retries stay disabled so durable agent steps and `llm/retry` events own every visible attempt, and direct `ctx.llm.stream()` calls remain single-attempt.
- **Streamed JSON tool-call arguments avoid per-delta reparsing** — the installed pi-ai carries [`patches/@[email protected]`](../../../patches/@[email protected]), which removes the per-delta re-parse of the whole accumulated argument JSON in every stream adapter (upstream [earendil-works/pi#9265](https://github.com/earendil-works/pi/issues/9265)); unpatched, a multi-megabyte argument stream costs O(n²) CPU on the event loop and stalls every session in the process. Completion events supply parsed arguments; Responses may parse at both argument completion and output-item completion. This adapter reads only delta strings and finalized arguments. Re-apply or retire the patch on every pi-ai upgrade.
- **Stale Anthropic thinking is omitted even where the endpoint would accept it** — prefix checks depend on the model, the account, and the request, which the adapter cannot observe, so every prefix change and every replay state without `prefixDigest` costs the model that reasoning, and Anthropic regenerates it as output tokens. The digest covers the pi-ai context objects, not the wire body: switching authentication or wire-changing compat settings mid-conversation is not detected, a pi-ai upgrade that changes Anthropic message serialization requires a new `DIGEST_SEED` in [`src/prefix-binding.ts`](src/prefix-binding.ts), and any change to how this package builds those objects drops recorded thinking once.
- **A manual-thinking tool round that spans a prefix change can still be rejected** — its open assistant message must keep its stale thinking to satisfy manual thinking, so an endpoint that enforces the prefix check rejects each request until a user message ends the round. Models that enforce the check use adaptive thinking in the installed catalog.
- **Gemini tool-call thought signatures are patched into Chat Completions** — the same patch reads `tool_calls[].extra_content.google.thought_signature` from a streamed OpenAI-compatible response into the tool call's replay signature and sends it back as `extra_content` when the same model resumes (upstream [earendil-works/pi#10157](https://github.com/earendil-works/pi/issues/10157)); unpatched, Gemini rejects the next step of a tool-calling turn with HTTP 400. Encrypted OpenRouter reasoning details keep their `reasoning_details` replay.
<a id="dev-note"></a>
+7 -2
View File
@@ -166,6 +166,7 @@ Config 更新严格验证发生变化的 provider。初始加载将已存储的
| [`src/context.ts`](src/context.ts) | Harness 到 pi-ai 的上下文转换、图片处理、回放恢复 |
| [`src/stream.ts`](src/stream.ts) | 把 pi-ai 事件转换为 harness `StreamChunk` 值 |
| [`src/replay.ts`](src/replay.ts) | 带版本的 `ReplayEnvelope` 存储与校验 |
| [`src/prefix-binding.ts`](src/prefix-binding.ts) | 请求前缀变化后省略 Anthropic thinking |
| [`src/discovery.ts`](src/discovery.ts) | 面向配置界面的端点询问 |
### 注册与目录
@@ -195,6 +196,8 @@ git add packages/llm/llm-pi-ai/src/provider-data.generated.ts
成功 assistant 响应会存储带版本的、无损 JSON 回放状态,与产生它们的提供方和模型放在一起——响应级事实加每个流式块一条逐块条目。请求时,`LlmRuntime` 仅当同一适配器实例拥有两条路由时才传递回放状态;适配器校验它并恢复原生响应 id、提供方签名与可选的 `providerThinkingLevel` effort 元数据,缺失的 effort 元数据仍保持缺失。回放会对照 assistant 来源校验请求模型身份,并用它保留同模型签名;提供方报告的响应模型仅作为诊断元数据。回放状态缺失或无法使用时会降级为提供方无关内容,并保留 assistant 来源中必需的提供方和模型。pi-ai 工具调用参数是解析后的对象,因此适配器解析输入并重新字符串化输出,以符合 harness 原始 JSON 约定;pi-ai 流内错误事件映射为终止 `finish` 分片。 协议转换仅作用于请求副本。从 A 切换到 B 再切回 A 时,存储的内容保持完整,包括空文本块、工具 id、原始块顺序和原生回放元数据;发给不同提供方、API 或模型的请求会省略不受支持的签名和推理块。缺少原生元数据的降级回放也会省略推理。切回原模型时,有效的原生回放会保留已存储的推理。
Anthropic 可以将每个已签名 thinking 块绑定到生成它时的请求前缀:系统提示词、工具和此前的消息。执行此检查的账户和请求会拒绝回放过期块的请求。因此,Anthropic Messages 响应会将其请求上下文的 SHA-256 摘要记录为 `prefixDigest`。后续 Anthropic Messages 请求会在每个回放响应之前重新计算重建前缀的摘要;摘要不同时,例如在压缩、工具结果裁剪、图片卸载或系统提示词或工具变化之后,省略该响应的已签名 thinking 与 redacted thinking。不含 `prefixDigest` 的回放状态(在此检查出现之前写入)按同样方式处理,从而修复已经失败的对话。文本、工具调用和未签名 thinking 保留;只含已签名 thinking 的响应不再发送。已省略的 thinking 在后续请求中保持省略。手动思考要求工具轮次的最后一个 assistant 轮次以 thinking 块开头,因此当请求启用 thinking 且模型的 compat 未设置 `forceAdaptiveThinking` 时,请求以工具结果延续的工具轮次对应的 assistant 消息会保留其 thinking。
</details>
-----
@@ -220,7 +223,7 @@ git add packages/llm/llm-pi-ai/src/provider-data.generated.ts
#### 模型看到什么
所选目录模型会收到系统消息文本、对话历史、工具以及支持的采样字段。目录模型声明 `supportsMidConvoSystemMessages` 时,适配器按历史顺序保留后续系统消息。否则,适配器在传给 pi-ai 之前,将每条非空的后续系统消息转换为以 `<system-reminder>` 包裹的用户消息。包裹文本不会赋予系统角色优先级。单次调用的 `GenerateOptions.system` 放在初始历史提示词之前。重复的系统消息仍会重复发送;空系统消息会被跳过,不会丢弃之前的文本。工具更新能力选择共享的历史声明投影;pi-ai 接收初始可用工具,以及按开发者消息分组的新增和受支持的移除更新。当包含工具更新的 Anthropic 请求中所有投影后的声明均为延迟加载时,初始可用工具为一个占位工具 `DeferredToolPlaceholder`,其描述要求模型永不调用它;若被调用,会得到普通的未知工具结果。Anthropic 在更新消息中以内联方式接收后续工具定义。回放会保留工具调用命名空间,以支持动态加载的 Responses 工具。每张保留图片前都会有文本,注明其完整附件 id 与实际请求尺寸。当前执行文件系统可以映射附件提供方的宿主对象时,该文本还会携带只读规范化对象路径,并警告规范化或请求投影可能缩放或重新编码上传内容。日志中的图片省略决策选中的每个出现位置都会在替换文本中保留自己的身份与当前已解析访问方式,其规范化附件不会读取或变换。当保留的出现位置数量仍超过路由的 `maxRequestImages`,或其精确 base64 载荷超过 `maxRequestImageBytes` 时,调用以 `IMAGE_OFFLOAD_REQUIRED` 失败,由 `dsh-compaction-image-offload` 用 `image/offload` 事件记录所选位置并重试步骤。提供方原生回放元数据只在适配器针对历史内容校验通过后恢复。 `imageRequestTarget(provider, model, image)` 向截图调用方提供与模型请求一致的图片尺寸和字节目标;不缩放图片的适配器保留归一化后的原图。
所选目录模型会收到系统消息文本、对话历史、工具以及支持的采样字段。目录模型声明 `supportsMidConvoSystemMessages` 时,适配器按历史顺序保留后续系统消息。否则,适配器在传给 pi-ai 之前,将每条非空的后续系统消息转换为以 `<system-reminder>` 包裹的用户消息。包裹文本不会赋予系统角色优先级。单次调用的 `GenerateOptions.system` 放在初始历史提示词之前。重复的系统消息仍会重复发送;空系统消息会被跳过,不会丢弃之前的文本。工具更新能力选择共享的历史声明投影;pi-ai 接收初始可用工具,以及按开发者消息分组的新增和受支持的移除更新。当包含工具更新的 Anthropic 请求中所有投影后的声明均为延迟加载时,初始可用工具为一个占位工具 `DeferredToolPlaceholder`,其描述要求模型永不调用它;若被调用,会得到普通的未知工具结果。Anthropic 在更新消息中以内联方式接收后续工具定义。回放会保留工具调用命名空间,以支持动态加载的 Responses 工具。每张保留图片前都会有文本,注明其完整附件 id 与实际请求尺寸。当前执行文件系统可以映射附件提供方的宿主对象时,该文本还会携带只读规范化对象路径,并警告规范化或请求投影可能缩放或重新编码上传内容。日志中的图片省略决策选中的每个出现位置都会在替换文本中保留自己的身份与当前已解析访问方式,其规范化附件不会读取或变换。当保留的出现位置数量仍超过路由的 `maxRequestImages`,或其精确 base64 载荷超过 `maxRequestImageBytes` 时,调用以 `IMAGE_OFFLOAD_REQUIRED` 失败,由 `dsh-compaction-image-offload` 用 `image/offload` 事件记录所选位置并重试步骤。提供方原生回放元数据只在适配器针对历史内容校验通过后恢复。在 Anthropic Messages 路由上,如果回放响应的请求前缀自生成后发生变化(例如经过压缩),该响应会丢失 thinking 块;其文本和工具调用保留。 `imageRequestTarget(provider, model, image)` 向截图调用方提供与模型请求一致的图片尺寸和字节目标;不缩放图片的适配器保留归一化后的原图。
#### Token 影响
@@ -228,7 +231,7 @@ git add packages/llm/llm-pi-ai/src/provider-data.generated.ts
#### KV Cache 影响
传给适配器的后续系统消息不会进入初始提示词,从而保留此前的请求前缀。这不改变 Agent loop 的提示词纳入行为;它仍在不支持原生系统更新的路由上合并更新。转换保持逻辑请求顺序,图片句柄与卸载占位符则会添加模型可见文本。即使附件身份与请求字节保持稳定,执行世界路径变化也会改写历史句柄,并可能从该图片起阻止复用。更换适配器实例、提供方、模型或其他上游 token 具有相同的后缀影响。一次省略决策会把较早图片换成占位文本,因此复用在该消息处结束;省略永不回退,在下一次决策前前缀保持稳定。数量超限时一次省略 `imageOffloadCountQuantum` 张图片,因此饱和路由每新增这么多张图片才让复用在最老的保留图片处结束一次。
传给适配器的后续系统消息不会进入初始提示词,从而保留此前的请求前缀。这不改变 Agent loop 的提示词纳入行为;它仍在不支持原生系统更新的路由上合并更新。转换保持逻辑请求顺序,图片句柄与卸载占位符则会添加模型可见文本。即使附件身份与请求字节保持稳定,执行世界路径变化也会改写历史句柄,并可能从该图片起阻止复用。更换适配器实例、提供方、模型或其他上游 token 具有相同的后缀影响。省略过期的 Anthropic thinking 发生在前缀变化之后,而该变化已在变化的消息处结束复用。第一个省略无摘要记录的 thinking 的请求会在第一条此类消息处结束复用,且仅发生一次。一次省略决策会把较早图片换成占位文本,因此复用在该消息处结束;省略永不回退,在下一次决策前前缀保持稳定。数量超限时一次省略 `imageOffloadCountQuantum` 张图片,因此饱和路由每新增这么多张图片才让复用在最老的保留图片处结束一次。
### 提供方响应
@@ -270,6 +273,8 @@ pi-ai 事件变成 harness 的推理、文本、工具调用、用量与 finish
- **提供方 HTTP 状态不可用**——pi-ai 错误事件不跨提供方暴露稳定 HTTP 状态。
- **重试策略由提供方自有,而非 SDK 重试**——pi-ai SDK 重试保持禁用,因此持久 agent(智能体)步骤与 `llm/retry` 事件拥有每个可见尝试,直接 `ctx.llm.stream()` 调用仍是单次尝试。
- **流式 JSON 工具调用参数避免逐 delta 重复解析**——安装的 pi-ai 带有 [`patches/@[email protected]`](../../../patches/@[email protected]),它移除了每个流适配器中对整段累计参数 JSON 的逐 delta 重新解析(上游 [earendil-works/pi#9265](https://github.com/earendil-works/pi/issues/9265));未打补丁时,数 MB 的参数流会在事件循环上消耗 O(n²) CPU,并使进程内所有会话停滞。完成事件提供解析后的参数;Responses 可能在参数完成和输出项完成时分别解析。本适配器只读取 delta 字符串与最终参数。每次升级 pi-ai 时都要重新应用或撤销该补丁。
- **即使端点会接受,过期的 Anthropic thinking 也会被省略**——前缀检查取决于模型、账户和请求,而适配器无法观测这些因素,因此每次前缀变化以及每个不含 `prefixDigest` 的回放状态都会让模型失去这部分推理,Anthropic 会以输出 token 重新生成它。摘要覆盖 pi-ai 上下文对象,而不是线上请求体:对话中途切换认证或会改变线上请求的 compat 设置不会被检测到;改变 Anthropic 消息序列化的 pi-ai 升级需要在 [`src/prefix-binding.ts`](src/prefix-binding.ts) 中使用新的 `DIGEST_SEED`;本包构建这些对象的方式发生任何变化,都会使已记录的 thinking 被丢弃一次。
- **跨越前缀变化的手动思考工具轮次仍可能被拒绝**——其未结束的 assistant 消息必须保留过期的 thinking 以满足手动思考要求,因此执行前缀检查的端点会拒绝每个请求,直到一条用户消息结束该轮次。在已安装的目录中,执行此检查的模型使用自适应思考。
- **Chat Completions 中的 Gemini 工具调用思考签名由补丁保留**——同一补丁把 OpenAI 兼容流式响应里的 `tool_calls[].extra_content.google.thought_signature` 读入工具调用的重放签名,并在同一模型继续时以 `extra_content` 发回(上游 [earendil-works/pi#10157](https://github.com/earendil-works/pi/issues/10157));未打补丁时,Gemini 会以 HTTP 400 拒绝工具调用轮次的下一步。OpenRouter 的加密推理详情仍通过 `reasoning_details` 重放。
<a id="dev-note"></a>
+19 -4
View File
@@ -62,6 +62,8 @@ import type { ResolvedPiAiProviderProfile } from './config.ts'
import { requestImageTarget, toPiContext } from './context.ts'
import { conversationUpdates, createModels, getSupportedThinkingLevels } from './models.ts'
import { toStreamChunks } from './stream.ts'
import { bindThinkingToPrefix } from './prefix-binding.ts'
import type { PrefixDigests } from './prefix-binding.ts'
import type { PiAiRequestHeaderContext } from './types.ts'
/**
@@ -126,6 +128,13 @@ export interface PiAiAdapterOptions {
* conversion because its stored replay state is unusable by this build.
*/
onReplayDegrade?: (detail: { provider: string; model: string; reason: string }) => void
/**
* Observe one request that sent replayed assistant messages without their
* signed Anthropic thinking because the request prefix changed or was never
* recorded. Stale messages stay omitted, so later requests of the
* conversation report them again.
*/
onThinkingOmitted?: (detail: { provider: string; model: string; count: number }) => void
}
/** The two auth injectables a pi-ai collection is built with. */
@@ -444,8 +453,9 @@ export class PiAiAdapter extends LlmAdapter {
const onReplayDegrade = (reason: string): void => {
this.config.onReplayDegrade?.({ provider: options.provider, model: options.model, reason })
}
const context = attachments === undefined
? toPiContext(request, undefined, onReplayDegrade, updates.systemPromptUpdate === 'in-history')
const prefixDigests: PrefixDigests = new Map()
const converted = attachments === undefined
? toPiContext(request, undefined, onReplayDegrade, updates.systemPromptUpdate === 'in-history', prefixDigests)
: await toPiContext({ ...request, signal: watchdog.signal }, {
attachments,
resolveImageAccess: ref => this.config.resolveImageAccess?.(attachments, ref),
@@ -456,7 +466,12 @@ export class PiAiAdapter extends LlmAdapter {
maxPixels: profile.requestImagePixelBudget,
maxBytes: profile.requestImageMaxBytes,
},
}, onReplayDegrade, updates.systemPromptUpdate === 'in-history')
}, onReplayDegrade, updates.systemPromptUpdate === 'in-history', prefixDigests)
const manualThinking = model.reasoning && reasoning !== undefined && reasoning !== 'off'
&& (model as Model<'anthropic-messages'>).compat?.forceAdaptiveThinking !== true
const { context, digest } = bindThinkingToPrefix(converted, model.api, prefixDigests, manualThinking, (count) => {
this.config.onThinkingOmitted?.({ provider: options.provider, model: options.model, count })
})
const events = snapshot.models.streamSimple(model, context, {
...profileOptions(profile, reasoning, apiKey),
...options.temperature === undefined ? {} : { temperature: options.temperature },
@@ -465,7 +480,7 @@ export class PiAiAdapter extends LlmAdapter {
signal: watchdog.signal,
headers: requestHeaders(profile.headers, options, model.baseUrl, this.config.prepareHeaders),
})
const iterator = toStreamChunks(events, model.contextWindow, options.signal, model.id)[Symbol.asyncIterator]()
const iterator = toStreamChunks(events, model.contextWindow, options.signal, model.id, digest)[Symbol.asyncIterator]()
let exhausted = false
try {
while (true) {
+16 -6
View File
@@ -16,6 +16,7 @@ import type {
} from '@deepseek-ai/dsh-attachment'
import type { Context as PiContext, ImageContent, Message as PiMessage, SystemMessage, TextContent, Tool as PiTool } from '@earendil-works/pi-ai'
import { toPiAssistant } from './replay.ts'
import type { PrefixDigests } from './prefix-binding.ts'
import { requestImageDimensions } from '@deepseek-ai/dsh-attachment'
import { DEFAULT_REQUEST_IMAGE_MAX_BYTES, DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET } from './config.ts'
@@ -205,8 +206,9 @@ function appendAssistant(
messages: PiMessage[],
toolNames: Map<ToolCallId, string>,
onReplayDegrade?: (reason: string) => void,
prefixDigests?: PrefixDigests,
): void {
const assistant = toPiAssistant(message, onReplayDegrade)
const assistant = toPiAssistant(message, onReplayDegrade, prefixDigests)
for (const block of assistant.content) {
if (block.type === 'toolCall') toolNames.set(brandString<ToolCallId>(block.id), block.name)
}
@@ -221,6 +223,7 @@ function appendSharedHistory(
toolNames: Map<ToolCallId, string>,
supportsMidConvoSystemMessages: boolean,
onReplayDegrade?: (reason: string) => void,
prefixDigests?: PrefixDigests,
): boolean {
if (message.role === 'developer') {
appendDeveloper(message, options, messages)
@@ -239,7 +242,7 @@ function appendSharedHistory(
return true
}
if (message.role === 'assistant') {
appendAssistant(message, messages, toolNames, onReplayDegrade)
appendAssistant(message, messages, toolNames, onReplayDegrade, prefixDigests)
return true
}
return false
@@ -249,6 +252,7 @@ function textOnlyContext(
options: GenerateOptions,
supportsMidConvoSystemMessages: boolean,
onReplayDegrade?: (reason: string) => void,
prefixDigests?: PrefixDigests,
): PiContext {
assertSupportedHistory(options.messages)
const split = splitSystemPrompt(options)
@@ -258,7 +262,7 @@ function textOnlyContext(
if (contentHasImage(message.content)) {
throw new LlmError('pi-ai image conversion requires the durable attachment service', 'UNSUPPORTED_CONTENT')
}
if (appendSharedHistory(message, options, messages, toolNames, supportsMidConvoSystemMessages, onReplayDegrade)) continue
if (appendSharedHistory(message, options, messages, toolNames, supportsMidConvoSystemMessages, onReplayDegrade, prefixDigests)) continue
if (message.role === 'tool') {
messages.push(toolResultOf(message, toolNames, flattenText(message)))
continue
@@ -309,6 +313,7 @@ export function requestImageTarget(ref: ImageAttachmentRef, budget: PiImageReque
* @param images - absent; selects the synchronous conversion.
* @param onReplayDegrade - forwarded to {@link toPiAssistant} for each assistant message.
* @param supportsMidConvoSystemMessages - native model capability; otherwise later system text becomes a user-role system-reminder.
* @param prefixDigests - receives the recorded request-prefix digest of each replayed assistant message that has one.
* @returns the pi-ai context; `tools` is omitted when the request declares none.
* @throws {LlmError} `UNSUPPORTED_CONTENT` for images in any history role, including a leading system message.
*/
@@ -317,6 +322,7 @@ export function toPiContext(
images?: undefined,
onReplayDegrade?: (reason: string) => void,
supportsMidConvoSystemMessages?: boolean,
prefixDigests?: PrefixDigests,
): PiContext
/**
* Convert harness history to a pi-ai Context while resolving durable images.
@@ -330,6 +336,7 @@ export function toPiContext(
* @param images - attachment provider, current path resolver, and request limits.
* @param onReplayDegrade - forwarded to {@link toPiAssistant} for each assistant message.
* @param supportsMidConvoSystemMessages - native model capability; otherwise later system text becomes a user-role system-reminder.
* @param prefixDigests - receives the recorded request-prefix digest of each replayed assistant message that has one.
* @returns the asynchronously resolved pi-ai context.
*/
export function toPiContext(
@@ -337,16 +344,18 @@ export function toPiContext(
images: PiImageRequestContext,
onReplayDegrade?: (reason: string) => void,
supportsMidConvoSystemMessages?: boolean,
prefixDigests?: PrefixDigests,
): Promise<PiContext>
export function toPiContext(
options: GenerateOptions,
images?: PiImageRequestContext,
onReplayDegrade?: (reason: string) => void,
supportsMidConvoSystemMessages = false,
prefixDigests?: PrefixDigests,
): PiContext | Promise<PiContext> {
return images === undefined
? textOnlyContext(options, supportsMidConvoSystemMessages, onReplayDegrade)
: toPiContextWithImages(options, images, supportsMidConvoSystemMessages, onReplayDegrade)
? textOnlyContext(options, supportsMidConvoSystemMessages, onReplayDegrade, prefixDigests)
: toPiContextWithImages(options, images, supportsMidConvoSystemMessages, onReplayDegrade, prefixDigests)
}
async function toPiContextWithImages(
@@ -354,6 +363,7 @@ async function toPiContextWithImages(
images: PiImageRequestContext,
supportsMidConvoSystemMessages: boolean,
onReplayDegrade?: (reason: string) => void,
prefixDigests?: PrefixDigests,
): Promise<PiContext> {
const { attachments, resolveImageAccess, maxRequestImageBytes, maxRequestImages, imageOffloadCountQuantum } = images
const requestImagePolicy = images.requestImagePolicy ?? {
@@ -388,7 +398,7 @@ async function toPiContextWithImages(
const messages: PiMessage[] = []
for (const message of exactMessages) {
if (appendSharedHistory(message, options, messages, toolNames, supportsMidConvoSystemMessages, onReplayDegrade)) continue
if (appendSharedHistory(message, options, messages, toolNames, supportsMidConvoSystemMessages, onReplayDegrade, prefixDigests)) continue
if (message.role === 'tool') {
messages.push(toolResultOf(message, toolNames, userContent(message.content, requestImages, resolveImageAccess)))
continue
+3
View File
@@ -252,6 +252,9 @@ export function apply(ctx: Context, config: Config): void {
+ ` sending that message as provider-neutral content (${reason})`,
)
},
onThinkingOmitted: ({ provider, model, count }) => {
ctx.logger.debug(`llm-pi-ai: omitted stale thinking from ${count} assistant message(s) for route "${provider}/${model}"`)
},
})
// Independent of the route set: signing in is what makes a route worth
// adding, so the flows are offered before any profile names their provider.
@@ -0,0 +1,110 @@
/**
* Request-prefix binding for replayed Anthropic thinking.
*
* Anthropic can bind each signed thinking block to the request prefix it was
* generated under: system prompt, tools, and every earlier message. Compaction
* and tool-result pruning rewrite that prefix, so a retained signed block
* becomes invalid and endpoints that enforce the check reject the whole
* request. Each Anthropic Messages response records a digest of the context it
* was generated from; a later request replays its signed thinking only while
* the reconstructed prefix still has the same digest. Replay state without a
* digest predates this check and is treated as stale.
*
* The digest covers the pi-ai context objects this package builds, including
* fields Anthropic never receives. Any change to their construction
* invalidates every recorded digest once; a pi-ai upgrade that changes how
* Anthropic messages are serialized without changing those objects requires a
* new {@link DIGEST_SEED}.
*
* @module dsh-llm-pi-ai/prefix-binding
*/
import { createHash } from 'node:crypto'
import type { Api, AssistantMessage, Context as PiContext } from '@earendil-works/pi-ai'
/** Recorded request-prefix digests of reconstructed assistant messages, owned by one context conversion. */
export type PrefixDigests = Map<AssistantMessage, string>
/** Wire protocol whose thinking signatures bind to the request prefix. */
const PREFIX_BOUND_API = 'anthropic-messages'
/** Seed of every digest chain; changing the digest input or its serialization requires a new seed. */
const DIGEST_SEED = 'dsh-llm-pi-ai/prefix-binding/v1'
function chain(previous: string, value: unknown): string {
return createHash('sha256').update(previous).update('\0').update(JSON.stringify(value)).digest('hex')
}
/** Index of the assistant message whose tool round the request continues with tool results, or -1. */
function openToolRoundAssistant(messages: PiContext['messages']): number {
let results = 0
for (let index = messages.length - 1; index >= 0; index--) {
const message = messages[index] as PiContext['messages'][number]
if (message.role === 'assistant') return results > 0 ? index : -1
if (message.role !== 'toolResult') return -1
results++
}
return -1
}
/** Whether a thinking block carries a signature or redacted payload that binds it to its prefix. */
function signed(block: AssistantMessage['content'][number]): boolean {
return block.type === 'thinking' && (block.redacted === true || (block.thinkingSignature ?? '').length > 0)
}
/** Copy of an assistant message without its signed thinking blocks. */
function withoutSignedThinking(message: AssistantMessage): AssistantMessage {
return { ...message, content: message.content.filter(block => !signed(block)) }
}
/** Context whose stale thinking is removed, with the digest of the complete request. */
export interface PrefixBoundContext {
/** Context to send; equal to the input when nothing was removed. */
context: PiContext
/** Digest of the sent context, recorded with the response; absent for other protocols. */
digest?: string
}
/**
* Remove signed thinking from replayed Anthropic Messages assistant messages
* whose request prefix changed since they were generated, or whose replay
* state records no prefix digest. Manual thinking requires the final
* assistant turn of a tool round to start with a thinking block, so with
* `manualThinking` the assistant message whose tool round the request
* continues keeps its thinking.
* @param context - converted request context.
* @param api - wire protocol of the requested model.
* @param prefixDigests - recorded digests the context conversion collected.
* @param manualThinking - whether the request enables budget-based rather than adaptive thinking.
* @param onOmitted - called once with the number of messages whose thinking was removed, when there are any.
* @returns the context to send and, for Anthropic Messages, its digest.
*/
export function bindThinkingToPrefix(
context: PiContext,
api: Api,
prefixDigests: PrefixDigests,
manualThinking: boolean,
onOmitted: (count: number) => void,
): PrefixBoundContext {
if (api !== PREFIX_BOUND_API) return { context }
const kept = manualThinking ? openToolRoundAssistant(context.messages) : -1
let digest = chain(DIGEST_SEED, { systemPrompt: context.systemPrompt ?? null, tools: context.tools ?? null })
let messages: PiContext['messages'] | undefined
let omitted = 0
for (const [index, original] of context.messages.entries()) {
let message = original
if (
message.role === 'assistant' && message.api === PREFIX_BOUND_API && index !== kept
&& prefixDigests.get(message) !== digest && message.content.some(signed)
) {
message = withoutSignedThinking(message)
messages ??= context.messages.slice()
messages[index] = message
omitted++
}
digest = chain(digest, message)
}
if (messages === undefined) return { context, digest }
onOmitted(omitted)
return { context: { ...context, messages }, digest }
}
+23 -5
View File
@@ -11,6 +11,7 @@
import { LlmError } from '@deepseek-ai/dsh-llm'
import type { AssistantMessage as HarnessAssistantMessage, ModelMessageSource, ReplayEnvelope } from '@deepseek-ai/dsh-llm'
import type { Api, AssistantMessage, ToolCall, Usage as PiUsage } from '@earendil-works/pi-ai'
import type { PrefixDigests } from './prefix-binding.ts'
/** Per-block half of the pi-ai replay envelope, one entry per content block. */
export type PiAiReplayBlock =
@@ -32,6 +33,8 @@ export interface PiAiReplayResponse {
/** Provider-native effort for historical replay; absence is preserved. */
providerThinkingLevel?: string
stopReason: AssistantMessage['stopReason']
/** Digest of the request context this response was generated from; see `prefix-binding.ts`. */
prefixDigest?: string
}
/** The validated halves of one pi-ai replay envelope. */
@@ -72,9 +75,10 @@ function emptyPiUsage(): PiUsage {
* removes one.
* @param message - completed native pi-ai assistant response.
* @param requestedModel - request identity stored in the assistant source; defaults to the native model.
* @param prefixDigest - digest of the request context, recorded when the protocol binds thinking to it.
* @returns the versioned lossless-JSON replay projection.
*/
export function toPiReplayState(message: AssistantMessage, requestedModel = message.model): ReplayEnvelope {
export function toPiReplayState(message: AssistantMessage, requestedModel = message.model, prefixDigest?: string): ReplayEnvelope {
const response: PiAiReplayResponse = {
kind: 'pi-ai',
version: 2,
@@ -85,6 +89,7 @@ export function toPiReplayState(message: AssistantMessage, requestedModel = mess
...message.responseId === undefined ? {} : { responseId: message.responseId },
...message.providerThinkingLevel === undefined ? {} : { providerThinkingLevel: message.providerThinkingLevel },
stopReason: message.stopReason,
...prefixDigest === undefined ? {} : { prefixDigest },
}
return {
response,
@@ -131,6 +136,7 @@ function readReplayState(value: unknown): PiAiReplayState {
if (response['responseModel'] !== undefined && typeof response['responseModel'] !== 'string') return invalidReplay('responseModel must be a string')
if (response['responseId'] !== undefined && typeof response['responseId'] !== 'string') return invalidReplay('responseId must be a string')
if (response['providerThinkingLevel'] !== undefined && typeof response['providerThinkingLevel'] !== 'string') return invalidReplay('providerThinkingLevel must be a string')
if (response['prefixDigest'] !== undefined && typeof response['prefixDigest'] !== 'string') return invalidReplay('prefixDigest must be a string')
const blocks = envelope['blocks']
if (!Array.isArray(blocks)) return invalidReplay('blocks must be an array')
for (const [index, value] of blocks.entries()) {
@@ -184,7 +190,12 @@ function foreignAssistant(message: HarnessAssistantMessage): AssistantMessage {
}
/** Recombine durable Harness content with validated pi-ai replay metadata. */
function replayedAssistant(message: HarnessAssistantMessage, source: ModelMessageSource, rawState: unknown): AssistantMessage {
function replayedAssistant(
message: HarnessAssistantMessage,
source: ModelMessageSource,
rawState: unknown,
prefixDigests?: PrefixDigests,
): AssistantMessage {
const state = readReplayState(rawState)
if (state.response.provider !== source.provider) return invalidReplay('provider does not match assistant source')
if (state.response.model !== source.model) return invalidReplay('model does not match assistant source')
@@ -216,7 +227,7 @@ function replayedAssistant(message: HarnessAssistantMessage, source: ModelMessag
default: return invalidReplay(`block ${index} has an unsupported Harness type`)
}
})
return {
const assistant: AssistantMessage = {
role: 'assistant',
content,
api: state.response.api,
@@ -229,6 +240,8 @@ function replayedAssistant(message: HarnessAssistantMessage, source: ModelMessag
stopReason: state.response.stopReason,
timestamp: 0,
}
if (state.response.prefixDigest !== undefined) prefixDigests?.set(assistant, state.response.prefixDigest)
return assistant
}
/**
@@ -242,13 +255,18 @@ function replayedAssistant(message: HarnessAssistantMessage, source: ModelMessag
* @param message - model-produced assistant content with provider, model, and optional adapter-owned replay metadata.
* @param onDegrade - called with the diagnostic reason when an unusable replay
* state falls back to provider-neutral conversion.
* @param prefixDigests - receives the recorded request-prefix digest of the returned message, when replay state has one.
* @returns a native pi-ai assistant message reconstructed from durable content.
*/
export function toPiAssistant(message: HarnessAssistantMessage, onDegrade?: (reason: string) => void): AssistantMessage {
export function toPiAssistant(
message: HarnessAssistantMessage,
onDegrade?: (reason: string) => void,
prefixDigests?: PrefixDigests,
): AssistantMessage {
const source = message.source
if (source.replayState === undefined) return foreignAssistant(message)
try {
return replayedAssistant(message, source, source.replayState)
return replayedAssistant(message, source, source.replayState, prefixDigests)
} catch (error: unknown) {
/* v8 ignore next -- replayedAssistant throws only INVALID_REPLAY_STATE LlmErrors; the
guard keeps a future non-replay failure loud instead of silently degrading it */
+3 -1
View File
@@ -136,6 +136,7 @@ export function mapStopReason(message: AssistantMessage, contextWindow?: number)
* @param callerSignal - caller cancellation state; an aborted caller makes any
* in-band terminal error an aborted finish.
* @param requestedModel - request model identity recorded for durable replay.
* @param prefixDigest - request-context digest recorded for durable replay.
* @returns the harness chunks, ending with `usage` then `finish`; throws
* `LlmError` (`STREAM_CLOSED`) if the source ends without a terminal event, or
* `MALFORMED_RESPONSE` when completed tool arguments are invalid. Length-limited
@@ -146,6 +147,7 @@ export async function* toStreamChunks(
contextWindow?: number,
callerSignal?: AbortSignal,
requestedModel?: string,
prefixDigest?: string,
): AsyncGenerator<StreamChunk> {
// pi-ai contentIndex ↔ our block index map 1:1 (both count blocks from 0
// in stream order), but we track ids per index for tool calls.
@@ -220,7 +222,7 @@ export async function* toStreamChunks(
yield {
type: 'finish',
reason: mapStopReason(event.message, contextWindow),
replayState: toPiReplayState(event.message, requestedModel),
replayState: toPiReplayState(event.message, requestedModel, prefixDigest),
}
return
case 'error':
@@ -670,6 +670,7 @@ describe('toPiContext', () => {
['non-string response id', { ...validReplay, response: { ...validResponse, responseId: 1 } }, 'responseId must be a string'],
['non-string provider thinking level', { ...validReplay, response: { ...validResponse, providerThinkingLevel: 1 } }, 'providerThinkingLevel must be a string'],
['null provider thinking level', { ...validReplay, response: { ...validResponse, providerThinkingLevel: null } }, 'providerThinkingLevel must be a string'],
['non-string prefix digest', { ...validReplay, response: { ...validResponse, prefixDigest: 1 } }, 'prefixDigest must be a string'],
['missing blocks', { response: validResponse }, 'blocks must be an array'],
['non-array blocks', { ...validReplay, blocks: 'text' }, 'blocks must be an array'],
['number block', { ...validReplay, blocks: [1] }, 'block 0 must be an object'],
@@ -0,0 +1,225 @@
/** Replayed Anthropic thinking is bound to the request prefix it was generated under. */
import { afterEach, describe, expect, it, vi } from 'vitest'
import { Context } from '@deepseek-ai/cordis'
import LlmRuntime, { createToolResultMessage, createUserMessage } from '@deepseek-ai/dsh-llm'
import type { Message, ToolCallId } from '@deepseek-ai/dsh-llm'
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
import { resolveProfiles } from '../src/config.ts'
import { memoryAuth } from './auth-double.ts'
import { assemble } from './assemble.ts'
import { anthropicFrame, closeMockServers, mockServer } from './mock-server.ts'
const contexts: Context[] = []
afterEach(async () => {
await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose()))
await closeMockServers()
vi.unstubAllEnvs()
})
const tool = { name: 'lookup', description: 'Look up a value.', parameters: { type: 'object', properties: { key: { type: 'number' } } } }
/** One Anthropic response: signed and redacted thinking, then a tool call or a reply. */
function frames(signature: string, toolUse: boolean): ReturnType<typeof anthropicFrame>[] {
return response(signature, [
{ type: 'thinking', thinking: `thinking ${signature}`, signature },
{ type: 'redacted_thinking', data: `redacted-${signature}` },
toolUse ? { type: 'tool_use', id: `toolu_${signature}`, name: 'lookup', input: { key: 1 } } : { type: 'text', text: 'done' },
], toolUse)
}
function response(signature: string, blocks: Record<string, unknown>[], toolUse = false): ReturnType<typeof anthropicFrame>[] {
return [
anthropicFrame({ type: 'message_start', message: {
id: `msg_${signature}`, type: 'message', role: 'assistant', model: 'model', content: [],
stop_reason: null, stop_sequence: null, usage: { input_tokens: 3, output_tokens: 1 },
} }),
...blocks.flatMap((content_block, index) => [
anthropicFrame({ type: 'content_block_start', index, content_block }),
anthropicFrame({ type: 'content_block_stop', index }),
]),
anthropicFrame({ type: 'message_delta', delta: { stop_reason: toolUse ? 'tool_use' : 'end_turn', stop_sequence: null }, usage: { output_tokens: 1 } }),
anthropicFrame({ type: 'message_stop' }),
]
}
async function setup(api: 'anthropic-messages' | 'openai-completions', script: Parameters<typeof mockServer>[0], adaptive = true, manual = false) {
vi.stubEnv('PREFIX_BINDING_KEY', 'test-key')
const server = await mockServer(script)
const ctx = new Context()
contexts.push(ctx)
await ctx.plugin(LlmRuntime)
await ctx.plugin(LlmPiAi, { providers: {
route: { api, baseURL: server.url, apiKeyEnv: 'PREFIX_BINDING_KEY', cacheRetention: 'none', models: [{ id: 'model', ...manual ? { reasoningEfforts: { high: 'high' } } : {} }],
...manual ? { reasoning: 'high' as const } : {},
...api === 'anthropic-messages' ? { compat: { forceAdaptiveThinking: adaptive } } : {} },
} })
const run = async (messages: Message[]) => {
const result = await assemble(ctx, { provider: 'route', model: 'model', messages, tools: [tool], system: 'system' })
return JSON.parse(JSON.stringify(result.message)) as Message
}
return { server, run }
}
function user(text: string): Message {
return createUserMessage({ source: { kind: 'user' }, content: [{ type: 'text', text }] })
}
function result(callId: ToolCallId, text: string): Message {
return JSON.parse(JSON.stringify(createToolResultMessage({ callId, content: [{ type: 'text', text }], isError: false }))) as Message
}
function callId(message: Message): ToolCallId {
const call = message.content.find(block => block.type === 'tool-call')
if (call === undefined) throw new Error('expected a tool call')
return call.id
}
function wire(request: unknown): string {
return JSON.stringify((request as { messages: unknown }).messages)
}
describe('Anthropic thinking prefix binding', () => {
it('keeps thinking while its prefix is unchanged and omits it after the prefix is rewritten', async () => {
const { server, run } = await setup('anthropic-messages', [
{ events: frames('first', true) },
{ events: frames('second', true) },
{ events: frames('third', false) },
{ events: frames('fourth', false) },
{ events: frames('fifth', false) },
])
const first = await run([user('Look up the value.')])
const prefixDigest: unknown = expect.stringMatching(/^[0-9a-f]{64}$/)
expect(first.source).toMatchObject({ replayState: { response: { prefixDigest } } })
const second = await run([user('Look up the value.'), first, result(callId(first), 'large output')])
expect(wire(server.requests[1])).toContain('"signature":"first"')
expect(wire(server.requests[1])).toContain('redacted-first')
// Pruning the first tool result rewrites the second response's prefix but not the first one's.
const tail = [first, result(callId(first), 'pruned'), second, result(callId(second), 'value')]
const third = await run([user('Look up the value.'), ...tail])
expect(wire(server.requests[2])).toContain('"signature":"first"')
expect(wire(server.requests[2])).not.toContain('"signature":"second"')
expect(wire(server.requests[2])).not.toContain('redacted-second')
expect(wire(server.requests[2])).toContain('toolu_second')
// Compaction replaces the head: every retained response loses its thinking, tool calls stay paired.
const compacted = [user('<compacted-summary>'), ...tail, third, user('Continue.')]
const fourth = await run(compacted)
const request = wire(server.requests[3])
for (const stale of ['"signature":"first"', 'redacted-first', '"signature":"second"', '"signature":"third"']) {
expect(request).not.toContain(stale)
}
expect(request).toContain('toolu_first')
expect(request).toContain('toolu_second')
// A response generated on the compacted prefix replays its own thinking.
await run([...compacted, fourth, user('Again.')])
expect(wire(server.requests[4])).toContain('"signature":"fourth"')
expect(wire(server.requests[4])).not.toContain('"signature":"third"')
})
it('treats a changed system prompt as a changed prefix', async () => {
const { server, run } = await setup('anthropic-messages', [{ events: frames('first', true) }, { events: frames('second', false) }])
const first = await run([user('Look up the value.')])
const ctx = contexts[0] as Context
await assemble(ctx, {
provider: 'route', model: 'model', tools: [tool], system: 'changed system',
messages: [user('Look up the value.'), first, result(callId(first), 'value')],
})
expect(wire(server.requests[1])).not.toContain('"signature":"first"')
expect(wire(server.requests[1])).toContain('toolu_first')
})
it('omits thinking whose replay state records no prefix digest', async () => {
const { server, run } = await setup('anthropic-messages', [{ events: frames('first', true) }, { events: frames('second', false) }])
const first = await run([user('Look up the value.')])
const replayState = (first.source as { replayState: { response: Record<string, unknown> } }).replayState
delete replayState.response['prefixDigest']
await run([user('Look up the value.'), first, result(callId(first), 'value')])
expect(wire(server.requests[1])).not.toContain('"signature":"first"')
expect(wire(server.requests[1])).toContain('toolu_first')
})
it('keeps stale thinking only on the open tool round of a manual-thinking request', async () => {
const { server, run } = await setup('anthropic-messages', [
{ events: frames('first', true) },
{ events: frames('second', true) },
{ events: frames('third', false) },
{ events: frames('fourth', false) },
], false, true)
const first = await run([user('Look up the value.')])
const second = await run([user('Look up the value.'), first, result(callId(first), 'value')])
await run([user('<compacted-summary>'), first, result(callId(first), 'value'), second, result(callId(second), 'value')])
expect(wire(server.requests[2])).not.toContain('"signature":"first"')
expect(wire(server.requests[2])).toContain('"signature":"second"')
expect(server.requests[2]).toMatchObject({ thinking: { type: 'enabled' } })
// Once a user message closes the round, the formerly kept thinking is stale too.
const compacted = [user('<compacted-summary>'), first, result(callId(first), 'value'), second, result(callId(second), 'value')]
await run([...compacted, user('Continue.')])
expect(wire(server.requests[3])).not.toContain('"signature":"second"')
})
it('sends an empty manual-thinking history unchanged', async () => {
const { server, run } = await setup('anthropic-messages', [{ events: frames('first', false) }], false, true)
await run([])
expect(server.requests[0]).toMatchObject({ messages: [] })
})
it('omits stale manual thinking from a final assistant message without tool results', async () => {
const { server, run } = await setup('anthropic-messages', [{ events: frames('first', false) }, { events: frames('second', false) }], false, true)
const first = await run([user('Answer.')])
await run([user('<compacted-summary>'), first])
expect(wire(server.requests[1])).not.toContain('"signature":"first"')
})
it('omits stale thinking of the open tool round when the request disables thinking', async () => {
const { server, run } = await setup('anthropic-messages', [{ events: frames('first', true) }, { events: frames('second', false) }], false)
const first = await run([user('Look up the value.')])
await run([user('<compacted-summary>'), first, result(callId(first), 'value')])
expect(wire(server.requests[1])).not.toContain('"signature":"first"')
})
it('drops a thinking-only message and keeps unsigned thinking', async () => {
const { server, run } = await setup('anthropic-messages', [
{ events: response('only', [{ type: 'thinking', thinking: 'thinking only', signature: 'only' }]) },
{ events: response('unsigned', [{ type: 'thinking', thinking: 'unsigned thinking', signature: '' }, { type: 'text', text: 'reply' }]) },
{ events: frames('third', false) },
])
const only = await run([user('Think.')])
const unsigned = await run([user('Think.'), only, user('Answer.')])
await run([user('<compacted-summary>'), only, user('Answer.'), unsigned, user('Again.')])
const messages = (server.requests[2] as { messages: { role: string }[] }).messages
expect(messages.map(message => message.role)).toEqual(['user', 'user', 'assistant', 'user'])
expect(wire(server.requests[2])).not.toContain('thinking only')
expect(wire(server.requests[2])).toContain('unsigned thinking')
})
it('omits stale thinking without an omission observer', async () => {
const { run } = await setup('anthropic-messages', [{ events: frames('first', true) }])
const first = await run([user('Look up the value.')])
const server = await mockServer([{ events: frames('second', false) }])
const adapter = new PiAiAdapter({
profiles: () => resolveProfiles({ route: {
api: 'anthropic-messages', baseURL: server.url, apiKeyEnv: 'PREFIX_BINDING_KEY', cacheRetention: 'none',
models: [{ id: 'model' }], compat: { forceAdaptiveThinking: true },
} }),
resolveApiKey: () => Promise.resolve('test-key'),
auth: memoryAuth(),
})
const messages = [user('<compacted-summary>'), first, result(callId(first), 'value')]
for await (const _chunk of adapter.stream({ provider: 'route', model: 'model', messages, tools: [tool], system: 'system' })) { /* drain */ }
expect(wire(server.requests[0])).not.toContain('"signature":"first"')
})
it('records no prefix digest for protocols without prefix binding', async () => {
const { run } = await setup('openai-completions', [{ events: [
JSON.stringify({ choices: [{ index: 0, delta: { role: 'assistant', content: 'reply' }, finish_reason: null }] }),
JSON.stringify({ choices: [{ index: 0, delta: {}, finish_reason: 'stop' }], usage: { prompt_tokens: 3, completion_tokens: 1 } }),
'[DONE]',
] }])
const reply = await run([user('Hello.')])
expect((reply.source as { replayState: { response: object } }).replayState.response).not.toHaveProperty('prefixDigest')
})
})