refactor(session): send canonical events directly
This commit is contained in:
Tianyi Cui committed
1 parent
3c0da7bef7
commit
9c7e142f79
21 files changed
+88
-671
No files matched your search
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-21-deepseek-llm-api-request-extensions.md
|
||||
2026-08-21-deepseek-llm-api-request-extensions.md: 8e4d1e375ecf4169ac8f7f93a3a341e4d4252191
|
||||
2026-08-21-deepseek-llm-api-request-extensions.zh.md: 05d68cdce547ff8bde059fd2d0fe1a1b44af67f0
|
||||
2026-08-21-deepseek-llm-api-request-extensions.md: 018b93115f5376affd86a4da3c76f0f367ba9ed0
|
||||
2026-08-21-deepseek-llm-api-request-extensions.zh.md: 4bc0f0c992445c5897069b68efa47fdba46dfdb4
|
||||
+19
-4
@@ -24,7 +24,7 @@ The provider-neutral `llm` package and `llm-pi-ai` contain no extension type, se
|
||||
|
||||
The failure direction is at least once. A transport or provider rejection records no watermark. A crash after remote acceptance but before the watermark persists causes replay after resume, never a skipped sequence. Existing session checkpoints persist the event; the upload plugin owns no second store.
|
||||
|
||||
Event strings are raw unless exact request-relative references reduce encoded bytes. A reference identifies one serialized DeepSeek message, a path to one parsed string value, and a half-open UTF-8 byte range. The encoder verifies that each candidate range decodes to the exact matched UTF-16 substring; surrogate-splitting and ill-formed UTF-16 candidates stay raw. The decoder reconstructs every literal and cited byte exactly and rejects invalid paths, ranges, or code-point splits. Fuzzy similarity, normalization, and lossy omission are absent.
|
||||
The `events` array contains complete canonical `SessionEvent` objects directly. The sender copies every present event member without projection or redaction; the field is self-contained and requires no reconstruction against `messages`.
|
||||
|
||||
## Plugin package field
|
||||
|
||||
@@ -50,7 +50,7 @@ The process-lifetime manifest-identity cache remains separate because in-process
|
||||
|
||||
## Verification
|
||||
|
||||
Registry tests pin duplicate ownership, effect-scoped disposal, detached field values, concurrent and abortable preparation, receiver-preserving acceptance, one acceptance settlement, and failure aggregation. Session tests pin the default-off policy, explicit full-first/suffix-later delivery, incremental watermark folding, persisted restart recovery, fork identity fencing, out-of-order acceptance, exact Unicode reconstruction, invalid references, surrogate-safe raw fallback, and late invariant loading. Package-inventory tests pin default-on and explicit-off policies, host and standing-preset discovery, conflicting Loader resolution bases, manifest resolution, lifecycle filtering, and exact name/version ordering. The direct adapter mock proves pre-HTTP preparation failure, cancellation, non-2xx non-acceptance, 2xx acceptance before a later stream failure, and field collision. Keyless replay pins post-2xx extension acceptance, and the TypeScript JSON-RPC plus Python packaged-runtime snapshots project the acceptance event through both SDKs. Real Loader composition pins default package metadata plus opt-in Session upload, one real-API request mounts both shipped extensions and proves the official endpoint accepts them, and pi-ai tests retain their unchanged wire requests.
|
||||
Registry tests pin duplicate ownership, effect-scoped disposal, detached field values, concurrent and abortable preparation, receiver-preserving acceptance, one acceptance settlement, and failure aggregation. Session tests pin the default-off policy, explicit full-first/suffix-later delivery, direct complete event envelopes independent of base-body messages, incremental watermark folding, persisted restart recovery, fork identity fencing, out-of-order acceptance, and late invariant loading. Package-inventory tests pin default-on and explicit-off policies, host and standing-preset discovery, conflicting Loader resolution bases, manifest resolution, lifecycle filtering, and exact name/version ordering. The direct adapter mock proves pre-HTTP preparation failure, cancellation, non-2xx non-acceptance, 2xx acceptance before a later stream failure, and field collision. Keyless replay pins post-2xx extension acceptance, and the TypeScript JSON-RPC plus Python packaged-runtime snapshots project the acceptance event through both SDKs. Real Loader composition pins default package metadata plus opt-in Session upload, one real-API request mounts both shipped extensions and proves the official endpoint accepts them, and pi-ai tests retain their unchanged wire requests.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -58,7 +58,22 @@ Registry tests pin duplicate ownership, effect-scoped disposal, detached field v
|
||||
|
||||
**Hard-wire the two producers into `llm-deepseek`.** Rejected because the adapter would import Session, Loader, preset, package-manifest, and cursor logic. The registry keeps transport responsible only for field merge and HTTP acceptance.
|
||||
|
||||
**Use fuzzy message similarity or omit overlapping event data.** Rejected because the receiver could not reconstruct the canonical log. Exact byte references with raw fallback preserve every value.
|
||||
### Why not request-relative message references?
|
||||
|
||||
A recursive tagged representation could replace exact event-string ranges with paths and UTF-8 byte offsets into the containing request's `messages`. Measurement used Node v24.16.0 on macOS arm64 and the three largest available local Zstandard Session artifacts, whose compressed artifact sizes were 2,437,052, 572,602, and 118,811 bytes. Late-enable replay used each final completed request boundary; steady replay covered 411 completed boundaries. The byte counts cover complete minified DeepSeek requests.
|
||||
|
||||
| Replay | Raw JSON | Referenced JSON | Saving | Synchronous encoder time |
|
||||
|---|---:|---:|---:|---:|
|
||||
| Late enable | 29,668,725 B | 27,645,825 B | 6.82% | 500.1 s total |
|
||||
| Steady state | 389,295,815 B | 387,180,848 B | 0.54% | 285.0 s total |
|
||||
|
||||
The three late-enable calls took 470.5, 29.4, and 0.158 seconds. Only 701 of 115,071 events (0.61%) selected references. A hypothetical level-6 whole-request gzip comparison reduced raw request bytes by 89.38% for late enable and 73.42% for steady state; message references added 21.68% and 0.59% respectively after gzip.
|
||||
|
||||
The receiver would also need to traverse the tagged tree, resolve paths into the exact request messages, validate UTF-8 ranges, and reconstruct every referenced event. Even treating that receiver cost as zero, the steady-state byte saving, synchronous sender cost, and dependence on another request field do not justify a versioned wire format.
|
||||
|
||||
### Why not omit assistant chunks or overlapping event data?
|
||||
|
||||
About 98% of the measured real-session events were `assistant/chunk`. Omitting chunks after reference encoding reduced the complete identity JSON by another 84.79% for late enable and 6.49% for steady state, but it prevents lossless canonical-log reconstruction and leaves `assistant/message.sourceEventSeqs` pointing to absent events. Fuzzy or normalized substitutions have the same reconstruction defect.
|
||||
|
||||
**Keep the upload cursor only in memory.** Rejected because a normal process restart would resend the entire Session. A canonical acceptance event makes restart recovery best-effort durable without another storage backend; the remaining crash window produces allowed duplicates.
|
||||
|
||||
@@ -70,7 +85,7 @@ Registry tests pin duplicate ownership, effect-scoped disposal, detached field v
|
||||
|
||||
## Consequences
|
||||
|
||||
Official DeepSeek requests carry active package versions to their resolved `baseURL`, including configured gateways. An explicit Session-log opt-in also carries the complete newly unaccepted Session suffix. The fields are model-hidden and add no prompt tokens or KV-cache changes, but can substantially increase HTTP body size. Request-relative packing synchronously compares suffix and message strings before dispatch, so a large first upload or retry backlog can delay the event loop until candidate indexing is implemented. Encoding, manifest resolution, field collision, acceptance logging, or provider schema rejection fails the model request rather than silently dropping metadata.
|
||||
Official DeepSeek requests carry active package versions to their resolved `baseURL`, including configured gateways. An explicit Session-log opt-in also carries the complete newly unaccepted Session suffix. The fields are model-hidden and add no prompt tokens or KV-cache changes, but can substantially increase HTTP body size. Manifest resolution, field collision, acceptance logging, or provider schema rejection fails the model request rather than silently dropping metadata.
|
||||
|
||||
The `delivery-accepted` event becomes part of the canonical log and is itself delivered on a later request. Crash recovery can duplicate a suffix but does not infer acceptance from assistant output or create a second local cursor store. Direct calls without a live Session omit the session field; host package inventory remains available.
|
||||
|
||||
|
||||
+19
-4
@@ -24,7 +24,7 @@ Status: implemented
|
||||
|
||||
失败方向为至少一次。传输失败或提供方拒绝不会记录水位。远端接受后、水位持久化前发生崩溃，会在恢复后触发重放，绝不会跳过序列。现有会话检查点会持久化该事件；上传插件不拥有第二份存储。
|
||||
|
||||
只有在确切的请求相对引用能够减少编码字节时，事件字符串才不使用原始形式。一个引用会标识一条已序列化 DeepSeek 消息、通向一个解析后字符串值的路径，以及半开 UTF-8 字节范围。编码器会验证每个候选范围都能解码为完全相同的已匹配 UTF-16 子串；拆分 surrogate 或非良构 UTF-16 的候选会保持原始形式。解码器会精确重建每个字面值与引用字节，并拒绝无效路径、范围或码点切分。系统不使用模糊相似度、规范化或有损省略。
|
||||
`events` 数组会直接包含完整的权威 `SessionEvent` 对象。发送方会复制事件的每个已有成员，不执行投影或脱敏；该字段自包含，无需根据 `messages` 重建内容。
|
||||
|
||||
## 插件包字段
|
||||
|
||||
@@ -50,7 +50,7 @@ Status: implemented
|
||||
|
||||
## 验证
|
||||
|
||||
注册表测试固定重复所有权、effect 作用域 dispose（资源释放）、分离字段值、并发且可取消的准备、保留接收者的接受操作、单次接受结算与失败聚合。会话测试固定默认关闭策略、显式启用后的首次完整／后续后缀交付、增量水位 fold、持久化重启恢复、fork 身份围栏、乱序接受、确切 Unicode 重建、无效引用、surrogate 安全原始回退与 invariant 延迟加载。插件包清单测试固定默认开启与显式关闭策略、宿主与 standing preset 发现、冲突的 Loader 解析基址、manifest 解析、生命周期过滤及确切名称／版本排序。直接适配器 mock 测试证明 HTTP 前准备失败、取消、非 2xx 不接受、2xx 在后续流失败前接受，以及字段冲突。无密钥回放会固定 2xx 后扩展接受，TypeScript JSON-RPC 与 Python 打包运行时快照则通过两套 SDK 投影接受事件。真实 Loader 组合会固定默认包元数据与显式启用的会话上传，一个真实 API 请求会挂载两个随附扩展并证明官方端点接受它们；pi-ai 测试保持其协议请求不变。
|
||||
注册表测试固定重复所有权、effect 作用域 dispose（资源释放）、分离字段值、并发且可取消的准备、保留接收者的接受操作、单次接受结算与失败聚合。会话测试固定默认关闭策略、显式启用后的首次完整／后续后缀交付、与基础正文消息无关的直接完整事件信封、增量水位 fold、持久化重启恢复、fork 身份围栏、乱序接受与 invariant 延迟加载。插件包清单测试固定默认开启与显式关闭策略、宿主与 standing preset 发现、冲突的 Loader 解析基址、manifest 解析、生命周期过滤及确切名称／版本排序。直接适配器 mock 测试证明 HTTP 前准备失败、取消、非 2xx 不接受、2xx 在后续流失败前接受，以及字段冲突。无密钥回放会固定 2xx 后扩展接受，TypeScript JSON-RPC 与 Python 打包运行时快照则通过两套 SDK 投影接受事件。真实 Loader 组合会固定默认包元数据与显式启用的会话上传，一个真实 API 请求会挂载两个随附扩展并证明官方端点接受它们；pi-ai 测试保持其协议请求不变。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
@@ -58,7 +58,22 @@ Status: implemented
|
||||
|
||||
**把两个提供方硬编码进 `llm-deepseek`。** 已否决，因为适配器将导入会话、Loader、preset、包 manifest 与游标逻辑。注册表让传输只负责字段合并与 HTTP 接受。
|
||||
|
||||
**使用模糊消息相似度，或省略重叠事件数据。** 已否决，因为接收方无法重建权威日志。带原始回退的确切字节引用会保留每个值。
|
||||
### 为什么不使用请求相对消息引用？
|
||||
|
||||
一种递归的带标签表示可以用所属请求 `messages` 中的路径与 UTF-8 字节偏移，替换事件字符串的确切范围。测量使用 Node v24.16.0、macOS arm64 与可用的三份最大本地 Zstandard 会话产物；其压缩产物大小分别为 2,437,052、572,602 与 118,811 字节。延迟启用回放使用各会话最后一个已完成请求边界；稳态回放覆盖 411 个已完成边界。字节数覆盖完整且最小化的 DeepSeek 请求。
|
||||
|
||||
| 回放方式 | 原始 JSON | 引用 JSON | 节省比例 | 同步编码器耗时 |
|
||||
|---|---:|---:|---:|---:|
|
||||
| 延迟启用 | 29,668,725 B | 27,645,825 B | 6.82% | 合计 500.1 s |
|
||||
| 稳态 | 389,295,815 B | 387,180,848 B | 0.54% | 合计 285.0 s |
|
||||
|
||||
三次延迟启用调用分别耗时 470.5、29.4 与 0.158 秒。115,071 个事件中只有 701 个（0.61%）选择引用。一项假设采用 level-6 整请求 gzip 的对照，使原始请求字节在延迟启用场景减少 89.38%，在稳态场景减少 73.42%；加入消息引用后，gzip 结果分别额外减少 21.68% 与 0.59%。
|
||||
|
||||
接收方还需要遍历带标签树、解析通向确切请求消息的路径、校验 UTF-8 范围，并重建每个引用事件。即使把接收方成本视为零，稳态字节节省、发送方同步成本以及对另一请求字段的依赖，也不足以支撑带版本的协议格式。
|
||||
|
||||
### 为什么不省略 assistant 分片或重叠事件数据？
|
||||
|
||||
实测真实会话事件中约 98% 为 `assistant/chunk`。在引用编码后省略分片，会让完整未压缩 JSON 在延迟启用场景进一步减少 84.79%，在稳态场景进一步减少 6.49%，但这会阻止权威日志的无损重建，并让 `assistant/message.sourceEventSeqs` 指向缺失事件。模糊替换或规范化替换也存在同一重建缺陷。
|
||||
|
||||
**只在内存中保留上传游标。** 已否决，因为普通进程重启会重发完整会话。权威接受事件让重启恢复获得尽力而为的持久性，无需另一存储后端；剩余崩溃窗口只会产生允许的重复。
|
||||
|
||||
@@ -70,7 +85,7 @@ Status: implemented
|
||||
|
||||
## 后果
|
||||
|
||||
DeepSeek 官方请求会把存活包版本发送到解析后的 `baseURL`，包括已配置 gateway。显式选择启用会话日志后，请求还会携带完整的未接受会话新后缀。这些字段对模型不可见，不增加提示词 token，也不改变 KV Cache，但可能显著增大 HTTP 正文。请求相对打包会在派发前同步比较后缀字符串与消息字符串，因此在实现候选索引前，较大的首次上传或重试积压可能延迟事件循环。编码、manifest 解析、字段冲突、接受记录或提供方 schema 拒绝会使模型请求失败，而不会静默丢弃元数据。
|
||||
DeepSeek 官方请求会把存活包版本发送到解析后的 `baseURL`，包括已配置 gateway。显式选择启用会话日志后，请求还会携带完整的未接受会话新后缀。这些字段对模型不可见，不增加提示词 token，也不改变 KV Cache，但可能显著增大 HTTP 正文。Manifest 解析、字段冲突、接受记录或提供方 schema 拒绝会使模型请求失败，而不会静默丢弃元数据。
|
||||
|
||||
`delivery-accepted` 事件会成为权威日志的一部分，并在后续请求中自行交付。崩溃恢复可能重复后缀，但不会根据 assistant 输出推断接受，也不会创建第二份本地游标存储。缺少存活会话的直接调用会省略会话字段；宿主包清单仍然可用。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/config-catalog.md
|
||||
config-catalog.md: 0159ddf433e3a32bb77864f4b3781d40c1815eb2
|
||||
config-catalog.zh.md: 7b0843c664d19a92b4122f92f0f283ab28e0e4a7
|
||||
config-catalog.md: 0e7244f84fe275c3f4f574c29a646cdbd9ea2ec8
|
||||
config-catalog.zh.md: b17c641400f8e73d526a42eeb521583f9d9b1b97
|
||||
@@ -1738,7 +1738,7 @@ export interface Config {
|
||||
}
|
||||
```
|
||||
|
||||
Source: [`packages/session/session-log-deepseek/src/index.ts:24`](../packages/session/session-log-deepseek/src/index.ts)
|
||||
Source: [`packages/session/session-log-deepseek/src/index.ts:22`](../packages/session/session-log-deepseek/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-session-persistence-jsonl"></a>
|
||||
|
||||
|
||||
@@ -1740,7 +1740,7 @@ export interface Config {
|
||||
}
|
||||
```
|
||||
|
||||
来源：[`packages/session/session-log-deepseek/src/index.ts:24`](../packages/session/session-log-deepseek/src/index.ts)
|
||||
来源：[`packages/session/session-log-deepseek/src/index.ts:22`](../packages/session/session-log-deepseek/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-session-persistence-jsonl"></a>
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/deepseek-llm-api-wire-extensions.md
|
||||
deepseek-llm-api-wire-extensions.md: 8a3f089c77a0879ef4056f3c16583935f2219248
|
||||
deepseek-llm-api-wire-extensions.zh.md: 1cf5ea1ce1cd410b23b011480a8c623827753464
|
||||
deepseek-llm-api-wire-extensions.md: fd42609693ac6fbf91dd82b6e73b2d1d06e65a54
|
||||
deepseek-llm-api-wire-extensions.zh.md: 61af718841c8778e8943a1a1c16621a9a6618add
|
||||
@@ -12,8 +12,8 @@ The adapter sends the additions to its resolved `baseURL`, including a configure
|
||||
|---|---|---|
|
||||
| HTTP field names | Lowercase kebab-case; HTTP matching remains case-insensitive | `user-agent`, `x-deepseek-harness-session-id` |
|
||||
| DeepSeek request-body extension fields | Snake case with the reserved `dsh_` prefix | `dsh_plugin_packages`, `dsh_session_log` |
|
||||
| DSH-owned nested JSON members | Camel case | `afterSeq`, `messageIndex`, `utf8Start` |
|
||||
| Tagged values | Kebab-case strings; durable events use `domain/action` | `message-references`, `session-log-deepseek/delivery-accepted` |
|
||||
| DSH-owned nested JSON members | Camel case | `afterSeq`, `throughSeq`, `sessionId` |
|
||||
| Tagged values | Kebab-case strings; durable events use `domain/action` | `session-log-deepseek/delivery-accepted` |
|
||||
|
||||
Each body extension owns its `version` independently. A version applies only to the object that contains it; no compatibility or ordering relationship exists between versions of different fields. JSON member order is not part of the protocol.
|
||||
|
||||
@@ -88,14 +88,11 @@ An enabled inventory with no qualifying entries sends `packages: []`; disabling
|
||||
"throughSeq": 0,
|
||||
"events": [
|
||||
{
|
||||
"encoding": "raw",
|
||||
"event": {
|
||||
"type": "turn/start",
|
||||
"seq": 0,
|
||||
"time": 1780000000001,
|
||||
"data": {
|
||||
"turn": 1
|
||||
}
|
||||
"type": "turn/start",
|
||||
"seq": 0,
|
||||
"time": 1780000000001,
|
||||
"data": {
|
||||
"turn": 1
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -129,106 +126,9 @@ The `session` member is the exact `Session.header`, not a complete runtime Sessi
|
||||
| `delegationDepth` | optional | Non-negative persisted subagent delegation depth |
|
||||
| `agentPreset` | optional | Agent preset id used to compose this Session |
|
||||
|
||||
### Event envelope and encodings
|
||||
### Canonical event envelopes
|
||||
|
||||
Each `events` item is an `EncodedSessionEvent` selected by its `encoding` value.
|
||||
|
||||
| `encoding` | `event` value |
|
||||
|---|---|
|
||||
| `raw` | Complete canonical `SessionEvent` |
|
||||
| `message-references` | `PackedJsonValue` that reconstructs the complete canonical `SessionEvent` against this request's `messages` |
|
||||
|
||||
A canonical event always carries `type`, `seq`, `time`, and `data`. It may carry `ignorable: true`; surface events may additionally carry `sourceEventSeqs` and `surfaceOp`. A raw encoding copies every present member without projection or redaction.
|
||||
|
||||
A `PackedJsonValue` uses one of four `kind` values:
|
||||
|
||||
| `kind` | Members | Decoded value |
|
||||
|---|---|---|
|
||||
| `literal` | `value` | The contained JSON value, which may itself be composite |
|
||||
| `string` | `parts` | Concatenation of decoded `PackedJsonStringPart` values |
|
||||
| `array` | `items` | Array of recursively decoded values |
|
||||
| `object` | `entries` | Object built from ordered `[key, value]` pairs |
|
||||
|
||||
A packed string contains these part variants:
|
||||
|
||||
| `kind` | `value` |
|
||||
|---|---|
|
||||
| `literal` | Literal string fragment |
|
||||
| `message-slice` | Exact request-relative `DeepSeekMessageStringSlice` |
|
||||
|
||||
```json
|
||||
{
|
||||
"encoding": "message-references",
|
||||
"event": {
|
||||
"kind": "object",
|
||||
"entries": [
|
||||
[
|
||||
"type",
|
||||
{
|
||||
"kind": "literal",
|
||||
"value": "plugin/test"
|
||||
}
|
||||
],
|
||||
[
|
||||
"seq",
|
||||
{
|
||||
"kind": "literal",
|
||||
"value": 0
|
||||
}
|
||||
],
|
||||
[
|
||||
"time",
|
||||
{
|
||||
"kind": "literal",
|
||||
"value": 1780000000001
|
||||
}
|
||||
],
|
||||
[
|
||||
"data",
|
||||
{
|
||||
"kind": "object",
|
||||
"entries": [
|
||||
[
|
||||
"text",
|
||||
{
|
||||
"kind": "string",
|
||||
"parts": [
|
||||
{
|
||||
"kind": "literal",
|
||||
"value": "prefix:"
|
||||
},
|
||||
{
|
||||
"kind": "message-slice",
|
||||
"value": {
|
||||
"messageIndex": 0,
|
||||
"path": [
|
||||
"content"
|
||||
],
|
||||
"utf8Start": 0,
|
||||
"utf8End": 12
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Slice member | Meaning |
|
||||
|---|---|
|
||||
| `messageIndex` | Zero-based index into the exact DeepSeek `messages` array in the containing request |
|
||||
| `path` | Array of string object keys and numeric array indexes from that message root to a string value |
|
||||
| `utf8Start` | Inclusive UTF-8 byte offset in the resolved string value |
|
||||
| `utf8End` | Exclusive UTF-8 byte offset in the resolved string value |
|
||||
|
||||
Offsets address the UTF-8 bytes of the parsed string value, not its JSON-escaped source text. A decoder rejects a missing path, a non-string target, an invalid range, or a range that splits a UTF-8 code point. Reconstruction therefore requires the exact `messages` array from the same request.
|
||||
|
||||
The encoder uses a reference only for an exact substring and only when the complete referenced event occupies fewer serialized UTF-8 bytes than its raw form. Literal fragments retain unmatched text. A candidate that cannot round-trip exactly stays raw; the protocol performs no fuzzy matching, truncation, or lossy omission.
|
||||
Each `events` item is a complete canonical `SessionEvent`, independent of every other request field. An event always carries `type`, `seq`, `time`, and `data`; it may carry `ignorable: true`, and surface events may additionally carry `sourceEventSeqs` and `surfaceOp`. The sender copies every present member without projection, redaction, or reconstruction.
|
||||
|
||||
### Acceptance watermark and at-least-once delivery
|
||||
|
||||
@@ -256,4 +156,4 @@ Transport and non-2xx failures append no watermark. A crash after endpoint accep
|
||||
|
||||
The request headers expose the Harness application version, one anonymous Harness-home identity, and an optional Session identity. `dsh_plugin_packages` exposes active npm package names and versions. When enabled, `dsh_session_log` may expose the Session working directory, system-prompt snapshots, user and assistant content, raw assistant chunks, tool arguments and results, compaction summaries, feedback, and plugin-owned events. Adapter API keys are not Session events and therefore do not enter the field. A gateway selected through `baseURL` receives the same values as the official endpoint.
|
||||
|
||||
Receivers address extension fields by name, dispatch each field by its own `version`, preserve distinct package versions, and ignore JSON member ordering. A session-log receiver validates the contiguous sequence range and reconstructs every referenced event against the containing request before interpreting event types. An unrecognized canonical event without `ignorable: true` prevents lossless reconstruction. The base request remains usable without either the registry or a particular contribution; field absence means that contribution did not apply to that request.
|
||||
Receivers address extension fields by name, dispatch each field by its own `version`, preserve distinct package versions, and ignore JSON member ordering. A session-log receiver validates the contiguous sequence range before interpreting event types. An unrecognized canonical event without `ignorable: true` prevents lossless reconstruction. The base request remains usable without either the registry or a particular contribution; field absence means that contribution did not apply to that request.
|
||||
@@ -12,8 +12,8 @@
|
||||
|---|---|---|
|
||||
| HTTP 字段名 | 小写 kebab-case；HTTP 匹配仍不区分大小写 | `user-agent`, `x-deepseek-harness-session-id` |
|
||||
| DeepSeek 请求正文扩展字段 | 使用保留 `dsh_` 前缀的 snake case | `dsh_plugin_packages`, `dsh_session_log` |
|
||||
| DSH 持有的嵌套 JSON 成员 | Camel case | `afterSeq`, `messageIndex`, `utf8Start` |
|
||||
| 带标签的值 | 使用 kebab-case 字符串；持久事件采用 `domain/action` | `message-references`, `session-log-deepseek/delivery-accepted` |
|
||||
| DSH 持有的嵌套 JSON 成员 | Camel case | `afterSeq`, `throughSeq`, `sessionId` |
|
||||
| 带标签的值 | 使用 kebab-case 字符串；持久事件采用 `domain/action` | `session-log-deepseek/delivery-accepted` |
|
||||
|
||||
每个正文扩展独立持有自身的 `version`。版本仅适用于包含该字段的对象；不同字段的版本之间不存在兼容或排序关系。JSON 成员顺序不属于协议。
|
||||
|
||||
@@ -88,14 +88,11 @@
|
||||
"throughSeq": 0,
|
||||
"events": [
|
||||
{
|
||||
"encoding": "raw",
|
||||
"event": {
|
||||
"type": "turn/start",
|
||||
"seq": 0,
|
||||
"time": 1780000000001,
|
||||
"data": {
|
||||
"turn": 1
|
||||
}
|
||||
"type": "turn/start",
|
||||
"seq": 0,
|
||||
"time": 1780000000001,
|
||||
"data": {
|
||||
"turn": 1
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -129,106 +126,9 @@
|
||||
| `delegationDepth` | 可选 | 持久化的非负 subagent 委派深度 |
|
||||
| `agentPreset` | 可选 | 用于组合该会话的 agent preset id |
|
||||
|
||||
### 事件信封与编码
|
||||
### 权威事件信封
|
||||
|
||||
每个 `events` 元素都是通过 `encoding` 值选择的 `EncodedSessionEvent`。
|
||||
|
||||
| `encoding` | `event` 值 |
|
||||
|---|---|
|
||||
| `raw` | 完整的权威 `SessionEvent` |
|
||||
| `message-references` | 根据本次请求 `messages` 重建完整权威 `SessionEvent` 的 `PackedJsonValue` |
|
||||
|
||||
权威事件始终携带 `type`、`seq`、`time` 和 `data`。它可以携带 `ignorable: true`；展示事件还可以携带 `sourceEventSeqs` 和 `surfaceOp`。raw 编码会复制全部已有成员，不做投影或脱敏。
|
||||
|
||||
`PackedJsonValue` 使用以下四种 `kind` 值之一：
|
||||
|
||||
| `kind` | 成员 | 解码值 |
|
||||
|---|---|---|
|
||||
| `literal` | `value` | 所含 JSON 值；该值自身也可以是复合值 |
|
||||
| `string` | `parts` | 已解码 `PackedJsonStringPart` 值的拼接结果 |
|
||||
| `array` | `items` | 递归解码值组成的数组 |
|
||||
| `object` | `entries` | 根据有序 `[key, value]` 对构建的对象 |
|
||||
|
||||
已打包字符串包含以下两种片段：
|
||||
|
||||
| `kind` | `value` |
|
||||
|---|---|
|
||||
| `literal` | 字面字符串片段 |
|
||||
| `message-slice` | 确切的请求相对 `DeepSeekMessageStringSlice` |
|
||||
|
||||
```json
|
||||
{
|
||||
"encoding": "message-references",
|
||||
"event": {
|
||||
"kind": "object",
|
||||
"entries": [
|
||||
[
|
||||
"type",
|
||||
{
|
||||
"kind": "literal",
|
||||
"value": "plugin/test"
|
||||
}
|
||||
],
|
||||
[
|
||||
"seq",
|
||||
{
|
||||
"kind": "literal",
|
||||
"value": 0
|
||||
}
|
||||
],
|
||||
[
|
||||
"time",
|
||||
{
|
||||
"kind": "literal",
|
||||
"value": 1780000000001
|
||||
}
|
||||
],
|
||||
[
|
||||
"data",
|
||||
{
|
||||
"kind": "object",
|
||||
"entries": [
|
||||
[
|
||||
"text",
|
||||
{
|
||||
"kind": "string",
|
||||
"parts": [
|
||||
{
|
||||
"kind": "literal",
|
||||
"value": "prefix:"
|
||||
},
|
||||
{
|
||||
"kind": "message-slice",
|
||||
"value": {
|
||||
"messageIndex": 0,
|
||||
"path": [
|
||||
"content"
|
||||
],
|
||||
"utf8Start": 0,
|
||||
"utf8End": 12
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| slice 成员 | 含义 |
|
||||
|---|---|
|
||||
| `messageIndex` | 包含该扩展的请求中，确切 DeepSeek `messages` 数组的零基序号 |
|
||||
| `path` | 从该消息根到某个字符串值的路径数组；对象键使用字符串，数组索引使用数字 |
|
||||
| `utf8Start` | 已解析字符串值中的包含性 UTF-8 字节偏移 |
|
||||
| `utf8End` | 已解析字符串值中的排他性 UTF-8 字节偏移 |
|
||||
|
||||
偏移量指向已解析字符串值的 UTF-8 字节，而不是经过 JSON 转义的源文本。路径缺失、目标不是字符串、范围无效或范围切开 UTF-8 码点时，解码器会拒绝。因此，重建必须使用同一次请求的确切 `messages` 数组。
|
||||
|
||||
编码器只会引用确切子字符串，并且仅在整个引用事件占用的序列化 UTF-8 字节少于 raw 形式时才会使用引用。字面片段保留未匹配文本。无法精确重建为原值的候选项会保留 raw 形式；协议不执行模糊匹配、截断或有损省略。
|
||||
每个 `events` 元素都是完整的权威 `SessionEvent`，不依赖任何其他请求字段。事件始终携带 `type`、`seq`、`time` 与 `data`；它可以携带 `ignorable: true`，展示事件还可携带 `sourceEventSeqs` 与 `surfaceOp`。发送方会复制每个已有成员，不执行投影、脱敏或重建。
|
||||
|
||||
### 接受水位与至少一次交付
|
||||
|
||||
@@ -256,4 +156,4 @@
|
||||
|
||||
请求标头会暴露 Harness 应用版本、一个匿名 Harness-home 身份和可选的会话身份。`dsh_plugin_packages` 会暴露存活 npm 包的名称与版本。启用后，`dsh_session_log` 可能暴露会话工作目录、系统提示词快照、用户与 assistant 内容、原始 assistant 分片、工具参数与结果、压缩摘要、反馈和插件持有的事件。适配器 API key 不是会话事件，因此不会进入该字段。通过 `baseURL` 选择的网关会收到与官方端点相同的值。
|
||||
|
||||
接收方按名称定位扩展字段，按各字段自己的 `version` 分派，保留不同的包版本，并忽略 JSON 成员顺序。会话日志接收方必须校验连续序号范围，并在解释事件类型前，根据包含该扩展的请求重建每个引用事件。遇到不带 `ignorable: true` 的未知权威事件时，接收方无法进行无损重建。即使缺少注册表或某项贡献，基础请求仍然可用；字段缺失表示该项贡献不适用于本次请求。
|
||||
接收方按名称定位扩展字段，按各字段自己的 `version` 分派，保留不同的包版本，并忽略 JSON 成员顺序。会话日志接收方必须先校验连续序号范围，再解释事件类型。遇到不带 `ignorable: true` 的未知权威事件时，接收方无法进行无损重建。即使缺少注册表或某项贡献，基础请求仍然可用；字段缺失表示该项贡献不适用于本次请求。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/persistence-catalog.md
|
||||
persistence-catalog.md: ad65bc6bd3971d204f9bce3c7ee822920ebe6a85
|
||||
persistence-catalog.zh.md: 3af5d7c46e7969b29a66b04adddc41d9d108553d
|
||||
persistence-catalog.md: 8b1206ba18a6f49a0eb809a1a3a12828fd94f827
|
||||
persistence-catalog.zh.md: 9a085ece1fd2df331f34e311c48bfe0c8ca92c91
|
||||
@@ -683,7 +683,7 @@ Source: [`packages/session/session-title-llm/src/index.ts:43`](../packages/sessi
|
||||
}
|
||||
```
|
||||
|
||||
Source: [`packages/session/session-log-deepseek/src/types.ts:54`](../packages/session/session-log-deepseek/src/types.ts)
|
||||
Source: [`packages/session/session-log-deepseek/src/types.ts:26`](../packages/session/session-log-deepseek/src/types.ts)
|
||||
|
||||
### `step/*`
|
||||
|
||||
|
||||
@@ -685,7 +685,7 @@ export type SessionEvent<T extends SessionEventType = SessionEventType> = {
|
||||
}
|
||||
```
|
||||
|
||||
来源：[`packages/session/session-log-deepseek/src/types.ts:54`](../packages/session/session-log-deepseek/src/types.ts)
|
||||
来源：[`packages/session/session-log-deepseek/src/types.ts:26`](../packages/session/session-log-deepseek/src/types.ts)
|
||||
|
||||
### `step/*`
|
||||
|
||||
|
||||
@@ -179,7 +179,7 @@ describe('llm-deepseek real dynamic composition', () => {
|
||||
session: { id: string }
|
||||
afterSeq: number
|
||||
throughSeq: number
|
||||
events: Array<{ event: { type: string; seq: number } }>
|
||||
events: Array<{ type: string; seq: number }>
|
||||
}
|
||||
}
|
||||
expect(request.dsh_session_log).toMatchObject({
|
||||
@@ -187,7 +187,7 @@ describe('llm-deepseek real dynamic composition', () => {
|
||||
session: { id: 'extension-composition-enabled' },
|
||||
afterSeq: -1,
|
||||
throughSeq: 0,
|
||||
events: [{ event: { type: 'turn/start', seq: 0 } }],
|
||||
events: [{ type: 'turn/start', seq: 0 }],
|
||||
})
|
||||
expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(0)
|
||||
})
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-log-deepseek/README.md
|
||||
README.md: 39ff31b8c33f2f8e0a12cb6ed0814d97f982e3c9
|
||||
README.zh.md: 0121644cc0b7ebbe2e6b80ac6c31ae21e088fe95
|
||||
README.md: 24cff61f77dab76caf086375931edd5ee45e3f04
|
||||
README.zh.md: 0411fcd298acae86aee93da7efe812ab6d589c29
|
||||
@@ -14,9 +14,7 @@ Shipped profiles mount the plugin so an overlay can enable it, but the default c
|
||||
|
||||
## Request field
|
||||
|
||||
For a request carrying a live `sessionId`, the plugin folds the greatest accepted watermark for that exact Session identity, snapshots `Session.events`, and sends the contiguous suffix after the watermark. A process-local fold scans each event once and consumes later appends incrementally; restart and HMR rebuild it from the durable log. The version-1 field contains the immutable `SessionHeader`, `afterSeq`, `throughSeq`, and every complete event envelope in that range. Forked sessions ignore inherited parent watermarks because each watermark records the Session id sent on the accepted request.
|
||||
|
||||
Each event is sent raw unless request-relative packing reduces its JSON byte size. The codec traverses the exact serialized DeepSeek `messages`, and a packed string can replace an exact substring with `{ messageIndex, path, utf8Start, utf8End }`. References use half-open UTF-8 offsets into parsed string values; literal fragments retain everything outside the match. The encoder verifies each candidate's local UTF-8 round trip, so a surrogate-splitting or ill-formed UTF-16 match stays raw. The exported decoder rejects missing paths, non-string targets, invalid ranges, and ranges that split a UTF-8 code point. Raw fallback and byte-size comparison make every representation lossless and prevent reference overhead from expanding an event.
|
||||
For a request carrying a live `sessionId`, the plugin folds the greatest accepted watermark for that exact Session identity, snapshots `Session.events`, and sends the contiguous suffix after the watermark. A process-local fold scans each event once and consumes later appends incrementally; restart and HMR rebuild it from the durable log. The version-1 field contains the immutable `SessionHeader`, `afterSeq`, `throughSeq`, and every complete canonical `SessionEvent` in that range as a direct array element. Forked sessions ignore inherited parent watermarks because each watermark records the Session id sent on the accepted request.
|
||||
|
||||
## Acceptance and retry
|
||||
|
||||
@@ -47,4 +45,3 @@ None; the model-visible request prefix remains unchanged.
|
||||
- **Crash-window duplicates** — a 2xx followed by process loss before the acceptance watermark persists causes conservative replay on resume.
|
||||
- **No live Session means no field** — direct or stale-session calls have no canonical log to snapshot; explicit absence semantics remain deferred.
|
||||
- **No independent request-size cap** — complete delivery is fail-closed; provider rejection leaves the cursor unchanged instead of truncating the log.
|
||||
- **Synchronous request-time packing** — the encoder compares suffix strings with request-message strings before dispatch. Large first uploads or retry backlogs can delay the event loop until candidate indexing is implemented.
|
||||
@@ -14,9 +14,7 @@
|
||||
|
||||
## 请求字段
|
||||
|
||||
对于携带存活 `sessionId` 的请求，插件会折叠该确切会话身份的最大已接受水位，对 `Session.events` 取快照，并发送水位之后的连续后缀。进程内 fold 会让每条事件只被扫描一次并增量消费后续追加；重启与 HMR 会从持久日志重建它。版本 1 字段包含不可变的 `SessionHeader`、`afterSeq`、`throughSeq`，以及该范围内每个完整事件信封。每个水位都会记录已接受请求发送的会话 id，因此 fork 会话会忽略从父会话继承的水位。
|
||||
|
||||
只有在请求相对打包能够减少 JSON 字节数时，事件才不以原始形式发送。Codec 会遍历确切的已序列化 DeepSeek `messages`，并可在打包字符串中用 `{ messageIndex, path, utf8Start, utf8End }` 替换完全匹配的子字符串。引用使用解析后字符串值的半开 UTF-8 偏移；字面分片保留匹配范围之外的全部内容。编码器会校验每个候选的局部 UTF-8 往返，因此拆分 surrogate 或非良构 UTF-16 的匹配会保持原始形式。导出的解码器会拒绝缺失路径、非字符串目标、无效范围和切断 UTF-8 码点的范围。原始回退与字节数比较让每种表示都保持无损，并阻止引用开销扩张事件。
|
||||
对于携带存活 `sessionId` 的请求，插件会折叠该确切会话身份的最大已接受水位，对 `Session.events` 取快照，并发送水位之后的连续后缀。进程内 fold 会让每条事件只被扫描一次并增量消费后续追加；重启与 HMR 会从持久日志重建它。版本 1 字段包含不可变的 `SessionHeader`、`afterSeq`、`throughSeq`，以及作为数组直接元素的该范围内每个完整权威 `SessionEvent`。每个水位都会记录已接受请求发送的会话 id，因此 fork 会话会忽略从父会话继承的水位。
|
||||
|
||||
## 接受与重试
|
||||
|
||||
@@ -47,4 +45,3 @@ DeepSeek 适配器会在 HTTP 2xx 后、消费 SSE（Server-Sent Events）正文
|
||||
- **崩溃窗口重复**——2xx 后、接受水位持久化前进程丢失，会在恢复时触发保守重放。
|
||||
- **缺少存活会话就没有字段**——直接调用或陈旧会话调用没有可供快照的权威日志；显式缺失语义仍暂缓处理。
|
||||
- **没有独立请求大小上限**——完整交付会快速失败；提供方拒绝会保持游标不变，而非截断日志。
|
||||
- **请求时同步打包**——编码器会在派发前比较后缀字符串与请求消息字符串。在实现候选索引前，较大的首次上传或重试积压可能延迟事件循环。
|
||||
@@ -1,231 +0,0 @@
|
||||
/** Lossless request-relative packing for canonical session events. */
|
||||
|
||||
import { Buffer } from 'node:buffer'
|
||||
import type { DeepSeekLlmApiJson } from '@deepseek-ai/dsh-deepseek-llm-api-extensions'
|
||||
import type { JsonValue, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type {
|
||||
DeepSeekMessageStringSlice,
|
||||
EncodedSessionEvent,
|
||||
PackedJsonStringPart,
|
||||
PackedJsonValue,
|
||||
} from './types.ts'
|
||||
|
||||
interface MessageStringSource {
|
||||
readonly messageIndex: number
|
||||
readonly path: readonly (string | number)[]
|
||||
readonly value: string
|
||||
readonly bytes: number
|
||||
}
|
||||
|
||||
interface PackedCandidate {
|
||||
readonly value: PackedJsonValue
|
||||
readonly references: boolean
|
||||
}
|
||||
|
||||
/** Return the serialized UTF-8 size used by the profitability decision. */
|
||||
function wireBytes(value: unknown): number {
|
||||
return Buffer.byteLength(JSON.stringify(value), 'utf8')
|
||||
}
|
||||
|
||||
/** Enumerate every string leaf of the exact wire messages. */
|
||||
function collectSources(
|
||||
value: DeepSeekLlmApiJson,
|
||||
messageIndex: number,
|
||||
path: readonly (string | number)[],
|
||||
output: MessageStringSource[],
|
||||
): void {
|
||||
if (typeof value === 'string') {
|
||||
if (value.length > 0) output.push({ messageIndex, path, value, bytes: Buffer.byteLength(value, 'utf8') })
|
||||
return
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
value.forEach((child, index) => { collectSources(child, messageIndex, [...path, index], output) })
|
||||
return
|
||||
}
|
||||
if (value === null || typeof value !== 'object') return
|
||||
for (const [key, child] of Object.entries(value)) collectSources(child, messageIndex, [...path, key], output)
|
||||
}
|
||||
|
||||
/** Build one exact slice from UTF-16 offsets, or reject a lossy UTF-8 conversion. */
|
||||
function sliceOf(source: MessageStringSource, start: number, end: number): DeepSeekMessageStringSlice | undefined {
|
||||
const candidate: DeepSeekMessageStringSlice = {
|
||||
messageIndex: source.messageIndex,
|
||||
path: source.path,
|
||||
utf8Start: Buffer.byteLength(source.value.slice(0, start), 'utf8'),
|
||||
utf8End: Buffer.byteLength(source.value.slice(0, end), 'utf8'),
|
||||
}
|
||||
const bytes = Buffer.from(source.value, 'utf8')
|
||||
const selected = bytes.subarray(candidate.utf8Start, candidate.utf8End)
|
||||
const decoded = selected.toString('utf8')
|
||||
return decoded === source.value.slice(start, end) && Buffer.from(decoded, 'utf8').equals(selected)
|
||||
? candidate
|
||||
: undefined
|
||||
}
|
||||
|
||||
/** Find a profitable exact whole-string or one-inner-string reference. */
|
||||
function packString(value: string, sources: readonly MessageStringSource[]): PackedCandidate {
|
||||
const literal: PackedJsonValue = { kind: 'literal', value }
|
||||
if (value.length === 0) return { value: literal, references: false }
|
||||
const valueBytes = Buffer.byteLength(value, 'utf8')
|
||||
|
||||
let best: PackedJsonValue | undefined
|
||||
let bestBytes = wireBytes(literal)
|
||||
for (const source of sources) {
|
||||
if (source.bytes < valueBytes) continue
|
||||
const start = source.value.indexOf(value)
|
||||
if (start < 0) continue
|
||||
const slice = sliceOf(source, start, start + value.length)
|
||||
if (slice === undefined) continue
|
||||
const candidate: PackedJsonValue = {
|
||||
kind: 'string',
|
||||
parts: [{ kind: 'message-slice', value: slice }],
|
||||
}
|
||||
const bytes = wireBytes(candidate)
|
||||
if (bytes < bestBytes) {
|
||||
best = candidate
|
||||
bestBytes = bytes
|
||||
}
|
||||
}
|
||||
if (best !== undefined) return { value: best, references: true }
|
||||
|
||||
for (const source of sources) {
|
||||
if (source.bytes > valueBytes) continue
|
||||
const start = value.indexOf(source.value)
|
||||
if (start < 0) continue
|
||||
const slice = sliceOf(source, 0, source.value.length)
|
||||
if (slice === undefined) continue
|
||||
const parts: PackedJsonStringPart[] = []
|
||||
if (start > 0) parts.push({ kind: 'literal', value: value.slice(0, start) })
|
||||
parts.push({ kind: 'message-slice', value: slice })
|
||||
const end = start + source.value.length
|
||||
if (end < value.length) parts.push({ kind: 'literal', value: value.slice(end) })
|
||||
const candidate: PackedJsonValue = { kind: 'string', parts }
|
||||
const bytes = wireBytes(candidate)
|
||||
if (bytes < bestBytes) {
|
||||
best = candidate
|
||||
bestBytes = bytes
|
||||
}
|
||||
}
|
||||
return best === undefined
|
||||
? { value: literal, references: false }
|
||||
: { value: best, references: true }
|
||||
}
|
||||
|
||||
/** Recursively pack one JSON value, retaining references only when the complete subtree shrinks. */
|
||||
function packValue(value: JsonValue, sources: readonly MessageStringSource[]): PackedCandidate {
|
||||
const literal: PackedJsonValue = { kind: 'literal', value }
|
||||
if (typeof value === 'string') return packString(value, sources)
|
||||
if (value === null || typeof value !== 'object') return { value: literal, references: false }
|
||||
|
||||
if (Array.isArray(value)) {
|
||||
const children = value.map(child => packValue(child, sources))
|
||||
if (!children.some(child => child.references)) return { value: literal, references: false }
|
||||
const candidate: PackedJsonValue = { kind: 'array', items: children.map(child => child.value) }
|
||||
return { value: candidate, references: true }
|
||||
}
|
||||
|
||||
const children = Object.entries(value).map(([key, child]) => [key, packValue(child, sources)] as const)
|
||||
if (!children.some(([, child]) => child.references)) return { value: literal, references: false }
|
||||
const candidate: PackedJsonValue = {
|
||||
kind: 'object',
|
||||
entries: children.map(([key, child]) => [key, child.value]),
|
||||
}
|
||||
return { value: candidate, references: true }
|
||||
}
|
||||
|
||||
/** Resolve one path in a parsed wire message. */
|
||||
function valueAt(root: DeepSeekLlmApiJson, path: readonly (string | number)[]): DeepSeekLlmApiJson {
|
||||
let value = root
|
||||
for (const segment of path) {
|
||||
if (typeof segment === 'number') {
|
||||
if (!Array.isArray(value) || segment < 0 || segment >= value.length) {
|
||||
throw new Error(`session-log-deepseek: message reference has invalid array segment ${segment}`)
|
||||
}
|
||||
value = value[segment] as DeepSeekLlmApiJson
|
||||
continue
|
||||
}
|
||||
if (value === null || typeof value !== 'object' || Array.isArray(value) || !Object.hasOwn(value, segment)) {
|
||||
throw new Error(`session-log-deepseek: message reference has invalid object segment ${JSON.stringify(segment)}`)
|
||||
}
|
||||
value = value[segment] as DeepSeekLlmApiJson
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
/** Decode and validate one exact UTF-8 message slice. */
|
||||
function decodeSlice(messages: readonly DeepSeekLlmApiJson[], slice: DeepSeekMessageStringSlice): string {
|
||||
const message = messages[slice.messageIndex]
|
||||
if (message === undefined) throw new Error(`session-log-deepseek: message reference index ${slice.messageIndex} is absent`)
|
||||
const source = valueAt(message, slice.path)
|
||||
if (typeof source !== 'string') throw new Error('session-log-deepseek: message reference path does not resolve to a string')
|
||||
const bytes = Buffer.from(source, 'utf8')
|
||||
if (!Number.isSafeInteger(slice.utf8Start) || !Number.isSafeInteger(slice.utf8End)
|
||||
|| slice.utf8Start < 0 || slice.utf8End <= slice.utf8Start || slice.utf8End > bytes.length) {
|
||||
throw new Error('session-log-deepseek: message reference byte range is invalid')
|
||||
}
|
||||
const selected = bytes.subarray(slice.utf8Start, slice.utf8End)
|
||||
const decoded = selected.toString('utf8')
|
||||
if (!Buffer.from(decoded, 'utf8').equals(selected)) {
|
||||
throw new Error('session-log-deepseek: message reference splits a UTF-8 code point')
|
||||
}
|
||||
return decoded
|
||||
}
|
||||
|
||||
/**
|
||||
* Decode one packed JSON value against the containing request's messages.
|
||||
* @param value - tagged literal/reference value to reconstruct.
|
||||
* @param messages - serialized messages from the same request.
|
||||
* @returns the exact reconstructed JSON value.
|
||||
*/
|
||||
export function unpackJsonValue(value: PackedJsonValue, messages: readonly DeepSeekLlmApiJson[]): JsonValue {
|
||||
switch (value.kind) {
|
||||
case 'literal':
|
||||
return structuredClone(value.value)
|
||||
case 'string':
|
||||
return value.parts.map(part => part.kind === 'literal' ? part.value : decodeSlice(messages, part.value)).join('')
|
||||
case 'array':
|
||||
return value.items.map(item => unpackJsonValue(item, messages))
|
||||
case 'object':
|
||||
return Object.fromEntries(value.entries.map(([key, child]) => [key, unpackJsonValue(child, messages)]))
|
||||
default:
|
||||
throw new Error('session-log-deepseek: unknown packed JSON value')
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Encode canonical session events against the exact DeepSeek wire messages.
|
||||
* @param events - immutable canonical event suffix.
|
||||
* @param messages - serialized request messages that the receiver also holds.
|
||||
* @returns one raw-or-referenced representation per event.
|
||||
*/
|
||||
export function packSessionEvents(
|
||||
events: readonly SessionEvent[],
|
||||
messages: readonly DeepSeekLlmApiJson[],
|
||||
): EncodedSessionEvent[] {
|
||||
// TODO: Index message strings if large opt-in suffixes show material request-preparation latency.
|
||||
const sources: MessageStringSource[] = []
|
||||
messages.forEach((message, messageIndex) => { collectSources(message, messageIndex, [], sources) })
|
||||
sources.sort((left, right) => right.bytes - left.bytes)
|
||||
return events.map((event) => {
|
||||
const packed = packValue(event as unknown as JsonValue, sources)
|
||||
const raw: EncodedSessionEvent = { encoding: 'raw', event }
|
||||
if (!packed.references) return raw
|
||||
const referenced: EncodedSessionEvent = { encoding: 'message-references', event: packed.value }
|
||||
return wireBytes(referenced) < wireBytes(raw) ? referenced : raw
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Reconstruct canonical session events from one request-relative representation.
|
||||
* @param events - encoded event list from the extension field.
|
||||
* @param messages - serialized messages from the same request.
|
||||
* @returns reconstructed event values in order.
|
||||
*/
|
||||
export function unpackSessionEvents(
|
||||
events: readonly EncodedSessionEvent[],
|
||||
messages: readonly DeepSeekLlmApiJson[],
|
||||
): SessionEvent[] {
|
||||
return events.map(encoded => encoded.encoding === 'raw'
|
||||
? structuredClone(encoded.event)
|
||||
: unpackJsonValue(encoded.event, messages) as unknown as SessionEvent)
|
||||
}
|
||||
@@ -7,12 +7,10 @@
|
||||
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import z from '@deepseek-ai/schemastery'
|
||||
import type { DeepSeekLlmApiJson } from '@deepseek-ai/dsh-deepseek-llm-api-extensions'
|
||||
import type {} from '@deepseek-ai/dsh-deepseek-llm-api-extensions'
|
||||
import { SessionId, type Session, type SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { packSessionEvents } from './codec.ts'
|
||||
import type { DeepSeekSessionLogExtension } from './types.ts'
|
||||
|
||||
export { packSessionEvents, unpackJsonValue, unpackSessionEvents } from './codec.ts'
|
||||
export type * from './types.ts'
|
||||
|
||||
/** Cordis plugin name. */
|
||||
@@ -63,13 +61,6 @@ export function acceptedThrough(session: Session): number {
|
||||
return throughSeq
|
||||
}
|
||||
|
||||
/** Resolve the serialized message list or fail before transport. */
|
||||
function requestMessages(body: Readonly<Record<string, DeepSeekLlmApiJson>>): readonly DeepSeekLlmApiJson[] {
|
||||
const messages = body.messages
|
||||
if (!Array.isArray(messages)) throw new Error('session-log-deepseek: DeepSeek request body has no messages array')
|
||||
return messages
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the incremental `dsh_session_log` request contribution when enabled.
|
||||
* @param ctx - plugin context carrying Sessions and the DeepSeek request-extension registry.
|
||||
@@ -94,7 +85,7 @@ export function apply(ctx: Context, config: Config): void {
|
||||
session: session.header,
|
||||
afterSeq,
|
||||
throughSeq,
|
||||
events: packSessionEvents(suffix, requestMessages(request.body)),
|
||||
events: suffix,
|
||||
}
|
||||
return {
|
||||
value,
|
||||
|
||||
@@ -1,34 +1,6 @@
|
||||
/** Wire types for lossless incremental DeepSeek session-log upload. */
|
||||
|
||||
import type { JsonValue, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/** Path from one DeepSeek wire message root to a string value. */
|
||||
export type DeepSeekMessageStringPath = readonly (string | number)[]
|
||||
|
||||
/** Exact half-open UTF-8 slice of one string in the containing request's messages. */
|
||||
export interface DeepSeekMessageStringSlice {
|
||||
readonly messageIndex: number
|
||||
readonly path: DeepSeekMessageStringPath
|
||||
readonly utf8Start: number
|
||||
readonly utf8End: number
|
||||
}
|
||||
|
||||
/** One literal or request-relative fragment of a packed JSON string. */
|
||||
export type PackedJsonStringPart =
|
||||
| { readonly kind: 'literal'; readonly value: string }
|
||||
| { readonly kind: 'message-slice'; readonly value: DeepSeekMessageStringSlice }
|
||||
|
||||
/** Tagged JSON representation whose string leaves may cite the containing request. */
|
||||
export type PackedJsonValue =
|
||||
| { readonly kind: 'literal'; readonly value: JsonValue }
|
||||
| { readonly kind: 'string'; readonly parts: readonly PackedJsonStringPart[] }
|
||||
| { readonly kind: 'array'; readonly items: readonly PackedJsonValue[] }
|
||||
| { readonly kind: 'object'; readonly entries: readonly (readonly [string, PackedJsonValue])[] }
|
||||
|
||||
/** One canonical session event, sent raw unless request-relative references reduce its encoded bytes. */
|
||||
export type EncodedSessionEvent =
|
||||
| { readonly encoding: 'raw'; readonly event: SessionEvent }
|
||||
| { readonly encoding: 'message-references'; readonly event: PackedJsonValue }
|
||||
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/** Versioned incremental session-log field carried by an official DeepSeek request. */
|
||||
export interface DeepSeekSessionLogExtension {
|
||||
@@ -38,8 +10,8 @@ export interface DeepSeekSessionLogExtension {
|
||||
readonly afterSeq: number
|
||||
/** Highest sequence represented by {@link events}. */
|
||||
readonly throughSeq: number
|
||||
/** Contiguous canonical events from `afterSeq + 1` through `throughSeq`. */
|
||||
readonly events: readonly EncodedSessionEvent[]
|
||||
/** Complete canonical event envelopes for every sequence from `afterSeq + 1` through `throughSeq`. */
|
||||
readonly events: readonly SessionEvent[]
|
||||
}
|
||||
|
||||
declare module '@deepseek-ai/dsh-deepseek-llm-api-extensions/types' {
|
||||
|
||||
@@ -1,139 +0,0 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import type { DeepSeekLlmApiJson } from '@deepseek-ai/dsh-deepseek-llm-api-extensions'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
packSessionEvents,
|
||||
unpackJsonValue,
|
||||
unpackSessionEvents,
|
||||
} from '../src/codec.ts'
|
||||
import type { PackedJsonValue } from '../src/types.ts'
|
||||
|
||||
function event(data: unknown, seq = 0): SessionEvent {
|
||||
return {
|
||||
type: 'plugin/test',
|
||||
seq,
|
||||
time: 1_700_000_000_000 + seq,
|
||||
data,
|
||||
} as unknown as SessionEvent
|
||||
}
|
||||
|
||||
describe('DeepSeek session-log codec', () => {
|
||||
it('replaces a raw assistant chunk with an exact slice of its assembled wire message', () => {
|
||||
const delta = 'streamed assistant content '.repeat(30)
|
||||
const source = {
|
||||
type: 'assistant/chunk',
|
||||
seq: 0,
|
||||
time: 1,
|
||||
data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: delta } },
|
||||
} as SessionEvent<'assistant/chunk'>
|
||||
const messages: DeepSeekLlmApiJson[] = [{ role: 'assistant', content: `prefix:${delta}:suffix` }]
|
||||
const packed = packSessionEvents([source], messages)
|
||||
|
||||
expect(packed[0]?.encoding).toBe('message-references')
|
||||
expect(unpackSessionEvents(packed, messages)).toEqual([source])
|
||||
})
|
||||
|
||||
it('references exact whole strings and reconstructs Unicode event values', () => {
|
||||
const text = `前缀-${'shared text '.repeat(40)}-结尾`
|
||||
const messages: DeepSeekLlmApiJson[] = [{ role: 'assistant', content: text }]
|
||||
const source = event({ nested: [{ text }], untouched: 3 })
|
||||
const packed = packSessionEvents([source], messages)
|
||||
|
||||
expect(packed[0]?.encoding).toBe('message-references')
|
||||
expect(JSON.stringify(packed)).toContain('message-slice')
|
||||
expect(unpackSessionEvents(packed, messages)).toEqual([source])
|
||||
})
|
||||
|
||||
it('keeps surrogate-splitting and ill-formed UTF-16 strings raw', () => {
|
||||
const high = String.fromCharCode(0xD83D)
|
||||
const low = String.fromCharCode(0xDE00)
|
||||
const tail = 'shared-tail-'.repeat(80)
|
||||
const cases = [
|
||||
{ message: `😀${tail}`, logged: `${low}${tail}` },
|
||||
{ message: `${tail}😀`, logged: `${tail}${high}` },
|
||||
{ message: `${high}${tail}`, logged: `${high}${tail}` },
|
||||
]
|
||||
|
||||
for (const item of cases) {
|
||||
const source = event({ text: item.logged })
|
||||
const messages: DeepSeekLlmApiJson[] = [{ role: 'assistant', content: item.message }]
|
||||
const packed = packSessionEvents([source], messages)
|
||||
expect(packed).toEqual([{ encoding: 'raw', event: source }])
|
||||
expect(unpackSessionEvents(packed, messages)).toEqual([source])
|
||||
}
|
||||
})
|
||||
|
||||
it('references one large inner message string and preserves literal prefix and suffix', () => {
|
||||
const shared = 'payload '.repeat(80)
|
||||
const messages: DeepSeekLlmApiJson[] = [{ role: 'tool', content: shared }]
|
||||
const source = event({ output: `before:${shared}:after` })
|
||||
const packed = packSessionEvents([source], messages)
|
||||
|
||||
expect(packed[0]?.encoding).toBe('message-references')
|
||||
expect(unpackSessionEvents(packed, messages)).toEqual([source])
|
||||
})
|
||||
|
||||
it('keeps short or unrelated events raw when references would expand them', () => {
|
||||
const messages: DeepSeekLlmApiJson[] = [{ role: 'user', content: 'tiny', empty: '', parts: [null, true, 5, 'nested'] }]
|
||||
const source = event({ text: 'tiny', empty: '', other: ['unrelated', true, null] })
|
||||
const packed = packSessionEvents([source], messages)
|
||||
expect(packed).toEqual([{ encoding: 'raw', event: source }])
|
||||
expect(unpackSessionEvents(packed, messages)).toEqual([source])
|
||||
})
|
||||
|
||||
it('falls back to a raw event when referenced children do not reduce the complete envelope', () => {
|
||||
let found = false
|
||||
for (let length = 40; length <= 240; length += 1) {
|
||||
const text = 'x'.repeat(length)
|
||||
const source = event({ text })
|
||||
const packed = packSessionEvents([source], [{ content: text }])
|
||||
if (packed[0]?.encoding === 'raw' && length > 80) found = true
|
||||
}
|
||||
expect(found).toBe(true)
|
||||
})
|
||||
|
||||
it('rejects missing paths, non-string targets, invalid ranges, and split UTF-8 code points', () => {
|
||||
const messages: DeepSeekLlmApiJson[] = [{ content: '😀abc', nested: [5] }]
|
||||
const packed = (value: object): PackedJsonValue => ({
|
||||
kind: 'string',
|
||||
parts: [{ kind: 'message-slice', value: value as never }],
|
||||
})
|
||||
expect(() => unpackJsonValue(packed({ messageIndex: 2, path: ['content'], utf8Start: 0, utf8End: 1 }), messages))
|
||||
.toThrow(/index 2 is absent/)
|
||||
expect(() => unpackJsonValue(packed({ messageIndex: 0, path: ['missing'], utf8Start: 0, utf8End: 1 }), messages))
|
||||
.toThrow(/invalid object segment/)
|
||||
expect(() => unpackJsonValue(packed({ messageIndex: 0, path: ['nested', 0], utf8Start: 0, utf8End: 1 }), messages))
|
||||
.toThrow(/does not resolve to a string/)
|
||||
expect(() => unpackJsonValue(packed({ messageIndex: 0, path: ['nested', 2], utf8Start: 0, utf8End: 1 }), messages))
|
||||
.toThrow(/invalid array segment/)
|
||||
expect(() => unpackJsonValue(packed({ messageIndex: 0, path: ['content'], utf8Start: -1, utf8End: 1 }), messages))
|
||||
.toThrow(/byte range is invalid/)
|
||||
expect(() => unpackJsonValue(packed({ messageIndex: 0, path: ['content'], utf8Start: 0, utf8End: 1 }), messages))
|
||||
.toThrow(/splits a UTF-8 code point/)
|
||||
})
|
||||
|
||||
it('decodes every packed JSON variant and rejects unknown tags', () => {
|
||||
const messages: DeepSeekLlmApiJson[] = [{ content: 'abcdef' }]
|
||||
const value: PackedJsonValue = {
|
||||
kind: 'object',
|
||||
entries: [[
|
||||
'items',
|
||||
{
|
||||
kind: 'array',
|
||||
items: [
|
||||
{ kind: 'literal', value: 1 },
|
||||
{
|
||||
kind: 'string',
|
||||
parts: [
|
||||
{ kind: 'literal', value: 'x' },
|
||||
{ kind: 'message-slice', value: { messageIndex: 0, path: ['content'], utf8Start: 1, utf8End: 4 } },
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
]],
|
||||
}
|
||||
expect(unpackJsonValue(value, messages)).toEqual({ items: [1, 'xbcd'] })
|
||||
expect(() => unpackJsonValue({ kind: 'future' } as never, messages)).toThrow(/unknown packed JSON value/)
|
||||
})
|
||||
})
|
||||
@@ -66,8 +66,8 @@ describe('incremental DeepSeek session-log upload', () => {
|
||||
expect(second.fields.dsh_session_log).toMatchObject({ afterSeq: 1, throughSeq: 3 })
|
||||
expect(second.fields.dsh_session_log?.events).toHaveLength(2)
|
||||
expect(second.fields.dsh_session_log?.events[0]).toMatchObject({
|
||||
encoding: 'raw',
|
||||
event: { type: 'session-log-deepseek/delivery-accepted', seq: 2 },
|
||||
type: 'session-log-deepseek/delivery-accepted',
|
||||
seq: 2,
|
||||
})
|
||||
})
|
||||
|
||||
@@ -147,15 +147,15 @@ describe('incremental DeepSeek session-log upload', () => {
|
||||
expect(current.fields.dsh_session_log).toMatchObject({
|
||||
afterSeq: 0,
|
||||
throughSeq: 1,
|
||||
events: [{ event: { type: 'session-log-deepseek/delivery-accepted' } }],
|
||||
events: [{ type: 'session-log-deepseek/delivery-accepted' }],
|
||||
})
|
||||
})
|
||||
|
||||
it('fails preparation when the DeepSeek body has no messages array', async () => {
|
||||
const { ctx, session } = await harness('bad-body')
|
||||
it('contributes complete events without reading request messages', async () => {
|
||||
const { ctx, session } = await harness('direct-events')
|
||||
session.append('turn/start', { turn: 1 })
|
||||
await expect(ctx.deepseekLlmApiExtensions.prepare({ body: {}, signal: SIGNAL, sessionId: session.id }))
|
||||
.rejects.toThrow(/no messages array/)
|
||||
const prepared = await ctx.deepseekLlmApiExtensions.prepare({ body: {}, signal: SIGNAL, sessionId: session.id })
|
||||
expect(prepared.fields.dsh_session_log?.events).toEqual(session.events)
|
||||
})
|
||||
|
||||
it('fails closed on a malformed persisted acceptance watermark', async () => {
|
||||
|
||||
Reference in new issue
Block a user