fix(telemetry): own byte-bounded session request scheduling

This commit is contained in:
07akioni committed 2026-09-25 18:39:21 +08:00
1 parent 7b8c7dfff2
commit 3dd52bfd9b
42 files changed
+672 -740

No files matched your search

@@ -9,11 +9,11 @@
en: 20f659a15fb55352
zh: 49d39c95da1b8aa5
/agent-note-session-log-otlp-strings-and-bounded-requests/decision:
en: eefcb4a88e478526
zh: b1f38662efcfddf8
en: 06c1b11a1666c42e
zh: 6319b67e8afe0d6f
/agent-note-session-log-otlp-strings-and-bounded-requests/alternatives-considered:
en: 0dc933b18a1a3573
zh: 77655740322c4e45
/agent-note-session-log-otlp-strings-and-bounded-requests/consequences:
en: ec1f5f8cf94e38d3
zh: 21dcfe6765881aa7
en: 3859569c9cdea30c
zh: 517449b8c49807f6
@@ -10,11 +10,11 @@ Session logs contain nested JSON and large tool results. The product collector a
## Decision
The product OTel package provides one Session-log reporter shared by its service and the feedback backend. Each canonical event becomes `eventName: "session-log"`, with `sessionId` and the JSON-stringified full event in `content` attributes. Redaction applies before serialization; the capture record carries an envelope without a duplicate payload.
The Session OTel backend owns the Session-log reporter. The product service owns only product events, with no Session types or unused Session queue. Each canonical event becomes `eventName: "session-log"`, with `sessionId` and JSON-stringified event values in `content`; serialization does not preserve JSONL key ordering. Redaction runs before serialization and the copied envelope excludes the payload.
Session logs use their own SDK provider and queue. The exporter measures complete uncompressed OTLP JSON using the transport's SDK serializer and splits batches to at most 4,000,000 bytes. A single oversized event is rejected with a diagnostic and does not prevent other events from being sent. Attribute truncation is disabled for this queue. The owner supplies a bounded shutdown that waits for split exports.
Session logs use a separate provider and byte/count queue. Each record is measured once with the SDK serializer including a full envelope; the sum conservatively bounds a combined request. Greedy packing avoids full-candidate serialization and recursive re-encoding. Single oversized records receive one diagnostic and do not block later events. One HTTP request owns the transport slot until settlement, even after its watchdog fires; shutdown expiry stops the remaining queue. Scope name/version and explicit transport options remain owned by the backend.
The [telemetry revival](../feature/2026-07-23-session-telemetry-otel-revival.md) remains authoritative for redaction, capture, and best-effort handoff. This decision replaces its SDK-only batching rule for Session-log request sizing; the SDK still owns queueing cadence and transport retry. Feedback authorization and the separate DeepSeek model-request contribution remain unchanged.
The [telemetry revival](../feature/2026-07-23-session-telemetry-otel-revival.md) remains authoritative for redaction, capture, and best-effort handoff. This decision replaces its SDK-only queue/batching rule: SDK batch deadlines cannot own an operation split into multiple HTTP requests. The backend schedules complete requests; the SDK still owns transport/retry. Feedback authorization and the separate DeepSeek model-request contribution remain unchanged.
## Alternatives considered
@@ -26,4 +26,4 @@ The [telemetry revival](../feature/2026-07-23-session-telemetry-otel-revival.md)
## Consequences
Product and Session records share routing configuration but never a request. Queue overflow, single-event rejection, process exit, and transport failures can still lose data; no durable outbox or collector acknowledgement is added. Tests observe exact byte limits, nested JSON round trips, separated requests, oversize continuation, and feedback-authorized recorded-session replay.
Product and Session records never share a request or queue, and neither package imports the other. Conservative per-record envelope accounting can produce smaller batches than exact combined accounting. Queue overflow, oversize rejection, exit, and transport failure can lose data; no durable outbox or acknowledgement is added. Tests cover byte limits, linear measurement, slow-request serialization, metadata, redaction-copy isolation, and feedback-authorized replay.
@@ -10,11 +10,11 @@ Session 日志包含嵌套 JSON 和大型工具结果。产品采集端接收带
## 决策
产品 OTel 包提供一个由其服务和反馈后端共享的 Session 日志上报器。每条规范事件转换为 `eventName: "session-log"`,属性包含 `sessionId` 和存放完整事件 JSON 字符串的 `content`。脱敏在序列化前执行;采集记录携带不重复包含载荷的信封。
Session OTel 后端拥有 Session 日志 reporter。产品服务仅负责产品事件,不依赖 Session 类型,也不创建无人使用的 Session 队列。每条规范事件对应 `eventName: "session-log"`,属性包含 `sessionId` 和 JSON 字符串形式的事件值 `content`;序列化不保留 JSONL 键顺序。脱敏在序列化前执行,复制的 envelope 不含 payload。
Session 日志使用专属 SDK provider 和队列。导出器使用传输的 SDK 序列化器计量完整未压缩 OTLP JSON,并将批次拆分到最多 4,000,000 字节。单条超限事件被拒绝并记录诊断,不阻止其他事件发送。该队列禁用属性截断。拥有方提供有界关闭过程,等待拆分导出完成。
Session 日志使用独立 provider 和字节/条数队列。每条记录使用 SDK 序列化器计量一次,包含完整 envelope;其总和保守约束合并请求大小。顺序组包避免候选批次的完整序列化和递归重复编码。单条超限记录只产生一次诊断,不阻塞后续事件。每个 HTTP 请求在结束前始终占有传输槽位,即使其监测期限已经触发;关闭期限到期后停止剩余队列。scope 名称/版本和显式传输选项仍由后端拥有。
[遥测恢复决策](../feature/2026-07-23-session-telemetry-otel-revival.zh.md) 仍负责脱敏、采集和尽力交接。本决策在 Session 日志请求计量方面替代其仅由 SDK 分批的规则;SDK 仍负责排队节奏和传输重试。反馈授权与独立的 DeepSeek 模型请求贡献保持不变。
[遥测恢复决策](../feature/2026-07-23-session-telemetry-otel-revival.zh.md) 继续约束脱敏、捕获和尽力交接。本决策替代其仅使用 SDK 队列/聚合的规则:SDK 批次期限无法拥有拆成多个 HTTP 请求的操作。后端调度完整请求,SDK 仍负责传输和重试。反馈授权和独立的 DeepSeek 模型请求附加路径不变。
## 考虑过的替代方案
@@ -26,4 +26,4 @@ Session 日志使用专属 SDK provider 和队列。导出器使用传输的 SDK
## 影响
产品与 Session 记录共享路由配置,但绝不共用请求。队列溢出、单条事件拒绝、进程退出和传输失败仍可能丢失数据;不增加持久化 outbox 或采集端确认。测试观察精确字节上限、嵌套 JSON 往返、请求分离、超限后继续发送,以及反馈授权的录制会话回放。
产品事件和 Session 记录不共享请求或队列,两个包也不相互导入。保守的逐记录 envelope 计量可能比精确合并计量产生更小批次。队列溢出、超限拒绝、退出和传输失败可能丢失数据;不新增持久 outbox 或确认机制。测试覆盖字节上限、线性计量、慢请求串行调度、元数据、脱敏复制隔离和反馈授权回放。
+7 -27
View File
@@ -1435,8 +1435,7 @@ export interface Config {
## `@deepseek-ai/dsh-host-product-telemetry-otel`
- `refs`: `Attributes` (`@opentelemetry/api`) · `BatchLogRecordProcessorOptions` (`@opentelemetry/sdk-logs`) · `OTLPExporterNodeConfigBase` (`@opentelemetry/otlp-exporter-base`)
- `source`: [`packages/host/product-telemetry-otel/src/index.ts:38`](../packages/host/product-telemetry-otel/src/index.ts)
- `source`: [`packages/host/product-telemetry-otel/src/index.ts:37`](../packages/host/product-telemetry-otel/src/index.ts)
```ts config-catalog
/** Collector routing, application identity, and bounded in-memory batch settings. */
@@ -1463,25 +1462,6 @@ export interface Config {
exportTimeoutMillis: number
/** Outer shutdown wait; pending exports may be lost after this deadline. */
shutdownTimeoutMillis: number
/** Independent Session-log batching and byte limit; product-event settings do not apply. */
sessionLog?: Pick<SessionLogOptions, 'processor' | 'maxRequestBytes'>
}
/** Independent Session-log transport and queue settings. */
export interface SessionLogOptions {
/** Explicit destination and SDK transport options. */
exporter: OTLPExporterNodeConfigBase & {
/** Full HTTP(S) logs destination. */
url: string
}
/** Session-only queue settings, independent of product-event aggregation. */
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
/** May lower, but never exceed, the collector's 4,000,000-byte limit. */
maxRequestBytes?: number
/** Application and anonymous identity carried on the OTLP resource. */
resourceAttributes: Attributes
/** Report rejected single records and network failures without recording their content. */
onFailure: (message: string, error?: Error) => void
}
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-host-product-telemetry-otel -->
@@ -2727,24 +2707,24 @@ export interface Config {
```ts config-catalog
/**
* Plugin configuration: one sharing policy, two verbatim SDK option objects,
* and one DSH-owned shutdown bound. Uploading modes validate their endpoint
* Plugin configuration: sharing policy, SDK transport options, byte/count queue
* settings, and an overall shutdown bound. Uploading modes validate their endpoint
* and shutdown deadline at plugin load; `DISABLED` reads neither.
*/
export interface Config {
/** Defaults to `FEEDBACK_ONLY`: capture session history only when feedback is explicitly submitted. */
mode?: SessionTelemetryMode
/**
* Explicit SDK HTTP transport settings. The reporter adds x-channel routing
* and does not inherit ambient credentials. URL is required while uploading.
* Explicit SDK HTTP transport settings, including optional routing headers.
* Ambient credentials are not inherited. URL is required while uploading.
*/
exporter?: OTLPExporterNodeConfigBase & {
/** Full logs endpoint (e.g. `https://collector.example.com/v1/logs`). Required outside `DISABLED`; validated at load. */
url?: string
}
/**
* Passed verbatim to `BatchLogRecordProcessor` (minus the exporter slot,
* which this plugin fills); the SDK owns and documents these knobs.
* Count, queue, cadence, and per-request watchdog settings for the byte-bounded
* processor. A watchdog warning never releases an unsettled transport slot.
*/
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
/** Maximum time spent awaiting the SDK provider's complete shutdown path. */
+7 -27
View File
@@ -1437,8 +1437,7 @@ export interface Config {
## `@deepseek-ai/dsh-host-product-telemetry-otel`
- `refs`: `Attributes` (`@opentelemetry/api`) · `BatchLogRecordProcessorOptions` (`@opentelemetry/sdk-logs`) · `OTLPExporterNodeConfigBase` (`@opentelemetry/otlp-exporter-base`)
- `source`: [`packages/host/product-telemetry-otel/src/index.ts:38`](../packages/host/product-telemetry-otel/src/index.ts)
- `source`: [`packages/host/product-telemetry-otel/src/index.ts:37`](../packages/host/product-telemetry-otel/src/index.ts)
```ts config-catalog
/** Collector routing, application identity, and bounded in-memory batch settings. */
@@ -1465,25 +1464,6 @@ export interface Config {
exportTimeoutMillis: number
/** Outer shutdown wait; pending exports may be lost after this deadline. */
shutdownTimeoutMillis: number
/** Independent Session-log batching and byte limit; product-event settings do not apply. */
sessionLog?: Pick<SessionLogOptions, 'processor' | 'maxRequestBytes'>
}
/** Independent Session-log transport and queue settings. */
export interface SessionLogOptions {
/** Explicit destination and SDK transport options. */
exporter: OTLPExporterNodeConfigBase & {
/** Full HTTP(S) logs destination. */
url: string
}
/** Session-only queue settings, independent of product-event aggregation. */
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
/** May lower, but never exceed, the collector's 4,000,000-byte limit. */
maxRequestBytes?: number
/** Application and anonymous identity carried on the OTLP resource. */
resourceAttributes: Attributes
/** Report rejected single records and network failures without recording their content. */
onFailure: (message: string, error?: Error) => void
}
```
<!-- END GENERATED config-catalog:@deepseek-ai/dsh-host-product-telemetry-otel -->
@@ -2729,24 +2709,24 @@ export interface Config {
```ts config-catalog
/**
* Plugin configuration: one sharing policy, two verbatim SDK option objects,
* and one DSH-owned shutdown bound. Uploading modes validate their endpoint
* Plugin configuration: sharing policy, SDK transport options, byte/count queue
* settings, and an overall shutdown bound. Uploading modes validate their endpoint
* and shutdown deadline at plugin load; `DISABLED` reads neither.
*/
export interface Config {
/** Defaults to `FEEDBACK_ONLY`: capture session history only when feedback is explicitly submitted. */
mode?: SessionTelemetryMode
/**
* Explicit SDK HTTP transport settings. The reporter adds x-channel routing
* and does not inherit ambient credentials. URL is required while uploading.
* Explicit SDK HTTP transport settings, including optional routing headers.
* Ambient credentials are not inherited. URL is required while uploading.
*/
exporter?: OTLPExporterNodeConfigBase & {
/** Full logs endpoint (e.g. `https://collector.example.com/v1/logs`). Required outside `DISABLED`; validated at load. */
url?: string
}
/**
* Passed verbatim to `BatchLogRecordProcessor` (minus the exporter slot,
* which this plugin fills); the SDK owns and documents these knobs.
* Count, queue, cadence, and per-request watchdog settings for the byte-bounded
* processor. A watchdog warning never releases an unsettled transport slot.
*/
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
/** Maximum time spent awaiting the SDK provider's complete shutdown path. */
+2 -4
View File
@@ -496,7 +496,6 @@ flowchart TD
pkg_experimental_webworker_runtime --> pkg_host_webserver
pkg_host_frontend_static --> pkg_client_connection
pkg_host_frontend_static --> pkg_host_webserver
pkg_host_product_telemetry_otel --> pkg_session
pkg_persona --> pkg_system_prompt
pkg_sandbox --> pkg_llm
pkg_sandbox --> pkg_session
@@ -1036,7 +1035,6 @@ flowchart TD
pkg_agent_preset --> pkg_agent_preset_registry
pkg_session_telemetry_otel --> pkg_anonymous_user_id
pkg_session_telemetry_otel --> pkg_command_feedback
pkg_session_telemetry_otel --> pkg_host_product_telemetry_otel
pkg_session_telemetry_otel --> pkg_llm
pkg_session_telemetry_otel --> pkg_message_feedback
pkg_session_telemetry_otel --> pkg_session
@@ -1464,6 +1462,7 @@ flowchart TD
| [`host-directory-picker-browse`](../packages/host/directory-picker-browse) | `host` | — |
| [`host-directory-picker-native`](../packages/host/directory-picker-native) | `host` | — |
| [`host-open-in-app`](../packages/host/open-in-app) | `host` | — |
| [`host-product-telemetry-otel`](../packages/host/product-telemetry-otel) | `host` | — |
| [`host-webserver`](../packages/host/webserver) | `host` | — |
| [`invariants`](../packages/runtime-diagnostics/invariants) | `runtime-diagnostics` | — |
| [`session-format`](../packages/session/session-format) | `session` | — |
@@ -1516,7 +1515,6 @@ flowchart TD
| [`app-boot`](../packages/boot/app-boot) | `boot` | [`home-paths`](../packages/util/home-paths), [`launch-environment`](../packages/util/launch-environment), [`system-prompt`](../packages/core/system-prompt) |
| [`experimental-webworker-runtime`](../packages/experimental/webworker-runtime) | `experimental` | [`client-connection`](../packages/client/connection), [`client-modules`](../packages/client/modules), [`host-webserver`](../packages/host/webserver) |
| [`host-frontend-static`](../packages/host/frontend-static) | `host` | [`client-connection`](../packages/client/connection), [`host-webserver`](../packages/host/webserver) |
| [`host-product-telemetry-otel`](../packages/host/product-telemetry-otel) | `host` | [`session`](../packages/core/session) |
| [`persona`](../packages/preset/persona) | `preset` | [`system-prompt`](../packages/core/system-prompt) |
| [`sandbox`](../packages/sandbox/sandbox) | `sandbox` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
| [`session-format-catalog`](../packages/session/session-format-catalog) | `session` | [`session`](../packages/core/session) |
@@ -1639,7 +1637,7 @@ flowchart TD
| [`host-plugin-inventory`](../packages/host/plugin-inventory) | `host` | [`agent-preset-registry`](../packages/preset/agent-preset-registry), [`brand`](../packages/util/brand), [`typert-protocol`](../packages/typert/protocol) |
| [`mcp-client`](../packages/mcp/mcp-client) | `mcp` | [`attachment`](../packages/attachment/attachment), [`llm`](../packages/llm/llm), [`mcp-resources`](../packages/mcp/mcp-resources), [`scope`](../packages/core/scope), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`timeout`](../packages/util/timeout), [`tools`](../packages/core/tools) |
| [`agent-preset`](../packages/preset/agent-preset) | `preset` | [`agent-preset-registry`](../packages/preset/agent-preset-registry) |
| [`session-telemetry-otel`](../packages/session/session-telemetry-otel) | `session` | [`anonymous-user-id`](../packages/identity/anonymous-user-id), [`command-feedback`](../packages/feedback/command-feedback), [`host-product-telemetry-otel`](../packages/host/product-telemetry-otel), [`llm`](../packages/llm/llm), [`message-feedback`](../packages/feedback/message-feedback), [`session`](../packages/core/session), [`session-telemetry`](../packages/session/session-telemetry) |
| [`session-telemetry-otel`](../packages/session/session-telemetry-otel) | `session` | [`anonymous-user-id`](../packages/identity/anonymous-user-id), [`command-feedback`](../packages/feedback/command-feedback), [`llm`](../packages/llm/llm), [`message-feedback`](../packages/feedback/message-feedback), [`session`](../packages/core/session), [`session-telemetry`](../packages/session/session-telemetry) |
| [`tool-bash`](../packages/shell/tool-bash) | `shell` | [`agent`](../packages/core/agent), [`jobs`](../packages/jobs/jobs), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`shell`](../packages/shell/shell), [`shell-env`](../packages/shell/shell-env), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
| [`tool-pwsh`](../packages/shell/tool-pwsh) | `shell` | [`agent`](../packages/core/agent), [`jobs`](../packages/jobs/jobs), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`shell`](../packages/shell/shell), [`shell-env`](../packages/shell/shell-env), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
| [`fs-ssh`](../packages/ssh/fs-ssh) | `ssh` | [`fs`](../packages/fs/fs), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`ssh`](../packages/ssh/ssh) |
+2 -4
View File
@@ -498,7 +498,6 @@ flowchart TD
pkg_experimental_webworker_runtime --> pkg_host_webserver
pkg_host_frontend_static --> pkg_client_connection
pkg_host_frontend_static --> pkg_host_webserver
pkg_host_product_telemetry_otel --> pkg_session
pkg_persona --> pkg_system_prompt
pkg_sandbox --> pkg_llm
pkg_sandbox --> pkg_session
@@ -1038,7 +1037,6 @@ flowchart TD
pkg_agent_preset --> pkg_agent_preset_registry
pkg_session_telemetry_otel --> pkg_anonymous_user_id
pkg_session_telemetry_otel --> pkg_command_feedback
pkg_session_telemetry_otel --> pkg_host_product_telemetry_otel
pkg_session_telemetry_otel --> pkg_llm
pkg_session_telemetry_otel --> pkg_message_feedback
pkg_session_telemetry_otel --> pkg_session
@@ -1466,6 +1464,7 @@ flowchart TD
| [`host-directory-picker-browse`](../packages/host/directory-picker-browse) | `host` | — |
| [`host-directory-picker-native`](../packages/host/directory-picker-native) | `host` | — |
| [`host-open-in-app`](../packages/host/open-in-app) | `host` | — |
| [`host-product-telemetry-otel`](../packages/host/product-telemetry-otel) | `host` | — |
| [`host-webserver`](../packages/host/webserver) | `host` | — |
| [`invariants`](../packages/runtime-diagnostics/invariants) | `runtime-diagnostics` | — |
| [`session-format`](../packages/session/session-format) | `session` | — |
@@ -1518,7 +1517,6 @@ flowchart TD
| [`app-boot`](../packages/boot/app-boot) | `boot` | [`home-paths`](../packages/util/home-paths), [`launch-environment`](../packages/util/launch-environment), [`system-prompt`](../packages/core/system-prompt) |
| [`experimental-webworker-runtime`](../packages/experimental/webworker-runtime) | `experimental` | [`client-connection`](../packages/client/connection), [`client-modules`](../packages/client/modules), [`host-webserver`](../packages/host/webserver) |
| [`host-frontend-static`](../packages/host/frontend-static) | `host` | [`client-connection`](../packages/client/connection), [`host-webserver`](../packages/host/webserver) |
| [`host-product-telemetry-otel`](../packages/host/product-telemetry-otel) | `host` | [`session`](../packages/core/session) |
| [`persona`](../packages/preset/persona) | `preset` | [`system-prompt`](../packages/core/system-prompt) |
| [`sandbox`](../packages/sandbox/sandbox) | `sandbox` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
| [`session-format-catalog`](../packages/session/session-format-catalog) | `session` | [`session`](../packages/core/session) |
@@ -1641,7 +1639,7 @@ flowchart TD
| [`host-plugin-inventory`](../packages/host/plugin-inventory) | `host` | [`agent-preset-registry`](../packages/preset/agent-preset-registry), [`brand`](../packages/util/brand), [`typert-protocol`](../packages/typert/protocol) |
| [`mcp-client`](../packages/mcp/mcp-client) | `mcp` | [`attachment`](../packages/attachment/attachment), [`llm`](../packages/llm/llm), [`mcp-resources`](../packages/mcp/mcp-resources), [`scope`](../packages/core/scope), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`timeout`](../packages/util/timeout), [`tools`](../packages/core/tools) |
| [`agent-preset`](../packages/preset/agent-preset) | `preset` | [`agent-preset-registry`](../packages/preset/agent-preset-registry) |
| [`session-telemetry-otel`](../packages/session/session-telemetry-otel) | `session` | [`anonymous-user-id`](../packages/identity/anonymous-user-id), [`command-feedback`](../packages/feedback/command-feedback), [`host-product-telemetry-otel`](../packages/host/product-telemetry-otel), [`llm`](../packages/llm/llm), [`message-feedback`](../packages/feedback/message-feedback), [`session`](../packages/core/session), [`session-telemetry`](../packages/session/session-telemetry) |
| [`session-telemetry-otel`](../packages/session/session-telemetry-otel) | `session` | [`anonymous-user-id`](../packages/identity/anonymous-user-id), [`command-feedback`](../packages/feedback/command-feedback), [`llm`](../packages/llm/llm), [`message-feedback`](../packages/feedback/message-feedback), [`session`](../packages/core/session), [`session-telemetry`](../packages/session/session-telemetry) |
| [`tool-bash`](../packages/shell/tool-bash) | `shell` | [`agent`](../packages/core/agent), [`jobs`](../packages/jobs/jobs), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`shell`](../packages/shell/shell), [`shell-env`](../packages/shell/shell-env), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
| [`tool-pwsh`](../packages/shell/tool-pwsh) | `shell` | [`agent`](../packages/core/agent), [`jobs`](../packages/jobs/jobs), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`shell`](../packages/shell/shell), [`shell-env`](../packages/shell/shell-env), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
| [`fs-ssh`](../packages/ssh/fs-ssh) | `ssh` | [`fs`](../packages/fs/fs), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`ssh`](../packages/ssh/ssh) |
+2 -2
View File
@@ -3,8 +3,8 @@
# After editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/product-telemetry.md
/product-telemetry:
en: 00f932c8124f931c
zh: 0607741db404976f
en: e1a8950c3b4743ed
zh: 3fff7e01a8f271e1
/product-telemetry/record-types:
en: bf5eadedbc896f5a
zh: 158fce1609a85d6d
+2 -42
View File
@@ -8,8 +8,6 @@ The [product telemetry plugin](../../packages/host/product-telemetry-otel/README
Enqueue is synchronous and does not acknowledge delivery. The SDK owns batching and retry; local diagnostics report export failures. See the package README for configuration, shutdown, and loss limits.
`reportSessionLog()` submits one complete Session event as `eventName: "session-log"` with string attributes `sessionId` and `content`. `SessionLogReporter` exposes the same independently queued sender to feedback backends. The sender measures full OTLP JSON bytes before gzip, splits requests at 4,000,000 bytes or a lower configured limit, and rejects oversized single events without truncation.
## Record types
```ts type-equiv
@@ -35,38 +33,6 @@ interface ProductTelemetryRecord {
Source: [`packages/host/product-telemetry-otel/src/index.ts`](../../packages/host/product-telemetry-otel/src/index.ts)
```ts type-equiv
/** One canonical event with its separately owned Session identity and redacted payload. */
interface SessionLogRecord {
sessionId: SessionId
/** Complete event envelope; data is the capture policy's exported copy. */
event: Omit<SessionEvent, 'data'> & { data: unknown }
/** Additional capture metadata; sessionId and content are always assigned by the reporter. */
attributes?: Attributes
/** Omitted values use INFO. */
severityNumber?: SeverityNumber
}
```
```ts type-equiv
/** Independent Session-log transport and queue settings. */
interface SessionLogOptions {
/** Explicit destination and SDK transport options. */
exporter: OTLPExporterNodeConfigBase & {
/** Full HTTP(S) logs destination. */
url: string
}
/** Session-only queue settings, independent of product-event aggregation. */
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
/** May lower, but never exceed, the collector's 4,000,000-byte limit. */
maxRequestBytes?: number
/** Application and anonymous identity carried on the OTLP resource. */
resourceAttributes: Attributes
/** Report rejected single records and network failures without recording their content. */
onFailure: (message: string, error?: Error) => void
}
```
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
<a id="cordis-surface"></a>
@@ -83,15 +49,9 @@ Host analytics sender. Mounting alone sends nothing; the owning fiber drains it
```ts cordis-catalog
/**
* Enqueue one explicitly authorized Session event in the separate byte-bounded queue.
* Enqueue one selected product event without waiting for network delivery.
* Queue admission and shutdown completion are not collector or warehouse acknowledgements.
* @param record - complete Session event and its owning Session id.
*/
reportSessionLog(record: SessionLogRecord): void
/**
* Enqueue a selected product event independently of Session-log reporting.
* @param record - approved analytics fields; no automatic Session collection.
* @param record - caller-owned event containing only approved analytics fields.
*/
emit(record: ProductTelemetryRecord): void
```
+2 -42
View File
@@ -8,8 +8,6 @@
入队同步完成,不代表送达确认。SDK 负责批量发送与重试;发送失败会产生本地诊断。配置、退出和丢失限制见包 README。
`reportSessionLog()` 将一条完整 Session 事件作为 `eventName: "session-log"` 提交,字符串属性为 `sessionId` 和 `content`。`SessionLogReporter` 向反馈后端提供同一个独立排队的发送器。发送器在 gzip 前计量完整 OTLP JSON 字节数,按 4,000,000 字节或更低配置上限拆分请求,并拒绝超限单条事件而不截断。
## 记录类型
```ts type-equiv
@@ -35,38 +33,6 @@ interface ProductTelemetryRecord {
Source: [`packages/host/product-telemetry-otel/src/index.ts`](../../packages/host/product-telemetry-otel/src/index.ts)
```ts type-equiv
/** One canonical event with its separately owned Session identity and redacted payload. */
interface SessionLogRecord {
sessionId: SessionId
/** Complete event envelope; data is the capture policy's exported copy. */
event: Omit<SessionEvent, 'data'> & { data: unknown }
/** Additional capture metadata; sessionId and content are always assigned by the reporter. */
attributes?: Attributes
/** Omitted values use INFO. */
severityNumber?: SeverityNumber
}
```
```ts type-equiv
/** Independent Session-log transport and queue settings. */
interface SessionLogOptions {
/** Explicit destination and SDK transport options. */
exporter: OTLPExporterNodeConfigBase & {
/** Full HTTP(S) logs destination. */
url: string
}
/** Session-only queue settings, independent of product-event aggregation. */
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
/** May lower, but never exceed, the collector's 4,000,000-byte limit. */
maxRequestBytes?: number
/** Application and anonymous identity carried on the OTLP resource. */
resourceAttributes: Attributes
/** Report rejected single records and network failures without recording their content. */
onFailure: (message: string, error?: Error) => void
}
```
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
<a id="cordis-surface"></a>
@@ -83,15 +49,9 @@ Host analytics sender. Mounting alone sends nothing; the owning fiber drains it
```ts cordis-catalog
/**
* Enqueue one explicitly authorized Session event in the separate byte-bounded queue.
* Enqueue one selected product event without waiting for network delivery.
* Queue admission and shutdown completion are not collector or warehouse acknowledgements.
* @param record - complete Session event and its owning Session id.
*/
reportSessionLog(record: SessionLogRecord): void
/**
* Enqueue a selected product event independently of Session-log reporting.
* @param record - approved analytics fields; no automatic Session collection.
* @param record - caller-owned event containing only approved analytics fields.
*/
emit(record: ProductTelemetryRecord): void
```
+2 -2
View File
@@ -3,8 +3,8 @@
# After editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/session-telemetry.md
/sessiontelemetrybackend:
en: b95249b397f4f069
zh: 5ddccf5c7596e6d3
en: b82e4db22a4d58aa
zh: 439181fa2c27542d
/sessiontelemetrybackend/the-logical-record:
en: 9695489076496a76
zh: 457c54698bc38c2f
+1 -1
View File
@@ -2,7 +2,7 @@
English | [中文](session-telemetry.zh.md)
The [capture service](../../packages/session/session-telemetry/README.md) owns canonical-event capture, redaction, and handoff cursors. The [feedback backend](../../packages/session/session-telemetry-otel/README.md) authorizes bounded Session prefixes and submits complete event strings through the [shared OTel reporter](../../packages/host/product-telemetry-otel/README.md), with an independent queue and a 4,000,000-byte request ceiling. No reporting data reaches model requests.
The [capture service](../../packages/session/session-telemetry/README.md) owns canonical-event capture, redaction, and handoff cursors. The [feedback backend](../../packages/session/session-telemetry-otel/README.md) authorizes Session prefixes and owns their independent queue, per-request scheduling, and 4,000,000-byte ceiling. No reporting data reaches model requests.
Source: [`packages/session/session-telemetry/src/index.ts`](../../packages/session/session-telemetry/src/index.ts)
+1 -1
View File
@@ -2,7 +2,7 @@
[English](session-telemetry.md) | 中文
[采集服务](../../packages/session/session-telemetry/README.zh.md) 负责规范事件采集、脱敏和交接游标。[反馈后端](../../packages/session/session-telemetry-otel/README.zh.md) 授权有界 Session 前缀,并通过[共享 OTel 上报器](../../packages/host/product-telemetry-otel/README.zh.md) 提交完整事件字符串,使用独立队列和 4,000,000 字节请求上限。上报数据不会进入模型请求。
[捕获服务](../../packages/session/session-telemetry/README.zh.md) 负责规范事件捕获、脱敏和交接游标。[反馈后端](../../packages/session/session-telemetry-otel/README.zh.md) 授权 Session 前缀,并负责独立队列、逐请求调度及 4,000,000 字节上限。上报数据不会进入模型请求。
源码:[`packages/session/session-telemetry/src/index.ts`](../../packages/session/session-telemetry/src/index.ts)
+6 -9
View File
@@ -192,15 +192,12 @@
# process out (the launchers patch the row disabled; config cannot disable
# a row). Exports carry the harness home's anonymous user id ($DSH_HOME/.anonymous-user-id,
# random UUID; delete the file to reset the identity) as the Resource's
# user.id. The exporter/processor values normally bound the shutdown drain
# to ~1s against an unreachable collector: exporter.timeoutMillis is both
# the per-attempt socket timeout and the retry deadline (1s effectively
# disables the SDK's 5-try backoff), while maxExportBatchSize == maxQueueSize
# (both explicit) makes the drain a single batch. The SDK awaits
# exporter.forceFlush() outside exportTimeoutMillis, so the backend's 3s
# shutdownTimeoutMillis is the load-bearing outer bound when a transport
# promise never settles. Every CLI exit path drains it by disposing the root
# on SIGINT/SIGTERM.
# user.id. Byte-bounded batching may turn an authorized prefix into
# several HTTP requests. Requests run serially; each has its own 1s
# transport timeout and 1.5s watchdog. A watchdog does not release the
# active request slot. The outer 3s shutdown deadline stops queued sends;
# large prefixes or an unsettled transport may leave records unsent.
# Every CLI exit path drains by disposing the root on SIGINT/SIGTERM.
- id: session-telemetry-otel
name: '@deepseek-ai/dsh-session-telemetry-otel'
config:
@@ -1702,15 +1702,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
summary: 'Host analytics sender.',
description: 'Host analytics sender. Mounting alone sends nothing; the owning fiber drains it on unload.',
methods: [
{
signature: 'reportSessionLog(record: SessionLogRecord): void',
description: 'Enqueue one explicitly authorized Session event in the separate byte-bounded queue. Queue admission and shutdown completion are not collector or warehouse acknowledgements.',
parameters: [{ name: 'record', description: 'complete Session event and its owning Session id.' }],
},
{
signature: 'emit(record: ProductTelemetryRecord): void',
description: 'Enqueue a selected product event independently of Session-log reporting.',
parameters: [{ name: 'record', description: 'approved analytics fields; no automatic Session collection.' }],
description: 'Enqueue one selected product event without waiting for network delivery. Queue admission and shutdown completion are not collector or warehouse acknowledgements.',
parameters: [{ name: 'record', description: 'caller-owned event containing only approved analytics fields.' }],
},
],
},
@@ -6566,10 +6561,6 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'SessionLogOffset',
declaration: 'export type SessionLogOffset = BrandedNumber<\'SessionLogOffset\'>;',
},
{
name: 'SessionLogRecord',
declaration: 'export interface SessionLogRecord {\n sessionId: SessionId;\n event: Omit<SessionEvent, \'data\'> & {\n data: unknown;\n };\n attributes?: Attributes;\n severityNumber?: SeverityNumber;\n}',
},
{
name: 'SessionLogSnapshot',
declaration: 'export interface SessionLogSnapshot {\n session: SessionHeader;\n inheritedEventCount: SessionLogOffset;\n events: SessionEvent[];\n}',
@@ -9,17 +9,17 @@
en: 18d7a8c944938416
zh: 4249a421e3d5f09a
/deepseek-ai-dsh-host-product-telemetry-otel/summary:
en: 791225419df7493e
zh: 728b142fcfe9d6e6
en: a969cf533f6649b5
zh: 8401a4263f70a8e0
/deepseek-ai-dsh-host-product-telemetry-otel/table-of-contents:
en: d152484eb41ac6b4
zh: 9aa34fb1dc95d8cf
/deepseek-ai-dsh-host-product-telemetry-otel/use-this-package:
en: a4793469bb1277c7
zh: c89d59ee4f87a256
en: 08e0bcbc259d131b
zh: 1d7cbde2688a903f
/deepseek-ai-dsh-host-product-telemetry-otel/understand-the-implementation:
en: db4d50ec33ae2ce5
zh: bb22ca3a232e1b76
en: 5d2c06a357a1c06a
zh: 9a1a17535889f5e3
/deepseek-ai-dsh-host-product-telemetry-otel/further-exploration:
en: 6bfb5e8a723ff0a9
zh: fffc6b02e2c1a3cb
@@ -30,8 +30,8 @@
en: b287f8e34f3634d1
zh: cd441f253ec0124b
/deepseek-ai-dsh-host-product-telemetry-otel/known-limitations-and-deferred-work:
en: 09804d496c1d03c4
zh: 80377ad0c371f0d6
en: 20b9ab75cb7e66a1
zh: a7dc4f4656eaf839
/deepseek-ai-dsh-host-product-telemetry-otel/known-limitations-and-deferred-work/dev-note:
en: 0a1253c6f7b5e8b9
zh: 9f56528562c7c914
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)
## Summary
Send selected product usage events and explicitly authorized Session logs to an OTLP/HTTP collector. Session logs use a separate queue and byte-bounded requests. Mounting the plugin collects nothing automatically. Delivery is best effort and does not confirm warehouse ingestion.
Send selected product usage events to an OTLP/HTTP collector. Events carry a name, string summary, occurrence time, and scalar or one-level object attributes. Mounting the plugin collects nothing automatically; applications explicitly submit each event. Delivery is best effort and does not confirm warehouse ingestion.
## Table of Contents
@@ -48,7 +48,6 @@ Mount the plugin in a Cordis composition with the application identity; override
| `timeoutMillis` | `15000` | Exporter HTTP and retry deadline |
| `exportTimeoutMillis` | `20000` | Processor batch export deadline |
| `shutdownTimeoutMillis` | `21000` | Outer wait for shutdown; expiry reports possible loss |
| `sessionLog.maxRequestBytes` / `sessionLog.processor` | `4,000,000` / SDK defaults | Independent Session-log request ceiling and batch settings; the byte ceiling cannot exceed 4,000,000 |
The default endpoint routes explicitly submitted events to the production product collector. Test and custom deployments must override it. Only `x-channel` and SDK protocol headers reach the collector; ambient OTel headers and client certificates are not inherited.
@@ -56,10 +55,6 @@ The 30-second interval batches product events; the exporter has a 15-second retr
Consumers inject `productTelemetry` and call `emit()` with explicitly selected analytics fields. Event names and field semantics belong to their product and analytics owners. The plugin reads no Session, account, credential, or device identifier. Callers must exclude prompts, responses, file contents, credentials, and other unapproved values.
`reportSessionLog(record)` accepts a Session id and one complete event. It writes `eventName: "session-log"`, a string body, and `attributes.sessionId` plus `attributes.content = JSON.stringify(event)`. Optional capture metadata remains on the record. Nested objects, arrays, booleans, nulls, and fractional numbers stay inside the JSON string. The exported `SessionLogReporter` provides the same pipeline to feedback-authorizing backends without mounting the product service. Callers must authorize Session sharing before submission.
Session records never share a request with product events. Each candidate batch is measured using the SDK OTLP JSON serializer before gzip, including resource, scope, attributes, UTF-8, and JSON escaping. Oversized batches split in event order; a single oversized record is rejected with a content-free diagnostic while other records continue. SDK attribute-length truncation is disabled for Session records. Shutdown awaits split requests as well as queued records, subject to the owner’s outer deadline.
The collector expects a string body and attributes containing strings, numbers, booleans, or objects of those scalars. Callers supply occurrence time in milliseconds; the plugin assigns observation time and defaults severity to INFO. Invalid transport configuration fails at activation. Export failures produce local warnings without making event submission wait for the network.
-----
@@ -70,7 +65,7 @@ The collector expects a string body and attributes containing strings, numbers,
<details>
<summary>Implementation internals — click to expand</summary>
Independent OTel providers keep product and Session-log queues separate. Both use the SDK JSON serializer and the same explicit HTTP transport configuration. The Session exporter splits oversized requests before transport; SDK batching cadence and retries remain SDK-owned. Plugin disposal drains both providers under one outer deadline. No global provider is installed.
A private OTel logger feeds `BatchLogRecordProcessor` and the SDK HTTP delegate with its JSON log serializer. The delegate receives explicit headers and an HTTP agent; only shared timeout and compression settings use SDK environment resolution. The direct `@opentelemetry/core` dependency matches `sdk-logs` at 2.9.0 so exporter result enums share one TypeScript identity. The SDK owns queueing, transient-error retries, and compression; plugin disposal drains pending records with a bounded wait. Export completion is observed separately because SDK shutdown can resolve after a rejected export. No global OTel provider is installed.
[`src/index.ts`](src/index.ts) owns configuration and submission. No runtime invariant companion is published: delivery has no independent local acknowledgement to compare with the SDK's queue.
@@ -104,7 +99,7 @@ Delivery and collection remain limited to the following capabilities.
- Callers own event selection, renderer-to-host transport, and any permitted identity attributes.
- The queue is memory-only; overflow, network failure, and process exit can lose events. There is no durable outbox or warehouse acknowledgement.
- Product events batch by record count. Session-log batches additionally obey the uncompressed byte ceiling; single-event rejection, queue overflow, and export failure have no durable retry guarantee.
- The SDK batches by record count, not encoded bytes. Callers must keep records within the collector's 4 MB limit and choose batch sizes appropriate to the receiver.
- Caller-selected strings are not redacted automatically. This package does not decide product disclosure or consent policy.
<a id="dev-note"></a>
@@ -9,7 +9,7 @@ kind: "package-reference"
## 概述
将选定的产品使用事件与明确授权的 Session 日志发送到 OTLP/HTTP 采集端。Session 日志使用独立队列和按字节限制的请求。挂载插件不会自动采集数据。投递为尽力而为,不确认数据仓库入库。
将选定的产品使用事件发送到 OTLP/HTTP 接收服务。事件包含名称、字符串摘要、发生时间,以及标量或单层对象属性。挂载插件不会自动采集信息;应用需要显式提交每个事件。发送采用尽力而为方式,不代表数据已入仓。
## 目录
@@ -48,7 +48,6 @@ kind: "package-reference"
| `timeoutMillis` | `15000` | Exporter HTTP 发送与重试的超时时间 |
| `exportTimeoutMillis` | `20000` | Processor 批次导出的超时时间 |
| `shutdownTimeoutMillis` | `21000` | 退出时的等待上限;超时会提示可能丢失数据 |
| `sessionLog.maxRequestBytes` / `sessionLog.processor` | `4,000,000` / SDK 默认值 | 独立的 Session 日志请求上限和批处理配置;字节上限不能超过 4,000,000 |
默认接收地址将显式提交的事件发送到生产产品 collector,测试和自定义部署必须覆盖该地址。只向 collector 发送 `x-channel` 和 SDK 协议请求头,不继承宿主 OTel 请求头或客户端证书。
@@ -56,10 +55,6 @@ kind: "package-reference"
消费方注入 `productTelemetry`,调用 `emit()` 提交明确选定的分析字段。事件名称与字段含义由产品和数据分析负责人定义。插件不读取 Session、账号、凭证或设备标识。调用方必须排除提示词、回答、文件内容、凭证及其他未经批准的数据。
`reportSessionLog(record)` 接收 Session id 和一条完整事件,写入 `eventName: "session-log"`、字符串 body,以及 `attributes.sessionId` 和 `attributes.content = JSON.stringify(event)`。可选的采集元数据保留在记录上。嵌套对象、数组、布尔值、null 与小数保留在 JSON 字符串内。导出的 `SessionLogReporter` 为负责反馈授权的后端提供同一管线,无需挂载产品服务。调用方必须先取得 Session 分享授权再提交。
Session 记录不会与产品事件共用一个请求。每个候选批次都使用 SDK 的 OTLP JSON 序列化器在 gzip 前计量,涵盖 resource、scope、attributes、UTF-8 和 JSON 转义。超限批次按事件顺序拆分;单条超限记录被拒绝并记录不含内容的诊断,其余记录继续发送。Session 记录禁用 SDK 属性长度截断。关闭时等待拆分请求和排队记录,受拥有方的外层截止时间约束。
接收服务要求 body 为字符串,attributes 的值为字符串、数字、布尔值或由这些标量组成的对象。调用方以毫秒提供发生时间;插件填写观测时间,默认严重程度为 INFO。无效传输配置在激活时报错。发送失败产生本地警告,事件提交不等待网络。
-----
@@ -70,7 +65,7 @@ Session 记录不会与产品事件共用一个请求。每个候选批次都使
<details>
<summary>实现细节——点击展开</summary>
独立的 OTel provider 将产品事件与 Session 日志队列分开。两者使用 SDK JSON 序列化器和相同的显式 HTTP 传输配置。Session 导出器在传输前拆分超限请求;批处理节奏和重试仍由 SDK 负责。插件卸载在同一个外层截止时间内排空两个 provider,不安装全局 provider。
私有 OTel logger 将记录交给 `BatchLogRecordProcessor`,再由 SDK HTTP delegate 和 JSON 日志序列化器发送。delegate 使用显式请求头和 HTTP agent,只有共享的超时与压缩配置使用 SDK 环境变量解析。直接依赖的 `@opentelemetry/core` 与 `sdk-logs` 对齐为 2.9.0,使导出结果枚举共享同一个 TypeScript 类型身份。SDK 负责队列、临时错误重试与压缩;插件卸载时在限定时间内发送剩余记录。发送完成结果单独观测,因为 SDK 在发送被拒绝后仍可能正常完成退出。不安装全局 OTel provider。
[`src/index.ts`](src/index.ts) 负责配置与提交。不发布运行时不变量伴随模块:本地没有独立的送达确认可与 SDK 队列比较。
@@ -104,7 +99,7 @@ Session 记录不会与产品事件共用一个请求。每个候选批次都使
- 调用方拥有事件选择、渲染进程到宿主的传输以及获准使用的标识属性。
- 队列仅存在于内存;队列溢出、网络故障和进程退出可能丢失事件。没有持久发件箱或入仓确认。
- 产品事件按记录数量分批。Session 日志批次还遵守未压缩字节上限;单条事件拒绝、队列溢出和导出失败均不提供持久化重试保证。
- SDK 按记录条数而非编码字节数分批。调用方必须确保记录不超过接收服务的 4 MB 限制,并选择接收服务适用的批次大小。
- 不自动脱敏调用方选择的字符串。本包不决定产品告知或同意策略。
<a id="dev-note"></a>
@@ -37,14 +37,12 @@
"@opentelemetry/sdk-logs": "^0.220.0"
},
"peerDependencies": {
"@deepseek-ai/cordis": "workspace:~",
"@deepseek-ai/dsh-session": "workspace:*"
"@deepseek-ai/cordis": "workspace:~"
},
"devDependencies": {
"@deepseek-ai/cordis": "workspace:~",
"@deepseek-ai/cordis-plugin-loader": "workspace:~",
"@deepseek-ai/dsh-app-boot": "workspace:*",
"@deepseek-ai/dsh-loader-smoke": "workspace:*",
"@deepseek-ai/dsh-session": "workspace:*"
"@deepseek-ai/dsh-loader-smoke": "workspace:*"
}
}
@@ -3,13 +3,12 @@ import { Context, Service } from '@deepseek-ai/cordis'
import z from '@deepseek-ai/schemastery'
import { SeverityNumber, type Logger } from '@opentelemetry/api-logs'
import { validateHeaderValue } from 'node:http'
import { CompressionAlgorithm } from '@opentelemetry/otlp-exporter-base'
import { JsonLogsSerializer } from '@opentelemetry/otlp-transformer'
import { createOtlpHttpExportDelegate, getSharedConfigurationFromEnvironment, httpAgentFactoryFromOptions } from '@opentelemetry/otlp-exporter-base/node-http'
import { CompressionAlgorithm, getSharedConfigurationDefaults, mergeOtlpSharedConfigurationWithDefaults, OTLPExporterBase } from '@opentelemetry/otlp-exporter-base'
import { ExportResultCode } from '@opentelemetry/core'
import { resourceFromAttributes } from '@opentelemetry/resources'
import { BatchLogRecordProcessor, LoggerProvider } from '@opentelemetry/sdk-logs'
import { createLogExporter } from './transport.ts'
import { resolveSessionLogLimits, SessionLogReporter, type SessionLogOptions, type SessionLogRecord } from './session-log.ts'
export { SessionLogReporter, SESSION_LOG_MAX_REQUEST_BYTES, type SessionLogOptions, type SessionLogRecord } from './session-log.ts'
declare module '@deepseek-ai/cordis' {
interface Context {
@@ -58,8 +57,6 @@ export interface Config {
exportTimeoutMillis: number
/** Outer shutdown wait; pending exports may be lost after this deadline. */
shutdownTimeoutMillis: number
/** Independent Session-log batching and byte limit; product-event settings do not apply. */
sessionLog?: Pick<SessionLogOptions, 'processor' | 'maxRequestBytes'>
}
const positiveInteger = () => z.number().step(1).min(1).max(2_147_483_647)
@@ -77,17 +74,12 @@ export const Config: z<Partial<Config>, Config> = z.object({
timeoutMillis: positiveInteger().default(15000),
exportTimeoutMillis: positiveInteger().default(20000),
shutdownTimeoutMillis: positiveInteger().default(21000),
sessionLog: z.object({
maxRequestBytes: positiveInteger().max(4_000_000),
processor: z.any(),
}),
})
/** Host analytics sender. Mounting alone sends nothing; the owning fiber drains it on unload. */
export default class ProductTelemetry extends Service {
static Config = Config
private readonly logger: Logger
private readonly sessionLogs: SessionLogReporter
constructor(ctx: Context, config: Config) {
let endpoint: URL
@@ -107,17 +99,21 @@ export default class ProductTelemetry extends Service {
if (config.maxExportBatchSize > config.maxQueueSize) {
throw new Error('product-telemetry-otel: maxExportBatchSize must not exceed maxQueueSize')
}
resolveSessionLogLimits(config.sessionLog ?? {})
super(ctx, 'productTelemetry')
const transport = {
url: config.endpoint,
headers: { 'x-channel': config.channel },
const shared = mergeOtlpSharedConfigurationWithDefaults({
timeoutMillis: config.timeoutMillis,
...(config.compression === undefined ? {} : {
compression: config.compression === 'gzip' ? CompressionAlgorithm.GZIP : CompressionAlgorithm.NONE,
}),
}, getSharedConfigurationFromEnvironment('LOGS'), getSharedConfigurationDefaults())
const transport = {
...shared,
url: config.endpoint,
// This collector must not inherit another endpoint's headers or TLS client identity.
headers: () => Promise.resolve({ 'Content-Type': 'application/json', 'x-channel': config.channel }),
agentFactory: httpAgentFactoryFromOptions({ keepAlive: true }),
}
const exporter = createLogExporter(transport)
const exporter = new OTLPExporterBase(createOtlpHttpExportDelegate(transport, JsonLogsSerializer))
const provider = new LoggerProvider({
resource: resourceFromAttributes({
'service.name': config.serviceName,
@@ -142,12 +138,6 @@ export default class ProductTelemetry extends Service {
})],
})
this.logger = provider.getLogger('@deepseek-ai/dsh-host-product-telemetry-otel')
this.sessionLogs = new SessionLogReporter({
...config.sessionLog,
exporter: transport,
resourceAttributes: { 'service.name': config.serviceName, 'service.version': config.serviceVersion },
onFailure: (message, error) => { ctx.logger.warn(message, error) },
})
ctx.effect(() => async () => {
let timer!: ReturnType<typeof setTimeout>
const deadline = new Promise<void>((resolve) => {
@@ -157,7 +147,7 @@ export default class ProductTelemetry extends Service {
}, config.shutdownTimeoutMillis)
})
try {
await Promise.race([Promise.all([provider.shutdown(), this.sessionLogs.shutdown()]), deadline])
await Promise.race([provider.shutdown(), deadline])
} finally {
clearTimeout(timer)
}
@@ -165,17 +155,9 @@ export default class ProductTelemetry extends Service {
}
/**
* Enqueue one explicitly authorized Session event in the separate byte-bounded queue.
* Enqueue one selected product event without waiting for network delivery.
* Queue admission and shutdown completion are not collector or warehouse acknowledgements.
* @param record - complete Session event and its owning Session id.
*/
reportSessionLog(record: SessionLogRecord): void {
this.sessionLogs.reportSessionLog(record)
}
/**
* Enqueue a selected product event independently of Session-log reporting.
* @param record - approved analytics fields; no automatic Session collection.
* @param record - caller-owned event containing only approved analytics fields.
*/
emit(record: ProductTelemetryRecord): void {
const severityNumber = record.severityNumber ?? SeverityNumber.INFO
@@ -1,164 +0,0 @@
/** Session-log records in an independent SDK queue with byte-bounded OTLP requests. */
import type { SessionEvent, SessionId } from '@deepseek-ai/dsh-session'
import type { Attributes } from '@opentelemetry/api'
import { SeverityNumber } from '@opentelemetry/api-logs'
import { ExportResultCode, type ExportResult } from '@opentelemetry/core'
import type { OTLPExporterNodeConfigBase } from '@opentelemetry/otlp-exporter-base'
import { JsonLogsSerializer } from '@opentelemetry/otlp-transformer'
import { resourceFromAttributes } from '@opentelemetry/resources'
import { BatchLogRecordProcessor, LoggerProvider, type BatchLogRecordProcessorOptions, type LogRecordExporter, type ReadableLogRecord } from '@opentelemetry/sdk-logs'
import { createLogExporter } from './transport.ts'
/** Collector request ceiling in uncompressed UTF-8 bytes, including the OTLP envelope. */
export const SESSION_LOG_MAX_REQUEST_BYTES = 4_000_000
/** One canonical event with its separately owned Session identity and redacted payload. */
export interface SessionLogRecord {
sessionId: SessionId
/** Complete event envelope; data is the capture policy's exported copy. */
event: Omit<SessionEvent, 'data'> & { data: unknown }
/** Additional capture metadata; sessionId and content are always assigned by the reporter. */
attributes?: Attributes
/** Omitted values use INFO. */
severityNumber?: SeverityNumber
}
/** Independent Session-log transport and queue settings. */
export interface SessionLogOptions {
/** Explicit destination and SDK transport options. */
exporter: OTLPExporterNodeConfigBase & {
/** Full HTTP(S) logs destination. */
url: string
}
/** Session-only queue settings, independent of product-event aggregation. */
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
/** May lower, but never exceed, the collector's 4,000,000-byte limit. */
maxRequestBytes?: number
/** Application and anonymous identity carried on the OTLP resource. */
resourceAttributes: Attributes
/** Report rejected single records and network failures without recording their content. */
onFailure: (message: string, error?: Error) => void
}
/**
* Validate byte and queue settings before constructing an SDK pipeline.
* @param options - Session-specific limits supplied by the owning composition.
* @returns the resolved collector request limit.
*/
export function resolveSessionLogLimits(options: Pick<SessionLogOptions, 'maxRequestBytes' | 'processor'>): number {
const limit = options.maxRequestBytes ?? SESSION_LOG_MAX_REQUEST_BYTES
if (!Number.isSafeInteger(limit) || limit < 1 || limit > SESSION_LOG_MAX_REQUEST_BYTES) {
throw new Error(`session log maxRequestBytes must be an integer between 1 and ${SESSION_LOG_MAX_REQUEST_BYTES}`)
}
for (const key of ['maxQueueSize', 'maxExportBatchSize', 'scheduledDelayMillis', 'exportTimeoutMillis'] as const) {
const value = options.processor?.[key]
if (value !== undefined && (!Number.isSafeInteger(value) || value < 1 || value > 2_147_483_647)) {
throw new Error(`session log processor.${key} must be a positive integer no greater than 2147483647`)
}
}
return limit
}
/** Split SDK batches using the same JSON serializer as the HTTP exporter. */
class ByteBoundedExporter implements LogRecordExporter {
private readonly pending = new Set<Promise<ExportResult>>()
constructor(private readonly exporter: LogRecordExporter, private readonly limit: number, private readonly warn: SessionLogOptions['onFailure']) {}
export(records: ReadableLogRecord[], callback: (result: ExportResult) => void): void {
const pending = this.send(records)
this.pending.add(pending)
void pending.then((result) => {
this.pending.delete(pending)
callback(result)
}, (error: unknown) => {
this.pending.delete(pending)
callback({ code: ExportResultCode.FAILED, error: error instanceof Error ? error : new Error(String(error)) })
})
}
private async send(records: ReadableLogRecord[]): Promise<ExportResult> {
const serialized = JsonLogsSerializer.serializeRequest(records)
if (serialized === undefined) throw new Error('Session log serialization produced no request')
const bytes = serialized.byteLength
if (bytes <= this.limit) {
return new Promise((resolve) => { this.exporter.export(records, resolve) })
}
if (records.length === 1) {
const error = new Error(`Session log record exceeds maxRequestBytes: ${bytes} > ${this.limit}`)
this.warn('Session log record rejected; content was not truncated', error)
return { code: ExportResultCode.FAILED, error }
}
const middle = Math.ceil(records.length / 2)
const first = await this.send(records.slice(0, middle))
const second = await this.send(records.slice(middle))
return first.code === ExportResultCode.FAILED ? first : second
}
async forceFlush(): Promise<void> {
await this.exporter.forceFlush()
await Promise.all(this.pending)
}
async shutdown(): Promise<void> {
await Promise.all(this.pending)
await this.exporter.shutdown()
}
}
/** Owns a Session-only queue; callers must drain it during their bounded shutdown. */
export class SessionLogReporter {
private readonly provider: LoggerProvider
private readonly logger: ReturnType<LoggerProvider['getLogger']>
/**
* @param options - transport, resource identity, queue limits, and failure observer.
*/
constructor(options: SessionLogOptions) {
const limit = resolveSessionLogLimits(options)
const exporter = new ByteBoundedExporter(createLogExporter(options.exporter), limit, options.onFailure)
this.provider = new LoggerProvider({
// Session content must not be truncated by ambient SDK attribute limits.
logRecordLimits: { attributeValueLengthLimit: Infinity, attributeCountLimit: Infinity },
resource: resourceFromAttributes(options.resourceAttributes),
processors: [new BatchLogRecordProcessor({
...options.processor,
exporter: {
export: (records, callback) => {
exporter.export(records, (result) => {
if (result.code !== ExportResultCode.SUCCESS) options.onFailure('Session log export failed', result.error)
callback(result)
})
},
forceFlush: () => exporter.forceFlush(),
shutdown: () => exporter.shutdown(),
},
})],
})
this.logger = this.provider.getLogger('@deepseek-ai/dsh-session-telemetry-otel')
}
/**
* Enqueue one event as a string-valued session-log record without acknowledging delivery.
* Oversized records are rejected during export; other records continue independently.
* @param record - canonical event and the Session it belongs to.
*/
reportSessionLog(record: SessionLogRecord): void {
const severityNumber = record.severityNumber ?? SeverityNumber.INFO
this.logger.emit({
eventName: 'session-log',
body: 'session-log',
timestamp: record.event.time,
observedTimestamp: record.event.time,
severityNumber,
severityText: SeverityNumber[severityNumber],
attributes: { ...record.attributes, sessionId: record.sessionId, content: JSON.stringify(record.event) },
})
}
/**
* Drain accepted records and stop the SDK pipeline; the owner supplies an outer deadline.
* @returns resolves when SDK shutdown completes, without guaranteeing collector acceptance.
*/
shutdown(): Promise<void> { return this.provider.shutdown() }
}
@@ -1,16 +1,13 @@
import { Agent, createServer, type IncomingHttpHeaders } from 'node:http'
import { createServer, type IncomingHttpHeaders } from 'node:http'
import { once } from 'node:events'
import { gunzipSync } from 'node:zlib'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { Context } from '@deepseek-ai/cordis'
import { LoggerProvider } from '@opentelemetry/sdk-logs'
import { SeverityNumber } from '@opentelemetry/api-logs'
import { JsonLogsSerializer } from '@opentelemetry/otlp-transformer'
import { SessionId, SessionSeq } from '@deepseek-ai/dsh-session'
import ProductTelemetry, { Config, SESSION_LOG_MAX_REQUEST_BYTES, SessionLogReporter, type SessionLogRecord } from '../src/index.ts'
import ProductTelemetry, { Config } from '../src/index.ts'
interface Capture {
bytes: number
headers: IncomingHttpHeaders
body: { resourceLogs: { resource: unknown; scopeLogs: { logRecords: Record<string, unknown>[] }[] }[] }
}
@@ -30,7 +27,7 @@ beforeEach(() => {
vi.stubEnv('OTEL_EXPORTER_OTLP_LOGS_COMPRESSION', undefined)
})
async function collector(statuses = [200], beforeRespond?: () => Promise<void>) {
async function collector(statuses = [200]) {
const captures: Capture[] = []
const server = createServer((req, res) => {
const chunks: Buffer[] = []
@@ -38,11 +35,8 @@ async function collector(statuses = [200], beforeRespond?: () => Promise<void>)
req.on('end', () => {
const raw = Buffer.concat(chunks)
const bytes = req.headers['content-encoding'] === 'gzip' ? gunzipSync(raw) : raw
captures.push({ bytes: bytes.length, headers: req.headers, body: JSON.parse(bytes.toString()) as Capture['body'] })
void (async () => {
await beforeRespond?.()
res.writeHead(statuses.shift() ?? 200, { 'content-type': 'application/json' }).end('{}')
})()
captures.push({ headers: req.headers, body: JSON.parse(bytes.toString()) as Capture['body'] })
res.writeHead(statuses.shift() ?? 200, { 'content-type': 'application/json' }).end('{}')
})
})
cleanup.push(async () => {
@@ -207,206 +201,3 @@ describe('explicit product telemetry', () => {
await vi.advanceTimersByTimeAsync(0)
})
})
function sessionRecord(text: string, seq = 0): SessionLogRecord {
return {
sessionId: SessionId('synthetic-session'),
event: { type: 'user/message', seq: SessionSeq(seq), time: 1_800_000_000_000, surfaceOp: 'append',
data: { content: [{ type: 'text', text }], nested: { items: [null, true, false, 0, 1.5, { text }] } } },
}
}
function logs(capture: Capture) {
return capture.body.resourceLogs.flatMap(resource => resource.scopeLogs.flatMap(scope => scope.logRecords))
}
function contents(captures: Capture[]): string[] {
return captures.flatMap(capture => logs(capture).map((record) => {
const attributes = record['attributes'] as { key: string; value: { stringValue: string } }[]
return attributes.find(attribute => attribute.key === 'content')!.value.stringValue
}))
}
function parseContent(content: string): unknown {
return JSON.parse(content)
}
describe('Session-log reporting', () => {
it('keeps product events in separate requests and preserves the complete nested event', async () => {
vi.stubEnv('OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT', '8')
vi.stubEnv('OTEL_EXPORTER_OTLP_HEADERS', 'Authorization=Bearer%20unrelated')
const target = await collector()
const ctx = context()
const fiber = await ctx.plugin(ProductTelemetry, config(target.endpoint))
const record = sessionRecord('中文\n"quoted"\\path 😀')
ctx.productTelemetry.emit(event)
ctx.productTelemetry.reportSessionLog(record)
await fiber.dispose()
expect(target.captures).toHaveLength(2)
const sessionCapture = target.captures.find(capture => logs(capture)[0]?.['eventName'] === 'session-log')!
expect(logs(sessionCapture)).toHaveLength(1)
expect(logs(sessionCapture)[0]).toMatchObject({
eventName: 'session-log', body: { stringValue: 'session-log' },
attributes: [
{ key: 'sessionId', value: { stringValue: record.sessionId } },
{ key: 'content', value: { stringValue: JSON.stringify(record.event) } },
],
})
expect(JSON.parse(contents([sessionCapture])[0]!)).toEqual(record.event)
expect(sessionCapture.headers).not.toHaveProperty('authorization')
})
it.each(['gzip', 'none'] as const)('splits real 4 MB batches before %s compression without losing event order', async (compression) => {
const target = await collector()
const ctx = context()
const fiber = await ctx.plugin(ProductTelemetry, config(target.endpoint, { compression }))
const records = [0, 1, 2].map(seq => sessionRecord('中"\\'.repeat(100_000), seq))
for (const record of records) ctx.productTelemetry.reportSessionLog(record)
await fiber.dispose()
expect(target.captures.length).toBeGreaterThan(1)
expect(target.captures.every(capture => capture.bytes <= SESSION_LOG_MAX_REQUEST_BYTES)).toBe(true)
expect(contents(target.captures).map(parseContent)).toEqual(records.map(record => record.event))
})
it('admits an exact byte-limit request and rejects a single event one byte above it', async () => {
const target = await collector()
const ctx = context()
const record = sessionRecord('边界"\\')
const baseline = await ctx.plugin(ProductTelemetry, config(target.endpoint))
ctx.productTelemetry.reportSessionLog(record)
await baseline.dispose()
const limit = target.captures[0]!.bytes
const exact = await ctx.plugin(ProductTelemetry, config(target.endpoint, { sessionLog: { maxRequestBytes: limit } }))
ctx.productTelemetry.reportSessionLog(record)
await exact.dispose()
expect(target.captures).toHaveLength(2)
expect(target.captures[1]!.bytes).toBe(limit)
const warn = vi.spyOn(ctx.logger, 'warn')
const tooSmall = await ctx.plugin(ProductTelemetry, config(target.endpoint, { sessionLog: { maxRequestBytes: limit - 1 } }))
ctx.productTelemetry.reportSessionLog(record)
await tooSmall.dispose()
expect(target.captures).toHaveLength(2)
expect(warn).toHaveBeenCalledWith('Session log record rejected; content was not truncated', expect.any(Error))
})
it('rejects an oversized event while still sending the events before and after it', async () => {
const target = await collector()
const ctx = context()
const warn = vi.spyOn(ctx.logger, 'warn')
const fiber = await ctx.plugin(ProductTelemetry, config(target.endpoint))
const records = [sessionRecord('before'), sessionRecord('x'.repeat(SESSION_LOG_MAX_REQUEST_BYTES), 1), sessionRecord('after', 2)]
for (const record of records) ctx.productTelemetry.reportSessionLog(record)
await fiber.dispose()
expect(contents(target.captures).map(parseContent)).toEqual([records[0]!.event, records[2]!.event])
expect(warn).toHaveBeenCalledWith('Session log record rejected; content was not truncated', expect.any(Error))
expect(target.captures.every(capture => capture.bytes <= SESSION_LOG_MAX_REQUEST_BYTES)).toBe(true)
})
it('continues later byte batches after a permanent rejection', async () => {
const target = await collector([400, 200])
const ctx = context()
const warn = vi.spyOn(ctx.logger, 'warn')
const fiber = await ctx.plugin(ProductTelemetry, config(target.endpoint, { sessionLog: { maxRequestBytes: 1300 } }))
ctx.productTelemetry.reportSessionLog(sessionRecord('first'.repeat(25)))
ctx.productTelemetry.reportSessionLog(sessionRecord('second'.repeat(25), 1))
await fiber.dispose()
expect(target.captures).toHaveLength(2)
expect(warn).toHaveBeenCalledWith('Session log export failed', expect.any(Error))
})
it.each([0, -1, 1.5, 4_000_001, Infinity])('refuses an invalid request limit %s', (maxRequestBytes) => {
expect(() => new SessionLogReporter({
exporter: { url: 'http://collector.test/v1/logs' }, resourceAttributes: {}, maxRequestBytes, onFailure: vi.fn(),
})).toThrow('maxRequestBytes')
})
})
it('honors explicit asynchronous headers and agent factories for Session logs', async () => {
const target = await collector()
const agent = new Agent({ keepAlive: false })
cleanup.push(async () => { agent.destroy() })
const failures = vi.fn()
const reporter = new SessionLogReporter({
exporter: { url: target.endpoint, userAgent: 'session-test', headers: async () => ({ 'x-channel': 'test-channel' }), httpAgentOptions: async () => agent },
resourceAttributes: {}, onFailure: failures,
})
cleanup.push(() => reporter.shutdown())
reporter.reportSessionLog(sessionRecord('explicit headers'))
await reporter.shutdown()
expect(failures).not.toHaveBeenCalled()
expect(target.captures[0]!.headers['x-channel']).toBe('test-channel')
expect(target.captures[0]!.headers['user-agent']).toContain('session-test')
})
it.each([new Error('serializer failed'), 'serializer failed', undefined])('reports serialization failure %s without sending a partial request', async (failure) => {
const target = await collector()
const failures = vi.fn()
const reporter = new SessionLogReporter({
exporter: { url: target.endpoint, keepAlive: false }, resourceAttributes: {}, onFailure: failures,
})
cleanup.push(() => reporter.shutdown())
reporter.reportSessionLog(sessionRecord('not exported'))
vi.spyOn(JsonLogsSerializer, 'serializeRequest').mockImplementationOnce(() => {
if (failure === undefined) return undefined
throw failure
})
await reporter.shutdown()
expect(target.captures).toEqual([])
expect(failures).toHaveBeenCalledWith('Session log export failed', expect.any(Error))
})
it.each([
['maxQueueSize', 0], ['maxExportBatchSize', 0], ['scheduledDelayMillis', 0], ['exportTimeoutMillis', 0],
['maxQueueSize', 1.5], ['exportTimeoutMillis', 2_147_483_648],
] as const)(
'rejects invalid Session queue %s=%s before registering a service', (key, value) => {
const ctx = context()
expect(() => new ProductTelemetry(ctx, config('http://collector.test/v1/logs', {
sessionLog: { processor: { [key]: value } },
}))).toThrow(`processor.${key}`)
expect(ctx.get('productTelemetry')).toBeUndefined()
},
)
it('drains every split request when disposal starts during the first request', async () => {
const received = Promise.withResolvers<undefined>()
const release = Promise.withResolvers<undefined>()
const target = await collector([200], async () => {
received.resolve(undefined)
await release.promise
})
const ctx = context()
cleanup.push(async () => { release.resolve(undefined) })
const fiber = await ctx.plugin(ProductTelemetry, config(target.endpoint, {
sessionLog: { maxRequestBytes: 1300, processor: { maxExportBatchSize: 3, scheduledDelayMillis: 60_000 } },
}))
const service = ctx.productTelemetry
const records = [0, 1, 2].map(seq => sessionRecord('content'.repeat(20), seq))
for (const record of records) service.reportSessionLog(record)
await received.promise
let disposed = false
const disposal = fiber.dispose().then(() => { disposed = true })
await Promise.resolve()
expect(disposed).toBe(false)
release.resolve(undefined)
await disposal
expect(target.captures).toHaveLength(3)
expect(contents(target.captures).map(parseContent)).toEqual(records.map(record => record.event))
service.reportSessionLog(sessionRecord('after shutdown'))
expect(target.captures).toHaveLength(3)
})
it('constructs and disposes the product service without Session-specific overrides', async () => {
const target = await collector()
const ctx = context()
const { sessionLog: _sessionLog, ...options } = config(target.endpoint)
const service = new ProductTelemetry(ctx, options)
service.reportSessionLog(sessionRecord('default Session queue'))
await ctx.fiber.dispose()
expect(target.captures).toHaveLength(1)
})
@@ -16,9 +16,6 @@
},
{
"path": "../../../vendor/schemastery"
},
{
"path": "../../core/session"
}
]
}
@@ -355,7 +355,7 @@ Every package below exports a Cordis plugin that a bundle patch can name in a Lo
| `@deepseek-ai/dsh-session-projection` | no | Session-projection seam: the merge-extensible projection type table, the provider contract, and the ctx.sessionProjections registry serving whole current values of log-derived per-session state |
| `@deepseek-ai/dsh-session-projection-cache` | yes | Persisted projection cache (ctx.sessionProjectionCache): durable per-session checkpoint records on the session_projcache storage domain (per-record layout), throttled write-behind, and the cached listing read |
| `@deepseek-ai/dsh-session-stats` | no | Whole-log conversation counts and wall times projection (sessionStats) for the DeepSeek Harness |
| `@deepseek-ai/dsh-session-telemetry-otel` | yes | OpenTelemetry backend for the DeepSeek Harness telemetry seam: hands captured session records to the OTel JS SDK's log pipeline |
| `@deepseek-ai/dsh-session-telemetry-otel` | yes | Feedback-authorized Session logs over byte-bounded OpenTelemetry HTTP requests |
| `@deepseek-ai/dsh-session-title` | yes | Log-backed session title service and provider registry for the DeepSeek Harness |
| `@deepseek-ai/dsh-session-title-all-prompts-llm` | yes | All-user-messages LLM provider plugin for DeepSeek Harness session titles |
| `@deepseek-ai/dsh-session-title-first-prompt-llm` | yes | First-message LLM provider plugin for DeepSeek Harness session titles |
@@ -9,8 +9,8 @@
en: f8ea869792c8b207
zh: bc076172ce22ba9b
/deepseek-ai-dsh-session-telemetry-otel/summary:
en: 1c5d2f15cb13e318
zh: 5e061a16cd38c4c5
en: 8f48640985c1f920
zh: 88446c8d7f95142c
/deepseek-ai-dsh-session-telemetry-otel/table-of-contents:
en: d152484eb41ac6b4
zh: 09388d293f9be9cb
@@ -21,26 +21,26 @@
en: bc7e598148a7f8c3
zh: 0209c6aa258ec134
/deepseek-ai-dsh-session-telemetry-otel/use-this-package/minimal-configuration:
en: afe9c8926a51303d
zh: a6cbf3a262484edf
en: 11845fd50dc0a494
zh: ee990bfca3621bd3
/deepseek-ai-dsh-session-telemetry-otel/use-this-package/what-leaves-the-machine:
en: d3008613a4c67cdb
zh: 4ff966d8ef8ea8d6
en: 78d933a0f391cc58
zh: 0552999de92573b1
/deepseek-ai-dsh-session-telemetry-otel/use-this-package/failures-and-shutdown:
en: 20c6e78df8ad2c2a
zh: fd6a00f98c4cc05c
en: 1e5379dd884037db
zh: f9f53bcd044ecb17
/deepseek-ai-dsh-session-telemetry-otel/understand-the-implementation:
en: ee672a6a6f0fa02d
zh: c3bab8e6b60e768c
/deepseek-ai-dsh-session-telemetry-otel/understand-the-implementation/design-concept:
en: d5e1f8249d3eea34
zh: 7a6915e33390f4ae
en: 22eccb2e5ab7c7d4
zh: 0a6eb06e46857745
/deepseek-ai-dsh-session-telemetry-otel/understand-the-implementation/source-map:
en: 0b99d2e5c27213cd
zh: 3cbdaa7f6a45d81a
/deepseek-ai-dsh-session-telemetry-otel/understand-the-implementation/capture-wiring:
en: 9bdc01d0fa130b3a
zh: 63bf30c4cc983ca0
en: 5c85eb0992eb5b2b
zh: a19cdd5814c0e2b9
/deepseek-ai-dsh-session-telemetry-otel/understand-the-implementation/field-mapping:
en: a42e9e081fb93bee
zh: b8e817c2455278b9
@@ -57,5 +57,5 @@
en: e43087aea7af4c9e
zh: f8b529e2012b4c63
/deepseek-ai-dsh-session-telemetry-otel/known-limitations-and-deferred-work/dev-note:
en: a770daec456212ab
zh: 1b861566d591837d
en: 0065df2648b09ac9
zh: b5bac41fc3aec70e
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)
## Summary
`dsh-session-telemetry-otel` exports session records through the OTel JS SDK only after new explicit feedback, for all users and providers, including `deepseek-official`. `FEEDBACK_ONLY` releases the canonical prefix through that feedback, including context; later records wait for the next explicit feedback. `DISABLED` constructs no transport. SDK batching can finish an authorized upload without another user interaction or model call. Deployments own their redaction rules.
`dsh-session-telemetry-otel` exports session records through the OTel JS SDK only after new explicit feedback, for all users and providers, including `deepseek-official`. `FEEDBACK_ONLY` releases the canonical prefix through that feedback, including context; later records wait for the next explicit feedback. `DISABLED` constructs no transport. Scheduled batching can finish an authorized upload without another user interaction or model call. Deployments own their redaction rules.
## Table of Contents
@@ -38,7 +38,7 @@ Programmatic TypeScript configuration uses the exported `SessionTelemetryMode` e
### Minimal configuration
Uploading modes require an exporter URL. Processor options configure the Session-only SDK queue; the shared reporter adds collector routing and byte-bounded export.
Uploading modes require an exporter URL. Processor settings control the independent Session-log queue; routing headers are explicit `exporter.headers` values.
```yaml
- id: sessionTelemetry-otel
@@ -50,30 +50,30 @@ Uploading modes require an exporter URL. Processor options configure the Session
url: https://collector.example.com/v1/logs
headers:
authorization: !!js `Bearer ${process.env.OTLP_TOKEN}`
processor: {} # optional; passed verbatim to BatchLogRecordProcessor
processor: {} # optional; byte/count batching and per-request watchdog
```
| Field | Default | Meaning |
|---|---|---|
| `mode` | `FEEDBACK_ONLY` | Sharing policy: `FEEDBACK_ONLY` or `DISABLED` |
| `exporter.url` | required in uploading modes | Full OTLP logs endpoint; must parse as `http(s)` |
| `exporter`, `processor` | — | Explicit SDK transport and Session-only batch settings; headers/TLS identity are not inherited from the environment |
| `shutdownTimeoutMillis` | `3,000` | Outer deadline for the SDK's complete shutdown sequence |
| `exporter`, `processor` | — | SDK transport plus byte/count batching; headers/TLS identity are not inherited from the environment. Agent factories own their returned agent settings, including keepAlive |
| `shutdownTimeoutMillis` | `3,000` | Outer deadline for all queued HTTP requests; remaining queued sends stop at expiry |
| `maxRequestBytes` | `4,000,000` | Maximum complete OTLP JSON request bytes before gzip; may only be lowered |
Direct `ctx.sessionTelemetry.emit()` calls are no-ops in every mode and cannot bypass feedback authorization. Inherited parent feedback does not authorize a child export: the child needs new feedback of its own. Its authorized prefix then includes inherited context.
Model requests, request headers, Session creation or adoption, restoration, and plugin mount or HMR do not authorize capture. Stored feedback alone triggers nothing. SDK scheduled flush and shutdown may finish batches authorized earlier, but never capture new records.
Model requests, request headers, Session creation or adoption, restoration, and plugin mount or HMR do not authorize capture. Stored feedback alone triggers nothing. Scheduled flush and shutdown may finish batches authorized earlier, but never capture new records.
### What leaves the machine
Each Session event becomes one `eventName: "session-log"` record. `attributes.sessionId` identifies its Session; `attributes.content` is the JSON string of the complete event envelope and the redacted `event.data`. The resource carries application and anonymous-user identity. Contents can include messages, tool inputs/results, prompts, schemas, feedback, and paths. `DISABLED` constructs no transport. The base profile sends to `https://dsh-otel-collector.deepseeksvc.com/v1/logs` with `x-channel: dsh_otel_report`; `DSH_TELEMETRY_OTLP_URL` can override the URL.
Each Session event becomes one `eventName: "session-log"` record. `attributes.sessionId` is the collector Session identity; `attributes.content` encodes the complete event envelope with redacted `event.data`. JSON values are preserved, not the original JSONL bytes or key ordering. Legacy `session.id`, `event.seq`, and `event.type` metadata remain for existing consumers. Resources carry application and anonymous-user identity; scope carries the backend package name and version. The base profile uses `https://dsh-otel-collector.deepseeksvc.com/v1/logs`; `DSH_TELEMETRY_OTLP_URL` overrides it. No channel header is added implicitly.
The [shared Session-log reporter](../../host/product-telemetry-otel/README.md) measures complete uncompressed requests, splits oversized batches, and rejects single events that exceed the limit without truncating them. Session logs never mix with product analytics in a request. Rejection and network failures produce local diagnostics; neither handoff nor shutdown confirms collector acceptance.
The backend measures each record once with the SDK OTLP JSON serializer, including its resource/scope envelope, then greedily packs requests using those conservative sizes. A single oversized event produces one rejection diagnostic without truncation. Session logs never mix with product analytics in a request. Capture handoff and shutdown are not collector acknowledgements.
### Failures and shutdown
Misconfiguration fails at plugin load: a missing or non-`http(s)` `exporter.url`, a non-positive-integer `processor.maxExportBatchSize` (which the SDK accepts but then hangs on at shutdown), and an invalid `shutdownTimeoutMillis` all reject before any record is exported. During shutdown, OTel awaits `exporter.forceFlush()` before the processor's bounded completion promise; if that transport promise never settles, this package abandons the wait at `shutdownTimeoutMillis`, logs the contained failure, and lets application teardown continue — records still pending then may be lost at process exit.
Invalid byte limits, non-positive queue/timer values, `maxExportBatchSize > maxQueueSize`, invalid endpoints, and invalid shutdown deadlines fail at load. Processor defaults are 2,048 queued records, 512 records per request, a 1,000 ms scheduling delay, and a 30,000 ms request watchdog. Byte limits may split a count batch into multiple serial HTTP requests. `exportTimeoutMillis` warns per request but never frees an unsettled transport slot; the SDK transport owns network timeout/retry. Shutdown drains these requests until `shutdownTimeoutMillis`; expiry discards the remaining queue and prevents further sends, while an active request may still settle. Large prefixes can therefore remain partially unsent at CLI exit.
-----
@@ -87,7 +87,7 @@ This section explains the backend's composition; the observable behavior is full
### Design concept
The backend is a thin adapter over the OTel JS SDK: it owns feedback authorization, resource identity, and an outer shutdown deadline. Canonical ledger records use the `@deepseek-ai/dsh-session-telemetry-otel` instrumentation scope; this backend captures no operational records. Resource identity carries `service.name`/`service.version` from `dsh-llm`'s `APP_IDENTITY` plus the anonymous `user.id` (from `$DSH_HOME/.anonymous-user-id`), once per export batch rather than per record.
The backend owns feedback authorization, identity, byte/count scheduling, and shutdown. OTel owns record construction, JSON transport, compression, and retries. Resource identity carries `service.name`/`service.version` from `APP_IDENTITY` and anonymous `user.id`; the scope retains this package’s name and version.
### Source map
@@ -97,7 +97,7 @@ The backend is a thin adapter over the OTel JS SDK: it owns feedback authorizati
### Capture wiring
The backend uses on-demand capture with stored history included. Only new own `feedback/record`, `feedback/message-put`, or `feedback/message-delete` events trigger live capture, bounded by that event. A cold `feedback/committed` notification supplies its committed canonical snapshot without publishing a live Session or Agent. Same-object handoff cursors suppress repeated capture. The backend implements no `flush()`; the SDK owns batching and shutdown drain.
The backend captures history on demand through new own feedback events or committed cold snapshots. Its private reporter serializes queued HTTP requests even after a watchdog fires; an outer shutdown deadline stops the remaining queue. It exposes no extra flush entry point.
### Field mapping
@@ -151,4 +151,4 @@ None.
</details>
**Runtime invariant:** No companion is published. Mode selection changes capture handoff, SDK setup, and local diagnostics without mutating session or service state an independent companion can compare. Export remains inside the SDK past the backend boundary.
**Runtime invariant:** No companion is published. Mode selection changes capture handoff, SDK setup, and local diagnostics without mutating session or service state an independent companion can compare. Collector delivery cannot be inferred from local queue state.
@@ -9,7 +9,7 @@ kind: "package-reference"
## 概述
`dsh-session-telemetry-otel` 仅在新的显式反馈后通过 OTel JS SDK 导出会话记录,适用于所有用户和提供方,包括 `deepseek-official`。`FEEDBACK_ONLY` 释放截至该反馈的权威日志前缀,包含上下文;后续记录等待下一次显式反馈。`DISABLED` 不构造传输。SDK 批处理可完成已授权的上传,无需另一次用户交互或模型调用。部署方负责脱敏规则。
`dsh-session-telemetry-otel` 仅在新的显式反馈后通过 OTel JS SDK 导出会话记录,适用于所有用户和提供方,包括 `deepseek-official`。`FEEDBACK_ONLY` 释放截至该反馈的权威日志前缀,包含上下文;后续记录等待下一次显式反馈。`DISABLED` 不构造传输。定时批处理可完成已授权的上传,无需另一次用户交互或模型调用。部署方负责脱敏规则。
## 目录
@@ -38,7 +38,7 @@ kind: "package-reference"
### 最小配置
上传模式要求提供导出器 URL。processor 选项配置 Session 专用 SDK 队列;共享上报器负责采集端路由和按字节限制的导出。
启用上传的模式必须提供 exporter URL。processor 设置控制独立的 Session 日志队列;路由头通过 `exporter.headers` 显式配置。
```yaml
- id: sessionTelemetry-otel
@@ -50,30 +50,30 @@ kind: "package-reference"
url: https://collector.example.com/v1/logs
headers:
authorization: !!js `Bearer ${process.env.OTLP_TOKEN}`
processor: {} # optional; passed verbatim to BatchLogRecordProcessor
processor: {} # optional; byte/count batching and per-request watchdog
```
| 字段 | 默认值 | 含义 |
|---|---|---|
| `mode` | `FEEDBACK_ONLY` | 共享策略:`FEEDBACK_ONLY` 或 `DISABLED` |
| `exporter.url` | 上传模式必填 | 完整 OTLP 日志端点;必须能解析为 `http(s)` |
| `exporter`, `processor` | — | 显式 SDK 传输和 Session 专用批处理配置;不从环境继承请求头或 TLS 身份 |
| `shutdownTimeoutMillis` | `3,000` | SDK 完整关闭序列的外层截止时间 |
| `exporter`, `processor` | — | SDK 传输及字节/条数聚合;不继承环境中的头部和 TLS 身份。agent 工厂负责返回实例的设置,包括 keepAlive |
| `shutdownTimeoutMillis` | `3,000` | 所有排队 HTTP 请求的外层期限;到期后停止剩余排队发送 |
| `maxRequestBytes` | `4,000,000` | gzip 前完整 OTLP JSON 请求的最大字节数;只能调低 |
直接调用 `ctx.sessionTelemetry.emit()` 在任何模式下都是空操作,不能绕过反馈授权。继承的父会话反馈不授权子会话导出:子会话需要新的自身反馈。授权后的前缀包含继承的上下文。
模型请求、请求头、Session 创建或接纳、恢复,以及插件挂载或 HMR(热模块替换)均不授权捕获。仅凭已存储的反馈不会触发任何操作。SDK 定时刷新和关闭可以完成先前已授权的批次,但绝不捕获新记录。
模型请求、请求头、Session 创建或接纳、恢复,以及插件挂载或 HMR(热模块替换)均不授权捕获。仅凭已存储的反馈不会触发任何操作。定时刷新和关闭可以完成先前已授权的批次,但绝不捕获新记录。
### 哪些数据会离开本机
每条 Session 事件对应一条 `eventName: "session-log"` 记录。`attributes.sessionId` 标识所属 Session;`attributes.content` 是完整事件信封与脱敏后的 `event.data` 的 JSON 字符串。resource 携带应用与匿名用户身份。内容可包含消息、工具输入和结果、提示词、schema、反馈及路径。`DISABLED` 不构造传输。基础 profile 使用 `x-channel: dsh_otel_report` 向 `https://dsh-otel-collector.deepseeksvc.com/v1/logs` 发送;`DSH_TELEMETRY_OTLP_URL` 可覆盖 URL。
每条 Session 事件对应一个 `eventName: "session-log"` 记录。`attributes.sessionId` 是 collector 使用的 Session 身份;`attributes.content` 编码完整事件 envelope 和脱敏后的 `event.data`。保留的是 JSON 值,不保证原 JSONL 字节或键顺序相同。为现有消费者保留 `session.id`、`event.seq` 和 `event.type` 元数据。Resource 携带应用和匿名用户身份;scope 携带后端包名和版本。基础配置使用 `https://dsh-otel-collector.deepseeksvc.com/v1/logs`,可用 `DSH_TELEMETRY_OTLP_URL` 覆盖。不隐式添加 channel 头。
[共享 Session 日志上报器](../../host/product-telemetry-otel/README.zh.md) 计量完整未压缩请求,拆分超限批次,并拒绝超限单条事件而不截断内容。Session 日志不会与产品埋点混入同一个请求。拒绝和网络失败会产生本地诊断;交接和关闭都不确认采集端接受。
后端使用 SDK 的 OTLP JSON 序列化器对每条记录计量一次,包含其 resource/scope envelope,再按保守大小顺序组包。单条超限事件产生一次拒绝诊断且不截断。Session 日志不会与产品埋点混在一个请求中。捕获交接和关闭完成不代表 collector 确认。
### 失败与关闭
配置错误会在插件加载时失败:缺少或非 `http(s)` 的 `exporter.url`、非正整数的 `processor.maxExportBatchSize`(SDK 会接受该值,随后却在关闭时挂起)以及无效的 `shutdownTimeoutMillis` 都会在任何记录导出前被拒绝。关闭期间,OTel 会先等待 `exporter.forceFlush()`,再等待处理器有界完成 promise;如果该传输 promise 始终不结算,本包会在 `shutdownTimeoutMillis` 到期时放弃等待、记录已隔离的失败,并让应用继续拆卸——届时仍待处理的记录可能在进程退出时丢失。
无效字节上限、非正队列/计时值、`maxExportBatchSize > maxQueueSize`、无效 endpoint 或关闭期限在加载时失败。processor 默认队列为 2,048 条、单请求最多 512 条、调度延迟 1,000 ms、请求监测期限 30,000 ms。字节上限可能将按条数划分的批次拆成多个串行 HTTP 请求。`exportTimeoutMillis` 针对每个请求告警,但不会释放未结束的传输槽位;SDK transport 负责网络超时和重试。关闭时发送这些请求直到 `shutdownTimeoutMillis`,到期后丢弃剩余队列并禁止后续发送,在途请求仍可能结束。因此较大的授权前缀在 CLI 退出时可能只发送了一部分。
-----
@@ -87,7 +87,7 @@ kind: "package-reference"
### 设计理念
后端是对 OTel JS SDK 的薄适配层:它拥有反馈授权、资源身份与外层关闭截止时间。权威 ledger 记录使用 `@deepseek-ai/dsh-session-telemetry-otel` 插桩作用域;此后端不捕获运维记录。资源身份携带 `service.name`/`service.version`(来自 `dsh-llm` 的 `APP_IDENTITY`)以及匿名 `user.id`(来自 `$DSH_HOME/.anonymous-user-id`),按导出批次携带一次,而非逐条记录。
后端负责反馈授权、身份、字节/条数调度和关闭。OTel 负责记录构造、JSON 传输、压缩和重试。Resource 身份包含 `APP_IDENTITY` 的 `service.name`/`service.version` 和匿名 `user.id`;scope 保留本包名称和版本。
### 源码地图
@@ -97,7 +97,7 @@ kind: "package-reference"
### 捕获接线
后端使用包含存储历史的按需捕获。只有新的自身 `feedback/record`、`feedback/message-put` 或 `feedback/message-delete` 事件触发活跃会话捕获,并以该事件为上限。冷会话 `feedback/committed` 通知提供已提交的权威快照,不发布存活 Session 或 Agent。同对象交接游标抑制重复捕获。后端不实现 `flush()`;SDK 负责批处理和关闭排空。
后端按需捕获历史,由新的自身反馈事件或已提交的冷会话快照触发。私有 reporter 串行发送排队 HTTP 请求,即使监测期限已触发也不会重叠;外层关闭期限停止剩余队列。不额外暴露 flush 入口。
### 字段映射
@@ -151,4 +151,4 @@ kind: "package-reference"
</details>
**运行时不变式:** 不发布伴生入口。模式选择只改变 capture handoff、SDK setup 与本地 diagnostics,不改变可由独立 companion 对照的会话或服务状态。导出在越过后端边界后仍由 SDK 内部处理。
**运行时不变式:** 不发布伴生入口。模式选择只改变 capture handoff、SDK setup 与本地 diagnostics,不改变可由独立 companion 对照的会话或服务状态。无法根据本地队列状态推断 collector 是否收到记录。
@@ -1,6 +1,6 @@
{
"name": "@deepseek-ai/dsh-session-telemetry-otel",
"description": "OpenTelemetry backend for the DeepSeek Harness telemetry seam: hands captured session records to the OTel JS SDK's log pipeline",
"description": "Feedback-authorized Session logs over byte-bounded OpenTelemetry HTTP requests",
"version": "0.1.7-rc.2",
"publishConfig": {
"access": "public"
@@ -32,7 +32,9 @@
"@opentelemetry/otlp-exporter-base": "^0.220.0",
"@opentelemetry/resources": "^2.9.0",
"@opentelemetry/sdk-logs": "^0.220.0",
"@deepseek-ai/schemastery": "workspace:~"
"@deepseek-ai/schemastery": "workspace:~",
"@opentelemetry/core": "2.9.0",
"@opentelemetry/otlp-transformer": "^0.220.0"
},
"peerDependencies": {
"@deepseek-ai/dsh-command-feedback": "workspace:*",
@@ -41,8 +43,7 @@
"@deepseek-ai/dsh-session": "workspace:*",
"@deepseek-ai/dsh-session-telemetry": "workspace:*",
"@deepseek-ai/dsh-anonymous-user-id": "workspace:*",
"@deepseek-ai/cordis": "workspace:~",
"@deepseek-ai/dsh-host-product-telemetry-otel": "workspace:*"
"@deepseek-ai/cordis": "workspace:~"
},
"devDependencies": {
"@deepseek-ai/cordis-plugin-logger-console": "workspace:~",
@@ -60,7 +61,6 @@
"@deepseek-ai/dsh-subprocess-local": "workspace:*",
"@deepseek-ai/dsh-anonymous-user-id": "workspace:*",
"@deepseek-ai/cordis": "workspace:~",
"@deepseek-ai/dsh-http-proxy": "workspace:*",
"@deepseek-ai/dsh-host-product-telemetry-otel": "workspace:*"
"@deepseek-ai/dsh-http-proxy": "workspace:*"
}
}
@@ -2,7 +2,7 @@
* OpenTelemetry Service Provider for the DeepSeek Harness telemetry capability.
*
* Authorizes feedback-bounded capture and hands complete event strings to the
* shared SessionLogReporter. This plugin owns resource identity and an outer
* Session-log reporter. This plugin owns resource identity and an outer
* shutdown deadline; the reporter owns byte-bounded SDK delivery.
*
* @module @deepseek-ai/dsh-session-telemetry-otel
@@ -23,7 +23,7 @@ import {
} from '@deepseek-ai/dsh-session-telemetry'
import { APP_IDENTITY } from '@deepseek-ai/dsh-llm'
import { getOrCreateAnonymousUserId } from '@deepseek-ai/dsh-anonymous-user-id'
import { SessionLogReporter } from '@deepseek-ai/dsh-host-product-telemetry-otel'
import { SessionLogReporter } from './session-log.ts'
import type { BatchLogRecordProcessorOptions } from '@opentelemetry/sdk-logs'
import type { OTLPExporterNodeConfigBase } from '@opentelemetry/otlp-exporter-base'
import { SeverityNumber } from '@opentelemetry/api-logs'
@@ -79,24 +79,24 @@ function sharingStatusFor(mode: SessionTelemetryMode): SessionTelemetrySharingSt
}
/**
* Plugin configuration: one sharing policy, two verbatim SDK option objects,
* and one DSH-owned shutdown bound. Uploading modes validate their endpoint
* Plugin configuration: sharing policy, SDK transport options, byte/count queue
* settings, and an overall shutdown bound. Uploading modes validate their endpoint
* and shutdown deadline at plugin load; `DISABLED` reads neither.
*/
export interface Config {
/** Defaults to `FEEDBACK_ONLY`: capture session history only when feedback is explicitly submitted. */
mode?: SessionTelemetryMode
/**
* Explicit SDK HTTP transport settings. The reporter adds x-channel routing
* and does not inherit ambient credentials. URL is required while uploading.
* Explicit SDK HTTP transport settings, including optional routing headers.
* Ambient credentials are not inherited. URL is required while uploading.
*/
exporter?: OTLPExporterNodeConfigBase & {
/** Full logs endpoint (e.g. `https://collector.example.com/v1/logs`). Required outside `DISABLED`; validated at load. */
url?: string
}
/**
* Passed verbatim to `BatchLogRecordProcessor` (minus the exporter slot,
* which this plugin fills); the SDK owns and documents these knobs.
* Count, queue, cadence, and per-request watchdog settings for the byte-bounded
* processor. A watchdog warning never releases an unsettled transport slot.
*/
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
/** Maximum time spent awaiting the SDK provider's complete shutdown path. */
@@ -108,7 +108,7 @@ export interface Config {
/**
* Schemastery validator for {@link Config}; cordis runs it before the plugin
* starts. The constructor validates endpoint and shutdown requirements; the
* shared reporter validates Session byte and queue limits. SDK transport and
* reporter validates Session byte and queue limits. SDK transport and
* processor settings retain their upstream types.
*/
export const Config: z<Config> = z.object({
@@ -127,10 +127,10 @@ export const DEFAULT_SHUTDOWN_TIMEOUT_MILLIS = 3_000
const MAX_TIMER_DELAY_MILLIS = 2_147_483_647
/** Severity mapping from the Service Definition's three-level vocabulary to OTel severity numbers. */
const SEVERITY: Record<SessionTelemetrySeverity, { severityNumber: SeverityNumber; severityText: string }> = {
info: { severityNumber: SeverityNumber.INFO, severityText: 'INFO' },
warn: { severityNumber: SeverityNumber.WARN, severityText: 'WARN' },
error: { severityNumber: SeverityNumber.ERROR, severityText: 'ERROR' },
const SEVERITY: Record<SessionTelemetrySeverity, SeverityNumber> = {
info: SeverityNumber.INFO,
warn: SeverityNumber.WARN,
error: SeverityNumber.ERROR,
}
/**
@@ -174,10 +174,7 @@ export class OpenTelemetrySessionBackend extends SessionTelemetryBackend {
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') {
throw new Error(`session-telemetry-otel: exporter.url must be http(s), got ${parsed.protocol}`)
}
// The one processor field checked beyond the SDK's own validation: the
// SDK accepts a non-positive batch size, but its shutdown drain then
// splices empty batches without consuming the queue — dispose would hang
// forever with records queued. Misconfiguration fails at load instead.
// Reject empty batches before allocating the reporter.
const batchSize = config.processor?.maxExportBatchSize
if (batchSize !== undefined && (!Number.isInteger(batchSize) || batchSize < 1)) {
throw new Error(`session-telemetry-otel: processor.maxExportBatchSize must be a positive integer, got ${String(batchSize)}`)
@@ -200,11 +197,14 @@ export class OpenTelemetrySessionBackend extends SessionTelemetryBackend {
})
this.provider = reporter
const enqueue: SessionTelemetrySink['emit'] = (record) => {
if (record.sourceEvent === undefined) return
if (record.sourceEvent === undefined) {
ctx.logger.warn('Session log record withheld: redaction removed sourceEvent')
return
}
reporter.reportSessionLog({
sessionId: record.sourceEvent.sessionId,
event: { ...record.sourceEvent.envelope, data: record.body },
severityNumber: SEVERITY[record.severity].severityNumber,
severityNumber: SEVERITY[record.severity],
attributes: record.attributes,
})
}
@@ -246,23 +246,11 @@ export class OpenTelemetrySessionBackend extends SessionTelemetryBackend {
*/
emit(_record: SessionTelemetryRecord): void {}
// The Service Definition's optional flush() hint is deliberately NOT implemented. The
// batch processor exports on its own cadence (`processor.scheduledDelayMillis`,
// the SDK's documented knob), and this backend is the SDK pipeline's only
// caller — forwarding the hint to `forceFlush()` would be the sole source of
// concurrent flushes, whose undocumented interactions with shutdown's
// internal drain (concurrent-flush guard, provider-level flush timeout)
// silently drop tail records. Rationale and the revival trigger: the
// revival Agent Note.
/**
* Ask the SDK to drain and quiesce, but reject after the backend-owned
* deadline. OTel's processor export timeout wraps `exportCompleted` only;
* shutdown awaits `exporter.forceFlush()` first, which can remain pending
* when the transport never obtains a socket. The provider promise remains
* observed after the deadline so a later rejection cannot become unhandled.
* `DISABLED` has no provider and resolves immediately.
* @returns resolves when the SDK pipeline quiesces or is disabled, or rejects at the configured deadline.
* Drain queued HTTP requests until the deployment deadline. The watchdog
* never releases an unsettled transport slot. At the outer deadline,
* queued records are abandoned and no further requests may start.
* @returns completion after transport shutdown, or rejection at the configured deadline.
*/
async shutdown(): Promise<void> {
if (this.provider === undefined) return
@@ -270,6 +258,7 @@ export class OpenTelemetrySessionBackend extends SessionTelemetryBackend {
let timer: ReturnType<typeof setTimeout> | undefined
const deadline = new Promise<never>((_resolve, reject) => {
timer = setTimeout(() => {
this.provider?.stopPending()
reject(new Error(`session-telemetry-otel: provider shutdown exceeded ${this.shutdownTimeoutMillis}ms`))
}, this.shutdownTimeoutMillis)
})
@@ -0,0 +1,222 @@
/** Session-log records in an independent byte-bounded OTLP queue. */
import { createRequire } from 'node:module'
import type { SessionEvent, SessionId } from '@deepseek-ai/dsh-session'
import type { Attributes } from '@opentelemetry/api'
import { SeverityNumber } from '@opentelemetry/api-logs'
import { ExportResultCode } from '@opentelemetry/core'
import type { OTLPExporterNodeConfigBase } from '@opentelemetry/otlp-exporter-base'
import { JsonLogsSerializer } from '@opentelemetry/otlp-transformer'
import { resourceFromAttributes } from '@opentelemetry/resources'
import { LoggerProvider, type BatchLogRecordProcessorOptions, type LogRecordExporter, type ReadableLogRecord, type LogRecordProcessor } from '@opentelemetry/sdk-logs'
import { createLogExporter } from './transport.ts'
/** Collector request ceiling in uncompressed UTF-8 bytes, including the OTLP envelope. */
export const SESSION_LOG_MAX_REQUEST_BYTES = 4_000_000
/** One canonical event with its separately owned Session identity and redacted payload. */
export interface SessionLogRecord {
sessionId: SessionId
/** Complete event envelope; data is the capture policy's exported copy. */
event: Omit<SessionEvent, 'data'> & { data: unknown }
/** Additional capture metadata; sessionId and content are always assigned by the reporter. */
attributes?: Attributes
/** Omitted values use INFO. */
severityNumber?: SeverityNumber
}
/** Session-log transport and byte/count queue settings. */
export interface SessionLogOptions {
/** Explicit destination and SDK transport options. */
exporter: OTLPExporterNodeConfigBase & {
/** Full HTTP(S) logs destination. */
url: string
}
/** Session-only queue settings, independent of product-event aggregation. */
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
/** May lower, but never exceed, the collector's 4,000,000-byte limit. */
maxRequestBytes?: number
/** Application and anonymous identity carried on the OTLP resource. */
resourceAttributes: Attributes
/** Report rejected single records and network failures without recording their content. */
onFailure: (message: string, error?: Error) => void
}
/**
* Validate byte and queue settings before constructing an SDK pipeline.
* @param options - Session-specific limits supplied by the owning composition.
* @returns the resolved collector request limit.
*/
export function resolveSessionLogLimits(options: Pick<SessionLogOptions, 'maxRequestBytes' | 'processor'>): number {
const limit = options.maxRequestBytes ?? SESSION_LOG_MAX_REQUEST_BYTES
if (!Number.isSafeInteger(limit) || limit < 1 || limit > SESSION_LOG_MAX_REQUEST_BYTES) {
throw new Error(`session log maxRequestBytes must be an integer between 1 and ${SESSION_LOG_MAX_REQUEST_BYTES}`)
}
for (const key of ['maxQueueSize', 'maxExportBatchSize', 'scheduledDelayMillis', 'exportTimeoutMillis'] as const) {
const value = options.processor?.[key]
if (value !== undefined && (!Number.isSafeInteger(value) || value < 1 || value > 2_147_483_647)) {
throw new Error(`session log processor.${key} must be a positive integer no greater than 2147483647`)
}
}
const queue = options.processor?.maxQueueSize ?? 2048
const batch = options.processor?.maxExportBatchSize ?? 512
if (batch > queue) throw new Error('session log maxExportBatchSize must not exceed maxQueueSize')
return limit
}
/** Byte/count batching with one transport request in flight, including after a watchdog warning. */
class SessionLogProcessor implements LogRecordProcessor {
private readonly queue: { record: ReadableLogRecord; bytes: number }[] = []
private bytes = 0
private timer: ReturnType<typeof setTimeout> | undefined
private active: Promise<void> | undefined
private shutdownPromise: Promise<void> | undefined
private stopped = false
constructor(
private readonly exporter: LogRecordExporter,
private readonly limit: number,
private readonly config: Required<Pick<BatchLogRecordProcessorOptions, 'maxQueueSize' | 'maxExportBatchSize' | 'scheduledDelayMillis' | 'exportTimeoutMillis'>>,
private readonly warn: SessionLogOptions['onFailure'],
) {}
onEmit(record: ReadableLogRecord): void {
if (this.stopped) return
if (this.queue.length >= this.config.maxQueueSize) {
this.warn('Session log queue is full; record rejected')
return
}
let bytes: number
try {
const serialized = JsonLogsSerializer.serializeRequest([record])
if (serialized === undefined) throw new Error('Session log serialization produced no request')
bytes = serialized.byteLength
} catch (error) {
this.warn('Session log serialization failed; record rejected', error instanceof Error ? error : new Error(String(error)))
return
}
if (bytes > this.limit) {
this.warn('Session log record rejected; content was not truncated', new Error(`Session log record exceeds maxRequestBytes: ${bytes} > ${this.limit}`))
return
}
this.queue.push({ record, bytes })
this.bytes += bytes
if (this.active !== undefined) return
if (this.queue.length >= this.config.maxExportBatchSize || this.bytes >= this.limit) {
void this.forceFlush()
} else if (this.timer === undefined) {
this.timer = setTimeout(() => { void this.forceFlush() }, this.config.scheduledDelayMillis)
this.timer.unref()
}
}
forceFlush(): Promise<void> {
clearTimeout(this.timer)
this.timer = undefined
if (this.active !== undefined) return this.active
if (this.queue.length === 0) return Promise.resolve()
this.active = this.drain()
return this.active
}
private async drain(): Promise<void> {
try {
while (!this.stopped && this.queue.length > 0) {
let bytes = 0
let count = 0
for (const entry of this.queue) {
if (count === this.config.maxExportBatchSize || bytes + entry.bytes > this.limit) break
bytes += entry.bytes
count++
}
// Each measured record includes its resource/scope envelope. The shared
// envelope in this provider's multi-record request cannot exceed their sum.
const records = this.queue.splice(0, count).map(entry => entry.record)
this.bytes -= bytes
await this.send(records)
}
} finally { this.active = undefined }
}
private send(records: ReadableLogRecord[]): Promise<void> {
return new Promise((resolve) => {
const timer = setTimeout(() => {
this.warn('Session log request exceeded exportTimeoutMillis; waiting for transport settlement')
}, this.config.exportTimeoutMillis)
timer.unref()
const finish = (error?: Error): void => {
clearTimeout(timer)
if (error !== undefined) this.warn('Session log export failed', error)
resolve()
}
try {
this.exporter.export(records, (result) => {
finish(result.code === ExportResultCode.SUCCESS
? undefined : result.error ?? new Error('Session log HTTP export failed'))
})
} catch (error) {
finish(error instanceof Error ? error : new Error(String(error)))
}
})
}
stopPending(): void {
this.stopped = true
this.queue.length = 0
this.bytes = 0
clearTimeout(this.timer)
this.timer = undefined
}
shutdown(): Promise<void> {
this.shutdownPromise ??= this.forceFlush().then(() => this.exporter.shutdown())
return this.shutdownPromise
}
}
/** Owns feedback-authorized Session logs; no product-event provider or queue is mounted. */
export class SessionLogReporter {
private readonly provider: LoggerProvider
private readonly processor: SessionLogProcessor
private readonly logger: ReturnType<LoggerProvider['getLogger']>
/** @param options - explicit transport, resource identity, queue limits, and diagnostics. */
constructor(options: SessionLogOptions) {
const limit = resolveSessionLogLimits(options)
this.processor = new SessionLogProcessor(createLogExporter(options.exporter), limit, {
maxQueueSize: options.processor?.maxQueueSize ?? 2048,
maxExportBatchSize: options.processor?.maxExportBatchSize ?? 512,
scheduledDelayMillis: options.processor?.scheduledDelayMillis ?? 1000,
exportTimeoutMillis: options.processor?.exportTimeoutMillis ?? 30000,
}, options.onFailure)
this.provider = new LoggerProvider({
logRecordLimits: { attributeValueLengthLimit: Infinity, attributeCountLimit: Infinity },
resource: resourceFromAttributes(options.resourceAttributes),
processors: [this.processor],
})
const { version } = createRequire(import.meta.url)('../package.json') as { version: string }
this.logger = this.provider.getLogger('@deepseek-ai/dsh-session-telemetry-otel', version)
}
/**
* Enqueue one complete event without acknowledging network delivery.
* @param record - event with redacted data and its original Session id.
*/
reportSessionLog(record: SessionLogRecord): void {
const severityNumber = record.severityNumber ?? SeverityNumber.INFO
this.logger.emit({
eventName: 'session-log', body: 'session-log',
timestamp: record.event.time, observedTimestamp: record.event.time,
severityNumber, severityText: SeverityNumber[severityNumber],
attributes: { ...record.attributes, sessionId: record.sessionId, content: JSON.stringify(record.event) },
})
}
/** Stop queued requests after the owning backend's shutdown deadline; an active transport may still settle. */
stopPending(): void { this.processor.stopPending() }
/**
* Drain queued requests and release the SDK transport.
* @returns completion after queued requests settle and the SDK transport shuts down.
*/
shutdown(): Promise<void> { return this.provider.shutdown() }
}
@@ -1,4 +1,4 @@
/** Explicit collector transport shared by product and Session-log pipelines. */
/** Explicit OTLP JSON transport for feedback-authorized Session logs. */
import { createOtlpHttpExportDelegate, getSharedConfigurationFromEnvironment, httpAgentFactoryFromOptions } from '@opentelemetry/otlp-exporter-base/node-http'
import { getSharedConfigurationDefaults, mergeOtlpSharedConfigurationWithDefaults, OTLPExporterBase, type OTLPExporterNodeConfigBase } from '@opentelemetry/otlp-exporter-base'
import { JsonLogsSerializer } from '@opentelemetry/otlp-transformer'
@@ -15,9 +15,10 @@ export function createLogExporter(options: OTLPExporterNodeConfigBase & { url: s
...shared,
url: options.url,
headers: async () => ({
'Content-Type': 'application/json', 'x-channel': 'dsh_otel_report',
'Content-Type': 'application/json',
...typeof options.headers === 'function' ? await options.headers() : options.headers,
}),
// An agent factory owns the returned agent, including its keepAlive setting.
agentFactory: typeof options.httpAgentOptions === 'function' ? options.httpAgentOptions
: httpAgentFactoryFromOptions({ keepAlive: options.keepAlive ?? true, ...options.httpAgentOptions }),
...(options.userAgent === undefined ? {} : { userAgent: options.userAgent }),
@@ -235,7 +235,7 @@ describe('OpenTelemetrySessionBackend wire', () => {
expect(allRecords(captures).some(r => r.scope.endsWith('/ops'))).toBe(false)
})
it('bounds the SDK forceFlush wait when an in-flight transport never settles', async () => {
it('stops queued sends at the shutdown deadline while observing the active transport', async () => {
const gate = Promise.withResolvers<boolean>()
const arrived = Promise.withResolvers<boolean>()
const { url, captures } = await mockCollector(async (index) => {
@@ -268,7 +268,8 @@ describe('OpenTelemetrySessionBackend wire', () => {
// the real provider promise remains clean after the test has proved the
// Cordis disposer no longer waits for it.
gate.resolve(true)
await expect.poll(() => captures.length).toBeGreaterThanOrEqual(2)
await expect.poll(() => captures.length).toBe(1)
expect(JSON.stringify(captures)).not.toContain('second report')
})
it('passes exporter options beyond url and headers through to the SDK exporter', async () => {
@@ -798,7 +799,6 @@ describe('dsh-session-telemetry-otel real-load-path guard', () => {
})
})
it('reports oversized records without failing feedback or exporting their contents', async () => {
const { url, captures } = await mockCollector()
const ctx = new Context()
@@ -812,7 +812,7 @@ it('reports oversized records without failing feedback or exporting their conten
expect(() => { recordFeedback(session, { text: 'feedback remains local' }) }).not.toThrow()
await fiber.dispose()
expect(captures).toEqual([])
expect(warn).toHaveBeenCalledWith('Session log export failed', expect.any(Error))
expect(warn).toHaveBeenCalledWith('Session log record rejected; content was not truncated', expect.any(Error))
} finally {
await ctx.fiber.dispose()
}
@@ -821,6 +821,7 @@ it('reports oversized records without failing feedback or exporting their conten
it('does not upload a record whose redaction policy removes its canonical envelope', async () => {
const { url, captures } = await mockCollector()
const { ctx, fiber } = await boot(url)
const warn = vi.spyOn(ctx.logger, 'warn')
try {
ctx.on('session-telemetry/record', (_record, next) => {
const { sourceEvent: _sourceEvent, ...redacted } = next()
@@ -829,6 +830,7 @@ it('does not upload a record whose redaction policy removes its canonical envelo
recordFeedback(ctx.sessions.create(SessionId('withheld-envelope')), { text: 'withheld' })
await fiber.dispose()
expect(captures).toEqual([])
expect(warn).toHaveBeenCalledWith('Session log record withheld: redaction removed sourceEvent')
} finally {
await ctx.fiber.dispose()
}
@@ -0,0 +1,272 @@
import { createRequire } from 'node:module'
import { Agent, createServer, type IncomingHttpHeaders } from 'node:http'
import { once } from 'node:events'
import { gunzipSync } from 'node:zlib'
import { afterEach, beforeEach, expect, it, vi } from 'vitest'
import { CompressionAlgorithm } from '@opentelemetry/otlp-exporter-base'
import { JsonLogsSerializer } from '@opentelemetry/otlp-transformer'
import { SessionId, SessionSeq } from '@deepseek-ai/dsh-session'
import { SESSION_LOG_MAX_REQUEST_BYTES, SessionLogReporter, type SessionLogRecord, type SessionLogOptions } from '../src/session-log.ts'
interface Capture {
bytes: number
headers: IncomingHttpHeaders
body: { resourceLogs: { resource: unknown; scopeLogs: { logRecords: Record<string, unknown>[] }[] }[] }
}
const cleanup: (() => Promise<void>)[] = []
afterEach(async () => {
try {
for (const dispose of cleanup.splice(0).reverse()) await dispose()
} finally {
vi.unstubAllEnvs()
vi.restoreAllMocks()
vi.useRealTimers()
}
})
beforeEach(() => {
vi.stubEnv('OTEL_EXPORTER_OTLP_COMPRESSION', undefined)
vi.stubEnv('OTEL_EXPORTER_OTLP_LOGS_COMPRESSION', undefined)
})
async function collector(statuses = [200]) {
const captures: Capture[] = []
const server = createServer((req, res) => {
const chunks: Buffer[] = []
req.on('data', chunk => chunks.push(chunk as Buffer))
req.on('end', () => {
const raw = Buffer.concat(chunks)
const bytes = req.headers['content-encoding'] === 'gzip' ? gunzipSync(raw) : raw
captures.push({ bytes: bytes.length, headers: req.headers, body: JSON.parse(bytes.toString()) as Capture['body'] })
res.writeHead(statuses.shift() ?? 200, { 'content-type': 'application/json' }).end('{}')
})
})
cleanup.push(async () => {
const closed = once(server, 'close')
server.close()
server.closeAllConnections()
await closed
})
server.listen(0, '127.0.0.1')
await once(server, 'listening')
const address = server.address()
if (address === null || typeof address === 'string') throw new Error('collector has no port')
return { captures, endpoint: `http://127.0.0.1:${address.port}/v1/logs` }
}
function sessionRecord(text: string, seq = 0): SessionLogRecord {
return {
sessionId: SessionId('synthetic-session'),
event: { type: 'user/message', seq: SessionSeq(seq), time: 1_800_000_000_000, surfaceOp: 'append',
data: { content: [{ type: 'text', text }], nested: { items: [null, true, false, 0, 1.5, { text }] } } },
}
}
function logs(capture: Capture) {
return capture.body.resourceLogs.flatMap(resource => resource.scopeLogs.flatMap(scope => scope.logRecords))
}
function contents(captures: Capture[]): string[] {
return captures.flatMap(capture => logs(capture).map((record) => {
const attributes = record['attributes'] as { key: string; value: { stringValue: string } }[]
return attributes.find(attribute => attribute.key === 'content')!.value.stringValue
}))
}
function parseContent(content: string): unknown {
return JSON.parse(content)
}
function reporter(endpoint: string, overrides: Partial<SessionLogOptions> = {}) {
const onFailure = vi.fn()
const sender = new SessionLogReporter({
exporter: { url: endpoint, timeoutMillis: 1000 }, resourceAttributes: { 'service.name': 'session-test' },
processor: { scheduledDelayMillis: 60000 }, onFailure, ...overrides,
})
cleanup.push(() => sender.shutdown())
return { sender, onFailure }
}
it('preserves full nested events without truncation or ambient headers and retains scope version', async () => {
vi.stubEnv('OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT', '8')
vi.stubEnv('OTEL_EXPORTER_OTLP_HEADERS', 'Authorization=Bearer%20unrelated')
const target = await collector()
const { sender } = reporter(target.endpoint)
const record = sessionRecord('中文\n"quote"\\path 😀')
sender.reportSessionLog(record)
await sender.shutdown()
expect(contents(target.captures).map(parseContent)).toEqual([record.event])
expect(logs(target.captures[0]!)[0]).toMatchObject({ eventName: 'session-log' })
expect(target.captures[0]!.headers).not.toHaveProperty('authorization')
expect(target.captures[0]!.headers).not.toHaveProperty('x-channel')
expect(target.captures[0]!.body.resourceLogs[0]!.scopeLogs[0]).toMatchObject({
scope: { name: '@deepseek-ai/dsh-session-telemetry-otel', version: (createRequire(import.meta.url)('../package.json') as { version: string }).version },
})
})
it.each([CompressionAlgorithm.GZIP, CompressionAlgorithm.NONE])('bounds %s requests and preserves nested content', async (compression) => {
const target = await collector()
const { sender } = reporter(target.endpoint, { exporter: { url: target.endpoint, compression } })
const records = [0,1,2].map(seq => sessionRecord('中"\\'.repeat(100000), seq))
for (const record of records) sender.reportSessionLog(record)
await sender.shutdown()
expect(target.captures.length).toBeGreaterThan(1)
expect(target.captures.every(c => c.bytes <= SESSION_LOG_MAX_REQUEST_BYTES)).toBe(true)
expect(contents(target.captures).map(parseContent)).toEqual(records.map(r => r.event))
})
it('admits an exact request and diagnoses an oversized record only once', async () => {
const target = await collector()
const record = sessionRecord('边界"\\')
const baseline = reporter(target.endpoint).sender
baseline.reportSessionLog(record)
await baseline.shutdown()
const limit = target.captures[0]!.bytes
const exact = reporter(target.endpoint, { maxRequestBytes: limit }).sender
exact.reportSessionLog(record)
await exact.shutdown()
const small = reporter(target.endpoint, { maxRequestBytes: limit - 1 })
small.sender.reportSessionLog(record)
await small.sender.shutdown()
expect(target.captures.map(c => c.bytes)).toEqual([limit, limit])
expect(small.onFailure).toHaveBeenCalledTimes(1)
expect(small.onFailure).toHaveBeenCalledWith('Session log record rejected; content was not truncated', expect.any(Error))
})
it('continues after an oversized record and a failed HTTP request', async () => {
const target = await collector([400,200])
const { sender, onFailure } = reporter(target.endpoint, { maxRequestBytes: 1300 })
const records = [sessionRecord('first'.repeat(25)), sessionRecord('x'.repeat(4000000),1), sessionRecord('last'.repeat(25),2)]
for(const record of records) sender.reportSessionLog(record)
await sender.shutdown()
expect(contents(target.captures).map(parseContent)).toEqual([records[0]!.event,records[2]!.event])
expect(onFailure).toHaveBeenCalledTimes(2)
})
it.each([0,-1,1.5,4000001,Infinity])('rejects byte limit %s', (maxRequestBytes) => {
expect(() => reporter('http://collector.test/logs',{ maxRequestBytes })).toThrow('maxRequestBytes')
})
it.each(['maxQueueSize','maxExportBatchSize','scheduledDelayMillis','exportTimeoutMillis'] as const)('rejects invalid %s', (key) => {
expect(() => reporter('http://collector.test/logs',{ processor:{ [key]:0 } })).toThrow(`processor.${key}`)
})
it('rejects a batch count above the queue capacity', () => {
expect(() => reporter('http://collector.test/logs',{ processor:{ maxQueueSize:1,maxExportBatchSize:2 } })).toThrow('maxExportBatchSize must not exceed maxQueueSize')
})
it('honors explicit routing headers and caller-owned agent settings', async () => {
const target = await collector()
const agent = new Agent({ keepAlive:false })
cleanup.push(async () => {agent.destroy()})
const { sender } = reporter(target.endpoint,{ exporter:{ url:target.endpoint,userAgent:'session-test',keepAlive:true,httpAgentOptions:async () => agent,headers:async () => ({ 'x-channel':'test-channel' }) } })
sender.reportSessionLog(sessionRecord('agent'))
await sender.shutdown()
expect(target.captures[0]!.headers.connection).toBe('close')
expect(target.captures[0]!.headers['x-channel']).toBe('test-channel')
expect(target.captures[0]!.headers['user-agent']).toContain('session-test')
})
it.each([new Error('serialize'), 'serialize', undefined])('contains serializer failures %s', (failure) => {
const { sender,onFailure } = reporter('http://collector.test/logs')
vi.spyOn(JsonLogsSerializer,'serializeRequest').mockImplementationOnce(() => { if(failure===undefined)return undefined; throw failure })
sender.reportSessionLog(sessionRecord('invalid'))
expect(onFailure).toHaveBeenCalledWith('Session log serialization failed; record rejected',expect.any(Error))
})
it('measures each record once instead of serializing the full candidate batch', async () => {
const target = await collector()
const { sender } = reporter(target.endpoint,{ maxRequestBytes:1300 })
const serialize = vi.spyOn(JsonLogsSerializer,'serializeRequest')
const records = Array.from({ length:100 },(_,seq) => sessionRecord('batch'.repeat(20),seq))
for(const record of records)sender.reportSessionLog(record)
await sender.shutdown()
expect(contents(target.captures).map(parseContent)).toEqual(records.map(r=>r.event))
expect(serialize.mock.calls.every(([records])=>records.length===1)).toBe(true)
expect(serialize).toHaveBeenCalledTimes(records.length+target.captures.length)
})
it('keeps one transport slot across multiple byte batches and times each request independently', async () => {
vi.useFakeTimers()
const transport = await import('../src/transport.ts')
const sent: number[] = []
let active = 0
let peak = 0
vi.spyOn(transport, 'createLogExporter').mockReturnValue({
export(records, callback) {
active++
peak = Math.max(peak, active)
sent.push(...records.map(r => (JSON.parse(r.attributes.content as string) as { seq: number }).seq))
setTimeout(() => { active--; callback({ code: 0 }) }, 800)
},
forceFlush: async () => {}, shutdown: async () => {},
})
const { sender, onFailure } = reporter('http://collector.test/logs', {
exporter: { url: 'http://collector.test/logs', concurrencyLimit: 1 },
maxRequestBytes: 1300,
processor: { maxExportBatchSize: 3, maxQueueSize: 6, scheduledDelayMillis: 60000, exportTimeoutMillis: 1500 },
})
for(let seq = 0; seq < 6; seq++) sender.reportSessionLog(sessionRecord('content'.repeat(20), seq))
const done = sender.shutdown()
await vi.advanceTimersByTimeAsync(4800)
await done
expect(sent).toEqual([0,1,2,3,4,5])
expect(peak).toBe(1)
expect(onFailure).not.toHaveBeenCalled()
})
it('does not release the transport slot on a watchdog timeout or send queued records after the shutdown deadline', async () => {
vi.useFakeTimers()
const transport = await import('../src/transport.ts')
const callback = Promise.withResolvers<(result: { code: 0 }) => void>()
const send = vi.fn((_records, complete: (result: { code: 0 }) => void) => { callback.resolve(complete) })
vi.spyOn(transport, 'createLogExporter').mockReturnValue({ export: send, forceFlush: async () => {}, shutdown: async () => {} })
const { sender, onFailure } = reporter('http://collector.test/logs', {
processor: { maxExportBatchSize: 1, maxQueueSize: 2, exportTimeoutMillis: 100, scheduledDelayMillis: 1000 },
})
sender.reportSessionLog(sessionRecord('first'))
const finish = await callback.promise
cleanup.push(async () => { sender.stopPending(); finish({ code: 0 }) })
sender.reportSessionLog(sessionRecord('queued',1))
await vi.advanceTimersByTimeAsync(500)
expect(send).toHaveBeenCalledTimes(1)
expect(onFailure).toHaveBeenCalledWith('Session log request exceeded exportTimeoutMillis; waiting for transport settlement')
sender.stopPending()
sender.reportSessionLog(sessionRecord('after deadline',2))
const done = sender.shutdown()
finish({ code:0 })
await done
sender.reportSessionLog(sessionRecord('late',2))
expect(send).toHaveBeenCalledTimes(1)
})
it.each([new Error('transport'), 'transport', undefined])('contains transport failures %s and continues later requests', async (failure) => {
const transport = await import('../src/transport.ts')
const send = vi.fn<import('@opentelemetry/sdk-logs').LogRecordExporter['export']>()
.mockImplementationOnce((_records, callback) => { if(failure === undefined) callback({ code:1 }); else throw failure })
.mockImplementation((_records, callback) => { callback({ code:0 }) })
vi.spyOn(transport,'createLogExporter').mockReturnValue({ export:send,forceFlush:async()=>{},shutdown:async()=>{} })
const { sender,onFailure } = reporter('http://collector.test/logs',{ processor:{ maxExportBatchSize:1 } })
sender.reportSessionLog(sessionRecord('failed'))
sender.reportSessionLog(sessionRecord('next',1))
await sender.shutdown()
expect(send).toHaveBeenCalledTimes(2)
expect(onFailure).toHaveBeenCalledTimes(1)
})
it('bounds the queued record count while a transport request is unsettled', async () => {
const transport = await import('../src/transport.ts')
const release = Promise.withResolvers<(result:{ code:0 })=>void>()
vi.spyOn(transport,'createLogExporter').mockReturnValue({
export(_records,callback){release.resolve(callback)},forceFlush:async()=>{},shutdown:async()=>{},
})
const { sender,onFailure } = reporter('http://collector.test/logs',{ processor:{ maxExportBatchSize:1,maxQueueSize:2 } })
for(let seq=0;seq<4;seq++)sender.reportSessionLog(sessionRecord('queued',seq))
const finish = await release.promise
cleanup.push(async () => { sender.stopPending(); finish({ code: 0 }) })
expect(onFailure).toHaveBeenCalledWith('Session log queue is full; record rejected')
sender.stopPending()
const done=sender.shutdown()
finish({ code:0 })
await done
})
@@ -34,9 +34,6 @@
},
{
"path": "../../identity/anonymous-user-id"
},
{
"path": "../../host/product-telemetry-otel"
}
]
}
@@ -36,8 +36,8 @@
en: cad1e9d195916553
zh: 2192f3ce5e2196bd
/deepseek-ai-dsh-session-telemetry/understand-the-implementation/design-concept:
en: 6a606a2a6549b063
zh: a6e23d4bf5c522ab
en: 56c449be11e6bac1
zh: 6b62969e93313c3e
/deepseek-ai-dsh-session-telemetry/understand-the-implementation/source-map:
en: 5dc345ec3dda38b8
zh: 189d673734a2c20e
+1 -1
View File
@@ -65,7 +65,7 @@ This section explains the capture design; the observable behavior is fully cover
### Design concept
The capture package owns complete event capture, redaction, and the handoff cursor. Backends own delivery; the OTel implementation uses SDK queues and retries with collector-specific byte-bounded requests. The [revival Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md) records the capture and redaction rationale.
The capture package owns complete event capture, redaction, and handoff cursors. Redaction rules must preserve `sourceEvent` for OTel upload; returning a fresh record without it withholds the event with a diagnostic. Its cloned envelope excludes `data`, which is carried only in `body`. The OTel backend owns byte/count scheduling and uses SDK transport/retries. The [revival Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md) owns capture and redaction rationale.
### Source map
@@ -65,7 +65,7 @@ ledger 记录携带 `sourceEvent`,其中包含所属 Session id 和不含 `dat
### 设计理念
采集包负责完整事件采集、脱敏和交接游标。后端负责投递;OTel 实现使用 SDK 队列和重试,并按采集端的字节上限限制请求。[恢复遥测的 Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md) 记录采集与脱敏的理由。
捕获包负责完整事件捕获、脱敏和交接游标。脱敏规则必须保留 `sourceEvent` 才能通过 OTel 上传;返回不含它的新记录会阻止该事件上传并产生诊断。复制的 envelope 不含 `data`,数据仅由 `body` 携带。OTel 后端负责字节/条数调度并使用 SDK 传输和重试。[恢复遥测决策记录](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md) 说明捕获和脱敏依据。
### 源码地图
@@ -135,6 +135,12 @@ describe('SessionTelemetryCoordinator capture', () => {
;(message.body as { content: { text: string }[] }).content[0]!.text = 'tampered'
const logged = session.snapshotEvents()[1] as SessionEvent<'user/message'>
expect(logged.data.content[0]).toMatchObject({ text: 'hello' })
expect(message.sourceEvent?.sessionId).toBe(session.id)
expect(message.sourceEvent?.envelope).not.toHaveProperty('data')
expect(message.sourceEvent?.envelope).toMatchObject({ seq: logged.seq, time: logged.time, type: logged.type, surfaceOp: 'append' })
expect(message.sourceEvent?.envelope).not.toBe(logged)
message.sourceEvent!.envelope.time = 0
expect(logged.time).not.toBe(0)
})
it('captures a live request header without replaying previously withheld events', async () => {
+6 -9
View File
@@ -9026,9 +9026,6 @@ importers:
'@deepseek-ai/dsh-loader-smoke':
specifier: workspace:*
version: link:../../test-support/loader-smoke
'@deepseek-ai/dsh-session':
specifier: workspace:*
version: link:../../core/session
packages/host/webserver:
dependencies:
@@ -10920,9 +10917,15 @@ importers:
'@opentelemetry/api-logs':
specifier: ^0.220.0
version: 0.220.0
'@opentelemetry/core':
specifier: 2.9.0
version: 2.9.0(@opentelemetry/[email protected])
'@opentelemetry/otlp-exporter-base':
specifier: ^0.220.0
version: 0.220.0(@opentelemetry/[email protected])
'@opentelemetry/otlp-transformer':
specifier: ^0.220.0
version: 0.220.0(@opentelemetry/[email protected])
'@opentelemetry/resources':
specifier: ^2.9.0
version: 2.10.0(@opentelemetry/[email protected])
@@ -10951,9 +10954,6 @@ importers:
'@deepseek-ai/dsh-command-feedback':
specifier: workspace:*
version: link:../../feedback/command-feedback
'@deepseek-ai/dsh-host-product-telemetry-otel':
specifier: workspace:*
version: link:../../host/product-telemetry-otel
'@deepseek-ai/dsh-http-proxy':
specifier: workspace:*
version: link:../../util/http-proxy
@@ -13983,9 +13983,6 @@ importers:
'@deepseek-ai/dsh-host-plugin-inventory':
specifier: workspace:*
version: link:../../packages/host/plugin-inventory
'@deepseek-ai/dsh-host-product-telemetry-otel':
specifier: workspace:*
version: link:../../packages/host/product-telemetry-otel
'@deepseek-ai/dsh-host-webserver':
specifier: workspace:*
version: link:../../packages/host/webserver
-1
View File
@@ -55,7 +55,6 @@
"@deepseek-ai/dsh-hooks-claude-code": "workspace:*",
"@deepseek-ai/dsh-hooks-codex": "workspace:*",
"@deepseek-ai/dsh-host-plugin-inventory": "workspace:*",
"@deepseek-ai/dsh-host-product-telemetry-otel": "workspace:*",
"@deepseek-ai/dsh-host-webserver": "workspace:*",
"@deepseek-ai/dsh-http-proxy": "workspace:*",
"@deepseek-ai/dsh-invariants": "workspace:*",
-1
View File
@@ -274,7 +274,6 @@ export const EVENT_WALK_EXEMPTIONS: Record<string, string> = {
*/
export const LINK_MAP: Readonly<Record<string, string>> = {
ProductTelemetryRecord: 'product-telemetry.md',
SessionLogRecord: 'product-telemetry.md',
ProductTelemetryScalar: 'product-telemetry.md',
WorkspaceChangesSummary: 'deliverables.md',
WorkspaceFileDiff: 'deliverables.md',
-10
View File
@@ -2336,16 +2336,6 @@
"doc": "docs/subsystems/product-telemetry.md",
"symbol": "ProductTelemetryRecord",
"source": "packages/host/product-telemetry-otel/src/index.ts"
},
{
"doc": "docs/subsystems/product-telemetry.md",
"symbol": "SessionLogRecord",
"source": "packages/host/product-telemetry-otel/src/session-log.ts"
},
{
"doc": "docs/subsystems/product-telemetry.md",
"symbol": "SessionLogOptions",
"source": "packages/host/product-telemetry-otel/src/session-log.ts"
}
]
}