perf(llm): avoid aggregate pi-ai runtime import

This commit is contained in:
imccyu committed 2026-09-15 20:45:49 +08:00
1 parent ee9f43bc5d
commit 75a56be10c
13 files changed
+159 -11

No files matched your search

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-15-narrow-pi-ai-runtime-imports.md
2026-09-15-narrow-pi-ai-runtime-imports.md: 307d880f3fce22b0e1b96efb9c3a220a511eff15
2026-09-15-narrow-pi-ai-runtime-imports.zh.md: 139983446a2f0cd5315629c4e0a632f8c125a837
@@ -0,0 +1,25 @@
# Agent Note: Narrow pi-ai runtime imports
Status: implemented
English | [中文](2026-09-15-narrow-pi-ai-runtime-imports.zh.md)
## Problem
The base bundle mounts `dsh-llm-pi-ai` with no configured routes so the Models settings page can offer pi-ai providers. Importing pi-ai's aggregate entry point for model helpers also evaluates its exported TypeBox namespace, adding hundreds of modules to every application startup even when all Sessions use `dsh-llm-deepseek`.
## Decision
`dsh-llm-pi-ai` has no runtime import of pi-ai's aggregate entry point. Catalog and login metadata continue through `providers/all`; protocol implementations use their existing `api/*.lazy` entries; overflow detection uses `utils/overflow`. Package-local `models.ts` supplies the three model helpers the adapter needs. Its collection comes from pi-ai's public `builtinModels()` implementation and is cleared before route providers are installed. Its provider constructor implements the static single-protocol case this adapter supplies. Its reasoning-level selection reads pi-ai's public `Model` metadata in pi-ai's escalation order.
Type-only imports from the aggregate entry point remain because TypeScript erases them. Import profiling of the built package resolves 153 pi-ai modules and no TypeBox modules or pi-ai aggregate entry.
## Alternatives considered
- **Add a pi-ai `models` export.** Rejected because this package does not need an upstream export-map change to consume the public provider, API, utility, and model metadata interfaces already available.
- **Dynamically import the aggregate entry point.** Rejected because the dormant adapter needs none of it; excluding the entry entirely removes the work instead of moving it to a later operation.
- **Copy pi-ai's complete Models implementation.** Rejected because `builtinModels()` already returns the upstream implementation with its authentication and storage behavior. Clearing its providers preserves that implementation without maintaining a fork.
## Consequences
Applications still load `providers/all` so configuration and authorization surfaces retain the complete installed provider directory. Constructing an adapter snapshot briefly constructs and then clears the built-in provider set before installing the resolved route providers. The package-local provider constructor deliberately accepts only static models and one protocol implementation; adding dynamic models, filtering, or multi-protocol custom routes requires extending that local function together with its tests.
@@ -0,0 +1,25 @@
# Agent Note:收窄 pi-ai 运行时 import
Status: implemented
[English](2026-09-15-narrow-pi-ai-runtime-imports.md) | 中文
## 问题
基础 bundle 会在没有配置路由时挂载 `dsh-llm-pi-ai`,让 Models 设置页能够提供 pi-ai provider。为了 model helper 而 import pi-ai 聚合入口还会求值其导出的 TypeBox namespace,即使所有 Session 都使用 `dsh-llm-deepseek`,每次应用启动也会额外加载数百个模块。
## 决策
`dsh-llm-pi-ai` 不再运行时 import pi-ai 聚合入口。Catalog 与登录元数据继续使用 `providers/all`;协议实现继续使用现有 `api/*.lazy` 入口;overflow 检测使用 `utils/overflow`。包内 `models.ts` 提供适配器所需的三个 model helper。Collection 来自 pi-ai 公开的 `builtinModels()` 实现,并在安装路由 provider 前清空。Provider constructor 实现本适配器传入的静态单协议分支。Reasoning level 选择按照 pi-ai 的升级顺序读取其公开 `Model` 元数据。
聚合入口的 type-only import 会被 TypeScript 擦除,因此予以保留。构建产物的 import profile 会解析 153 个 pi-ai 模块,不包含 TypeBox 模块或 pi-ai 聚合入口。
## 考虑过的替代方案
- **给 pi-ai 增加 `models` export。** 拒绝,因为本包可以直接使用已经公开的 provider、API、utility 与 model metadata interface,无需修改上游 export map。
- **动态 import 聚合入口。** 拒绝,因为休眠适配器不需要其中任何内容;完全排除该入口会直接删除工作,而不是把工作移动到之后的操作。
- **复制 pi-ai 完整的 Models 实现。** 拒绝,因为 `builtinModels()` 已经返回包含其认证与存储行为的上游实现。清空其中的 provider 可以保留该实现,无需维护 fork。
## 后果
应用仍会加载 `providers/all`,因此配置与授权界面保留完整的已安装 provider 目录。构建 adapter snapshot 时会短暂构造并清空内置 provider set,再安装已解析的路由 provider。包内 provider constructor 只接受静态 model 和一个协议实现;若要增加动态 model、filter 或多协议自定义路由,必须同时扩展该本地函数及其测试。
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
README.md: a49281b5ddc63c1fe4ce628857ff6f267dc498cd
README.zh.md: 78aed7db98e217691351f49f3f8ee41e74177e17
README.md: ceb6c75ad22534db7af62e915e1aee1f11d55390
README.zh.md: 3c6697fea5c5c185af29da8b7156f74ca322d339
+2 -1
View File
@@ -130,7 +130,7 @@ This section explains the design behind the adapter; the observable behavior is
### Design philosophy
The adapter is built on immutable snapshots and per-operation resolution. Each operation captures a whole snapshot — the profiles plus a `createModels()` collection holding the `Provider` each route built — before its first `await`, and a configuration change builds a new collection rather than mutating the one in use, so a request that started under one configuration never finishes under another. A route's own credential reference resolves through the harness seam and rides as the request's `apiKey` option, which pi-ai treats as the highest-priority auth override — that is what keeps the fail-loud reference semantics. Everything that override does not cover reaches pi-ai through the collection's own auth: the credential store holds the records a login wrote and a refresh rotates (addressed as `llm-pi-ai/<provider id>`), and the auth context answers the ambient questions a provider asks while resolving. Both are stable across snapshots, so a configuration change rebuilds the collection without forgetting who is signed in.
The adapter is built on immutable snapshots and per-operation resolution. Each operation captures a whole snapshot — the profiles plus a `createModels()` collection holding the `Provider` each route built — before its first `await`, and a configuration change builds a new collection rather than mutating the one in use, so a request that started under one configuration never finishes under another. A route's own credential reference resolves through the harness seam and rides as the request's `apiKey` option, which pi-ai treats as the highest-priority auth override — that is what keeps the fail-loud reference semantics. Everything that override does not cover reaches pi-ai through the collection's own auth: the credential store holds the records a login wrote and a refresh rotates (addressed as `llm-pi-ai/<provider id>`), and the auth context answers the ambient questions a provider asks while resolving. Both are stable across snapshots, so a configuration change rebuilds the collection without forgetting who is signed in. Runtime imports use pi-ai's provider, API, and utility entry points; `src/models.ts` supplies the small model-helper subset this adapter needs without evaluating pi-ai's aggregate entry point.
### Source map
@@ -141,6 +141,7 @@ The adapter is built on immutable snapshots and per-operation resolution. Each o
| [`src/login.ts`](src/login.ts) | Authorization flows for the installed providers that ship a login |
| [`src/config.ts`](src/config.ts) | Profile schema, resolution, and serviceability checks |
| [`src/catalog.ts`](src/catalog.ts) | Installed-catalog integration and drift gates |
| [`src/models.ts`](src/models.ts) | Model collections, static providers, and reasoning levels over narrow pi-ai entry points |
| [`src/provider.ts`](src/provider.ts) | The supported-protocol table and provider construction |
| [`src/context.ts`](src/context.ts) | Harness-to-pi-ai context conversion, image handling, replay restore |
| [`src/stream.ts`](src/stream.ts) | pi-ai event conversion into harness `StreamChunk` values |
+2 -1
View File
@@ -130,7 +130,7 @@ Settings 写入会在合并组合层与用户层后严格校验每个新增或
### 设计理念
适配器建立在不可变快照与按操作解析之上。每个操作都会在第一次 `await` 前捕获整个快照——profile 加一个持有每条路由所构建 `Provider` 的 `createModels()` 集合——配置变更会构建新集合而非修改使用中的集合,因此在一个配置下开始的请求绝不会在另一个配置下结束。路由自己的凭据引用经 harness seam 解析,并以请求 `apiKey` 选项传入,pi-ai 将其视为优先级最高的 auth 覆盖——这正是明确失败引用语义的所在。该覆盖未覆盖的一切都经集合自身的 auth 到达 pi-ai:凭据存储持有登录写入、刷新轮换的记录(以 `llm-pi-ai/<provider id>` 寻址),auth context 回答提供方解析时提出的 ambient 问题。两者跨快照保持稳定,因此配置变更重建集合时不会忘记谁已登录。
适配器建立在不可变快照与按操作解析之上。每个操作都会在第一次 `await` 前捕获整个快照——profile 加一个持有每条路由所构建 `Provider` 的 `createModels()` 集合——配置变更会构建新集合而非修改使用中的集合,因此在一个配置下开始的请求绝不会在另一个配置下结束。路由自己的凭据引用经 harness seam 解析,并以请求 `apiKey` 选项传入,pi-ai 将其视为优先级最高的 auth 覆盖——这正是明确失败引用语义的所在。该覆盖未覆盖的一切都经集合自身的 auth 到达 pi-ai:凭据存储持有登录写入、刷新轮换的记录(以 `llm-pi-ai/<provider id>` 寻址),auth context 回答提供方解析时提出的 ambient 问题。两者跨快照保持稳定,因此配置变更重建集合时不会忘记谁已登录。运行时 import 使用 pi-ai 的 provider、API 与 utility 入口;`src/models.ts` 提供本适配器所需的少量 model helper,而不会求值 pi-ai 聚合入口。
### 源码地图
@@ -141,6 +141,7 @@ Settings 写入会在合并组合层与用户层后严格校验每个新增或
| [`src/login.ts`](src/login.ts) | 面向提供登录的已安装提供方的授权流程 |
| [`src/config.ts`](src/config.ts) | Profile schema、解析与可服务性校验 |
| [`src/catalog.ts`](src/catalog.ts) | 已安装目录集成与漂移门禁 |
| [`src/models.ts`](src/models.ts) | 基于 pi-ai 窄入口的 model collection、静态 provider 与 reasoning level |
| [`src/provider.ts`](src/provider.ts) | 受支持协议表与提供方构建 |
| [`src/context.ts`](src/context.ts) | Harness 到 pi-ai 的上下文转换、图片处理、回放恢复 |
| [`src/stream.ts`](src/stream.ts) | 把 pi-ai 事件转换为 harness `StreamChunk` 值 |
+1 -1
View File
@@ -26,7 +26,6 @@
* @module dsh-llm-pi-ai/adapter
*/
import { createModels, getSupportedThinkingLevels } from '@earendil-works/pi-ai'
import type {
Api,
AuthContext,
@@ -60,6 +59,7 @@ import type { AttachmentStore, ImageAttachmentRef } from '@deepseek-ai/dsh-attac
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
import type { ResolvedPiAiProviderProfile } from './config.ts'
import { toPiContext } from './context.ts'
import { createModels, getSupportedThinkingLevels } from './models.ts'
import { toStreamChunks } from './stream.ts'
/** One resolution's frozen view: the profiles and the collection built from them. */
+1 -1
View File
@@ -7,7 +7,6 @@
* @module dsh-llm-pi-ai/login
*/
import { createModels } from '@earendil-works/pi-ai'
import type { AuthEvent, AuthPrompt, AuthType, Provider } from '@earendil-works/pi-ai'
import type { Context } from '@deepseek-ai/cordis'
import type { AuthorizationMethod, AuthorizationPrompt, AuthorizationSession } from '@deepseek-ai/dsh-authorization'
@@ -15,6 +14,7 @@ import { isCredentialKeySegment } from '@deepseek-ai/dsh-credentials'
import { catalogProvider, catalogProviderIds } from './catalog.ts'
import { recordKeyFor } from './auth.ts'
import type { PiAiAuthInjection } from './adapter.ts'
import { createModels } from './models.ts'
/**
* The login methods one catalog provider offers.
+67
View File
@@ -0,0 +1,67 @@
/** pi-ai model helpers assembled from public narrow entry points. */
import { builtinModels } from '@earendil-works/pi-ai/providers/all'
import type {
Api,
CreateModelsOptions,
Model,
ModelThinkingLevel,
MutableModels,
Provider,
ProviderAuth,
ProviderStreams,
} from '@earendil-works/pi-ai'
import { THINKING_LEVELS } from './catalog.ts'
/** Input accepted by the static, single-protocol providers this package builds. */
interface StaticProviderOptions {
id: string
name: string
baseUrl?: string
auth: ProviderAuth
models: readonly Model<Api>[]
api: ProviderStreams
}
/**
* Create an empty pi-ai collection without importing its aggregate entry point.
* @param options - credential storage and ambient authentication integrations.
* @returns a mutable collection with no registered providers.
*/
export function createModels(options?: CreateModelsOptions): MutableModels {
const models = builtinModels(options)
models.clearProviders()
return models
}
/**
* Create the static, single-protocol provider used by configured custom routes.
* @param input - provider identity, models, authentication, and protocol implementation.
* @returns a provider that delegates each operation to the supplied protocol.
*/
export function createProvider(input: StaticProviderOptions): Provider {
return {
id: input.id,
name: input.name,
...input.baseUrl === undefined ? {} : { baseUrl: input.baseUrl },
auth: input.auth,
getModels: () => input.models,
stream: (model, context, options) => input.api.stream(model, context, options),
streamSimple: (model, context, options) => input.api.streamSimple(model, context, options),
}
}
/**
* Resolve selectable reasoning levels from pi-ai's public model metadata.
* @param model - model descriptor carrying reasoning support and wire mappings.
* @returns supported levels in pi-ai's escalation order.
*/
export function getSupportedThinkingLevels(model: Model<Api>): ModelThinkingLevel[] {
if (!model.reasoning) return ['off']
return THINKING_LEVELS.filter((level) => {
const mapped = model.thinkingLevelMap?.[level]
if (mapped === null) return false
if (level === 'xhigh' || level === 'max') return mapped !== undefined
return true
})
}
+1 -1
View File
@@ -19,12 +19,12 @@
* @module dsh-llm-pi-ai/provider
*/
import { createProvider } from '@earendil-works/pi-ai'
import type { Api, ApiKeyAuth, Model, Provider, ProviderStreams } from '@earendil-works/pi-ai'
import { anthropicMessagesApi } from '@earendil-works/pi-ai/api/anthropic-messages.lazy'
import { openAICompletionsApi } from '@earendil-works/pi-ai/api/openai-completions.lazy'
import { openAIResponsesApi } from '@earendil-works/pi-ai/api/openai-responses.lazy'
import { catalogProvider, PiAiCatalogError } from './catalog.ts'
import { createProvider } from './models.ts'
/**
* Wire protocols a configured route may name, mapped to pi-ai's lazily loaded
+1 -1
View File
@@ -11,7 +11,7 @@
import { brandString } from '@deepseek-ai/dsh-brand'
import { CONTEXT_WINDOW_EXCEEDED_CODE, EMPTY_RESPONSE_CODE, isContextWindowExceededError, isQuotaExceededError, LlmError, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
import type { FinishReason, StreamChunk, TokenUsage, ToolCallId } from '@deepseek-ai/dsh-llm'
import { isContextOverflow } from '@earendil-works/pi-ai'
import { isContextOverflow } from '@earendil-works/pi-ai/utils/overflow'
import type { AssistantMessage, AssistantMessageEvent, Usage as PiUsage } from '@earendil-works/pi-ai'
import { toPiReplayState } from './replay.ts'
+24 -1
View File
@@ -9,9 +9,10 @@ import FileSettingsProvider from '@deepseek-ai/dsh-settings-file'
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all'
import { createModels, getSupportedThinkingLevels } from '@earendil-works/pi-ai'
import { AssistantMessageEventStream } from '@earendil-works/pi-ai/utils/event-stream'
import type { Api, Model, OpenAICompletionsCompat, Provider } from '@earendil-works/pi-ai'
import { resolveProfiles } from '../src/config.ts'
import { createModels, createProvider, getSupportedThinkingLevels } from '../src/models.ts'
import { buildProvider, supportedProtocols } from '../src/provider.ts'
import { assemble } from './assemble.ts'
import { memoryAuth } from './auth-double.ts'
@@ -355,6 +356,28 @@ describe('hand-declared providers', () => {
expect(() => buildProvider(spec)).toThrow(/cannot serve; supported protocols are/)
})
it('delegates both stream methods from a static provider', () => {
const [model] = getBuiltinModels('deepseek')
if (model === undefined) throw new Error('the installed catalog ships no deepseek model')
const direct = new AssistantMessageEventStream()
const simple = new AssistantMessageEventStream()
const stream = vi.fn(() => direct)
const streamSimple = vi.fn(() => simple)
const provider = createProvider({
id: 'local',
name: 'Local',
models: [model],
auth: { apiKey: { name: 'Local', resolve: () => Promise.resolve({ auth: {}, source: 'Local' }) } },
api: { stream, streamSimple },
})
const context = { messages: [] }
expect(provider.stream(model, context)).toBe(direct)
expect(provider.streamSimple(model, context)).toBe(simple)
expect(stream).toHaveBeenCalledOnce()
expect(streamSimple).toHaveBeenCalledOnce()
})
it('leaves an unauthenticated route to its protocol rather than inventing a credential', async () => {
const server = await mockServer([{ events: textEvents }])
// Naming no credential is the deliberately unauthenticated posture — a
+2 -2
View File
@@ -14,8 +14,8 @@ const login = vi.hoisted(() => vi.fn())
// The whole of what this module does with pi-ai is run one provider's login
// against a collection built with the harness store, so the collection is the
// boundary worth observing; a real login would open a browser.
vi.mock('@earendil-works/pi-ai', async importOriginal => ({
...await importOriginal<typeof import('@earendil-works/pi-ai')>(),
vi.mock('../src/models.ts', async importOriginal => ({
...await importOriginal<typeof import('../src/models.ts')>(),
createModels: () => ({ setProvider: () => {}, login }),
}))