perf(llm): avoid aggregate pi-ai runtime import
This commit is contained in:
1 parent
ee9f43bc5d
commit
75a56be10c
13 files changed
+159
-11
No files matched your search
+6
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-15-narrow-pi-ai-runtime-imports.md
|
||||
2026-09-15-narrow-pi-ai-runtime-imports.md: 307d880f3fce22b0e1b96efb9c3a220a511eff15
|
||||
2026-09-15-narrow-pi-ai-runtime-imports.zh.md: 139983446a2f0cd5315629c4e0a632f8c125a837
|
||||
@@ -0,0 +1,25 @@
|
||||
# Agent Note: Narrow pi-ai runtime imports
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-09-15-narrow-pi-ai-runtime-imports.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The base bundle mounts `dsh-llm-pi-ai` with no configured routes so the Models settings page can offer pi-ai providers. Importing pi-ai's aggregate entry point for model helpers also evaluates its exported TypeBox namespace, adding hundreds of modules to every application startup even when all Sessions use `dsh-llm-deepseek`.
|
||||
|
||||
## Decision
|
||||
|
||||
`dsh-llm-pi-ai` has no runtime import of pi-ai's aggregate entry point. Catalog and login metadata continue through `providers/all`; protocol implementations use their existing `api/*.lazy` entries; overflow detection uses `utils/overflow`. Package-local `models.ts` supplies the three model helpers the adapter needs. Its collection comes from pi-ai's public `builtinModels()` implementation and is cleared before route providers are installed. Its provider constructor implements the static single-protocol case this adapter supplies. Its reasoning-level selection reads pi-ai's public `Model` metadata in pi-ai's escalation order.
|
||||
|
||||
Type-only imports from the aggregate entry point remain because TypeScript erases them. Import profiling of the built package resolves 153 pi-ai modules and no TypeBox modules or pi-ai aggregate entry.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Add a pi-ai `models` export.** Rejected because this package does not need an upstream export-map change to consume the public provider, API, utility, and model metadata interfaces already available.
|
||||
- **Dynamically import the aggregate entry point.** Rejected because the dormant adapter needs none of it; excluding the entry entirely removes the work instead of moving it to a later operation.
|
||||
- **Copy pi-ai's complete Models implementation.** Rejected because `builtinModels()` already returns the upstream implementation with its authentication and storage behavior. Clearing its providers preserves that implementation without maintaining a fork.
|
||||
|
||||
## Consequences
|
||||
|
||||
Applications still load `providers/all` so configuration and authorization surfaces retain the complete installed provider directory. Constructing an adapter snapshot briefly constructs and then clears the built-in provider set before installing the resolved route providers. The package-local provider constructor deliberately accepts only static models and one protocol implementation; adding dynamic models, filtering, or multi-protocol custom routes requires extending that local function together with its tests.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Agent Note:收窄 pi-ai 运行时 import
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-09-15-narrow-pi-ai-runtime-imports.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
基础 bundle 会在没有配置路由时挂载 `dsh-llm-pi-ai`,让 Models 设置页能够提供 pi-ai provider。为了 model helper 而 import pi-ai 聚合入口还会求值其导出的 TypeBox namespace,即使所有 Session 都使用 `dsh-llm-deepseek`,每次应用启动也会额外加载数百个模块。
|
||||
|
||||
## 决策
|
||||
|
||||
`dsh-llm-pi-ai` 不再运行时 import pi-ai 聚合入口。Catalog 与登录元数据继续使用 `providers/all`;协议实现继续使用现有 `api/*.lazy` 入口;overflow 检测使用 `utils/overflow`。包内 `models.ts` 提供适配器所需的三个 model helper。Collection 来自 pi-ai 公开的 `builtinModels()` 实现,并在安装路由 provider 前清空。Provider constructor 实现本适配器传入的静态单协议分支。Reasoning level 选择按照 pi-ai 的升级顺序读取其公开 `Model` 元数据。
|
||||
|
||||
聚合入口的 type-only import 会被 TypeScript 擦除,因此予以保留。构建产物的 import profile 会解析 153 个 pi-ai 模块,不包含 TypeBox 模块或 pi-ai 聚合入口。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **给 pi-ai 增加 `models` export。** 拒绝,因为本包可以直接使用已经公开的 provider、API、utility 与 model metadata interface,无需修改上游 export map。
|
||||
- **动态 import 聚合入口。** 拒绝,因为休眠适配器不需要其中任何内容;完全排除该入口会直接删除工作,而不是把工作移动到之后的操作。
|
||||
- **复制 pi-ai 完整的 Models 实现。** 拒绝,因为 `builtinModels()` 已经返回包含其认证与存储行为的上游实现。清空其中的 provider 可以保留该实现,无需维护 fork。
|
||||
|
||||
## 后果
|
||||
|
||||
应用仍会加载 `providers/all`,因此配置与授权界面保留完整的已安装 provider 目录。构建 adapter snapshot 时会短暂构造并清空内置 provider set,再安装已解析的路由 provider。包内 provider constructor 只接受静态 model 和一个协议实现;若要增加动态 model、filter 或多协议自定义路由,必须同时扩展该本地函数及其测试。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: a49281b5ddc63c1fe4ce628857ff6f267dc498cd
|
||||
README.zh.md: 78aed7db98e217691351f49f3f8ee41e74177e17
|
||||
README.md: ceb6c75ad22534db7af62e915e1aee1f11d55390
|
||||
README.zh.md: 3c6697fea5c5c185af29da8b7156f74ca322d339
|
||||
@@ -130,7 +130,7 @@ This section explains the design behind the adapter; the observable behavior is
|
||||
|
||||
### Design philosophy
|
||||
|
||||
The adapter is built on immutable snapshots and per-operation resolution. Each operation captures a whole snapshot — the profiles plus a `createModels()` collection holding the `Provider` each route built — before its first `await`, and a configuration change builds a new collection rather than mutating the one in use, so a request that started under one configuration never finishes under another. A route's own credential reference resolves through the harness seam and rides as the request's `apiKey` option, which pi-ai treats as the highest-priority auth override — that is what keeps the fail-loud reference semantics. Everything that override does not cover reaches pi-ai through the collection's own auth: the credential store holds the records a login wrote and a refresh rotates (addressed as `llm-pi-ai/<provider id>`), and the auth context answers the ambient questions a provider asks while resolving. Both are stable across snapshots, so a configuration change rebuilds the collection without forgetting who is signed in.
|
||||
The adapter is built on immutable snapshots and per-operation resolution. Each operation captures a whole snapshot — the profiles plus a `createModels()` collection holding the `Provider` each route built — before its first `await`, and a configuration change builds a new collection rather than mutating the one in use, so a request that started under one configuration never finishes under another. A route's own credential reference resolves through the harness seam and rides as the request's `apiKey` option, which pi-ai treats as the highest-priority auth override — that is what keeps the fail-loud reference semantics. Everything that override does not cover reaches pi-ai through the collection's own auth: the credential store holds the records a login wrote and a refresh rotates (addressed as `llm-pi-ai/<provider id>`), and the auth context answers the ambient questions a provider asks while resolving. Both are stable across snapshots, so a configuration change rebuilds the collection without forgetting who is signed in. Runtime imports use pi-ai's provider, API, and utility entry points; `src/models.ts` supplies the small model-helper subset this adapter needs without evaluating pi-ai's aggregate entry point.
|
||||
|
||||
### Source map
|
||||
|
||||
@@ -141,6 +141,7 @@ The adapter is built on immutable snapshots and per-operation resolution. Each o
|
||||
| [`src/login.ts`](src/login.ts) | Authorization flows for the installed providers that ship a login |
|
||||
| [`src/config.ts`](src/config.ts) | Profile schema, resolution, and serviceability checks |
|
||||
| [`src/catalog.ts`](src/catalog.ts) | Installed-catalog integration and drift gates |
|
||||
| [`src/models.ts`](src/models.ts) | Model collections, static providers, and reasoning levels over narrow pi-ai entry points |
|
||||
| [`src/provider.ts`](src/provider.ts) | The supported-protocol table and provider construction |
|
||||
| [`src/context.ts`](src/context.ts) | Harness-to-pi-ai context conversion, image handling, replay restore |
|
||||
| [`src/stream.ts`](src/stream.ts) | pi-ai event conversion into harness `StreamChunk` values |
|
||||
|
||||
@@ -130,7 +130,7 @@ Settings 写入会在合并组合层与用户层后严格校验每个新增或
|
||||
|
||||
### 设计理念
|
||||
|
||||
适配器建立在不可变快照与按操作解析之上。每个操作都会在第一次 `await` 前捕获整个快照——profile 加一个持有每条路由所构建 `Provider` 的 `createModels()` 集合——配置变更会构建新集合而非修改使用中的集合,因此在一个配置下开始的请求绝不会在另一个配置下结束。路由自己的凭据引用经 harness seam 解析,并以请求 `apiKey` 选项传入,pi-ai 将其视为优先级最高的 auth 覆盖——这正是明确失败引用语义的所在。该覆盖未覆盖的一切都经集合自身的 auth 到达 pi-ai:凭据存储持有登录写入、刷新轮换的记录(以 `llm-pi-ai/<provider id>` 寻址),auth context 回答提供方解析时提出的 ambient 问题。两者跨快照保持稳定,因此配置变更重建集合时不会忘记谁已登录。
|
||||
适配器建立在不可变快照与按操作解析之上。每个操作都会在第一次 `await` 前捕获整个快照——profile 加一个持有每条路由所构建 `Provider` 的 `createModels()` 集合——配置变更会构建新集合而非修改使用中的集合,因此在一个配置下开始的请求绝不会在另一个配置下结束。路由自己的凭据引用经 harness seam 解析,并以请求 `apiKey` 选项传入,pi-ai 将其视为优先级最高的 auth 覆盖——这正是明确失败引用语义的所在。该覆盖未覆盖的一切都经集合自身的 auth 到达 pi-ai:凭据存储持有登录写入、刷新轮换的记录(以 `llm-pi-ai/<provider id>` 寻址),auth context 回答提供方解析时提出的 ambient 问题。两者跨快照保持稳定,因此配置变更重建集合时不会忘记谁已登录。运行时 import 使用 pi-ai 的 provider、API 与 utility 入口;`src/models.ts` 提供本适配器所需的少量 model helper,而不会求值 pi-ai 聚合入口。
|
||||
|
||||
### 源码地图
|
||||
|
||||
@@ -141,6 +141,7 @@ Settings 写入会在合并组合层与用户层后严格校验每个新增或
|
||||
| [`src/login.ts`](src/login.ts) | 面向提供登录的已安装提供方的授权流程 |
|
||||
| [`src/config.ts`](src/config.ts) | Profile schema、解析与可服务性校验 |
|
||||
| [`src/catalog.ts`](src/catalog.ts) | 已安装目录集成与漂移门禁 |
|
||||
| [`src/models.ts`](src/models.ts) | 基于 pi-ai 窄入口的 model collection、静态 provider 与 reasoning level |
|
||||
| [`src/provider.ts`](src/provider.ts) | 受支持协议表与提供方构建 |
|
||||
| [`src/context.ts`](src/context.ts) | Harness 到 pi-ai 的上下文转换、图片处理、回放恢复 |
|
||||
| [`src/stream.ts`](src/stream.ts) | 把 pi-ai 事件转换为 harness `StreamChunk` 值 |
|
||||
|
||||
@@ -26,7 +26,6 @@
|
||||
* @module dsh-llm-pi-ai/adapter
|
||||
*/
|
||||
|
||||
import { createModels, getSupportedThinkingLevels } from '@earendil-works/pi-ai'
|
||||
import type {
|
||||
Api,
|
||||
AuthContext,
|
||||
@@ -60,6 +59,7 @@ import type { AttachmentStore, ImageAttachmentRef } from '@deepseek-ai/dsh-attac
|
||||
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
import type { ResolvedPiAiProviderProfile } from './config.ts'
|
||||
import { toPiContext } from './context.ts'
|
||||
import { createModels, getSupportedThinkingLevels } from './models.ts'
|
||||
import { toStreamChunks } from './stream.ts'
|
||||
|
||||
/** One resolution's frozen view: the profiles and the collection built from them. */
|
||||
|
||||
@@ -7,7 +7,6 @@
|
||||
* @module dsh-llm-pi-ai/login
|
||||
*/
|
||||
|
||||
import { createModels } from '@earendil-works/pi-ai'
|
||||
import type { AuthEvent, AuthPrompt, AuthType, Provider } from '@earendil-works/pi-ai'
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import type { AuthorizationMethod, AuthorizationPrompt, AuthorizationSession } from '@deepseek-ai/dsh-authorization'
|
||||
@@ -15,6 +14,7 @@ import { isCredentialKeySegment } from '@deepseek-ai/dsh-credentials'
|
||||
import { catalogProvider, catalogProviderIds } from './catalog.ts'
|
||||
import { recordKeyFor } from './auth.ts'
|
||||
import type { PiAiAuthInjection } from './adapter.ts'
|
||||
import { createModels } from './models.ts'
|
||||
|
||||
/**
|
||||
* The login methods one catalog provider offers.
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
/** pi-ai model helpers assembled from public narrow entry points. */
|
||||
|
||||
import { builtinModels } from '@earendil-works/pi-ai/providers/all'
|
||||
import type {
|
||||
Api,
|
||||
CreateModelsOptions,
|
||||
Model,
|
||||
ModelThinkingLevel,
|
||||
MutableModels,
|
||||
Provider,
|
||||
ProviderAuth,
|
||||
ProviderStreams,
|
||||
} from '@earendil-works/pi-ai'
|
||||
import { THINKING_LEVELS } from './catalog.ts'
|
||||
|
||||
/** Input accepted by the static, single-protocol providers this package builds. */
|
||||
interface StaticProviderOptions {
|
||||
id: string
|
||||
name: string
|
||||
baseUrl?: string
|
||||
auth: ProviderAuth
|
||||
models: readonly Model<Api>[]
|
||||
api: ProviderStreams
|
||||
}
|
||||
|
||||
/**
|
||||
* Create an empty pi-ai collection without importing its aggregate entry point.
|
||||
* @param options - credential storage and ambient authentication integrations.
|
||||
* @returns a mutable collection with no registered providers.
|
||||
*/
|
||||
export function createModels(options?: CreateModelsOptions): MutableModels {
|
||||
const models = builtinModels(options)
|
||||
models.clearProviders()
|
||||
return models
|
||||
}
|
||||
|
||||
/**
|
||||
* Create the static, single-protocol provider used by configured custom routes.
|
||||
* @param input - provider identity, models, authentication, and protocol implementation.
|
||||
* @returns a provider that delegates each operation to the supplied protocol.
|
||||
*/
|
||||
export function createProvider(input: StaticProviderOptions): Provider {
|
||||
return {
|
||||
id: input.id,
|
||||
name: input.name,
|
||||
...input.baseUrl === undefined ? {} : { baseUrl: input.baseUrl },
|
||||
auth: input.auth,
|
||||
getModels: () => input.models,
|
||||
stream: (model, context, options) => input.api.stream(model, context, options),
|
||||
streamSimple: (model, context, options) => input.api.streamSimple(model, context, options),
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve selectable reasoning levels from pi-ai's public model metadata.
|
||||
* @param model - model descriptor carrying reasoning support and wire mappings.
|
||||
* @returns supported levels in pi-ai's escalation order.
|
||||
*/
|
||||
export function getSupportedThinkingLevels(model: Model<Api>): ModelThinkingLevel[] {
|
||||
if (!model.reasoning) return ['off']
|
||||
return THINKING_LEVELS.filter((level) => {
|
||||
const mapped = model.thinkingLevelMap?.[level]
|
||||
if (mapped === null) return false
|
||||
if (level === 'xhigh' || level === 'max') return mapped !== undefined
|
||||
return true
|
||||
})
|
||||
}
|
||||
@@ -19,12 +19,12 @@
|
||||
* @module dsh-llm-pi-ai/provider
|
||||
*/
|
||||
|
||||
import { createProvider } from '@earendil-works/pi-ai'
|
||||
import type { Api, ApiKeyAuth, Model, Provider, ProviderStreams } from '@earendil-works/pi-ai'
|
||||
import { anthropicMessagesApi } from '@earendil-works/pi-ai/api/anthropic-messages.lazy'
|
||||
import { openAICompletionsApi } from '@earendil-works/pi-ai/api/openai-completions.lazy'
|
||||
import { openAIResponsesApi } from '@earendil-works/pi-ai/api/openai-responses.lazy'
|
||||
import { catalogProvider, PiAiCatalogError } from './catalog.ts'
|
||||
import { createProvider } from './models.ts'
|
||||
|
||||
/**
|
||||
* Wire protocols a configured route may name, mapped to pi-ai's lazily loaded
|
||||
|
||||
@@ -11,7 +11,7 @@
|
||||
import { brandString } from '@deepseek-ai/dsh-brand'
|
||||
import { CONTEXT_WINDOW_EXCEEDED_CODE, EMPTY_RESPONSE_CODE, isContextWindowExceededError, isQuotaExceededError, LlmError, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
|
||||
import type { FinishReason, StreamChunk, TokenUsage, ToolCallId } from '@deepseek-ai/dsh-llm'
|
||||
import { isContextOverflow } from '@earendil-works/pi-ai'
|
||||
import { isContextOverflow } from '@earendil-works/pi-ai/utils/overflow'
|
||||
import type { AssistantMessage, AssistantMessageEvent, Usage as PiUsage } from '@earendil-works/pi-ai'
|
||||
import { toPiReplayState } from './replay.ts'
|
||||
|
||||
|
||||
@@ -9,9 +9,10 @@ import FileSettingsProvider from '@deepseek-ai/dsh-settings-file'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all'
|
||||
import { createModels, getSupportedThinkingLevels } from '@earendil-works/pi-ai'
|
||||
import { AssistantMessageEventStream } from '@earendil-works/pi-ai/utils/event-stream'
|
||||
import type { Api, Model, OpenAICompletionsCompat, Provider } from '@earendil-works/pi-ai'
|
||||
import { resolveProfiles } from '../src/config.ts'
|
||||
import { createModels, createProvider, getSupportedThinkingLevels } from '../src/models.ts'
|
||||
import { buildProvider, supportedProtocols } from '../src/provider.ts'
|
||||
import { assemble } from './assemble.ts'
|
||||
import { memoryAuth } from './auth-double.ts'
|
||||
@@ -355,6 +356,28 @@ describe('hand-declared providers', () => {
|
||||
expect(() => buildProvider(spec)).toThrow(/cannot serve; supported protocols are/)
|
||||
})
|
||||
|
||||
it('delegates both stream methods from a static provider', () => {
|
||||
const [model] = getBuiltinModels('deepseek')
|
||||
if (model === undefined) throw new Error('the installed catalog ships no deepseek model')
|
||||
const direct = new AssistantMessageEventStream()
|
||||
const simple = new AssistantMessageEventStream()
|
||||
const stream = vi.fn(() => direct)
|
||||
const streamSimple = vi.fn(() => simple)
|
||||
const provider = createProvider({
|
||||
id: 'local',
|
||||
name: 'Local',
|
||||
models: [model],
|
||||
auth: { apiKey: { name: 'Local', resolve: () => Promise.resolve({ auth: {}, source: 'Local' }) } },
|
||||
api: { stream, streamSimple },
|
||||
})
|
||||
const context = { messages: [] }
|
||||
|
||||
expect(provider.stream(model, context)).toBe(direct)
|
||||
expect(provider.streamSimple(model, context)).toBe(simple)
|
||||
expect(stream).toHaveBeenCalledOnce()
|
||||
expect(streamSimple).toHaveBeenCalledOnce()
|
||||
})
|
||||
|
||||
it('leaves an unauthenticated route to its protocol rather than inventing a credential', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
// Naming no credential is the deliberately unauthenticated posture — a
|
||||
|
||||
@@ -14,8 +14,8 @@ const login = vi.hoisted(() => vi.fn())
|
||||
// The whole of what this module does with pi-ai is run one provider's login
|
||||
// against a collection built with the harness store, so the collection is the
|
||||
// boundary worth observing; a real login would open a browser.
|
||||
vi.mock('@earendil-works/pi-ai', async importOriginal => ({
|
||||
...await importOriginal<typeof import('@earendil-works/pi-ai')>(),
|
||||
vi.mock('../src/models.ts', async importOriginal => ({
|
||||
...await importOriginal<typeof import('../src/models.ts')>(),
|
||||
createModels: () => ({ setProvider: () => {}, login }),
|
||||
}))
|
||||
|
||||
|
||||
Reference in new issue
Block a user