refactor(llm): request minimal reasoning without naming a caller purpose
`purpose` is a provider-neutral label for replay and transport metadata, so an adapter must not branch on one caller's value. `GenerateOptions` gains `minimizeReasoning`, which any auxiliary caller sets when reasoning would consume the output budget it reserved for a short visible result; both shipped adapters dispatch such a request at the least reasoning the model supports, and `session-title-llm` sets it instead of relying on the adapter recognizing `'session-title'`. The pasted type in the subsystem page and the generated Cordis API catalog follow. Prompted by review: llm-pi-ai must not know about session-title.
This commit is contained in:
1 parent
8a199f81eb
commit
32a561f3a6
19 files changed
+67
-44
No files matched your search
@@ -634,12 +634,19 @@ interface GenerateOptions {
|
||||
sessionId?: Branded<'SessionId'>
|
||||
/**
|
||||
* Provider-neutral classification for an auxiliary model call. Adapters may
|
||||
* map the purpose to model-hidden transport metadata or purpose-specific
|
||||
* generation policy. A `session-title` call caps output for one short visible
|
||||
* title, so adapters dispatch it with the least reasoning the model supports.
|
||||
* Ordinary conversation requests leave it unset.
|
||||
* map the purpose to model-hidden transport metadata. Ordinary conversation
|
||||
* requests leave it unset.
|
||||
*/
|
||||
purpose?: 'compaction' | 'session-title'
|
||||
/**
|
||||
* Dispatch with the least reasoning the routed model supports: `off` when
|
||||
* the model offers it, otherwise its lowest declared level. An auxiliary
|
||||
* caller sets this when reasoning would consume the output budget it
|
||||
* reserved for a short visible result. The request's own effort and the
|
||||
* profile default do not apply, and an effort the model does not support is
|
||||
* not validated for such a request.
|
||||
*/
|
||||
minimizeReasoning?: boolean
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
@@ -640,12 +640,19 @@ interface GenerateOptions {
|
||||
sessionId?: Branded<'SessionId'>
|
||||
/**
|
||||
* Provider-neutral classification for an auxiliary model call. Adapters may
|
||||
* map the purpose to model-hidden transport metadata or purpose-specific
|
||||
* generation policy. A `session-title` call caps output for one short visible
|
||||
* title, so adapters dispatch it with the least reasoning the model supports.
|
||||
* Ordinary conversation requests leave it unset.
|
||||
* map the purpose to model-hidden transport metadata. Ordinary conversation
|
||||
* requests leave it unset.
|
||||
*/
|
||||
purpose?: 'compaction' | 'session-title'
|
||||
/**
|
||||
* Dispatch with the least reasoning the routed model supports: `off` when
|
||||
* the model offers it, otherwise its lowest declared level. An auxiliary
|
||||
* caller sets this when reasoning would consume the output budget it
|
||||
* reserved for a short visible result. The request's own effort and the
|
||||
* profile default do not apply, and an effort the model does not support is
|
||||
* not validated for such a request.
|
||||
*/
|
||||
minimizeReasoning?: boolean
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
@@ -5389,7 +5389,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'GenerateOptions',
|
||||
declaration: 'export interface GenerateOptions {\n provider: string;\n model: string;\n reasoningEffort?: ReasoningEffortId;\n messages: RequestMessage[];\n system?: string;\n tools?: ToolSchema[];\n toolHistory?: ToolHistory;\n temperature?: number;\n maxTokens?: number;\n stop?: string[];\n signal?: AbortSignal;\n sessionId?: Branded<\'SessionId\'>;\n purpose?: \'compaction\' | \'session-title\';\n}',
|
||||
declaration: 'export interface GenerateOptions {\n provider: string;\n model: string;\n reasoningEffort?: ReasoningEffortId;\n messages: RequestMessage[];\n system?: string;\n tools?: ToolSchema[];\n toolHistory?: ToolHistory;\n temperature?: number;\n maxTokens?: number;\n stop?: string[];\n signal?: AbortSignal;\n sessionId?: Branded<\'SessionId\'>;\n purpose?: \'compaction\' | \'session-title\';\n minimizeReasoning?: boolean;\n}',
|
||||
},
|
||||
{
|
||||
name: 'GenericCallView',
|
||||
|
||||
@@ -30,8 +30,8 @@
|
||||
en: 3853ba1a1ffbb46c
|
||||
zh: 97203e80ad933e59
|
||||
/deepseek-ai-dsh-llm-deepseek/use-this-package/streaming-with-thinking-and-images:
|
||||
en: 9b82549c6ce25261
|
||||
zh: 22a3978c5022576b
|
||||
en: 1d7ba1bedfd11a18
|
||||
zh: 9e1a3372f610e9aa
|
||||
/deepseek-ai-dsh-llm-deepseek/use-this-package/dynamic-configuration:
|
||||
en: 49290fc5647136b4
|
||||
zh: 96fa9ce0f9e83c13
|
||||
|
||||
@@ -99,7 +99,7 @@ Cached ids are scoped by endpoint and credential, refreshed before expiry, inval
|
||||
|
||||
Files mode bounds retained request versions by `maxRequestFilesBytes` and `maxImagesPerRequest`; inline fallback has its own base64 budget. Both remove an oldest prefix in configured byte or count quanta. Each omitted image gets its own model-visible placeholder with its display name or attachment id and, when available, normalized dimensions, media type, and current read-only path. The stepped high-watermark policy avoids rewriting an old request prefix after every new image.
|
||||
|
||||
`reasoningEffort` selects the advertised default. Exact-model metadata exposes ordered `off`, `low`, `high`, and `max` efforts with selection guidance when deployment policy permits thinking. `low`, `high`, and `max` enable thinking and serialize as `output_config.effort`, while adapter-owned `off` sends `thinking.type: disabled` instead. An unsupported value fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O, and `thinking: disabled` rejects any non-`off` effort at plugin load. Requests with `purpose: 'session-title'` force thinking off to reserve output for visible title text. The adapter forwards an explicit `temperature`; DeepSeek accepts it with thinking enabled but ignores its value in that mode.
|
||||
`reasoningEffort` selects the advertised default. Exact-model metadata exposes ordered `off`, `low`, `high`, and `max` efforts with selection guidance when deployment policy permits thinking. `low`, `high`, and `max` enable thinking and serialize as `output_config.effort`, while adapter-owned `off` sends `thinking.type: disabled` instead. An unsupported value fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O, and `thinking: disabled` rejects any non-`off` effort at plugin load. Requests with `GenerateOptions.minimizeReasoning` force thinking off to reserve output for a short visible result. The adapter forwards an explicit `temperature`; DeepSeek accepts it with thinking enabled but ignores its value in that mode.
|
||||
|
||||
### Dynamic configuration
|
||||
|
||||
|
||||
@@ -99,7 +99,7 @@ Messages 和 Files 请求通过 `x-dsh-auth-token` 发送账号 token,不加 B
|
||||
|
||||
Files 模式通过 `maxRequestFilesBytes` 与 `maxImagesPerRequest` 限制保留请求版本;内联回退有独立 base64 预算。两种模式都按配置的字节或数量量子移除最旧前缀。每张省略图片都有自己的模型可见占位符,包含显示名或附件 id,以及可用时的规范化尺寸、媒体类型与当前只读路径。分阶高水位策略避免每新增一张图片都改写旧请求前缀。
|
||||
|
||||
`reasoningEffort` 选择公布的默认值。当部署策略允许 thinking 时,确切模型元数据会按顺序公开 `off`、`low`、`high` 与 `max` 强度及选择指引。`low`、`high` 与 `max` 启用 thinking,并以 `output_config.effort` 序列化,适配器自有的 `off` 则发送 `thinking.type: disabled`。不支持的取值会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败;`thinking: disabled` 会在插件加载时拒绝任何非 `off` 强度。`purpose: 'session-title'` 的请求会强制关闭 thinking,把有界输出留给可见标题文本。适配器转发显式 `temperature`;DeepSeek 在启用 thinking 时接受该参数,但忽略其值。
|
||||
`reasoningEffort` 选择公布的默认值。当部署策略允许 thinking 时,确切模型元数据会按顺序公开 `off`、`low`、`high` 与 `max` 强度及选择指引。`low`、`high` 与 `max` 启用 thinking,并以 `output_config.effort` 序列化,适配器自有的 `off` 则发送 `thinking.type: disabled`。不支持的取值会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败;`thinking: disabled` 会在插件加载时拒绝任何非 `off` 强度。携带 `GenerateOptions.minimizeReasoning` 的请求会强制关闭 thinking,把输出留给简短的可见结果。适配器转发显式 `temperature`;DeepSeek 在启用 thinking 时接受该参数,但忽略其值。
|
||||
|
||||
### 动态配置
|
||||
|
||||
|
||||
@@ -143,7 +143,7 @@ export function serialize(
|
||||
}
|
||||
}
|
||||
if (pending.size > 0) throw new LlmError('DeepSeek Messages history ends with unresolved tools', 'INVALID_REQUEST')
|
||||
const effort = options.purpose === 'session-title' ? 'off' : options.reasoningEffort ?? (connection.defaults.reasoningEffort ?? (connection.defaults.thinking === 'disabled' ? 'off' : 'high'))
|
||||
const effort = options.minimizeReasoning === true ? 'off' : options.reasoningEffort ?? (connection.defaults.reasoningEffort ?? (connection.defaults.thinking === 'disabled' ? 'off' : 'high'))
|
||||
if (!['off', 'low', 'high', 'max'].includes(effort) || (connection.defaults.thinking === 'disabled' && effort !== 'off')) {
|
||||
throw new LlmError(`DeepSeek Messages does not support reasoning effort ${effort}`, 'UNSUPPORTED_REASONING_EFFORT')
|
||||
}
|
||||
|
||||
@@ -285,8 +285,8 @@ describe('Messages request conversion', () => {
|
||||
expect(request.output_config).toEqual(effort === 'off' ? undefined : { effort })
|
||||
})
|
||||
|
||||
it('disables thinking for titles, passes temperature with thinking and refuses unsupported effort', () => {
|
||||
expect(body([user()], { purpose: 'session-title', temperature: 0 })).toMatchObject({ thinking: { type: 'disabled' }, temperature: 0 })
|
||||
it('disables thinking for a minimized-reasoning request, passes temperature with thinking and refuses unsupported effort', () => {
|
||||
expect(body([user()], { minimizeReasoning: true, temperature: 0 })).toMatchObject({ thinking: { type: 'disabled' }, temperature: 0 })
|
||||
expect(body([user()], { temperature: 0 })).toMatchObject({ thinking: { type: 'enabled' }, temperature: 0 })
|
||||
expect(() => body([user()], { reasoningEffort: ReasoningEffortId('medium') })).toThrow(/effort/)
|
||||
const disabled = resolveAdapterOptions({ thinking: 'disabled' })
|
||||
|
||||
@@ -30,8 +30,8 @@
|
||||
en: 376f83206858b601
|
||||
zh: 38bb6993609c96a6
|
||||
/deepseek-ai-dsh-llm-pi-ai/use-this-package/run-with-reasoning-and-wire-compatibility:
|
||||
en: 803f9e75f71fa86e
|
||||
zh: b17d985dd5267163
|
||||
en: 38f5251e412f2445
|
||||
zh: f492ef40cdafc20e
|
||||
/deepseek-ai-dsh-llm-pi-ai/use-this-package/change-configuration-at-runtime:
|
||||
en: 92f554e1718614ca
|
||||
zh: f17ed3ad3e5d28e2
|
||||
|
||||
@@ -102,7 +102,7 @@ A profile's `models` list replaces the route's installed catalog rather than ext
|
||||
|
||||
`reasoningEfforts` declares a model's selectable thinking levels: each key is a level selectors offer, its value the spelling dispatch sends on the wire, so `max: ultra` renames a level for a gateway with its own vocabulary. Omitting the field keeps the installed catalog entry's capability; `false` declares a non-reasoning model. `compat` switches reshape the request for endpoints pi-ai cannot recognize — which role carries the system prompt, which field caps output, how a thinking level travels — configurable per route and per model. A model neither the entry nor the installed catalog sizes takes the route's `defaultContextWindow` and `defaultMaxTokens` fallbacks.
|
||||
|
||||
A request carrying `GenerateOptions.purpose: 'session-title'` dispatches at the lowest level its model supports, whatever effort the request or profile names: `off` when the model offers it, otherwise the lowest declared level. A title is a short delayed one-off, so it does not spend its output ceiling on reasoning. `off` dispatches exactly as a selected `off` does, so a protocol that expresses it by omitting the reasoning field leaves the gateway's default in effect; reasoning tokens at any of these levels count toward the title cap. The requested effort is not validated for this purpose, so a name the model does not support fails the main call rather than the title call.
|
||||
A request carrying `GenerateOptions.minimizeReasoning` dispatches at the lowest level its model supports, whatever effort the request or profile names: `off` when the model offers it, otherwise the lowest declared level. Such a caller reserves its output budget for a short visible result, so it does not spend that budget on reasoning. `off` dispatches exactly as a selected `off` does, so a protocol that expresses it by omitting the reasoning field leaves the gateway's default in effect; reasoning tokens at any of these levels count toward the request's output cap. The requested effort is not validated for such a request, so a name the model does not support fails an ordinary call rather than this one.
|
||||
|
||||
For self-hosted Chat Completions endpoints, `thinkingTokenBudgetField` selects the reasoning-budget parameter, and `vllmPriority` sets an integer scheduler priority when the server enables priority scheduling. Template arguments accept `$var: thinking.budget`. `openai-responses` gateways can set `supportsMaxOutputTokens: false` to omit `max_output_tokens`; Azure and Codex transports ignore this shared compatibility field. These controls are opt-in; catalog-owned Anthropic effort and fallback capabilities are not configurable switches.
|
||||
|
||||
|
||||
@@ -102,7 +102,7 @@ profile 的 `models` 列表会替换而非扩展路由的已安装目录;每
|
||||
|
||||
`reasoningEfforts` 声明模型可选择的 thinking 等级:每个键都是选择器提供的等级,其值是分派时在协议中发送的拼写,因此 `max: ultra` 可以为拥有自有词汇的网关重命名等级。省略该字段时保留已安装目录条目的能力;`false` 声明非推理模型。对于 pi-ai 无法识别的端点,`compat` 开关重塑请求——哪个角色携带系统提示词、哪个字段限制输出、thinking 等级如何传递——可逐路由、逐模型配置。条目与已安装目录都没有尺寸的模型,会采用路由的 `defaultContextWindow` 与 `defaultMaxTokens` 回退值。
|
||||
|
||||
携带 `GenerateOptions.purpose: 'session-title'` 的请求会以其模型支持的最低等级分派,不论请求或 profile 指定何种推理强度:模型提供 `off` 时使用 `off`,否则使用声明的最低等级。标题是延迟敏感的短小一次性请求,不把输出上限花在推理上。`off` 的分派方式与手动选择 `off` 完全相同,因此以省略推理字段表达 `off` 的协议会沿用网关默认行为;这些等级下产生的推理 token 都计入标题上限。该用途不校验所请求的强度,因此模型不支持的名称会让主请求失败,而不是标题请求。
|
||||
携带 `GenerateOptions.minimizeReasoning` 的请求会以其模型支持的最低等级分派,不论请求或 profile 指定何种推理强度:模型提供 `off` 时使用 `off`,否则使用声明的最低等级。这类调用方把输出预算留给简短的可见结果,因此不把该预算花在推理上。`off` 的分派方式与手动选择 `off` 完全相同,因此以省略推理字段表达 `off` 的协议会沿用网关默认行为;这些等级下产生的推理 token 都计入该请求的输出上限。该请求不校验所请求的强度,因此模型不支持的名称会让普通调用失败,而不是该请求。
|
||||
|
||||
对于自托管 Chat Completions 端点,`thinkingTokenBudgetField` 选择推理预算参数,`vllmPriority` 在服务端启用优先级调度时设置整数调度优先级。模板参数接受 `$var: thinking.budget`。`openai-responses` 网关可设置 `supportsMaxOutputTokens: false` 来省略 `max_output_tokens`;Azure 与 Codex 传输会忽略这个共享兼容字段。这些控制均需显式启用;目录拥有的 Anthropic effort 和回退能力不是可配置开关。
|
||||
|
||||
|
||||
@@ -169,26 +169,26 @@ function resolveReasoningLevel(
|
||||
}
|
||||
|
||||
/**
|
||||
* The level one request dispatches with. A `session-title` request takes the
|
||||
* model's lowest supported level, which is `off` whenever the model can stop
|
||||
* reasoning, because reasoning a model cannot switch off still counts toward
|
||||
* the dispatch cap. That request deliberately skips the requested-effort
|
||||
* check, so a name the model does not support fails the main call rather than
|
||||
* the auxiliary one. Every other request validates the requested or profile
|
||||
* effort.
|
||||
* The level one request dispatches with. A request asking to minimize reasoning
|
||||
* takes the model's lowest supported level, which is `off` whenever the model
|
||||
* can stop reasoning, because reasoning a model cannot switch off still counts
|
||||
* toward the output budget the caller reserved. That request deliberately
|
||||
* skips the requested-effort check, so a name the model does not support fails
|
||||
* an ordinary call rather than this one. Every other request validates the
|
||||
* requested or profile effort.
|
||||
* @param model - the resolved model descriptor.
|
||||
* @param options - the request's purpose and requested effort.
|
||||
* @param options - the request's reasoning preference and requested effort.
|
||||
* @param profile - the route profile carrying the default effort.
|
||||
* @returns the level to dispatch, or undefined to send none, which a
|
||||
* `session-title` request gets only for a model whose metadata leaves it no
|
||||
* supported level.
|
||||
* minimized-reasoning request gets only for a model whose metadata leaves it
|
||||
* no supported level.
|
||||
*/
|
||||
function requestReasoningLevel(
|
||||
model: Model<Api>,
|
||||
options: GenerateOptions,
|
||||
profile: ResolvedPiAiProviderProfile,
|
||||
): ModelThinkingLevel | undefined {
|
||||
if (options.purpose === 'session-title') return getSupportedThinkingLevels(model)[0]
|
||||
if (options.minimizeReasoning === true) return getSupportedThinkingLevels(model)[0]
|
||||
return resolveReasoningLevel(model, options.reasoningEffort ?? profile.reasoning)
|
||||
}
|
||||
|
||||
|
||||
@@ -788,7 +788,7 @@ describe('provider profile lifecycle', () => {
|
||||
expect(server.requests[0]).not.toHaveProperty('reasoning_effort')
|
||||
})
|
||||
|
||||
it('dispatches session-title requests at the lowest level each model supports', async () => {
|
||||
it('dispatches a minimized-reasoning request at the lowest level each model supports', async () => {
|
||||
vi.stubEnv('PI_TEST_KEY', 'test-key')
|
||||
const server = await mockServer([
|
||||
{ events: anthropicTextEvents },
|
||||
@@ -813,18 +813,18 @@ describe('provider profile lifecycle', () => {
|
||||
},
|
||||
},
|
||||
})
|
||||
const request = (model: string, purpose?: 'session-title'): Promise<unknown> => assemble(ctx, {
|
||||
const request = (model: string, minimizeReasoning?: true): Promise<unknown> => assemble(ctx, {
|
||||
provider: 'adaptive-gateway',
|
||||
model,
|
||||
reasoningEffort: ReasoningEffortId('max'),
|
||||
maxTokens: 64,
|
||||
messages: [],
|
||||
...purpose === undefined ? {} : { purpose },
|
||||
...minimizeReasoning === undefined ? {} : { minimizeReasoning },
|
||||
})
|
||||
|
||||
await request('always-thinks')
|
||||
await request('always-thinks', 'session-title')
|
||||
await request('may-think', 'session-title')
|
||||
await request('always-thinks', true)
|
||||
await request('may-think', true)
|
||||
|
||||
expect(server.requests[0]).toMatchObject({ thinking: { type: 'adaptive' }, output_config: { effort: 'max' } })
|
||||
expect(server.requests[1]).toMatchObject({
|
||||
|
||||
@@ -546,10 +546,17 @@ export interface GenerateOptions {
|
||||
sessionId?: Branded<'SessionId'>
|
||||
/**
|
||||
* Provider-neutral classification for an auxiliary model call. Adapters may
|
||||
* map the purpose to model-hidden transport metadata or purpose-specific
|
||||
* generation policy. A `session-title` call caps output for one short visible
|
||||
* title, so adapters dispatch it with the least reasoning the model supports.
|
||||
* Ordinary conversation requests leave it unset.
|
||||
* map the purpose to model-hidden transport metadata. Ordinary conversation
|
||||
* requests leave it unset.
|
||||
*/
|
||||
purpose?: 'compaction' | 'session-title'
|
||||
/**
|
||||
* Dispatch with the least reasoning the routed model supports: `off` when
|
||||
* the model offers it, otherwise its lowest declared level. An auxiliary
|
||||
* caller sets this when reasoning would consume the output budget it
|
||||
* reserved for a short visible result. The request's own effort and the
|
||||
* profile default do not apply, and an effort the model does not support is
|
||||
* not validated for such a request.
|
||||
*/
|
||||
minimizeReasoning?: boolean
|
||||
}
|
||||
@@ -36,8 +36,8 @@
|
||||
en: 0fdc928704abdd3a
|
||||
zh: bd4c0a7dd47578cf
|
||||
/deepseek-ai-dsh-session-title-llm/understand-the-implementation/request-flow:
|
||||
en: 9103f1df0e62575f
|
||||
zh: fda4aa05f54f5c69
|
||||
en: d60754b3baf4e947
|
||||
zh: e347c55c9dba1987
|
||||
/deepseek-ai-dsh-session-title-llm/further-exploration:
|
||||
en: 641f31c35c48916b
|
||||
zh: 8e809fbdfef61b61
|
||||
|
||||
@@ -72,7 +72,7 @@ One shared policy so provider plugins cannot drift: config validation, route res
|
||||
|
||||
### Request flow
|
||||
|
||||
A generation validates the config once at registration; each revision frames the selected messages as JSON, measures the framed prompt's UTF-8 bytes against `maxInputBytes`, resolves the route (the explicit pair or the logged `request/header`), appends a log-only `session/title-llm-request` event carrying the exact dispatchable request, then streams through `ctx.llm` under a composed timeout and cancellation deadline. The dispatched envelope carries `purpose: 'session-title'` and deliberately lacks the agent loop's process-local request identity; adapters dispatch that purpose with the least reasoning the model supports, so a title request does not spend its output ceiling on reasoning: the DeepSeek adapter disables thinking, and the pi-ai adapter selects the model's lowest supported level. Output assembles into text blocks only; tool calls, malformed or empty output, and non-stop finish reasons reject, and a later model failure leaves the request record intact.
|
||||
A generation validates the config once at registration; each revision frames the selected messages as JSON, measures the framed prompt's UTF-8 bytes against `maxInputBytes`, resolves the route (the explicit pair or the logged `request/header`), appends a log-only `session/title-llm-request` event carrying the exact dispatchable request, then streams through `ctx.llm` under a composed timeout and cancellation deadline. The dispatched envelope carries `purpose: 'session-title'` for replay labeling and `minimizeReasoning`, and deliberately lacks the agent loop's process-local request identity; adapters dispatch a minimized-reasoning request with the least reasoning the model supports, so a title request does not spend its output ceiling on reasoning: the DeepSeek adapter disables thinking, and the pi-ai adapter selects the model's lowest supported level. Output assembles into text blocks only; tool calls, malformed or empty output, and non-stop finish reasons reject, and a later model failure leaves the request record intact.
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
@@ -72,7 +72,7 @@ kind: "package-library"
|
||||
|
||||
### 请求流程
|
||||
|
||||
生成在注册时校验一次配置;每次修订把选中的消息封装为 JSON,依据 `maxInputBytes` 检查封装提示词的 UTF-8 字节数,解析路由(显式对或已记录 `request/header`),追加一条携带确切可分发请求的仅日志 `session/title-llm-request` 事件,然后在组合的超时与取消截止时间内通过 `ctx.llm` 流式生成。分发的封套携带 `purpose: 'session-title'`,且有意不包含 agent loop 的进程本地请求身份;适配器以模型支持的最少推理分发该用途,使标题请求不把输出上限花在推理上:DeepSeek 适配器禁用思考,pi-ai 适配器选择模型支持的最低等级。输出只组装为文本块;工具调用、格式错误或空输出与非 stop 结束原因都会拒绝,后续模型失败会保留请求记录。
|
||||
生成在注册时校验一次配置;每次修订把选中的消息封装为 JSON,依据 `maxInputBytes` 检查封装提示词的 UTF-8 字节数,解析路由(显式对或已记录 `request/header`),追加一条携带确切可分发请求的仅日志 `session/title-llm-request` 事件,然后在组合的超时与取消截止时间内通过 `ctx.llm` 流式生成。分发的封套携带用于重放标注的 `purpose: 'session-title'` 与 `minimizeReasoning`,且有意不包含 agent loop 的进程本地请求身份;适配器以模型支持的最少推理分发最小化推理请求,使标题请求不把输出上限花在推理上:DeepSeek 适配器禁用思考,pi-ai 适配器选择模型支持的最低等级。输出只组装为文本块;工具调用、格式错误或空输出与非 stop 结束原因都会拒绝,后续模型失败会保留请求记录。
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
@@ -304,6 +304,7 @@ export async function generateSessionTitleWithLlm(
|
||||
maxTokens,
|
||||
sessionId: request.session.id,
|
||||
purpose: 'session-title',
|
||||
minimizeReasoning: true,
|
||||
signal: callDeadline.signal,
|
||||
})
|
||||
request.session.append('session/title-llm-request', {
|
||||
|
||||
@@ -166,6 +166,7 @@ describe('generateSessionTitleWithLlm', () => {
|
||||
maxTokens: 32,
|
||||
sessionId: providerRequest.session.id,
|
||||
purpose: 'session-title',
|
||||
minimizeReasoning: true,
|
||||
})
|
||||
expect(options.system).toContain('5 words')
|
||||
expect(options.system).toContain('10 CJK characters')
|
||||
|
||||
Reference in new issue
Block a user